2018 XCSP3 competition: sequential solvers tracks: results by benchmark

Results by benchmark for category all categories

This page displays the results of the different solvers for each benchmark for category all categories

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell example	Meaning
Answer	Solver result
f=...	value of the objective function for the model reported by the solver
TT=...	Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out

Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

Abbreviation	Meaning
f=...	Value of the objective function
TO	Time Out
MO	Mem. Out (out of memory)

Meaning of the different colors:

Color	Meaning
text	the solver cannot handle this instance
text	the solver gave no answer
text	the solver could give an answer (SAT)
text	the solver gave a definitive answer (OPTIMUM FOUND or UNSAT)
text	the solver performed better than the other ones on that instance (complete solvers point of view)
text	the solver performed better than the other ones on that instance (incomplete solvers point of view)
text	the solver was ended by a signal or other problem
text	the solver gave an incomplete answer
text	the solver gave a wrong answer

For better readability, you may choose to hide some solvers:
BTD 2018.06.11_3 (complete)
BTD 2018.07.27_3 (complete)
BTD_12 2018-06-11_12 (complete)
BTD_12 2018.07.27_12 (complete)
Choco-solver 4.0.7 seq (493a269) (complete)
Choco-solver 4.0.7b seq (e747e1e) (complete)
Concrete 3.9.2 (complete)
Concrete 3.9.2-SuperNG (complete)
Concrete 3.8 2018-06-13 (complete)
Concrete 3.8-SuperNG 2018-06-13 (complete)
cosoco 1.12 (complete)
macht 2018.06.11 (complete)
macht 2018.07.27 (complete)
Mistral-2.0 2018-06-15 (complete)
Mistral-2.0 2018-08-01 (complete)
OscaR - Conflict Ordering 2018-07-02 (complete)
OscaR - Conflict Ordering 2018-08-14 (complete)
OscaR - Conflict Ordering with restarts 2018-07-02 (complete)
OscaR - Conflict Ordering with restarts 2018-08-14 (complete)
OscaR - Conflict Ordering with restarts 2018-08-17 (complete)
OscaR - Hybrid 2018-07-02 (complete)
OscaR - Hybrid 2018-08-14 (complete)
PicatSAT 2018-06-15 (complete)
PicatSAT 2018-08-02 (complete)
PicatSAT 2018-08-14 (complete)
Sat4j-CSP 2018-07-11 (complete)
scop both+MapleCOMSPS (2018-06-13) (complete)
scop both+MapleCOMSPS (2018-07-31) (complete)
scop order+MapleCOMSPS (2018-06-13) (complete)
scop order+MapleCOMSPS (2018-07-31) (complete)

Bench name	Best results on this instance	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
Auction/Auction-cnt/ Auction-cnt-decay100_c18.xml	SAT f=759775 TT=2520.05 T1=1381.83	No data	No data	No data	No data	SAT (TO) f=623249 TT=2400.07 T1=2231.65	SAT (TO) f=626376 TT=2520.07 T1=2398.02	SAT (TO) f=472590 TT=2520.13 T1=248.46	SAT (TO) f=474788 TT=2520.14 T1=717.44	SAT (TO) f=488648 TT=2520.14 T1=58.47	SAT (TO) f=536318 TT=2520.08 T1=293.28	SAT (TO) f=536035 TT=2520.07 T1=2378.69	No data	No data	SAT (TO) f=541558 TT=2400.05 T1=1168.66	SAT (TO) f=541558 TT=2520.09 T1=1174.9	No data	No data	SAT (TO) f=521955 TT=2400.09 T1=24.85	SAT (TO) f=521656 TT=2520.1 T1=1288.92	SAT (TO) f=513528 TT=2520.03 T1=32.5	SAT (TO) f=809405 TT=2400.03 T1=1522.05	SAT (TO) f=759775 TT=2520.05 T1=1381.83	? (TO) TT=2519.74	? (TO) TT=2520	? (TO) TT=2519.73	SAT (TO) f=243159 TT=2400.16 T1=1983.26	No data	No data	No data	No data
Auction/Auction-cnt/ Auction-cnt-decay200_c18.xml	SAT f=1458346 TT=2520.1 T1=2164.55	No data	No data	No data	No data	SAT (TO) f=1281904 TT=2400.06 T1=2192.61	SAT (TO) f=1281904 TT=2520.03 T1=2132.02	SAT (TO) f=896738 TT=2520.1 T1=771.66	SAT (TO) f=928237 TT=2520.08 T1=1626.53	SAT (TO) f=881244 TT=2520.16 T1=841.8	SAT (TO) f=883182 TT=2520.09 T1=2431.6	SAT (TO) f=716515 TT=2520.01 T1=2519.62	No data	No data	SAT (TO) f=999185 TT=2400.09 T1=787.94	SAT (TO) f=999185 TT=2519.74 T1=792.6	No data	No data	SAT (TO) f=831478 TT=2400.05 T1=611.78	SAT (TO) f=796204 TT=2520.12 T1=14.18	SAT (TO) f=810394 TT=2520.09 T1=21.84	SAT (TO) f=1402863 TT=2400.04 T1=1930.31	SAT (TO) f=1458346 TT=2520.1 T1=2164.55	? (TO) TT=2520.01	? (TO) TT=2519.97	? (TO) TT=2519.93	SAT (TO) f=534878 TT=2400.14 T1=1126.25	No data	No data	No data	No data
Auction/Auction-cnt/ Auction-cnt-decay300_c18.xml	SAT f=1958918 TT=2520.08 T1=2087.2	No data	No data	No data	No data	SAT (TO) f=1668675 TT=2400.1 T1=999.58	SAT (TO) f=1668675 TT=2520.1 T1=1049.14	SAT (TO) f=1325626 TT=2520.15 T1=415.06	SAT (TO) f=1274377 TT=2520.13 T1=101.77	SAT (TO) f=1303313 TT=2520.1 T1=1606.36	SAT (TO) f=1242537 TT=2520.14 T1=133.31	SAT (TO) f=816503 TT=2519.96 T1=2519.66	No data	No data	SAT (TO) f=1277057 TT=2400.1 T1=1957.93	SAT (TO) f=1277057 TT=2519.93 T1=1984.67	No data	No data	SAT (TO) f=1207542 TT=2400.04 T1=808.02	SAT (TO) f=1126437 TT=2520.07 T1=1881.83	SAT (TO) f=1168912 TT=2520.08 T1=2512.57	SAT (TO) f=2015882 TT=2400.09 T1=2145.07	SAT (TO) f=1958918 TT=2520.08 T1=2087.2	? (TO) TT=2519.9	? (TO) TT=2520.04	? (TO) TT=2519.97	SAT (TO) f=694246 TT=2400.11 T1=2245.58	No data	No data	No data	No data
Auction/Auction-cnt/ Auction-cnt-decay400_c18.xml	SAT f=2494709 TT=2520.12 T1=1994.82	No data	No data	No data	No data	SAT (TO) f=2177494 TT=2400.06 T1=1718.47	SAT (TO) f=2177494 TT=2520.09 T1=1745.99	SAT (TO) f=1570123 TT=2520.12 T1=1507.42	SAT (TO) f=1612254 TT=2520.16 T1=9.1	SAT (TO) f=1645472 TT=2520.12 T1=642.92	SAT (TO) f=1583043 TT=2520.06 T1=134.92	SAT (TO) f=878545 TT=2519.97 T1=2519.68	No data	No data	SAT (TO) f=1490528 TT=2400.07 T1=308.66	SAT (TO) f=1490528 TT=2520.02 T1=310.77	No data	No data	SAT (TO) f=1604224 TT=2400.09 T1=1976.56	SAT (TO) f=1573030 TT=2520.08 T1=2267.63	SAT (TO) f=1506881 TT=2520.1 T1=803.93	SAT (TO) f=2493564 TT=2400.1 T1=1961.05	SAT (TO) f=2494709 TT=2520.12 T1=1994.82	? (TO) TT=2520.04	? (TO) TT=2520.06	? (TO) TT=2519.94	SAT (TO) f=769415 TT=2400.22 T1=33.26	No data	No data	No data	No data
Auction/Auction-cnt/ Auction-cnt-decay500_c18.xml	SAT f=3017966 TT=2520.12 T1=1890.16	No data	No data	No data	No data	SAT (TO) f=2564900 TT=2400.11 T1=2234.24	SAT (TO) f=2564900 TT=2520.11 T1=2225.44	SAT (TO) f=2074716 TT=2520.09 T1=48.59	SAT (TO) f=1869998 TT=2520.14 T1=756.81	SAT (TO) f=2136756 TT=2520.11 T1=9.9	SAT (TO) f=2163655 TT=2520.08 T1=9.52	SAT (TO) f=1159861 TT=2520.07 T1=2519.79	No data	No data	SAT (TO) f=1737928 TT=2400.07 T1=1008.93	SAT (TO) f=1737928 TT=2520.06 T1=1016.08	No data	No data	SAT (TO) f=1836605 TT=2400.1 T1=1408.71	SAT (TO) f=1825991 TT=2520.03 T1=2486.12	SAT (TO) f=1932606 TT=2520.06 T1=2490.89	SAT (TO) f=2941882 TT=2400.1 T1=2335.33	SAT (TO) f=3017966 TT=2520.12 T1=1890.16	? (TO) TT=2520.02	? (TO) TT=2520	? (TO) TT=2519.9	SAT (TO) f=1124804 TT=2400.18 T1=1697.6	No data	No data	No data	No data
Auction/Auction-cnt/ Auction-cnt-example_c18.xml	OPT f=54 TT=0.005 T1=0	No data	No data	No data	No data	OPT f=54 TT=0.82 T1=0.45	OPT f=54 TT=0.826 T1=0.44	OPT f=54 TT=5.935 T1=3.09	OPT f=54 TT=5.923 T1=3.11	OPT f=54 TT=5.995 T1=3.06	OPT f=54 TT=5.984 T1=3.66	OPT f=54 TT=0.005 T1=0	No data	No data	OPT f=54 TT=0.005 T1=0	OPT f=54 TT=0.005 T1=0	No data	No data	SAT (TO) f=54 TT=2400.06 T1=1.02	SAT (TO) f=54 TT=2520.08 T1=0.99	OPT f=54 TT=1.469 T1=0.99	OPT f=54 TT=1.525 T1=1.01	OPT f=54 TT=1.452 T1=0.98	OPT f=54 TT=0.019 T1=0.01	OPT f=54 TT=0.017 T1=0.01	OPT f=54 TT=0.017 T1=0.01	OPT f=54 TT=0.578 T1=0.34	No data	No data	No data	No data
Auction/Auction-cnt/ Auction-cnt-matching040_c18.xml	SAT f=12922392 TT=2520.04 T1=942.08	No data	No data	No data	No data	SAT (TO) f=7925214 TT=2400.06 T1=1973.28	SAT (TO) f=7925214 TT=2520.06 T1=1980.94	SAT (TO) f=9581418 TT=2520.13 T1=1393.81	SAT (TO) f=9482748 TT=2520.08 T1=215.94	SAT (TO) f=9552934 TT=2520.09 T1=1615.12	SAT (TO) f=8217494 TT=2520.07 T1=1544.86	SAT (TO) f=4315461 TT=2520.09 T1=2021.27	No data	No data	SAT (TO) f=9704465 TT=2400.02 T1=198.01	SAT (TO) f=9704465 TT=2519.93 T1=199.85	No data	No data	SAT (TO) f=7765907 TT=2400.04 T1=56.35	SAT (TO) f=7765907 TT=2520.11 T1=89.8	SAT (TO) f=7894756 TT=2520.03 T1=1114.74	SAT (TO) f=12882352 TT=2400.06 T1=1867.17	SAT (TO) f=12922392 TT=2520.04 T1=942.08	? (TO) TT=2519.92	? (TO) TT=2519.77	? (TO) TT=2520.05	SAT (TO) f=8923883 TT=2400.3 T1=94.62	No data	No data	No data	No data
Auction/Auction-cnt/ Auction-cnt-matching080_c18.xml	SAT f=49734878 TT=2520.08 T1=1583.34	No data	No data	No data	No data	SAT (TO) f=30568459 TT=2400.12 T1=1.38	SAT (TO) f=30568459 TT=2520.06 T1=1.41	SAT (TO) f=31947930 TT=2520.09 T1=2089.92	SAT (TO) f=31649831 TT=2520.11 T1=129.05	SAT (TO) f=31206797 TT=2520.06 T1=223.15	SAT (TO) f=32442777 TT=2520.07 T1=729.28	SAT (TO) f=8197230 TT=2520.06 T1=2324.98	No data	No data	SAT (TO) f=21755197 TT=2400.03 T1=1952.17	SAT (TO) f=21755197 TT=2519.95 T1=1950.27	No data	No data	SAT (TO) f=30255533 TT=2400.03 T1=1667.21	SAT (TO) f=29942494 TT=2520.12 T1=2205.11	SAT (TO) f=30759348 TT=2520.03 T1=884.55	SAT (TO) f=45437910 TT=2400.41 T1=648	SAT (TO) f=49734878 TT=2520.08 T1=1583.34	? (TO) TT=2519.79	? (TO) TT=2520.07	? (TO) TT=2519.99	SAT (TO) f=26851162 TT=2400.35 T1=528.46	No data	No data	No data	No data
Auction/Auction-sum/ Auction-sum-decay100_c18.xml	SAT f=750618 TT=2520.06 T1=2090.59	No data	No data	No data	No data	SAT (TO) f=651593 TT=2400.08 T1=937.23	SAT (TO) f=651593 TT=2520.13 T1=967.32	SAT (TO) f=559117 TT=2520.12 T1=511.08	SAT (TO) f=550601 TT=2520.16 T1=1917.85	? (TO) TT=2522.11	? (TO) TT=2522.11	SAT (TO) f=539206 TT=2520.07 T1=2348.02	No data	No data	SAT (TO) f=566218 TT=2400.07 T1=1646.5	SAT (TO) f=566218 TT=2520.06 T1=1649.87	No data	No data	SAT (TO) f=573921 TT=2400.08 T1=366.5	SAT (TO) f=534256 TT=2520.35 T1=25.52	SAT (TO) f=543786 TT=2520.08 T1=1352.35	SAT (TO) f=783852 TT=2400.12 T1=2227.94	SAT (TO) f=750618 TT=2520.06 T1=2090.59	? (TO) TT=2519.93	? (TO) TT=2519.88	? (TO) TT=2520.06	? (TO) TT=2400.11	No data	No data	No data	No data
Auction/Auction-sum/ Auction-sum-decay200_c18.xml	SAT f=1433796 TT=2520.12 T1=1979.84	No data	No data	No data	No data	SAT (TO) f=1245380 TT=2400.13 T1=782.4	SAT (TO) f=1245380 TT=2520.11 T1=754.27	SAT (TO) f=1035551 TT=2520.08 T1=1128.52	SAT (TO) f=1048656 TT=2520.14 T1=2032.66	? (TO) TT=2522.12	? (TO) TT=2522.07	SAT (TO) f=754804 TT=2519.94 T1=2519.55	No data	No data	SAT (TO) f=1014918 TT=2400.08 T1=196.34	SAT (TO) f=1014918 TT=2519.93 T1=197.15	No data	No data	SAT (TO) f=835314 TT=2400.1 T1=2047.83	SAT (TO) f=811537 TT=2520.08 T1=678.07	SAT (TO) f=835467 TT=2520.09 T1=494.08	SAT (TO) f=1440749 TT=2400.03 T1=1129.75	SAT (TO) f=1433796 TT=2520.12 T1=1979.84	? (TO) TT=2519.73	? (TO) TT=2520.09	? (TO) TT=2520.07	? (TO) TT=2400.09	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
Auction/Auction-sum/ Auction-sum-decay300_c18.xml	SAT f=1955977 TT=2520.18 T1=2385.68	No data	No data	No data	No data	SAT (TO) f=1724240 TT=2400.03 T1=428.44	SAT (TO) f=1724240 TT=2520.1 T1=465.69	SAT (TO) f=1437447 TT=2520.07 T1=2394.21	SAT (TO) f=1454405 TT=2520.09 T1=291.41	SAT (TO) f=1449037 TT=2520.07 T1=2475.1	SAT (TO) f=1334005 TT=2520.12 T1=2344.07	SAT (TO) f=894399 TT=2520.08 T1=2519.61	No data	No data	SAT (TO) f=1247216 TT=2400.06 T1=1106.15	SAT (TO) f=1247216 TT=2520.08 T1=1110.71	No data	No data	SAT (TO) f=1153856 TT=2400.07 T1=2181.81	SAT (TO) f=1143084 TT=2520.08 T1=633.51	SAT (TO) f=1149908 TT=2520.05 T1=1527.17	SAT (TO) f=2005187 TT=2400.11 T1=2313.65	SAT (TO) f=1955977 TT=2520.18 T1=2385.68	? (TO) TT=2519.91	? (TO) TT=2519.62	? (TO) TT=2520.08	SAT (TO) f=578688 TT=2400.16 T1=1434.09	No data	No data	No data	No data
Auction/Auction-sum/ Auction-sum-decay400_c18.xml	SAT f=2501073 TT=2520.03 T1=1847.02	No data	No data	No data	No data	SAT (TO) f=2255440 TT=2400.04 T1=2213.82	SAT (TO) f=2255440 TT=2520.12 T1=2211.73	SAT (TO) f=1844720 TT=2520.14 T1=143.85	SAT (TO) f=1805580 TT=2520.12 T1=703.59	? (TO) TT=2522.1	SAT (TO) f=1776807 TT=2520.16 T1=765.08	SAT (TO) f=869281 TT=2520.09 T1=2519.66	No data	No data	SAT (TO) f=1444085 TT=2400.08 T1=2148.43	SAT (TO) f=1444085 TT=2519.91 T1=2143.45	No data	No data	SAT (TO) f=1487688 TT=2400.12 T1=2364.32	SAT (TO) f=1522733 TT=2520.05 T1=2294.95	SAT (TO) f=1541247 TT=2520.12 T1=365.91	SAT (TO) f=2482439 TT=2400.12 T1=2228.86	SAT (TO) f=2501073 TT=2520.03 T1=1847.02	? (TO) TT=2520.02	? (TO) TT=2519.93	? (TO) TT=2519.79	SAT (TO) f=702676 TT=2401.4 T1=1673.81	No data	No data	No data	No data
Auction/Auction-sum/ Auction-sum-decay500_c18.xml	SAT f=3125540 TT=2520.06 T1=1924.43	No data	No data	No data	No data	SAT (TO) f=2788055 TT=2400.15 T1=209.93	SAT (TO) f=2788055 TT=2520.11 T1=219.52	SAT (TO) f=2180137 TT=2520.13 T1=758.96	SAT (TO) f=2349526 TT=2520.1 T1=535.11	SAT (TO) f=2071736 TT=2520.13 T1=2064.04	SAT (TO) f=2173290 TT=2520.13 T1=2241.79	SAT (TO) f=1043766 TT=2520.06 T1=2519.85	No data	No data	SAT (TO) f=1691567 TT=2400.08 T1=1997.52	SAT (TO) f=1691567 TT=2519.91 T1=2001.2	No data	No data	SAT (TO) f=1882828 TT=2400.09 T1=510.38	SAT (TO) f=1865085 TT=2520.11 T1=430.39	SAT (TO) f=1902586 TT=2520.12 T1=2486.11	SAT (TO) f=3048435 TT=2400.09 T1=2155.12	SAT (TO) f=3125540 TT=2520.06 T1=1924.43	? (TO) TT=2519.89	? (TO) TT=2519.95	? (TO) TT=2520.06	SAT (TO) f=1170840 TT=2400.54 T1=73.64	No data	No data	No data	No data
Auction/Auction-sum/ Auction-sum-example_c18.xml	OPT f=54 TT=0.004 T1=0	No data	No data	No data	No data	OPT f=54 TT=0.824 T1=0.45	OPT f=54 TT=0.805 T1=0.44	OPT f=54 TT=6.051 T1=3.1	OPT f=54 TT=6.119 T1=3.15	OPT f=54 TT=5.97 T1=3.12	OPT f=54 TT=6.116 T1=3.13	OPT f=54 TT=0.004 T1=0	No data	No data	OPT f=54 TT=0.004 T1=0	OPT f=54 TT=0.005 T1=0	No data	No data	SAT (TO) f=54 TT=2400.05 T1=1.01	SAT (TO) f=54 TT=2520.12 T1=1	OPT f=54 TT=1.486 T1=1	OPT f=54 TT=1.514 T1=1.01	OPT f=54 TT=1.461 T1=0.99	OPT f=54 TT=0.018 T1=0.01	OPT f=54 TT=0.016 T1=0.01	OPT f=54 TT=0.018 T1=0.01	OPT f=54 TT=0.562 T1=0.34	No data	No data	No data	No data
Auction/Auction-sum/ Auction-sum-matching040_c18.xml	SAT f=12918598 TT=2520.13 T1=773.68	No data	No data	No data	No data	SAT (TO) f=8642176 TT=2400.06 T1=1789.51	SAT (TO) f=8908119 TT=2520.11 T1=2437.26	SAT (TO) f=9412952 TT=2520.15 T1=1552.06	SAT (TO) f=9093424 TT=2520.16 T1=1485.01	? (TO) TT=2528.38	? (TO) TT=2528.07	SAT (TO) f=3881017 TT=2520.03 T1=1810.11	No data	No data	SAT (TO) f=7565890 TT=2400.03 T1=2334.65	SAT (TO) f=8700950 TT=2519.67 T1=2469.55	No data	No data	SAT (TO) f=7820295 TT=2400.08 T1=1519.21	SAT (TO) f=8187094 TT=2520.08 T1=2430.57	SAT (TO) f=8060347 TT=2520.13 T1=869.3	SAT (TO) f=12959841 TT=2400.1 T1=301.74	SAT (TO) f=12918598 TT=2520.13 T1=773.68	? (TO) TT=2520.04	? (TO) TT=2520.03	? (TO) TT=2520.1	? (TO) TT=2400.1	No data	No data	No data	No data
Auction/Auction-sum/ Auction-sum-matching080_c18.xml	SAT f=46256429 TT=2520.05 T1=2263.93	No data	No data	No data	No data	SAT (TO) f=31584941 TT=2400.06 T1=1893.06	SAT (TO) f=32190529 TT=2520.06 T1=2436.49	SAT (TO) f=35187110 TT=2520.11 T1=1397.18	SAT (TO) f=37264982 TT=2520.09 T1=306.22	? (TO) TT=2527.96	? (TO) TT=2522.05	SAT (TO) f=9743707 TT=2520 T1=2458.25	No data	No data	SAT (TO) f=17146843 TT=2400.02 T1=1973.21	SAT (TO) f=17146843 TT=2519.97 T1=1970.56	No data	No data	SAT (TO) f=29644484 TT=2400.13 T1=77.04	SAT (TO) f=29816463 TT=2520.04 T1=415.72	SAT (TO) f=29857707 TT=2520.04 T1=2089.62	SAT (TO) f=46810432 TT=2400.06 T1=1047.75	SAT (TO) f=46256429 TT=2520.05 T1=2263.93	? (TO) TT=2519.92	? (TO) TT=2520.02	? (TO) TT=2520.07	SAT (TO) f=22768808 TT=2400.15 T1=2360.79	No data	No data	No data	No data
Bacp/Bacp-m1/ Bacp-m1-05_c18.xml	OPT f=9 TT=0.024 T1=0	No data	No data	No data	No data	OPT f=9 TT=1.732 T1=0.54	OPT f=9 TT=1.777 T1=0.54	OPT f=9 TT=8.914 T1=3.6	OPT f=9 TT=8.815 T1=3.67	OPT f=9 TT=9.168 T1=3.62	OPT f=9 TT=8.791 T1=3.55	OPT f=9 TT=0.024 T1=0	No data	No data	OPT f=9 TT=0.024 T1=0.01	OPT f=9 TT=0.024 T1=0.01	No data	No data	SAT (TO) f=9 TT=2400.04 T1=1.21	SAT (TO) f=9 TT=2520.08 T1=1.13	OPT f=9 TT=2.046 T1=1.13	OPT f=9 TT=2.149 T1=1.2	OPT f=9 TT=2.06 T1=1.15	OPT f=9 TT=6.837 T1=6.83	OPT f=9 TT=7.776 T1=7.77	OPT f=9 TT=7.807 T1=7.81	OPT f=9 TT=2.284 T1=0.92	No data	No data	No data	No data
Bacp/Bacp-m1/ Bacp-m1-06_c18.xml	OPT f=10 TT=1.075 T1=0.57	No data	No data	No data	No data	OPT f=10 TT=9.528 T1=0.56	OPT f=10 TT=9.636 T1=0.57	OPT f=10 TT=32.886 T1=3.72	OPT f=10 TT=25.337 T1=3.89	OPT f=10 TT=39.325 T1=3.71	OPT f=10 TT=27.481 T1=3.77	Signal TT=0.012	No data	No data	OPT f=10 TT=1.959 T1=0.01	OPT f=10 TT=1.929 T1=0.01	No data	No data	SAT (TO) f=10 TT=2400.11 T1=1.23	SAT (TO) f=10 TT=2520.03 T1=1.17	OPT f=10 TT=6.515 T1=1.17	SAT (TO) f=10 TT=2400.12 T1=1.23	SAT (TO) f=10 TT=2520.1 T1=1.17	OPT f=10 TT=0.841 T1=0.84	OPT f=10 TT=0.856 T1=0.85	OPT f=10 TT=1.075 T1=1.07	OPT f=10 TT=8.923 T1=1.42	No data	No data	No data	No data
Bacp/Bacp-m1/ Bacp-m1-07a_c18.xml	OPT f=12 TT=107.185 T1=0.03	No data	No data	No data	No data	SAT (TO) f=12 TT=2400.08 T1=0.73	SAT (TO) f=12 TT=2520.09 T1=0.72	SAT (TO) f=12 TT=2520.11 T1=4.03	SAT (TO) f=12 TT=2520.11 T1=3.97	SAT (TO) f=12 TT=2520.05 T1=4.11	SAT (TO) f=12 TT=2520.12 T1=3.9	SAT (TO) f=12 TT=2520 T1=0.03	No data	No data	SAT (TO) f=12 TT=2400.05 T1=0.02	SAT (TO) f=12 TT=2519.79 T1=0.02	No data	No data	SAT (TO) f=12 TT=2400.06 T1=1.29	SAT (TO) f=12 TT=2520.12 T1=1.21	SAT (TO) f=12 TT=2520.05 T1=1.22	SAT (TO) f=12 TT=2400.05 T1=1.27	SAT (TO) f=12 TT=2520.1 T1=1.21	OPT f=12 TT=97.963 T1=97.97	OPT f=12 TT=131.966 T1=131.96	OPT f=12 TT=107.185 T1=107.17	? (TO) TT=2400.32	No data	No data	No data	No data
Bacp/Bacp-m1/ Bacp-m1-07b_c18.xml	OPT f=13 TT=1278.93 T1=0.02	No data	No data	No data	No data	SAT (TO) f=13 TT=2400.12 T1=0.78	SAT (TO) f=13 TT=2520.1 T1=0.68	SAT (TO) f=13 TT=2520.11 T1=4.25	SAT (TO) f=13 TT=2520.13 T1=4.16	SAT (TO) f=13 TT=2520.13 T1=4.31	SAT (TO) f=13 TT=2520.04 T1=4.2	SAT (TO) f=13 TT=2519.79 T1=0.02	No data	No data	SAT (TO) f=13 TT=2400.06 T1=0.01	SAT (TO) f=13 TT=2520.07 T1=0.01	No data	No data	SAT (TO) f=13 TT=2400.09 T1=1.31	SAT (TO) f=13 TT=2520.11 T1=1.51	SAT (TO) f=13 TT=2520.08 T1=1.27	SAT (TO) f=13 TT=2400.03 T1=1.5	SAT (TO) f=13 TT=2520.02 T1=1.45	OPT f=13 TT=2163.98 T1=2163.89	OPT f=13 TT=1612.92 T1=1612.75	OPT f=13 TT=1278.93 T1=1278.97	SAT (TO) f=13 TT=2400.29 T1=6.49	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
Bacp/Bacp-m1/ Bacp-m1-08a_c18.xml	SAT f=17 TT=2400.26 T1=0.04	No data	No data	No data	No data	SAT (TO) f=17 TT=2400.08 T1=0.66	SAT (TO) f=17 TT=2520.06 T1=1.12	SAT (TO) f=17 TT=2520.04 T1=4.13	SAT (TO) f=17 TT=2520.12 T1=4.2	SAT (TO) f=17 TT=2520.12 T1=4.31	SAT (TO) f=17 TT=2520.07 T1=4.23	SAT (TO) f=17 TT=2519.87 T1=0.04	No data	No data	SAT (TO) f=17 TT=2400.09 T1=0.03	SAT (TO) f=17 TT=2519.98 T1=0.03	No data	No data	SAT (TO) f=17 TT=2400.04 T1=1.37	SAT (TO) f=17 TT=2520.11 T1=1.31	SAT (TO) f=17 TT=2520.08 T1=1.32	SAT (TO) f=17 TT=2400.06 T1=2.73	SAT (TO) f=17 TT=2520.03 T1=2.53	? (TO) TT=2519.89	? (TO) TT=2520.06	? (TO) TT=2519.9	SAT (TO) f=17 TT=2400.26 T1=26.77	No data	No data	No data	No data
Bacp/Bacp-m1/ Bacp-m1-08b_c18.xml	SAT f=17 TT=2400.18 T1=0.04	No data	No data	No data	No data	SAT (TO) f=17 TT=2400.14 T1=0.67	SAT (TO) f=17 TT=2520.1 T1=0.66	SAT (TO) f=17 TT=2520.08 T1=4.25	SAT (TO) f=17 TT=2520.06 T1=4.16	SAT (TO) f=17 TT=2520.12 T1=4.28	SAT (TO) f=17 TT=2520.07 T1=4.54	SAT (TO) f=17 TT=2519.93 T1=0.04	No data	No data	SAT (TO) f=17 TT=2400.05 T1=0.06	SAT (TO) f=17 TT=2519.94 T1=0.06	No data	No data	SAT (TO) f=17 TT=2400.08 T1=1.39	SAT (TO) f=17 TT=2520.07 T1=1.34	SAT (TO) f=17 TT=2520.12 T1=1.31	SAT (TO) f=17 TT=2400.09 T1=3.57	SAT (TO) f=17 TT=2520.07 T1=3.79	? (TO) TT=2520.05	? (TO) TT=2520.07	? (TO) TT=2520.01	SAT (TO) f=17 TT=2400.18 T1=18.68	No data	No data	No data	No data
Bacp/Bacp-m1/ Bacp-m1-10a-nodv_c18.xml	SAT f=14 TT=2400.15 T1=0.05	No data	No data	No data	No data	SAT (TO) f=14 TT=2400.13 T1=0.65	SAT (TO) f=14 TT=2520.03 T1=0.65	SAT (TO) f=14 TT=2520.09 T1=4.66	SAT (TO) f=14 TT=2520.07 T1=4.15	SAT (TO) f=14 TT=2520.08 T1=4.49	SAT (TO) f=14 TT=2520.13 T1=5.31	SAT (TO) f=14 TT=2520.01 T1=0.05	No data	No data	SAT (TO) f=14 TT=2400.01 T1=0.07	SAT (TO) f=14 TT=2520.09 T1=0.07	No data	No data	SAT (TO) f=14 TT=2400.04 T1=1.43	SAT (TO) f=14 TT=2520.05 T1=1.37	SAT (TO) f=14 TT=2520.11 T1=1.35	SAT (TO) f=14 TT=2400.07 T1=1.4	SAT (TO) f=14 TT=2520.02 T1=1.29	? (TO) TT=2519.86	? (TO) TT=2519.92	? (TO) TT=2519.91	SAT (TO) f=14 TT=2400.15 T1=28.36	No data	No data	No data	No data
Bacp/Bacp-m1/ Bacp-m1-10a_c18.xml	SAT f=14 TT=2400.18 T1=0.33	No data	No data	No data	No data	SAT (TO) f=14 TT=2400.14 T1=0.64	SAT (TO) f=14 TT=2520.12 T1=0.65	SAT (TO) f=14 TT=2520.14 T1=4.14	SAT (TO) f=14 TT=2520.07 T1=4.14	SAT (TO) f=14 TT=2520.04 T1=5.26	SAT (TO) f=14 TT=2520.08 T1=4.58	SAT (TO) f=14 TT=2520 T1=0.33	No data	No data	SAT (TO) f=14 TT=2400.09 T1=0.07	SAT (TO) f=14 TT=2519.82 T1=0.07	No data	No data	SAT (TO) f=14 TT=2400.03 T1=2.37	SAT (TO) f=14 TT=2520.08 T1=2.4	SAT (TO) f=14 TT=2520.08 T1=2.56	SAT (TO) f=14 TT=2400.11 T1=122.51	SAT (TO) f=14 TT=2520.09 T1=133.77	? (TO) TT=2519.47	? (TO) TT=2519.97	? (TO) TT=2520.04	SAT (TO) f=14 TT=2400.18 T1=28.53	No data	No data	No data	No data
Bacp/Bacp-m1/ Bacp-m1-10b_c18.xml	SAT f=14 TT=2400.22 T1=0.02	No data	No data	No data	No data	SAT (TO) f=14 TT=2400.07 T1=0.64	SAT (TO) f=14 TT=2520.14 T1=0.65	SAT (TO) f=14 TT=2520.1 T1=4.13	SAT (TO) f=14 TT=2520.1 T1=4.24	SAT (TO) f=14 TT=2520.15 T1=4.13	SAT (TO) f=14 TT=2520.09 T1=4.31	SAT (TO) f=14 TT=2519.65 T1=0.02	No data	No data	SAT (TO) f=14 TT=2400.02 T1=0.03	SAT (TO) f=14 TT=2519.92 T1=0.03	No data	No data	SAT (TO) f=14 TT=2400.05 T1=1.39	SAT (TO) f=14 TT=2520.02 T1=1.32	SAT (TO) f=14 TT=2520.04 T1=1.32	SAT (TO) f=14 TT=2400.11 T1=1.48	SAT (TO) f=14 TT=2520.04 T1=1.43	? (TO) TT=2519.86	? (TO) TT=2519.93	? (TO) TT=2519.88	SAT (TO) f=14 TT=2400.22 T1=92.36	No data	No data	No data	No data
Bacp/Bacp-m1/ Bacp-m1-12a-nodv_c18.xml	SAT f=17 TT=2520.02 T1=1.58	No data	No data	No data	No data	SAT (TO) f=17 TT=2400.11 T1=8.36	SAT (TO) f=17 TT=2520.11 T1=8.29	SAT (TO) f=18 TT=2520.06 T1=4.65	SAT (TO) f=18 TT=2520.14 T1=4.71	SAT (TO) f=17 TT=2520.12 T1=1249.37	SAT (TO) f=18 TT=2520.08 T1=5.29	SAT (TO) f=17 TT=2520.02 T1=3.92	No data	No data	SAT (TO) f=17 TT=2400.06 T1=0.18	SAT (TO) f=17 TT=2519.9 T1=0.25	No data	No data	SAT (TO) f=17 TT=2400.1 T1=1.65	SAT (TO) f=17 TT=2520.04 T1=1.89	SAT (TO) f=17 TT=2520.04 T1=1.58	SAT (TO) f=17 TT=2400.08 T1=1.64	SAT (TO) f=17 TT=2520.06 T1=1.68	? (TO) TT=2519.98	? (TO) TT=2519.92	? (TO) TT=2519.91	? (TO) TT=2400.07	No data	No data	No data	No data
Bacp/Bacp-m1/ Bacp-m1-12a_c18.xml	SAT f=17 TT=2520 T1=0.31	No data	No data	No data	No data	SAT (TO) f=17 TT=2400.08 T1=8.23	SAT (TO) f=17 TT=2520.05 T1=8.31	SAT (TO) f=17 TT=2520.05 T1=4.69	SAT (TO) f=17 TT=2520.14 T1=5.72	SAT (TO) f=17 TT=2520.04 T1=29.51	SAT (TO) f=17 TT=2520.1 T1=174.56	SAT (TO) f=17 TT=2520 T1=0.31	No data	No data	SAT (TO) f=17 TT=2400.08 T1=0.18	SAT (TO) f=17 TT=2519.84 T1=0.18	No data	No data	SAT (TO) f=17 TT=2400.09 T1=1.95	SAT (TO) f=17 TT=2520.1 T1=1.49	SAT (TO) f=17 TT=2520.13 T1=1.48	? (TO) TT=2400.05	? (TO) TT=2520.07	? (TO) TT=2519.82	? (TO) TT=2520.02	? (TO) TT=2520.02	? (TO) TT=2400.07	No data	No data	No data	No data
Bacp/Bacp-m1/ Bacp-m1-12b_c18.xml	SAT f=17 TT=2519.83 T1=0.06	No data	No data	No data	No data	SAT (TO) f=17 TT=2400.08 T1=13.62	SAT (TO) f=17 TT=2520.12 T1=13.57	SAT (TO) f=17 TT=2520.1 T1=4.79	SAT (TO) f=17 TT=2520.12 T1=4.69	SAT (TO) f=17 TT=2520.1 T1=4.6	SAT (TO) f=17 TT=2520.14 T1=4.92	SAT (TO) f=17 TT=2519.83 T1=0.06	No data	No data	SAT (TO) f=17 TT=2400.02 T1=0.25	SAT (TO) f=17 TT=2519.82 T1=0.25	No data	No data	SAT (TO) f=17 TT=2400.12 T1=1.56	SAT (TO) f=17 TT=2520.04 T1=1.47	SAT (TO) f=17 TT=2520.04 T1=1.81	SAT (TO) f=17 TT=2400.08 T1=1.55	SAT (TO) f=17 TT=2520.07 T1=1.51	? (TO) TT=2519.78	? (TO) TT=2519.88	? (TO) TT=2520	? (TO) TT=2400.16	No data	No data	No data	No data
Bacp/Bacp-m2/ Bacp-m2-05_c18.xml	OPT f=9 TT=0.027 T1=0.01	No data	No data	No data	No data	OPT f=9 TT=1.949 T1=0.55	OPT f=9 TT=1.968 T1=0.79	OPT f=9 TT=11.753 T1=4.49	OPT f=9 TT=11.922 T1=4.5	OPT f=9 TT=9.7 T1=4.04	OPT f=9 TT=10.011 T1=4.03	OPT f=9 TT=0.027 T1=0.01	No data	No data	OPT f=9 TT=0.1 T1=0.01	OPT f=9 TT=0.1 T1=0.01	No data	No data	SAT (TO) f=9 TT=2400.12 T1=1.24	SAT (TO) f=9 TT=2520.08 T1=1.19	OPT f=9 TT=2.196 T1=1.19	OPT f=9 TT=2.305 T1=1.24	OPT f=9 TT=2.239 T1=1.19	OPT f=9 TT=0.255 T1=0.25	OPT f=9 TT=0.276 T1=0.27	OPT f=9 TT=0.273 T1=0.27	OPT f=9 TT=2.788 T1=0.81	No data	No data	No data	No data
Bacp/Bacp-m2/ Bacp-m2-06_c18.xml	OPT f=10 TT=0.712 T1=0.01	No data	No data	No data	No data	OPT f=10 TT=8.074 T1=0.59	OPT f=10 TT=7.834 T1=0.59	OPT f=10 TT=40.737 T1=4.84	OPT f=10 TT=43.612 T1=4.67	OPT f=10 TT=21.085 T1=4.22	OPT f=10 TT=26.307 T1=4.31	OPT f=10 TT=3.282 T1=0.01	No data	No data	Wrong UNSAT TT=0.019	OPT f=10 TT=2.718 T1=0.01	No data	No data	SAT (TO) f=10 TT=2400.05 T1=1.29	SAT (TO) f=10 TT=2520.03 T1=1.25	OPT f=10 TT=10.36 T1=1.25	SAT (TO) f=10 TT=2400.03 T1=1.32	SAT (TO) f=10 TT=2520.12 T1=1.25	OPT f=10 TT=0.684 T1=0.68	OPT f=10 TT=0.716 T1=0.71	OPT f=10 TT=0.712 T1=0.71	OPT f=10 TT=19.334 T1=1.75	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
Bacp/Bacp-m2/ Bacp-m2-07a_c18.xml	OPT f=12 TT=115.296 T1=0.02	No data	No data	No data	No data	SAT (TO) f=12 TT=2400.05 T1=0.63	SAT (TO) f=12 TT=2520.03 T1=0.65	SAT (TO) f=12 TT=2520.04 T1=5.51	SAT (TO) f=12 TT=2520.1 T1=5.57	SAT (TO) f=12 TT=2520.1 T1=5.1	SAT (TO) f=12 TT=2520.07 T1=4.85	SAT (TO) f=12 TT=2519.91 T1=0.02	No data	No data	SAT (TO) f=12 TT=2400.05 T1=0.1	SAT (TO) f=12 TT=2519.95 T1=0.1	No data	No data	SAT (TO) f=12 TT=2400.05 T1=1.38	SAT (TO) f=12 TT=2520.05 T1=1.29	SAT (TO) f=12 TT=2520.11 T1=1.3	SAT (TO) f=12 TT=2400.11 T1=1.38	SAT (TO) f=12 TT=2520.04 T1=1.32	OPT f=12 TT=131.143 T1=131.14	OPT f=12 TT=114.916 T1=114.92	OPT f=12 TT=115.296 T1=115.29	SAT (TO) f=12 TT=2400.18 T1=3.83	No data	No data	No data	No data
Bacp/Bacp-m2/ Bacp-m2-07b_c18.xml	OPT f=13 TT=1198.76 T1=0.03	No data	No data	No data	No data	SAT (TO) f=13 TT=2400.07 T1=1.59	SAT (TO) f=13 TT=2520.05 T1=1.65	SAT (TO) f=13 TT=2520.06 T1=5.77	SAT (TO) f=13 TT=2520.09 T1=5.76	SAT (TO) f=13 TT=2520.13 T1=5.02	SAT (TO) f=13 TT=2520.13 T1=4.8	SAT (TO) f=13 TT=2519.8 T1=0.03	No data	No data	SAT (TO) f=13 TT=2400.1 T1=0.02	SAT (TO) f=13 TT=2520.03 T1=0.02	No data	No data	SAT (TO) f=13 TT=2400.08 T1=1.43	SAT (TO) f=13 TT=2520.05 T1=1.35	SAT (TO) f=13 TT=2520.1 T1=1.35	SAT (TO) f=13 TT=2400.03 T1=1.71	SAT (TO) f=13 TT=2520.11 T1=1.64	OPT f=13 TT=1521 T1=1521	OPT f=13 TT=957.984 T1=957.97	OPT f=13 TT=1198.76 T1=1198.79	SAT (TO) f=13 TT=2400.08 T1=5.72	No data	No data	No data	No data
Bacp/Bacp-m2/ Bacp-m2-08a_c18.xml	SAT f=17 TT=2400.16 T1=0.27	No data	No data	No data	No data	SAT (TO) f=17 TT=2400.03 T1=0.71	SAT (TO) f=17 TT=2520.13 T1=0.7	SAT (TO) f=17 TT=2520.1 T1=7.63	SAT (TO) f=17 TT=2520.14 T1=8.35	SAT (TO) f=17 TT=2520.07 T1=5.83	SAT (TO) f=17 TT=2520.11 T1=5.84	SAT (TO) f=17 TT=2519.89 T1=0.27	No data	No data	Wrong UNSAT TT=0.044	SAT (TO) f=17 TT=2519.93 T1=0.05	No data	No data	SAT (TO) f=17 TT=2400.09 T1=1.52	SAT (TO) f=17 TT=2520.09 T1=1.47	SAT (TO) f=17 TT=2520.07 T1=1.58	SAT (TO) f=17 TT=2400.12 T1=2.97	SAT (TO) f=17 TT=2520.12 T1=2.98	? (TO) TT=2519.97	? (TO) TT=2520.07	? (TO) TT=2520.1	SAT (TO) f=17 TT=2400.16 T1=21.7	No data	No data	No data	No data
Bacp/Bacp-m2/ Bacp-m2-08b_c18.xml	SAT f=17 TT=2400.19 T1=0.69	No data	No data	No data	No data	SAT (TO) f=17 TT=2400.05 T1=0.7	SAT (TO) f=17 TT=2520.03 T1=0.69	SAT (TO) f=17 TT=2520.11 T1=7.44	SAT (TO) f=17 TT=2520.04 T1=7.54	SAT (TO) f=17 TT=2520.12 T1=5.64	SAT (TO) f=17 TT=2520.05 T1=5.79	SAT (TO) f=17 TT=2520.04 T1=1.84	No data	No data	Wrong UNSAT TT=0.043	SAT (TO) f=17 TT=2520.06 T1=0.06	No data	No data	SAT (TO) f=17 TT=2400.03 T1=1.56	SAT (TO) f=17 TT=2520.03 T1=1.44	SAT (TO) f=17 TT=2520.06 T1=1.83	SAT (TO) f=17 TT=2400.06 T1=4.45	SAT (TO) f=17 TT=2520.03 T1=4.42	? (TO) TT=2519.83	? (TO) TT=2519.77	? (TO) TT=2520.04	SAT (TO) f=17 TT=2400.19 T1=23.29	No data	No data	No data	No data
Bacp/Bacp-m2/ Bacp-m2-10a-nodv_c18.xml	SAT f=14 TT=2400.17 T1=0.12	No data	No data	No data	No data	SAT (TO) f=14 TT=2400.11 T1=0.71	SAT (TO) f=14 TT=2520.08 T1=0.72	SAT (TO) f=14 TT=2520.12 T1=7.13	SAT (TO) f=14 TT=2520.06 T1=7.02	SAT (TO) f=15 TT=2520.06 T1=6.11	SAT (TO) f=14 TT=2520.1 T1=5.98	SAT (TO) f=14 TT=2519.95 T1=0.12	No data	No data	Wrong UNSAT TT=0.048	SAT (TO) f=14 TT=2520.05 T1=0.2	No data	No data	SAT (TO) f=14 TT=2400.04 T1=1.62	SAT (TO) f=14 TT=2520.05 T1=1.54	SAT (TO) f=14 TT=2520.1 T1=1.55	SAT (TO) f=14 TT=2400.11 T1=2.99	SAT (TO) f=14 TT=2520.12 T1=2.67	? (TO) TT=2519.89	? (TO) TT=2520.01	? (TO) TT=2520.1	SAT (TO) f=14 TT=2400.17 T1=40.25	No data	No data	No data	No data
Bacp/Bacp-m2/ Bacp-m2-10a_c18.xml	SAT f=14 TT=2400.16 T1=0.71	No data	No data	No data	No data	SAT (TO) f=14 TT=2400.04 T1=0.71	SAT (TO) f=14 TT=2520.12 T1=0.71	SAT (TO) f=14 TT=2520.11 T1=6.88	SAT (TO) f=14 TT=2520.11 T1=7.98	SAT (TO) f=14 TT=2520.1 T1=6.74	SAT (TO) f=14 TT=2520.09 T1=6.04	? (TO) TT=2519.94	No data	No data	Wrong UNSAT TT=0.049	SAT (TO) f=14 TT=2520.05 T1=0.2	No data	No data	SAT (TO) f=14 TT=2400.13 T1=2.58	SAT (TO) f=14 TT=2520.1 T1=2.85	SAT (TO) f=14 TT=2520.12 T1=2.66	SAT (TO) f=14 TT=2400.09 T1=199.28	SAT (TO) f=14 TT=2520.09 T1=203.02	? (TO) TT=2519.84	? (TO) TT=2519.88	? (TO) TT=2520.02	SAT (TO) f=14 TT=2400.16 T1=41.64	No data	No data	No data	No data
Bacp/Bacp-m2/ Bacp-m2-10b_c18.xml	SAT f=14 TT=2400.19 T1=0.19	No data	No data	No data	No data	SAT (TO) f=14 TT=2400.16 T1=0.7	SAT (TO) f=14 TT=2520.08 T1=0.71	SAT (TO) f=14 TT=2520.11 T1=6.88	SAT (TO) f=14 TT=2520.09 T1=7.3	SAT (TO) f=14 TT=2520.1 T1=5.77	SAT (TO) f=14 TT=2520.08 T1=5.94	SAT (TO) f=14 TT=2519.89 T1=0.19	No data	No data	Wrong UNSAT TT=0.049	SAT (TO) f=14 TT=2520.08 T1=3.68	No data	No data	SAT (TO) f=14 TT=2400.1 T1=1.52	SAT (TO) f=14 TT=2520.02 T1=1.48	SAT (TO) f=14 TT=2520.11 T1=1.49	SAT (TO) f=14 TT=2400.03 T1=1.67	SAT (TO) f=14 TT=2520.07 T1=1.58	? (TO) TT=2519.92	? (TO) TT=2520.04	? (TO) TT=2519.75	SAT (TO) f=14 TT=2400.19 T1=108.87	No data	No data	No data	No data
Bacp/Bacp-m2/ Bacp-m2-12a-nodv_c18.xml	SAT f=17 TT=2400.15 T1=0.73	No data	No data	No data	No data	SAT (TO) f=17 TT=2400.1 T1=5.64	SAT (TO) f=17 TT=2520.05 T1=5.64	SAT (TO) f=18 TT=2520.1 T1=11.32	SAT (TO) f=17 TT=2520.13 T1=11.45	SAT (TO) f=19 TT=2520.07 T1=9.57	SAT (TO) f=19 TT=2520.12 T1=10.12	SAT (TO) f=17 TT=2519.81 T1=0.73	No data	No data	Wrong UNSAT TT=0.085	SAT (TO) f=18 TT=2519.9 T1=0.12	No data	No data	SAT (TO) f=17 TT=2400.03 T1=60.12	SAT (TO) f=17 TT=2520.12 T1=99.28	SAT (TO) f=17 TT=2520.07 T1=20.05	SAT (TO) f=17 TT=2400.07 T1=14.69	SAT (TO) f=17 TT=2520.12 T1=12.06	? (TO) TT=2519.82	? (TO) TT=2520.05	? (TO) TT=2520.05	SAT (TO) f=17 TT=2400.15 T1=462.95	No data	No data	No data	No data
Bacp/Bacp-m2/ Bacp-m2-12a_c18.xml	SAT f=17 TT=2400.23 T1=1.7	No data	No data	No data	No data	SAT (TO) f=17 TT=2400.07 T1=5.61	SAT (TO) f=17 TT=2520.08 T1=5.52	SAT (TO) f=17 TT=2520.08 T1=14.02	SAT (TO) f=17 TT=2520.08 T1=13.6	SAT (TO) f=17 TT=2520.09 T1=11.46	SAT (TO) f=17 TT=2520.09 T1=26.84	? (TO) TT=2519.91	No data	No data	Wrong UNSAT TT=0.087	SAT (TO) f=18 TT=2519.65 T1=0.11	No data	No data	SAT (TO) f=17 TT=2400.11 T1=1.79	SAT (TO) f=17 TT=2520.13 T1=1.7	SAT (TO) f=17 TT=2520.05 T1=1.71	? (TO) TT=2400.04	? (TO) TT=2520.08	? (TO) TT=2520.07	? (TO) TT=2519.93	? (TO) TT=2520.04	SAT (TO) f=17 TT=2400.23 T1=472.89	No data	No data	No data	No data
Bacp/Bacp-m2/ Bacp-m2-12b_c18.xml	SAT f=17 TT=2401.24 T1=0.07	No data	No data	No data	No data	SAT (TO) f=17 TT=2400.06 T1=5.87	SAT (TO) f=17 TT=2520.11 T1=6.16	SAT (TO) f=17 TT=2520.07 T1=12.42	SAT (TO) f=17 TT=2520.18 T1=12.34	SAT (TO) f=17 TT=2520.1 T1=10.46	SAT (TO) f=17 TT=2520.14 T1=16.5	SAT (TO) f=17 TT=2519.98 T1=0.07	No data	No data	Wrong UNSAT TT=0.086	SAT (TO) f=18 TT=2520.03 T1=0.11	No data	No data	SAT (TO) f=17 TT=2400.04 T1=1.76	SAT (TO) f=17 TT=2520.07 T1=1.7	SAT (TO) f=17 TT=2520.12 T1=1.69	SAT (TO) f=17 TT=2400.12 T1=1.77	SAT (TO) f=17 TT=2520.04 T1=1.68	? (TO) TT=2520.06	? (TO) TT=2519.83	? (TO) TT=2519.93	SAT (TO) f=17 TT=2401.24 T1=913.89	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
Bibd/ Bibd-sc-07-03-01_c18.xml	SAT TT=0.049	? (NS) TT=0.028	? (NS) TT=0.029	? (NS) TT=0.028	? (NS) TT=0.029	SAT TT=1.433	SAT TT=1.423	SAT TT=9.118	SAT TT=9.622	SAT TT=9.3	SAT TT=9.456	SAT TT=0.109	? (NS) TT=0.028	? (NS) TT=0.028	SAT TT=0.063	SAT TT=0.063	SAT TT=2.602	SAT TT=2.153	SAT TT=2.695	SAT TT=2.353	SAT TT=2.289	No data	No data	SAT TT=0.049	SAT TT=0.049	SAT TT=0.049	SAT TT=1.956	SAT TT=4.741	SAT TT=6.11	SAT TT=4.682	SAT TT=6.192
Bibd/ Bibd-sc-09-03-09_c18.xml	SAT TT=5.901	? (NS) TT=1.258	? (NS) TT=1.277	? (NS) TT=1.259	? (NS) TT=1.281	? (TO) TT=2400.07	? (TO) TT=2520.1	? (TO) TT=2520.13	? (TO) TT=2520.09	? (TO) TT=2520.06	? (TO) TT=2520.15	? (TO) TT=2519.94	? (NS) TT=1.273	? (NS) TT=1.259	SAT TT=6.263	SAT TT=5.901	? (TO) TT=2400.04	? (TO) TT=2520.09	? (TO) TT=2400.03	? (TO) TT=2520.1	? (TO) TT=2520.04	No data	No data	SAT TT=7.071	SAT TT=7.153	SAT TT=9.119	SAT TT=1360.6	SAT TT=40.768	SAT TT=44.263	SAT TT=41.532	SAT TT=44.601
Bibd/ Bibd-sc-10-03-08_c18.xml	SAT TT=14.044	? (NS) TT=2.105	? (NS) TT=2.092	? (NS) TT=2.049	? (NS) TT=2.108	? (TO) TT=2400.09	? (TO) TT=2520.1	SAT TT=113.656	SAT TT=1348.88	? (TO) TT=2520.08	? (TO) TT=2520.09	? (TO) TT=2520.02	? (NS) TT=2.074	? (NS) TT=2.059	SAT TT=75.874	SAT TT=25.329	? (TO) TT=2400.03	? (TO) TT=2520.12	? (TO) TT=2400.03	SAT TT=581.524	? (TO) TT=2520.08	No data	No data	SAT TT=14.132	SAT TT=14.318	SAT TT=14.044	? (TO) TT=2400.17	SAT TT=53.2	SAT TT=54.824	? (MO) TT=51.127	SAT TT=64.171
Bibd/ Bibd-sc-12-06-05_c18.xml	SAT TT=1.292	? (NS) TT=0.299	? (NS) TT=0.308	? (NS) TT=0.303	? (NS) TT=0.308	SAT TT=37.742	SAT TT=38.327	? (TO) TT=2520.08	SAT TT=1124.06	? (TO) TT=2520.08	? (TO) TT=2520.11	? (TO) TT=2519.97	? (NS) TT=0.303	? (NS) TT=0.303	SAT TT=4.892	SAT TT=4.9	? (TO) TT=2400.03	? (TO) TT=2520.05	? (TO) TT=2400.07	? (TO) TT=2520.09	? (TO) TT=2520.08	No data	No data	SAT TT=2.048	SAT TT=2.054	SAT TT=1.292	SAT TT=218.498	? (MO) TT=14.77	SAT TT=16.783	SAT TT=15.121	SAT TT=16.848
Bibd/ Bibd-sc-22-033-12-08-04_c18.xml		? (NS) TT=3.179	? (NS) TT=3.146	? (NS) TT=3.202	? (NS) TT=3.245	? (TO) TT=2400.12	? (TO) TT=2520.12	? (TO) TT=2520.13	? (TO) TT=2520.14	? (TO) TT=2520.07	? (TO) TT=2520.08	? (TO) TT=2519.93	? (NS) TT=3.273	? (NS) TT=3.081	? (TO) TT=2400.04	? (TO) TT=2520	? (TO) TT=2400.11	? (TO) TT=2520.1	? (TO) TT=2400.07	? (TO) TT=2520.03	? (TO) TT=2520.06	No data	No data	? (TO) TT=2520	? (TO) TT=2520.07	? (TO) TT=2519.81	? (TO) TT=2400.21	? (MO) TT=1789.51	? (TO) TT=2520.05	? (MO) TT=1787.52	? (TO) TT=2520.07
Bibd/ Bibd-sc-34-085-30-12-10_c18.xml		? (NS) TT=168.17	? (NS) TT=166.715	? (NS) TT=166.953	? (NS) TT=166.609	? (TO) TT=2400.05	? (TO) TT=2520.11	? (TO) TT=2520.06	? (TO) TT=2520.16	? (TO) TT=2520.13	? (TO) TT=2520.14	? (MO) TT=13.762	? (NS) TT=167.798	? (NS) TT=167.578	? (TO) TT=2400.06	? (TO) TT=2519.53	? (TO) TT=2400.1	? (TO) TT=2520.09	? (TO) TT=2400.05	? (TO) TT=2520.06	? (TO) TT=2520.08	No data	No data	? (TO) TT=2520.01	? (TO) TT=2519.83	? (TO) TT=2519.78	? (TO) TT=2400.22	? (MO) TT=113.076	? (TO) TT=2520.16	? (MO) TT=108.753	? (TO) TT=2520.14
Bibd/ Bibd-sum-07-03-01_c18.xml	SAT TT=0.013	? (NS) TT=0.008	? (NS) TT=0.008	? (NS) TT=0.008	? (NS) TT=0.009	SAT TT=1.066	SAT TT=1.091	SAT TT=8.593	SAT TT=8.694	SAT TT=8.811	SAT TT=9.05	SAT TT=1.205	? (NS) TT=0.008	? (NS) TT=0.008	SAT TT=0.013	SAT TT=0.014	SAT TT=1.759	SAT TT=1.631	SAT TT=1.907	SAT TT=1.8	SAT TT=1.796	No data	No data	SAT TT=0.046	SAT TT=0.047	SAT TT=0.047	SAT TT=1.442	SAT TT=4.091	SAT TT=5.258	SAT TT=4.039	SAT TT=5.282
Bibd/ Bibd-sum-09-03-09_c18.xml	SAT TT=6	? (NS) TT=0.427	? (NS) TT=0.429	? (NS) TT=0.427	? (NS) TT=0.431	? (TO) TT=2400.03	? (TO) TT=2520.07	? (TO) TT=2520.1	? (TO) TT=2520.06	? (TO) TT=2520.08	? (TO) TT=2520.26	? (TO) TT=2520.02	? (NS) TT=0.428	? (NS) TT=0.424	SAT TT=22.083	SAT TT=12.704	? (TO) TT=2400.08	? (TO) TT=2520.11	? (TO) TT=2400.05	? (TO) TT=2520.07	? (TO) TT=2520.02	No data	No data	SAT TT=6	SAT TT=6.035	SAT TT=7.204	? (TO) TT=2400.1	SAT TT=35.025	SAT TT=37.346	SAT TT=35.028	SAT TT=37.346
Bibd/ Bibd-sum-10-03-08_c18.xml	SAT TT=10.895	? (NS) TT=0.756	? (NS) TT=0.763	? (NS) TT=0.756	? (NS) TT=0.797	? (TO) TT=2400.06	? (TO) TT=2520.07	SAT TT=352.146	? (TO) TT=2520.12	? (TO) TT=2520.07	SAT TT=702.93	? (TO) TT=2520.08	? (NS) TT=0.762	? (NS) TT=0.757	SAT TT=432.654	SAT TT=29.209	? (TO) TT=2400.1	? (TO) TT=2520.06	? (TO) TT=2400.04	? (TO) TT=2520.08	? (TO) TT=2520.11	No data	No data	SAT TT=10.895	SAT TT=10.91	SAT TT=13.71	? (TO) TT=2400.17	SAT TT=45.719	SAT TT=49.481	? (MO) TT=44.171	SAT TT=57.261
Bibd/ Bibd-sum-12-06-05_c18.xml	SAT TT=1.23	? (NS) TT=0.067	? (NS) TT=0.068	? (NS) TT=0.067	? (NS) TT=0.068	SAT TT=1280.81	SAT TT=1306.34	? (TO) TT=2520.06	? (TO) TT=2520.16	? (TO) TT=2520.13	? (TO) TT=2520.07	? (TO) TT=2520.1	? (NS) TT=0.067	? (NS) TT=0.067	SAT TT=1.769	SAT TT=1.872	? (TO) TT=2400.08	? (TO) TT=2520.11	? (TO) TT=2400.08	? (TO) TT=2520.08	? (TO) TT=2520.03	No data	No data	SAT TT=2.771	SAT TT=2.797	SAT TT=1.23	SAT TT=73.425	SAT TT=13.862	SAT TT=13.971	SAT TT=13.77	SAT TT=14.078
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
Bibd/ Bibd-sum-22-033-12-08-04_c18.xml		? (NS) TT=1.269	? (NS) TT=1.266	? (NS) TT=1.262	? (NS) TT=1.276	? (TO) TT=2400.07	? (TO) TT=2520.04	? (TO) TT=2520.08	? (TO) TT=2520.16	? (TO) TT=2520.12	? (TO) TT=2520.13	? (TO) TT=2520.02	? (NS) TT=1.411	? (NS) TT=1.263	? (TO) TT=2400.01	? (TO) TT=2520.04	? (TO) TT=2400.07	? (TO) TT=2520.11	? (TO) TT=2400.04	? (TO) TT=2520.09	? (TO) TT=2520.02	No data	No data	? (TO) TT=2520.02	? (TO) TT=2519.89	? (TO) TT=2520.01	? (TO) TT=2400.2	? (TO) TT=2520.11	? (TO) TT=2520.15	? (TO) TT=2520.11	? (TO) TT=2520.13
Bibd/ Bibd-sum-34-085-30-12-10_c18.xml		? (NS) TT=99.786	? (NS) TT=104.006	? (NS) TT=98.323	? (NS) TT=94.963	? (TO) TT=2400.05	? (TO) TT=2520.1	? (TO) TT=2520.11	? (TO) TT=2520.1	? (TO) TT=2520.16	? (TO) TT=2520.1	? (TO) TT=2519.98	? (NS) TT=94.634	? (NS) TT=95.406	? (TO) TT=2400.04	? (TO) TT=2519.81	? (TO) TT=2400.04	? (TO) TT=2520.05	? (TO) TT=2400.06	? (TO) TT=2520.08	? (TO) TT=2520.08	No data	No data	? (TO) TT=2519.83	? (TO) TT=2519.94	? (TO) TT=2520.06	? (TO) TT=2400.13	? (MO) TT=93.944	? (TO) TT=2520.2	? (MO) TT=89.418	? (TO) TT=2520.18
CarSequencing/ CarSequencing-04-72_c18.xml	SAT TT=18.681	? (NS) TT=0.008	? (NS) TT=0.008	? (NS) TT=0.009	? (NS) TT=0.009	? (TO) TT=2400.13	? (TO) TT=2520.09	? (TO) TT=2520.08	? (TO) TT=2520.11	? (TO) TT=2520.13	? (TO) TT=2520.14	? (TO) TT=2519.92	? (NS) TT=0.008	? (NS) TT=0.008	SAT TT=2366.84	? (TO) TT=2519.95	? (TO) TT=2400.02	? (TO) TT=2520.04	? (TO) TT=2400.04	? (TO) TT=2520.04	? (TO) TT=2520.08	No data	No data	SAT TT=18.77	SAT TT=18.801	SAT TT=18.681	SAT TT=1603.45	SAT TT=42.384	SAT TT=66.55	SAT TT=41.612	SAT TT=66.247
CarSequencing/ CarSequencing-06-76_c18.xml	UNSAT TT=110.197	? (NS) TT=0.008	? (NS) TT=0.009	? (NS) TT=0.009	? (NS) TT=0.009	? (TO) TT=2400.12	? (TO) TT=2520.11	? (TO) TT=2520.07	? (TO) TT=2520.17	? (TO) TT=2520.13	? (TO) TT=2520.14	? (TO) TT=2519.9	? (NS) TT=0.008	? (NS) TT=0.008	? (TO) TT=2400.07	? (TO) TT=2520	? (TO) TT=2400.09	? (TO) TT=2520.06	? (TO) TT=2400.1	? (TO) TT=2520.1	? (TO) TT=2520.04	No data	No data	UNSAT TT=343.052	UNSAT TT=343.194	UNSAT TT=573.628	? (TO) TT=2400.54	UNSAT TT=118.467	UNSAT TT=111.829	UNSAT TT=118.754	UNSAT TT=110.197
CarSequencing/ CarSequencing-60-01_c18.xml	SAT TT=20.494	? (NS) TT=0.011	? (NS) TT=0.011	? (NS) TT=0.011	? (NS) TT=0.011	SAT TT=395.828	SAT TT=388.102	? (TO) TT=2520.12	? (TO) TT=2520.78	SAT TT=822.972	? (TO) TT=2520.09	? (TO) TT=2519.95	? (NS) TT=0.011	? (NS) TT=0.011	? (TO) TT=2400.1	SAT TT=2300.57	? (TO) TT=2400.11	? (TO) TT=2520.1	SAT TT=20.557	SAT TT=25.542	SAT TT=20.494	No data	No data	SAT TT=42.697	SAT TT=43.022	SAT TT=42.475	? (TO) TT=2400.11	? (MO) TT=41.514	SAT TT=846.345	? (MO) TT=37.23	SAT TT=45.417
CarSequencing/ CarSequencing-60-02_c18.xml	SAT TT=0.228	? (NS) TT=0.01	? (NS) TT=0.011	? (NS) TT=0.011	? (NS) TT=0.012	SAT TT=5.336	SAT TT=5.075	? (TO) TT=2520.17	? (TO) TT=2520.16	SAT TT=370.009	SAT TT=1575.56	SAT TT=1.265	? (NS) TT=0.011	? (NS) TT=0.011	SAT TT=0.228	SAT TT=1.074	? (TO) TT=2400.04	? (TO) TT=2520.08	? (TO) TT=2400.02	? (TO) TT=2520.03	? (TO) TT=2520.02	No data	No data	SAT TT=22.815	SAT TT=23.024	SAT TT=24.865	? (TO) TT=2400.23	? (MO) TT=36.038	SAT TT=805.862	? (MO) TT=35.731	SAT TT=42.374
CarSequencing/ CarSequencing-65-01_c18.xml	SAT TT=46.409	? (NS) TT=0.011	? (NS) TT=0.011	? (NS) TT=0.012	? (NS) TT=0.011	SAT TT=431.61	SAT TT=504.568	? (TO) TT=2520.22	? (TO) TT=2520.23	SAT TT=2064.3	? (TO) TT=2520.15	? (TO) TT=2519.94	? (NS) TT=0.011	? (NS) TT=0.011	? (TO) TT=2400.07	? (TO) TT=2519.91	? (TO) TT=2400.08	? (TO) TT=2520.11	SAT TT=30.189	SAT TT=302.898	SAT TT=189.728	No data	No data	SAT TT=56.188	SAT TT=56.302	SAT TT=57.255	? (TO) TT=2400.22	? (MO) TT=41.887	SAT TT=1130.78	? (MO) TT=41.286	SAT TT=46.409
CarSequencing/ CarSequencing-65-02_c18.xml	SAT TT=0.252	? (NS) TT=0.011	? (NS) TT=0.011	? (NS) TT=0.011	? (NS) TT=0.012	SAT TT=4.939	SAT TT=5.334	SAT TT=1412.21	? (TO) TT=2520.16	SAT TT=285.211	SAT TT=283.045	? (TO) TT=2519.6	? (NS) TT=0.011	? (NS) TT=0.011	SAT TT=0.252	SAT TT=0.252	? (TO) TT=2400.05	? (TO) TT=2520.06	SAT TT=146.426	? (TO) TT=2520.03	? (TO) TT=2520.02	No data	No data	SAT TT=37.121	SAT TT=37.272	SAT TT=44.755	? (TO) TT=2402.63	? (MO) TT=37.801	SAT TT=1077.03	? (MO) TT=35.43	SAT TT=46.428
CarSequencing/ CarSequencing-70-01_c18.xml	SAT TT=5.095	? (NS) TT=0.011	? (NS) TT=0.011	? (NS) TT=0.011	? (NS) TT=0.011	SAT TT=131.53	SAT TT=129.746	? (TO) TT=2520.25	SAT TT=1222.22	? (TO) TT=2520.15	? (TO) TT=2520.54	? (TO) TT=2519.95	? (NS) TT=0.011	? (NS) TT=0.011	SAT TT=5.095	? (TO) TT=2519.78	? (TO) TT=2400.08	? (TO) TT=2520.09	SAT TT=29.262	SAT TT=21.346	SAT TT=72.918	No data	No data	SAT TT=47.336	SAT TT=47.526	SAT TT=64.956	? (TO) TT=2400.15	? (MO) TT=42.822	SAT TT=596.461	? (MO) TT=41.182	SAT TT=48.341
CarSequencing/ CarSequencing-70-02_c18.xml	SAT TT=0.329	? (NS) TT=0.011	? (NS) TT=0.011	? (NS) TT=0.012	? (NS) TT=0.012	SAT TT=16.416	SAT TT=16.809	? (TO) TT=2520.11	? (TO) TT=2520.16	SAT TT=522.271	SAT TT=410.074	? (TO) TT=2519.94	? (NS) TT=0.011	? (NS) TT=0.011	SAT TT=0.329	SAT TT=1.206	? (TO) TT=2400.08	? (TO) TT=2520.07	? (TO) TT=2400.03	? (TO) TT=2520.1	? (TO) TT=2520.1	No data	No data	SAT TT=46.392	SAT TT=46.801	SAT TT=41.824	? (TO) TT=2400.14	? (MO) TT=42.014	SAT TT=808.362	? (MO) TT=38.623	SAT TT=54.086
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
CarSequencing/ CarSequencing-75-01_c18.xml	SAT TT=35.195	? (NS) TT=0.011	? (NS) TT=0.011	? (NS) TT=0.011	? (NS) TT=0.011	SAT TT=35.697	SAT TT=35.195	? (TO) TT=2520.13	SAT TT=1135.04	? (TO) TT=2520.12	? (TO) TT=2520.11	? (TO) TT=2519.94	? (NS) TT=0.011	? (NS) TT=0.011	? (TO) TT=2400.04	? (TO) TT=2519.73	? (TO) TT=2400.03	? (TO) TT=2520.09	SAT TT=90.689	SAT TT=60.473	SAT TT=90.736	No data	No data	SAT TT=104.811	SAT TT=105.08	SAT TT=69.14	? (TO) TT=2400.18	? (MO) TT=45.061	SAT TT=1749.15	? (MO) TT=39.308	SAT TT=50.379
CarSequencing/ CarSequencing-75-02_c18.xml	SAT TT=2.305	? (NS) TT=0.011	? (NS) TT=0.011	? (NS) TT=0.012	? (NS) TT=0.011	SAT TT=5.178	SAT TT=5.155	? (TO) TT=2520.12	SAT TT=1428.64	SAT TT=291.283	SAT TT=489.807	SAT TT=7.563	? (NS) TT=0.011	? (NS) TT=0.011	? (TO) TT=2400.01	SAT TT=2.305	? (TO) TT=2400.03	? (TO) TT=2520.02	SAT TT=12.055	SAT TT=11.801	SAT TT=12.089	No data	No data	SAT TT=41.099	SAT TT=41.569	SAT TT=50.345	? (TO) TT=2400.19	? (MO) TT=43.923	SAT TT=1318.1	? (MO) TT=39.682	SAT TT=79.991
CarSequencing/ CarSequencing-80-01_c18.xml	SAT TT=48.332	? (NS) TT=0.011	? (NS) TT=0.011	? (NS) TT=0.012	? (NS) TT=0.011	? (TO) TT=2400.08	? (TO) TT=2520.14	? (TO) TT=2520.13	? (TO) TT=2520.2	? (TO) TT=2520.12	? (TO) TT=2520.13	? (TO) TT=2519.92	? (NS) TT=0.011	? (NS) TT=0.011	? (TO) TT=2400.05	? (TO) TT=2519.93	? (TO) TT=2400.04	? (TO) TT=2520.04	? (TO) TT=2400.03	? (TO) TT=2520.03	SAT TT=48.332	No data	No data	SAT TT=104.977	SAT TT=105.626	SAT TT=120.963	? (TO) TT=2400.15	? (MO) TT=44.583	SAT TT=2449.31	? (MO) TT=40.128	SAT TT=67.462
CarSequencing/ CarSequencing-80-02_c18.xml	SAT TT=19.669	? (NS) TT=0.011	? (NS) TT=0.011	? (NS) TT=0.011	? (NS) TT=0.011	SAT TT=19.669	SAT TT=20.923	? (TO) TT=2520.27	? (TO) TT=2520.19	SAT TT=319.037	? (TO) TT=2520.18	? (TO) TT=2519.93	? (NS) TT=0.011	? (NS) TT=0.011	? (TO) TT=2400.1	? (TO) TT=2519.89	? (TO) TT=2400.09	? (TO) TT=2520.1	? (TO) TT=2400.07	? (TO) TT=2520.08	? (TO) TT=2520.11	No data	No data	SAT TT=76.231	SAT TT=76.579	SAT TT=67.799	? (TO) TT=2400.18	? (MO) TT=45.028	SAT TT=1697.71	? (MO) TT=39.541	SAT TT=90.498
CarSequencing/ CarSequencing-85-01_c18.xml	SAT TT=84.343	? (NS) TT=0.011	? (NS) TT=0.011	? (NS) TT=0.011	? (NS) TT=0.011	? (TO) TT=2400.07	? (TO) TT=2520.12	? (TO) TT=2520.29	? (TO) TT=2521.05	? (TO) TT=2520.15	? (TO) TT=2520.11	? (TO) TT=2519.9	? (NS) TT=0.011	? (NS) TT=0.011	? (TO) TT=2400.02	? (TO) TT=2519.94	? (TO) TT=2400.11	? (TO) TT=2520.07	? (TO) TT=2400.06	? (TO) TT=2520.04	? (TO) TT=2520.05	No data	No data	SAT TT=126.99	SAT TT=127.896	SAT TT=116.963	? (TO) TT=2400.17	? (MO) TT=46.242	? (TO) TT=2520.09	? (MO) TT=41.83	SAT TT=84.343
CarSequencing/ CarSequencing-85-02_c18.xml	SAT TT=105.226	? (NS) TT=0.011	? (NS) TT=0.011	? (NS) TT=0.012	? (NS) TT=0.012	SAT TT=1752.48	SAT TT=1777.83	? (TO) TT=2520.17	? (TO) TT=2520.23	? (TO) TT=2520.11	? (TO) TT=2520.11	SAT TT=1926.4	? (NS) TT=0.011	? (NS) TT=0.011	? (TO) TT=2400.06	? (TO) TT=2520.05	? (TO) TT=2400.02	? (TO) TT=2520.06	? (TO) TT=2400.05	? (TO) TT=2520.03	? (TO) TT=2520.06	No data	No data	SAT TT=122.881	SAT TT=124.825	SAT TT=138.787	? (TO) TT=2400.18	? (MO) TT=46.034	? (TO) TT=2520.14	? (MO) TT=39.603	SAT TT=105.226
CarSequencing/ CarSequencing-90-01_c18.xml	SAT TT=156.426	? (NS) TT=0.011	? (NS) TT=0.011	? (NS) TT=0.011	? (NS) TT=0.011	? (TO) TT=2400.1	? (TO) TT=2520.09	? (TO) TT=2520.26	? (TO) TT=2520.12	? (TO) TT=2520.16	? (TO) TT=2520.36	? (TO) TT=2519.96	? (NS) TT=0.011	? (NS) TT=0.011	? (TO) TT=2400.07	? (TO) TT=2520.07	? (TO) TT=2400.02	? (TO) TT=2520.11	? (TO) TT=2400.06	? (TO) TT=2520.04	? (TO) TT=2520.11	No data	No data	SAT TT=180.896	SAT TT=181.464	SAT TT=156.426	? (TO) TT=2400.11	? (MO) TT=46.955	? (TO) TT=2520.18	? (MO) TT=41.902	SAT TT=446.387
CarSequencing/ CarSequencing-90-02_c18.xml	SAT TT=129.751	? (NS) TT=0.011	? (NS) TT=0.011	? (NS) TT=0.012	? (NS) TT=0.012	? (TO) TT=2400.12	? (TO) TT=2520.09	? (TO) TT=2520.14	? (TO) TT=2520.19	? (TO) TT=2520.1	? (TO) TT=2520.09	? (TO) TT=2519.95	? (NS) TT=0.011	? (NS) TT=0.011	? (TO) TT=2400.05	? (TO) TT=2519.75	? (TO) TT=2400.07	? (TO) TT=2520.1	? (TO) TT=2400.07	? (TO) TT=2520.11	? (TO) TT=2520.06	No data	No data	SAT TT=129.751	SAT TT=130.246	SAT TT=246.37	? (TO) TT=2400.18	? (MO) TT=46.155	? (TO) TT=2520.12	? (MO) TT=41.542	SAT TT=212.261
CarSequencing/ CarSequencing-dingbas_c18.xml	SAT TT=0.009	? (NS) TT=0.006	? (NS) TT=0.006	? (NS) TT=0.007	? (NS) TT=0.006	SAT TT=1	SAT TT=1.008	SAT TT=9.652	SAT TT=9.355	SAT TT=7.926	SAT TT=8.44	SAT TT=0.009	? (NS) TT=0.006	? (NS) TT=0.006	SAT TT=0.01	SAT TT=0.011	SAT TT=1.712	SAT TT=1.708	SAT TT=1.78	SAT TT=1.77	SAT TT=1.793	No data	No data	SAT TT=0.065	SAT TT=0.064	SAT TT=0.064	SAT TT=1.339	SAT TT=3.953	SAT TT=5.341	SAT TT=3.817	SAT TT=5.344
ColouredQueens/ ColouredQueens-05_c18.xml	SAT TT=0.005	SAT TT=0.006	SAT TT=0.007	SAT TT=0.007	SAT TT=0.007	SAT TT=0.859	SAT TT=0.857	SAT TT=6.459	SAT TT=6.311	SAT TT=5.791	SAT TT=5.818	SAT TT=0.005	SAT TT=0.007	SAT TT=0.007	SAT TT=0.006	SAT TT=0.006	SAT TT=1.566	SAT TT=1.592	SAT TT=1.549	SAT TT=1.558	SAT TT=1.578	No data	No data	SAT TT=0.016	SAT TT=0.016	SAT TT=0.015	SAT TT=0.589	SAT TT=2.929	SAT TT=4.106	SAT TT=2.885	SAT TT=4.069
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
ColouredQueens/ ColouredQueens-06_c18.xml	UNSAT TT=0.016	UNSAT TT=0.589	UNSAT TT=0.593	UNSAT TT=0.864	UNSAT TT=0.867	UNSAT TT=1.823	UNSAT TT=1.842	UNSAT TT=10.409	UNSAT TT=11.127	UNSAT TT=10.217	UNSAT TT=9.297	UNSAT TT=2.028	UNSAT TT=0.815	UNSAT TT=0.797	UNSAT TT=0.016	UNSAT TT=0.02	UNSAT TT=1.878	UNSAT TT=1.873	? (TO) TT=2400.04	? (TO) TT=2520.05	UNSAT TT=1.785	No data	No data	UNSAT TT=0.027	UNSAT TT=0.027	UNSAT TT=0.027	UNSAT TT=4.21	UNSAT TT=3.273	UNSAT TT=4.409	UNSAT TT=3.33	UNSAT TT=4.584
ColouredQueens/ ColouredQueens-07_c18.xml	SAT TT=0.007	SAT TT=0.016	SAT TT=0.016	SAT TT=0.008	SAT TT=0.008	SAT TT=1.036	SAT TT=1.054	SAT TT=6.945	SAT TT=6.494	SAT TT=6.736	SAT TT=7.019	SAT TT=0.065	SAT TT=0.008	SAT TT=0.008	SAT TT=0.009	SAT TT=0.007	SAT TT=1.679	SAT TT=1.715	SAT TT=1.71	SAT TT=1.691	SAT TT=1.676	No data	No data	SAT TT=0.038	SAT TT=0.038	SAT TT=0.038	SAT TT=0.756	SAT TT=3.965	SAT TT=5.161	SAT TT=3.863	SAT TT=5.164
ColouredQueens/ ColouredQueens-08_c18.xml	UNSAT TT=0.18	? (TO) TT=2519.92	? (TO) TT=2519.98	? TT=2520.01	? (TO) TT=2520	? (TO) TT=2400.11	? (TO) TT=2520.35	? (TO) TT=2520.14	? (TO) TT=2520.14	? (TO) TT=2520.14	? (TO) TT=2520.11	? (TO) TT=2519.94	? (TO) TT=2519.77	? (TO) TT=2519.98	UNSAT TT=219.755	UNSAT TT=215.174	? (TO) TT=2400.04	? (TO) TT=2520.07	? (TO) TT=2400.09	? (TO) TT=2520.03	? (TO) TT=2520.1	No data	No data	UNSAT TT=0.181	UNSAT TT=0.183	UNSAT TT=0.18	? (TO) TT=2400.16	UNSAT TT=1213.14	? (TO) TT=2520.09	UNSAT TT=1192.23	? (TO) TT=2520.06
ColouredQueens/ ColouredQueens-09_c18.xml		? (TO) TT=2519.91	? TT=2520.01	? (TO) TT=2519.93	? (TO) TT=2519.92	? (TO) TT=2400.12	? (TO) TT=2520.06	? (TO) TT=2520.05	? (TO) TT=2520.14	? (TO) TT=2520.05	? (TO) TT=2520.12	? (TO) TT=2520.02	? TT=2520.01	? (TO) TT=2519.99	? (TO) TT=2400.07	? (TO) TT=2519.92	? (TO) TT=2400.04	? (TO) TT=2520.05	? (TO) TT=2400.09	? (TO) TT=2520.05	? (TO) TT=2520.12	No data	No data	? (TO) TT=2519.93	? (TO) TT=2519.78	? (TO) TT=2519.99	? (TO) TT=2400.08	? (TO) TT=2520.05	? (TO) TT=2520.15	? (TO) TT=2520.05	? (TO) TT=2520.12
ColouredQueens/ ColouredQueens-10_c18.xml		? (TO) TT=2519.91	? (TO) TT=2519.84	? TT=2520.01	? TT=2520.01	? (TO) TT=2400.12	? (TO) TT=2520.13	? (TO) TT=2520.08	? (TO) TT=2520.08	? (TO) TT=2520.1	? (TO) TT=2520.08	? (TO) TT=2520.08	? (TO) TT=2519.96	? (TO) TT=2519.8	? (TO) TT=2400.1	? (TO) TT=2520.06	? (TO) TT=2400.04	? (TO) TT=2520.11	? (TO) TT=2400.1	? (TO) TT=2520.06	? (TO) TT=2520.06	No data	No data	? (TO) TT=2519.95	? (TO) TT=2520.06	? (TO) TT=2519.76	? (TO) TT=2400.07	? (TO) TT=2520.08	? (TO) TT=2520.08	? (TO) TT=2520.12	? (TO) TT=2520.14
ColouredQueens/ ColouredQueens-11_c18.xml		? (TO) TT=2519.91	? (TO) TT=2519.96	? TT=2520.01	? (TO) TT=2519.97	? (TO) TT=2400.12	? (TO) TT=2520.14	? (TO) TT=2520.13	? (TO) TT=2520.05	? (TO) TT=2520.11	? (TO) TT=2520.06	? (TO) TT=2520.04	? TT=2520.01	? (TO) TT=2519.92	? (TO) TT=2400.03	? (TO) TT=2519.78	? (TO) TT=2400.09	? (TO) TT=2520.04	? (TO) TT=2400.09	? (TO) TT=2520.08	? (TO) TT=2520.03	No data	No data	? (TO) TT=2520.06	? (TO) TT=2519.93	? (TO) TT=2519.93	? (TO) TT=2400.09	? (TO) TT=2520.1	? (TO) TT=2520.11	? (TO) TT=2520.08	? (TO) TT=2520.08
ColouredQueens/ ColouredQueens-12_c18.xml		? (TO) TT=2519.9	? (TO) TT=2519.95	? TT=2520.01	? (TO) TT=2519.95	? (TO) TT=2400.08	? (TO) TT=2520.08	? (TO) TT=2520.08	? (TO) TT=2520.12	? (TO) TT=2520.05	? (TO) TT=2520.12	? (TO) TT=2520.07	? TT=2520.01	? (TO) TT=2519.82	? (TO) TT=2400.04	? (TO) TT=2519.94	? (TO) TT=2400.05	? (TO) TT=2520.08	? (TO) TT=2400.04	? (TO) TT=2520.09	? (TO) TT=2520.1	No data	No data	? (TO) TT=2520.07	? (TO) TT=2519.97	? (TO) TT=2519.88	? (TO) TT=2400.1	? (TO) TT=2520.1	? (TO) TT=2520.05	? (TO) TT=2520.06	? (TO) TT=2520.16
ColouredQueens/ ColouredQueens-13_c18.xml		? (TO) TT=2519.94	? TT=2520.01	? TT=2520.01	? (TO) TT=2519.82	? (TO) TT=2400.1	? (TO) TT=2520.08	? (TO) TT=2520.09	? (TO) TT=2520.13	? (TO) TT=2520.13	? (TO) TT=2520.09	? (TO) TT=2519.98	? TT=2520.01	? (TO) TT=2519.97	? (TO) TT=2400.07	? (TO) TT=2519.29	? (TO) TT=2400.06	? (TO) TT=2520.07	? (TO) TT=2400.05	? (TO) TT=2520.08	? (TO) TT=2520.04	No data	No data	? (TO) TT=2519.63	? (TO) TT=2519.95	? (TO) TT=2519.83	? (TO) TT=2400.11	? (TO) TT=2520.12	? (TO) TT=2520.07	? (TO) TT=2520.05	? (TO) TT=2520.11
ColouredQueens/ ColouredQueens-14_c18.xml		? (TO) TT=2519.91	? TT=2520.01	? TT=2520.01	? (TO) TT=2519.99	? (TO) TT=2400.12	? (TO) TT=2520.07	? (TO) TT=2520.05	? (TO) TT=2520.08	? (TO) TT=2520.14	? (TO) TT=2520.09	? (TO) TT=2520.01	? TT=2520.01	? (TO) TT=2520	? (TO) TT=2400.06	? (TO) TT=2519.5	? (TO) TT=2400.04	? (TO) TT=2520.1	? (TO) TT=2400.08	? (TO) TT=2520.02	? (TO) TT=2520.09	No data	No data	? (TO) TT=2519.79	? (TO) TT=2520.06	? (TO) TT=2519.88	? (TO) TT=2400.08	? (TO) TT=2520.1	? (TO) TT=2520.11	? (TO) TT=2520.06	? (TO) TT=2520.08
ColouredQueens/ ColouredQueens-15_c18.xml		? (TO) TT=2519.93	? TT=2520.01	? (TO) TT=2519.96	? (TO) TT=2519.67	? (TO) TT=2400.09	? (TO) TT=2520.09	? (TO) TT=2520.05	? (TO) TT=2520.06	? (TO) TT=2520.05	? (TO) TT=2520.1	? (TO) TT=2519.96	? TT=2520.01	? (TO) TT=2519.99	? (TO) TT=2400.01	? (TO) TT=2519.83	? (TO) TT=2400.05	? (TO) TT=2520.11	? (TO) TT=2400.07	? (TO) TT=2520.11	? (TO) TT=2520.05	No data	No data	? (TO) TT=2519.71	? (TO) TT=2519.77	? (TO) TT=2519.77	? (TO) TT=2400.13	? (TO) TT=2520.06	? (TO) TT=2520.13	? (TO) TT=2520.05	? (TO) TT=2520.12
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
ColouredQueens/ ColouredQueens-16_c18.xml		? (TO) TT=2519.93	? (TO) TT=2519.84	? (TO) TT=2519.79	? TT=2520.01	? (TO) TT=2400.1	? (TO) TT=2520.03	? (TO) TT=2520.13	? (TO) TT=2520.1	? (TO) TT=2520.04	? (TO) TT=2520.08	? (TO) TT=2520.1	? (TO) TT=2519.87	? (TO) TT=2519.94	? (TO) TT=2400.06	? (TO) TT=2519.92	? (TO) TT=2400.03	? (TO) TT=2520.1	? (TO) TT=2400.08	? (TO) TT=2520.11	? (TO) TT=2520.07	No data	No data	? (TO) TT=2519.73	? (TO) TT=2519.76	? (TO) TT=2519.93	? (TO) TT=2400.12	? (TO) TT=2520.05	? (TO) TT=2520.11	? (TO) TT=2520.08	? (TO) TT=2520.12
Crossword/ Crossword-m18-ogd2008-vg-04-05_c18.xml	SAT TT=0.045	SAT TT=0.045	SAT TT=0.046	SAT TT=0.045	SAT TT=0.045	SAT TT=1.84	SAT TT=1.73	SAT TT=10.105	SAT TT=10.251	SAT TT=9.966	SAT TT=9.704	SAT TT=0.139	SAT TT=0.045	SAT TT=0.046	SAT TT=0.152	SAT TT=0.152	SAT TT=2.932	SAT TT=2.932	SAT TT=3.021	SAT TT=3.023	SAT TT=2.751	No data	No data	SAT TT=5.628	SAT TT=5.59	SAT TT=5.596	? (TO) TT=2400.17	SAT TT=39.607	SAT TT=45.271	SAT TT=39.1	SAT TT=41.398
Crossword/ Crossword-m18-ogd2008-vg-05-06_c18.xml	SAT TT=0.12	SAT TT=0.12	SAT TT=0.12	SAT TT=0.12	SAT TT=0.12	SAT TT=2.176	SAT TT=2.36	SAT TT=12.742	SAT TT=12.61	SAT TT=12.874	SAT TT=13.016	SAT TT=0.355	SAT TT=0.121	SAT TT=0.12	SAT TT=0.405	SAT TT=0.423	SAT TT=3.275	SAT TT=3.261	SAT TT=3.316	SAT TT=3.405	SAT TT=3.272	No data	No data	SAT TT=124.227	SAT TT=124.456	SAT TT=124.42	? (TO) TT=2400.12	SAT TT=114.813	SAT TT=118.556	SAT TT=117.469	SAT TT=117.101
Crossword/ Crossword-m18-ogd2008-vg-06-07_c18.xml	SAT TT=0.268	SAT TT=0.273	SAT TT=0.272	SAT TT=0.268	SAT TT=0.268	SAT TT=3.367	SAT TT=3.263	SAT TT=25.789	SAT TT=26.765	SAT TT=21.135	SAT TT=19.785	SAT TT=1.157	SAT TT=0.5	SAT TT=0.496	SAT TT=0.98	SAT TT=1.142	SAT TT=5.136	SAT TT=4.983	SAT TT=4.442	SAT TT=4.312	SAT TT=4.405	No data	No data	? (TO) TT=2520.04	? (TO) TT=2520.06	? (TO) TT=2520.04	? (TO) TT=2407.71	? (MO) TT=224.913	SAT TT=285.641	? (MO) TT=230.787	SAT TT=282.063
Crossword/ Crossword-m18-ogd2008-vg-07-08_c18.xml	SAT TT=40.017	SAT TT=40.3	SAT TT=40.017	SAT TT=683.174	SAT TT=682.129	SAT TT=130.038	SAT TT=130.572	? (TO) TT=2520.14	? (TO) TT=2520.1	? (TO) TT=2520.04	? (TO) TT=2520.11	SAT TT=949.108	SAT TT=576.168	SAT TT=572.951	? (TO) TT=2400.09	? (TO) TT=2519.87	? TT=2399.69	? (TO) TT=2520.13	SAT TT=2383.31	SAT TT=773.832	SAT TT=1753.09	No data	No data	? (TO) TT=2519.99	? (TO) TT=2520.05	? (TO) TT=2519.76	? TT=1608.99	? (MO) TT=443.122	? (TO) TT=2520.1	? (MO) TT=481.007	? (TO) TT=2520.13
Crossword/ Crossword-m18-ogd2008-vg-08-09_c18.xml		? (TO) TT=2520.03	? (TO) TT=2519.96	? (TO) TT=2519.98	? (TO) TT=2519.98	? (TO) TT=2400.03	? (TO) TT=2520.04	? (TO) TT=2520.08	? (TO) TT=2520.1	? (TO) TT=2520.05	? (TO) TT=2520.1	? (TO) TT=2519.96	? (TO) TT=2519.88	? (TO) TT=2519.85	? (TO) TT=2400.01	? (TO) TT=2520.02	? (TO) TT=2400.09	? (TO) TT=2520.11	? (TO) TT=2400.14	? (TO) TT=2520.06	? (TO) TT=2520.1	No data	No data	? (TO) TT=2519.68	? (TO) TT=2519.89	? (TO) TT=2519.83	? TT=2144.49	? (MO) TT=868.706	? (TO) TT=2520.13	? (MO) TT=862.43	? (TO) TT=2520.1
Crossword/ Crossword-m18-ogd2008-vg-09-10_c18.xml		? (TO) TT=2520.04	? (TO) TT=2520.08	? (TO) TT=2519.79	? (TO) TT=2520.06	? (TO) TT=2400.08	? (TO) TT=2520.12	? (TO) TT=2520.17	? (TO) TT=2520.17	? (TO) TT=2520.12	? (TO) TT=2520.11	? (TO) TT=2519.95	? (TO) TT=2519.46	? (TO) TT=2519.91	? (TO) TT=2400.06	? (TO) TT=2519.54	? (TO) TT=2400.08	? (TO) TT=2520.13	? (TO) TT=2400.1	? (TO) TT=2520.12	? (TO) TT=2520.05	No data	No data	? (TO) TT=2519.96	? (TO) TT=2520.07	? (TO) TT=2519.99	? TT=716.736	? (TO) TT=2528.28	? (TO) TT=2528.48	? (TO) TT=2528.34	? (TO) TT=2528.14
Crossword/ Crossword-m18-ogd2008-vg-10-11_c18.xml		? (TO) TT=2519.91	? (TO) TT=2519.96	? (TO) TT=2519.81	? (TO) TT=2520.05	? (TO) TT=2400.12	? (TO) TT=2520.08	? (TO) TT=2520.14	? (TO) TT=2520.1	? (TO) TT=2520.11	? (TO) TT=2520.14	? (TO) TT=2519.92	? (TO) TT=2519.76	? (TO) TT=2519.89	? (TO) TT=2400.03	? (TO) TT=2520.05	? (TO) TT=2400.05	? (TO) TT=2520.11	? (TO) TT=2400.07	? (TO) TT=2520.1	? (TO) TT=2520.12	No data	No data	? (TO) TT=2519.7	? (TO) TT=2519.99	? (TO) TT=2519.96	? TT=574.491	? (TO) TT=2528.54	? (TO) TT=2528.42	? (TO) TT=2528.31	? (TO) TT=2528.56
Crossword/ Crossword-m18-ogd2008-vg-11-12_c18.xml		? (TO) TT=2520.04	? (TO) TT=2520.04	? (TO) TT=2519.97	? (TO) TT=2519.82	? (TO) TT=2400.1	? (TO) TT=2520.09	? (TO) TT=2520.11	? (TO) TT=2520.15	? (TO) TT=2520.11	? (TO) TT=2520.17	? (TO) TT=2519.81	? (TO) TT=2519.87	? (TO) TT=2519.95	? (TO) TT=2400.05	? (TO) TT=2520.06	? (TO) TT=2400.12	? (TO) TT=2520.06	? (TO) TT=2400.13	? (TO) TT=2520.44	? (TO) TT=2520.06	No data	No data	? (TO) TT=2519.77	? (TO) TT=2519.91	? (TO) TT=2520.18	? TT=1357.54	? (TO) TT=2526.67	? (TO) TT=2520.92	? (TO) TT=2528.4	? (TO) TT=2528.14
Crossword/ Crossword-m18-ogd2008-vg-12-13_c18.xml		? (TO) TT=2519.91	? (TO) TT=2519.91	? (TO) TT=2519.97	? (TO) TT=2519.96	? (TO) TT=2400.04	? (TO) TT=2520.06	? (TO) TT=2520.28	? (TO) TT=2520.2	? (TO) TT=2520.17	? (TO) TT=2520.1	? (TO) TT=2519.79	? (TO) TT=2519.51	? (TO) TT=2519.74	? (TO) TT=2400.05	? (TO) TT=2519.88	? (TO) TT=2400.13	? (TO) TT=2520.12	? (TO) TT=2400.07	? (TO) TT=2520.05	? (TO) TT=2520.07	No data	No data	? (TO) TT=2519.99	? (TO) TT=2520.12	? (TO) TT=2519.59	? TT=1897.03	? (TO) TT=2528.25	? (TO) TT=2528.37	? (TO) TT=2528.3	? (TO) TT=2520.36
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
Crossword/ Crossword-m18-ogd2008-vg-13-14_c18.xml		? (TO) TT=2519.9	? (TO) TT=2519.88	? (TO) TT=2519.79	? (TO) TT=2519.95	? (TO) TT=2400.1	? (TO) TT=2520.12	? (TO) TT=2520.15	? (TO) TT=2520.16	? (TO) TT=2520.15	? (TO) TT=2520.13	? (TO) TT=2519.93	? (TO) TT=2519.5	? (TO) TT=2519.98	? (TO) TT=2400.02	? (TO) TT=2519.95	? (TO) TT=2400.1	? (TO) TT=2520.11	? (TO) TT=2400.06	? (TO) TT=2520.09	? (TO) TT=2520.1	No data	No data	? (TO) TT=2519.77	? (TO) TT=2519.86	? (TO) TT=2519.78	? TT=1726.22	? (TO) TT=2528.12	? (TO) TT=2528.24	? (TO) TT=2520.3	? (TO) TT=2528.43
Crossword/ Crossword-m18-ogd2008-vg-14-15_c18.xml		? (TO) TT=2520.03	? (TO) TT=2520.04	? (TO) TT=2519.98	? (TO) TT=2520.04	? (TO) TT=2400.05	? (TO) TT=2520.06	? (TO) TT=2520.08	? (TO) TT=2520.11	? (TO) TT=2520.11	? (TO) TT=2520.14	? (TO) TT=2519.96	? (TO) TT=2519.89	? (TO) TT=2519.74	? (TO) TT=2400.01	? (TO) TT=2519.92	? (TO) TT=2400.13	? (TO) TT=2520.06	? (TO) TT=2400.1	? (TO) TT=2520.11	? (TO) TT=2520.08	No data	No data	? (TO) TT=2519.93	? (TO) TT=2520.02	? (TO) TT=2520.04	? (TO) TT=2408.15	? (TO) TT=2528.36	? (TO) TT=2528.26	? (TO) TT=2525.24	? (TO) TT=2528.28
Crossword/ Crossword-m18-ogd2008-vg-15-16_c18.xml		? (TO) TT=2520.03	? (TO) TT=2519.78	? (TO) TT=2519.98	? (TO) TT=2520.05	? (TO) TT=2400.09	? (TO) TT=2520.06	? (TO) TT=2520.27	? (TO) TT=2520.2	? (TO) TT=2520.14	? (TO) TT=2520.05	? (TO) TT=2519.81	? (TO) TT=2520.02	? (TO) TT=2519.88	? (TO) TT=2400.09	? (TO) TT=2519.97	? (TO) TT=2400.06	? (TO) TT=2520.11	? (TO) TT=2400.11	? (TO) TT=2520.05	? TT=2519.99	No data	No data	? (TO) TT=2519.98	? (TO) TT=2519.78	? (TO) TT=2519.98	? TT=1974.23	? (TO) TT=2528.15	? (TO) TT=2528.37	? (TO) TT=2528.36	? (TO) TT=2528.25
Crossword/ Crossword-m18-ogd2008-vg-16-17_c18.xml	UNSAT TT=368.785	UNSAT TT=367.118	UNSAT TT=368.785	UNSAT TT=630.506	UNSAT TT=628.721	UNSAT TT=715.791	UNSAT TT=713.856	? (TO) TT=2520.09	? (TO) TT=2520.18	? (TO) TT=2520.1	? (TO) TT=2520.18	? (TO) TT=2519.93	UNSAT TT=749.849	UNSAT TT=747.785	? (TO) TT=2400.09	? (TO) TT=2519.9	? (TO) TT=2400.08	? (TO) TT=2520.07	? (TO) TT=2400.1	? (TO) TT=2520.03	? (TO) TT=2520.07	No data	No data	? (TO) TT=2519.64	? (TO) TT=2519.75	? (TO) TT=2519.88	? TT=2213.35	? (TO) TT=2528.21	? (TO) TT=2523.04	? (TO) TT=2525.57	? (TO) TT=2521.3
CrosswordDesign/ CrosswordDesign-03-4-rom_c18.xml	OPT f=12 TT=0.052 T1=0.05	No data	No data	No data	No data	OPT f=12 TT=1.794 T1=0.76	OPT f=12 TT=1.774 T1=0.77	OPT f=12 TT=9.886 T1=4.3	OPT f=12 TT=9.977 T1=4.34	OPT f=12 TT=9.485 T1=4.21	OPT f=12 TT=10.32 T1=4.4	OPT f=12 TT=0.052 T1=0.05	No data	No data	OPT f=12 TT=0.228 T1=0.21	OPT f=12 TT=0.234 T1=0.21	No data	No data	SAT (TO) f=12 TT=2400.13 T1=1.3	SAT (TO) f=12 TT=2520.06 T1=1.27	OPT f=12 TT=2.544 T1=1.25	OPT f=12 TT=2.675 T1=1.29	OPT f=12 TT=2.509 T1=1.25	OPT f=12 TT=12.981 T1=12.97	OPT f=12 TT=16.3 T1=16.3	OPT f=12 TT=16.282 T1=16.28	? (TO) TT=2400.13	No data	No data	No data	No data
CrosswordDesign/ CrosswordDesign-04-4-rom_c18.xml	OPT f=24 TT=0.401 T1=0.32	No data	No data	No data	No data	OPT f=24 TT=3.327 T1=1.5	OPT f=24 TT=3.672 T1=1.72	OPT f=24 TT=14.681 T1=6.28	OPT f=24 TT=16.28 T1=5.9	OPT f=24 TT=14.718 T1=5.9	OPT f=24 TT=14.533 T1=5.62	OPT f=24 TT=0.401 T1=0.32	No data	No data	OPT f=24 TT=13.313 T1=9.98	OPT f=24 TT=13.362 T1=10.04	No data	No data	SAT (TO) f=24 TT=2400.03 T1=1.72	SAT (TO) f=24 TT=2520.08 T1=1.65	OPT f=24 TT=4.391 T1=1.68	OPT f=24 TT=4.321 T1=1.64	OPT f=24 TT=4.23 T1=1.62	OPT f=24 TT=148.762 T1=148.77	OPT f=24 TT=144.827 T1=144.83	OPT f=24 TT=136.819 T1=136.81	? (TO) TT=2407.88	No data	No data	No data	No data
CrosswordDesign/ CrosswordDesign-05-4-rom_c18.xml	OPT f=38 TT=24.244 T1=10.83	No data	No data	No data	No data	OPT f=38 TT=65.942 T1=14.4	OPT f=38 TT=70.08 T1=16.78	OPT f=38 TT=195.377 T1=28.64	OPT f=38 TT=170.764 T1=128.49	OPT f=38 TT=149.943 T1=35	OPT f=38 TT=99.078 T1=17.94	OPT f=38 TT=24.244 T1=10.83	No data	No data	? (NS) TT=21.294	? (NS) TT=26.172	No data	No data	SAT (TO) f=38 TT=2400.1 T1=32.32	SAT (TO) f=38 TT=2520.11 T1=65.78	OPT f=38 TT=322.064 T1=38.07	SAT (TO) f=38 TT=2400.12 T1=87.65	SAT (TO) f=38 TT=2520.13 T1=53	? (TO) TT=2520.02	? (TO) TT=2519.93	? (TO) TT=2519.95	? (TO) TT=2408.2	No data	No data	No data	No data
CrosswordDesign/ CrosswordDesign-06-4-rom_c18.xml	SAT f=54 TT=2519.97 T1=131.44	No data	No data	No data	No data	SAT (TO) f=54 TT=2400.1 T1=316.53	SAT (TO) f=54 TT=2520.14 T1=317.89	SAT (TO) f=54 TT=2520.43 T1=131.44	SAT (TO) f=48 TT=2520.13 T1=359.1	SAT (TO) f=48 TT=2520.21 T1=1040.62	SAT (TO) f=54 TT=2520.31 T1=370.44	SAT (TO) f=54 TT=2519.97 T1=1578.49	No data	No data	? (MO) TT=27.466	? (MO) TT=28.612	No data	No data	SAT (TO) f=43 TT=2400.08 T1=932.67	SAT (TO) f=43 TT=2524.1 T1=2338.82	SAT (TO) f=41 TT=2520.11 T1=20.63	SAT (TO) f=50 TT=2400.14 T1=611.01	SAT (TO) f=52 TT=2520.05 T1=1098.85	? (exit code) TT=219.334	? (TO) TT=2520.19	? (TO) TT=2520.13	? (TO) TT=2408.15	No data	No data	No data	No data
CrosswordDesign/ CrosswordDesign-07-4-rom_c18.xml	SAT f=55 TT=2520.01 T1=234.03	No data	No data	No data	No data	? TT=187.379	? TT=194.512	SAT (TO) f=53 TT=2520.34 T1=1461.26	SAT (TO) f=55 TT=2520.25 T1=234.03	SAT (TO) f=60 TT=2520.38 T1=1968.22	SAT (TO) f=57 TT=2520.23 T1=361.56	SAT (TO) f=55 TT=2520.01 T1=2424.33	No data	No data	? (NS) TT=15.589	? (NS) TT=16.145	No data	No data	? TT=72.347	? TT=81.647	? TT=73.377	? TT=72.808	? TT=72.709	? (exit code) TT=90.993	? (MO) TT=1574.34	? (MO) TT=1502.02	? TT=1595.95	No data	No data	No data	No data
CrosswordDesign/ CrosswordDesign-08-4-rom_c18.xml	SAT f=70 TT=2520.2 T1=2295.61	No data	No data	No data	No data	? TT=224.357	? TT=254.319	SAT (TO) f=70 TT=2520.2 T1=2295.61	SAT (TO) f=67 TT=2520.32 T1=174.47	SAT (TO) f=66 TT=2520.52 T1=720.35	SAT (TO) f=63 TT=2520.47 T1=464.77	SAT (TO) f=4 TT=2520.07 T1=59.6	No data	No data	? (NS) TT=16.755	? (NS) TT=16.817	No data	No data	? TT=70.534	? TT=66.928	? TT=63.947	? TT=64.601	? TT=62.692	? (exit code) TT=1341.36	? (exit code) TT=85.965	? (exit code) TT=67.891	? (TO) TT=2407.54	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
CrosswordDesign/ CrosswordDesign-09-4-rom_c18.xml	SAT f=77 TT=2520.58 T1=1999.76	No data	No data	No data	No data	? TT=347.512	? TT=402.979	SAT (TO) f=77 TT=2520.58 T1=1999.76	SAT (TO) f=76 TT=2520.19 T1=1160.2	SAT (TO) f=79 TT=2520.09 T1=1468.55	SAT (TO) f=76 TT=2520.24 T1=1853.9	? (TO) TT=2520.08	No data	No data	? (NS) TT=17.205	? (NS) TT=1.851	No data	No data	? TT=73.293	? TT=69.563	? TT=70.065	? TT=68.243	? TT=68.914	? (exit code) TT=80.134	? (exit code) TT=81.876	? (exit code) TT=68.101	? (TO) TT=2400.39	No data	No data	No data	No data
CrosswordDesign/ CrosswordDesign-10-4-rom_c18.xml	SAT f=84 TT=2520.4 T1=2105.71	No data	No data	No data	No data	? TT=201.607	? TT=219.208	SAT (TO) f=81 TT=2520.59 T1=1826.4	SAT (TO) f=84 TT=2520.4 T1=2105.71	? TT=310.371	SAT (TO) f=74 TT=2520.35 T1=2334.62	? (TO) TT=2519.87	No data	No data	? (NS) TT=17.648	? (NS) TT=2.488	No data	No data	? TT=76.604	? TT=69.759	? TT=72.092	? TT=75.306	? TT=69.148	? (exit code) TT=69.58	? (exit code) TT=74.635	? (exit code) TT=69.092	? (TO) TT=2400.36	No data	No data	No data	No data
CrosswordDesign/ CrosswordDesign-11-4-rom_c18.xml	SAT f=65 TT=2520.1 T1=1911.15	No data	No data	No data	No data	? TT=297.566	? TT=271.533	? TT=1028.88	? TT=746.242	? TT=328.036	? TT=957.206	SAT (TO) f=65 TT=2520.1 T1=1911.15	No data	No data	? (MO) TT=18.664	? (NS) TT=3.084	No data	No data	? TT=86.479	? TT=76.196	? TT=72.286	? TT=82.708	? TT=72.268	? (exit code) TT=88.503	? (exit code) TT=88.598	? (exit code) TT=86.529	? (TO) TT=2400.28	No data	No data	No data	No data
CrosswordDesign/ CrosswordDesign-12-4-rom_c18.xml	SAT f=76 TT=2519.97 T1=1507.98	No data	No data	No data	No data	? TT=308.224	? TT=338.297	? TT=1664.48	? TT=1697.98	? TT=599.633	? TT=1819.33	SAT (TO) f=76 TT=2519.97 T1=1507.98	No data	No data	? (NS) TT=18.845	? (NS) TT=3.591	No data	No data	? TT=83.478	? TT=83.303	? TT=75.467	? TT=81.808	? TT=75.072	? (exit code) TT=1561.21	? (exit code) TT=1645.43	? (exit code) TT=1667.7	? (TO) TT=2400.34	No data	No data	No data	No data
CrosswordDesign/ CrosswordDesign-13-4-rom_c18.xml	SAT f=71 TT=2520.08 T1=2058.84	No data	No data	No data	No data	? TT=348.261	? TT=391.175	? (TO) TT=2528.31	? (TO) TT=2528.24	? TT=1073.2	? (TO) TT=2527.87	SAT (TO) f=71 TT=2520.08 T1=2058.84	No data	No data	? (NS) TT=19.259	? (NS) TT=3.995	No data	No data	? TT=78.947	? TT=70.255	? TT=70.287	? TT=77.527	? TT=71.465	? (MO) TT=1801.46	? (MO) TT=1986.77	? (MO) TT=1951.24	? (TO) TT=2400.39	No data	No data	No data	No data
CrosswordDesign/ CrosswordDesign-14-4-rom_c18.xml	SAT f=57 TT=2519.99 T1=2470.63	No data	No data	No data	No data	? TT=388.854	? TT=377.848	? TT=2122.01	? (TO) TT=2528.37	? TT=1790.46	? TT=1759.88	SAT (TO) f=57 TT=2519.99 T1=2470.63	No data	No data	? (MO) TT=19.564	? (NS) TT=4.317	No data	No data	? TT=78.701	? TT=72.844	? TT=69.088	? TT=80.238	? TT=73.821	? (exit code) TT=73.553	? (exit code) TT=74.781	? (exit code) TT=68.839	? (TO) TT=2400.33	No data	No data	No data	No data
CrosswordDesign/ CrosswordDesign-15-4-rom_c18.xml	SAT f=43 TT=2520.1 T1=1953.76	No data	No data	No data	No data	? TT=58.64	? TT=61.263	? (TO) TT=2528.14	? (TO) TT=2528.07	? (TO) TT=2527.97	? (TO) TT=2527.98	SAT (TO) f=43 TT=2520.1 T1=1953.76	No data	No data	? (NS) TT=18.623	? (NS) TT=4.624	No data	No data	? TT=86.263	? TT=73.176	? TT=73.586	? TT=86.121	? TT=76.212	? (exit code) TT=74.627	? (exit code) TT=72.407	? (exit code) TT=69.423	? (TO) TT=2400.42	No data	No data	No data	No data
Dubois/ Dubois-018_c18.xml	UNSAT TT=0.007	UNSAT TT=0.008	UNSAT TT=0.008	UNSAT TT=0.008	UNSAT TT=0.008	UNSAT TT=6.96	UNSAT TT=7.269	UNSAT TT=84.29	UNSAT TT=50.393	UNSAT TT=83.064	UNSAT TT=49.377	UNSAT TT=20.539	UNSAT TT=17.788	UNSAT TT=17.873	UNSAT TT=4.455	UNSAT TT=4.922	UNSAT TT=6.015	UNSAT TT=5.886	? (TO) TT=2400.04	? (TO) TT=2520.02	UNSAT TT=6.098	No data	No data	UNSAT TT=0.008	UNSAT TT=0.007	UNSAT TT=0.007	UNSAT TT=1.19	UNSAT TT=3.148	UNSAT TT=4.297	UNSAT TT=3.062	UNSAT TT=4.279
Dubois/ Dubois-020_c18.xml	UNSAT TT=0.007	UNSAT TT=0.009	UNSAT TT=0.009	UNSAT TT=0.009	UNSAT TT=0.009	UNSAT TT=23.68	UNSAT TT=23.077	UNSAT TT=347.652	UNSAT TT=142.89	UNSAT TT=325.745	UNSAT TT=155.12	UNSAT TT=82.531	UNSAT TT=50.295	UNSAT TT=47.88	UNSAT TT=13.109	UNSAT TT=10.562	UNSAT TT=16.786	UNSAT TT=16.774	? (TO) TT=2400.04	? (TO) TT=2520.04	UNSAT TT=84.961	No data	No data	UNSAT TT=0.008	UNSAT TT=0.009	UNSAT TT=0.007	UNSAT TT=1.169	UNSAT TT=3.197	UNSAT TT=4.357	UNSAT TT=3.128	UNSAT TT=4.381
Dubois/ Dubois-021_c18.xml	UNSAT TT=0.007	UNSAT TT=0.009	UNSAT TT=0.009	UNSAT TT=0.009	UNSAT TT=0.009	UNSAT TT=50.646	UNSAT TT=51.781	UNSAT TT=708.665	UNSAT TT=382.362	UNSAT TT=707.582	UNSAT TT=359.282	UNSAT TT=176.532	UNSAT TT=61.986	UNSAT TT=59.728	UNSAT TT=32.607	UNSAT TT=26.151	UNSAT TT=33.647	UNSAT TT=33.54	? (TO) TT=2400.05	? (TO) TT=2520.03	UNSAT TT=172.474	No data	No data	UNSAT TT=0.007	UNSAT TT=0.008	UNSAT TT=0.008	UNSAT TT=1.196	UNSAT TT=3.432	UNSAT TT=4.498	UNSAT TT=3.355	UNSAT TT=4.367
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
Dubois/ Dubois-022_c18.xml	UNSAT TT=0.008	UNSAT TT=0.009	UNSAT TT=0.009	UNSAT TT=0.009	UNSAT TT=0.009	UNSAT TT=92.192	UNSAT TT=95.623	UNSAT TT=1266.34	UNSAT TT=649.772	UNSAT TT=1241.84	UNSAT TT=788.638	UNSAT TT=385.936	UNSAT TT=136.168	UNSAT TT=135.243	UNSAT TT=60.59	UNSAT TT=66.97	UNSAT TT=64.526	UNSAT TT=63.397	? (TO) TT=2400.07	? (TO) TT=2520.1	UNSAT TT=364.043	No data	No data	UNSAT TT=0.008	UNSAT TT=0.009	UNSAT TT=0.008	UNSAT TT=1.179	UNSAT TT=3.474	UNSAT TT=4.705	UNSAT TT=3.419	UNSAT TT=4.649
Dubois/ Dubois-023_c18.xml	UNSAT TT=0.008	UNSAT TT=0.009	UNSAT TT=0.009	UNSAT TT=0.009	UNSAT TT=0.009	UNSAT TT=192.977	UNSAT TT=188.121	? (TO) TT=2520.05	UNSAT TT=1275.38	UNSAT TT=2518.93	UNSAT TT=1479.7	UNSAT TT=631.173	UNSAT TT=306.19	UNSAT TT=298.18	UNSAT TT=124.016	UNSAT TT=93.3	UNSAT TT=135.312	UNSAT TT=130.127	? (TO) TT=2400.08	? (TO) TT=2520.06	UNSAT TT=633.327	No data	No data	UNSAT TT=0.008	UNSAT TT=0.008	UNSAT TT=0.008	UNSAT TT=1.193	UNSAT TT=3.312	UNSAT TT=4.542	UNSAT TT=3.425	UNSAT TT=4.686
Dubois/ Dubois-024_c18.xml	UNSAT TT=0.008	UNSAT TT=0.009	UNSAT TT=0.009	UNSAT TT=0.011	UNSAT TT=0.009	UNSAT TT=310.829	UNSAT TT=309.738	? (TO) TT=2520.11	? (TO) TT=2520.05	? (TO) TT=2520.04	? (TO) TT=2520.1	UNSAT TT=1129.19	UNSAT TT=741.778	UNSAT TT=757.22	UNSAT TT=221.993	UNSAT TT=215.21	UNSAT TT=268.842	UNSAT TT=274.585	? (TO) TT=2400.08	? (TO) TT=2520.1	UNSAT TT=1369.04	No data	No data	UNSAT TT=0.01	UNSAT TT=0.008	UNSAT TT=0.008	UNSAT TT=1.211	UNSAT TT=3.501	UNSAT TT=4.626	UNSAT TT=3.323	UNSAT TT=4.715
Dubois/ Dubois-025_c18.xml	UNSAT TT=0.009	UNSAT TT=0.009	UNSAT TT=0.009	UNSAT TT=0.009	UNSAT TT=0.009	UNSAT TT=673.053	UNSAT TT=683.741	? (TO) TT=2520.09	? (TO) TT=2520.11	? (TO) TT=2520.1	? (TO) TT=2520.06	? (TO) TT=2520.1	UNSAT TT=906.646	UNSAT TT=906.527	UNSAT TT=390.101	UNSAT TT=470.542	UNSAT TT=542.898	UNSAT TT=547.971	? (TO) TT=2400.04	? (TO) TT=2520.08	UNSAT TT=2388.04	No data	No data	UNSAT TT=0.009	UNSAT TT=0.009	UNSAT TT=0.009	UNSAT TT=1.269	UNSAT TT=3.615	UNSAT TT=4.622	UNSAT TT=3.518	UNSAT TT=4.732
Dubois/ Dubois-030_c18.xml	UNSAT TT=0.009	UNSAT TT=0.01	UNSAT TT=0.01	UNSAT TT=0.01	UNSAT TT=0.01	? (TO) TT=2400.09	? (TO) TT=2520.06	? (TO) TT=2520.14	? (TO) TT=2520.07	? (TO) TT=2520.14	? (TO) TT=2520.11	? (TO) TT=2519.93	? (TO) TT=2519.83	? (TO) TT=2519.99	? (TO) TT=2400.07	? (TO) TT=2520.08	? (TO) TT=2400.04	? (TO) TT=2520.04	? (TO) TT=2400.05	? (TO) TT=2520.04	? (TO) TT=2520.03	No data	No data	UNSAT TT=0.009	UNSAT TT=0.009	UNSAT TT=0.009	UNSAT TT=1.362	UNSAT TT=3.616	UNSAT TT=4.816	UNSAT TT=3.675	UNSAT TT=4.7
Dubois/ Dubois-040_c18.xml	UNSAT TT=0.01	UNSAT TT=0.011	UNSAT TT=0.012	UNSAT TT=0.013	UNSAT TT=0.012	? (TO) TT=2400.07	? (TO) TT=2520.12	? (TO) TT=2520.07	? (TO) TT=2520.06	? (TO) TT=2520.08	? (TO) TT=2520.07	? (TO) TT=2519.81	? (TO) TT=2519.94	? (TO) TT=2519.85	? (TO) TT=2400.04	? (TO) TT=2519.96	? (TO) TT=2400.04	? (TO) TT=2520.06	? (TO) TT=2400.11	? (TO) TT=2520.09	? (TO) TT=2520.08	No data	No data	UNSAT TT=0.012	UNSAT TT=0.01	UNSAT TT=0.01	UNSAT TT=1.579	UNSAT TT=3.94	UNSAT TT=5.123	UNSAT TT=4.073	UNSAT TT=5.182
Dubois/ Dubois-050_c18.xml	UNSAT TT=0.011	UNSAT TT=0.013	UNSAT TT=0.014	UNSAT TT=0.014	UNSAT TT=0.014	? (TO) TT=2400.04	? (TO) TT=2520.09	? (TO) TT=2520.08	? (TO) TT=2520.1	? (TO) TT=2520.13	? (TO) TT=2520.07	? (TO) TT=2519.8	? (TO) TT=2519.61	? (TO) TT=2519.91	? (TO) TT=2400.01	? (TO) TT=2519.67	? (TO) TT=2400.02	? (TO) TT=2520.03	? (TO) TT=2400.07	? (TO) TT=2520.09	? (TO) TT=2520.06	No data	No data	UNSAT TT=0.012	UNSAT TT=0.011	UNSAT TT=0.011	UNSAT TT=1.623	UNSAT TT=4.243	UNSAT TT=5.486	UNSAT TT=4.19	UNSAT TT=5.482
Dubois/ Dubois-075_c18.xml	UNSAT TT=0.014	UNSAT TT=0.019	UNSAT TT=0.019	UNSAT TT=0.021	UNSAT TT=0.02	? (TO) TT=2400.11	? (TO) TT=2520.08	? (TO) TT=2520.15	? (TO) TT=2520.13	? (TO) TT=2520.13	? (TO) TT=2520.13	? (TO) TT=2519.93	? (TO) TT=2519.91	? (TO) TT=2520	? (TO) TT=2400.08	? (TO) TT=2520.08	? (TO) TT=2400.11	? (TO) TT=2520.09	? (TO) TT=2400.08	? (TO) TT=2520.02	? (TO) TT=2520.04	No data	No data	UNSAT TT=0.015	UNSAT TT=0.014	UNSAT TT=0.015	UNSAT TT=2.481	UNSAT TT=4.844	UNSAT TT=6.158	UNSAT TT=4.808	UNSAT TT=6.156
Dubois/ Dubois-100_c18.xml	UNSAT TT=0.018	UNSAT TT=0.026	UNSAT TT=0.026	UNSAT TT=0.027	UNSAT TT=0.027	? (TO) TT=2400.11	? (TO) TT=2520.12	? (TO) TT=2520.1	? (TO) TT=2520.07	? (TO) TT=2520.1	? (TO) TT=2520.14	? (TO) TT=2519.89	? (TO) TT=2519.94	? (TO) TT=2520	? (TO) TT=2400.03	? (TO) TT=2520.05	? (TO) TT=2400.05	? (TO) TT=2520.1	? (TO) TT=2400.11	? (TO) TT=2520.06	? (TO) TT=2520.05	No data	No data	UNSAT TT=0.018	UNSAT TT=0.018	UNSAT TT=0.018	UNSAT TT=2.76	UNSAT TT=5.391	UNSAT TT=6.777	UNSAT TT=5.266	UNSAT TT=6.684
Eternity/ Eternity-04-04_c18.xml	SAT TT=0.009	SAT TT=0.01	SAT TT=0.01	SAT TT=0.009	SAT TT=0.009	SAT TT=1.158	SAT TT=1.159	SAT TT=7.223	SAT TT=7.54	SAT TT=7.417	SAT TT=7.399	SAT TT=0.014	SAT TT=0.01	SAT TT=0.01	SAT TT=0.015	SAT TT=0.015	SAT TT=2.081	SAT TT=1.927	SAT TT=2.101	SAT TT=1.928	SAT TT=1.956	No data	No data	SAT TT=0.541	SAT TT=0.602	SAT TT=0.604	SAT TT=7.625	SAT TT=8.456	SAT TT=10.266	SAT TT=8.577	SAT TT=10.3
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
Eternity/ Eternity-05-05_c18.xml	SAT TT=0.048	SAT TT=0.084	SAT TT=0.084	SAT TT=0.112	SAT TT=0.112	SAT TT=1.504	SAT TT=1.538	SAT TT=9.206	SAT TT=8.619	SAT TT=8.757	SAT TT=10.454	SAT TT=0.14	SAT TT=0.127	SAT TT=0.126	SAT TT=0.048	SAT TT=0.17	SAT TT=2.491	SAT TT=2.377	SAT TT=2.29	SAT TT=2.17	SAT TT=2.133	No data	No data	SAT TT=1.028	SAT TT=1.037	SAT TT=1.026	SAT TT=136.898	SAT TT=14.728	SAT TT=14.939	SAT TT=14.519	SAT TT=15.395
Eternity/ Eternity-06-06_c18.xml	SAT TT=0.139	SAT TT=3.756	SAT TT=3.537	SAT TT=0.642	SAT TT=0.612	SAT TT=13.232	SAT TT=12.98	SAT TT=21.122	SAT TT=25.876	SAT TT=22.854	SAT TT=15.416	SAT TT=1.039	SAT TT=4.419	SAT TT=4.225	SAT TT=0.139	SAT TT=0.637	SAT TT=7.648	SAT TT=7.551	SAT TT=2.757	SAT TT=2.672	SAT TT=2.672	No data	No data	SAT TT=6.058	SAT TT=7.346	SAT TT=3.997	? (TO) TT=2400.12	SAT TT=21.906	SAT TT=25.903	SAT TT=21.932	SAT TT=27.3
Eternity/ Eternity-07-07_c18.xml	SAT TT=0.989	SAT TT=386.79	SAT TT=341.085	SAT TT=10.067	SAT TT=8.764	SAT TT=438.201	SAT TT=447.843	SAT TT=886.756	SAT TT=872.077	SAT TT=86.512	SAT TT=297.691	SAT TT=13.122	SAT TT=614.124	SAT TT=633.878	SAT TT=61.757	SAT TT=0.989	? (TO) TT=2400.09	? (TO) TT=2520.04	SAT TT=341.565	SAT TT=1109.93	SAT TT=163.521	No data	No data	SAT TT=577.262	SAT TT=216.256	SAT TT=20.128	? (TO) TT=2400.22	SAT TT=29.201	SAT TT=84.37	SAT TT=29.145	SAT TT=83.157
Eternity/ Eternity-08-08_c18.xml		? TT=2520.03	? (TO) TT=2519.99	? TT=2520.03	? TT=2520.03	? (TO) TT=2400.07	? (TO) TT=2520.08	? (TO) TT=2520.11	? (TO) TT=2520.07	? (TO) TT=2520.11	? (TO) TT=2520.14	? (TO) TT=2519.91	? TT=2520.03	? TT=2520.03	? (TO) TT=2400.06	? (TO) TT=2519.74	? (TO) TT=2400.02	? (TO) TT=2520.05	? (TO) TT=2400.06	? (TO) TT=2520.06	? (TO) TT=2520.11	No data	No data	? (TO) TT=2519.89	? (TO) TT=2520.08	? (TO) TT=2519.66	? (TO) TT=2400.2	? (MO) TT=202.551	? (TO) TT=2520.08	? (MO) TT=209.562	? (TO) TT=2520.06
Eternity/ Eternity-09-09_c18.xml		? (TO) TT=2519.91	? (TO) TT=2520.04	? TT=2520.05	? (TO) TT=2519.77	? (TO) TT=2400.1	? (TO) TT=2520.03	? (TO) TT=2520.05	? (TO) TT=2520.06	? (TO) TT=2520.11	? (TO) TT=2520.11	? (TO) TT=2519.82	? TT=2520.05	? (TO) TT=2519.82	? (TO) TT=2400.08	? (TO) TT=2519.92	? (TO) TT=2400.05	? (TO) TT=2520.08	? (TO) TT=2400.09	? (TO) TT=2520.04	? (TO) TT=2520.09	No data	No data	? (TO) TT=2520.08	? (TO) TT=2519.97	? (TO) TT=2519.96	? (TO) TT=2400.22	? (MO) TT=105.48	? (TO) TT=2520.15	? (MO) TT=127.514	? (TO) TT=2520.13
Eternity/ Eternity-10-04_c18.xml	SAT TT=0.108	SAT TT=2.019	SAT TT=1.767	SAT TT=4.103	SAT TT=3.712	SAT TT=5.352	SAT TT=5.536	SAT TT=18.228	SAT TT=12.12	SAT TT=10.477	SAT TT=13.644	SAT TT=2.062	SAT TT=1.936	SAT TT=1.914	SAT TT=0.108	SAT TT=0.922	SAT TT=4.734	SAT TT=4.499	SAT TT=5.464	SAT TT=5.22	SAT TT=5.318	No data	No data	SAT TT=4.613	SAT TT=6.56	SAT TT=4.903	? (TO) TT=2400.12	SAT TT=21.452	SAT TT=27.796	SAT TT=21.788	SAT TT=27.873
Eternity/ Eternity-10-05_c18.xml	SAT TT=1.084	SAT TT=1503.01	SAT TT=1294.02	SAT TT=37.608	SAT TT=32.842	SAT TT=158.812	SAT TT=159.811	SAT TT=76.45	SAT TT=193.189	SAT TT=98.71	SAT TT=670.342	SAT TT=24.913	SAT TT=4.673	SAT TT=4.861	SAT TT=1.084	SAT TT=1.428	SAT TT=325.049	SAT TT=327.978	SAT TT=273.489	? (TO) TT=2520.07	SAT TT=520.734	No data	No data	SAT TT=28.446	SAT TT=28.188	SAT TT=56.014	? (TO) TT=2400.2	SAT TT=25.987	SAT TT=53.56	SAT TT=25.959	SAT TT=55.327
Eternity/ Eternity-10-06_c18.xml	SAT TT=211.828	? (TO) TT=2519.92	? (TO) TT=2519.89	? TT=2520.03	? TT=2520.03	SAT TT=211.828	SAT TT=215.496	? (TO) TT=2520.16	? (TO) TT=2520.08	? (TO) TT=2520.08	? (TO) TT=2520.14	? (TO) TT=2519.8	? (TO) TT=2519.92	? (TO) TT=2519.67	SAT TT=601.874	SAT TT=1118.91	? (TO) TT=2400.03	? (TO) TT=2520.08	? (TO) TT=2400.1	? (TO) TT=2520.02	? (TO) TT=2520.03	No data	No data	? (TO) TT=2520.1	? (TO) TT=2519.95	? (TO) TT=2519.92	? (TO) TT=2400.19	? (MO) TT=1055.52	? (TO) TT=2520.09	? (MO) TT=1048.12	? (TO) TT=2520.12
Eternity/ Eternity-10-10_c18.xml		? TT=2520.07	? (TO) TT=2520	? TT=2520.06	? (TO) TT=2520.07	? (TO) TT=2400.09	? (TO) TT=2520.1	? (TO) TT=2520.06	? (TO) TT=2520.06	? (TO) TT=2520.11	? (TO) TT=2520.08	? (TO) TT=2519.83	? (TO) TT=2519.96	? (TO) TT=2519.91	? (TO) TT=2400.03	? (TO) TT=2519.98	? (TO) TT=2400.05	? (TO) TT=2520.08	? (TO) TT=2400.05	? (TO) TT=2520.09	? (TO) TT=2520.08	No data	No data	? (TO) TT=2520.04	? (TO) TT=2520.01	? (TO) TT=2519.95	? (TO) TT=2400.14	? (MO) TT=96.604	? (TO) TT=2520.06	? (MO) TT=106.752	? (TO) TT=2520.13
Eternity/ Eternity-11-11_c18.xml		? (TO) TT=2520.06	? (TO) TT=2519.83	? (TO) TT=2520.07	? (TO) TT=2519.77	? (TO) TT=2400.05	? (TO) TT=2520.11	? (TO) TT=2520.12	? (TO) TT=2520.07	? (TO) TT=2520.07	? (TO) TT=2520.06	? (TO) TT=2520.01	? (TO) TT=2519.94	? (TO) TT=2520.05	? (TO) TT=2400.05	? (TO) TT=2520.09	? (TO) TT=2400.05	? (TO) TT=2520.07	? (TO) TT=2400.07	? (TO) TT=2520.09	? (TO) TT=2520.09	No data	No data	? (TO) TT=2519.97	? (TO) TT=2519.98	? (TO) TT=2520.05	? (TO) TT=2400.15	? (MO) TT=95.348	? (TO) TT=2520.11	? (MO) TT=96.511	? (TO) TT=2520.18
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
Eternity/ Eternity-12-12_c18.xml		? (TO) TT=2519.91	? (TO) TT=2519.83	? (TO) TT=2519.91	? (TO) TT=2520.06	? (TO) TT=2400.12	? (TO) TT=2520.04	? (TO) TT=2520.07	? (TO) TT=2520.1	? (TO) TT=2520.12	? (TO) TT=2520.09	? (TO) TT=2519.81	? (TO) TT=2520.04	? (TO) TT=2519.79	? (TO) TT=2400.07	? (TO) TT=2519.7	? (TO) TT=2400.12	? (TO) TT=2520.11	? (TO) TT=2400.09	? (TO) TT=2520.06	? (TO) TT=2520.05	No data	No data	? (TO) TT=2520.09	? (TO) TT=2519.86	? (TO) TT=2519.88	? (TO) TT=2400.14	? (MO) TT=140.058	? (TO) TT=2520.1	? (MO) TT=132.528	? (TO) TT=2520.12
Eternity/ Eternity-13-13_c18.xml		? (TO) TT=2520	? (TO) TT=2520.04	? (TO) TT=2519.94	? (TO) TT=2520.06	? (TO) TT=2400.1	? (TO) TT=2520.04	? (TO) TT=2520.12	? (TO) TT=2520.08	? (TO) TT=2520.07	? (TO) TT=2520.11	? (TO) TT=2519.91	? (TO) TT=2519.77	? (TO) TT=2520.05	? (TO) TT=2400.03	? (TO) TT=2519.66	? (TO) TT=2400.05	? (TO) TT=2520.11	? (TO) TT=2400.04	? (TO) TT=2520.06	? (TO) TT=2520.11	No data	No data	? (TO) TT=2519.91	? (TO) TT=2520.05	? (TO) TT=2519.97	? (TO) TT=2400.14	? (MO) TT=181.714	? (TO) TT=2520.16	? (MO) TT=179.635	? (TO) TT=2520.17
Eternity/ Eternity-14-14_c18.xml		? (TO) TT=2520.06	? (TO) TT=2519.96	? (TO) TT=2520.06	? (TO) TT=2519.75	? (TO) TT=2400.05	? (TO) TT=2520.11	? (TO) TT=2520.14	? (TO) TT=2520.09	? (TO) TT=2520.13	? (TO) TT=2520.07	? (TO) TT=2520.01	? (TO) TT=2520.02	? (TO) TT=2520.03	? (TO) TT=2400.03	? (TO) TT=2519.9	? (TO) TT=2400.07	? (TO) TT=2520.03	? (TO) TT=2400.11	? (TO) TT=2520.13	? (TO) TT=2520.05	No data	No data	? (TO) TT=2519.92	? (TO) TT=2520.11	? (TO) TT=2520.09	? (TO) TT=2400.15	? (MO) TT=253.612	? (TO) TT=2520.15	? (MO) TT=245.847	? (TO) TT=2520.14
Eternity/ Eternity-15-15_c18.xml		? (TO) TT=2520.06	? (TO) TT=2519.97	? (TO) TT=2520.06	? (TO) TT=2520.04	? (TO) TT=2400.07	? (TO) TT=2520.04	? (TO) TT=2520.13	? (TO) TT=2520.1	? (TO) TT=2520.12	? (TO) TT=2520.06	? (TO) TT=2519.83	? (TO) TT=2520.04	? (TO) TT=2519.94	? (TO) TT=2400.04	? (TO) TT=2519.93	? (TO) TT=2400.02	? (TO) TT=2520.08	? (TO) TT=2400.1	? (TO) TT=2520.02	? (TO) TT=2520.1	No data	No data	? (TO) TT=2520.08	? (TO) TT=2519.93	? (TO) TT=2520.08	? (TO) TT=2400.19	? (MO) TT=352.997	? (TO) TT=2520.21	? (MO) TT=354.558	? (TO) TT=2520.14
Fapp/Fapp-m2s/ Fapp-m2s-01-0200_c18.xml	SAT f=35787314 TT=2520.08 T1=656.09	No data	No data	No data	No data	SAT (TO) f=35787314 TT=2400.09 T1=662.71	SAT (TO) f=35787314 TT=2520.08 T1=656.09	SAT (TO) f=98763724 TT=2520.12 T1=29.9	SAT (TO) f=62597228 TT=2520.08 T1=34.88	SAT (TO) f=80438920 TT=2520.12 T1=31.28	SAT (TO) f=44783802 TT=2520.09 T1=31.3	? TT=16.859	No data	No data	SAT (TO) f=100248846 TT=2400.03 T1=2118.75	SAT (TO) f=100248846 TT=2519.97 T1=2139.69	No data	No data	SAT (TO) f=100248967 TT=2400.05 T1=2097.45	? (TO) TT=2520.12	? (TO) TT=2520.11	SAT (TO) f=99265427 TT=2400.09 T1=2336.4	SAT (TO) f=99057185 TT=2520.17 T1=2449.24	? (TO) TT=2519.93	? (TO) TT=2519.97	? (TO) TT=2519.98	? (TO) TT=2407.43	No data	No data	No data	No data
Fapp/Fapp-m2s/ Fapp-m2s-02-0250_c18.xml	SAT f=40498440 TT=2520.15 T1=151.32	No data	No data	No data	No data	SAT (TO) f=141134191 TT=2400.13 T1=919.89	SAT (TO) f=141134191 TT=2520.05 T1=916.45	SAT (TO) f=40498440 TT=2520.15 T1=151.32	SAT (TO) f=60577345 TT=2520.1 T1=706.03	SAT (TO) f=223240009 TT=2520.18 T1=49.23	SAT (TO) f=223154775 TT=2520.16 T1=69.51	? TT=28.418	No data	No data	SAT (TO) f=225497794 TT=2400.06 T1=1451.14	SAT (TO) f=225497794 TT=2520.07 T1=1590.16	No data	No data	? (TO) TT=2400.06	? (TO) TT=2520.09	? (TO) TT=2520.03	SAT (TO) f=226933202 TT=2400.12 T1=2286.1	SAT (TO) f=226876343 TT=2520.96 T1=2378.74	? (TO) TT=2520.04	? (TO) TT=2520.06	? (TO) TT=2520.09	? TT=1903.84	No data	No data	No data	No data
Fapp/Fapp-m2s/ Fapp-m2s-03-0300_c18.xml	SAT f=294770329 TT=2520.17 T1=82.59	No data	No data	No data	No data	SAT (TO) f=294934580 TT=2400.05 T1=1205.78	SAT (TO) f=294934580 TT=2520.16 T1=1200.49	SAT (TO) f=294770329 TT=2520.17 T1=82.59	SAT (TO) f=465087373 TT=2520.19 T1=87.6	SAT (TO) f=464984334 TT=2520.16 T1=95.13	SAT (TO) f=294524152 TT=2520.22 T1=93.33	? (TO) TT=2520.09	No data	No data	SAT (TO) f=470193793 TT=2400.03 T1=1640.67	SAT (TO) f=470152795 TT=2519.87 T1=2495.45	No data	No data	? (TO) TT=2400.11	? (TO) TT=2520.1	SAT (TO) f=474748253 TT=2520.17 T1=81.66	SAT (TO) f=474954140 TT=2400.27 T1=2139.29	SAT (TO) f=474810662 TT=2520.12 T1=2323.07	? (TO) TT=2520.05	? (TO) TT=2520.1	? (TO) TT=2520.12	? TT=2261.95	No data	No data	No data	No data
Fapp/Fapp-m2s/ Fapp-m2s-04-0300_c18.xml	SAT f=26739930 TT=2520.12 T1=2107.47	No data	No data	No data	No data	SAT (TO) f=26739930 TT=2400.13 T1=2106.37	SAT (TO) f=26739930 TT=2520.12 T1=2107.47	SAT (TO) f=27301890 TT=2520.08 T1=99.34	SAT (TO) f=269649874 TT=2520.28 T1=110.82	SAT (TO) f=27317500 TT=2520.1 T1=158.23	SAT (TO) f=122023784 TT=2520.17 T1=796.92	SAT (TO) f=275085566 TT=2519.95 T1=425.12	No data	No data	SAT (TO) f=272102214 TT=2400.09 T1=2338.21	SAT (TO) f=272086604 TT=2520.08 T1=2403.93	No data	No data	? (TO) TT=2400.03	? (TO) TT=2520.03	SAT (TO) f=274414589 TT=2520.05 T1=2469.42	SAT (TO) f=276102473 TT=2400.09 T1=2250.86	SAT (TO) f=276680333 TT=2520.1 T1=2334.16	? (TO) TT=2520.06	? (TO) TT=2519.8	? (TO) TT=2519.95	? TT=1864.21	No data	No data	No data	No data
Fapp/Fapp-m2s/ Fapp-m2s-05-0350_c18.xml	SAT f=524813477 TT=2527.64 T1=116.16	No data	No data	No data	No data	? (TO) TT=2400.14	? (TO) TT=2520.1	SAT (TO) f=525183769 TT=2520.11 T1=107.88	SAT (TO) f=524813477 TT=2527.64 T1=116.16	SAT (TO) f=526229521 TT=2520.19 T1=156.46	? (TO) TT=2528.03	SAT (TO) f=535158753 TT=2520.03 T1=672.15	No data	No data	SAT (TO) f=533394466 TT=2400.08 T1=1266.17	SAT (TO) f=533394466 TT=2519.79 T1=1264.68	No data	No data	? (TO) TT=2400.05	? (TO) TT=2520.04	? (TO) TT=2520.1	? (TO) TT=2400.12	? (TO) TT=2520.09	? (TO) TT=2519.92	? (TO) TT=2520.14	? (TO) TT=2520.09	? TT=1225.52	No data	No data	No data	No data
Fapp/Fapp-m2s/ Fapp-m2s-06-0500_c18.xml	SAT f=840126924 TT=2520.18 T1=367.56	No data	No data	No data	No data	SAT (TO) f=1027918899 TT=2400.14 T1=1169.8	SAT (TO) f=1027918899 TT=2520.07 T1=1174.23	SAT (TO) f=840126924 TT=2520.18 T1=367.56	SAT (TO) f=1032285803 TT=2520.15 T1=624.67	SAT (TO) f=560654021 TT=2520.22 T1=916.67	SAT (TO) f=934710053 TT=2520.14 T1=190.26	SAT (TO) f=1054458226 TT=2519.89 T1=2252.32	No data	No data	SAT (TO) f=1049785785 TT=2400.09 T1=2306.62	SAT (TO) f=1049785773 TT=2519.53 T1=2440.54	No data	No data	? (TO) TT=2400.09	? (TO) TT=2520.05	? (TO) TT=2520.13	SAT (TO) f=1061881838 TT=2400.12 T1=1918.05	SAT (TO) f=1064813911 TT=2520.13 T1=2005.66	? (TO) TT=2520.01	? (TO) TT=2519.75	? (TO) TT=2520.1	? (TO) TT=2408.04	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
Fapp/Fapp-m2s/ Fapp-m2s-07-0600_c18.xml	SAT f=1603939205 TT=2520.11 T1=322.85	No data	No data	No data	No data	? (TO) TT=2400.05	? (TO) TT=2520.1	SAT (TO) f=1603981165 TT=2520.16 T1=226.9	SAT (TO) f=1603939205 TT=2520.11 T1=322.85	SAT (TO) f=1782066163 TT=2520.16 T1=252.4	SAT (TO) f=1603938833 TT=2520.15 T1=279.73	? (TO) TT=2520.06	No data	No data	Wrong UNSAT TT=168.362	Wrong UNSAT TT=166.953	No data	No data	? (TO) TT=2400.07	? (TO) TT=2520.14	? (TO) TT=2520.1	? (TO) TT=2407.79	SAT (TO) f=2041539264 TT=2527.21 T1=1160.41	? (TO) TT=2520.09	? (TO) TT=2520.12	? (TO) TT=2520.14	? (TO) TT=2408.27	No data	No data	No data	No data
Fapp/Fapp-m2s/ Fapp-m2s-08-0700_c18.xml	SAT f=542126384 TT=2521 T1=192.56	No data	No data	No data	No data	? (TO) TT=2400.11	? (TO) TT=2520.12	SAT (TO) f=542225012 TT=2520.12 T1=148.81	SAT (TO) f=542126384 TT=2521 T1=192.56	SAT (TO) f=542093498 TT=2520.13 T1=164.39	SAT (TO) f=541961926 TT=2520.17 T1=192.45	SAT (TO) f=1220815028 TT=2519.84 T1=2068.32	No data	No data	? (TO) TT=2400.1	? (TO) TT=2519.93	No data	No data	? (TO) TT=2400.09	? (TO) TT=2520.13	? (TO) TT=2520.12	? (TO) TT=2400.12	? (TO) TT=2520.1	? (TO) TT=2520.04	? (TO) TT=2520.17	? (TO) TT=2520.11	? (TO) TT=2408.32	No data	No data	No data	No data
Fapp/Fapp-m2s/ Fapp-m2s-16-0260_c18.xml	SAT f=372103316 TT=2520.09 T1=561.08	No data	No data	No data	No data	SAT (TO) f=372103316 TT=2400.06 T1=572.23	SAT (TO) f=372103316 TT=2520.09 T1=561.08	SAT (TO) f=375673011 TT=2520.11 T1=54.64	SAT (TO) f=377292294 TT=2520.11 T1=54.54	SAT (TO) f=375930692 TT=2520.15 T1=54.47	SAT (TO) f=376850568 TT=2520.16 T1=64.69	? TT=37.937	No data	No data	SAT (TO) f=387393741 TT=2400.04 T1=105.06	SAT (TO) f=387393741 TT=2519.72 T1=112.19	No data	No data	SAT (TO) f=387430773 TT=2400.1 T1=174.53	SAT (TO) f=386915602 TT=2520.05 T1=688.26	SAT (TO) f=386694797 TT=2520.14 T1=1829.74	SAT (TO) f=389436243 TT=2400.16 T1=2220.78	SAT (TO) f=388718691 TT=2520.16 T1=2380.94	Wrong UNSAT TT=174.006	? (TO) TT=2520.11	? (TO) TT=2519.91	? TT=1622.25	No data	No data	No data	No data
Fapp/Fapp-m2s/ Fapp-m2s-17-0300_c18.xml	OPT f=130971634 TT=82.131 T1=55.15	No data	No data	No data	No data	OPT f=130971634 TT=253.034 T1=233.7	OPT f=130971634 TT=247.57 T1=232.16	OPT f=130971634 TT=82.131 T1=55.15	OPT f=130971634 TT=85.05 T1=57.35	OPT f=130971634 TT=97.159 T1=59.56	OPT f=130971634 TT=93.963 T1=58.31	SAT (TO) f=377783380 TT=2520.06 T1=377.81	No data	No data	SAT (TO) f=376480179 TT=2400.05 T1=1130.55	SAT (TO) f=376480179 TT=2519.92 T1=1136.3	No data	No data	? (TO) TT=2400.1	SAT (TO) f=372860365 TT=2520.07 T1=1152.09	? (TO) TT=2520.06	SAT (TO) f=382326506 TT=2400.08 T1=2246.22	SAT (TO) f=379665800 TT=2520.13 T1=2337.96	OPT f=130971634 TT=1162.56 T1=1162.63	OPT f=130971634 TT=1120.52 T1=1120.74	OPT f=130971634 TT=1121.15 T1=1121.15	? TT=1590.51	No data	No data	No data	No data
Fapp/Fapp-m2s/ Fapp-m2s-18-0350_c18.xml	OPT f=162507055 TT=68.442 T1=47.11	No data	No data	No data	No data	OPT f=162507055 TT=67.375 T1=54.13	OPT f=162507055 TT=68.442 T1=54.33	OPT f=162507055 TT=75.295 T1=47.11	OPT f=162507055 TT=74.746 T1=48.68	OPT f=162507055 TT=69.339 T1=42.92	OPT f=162507055 TT=69.799 T1=43.65	? TT=34.014	No data	No data	SAT (TO) f=232566519 TT=2400.04 T1=662.55	SAT (TO) f=232566519 TT=2519.97 T1=691.65	No data	No data	? (TO) TT=2400.09	? (TO) TT=2520.04	? (TO) TT=2520.11	SAT (TO) f=231924910 TT=2400.14 T1=2291.81	SAT (TO) f=231183439 TT=2520.12 T1=2289.25	OPT f=162507055 TT=1273.05 T1=1273.11	OPT f=162507055 TT=1183.43 T1=1183.41	OPT f=162507055 TT=1116.8 T1=1116.76	? TT=1714.84	No data	No data	No data	No data
Fapp/Fapp-m2s/ Fapp-m2s-20-0420_c18.xml	OPT f=506362597 TT=92.837 T1=63.23	No data	No data	No data	No data	OPT f=506362597 TT=184.923 T1=151.5	OPT f=506362597 TT=185.813 T1=151.66	OPT f=506362597 TT=122.535 T1=73.49	OPT f=506362597 TT=92.837 T1=63.23	SAT (TO) f=566554501 TT=2520.13 T1=77.25	SAT (TO) f=565429095 TT=2520.16 T1=74.02	SAT (TO) f=581973846 TT=2519.97 T1=797.36	No data	No data	SAT (TO) f=579092427 TT=2400.05 T1=104.54	SAT (TO) f=579092427 TT=2520.01 T1=106.23	No data	No data	? (TO) TT=2400.11	? (TO) TT=2520.05	? (TO) TT=2520.08	? (TO) TT=2400.15	? (TO) TT=2520.05	Wrong UNSAT TT=208.962	OPT f=506362597 TT=1682.3 T1=1682.17	OPT f=506362597 TT=1710.8 T1=1710.97	? TT=1415.96	No data	No data	No data	No data
Fapp/Fapp-m2s/ Fapp-m2s-21-0500_c18.xml	OPT f=63832030 TT=61.434 T1=38.96	No data	No data	No data	No data	OPT f=63832030 TT=161.182 T1=146.11	OPT f=63832030 TT=164.231 T1=149.02	OPT f=63832030 TT=61.434 T1=38.96	OPT f=63832030 TT=64.489 T1=40.19	OPT f=63832030 TT=62.568 T1=38.89	OPT f=63832030 TT=62.843 T1=38.36	? TT=29.922	No data	No data	SAT (TO) f=182559672 TT=2400.04 T1=2333.85	SAT (TO) f=182559672 TT=2520.02 T1=2345.79	No data	No data	? (TO) TT=2400.11	? (TO) TT=2520.07	? (TO) TT=2520.07	SAT (TO) f=184253556 TT=2400.14 T1=2272.54	SAT (TO) f=184000747 TT=2520.08 T1=2394.01	OPT f=63832030 TT=775.235 T1=775.25	OPT f=63832030 TT=843.688 T1=843.59	OPT f=63832030 TT=753.68 T1=753.74	? TT=1756.91	No data	No data	No data	No data
Fapp/Fapp-m2s/ Fapp-m2s-ex1_c18.xml	OPT f=523 TT=0.877 T1=0.87	No data	No data	No data	No data	OPT f=523 TT=3.034 T1=1.09	OPT f=523 TT=3.319 T1=1.17	OPT f=523 TT=9.983 T1=4.6	OPT f=523 TT=10.469 T1=4.85	OPT f=523 TT=10.499 T1=4.39	OPT f=523 TT=10.696 T1=4.44	OPT f=523 TT=18.774 T1=11.72	No data	No data	OPT f=523 TT=0.082 T1=0.08	OPT f=523 TT=0.081 T1=0.08	No data	No data	SAT (TO) f=523 TT=2400.08 T1=9.51	SAT (TO) f=523 TT=2520.02 T1=10.63	OPT f=523 TT=11.478 T1=6.76	SAT (TO) f=523 TT=2400.11 T1=2.45	SAT (TO) f=523 TT=2520.11 T1=2.36	OPT f=523 TT=0.902 T1=0.9	OPT f=523 TT=0.786 T1=0.78	OPT f=523 TT=0.877 T1=0.87	SAT (TO) f=524 TT=2400.14 T1=2308.57	No data	No data	No data	No data
Fapp/Fapp-m2s/ Fapp-m2s-ex2_c18.xml	OPT f=13871 TT=2.41 T1=0.88	No data	No data	No data	No data	OPT f=13871 TT=2.359 T1=0.89	OPT f=13871 TT=2.41 T1=0.88	OPT f=13871 TT=9.948 T1=4.33	OPT f=13871 TT=11.029 T1=5.11	OPT f=13871 TT=10.205 T1=4.31	OPT f=13871 TT=10.787 T1=4.49	OPT f=13871 TT=4.609 T1=4.35	No data	No data	OPT f=13871 TT=0.215 T1=0.21	OPT f=13871 TT=0.217 T1=0.21	No data	No data	SAT (TO) f=13871 TT=2400.11 T1=5.74	SAT (TO) f=13871 TT=2520.04 T1=4.04	OPT f=13871 TT=8.713 T1=3.97	OPT f=13871 TT=3.88 T1=1.7	OPT f=13871 TT=3.871 T1=1.69	OPT f=13871 TT=4.278 T1=4.27	OPT f=13871 TT=4.863 T1=4.86	OPT f=13871 TT=3.017 T1=3.02	SAT (TO) f=15845 TT=2400.26 T1=295.57	No data	No data	No data	No data
Fapp/Fapp-m2s/ Fapp-m2s-test01-0150_c18.xml	OPT f=18292585 TT=2058.63 T1=2058.71	No data	No data	No data	No data	SAT (TO) f=18292601 TT=2400.06 T1=129.41	SAT (TO) f=18292601 TT=2520.13 T1=128.4	SAT (TO) f=18333277 TT=2520.1 T1=59.56	SAT (TO) f=18346799 TT=2520.16 T1=1006.29	SAT (TO) f=18333290 TT=2520.08 T1=291.64	SAT (TO) f=22855709 TT=2520.15 T1=27.12	SAT (TO) f=51087252 TT=2519.97 T1=1911.01	No data	No data	SAT (TO) f=51060217 TT=2400.09 T1=1858.1	SAT (TO) f=51060217 TT=2519.8 T1=1863.91	No data	No data	SAT (TO) f=51561535 TT=2400.06 T1=2049.84	SAT (TO) f=51331220 TT=2520.13 T1=1439.1	? (TO) TT=2520.07	SAT (TO) f=50606891 TT=2400.08 T1=2335.19	SAT (TO) f=50437546 TT=2520.11 T1=2438	OPT f=18292585 TT=2231.39 T1=2231.34	OPT f=18292585 TT=2048.72 T1=2048.72	OPT f=18292585 TT=2058.63 T1=2058.71	? TT=2083.86	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
Fapp/Fapp-m2s/ Fapp-m2s-test02-0250_c18.xml	SAT f=89666091 TT=2520.12 T1=1242.47	No data	No data	No data	No data	SAT (TO) f=89666091 TT=2400.15 T1=1264.3	SAT (TO) f=89666091 TT=2520.12 T1=1242.47	SAT (TO) f=89790742 TT=2520.11 T1=66.91	SAT (TO) f=141863787 TT=2520.14 T1=81.3	SAT (TO) f=115498750 TT=2520.14 T1=60.83	SAT (TO) f=89677395 TT=2520.13 T1=98.26	SAT (TO) f=145576714 TT=2519.99 T1=215.62	No data	No data	SAT (TO) f=143256229 TT=2400.05 T1=1819.95	SAT (TO) f=143256229 TT=2520.09 T1=1920.59	No data	No data	? (TO) TT=2400.04	? (TO) TT=2520.08	? (TO) TT=2520.13	SAT (TO) f=143641913 TT=2400.15 T1=2253.79	SAT (TO) f=142736257 TT=2520.15 T1=2402	? (TO) TT=2520.04	? (TO) TT=2520.12	? (TO) TT=2520.08	? TT=1721.28	No data	No data	No data	No data
Fapp/Fapp-m2s/ Fapp-m2s-test03-0400_c18.xml	SAT f=164004631 TT=2520.07 T1=2075.03	No data	No data	No data	No data	SAT (TO) f=449431801 TT=2400.12 T1=2230.83	SAT (TO) f=449431801 TT=2520.07 T1=2255.67	SAT (TO) f=164004631 TT=2520.07 T1=2075.03	SAT (TO) f=164105684 TT=2520.14 T1=1019.96	SAT (TO) f=163762036 TT=2520.1 T1=1712.53	SAT (TO) f=163984430 TT=2520.08 T1=734.71	? TT=60.708	No data	No data	SAT (TO) f=456145389 TT=2400.09 T1=854.62	SAT (TO) f=456145389 TT=2520.14 T1=915.1	No data	No data	? (TO) TT=2400.09	? (TO) TT=2520.08	SAT (TO) f=458147641 TT=2520.16 T1=516.05	? (TO) TT=2400.11	? (TO) TT=2520.11	? (TO) TT=2519.92	? (TO) TT=2520.03	? (TO) TT=2519.81	? TT=1366.26	No data	No data	No data	No data
Frb/ frb-30-15-1_c18.xml	SAT TT=0.158	SAT TT=0.42	SAT TT=0.359	SAT TT=1.028	SAT TT=0.875	SAT TT=3.272	SAT TT=3.098	SAT TT=11.31	SAT TT=12.395	SAT TT=11.769	SAT TT=13.32	SAT TT=0.158	SAT TT=0.72	SAT TT=0.84	SAT TT=0.273	SAT TT=1.956	SAT TT=9.442	SAT TT=9.469	SAT TT=8.251	SAT TT=8.209	SAT TT=8.152	No data	No data	SAT TT=6.429	SAT TT=6.422	SAT TT=6.429	SAT TT=3.07	SAT TT=10.334	SAT TT=10.979	SAT TT=9.936	SAT TT=11.627
Frb/ frb-35-17-1_c18.xml	SAT TT=2.374	SAT TT=8.517	SAT TT=7.276	SAT TT=14.426	SAT TT=12.241	SAT TT=7.915	SAT TT=8.517	SAT TT=25.477	SAT TT=24.579	SAT TT=15.342	SAT TT=24.997	SAT TT=2.374	SAT TT=10.976	SAT TT=12.721	SAT TT=14.011	SAT TT=11.376	SAT TT=68.789	SAT TT=68.674	SAT TT=206.09	SAT TT=92.17	SAT TT=107.337	No data	No data	SAT TT=29.308	SAT TT=29.576	SAT TT=29.297	SAT TT=8.393	SAT TT=13.855	SAT TT=18.832	SAT TT=14.311	SAT TT=18.551
Frb/ frb-40-19-1-mgd_c18.xml	SAT TT=5.221	SAT TT=31.961	SAT TT=26.835	SAT TT=59.673	SAT TT=50.277	SAT TT=10.801	SAT TT=10.466	SAT TT=22.091	SAT TT=42.445	SAT TT=36.472	SAT TT=42.51	SAT TT=5.221	SAT TT=58.749	SAT TT=69.588	SAT TT=31.311	SAT TT=42.04	SAT TT=12.268	SAT TT=12.2	SAT TT=5.747	SAT TT=5.423	SAT TT=5.422	No data	No data	SAT TT=90.735	SAT TT=90.729	SAT TT=90.565	SAT TT=197.624	SAT TT=20.431	SAT TT=29.574	SAT TT=20.687	SAT TT=29.35
Frb/ frb-45-21-1_c18.xml	SAT TT=1151.63	? (TO) TT=2519.82	? TT=2520.04	? TT=2520.04	? (TO) TT=2519.94	? (TO) TT=2400.08	? (TO) TT=2520.08	? (TO) TT=2520.05	? (TO) TT=2520.07	? (TO) TT=2520.13	? (TO) TT=2520.11	SAT TT=1882.43	? TT=2520.04	? (TO) TT=2520.04	? (TO) TT=2400.03	? (TO) TT=2520.08	? (TO) TT=2400.09	? (TO) TT=2520.04	? (TO) TT=2400.12	? (TO) TT=2520.1	? (TO) TT=2520.03	No data	No data	? (TO) TT=2519.92	? (TO) TT=2519.94	? (TO) TT=2520.07	? (TO) TT=2400.33	SAT TT=1775.47	SAT TT=1151.63	SAT TT=1771.15	SAT TT=1161.04
Frb/ frb-50-23-1-mgd_c18.xml	SAT TT=1513.36	? (TO) TT=2519.92	? (TO) TT=2520	? TT=2520.05	? (TO) TT=2519.94	? (TO) TT=2400.09	? (TO) TT=2520.12	SAT TT=1513.36	? (TO) TT=2520.09	? (TO) TT=2520.06	? (TO) TT=2520.12	? (TO) TT=2520.08	? (TO) TT=2520.05	? (TO) TT=2519.91	? (TO) TT=2400.08	? (TO) TT=2520.06	? (TO) TT=2400.08	? (TO) TT=2520.12	? (TO) TT=2400.11	? (TO) TT=2520.05	? (TO) TT=2520.12	No data	No data	? (TO) TT=2519.98	? (TO) TT=2520.06	? (TO) TT=2520.03	? (TO) TT=2400.1	? (TO) TT=2520.13	? (TO) TT=2520.05	? (TO) TT=2520.05	? (TO) TT=2520.07
Frb/ frb-50-23-2-mgd_c18.xml	SAT TT=844.678	? (TO) TT=2519.8	? (TO) TT=2520.03	? (TO) TT=2520.05	? TT=2520.05	? (TO) TT=2400.1	? (TO) TT=2520.06	? (TO) TT=2520.1	? (TO) TT=2520.67	? (TO) TT=2520.12	? (TO) TT=2520.07	? (TO) TT=2519.7	? (TO) TT=2520.04	? TT=2520.05	? (TO) TT=2400.1	? (TO) TT=2520.04	? (TO) TT=2400.1	? (TO) TT=2520.09	? (TO) TT=2400.03	? (TO) TT=2520.05	? (TO) TT=2520.03	No data	No data	? (TO) TT=2519.85	? (TO) TT=2520.12	? (TO) TT=2520.02	? (TO) TT=2400.09	SAT TT=2249.59	SAT TT=844.678	SAT TT=2245.14	SAT TT=852.32
Frb/ frb-50-23-3-mgd_c18.xml		? (TO) TT=2520.05	? (TO) TT=2520.03	? (TO) TT=2519.94	? (TO) TT=2520.05	? (TO) TT=2400.08	? (TO) TT=2520.09	? (TO) TT=2520.1	? (TO) TT=2520.1	? (TO) TT=2520.12	? (TO) TT=2520.14	? (TO) TT=2519.79	? (TO) TT=2519.66	? (TO) TT=2520.05	? (TO) TT=2400.04	? (TO) TT=2520.05	? (TO) TT=2400.05	? (TO) TT=2520.1	? (TO) TT=2400.09	? (TO) TT=2520.03	? (TO) TT=2520.08	No data	No data	? (TO) TT=2519.9	? (TO) TT=2520.08	? (TO) TT=2520.07	? (TO) TT=2400.11	? (TO) TT=2520.12	? (TO) TT=2520.08	? (TO) TT=2520.05	? (TO) TT=2520.08
Frb/ frb-53-24-1-mgd_c18.xml		? (TO) TT=2520.03	? (TO) TT=2520.05	? (TO) TT=2519.98	? (TO) TT=2519.94	? (TO) TT=2400.08	? (TO) TT=2520.07	? (TO) TT=2520.05	? (TO) TT=2520.07	? (TO) TT=2520.09	? (TO) TT=2520.11	? (TO) TT=2519.82	? (TO) TT=2520.03	? (TO) TT=2519.92	? (TO) TT=2400.06	? (TO) TT=2520.05	? (TO) TT=2400.06	? (TO) TT=2520.02	? (TO) TT=2400.11	? (TO) TT=2520.04	? (TO) TT=2520.11	No data	No data	? (TO) TT=2519.92	? (TO) TT=2520.02	? (TO) TT=2519.74	? (TO) TT=2400.11	? (TO) TT=2520.08	? (TO) TT=2520.13	? (TO) TT=2520.12	? (TO) TT=2520.07
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
Frb/ frb-53-24-2-mgd_c18.xml	SAT TT=1964.13	? (TO) TT=2520.05	? (TO) TT=2520.04	? (TO) TT=2519.94	? TT=2520.06	? (TO) TT=2400.08	? (TO) TT=2520.03	? (TO) TT=2520.12	? (TO) TT=2520.1	? (TO) TT=2520.12	? (TO) TT=2520.18	? (TO) TT=2519.65	? (TO) TT=2520.04	? TT=2520.06	? (TO) TT=2400.09	? (TO) TT=2520.06	? (TO) TT=2400.09	? (TO) TT=2520.09	? (TO) TT=2400.1	? (TO) TT=2520.11	SAT TT=1964.13	No data	No data	? (TO) TT=2519.85	? (TO) TT=2519.95	? (TO) TT=2520.05	? (TO) TT=2400.08	? (TO) TT=2520.06	? (TO) TT=2520.13	? (TO) TT=2520.07	? (TO) TT=2520.06
Frb/ frb-53-24-3-mgd_c18.xml		? (TO) TT=2520.05	? (TO) TT=2520.01	? (TO) TT=2519.99	? TT=2520.06	? (TO) TT=2400.05	? (TO) TT=2520.1	? (TO) TT=2520.07	? (TO) TT=2520.1	? (TO) TT=2520.07	? (TO) TT=2520.1	? (TO) TT=2520.08	? (TO) TT=2520.03	? (TO) TT=2520.04	? (TO) TT=2400.08	? (TO) TT=2519.72	? (TO) TT=2400.03	? (TO) TT=2520.04	? (TO) TT=2400.11	? (TO) TT=2520.03	? (TO) TT=2520.1	No data	No data	? (TO) TT=2520	? (TO) TT=2520	? (TO) TT=2519.97	? (TO) TT=2400.08	? (TO) TT=2520.1	? (TO) TT=2520.05	? (TO) TT=2520.05	? (TO) TT=2520.09
Frb/ frb-56-25-1-mgd_c18.xml		? (TO) TT=2520.06	? (TO) TT=2520.02	? (TO) TT=2520.04	? TT=2520.07	? (TO) TT=2400.13	? (TO) TT=2520.05	? (TO) TT=2520.05	? (TO) TT=2520.13	? (TO) TT=2520.11	? (TO) TT=2520.07	? (TO) TT=2520.08	? (TO) TT=2519.92	? (TO) TT=2519.99	? (TO) TT=2400.05	? (TO) TT=2519.68	? (TO) TT=2400.07	? (TO) TT=2520.08	? (TO) TT=2400.04	? (TO) TT=2520.09	? (TO) TT=2520.05	No data	No data	? (TO) TT=2519.98	? (TO) TT=2519.88	? (TO) TT=2520.03	? (TO) TT=2400.1	? (TO) TT=2520.06	? (TO) TT=2520.14	? (TO) TT=2520.13	? (TO) TT=2520.07
Frb/ frb-56-25-2-mgd_c18.xml		? (TO) TT=2520.04	? (TO) TT=2520.06	? (TO) TT=2520	? TT=2520.07	? (TO) TT=2400.1	? (TO) TT=2520.1	? (TO) TT=2520.1	? (TO) TT=2520.09	? (TO) TT=2520.13	? (TO) TT=2520.1	? (TO) TT=2519.71	? (TO) TT=2520.06	? (TO) TT=2520.03	? (TO) TT=2400.02	? (TO) TT=2519.8	? (TO) TT=2400.12	? (TO) TT=2520.05	? (TO) TT=2400.07	? (TO) TT=2520.06	? (TO) TT=2520.1	No data	No data	? (TO) TT=2519.82	? (TO) TT=2520.08	? (TO) TT=2520.08	? (TO) TT=2400.09	? (TO) TT=2520.09	? (TO) TT=2520.04	? (TO) TT=2520.1	? (TO) TT=2520.12
Frb/ frb-56-25-3-mgd_c18.xml		? (TO) TT=2519.9	? (TO) TT=2520.01	? (TO) TT=2520.06	? (TO) TT=2520.06	? (TO) TT=2400.12	? (TO) TT=2520.05	? (TO) TT=2520.12	? (TO) TT=2520.14	? (TO) TT=2520.09	? (TO) TT=2520.15	? (TO) TT=2519.97	? (TO) TT=2520.03	? (TO) TT=2519.94	? (TO) TT=2400.04	? (TO) TT=2520.01	? (TO) TT=2400.03	? (TO) TT=2520.08	? (TO) TT=2400.01	? (TO) TT=2520.1	? (TO) TT=2520.07	No data	No data	? (TO) TT=2519.86	? (TO) TT=2519.81	? (TO) TT=2519.92	? (TO) TT=2400.15	? (TO) TT=2520.07	? (TO) TT=2520.07	? (TO) TT=2520.06	? (TO) TT=2520.1
Frb/ frb-59-26-1-mgd_c18.xml		? (TO) TT=2519.93	? (TO) TT=2520	? (TO) TT=2519.87	? (TO) TT=2519.93	? (TO) TT=2400.05	? (TO) TT=2520.13	? (TO) TT=2520.1	? (TO) TT=2520.09	? (TO) TT=2520.12	? (TO) TT=2520.04	? (TO) TT=2519.82	? (TO) TT=2519.92	? (TO) TT=2520.02	? (TO) TT=2400.05	? (TO) TT=2519.91	? (TO) TT=2400.07	? (TO) TT=2520.09	? (TO) TT=2400.11	? (TO) TT=2520.11	? (TO) TT=2520.1	No data	No data	? (TO) TT=2519.87	? (TO) TT=2519.92	? (TO) TT=2519.75	? (TO) TT=2400.12	? (TO) TT=2520.08	? (TO) TT=2520.09	? (TO) TT=2520.14	? (TO) TT=2520.05
Frb/ frb-59-26-2-mgd_c18.xml		? (TO) TT=2519.93	? (TO) TT=2520.01	? (TO) TT=2520.08	? (TO) TT=2520.07	? (TO) TT=2400.07	? (TO) TT=2520.09	? (TO) TT=2520.08	? (TO) TT=2520.14	? (TO) TT=2520.09	? (TO) TT=2520.08	? (TO) TT=2520.06	? (TO) TT=2519.94	? (TO) TT=2520.05	? (TO) TT=2400.06	? (TO) TT=2519.89	? (TO) TT=2400.08	? (TO) TT=2520.04	? (TO) TT=2400.02	? (TO) TT=2520.01	? (TO) TT=2520.06	No data	No data	? (TO) TT=2519.95	? (TO) TT=2520.07	? (TO) TT=2519.76	? (TO) TT=2400.08	? (TO) TT=2520.1	? (TO) TT=2520.06	? (TO) TT=2520.06	? (TO) TT=2520.06
Frb/ frb-59-26-3-mgd_c18.xml		? (TO) TT=2520.06	? (TO) TT=2520.06	? (TO) TT=2519.97	? (TO) TT=2520.05	? (TO) TT=2400.09	? (TO) TT=2520.1	? (TO) TT=2520.15	? (TO) TT=2520.05	? (TO) TT=2520.08	? (TO) TT=2520.12	? (TO) TT=2519.79	? (TO) TT=2520.06	? (TO) TT=2520.06	? (TO) TT=2400.08	? (TO) TT=2519.97	? (TO) TT=2400.05	? (TO) TT=2520.07	? (TO) TT=2400.04	? (TO) TT=2520.05	? (TO) TT=2520.1	No data	No data	? (TO) TT=2519.88	? (TO) TT=2519.98	? (TO) TT=2519.94	? (TO) TT=2400.12	? (TO) TT=2520.06	? (TO) TT=2520.15	? (TO) TT=2520.04	? (TO) TT=2520.06
GolombRuler/ GolombRuler-a3v18-07_c18.xml	OPT f=25 TT=0.04 T1=0.02	No data	No data	No data	No data	OPT f=25 TT=2.82 T1=0.8	OPT f=25 TT=2.867 T1=0.82	OPT f=25 TT=7.76 T1=3.48	OPT f=25 TT=8.107 T1=3.54	OPT f=25 TT=7.55 T1=3.47	OPT f=25 TT=7.908 T1=3.78	OPT f=25 TT=0.04 T1=0.02	No data	No data	OPT f=25 TT=0.03 T1=0.01	OPT f=25 TT=0.03 T1=0.01	No data	No data	SAT (TO) f=25 TT=2400.07 T1=1.14	SAT (TO) f=25 TT=2520.04 T1=1.09	OPT f=25 TT=1.83 T1=1.45	OPT f=25 TT=1.967 T1=1.16	OPT f=25 TT=1.905 T1=1.13	OPT f=25 TT=16.126 T1=16.12	OPT f=25 TT=16.156 T1=16.15	OPT f=25 TT=16.199 T1=16.19	OPT f=25 TT=13.101 T1=4.24	No data	No data	No data	No data
GolombRuler/ GolombRuler-a3v18-08_c18.xml	OPT f=34 TT=0.295 T1=0.1	No data	No data	No data	No data	OPT f=34 TT=8.26 T1=2.52	OPT f=34 TT=7.983 T1=2.52	OPT f=34 TT=10.078 T1=3.95	OPT f=34 TT=11.453 T1=4.45	OPT f=34 TT=9.729 T1=4.12	OPT f=34 TT=10.239 T1=4.26	OPT f=34 TT=0.295 T1=0.1	No data	No data	OPT f=34 TT=0.171 T1=0.03	OPT f=34 TT=0.171 T1=0.03	No data	No data	SAT (TO) f=34 TT=2400.08 T1=1.31	SAT (TO) f=34 TT=2520.06 T1=1.28	OPT f=34 TT=2.708 T1=1.28	OPT f=34 TT=2.454 T1=1.21	OPT f=34 TT=2.308 T1=1.14	OPT f=34 TT=128.998 T1=128.99	OPT f=34 TT=125.737 T1=125.73	OPT f=34 TT=126.897 T1=126.91	OPT f=34 TT=57.969 T1=12.77	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
GolombRuler/ GolombRuler-a3v18-09_c18.xml	OPT f=44 TT=1.814 T1=1.34	No data	No data	No data	No data	OPT f=44 TT=71.518 T1=26.74	OPT f=44 TT=71.057 T1=26.57	OPT f=44 TT=15.378 T1=6.46	OPT f=44 TT=17.868 T1=6.62	OPT f=44 TT=15.351 T1=6.75	OPT f=44 TT=14.762 T1=4.93	OPT f=44 TT=1.814 T1=1.34	No data	No data	OPT f=44 TT=1.665 T1=0.05	OPT f=44 TT=1.673 T1=0.05	No data	No data	SAT (TO) f=44 TT=2400.08 T1=2.96	SAT (TO) f=44 TT=2520.09 T1=2.06	OPT f=44 TT=4.397 T1=2.04	OPT f=44 TT=4.029 T1=1.78	OPT f=44 TT=3.945 T1=1.71	OPT f=44 TT=598.656 T1=598.7	OPT f=44 TT=655.093 T1=655.1	OPT f=44 TT=657.285 T1=657.3	OPT f=44 TT=806.516 T1=129.32	No data	No data	No data	No data
GolombRuler/ GolombRuler-a3v18-10_c18.xml	OPT f=55 TT=7.689 T1=4.07	No data	No data	No data	No data	OPT f=55 TT=365.061 T1=293.67	OPT f=55 TT=369.069 T1=297.28	OPT f=55 TT=40.154 T1=18.94	OPT f=55 TT=114.648 T1=95.17	OPT f=55 TT=49.317 T1=14.53	OPT f=55 TT=47.295 T1=9.61	OPT f=55 TT=14.977 T1=8.84	No data	No data	OPT f=55 TT=12.626 T1=0.43	OPT f=55 TT=12.604 T1=0.43	No data	No data	SAT (TO) f=55 TT=2400.07 T1=7.24	SAT (TO) f=55 TT=2520.02 T1=9.81	OPT f=55 TT=24.211 T1=5.74	OPT f=55 TT=7.771 T1=4.2	OPT f=55 TT=7.689 T1=4.07	OPT f=55 TT=881.948 T1=882	OPT f=55 TT=713.984 T1=713.91	OPT f=55 TT=707.903 T1=707.88	SAT (TO) f=55 TT=2400.26 T1=1333.9	No data	No data	No data	No data
GolombRuler/ GolombRuler-a3v18-11_c18.xml	OPT f=72 TT=368.702 T1=5.84	No data	No data	No data	No data	SAT (TO) f=72 TT=2400.07 T1=2204.6	SAT (TO) f=72 TT=2520.11 T1=2235.79	OPT f=72 TT=1068.27 T1=5.84	SAT (TO) f=72 TT=2520.12 T1=716.9	OPT f=72 TT=1006.29 T1=390.47	OPT f=72 TT=2290.36 T1=44.53	OPT f=72 TT=368.702 T1=76.52	No data	No data	OPT f=72 TT=490.463 T1=7.21	OPT f=72 TT=493.547 T1=7.28	No data	No data	SAT (TO) f=72 TT=2400.06 T1=187.79	SAT (TO) f=72 TT=2520.09 T1=24.31	OPT f=72 TT=840.76 T1=40.27	SAT (TO) f=76 TT=2400.12 T1=17.01	SAT (TO) f=76 TT=2520.03 T1=15.17	? (TO) TT=2519.82	? (TO) TT=2520.04	? (TO) TT=2520.07	SAT (TO) f=80 TT=2400.21 T1=1440.9	No data	No data	No data	No data
GolombRuler/ GolombRuler-a3v18-12_c18.xml	SAT f=85 TT=2520.06 T1=326.64	No data	No data	No data	No data	SAT (TO) f=98 TT=2400.06 T1=988.23	SAT (TO) f=98 TT=2520.07 T1=979.84	SAT (TO) f=85 TT=2520.07 T1=713.5	SAT (TO) f=91 TT=2520.14 T1=1032.24	SAT (TO) f=85 TT=2520.05 T1=505.38	SAT (TO) f=92 TT=2520.12 T1=2267.48	SAT (TO) f=91 TT=2519.79 T1=294.57	No data	No data	SAT (TO) f=85 TT=2400.04 T1=1675.1	SAT (TO) f=85 TT=2519.94 T1=1670.18	No data	No data	SAT (TO) f=85 TT=2400.02 T1=745.04	SAT (TO) f=85 TT=2520.06 T1=326.64	SAT (TO) f=85 TT=2520.07 T1=1368.93	SAT (TO) f=97 TT=2400.09 T1=22.97	SAT (TO) f=96 TT=2520.05 T1=47.51	? (TO) TT=2519.96	? (TO) TT=2520.05	? (TO) TT=2519.97	SAT (TO) f=107 TT=2400.37 T1=2136.4	No data	No data	No data	No data
GolombRuler/ GolombRuler-a3v18-13_c18.xml	SAT f=107 TT=2520.11 T1=1076.11	No data	No data	No data	No data	SAT (TO) f=119 TT=2400.05 T1=1914.07	SAT (TO) f=119 TT=2520.02 T1=1915.15	SAT (TO) f=114 TT=2520.07 T1=731.66	SAT (TO) f=116 TT=2520.1 T1=609.69	SAT (TO) f=111 TT=2520.1 T1=1997.28	SAT (TO) f=117 TT=2520.13 T1=59.38	SAT (TO) f=113 TT=2519.79 T1=880.58	No data	No data	SAT (TO) f=110 TT=2400.09 T1=21.15	SAT (TO) f=110 TT=2519.91 T1=21.29	No data	No data	SAT (TO) f=111 TT=2400.04 T1=1119.94	SAT (TO) f=112 TT=2520.05 T1=2252.26	SAT (TO) f=107 TT=2520.11 T1=1076.11	SAT (TO) f=122 TT=2400.08 T1=70.08	SAT (TO) f=129 TT=2520.12 T1=10.66	? (TO) TT=2519.92	? (TO) TT=2519.82	? (TO) TT=2519.93	? (TO) TT=2400.29	No data	No data	No data	No data
GolombRuler/ GolombRuler-a3v18-14_c18.xml	SAT f=135 TT=2520.06 T1=355.42	No data	No data	No data	No data	SAT (TO) f=153 TT=2400.03 T1=2279.19	SAT (TO) f=153 TT=2520.08 T1=2283.28	SAT (TO) f=135 TT=2520.08 T1=355.42	SAT (TO) f=135 TT=2520.06 T1=686.54	SAT (TO) f=137 TT=2520.11 T1=36.78	SAT (TO) f=136 TT=2520.06 T1=2202.06	SAT (TO) f=141 TT=2519.81 T1=1716.42	No data	No data	SAT (TO) f=139 TT=2400.03 T1=408.58	SAT (TO) f=139 TT=2519.92 T1=409.92	No data	No data	SAT (TO) f=140 TT=2400.05 T1=1901.79	SAT (TO) f=140 TT=2520.08 T1=1407.06	SAT (TO) f=140 TT=2520.11 T1=384.82	SAT (TO) f=142 TT=2400.03 T1=321.41	SAT (TO) f=145 TT=2520.12 T1=1734.76	? (TO) TT=2519.77	? (TO) TT=2519.95	? (TO) TT=2519.77	? (TO) TT=2400.18	No data	No data	No data	No data
GolombRuler/ GolombRuler-a3v18-15_c18.xml	SAT f=168 TT=2520.1 T1=645.7	No data	No data	No data	No data	SAT (TO) f=189 TT=2400.1 T1=1657.22	SAT (TO) f=189 TT=2520.03 T1=1651.43	SAT (TO) f=168 TT=2520.13 T1=2284.49	SAT (TO) f=173 TT=2520.07 T1=693.83	SAT (TO) f=165 TT=2520.38 T1=908.6	SAT (TO) f=170 TT=2520.08 T1=2384.69	SAT (TO) f=176 TT=2519.69 T1=1606.59	No data	No data	SAT (TO) f=160 TT=2400.02 T1=2201.82	SAT (TO) f=160 TT=2519.98 T1=2230.61	No data	No data	SAT (TO) f=170 TT=2400.11 T1=266.16	SAT (TO) f=170 TT=2520.12 T1=824.19	SAT (TO) f=168 TT=2520.1 T1=645.7	SAT (TO) f=187 TT=2400.04 T1=502.19	SAT (TO) f=177 TT=2520.05 T1=188.85	? (TO) TT=2519.86	? (TO) TT=2520.12	? (TO) TT=2519.92	? (TO) TT=2400.08	No data	No data	No data	No data
GolombRuler/ GolombRuler-a3v18-16_c18.xml	SAT f=202 TT=2520.06 T1=1480.94	No data	No data	No data	No data	SAT (TO) f=227 TT=2400.09 T1=35.96	SAT (TO) f=227 TT=2520.03 T1=35.97	SAT (TO) f=202 TT=2520.08 T1=1480.94	SAT (TO) f=204 TT=2520.14 T1=2160.24	SAT (TO) f=206 TT=2520.08 T1=2467.87	SAT (TO) f=210 TT=2520.13 T1=1040.97	SAT (TO) f=212 TT=2519.79 T1=336.95	No data	No data	SAT (TO) f=185 TT=2400.08 T1=1999.74	SAT (TO) f=185 TT=2519.82 T1=2048.24	No data	No data	SAT (TO) f=203 TT=2400.07 T1=1052.55	SAT (TO) f=207 TT=2520.12 T1=736.62	SAT (TO) f=207 TT=2520.05 T1=1610.07	SAT (TO) f=221 TT=2400.06 T1=1707.75	SAT (TO) f=202 TT=2520.06 T1=2445.56	? (TO) TT=2519.98	? (TO) TT=2520.04	? (TO) TT=2520.03	? (TO) TT=2400.13	No data	No data	No data	No data
GolombRuler/ GolombRuler-a3v18-nodv-09_c18.xml	OPT f=44 TT=11.569 T1=7.1	No data	No data	No data	No data	OPT f=44 TT=69.167 T1=25.65	OPT f=44 TT=70.331 T1=26.21	OPT f=44 TT=104.7 T1=7.1	OPT f=44 TT=237.475 T1=60.13	OPT f=44 TT=75.46 T1=6.93	OPT f=44 TT=304.721 T1=165.85	OPT f=44 TT=11.569 T1=9.04	No data	No data	OPT f=44 TT=1.659 T1=0.05	OPT f=44 TT=1.659 T1=0.05	No data	No data	SAT (TO) f=44 TT=2400.02 T1=23.11	SAT (TO) f=44 TT=2520.07 T1=29.15	OPT f=44 TT=213.675 T1=45.54	SAT (TO) f=45 TT=2400.08 T1=26.11	SAT (TO) f=47 TT=2520.05 T1=97.28	OPT f=44 TT=598.995 T1=599.02	OPT f=44 TT=657.093 T1=657.08	OPT f=44 TT=656.308 T1=656.26	OPT f=44 TT=802.259 T1=128.86	No data	No data	No data	No data
GolombRuler/ GolombRuler-a3v18-nodv-10_c18.xml	OPT f=55 TT=174.248 T1=144.16	No data	No data	No data	No data	OPT f=55 TT=364.623 T1=293.64	OPT f=55 TT=370.921 T1=298.74	OPT f=55 TT=1138.06 T1=176.85	SAT (TO) f=58 TT=2520.07 T1=2408.8	OPT f=55 TT=1143.3 T1=37.06	SAT (TO) f=59 TT=2520.09 T1=811.69	OPT f=55 TT=174.248 T1=144.16	No data	No data	OPT f=55 TT=12.679 T1=0.43	OPT f=55 TT=12.724 T1=0.44	No data	No data	SAT (TO) f=55 TT=2400.1 T1=55.47	SAT (TO) f=60 TT=2520.12 T1=882.41	SAT (TO) f=61 TT=2520.07 T1=2030.42	SAT (TO) f=74 TT=2400.08 T1=2.25	SAT (TO) f=60 TT=2520.04 T1=114.79	OPT f=55 TT=848.852 T1=848.85	OPT f=55 TT=715.917 T1=715.93	OPT f=55 TT=715.455 T1=715.43	SAT (TO) f=55 TT=2400.98 T1=1316.23	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
GolombRuler/ GolombRuler-a3v18-nodv-11_c18.xml	SAT f=72 TT=2519.5 T1=379.72	No data	No data	No data	No data	SAT (TO) f=72 TT=2400.05 T1=2237.52	SAT (TO) f=72 TT=2520.09 T1=2200.54	SAT (TO) f=75 TT=2520.04 T1=1506.64	SAT (TO) f=72 TT=2520.05 T1=1656.19	SAT (TO) f=76 TT=2520.06 T1=61.59	SAT (TO) f=72 TT=2520.16 T1=187.85	SAT (TO) f=72 TT=2519.5 T1=379.72	No data	No data	OPT f=72 TT=489.046 T1=7.23	OPT f=72 TT=491.373 T1=7.25	No data	No data	SAT (TO) f=81 TT=2400.07 T1=263.64	SAT (TO) f=81 TT=2520.08 T1=1313.56	SAT (TO) f=79 TT=2520.02 T1=87.15	SAT (TO) f=79 TT=2400.12 T1=277.1	SAT (TO) f=77 TT=2520.05 T1=92.94	? (TO) TT=2519.85	? (TO) TT=2520.08	? (TO) TT=2519.82	SAT (TO) f=80 TT=2400.26 T1=1425.38	No data	No data	No data	No data
GracefulGraph/ GracefulGraph-K02-P09_c18.xml	SAT TT=0.014	SAT TT=1.132	SAT TT=1.121	SAT TT=0.022	SAT TT=0.022	SAT TT=1.736	SAT TT=1.761	SAT TT=7.402	SAT TT=9.186	SAT TT=17.169	SAT TT=11.553	? (TO) TT=2520.07	SAT TT=0.022	SAT TT=0.022	SAT TT=0.04	SAT TT=0.014	SAT TT=2.24	SAT TT=2.139	SAT TT=26.129	SAT TT=7.334	SAT TT=7.834	No data	No data	SAT TT=20.69	SAT TT=20.72	SAT TT=15.813	SAT TT=10.184	SAT TT=5.81	SAT TT=6.62	SAT TT=5.805	SAT TT=6.59
GracefulGraph/ GracefulGraph-K03-P03_c18.xml	SAT TT=0.01	SAT TT=0.016	SAT TT=0.017	SAT TT=0.011	SAT TT=0.011	SAT TT=1.105	SAT TT=1.115	SAT TT=6.964	SAT TT=6.862	SAT TT=7.489	SAT TT=10.146	SAT TT=0.581	SAT TT=0.011	SAT TT=0.011	SAT TT=0.01	SAT TT=0.01	SAT TT=1.776	SAT TT=1.674	SAT TT=1.952	SAT TT=2.044	SAT TT=2.036	No data	No data	SAT TT=0.835	SAT TT=0.378	SAT TT=0.831	SAT TT=1.573	SAT TT=3.304	SAT TT=4.491	SAT TT=3.268	SAT TT=4.504
GracefulGraph/ GracefulGraph-K04-P02_c18.xml	SAT TT=0.024	SAT TT=0.386	SAT TT=0.39	SAT TT=0.06	SAT TT=0.061	SAT TT=1.255	SAT TT=1.275	SAT TT=6.992	SAT TT=6.383	SAT TT=11.223	SAT TT=7.16	SAT TT=2.525	SAT TT=0.03	SAT TT=0.028	SAT TT=0.024	SAT TT=0.063	SAT TT=1.86	SAT TT=1.778	SAT TT=3.869	SAT TT=4.087	SAT TT=4.107	No data	No data	SAT TT=0.196	SAT TT=0.226	SAT TT=0.196	SAT TT=3.466	SAT TT=3.338	SAT TT=4.54	SAT TT=3.325	SAT TT=4.481
GracefulGraph/ GracefulGraph-K04-P03_c18.xml	SAT TT=0.556	SAT TT=429.053	SAT TT=426.527	SAT TT=5.229	SAT TT=5.222	SAT TT=5.476	SAT TT=5.464	SAT TT=10.119	SAT TT=13.232	SAT TT=21.147	SAT TT=13.575	SAT TT=1316.58	SAT TT=0.776	SAT TT=0.774	SAT TT=4.051	SAT TT=0.556	SAT TT=9.694	SAT TT=8.823	SAT TT=15.358	SAT TT=45.878	SAT TT=44.779	No data	No data	SAT TT=12.231	SAT TT=6.014	SAT TT=14.866	SAT TT=9.008	SAT TT=24.801	SAT TT=11.275	SAT TT=24.491	SAT TT=11.128
GracefulGraph/ GracefulGraph-K04-P04_c18.xml	SAT TT=8.522	SAT TT=1995.84	SAT TT=1955.42	SAT TT=402.536	SAT TT=395.867	SAT TT=65.269	SAT TT=65.507	SAT TT=17.853	SAT TT=8.522	SAT TT=136.582	SAT TT=313.129	? (TO) TT=2520.11	SAT TT=52.19	SAT TT=51.623	SAT TT=50.653	SAT TT=147.18	SAT TT=1163.63	SAT TT=1175.71	SAT TT=487.191	SAT TT=1617.31	SAT TT=772.262	No data	No data	SAT TT=1254.13	SAT TT=61.912	SAT TT=1267.58	? (TO) TT=2400.13	SAT TT=254.842	SAT TT=456.132	SAT TT=252.87	SAT TT=459.732
GracefulGraph/ GracefulGraph-K05-P02_c18.xml	SAT TT=21.896	SAT TT=440.608	SAT TT=438.617	SAT TT=83.832	SAT TT=84.353	SAT TT=56.824	SAT TT=56.197	SAT TT=45.189	SAT TT=62.908	SAT TT=464.988	SAT TT=238.51	? (TO) TT=2519.98	SAT TT=335.846	SAT TT=339.083	SAT TT=60.507	SAT TT=33.064	SAT TT=22.539	SAT TT=21.896	SAT TT=115.079	SAT TT=101.752	SAT TT=23.96	No data	No data	SAT TT=2217	SAT TT=2315.77	SAT TT=2274.87	? (TO) TT=2400.08	SAT TT=332.4	SAT TT=43.836	SAT TT=331.888	SAT TT=44.134
GracefulGraph/ GracefulGraph-K05-P03_c18.xml	SAT TT=89.932	? (TO) TT=2519.91	? (TO) TT=2519.98	? TT=2520.01	? (TO) TT=2519.99	? (TO) TT=2400.15	? (TO) TT=2520.09	? (TO) TT=2520.06	? (TO) TT=2520.12	? (TO) TT=2520.13	? (TO) TT=2520.08	? (TO) TT=2520.08	? TT=2520.02	? (TO) TT=2520	SAT TT=89.932	? (TO) TT=2519.81	? (TO) TT=2400.1	? (TO) TT=2520.1	? (TO) TT=2400.05	? (TO) TT=2520.11	? (TO) TT=2520.07	No data	No data	? (TO) TT=2520.11	? (TO) TT=2519.95	? (TO) TT=2519.88	? (TO) TT=2400.15	? (TO) TT=2520.13	? (TO) TT=2520.09	? (TO) TT=2520.1	? (TO) TT=2520.06
GracefulGraph/ GracefulGraph-K06-P02_c18.xml		? (TO) TT=2519.93	? (TO) TT=2519.84	? TT=2520.01	? TT=2520.01	? (TO) TT=2400.04	? (TO) TT=2520.1	? (TO) TT=2520.07	? (TO) TT=2520.15	? (TO) TT=2520.07	? (TO) TT=2520.07	? (TO) TT=2520.02	? (TO) TT=2519.93	? (TO) TT=2519.78	? (TO) TT=2400.03	? (TO) TT=2520.08	? (TO) TT=2400.07	? (TO) TT=2520.12	? (TO) TT=2400.03	? (TO) TT=2520.09	? (TO) TT=2520.05	No data	No data	? (TO) TT=2520.06	? (TO) TT=2520.06	? (TO) TT=2520.09	? (TO) TT=2400.13	? (TO) TT=2520.07	? (TO) TT=2520.1	? (TO) TT=2520.05	? (TO) TT=2520.06
GracefulGraph/ GracefulGraph-K06-P03_c18.xml		? (TO) TT=2519.91	? TT=2520.02	? (TO) TT=2520	? TT=2520.02	? (TO) TT=2400.12	? (TO) TT=2520.04	? (TO) TT=2520.12	? (TO) TT=2520.11	? (TO) TT=2520.09	? (TO) TT=2520.1	? (TO) TT=2519.71	? (TO) TT=2519.88	? (TO) TT=2519.92	? (TO) TT=2400.03	? (TO) TT=2519.96	? (TO) TT=2400.11	? (TO) TT=2520.08	? (TO) TT=2400.03	? (TO) TT=2520.1	? (TO) TT=2520.06	No data	No data	? (TO) TT=2519.74	? (TO) TT=2519.92	? (TO) TT=2520.02	? (TO) TT=2400.22	? (MO) TT=2237.57	? (TO) TT=2520.08	? (MO) TT=2214.41	? (TO) TT=2520.1
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
GracefulGraph/ GracefulGraph-K07-P02_c18.xml		? (TO) TT=2519.91	? (TO) TT=2519.96	? TT=2520.02	? (TO) TT=2519.99	? (TO) TT=2400.12	? (TO) TT=2520.1	? (TO) TT=2520.07	? (TO) TT=2520.06	? (TO) TT=2520.07	? (TO) TT=2520.13	? (TO) TT=2519.95	? (TO) TT=2519.78	? (TO) TT=2520	? (TO) TT=2400.08	? (TO) TT=2520.08	? (TO) TT=2400.11	? (TO) TT=2520.09	? (TO) TT=2400.05	? (TO) TT=2520.07	? (TO) TT=2520.11	No data	No data	? (TO) TT=2519.82	? (TO) TT=2520.03	? (TO) TT=2520.05	? (TO) TT=2400.12	? (TO) TT=2520.09	? (TO) TT=2520.07	? (TO) TT=2520.09	? (TO) TT=2520.15
GracefulGraph/ GracefulGraph-K07-P03_c18.xml		? (TO) TT=2519.92	? (TO) TT=2519.99	? TT=2520.02	? (TO) TT=2519.96	? (TO) TT=2400.13	? (TO) TT=2520.1	? (TO) TT=2520.1	? (TO) TT=2520.1	? (TO) TT=2520.11	? (TO) TT=2520.11	? (TO) TT=2520.05	? (TO) TT=2520	? (TO) TT=2519.99	? (TO) TT=2400.06	? (TO) TT=2520	? (TO) TT=2400.03	? (TO) TT=2520.04	? (TO) TT=2400.11	? (TO) TT=2520.09	? (TO) TT=2520.04	No data	No data	? (TO) TT=2519.98	? (TO) TT=2520.06	? (TO) TT=2520	? (TO) TT=2400.14	? (MO) TT=15.201	? (TO) TT=2520.08	? (MO) TT=15.453	? (TO) TT=2520.08
GraphColoring/ GraphColoring-2-fullins-5_c18.xml	OPT f=6 TT=352.426 T1=1.99	No data	No data	No data	No data	OPT f=6 TT=418.213 T1=2.21	OPT f=6 TT=420.793 T1=2.24	OPT f=6 TT=1759.38 T1=31.97	SAT (TO) f=6 TT=2520.15 T1=44.11	OPT f=6 TT=1191.92 T1=35.67	SAT (TO) f=6 TT=2520.08 T1=49.84	OPT f=6 TT=352.426 T1=1.99	No data	No data	OPT f=6 TT=423.487 T1=2.24	OPT f=6 TT=463.686 T1=2.22	No data	No data	SAT (TO) f=6 TT=2400.13 T1=53.13	SAT (TO) f=6 TT=2520.13 T1=56.19	SAT (TO) f=6 TT=2520.04 T1=55.04	SAT (TO) f=6 TT=2400.06 T1=69.14	SAT (TO) f=6 TT=2520.07 T1=69.09	OPT f=6 TT=443.011 T1=443.03	OPT f=6 TT=403.757 T1=403.71	OPT f=6 TT=388.688 T1=388.7	? TT=1461.12	No data	No data	No data	No data
GraphColoring/ GraphColoring-3-fullins-5_c18.xml	OPT f=7 TT=1856.54 T1=5.59	No data	No data	No data	No data	SAT (TO) f=7 TT=2400.11 T1=6.07	SAT (TO) f=7 TT=2520.09 T1=5.97	SAT (TO) f=7 TT=2520.16 T1=237.61	SAT (TO) f=7 TT=2520.31 T1=403.94	SAT (TO) f=7 TT=2520.09 T1=306.98	SAT (TO) f=7 TT=2520.07 T1=472.04	SAT (TO) f=7 TT=2520.03 T1=5.59	No data	No data	SAT (TO) f=7 TT=2400.08 T1=7.02	SAT (TO) f=7 TT=2519.91 T1=6.8	No data	No data	SAT (TO) f=7 TT=2400.08 T1=862.2	SAT (TO) f=7 TT=2520.05 T1=947.63	SAT (TO) f=7 TT=2520.09 T1=867.46	SAT (TO) f=1142 TT=2400.11 T1=2246.04	SAT (TO) f=1093 TT=2520.09 T1=2350.52	OPT f=7 TT=1972.24 T1=1972.12	OPT f=7 TT=1863.81 T1=1863.69	OPT f=7 TT=1856.54 T1=1856.75	? (problem) TT=1597.18	No data	No data	No data	No data
GraphColoring/ GraphColoring-4-fullins-4_c18.xml	OPT f=7 TT=18.483 T1=1.06	No data	No data	No data	No data	OPT f=7 TT=75.415 T1=1.47	OPT f=7 TT=75.726 T1=1.5	OPT f=7 TT=130.931 T1=19.06	OPT f=7 TT=244.725 T1=25.11	OPT f=7 TT=126.234 T1=22.77	OPT f=7 TT=272.291 T1=28.69	OPT f=7 TT=18.483 T1=1.06	No data	No data	OPT f=7 TT=34.325 T1=1.23	OPT f=7 TT=38.548 T1=1.18	No data	No data	SAT (TO) f=7 TT=2400.07 T1=20.26	SAT (TO) f=7 TT=2520.1 T1=19.99	OPT f=7 TT=120.575 T1=20.6	OPT f=7 TT=54.43 T1=30.39	OPT f=7 TT=51.281 T1=28.09	OPT f=7 TT=250.005 T1=250.03	OPT f=7 TT=259.467 T1=259.5	OPT f=7 TT=247.066 T1=247.07	? TT=1334.2	No data	No data	No data	No data
GraphColoring/ GraphColoring-4-fullins-5_c18.xml	SAT f=8 TT=2519.94 T1=13.41	No data	No data	No data	No data	SAT (TO) f=8 TT=2400.12 T1=23.33	SAT (TO) f=8 TT=2520.07 T1=23.71	SAT (TO) f=8 TT=2520.14 T1=1833.56	SAT (TO) f=3369 TT=2520.12 T1=2021.37	SAT (TO) f=3322 TT=2520.07 T1=2363.56	? (TO) TT=2523.13	SAT (TO) f=8 TT=2519.94 T1=13.41	No data	No data	SAT (TO) f=8 TT=2400.01 T1=19.23	SAT (TO) f=8 TT=2520.02 T1=19.05	No data	No data	SAT (TO) f=3733 TT=2400.13 T1=2332.84	SAT (TO) f=3714 TT=2520.32 T1=2450.06	SAT (TO) f=3757 TT=2520.1 T1=2449.31	SAT (TO) f=3844 TT=2406.2 T1=1997.29	SAT (TO) f=3861 TT=2520.11 T1=2011.25	? (TO) TT=2520.03	? (TO) TT=2520.09	? (TO) TT=2519.94	? TT=2087.04	No data	No data	No data	No data
GraphColoring/ GraphColoring-5-fullins-3_c18.xml	OPT f=7 TT=0.362 T1=0.13	No data	No data	No data	No data	OPT f=7 TT=4.862 T1=0.76	OPT f=7 TT=5.772 T1=0.74	OPT f=7 TT=15.786 T1=4.96	OPT f=7 TT=18.729 T1=5.32	OPT f=7 TT=19.08 T1=5.03	OPT f=7 TT=19.189 T1=5.38	OPT f=7 TT=0.362 T1=0.13	No data	No data	OPT f=7 TT=0.493 T1=0.14	OPT f=7 TT=0.486 T1=0.14	No data	No data	SAT (TO) f=7 TT=2400.1 T1=1.94	SAT (TO) f=7 TT=2520.11 T1=1.72	OPT f=7 TT=36.793 T1=1.7	SAT (TO) f=7 TT=2400.08 T1=2.07	SAT (TO) f=7 TT=2520.04 T1=1.96	OPT f=7 TT=23.124 T1=23.12	OPT f=7 TT=25.323 T1=25.33	OPT f=7 TT=24.309 T1=24.31	OPT f=7 TT=677.114 T1=612.58	No data	No data	No data	No data
GraphColoring/ GraphColoring-5-fullins-4_c18.xml	OPT f=8 TT=141.733 T1=1.86	No data	No data	No data	No data	OPT f=8 TT=1520.94 T1=2.37	OPT f=8 TT=1534.02 T1=2.45	OPT f=8 TT=1429.03 T1=38.43	SAT (TO) f=8 TT=2520.08 T1=67.15	OPT f=8 TT=1716.21 T1=45.2	SAT (TO) f=8 TT=2520.13 T1=86.94	OPT f=8 TT=141.733 T1=1.86	No data	No data	OPT f=8 TT=679.826 T1=2.08	OPT f=8 TT=628.282 T1=2.11	No data	No data	SAT (TO) f=8 TT=2400.13 T1=79.3	SAT (TO) f=8 TT=2520.09 T1=81.25	SAT (TO) f=8 TT=2520.08 T1=78.47	SAT (TO) f=8 TT=2400.11 T1=104.01	SAT (TO) f=8 TT=2520.13 T1=102.23	OPT f=8 TT=387.453 T1=387.47	OPT f=8 TT=402.843 T1=402.79	OPT f=8 TT=385.347 T1=385.34	? TT=1576.75	No data	No data	No data	No data
GraphColoring/ GraphColoring-qg-order100_c18.xml	SAT f=101 TT=2520.13 T1=663.17	No data	No data	No data	No data	SAT (TO) f=101 TT=2401.76 T1=661.94	SAT (TO) f=101 TT=2520.13 T1=663.17	? (TO) TT=2528.58	? (TO) TT=2528.04	? (TO) TT=2522.21	? (TO) TT=2522.29	? (MO) TT=70.579	No data	No data	SAT (TO) f=107 TT=2400.17 T1=1254.8	SAT (TO) f=107 TT=2520.07 T1=1289.41	No data	No data	? TT=603.196	? TT=693.873	? TT=655.225	? TT=726.303	? TT=677.957	? (TO) TT=2519.94	? (TO) TT=2520.31	? (TO) TT=2519.93	? (TO) TT=2408.3	No data	No data	No data	No data
GraphColoring/ GraphColoring-qg-order30_c18.xml	OPT f=29 TT=303.273 T1=3.18	No data	No data	No data	No data	SAT (TO) f=29 TT=2400.09 T1=3.19	SAT (TO) f=29 TT=2520.08 T1=3.18	OPT f=29 TT=303.273 T1=281.97	OPT f=29 TT=1902.07 T1=1861.94	OPT f=29 TT=103.443 T1=84.12	OPT f=29 TT=358.86 T1=320.17	SAT (TO) f=29 TT=2519.9 T1=4.46	No data	No data	SAT (TO) f=29 TT=2400.06 T1=5.17	SAT (TO) f=29 TT=2519.96 T1=5.13	No data	No data	SAT (TO) f=29 TT=2400.06 T1=153.89	SAT (TO) f=29 TT=2520.06 T1=209.93	SAT (TO) f=29 TT=2520.04 T1=113.59	SAT (TO) f=29 TT=2400.14 T1=312.06	SAT (TO) f=29 TT=2520.14 T1=459.76	? (TO) TT=2519.84	? (TO) TT=2519.8	? (TO) TT=2519.98	? TT=1459.8	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
GraphColoring/ GraphColoring-qg-order40_c18.xml	OPT f=39 TT=1305.33 T1=9.52	No data	No data	No data	No data	SAT (TO) f=39 TT=2400.09 T1=9.1	SAT (TO) f=39 TT=2520.12 T1=9.52	OPT f=39 TT=1305.33 T1=1264.52	SAT (TO) f=345 TT=2520.07 T1=2466.37	OPT f=39 TT=433.23 T1=401.31	OPT f=39 TT=2136.6 T1=2001.81	SAT (TO) f=39 TT=2519.82 T1=10.74	No data	No data	SAT (TO) f=39 TT=2400.02 T1=15.41	SAT (TO) f=39 TT=2519.6 T1=15.61	No data	No data	SAT (TO) f=41 TT=2400.06 T1=1302.21	SAT (TO) f=41 TT=2520.13 T1=1315.61	SAT (TO) f=41 TT=2520.06 T1=1391.91	SAT (TO) f=143 TT=2400.07 T1=2309.97	SAT (TO) f=558 TT=2520.11 T1=2386.65	? (TO) TT=2519.63	? (TO) TT=2520.05	? (TO) TT=2520.07	? TT=1393.73	No data	No data	No data	No data
GraphColoring/ GraphColoring-qg-order60_c18.xml	SAT f=59 TT=2519.69 T1=74.09	No data	No data	No data	No data	SAT (TO) f=59 TT=2400.1 T1=223.44	SAT (TO) f=59 TT=2520.08 T1=220.88	SAT (TO) f=2842 TT=2520.12 T1=2346.03	SAT (TO) f=3207 TT=2520.1 T1=2371.39	SAT (TO) f=3002 TT=2520.15 T1=2330.69	SAT (TO) f=3275 TT=2520.23 T1=2365.72	SAT (TO) f=59 TT=2519.69 T1=74.09	No data	No data	SAT (TO) f=59 TT=2400.09 T1=142.59	SAT (TO) f=59 TT=2519.72 T1=142.02	No data	No data	? TT=505.368	? TT=502.799	? TT=391.083	? TT=421.082	? (TO) TT=2527.14	? (TO) TT=2520.11	? (TO) TT=2520.08	? (TO) TT=2519.86	? TT=2025.01	No data	No data	No data	No data
GraphColoring/ GraphColoring-queen5-5_c18.xml	OPT f=4 TT=0.03 T1=0.03	No data	No data	No data	No data	OPT f=4 TT=1.225 T1=0.57	OPT f=4 TT=1.197 T1=0.58	OPT f=4 TT=7.749 T1=3.65	OPT f=4 TT=7.592 T1=3.58	OPT f=4 TT=7.426 T1=3.65	OPT f=4 TT=7.2 T1=4.16	OPT f=4 TT=0.03 T1=0.03	No data	No data	OPT f=4 TT=0.032 T1=0.03	OPT f=4 TT=0.032 T1=0.03	No data	No data	SAT (TO) f=4 TT=2400.04 T1=1.48	SAT (TO) f=4 TT=2520.05 T1=1.14	OPT f=4 TT=1.934 T1=1.14	OPT f=4 TT=2.071 T1=1.19	OPT f=4 TT=1.996 T1=1.14	OPT f=4 TT=1.195 T1=1.19	OPT f=4 TT=1.512 T1=1.51	OPT f=4 TT=1.517 T1=1.52	OPT f=4 TT=3.126 T1=1.34	No data	No data	No data	No data
Haystacks/ Haystacks-ext-04_c18.xml	UNSAT TT=0.005	UNSAT TT=0.007	UNSAT TT=0.007	UNSAT TT=0.007	UNSAT TT=0.007	UNSAT TT=0.895	UNSAT TT=0.861	UNSAT TT=6.468	UNSAT TT=6.43	UNSAT TT=6.163	UNSAT TT=6.245	UNSAT TT=0.005	UNSAT TT=0.009	UNSAT TT=0.01	UNSAT TT=0.009	UNSAT TT=0.009	UNSAT TT=1.556	UNSAT TT=1.533	? (TO) TT=2400.05	? (TO) TT=2520.05	UNSAT TT=1.57	No data	No data	UNSAT TT=0.008	UNSAT TT=0.007	UNSAT TT=0.007	UNSAT TT=0.63	UNSAT TT=2.33	UNSAT TT=3.493	UNSAT TT=2.275	UNSAT TT=3.437
Haystacks/ Haystacks-ext-05_c18.xml	UNSAT TT=0.008	UNSAT TT=0.008	UNSAT TT=0.008	UNSAT TT=0.009	UNSAT TT=0.008	UNSAT TT=3.612	UNSAT TT=3.711	UNSAT TT=13.701	UNSAT TT=19.718	UNSAT TT=7.802	UNSAT TT=9.692	UNSAT TT=3.643	UNSAT TT=0.617	UNSAT TT=0.646	UNSAT TT=0.399	UNSAT TT=0.588	UNSAT TT=2.512	UNSAT TT=2.458	? (TO) TT=2400.07	? (TO) TT=2520.03	UNSAT TT=3.212	No data	No data	UNSAT TT=0.013	UNSAT TT=0.012	UNSAT TT=0.012	UNSAT TT=0.752	UNSAT TT=2.768	UNSAT TT=3.948	UNSAT TT=2.735	UNSAT TT=3.93
Haystacks/ Haystacks-ext-06_c18.xml	UNSAT TT=0.012	UNSAT TT=0.013	UNSAT TT=0.012	UNSAT TT=0.015	UNSAT TT=0.014	UNSAT TT=32.326	UNSAT TT=32.108	UNSAT TT=701.228	UNSAT TT=420.524	UNSAT TT=26.088	UNSAT TT=19.661	? (TO) TT=2519.81	? (TO) TT=2519.81	? (TO) TT=2519.94	UNSAT TT=12.934	? (TO) TT=2519.91	UNSAT TT=460.911	UNSAT TT=453.202	? (TO) TT=2400.09	? (TO) TT=2520.09	? (TO) TT=2520.06	No data	No data	UNSAT TT=0.02	UNSAT TT=0.019	UNSAT TT=0.019	UNSAT TT=1.003	UNSAT TT=3.242	UNSAT TT=4.443	UNSAT TT=3.343	UNSAT TT=4.542
Haystacks/ Haystacks-ext-07_c18.xml	UNSAT TT=0.037	UNSAT TT=0.039	UNSAT TT=0.037	UNSAT TT=0.042	UNSAT TT=0.039	UNSAT TT=96.702	UNSAT TT=99.31	? (TO) TT=2520.11	? (TO) TT=2520.05	? (TO) TT=2520.04	? (TO) TT=2520.09	? (TO) TT=2519.94	? (TO) TT=2519.93	? (TO) TT=2519.81	? (TO) TT=2400.09	? (TO) TT=2520.01	? (TO) TT=2400.05	? (TO) TT=2520.08	? (TO) TT=2400.08	? (TO) TT=2520.04	? (TO) TT=2520.04	No data	No data	UNSAT TT=0.059	UNSAT TT=0.058	UNSAT TT=0.057	UNSAT TT=1.115	UNSAT TT=3.92	UNSAT TT=5.085	UNSAT TT=3.986	UNSAT TT=5.109
Haystacks/ Haystacks-ext-08_c18.xml	UNSAT TT=0.122	UNSAT TT=0.242	UNSAT TT=0.226	UNSAT TT=0.245	UNSAT TT=0.225	? (TO) TT=2400.04	? (TO) TT=2520.02	? (TO) TT=2520.05	? (TO) TT=2520.07	? (TO) TT=2520.11	? (TO) TT=2520.1	? (TO) TT=2519.94	? (TO) TT=2519.94	? (TO) TT=2519.99	? (TO) TT=2400.05	? (TO) TT=2519.91	? (TO) TT=2400.09	? (TO) TT=2520.02	? (TO) TT=2400.07	? (TO) TT=2520.07	? (TO) TT=2520.09	No data	No data	UNSAT TT=0.122	UNSAT TT=0.122	UNSAT TT=0.122	UNSAT TT=1.788	UNSAT TT=4.764	UNSAT TT=5.884	UNSAT TT=4.679	UNSAT TT=5.882
Haystacks/ Haystacks-ext-09_c18.xml	UNSAT TT=0.361	UNSAT TT=1.952	UNSAT TT=1.802	UNSAT TT=2.06	UNSAT TT=1.975	? (TO) TT=2400.12	? (TO) TT=2520.06	? (TO) TT=2520.12	? (TO) TT=2520.1	? (TO) TT=2520.1	? (TO) TT=2520.06	? (TO) TT=2519.94	? (TO) TT=2519.94	? (TO) TT=2519.81	? (TO) TT=2400.08	? (TO) TT=2519.98	? (TO) TT=2400.02	? (TO) TT=2520.06	? (TO) TT=2400.04	? (TO) TT=2520.05	? (TO) TT=2520.12	No data	No data	UNSAT TT=0.361	UNSAT TT=0.361	UNSAT TT=0.361	UNSAT TT=4.072	UNSAT TT=5.674	UNSAT TT=6.859	UNSAT TT=5.476	UNSAT TT=6.752
Haystacks/ Haystacks-ext-10_c18.xml	UNSAT TT=0.957	UNSAT TT=18.064	UNSAT TT=16.993	UNSAT TT=19.406	UNSAT TT=18.199	? (TO) TT=2400.1	? (TO) TT=2520.1	? (TO) TT=2520.09	? (TO) TT=2520.1	? (TO) TT=2520.08	? (TO) TT=2520.08	? (TO) TT=2519.94	? (TO) TT=2519.91	? (TO) TT=2519.97	? (TO) TT=2400.05	? (TO) TT=2519.83	? (TO) TT=2400.12	? (TO) TT=2520.11	? (TO) TT=2400.12	? (TO) TT=2520.06	? (TO) TT=2520.07	No data	No data	UNSAT TT=0.966	UNSAT TT=0.958	UNSAT TT=0.957	UNSAT TT=39.312	UNSAT TT=7.111	UNSAT TT=8.428	UNSAT TT=7.037	UNSAT TT=8.358
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
Haystacks/ Haystacks-ext-11_c18.xml	UNSAT TT=3.066	UNSAT TT=199.16	UNSAT TT=184.419	UNSAT TT=213.531	UNSAT TT=195.961	? (TO) TT=2400.09	? (TO) TT=2520.17	? (TO) TT=2520.07	? (TO) TT=2520.13	? (TO) TT=2520.14	? (TO) TT=2520.07	? (TO) TT=2519.79	? (TO) TT=2519.82	? (TO) TT=2519.91	? (TO) TT=2400.04	? (TO) TT=2520.04	? (TO) TT=2400.05	? (TO) TT=2520.05	? (TO) TT=2400.03	? (TO) TT=2520.11	? (TO) TT=2520.07	No data	No data	UNSAT TT=3.085	UNSAT TT=3.095	UNSAT TT=3.066	UNSAT TT=1437.13	UNSAT TT=8.695	UNSAT TT=11.213	UNSAT TT=8.532	UNSAT TT=11.195
Haystacks/ Haystacks-ext-12_c18.xml	UNSAT TT=8.231	UNSAT TT=2436.75	UNSAT TT=2260.86	? (TO) TT=2519.95	UNSAT TT=2404.68	? (TO) TT=2400.13	? (TO) TT=2520.04	? (TO) TT=2520.07	? (TO) TT=2520.03	? (TO) TT=2520.05	? (TO) TT=2520.13	? (TO) TT=2519.74	? (TO) TT=2519.77	? (TO) TT=2520.01	? (TO) TT=2400.08	? (TO) TT=2519.81	? (TO) TT=2400.07	? (TO) TT=2520.07	? (TO) TT=2400.05	? (TO) TT=2520.05	? (TO) TT=2520.04	No data	No data	UNSAT TT=8.259	UNSAT TT=8.284	UNSAT TT=8.231	? (TO) TT=2400.1	UNSAT TT=12.378	UNSAT TT=16.446	UNSAT TT=12.152	UNSAT TT=16.4
Haystacks/ Haystacks-ext-13_c18.xml	UNSAT TT=22.866	? TT=2520.02	? (TO) TT=2519.96	? TT=2520.02	? (TO) TT=2519.9	? (TO) TT=2400.08	? (TO) TT=2520.08	? (TO) TT=2520.1	? (TO) TT=2520.07	? (TO) TT=2520.12	? (TO) TT=2520.12	? (TO) TT=2519.78	? (TO) TT=2519.81	? (TO) TT=2519.95	? (TO) TT=2400.06	? (TO) TT=2519.82	? (TO) TT=2400.07	? (TO) TT=2520.07	? (TO) TT=2400.1	? (TO) TT=2520.04	? (TO) TT=2520.07	No data	No data	UNSAT TT=62.407	UNSAT TT=62.068	UNSAT TT=61.969	? (TO) TT=2400.08	UNSAT TT=22.956	UNSAT TT=33.332	UNSAT TT=22.866	UNSAT TT=33.339
Knapsack/ Knapsack-30-100-00_c18.xml	OPT f=709 TT=0.748 T1=0.43	No data	No data	No data	No data	OPT f=709 TT=4.461 T1=1.3	OPT f=709 TT=5.467 T1=1.27	OPT f=709 TT=15.253 T1=3.56	OPT f=709 TT=14.911 T1=4.05	OPT f=709 TT=14.656 T1=3.92	OPT f=709 TT=13.074 T1=3.67	OPT f=709 TT=0.748 T1=0.43	No data	No data	OPT f=709 TT=0.097 T1=0.01	OPT f=709 TT=0.098 T1=0.01	No data	No data	SAT (TO) f=709 TT=2400.04 T1=1.16	SAT (TO) f=709 TT=2520.11 T1=1.14	OPT f=709 TT=2.228 T1=1.43	OPT f=709 TT=2.272 T1=1.09	OPT f=709 TT=2.208 T1=1.06	OPT f=709 TT=4.505 T1=4.5	OPT f=709 TT=3.5 T1=3.49	OPT f=709 TT=3.517 T1=3.51	OPT f=709 TT=107.175 T1=21.24	No data	No data	No data	No data
Knapsack/ Knapsack-30-100-01_c18.xml	OPT f=871 TT=0.457 T1=0.35	No data	No data	No data	No data	OPT f=871 TT=3.502 T1=1.25	OPT f=871 TT=4.079 T1=1.7	OPT f=871 TT=10.862 T1=3.46	OPT f=871 TT=12.23 T1=3.64	OPT f=871 TT=12.593 T1=3.51	OPT f=871 TT=12.342 T1=3.58	OPT f=871 TT=0.457 T1=0.35	No data	No data	OPT f=871 TT=0.07 T1=0	OPT f=871 TT=0.072 T1=0.01	No data	No data	SAT (TO) f=871 TT=2400.07 T1=1.22	SAT (TO) f=871 TT=2520.07 T1=1.23	OPT f=871 TT=2.172 T1=1.22	OPT f=871 TT=2.329 T1=1.24	OPT f=871 TT=2.292 T1=1.21	OPT f=871 TT=2.907 T1=2.9	OPT f=871 TT=4.457 T1=4.45	OPT f=871 TT=4.488 T1=4.49	OPT f=871 TT=88.779 T1=5.55	No data	No data	No data	No data
Knapsack/ Knapsack-40-150-00_c18.xml	OPT f=932 TT=6.7 T1=1.89	No data	No data	No data	No data	OPT f=932 TT=81.392 T1=44.77	OPT f=932 TT=79.985 T1=44.55	OPT f=932 TT=149.288 T1=4.84	OPT f=932 TT=257.173 T1=5.04	OPT f=932 TT=153.291 T1=4.72	OPT f=932 TT=282.717 T1=3.54	OPT f=932 TT=27.044 T1=20.6	No data	No data	OPT f=932 TT=3.39 T1=0.01	OPT f=932 TT=3.369 T1=0.01	No data	No data	SAT (TO) f=932 TT=2400.03 T1=1.89	SAT (TO) f=932 TT=2520.04 T1=1.89	OPT f=932 TT=9.678 T1=1.92	OPT f=932 TT=6.548 T1=4.35	OPT f=932 TT=6.7 T1=4.19	OPT f=932 TT=77.622 T1=77.62	OPT f=932 TT=60.92 T1=60.91	OPT f=932 TT=61.198 T1=61.19	SAT (TO) f=923 TT=2400.37 T1=1110.01	No data	No data	No data	No data
Knapsack/ Knapsack-40-150-01_c18.xml	OPT f=1128 TT=4.069 T1=1.78	No data	No data	No data	No data	OPT f=1128 TT=35.342 T1=18.46	OPT f=1128 TT=39.42 T1=19.86	OPT f=1128 TT=67.957 T1=3.81	OPT f=1128 TT=117.445 T1=4.76	OPT f=1128 TT=114.884 T1=3.92	OPT f=1128 TT=129.606 T1=4.29	OPT f=1128 TT=19.243 T1=18.05	No data	No data	OPT f=1128 TT=1.921 T1=0.02	OPT f=1128 TT=1.947 T1=0.02	No data	No data	SAT (TO) f=1128 TT=2400.11 T1=1.78	SAT (TO) f=1128 TT=2520.06 T1=1.78	OPT f=1128 TT=5.451 T1=1.79	OPT f=1128 TT=4.138 T1=2.18	OPT f=1128 TT=4.069 T1=2.16	OPT f=1128 TT=23.717 T1=23.71	OPT f=1128 TT=31.914 T1=31.91	OPT f=1128 TT=32.142 T1=32.14	SAT (TO) f=1128 TT=2400.36 T1=1656.35	No data	No data	No data	No data
Knapsack/ Knapsack-50-200-00_c18.xml	OPT f=960 TT=27.401 T1=3.99	No data	No data	No data	No data	OPT f=960 TT=295.691 T1=105.26	OPT f=960 TT=288.083 T1=101.91	OPT f=960 TT=878.562 T1=4.99	OPT f=960 TT=1621.03 T1=3.99	OPT f=960 TT=785.663 T1=9.29	OPT f=960 TT=1366.92 T1=4.1	OPT f=960 TT=59.688 T1=59.31	No data	No data	OPT f=960 TT=7.876 T1=0.43	OPT f=960 TT=8.104 T1=0.45	No data	No data	SAT (TO) f=960 TT=2400.07 T1=4.43	SAT (TO) f=960 TT=2520.11 T1=5.04	OPT f=960 TT=27.401 T1=4.13	SAT (TO) f=960 TT=2400.1 T1=6.01	SAT (TO) f=960 TT=2520.05 T1=6.01	OPT f=960 TT=55.061 T1=55.05	OPT f=960 TT=95.854 T1=95.86	OPT f=960 TT=95.966 T1=95.96	SAT (TO) f=946 TT=2400.21 T1=732.67	No data	No data	No data	No data
Knapsack/ Knapsack-50-200-01_c18.xml	OPT f=1297 TT=21.215 T1=3.42	No data	No data	No data	No data	OPT f=1297 TT=409.716 T1=52.47	OPT f=1297 TT=412.248 T1=52.92	OPT f=1297 TT=860.654 T1=5.98	OPT f=1297 TT=1368.52 T1=42.34	OPT f=1297 TT=1001.17 T1=8.04	OPT f=1297 TT=1274.14 T1=5.72	OPT f=1297 TT=84.522 T1=78.59	No data	No data	OPT f=1297 TT=6.188 T1=0.01	OPT f=1297 TT=6.297 T1=0.05	No data	No data	SAT (TO) f=1297 TT=2400.07 T1=9.21	SAT (TO) f=1297 TT=2520.03 T1=3.94	OPT f=1297 TT=21.215 T1=3.42	SAT (TO) f=1297 TT=2400.09 T1=7.77	SAT (TO) f=1297 TT=2520.09 T1=4.54	OPT f=1297 TT=213.388 T1=213.4	OPT f=1297 TT=214.003 T1=214.01	OPT f=1297 TT=214.748 T1=214.73	? (TO) TT=2400.08	No data	No data	No data	No data
Knapsack/ Knapsack-60-250-00_c18.xml	OPT f=1404 TT=897.822 T1=3.61	No data	No data	No data	No data	SAT (TO) f=1404 TT=2400.05 T1=1158.49	SAT (TO) f=1404 TT=2520.07 T1=1141.13	SAT (TO) f=1404 TT=2520.09 T1=19.86	SAT (TO) f=1404 TT=2520.12 T1=5.42	SAT (TO) f=1404 TT=2520.05 T1=6.61	SAT (TO) f=1404 TT=2520.08 T1=20.72	SAT (TO) f=1378 TT=2520.06 T1=2414.55	No data	No data	OPT f=1404 TT=213.15 T1=0.06	OPT f=1404 TT=213.668 T1=0.06	No data	No data	SAT (TO) f=1404 TT=2400.05 T1=28.53	SAT (TO) f=1404 TT=2520.07 T1=68.32	OPT f=1404 TT=897.822 T1=124.75	SAT (TO) f=1404 TT=2400.07 T1=12.34	SAT (TO) f=1404 TT=2520.05 T1=3.61	? (TO) TT=2520.04	? (TO) TT=2520.06	? (TO) TT=2519.98	? (TO) TT=2400.08	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
Knapsack/ Knapsack-60-250-01_c18.xml	OPT f=1323 TT=415.451 T1=2.5	No data	No data	No data	No data	SAT (TO) f=1323 TT=2400.05 T1=1074.65	SAT (TO) f=1323 TT=2520.09 T1=1055.31	SAT (TO) f=1323 TT=2520.07 T1=16.27	SAT (TO) f=1323 TT=2520.13 T1=28.3	SAT (TO) f=1323 TT=2520.09 T1=45.68	SAT (TO) f=1323 TT=2520.07 T1=852.06	OPT f=1323 TT=1608.41 T1=1506.58	No data	No data	OPT f=1323 TT=157.452 T1=0.05	OPT f=1323 TT=157.308 T1=0.05	No data	No data	SAT (TO) f=1323 TT=2400.12 T1=270.13	SAT (TO) f=1323 TT=2520.45 T1=33.06	OPT f=1323 TT=415.451 T1=44.9	OPT f=1323 TT=135.152 T1=85.33	SAT (TO) f=1323 TT=2520.12 T1=2.5	? (TO) TT=2520.08	? (TO) TT=2519.83	? (TO) TT=2519.97	? (TO) TT=2400.08	No data	No data	No data	No data
Knapsack/ Knapsack-70-300-00_c18.xml	SAT f=1425 TT=2519.75 T1=56.01	No data	No data	No data	No data	SAT (TO) f=1366 TT=2400.12 T1=1832.17	SAT (TO) f=1366 TT=2520.07 T1=1851.26	SAT (TO) f=1425 TT=2520.11 T1=460.96	SAT (TO) f=1422 TT=2520.11 T1=7.43	SAT (TO) f=1425 TT=2520.07 T1=5.56	SAT (TO) f=1425 TT=2520.08 T1=2355.06	SAT (TO) f=1327 TT=2519.79 T1=2470.52	No data	No data	OPT f=1425 TT=774.95 T1=0.11	OPT f=1425 TT=774.765 T1=0.11	No data	No data	SAT f=1425 TT=2399.67 T1=65.72	SAT f=1425 TT=2519.75 T1=194.81	SAT (TO) f=1425 TT=2520.09 T1=290.01	SAT (TO) f=1425 TT=2400.1 T1=188.15	SAT (TO) f=1425 TT=2520.11 T1=56.01	? (TO) TT=2520.02	? (TO) TT=2519.95	? (TO) TT=2520.01	? (TO) TT=2400.29	No data	No data	No data	No data
Knapsack/ Knapsack-70-300-01_c18.xml	OPT f=1465 TT=2288.75 T1=21.5	No data	No data	No data	No data	SAT (TO) f=1406 TT=2400.12 T1=141.95	SAT (TO) f=1406 TT=2520.04 T1=142.28	SAT (TO) f=1465 TT=2520.05 T1=33.59	SAT (TO) f=1465 TT=2520.09 T1=21.5	SAT (TO) f=1464 TT=2520.06 T1=4.81	SAT (TO) f=1465 TT=2520.1 T1=19.76	SAT (TO) f=1378 TT=2520.06 T1=2300.8	No data	No data	OPT f=1465 TT=746.268 T1=0.02	OPT f=1465 TT=753.757 T1=0.02	No data	No data	SAT f=1465 TT=2399.62 T1=67.07	SAT (TO) f=1465 TT=2520.12 T1=57.12	OPT f=1465 TT=2288.75 T1=113.56	SAT (TO) f=1465 TT=2400.05 T1=37.1	SAT (TO) f=1465 TT=2520.03 T1=26.94	? (TO) TT=2520.02	? (TO) TT=2520.11	? (TO) TT=2519.7	? (TO) TT=2400.09	No data	No data	No data	No data
Knapsack/ Knapsack-80-350-00_c18.xml	SAT f=1564 TT=2519.53 T1=260.18	No data	No data	No data	No data	SAT (TO) f=1398 TT=2400.07 T1=1788.56	SAT (TO) f=1398 TT=2520.05 T1=1799.29	SAT (TO) f=1564 TT=2520.05 T1=1874.15	SAT (TO) f=1564 TT=2520.13 T1=334.5	SAT (TO) f=1564 TT=2520.1 T1=8.22	SAT (TO) f=1564 TT=2520.15 T1=212.02	SAT (TO) f=1375 TT=2519.97 T1=2511.98	No data	No data	SAT (TO) f=1564 TT=2400.06 T1=0.03	SAT (TO) f=1564 TT=2520.1 T1=0.03	No data	No data	SAT f=1560 TT=2399.79 T1=167.46	SAT f=1564 TT=2519.53 T1=260.18	SAT (TO) f=1564 TT=2520.04 T1=891.45	SAT (TO) f=1564 TT=2400.07 T1=25.14	SAT (TO) f=1561 TT=2520.09 T1=37.73	? (TO) TT=2519.84	? (TO) TT=2520.06	? (TO) TT=2519.79	? (TO) TT=2400.05	No data	No data	No data	No data
Knapsack/ Knapsack-80-350-01_c18.xml	SAT f=1668 TT=2519.87 T1=8.51	No data	No data	No data	No data	SAT (TO) f=1533 TT=2400.02 T1=2156.46	SAT (TO) f=1533 TT=2520.1 T1=2089.26	SAT (TO) f=1668 TT=2520.04 T1=8.51	SAT (TO) f=1668 TT=2520.1 T1=315.32	SAT (TO) f=1668 TT=2520.08 T1=284.92	SAT (TO) f=1668 TT=2520.07 T1=794.07	SAT (TO) f=1535 TT=2519.53 T1=416.17	No data	No data	OPT f=1668 TT=793.786 T1=0.02	OPT f=1668 TT=792.873 T1=0.02	No data	No data	SAT f=1668 TT=2399.53 T1=213.03	SAT f=1668 TT=2519.87 T1=176.12	SAT f=1668 TT=2519.89 T1=165.65	SAT (TO) f=1668 TT=2400.05 T1=25.5	SAT (TO) f=1668 TT=2520.12 T1=21.62	? (TO) TT=2520.03	? (TO) TT=2519.76	? (TO) TT=2519.73	? (TO) TT=2400.06	No data	No data	No data	No data
Knapsack/ Knapsack-90-400-00_c18.xml	SAT f=1551 TT=2520.07 T1=522.36	No data	No data	No data	No data	SAT (TO) f=1407 TT=2400.04 T1=1015.8	SAT (TO) f=1407 TT=2520.06 T1=1031.86	SAT (TO) f=1547 TT=2520.11 T1=1055.31	SAT (TO) f=1544 TT=2520.08 T1=572.05	SAT (TO) f=1551 TT=2520.12 T1=808.35	SAT (TO) f=1534 TT=2520.12 T1=892.92	SAT (TO) f=1286 TT=2520.07 T1=2511.65	No data	No data	SAT (TO) f=1551 TT=2400.07 T1=0.05	SAT (TO) f=1551 TT=2520.02 T1=0.1	No data	No data	SAT f=1551 TT=2399.84 T1=702.26	SAT (TO) f=1551 TT=2520.11 T1=522.76	SAT (TO) f=1547 TT=2520.05 T1=1486.94	SAT (TO) f=1551 TT=2400.03 T1=219.5	SAT (TO) f=1551 TT=2520.07 T1=522.36	? (TO) TT=2519.96	? (TO) TT=2520.06	? (TO) TT=2519.92	? (TO) TT=2400.07	No data	No data	No data	No data
Knapsack/ Knapsack-90-400-01_c18.xml	SAT f=1681 TT=2519.97 T1=6.32	No data	No data	No data	No data	SAT (TO) f=1536 TT=2400.11 T1=2312.56	SAT (TO) f=1543 TT=2520.05 T1=2423.48	SAT (TO) f=1681 TT=2520.04 T1=33.54	SAT (TO) f=1681 TT=2520.07 T1=6.32	SAT (TO) f=1681 TT=2520.08 T1=70.71	SAT (TO) f=1681 TT=2520.08 T1=118.19	SAT (TO) f=1292 TT=2519.81 T1=2410.53	No data	No data	SAT (TO) f=1681 TT=2400.02 T1=0.08	SAT (TO) f=1681 TT=2519.9 T1=0.08	No data	No data	SAT f=1681 TT=2399.66 T1=2019.24	SAT f=1681 TT=2519.97 T1=78.76	SAT (TO) f=1681 TT=2520.04 T1=421.81	SAT (TO) f=1681 TT=2400.07 T1=7.8	SAT (TO) f=1681 TT=2520.04 T1=8.04	? (TO) TT=2520.03	? (TO) TT=2520.09	? (TO) TT=2519.95	? (TO) TT=2400.11	No data	No data	No data	No data
Langford/ LangfordBin-06_c18.xml	UNSAT TT=0.009	UNSAT TT=0.223	UNSAT TT=0.222	UNSAT TT=0.217	UNSAT TT=0.212	UNSAT TT=1.868	UNSAT TT=1.915	UNSAT TT=12.389	UNSAT TT=13.518	UNSAT TT=11.132	UNSAT TT=11.218	UNSAT TT=1.051	UNSAT TT=0.195	UNSAT TT=0.193	UNSAT TT=0.01	UNSAT TT=0.009	UNSAT TT=1.654	UNSAT TT=1.577	? (TO) TT=2400.08	? (TO) TT=2520.1	UNSAT TT=1.589	No data	No data	UNSAT TT=0.024	UNSAT TT=0.022	UNSAT TT=0.023	UNSAT TT=1.57	UNSAT TT=2.58	UNSAT TT=3.669	UNSAT TT=2.587	UNSAT TT=3.667
Langford/ LangfordBin-07_c18.xml	SAT TT=0.007	SAT TT=0.157	SAT TT=0.155	SAT TT=0.157	SAT TT=0.154	SAT TT=0.932	SAT TT=0.955	SAT TT=9.175	SAT TT=13.53	SAT TT=11.388	SAT TT=13.067	SAT TT=0.007	SAT TT=0.13	SAT TT=0.127	SAT TT=0.009	SAT TT=0.009	SAT TT=1.682	SAT TT=1.516	SAT TT=1.668	SAT TT=1.555	SAT TT=1.562	No data	No data	SAT TT=0.019	SAT TT=0.02	SAT TT=0.019	SAT TT=1.434	SAT TT=2.75	SAT TT=3.943	SAT TT=2.766	SAT TT=3.934
Langford/ LangfordBin-08_c18.xml	SAT TT=0.008	SAT TT=3.226	SAT TT=3.184	SAT TT=2.499	SAT TT=2.464	SAT TT=0.948	SAT TT=0.962	SAT TT=26.587	SAT TT=24.097	SAT TT=30.219	SAT TT=42.378	SAT TT=0.008	SAT TT=2.825	SAT TT=2.8	SAT TT=0.016	SAT TT=0.012	SAT TT=1.694	SAT TT=1.546	SAT TT=1.7	SAT TT=1.571	SAT TT=1.533	No data	No data	SAT TT=0.028	SAT TT=0.027	SAT TT=0.027	SAT TT=1.902	SAT TT=2.946	SAT TT=4.119	SAT TT=2.924	SAT TT=4.098
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
Langford/ LangfordBin-09_c18.xml	UNSAT TT=1.009	? TT=2520.01	? TT=2520.01	? (TO) TT=2519.84	? TT=2520.01	UNSAT TT=175.169	UNSAT TT=175.766	? (TO) TT=2520.04	? (TO) TT=2520.07	? (TO) TT=2520.1	? (TO) TT=2520.12	? (TO) TT=2520.09	? (TO) TT=2519.94	? (TO) TT=2519.99	UNSAT TT=1.009	UNSAT TT=1.133	UNSAT TT=3.129	UNSAT TT=2.689	? (TO) TT=2400.06	? (TO) TT=2520.04	UNSAT TT=2.688	No data	No data	UNSAT TT=1.971	UNSAT TT=1.967	UNSAT TT=1.96	UNSAT TT=11.652	UNSAT TT=4.573	UNSAT TT=7.047	UNSAT TT=4.6	UNSAT TT=7.033
Langford/ LangfordBin-10_c18.xml	UNSAT TT=3.887	? TT=2520.01	? (TO) TT=2519.96	? TT=2520.01	? (TO) TT=2519.81	? (TO) TT=2400.07	? (TO) TT=2520.05	? (TO) TT=2520.09	? (TO) TT=2520.06	? (TO) TT=2520.12	? (TO) TT=2520.08	? (TO) TT=2520.02	? (TO) TT=2519.84	? (TO) TT=2519.91	UNSAT TT=9.385	UNSAT TT=8.648	UNSAT TT=4.639	UNSAT TT=3.887	? (TO) TT=2400.06	? (TO) TT=2520.09	UNSAT TT=4.263	No data	No data	UNSAT TT=43.056	UNSAT TT=43.493	UNSAT TT=43.753	UNSAT TT=82.093	UNSAT TT=10.935	UNSAT TT=26.416	UNSAT TT=10.967	UNSAT TT=27.235
Langford/ LangfordBin-11_c18.xml	SAT TT=0.014	? (TO) TT=2519.93	? (TO) TT=2519.84	? TT=2520.01	? TT=2520.01	SAT TT=0.974	SAT TT=0.985	? (TO) TT=2520.12	? (TO) TT=2520.09	? (TO) TT=2520.09	? (TO) TT=2520.09	SAT TT=0.014	? TT=2520.01	? (TO) TT=2519.95	SAT TT=0.048	SAT TT=0.019	SAT TT=1.725	SAT TT=1.616	SAT TT=1.764	SAT TT=1.606	SAT TT=1.594	No data	No data	SAT TT=0.051	SAT TT=0.051	SAT TT=0.051	SAT TT=2.493	SAT TT=3.497	SAT TT=4.583	SAT TT=3.378	SAT TT=4.592
Langford/ LangfordBin-12_c18.xml	SAT TT=0.018	? (TO) TT=2519.79	? (TO) TT=2519.97	? (TO) TT=2519.78	? (TO) TT=2519.88	SAT TT=0.98	SAT TT=1.006	? (TO) TT=2520.09	? (TO) TT=2520.08	? (TO) TT=2520.08	? (TO) TT=2520.1	SAT TT=0.018	? TT=2520.01	? (TO) TT=2519.91	SAT TT=0.053	SAT TT=0.08	SAT TT=1.722	SAT TT=1.591	SAT TT=1.75	SAT TT=1.606	SAT TT=1.607	No data	No data	SAT TT=0.06	SAT TT=0.221	SAT TT=0.06	SAT TT=2.663	SAT TT=3.628	SAT TT=4.831	SAT TT=3.536	SAT TT=4.836
Langford/ LangfordBin-13_c18.xml	UNSAT TT=674.478	? TT=2520.01	? TT=2520.01	? (TO) TT=2519.98	? TT=2520.01	? (TO) TT=2400.11	? (TO) TT=2520.1	? (TO) TT=2520.15	? (TO) TT=2520.09	? (TO) TT=2520.08	? (TO) TT=2520.09	? (TO) TT=2520.07	? (TO) TT=2519.78	? (TO) TT=2519.82	? (TO) TT=2400.04	? (TO) TT=2519.94	UNSAT TT=699.677	UNSAT TT=674.478	? (TO) TT=2400.09	? (TO) TT=2520.08	? (TO) TT=2520.09	No data	No data	? (TO) TT=2519.76	? (TO) TT=2520	? (TO) TT=2520.06	? (TO) TT=2400.15	? (TO) TT=2520.09	? (TO) TT=2520.13	? (TO) TT=2520.1	? (TO) TT=2520.14
Langford/ LangfordBin-14_c18.xml		? TT=2520.01	? TT=2520.01	? (TO) TT=2519.91	? TT=2520.01	? (TO) TT=2400.08	? (TO) TT=2520.06	? (TO) TT=2520.1	? (TO) TT=2520.05	? (TO) TT=2520.11	? (TO) TT=2520.1	? (TO) TT=2520.01	? (TO) TT=2519.8	? (TO) TT=2519.79	? (TO) TT=2400.02	? (TO) TT=2520	? (TO) TT=2400.04	? (TO) TT=2520.04	? (TO) TT=2400.07	? (TO) TT=2520.05	? (TO) TT=2520.03	No data	No data	? (TO) TT=2519.95	? (TO) TT=2520.01	? (TO) TT=2520.05	? (TO) TT=2400.13	? (TO) TT=2520.08	? (TO) TT=2520.12	? (TO) TT=2520.1	? (TO) TT=2520.12
Langford/ LangfordBin-15_c18.xml	SAT TT=0.018	? TT=2520.01	? (TO) TT=2519.74	? TT=2520.01	? TT=2520.01	? (TO) TT=2400.08	? (TO) TT=2520.05	? (TO) TT=2520.12	? (TO) TT=2520.05	? (TO) TT=2520.13	? (TO) TT=2520.11	SAT TT=0.018	? (TO) TT=2519.93	? (TO) TT=2520.01	SAT TT=0.402	SAT TT=0.088	SAT TT=1.919	SAT TT=1.729	SAT TT=1.794	SAT TT=1.644	SAT TT=1.639	No data	No data	SAT TT=1.268	SAT TT=1.262	SAT TT=1.259	SAT TT=5.264	SAT TT=4.114	SAT TT=5.273	SAT TT=4.025	SAT TT=5.422
Langford/ LangfordBin-16_c18.xml	SAT TT=0.261	? TT=2520.01	? (TO) TT=2519.97	? TT=2520.01	? (TO) TT=2520.01	SAT TT=1.234	SAT TT=1.243	? (TO) TT=2520.1	? (TO) TT=2520.13	? (TO) TT=2520.08	? (TO) TT=2520.05	? (TO) TT=2520	? (TO) TT=2519.91	? (TO) TT=2519.98	SAT TT=409.963	SAT TT=9.869	SAT TT=1.995	SAT TT=1.799	SAT TT=1.833	SAT TT=1.631	SAT TT=1.649	No data	No data	SAT TT=0.262	SAT TT=0.261	SAT TT=0.306	SAT TT=5.459	SAT TT=4.328	SAT TT=5.616	SAT TT=4.487	SAT TT=5.442
LowAutocorrelation/ LowAutocorrelation-015_c18.xml	OPT f=15 TT=1.007 T1=0.25	No data	No data	No data	No data	OPT f=15 TT=10.579 T1=1.58	OPT f=15 TT=9.75 T1=1.6	OPT f=15 TT=23.554 T1=4.49	OPT f=15 TT=25.284 T1=11.81	OPT f=15 TT=24.914 T1=5.24	OPT f=15 TT=18.778 T1=4.74	OPT f=15 TT=7.822 T1=0.25	No data	No data	OPT f=15 TT=1.574 T1=0.54	OPT f=15 TT=1.588 T1=0.55	No data	No data	SAT (TO) f=15 TT=2400.09 T1=1.44	SAT (TO) f=15 TT=2520.02 T1=1.32	OPT f=15 TT=4.273 T1=1.33	OPT f=15 TT=4.874 T1=1.37	SAT (TO) f=15 TT=2520.06 T1=1.24	OPT f=15 TT=1.051 T1=1.05	OPT f=15 TT=1.292 T1=1.29	OPT f=15 TT=1.007 T1=1	OPT f=15 TT=2.494 T1=0.78	No data	No data	No data	No data
LowAutocorrelation/ LowAutocorrelation-020_c18.xml	OPT f=26 TT=5.655 T1=2.27	No data	No data	No data	No data	OPT f=26 TT=335.15 T1=11.69	OPT f=26 TT=331.112 T1=11.58	OPT f=26 TT=638.053 T1=17.18	OPT f=26 TT=256.957 T1=30.18	OPT f=26 TT=539.991 T1=101.85	OPT f=26 TT=283.125 T1=64.49	OPT f=26 TT=544.675 T1=287.87	No data	No data	OPT f=26 TT=37.816 T1=5.98	OPT f=26 TT=37.518 T1=5.92	No data	No data	SAT (TO) f=26 TT=2400.03 T1=6.47	SAT (TO) f=26 TT=2520.07 T1=3.34	OPT f=26 TT=112.438 T1=17.58	SAT (TO) f=26 TT=2400.03 T1=2.24	SAT (TO) f=26 TT=2520.09 T1=2.27	OPT f=26 TT=6.509 T1=6.5	OPT f=26 TT=6.449 T1=6.44	OPT f=26 TT=5.655 T1=5.65	OPT f=26 TT=33.907 T1=7.82	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
LowAutocorrelation/ LowAutocorrelation-021_c18.xml	OPT f=26 TT=8.566 T1=8.56	No data	No data	No data	No data	OPT f=26 TT=887.516 T1=513.02	OPT f=26 TT=885.76 T1=510.94	OPT f=26 TT=1346.5 T1=63.1	OPT f=26 TT=794.127 T1=106.89	OPT f=26 TT=2067.98 T1=404.8	OPT f=26 TT=688.574 T1=100.45	OPT f=26 TT=1018.48 T1=76.67	No data	No data	OPT f=26 TT=33.046 T1=7.21	OPT f=26 TT=33.003 T1=7.19	No data	No data	SAT (TO) f=26 TT=2400.1 T1=2.59	SAT (TO) f=26 TT=2520.11 T1=16.04	OPT f=26 TT=167.927 T1=11.15	SAT (TO) f=26 TT=2400.1 T1=69.23	SAT (TO) f=26 TT=2520.04 T1=457.38	OPT f=26 TT=6.789 T1=6.78	OPT f=26 TT=5.677 T1=5.67	OPT f=26 TT=8.566 T1=8.56	OPT f=26 TT=58.299 T1=23.37	No data	No data	No data	No data
LowAutocorrelation/ LowAutocorrelation-022_c18.xml	OPT f=39 TT=55.256 T1=1.77	No data	No data	No data	No data	SAT (TO) f=39 TT=2400.1 T1=3.74	SAT (TO) f=39 TT=2520.04 T1=3.8	SAT (TO) f=39 TT=2520.1 T1=99.79	OPT f=39 TT=2004.21 T1=220.7	SAT (TO) f=39 TT=2520.12 T1=489.59	SAT (TO) f=39 TT=2520.09 T1=206.52	SAT (TO) f=39 TT=2519.79 T1=524.12	No data	No data	OPT f=39 TT=268.376 T1=2.23	OPT f=39 TT=268.6 T1=2.23	No data	No data	SAT (TO) f=39 TT=2400.1 T1=7.64	SAT (TO) f=39 TT=2520.09 T1=18.55	OPT f=39 TT=596.725 T1=3.02	SAT (TO) f=39 TT=2400.06 T1=2.55	SAT (TO) f=39 TT=2520.07 T1=2.29	OPT f=39 TT=35.093 T1=35.09	OPT f=39 TT=45.286 T1=45.29	OPT f=39 TT=55.256 T1=55.25	OPT f=39 TT=151.698 T1=1.77	No data	No data	No data	No data
LowAutocorrelation/ LowAutocorrelation-023_c18.xml	OPT f=47 TT=117.237 T1=2.19	No data	No data	No data	No data	SAT (TO) f=51 TT=2400.13 T1=237.75	SAT (TO) f=47 TT=2520.09 T1=2425.81	SAT (TO) f=47 TT=2520.03 T1=581.71	SAT (TO) f=47 TT=2520.05 T1=1369.17	SAT (TO) f=47 TT=2520.11 T1=193.24	SAT (TO) f=47 TT=2520.07 T1=122.94	SAT (TO) f=47 TT=2519.79 T1=193.84	No data	No data	OPT f=47 TT=519.456 T1=18.54	OPT f=47 TT=518.004 T1=18.74	No data	No data	SAT (TO) f=47 TT=2400.09 T1=4.1	SAT (TO) f=47 TT=2520.03 T1=5.05	OPT f=47 TT=814.849 T1=9.6	SAT (TO) f=47 TT=2400.05 T1=2.17	SAT (TO) f=47 TT=2520.06 T1=2.19	OPT f=47 TT=128.993 T1=129.01	OPT f=47 TT=116.925 T1=116.92	OPT f=47 TT=117.237 T1=117.23	OPT f=47 TT=606.997 T1=18.94	No data	No data	No data	No data
LowAutocorrelation/ LowAutocorrelation-024_c18.xml	OPT f=36 TT=70.833 T1=63.57	No data	No data	No data	No data	SAT (TO) f=36 TT=2400.09 T1=1183.17	SAT (TO) f=36 TT=2520.08 T1=1160.99	SAT (TO) f=36 TT=2520.07 T1=1483.79	SAT (TO) f=52 TT=2520.05 T1=96.59	SAT (TO) f=36 TT=2520.06 T1=861.43	SAT (TO) f=36 TT=2520.11 T1=287.66	SAT (TO) f=56 TT=2519.94 T1=75	No data	No data	OPT f=36 TT=212.155 T1=16.43	OPT f=36 TT=210.926 T1=16.47	No data	No data	SAT (TO) f=36 TT=2400.1 T1=5.39	SAT (TO) f=36 TT=2520.09 T1=79.65	OPT f=36 TT=1552.85 T1=223.9	SAT (TO) f=44 TT=2400.07 T1=6.19	SAT (TO) f=44 TT=2520.04 T1=6.75	OPT f=36 TT=63.477 T1=63.47	OPT f=36 TT=69.937 T1=69.94	OPT f=36 TT=70.833 T1=70.84	OPT f=36 TT=381.383 T1=63.57	No data	No data	No data	No data
LowAutocorrelation/ LowAutocorrelation-025_c18.xml	OPT f=36 TT=68.358 T1=68.36	No data	No data	No data	No data	SAT (TO) f=44 TT=2400.07 T1=184.81	SAT (TO) f=44 TT=2520.04 T1=183.84	SAT (TO) f=36 TT=2520.06 T1=192.67	SAT (TO) f=36 TT=2520.09 T1=955.28	SAT (TO) f=36 TT=2520.07 T1=1989.51	SAT (TO) f=48 TT=2520.09 T1=2224.11	SAT (TO) f=48 TT=2519.94 T1=686.36	No data	No data	OPT f=36 TT=483.191 T1=264.09	OPT f=36 TT=488.95 T1=265.5	No data	No data	SAT (TO) f=36 TT=2400.04 T1=294.96	SAT (TO) f=36 TT=2520.04 T1=1436.64	SAT (TO) f=36 TT=2520.04 T1=168.18	SAT (TO) f=36 TT=2400.1 T1=23.28	SAT (TO) f=44 TT=2520.09 T1=11.95	OPT f=36 TT=95.306 T1=95.3	OPT f=36 TT=80.076 T1=80.07	OPT f=36 TT=68.358 T1=68.36	OPT f=36 TT=741.95 T1=236.21	No data	No data	No data	No data
LowAutocorrelation/ LowAutocorrelation-026_c18.xml	OPT f=45 TT=341.811 T1=38.73	No data	No data	No data	No data	SAT (TO) f=53 TT=2400.04 T1=817.33	SAT (TO) f=53 TT=2520.08 T1=795.18	SAT (TO) f=45 TT=2520.09 T1=1562.37	SAT (TO) f=45 TT=2520.07 T1=605.78	SAT (TO) f=45 TT=2520.05 T1=57.83	SAT (TO) f=53 TT=2520.13 T1=395.51	SAT (TO) f=53 TT=2519.94 T1=1457.07	No data	No data	OPT f=45 TT=1874.89 T1=37.76	OPT f=45 TT=1887.49 T1=38.1	No data	No data	SAT (TO) f=45 TT=2400.08 T1=293.69	SAT (TO) f=45 TT=2520.02 T1=53.69	SAT (TO) f=45 TT=2520.08 T1=38.73	SAT (TO) f=53 TT=2400.12 T1=4.48	SAT (TO) f=53 TT=2520.05 T1=4.67	OPT f=45 TT=355.134 T1=355.14	OPT f=45 TT=342.352 T1=342.34	OPT f=45 TT=341.811 T1=341.82	SAT (TO) f=45 TT=2400.17 T1=188.32	No data	No data	No data	No data
LowAutocorrelation/ LowAutocorrelation-027_c18.xml	OPT f=37 TT=190.42 T1=190.43	No data	No data	No data	No data	SAT (TO) f=73 TT=2400.09 T1=451.19	SAT (TO) f=73 TT=2520.03 T1=430.83	SAT (TO) f=73 TT=2520.04 T1=452.57	SAT (TO) f=37 TT=2520.15 T1=1077.03	SAT (TO) f=65 TT=2520.11 T1=768.69	SAT (TO) f=69 TT=2520.13 T1=1969.1	SAT (TO) f=61 TT=2519.94 T1=1174.63	No data	No data	OPT f=37 TT=1469.42 T1=1093.37	OPT f=37 TT=1471.13 T1=1092.86	No data	No data	SAT (TO) f=37 TT=2400.04 T1=1644.71	SAT (TO) f=37 TT=2520.11 T1=828.67	SAT (TO) f=37 TT=2520.04 T1=536.7	SAT (TO) f=77 TT=2400.05 T1=11.28	SAT (TO) f=65 TT=2520.03 T1=2452.11	OPT f=37 TT=200.304 T1=200.31	OPT f=37 TT=191.86 T1=191.87	OPT f=37 TT=190.42 T1=190.43	OPT f=37 TT=2338.88 T1=1175.33	No data	No data	No data	No data
LowAutocorrelation/ LowAutocorrelation-028_c18.xml	OPT f=50 TT=1596.51 T1=1067.94	No data	No data	No data	No data	SAT (TO) f=86 TT=2400.04 T1=572.71	SAT (TO) f=86 TT=2520.04 T1=555.2	SAT (TO) f=58 TT=2520.11 T1=1168.28	SAT (TO) f=74 TT=2520.03 T1=1107.8	SAT (TO) f=74 TT=2520.1 T1=1092.26	SAT (TO) f=74 TT=2520.06 T1=2408.51	SAT (TO) f=74 TT=2519.96 T1=1267.86	No data	No data	SAT (TO) f=50 TT=2400.04 T1=1766.66	SAT (TO) f=50 TT=2519.96 T1=1763.87	No data	No data	SAT (TO) f=62 TT=2400.07 T1=1446.18	SAT (TO) f=58 TT=2520.06 T1=1517.66	SAT (TO) f=50 TT=2520.04 T1=1067.94	SAT (TO) f=74 TT=2400.06 T1=1542.29	SAT (TO) f=98 TT=2520.08 T1=5.44	OPT f=50 TT=1444.11 T1=1444.2	OPT f=50 TT=1597.11 T1=1597.3	OPT f=50 TT=1596.51 T1=1596.49	SAT (TO) f=50 TT=2400.16 T1=1763.95	No data	No data	No data	No data
LowAutocorrelation/ LowAutocorrelation-030_c18.xml	SAT f=67 TT=2520.05 T1=682.96	No data	No data	No data	No data	SAT (TO) f=107 TT=2400.07 T1=709.61	SAT (TO) f=107 TT=2520.09 T1=698.99	SAT (TO) f=91 TT=2520.08 T1=773.02	SAT (TO) f=91 TT=2520.12 T1=239.95	SAT (TO) f=91 TT=2520.06 T1=11.21	SAT (TO) f=83 TT=2520.09 T1=248.45	SAT (TO) f=131 TT=2519.93 T1=189.27	No data	No data	SAT (TO) f=59 TT=2400.08 T1=587.19	SAT (TO) f=59 TT=2520.08 T1=595.41	No data	No data	SAT (TO) f=59 TT=2400.1 T1=467.81	SAT (TO) f=67 TT=2520.05 T1=682.96	SAT (TO) f=75 TT=2520.1 T1=555.9	SAT (TO) f=83 TT=2400.09 T1=2019.89	SAT (TO) f=147 TT=2520.05 T1=9.57	? (TO) TT=2520.01	? (TO) TT=2519.93	? (TO) TT=2519.93	SAT (TO) f=75 TT=2400.16 T1=2036.67	No data	No data	No data	No data
LowAutocorrelation/ LowAutocorrelation-035_c18.xml	SAT f=113 TT=2520.12 T1=689.35	No data	No data	No data	No data	SAT (TO) f=137 TT=2400.11 T1=1.65	SAT (TO) f=137 TT=2520.05 T1=1.68	SAT (TO) f=113 TT=2520.12 T1=689.35	SAT (TO) f=141 TT=2520.06 T1=92.78	SAT (TO) f=109 TT=2520.11 T1=845.59	SAT (TO) f=133 TT=2520.13 T1=2311.53	SAT (TO) f=197 TT=2519.78 T1=1633.92	No data	No data	SAT (TO) f=113 TT=2400.05 T1=1590.07	SAT (TO) f=113 TT=2519.75 T1=1572.73	No data	No data	SAT (TO) f=101 TT=2400.04 T1=1445.12	SAT (TO) f=129 TT=2520.06 T1=619.77	SAT (TO) f=117 TT=2520.08 T1=1230.57	SAT (TO) f=241 TT=2400.07 T1=10.15	SAT (TO) f=257 TT=2520.07 T1=11.89	? (TO) TT=2520.01	? (TO) TT=2520.02	? (TO) TT=2520.07	SAT (TO) f=133 TT=2400.11 T1=1217.95	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
LowAutocorrelation/ LowAutocorrelation-040_c18.xml	SAT f=160 TT=2520.04 T1=463.92	No data	No data	No data	No data	SAT (TO) f=212 TT=2400.05 T1=1087.52	SAT (TO) f=212 TT=2520.03 T1=1104.62	SAT (TO) f=188 TT=2520.06 T1=2382.1	SAT (TO) f=200 TT=2520.05 T1=180.07	SAT (TO) f=152 TT=2520.13 T1=905.68	SAT (TO) f=216 TT=2520.11 T1=1223.81	SAT (TO) f=500 TT=2519.78 T1=2.7	No data	No data	SAT (TO) f=144 TT=2400.09 T1=1322.21	SAT (TO) f=144 TT=2519.95 T1=1325.2	No data	No data	SAT (TO) f=172 TT=2400.08 T1=939.4	SAT (TO) f=160 TT=2520.04 T1=463.92	SAT (TO) f=160 TT=2520.07 T1=527.79	SAT (TO) f=216 TT=2400.06 T1=410.92	SAT (TO) f=200 TT=2520.13 T1=268.3	? (TO) TT=2519.78	? (TO) TT=2520.06	? (TO) TT=2519.92	SAT (TO) f=180 TT=2400.14 T1=1680.34	No data	No data	No data	No data
LowAutocorrelation/ LowAutocorrelation-050_c18.xml	SAT f=289 TT=2520.12 T1=789.39	No data	No data	No data	No data	SAT (TO) f=409 TT=2400.08 T1=308.25	SAT (TO) f=401 TT=2520.09 T1=2374.67	SAT (TO) f=329 TT=2520.12 T1=242.06	SAT (TO) f=353 TT=2520.11 T1=66.67	SAT (TO) f=361 TT=2520.13 T1=1177.26	SAT (TO) f=385 TT=2520.08 T1=33.05	SAT (TO) f=649 TT=2519.92 T1=276.32	No data	No data	SAT (TO) f=321 TT=2400.03 T1=53.35	SAT (TO) f=321 TT=2519.67 T1=52.79	No data	No data	SAT (TO) f=281 TT=2400.07 T1=118.91	SAT (TO) f=313 TT=2520.05 T1=1806.96	SAT (TO) f=305 TT=2520.08 T1=859.95	SAT (TO) f=753 TT=2400.05 T1=5.71	SAT (TO) f=289 TT=2520.12 T1=789.39	? (TO) TT=2519.97	? (TO) TT=2519.77	? (TO) TT=2519.89	SAT (TO) f=353 TT=2400.13 T1=2092.12	No data	No data	No data	No data
MagicHexagon/ MagicHexagon-04-0003_c18.xml	SAT TT=0.108	SAT TT=42.362	SAT TT=41.311	SAT TT=38.003	SAT TT=38.029	SAT TT=1.087	SAT TT=1.097	SAT TT=21.145	SAT TT=11.323	SAT TT=38.162	SAT TT=38.977	SAT TT=0.339	SAT TT=37.952	SAT TT=37.887	SAT TT=0.108	SAT TT=2.638	SAT TT=2192.13	SAT TT=2205.65	SAT TT=4.659	SAT TT=4.228	SAT TT=4.098	No data	No data	SAT TT=63.46	SAT TT=64.099	SAT TT=63.623	? (TO) TT=2400.12	SAT TT=6.54	SAT TT=267.223	SAT TT=6.874	SAT TT=265.526
MagicHexagon/ MagicHexagon-04-0017_c18.xml	UNSAT TT=0.172	UNSAT TT=354.838	UNSAT TT=352.424	UNSAT TT=215.222	UNSAT TT=216.092	UNSAT TT=5.067	UNSAT TT=4.564	UNSAT TT=13.26	UNSAT TT=15.536	UNSAT TT=14.194	UNSAT TT=15.785	UNSAT TT=3.4	UNSAT TT=215.288	UNSAT TT=215.353	UNSAT TT=0.205	UNSAT TT=0.172	UNSAT TT=5.195	UNSAT TT=5.661	? (TO) TT=2400.03	? (TO) TT=2520.05	UNSAT TT=6.762	No data	No data	UNSAT TT=150.061	UNSAT TT=149.711	UNSAT TT=151.359	UNSAT TT=283.675	UNSAT TT=8.823	UNSAT TT=10.275	UNSAT TT=8.545	UNSAT TT=10.467
MagicHexagon/ MagicHexagon-04-m004_c18.xml	SAT TT=0.113	SAT TT=268.174	SAT TT=255.752	SAT TT=183.754	SAT TT=182.494	SAT TT=1.466	SAT TT=1.435	SAT TT=17.42	SAT TT=19.683	SAT TT=20.149	SAT TT=17.108	SAT TT=13.094	SAT TT=183.746	SAT TT=183.432	SAT TT=0.113	SAT TT=0.504	SAT TT=35.591	SAT TT=35.027	SAT TT=4.875	SAT TT=12.088	SAT TT=12.17	No data	No data	SAT TT=48.018	SAT TT=47.874	SAT TT=47.799	SAT TT=808.52	SAT TT=18.059	SAT TT=964.865	SAT TT=17.829	SAT TT=967.309
MagicHexagon/ MagicHexagon-05-0006_c18.xml	SAT TT=10.191	? TT=2520.03	? (TO) TT=2519.99	? (TO) TT=2519.99	? (TO) TT=2520.08	SAT TT=333.425	SAT TT=331.187	SAT TT=872.305	? (TO) TT=2520.05	SAT TT=24.348	? (TO) TT=2520.06	SAT TT=10.191	? (TO) TT=2520.04	? (TO) TT=2519.94	? (TO) TT=2400.09	SAT TT=1953.18	? (TO) TT=2400.03	? (TO) TT=2520.06	? (TO) TT=2400.06	? (TO) TT=2520.04	? (TO) TT=2520.11	No data	No data	? (TO) TT=2519.92	? (TO) TT=2519.79	? (TO) TT=2520.03	? (TO) TT=2400.19	? (MO) TT=13.533	? (TO) TT=2520.16	? (MO) TT=13.643	? (TO) TT=2520.12
MagicHexagon/ MagicHexagon-05-m003_c18.xml	SAT TT=97.246	? (TO) TT=2520.07	? (TO) TT=2520.04	? (TO) TT=2519.99	? (TO) TT=2519.85	? (TO) TT=2400.09	? (TO) TT=2520.05	SAT TT=97.246	? (TO) TT=2520.12	SAT TT=253.812	? (TO) TT=2520.12	? (TO) TT=2520.08	? (TO) TT=2520.03	? (TO) TT=2520.06	? (TO) TT=2400.02	SAT TT=861.993	? (TO) TT=2400.08	? (TO) TT=2520.06	? (TO) TT=2400.12	? (TO) TT=2520.11	SAT TT=1683.7	No data	No data	? (TO) TT=2519.76	? (TO) TT=2520.09	? (TO) TT=2520.07	? (TO) TT=2400.21	? (MO) TT=13.673	? (TO) TT=2520.13	? (MO) TT=13.732	? (TO) TT=2520.09
MagicHexagon/ MagicHexagon-06-0010_c18.xml		? (TO) TT=2520.09	? (TO) TT=2520.06	? (TO) TT=2520.01	? (TO) TT=2520.06	? (TO) TT=2400.1	? (TO) TT=2520.06	? (TO) TT=2520.13	? (TO) TT=2520.11	? (TO) TT=2520.1	? (TO) TT=2520.07	? (TO) TT=2519.79	? (TO) TT=2520.06	? (TO) TT=2520.07	? (TO) TT=2400.07	? (TO) TT=2519.96	? (TO) TT=2400.07	? (TO) TT=2520.1	? (TO) TT=2400.05	? (TO) TT=2520.07	? (TO) TT=2520.12	No data	No data	? (TO) TT=2520.02	? (TO) TT=2519.76	? (TO) TT=2520.06	? (TO) TT=2400.13	? (MO) TT=18.72	? (TO) TT=2520.13	? (MO) TT=19.632	? (TO) TT=2520.1
MagicHexagon/ MagicHexagon-06-m001_c18.xml	SAT TT=816.032	? (TO) TT=2520.1	? (TO) TT=2519.8	? (TO) TT=2519.73	? (TO) TT=2520.1	? (TO) TT=2400.13	? (TO) TT=2520.12	? (TO) TT=2520.06	? (TO) TT=2520.06	? (TO) TT=2520.08	? (TO) TT=2520.07	SAT TT=816.032	? (TO) TT=2520.05	? (TO) TT=2520.06	? (TO) TT=2400.07	? (TO) TT=2519.94	? (TO) TT=2400.08	? (TO) TT=2520.09	? (TO) TT=2400.07	? (TO) TT=2520.04	? (TO) TT=2520.09	No data	No data	? (TO) TT=2519.9	? (TO) TT=2519.84	? (TO) TT=2519.86	? (TO) TT=2400.12	? (MO) TT=18.289	? (TO) TT=2520.13	? (MO) TT=19.938	? (TO) TT=2520.19
MagicHexagon/ MagicHexagon-07-0002_c18.xml		? (TO) TT=2520.09	? (TO) TT=2519.97	? (TO) TT=2520.01	? (TO) TT=2520.1	? (TO) TT=2400.1	? (TO) TT=2520.13	? (TO) TT=2520.07	? (TO) TT=2520.06	? (TO) TT=2520.15	? (TO) TT=2520.09	? (TO) TT=2520.05	? (TO) TT=2520.06	? (TO) TT=2520.07	? (TO) TT=2400.05	? (TO) TT=2520.03	? (TO) TT=2400.06	? (TO) TT=2520.03	? (TO) TT=2400.07	? (TO) TT=2520.08	? (TO) TT=2520.05	No data	No data	? (TO) TT=2520.08	? (TO) TT=2520.13	? (TO) TT=2520.16	? (TO) TT=2400.13	? (MO) TT=17.157	? (TO) TT=2520.11	? (MO) TT=43.355	? (TO) TT=2520.2
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
MagicHexagon/ MagicHexagon-07-m011_c18.xml		? (TO) TT=2519.77	? (TO) TT=2520.11	? (TO) TT=2520	? (TO) TT=2520.08	? (TO) TT=2400.1	? (TO) TT=2520.07	? (TO) TT=2520.12	? (TO) TT=2520.15	? (TO) TT=2520.05	? (TO) TT=2520.09	? (TO) TT=2519.95	? (TO) TT=2520.07	? (TO) TT=2520.04	? (TO) TT=2400.11	? (TO) TT=2519.76	? (TO) TT=2400.08	? (TO) TT=2520.06	? (TO) TT=2400.11	? (TO) TT=2520.1	? (TO) TT=2520.07	No data	No data	? (TO) TT=2519.9	? (TO) TT=2519.92	? (TO) TT=2520.08	? (TO) TT=2400.22	? (MO) TT=18.53	? (TO) TT=2520.15	? (MO) TT=46.548	? (TO) TT=2520.23
MagicHexagon/ MagicHexagon-08-0006_c18.xml		? (TO) TT=2520.11	? (TO) TT=2519.99	? (TO) TT=2520.01	? (TO) TT=2520.06	? (TO) TT=2400.1	? (TO) TT=2520.06	? (TO) TT=2520.1	? (TO) TT=2520.14	? (TO) TT=2520.06	? (TO) TT=2520.05	? (TO) TT=2520.08	? (TO) TT=2520.07	? (TO) TT=2520.07	? (TO) TT=2400.1	? (TO) TT=2520.02	? (TO) TT=2400.09	? (TO) TT=2520.02	? (TO) TT=2400.09	? (TO) TT=2520.07	? (TO) TT=2520.08	No data	No data	? (TO) TT=2519.97	? (TO) TT=2519.81	? (TO) TT=2519.98	? TT=1405.38	? (MO) TT=25.371	? (TO) TT=2520.09	? (MO) TT=103.846	? (MO) TT=710.105
MagicHexagon/ MagicHexagon-08-m009_c18.xml		? (TO) TT=2520.08	? (TO) TT=2519.98	? (TO) TT=2519.99	? (TO) TT=2520.01	? (TO) TT=2400.06	? (TO) TT=2520.07	? (TO) TT=2520.05	? (TO) TT=2520.04	? (TO) TT=2520.06	? (TO) TT=2520.1	? (TO) TT=2519.8	? (TO) TT=2520.06	? (TO) TT=2520.04	? (TO) TT=2400.09	? (TO) TT=2520.08	? (TO) TT=2400.13	? (TO) TT=2520.1	? (TO) TT=2400.02	? (TO) TT=2520.02	? (TO) TT=2520.04	No data	No data	? (TO) TT=2519.87	? (TO) TT=2520.12	? (TO) TT=2519.92	? TT=1394	? (MO) TT=24.152	? (TO) TT=2520.14	? (MO) TT=104.494	? (MO) TT=731.663
Mario/ Mario-easy-2_c18.xml	OPT f=628 TT=1.314 T1=0.61	No data	No data	No data	No data	OPT f=628 TT=1.323 T1=0.6	OPT f=628 TT=1.314 T1=0.61	OPT f=628 TT=7.768 T1=4.33	OPT f=628 TT=9.582 T1=4.29	OPT f=628 TT=9.256 T1=4.41	OPT f=628 TT=9.625 T1=4.34	? (NS) TT=0.011	No data	No data	OPT f=628 TT=0.045 T1=0.04	OPT f=628 TT=0.044 T1=0.04	No data	No data	SAT (TO) f=628 TT=2400.13 T1=1.32	SAT (TO) f=628 TT=2520.03 T1=1.27	OPT f=628 TT=2.283 T1=1.27	OPT f=628 TT=2.402 T1=1.32	OPT f=628 TT=2.328 T1=1.29	OPT f=628 TT=2.991 T1=2.99	OPT f=628 TT=2.644 T1=2.64	OPT f=628 TT=2.667 T1=2.66	? (TO) TT=2407.96	No data	No data	No data	No data
Mario/ Mario-easy-4_c18.xml	OPT f=545 TT=2.831 T1=1.41	No data	No data	No data	No data	OPT f=545 TT=4.186 T1=1.57	OPT f=545 TT=4.236 T1=1.57	OPT f=545 TT=15.322 T1=7.42	OPT f=545 TT=13.754 T1=4.94	OPT f=545 TT=13.2 T1=5.41	OPT f=545 TT=14.055 T1=5.5	? (NS) TT=0.011	No data	No data	OPT f=545 TT=9.691 T1=9.55	OPT f=545 TT=8.232 T1=8.05	No data	No data	SAT (TO) f=545 TT=2400.06 T1=1.53	SAT (TO) f=545 TT=2520.09 T1=1.48	OPT f=545 TT=3.008 T1=1.48	OPT f=545 TT=2.873 T1=1.46	OPT f=545 TT=2.831 T1=1.41	OPT f=545 TT=6.914 T1=6.91	OPT f=545 TT=7.151 T1=7.15	OPT f=545 TT=5.166 T1=5.16	? (TO) TT=2407.98	No data	No data	No data	No data
Mario/ Mario-easy-5_c18.xml	OPT f=445 TT=2.481 T1=1	No data	No data	No data	No data	OPT f=445 TT=2.782 T1=0.99	OPT f=445 TT=2.804 T1=1	OPT f=445 TT=13.788 T1=4.35	OPT f=445 TT=13.807 T1=4.65	OPT f=445 TT=13.198 T1=4.47	OPT f=445 TT=13.781 T1=4.57	? (NS) TT=0.011	No data	No data	OPT f=445 TT=2.662 T1=0.03	OPT f=445 TT=2.695 T1=0.03	No data	No data	SAT (TO) f=445 TT=2400.08 T1=1.35	SAT (TO) f=445 TT=2520.08 T1=1.31	OPT f=445 TT=2.84 T1=1.31	OPT f=445 TT=2.591 T1=1.29	OPT f=445 TT=2.481 T1=1.24	OPT f=445 TT=4.134 T1=4.13	OPT f=445 TT=3.941 T1=3.94	OPT f=445 TT=3.957 T1=3.95	? (TO) TT=2407.59	No data	No data	No data	No data
Mario/ Mario-n-medium-2_c18.xml	OPT f=1053 TT=170.602 T1=18.34	No data	No data	No data	No data	OPT f=1053 TT=170.619 T1=94.39	OPT f=1053 TT=170.602 T1=94.42	OPT f=1053 TT=2275.45 T1=2014.47	OPT f=1053 TT=1292.15 T1=159.99	OPT f=1053 TT=1898 T1=1716.25	SAT (TO) f=1053 TT=2520.09 T1=2111.92	? (NS) TT=0.02	No data	No data	SAT (TO) f=1053 TT=2400.02 T1=96.43	SAT (TO) f=1053 TT=2519.98 T1=94.1	No data	No data	SAT (TO) f=1053 TT=2400.03 T1=3.08	SAT (TO) f=1053 TT=2520.09 T1=279.92	OPT f=1053 TT=467.198 T1=18.34	SAT (TO) f=886 TT=2400.07 T1=13.68	SAT (TO) f=990 TT=2520.1 T1=584.22	? (TO) TT=2520.09	? (TO) TT=2519.73	OPT f=1053 TT=2369.06 T1=2369.22	? (TO) TT=2407.47	No data	No data	No data	No data
Mario/ Mario-n-medium-3_c18.xml	OPT f=943 TT=83.701 T1=11.78	No data	No data	No data	No data	OPT f=943 TT=82.699 T1=58.2	OPT f=943 TT=83.701 T1=59	OPT f=943 TT=458.251 T1=13.88	OPT f=943 TT=955.883 T1=428.89	OPT f=943 TT=716.608 T1=27.94	OPT f=943 TT=882.874 T1=8.26	? (NS) TT=0.02	No data	No data	OPT f=943 TT=766.903 T1=127.79	OPT f=943 TT=775.312 T1=124.44	No data	No data	SAT (TO) f=943 TT=2400.08 T1=45.8	SAT (TO) f=943 TT=2520.12 T1=19.47	OPT f=943 TT=116.889 T1=17.84	SAT (TO) f=943 TT=2400.07 T1=212.05	SAT (TO) f=943 TT=2520.12 T1=11.78	OPT f=943 TT=1443.94 T1=1443.89	OPT f=943 TT=1372.1 T1=1372.07	OPT f=943 TT=1379.87 T1=1379.75	? (TO) TT=2407.87	No data	No data	No data	No data
Mario/ Mario-n-medium-4_c18.xml	OPT f=832 TT=125.955 T1=13.01	No data	No data	No data	No data	OPT f=832 TT=125.138 T1=98.55	OPT f=832 TT=125.955 T1=98.83	OPT f=832 TT=445.943 T1=105.74	OPT f=832 TT=629.804 T1=263.34	OPT f=832 TT=404.274 T1=42.21	OPT f=832 TT=733.536 T1=183.54	? (NS) TT=0.02	No data	No data	OPT f=832 TT=318.952 T1=30.04	OPT f=832 TT=327.554 T1=29.4	No data	No data	SAT (TO) f=832 TT=2400.05 T1=44.74	SAT (TO) f=832 TT=2520.1 T1=13.01	OPT f=832 TT=277.464 T1=68.83	SAT (TO) f=832 TT=2400.12 T1=20.89	SAT (TO) f=832 TT=2520.04 T1=20.96	OPT f=832 TT=386.519 T1=386.49	OPT f=832 TT=283.161 T1=283.16	OPT f=832 TT=283.796 T1=283.79	? (TO) TT=2407.84	No data	No data	No data	No data
Mario/ Mario-n-medium-5_c18.xml	OPT f=771 TT=86.991 T1=15.23	No data	No data	No data	No data	OPT f=771 TT=87.604 T1=34.01	OPT f=771 TT=86.991 T1=33.96	OPT f=771 TT=821.393 T1=37.62	OPT f=771 TT=1204.19 T1=132.18	OPT f=771 TT=942.065 T1=45.41	OPT f=771 TT=1284 T1=585.4	? (NS) TT=0.02	No data	No data	OPT f=771 TT=2306.96 T1=11.21	OPT f=771 TT=2246.27 T1=11.03	No data	No data	SAT (TO) f=771 TT=2400.11 T1=35.01	SAT (TO) f=771 TT=2520.09 T1=15.23	OPT f=771 TT=165.26 T1=32.67	SAT (TO) f=762 TT=2400.12 T1=10.73	SAT (TO) f=769 TT=2520.1 T1=24.22	OPT f=771 TT=1195.92 T1=1195.85	OPT f=771 TT=1139.24 T1=1139.2	OPT f=771 TT=1138.98 T1=1138.95	? (TO) TT=2408.15	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
Mario/ Mario-t-hard-1_c18.xml	OPT f=4783 TT=4.549 T1=1.89	No data	No data	No data	No data	OPT f=4783 TT=14.706 T1=11.24	OPT f=4783 TT=14.545 T1=11.23	OPT f=4783 TT=51.24 T1=39.59	OPT f=4783 TT=37.327 T1=23.07	OPT f=4783 TT=27.189 T1=14.77	OPT f=4783 TT=91.549 T1=75.97	? (NS) TT=0.087	No data	No data	OPT f=4783 TT=6.276 T1=6.26	OPT f=4783 TT=5.817 T1=5.8	No data	No data	SAT (TO) f=4783 TT=2400.03 T1=2.07	SAT (TO) f=4783 TT=2520.12 T1=2.45	OPT f=4783 TT=6.628 T1=2.62	OPT f=4783 TT=4.611 T1=1.95	OPT f=4783 TT=4.549 T1=1.89	? (TO) TT=2520.02	? (TO) TT=2520.14	? (TO) TT=2520.08	? TT=1494.73	No data	No data	No data	No data
Mario/ Mario-t-hard-2_c18.xml	OPT f=4850 TT=6.373 T1=2.62	No data	No data	No data	No data	OPT f=4850 TT=16.249 T1=12.94	OPT f=4850 TT=16.031 T1=12.92	OPT f=4850 TT=43.067 T1=29.74	OPT f=4850 TT=39.151 T1=25.26	OPT f=4850 TT=32.479 T1=19.46	OPT f=4850 TT=39.073 T1=24.4	? (NS) TT=0.087	No data	No data	OPT f=4850 TT=13.671 T1=13.67	OPT f=4850 TT=12.619 T1=12.61	No data	No data	SAT (TO) f=4850 TT=2400.03 T1=2.69	SAT (TO) f=4850 TT=2520.05 T1=2.64	OPT f=4850 TT=6.373 T1=2.62	OPT f=4850 TT=8.46 T1=4.84	OPT f=4850 TT=8.916 T1=4.89	? (TO) TT=2519.86	? (TO) TT=2520.07	? (TO) TT=2520	? TT=1422.68	No data	No data	No data	No data
Mario/ Mario-t-hard-5_c18.xml	OPT f=4482 TT=3.687 T1=1.62	No data	No data	No data	No data	OPT f=4482 TT=14.599 T1=11.41	OPT f=4482 TT=14.471 T1=11.3	OPT f=4482 TT=23.853 T1=12.89	OPT f=4482 TT=33.304 T1=18.63	OPT f=4482 TT=28.883 T1=16.29	OPT f=4482 TT=51.262 T1=36.71	? (NS) TT=0.087	No data	No data	OPT f=4482 TT=7.575 T1=7.56	OPT f=4482 TT=7.078 T1=7.07	No data	No data	SAT (TO) f=4482 TT=2400.11 T1=2.79	SAT (TO) f=4482 TT=2520.08 T1=1.66	OPT f=4482 TT=3.687 T1=1.62	OPT f=4482 TT=4.047 T1=1.7	OPT f=4482 TT=3.81 T1=1.62	? (TO) TT=2520.08	? (TO) TT=2520.08	? (TO) TT=2520.11	? (problem) TT=1449.74	No data	No data	No data	No data
MisteryShopper/ MisteryShopper-04_c18.xml	SAT TT=0.026	? (NS) TT=0.007	? (NS) TT=0.007	? (NS) TT=0.008	? (NS) TT=0.008	SAT TT=1.367	SAT TT=1.346	SAT TT=8.419	SAT TT=8.428	SAT TT=8.588	SAT TT=8.56	SAT TT=0.026	? (NS) TT=0.007	? (NS) TT=0.007	SAT TT=0.026	SAT TT=0.026	SAT TT=2.243	SAT TT=2.18	SAT TT=2.219	SAT TT=2.233	SAT TT=2.209	No data	No data	SAT TT=0.336	SAT TT=0.331	SAT TT=0.339	? TT=0.875	SAT TT=6.805	SAT TT=8.217	SAT TT=6.707	SAT TT=8.449
MisteryShopper/ MisteryShopper-06a_c18.xml	SAT TT=0.168	? (NS) TT=0.011	? (NS) TT=0.011	? (NS) TT=0.011	? (NS) TT=0.011	SAT TT=3.102	SAT TT=3.111	SAT TT=11.731	SAT TT=11.685	SAT TT=12.12	SAT TT=11.626	SAT TT=435.117	? (NS) TT=0.011	? (NS) TT=0.011	SAT TT=0.171	SAT TT=0.168	SAT TT=4.238	SAT TT=4.367	SAT TT=3.302	SAT TT=3.332	SAT TT=3.455	No data	No data	SAT TT=3.307	SAT TT=3.286	SAT TT=3.318	? TT=1.541	SAT TT=14.637	SAT TT=15.644	SAT TT=14.084	SAT TT=16.293
MisteryShopper/ MisteryShopper-06b_c18.xml	SAT TT=0.765	? (NS) TT=0.015	? (NS) TT=0.015	? (NS) TT=0.015	? (NS) TT=0.015	SAT TT=4.344	SAT TT=4.141	SAT TT=15.886	SAT TT=19.87	SAT TT=14.207	SAT TT=13.962	SAT TT=230.193	? (NS) TT=0.015	? (NS) TT=0.015	SAT TT=0.765	SAT TT=2.213	SAT TT=3.726	SAT TT=3.704	SAT TT=20.65	SAT TT=17.17	SAT TT=30.166	No data	No data	SAT TT=7.711	SAT TT=7.671	SAT TT=7.681	? TT=2.009	SAT TT=21.175	SAT TT=21.269	SAT TT=21.1	SAT TT=22.621
MisteryShopper/ MisteryShopper-06c_c18.xml	SAT TT=0.972	? (NS) TT=0.022	? (NS) TT=0.023	? (NS) TT=0.022	? (NS) TT=0.023	SAT TT=10.68	SAT TT=10.727	SAT TT=17.155	SAT TT=27.604	SAT TT=17.715	SAT TT=19.752	? (TO) TT=2519.94	? (NS) TT=0.022	? (NS) TT=0.022	SAT TT=0.972	SAT TT=1.276	SAT TT=240.344	SAT TT=237.055	SAT TT=4.393	SAT TT=4.497	SAT TT=4.358	No data	No data	SAT TT=21.855	SAT TT=18.129	SAT TT=18.391	? TT=2.542	SAT TT=28.871	SAT TT=32.121	SAT TT=27.909	SAT TT=33.781
MisteryShopper/ MisteryShopper-06d_c18.xml	SAT TT=8.062	? (NS) TT=0.034	? (NS) TT=0.034	? (NS) TT=0.034	? (NS) TT=0.035	SAT TT=9.025	SAT TT=8.062	SAT TT=30.175	SAT TT=25.884	SAT TT=19.752	SAT TT=31.664	? (TO) TT=2519.81	? (NS) TT=0.034	? (NS) TT=0.034	SAT TT=42.043	SAT TT=232.112	SAT TT=13.343	SAT TT=12.885	SAT TT=22.969	SAT TT=59.741	SAT TT=189.237	No data	No data	SAT TT=89.985	SAT TT=85.617	SAT TT=92.879	? TT=2.753	? (MO) TT=33.114	SAT TT=52.508	? (MO) TT=32.534	SAT TT=52.994
MisteryShopper/ MisteryShopper-10a_c18.xml	SAT TT=0.045	? (NS) TT=0.01	? (NS) TT=0.011	? (NS) TT=0.01	? (NS) TT=0.011	SAT TT=1.988	SAT TT=1.939	SAT TT=11.575	SAT TT=11.253	SAT TT=12.426	SAT TT=12.098	SAT TT=0.045	? (NS) TT=0.011	? (NS) TT=0.011	SAT TT=0.181	SAT TT=0.18	SAT TT=3.798	SAT TT=3.805	SAT TT=3.233	SAT TT=3.24	SAT TT=3.168	No data	No data	SAT TT=3.147	SAT TT=3.12	SAT TT=3.132	? TT=1.548	SAT TT=15.253	SAT TT=16.512	SAT TT=14.885	SAT TT=16.198
MisteryShopper/ MisteryShopper-10b_c18.xml	SAT TT=0.756	? (NS) TT=0.016	? (NS) TT=0.017	? (NS) TT=0.017	? (NS) TT=0.017	SAT TT=3.533	SAT TT=3.406	SAT TT=18.834	SAT TT=20.532	SAT TT=14.996	SAT TT=16.707	? (TO) TT=2519.91	? (NS) TT=0.017	? (NS) TT=0.017	SAT TT=0.756	SAT TT=0.919	? (TO) TT=2400.1	? (TO) TT=2520.08	SAT TT=145.494	SAT TT=22.245	SAT TT=26.783	No data	No data	SAT TT=5.58	SAT TT=5.921	SAT TT=5.848	? TT=2.462	SAT TT=24.282	SAT TT=24.833	SAT TT=22.995	SAT TT=25.445
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
MisteryShopper/ MisteryShopper-10c_c18.xml	SAT TT=5.952	? (NS) TT=0.028	? (NS) TT=0.029	? (NS) TT=0.03	? (NS) TT=0.029	SAT TT=12.574	SAT TT=12.938	SAT TT=26.891	? (TO) TT=2520.06	SAT TT=2216.12	? (TO) TT=2520.11	? (TO) TT=2519.94	? (NS) TT=0.028	? (NS) TT=0.028	SAT TT=9.995	SAT TT=5.952	? (TO) TT=2400.09	? (TO) TT=2520.04	SAT TT=51.829	SAT TT=503.639	SAT TT=79.824	No data	No data	SAT TT=25.479	SAT TT=25.395	SAT TT=25.536	? TT=2.722	? (MO) TT=33.159	SAT TT=37.589	? (MO) TT=32.857	SAT TT=38.503
MisteryShopper/ MisteryShopper-10d_c18.xml	SAT TT=31.059	? (NS) TT=0.048	? (NS) TT=0.049	? (NS) TT=0.049	? (NS) TT=0.049	SAT TT=657.127	SAT TT=651.882	SAT TT=31.059	? (TO) TT=2520.08	SAT TT=53.972	? (TO) TT=2520.09	? (TO) TT=2519.76	? (NS) TT=0.048	? (NS) TT=0.048	SAT TT=217.765	SAT TT=60.578	? (TO) TT=2400.07	? (TO) TT=2520.08	? (TO) TT=2400.04	? (TO) TT=2520.03	? (TO) TT=2520.02	No data	No data	SAT TT=139.899	SAT TT=136.098	SAT TT=155.813	? TT=4.441	? (MO) TT=44.789	SAT TT=138.117	? (MO) TT=43.934	SAT TT=137.543
MisteryShopper/ MisteryShopper-12_c18.xml	SAT TT=8.631	? (NS) TT=0.037	? (NS) TT=0.038	? (NS) TT=0.037	? (NS) TT=0.038	SAT TT=10.295	SAT TT=9.661	SAT TT=39.903	? (TO) TT=2520.05	SAT TT=61.058	? (TO) TT=2520.07	? (TO) TT=2519.79	? (NS) TT=0.038	? (NS) TT=0.038	SAT TT=8.631	SAT TT=53.578	? (TO) TT=2400.12	? (TO) TT=2520.09	? (TO) TT=2400.06	? (TO) TT=2520.05	? (TO) TT=2520.1	No data	No data	SAT TT=63.538	SAT TT=62.898	SAT TT=59.316	? TT=3.853	? (MO) TT=39.452	SAT TT=64.16	? (MO) TT=37.082	SAT TT=64.312
NurseRostering/ NurseRostering-00_c18.xml	OPT f=1202 TT=5.356 T1=2.76	No data	No data	No data	No data	OPT f=1202 TT=22.798 T1=9.22	OPT f=1202 TT=22.682 T1=9.3	OPT f=1202 TT=73.557 T1=10.72	OPT f=1202 TT=48.751 T1=17.16	OPT f=1202 TT=52.872 T1=10.44	OPT f=1202 TT=50.983 T1=7.59	OPT f=1202 TT=11.419 T1=3.56	No data	No data	OPT f=1202 TT=4.924 T1=0.63	OPT f=1202 TT=5.013 T1=0.66	No data	No data	SAT (TO) f=1202 TT=2400.11 T1=5.58	SAT (TO) f=1202 TT=2520.04 T1=8.87	OPT f=1202 TT=16.637 T1=6.82	OPT f=1202 TT=15.139 T1=9.69	OPT f=1202 TT=14.143 T1=9.16	OPT f=1202 TT=34.173 T1=34.17	OPT f=1202 TT=46.196 T1=46.19	OPT f=1202 TT=46.021 T1=46.01	OPT f=1202 TT=5.356 T1=2.76	No data	No data	No data	No data
NurseRostering/ NurseRostering-01_c18.xml	OPT f=607 TT=7.029 T1=2.93	No data	No data	No data	No data	OPT f=607 TT=134.022 T1=9.93	OPT f=607 TT=134.962 T1=9.92	OPT f=607 TT=339.207 T1=6.45	OPT f=607 TT=234.792 T1=5.58	OPT f=607 TT=232.457 T1=9.83	OPT f=607 TT=142.389 T1=6.7	OPT f=607 TT=20.858 T1=11.26	No data	No data	OPT f=607 TT=7.847 T1=0.45	OPT f=607 TT=7.778 T1=0.43	No data	No data	SAT (TO) f=607 TT=2400.04 T1=16.84	SAT (TO) f=607 TT=2520.03 T1=30.79	OPT f=607 TT=44.103 T1=18.73	OPT f=607 TT=48.242 T1=42.13	OPT f=607 TT=47.798 T1=41.19	OPT f=607 TT=18.025 T1=18.03	OPT f=607 TT=21.034 T1=21.04	OPT f=607 TT=18.42 T1=18.41	OPT f=607 TT=7.029 T1=2.93	No data	No data	No data	No data
NurseRostering/ NurseRostering-02_c18.xml	SAT f=1025 TT=2520.04 T1=83.24	No data	No data	No data	No data	SAT (TO) f=1427 TT=2400.05 T1=1539.13	SAT (TO) f=1427 TT=2520.05 T1=1536.22	SAT (TO) f=1227 TT=2520.12 T1=1975.96	SAT (TO) f=1356 TT=2520.1 T1=1394.75	SAT (TO) f=1439 TT=2520.09 T1=1224.54	SAT (TO) f=1539 TT=2520.06 T1=1842.32	SAT (TO) f=1558 TT=2520.04 T1=1955.68	No data	No data	SAT (TO) f=934 TT=2400.06 T1=2036.3	SAT (TO) f=934 TT=2519.98 T1=2047.8	No data	No data	SAT (TO) f=1458 TT=2400.04 T1=790.91	SAT (TO) f=2258 TT=2520.03 T1=876.37	SAT (TO) f=1753 TT=2520.07 T1=689.5	SAT (TO) f=1223 TT=2400.04 T1=1255.31	SAT (TO) f=1025 TT=2520.04 T1=83.24	? (TO) TT=2519.83	? (TO) TT=2519.98	? (TO) TT=2520.11	SAT (TO) f=1045 TT=2400.09 T1=2019.08	No data	No data	No data	No data
NurseRostering/ NurseRostering-03_c18.xml	SAT f=1249 TT=2400.37 T1=1967.68	No data	No data	No data	No data	SAT (TO) f=1502 TT=2400.13 T1=71.67	SAT (TO) f=1502 TT=2520.11 T1=71.18	SAT (TO) f=1745 TT=2520.11 T1=1764.92	SAT (TO) f=2345 TT=2520.09 T1=559.72	SAT (TO) f=1749 TT=2520.04 T1=1559.91	SAT (TO) f=1730 TT=2520.07 T1=220.54	SAT (TO) f=4453 TT=2519.88 T1=0.2	No data	No data	SAT (TO) f=1109 TT=2400.01 T1=1352.66	SAT (TO) f=1109 TT=2520.04 T1=1386.79	No data	No data	SAT (TO) f=3576 TT=2400.07 T1=1399.42	SAT (TO) f=3481 TT=2520.11 T1=404.68	SAT (TO) f=3682 TT=2520.09 T1=128.98	SAT (TO) f=1303 TT=2400.12 T1=1570.49	SAT (TO) f=1403 TT=2520.1 T1=131.59	? (TO) TT=2519.7	? (TO) TT=2519.94	? (TO) TT=2519.89	SAT (TO) f=1249 TT=2400.37 T1=1967.68	No data	No data	No data	No data
NurseRostering/ NurseRostering-04_c18.xml	SAT f=2070 TT=2400.07 T1=2328.19	No data	No data	No data	No data	SAT (TO) f=2424 TT=2400.09 T1=461.51	SAT (TO) f=2424 TT=2520.05 T1=461.14	SAT (TO) f=3369 TT=2520.07 T1=2260.98	SAT (TO) f=3753 TT=2520.08 T1=872.61	Wrong Cert. TT=2520.07	Wrong Cert. TT=2520.15	SAT (TO) f=2979 TT=2520.07 T1=192.15	No data	No data	SAT (TO) f=1832 TT=2400.06 T1=2298.72	SAT (TO) f=1832 TT=2519.99 T1=2297.84	No data	No data	SAT (TO) f=4285 TT=2400.07 T1=2103.47	SAT (TO) f=3807 TT=2520.1 T1=1068.38	SAT (TO) f=4502 TT=2520.12 T1=922.44	SAT (TO) f=2130 TT=2400.04 T1=1326.17	SAT (TO) f=2437 TT=2520.03 T1=1937.5	? (TO) TT=2519.74	Wrong Cert. TT=879.531	? (TO) TT=2519.94	SAT (TO) f=2070 TT=2400.07 T1=2328.19	No data	No data	No data	No data
NurseRostering/ NurseRostering-05_c18.xml	SAT f=1819 TT=2400.21 T1=2321.98	No data	No data	No data	No data	SAT (TO) f=2935 TT=2400.04 T1=556.2	SAT (TO) f=2935 TT=2520.06 T1=551.5	SAT (TO) f=3784 TT=2520.09 T1=746.53	SAT (TO) f=3966 TT=2520.11 T1=459.79	Wrong Cert. TT=2520.09	Wrong Cert. TT=2520.16	SAT (TO) f=6859 TT=2520.06 T1=1756.2	No data	No data	SAT (TO) f=2386 TT=2400 T1=1992.9	SAT (TO) f=2386 TT=2519.94 T1=1982.33	No data	No data	SAT (TO) f=5134 TT=2400.11 T1=1273.2	SAT (TO) f=5861 TT=2520.11 T1=778.41	SAT (TO) f=5648 TT=2520.08 T1=198.7	SAT (TO) f=2466 TT=2400.07 T1=1443.47	SAT (TO) f=2061 TT=2520.12 T1=925.4	? (TO) TT=2519.82	? (TO) TT=2520.06	? (TO) TT=2520.07	SAT (TO) f=1819 TT=2400.21 T1=2321.98	No data	No data	No data	No data
NurseRostering/ NurseRostering-06_c18.xml	SAT f=4362 TT=2520.1 T1=1094.45	No data	No data	No data	No data	SAT (TO) f=4362 TT=2400.06 T1=1077.8	SAT (TO) f=4362 TT=2520.1 T1=1094.45	SAT (TO) f=5702 TT=2520.11 T1=936.59	SAT (TO) f=6713 TT=2520.14 T1=543.43	SAT (TO) f=5987 TT=2520.09 T1=788.83	SAT (TO) f=5689 TT=2520.14 T1=18.33	SAT (TO) f=6854 TT=2520.06 T1=753.12	No data	No data	SAT (TO) f=3903 TT=2400.1 T1=1995.19	SAT (TO) f=3903 TT=2519.93 T1=2069.9	No data	No data	? (TO) TT=2400.09	SAT (TO) f=7585 TT=2520.07 T1=2318.33	? (TO) TT=2520.02	? (TO) TT=2400.06	? (TO) TT=2520.1	? (TO) TT=2520.03	? (TO) TT=2520.02	? (TO) TT=2520.04	SAT (TO) f=6222 TT=2400.19 T1=2390.43	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
NurseRostering/ NurseRostering-07_c18.xml	SAT f=5606 TT=2520.14 T1=1547.73	No data	No data	No data	No data	SAT (TO) f=5606 TT=2400.11 T1=1565.75	SAT (TO) f=5606 TT=2520.14 T1=1547.73	SAT (TO) f=6561 TT=2520.05 T1=713.58	SAT (TO) f=5928 TT=2520.12 T1=2183.75	Wrong Cert. TT=2520.12	Wrong Cert. TT=2520.07	SAT (TO) f=9193 TT=2520.09 T1=38.23	No data	No data	SAT (TO) f=6408 TT=2400.09 T1=2048.15	SAT (TO) f=6408 TT=2519.79 T1=2051.91	No data	No data	SAT (TO) f=7225 TT=2400.07 T1=867.15	SAT (TO) f=8227 TT=2520.05 T1=520.72	SAT (TO) f=7834 TT=2520.1 T1=210.05	? (TO) TT=2400.04	? (TO) TT=2520.09	? (TO) TT=2520.03	? (TO) TT=2519.94	? (TO) TT=2519.97	SAT (TO) f=6970 TT=2400.3 T1=2364.25	No data	No data	No data	No data
NurseRostering/ NurseRostering-08_c18.xml	SAT f=9900 TT=2520.05 T1=1576.42	No data	No data	No data	No data	SAT (TO) f=9900 TT=2400.08 T1=1577.81	SAT (TO) f=9900 TT=2520.05 T1=1576.42	? (TO) TT=2520.11	? (TO) TT=2520.11	? (TO) TT=2520.09	? (TO) TT=2520.16	SAT (TO) f=17697 TT=2520.07 T1=148.73	No data	No data	? (TO) TT=2400.06	? (TO) TT=2519.61	No data	No data	? (TO) TT=2400.12	? (TO) TT=2520.12	? (TO) TT=2520.03	? (TO) TT=2400.07	? (TO) TT=2520.11	? (TO) TT=2519.92	? (TO) TT=2519.92	? (TO) TT=2520.03	? (TO) TT=2400.34	No data	No data	No data	No data
NurseRostering/ NurseRostering-09_c18.xml	SAT f=2750 TT=2520.08 T1=393.15	No data	No data	No data	No data	SAT (TO) f=2750 TT=2400.08 T1=415.99	SAT (TO) f=2750 TT=2520.08 T1=393.15	SAT (TO) f=6148 TT=2520.08 T1=849.18	SAT (TO) f=6804 TT=2520.07 T1=2008.48	SAT (TO) f=6467 TT=2520.12 T1=10.71	SAT (TO) f=6485 TT=2520.14 T1=1242.26	SAT (TO) f=10554 TT=2520.03 T1=225.91	No data	No data	SAT (TO) f=6187 TT=2400.02 T1=2156.13	SAT (TO) f=6187 TT=2519.77 T1=2148.39	No data	No data	? (TO) TT=2400.02	SAT (TO) f=9909 TT=2520.05 T1=110.09	SAT (TO) f=10816 TT=2520.09 T1=32.31	? (TO) TT=2400.05	? (TO) TT=2520.08	? (TO) TT=2520.04	? (TO) TT=2520.08	? (TO) TT=2520.03	? (TO) TT=2400.4	No data	No data	No data	No data
NurseRostering/ NurseRostering-10_c18.xml	SAT f=10181 TT=2520.06 T1=1397.48	No data	No data	No data	No data	SAT (TO) f=10181 TT=2400.07 T1=1242.45	SAT (TO) f=10181 TT=2520.06 T1=1397.48	? (TO) TT=2520.13	? (TO) TT=2520.09	SAT (TO) f=13519 TT=2520.14 T1=762.43	SAT (TO) f=13343 TT=2520.13 T1=634.29	SAT (TO) f=24909 TT=2519.93 T1=1954.65	No data	No data	SAT (TO) f=19811 TT=2400.04 T1=1809.98	SAT (TO) f=19811 TT=2519.88 T1=1833.5	No data	No data	? (TO) TT=2400.05	? (TO) TT=2520.07	? (TO) TT=2520.14	? (TO) TT=2400.07	? (TO) TT=2520.05	? (TO) TT=2520.11	? (TO) TT=2520.1	? (TO) TT=2520.06	? (TO) TT=2400.26	No data	No data	No data	No data
NurseRostering/ NurseRostering-11_c18.xml	SAT f=9322 TT=2520.13 T1=1590.35	No data	No data	No data	No data	SAT (TO) f=9322 TT=2400.14 T1=1404.2	SAT (TO) f=9322 TT=2520.13 T1=1590.35	? (TO) TT=2520.14	? (TO) TT=2520.06	Wrong Cert. TT=2520.08	Wrong Cert. TT=2520.14	SAT (TO) f=37778 TT=2519.79 T1=479.44	No data	No data	? (TO) TT=2400.07	? (TO) TT=2520.03	No data	No data	? (TO) TT=2400.08	? (TO) TT=2520.1	? (TO) TT=2520.09	? (TO) TT=2400.05	? (TO) TT=2520.08	? (TO) TT=2519.95	? (TO) TT=2519.84	? (TO) TT=2519.69	? (TO) TT=2400.21	No data	No data	No data	No data
NurseRostering/ NurseRostering-12_c18.xml	SAT f=56621 TT=2519.9 T1=125.06	No data	No data	No data	No data	? (TO) TT=2400.13	? (TO) TT=2520.1	? (TO) TT=2520.15	? (TO) TT=2520.06	? (TO) TT=2520.05	? (TO) TT=2520.09	SAT (TO) f=56621 TT=2519.9 T1=125.06	No data	No data	? (TO) TT=2400.03	? (TO) TT=2520.02	No data	No data	? (TO) TT=2400.04	? (TO) TT=2520.09	? (TO) TT=2520.03	? (TO) TT=2400.11	? (TO) TT=2520.05	? (TO) TT=2520.04	? (TO) TT=2520	? (TO) TT=2520.01	? (TO) TT=2408.04	No data	No data	No data	No data
NurseRostering/ NurseRostering-13_c18.xml	SAT f=62102 TT=2520.11 T1=1851.25	No data	No data	No data	No data	? (TO) TT=2400.05	? (TO) TT=2520.06	? (TO) TT=2520.05	? (TO) TT=2520.15	? (TO) TT=2520.09	? (TO) TT=2520.14	SAT (TO) f=127588 TT=2519.95 T1=326.17	No data	No data	? (TO) TT=2400.06	? (TO) TT=2519.77	No data	No data	? (TO) TT=2400.08	SAT (TO) f=62102 TT=2520.11 T1=1851.25	? (TO) TT=2520.1	? (TO) TT=2400.06	? (TO) TT=2520.04	? (TO) TT=2520.08	? (TO) TT=2520.1	? (TO) TT=2520.08	? (TO) TT=2400.25	No data	No data	No data	No data
NurseRostering/ NurseRostering-14_c18.xml	SAT f=27589 TT=2519.95 T1=2456.38	No data	No data	No data	No data	? (TO) TT=2400.02	? (TO) TT=2520.05	? (TO) TT=2520.05	? (TO) TT=2520.06	? (TO) TT=2520.09	? (TO) TT=2520.06	SAT (TO) f=27589 TT=2519.95 T1=2456.38	No data	No data	? (TO) TT=2400.06	? (TO) TT=2520.02	No data	No data	? (TO) TT=2400.05	? (TO) TT=2520.06	? (TO) TT=2520.03	? (TO) TT=2400.04	? (TO) TT=2520.09	? (TO) TT=2519.89	? (TO) TT=2520.11	? (TO) TT=2519.89	? (TO) TT=2400.17	No data	No data	No data	No data
NurseRostering/ NurseRostering-15_c18.xml	SAT f=43817 TT=2519.82 T1=76.02	No data	No data	No data	No data	? (TO) TT=2400.03	? (TO) TT=2520.1	? (TO) TT=2520.1	? (TO) TT=2520.09	? (TO) TT=2520.08	? (TO) TT=2520.07	SAT (TO) f=43817 TT=2519.82 T1=76.02	No data	No data	? (TO) TT=2400.03	? (TO) TT=2519.86	No data	No data	? (TO) TT=2400.04	? (TO) TT=2520.11	? (TO) TT=2520.05	? (TO) TT=2400.1	? (TO) TT=2520.07	? (TO) TT=2520.03	? (TO) TT=2520.07	? (TO) TT=2520.05	? (TO) TT=2400.22	No data	No data	No data	No data
NurseRostering/ NurseRostering-16_c18.xml		No data	No data	No data	No data	? (TO) TT=2400.08	? (TO) TT=2520.03	? (TO) TT=2520.14	? (TO) TT=2520.13	? (TO) TT=2520.08	? (TO) TT=2520.14	? (TO) TT=2519.99	No data	No data	? (TO) TT=2400.1	? (TO) TT=2520.02	No data	No data	? (TO) TT=2400.1	? (TO) TT=2520.11	? (TO) TT=2520.08	? (TO) TT=2400.03	? (TO) TT=2520.11	? (TO) TT=2519.88	? (TO) TT=2519.63	? (TO) TT=2520.06	? (TO) TT=2400.35	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
NurseRostering/ NurseRostering-17_c18.xml		No data	No data	No data	No data	? (TO) TT=2400.04	? (TO) TT=2520.1	? (TO) TT=2520.1	? (TO) TT=2520.05	? (TO) TT=2520.12	? (TO) TT=2520.13	? (TO) TT=2519.93	No data	No data	? (TO) TT=2400.06	? (TO) TT=2519.97	No data	No data	? (TO) TT=2400.06	? (TO) TT=2520.04	? (TO) TT=2520.05	? (TO) TT=2400.03	? (TO) TT=2520.12	? (TO) TT=2520.12	? (TO) TT=2519.65	? (TO) TT=2520.05	? (TO) TT=2400.22	No data	No data	No data	No data
NurseRostering/ NurseRostering-18_c18.xml		No data	No data	No data	No data	? (TO) TT=2400.06	? (TO) TT=2520.06	? (TO) TT=2520.07	? (TO) TT=2520.14	? (TO) TT=2520.1	? (TO) TT=2520.07	? (TO) TT=2520.05	No data	No data	? (TO) TT=2400.09	? (TO) TT=2520.04	No data	No data	? (TO) TT=2400.1	? (TO) TT=2520.1	? (TO) TT=2520.07	? (TO) TT=2400.1	? (TO) TT=2520.11	? (TO) TT=2519.84	? (TO) TT=2520.03	? (TO) TT=2520.09	? (TO) TT=2400.23	No data	No data	No data	No data
NurseRostering/ NurseRostering-19_c18.xml		No data	No data	No data	No data	? (TO) TT=2400.04	? (TO) TT=2520.22	? (TO) TT=2520.07	? (TO) TT=2520.07	? (TO) TT=2520.15	? (TO) TT=2520.05	? (TO) TT=2520.01	No data	No data	? (TO) TT=2400.06	? (TO) TT=2519.93	No data	No data	? (TO) TT=2400.03	? (TO) TT=2520.08	? (TO) TT=2520.11	? (TO) TT=2400.09	? (TO) TT=2520.12	? (TO) TT=2520	? (TO) TT=2520.09	? (TO) TT=2519.92	? (TO) TT=2400.21	No data	No data	No data	No data
NurseRostering/ NurseRostering-20_c18.xml		No data	No data	No data	No data	? (TO) TT=2400.06	? (TO) TT=2520.06	? (TO) TT=2520.19	? (TO) TT=2520.1	? (TO) TT=2520.15	? (TO) TT=2520.08	? (TO) TT=2519.63	No data	No data	? (TO) TT=2400.05	? (TO) TT=2519.85	No data	No data	? (TO) TT=2400.14	? (TO) TT=2520.11	? (TO) TT=2520.13	? (TO) TT=2400.11	? (TO) TT=2520.11	? (TO) TT=2520.07	? (TO) TT=2520.06	? (TO) TT=2520.05	? (TO) TT=2400.18	No data	No data	No data	No data
PeacableArmies/ PeacableArmies-m1-05_c18.xml	OPT f=4 TT=0.193 T1=0.13	No data	No data	No data	No data	OPT f=4 TT=1.935 T1=0.66	OPT f=4 TT=1.939 T1=0.65	OPT f=4 TT=14.773 T1=6.08	OPT f=4 TT=15.339 T1=5.83	OPT f=4 TT=13.347 T1=4.91	OPT f=4 TT=13.02 T1=4.87	OPT f=4 TT=1.54 T1=0.13	No data	No data	OPT f=4 TT=0.27 T1=0.11	OPT f=4 TT=0.271 T1=0.11	No data	No data	SAT (TO) f=4 TT=2400.1 T1=1.28	SAT (TO) f=4 TT=2520.09 T1=1.24	OPT f=4 TT=2.506 T1=1.24	OPT f=4 TT=2.558 T1=1.32	OPT f=4 TT=2.528 T1=1.29	OPT f=4 TT=0.192 T1=0.19	OPT f=4 TT=0.193 T1=0.19	OPT f=4 TT=0.193 T1=0.19	OPT f=4 TT=2.098 T1=0.71	No data	No data	No data	No data
PeacableArmies/ PeacableArmies-m1-07_c18.xml	OPT f=7 TT=3.049 T1=1.23	No data	No data	No data	No data	OPT f=7 TT=33.257 T1=1.24	OPT f=7 TT=33.403 T1=1.23	OPT f=7 TT=645.446 T1=7.58	OPT f=7 TT=388.204 T1=8.69	OPT f=7 TT=128.467 T1=6.9	OPT f=7 TT=82.957 T1=7.11	OPT f=7 TT=1444.6 T1=9.4	No data	No data	OPT f=7 TT=217.533 T1=2.45	OPT f=7 TT=218.191 T1=2.46	No data	No data	SAT (TO) f=7 TT=2400.1 T1=1.55	SAT (TO) f=7 TT=2520.11 T1=1.49	OPT f=7 TT=163.025 T1=1.49	SAT (TO) f=7 TT=2400.11 T1=5.22	SAT (TO) f=7 TT=2520.07 T1=4.44	OPT f=7 TT=2.919 T1=2.91	OPT f=7 TT=3.06 T1=3.06	OPT f=7 TT=3.049 T1=3.05	OPT f=7 TT=461.421 T1=4.93	No data	No data	No data	No data
PeacableArmies/ PeacableArmies-m1-10_c18.xml	SAT f=14 TT=2520.05 T1=1737.61	No data	No data	No data	No data	SAT (TO) f=13 TT=2400.05 T1=187.34	SAT (TO) f=13 TT=2520.11 T1=199.66	SAT (TO) f=13 TT=2520.18 T1=81.16	SAT (TO) f=13 TT=2520.06 T1=351.03	SAT (TO) f=14 TT=2520.09 T1=1115.23	SAT (TO) f=13 TT=2520.17 T1=67.99	SAT (TO) f=11 TT=2519.78 T1=201.18	No data	No data	SAT (TO) f=14 TT=2400.01 T1=188.82	SAT (TO) f=14 TT=2519.95 T1=174.53	No data	No data	SAT (TO) f=13 TT=2400.02 T1=515.66	SAT (TO) f=12 TT=2520.12 T1=18.97	SAT (TO) f=13 TT=2520.11 T1=611.64	SAT (TO) f=13 TT=2400.1 T1=1049.04	SAT (TO) f=14 TT=2520.05 T1=1737.61	? (TO) TT=2519.99	? (TO) TT=2520.07	? (TO) TT=2519.76	SAT (TO) f=13 TT=2400.25 T1=1758.32	No data	No data	No data	No data
PeacableArmies/ PeacableArmies-m1-15_c18.xml	SAT f=25 TT=2520.09 T1=2.97	No data	No data	No data	No data	SAT (TO) f=21 TT=2400.13 T1=663.43	SAT (TO) f=21 TT=2520.09 T1=524.74	SAT (TO) f=22 TT=2520.06 T1=43.74	SAT (TO) f=22 TT=2520.73 T1=720.6	SAT (TO) f=25 TT=2520.13 T1=1960.9	SAT (TO) f=24 TT=2520.14 T1=220.35	SAT (TO) f=18 TT=2519.56 T1=269.86	No data	No data	SAT (TO) f=24 TT=2400.06 T1=50.84	SAT (TO) f=24 TT=2519.7 T1=50.13	No data	No data	SAT (TO) f=25 TT=2400.03 T1=2.93	SAT (TO) f=25 TT=2520.12 T1=2.97	SAT (TO) f=25 TT=2520.09 T1=2.99	SAT (TO) f=28 TT=2400.12 T1=260.82	SAT (TO) f=24 TT=2520.06 T1=78.82	? (TO) TT=2520.01	? (TO) TT=2519.92	? (TO) TT=2520.08	? (TO) TT=2400.13	No data	No data	No data	No data
PeacableArmies/ PeacableArmies-m1-20_c18.xml	SAT f=46 TT=2520.1 T1=582.92	No data	No data	No data	No data	SAT (TO) f=26 TT=2400.08 T1=359.46	SAT (TO) f=26 TT=2520.11 T1=330.79	? (TO) TT=2522.15	SAT (TO) f=33 TT=2520.32 T1=198.92	SAT (TO) f=38 TT=2520.38 T1=858.3	SAT (TO) f=35 TT=2520.15 T1=1180.81	SAT (TO) f=25 TT=2519.6 T1=185.66	No data	No data	SAT (TO) f=41 TT=2400.09 T1=886.47	SAT (TO) f=41 TT=2519.82 T1=854.99	No data	No data	SAT (TO) f=34 TT=2400.11 T1=4.56	SAT (TO) f=35 TT=2520.13 T1=4.87	SAT (TO) f=36 TT=2520.09 T1=14.16	SAT (TO) f=46 TT=2400.06 T1=36.59	SAT (TO) f=46 TT=2520.1 T1=582.92	? (TO) TT=2519.94	? (TO) TT=2520.04	? (TO) TT=2519.92	? (TO) TT=2400.12	No data	No data	No data	No data
PeacableArmies/ PeacableArmies-m1-30_c18.xml	SAT f=96 TT=2520.14 T1=195.65	No data	No data	No data	No data	SAT (TO) f=44 TT=2400.22 T1=756.04	SAT (TO) f=44 TT=2520.12 T1=585.97	SAT f=32 TT=1735.8 T1=1189.2	? (TO) TT=2527.38	? (TO) TT=2528.18	? (TO) TT=2527.97	SAT (TO) f=44 TT=2519.72 T1=219.37	No data	No data	SAT (TO) f=71 TT=2400.04 T1=224.52	SAT (TO) f=71 TT=2519.56 T1=232.08	No data	No data	SAT (TO) f=69 TT=2400.11 T1=17.02	SAT (TO) f=68 TT=2520.14 T1=8.54	SAT (TO) f=68 TT=2520.13 T1=8.64	SAT (TO) f=99 TT=2400.12 T1=822.22	SAT (TO) f=96 TT=2520.14 T1=195.65	? (TO) TT=2519.81	? (TO) TT=2519.49	? (TO) TT=2520.09	? (TO) TT=2400.17	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
PeacableArmies/ PeacableArmies-m1-50_c18.xml	SAT f=184 TT=2520.37 T1=598.55	No data	No data	No data	No data	SAT (TO) f=179 TT=2400.07 T1=410.36	SAT (TO) f=179 TT=2520.09 T1=403.77	? (TO) TT=2522.09	? (TO) TT=2522.11	? (TO) TT=2527.88	? (TO) TT=2528.12	? (MO) TT=101.073	No data	No data	SAT (TO) f=99 TT=2400.07 T1=1726.93	SAT (TO) f=99 TT=2520.21 T1=1888.69	No data	No data	SAT (TO) f=173 TT=2400.09 T1=92.03	SAT (TO) f=173 TT=2520.06 T1=160.11	SAT (TO) f=177 TT=2520.11 T1=668.99	SAT (TO) f=190 TT=2400.16 T1=1465.95	SAT (TO) f=184 TT=2520.37 T1=598.55	? (TO) TT=2520.05	? (TO) TT=2519.95	? (TO) TT=2519.92	? TT=48.435	No data	No data	No data	No data
PeacableArmies/ PeacableArmies-m2-05_c18.xml	OPT f=4 TT=0.159 T1=0.1	No data	No data	No data	No data	OPT f=4 TT=1.704 T1=0.69	OPT f=4 TT=1.801 T1=0.7	OPT f=4 TT=32.048 T1=5.41	OPT f=4 TT=32.817 T1=6.79	OPT f=4 TT=30.628 T1=5.83	OPT f=4 TT=47.34 T1=4.42	OPT f=4 TT=0.518 T1=0.1	No data	No data	OPT f=4 TT=0.449 T1=0.07	OPT f=4 TT=0.448 T1=0.07	No data	No data	SAT (TO) f=4 TT=2400.11 T1=1.38	SAT (TO) f=4 TT=2520.04 T1=1.31	OPT f=4 TT=16.277 T1=1.6	SAT (TO) f=4 TT=2400.07 T1=2.7	SAT (TO) f=4 TT=2520.09 T1=2.06	OPT f=4 TT=0.115 T1=0.11	OPT f=4 TT=0.161 T1=0.16	OPT f=4 TT=0.159 T1=0.16	OPT f=4 TT=1.804 T1=0.59	No data	No data	No data	No data
PeacableArmies/ PeacableArmies-m2-07_c18.xml	OPT f=7 TT=2.866 T1=1.29	No data	No data	No data	No data	OPT f=7 TT=23.567 T1=8.05	OPT f=7 TT=22.233 T1=8.14	? (TO) TT=2520.13	SAT (TO) f=7 TT=2520.11 T1=1062.31	SAT (TO) f=7 TT=2520.07 T1=6.1	SAT (TO) f=6 TT=2520.12 T1=5.33	OPT f=7 TT=369.984 T1=29.76	No data	No data	OPT f=7 TT=589.116 T1=0.29	OPT f=7 TT=596.717 T1=0.28	No data	No data	SAT (TO) f=6 TT=2400.04 T1=326.08	SAT (TO) f=7 TT=2520.1 T1=1456.17	SAT (TO) f=6 TT=2520.08 T1=418.76	SAT (TO) f=7 TT=2400.1 T1=3.49	SAT (TO) f=7 TT=2520.11 T1=4.19	OPT f=7 TT=3.249 T1=3.24	OPT f=7 TT=2.893 T1=2.89	OPT f=7 TT=2.866 T1=2.86	OPT f=7 TT=29.687 T1=1.29	No data	No data	No data	No data
PeacableArmies/ PeacableArmies-m2-10_c18.xml	SAT f=14 TT=2400.25 T1=66.74	No data	No data	No data	No data	SAT (TO) f=11 TT=2400.11 T1=822.7	SAT (TO) f=11 TT=2520.03 T1=854.24	? (TO) TT=2520.14	? (TO) TT=2520.14	SAT (TO) f=13 TT=2520.05 T1=269.26	SAT (TO) f=12 TT=2520.05 T1=15.58	SAT (TO) f=11 TT=2519.8 T1=1935.69	No data	No data	SAT (TO) f=14 TT=2400.05 T1=340.68	SAT (TO) f=14 TT=2520.06 T1=341.46	No data	No data	? (TO) TT=2400.07	? (TO) TT=2520.09	? (TO) TT=2520.09	SAT (TO) f=9 TT=2400.03 T1=4.78	SAT (TO) f=13 TT=2520.07 T1=165.01	? (TO) TT=2519.9	? (TO) TT=2520.06	? (TO) TT=2519.95	SAT (TO) f=14 TT=2400.25 T1=66.74	No data	No data	No data	No data
PeacableArmies/ PeacableArmies-m2-15_c18.xml	SAT f=29 TT=2400.29 T1=827.81	No data	No data	No data	No data	SAT (TO) f=13 TT=2400.04 T1=1504.24	SAT (TO) f=13 TT=2520.04 T1=1506.9	? (TO) TT=2520.06	? (TO) TT=2520.13	SAT (TO) f=26 TT=2520.08 T1=2340.73	SAT (TO) f=21 TT=2520.14 T1=381.92	SAT (TO) f=12 TT=2519.92 T1=503.37	No data	No data	SAT (TO) f=24 TT=2400.08 T1=13.06	SAT (TO) f=24 TT=2519.89 T1=12.97	No data	No data	? (TO) TT=2400.12	? (TO) TT=2520.03	? (TO) TT=2520.08	SAT (TO) f=13 TT=2400.09 T1=14.82	SAT (TO) f=23 TT=2520.04 T1=20.8	? (TO) TT=2519.89	? (TO) TT=2519.95	? (TO) TT=2519.96	SAT (TO) f=29 TT=2400.29 T1=827.81	No data	No data	No data	No data
PeacableArmies/ PeacableArmies-m2-20_c18.xml	SAT f=42 TT=2400.21 T1=1164.53	No data	No data	No data	No data	SAT (TO) f=12 TT=2400.2 T1=345.7	SAT (TO) f=12 TT=2520.06 T1=349.17	? (TO) TT=2520.08	? (TO) TT=2520.14	? (TO) TT=2520.15	? (TO) TT=2520.08	SAT (TO) f=13 TT=2519.67 T1=2383.19	No data	No data	SAT (TO) f=38 TT=2400.02 T1=873.3	SAT (TO) f=38 TT=2519.96 T1=871.66	No data	No data	? (TO) TT=2400.03	? (TO) TT=2520.04	? (TO) TT=2520.07	SAT (TO) f=28 TT=2400.06 T1=102.83	SAT (TO) f=22 TT=2520.12 T1=72.72	? (TO) TT=2519.86	? (TO) TT=2520.01	? (TO) TT=2519.69	SAT (TO) f=42 TT=2400.21 T1=1164.53	No data	No data	No data	No data
PeacableArmies/ PeacableArmies-m2-30_c18.xml	SAT f=83 TT=2520.1 T1=1756.76	No data	No data	No data	No data	SAT (TO) f=12 TT=2400.19 T1=2348.91	SAT (TO) f=12 TT=2520.05 T1=2364.47	? (TO) TT=2520.1	? (TO) TT=2520.12	? (TO) TT=2520.11	? (TO) TT=2520.25	SAT (TO) f=15 TT=2519.71 T1=1333.07	No data	No data	SAT (TO) f=70 TT=2400.02 T1=135.22	SAT (TO) f=70 TT=2519.92 T1=137.58	No data	No data	? (TO) TT=2400.04	? (TO) TT=2520.1	? (TO) TT=2520.11	SAT (TO) f=17 TT=2400.07 T1=842.03	SAT (TO) f=83 TT=2520.1 T1=1756.76	? (TO) TT=2519.97	? (TO) TT=2520.04	? (TO) TT=2520.06	SAT (TO) f=60 TT=2400.25 T1=2351.8	No data	No data	No data	No data
PeacableArmies/ PeacableArmies-m2-50_c18.xml	SAT f=113 TT=2520.11 T1=1520.64	No data	No data	No data	No data	SAT (TO) f=9 TT=2400.08 T1=2199.45	SAT (TO) f=9 TT=2520.04 T1=2183.14	? (TO) TT=2522.11	? (TO) TT=2522.08	? (TO) TT=2522.1	? (TO) TT=2522.11	SAT (TO) f=13 TT=2520.13 T1=1135.96	No data	No data	SAT (TO) f=196 TT=2400.01 T1=2050.31	SAT (TO) f=196 TT=2519.92 T1=2036.96	No data	No data	SAT (TO) f=142 TT=2400.09 T1=2328.81	? (TO) TT=2520.14	SAT (TO) f=113 TT=2520.11 T1=1520.64	SAT (TO) f=41 TT=2400.08 T1=1823.05	SAT (TO) f=35 TT=2520.14 T1=1879.83	? (TO) TT=2520.01	? (TO) TT=2520.09	? (TO) TT=2520.09	? (TO) TT=2400.21	No data	No data	No data	No data
PizzaVoucher/ PizzaVoucher-10a_c18.xml	OPT f=210 TT=4.349 T1=1.01	No data	No data	No data	No data	OPT f=210 TT=11.484 T1=1.26	OPT f=210 TT=11.839 T1=1.01	OPT f=210 TT=18.634 T1=7.79	OPT f=210 TT=14.799 T1=5.46	OPT f=210 TT=21.225 T1=4.12	OPT f=210 TT=16.879 T1=4.74	OPT f=210 TT=187.29 T1=100.04	No data	No data	OPT f=210 TT=20.409 T1=5.12	OPT f=210 TT=20.22 T1=5.08	No data	No data	SAT (TO) f=210 TT=2400.04 T1=59.39	SAT (TO) f=210 TT=2520.03 T1=149.78	OPT f=210 TT=340.163 T1=54.65	SAT (TO) f=210 TT=2400.09 T1=274.72	SAT (TO) f=210 TT=2520.12 T1=275.57	OPT f=210 TT=6.552 T1=6.55	OPT f=210 TT=6.57 T1=6.56	OPT f=210 TT=6.617 T1=6.62	OPT f=210 TT=4.349 T1=1.74	No data	No data	No data	No data
PizzaVoucher/ PizzaVoucher-10b_c18.xml	OPT f=28 TT=5.228 T1=1.4	No data	No data	No data	No data	OPT f=28 TT=235.003 T1=8.43	OPT f=28 TT=241.833 T1=8.79	OPT f=28 TT=119.986 T1=5.17	OPT f=28 TT=88.422 T1=5.52	OPT f=28 TT=75.108 T1=4.37	OPT f=28 TT=56.712 T1=14.31	OPT f=28 TT=614.685 T1=206.14	No data	No data	OPT f=28 TT=19.608 T1=2.54	OPT f=28 TT=19.57 T1=2.53	No data	No data	SAT (TO) f=28 TT=2400.09 T1=6.68	SAT (TO) f=28 TT=2520.04 T1=8.61	OPT f=28 TT=471.92 T1=11.1	SAT (TO) f=28 TT=2400.07 T1=242.69	SAT (TO) f=28 TT=2520.08 T1=243.14	OPT f=28 TT=5.198 T1=5.19	OPT f=28 TT=5.118 T1=5.11	OPT f=28 TT=5.228 T1=5.22	OPT f=28 TT=10.074 T1=1.4	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
PizzaVoucher/ PizzaVoucher-12a_c18.xml	OPT f=40 TT=6.445 T1=3.38	No data	No data	No data	No data	OPT f=40 TT=1903.72 T1=129.44	OPT f=40 TT=1914.34 T1=129.07	OPT f=40 TT=204.046 T1=93.37	OPT f=40 TT=453.889 T1=253.24	OPT f=40 TT=204.472 T1=59.15	OPT f=40 TT=235.362 T1=155.76	SAT (TO) f=41 TT=2520 T1=9.21	No data	No data	OPT f=40 TT=362.191 T1=87.48	OPT f=40 TT=366.443 T1=88.64	No data	No data	SAT (TO) f=40 TT=2400.03 T1=47.68	SAT (TO) f=40 TT=2520.06 T1=1139.62	SAT (TO) f=43 TT=2520.09 T1=267.83	? (TO) TT=2400.07	? (TO) TT=2520.06	OPT f=40 TT=8.204 T1=8.2	OPT f=40 TT=8.022 T1=8.02	OPT f=40 TT=7.579 T1=7.57	OPT f=40 TT=6.445 T1=3.38	No data	No data	No data	No data
PizzaVoucher/ PizzaVoucher-12b_c18.xml	OPT f=40 TT=8.127 T1=6.34	No data	No data	No data	No data	SAT (TO) f=40 TT=2400.12 T1=322.9	SAT (TO) f=40 TT=2520.07 T1=319.21	OPT f=40 TT=1435.49 T1=209.01	OPT f=40 TT=682.204 T1=456.82	OPT f=40 TT=606.653 T1=528.39	OPT f=40 TT=313.287 T1=253.84	SAT (TO) f=41 TT=2520.1 T1=35.02	No data	No data	OPT f=40 TT=1365.99 T1=5.05	OPT f=40 TT=1356.03 T1=4.96	No data	No data	SAT (TO) f=45 TT=2400.11 T1=19.98	SAT (TO) f=42 TT=2520.11 T1=1616.79	SAT (TO) f=49 TT=2520.1 T1=5.35	? (TO) TT=2400.04	? (TO) TT=2520.04	OPT f=40 TT=7.645 T1=7.64	OPT f=40 TT=7.732 T1=7.73	OPT f=40 TT=8.127 T1=8.13	OPT f=40 TT=10.45 T1=6.34	No data	No data	No data	No data
PizzaVoucher/ PizzaVoucher-15a_c18.xml	OPT f=42 TT=58.138 T1=12.3	No data	No data	No data	No data	SAT (TO) f=43 TT=2400.09 T1=551.27	SAT (TO) f=43 TT=2520.26 T1=550.04	SAT (TO) f=42 TT=2520.07 T1=12.73	SAT (TO) f=43 TT=2520.11 T1=2456.49	SAT (TO) f=57 TT=2520.07 T1=14.99	SAT (TO) f=44 TT=2520.04 T1=34.26	SAT (TO) f=48 TT=2520.01 T1=350.18	No data	No data	SAT (TO) f=42 TT=2400.03 T1=9.93	SAT (TO) f=42 TT=2519.82 T1=9.88	No data	No data	SAT (TO) f=54 TT=2400.05 T1=643.88	SAT (TO) f=45 TT=2520.12 T1=16.09	SAT (TO) f=55 TT=2520.12 T1=9.78	? (TO) TT=2400.03	? (TO) TT=2520.03	OPT f=42 TT=62.118 T1=62.12	OPT f=42 TT=59.272 T1=59.26	OPT f=42 TT=58.138 T1=58.14	OPT f=42 TT=273.846 T1=12.3	No data	No data	No data	No data
PizzaVoucher/ PizzaVoucher-15b_c18.xml	OPT f=42 TT=517.492 T1=7.27	No data	No data	No data	No data	SAT (TO) f=44 TT=2400.02 T1=363.08	SAT (TO) f=44 TT=2520.06 T1=363.57	SAT (TO) f=42 TT=2520.07 T1=41.51	SAT (TO) f=42 TT=2520.05 T1=7.27	SAT (TO) f=47 TT=2520.14 T1=1648.6	SAT (TO) f=46 TT=2520.03 T1=563.42	SAT (TO) f=47 TT=2520.1 T1=2139.1	No data	No data	SAT (TO) f=42 TT=2400.09 T1=6.41	SAT (TO) f=42 TT=2520.04 T1=6.25	No data	No data	SAT (TO) f=47 TT=2400.04 T1=43.83	SAT (TO) f=47 TT=2520.04 T1=1032.79	SAT (TO) f=56 TT=2520.06 T1=22.71	? (TO) TT=2400.11	? (TO) TT=2520.14	OPT f=42 TT=577.122 T1=577.14	OPT f=42 TT=476.221 T1=476.2	OPT f=42 TT=517.492 T1=517.48	SAT (TO) f=42 TT=2400.19 T1=9.28	No data	No data	No data	No data
PizzaVoucher/ PizzaVoucher-18a_c18.xml	OPT f=49 TT=1175.91 T1=78.05	No data	No data	No data	No data	SAT (TO) f=56 TT=2400.1 T1=479.09	SAT (TO) f=56 TT=2520.08 T1=474.76	SAT (TO) f=53 TT=2520.14 T1=127.87	SAT (TO) f=55 TT=2520.15 T1=61.96	SAT (TO) f=55 TT=2520.08 T1=1863.38	SAT (TO) f=56 TT=2520.09 T1=872.13	SAT (TO) f=58 TT=2519.98 T1=715.94	No data	No data	SAT (TO) f=49 TT=2400.07 T1=45.77	SAT (TO) f=49 TT=2520.01 T1=48.08	No data	No data	SAT (TO) f=71 TT=2400.08 T1=84.55	SAT (TO) f=71 TT=2520.05 T1=889.31	SAT (TO) f=97 TT=2520.08 T1=2360.15	? (TO) TT=2400.12	? (TO) TT=2520.04	OPT f=49 TT=1443.27 T1=1443.25	OPT f=49 TT=1330.83 T1=1331.14	OPT f=49 TT=1175.91 T1=1175.96	SAT (TO) f=49 TT=2400.23 T1=78.05	No data	No data	No data	No data
PizzaVoucher/ PizzaVoucher-18b_c18.xml	SAT f=49 TT=2400.23 T1=43.24	No data	No data	No data	No data	SAT (TO) f=70 TT=2400.12 T1=366.69	SAT (TO) f=70 TT=2520.07 T1=379.78	SAT (TO) f=61 TT=2520.13 T1=76.5	SAT (TO) f=57 TT=2520.11 T1=493.11	SAT (TO) f=62 TT=2520.07 T1=92.13	SAT (TO) f=62 TT=2520.11 T1=30.1	? (TO) TT=2520.01	No data	No data	SAT (TO) f=50 TT=2400.03 T1=29.48	SAT (TO) f=50 TT=2519.92 T1=28.71	No data	No data	? (TO) TT=2400.12	? (TO) TT=2520.12	? (TO) TT=2520.13	? (TO) TT=2400.03	? (TO) TT=2520.07	? (TO) TT=2519.87	? (TO) TT=2520.06	? (TO) TT=2520.04	SAT (TO) f=49 TT=2400.23 T1=43.24	No data	No data	No data	No data
PizzaVoucher/ PizzaVoucher-22a_c18.xml	SAT f=57 TT=2400.29 T1=462.3	No data	No data	No data	No data	SAT (TO) f=91 TT=2400.14 T1=734.97	SAT (TO) f=91 TT=2520.07 T1=755.94	SAT (TO) f=74 TT=2520.08 T1=172.42	SAT (TO) f=62 TT=2520.12 T1=212.59	SAT (TO) f=77 TT=2520.12 T1=8.24	SAT (TO) f=74 TT=2520.09 T1=520.68	? (TO) TT=2519.84	No data	No data	SAT (TO) f=61 TT=2400.06 T1=356.16	SAT (TO) f=61 TT=2519.82 T1=331.23	No data	No data	? (TO) TT=2400.1	? (TO) TT=2520.04	? (TO) TT=2520.07	? (TO) TT=2400.08	? (TO) TT=2520.06	? (TO) TT=2519.76	? (TO) TT=2520.07	? (TO) TT=2520.03	SAT (TO) f=57 TT=2400.29 T1=462.3	No data	No data	No data	No data
PizzaVoucher/ PizzaVoucher-22b_c18.xml	SAT f=54 TT=2400.26 T1=306.24	No data	No data	No data	No data	? (TO) TT=2400.05	? (TO) TT=2520.11	SAT (TO) f=65 TT=2520.12 T1=71.47	SAT (TO) f=61 TT=2520.05 T1=1835.5	SAT (TO) f=65 TT=2520.05 T1=451.2	SAT (TO) f=68 TT=2520.08 T1=862.23	? (TO) TT=2519.98	No data	No data	SAT (TO) f=58 TT=2400.06 T1=1816.17	SAT (TO) f=58 TT=2519.91 T1=1818.44	No data	No data	? (TO) TT=2400.13	? (TO) TT=2520.06	? (TO) TT=2520.03	? (TO) TT=2400.04	? (TO) TT=2520.1	? (TO) TT=2519.93	? (TO) TT=2520.05	? (TO) TT=2519.87	SAT (TO) f=54 TT=2400.26 T1=306.24	No data	No data	No data	No data
PseudoBoolean-dec/ Pb-chnl10-11_c18.xml	UNSAT TT=4.242	? (TO) TT=2519.91	? (TO) TT=2519.97	? (TO) TT=2519.99	? (TO) TT=2519.87	? (TO) TT=2400.07	? (TO) TT=2520.12	UNSAT TT=967.168	UNSAT TT=782.014	UNSAT TT=1770.84	UNSAT TT=2156.55	? (TO) TT=2519.96	? TT=2520.01	? (TO) TT=2519.93	UNSAT TT=391.903	UNSAT TT=373.18	UNSAT TT=117.824	UNSAT TT=115.284	? (TO) TT=2400.09	? (TO) TT=2520.03	UNSAT TT=354.279	No data	No data	UNSAT TT=2225.79	UNSAT TT=2286.07	UNSAT TT=2292.45	UNSAT TT=4.242	UNSAT TT=22.519	UNSAT TT=38.877	UNSAT TT=49.948	UNSAT TT=17.068
PseudoBoolean-dec/ Pb-chnl15-16_c18.xml	UNSAT TT=166.895	? (TO) TT=2519.93	? TT=2520.01	? TT=2520.01	? TT=2520.01	? (TO) TT=2400.11	? (TO) TT=2520.04	? (TO) TT=2520.05	? (TO) TT=2520.04	? (TO) TT=2520.09	? (TO) TT=2520.06	? (TO) TT=2520.08	? TT=2520.01	? (TO) TT=2519.98	? (TO) TT=2400.05	? (TO) TT=2520.01	? (TO) TT=2400.11	? (TO) TT=2520.06	? (TO) TT=2400.07	? (TO) TT=2520.11	? (TO) TT=2520.1	No data	No data	? (TO) TT=2520.04	? (TO) TT=2520.08	? (TO) TT=2519.85	? (TO) TT=2400.12	? (TO) TT=2520.1	? (TO) TT=2520.13	UNSAT TT=166.895	UNSAT TT=358.384
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
PseudoBoolean-dec/ Pb-chnl20-21_c18.xml		? (TO) TT=2519.9	? (TO) TT=2519.97	? TT=2520.02	? (TO) TT=2520.02	? (TO) TT=2400.03	? (TO) TT=2520.09	? (TO) TT=2520.1	? (TO) TT=2520.14	? (TO) TT=2520.06	? (TO) TT=2520.07	? (TO) TT=2520.04	? (TO) TT=2519.95	? (TO) TT=2519.99	? (TO) TT=2400.01	? (TO) TT=2519.91	? (TO) TT=2400.09	? (TO) TT=2520.07	? (TO) TT=2400.09	? (TO) TT=2520.1	? (TO) TT=2520.04	No data	No data	? (TO) TT=2519.68	? (TO) TT=2520.05	? (TO) TT=2519.9	? (TO) TT=2400.15	? (TO) TT=2520.13	? (TO) TT=2520.14	? (TO) TT=2520.07	? (TO) TT=2520.09
PseudoBoolean-dec/ Pb-chnl30-31_c18.xml		? (TO) TT=2519.91	? (TO) TT=2519.84	? (TO) TT=2520.05	? (TO) TT=2519.99	? (TO) TT=2400.04	? (TO) TT=2520.04	? (TO) TT=2520.06	? (TO) TT=2520.15	? (TO) TT=2520.09	? (TO) TT=2520.11	? (TO) TT=2520.09	? TT=2520.07	? (TO) TT=2519.9	? (TO) TT=2400.02	? (TO) TT=2519.61	? (TO) TT=2400.1	? (TO) TT=2520.12	? (TO) TT=2400.06	? (TO) TT=2520.02	? (TO) TT=2520.04	No data	No data	? (TO) TT=2520.06	? (TO) TT=2519.54	? (TO) TT=2519.61	? (TO) TT=2400.14	? (TO) TT=2520.11	? (TO) TT=2520.12	? (MO) TT=2251.68	? (TO) TT=2520.14
PseudoBoolean-dec/ Pb-chnl50-51_c18.xml		? (TO) TT=2519.91	? (TO) TT=2519.95	? (TO) TT=2519.92	? (TO) TT=2519.98	? (TO) TT=2400.06	? (TO) TT=2520.15	? (TO) TT=2520.13	? (TO) TT=2520.05	? (TO) TT=2520.12	? (TO) TT=2520.12	? (TO) TT=2520.08	? (TO) TT=2520.04	? (TO) TT=2519.91	? (TO) TT=2400.07	? (TO) TT=2520.08	? (TO) TT=2400.02	? (TO) TT=2520.1	? (TO) TT=2400.08	? (TO) TT=2520.09	? (TO) TT=2520.12	No data	No data	? (TO) TT=2519.74	? (TO) TT=2520.03	? (TO) TT=2520.14	? (TO) TT=2400.18	? (TO) TT=2520.08	? (TO) TT=2520.13	? (MO) TT=322.44	? (TO) TT=2520.15
PseudoBoolean-dec/ Pb-fpga25-25-sat_c18.xml	SAT TT=0.216	? (TO) TT=2519.78	? (TO) TT=2520.08	SAT TT=0.219	SAT TT=0.219	SAT TT=89.601	SAT TT=89.162	SAT TT=45.854	? (TO) TT=2520.07	? (TO) TT=2520.06	SAT TT=15.816	SAT TT=22.355	SAT TT=0.215	SAT TT=0.216	SAT TT=1.236	SAT TT=0.347	? (TO) TT=2400.03	? (TO) TT=2520.03	? (TO) TT=2400.07	? (TO) TT=2520.06	? (TO) TT=2520.03	No data	No data	? (TO) TT=2520.06	? (TO) TT=2520.12	? (TO) TT=2519.88	SAT TT=8.431	SAT TT=7.953	SAT TT=9.486	SAT TT=8.812	SAT TT=10.445
PseudoBoolean-dec/ Pb-fpga45-45-sat_c18.xml	SAT TT=3.359	? (TO) TT=2519.92	? (TO) TT=2519.85	SAT TT=3.596	SAT TT=3.359	? (TO) TT=2400.04	? (TO) TT=2520.4	? (TO) TT=2520.06	? (TO) TT=2520.17	? (TO) TT=2520.53	? (TO) TT=2520.11	SAT TT=1534.68	SAT TT=4.592	SAT TT=3.799	? (TO) TT=2400.02	? (TO) TT=2519.77	? (TO) TT=2400.04	? (TO) TT=2520.06	? (TO) TT=2400.04	? (TO) TT=2520.07	? (TO) TT=2520.09	No data	No data	? (TO) TT=2520.08	? (TO) TT=2519.58	? (TO) TT=2519.9	SAT TT=351.217	SAT TT=12.962	SAT TT=15.503	SAT TT=24.543	SAT TT=26.631
PseudoBoolean-dec/ Pb-robin08_c18.xml	SAT TT=0.096	SAT TT=2.334	SAT TT=2.315	SAT TT=0.349	SAT TT=0.347	SAT TT=2.216	SAT TT=2.291	SAT TT=48.2	SAT TT=70.146	SAT TT=39.644	SAT TT=44.086	SAT TT=0.439	SAT TT=0.74	SAT TT=0.736	SAT TT=0.208	SAT TT=0.096	SAT TT=2.858	SAT TT=2.872	SAT TT=2.873	SAT TT=2.951	SAT TT=2.877	No data	No data	SAT TT=11.196	SAT TT=10.976	SAT TT=11.105	SAT TT=71.263	SAT TT=7.409	SAT TT=9.357	SAT TT=10.321	SAT TT=11.74
PseudoBoolean-dec/ Pb-robin10_c18.xml	SAT TT=1.095	SAT TT=23.255	SAT TT=23.143	SAT TT=97.707	SAT TT=98.738	SAT TT=11.217	SAT TT=11.714	SAT TT=366.737	? (TO) TT=2520.15	SAT TT=706.625	SAT TT=1191.96	SAT TT=42.899	SAT TT=54.18	SAT TT=48.785	SAT TT=1.095	SAT TT=2.134	SAT TT=12.311	SAT TT=12.033	SAT TT=5.525	SAT TT=5.687	SAT TT=5.664	No data	No data	SAT TT=4.77	SAT TT=4.743	SAT TT=4.774	? (TO) TT=2400.15	SAT TT=11.878	SAT TT=14.873	SAT TT=17.682	SAT TT=18.39
PseudoBoolean-dec/ Pb-robin12_c18.xml	SAT TT=1.846	? (TO) TT=2519.91	? (TO) TT=2519.86	? (TO) TT=2519.96	? (TO) TT=2520.07	SAT TT=18.588	SAT TT=18.673	? (TO) TT=2520.28	? (TO) TT=2520.31	? (TO) TT=2520.17	? (TO) TT=2520.12	SAT TT=1246.44	? (TO) TT=2520.03	? (TO) TT=2519.78	SAT TT=134.758	SAT TT=1.846	SAT TT=52.311	SAT TT=54.735	SAT TT=177.961	? (TO) TT=2520.03	SAT TT=670.303	No data	No data	SAT TT=11.602	SAT TT=11.565	SAT TT=11.651	? (TO) TT=2400.12	SAT TT=27.873	SAT TT=37.556	SAT TT=42.641	SAT TT=30.935
PseudoBoolean-dec/ Pb-robin14_c18.xml	SAT TT=87.425	? (TO) TT=2519.9	? (TO) TT=2520.07	? (TO) TT=2520.06	? (TO) TT=2520.08	? (TO) TT=2400.09	? (TO) TT=2520.12	? (TO) TT=2522.64	? (TO) TT=2522.22	? (TO) TT=2522.1	? (TO) TT=2522.15	? (TO) TT=2519.92	? (TO) TT=2520.11	? (TO) TT=2519.97	SAT TT=116.823	SAT TT=328.087	? (TO) TT=2400.05	? (TO) TT=2520.04	? (TO) TT=2400.05	? (TO) TT=2520.03	? (TO) TT=2520.04	No data	No data	? (TO) TT=2520.1	? (TO) TT=2519.92	? (TO) TT=2520.09	? (TO) TT=2400.1	? (MO) TT=34.504	SAT TT=87.425	? (MO) TT=40.193	SAT TT=264.364
PseudoBoolean-dec/ Pb-robin16_c18.xml		? (TO) TT=2519.89	? (TO) TT=2519.81	? (TO) TT=2520.05	? (TO) TT=2519.98	? (TO) TT=2400.1	? (TO) TT=2520.07	? (TO) TT=2522.07	? (TO) TT=2522.09	? (TO) TT=2528.12	? (TO) TT=2528.16	? (TO) TT=2519.96	? (TO) TT=2519.99	? (TO) TT=2519.97	? (TO) TT=2400.04	? (TO) TT=2520.06	? (TO) TT=2400.1	? (TO) TT=2520.04	? (TO) TT=2400.03	? (TO) TT=2520.08	? (TO) TT=2520.11	No data	No data	? (TO) TT=2519.94	? (TO) TT=2519.98	? (TO) TT=2519.96	? (TO) TT=2400.18	? (MO) TT=35.199	? (TO) TT=2520.11	? (MO) TT=98.656	? (TO) TT=2520.28
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
PseudoBoolean-dec/ Pb-robin18_c18.xml		? (TO) TT=2519.8	? (TO) TT=2519.95	? (TO) TT=2519.93	? (TO) TT=2519.96	? (TO) TT=2400.05	? (TO) TT=2520.07	? (TO) TT=2522.08	? (TO) TT=2522.14	? (TO) TT=2527.87	? (TO) TT=2527.44	? (TO) TT=2520.05	? (TO) TT=2519.88	? (TO) TT=2519.77	? (TO) TT=2400.02	? (TO) TT=2519.88	? (TO) TT=2400.04	? (TO) TT=2520.12	? (TO) TT=2400.08	? (TO) TT=2520.11	? (TO) TT=2520.04	No data	No data	? (TO) TT=2519.99	? (TO) TT=2519.76	? (TO) TT=2519.88	? TT=353.132	? (MO) TT=53.138	? (TO) TT=2520.17	? (MO) TT=82.552	? (TO) TT=2520.22
PseudoBoolean-opt/ Pb-circ10-3_c18.xml	SAT f=360 TT=2520.06 T1=1165.63	No data	No data	No data	No data	SAT (TO) f=360 TT=2400.07 T1=1139.04	SAT (TO) f=360 TT=2520.06 T1=1165.63	? (TO) TT=2520.18	? (TO) TT=2520.26	? (TO) TT=2522.08	? (TO) TT=2522.06	SAT (TO) f=364 TT=2519.97 T1=918.6	No data	No data	SAT (TO) f=332 TT=2400.08 T1=2167.52	SAT (TO) f=332 TT=2519.9 T1=2189.5	No data	No data	? (TO) TT=2400.08	? (TO) TT=2520.11	? (TO) TT=2520.12	? (TO) TT=2400.07	? (TO) TT=2520.06	? (TO) TT=2519.8	? (TO) TT=2520.11	? (TO) TT=2519.76	SAT (TO) f=378 TT=2400.2 T1=1441.23	No data	No data	No data	No data
PseudoBoolean-opt/ Pb-circ4-3_c18.xml	OPT f=20 TT=0.837 T1=0.07	No data	No data	No data	No data	OPT f=20 TT=5.771 T1=0.65	OPT f=20 TT=5.822 T1=0.65	OPT f=20 TT=21.225 T1=6.27	OPT f=20 TT=19.933 T1=6.81	OPT f=20 TT=19.466 T1=5.71	OPT f=20 TT=16.605 T1=5.27	OPT f=20 TT=12.941 T1=0.07	No data	No data	OPT f=20 TT=3.234 T1=0.11	OPT f=20 TT=3.241 T1=0.11	No data	No data	SAT (TO) f=20 TT=2400.11 T1=1.76	SAT (TO) f=20 TT=2520.07 T1=1.69	OPT f=20 TT=4.513 T1=1.73	OPT f=20 TT=4.416 T1=1.78	OPT f=20 TT=4.268 T1=1.74	OPT f=20 TT=0.784 T1=0.78	OPT f=20 TT=0.843 T1=0.84	OPT f=20 TT=0.837 T1=0.83	OPT f=20 TT=4.483 T1=1.11	No data	No data	No data	No data
PseudoBoolean-opt/ Pb-circ6-3_c18.xml	SAT f=66 TT=2400.09 T1=817.39	No data	No data	No data	No data	SAT (TO) f=66 TT=2400.03 T1=1651.13	SAT (TO) f=66 TT=2520.06 T1=1667.05	SAT (TO) f=70 TT=2520.19 T1=395.73	SAT (TO) f=68 TT=2520.15 T1=1731.09	SAT (TO) f=70 TT=2520.09 T1=454.72	SAT (TO) f=72 TT=2520.12 T1=1591.44	SAT (TO) f=66 TT=2519.88 T1=817.39	No data	No data	SAT (TO) f=64 TT=2400.07 T1=2242.11	SAT (TO) f=64 TT=2519.96 T1=2253.27	No data	No data	SAT (TO) f=72 TT=2400.12 T1=422.93	SAT (TO) f=76 TT=2520.04 T1=470.91	SAT (TO) f=70 TT=2520.06 T1=76.74	SAT (TO) f=72 TT=2400.09 T1=266.27	SAT (TO) f=72 TT=2520.08 T1=261.91	? (TO) TT=2520.1	? (TO) TT=2520.06	? (TO) TT=2519.94	SAT (TO) f=66 TT=2400.09 T1=1986.93	No data	No data	No data	No data
PseudoBoolean-opt/ Pb-circ8-3_c18.xml	SAT f=176 TT=2520.05 T1=967.17	No data	No data	No data	No data	SAT (TO) f=178 TT=2400.1 T1=35.94	SAT (TO) f=178 TT=2520.09 T1=36.01	SAT (TO) f=184 TT=2520.2 T1=959.6	SAT (TO) f=180 TT=2520.1 T1=495.93	SAT (TO) f=176 TT=2520.17 T1=2041.1	SAT (TO) f=182 TT=2520.18 T1=1205.47	SAT (TO) f=176 TT=2520.05 T1=967.17	No data	No data	SAT (TO) f=168 TT=2400.09 T1=1418.97	SAT (TO) f=168 TT=2520.05 T1=1433.92	No data	No data	? (TO) TT=2400.12	? (TO) TT=2520.04	? (TO) TT=2520.04	? (TO) TT=2400.1	? (TO) TT=2520.04	? (TO) TT=2519.74	? (TO) TT=2520.05	? (TO) TT=2520.07	SAT (TO) f=178 TT=2400.4 T1=1212.47	No data	No data	No data	No data
PseudoBoolean-opt/ Pb-garden-100x100_c18.xml	SAT f=2675 TT=2519.94 T1=21.46	No data	No data	No data	No data	SAT (TO) f=2864 TT=2400.05 T1=11.7	SAT (TO) f=2864 TT=2520.09 T1=11.59	? (TO) TT=2528.49	? (TO) TT=2528.41	? (TO) TT=2527.97	? (TO) TT=2528.15	SAT (TO) f=2675 TT=2519.94 T1=21.46	No data	No data	SAT (TO) f=2747 TT=2400.09 T1=2388.63	SAT (TO) f=2747 TT=2520.07 T1=2402.7	No data	No data	SAT (TO) f=5265 TT=2400.08 T1=978.03	SAT (TO) f=5263 TT=2520.15 T1=2153.62	SAT (TO) f=5269 TT=2520.1 T1=810.44	SAT (TO) f=3444 TT=2400.14 T1=1997.79	? (TO) TT=2528.29	? (TO) TT=2520.11	? (TO) TT=2519.61	? (TO) TT=2520.1	SAT (TO) f=3473 TT=2400.78 T1=1235.12	No data	No data	No data	No data
PseudoBoolean-opt/ Pb-garden-15x15_c18.xml	OPT f=53 TT=1136.33 T1=1136.37	No data	No data	No data	No data	SAT (TO) f=63 TT=2400.09 T1=220.47	SAT (TO) f=63 TT=2520.15 T1=216.15	SAT (TO) f=65 TT=2520.09 T1=1222.73	SAT (TO) f=63 TT=2520.16 T1=208.94	SAT (TO) f=62 TT=2520.08 T1=532.03	SAT (TO) f=62 TT=2520.13 T1=796.51	SAT (TO) f=66 TT=2520.09 T1=0.02	No data	No data	SAT (TO) f=62 TT=2400.09 T1=20.64	SAT (TO) f=62 TT=2520.07 T1=20.85	No data	No data	SAT (TO) f=88 TT=2400.07 T1=161.22	SAT (TO) f=85 TT=2520.07 T1=2088.35	SAT (TO) f=87 TT=2520.08 T1=795.6	SAT (TO) f=58 TT=2400.11 T1=388.97	SAT (TO) f=58 TT=2520.08 T1=106.14	OPT f=53 TT=1096.28 T1=1096.25	OPT f=53 TT=1183.23 T1=1183.34	OPT f=53 TT=1136.33 T1=1136.37	SAT (TO) f=76 TT=2400.38 T1=1392.73	No data	No data	No data	No data
PseudoBoolean-opt/ Pb-garden-4x4_c18.xml	OPT f=4 TT=0.006 T1=0	No data	No data	No data	No data	OPT f=4 TT=0.864 T1=0.45	OPT f=4 TT=0.877 T1=0.45	OPT f=4 TT=7.235 T1=3.56	OPT f=4 TT=7.282 T1=3.53	OPT f=4 TT=6.87 T1=3.43	OPT f=4 TT=6.804 T1=3.43	OPT f=4 TT=0.006 T1=0	No data	No data	OPT f=4 TT=0.006 T1=0	OPT f=4 TT=0.006 T1=0	No data	No data	SAT (TO) f=4 TT=2400.08 T1=1.04	SAT (TO) f=4 TT=2520.02 T1=1.01	OPT f=4 TT=1.507 T1=1.01	OPT f=4 TT=1.625 T1=1.06	OPT f=4 TT=1.563 T1=1.02	OPT f=4 TT=0.021 T1=0.02	OPT f=4 TT=0.026 T1=0.02	OPT f=4 TT=0.026 T1=0.02	OPT f=4 TT=0.623 T1=0.35	No data	No data	No data	No data
PseudoBoolean-opt/ Pb-garden-7x7_c18.xml	OPT f=12 TT=0.183 T1=0.18	No data	No data	No data	No data	OPT f=12 TT=48.242 T1=12.63	OPT f=12 TT=45.642 T1=12.21	OPT f=12 TT=205.724 T1=31.99	OPT f=12 TT=39.314 T1=22.3	OPT f=12 TT=32.139 T1=20.29	OPT f=12 TT=13.547 T1=4.97	OPT f=12 TT=285.688 T1=237.17	No data	No data	OPT f=12 TT=43.487 T1=0.13	OPT f=12 TT=43.463 T1=0.13	No data	No data	SAT (TO) f=12 TT=2400.06 T1=25.51	SAT (TO) f=12 TT=2520.07 T1=13.43	OPT f=12 TT=88.352 T1=35.3	SAT (TO) f=13 TT=2400.09 T1=4.48	SAT (TO) f=12 TT=2520.03 T1=240.29	OPT f=12 TT=0.158 T1=0.16	OPT f=12 TT=0.183 T1=0.18	OPT f=12 TT=0.183 T1=0.18	OPT f=12 TT=41.396 T1=23.48	No data	No data	No data	No data
PseudoBoolean-opt/ Pb-garden-9x9_c18.xml	OPT f=20 TT=0.917 T1=0.91	No data	No data	No data	No data	SAT (TO) f=21 TT=2400.08 T1=1363.54	SAT (TO) f=21 TT=2520.08 T1=1298.69	SAT (TO) f=21 TT=2520.06 T1=52	SAT (TO) f=21 TT=2520.13 T1=23.53	SAT (TO) f=20 TT=2520.11 T1=308.95	SAT (TO) f=20 TT=2520.04 T1=479.97	SAT (TO) f=22 TT=2520.07 T1=0.01	No data	No data	SAT (TO) f=21 TT=2400.07 T1=0.11	SAT (TO) f=21 TT=2519.92 T1=0.11	No data	No data	SAT (TO) f=22 TT=2400.05 T1=607.96	SAT (TO) f=22 TT=2520.11 T1=961.55	SAT (TO) f=22 TT=2520.04 T1=1397.78	SAT (TO) f=21 TT=2400.07 T1=241.06	SAT (TO) f=21 TT=2520.1 T1=254.54	OPT f=20 TT=0.91 T1=0.91	OPT f=20 TT=0.923 T1=0.92	OPT f=20 TT=0.917 T1=0.91	SAT (TO) f=22 TT=2400.26 T1=2342.23	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
PseudoBoolean-opt/ Pb-radar-10-10-45-050-100_c18.xml	SAT f=4 TT=2520.09 T1=2.89	No data	No data	No data	No data	SAT (TO) f=4 TT=2400.06 T1=2.87	SAT (TO) f=4 TT=2520.09 T1=2.89	SAT (TO) f=11 TT=2520.09 T1=80.43	SAT (TO) f=11 TT=2520.12 T1=93.06	SAT (TO) f=13 TT=2520.12 T1=1177.49	SAT (TO) f=11 TT=2520.05 T1=216.3	SAT (TO) f=11 TT=2520.07 T1=1.44	No data	No data	SAT (TO) f=8 TT=2400.07 T1=780.31	SAT (TO) f=8 TT=2519.92 T1=787.75	No data	No data	SAT (TO) f=43 TT=2400.11 T1=126.91	SAT (TO) f=36 TT=2520.12 T1=221.81	SAT (TO) f=37 TT=2520.08 T1=471.33	SAT (TO) f=4 TT=2400.03 T1=223.03	SAT (TO) f=5 TT=2520.12 T1=1221.27	? (TO) TT=2519.81	? (TO) TT=2519.89	? (TO) TT=2519.84	SAT (TO) f=13 TT=2400.47 T1=0.6	No data	No data	No data	No data
PseudoBoolean-opt/ Pb-radar-10-20-45-050-100_c18.xml	SAT f=4 TT=2520.06 T1=1708.02	No data	No data	No data	No data	SAT (TO) f=11 TT=2400.14 T1=805.29	SAT (TO) f=11 TT=2520.12 T1=772.62	SAT (TO) f=44 TT=2520.13 T1=24.26	SAT (TO) f=32 TT=2520.13 T1=8.72	SAT (TO) f=29 TT=2520.15 T1=1933.17	SAT (TO) f=37 TT=2520.1 T1=1579.77	SAT (TO) f=19 TT=2520.07 T1=2272.49	No data	No data	SAT (TO) f=18 TT=2400.07 T1=0.1	SAT (TO) f=18 TT=2519.91 T1=0.1	No data	No data	SAT (TO) f=94 TT=2400.39 T1=1950.91	SAT (TO) f=102 TT=2520.11 T1=1961.92	SAT (TO) f=99 TT=2520.04 T1=2508.95	SAT (TO) f=5 TT=2400.1 T1=94.03	SAT (TO) f=4 TT=2520.06 T1=1708.02	? (TO) TT=2519.61	? (TO) TT=2520.06	? (TO) TT=2520.05	SAT (TO) f=15 TT=2400.34 T1=1.01	No data	No data	No data	No data
PseudoBoolean-opt/ Pb-radar-30-30-45-050-100_c18.xml	SAT f=49 TT=2400.3 T1=1680.76	No data	No data	No data	No data	SAT (TO) f=97 TT=2400.22 T1=2381.97	SAT (TO) f=97 TT=2520.1 T1=2370.27	SAT (TO) f=188 TT=2520.15 T1=178.79	? (TO) TT=2528.58	? (TO) TT=2528.19	? (TO) TT=2528.13	SAT (TO) f=124 TT=2519.77 T1=1568.8	No data	No data	SAT (TO) f=169 TT=2400.09 T1=713.15	SAT (TO) f=169 TT=2519.91 T1=713.4	No data	No data	SAT (TO) f=1312 TT=2400.09 T1=431.47	SAT (TO) f=1326 TT=2520.13 T1=1447.58	SAT (TO) f=1293 TT=2520.11 T1=387.47	SAT (TO) f=63 TT=2400.12 T1=2243.89	SAT (TO) f=95 TT=2520.12 T1=2421.51	? (TO) TT=2520.02	? (TO) TT=2519.81	? (TO) TT=2519.89	SAT (TO) f=49 TT=2400.3 T1=1680.76	No data	No data	No data	No data
PseudoBoolean-opt/ Pb-radar-30-70-45-050-100_c18.xml	SAT f=158 TT=2400.15 T1=17.75	No data	No data	No data	No data	SAT (TO) f=422 TT=2400.05 T1=1019.76	SAT (TO) f=422 TT=2520.05 T1=1014.54	? (TO) TT=2527.98	? (TO) TT=2528.28	? (TO) TT=2528.37	? (TO) TT=2528.3	SAT (TO) f=491 TT=2519.98 T1=1524.14	No data	No data	SAT (TO) f=445 TT=2400.05 T1=56.18	SAT (TO) f=445 TT=2519.77 T1=57.61	No data	No data	SAT (TO) f=3013 TT=2400.08 T1=2339.55	SAT (TO) f=3095 TT=2520.11 T1=2450.5	SAT (TO) f=3008 TT=2520.1 T1=2464.92	? (TO) TT=2408.09	? (TO) TT=2528.15	? (TO) TT=2520.12	? (TO) TT=2519.96	? (TO) TT=2520.05	SAT (TO) f=158 TT=2400.15 T1=17.75	No data	No data	No data	No data
QuadraticAssignment/ QuadraticAssignment-esc128_c18.xml	SAT f=32 TT=2520.03 T1=143.77	No data	No data	No data	No data	SAT (TO) f=32 TT=2400.06 T1=144.12	SAT (TO) f=32 TT=2520.03 T1=143.77	SAT (TO) f=59 TT=2520.06 T1=483.23	SAT (TO) f=38 TT=2520.06 T1=177.88	SAT (TO) f=37 TT=2520.07 T1=2002.56	SAT (TO) f=37 TT=2520.09 T1=1311.9	SAT (TO) f=42 TT=2520.02 T1=1064.77	No data	No data	SAT (TO) f=32 TT=2400.08 T1=111.37	SAT (TO) f=32 TT=2520.03 T1=110.42	No data	No data	SAT (TO) f=33 TT=2400.09 T1=71.95	SAT (TO) f=33 TT=2520.08 T1=439.19	SAT (TO) f=33 TT=2520.13 T1=1214.54	SAT (TO) f=32 TT=2400.04 T1=30.12	SAT (TO) f=32 TT=2520.06 T1=294.73	? (TO) TT=2519.84	? (TO) TT=2520.09	? (TO) TT=2520.06	? TT=417.661	No data	No data	No data	No data
QuadraticAssignment/ QuadraticAssignment-esc16a_c18.xml	SAT f=34 TT=2400.22 T1=1.43	No data	No data	No data	No data	SAT (TO) f=34 TT=2400.11 T1=1.71	SAT (TO) f=34 TT=2520.09 T1=1.7	SAT (TO) f=34 TT=2520.09 T1=7.71	SAT (TO) f=34 TT=2520.12 T1=102.38	SAT (TO) f=34 TT=2520.08 T1=884.6	SAT (TO) f=34 TT=2520.06 T1=293.23	SAT (TO) f=34 TT=2519.99 T1=1.43	No data	No data	SAT (TO) f=34 TT=2400.08 T1=0.02	SAT (TO) f=34 TT=2519.8 T1=0.02	No data	No data	SAT (TO) f=34 TT=2400.09 T1=7.25	SAT (TO) f=34 TT=2520.11 T1=9.96	SAT (TO) f=34 TT=2520.12 T1=7.55	SAT (TO) f=35 TT=2400.12 T1=3.11	SAT (TO) f=34 TT=2520.12 T1=3.63	? (TO) TT=2519.99	? (TO) TT=2519.62	? (TO) TT=2520.05	SAT (TO) f=34 TT=2400.22 T1=1256.74	No data	No data	No data	No data
QuadraticAssignment/ QuadraticAssignment-esc16d_c18.xml	OPT f=8 TT=14.54 T1=0.03	No data	No data	No data	No data	OPT f=8 TT=55.393 T1=0.77	OPT f=8 TT=54.335 T1=0.76	OPT f=8 TT=94.198 T1=3.9	OPT f=8 TT=56.141 T1=4.48	OPT f=8 TT=77.216 T1=3.75	OPT f=8 TT=81.401 T1=3.88	OPT f=8 TT=138.957 T1=0.03	No data	No data	OPT f=8 TT=52.632 T1=0.01	OPT f=8 TT=51.849 T1=0.01	No data	No data	SAT (TO) f=8 TT=2400.03 T1=1.97	SAT (TO) f=8 TT=2520.11 T1=1.98	OPT f=8 TT=163.029 T1=2	SAT (TO) f=8 TT=2400.08 T1=2.49	SAT (TO) f=8 TT=2520.09 T1=2.44	OPT f=8 TT=13.445 T1=13.45	OPT f=8 TT=14.766 T1=14.77	OPT f=8 TT=14.54 T1=14.53	OPT f=8 TT=176.837 T1=12.26	No data	No data	No data	No data
QuadraticAssignment/ QuadraticAssignment-esc32a_c18.xml	SAT f=69 TT=2520.02 T1=85.85	No data	No data	No data	No data	SAT (TO) f=69 TT=2400.12 T1=87.2	SAT (TO) f=69 TT=2520.02 T1=85.85	SAT (TO) f=80 TT=2520.13 T1=2412.57	SAT (TO) f=85 TT=2520.13 T1=1401.8	SAT (TO) f=72 TT=2520.08 T1=599.22	SAT (TO) f=73 TT=2520.1 T1=659.18	SAT (TO) f=93 TT=2520.01 T1=2053.81	No data	No data	SAT (TO) f=66 TT=2400.06 T1=1459.03	SAT (TO) f=65 TT=2520.02 T1=2511.46	No data	No data	SAT (TO) f=86 TT=2400.08 T1=113.27	SAT (TO) f=87 TT=2520.12 T1=2077.06	SAT (TO) f=94 TT=2520.09 T1=595.57	SAT (TO) f=76 TT=2400.1 T1=67.48	SAT (TO) f=73 TT=2520.08 T1=780.56	? (TO) TT=2520.04	? (TO) TT=2519.99	? (TO) TT=2519.91	SAT (TO) f=180 TT=2400.3 T1=1028.69	No data	No data	No data	No data
QuadraticAssignment/ QuadraticAssignment-esc32d_c18.xml	SAT f=100 TT=2520.08 T1=12.7	No data	No data	No data	No data	SAT (TO) f=101 TT=2400.1 T1=938.32	SAT (TO) f=101 TT=2520.07 T1=948.63	SAT (TO) f=111 TT=2520.09 T1=1103.75	SAT (TO) f=106 TT=2520.03 T1=42.17	SAT (TO) f=116 TT=2520.11 T1=1179.66	SAT (TO) f=110 TT=2520.09 T1=1607.83	SAT (TO) f=110 TT=2520 T1=24.76	No data	No data	SAT (TO) f=101 TT=2400.02 T1=64.49	SAT (TO) f=101 TT=2519.98 T1=64.67	No data	No data	SAT (TO) f=108 TT=2400.08 T1=1729.46	SAT (TO) f=105 TT=2520.05 T1=533.22	SAT (TO) f=107 TT=2520.09 T1=1225.47	SAT (TO) f=100 TT=2400.03 T1=389.54	SAT (TO) f=100 TT=2520.08 T1=12.7	? (TO) TT=2519.8	? (TO) TT=2520.05	? (TO) TT=2519.93	SAT (TO) f=151 TT=2400.33 T1=59.69	No data	No data	No data	No data
QuadraticAssignment/ QuadraticAssignment-esc64a_c18.xml	SAT f=58 TT=2520.01 T1=0.68	No data	No data	No data	No data	SAT (TO) f=58 TT=2400.31 T1=19.14	SAT (TO) f=58 TT=2520.05 T1=19.62	SAT (TO) f=59 TT=2520.11 T1=231.61	SAT (TO) f=58 TT=2520.06 T1=2215.24	SAT (TO) f=58 TT=2520.13 T1=216.03	SAT (TO) f=58 TT=2520.08 T1=13.13	SAT (TO) f=58 TT=2520.01 T1=0.68	No data	No data	SAT (TO) f=58 TT=2400.08 T1=0.64	SAT (TO) f=58 TT=2520.03 T1=0.65	No data	No data	SAT (TO) f=58 TT=2400.07 T1=9.05	SAT (TO) f=58 TT=2520.12 T1=7.11	SAT (TO) f=58 TT=2520.06 T1=13.36	SAT (TO) f=58 TT=2400.04 T1=10.29	SAT (TO) f=58 TT=2520.07 T1=7.68	? (TO) TT=2520.02	? (TO) TT=2520.07	? (TO) TT=2519.94	SAT (TO) f=107 TT=2400.35 T1=824.63	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
QuadraticAssignment/ QuadraticAssignment-had12_c18.xml	SAT f=826 TT=2520.05 T1=143.46	No data	No data	No data	No data	SAT (TO) f=835 TT=2400.08 T1=2281.83	SAT (TO) f=835 TT=2520.16 T1=2277.36	SAT (TO) f=830 TT=2520.15 T1=325.06	SAT (TO) f=826 TT=2520.12 T1=143.46	SAT (TO) f=831 TT=2520.08 T1=1760.41	SAT (TO) f=827 TT=2520.05 T1=10.43	SAT (TO) f=832 TT=2519.95 T1=1746.64	No data	No data	SAT (TO) f=826 TT=2400.08 T1=3.85	SAT (TO) f=826 TT=2520.08 T1=3.8	No data	No data	SAT (TO) f=826 TT=2400.08 T1=1991.88	SAT (TO) f=826 TT=2520.05 T1=317.76	SAT (TO) f=826 TT=2520.09 T1=654.13	SAT (TO) f=826 TT=2400.09 T1=10.05	SAT (TO) f=826 TT=2520.05 T1=599.74	? (TO) TT=2519.95	? (TO) TT=2519.92	? (TO) TT=2520.01	SAT (TO) f=855 TT=2400.1 T1=2282.14	No data	No data	No data	No data
QuadraticAssignment/ QuadraticAssignment-had14_c18.xml	SAT f=1372 TT=2520.1 T1=7.7	No data	No data	No data	No data	SAT (TO) f=1390 TT=2400.05 T1=1672.41	SAT (TO) f=1390 TT=2520.11 T1=1663.11	SAT (TO) f=1379 TT=2520.13 T1=174.14	SAT (TO) f=1412 TT=2520.12 T1=325.95	SAT (TO) f=1370 TT=2520.11 T1=1390.14	SAT (TO) f=1378 TT=2520.15 T1=38.26	SAT (TO) f=1385 TT=2520.1 T1=205.04	No data	No data	SAT (TO) f=1362 TT=2400.05 T1=21.98	SAT (TO) f=1362 TT=2519.96 T1=22.02	No data	No data	SAT (TO) f=1378 TT=2400.07 T1=1001.35	SAT (TO) f=1372 TT=2520.12 T1=790.9	SAT (TO) f=1387 TT=2520.04 T1=1045.08	SAT (TO) f=1372 TT=2400.1 T1=3.55	SAT (TO) f=1372 TT=2520.1 T1=7.7	? (TO) TT=2519.95	? (TO) TT=2519.92	? (TO) TT=2520.06	SAT (TO) f=1430 TT=2400.16 T1=1170.04	No data	No data	No data	No data
QuadraticAssignment/ QuadraticAssignment-had16_c18.xml	SAT f=1860 TT=2520.08 T1=478.92	No data	No data	No data	No data	SAT (TO) f=1928 TT=2400.05 T1=1703.8	SAT (TO) f=1928 TT=2520.04 T1=1635.99	SAT (TO) f=1871 TT=2520.06 T1=373.99	SAT (TO) f=1894 TT=2520.08 T1=961.91	SAT (TO) f=1870 TT=2520.04 T1=67.14	SAT (TO) f=1871 TT=2520.06 T1=1003.7	SAT (TO) f=1934 TT=2520.01 T1=1918.51	No data	No data	SAT (TO) f=1861 TT=2400.06 T1=86.5	SAT (TO) f=1861 TT=2519.96 T1=87.13	No data	No data	SAT (TO) f=1906 TT=2400.06 T1=680.9	SAT (TO) f=1898 TT=2520.04 T1=914.45	SAT (TO) f=1895 TT=2520.1 T1=16.36	SAT (TO) f=1885 TT=2400.08 T1=920.27	SAT (TO) f=1860 TT=2520.08 T1=478.92	? (TO) TT=2519.85	? (TO) TT=2520.11	? (TO) TT=2519.99	SAT (TO) f=1994 TT=2400.15 T1=2181.62	No data	No data	No data	No data
QuadraticAssignment/ QuadraticAssignment-had18_c18.xml	SAT f=2679 TT=2520.04 T1=195.2	No data	No data	No data	No data	SAT (TO) f=2760 TT=2400.03 T1=396.38	SAT (TO) f=2760 TT=2520.12 T1=403.71	SAT (TO) f=2721 TT=2520.04 T1=34.19	SAT (TO) f=2751 TT=2520.08 T1=23.09	SAT (TO) f=2680 TT=2520.13 T1=1121.99	SAT (TO) f=2685 TT=2520.14 T1=1197.31	SAT (TO) f=2775 TT=2519.92 T1=1882.63	No data	No data	SAT (TO) f=2680 TT=2400.03 T1=435.83	SAT (TO) f=2680 TT=2520.09 T1=432.93	No data	No data	SAT (TO) f=2713 TT=2400.04 T1=2249.36	SAT (TO) f=2762 TT=2520.1 T1=727.59	SAT (TO) f=2759 TT=2520.11 T1=1334.07	SAT (TO) f=2679 TT=2400.12 T1=1869.14	SAT (TO) f=2679 TT=2520.04 T1=195.2	? (TO) TT=2520	? (TO) TT=2519.91	? (TO) TT=2519.72	SAT (TO) f=2862 TT=2400.23 T1=662.74	No data	No data	No data	No data
QuadraticAssignment/ QuadraticAssignment-had20_c18.xml	SAT f=3467 TT=2520.08 T1=459.98	No data	No data	No data	No data	SAT (TO) f=3594 TT=2400.11 T1=560.92	SAT (TO) f=3594 TT=2520.06 T1=564.6	SAT (TO) f=3545 TT=2520.04 T1=1238.63	SAT (TO) f=3512 TT=2520.12 T1=936.28	SAT (TO) f=3492 TT=2520.13 T1=1664.88	SAT (TO) f=3500 TT=2520.11 T1=1185.49	SAT (TO) f=3668 TT=2519.94 T1=1719.61	No data	No data	SAT (TO) f=3512 TT=2400.05 T1=2210.51	SAT (TO) f=3512 TT=2519.93 T1=2191.49	No data	No data	SAT (TO) f=3634 TT=2400.12 T1=21.42	SAT (TO) f=3600 TT=2520.04 T1=261.03	SAT (TO) f=3594 TT=2520.07 T1=1128.7	SAT (TO) f=3525 TT=2400.08 T1=1158.42	SAT (TO) f=3467 TT=2520.08 T1=459.98	? (TO) TT=2520.01	? (TO) TT=2520.11	? (TO) TT=2520.1	SAT (TO) f=3716 TT=2400.15 T1=2230.33	No data	No data	No data	No data
QuadraticAssignment/ QuadraticAssignment-qap_c18.xml	OPT f=4776 TT=0.377 T1=0.35	No data	No data	No data	No data	OPT f=4776 TT=2.383 T1=0.72	OPT f=4776 TT=2.346 T1=0.71	OPT f=4776 TT=9.868 T1=4.31	OPT f=4776 TT=10.07 T1=4.28	OPT f=4776 TT=8.976 T1=3.85	OPT f=4776 TT=9.548 T1=3.96	OPT f=4776 TT=0.377 T1=0.35	No data	No data	OPT f=4776 TT=0.069 T1=0.03	OPT f=4776 TT=0.068 T1=0.03	No data	No data	SAT (TO) f=4776 TT=2400.1 T1=1.48	SAT (TO) f=4776 TT=2520.12 T1=1.43	OPT f=4776 TT=3.039 T1=1.46	OPT f=4776 TT=3.376 T1=1.45	OPT f=4776 TT=3.382 T1=1.42	OPT f=4776 TT=26.584 T1=26.58	OPT f=4776 TT=24.902 T1=24.9	OPT f=4776 TT=24.974 T1=24.98	OPT f=4776 TT=714.874 T1=614.93	No data	No data	No data	No data
QuadraticAssignment/ QuadraticAssignment-sko42_c18.xml	SAT f=8186 TT=2520.13 T1=913.42	No data	No data	No data	No data	SAT (TO) f=9220 TT=2400.11 T1=171.55	SAT (TO) f=9220 TT=2520.04 T1=174.18	SAT (TO) f=9200 TT=2520.12 T1=34.33	SAT (TO) f=9156 TT=2520.12 T1=36.2	SAT (TO) f=9183 TT=2520.06 T1=1626.84	SAT (TO) f=9173 TT=2520.13 T1=2270.48	SAT (TO) f=9512 TT=2519.86 T1=2175.21	No data	No data	SAT (TO) f=8596 TT=2400.02 T1=2297.9	SAT (TO) f=8582 TT=2519.92 T1=2513.83	No data	No data	SAT (TO) f=9298 TT=2400.14 T1=1777.55	SAT (TO) f=9349 TT=2520.1 T1=801.43	SAT (TO) f=9289 TT=2520.12 T1=153.52	SAT (TO) f=8120 TT=2400.07 T1=1793.04	SAT (TO) f=8186 TT=2520.13 T1=913.42	? (TO) TT=2519.86	? (TO) TT=2520.06	? (TO) TT=2519.95	? TT=2131.72	No data	No data	No data	No data
QuadraticAssignment/ QuadraticAssignment-sko49_c18.xml	SAT f=12137 TT=2520.09 T1=1750.82	No data	No data	No data	No data	SAT (TO) f=13022 TT=2400.06 T1=588.02	SAT (TO) f=13022 TT=2520.06 T1=580.52	SAT (TO) f=13198 TT=2520.09 T1=263.78	SAT (TO) f=13633 TT=2520.1 T1=1011.35	SAT (TO) f=13494 TT=2520.07 T1=122.96	SAT (TO) f=13709 TT=2520.15 T1=2443.73	SAT (TO) f=13783 TT=2519.96 T1=763.48	No data	No data	SAT (TO) f=12766 TT=2400.03 T1=2091.82	SAT (TO) f=12766 TT=2519.85 T1=2115.08	No data	No data	SAT (TO) f=13676 TT=2400.06 T1=1888.98	SAT (TO) f=13478 TT=2520.13 T1=590.36	SAT (TO) f=13675 TT=2520.1 T1=82.01	SAT (TO) f=11955 TT=2400.08 T1=2125.29	SAT (TO) f=12137 TT=2520.09 T1=1750.82	? (TO) TT=2519.84	? (TO) TT=2519.92	? (TO) TT=2519.87	? TT=1929.37	No data	No data	No data	No data
QuadraticAssignment/ QuadraticAssignment-sko56_c18.xml	SAT f=17822 TT=2520.11 T1=2312.72	No data	No data	No data	No data	SAT (TO) f=19034 TT=2400.04 T1=1724.2	SAT (TO) f=19034 TT=2520.04 T1=1725.18	SAT (TO) f=20038 TT=2520.13 T1=132.93	SAT (TO) f=19505 TT=2520.1 T1=118.49	SAT (TO) f=19645 TT=2520.22 T1=121.08	SAT (TO) f=20262 TT=2520.13 T1=295.52	SAT (TO) f=20529 TT=2520.06 T1=2257.57	No data	No data	SAT (TO) f=18633 TT=2400.1 T1=2221.04	SAT (TO) f=18633 TT=2519.96 T1=2316.45	No data	No data	SAT (TO) f=20405 TT=2400.06 T1=522.28	SAT (TO) f=20340 TT=2520.05 T1=478.12	SAT (TO) f=20337 TT=2520.12 T1=1228.03	SAT (TO) f=18159 TT=2400.06 T1=1994.53	SAT (TO) f=17822 TT=2520.11 T1=2312.72	? (TO) TT=2519.99	? (TO) TT=2520.08	? (TO) TT=2519.99	? TT=1831.04	No data	No data	No data	No data
QuadraticAssignment/ QuadraticAssignment-sko64_c18.xml	SAT f=25546 TT=2520.1 T1=2361.94	No data	No data	No data	No data	SAT (TO) f=26354 TT=2400.05 T1=1754.45	SAT (TO) f=26354 TT=2520.03 T1=1735.93	SAT (TO) f=27593 TT=2520.13 T1=241.81	SAT (TO) f=27388 TT=2520.1 T1=271.73	SAT (TO) f=27146 TT=2520.19 T1=280.37	SAT (TO) f=27649 TT=2520.14 T1=281.36	SAT (TO) f=28923 TT=2519.98 T1=277.88	No data	No data	SAT (TO) f=26460 TT=2400.08 T1=2126.9	SAT (TO) f=26460 TT=2519.97 T1=2233.03	No data	No data	SAT (TO) f=28114 TT=2400.07 T1=1745.32	SAT (TO) f=27937 TT=2520.09 T1=37.2	SAT (TO) f=27994 TT=2520.08 T1=110.11	SAT (TO) f=26020 TT=2400.14 T1=2325.33	SAT (TO) f=25546 TT=2520.1 T1=2361.94	? (TO) TT=2519.9	? (TO) TT=2519.97	? (TO) TT=2519.96	? TT=1919.24	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
QuadraticAssignment/ QuadraticAssignment-sko72_c18.xml	SAT f=35807 TT=2520.07 T1=2171.51	No data	No data	No data	No data	SAT (TO) f=36779 TT=2400.04 T1=1570.77	SAT (TO) f=36707 TT=2520.11 T1=2435.92	SAT (TO) f=37704 TT=2520.33 T1=457.81	SAT (TO) f=37362 TT=2520.22 T1=525.66	SAT (TO) f=37856 TT=2520.29 T1=508.64	SAT (TO) f=37447 TT=2520.06 T1=524.21	SAT (TO) f=38953 TT=2520.05 T1=391.32	No data	No data	SAT (TO) f=36277 TT=2400.05 T1=2333.98	SAT (TO) f=36277 TT=2519.62 T1=2481.53	No data	No data	SAT (TO) f=38399 TT=2400.11 T1=937.67	SAT (TO) f=38672 TT=2520.1 T1=95.2	SAT (TO) f=38289 TT=2520.12 T1=1188.77	SAT (TO) f=35373 TT=2400.13 T1=2213.61	SAT (TO) f=35807 TT=2520.07 T1=2171.51	? (TO) TT=2519.93	? (TO) TT=2520.14	? (TO) TT=2519.86	? TT=1660.86	No data	No data	No data	No data
QuadraticAssignment/ QuadraticAssignment-sko81_c18.xml	SAT f=49154 TT=2520.11 T1=2319.6	No data	No data	No data	No data	SAT (TO) f=50058 TT=2400.11 T1=1479.89	SAT (TO) f=50058 TT=2520.09 T1=1504.6	SAT (TO) f=50462 TT=2520.28 T1=964.96	SAT (TO) f=51101 TT=2520.54 T1=971.6	SAT (TO) f=51134 TT=2520.87 T1=1044.83	SAT (TO) f=50604 TT=2520.58 T1=1047.69	SAT (TO) f=53487 TT=2520.08 T1=1016.26	No data	No data	SAT (TO) f=50204 TT=2400.11 T1=2103.42	SAT (TO) f=50204 TT=2519.81 T1=2104.8	No data	No data	SAT (TO) f=52894 TT=2400.18 T1=96.33	SAT (TO) f=53052 TT=2520.1 T1=24.5	SAT (TO) f=53322 TT=2520.16 T1=115.21	SAT (TO) f=49152 TT=2400.15 T1=2344.29	SAT (TO) f=49154 TT=2520.11 T1=2319.6	? (TO) TT=2519.99	? (TO) TT=2520.03	? (TO) TT=2520.11	? TT=1686.95	No data	No data	No data	No data
QuadraticAssignment/ QuadraticAssignment-sko90_c18.xml	SAT f=62971 TT=2520.12 T1=2352.37	No data	No data	No data	No data	? (TO) TT=2400.08	? (TO) TT=2520.1	SAT (TO) f=64449 TT=2520.22 T1=2081.64	SAT (TO) f=64647 TT=2520.42 T1=2064.71	SAT (TO) f=64885 TT=2520.24 T1=1825.38	SAT (TO) f=64009 TT=2520.42 T1=1644.49	SAT (TO) f=67850 TT=2520.04 T1=1264.38	No data	No data	SAT (TO) f=63495 TT=2400.13 T1=2176.47	SAT (TO) f=63495 TT=2520.07 T1=2315.21	No data	No data	SAT (TO) f=66989 TT=2400.11 T1=1447.91	SAT (TO) f=67762 TT=2520.15 T1=37.67	SAT (TO) f=67528 TT=2520.32 T1=2402.06	SAT (TO) f=62835 TT=2400.12 T1=2060.66	SAT (TO) f=62971 TT=2520.12 T1=2352.37	? (TO) TT=2520.16	? (TO) TT=2519.67	? (TO) TT=2519.94	? TT=665.289	No data	No data	No data	No data
Quasigroups/ QuasiGroup-4-07_c18.xml	UNSAT TT=1.006	UNSAT TT=8.96	UNSAT TT=9.074	UNSAT TT=16.854	UNSAT TT=16.941	UNSAT TT=5.532	UNSAT TT=5.325	UNSAT TT=16.477	UNSAT TT=17.426	UNSAT TT=16.301	UNSAT TT=17.038	UNSAT TT=26.155	UNSAT TT=4.03	UNSAT TT=4.027	UNSAT TT=2.71	UNSAT TT=2.585	UNSAT TT=3.883	UNSAT TT=3.335	? (TO) TT=2400.07	? (TO) TT=2520.04	UNSAT TT=3.584	No data	No data	UNSAT TT=1.376	UNSAT TT=1.388	UNSAT TT=1.006	UNSAT TT=2267.45	UNSAT TT=6.953	UNSAT TT=7.712	UNSAT TT=7.075	UNSAT TT=7.743
Quasigroups/ QuasiGroup-4-08_c18.xml	UNSAT TT=30.456	UNSAT TT=1567.21	UNSAT TT=1572.09	UNSAT TT=498.695	UNSAT TT=494.722	UNSAT TT=290.237	UNSAT TT=288.943	UNSAT TT=605.789	UNSAT TT=848.037	UNSAT TT=643.279	UNSAT TT=786.38	? (TO) TT=2520.07	UNSAT TT=362.113	UNSAT TT=370.927	UNSAT TT=461.008	UNSAT TT=510.389	UNSAT TT=102.589	UNSAT TT=99.887	? (TO) TT=2400.03	? (TO) TT=2520.11	UNSAT TT=604.658	No data	No data	UNSAT TT=30.475	UNSAT TT=30.456	UNSAT TT=30.477	? (TO) TT=2400.13	UNSAT TT=55.745	UNSAT TT=93.678	UNSAT TT=55.5	UNSAT TT=95.874
Quasigroups/ QuasiGroup-4-09_c18.xml	SAT TT=0.138	SAT TT=1.052	SAT TT=1.152	SAT TT=0.738	SAT TT=0.76	SAT TT=1.609	SAT TT=1.667	SAT TT=13.706	SAT TT=13.292	SAT TT=11.996	SAT TT=14.231	SAT TT=16.932	SAT TT=0.202	SAT TT=0.202	SAT TT=0.33	SAT TT=0.138	SAT TT=13.57	SAT TT=12.491	SAT TT=3.609	SAT TT=3.29	SAT TT=3.436	No data	No data	SAT TT=3.207	SAT TT=3.193	SAT TT=3.07	? (TO) TT=2400.07	SAT TT=9.435	SAT TT=10.786	SAT TT=9.454	SAT TT=10.498
Quasigroups/ QuasiGroup-4-10_c18.xml		? TT=2520.05	? TT=2520.05	? (TO) TT=2519.96	? (TO) TT=2520.05	? (TO) TT=2400.05	? (TO) TT=2520.06	? (TO) TT=2520.12	? (TO) TT=2520.04	? (TO) TT=2520.09	? (TO) TT=2520.04	? (TO) TT=2519.91	? TT=2520.05	? (TO) TT=2520.03	? (TO) TT=2400.07	? (TO) TT=2519.99	? (TO) TT=2400.07	? (TO) TT=2520.11	? (TO) TT=2400.02	? (TO) TT=2520.06	? (TO) TT=2520.04	No data	No data	? (TO) TT=2519.93	? (TO) TT=2520.05	? (TO) TT=2519.88	? (TO) TT=2400.12	? (MO) TT=150.722	? (TO) TT=2520.1	? (TO) TT=2520.06	? (TO) TT=2520.05
Quasigroups/ QuasiGroup-4-12_c18.xml	SAT TT=394.459	? (TO) TT=2520.1	? (TO) TT=2519.95	? (TO) TT=2520.04	? (TO) TT=2520.07	? (TO) TT=2400.12	? (TO) TT=2520.1	SAT TT=1602.11	? (TO) TT=2520.13	? (TO) TT=2520.14	? (TO) TT=2520.16	? (TO) TT=2520.07	SAT TT=790.071	SAT TT=791.473	? (TO) TT=2400.06	? (TO) TT=2519.74	? (TO) TT=2400.05	? (TO) TT=2520.09	? (TO) TT=2400.04	SAT TT=589.174	? (TO) TT=2520.02	No data	No data	SAT TT=2500.6	? (TO) TT=2520.05	? (TO) TT=2519.92	? (TO) TT=2400.14	SAT TT=397.511	SAT TT=1135.64	SAT TT=394.459	SAT TT=1133.94
Quasigroups/ QuasiGroup-4-14_c18.xml		? (TO) TT=2520.06	? (TO) TT=2519.89	? (TO) TT=2520.06	? (TO) TT=2519.95	? (TO) TT=2400.09	? (TO) TT=2520.03	? (TO) TT=2520.11	? (TO) TT=2520.1	? (TO) TT=2520.14	? (TO) TT=2520.09	? (TO) TT=2519.79	? (TO) TT=2520.05	? (TO) TT=2520.05	? (TO) TT=2400.09	? (TO) TT=2520.05	? (TO) TT=2400.09	? (TO) TT=2520.03	? (TO) TT=2400.08	? (TO) TT=2520.1	? (TO) TT=2520.06	No data	No data	? (TO) TT=2519.93	? (TO) TT=2520.1	? (TO) TT=2519.83	? (TO) TT=2400.09	? (TO) TT=2520.08	? (TO) TT=2520.07	? (TO) TT=2520.11	? (TO) TT=2520.05
Quasigroups/ QuasiGroup-4-16_c18.xml		? (TO) TT=2520.04	? (TO) TT=2519.98	? (TO) TT=2520.07	? (TO) TT=2520.08	? (TO) TT=2400.1	? (TO) TT=2520.04	? (TO) TT=2520.08	? (TO) TT=2520.12	? (TO) TT=2520.13	? (TO) TT=2520.09	? (TO) TT=2519.8	? (TO) TT=2520.02	? (TO) TT=2519.65	? (TO) TT=2400.05	? (TO) TT=2519.95	? (TO) TT=2400.11	? (TO) TT=2520.03	? (TO) TT=2400.11	? (TO) TT=2520.1	? (TO) TT=2520.06	No data	No data	? (TO) TT=2519.97	? (TO) TT=2520.07	? (TO) TT=2520.08	? (TO) TT=2400.16	? (MO) TT=797.886	? (TO) TT=2520.06	? (MO) TT=797.465	? (TO) TT=2520.13
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
Quasigroups/ QuasiGroup-4-20_c18.xml		? (TO) TT=2520.06	? (TO) TT=2519.96	? (TO) TT=2519.94	? (TO) TT=2519.94	? (TO) TT=2400.07	? (TO) TT=2520.09	? (TO) TT=2520.07	? (TO) TT=2520.06	? (TO) TT=2520.1	? (TO) TT=2520.11	? (TO) TT=2519.96	? (TO) TT=2520.04	? (TO) TT=2520.05	? (TO) TT=2400.09	? (TO) TT=2519.98	? (TO) TT=2400.03	? (TO) TT=2520.11	? (TO) TT=2400.03	? (TO) TT=2520.1	? (TO) TT=2520.04	No data	No data	? (TO) TT=2519.92	? (TO) TT=2520.03	? (TO) TT=2519.97	? (problem) TT=1545.13	? (MO) TT=40.344	? (TO) TT=2520.09	? (MO) TT=39.623	? (TO) TT=2520.14
Quasigroups/ QuasiGroup-7-07_c18.xml	UNSAT TT=0.287	UNSAT TT=0.02	UNSAT TT=0.821	UNSAT TT=0.021	UNSAT TT=3.704	UNSAT TT=2.395	UNSAT TT=2.395	UNSAT TT=12.562	UNSAT TT=12.307	UNSAT TT=12.34	UNSAT TT=12.716	UNSAT TT=0.569	UNSAT TT=0.041	UNSAT TT=2.957	UNSAT TT=0.541	UNSAT TT=0.613	UNSAT TT=2.862	UNSAT TT=2.794	? (TO) TT=2400.05	? (TO) TT=2520.08	UNSAT TT=3.261	No data	No data	UNSAT TT=0.289	UNSAT TT=0.289	UNSAT TT=0.287	UNSAT TT=8.822	UNSAT TT=4.857	UNSAT TT=6.082	UNSAT TT=4.806	UNSAT TT=6.003
Quasigroups/ QuasiGroup-7-08_c18.xml	UNSAT TT=1.556	UNSAT TT=0.027	UNSAT TT=40.323	UNSAT TT=0.026	UNSAT TT=525.579	UNSAT TT=13.19	UNSAT TT=13.311	UNSAT TT=83.587	UNSAT TT=69.525	UNSAT TT=73.749	UNSAT TT=84.986	UNSAT TT=47.872	UNSAT TT=0.019	UNSAT TT=567.455	UNSAT TT=31.19	UNSAT TT=28.968	UNSAT TT=24.006	UNSAT TT=26.9	? (TO) TT=2400.05	? (TO) TT=2520.08	UNSAT TT=174.568	No data	No data	UNSAT TT=1.567	UNSAT TT=1.556	UNSAT TT=1.561	? (TO) TT=2400.09	UNSAT TT=6.663	UNSAT TT=7.536	UNSAT TT=6.52	UNSAT TT=7.53
Quasigroups/ QuasiGroup-7-09_c18.xml	SAT TT=0.028	Wrong UNSAT TT=0.046	SAT TT=14.301	Wrong UNSAT TT=0.034	SAT TT=0.231	SAT TT=1.913	SAT TT=1.85	SAT TT=11.906	SAT TT=9.326	SAT TT=9.325	SAT TT=14	SAT TT=0.43	Wrong UNSAT TT=0.034	SAT TT=14.045	SAT TT=0.028	SAT TT=0.06	SAT TT=7.088	SAT TT=7.377	SAT TT=2.37	SAT TT=2.163	SAT TT=2.16	No data	No data	SAT TT=1.479	SAT TT=1.474	SAT TT=1.479	SAT TT=172.461	SAT TT=6.578	SAT TT=7.913	SAT TT=6.625	SAT TT=7.902
Quasigroups/ QuasiGroup-7-10_c18.xml	UNSAT TT=1431.43	UNSAT TT=0.076	? TT=2520.01	UNSAT TT=0.05	? TT=2520.01	? (TO) TT=2400.05	? (TO) TT=2520.11	? (TO) TT=2520.1	? (TO) TT=2520.11	? (TO) TT=2520.08	? (TO) TT=2520.08	? (TO) TT=2519.91	UNSAT TT=0.048	? TT=2520.01	? (TO) TT=2400.11	? (TO) TT=2520.06	? (TO) TT=2400.09	? (TO) TT=2520.08	? (TO) TT=2400.05	? (TO) TT=2520.08	? (TO) TT=2520.09	No data	No data	UNSAT TT=1478.03	UNSAT TT=1499.06	UNSAT TT=1478.01	? (TO) TT=2400.1	UNSAT TT=1623.38	UNSAT TT=1431.43	UNSAT TT=1602.04	UNSAT TT=1434.05
Quasigroups/ QuasiGroup-7-12_c18.xml		UNSAT TT=0.118	? TT=2520.02	UNSAT TT=0.088	? TT=2520.02	? (TO) TT=2400.08	? (TO) TT=2520.08	? (TO) TT=2520.11	? (TO) TT=2520.1	? (TO) TT=2520.07	? (TO) TT=2520.05	? (TO) TT=2519.96	UNSAT TT=0.097	? (TO) TT=2519.76	? (TO) TT=2400.07	? (TO) TT=2519.98	? (TO) TT=2400.08	? (TO) TT=2520.05	? (TO) TT=2400.06	? (TO) TT=2520.03	? (TO) TT=2520.08	No data	No data	? (TO) TT=2519.9	? (TO) TT=2519.99	? (TO) TT=2520.03	? (TO) TT=2400.12	? (TO) TT=2520.05	? (TO) TT=2520.05	? (TO) TT=2520.13	? (TO) TT=2520.05
Quasigroups/ QuasiGroup-7-14_c18.xml		UNSAT TT=0.28	? TT=2520.03	UNSAT TT=0.152	? TT=2520.03	? (TO) TT=2400.07	? (TO) TT=2520.07	? (TO) TT=2520.13	? (TO) TT=2520.13	? (TO) TT=2520.04	? (TO) TT=2520.08	? (TO) TT=2520.1	UNSAT TT=0.174	? TT=2520.03	? (TO) TT=2400.08	? (TO) TT=2519.94	? (TO) TT=2400.06	? (TO) TT=2520.04	? (TO) TT=2400.07	? (TO) TT=2520.09	? (TO) TT=2520.08	No data	No data	? (TO) TT=2519.84	? (TO) TT=2519.82	? (TO) TT=2519.93	? (TO) TT=2400.17	? (TO) TT=2520.08	? (TO) TT=2520.11	? (TO) TT=2520.05	? (TO) TT=2520.09
Quasigroups/ QuasiGroup-7-16_c18.xml		UNSAT TT=0.491	? (TO) TT=2520.03	UNSAT TT=0.337	? TT=2520.05	? (TO) TT=2400.06	? (TO) TT=2520.07	? (TO) TT=2520.14	? (TO) TT=2520.15	? (TO) TT=2520.09	? (TO) TT=2520.12	? (TO) TT=2519.91	UNSAT TT=1.284	? (TO) TT=2519.91	? (TO) TT=2400.05	? (TO) TT=2520.07	? (TO) TT=2400.07	? (TO) TT=2520.03	? (TO) TT=2400.06	? (TO) TT=2520.09	? (TO) TT=2520.06	No data	No data	? (TO) TT=2519.97	? (TO) TT=2520.04	? (TO) TT=2520	? (TO) TT=2400.18	? (TO) TT=2520.04	? (TO) TT=2520.06	? (TO) TT=2520.14	? (TO) TT=2520.11
Quasigroups/ QuasiGroup-7-20_c18.xml		UNSAT TT=1.021	? (TO) TT=2519.94	UNSAT TT=0.873	? (TO) TT=2519.92	? (TO) TT=2400.05	? (TO) TT=2520.03	? (TO) TT=2520.14	? (TO) TT=2520.08	? (TO) TT=2520.08	? (TO) TT=2520.09	? (TO) TT=2520.06	UNSAT TT=0.625	? (TO) TT=2519.8	? (TO) TT=2400.02	? (TO) TT=2520.09	? (TO) TT=2400.1	? (TO) TT=2520.02	? (TO) TT=2400.06	? (TO) TT=2520.05	? (TO) TT=2520.05	No data	No data	? (TO) TT=2519.93	? (TO) TT=2520.05	? (TO) TT=2520.07	? (TO) TT=2400.19	? (MO) TT=1386.95	? (TO) TT=2520.09	? (MO) TT=1390.13	? (TO) TT=2520.09
Rcpsp/ Rcpsp-j120-01-01_c18.xml	SAT f=105 TT=2520.09 T1=1011.33	No data	No data	No data	No data	SAT (TO) f=114 TT=2400.06 T1=1793.52	SAT (TO) f=114 TT=2520.08 T1=1829.44	SAT (TO) f=112 TT=2520.06 T1=106.69	SAT (TO) f=114 TT=2520.09 T1=538.69	SAT (TO) f=111 TT=2520.06 T1=296.62	SAT (TO) f=113 TT=2520.1 T1=889.13	? (NS) TT=0.076	No data	No data	SAT (TO) f=105 TT=2400.04 T1=1720.95	SAT (TO) f=105 TT=2519.78 T1=1721.39	No data	No data	SAT (TO) f=107 TT=2400.07 T1=138.9	SAT (TO) f=105 TT=2520.09 T1=1011.33	SAT (TO) f=107 TT=2520.08 T1=30.13	SAT (TO) f=108 TT=2400.03 T1=1073.59	SAT (TO) f=107 TT=2520.07 T1=143.48	? (TO) TT=2519.85	? (TO) TT=2520.01	? (TO) TT=2519.94	? TT=1458.93	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
Rcpsp/ Rcpsp-j120-01-02_c18.xml	OPT f=109 TT=166.12 T1=39.9	No data	No data	No data	No data	SAT (TO) f=111 TT=2400.11 T1=1864.09	SAT (TO) f=111 TT=2520.05 T1=1913.79	SAT (TO) f=110 TT=2520.06 T1=883.88	SAT (TO) f=111 TT=2520.07 T1=156.77	SAT (TO) f=111 TT=2520.09 T1=28.46	SAT (TO) f=111 TT=2520.1 T1=29.2	? (NS) TT=0.076	No data	No data	OPT f=109 TT=2383.57 T1=2367.44	OPT f=109 TT=2401.62 T1=2385.24	No data	No data	SAT (TO) f=109 TT=2400.1 T1=29.74	SAT (TO) f=109 TT=2520.09 T1=39.9	OPT f=109 TT=166.12 T1=66.15	SAT (TO) f=109 TT=2400.06 T1=363.06	SAT (TO) f=112 TT=2520.1 T1=5.75	OPT f=109 TT=339.937 T1=339.93	OPT f=109 TT=351.667 T1=351.66	OPT f=109 TT=351.066 T1=351.11	? (problem) TT=1448.71	No data	No data	No data	No data
Rcpsp/ Rcpsp-j120-01-03_c18.xml	SAT f=126 TT=2520.03 T1=21.26	No data	No data	No data	No data	SAT (TO) f=129 TT=2400.06 T1=626.51	SAT (TO) f=129 TT=2520.1 T1=643.87	SAT (TO) f=128 TT=2520.09 T1=608.18	SAT (TO) f=126 TT=2520.08 T1=1831.02	SAT (TO) f=130 TT=2520.1 T1=18.36	SAT (TO) f=128 TT=2520.15 T1=1956.63	? (NS) TT=0.076	No data	No data	SAT (TO) f=126 TT=2400.01 T1=183.97	SAT (TO) f=126 TT=2519.91 T1=181.32	No data	No data	SAT (TO) f=126 TT=2400.05 T1=43.04	SAT (TO) f=126 TT=2520.03 T1=21.26	SAT (TO) f=126 TT=2520.1 T1=751.87	SAT (TO) f=128 TT=2400.09 T1=3.49	SAT (TO) f=127 TT=2520.07 T1=153.31	? (TO) TT=2519.91	? (TO) TT=2519.94	? (TO) TT=2520.05	? TT=1420.14	No data	No data	No data	No data
Rcpsp/ Rcpsp-j120-01-04_c18.xml	OPT f=97 TT=8.173 T1=3.59	No data	No data	No data	No data	SAT (TO) f=97 TT=2400.03 T1=2265.79	SAT (TO) f=97 TT=2520.07 T1=2408.14	SAT (TO) f=97 TT=2520.14 T1=20.91	SAT (TO) f=97 TT=2520.05 T1=109.45	SAT (TO) f=97 TT=2520.08 T1=42.92	SAT (TO) f=97 TT=2520.1 T1=25.23	? (NS) TT=0.077	No data	No data	OPT f=97 TT=31.961 T1=14.26	OPT f=97 TT=31.839 T1=14.24	No data	No data	SAT (TO) f=97 TT=2400.1 T1=6.49	SAT (TO) f=97 TT=2520.08 T1=3.59	OPT f=97 TT=8.173 T1=3.62	SAT (TO) f=97 TT=2400.05 T1=9.52	SAT (TO) f=97 TT=2520.04 T1=6.58	OPT f=97 TT=336.365 T1=336.36	OPT f=97 TT=336.107 T1=336.09	OPT f=97 TT=335.948 T1=335.99	? TT=1322.49	No data	No data	No data	No data
Rcpsp/ Rcpsp-j30-01-01_c18.xml	OPT f=43 TT=1.306 T1=0.6	No data	No data	No data	No data	OPT f=43 TT=1.275 T1=0.63	OPT f=43 TT=1.306 T1=0.6	OPT f=43 TT=7.646 T1=3.71	OPT f=43 TT=7.901 T1=3.74	OPT f=43 TT=7.804 T1=3.69	OPT f=43 TT=7.839 T1=3.71	? (NS) TT=0.023	No data	No data	OPT f=43 TT=0.078 T1=0.07	OPT f=43 TT=0.078 T1=0.07	No data	No data	SAT (TO) f=43 TT=2400.05 T1=1.31	SAT (TO) f=43 TT=2520.08 T1=1.26	OPT f=43 TT=2.108 T1=1.24	OPT f=43 TT=2.352 T1=1.33	OPT f=43 TT=2.277 T1=1.26	OPT f=43 TT=4.794 T1=4.79	OPT f=43 TT=4.757 T1=4.75	OPT f=43 TT=4.749 T1=4.74	OPT f=43 TT=51.443 T1=40.93	No data	No data	No data	No data
Rcpsp/ Rcpsp-j30-01-02_c18.xml	OPT f=47 TT=1.21 T1=0.58	No data	No data	No data	No data	OPT f=47 TT=1.2 T1=0.58	OPT f=47 TT=1.21 T1=0.58	OPT f=47 TT=7.555 T1=3.82	OPT f=47 TT=7.799 T1=3.7	OPT f=47 TT=7.556 T1=3.61	OPT f=47 TT=7.705 T1=3.67	? (NS) TT=0.023	No data	No data	OPT f=47 TT=0.08 T1=0.07	OPT f=47 TT=0.08 T1=0.07	No data	No data	SAT (TO) f=47 TT=2400.1 T1=1.29	SAT (TO) f=47 TT=2520.1 T1=1.21	OPT f=47 TT=2.02 T1=1.2	OPT f=47 TT=2.429 T1=1.36	OPT f=47 TT=2.375 T1=1.3	OPT f=47 TT=3.769 T1=3.76	OPT f=47 TT=3.936 T1=3.93	OPT f=47 TT=3.93 T1=3.93	OPT f=47 TT=64.856 T1=53.97	No data	No data	No data	No data
Rcpsp/ Rcpsp-j30-01-03_c18.xml	OPT f=47 TT=1.353 T1=0.62	No data	No data	No data	No data	OPT f=47 TT=1.33 T1=0.61	OPT f=47 TT=1.353 T1=0.62	OPT f=47 TT=8.334 T1=3.91	OPT f=47 TT=8.748 T1=4.02	OPT f=47 TT=8.198 T1=3.92	OPT f=47 TT=8.167 T1=3.85	? (NS) TT=0.023	No data	No data	OPT f=47 TT=0.067 T1=0.06	OPT f=47 TT=0.067 T1=0.06	No data	No data	SAT (TO) f=47 TT=2400.05 T1=1.3	SAT (TO) f=47 TT=2520.11 T1=1.21	OPT f=47 TT=2.049 T1=1.21	OPT f=47 TT=2.322 T1=1.31	OPT f=47 TT=2.218 T1=1.25	OPT f=47 TT=2.519 T1=2.51	OPT f=47 TT=2.394 T1=2.4	OPT f=47 TT=2.392 T1=2.39	OPT f=47 TT=42.265 T1=33.77	No data	No data	No data	No data
Rcpsp/ Rcpsp-j30-01-04_c18.xml	OPT f=62 TT=2.06 T1=0.83	No data	No data	No data	No data	OPT f=62 TT=2.039 T1=0.81	OPT f=62 TT=2.06 T1=0.83	OPT f=62 TT=7.697 T1=3.72	OPT f=62 TT=8.241 T1=3.82	OPT f=62 TT=7.911 T1=3.73	OPT f=62 TT=7.807 T1=3.7	? (NS) TT=0.023	No data	No data	OPT f=62 TT=0.093 T1=0.08	OPT f=62 TT=0.093 T1=0.08	No data	No data	SAT (TO) f=62 TT=2400.09 T1=1.34	SAT (TO) f=62 TT=2520.11 T1=1.26	OPT f=62 TT=2.19 T1=1.25	OPT f=62 TT=2.337 T1=1.33	OPT f=62 TT=2.249 T1=1.26	OPT f=62 TT=5.37 T1=5.36	OPT f=62 TT=4.576 T1=4.57	OPT f=62 TT=4.573 T1=4.57	OPT f=62 TT=204.305 T1=185.11	No data	No data	No data	No data
Rcpsp/ Rcpsp-j60-01-01_c18.xml	OPT f=77 TT=1.714 T1=0.72	No data	No data	No data	No data	OPT f=77 TT=1.71 T1=0.72	OPT f=77 TT=1.714 T1=0.72	OPT f=77 TT=9.524 T1=4.36	OPT f=77 TT=8.903 T1=4.06	OPT f=77 TT=8.826 T1=4.28	OPT f=77 TT=8.773 T1=4.05	? (NS) TT=0.041	No data	No data	OPT f=77 TT=0.654 T1=0.63	OPT f=77 TT=0.657 T1=0.63	No data	No data	SAT (TO) f=77 TT=2400.06 T1=1.63	SAT (TO) f=77 TT=2520.09 T1=1.52	OPT f=77 TT=3.006 T1=1.51	OPT f=77 TT=3.834 T1=1.75	OPT f=77 TT=3.735 T1=1.69	OPT f=77 TT=28.064 T1=28.06	OPT f=77 TT=28.76 T1=28.76	OPT f=77 TT=28.698 T1=28.69	? (TO) TT=2400.31	No data	No data	No data	No data
Rcpsp/ Rcpsp-j60-01-02_c18.xml	OPT f=68 TT=3.414 T1=1.38	No data	No data	No data	No data	OPT f=68 TT=3.805 T1=1.39	OPT f=68 TT=3.805 T1=1.38	OPT f=68 TT=9.342 T1=4.45	OPT f=68 TT=9.482 T1=4.22	OPT f=68 TT=10 T1=4.41	OPT f=68 TT=10.055 T1=4.45	? (NS) TT=0.041	No data	No data	OPT f=68 TT=0.766 T1=0.67	OPT f=68 TT=0.763 T1=0.67	No data	No data	SAT (TO) f=68 TT=2400.05 T1=1.73	SAT (TO) f=68 TT=2520.09 T1=1.67	OPT f=68 TT=3.414 T1=1.63	OPT f=68 TT=4.041 T1=1.85	OPT f=68 TT=3.912 T1=1.76	OPT f=68 TT=34.232 T1=34.23	OPT f=68 TT=33.609 T1=33.6	OPT f=68 TT=33.791 T1=33.79	SAT (TO) f=246 TT=2400.29 T1=2245.96	No data	No data	No data	No data
Rcpsp/ Rcpsp-j60-01-03_c18.xml	OPT f=68 TT=1.748 T1=0.74	No data	No data	No data	No data	OPT f=68 TT=1.754 T1=0.73	OPT f=68 TT=1.748 T1=0.74	OPT f=68 TT=9.25 T1=4.2	OPT f=68 TT=9.114 T1=4.21	OPT f=68 TT=8.841 T1=4.04	OPT f=68 TT=8.739 T1=4.06	? (NS) TT=0.041	No data	No data	OPT f=68 TT=0.547 T1=0.52	OPT f=68 TT=0.543 T1=0.52	No data	No data	SAT (TO) f=68 TT=2400.06 T1=1.57	SAT (TO) f=68 TT=2520.03 T1=1.49	OPT f=68 TT=2.936 T1=1.48	OPT f=68 TT=3.671 T1=1.74	OPT f=68 TT=3.454 T1=1.64	OPT f=68 TT=27.411 T1=27.41	OPT f=68 TT=27.932 T1=27.93	OPT f=68 TT=27.952 T1=27.95	SAT (TO) f=83 TT=2400.35 T1=2260.44	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
Rcpsp/ Rcpsp-j60-01-04_c18.xml	OPT f=91 TT=2.87 T1=1.33	No data	No data	No data	No data	OPT f=91 TT=3.664 T1=1.28	OPT f=91 TT=3.755 T1=1.33	OPT f=91 TT=9.306 T1=4.2	OPT f=91 TT=9.474 T1=4.72	OPT f=91 TT=9.278 T1=4.43	OPT f=91 TT=9.619 T1=4.83	? (NS) TT=0.041	No data	No data	OPT f=91 TT=0.635 T1=0.55	OPT f=91 TT=0.633 T1=0.54	No data	No data	SAT (TO) f=91 TT=2400.08 T1=1.57	SAT (TO) f=91 TT=2520.03 T1=1.48	OPT f=91 TT=2.87 T1=1.46	OPT f=91 TT=3.69 T1=1.73	OPT f=91 TT=3.439 T1=1.94	OPT f=91 TT=31.63 T1=31.63	OPT f=91 TT=31.908 T1=31.9	OPT f=91 TT=31.946 T1=31.94	? (TO) TT=2400.34	No data	No data	No data	No data
Rcpsp/ Rcpsp-j90-01-01_c18.xml	OPT f=73 TT=5.482 T1=2.3	No data	No data	No data	No data	OPT f=73 TT=13.091 T1=6.94	OPT f=73 TT=12.264 T1=6.87	OPT f=73 TT=25.141 T1=16.21	OPT f=73 TT=12.008 T1=5.03	OPT f=73 TT=15.843 T1=7.39	OPT f=73 TT=14.428 T1=6.57	? (NS) TT=0.058	No data	No data	OPT f=73 TT=5.339 T1=5.07	OPT f=73 TT=5.348 T1=5.07	No data	No data	SAT (TO) f=73 TT=2400.11 T1=2.75	SAT (TO) f=73 TT=2520.05 T1=3.02	OPT f=73 TT=5.482 T1=2.3	OPT f=73 TT=6.716 T1=3.14	OPT f=73 TT=6.033 T1=2.95	OPT f=73 TT=99.645 T1=99.64	OPT f=73 TT=101.108 T1=101.1	OPT f=73 TT=101.015 T1=101	? TT=1563.29	No data	No data	No data	No data
Rcpsp/ Rcpsp-j90-01-02_c18.xml	OPT f=92 TT=2.97 T1=1.16	No data	No data	No data	No data	OPT f=92 TT=3.027 T1=1.13	OPT f=92 TT=2.97 T1=1.16	OPT f=92 TT=11.466 T1=5.4	OPT f=92 TT=11.193 T1=5.35	OPT f=92 TT=10.25 T1=4.55	OPT f=92 TT=10.56 T1=5.04	? (NS) TT=0.059	No data	No data	OPT f=92 TT=2.241 T1=2.12	OPT f=92 TT=2.229 T1=2.11	No data	No data	SAT (TO) f=92 TT=2400.03 T1=1.96	SAT (TO) f=92 TT=2520.05 T1=1.87	OPT f=92 TT=4.056 T1=1.82	OPT f=92 TT=5.091 T1=2.16	OPT f=92 TT=5.073 T1=2.15	OPT f=92 TT=109.632 T1=109.62	OPT f=92 TT=110.793 T1=110.78	OPT f=92 TT=110.192 T1=110.2	? TT=1842.5	No data	No data	No data	No data
Rcpsp/ Rcpsp-j90-01-03_c18.xml	OPT f=66 TT=5.058 T1=2.14	No data	No data	No data	No data	OPT f=66 TT=9.92 T1=3.23	OPT f=66 TT=10.833 T1=3.35	OPT f=66 TT=13.554 T1=5.57	OPT f=66 TT=13.317 T1=5.33	OPT f=66 TT=12.588 T1=5.2	OPT f=66 TT=13.489 T1=5.46	? (NS) TT=0.06	No data	No data	OPT f=66 TT=4.478 T1=4	OPT f=66 TT=4.481 T1=4	No data	No data	SAT (TO) f=66 TT=2400.03 T1=2.31	SAT (TO) f=66 TT=2520.03 T1=2.26	OPT f=66 TT=5.157 T1=2.49	OPT f=66 TT=5.332 T1=2.26	OPT f=66 TT=5.058 T1=2.14	OPT f=66 TT=123.198 T1=123.19	OPT f=66 TT=124.859 T1=124.87	OPT f=66 TT=124.528 T1=124.52	? TT=1654.38	No data	No data	No data	No data
Rcpsp/ Rcpsp-j90-01-04_c18.xml	OPT f=86 TT=6.883 T1=1.18	No data	No data	No data	No data	OPT f=86 TT=1083.67 T1=1	OPT f=86 TT=1105.99 T1=1.18	OPT f=86 TT=49.411 T1=5.97	OPT f=86 TT=98.119 T1=6.36	OPT f=86 TT=48.606 T1=5.34	OPT f=86 TT=128.212 T1=4.88	? (NS) TT=0.058	No data	No data	OPT f=86 TT=16.662 T1=3.06	OPT f=86 TT=16.66 T1=3.05	No data	No data	SAT (TO) f=86 TT=2400.09 T1=2.5	SAT (TO) f=86 TT=2520.12 T1=2.7	OPT f=86 TT=6.883 T1=2.55	OPT f=86 TT=7.283 T1=2.58	OPT f=86 TT=7.209 T1=2.43	OPT f=86 TT=111.394 T1=111.39	OPT f=86 TT=111.948 T1=111.93	OPT f=86 TT=112.13 T1=112.13	? TT=1691.83	No data	No data	No data	No data
Rlfap/Rlfap-dec-scens11/ Rlfap-scen-11-f01_c18.xml	UNSAT TT=17.385	UNSAT TT=1199.08	UNSAT TT=1216.92	? (TO) TT=2519.81	? (TO) TT=2520.06	? (TO) TT=2400.09	? (TO) TT=2520.05	? (TO) TT=2520.07	? (TO) TT=2520.12	? (TO) TT=2520.07	? (TO) TT=2520.16	? (TO) TT=2519.96	? (TO) TT=2520.03	? (TO) TT=2519.77	? (TO) TT=2400.05	? (TO) TT=2519.91	? (TO) TT=2400.08	? (TO) TT=2520.07	? (TO) TT=2400.02	? (TO) TT=2520.12	? (TO) TT=2520.09	No data	No data	UNSAT TT=271.731	UNSAT TT=297.152	UNSAT TT=293.888	? (TO) TT=2400.16	UNSAT TT=17.385	UNSAT TT=21.206	UNSAT TT=17.743	UNSAT TT=20.112
Rlfap/Rlfap-dec-scens11/ Rlfap-scen-11-f02_c18.xml	UNSAT TT=18.41	UNSAT TT=609.291	UNSAT TT=624.897	UNSAT TT=655.588	UNSAT TT=661.857	UNSAT TT=1957.72	UNSAT TT=1805.37	? (TO) TT=2520.15	? (TO) TT=2520.17	? (TO) TT=2520.11	? (TO) TT=2520.07	? (TO) TT=2520.08	UNSAT TT=1168.6	UNSAT TT=1156.08	? (TO) TT=2400.08	? (TO) TT=2519.89	? (TO) TT=2400.06	? (TO) TT=2520.1	? (TO) TT=2400.07	? (TO) TT=2520.12	? (TO) TT=2520.11	No data	No data	UNSAT TT=224.691	UNSAT TT=226.347	UNSAT TT=197.532	? (TO) TT=2400.15	UNSAT TT=18.68	UNSAT TT=22.039	UNSAT TT=18.41	UNSAT TT=21.06
Rlfap/Rlfap-dec-scens11/ Rlfap-scen-11-f03_c18.xml	UNSAT TT=17.001	UNSAT TT=226.071	UNSAT TT=216.376	UNSAT TT=160.087	UNSAT TT=175.455	UNSAT TT=701.871	UNSAT TT=694.718	? (TO) TT=2520.06	? (TO) TT=2520.17	? (TO) TT=2520.14	? (TO) TT=2520.09	? (TO) TT=2519.79	UNSAT TT=329.17	UNSAT TT=325.57	UNSAT TT=825.882	UNSAT TT=859.258	? (TO) TT=2400.09	? (TO) TT=2520.05	? (TO) TT=2400.11	? (TO) TT=2520.04	? (TO) TT=2520.06	No data	No data	UNSAT TT=96.366	UNSAT TT=125.589	UNSAT TT=110.935	UNSAT TT=850.227	UNSAT TT=17.574	UNSAT TT=19.207	UNSAT TT=17.001	UNSAT TT=19.741
Rlfap/Rlfap-dec-scens11/ Rlfap-scen-11-f04_c18.xml	UNSAT TT=15.989	UNSAT TT=65.664	UNSAT TT=59.096	UNSAT TT=46.56	UNSAT TT=56.367	UNSAT TT=238.566	UNSAT TT=280.828	? (TO) TT=2520.09	? (TO) TT=2520.15	? (TO) TT=2520.11	? (TO) TT=2520.14	UNSAT TT=1429.97	UNSAT TT=93.111	UNSAT TT=90.888	UNSAT TT=285.997	UNSAT TT=264.786	? (TO) TT=2400.09	? (TO) TT=2520.07	? (TO) TT=2400.09	? (TO) TT=2520.09	? (TO) TT=2520.06	No data	No data	UNSAT TT=92.404	UNSAT TT=96.348	UNSAT TT=100.751	UNSAT TT=191.862	UNSAT TT=17.182	UNSAT TT=19.433	UNSAT TT=15.989	UNSAT TT=18.801
Rlfap/Rlfap-dec-scens11/ Rlfap-scen-11-f05_c18.xml	UNSAT TT=12.302	UNSAT TT=18.776	UNSAT TT=20.256	UNSAT TT=11.556	UNSAT TT=12.302	UNSAT TT=67.291	UNSAT TT=76.517	UNSAT TT=1552.24	UNSAT TT=1566.32	UNSAT TT=1685.77	UNSAT TT=1547.24	UNSAT TT=321.029	UNSAT TT=26.048	UNSAT TT=25.782	UNSAT TT=60.801	UNSAT TT=59.304	? (TO) TT=2400.1	? (TO) TT=2520.09	? (TO) TT=2400.1	? (TO) TT=2520.11	? (TO) TT=2520.08	No data	No data	UNSAT TT=200.067	UNSAT TT=182.315	UNSAT TT=188.658	UNSAT TT=28.214	UNSAT TT=16.398	UNSAT TT=18.309	UNSAT TT=16.078	UNSAT TT=18.314
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
Rlfap/Rlfap-dec-scens11/ Rlfap-scen-11-f06_c18.xml	UNSAT TT=2.897	UNSAT TT=6.474	UNSAT TT=6.551	UNSAT TT=2.864	UNSAT TT=2.897	UNSAT TT=23.496	UNSAT TT=20.99	UNSAT TT=422.513	UNSAT TT=376.99	UNSAT TT=397.263	UNSAT TT=392.692	UNSAT TT=60.056	UNSAT TT=5.495	UNSAT TT=5.474	UNSAT TT=14.734	UNSAT TT=16.085	? (TO) TT=2400.06	? (TO) TT=2520.12	? (TO) TT=2400.1	? (TO) TT=2520.07	? (TO) TT=2520.06	No data	No data	UNSAT TT=65.495	UNSAT TT=57.31	UNSAT TT=66.246	UNSAT TT=17.026	UNSAT TT=16.509	UNSAT TT=18.748	UNSAT TT=16.153	UNSAT TT=18.349
Rlfap/Rlfap-dec-scens11/ Rlfap-scen-11-f07_c18.xml	UNSAT TT=2.154	UNSAT TT=3.974	UNSAT TT=3.999	UNSAT TT=2.122	UNSAT TT=2.154	UNSAT TT=22.591	UNSAT TT=23.463	UNSAT TT=236.879	UNSAT TT=199.792	UNSAT TT=221.429	UNSAT TT=230.286	UNSAT TT=30.013	UNSAT TT=3.245	UNSAT TT=3.282	UNSAT TT=11.36	UNSAT TT=11.96	UNSAT TT=789.479	UNSAT TT=780.23	? (TO) TT=2400.06	? (TO) TT=2520.07	? (TO) TT=2520.12	No data	No data	UNSAT TT=115.826	UNSAT TT=86.554	UNSAT TT=122.031	UNSAT TT=15.854	UNSAT TT=16.164	UNSAT TT=17.674	UNSAT TT=15.974	UNSAT TT=18.12
Rlfap/Rlfap-dec-scens11/ Rlfap-scen-11-f08_c18.xml	UNSAT TT=1.329	UNSAT TT=1.614	UNSAT TT=1.651	UNSAT TT=1.283	UNSAT TT=1.329	UNSAT TT=16.075	UNSAT TT=16.122	UNSAT TT=69.254	UNSAT TT=62.397	UNSAT TT=64.679	UNSAT TT=73.215	UNSAT TT=4.463	UNSAT TT=1.455	UNSAT TT=1.45	UNSAT TT=3.076	UNSAT TT=3.159	? (TO) TT=2400.08	? (TO) TT=2520.09	? (TO) TT=2400.12	? (TO) TT=2520.05	UNSAT TT=489.54	No data	No data	UNSAT TT=103.421	UNSAT TT=126.418	UNSAT TT=95.147	UNSAT TT=16.192	UNSAT TT=16.237	UNSAT TT=18.115	UNSAT TT=15.74	UNSAT TT=17.98
Rlfap/Rlfap-dec-scens11/ Rlfap-scen-11-f09_c18.xml	UNSAT TT=1.269	UNSAT TT=1.593	UNSAT TT=1.617	UNSAT TT=1.241	UNSAT TT=1.269	UNSAT TT=82.115	UNSAT TT=88.712	UNSAT TT=57.77	UNSAT TT=49.455	UNSAT TT=60.628	UNSAT TT=72.584	UNSAT TT=3.619	UNSAT TT=1.351	UNSAT TT=1.357	UNSAT TT=3.052	UNSAT TT=3.05	UNSAT TT=1219.27	UNSAT TT=1124.26	? (TO) TT=2400.09	? (TO) TT=2520.07	UNSAT TT=208.435	No data	No data	UNSAT TT=117.047	UNSAT TT=94.634	UNSAT TT=87.805	UNSAT TT=16.076	UNSAT TT=16.216	UNSAT TT=18.235	UNSAT TT=16.414	UNSAT TT=18.449
Rlfap/Rlfap-dec-scens11/ Rlfap-scen-11-f10_c18.xml	UNSAT TT=1.247	UNSAT TT=1.262	UNSAT TT=1.279	UNSAT TT=1.229	UNSAT TT=1.248	UNSAT TT=8.753	UNSAT TT=6.966	UNSAT TT=35.707	UNSAT TT=38.43	UNSAT TT=38.764	UNSAT TT=39.807	UNSAT TT=2.716	UNSAT TT=1.241	UNSAT TT=1.247	UNSAT TT=2.619	UNSAT TT=2.564	UNSAT TT=11.512	UNSAT TT=11.226	? (TO) TT=2400.08	? (TO) TT=2520.04	UNSAT TT=12.184	No data	No data	UNSAT TT=56.197	UNSAT TT=61.582	UNSAT TT=44.011	UNSAT TT=13.496	UNSAT TT=15.527	UNSAT TT=18.352	UNSAT TT=15.065	UNSAT TT=17.406
Rlfap/Rlfap-dec-scens11/ Rlfap-scen-11-f11_c18.xml	UNSAT TT=1.201	UNSAT TT=1.23	UNSAT TT=1.257	UNSAT TT=1.166	UNSAT TT=1.201	UNSAT TT=7.367	UNSAT TT=9.919	UNSAT TT=30.719	UNSAT TT=34.66	UNSAT TT=34.59	UNSAT TT=31.874	UNSAT TT=2.721	UNSAT TT=1.237	UNSAT TT=1.249	UNSAT TT=2.65	UNSAT TT=2.57	UNSAT TT=11.044	UNSAT TT=10.689	? (TO) TT=2400.1	? (TO) TT=2520.05	UNSAT TT=8.873	No data	No data	UNSAT TT=100.404	UNSAT TT=97.479	UNSAT TT=106.43	UNSAT TT=9.246	UNSAT TT=15.236	UNSAT TT=17.956	UNSAT TT=15.65	UNSAT TT=17.089
Rlfap/Rlfap-dec-scens11/ Rlfap-scen-11-f12_c18.xml	UNSAT TT=1.212	UNSAT TT=1.221	UNSAT TT=1.255	UNSAT TT=1.169	UNSAT TT=1.212	UNSAT TT=7.783	UNSAT TT=7.188	UNSAT TT=31.741	UNSAT TT=40.6	UNSAT TT=33.157	UNSAT TT=33.55	UNSAT TT=2.685	UNSAT TT=1.242	UNSAT TT=1.232	UNSAT TT=2.609	UNSAT TT=2.596	UNSAT TT=11.44	UNSAT TT=10.267	? (TO) TT=2400.06	? (TO) TT=2520.09	UNSAT TT=9.06	No data	No data	UNSAT TT=109.213	UNSAT TT=120.268	UNSAT TT=120.61	UNSAT TT=10.19	UNSAT TT=15.238	UNSAT TT=17.381	UNSAT TT=15.458	UNSAT TT=17.479
Rlfap/Rlfap-opt/ Rlfap-graph-01-opt_c18.xml	SAT f=18 TT=2520.16 T1=59.03	No data	No data	No data	No data	SAT (TO) f=20 TT=2400.06 T1=118.92	SAT (TO) f=20 TT=2520.08 T1=105.4	SAT (TO) f=20 TT=2520.1 T1=61.26	SAT (TO) f=18 TT=2520.16 T1=59.03	SAT (TO) f=20 TT=2520.08 T1=34.97	SAT (TO) f=20 TT=2520.09 T1=98.35	SAT (TO) f=20 TT=2519.93 T1=211.48	No data	No data	SAT (TO) f=30 TT=2400.01 T1=2400.1	SAT (TO) f=30 TT=2519.84 T1=2520	No data	No data	SAT (TO) f=48 TT=2400.05 T1=2368.1	SAT (TO) f=48 TT=2520.03 T1=15.27	SAT (TO) f=48 TT=2520.09 T1=13.88	Wrong Opt. f=34 TT=4.186 T1=1.71	SAT (TO) f=34 TT=2520.11 T1=77.64	? (TO) TT=2519.86	? (TO) TT=2519.94	? (TO) TT=2520.06	SAT (TO) f=40 TT=2400.43 T1=367.06	No data	No data	No data	No data
Rlfap/Rlfap-opt/ Rlfap-graph-02-opt_c18.xml	SAT f=14 TT=2520.07 T1=28.31	No data	No data	No data	No data	SAT (TO) f=16 TT=2400.12 T1=497.16	SAT (TO) f=16 TT=2520.1 T1=515.79	SAT (TO) f=16 TT=2520.12 T1=187.22	SAT (TO) f=16 TT=2520.11 T1=236.53	SAT (TO) f=16 TT=2520.12 T1=243.91	SAT (TO) f=16 TT=2520.16 T1=194.93	SAT (TO) f=14 TT=2520.07 T1=28.31	No data	No data	SAT (TO) f=32 TT=2400.01 T1=2399.9	SAT (TO) f=32 TT=2519.92 T1=2520	No data	No data	SAT (TO) f=48 TT=2400.03 T1=2368.7	SAT (TO) f=48 TT=2520.08 T1=80.9	SAT (TO) f=48 TT=2520.07 T1=79.37	Wrong Opt. f=32 TT=5.007 T1=1.99	SAT (TO) f=44 TT=2520.09 T1=18.35	? (TO) TT=2519.83	? (TO) TT=2519.92	? (TO) TT=2519.85	? TT=756.812	No data	No data	No data	No data
Rlfap/Rlfap-opt/ Rlfap-graph-03-opt_c18.xml	OPT f=380 TT=0.894 T1=0.89	No data	No data	No data	No data	OPT f=380 TT=5.434 T1=2.32	OPT f=380 TT=6.992 T1=2.62	OPT f=380 TT=23.444 T1=10.31	OPT f=380 TT=23.582 T1=10.43	OPT f=380 TT=23.46 T1=11.52	OPT f=380 TT=26.462 T1=12.11	OPT f=380 TT=0.894 T1=0.89	No data	No data	OPT f=380 TT=0.973 T1=0.96	OPT f=380 TT=0.967 T1=0.95	No data	No data	SAT (TO) f=380 TT=2400.03 T1=23.03	SAT (TO) f=380 TT=2520.05 T1=192.69	OPT f=380 TT=926.43 T1=916.26	SAT (TO) f=736 TT=2400.03 T1=72.11	SAT (TO) f=736 TT=2520.08 T1=73.93	OPT f=380 TT=110.593 T1=110.59	OPT f=380 TT=117.543 T1=117.54	OPT f=380 TT=117.915 T1=117.91	SAT (TO) f=764 TT=2400.28 T1=872.12	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
Rlfap/Rlfap-opt/ Rlfap-graph-04-opt_c18.xml	OPT f=394 TT=3.892 T1=2.72	No data	No data	No data	No data	OPT f=394 TT=6.986 T1=2.4	OPT f=394 TT=7.295 T1=2.72	OPT f=394 TT=34.72 T1=18.95	OPT f=394 TT=39.968 T1=18.86	OPT f=394 TT=33.59 T1=17.68	OPT f=394 TT=38.447 T1=20.16	OPT f=394 TT=3.892 T1=3.88	No data	No data	OPT f=394 TT=2.92 T1=2.89	OPT f=394 TT=2.821 T1=2.79	No data	No data	SAT (TO) f=792 TT=2400.09 T1=1082.43	? (TO) TT=2520.08	? (TO) TT=2520.05	? (TO) TT=2400.08	? (TO) TT=2520.04	OPT f=394 TT=228.615 T1=228.62	OPT f=394 TT=242.93 T1=242.92	OPT f=394 TT=242.352 T1=242.36	? (TO) TT=2400.38	No data	No data	No data	No data
Rlfap/Rlfap-opt/ Rlfap-graph-05-opt_c18.xml	SAT f=23688 TT=2519.94 T1=834.49	No data	No data	No data	No data	SAT (TO) f=29741 TT=2400.08 T1=91.77	SAT (TO) f=29741 TT=2520.03 T1=90.55	SAT (TO) f=26422 TT=2520.12 T1=11.61	SAT (TO) f=29155 TT=2520.13 T1=13.2	SAT (TO) f=29768 TT=2520.13 T1=11.62	SAT (TO) f=28082 TT=2520.11 T1=11.17	SAT (TO) f=23688 TT=2519.94 T1=834.49	No data	No data	Signal TT=0.071	SAT (TO) f=79099 TT=2519.93 T1=907.1	No data	No data	SAT (TO) f=75309 TT=2400.04 T1=187.43	SAT (TO) f=64228 TT=2520.04 T1=1728.63	SAT (TO) f=69809 TT=2520.05 T1=852.36	SAT (TO) f=36368 TT=2400.06 T1=2288.77	SAT (TO) f=35159 TT=2520.1 T1=2016.23	? (TO) TT=2520.07	? (TO) TT=2519.97	? (TO) TT=2520.07	SAT (TO) f=31154 TT=2400.27 T1=2.33	No data	No data	No data	No data
Rlfap/Rlfap-opt/ Rlfap-graph-06-opt_c18.xml	SAT f=41530 TT=2520.08 T1=62.46	No data	No data	No data	No data	SAT (TO) f=69941 TT=2400.11 T1=121.18	SAT (TO) f=69941 TT=2520.09 T1=122.2	SAT (TO) f=66580 TT=2520.13 T1=18.61	SAT (TO) f=74144 TT=2520.07 T1=28.94	SAT (TO) f=72477 TT=2520.06 T1=20.52	SAT (TO) f=66678 TT=2520.18 T1=32.07	SAT (TO) f=41530 TT=2520.08 T1=62.46	No data	No data	Signal TT=0.127	SAT (TO) f=151052 TT=2520.11 T1=2099.74	No data	No data	SAT (TO) f=182575 TT=2400.06 T1=1240.58	SAT (TO) f=175204 TT=2520.11 T1=1459.79	SAT (TO) f=166898 TT=2520.05 T1=451.31	SAT (TO) f=112921 TT=2400.09 T1=2255.73	SAT (TO) f=113744 TT=2520.05 T1=2167.27	? (TO) TT=2520.04	? (TO) TT=2520.03	? (TO) TT=2520.06	SAT (TO) f=49463 TT=2400.23 T1=4.93	No data	No data	No data	No data
Rlfap/Rlfap-opt/ Rlfap-graph-07-opt_c18.xml	SAT f=14479 TT=2520.05 T1=176.96	No data	No data	No data	No data	SAT (TO) f=14479 TT=2400.03 T1=176.12	SAT (TO) f=14479 TT=2520.05 T1=176.96	SAT (TO) f=43722 TT=2520.17 T1=16.85	SAT (TO) f=55632 TT=2520.13 T1=17.32	SAT (TO) f=67764 TT=2520.15 T1=15.88	SAT (TO) f=44215 TT=2520.08 T1=16.55	SAT (TO) f=27304 TT=2520.05 T1=612.79	No data	No data	Signal TT=0.094	SAT (TO) f=100238 TT=2519.89 T1=1364.79	No data	No data	SAT (TO) f=162991 TT=2400.12 T1=518.8	SAT (TO) f=168802 TT=2520.11 T1=207.57	SAT (TO) f=168307 TT=2520.12 T1=223.2	SAT (TO) f=101845 TT=2400.12 T1=2172.5	SAT (TO) f=107644 TT=2520.09 T1=2438.17	? (TO) TT=2520.06	? (TO) TT=2519.95	? (TO) TT=2520.1	SAT (TO) f=25976 TT=2400.18 T1=4.43	No data	No data	No data	No data
Rlfap/Rlfap-opt/ Rlfap-graph-08-opt_c18.xml	SAT f=18 TT=2520.14 T1=1736.09	No data	No data	No data	No data	SAT (TO) f=22 TT=2400.02 T1=283.1	SAT (TO) f=22 TT=2520.11 T1=288.52	SAT (TO) f=18 TT=2520.14 T1=1736.09	SAT (TO) f=20 TT=2520.14 T1=617.51	SAT (TO) f=18 TT=2520.15 T1=945.37	SAT (TO) f=20 TT=2520.18 T1=560.51	SAT (TO) f=20 TT=2520.08 T1=47.96	No data	No data	SAT (TO) f=48 TT=2400.05 T1=2400.1	SAT (TO) f=48 TT=2519.86 T1=2520	No data	No data	SAT (TO) f=48 TT=2400.06 T1=2365.2	SAT (TO) f=48 TT=2520.04 T1=251.69	SAT (TO) f=48 TT=2520.04 T1=496.83	Wrong Opt. f=48 TT=5.89 T1=2.35	SAT (TO) f=46 TT=2520.11 T1=57.76	? (TO) TT=2519.93	? (TO) TT=2519.78	? (TO) TT=2519.92	? TT=747.689	No data	No data	No data	No data
Rlfap/Rlfap-opt/ Rlfap-graph-09-opt_c18.xml	SAT f=18 TT=2520.08 T1=470.35	No data	No data	No data	No data	SAT (TO) f=22 TT=2400.03 T1=1169.66	SAT (TO) f=22 TT=2520.07 T1=1178.78	SAT (TO) f=18 TT=2520.12 T1=1821.58	SAT (TO) f=20 TT=2520.13 T1=2104.08	SAT (TO) f=18 TT=2520.17 T1=1321.52	SAT (TO) f=18 TT=2520.18 T1=1139.86	SAT (TO) f=18 TT=2520.08 T1=470.35	No data	No data	? (TO) TT=2400.01	? (TO) TT=2520.01	No data	No data	SAT (TO) f=46 TT=2400.12 T1=2361.81	? (TO) TT=2520.08	? (TO) TT=2520.05	Wrong Opt. f=46 TT=6.864 T1=2.67	SAT (TO) f=46 TT=2520.04 T1=177.24	? (TO) TT=2520.08	? (TO) TT=2520.11	? (TO) TT=2520.1	? TT=1882.84	No data	No data	No data	No data
Rlfap/Rlfap-opt/ Rlfap-graph-10-opt_c18.xml	OPT f=394 TT=11.858 T1=5.41	No data	No data	No data	No data	OPT f=394 TT=13.83 T1=5.88	OPT f=394 TT=11.858 T1=5.41	OPT f=394 TT=52.332 T1=34.28	OPT f=394 TT=59.715 T1=37.66	OPT f=394 TT=57.438 T1=38.64	OPT f=394 TT=60.955 T1=40.34	OPT f=394 TT=20.231 T1=20.22	No data	No data	OPT f=394 TT=5.893 T1=5.84	OPT f=394 TT=5.901 T1=5.85	No data	No data	? (TO) TT=2400.08	? (TO) TT=2520.06	? (TO) TT=2520.08	? (TO) TT=2400.05	? (TO) TT=2520.11	OPT f=394 TT=393.303 T1=393.27	OPT f=394 TT=417.829 T1=417.83	OPT f=394 TT=417.495 T1=417.52	SAT (TO) f=792 TT=2400.54 T1=108.97	No data	No data	No data	No data
Rlfap/Rlfap-opt/ Rlfap-graph-11-opt_c18.xml	SAT f=101994 TT=2400.14 T1=7.98	No data	No data	No data	No data	SAT (TO) f=109939 TT=2400.12 T1=474.72	SAT (TO) f=109939 TT=2520.06 T1=471.81	SAT (TO) f=116650 TT=2520.09 T1=57.64	SAT (TO) f=135618 TT=2520.38 T1=54.89	SAT (TO) f=125709 TT=2520.07 T1=34.63	SAT (TO) f=135077 TT=2520.18 T1=67.99	SAT (TO) f=138606 TT=2519.92 T1=805.61	No data	No data	Signal TT=0.217	SAT (TO) f=309439 TT=2519.76 T1=1079.41	No data	No data	SAT (TO) f=309345 TT=2400.1 T1=2174.22	SAT (TO) f=311901 TT=2520.11 T1=2404.56	? (TO) TT=2520.12	SAT (TO) f=227490 TT=2401.27 T1=2342.19	SAT (TO) f=232059 TT=2520.11 T1=2462.37	? (TO) TT=2520.08	? (TO) TT=2520.04	? (TO) TT=2519.92	SAT (TO) f=101994 TT=2400.14 T1=7.98	No data	No data	No data	No data
Rlfap/Rlfap-opt/ Rlfap-graph-12-opt_c18.xml	SAT f=21762 TT=2400.18 T1=9.37	No data	No data	No data	No data	SAT (TO) f=67108 TT=2400.08 T1=412.24	SAT (TO) f=67108 TT=2520.05 T1=412.63	SAT (TO) f=133151 TT=2520.14 T1=35.98	SAT (TO) f=109768 TT=2520.25 T1=43.32	SAT (TO) f=106266 TT=2520.12 T1=49.57	SAT (TO) f=124840 TT=2520.11 T1=45.35	SAT (TO) f=66810 TT=2520.02 T1=2118.7	No data	No data	Signal TT=0.159	SAT (TO) f=258385 TT=2519.65 T1=876.93	No data	No data	SAT (TO) f=354730 TT=2400.44 T1=1030.69	SAT (TO) f=380148 TT=2520.1 T1=955.51	SAT (TO) f=354960 TT=2520.1 T1=2417	SAT (TO) f=295440 TT=2400.15 T1=2334.04	SAT (TO) f=279879 TT=2520.07 T1=2445.85	? (TO) TT=2520.05	? (TO) TT=2520.08	? (TO) TT=2519.95	SAT (TO) f=21762 TT=2400.18 T1=9.37	No data	No data	No data	No data
Rlfap/Rlfap-opt/ Rlfap-graph-13-opt_c18.xml	SAT f=135769 TT=2400.21 T1=9.63	No data	No data	No data	No data	SAT (TO) f=162439 TT=2400.05 T1=486.75	SAT (TO) f=162439 TT=2520.04 T1=480.42	SAT (TO) f=183949 TT=2520.14 T1=114.4	SAT f=181646 TT=2175.55 T1=241.96	SAT (TO) f=188698 TT=2520.14 T1=111.33	? (TO) TT=2528.17	SAT (TO) f=250884 TT=2520.04 T1=763	No data	No data	Signal TT=0.302	SAT (TO) f=502955 TT=2519.99 T1=2120.98	No data	No data	? (TO) TT=2400.1	? (TO) TT=2520.1	? (TO) TT=2520.03	? (TO) TT=2400.05	? (TO) TT=2520.12	? (TO) TT=2519.92	? (TO) TT=2520.1	? (TO) TT=2519.78	SAT (TO) f=135769 TT=2400.21 T1=9.63	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
Rlfap/Rlfap-opt/ Rlfap-graph-14-opt_c18.xml	SAT f=12 TT=2520.09 T1=1305.38	No data	No data	No data	No data	SAT (TO) f=14 TT=2400.06 T1=687.38	SAT (TO) f=12 TT=2520.11 T1=2502.44	SAT (TO) f=12 TT=2520.22 T1=1323.12	SAT (TO) f=12 TT=2520.09 T1=1305.38	SAT (TO) f=16 TT=2520.12 T1=1136.27	SAT (TO) f=10 TT=2520.13 T1=2279.9	SAT (TO) f=14 TT=2520.09 T1=37.36	No data	No data	? (TO) TT=2400.04	? (TO) TT=2520.01	No data	No data	SAT (TO) f=44 TT=2400.06 T1=2364.6	SAT (TO) f=44 TT=2520.03 T1=919.83	? (TO) TT=2520.05	Wrong Opt. f=30 TT=6.346 T1=2.51	SAT (TO) f=40 TT=2520.09 T1=134.45	? (TO) TT=2519.86	? (TO) TT=2519.94	? (TO) TT=2519.94	? TT=1814.55	No data	No data	No data	No data
Rlfap/Rlfap-opt/ Rlfap-scen-01-opt_c18.xml	SAT f=16 TT=2520.08 T1=44.57	No data	No data	No data	No data	SAT (TO) f=16 TT=2400.12 T1=1873.38	SAT (TO) f=16 TT=2520.13 T1=1865.89	SAT (TO) f=16 TT=2520.36 T1=2042.81	SAT (TO) f=16 TT=2520.27 T1=1436.26	SAT (TO) f=16 TT=2520.15 T1=1393.82	SAT (TO) f=16 TT=2520.14 T1=2303.71	SAT (TO) f=16 TT=2520.08 T1=44.57	No data	No data	? (TO) TT=2400.03	? (TO) TT=2520.11	No data	No data	SAT (TO) f=46 TT=2400.07 T1=2364.6	SAT (TO) f=46 TT=2520.07 T1=396.56	SAT (TO) f=48 TT=2520.12 T1=267.58	Wrong Opt. f=42 TT=7.088 T1=2.86	SAT (TO) f=46 TT=2520.1 T1=57.4	? (TO) TT=2520.1	? (TO) TT=2519.99	? (TO) TT=2519.86	? TT=1766.94	No data	No data	No data	No data
Rlfap/Rlfap-opt/ Rlfap-scen-02-opt_c18.xml	SAT f=14 TT=2519.65 T1=4.74	No data	No data	No data	No data	SAT (TO) f=14 TT=2400.04 T1=28.74	SAT (TO) f=14 TT=2520.12 T1=30.75	SAT (TO) f=14 TT=2520.21 T1=48.69	SAT (TO) f=14 TT=2520.07 T1=50.9	SAT (TO) f=14 TT=2520.08 T1=211.26	SAT (TO) f=14 TT=2520.1 T1=73.17	SAT (TO) f=14 TT=2519.65 T1=4.74	No data	No data	SAT (TO) f=28 TT=2400.01 T1=2400.3	SAT (TO) f=28 TT=2520.05 T1=2520	No data	No data	SAT (TO) f=44 TT=2400.12 T1=2369.01	SAT (TO) f=42 TT=2520.08 T1=19.03	SAT (TO) f=44 TT=2520.08 T1=4.62	Wrong Opt. f=28 TT=4.059 T1=1.69	SAT (TO) f=40 TT=2520.08 T1=4.16	? (TO) TT=2519.88	? (TO) TT=2520.06	? (TO) TT=2520.06	SAT (TO) f=38 TT=2400.36 T1=43.43	No data	No data	No data	No data
Rlfap/Rlfap-opt/ Rlfap-scen-03-opt_c18.xml	SAT f=14 TT=2520.06 T1=277.26	No data	No data	No data	No data	SAT (TO) f=16 TT=2400.02 T1=455.24	SAT (TO) f=16 TT=2520.07 T1=451.81	SAT (TO) f=14 TT=2520.19 T1=931.71	SAT (TO) f=14 TT=2520.31 T1=456.45	SAT (TO) f=14 TT=2520.2 T1=266.7	SAT (TO) f=16 TT=2520.16 T1=839.31	SAT (TO) f=14 TT=2520.06 T1=277.26	No data	No data	SAT (TO) f=34 TT=2400.05 T1=2400.1	SAT (TO) f=34 TT=2519.98 T1=2520	No data	No data	SAT (TO) f=46 TT=2400.12 T1=2366.6	SAT (TO) f=46 TT=2520.12 T1=123.96	SAT (TO) f=46 TT=2520.05 T1=131.81	Wrong Opt. f=40 TT=5.127 T1=2.02	? (TO) TT=2520.04	? (TO) TT=2520.09	? (TO) TT=2520.09	? (TO) TT=2519.69	? TT=1299.82	No data	No data	No data	No data
Rlfap/Rlfap-opt/ Rlfap-scen-04-opt_c18.xml	OPT f=46 TT=2.112 T1=2.1	No data	No data	No data	No data	OPT f=46 TT=24.714 T1=18.44	OPT f=46 TT=24.817 T1=18.52	OPT f=46 TT=29.525 T1=13.05	OPT f=46 TT=29.433 T1=13.11	OPT f=46 TT=30.06 T1=13.25	OPT f=46 TT=29.132 T1=12.92	OPT f=46 TT=2.112 T1=2.1	No data	No data	SAT (TO) f=46 TT=2400.08 T1=2400.2	SAT (TO) f=46 TT=2520.04 T1=2520	No data	No data	SAT (TO) f=46 TT=2400.11 T1=2367.4	SAT (TO) f=46 TT=2520.12 T1=1202.62	OPT f=46 TT=1095.85 T1=1090.86	OPT f=46 TT=7.149 T1=2.69	? (TO) TT=2520.05	OPT f=46 TT=48.617 T1=48.61	OPT f=46 TT=51.599 T1=51.6	OPT f=46 TT=51.886 T1=51.89	? (TO) TT=2408	No data	No data	No data	No data
Rlfap/Rlfap-opt/ Rlfap-scen-05-opt_c18.xml	OPT f=792 TT=1.448 T1=1.44	No data	No data	No data	No data	OPT f=792 TT=4.755 T1=1.61	OPT f=792 TT=4.434 T1=1.5	OPT f=792 TT=21.249 T1=8.09	OPT f=792 TT=22.19 T1=8.36	OPT f=792 TT=21.572 T1=8.52	OPT f=792 TT=22.425 T1=8.71	OPT f=792 TT=1.448 T1=1.44	No data	No data	OPT f=792 TT=1.501 T1=1.47	OPT f=792 TT=1.494 T1=1.47	No data	No data	SAT (TO) f=792 TT=2400.05 T1=9.19	SAT (TO) f=792 TT=2520.07 T1=14.09	OPT f=792 TT=19.513 T1=14.11	OPT f=792 TT=12.138 T1=6.32	OPT f=792 TT=11.506 T1=6.15	OPT f=792 TT=264.381 T1=264.39	OPT f=792 TT=278.906 T1=278.89	OPT f=792 TT=280.128 T1=280.1	OPT f=792 TT=39.487 T1=5.63	No data	No data	No data	No data
Rlfap/Rlfap-opt/ Rlfap-scen-06-opt_c18.xml	SAT f=10458 TT=2400.27 T1=288	No data	No data	No data	No data	SAT (TO) f=34635 TT=2400.02 T1=148.26	SAT (TO) f=34635 TT=2520.12 T1=148.15	SAT (TO) f=29749 TT=2520.08 T1=14.96	SAT (TO) f=37880 TT=2520.2 T1=24.26	SAT (TO) f=27294 TT=2520.22 T1=18.57	SAT (TO) f=27280 TT=2520.25 T1=17.91	SAT (TO) f=14038 TT=2520.04 T1=673.23	No data	No data	Signal TT=0.072	SAT (TO) f=33560 TT=2519.89 T1=2459.68	No data	No data	SAT (TO) f=85326 TT=2400.09 T1=659.65	SAT (TO) f=78342 TT=2520.08 T1=2493.55	SAT (TO) f=83876 TT=2520.1 T1=430.92	SAT (TO) f=21768 TT=2400.05 T1=1669.89	SAT (TO) f=28280 TT=2520.09 T1=2451.48	? (TO) TT=2519.92	? (TO) TT=2520.07	? (TO) TT=2519.97	SAT (TO) f=10458 TT=2400.27 T1=288	No data	No data	No data	No data
Rlfap/Rlfap-opt/ Rlfap-scen-07-opt_c18.xml	SAT f=5249896 TT=2400.22 T1=1939.74	No data	No data	No data	No data	SAT (TO) f=70314748 TT=2400.07 T1=1349.91	SAT (TO) f=70314748 TT=2520.03 T1=1382.07	SAT (TO) f=43944112 TT=2520.16 T1=52.88	SAT (TO) f=68247852 TT=2520.32 T1=98.77	SAT (TO) f=55644890 TT=2520.09 T1=59.81	SAT (TO) f=48997251 TT=2520.19 T1=99.83	SAT (TO) f=40901981 TT=2520.07 T1=1800.25	No data	No data	Signal TT=2.447	SAT (TO) f=120832600 TT=2519.75 T1=2341.35	No data	No data	SAT (TO) f=182404513 TT=2400.1 T1=1472.59	SAT (TO) f=185264734 TT=2520.13 T1=819.28	SAT (TO) f=191483866 TT=2520.16 T1=825.82	SAT (TO) f=87071194 TT=2400.13 T1=2169.36	SAT (TO) f=90810905 TT=2520.71 T1=2274.16	? (TO) TT=2520.05	? (TO) TT=2520.08	? (TO) TT=2520	SAT (TO) f=5249896 TT=2400.22 T1=1939.74	No data	No data	No data	No data
Rlfap/Rlfap-opt/ Rlfap-scen-08-opt_c18.xml	SAT f=1997 TT=2519.99 T1=650.07	No data	No data	No data	No data	SAT (TO) f=2273 TT=2400.12 T1=620.75	SAT (TO) f=2273 TT=2520.07 T1=618.33	SAT (TO) f=2087 TT=2520.12 T1=184.77	SAT (TO) f=2257 TT=2520.59 T1=482.84	SAT (TO) f=2458 TT=2520.16 T1=191.32	SAT (TO) f=2280 TT=2520.82 T1=607.24	SAT (TO) f=1997 TT=2519.99 T1=650.07	No data	No data	Signal TT=0.3	SAT (TO) f=4433 TT=2519.83 T1=1104.15	No data	No data	? (TO) TT=2400.12	? (TO) TT=2520.07	? (TO) TT=2520.11	SAT (TO) f=4903 TT=2400.15 T1=2331.98	SAT (TO) f=4722 TT=2520.1 T1=2470.04	? (TO) TT=2519.96	? (TO) TT=2519.8	? (TO) TT=2520.02	SAT (TO) f=2538 TT=2400.22 T1=17.24	No data	No data	No data	No data
Rlfap/Rlfap-opt/ Rlfap-scen-09-opt_c18.xml	SAT f=19049 TT=2400.17 T1=8.48	No data	No data	No data	No data	SAT (TO) f=37373 TT=2400.08 T1=10.37	SAT (TO) f=37373 TT=2520.04 T1=10.3	SAT (TO) f=70761 TT=2520.11 T1=37.86	SAT (TO) f=126058 TT=2520.12 T1=37.82	SAT (TO) f=101561 TT=2520.12 T1=32.4	SAT (TO) f=85230 TT=2520.18 T1=37.61	SAT (TO) f=33591 TT=2519.64 T1=1970.38	No data	No data	Signal TT=0.158	SAT (TO) f=77189 TT=2519.75 T1=1290.4	No data	No data	SAT (TO) f=294555 TT=2400.14 T1=1302.88	SAT (TO) f=316240 TT=2520.06 T1=2328.98	SAT (TO) f=330897 TT=2520.07 T1=2181.55	SAT (TO) f=229840 TT=2400.15 T1=2331.2	SAT (TO) f=250507 TT=2520.16 T1=2452.41	? (TO) TT=2519.91	? (TO) TT=2519.78	? (TO) TT=2520.06	SAT (TO) f=19049 TT=2400.17 T1=8.48	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
Rlfap/Rlfap-opt/ Rlfap-scen-10-opt_c18.xml	OPT f=31516 TT=654.472 T1=540.08	No data	No data	No data	No data	SAT (TO) f=107493 TT=2400.04 T1=518.4	SAT (TO) f=107493 TT=2520.04 T1=524.63	SAT (TO) f=1414674 TT=2520.09 T1=2120.7	SAT (TO) f=2626141 TT=2520.22 T1=1769.74	SAT (TO) f=2840075 TT=2520.07 T1=1299.06	SAT (TO) f=1226559 TT=2520.16 T1=1191.03	SAT (TO) f=415454 TT=2520 T1=2484.14	No data	No data	Signal TT=0.221	SAT (TO) f=196801 TT=2519.88 T1=2491.26	No data	No data	SAT (TO) f=10133272 TT=2400.12 T1=1290.49	SAT (TO) f=9135127 TT=2520.15 T1=412	SAT (TO) f=9402052 TT=2520.14 T1=861.16	SAT (TO) f=6695055 TT=2400.06 T1=2331.32	SAT (TO) f=6060752 TT=2520.1 T1=2486.44	? (TO) TT=2519.86	? (TO) TT=2519.6	? (TO) TT=2520.05	OPT f=31516 TT=654.472 T1=540.08	No data	No data	No data	No data
Rlfap/Rlfap-opt/ Rlfap-scen-11-opt_c18.xml	SAT f=22 TT=2519.96 T1=49.38	No data	No data	No data	No data	? (TO) TT=2400.08	? (TO) TT=2520.12	SAT (TO) f=26 TT=2520.15 T1=1488.9	SAT (TO) f=28 TT=2520.22 T1=2382.75	SAT (TO) f=22 TT=2520.23 T1=1868.01	SAT (TO) f=22 TT=2520.21 T1=711.9	SAT (TO) f=22 TT=2519.96 T1=49.38	No data	No data	SAT (TO) f=42 TT=2400.06 T1=2400.11	SAT (TO) f=42 TT=2519.91 T1=2520	No data	No data	SAT (TO) f=46 TT=2400.07 T1=2365.9	? (TO) TT=2520.02	? TT=2519.95	Wrong Opt. f=46 TT=18.065 T1=12.65	? (TO) TT=2520.07	? (TO) TT=2519.7	? (TO) TT=2520.06	? (TO) TT=2519.89	? TT=1020.69	No data	No data	No data	No data
SocialGolfers/ SocialGolfers-4-4-4-cp_c18.xml	SAT TT=0.169	? (NS) TT=0.147	? (NS) TT=0.152	? (NS) TT=0.151	? (NS) TT=0.153	SAT TT=2.571	SAT TT=2.6	SAT TT=13.38	SAT TT=13.308	SAT TT=12.608	SAT TT=12.685	SAT TT=0.407	? (NS) TT=0.149	? (NS) TT=0.149	SAT TT=0.354	SAT TT=0.37	SAT TT=3.046	SAT TT=2.884	SAT TT=3.014	SAT TT=3.04	SAT TT=2.913	No data	No data	SAT TT=0.175	SAT TT=0.175	SAT TT=0.169	SAT TT=2.721	SAT TT=6.665	SAT TT=7.689	SAT TT=6.532	SAT TT=7.84
SocialGolfers/ SocialGolfers-4-4-5-cp_c18.xml	SAT TT=0.255	? (NS) TT=0.248	? (NS) TT=0.255	? (NS) TT=0.251	? (NS) TT=0.257	SAT TT=3.858	SAT TT=3.834	SAT TT=15.827	SAT TT=15.469	SAT TT=14.789	SAT TT=15.05	SAT TT=1.124	? (NS) TT=0.248	? (NS) TT=0.25	SAT TT=0.61	SAT TT=0.606	SAT TT=4.393	SAT TT=4.056	SAT TT=3.917	SAT TT=3.69	SAT TT=3.69	No data	No data	SAT TT=0.27	SAT TT=0.272	SAT TT=0.255	SAT TT=4.271	SAT TT=8.077	SAT TT=9.107	SAT TT=8.08	SAT TT=9.191
SocialGolfers/ SocialGolfers-4-4-6-cp_c18.xml	UNSAT TT=0.372	? (NS) TT=0.381	? (NS) TT=0.389	? (NS) TT=0.38	? (NS) TT=0.391	UNSAT TT=194.51	UNSAT TT=199.584	UNSAT TT=487.382	UNSAT TT=427.942	UNSAT TT=556.159	? (TO) TT=2520.15	? (TO) TT=2519.79	? (NS) TT=0.378	? (NS) TT=0.382	UNSAT TT=30.831	UNSAT TT=32.413	UNSAT TT=181.258	UNSAT TT=182.543	? (TO) TT=2400.08	? (TO) TT=2520.04	UNSAT TT=163.715	No data	No data	UNSAT TT=0.372	UNSAT TT=0.379	UNSAT TT=0.372	UNSAT TT=10.277	UNSAT TT=9.724	UNSAT TT=10.409	UNSAT TT=9.746	UNSAT TT=10.276
SocialGolfers/ SocialGolfers-4-4-7-cp_c18.xml	UNSAT TT=0.474	? (NS) TT=0.54	? (NS) TT=0.557	? (NS) TT=0.548	? (NS) TT=0.558	UNSAT TT=1155.38	UNSAT TT=1139.22	UNSAT TT=1985.42	UNSAT TT=1526.07	UNSAT TT=58.519	UNSAT TT=217.194	? (TO) TT=2519.78	? (NS) TT=0.542	? (NS) TT=0.544	UNSAT TT=40.4	UNSAT TT=43.035	UNSAT TT=35.721	UNSAT TT=36.31	? (TO) TT=2400.07	? (TO) TT=2520.06	UNSAT TT=110.266	No data	No data	UNSAT TT=0.491	UNSAT TT=0.493	UNSAT TT=0.474	UNSAT TT=7.958	UNSAT TT=10.379	UNSAT TT=11.223	UNSAT TT=10.553	UNSAT TT=11.344
SocialGolfers/ SocialGolfers-5-5-5-cp_c18.xml	SAT TT=1.234	? (NS) TT=0.653	? (NS) TT=0.672	? (NS) TT=0.655	? (NS) TT=0.677	SAT TT=7.372	SAT TT=6.696	SAT TT=22.606	SAT TT=21.401	SAT TT=21.452	SAT TT=20.209	SAT TT=6.048	? (NS) TT=0.656	? (NS) TT=0.66	SAT TT=1.523	SAT TT=1.539	SAT TT=4.948	SAT TT=4.869	SAT TT=4.766	SAT TT=4.636	SAT TT=4.657	No data	No data	SAT TT=1.34	SAT TT=1.347	SAT TT=1.234	SAT TT=6.048	SAT TT=12.708	SAT TT=13.676	SAT TT=12.383	SAT TT=13.713
SocialGolfers/ SocialGolfers-5-5-6-cp_c18.xml	SAT TT=1.626	? (NS) TT=1.028	? (NS) TT=1.056	? (NS) TT=1.029	? (NS) TT=1.06	SAT TT=12.599	SAT TT=12.157	SAT TT=22.006	SAT TT=26.825	SAT TT=27.082	SAT TT=22.864	SAT TT=11.892	? (NS) TT=1.038	? (NS) TT=1.035	SAT TT=2.367	SAT TT=2.512	SAT TT=7.64	SAT TT=7.675	SAT TT=5.415	SAT TT=5.241	SAT TT=5.108	No data	No data	SAT TT=1.722	SAT TT=1.703	SAT TT=1.626	SAT TT=9.55	SAT TT=14.814	SAT TT=16.518	SAT TT=14.796	SAT TT=15.592
SocialGolfers/ SocialGolfers-5-5-7-cp_c18.xml	UNSAT TT=4.553	? (NS) TT=1.518	? (NS) TT=1.553	? (NS) TT=1.53	? (NS) TT=1.545	? (TO) TT=2400.07	? (TO) TT=2520.04	? (TO) TT=2520.16	? (TO) TT=2520.16	? (TO) TT=2520.15	? (TO) TT=2520.09	? (TO) TT=2519.78	? (NS) TT=1.52	? (NS) TT=1.528	? (TO) TT=2400.04	? (TO) TT=2520.01	? (TO) TT=2400.08	? (TO) TT=2520.07	? (TO) TT=2400.03	? (TO) TT=2520.06	? (TO) TT=2520.08	No data	No data	UNSAT TT=4.566	UNSAT TT=4.553	UNSAT TT=11.198	UNSAT TT=22.043	UNSAT TT=64.542	UNSAT TT=24.538	UNSAT TT=63.7	UNSAT TT=24.18
SocialGolfers/ SocialGolfers-5-5-8-cp_c18.xml	UNSAT TT=7.371	? (NS) TT=2.238	? (NS) TT=2.202	? (NS) TT=2.152	? (NS) TT=2.212	? (TO) TT=2400.09	? (TO) TT=2520.1	? (TO) TT=2520.12	? (TO) TT=2520.11	? (TO) TT=2520.09	? (TO) TT=2520.09	? (TO) TT=2519.8	? (NS) TT=2.219	? (NS) TT=2.153	? (TO) TT=2400.04	? (TO) TT=2520.02	? (TO) TT=2400.09	? (TO) TT=2520.02	? (TO) TT=2400.06	? (TO) TT=2520.08	? (TO) TT=2520.11	No data	No data	UNSAT TT=7.686	UNSAT TT=7.589	UNSAT TT=7.371	UNSAT TT=268.754	UNSAT TT=34.645	UNSAT TT=19.831	UNSAT TT=34.665	UNSAT TT=19.985
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
SocialGolfers/ SocialGolfers-6-6-6-cp_c18.xml		? (NS) TT=2.493	? (NS) TT=2.551	? (NS) TT=2.526	? (NS) TT=2.564	? (TO) TT=2400.3	? (TO) TT=2520.06	? (TO) TT=2520.1	? (TO) TT=2520.04	? (TO) TT=2520.14	? (TO) TT=2520.1	? (TO) TT=2519.77	? (NS) TT=2.501	? (NS) TT=2.505	? (TO) TT=2400.1	? (TO) TT=2520.06	? (TO) TT=2400.12	? (TO) TT=2520.11	? (TO) TT=2400.08	? (TO) TT=2520.07	? (TO) TT=2520.07	No data	No data	? (TO) TT=2519.95	? (TO) TT=2520.03	? (TO) TT=2519.91	? (TO) TT=2400.14	? (TO) TT=2520.12	? (TO) TT=2520.14	? (TO) TT=2520.11	? (TO) TT=2520.14
SocialGolfers/ SocialGolfers-6-6-7-cp_c18.xml		? (NS) TT=3.812	? (NS) TT=3.896	? (NS) TT=4.041	? (NS) TT=3.89	? (TO) TT=2400.07	? (TO) TT=2520.06	? (TO) TT=2520.12	? (TO) TT=2520.07	? (TO) TT=2520.09	? (TO) TT=2520.14	? (TO) TT=2519.78	? (NS) TT=3.807	? (NS) TT=3.833	? (TO) TT=2400.1	? (TO) TT=2519.73	? (TO) TT=2400.07	? (TO) TT=2520.04	? (TO) TT=2400.08	? (TO) TT=2520.11	? (TO) TT=2520.08	No data	No data	? (TO) TT=2519.89	? (TO) TT=2520.08	? (TO) TT=2520.04	? (TO) TT=2400.19	? (TO) TT=2520.11	? (TO) TT=2520.1	? (TO) TT=2520.1	? (TO) TT=2520.13
SocialGolfers/ SocialGolfers-6-6-8-cp_c18.xml		? (NS) TT=5.637	? (NS) TT=5.75	? (NS) TT=5.779	? (NS) TT=5.825	? (TO) TT=2400.07	? (TO) TT=2520.06	? (TO) TT=2520.13	? (TO) TT=2520.11	? (TO) TT=2520.15	? (TO) TT=2520.1	? (TO) TT=2519.76	? (NS) TT=5.691	? (NS) TT=5.654	? (TO) TT=2400.04	? (TO) TT=2519.82	? (TO) TT=2400.12	? (TO) TT=2520.08	? (TO) TT=2400.09	? (TO) TT=2520.07	? (TO) TT=2520.04	No data	No data	? (TO) TT=2519.8	? (TO) TT=2520.06	? (TO) TT=2519.92	? (TO) TT=2400.12	? (TO) TT=2520.08	? (TO) TT=2520.09	? (MO) TT=25.03	? (TO) TT=2520.09
SocialGolfers/ SocialGolfers-6-6-9-cp_c18.xml		? (NS) TT=8.092	? (NS) TT=8.399	? (NS) TT=8.265	? (NS) TT=8.454	? (TO) TT=2400.12	? (TO) TT=2520.05	? (TO) TT=2520.07	? (TO) TT=2520.13	? (TO) TT=2520.1	? (TO) TT=2520.11	? (TO) TT=2519.79	? (NS) TT=8.14	? (NS) TT=8.328	? (TO) TT=2400	? (TO) TT=2519.58	? (TO) TT=2400.12	? (TO) TT=2520.1	? (TO) TT=2400.12	? (TO) TT=2520.11	? (TO) TT=2520.08	No data	No data	? (TO) TT=2519.79	? (TO) TT=2520.09	? (TO) TT=2520	? (TO) TT=2400.12	? (TO) TT=2520.06	? (TO) TT=2520.13	? (TO) TT=2520.09	? (TO) TT=2520.1
SportsScheduling/ SportsScheduling-08_c18.xml	SAT TT=0.02	? (NS) TT=0.007	? (NS) TT=0.007	? (NS) TT=0.007	? (NS) TT=0.007	SAT TT=1.88	SAT TT=1.892	SAT TT=8.583	SAT TT=9.087	SAT TT=8.492	SAT TT=9.385	SAT TT=0.039	? (NS) TT=0.007	? (NS) TT=0.007	SAT TT=0.02	SAT TT=0.022	SAT TT=2.078	SAT TT=2.019	SAT TT=2.155	SAT TT=2.054	SAT TT=2.057	No data	No data	SAT TT=1.773	SAT TT=1.845	SAT TT=1.853	SAT TT=24.529	SAT TT=9.012	SAT TT=10.455	SAT TT=8.837	SAT TT=10.42
SportsScheduling/ SportsScheduling-10_c18.xml	SAT TT=0.052	? (NS) TT=0.008	? (NS) TT=0.009	? (NS) TT=0.009	? (NS) TT=0.009	SAT TT=3.116	SAT TT=3.164	SAT TT=13.653	SAT TT=13.567	SAT TT=14.215	SAT TT=19.869	SAT TT=27.498	? (NS) TT=0.009	? (NS) TT=0.009	SAT TT=0.094	SAT TT=0.052	SAT TT=2.319	SAT TT=2.238	SAT TT=7.834	SAT TT=7.74	SAT TT=7.755	No data	No data	SAT TT=3.598	SAT TT=6.067	SAT TT=6.071	SAT TT=1267.85	SAT TT=14.933	SAT TT=15.744	SAT TT=14.728	SAT TT=16.578
SportsScheduling/ SportsScheduling-12_c18.xml	SAT TT=1.545	? (NS) TT=0.011	? (NS) TT=0.011	? (NS) TT=0.011	? (NS) TT=0.011	SAT TT=11.157	SAT TT=10.52	SAT TT=39.917	SAT TT=41.18	SAT TT=125.286	SAT TT=60.07	SAT TT=948.179	? (NS) TT=0.011	? (NS) TT=0.011	SAT TT=1.545	SAT TT=2.794	SAT TT=5.901	SAT TT=5.36	SAT TT=13.683	SAT TT=23.003	SAT TT=13.744	No data	No data	SAT TT=94.998	SAT TT=76.117	SAT TT=140.906	? (TO) TT=2400.22	SAT TT=53.526	SAT TT=120.565	SAT TT=53.867	SAT TT=118.629
SportsScheduling/ SportsScheduling-14_c18.xml	SAT TT=49.912	? (NS) TT=0.014	? (NS) TT=0.014	? (NS) TT=0.014	? (NS) TT=0.014	? (TO) TT=2400.04	? (TO) TT=2520.41	SAT TT=217.012	? (TO) TT=2520.13	? (TO) TT=2520.14	? (TO) TT=2520.05	? (TO) TT=2520.08	? (NS) TT=0.014	? (NS) TT=0.015	SAT TT=50.189	SAT TT=49.912	SAT TT=1224.02	SAT TT=1198.05	SAT TT=98.719	SAT TT=114.747	SAT TT=201.721	No data	No data	SAT TT=408.272	SAT TT=172.754	SAT TT=369.601	? (TO) TT=2400.15	? (MO) TT=459.575	SAT TT=1845.37	? (MO) TT=463.24	SAT TT=1842.15
SportsScheduling/ SportsScheduling-16_c18.xml	SAT TT=321.729	? (NS) TT=0.018	? (NS) TT=0.019	? (NS) TT=0.019	? (NS) TT=0.019	? (TO) TT=2400.1	? (TO) TT=2520.12	? (TO) TT=2520.12	? (TO) TT=2520.06	? (TO) TT=2520.12	? (TO) TT=2520.1	? (TO) TT=2519.79	? (NS) TT=0.019	? (NS) TT=0.019	? (TO) TT=2400.08	? (TO) TT=2520.09	SAT TT=321.729	SAT TT=326.745	? (TO) TT=2400.02	? (TO) TT=2520.09	? (TO) TT=2520.09	No data	No data	? (TO) TT=2519.98	? (TO) TT=2519.93	? (TO) TT=2519.79	? (TO) TT=2400.23	? (MO) TT=35.394	? (TO) TT=2520.13	? (MO) TT=34.89	? (TO) TT=2520.13
SportsScheduling/ SportsScheduling-18_c18.xml		? (NS) TT=0.028	? (NS) TT=0.028	? (NS) TT=0.029	? (NS) TT=0.028	? (TO) TT=2400.05	? (TO) TT=2520.04	? (TO) TT=2520.04	? (TO) TT=2520.06	? (TO) TT=2520.06	? (TO) TT=2520.1	? (TO) TT=2519.75	? (NS) TT=0.028	? (NS) TT=0.028	? (TO) TT=2400.04	? (TO) TT=2519.56	? (TO) TT=2400.05	? (TO) TT=2520.07	? (TO) TT=2400.03	? (TO) TT=2520.05	? (TO) TT=2520.1	No data	No data	? (TO) TT=2520.07	? (TO) TT=2520.07	? (TO) TT=2520.02	? (TO) TT=2400.22	? (MO) TT=48.152	? (TO) TT=2520.14	? (MO) TT=48.328	? (TO) TT=2520.14
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
SportsScheduling/ SportsScheduling-20_c18.xml		? (NS) TT=0.039	? (NS) TT=0.039	? (NS) TT=0.04	? (NS) TT=0.039	? (TO) TT=2400.08	? (TO) TT=2520.03	? (TO) TT=2520.05	? (TO) TT=2520.08	? (TO) TT=2520.08	? (TO) TT=2520.06	? (TO) TT=2520.07	? (NS) TT=0.039	? (NS) TT=0.039	? (TO) TT=2400.09	? (TO) TT=2519.91	? (TO) TT=2400.02	? (TO) TT=2520.03	? (TO) TT=2400.03	? (TO) TT=2520.08	? (TO) TT=2520.06	No data	No data	? (TO) TT=2519.91	? (TO) TT=2519.94	? (TO) TT=2519.9	? (TO) TT=2400.13	? (MO) TT=66.581	? (TO) TT=2520.16	? (MO) TT=64.362	? (TO) TT=2520.18
SportsScheduling/ SportsScheduling-22_c18.xml		? (NS) TT=0.054	? (NS) TT=0.055	? (NS) TT=0.056	? (NS) TT=0.055	? (TO) TT=2400.08	? (TO) TT=2520.08	? (TO) TT=2520.12	? (TO) TT=2520.19	? (TO) TT=2520.61	? (TO) TT=2520.11	? (TO) TT=2519.79	? (NS) TT=0.054	? (NS) TT=0.055	? (TO) TT=2400.08	? (TO) TT=2520.05	? (TO) TT=2400.09	? (TO) TT=2520.04	? (TO) TT=2400.05	? (TO) TT=2520.08	? (TO) TT=2520.03	No data	No data	? (TO) TT=2520.01	? (TO) TT=2519.89	? (TO) TT=2520.05	? (TO) TT=2400.18	? (MO) TT=89.74	? (TO) TT=2520.17	? (MO) TT=88.176	? (TO) TT=2520.15
SportsScheduling/ SportsScheduling-24_c18.xml		? (NS) TT=0.082	? (NS) TT=0.084	? (NS) TT=0.084	? (NS) TT=0.084	? (TO) TT=2400.09	? (TO) TT=2520.1	? (TO) TT=2520.05	? (TO) TT=2520.07	? (TO) TT=2520.05	? (TO) TT=2520.16	? (TO) TT=2519.81	? (NS) TT=0.082	? (NS) TT=0.084	? (TO) TT=2400.07	? (TO) TT=2519.89	? (TO) TT=2400.09	? (TO) TT=2520.1	? (TO) TT=2400.09	? (TO) TT=2520.08	? (TO) TT=2520.11	No data	No data	? (TO) TT=2520.06	? (TO) TT=2519.58	? (TO) TT=2519.98	? (TO) TT=2400.18	? (MO) TT=122.534	? (TO) TT=2520.28	? (MO) TT=124.901	? (TO) TT=2520.22
SportsScheduling/ SportsScheduling-26_c18.xml		? (NS) TT=0.113	? (NS) TT=0.115	? (NS) TT=0.114	? (NS) TT=0.115	? (TO) TT=2400.07	? (TO) TT=2520.11	? (TO) TT=2520.05	? (TO) TT=2520.15	? (TO) TT=2520.06	? (TO) TT=2520.08	? (TO) TT=2520.06	? (NS) TT=0.113	? (NS) TT=0.114	? (TO) TT=2400.08	? (TO) TT=2519.74	? (TO) TT=2400.05	? (TO) TT=2520.09	? (TO) TT=2400.12	? (TO) TT=2520.11	? (TO) TT=2520.1	No data	No data	? (TO) TT=2519.96	? (TO) TT=2519.98	? (TO) TT=2520.03	? (TO) TT=2400.19	? (MO) TT=175.206	? (TO) TT=2520.38	? (MO) TT=173.099	? (TO) TT=2520.35
SteelMillSlab/ SteelMillSlab-m1-2-0_c18.xml		No data	No data	No data	No data	? (TO) TT=2400.13	? (TO) TT=2520.08	? (TO) TT=2520.21	? (TO) TT=2520.13	? (TO) TT=2520.2	? (TO) TT=2520.14	? (NS) TT=0.02	No data	No data	? (NS) TT=0.053	? (NS) TT=0.054	No data	No data	? (TO) TT=2400.1	? (TO) TT=2520.12	? (TO) TT=2520.05	? (TO) TT=2400.1	? (TO) TT=2520.12	? (TO) TT=2520.03	? (TO) TT=2520.03	? (TO) TT=2519.89	? TT=1733.02	No data	No data	No data	No data
SteelMillSlab/ SteelMillSlab-m1-2-2_c18.xml		No data	No data	No data	No data	? (TO) TT=2400.12	? (TO) TT=2520.08	? (TO) TT=2520.07	? (TO) TT=2520.2	? (TO) TT=2520.07	? (TO) TT=2520.11	? (NS) TT=0.02	No data	No data	? (NS) TT=0.054	? (NS) TT=0.053	No data	No data	? (TO) TT=2400.07	? (TO) TT=2520.05	? (TO) TT=2520.1	? (TO) TT=2400.08	? (TO) TT=2520.06	? (TO) TT=2519.95	? (TO) TT=2519.94	? (TO) TT=2520.06	? TT=1643.89	No data	No data	No data	No data
SteelMillSlab/ SteelMillSlab-m1-3-0_c18.xml		No data	No data	No data	No data	? (TO) TT=2400.06	? (TO) TT=2520.09	? (TO) TT=2520.1	? (TO) TT=2521.23	? (TO) TT=2520.12	? (TO) TT=2520.22	? (NS) TT=0.02	No data	No data	? (NS) TT=0.053	? (NS) TT=0.054	No data	No data	? (TO) TT=2400.06	? (TO) TT=2520.1	? (TO) TT=2520.03	? (TO) TT=2400.04	? (TO) TT=2520.1	? (TO) TT=2520.06	? (TO) TT=2520.05	? (TO) TT=2520.06	? TT=1661.09	No data	No data	No data	No data
SteelMillSlab/ SteelMillSlab-m1-3-2_c18.xml		No data	No data	No data	No data	? (TO) TT=2400.12	? (TO) TT=2520.04	? (TO) TT=2520.05	? (TO) TT=2520.13	? (TO) TT=2520.14	? (TO) TT=2520.1	? (NS) TT=0.02	No data	No data	? (NS) TT=0.054	? (NS) TT=0.052	No data	No data	? (TO) TT=2400.1	? (TO) TT=2520.11	? (TO) TT=2520.06	? (TO) TT=2400.02	? (TO) TT=2520.1	? (TO) TT=2520.07	? (TO) TT=2520.07	? (TO) TT=2520.04	? TT=1632.48	No data	No data	No data	No data
SteelMillSlab/ SteelMillSlab-m1-4-0_c18.xml		No data	No data	No data	No data	? (TO) TT=2400.1	? (TO) TT=2520.07	? (TO) TT=2520.13	? (TO) TT=2520.36	? (TO) TT=2520.09	? (TO) TT=2520.08	? (NS) TT=0.021	No data	No data	? (NS) TT=0.054	? (NS) TT=0.053	No data	No data	? (TO) TT=2400.03	? (TO) TT=2520.02	? (TO) TT=2520.08	? (TO) TT=2400.09	? (TO) TT=2520.04	? (TO) TT=2520.04	? (TO) TT=2520.07	? (TO) TT=2520.01	? TT=1747.89	No data	No data	No data	No data
SteelMillSlab/ SteelMillSlab-m1-simple_c18.xml	OPT f=0 TT=0.4 T1=0.4	No data	No data	No data	No data	OPT f=0 TT=1.395 T1=0.63	OPT f=0 TT=1.392 T1=0.62	OPT f=0 TT=9.219 T1=4.2	OPT f=0 TT=9.455 T1=4.22	OPT f=0 TT=9.153 T1=4.2	OPT f=0 TT=9.436 T1=4.59	? (NS) TT=0.006	No data	No data	OPT f=0 TT=0.041 T1=0.04	OPT f=0 TT=0.041 T1=0.04	No data	No data	SAT (TO) f=0 TT=2400.11 T1=1.27	SAT (TO) f=0 TT=2520.09 T1=1.21	OPT f=0 TT=1.993 T1=1.59	OPT f=0 TT=2.043 T1=1.25	OPT f=0 TT=1.936 T1=1.19	OPT f=0 TT=0.542 T1=0.54	OPT f=0 TT=0.396 T1=0.39	OPT f=0 TT=0.4 T1=0.4	OPT f=0 TT=2.634 T1=1.11	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
SteelMillSlab/ SteelMillSlab-m2-2-0_c18.xml		No data	No data	No data	No data	? (TO) TT=2400.09	? (TO) TT=2520.08	? (TO) TT=2520.11	? (TO) TT=2520.07	? (TO) TT=2520.12	? (TO) TT=2520.16	? (TO) TT=2519.87	No data	No data	? (TO) TT=2400.09	? (TO) TT=2519.92	No data	No data	? (TO) TT=2400.09	? (TO) TT=2520.06	? (TO) TT=2520.06	? (TO) TT=2400.11	? (TO) TT=2520.11	? (TO) TT=2519.83	? (TO) TT=2519.98	? (TO) TT=2520.04	? TT=1771.87	No data	No data	No data	No data
SteelMillSlab/ SteelMillSlab-m2-2-2_c18.xml		No data	No data	No data	No data	? (TO) TT=2400.09	? (TO) TT=2520.03	? (TO) TT=2520.15	? (TO) TT=2520.1	? (TO) TT=2520.06	? (TO) TT=2520.13	? (TO) TT=2520.24	No data	No data	? (TO) TT=2400.04	? (TO) TT=2519.95	No data	No data	? (TO) TT=2400.08	? (TO) TT=2520.09	? (TO) TT=2520.09	? (TO) TT=2400.11	? (TO) TT=2520.06	? (TO) TT=2520.02	? (TO) TT=2520.1	? (TO) TT=2520.02	? TT=1731.75	No data	No data	No data	No data
SteelMillSlab/ SteelMillSlab-m2-3-0_c18.xml		No data	No data	No data	No data	? (TO) TT=2400.1	? (TO) TT=2520.11	? (TO) TT=2520.17	? (TO) TT=2520.11	? (TO) TT=2520.15	? (TO) TT=2520.22	? (TO) TT=2519.96	No data	No data	? (TO) TT=2400.06	? (TO) TT=2520.03	No data	No data	? (TO) TT=2400.13	? (TO) TT=2520.07	? (TO) TT=2520.12	? (TO) TT=2400.11	? (TO) TT=2520.12	? (TO) TT=2519.83	? (TO) TT=2520.02	? (TO) TT=2519.83	? TT=526.55	No data	No data	No data	No data
SteelMillSlab/ SteelMillSlab-m2-3-2_c18.xml		No data	No data	No data	No data	? (TO) TT=2400.06	? (TO) TT=2520.13	? (TO) TT=2520.13	? (TO) TT=2520.12	? (TO) TT=2520.14	? (TO) TT=2520.11	? (TO) TT=2519.94	No data	No data	? (TO) TT=2400.06	? (TO) TT=2519.79	No data	No data	? (TO) TT=2400.08	? (TO) TT=2520.09	? (TO) TT=2520.03	? (TO) TT=2400.1	? (TO) TT=2520.04	? (TO) TT=2519.94	? (TO) TT=2519.94	? (TO) TT=2519.93	? TT=1791.12	No data	No data	No data	No data
SteelMillSlab/ SteelMillSlab-m2-4-0_c18.xml		No data	No data	No data	No data	? (TO) TT=2400.04	? (TO) TT=2520.12	? (TO) TT=2520.08	? (TO) TT=2520.18	? (TO) TT=2520.12	? (TO) TT=2520.15	? (TO) TT=2519.76	No data	No data	? (TO) TT=2400.02	? (TO) TT=2519.91	No data	No data	? (TO) TT=2400.09	? (TO) TT=2520.07	? (TO) TT=2520.11	? (TO) TT=2400.11	? (TO) TT=2520.07	? (TO) TT=2519.93	? (TO) TT=2519.86	? (TO) TT=2519.9	? TT=437.186	No data	No data	No data	No data
SteelMillSlab/ SteelMillSlab-m2-simple_c18.xml	OPT f=0 TT=0.535 T1=0.53	No data	No data	No data	No data	OPT f=0 TT=1.849 T1=0.77	OPT f=0 TT=1.823 T1=0.76	OPT f=0 TT=9.568 T1=4.36	OPT f=0 TT=9.74 T1=4.45	OPT f=0 TT=9.626 T1=4.6	OPT f=0 TT=9.653 T1=4.4	SAT f=8 TT=0.094 T1=0.09	No data	No data	Wrong UNSAT TT=0.1	OPT f=0 TT=0.098 T1=0.09	No data	No data	SAT (TO) f=0 TT=2400.11 T1=1.45	SAT (TO) f=0 TT=2520.02 T1=1.35	OPT f=0 TT=2.493 T1=1.35	OPT f=0 TT=2.65 T1=1.42	OPT f=0 TT=2.454 T1=1.33	OPT f=0 TT=0.528 T1=0.53	OPT f=0 TT=0.533 T1=0.53	OPT f=0 TT=0.535 T1=0.53	OPT f=0 TT=2.783 T1=1.25	No data	No data	No data	No data
SteelMillSlab/ SteelMillSlab-m2s-2-0_c18.xml	SAT f=282 TT=2520.34 T1=704.24	No data	No data	No data	No data	SAT (TO) f=284 TT=2400.16 T1=962.69	SAT (TO) f=284 TT=2520.07 T1=947.32	SAT (TO) f=319 TT=2520.1 T1=65.32	SAT (TO) f=349 TT=2520.12 T1=726.3	SAT (TO) f=319 TT=2520.14 T1=246.36	SAT f=379 TT=2427.1 T1=45.74	SAT (TO) f=282 TT=2520.34 T1=704.24	No data	No data	SAT (TO) f=94 TT=2400.11 T1=84.82	SAT (TO) f=94 TT=2520.08 T1=86.94	No data	No data	SAT (TO) f=370 TT=2400.17 T1=2151.02	SAT (TO) f=339 TT=2520.16 T1=1465.46	? (TO) TT=2520.04	? (TO) TT=2400.05	? (TO) TT=2520.03	? (TO) TT=2519.98	? (TO) TT=2520.07	? (TO) TT=2520.02	? (TO) TT=2400.12	No data	No data	No data	No data
SteelMillSlab/ SteelMillSlab-m2s-2-2_c18.xml	SAT f=264 TT=2520.34 T1=1407.35	No data	No data	No data	No data	SAT (TO) f=694 TT=2400.1 T1=445.61	SAT (TO) f=694 TT=2520.08 T1=446.39	SAT (TO) f=650 TT=2520.12 T1=63.07	SAT (TO) f=725 TT=2520.13 T1=161.65	SAT (TO) f=611 TT=2520.1 T1=95.57	SAT f=772 TT=1729.67 T1=39.87	SAT (TO) f=264 TT=2520.34 T1=1407.35	No data	No data	SAT (TO) f=180 TT=2400.09 T1=793.64	SAT (TO) f=180 TT=2519.75 T1=825.98	No data	No data	SAT (TO) f=923 TT=2400.2 T1=1411.47	SAT (TO) f=1003 TT=2520.11 T1=2495.95	SAT (TO) f=1117 TT=2520.19 T1=1908.93	? (TO) TT=2400.08	? (TO) TT=2520.1	? (TO) TT=2519.99	? (TO) TT=2519.92	? (TO) TT=2519.9	? (TO) TT=2408.02	No data	No data	No data	No data
SteelMillSlab/ SteelMillSlab-m2s-3-0_c18.xml	SAT f=212 TT=2520.13 T1=133.4	No data	No data	No data	No data	SAT (TO) f=260 TT=2400.11 T1=528.52	SAT (TO) f=260 TT=2520.08 T1=516.88	SAT (TO) f=214 TT=2520.09 T1=2356.69	SAT (TO) f=212 TT=2520.13 T1=133.4	SAT (TO) f=133 TT=2520.12 T1=39.13	? (TO) TT=2528.32	SAT (TO) f=252 TT=2519.89 T1=807.28	No data	No data	SAT (TO) f=36 TT=2400.02 T1=627.32	SAT (TO) f=36 TT=2520 T1=642.92	No data	No data	SAT (TO) f=244 TT=2400.14 T1=1593.02	SAT (TO) f=251 TT=2520.2 T1=872.58	SAT (TO) f=234 TT=2520.14 T1=2452.91	? (TO) TT=2400.08	? (TO) TT=2520.09	? (TO) TT=2520.05	? (TO) TT=2519.73	? (TO) TT=2520.05	? (TO) TT=2400.11	No data	No data	No data	No data
SteelMillSlab/ SteelMillSlab-m2s-3-2_c18.xml	SAT f=296 TT=2520.08 T1=898.48	No data	No data	No data	No data	SAT (TO) f=358 TT=2400.09 T1=380.78	SAT (TO) f=358 TT=2520.14 T1=377.04	SAT (TO) f=355 TT=2520.15 T1=917.2	SAT (TO) f=296 TT=2520.08 T1=898.48	SAT (TO) f=224 TT=2520.07 T1=35.09	? (TO) TT=2527.9	SAT (TO) f=410 TT=2520.17 T1=497.49	No data	No data	SAT (TO) f=59 TT=2400.06 T1=806.88	SAT (TO) f=59 TT=2519.83 T1=876.47	No data	No data	SAT (TO) f=429 TT=2400.15 T1=1368.59	SAT (TO) f=455 TT=2520.11 T1=620.08	SAT (TO) f=450 TT=2520.13 T1=1420.64	? (TO) TT=2400.1	? (TO) TT=2520.03	? (TO) TT=2520.07	? (TO) TT=2519.8	? (TO) TT=2520.06	? (TO) TT=2400.14	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
SteelMillSlab/ SteelMillSlab-m2s-4-0_c18.xml	SAT f=168 TT=2519.91 T1=696.4	No data	No data	No data	No data	SAT (TO) f=243 TT=2400.11 T1=560.23	SAT (TO) f=243 TT=2520.13 T1=559.14	SAT (TO) f=238 TT=2520.08 T1=503.52	SAT (TO) f=214 TT=2520.1 T1=228.34	SAT (TO) f=211 TT=2520.13 T1=39.64	? (TO) TT=2527.99	SAT (TO) f=168 TT=2519.91 T1=696.4	No data	No data	SAT (TO) f=53 TT=2400.1 T1=1956.47	SAT (TO) f=52 TT=2519.98 T1=2511.12	No data	No data	SAT (TO) f=284 TT=2400.18 T1=824.11	SAT (TO) f=253 TT=2520.09 T1=2099.25	SAT (TO) f=255 TT=2520.13 T1=1917.79	? (TO) TT=2400.11	? (TO) TT=2520.09	? (TO) TT=2519.82	? (TO) TT=2519.95	? (TO) TT=2519.8	? (TO) TT=2407.85	No data	No data	No data	No data
StillLife/ StillLife-wastage-08_c18.xml	OPT f=36 TT=3.651 T1=3.28	No data	No data	No data	No data	OPT f=36 TT=13.955 T1=10.28	OPT f=36 TT=13.083 T1=10.15	OPT f=36 TT=26.288 T1=14.62	OPT f=36 TT=32.32 T1=19.9	OPT f=36 TT=22.002 T1=12.3	OPT f=36 TT=25.502 T1=13.85	OPT f=36 TT=3.651 T1=3.28	No data	No data	OPT f=36 TT=5.118 T1=5.03	OPT f=36 TT=5.153 T1=5.06	No data	No data	SAT (TO) f=36 TT=2400.09 T1=33.05	SAT (TO) f=36 TT=2520.02 T1=10.68	OPT f=36 TT=23.196 T1=19.2	SAT (TO) f=36 TT=2400.05 T1=6.88	SAT (TO) f=36 TT=2520.03 T1=6.29	OPT f=36 TT=7.629 T1=7.63	OPT f=36 TT=8.783 T1=8.78	OPT f=36 TT=8.74 T1=8.74	? (TO) TT=2400.11	No data	No data	No data	No data
StillLife/ StillLife-wastage-09_c18.xml	OPT f=43 TT=15.45 T1=10.91	No data	No data	No data	No data	OPT f=43 TT=101.736 T1=20.68	OPT f=43 TT=101.396 T1=20.47	OPT f=43 TT=464.089 T1=51.37	OPT f=43 TT=86.376 T1=10.91	OPT f=43 TT=150.245 T1=6.35	OPT f=43 TT=182.362 T1=16.52	OPT f=43 TT=153.666 T1=25.17	No data	No data	OPT f=43 TT=100.233 T1=15.06	OPT f=43 TT=100.356 T1=15.14	No data	No data	SAT (TO) f=43 TT=2400.07 T1=192.78	SAT (TO) f=43 TT=2520.03 T1=141.98	OPT f=43 TT=941.448 T1=318.35	SAT (TO) f=42 TT=2400.12 T1=30.74	SAT (TO) f=43 TT=2520.05 T1=29.47	OPT f=43 TT=12.144 T1=12.14	OPT f=43 TT=15.878 T1=15.88	OPT f=43 TT=15.45 T1=15.49	? (TO) TT=2400.25	No data	No data	No data	No data
StillLife/ StillLife-wastage-10_c18.xml	OPT f=54 TT=17.403 T1=17.44	No data	No data	No data	No data	OPT f=54 TT=912.101 T1=775.08	OPT f=54 TT=911.726 T1=775.4	OPT f=54 TT=1404.48 T1=778.05	OPT f=54 TT=1909.31 T1=1280.17	OPT f=54 TT=603.104 T1=518.07	OPT f=54 TT=1678.71 T1=820.48	OPT f=54 TT=1907.63 T1=1697.18	No data	No data	OPT f=54 TT=324.149 T1=123.6	OPT f=54 TT=322.382 T1=122.87	No data	No data	SAT (TO) f=54 TT=2400.1 T1=40.29	SAT (TO) f=54 TT=2520.08 T1=18.57	OPT f=54 TT=1034.28 T1=58.1	SAT (TO) f=54 TT=2400.04 T1=44.45	SAT (TO) f=54 TT=2520.07 T1=44.51	OPT f=54 TT=19.124 T1=19.12	OPT f=54 TT=16.502 T1=16.5	OPT f=54 TT=17.403 T1=17.44	? (TO) TT=2400.17	No data	No data	No data	No data
StillLife/ StillLife-wastage-11_c18.xml	OPT f=64 TT=30.912 T1=30.91	No data	No data	No data	No data	SAT (TO) f=64 TT=2400.1 T1=1616.89	SAT (TO) f=64 TT=2520.09 T1=1600.68	SAT (TO) f=64 TT=2520.09 T1=147.39	SAT (TO) f=64 TT=2520.07 T1=2161.62	SAT (TO) f=64 TT=2520.11 T1=607.51	SAT (TO) f=64 TT=2520.06 T1=136.11	SAT (TO) f=61 TT=2520.08 T1=1742.83	No data	No data	SAT (TO) f=64 TT=2400.04 T1=187.2	SAT (TO) f=64 TT=2520.03 T1=187.15	No data	No data	SAT (TO) f=62 TT=2400.05 T1=110.93	SAT (TO) f=63 TT=2520.07 T1=2280.58	SAT (TO) f=59 TT=2520.07 T1=2229.61	SAT (TO) f=56 TT=2400.1 T1=13.95	SAT (TO) f=56 TT=2520.1 T1=62.01	OPT f=64 TT=31.936 T1=31.93	OPT f=64 TT=31.185 T1=31.19	OPT f=64 TT=30.912 T1=30.91	? (TO) TT=2400.2	No data	No data	No data	No data
StillLife/ StillLife-wastage-12_c18.xml	OPT f=76 TT=39.921 T1=39.92	No data	No data	No data	No data	SAT (TO) f=73 TT=2400.03 T1=2369.05	SAT (TO) f=73 TT=2520.09 T1=2334.19	SAT (TO) f=76 TT=2520.06 T1=163.5	SAT (TO) f=74 TT=2520.09 T1=461.93	SAT (TO) f=75 TT=2520.11 T1=1639.23	SAT (TO) f=75 TT=2520.12 T1=514.05	SAT (TO) f=68 TT=2520.09 T1=2177.12	No data	No data	SAT (TO) f=76 TT=2400.08 T1=163.02	SAT (TO) f=76 TT=2519.64 T1=165.19	No data	No data	SAT (TO) f=68 TT=2400.02 T1=1227.95	SAT (TO) f=72 TT=2520.05 T1=1765.12	SAT (TO) f=74 TT=2520.04 T1=845.13	SAT (TO) f=73 TT=2400.12 T1=14.32	SAT (TO) f=73 TT=2520.05 T1=25.93	OPT f=76 TT=33.379 T1=33.38	OPT f=76 TT=39.975 T1=39.97	OPT f=76 TT=39.921 T1=39.92	? (TO) TT=2400.16	No data	No data	No data	No data
StillLife/ StillLife-wastage-13_c18.xml	OPT f=90 TT=48.232 T1=48.27	No data	No data	No data	No data	SAT (TO) f=77 TT=2400.11 T1=1139.62	SAT (TO) f=77 TT=2520.12 T1=1145.92	SAT (TO) f=89 TT=2520.08 T1=1351.7	SAT (TO) f=86 TT=2520.1 T1=1078.64	SAT (TO) f=88 TT=2520.1 T1=771.09	SAT (TO) f=88 TT=2520.13 T1=1508.6	SAT (TO) f=72 TT=2520.05 T1=2452.35	No data	No data	SAT (TO) f=90 TT=2400.06 T1=378.92	SAT (TO) f=90 TT=2519.97 T1=378.29	No data	No data	SAT (TO) f=85 TT=2400.11 T1=80.23	SAT (TO) f=83 TT=2520.06 T1=48.71	SAT (TO) f=85 TT=2520.06 T1=1637.42	SAT (TO) f=71 TT=2400.08 T1=8.02	SAT (TO) f=71 TT=2520.06 T1=6.99	OPT f=90 TT=50.43 T1=50.43	OPT f=90 TT=43.384 T1=43.38	OPT f=90 TT=48.232 T1=48.27	? (TO) TT=2400.26	No data	No data	No data	No data
StillLife/ StillLife-wastage-14_c18.xml	OPT f=104 TT=59.31 T1=59.31	No data	No data	No data	No data	SAT (TO) f=91 TT=2400.05 T1=1285.63	SAT (TO) f=91 TT=2520.04 T1=1288.92	SAT (TO) f=100 TT=2520.14 T1=1207.97	SAT (TO) f=96 TT=2520.07 T1=918.89	SAT (TO) f=100 TT=2520.12 T1=531.75	SAT (TO) f=102 TT=2520.06 T1=2300.15	SAT (TO) f=68 TT=2520.07 T1=2282.57	No data	No data	SAT (TO) f=103 TT=2400.01 T1=878.85	SAT (TO) f=103 TT=2520.03 T1=881.99	No data	No data	SAT (TO) f=88 TT=2400.12 T1=562.38	SAT (TO) f=86 TT=2520.04 T1=168.5	SAT (TO) f=93 TT=2520.1 T1=48.18	SAT (TO) f=101 TT=2400.13 T1=2115.48	SAT (TO) f=96 TT=2520.1 T1=2016.11	OPT f=104 TT=54.236 T1=54.24	OPT f=104 TT=45.72 T1=45.72	OPT f=104 TT=59.31 T1=59.31	? (TO) TT=2400.11	No data	No data	No data	No data
StillLife/ StillLife-wastage-15_c18.xml	OPT f=119 TT=66.012 T1=66.01	No data	No data	No data	No data	SAT (TO) f=103 TT=2400.04 T1=1450.44	SAT (TO) f=103 TT=2520.12 T1=1468.01	SAT (TO) f=113 TT=2520.08 T1=455.51	SAT (TO) f=113 TT=2520.08 T1=1678.56	SAT (TO) f=112 TT=2520.09 T1=500.49	SAT (TO) f=113 TT=2520.1 T1=1129.51	SAT (TO) f=77 TT=2520.08 T1=1769.43	No data	No data	SAT (TO) f=119 TT=2400.08 T1=533.59	SAT (TO) f=119 TT=2520.07 T1=533.16	No data	No data	SAT (TO) f=104 TT=2400.09 T1=89.26	SAT (TO) f=106 TT=2520.08 T1=248.2	SAT (TO) f=104 TT=2520.05 T1=201.43	SAT (TO) f=107 TT=2400.05 T1=720.28	SAT (TO) f=111 TT=2520.05 T1=2404.29	OPT f=119 TT=70.204 T1=70.2	OPT f=119 TT=66.006 T1=66.02	OPT f=119 TT=66.012 T1=66.01	? (TO) TT=2400.32	No data	No data	No data	No data
StillLife/ StillLife-wastage-17_c18.xml	OPT f=152 TT=148.537 T1=148.54	No data	No data	No data	No data	SAT (TO) f=121 TT=2400.11 T1=1315.84	SAT (TO) f=121 TT=2520.15 T1=1254.89	SAT (TO) f=142 TT=2520.14 T1=2201.44	SAT (TO) f=142 TT=2520.04 T1=661.16	SAT (TO) f=144 TT=2520.07 T1=1884.14	SAT (TO) f=144 TT=2520.11 T1=1941.51	SAT (TO) f=92 TT=2519.98 T1=0.13	No data	No data	SAT (TO) f=150 TT=2400.02 T1=1586.62	SAT (TO) f=150 TT=2520.05 T1=1585.24	No data	No data	SAT (TO) f=128 TT=2400.11 T1=59.78	SAT (TO) f=133 TT=2520.13 T1=999.87	SAT (TO) f=131 TT=2520.08 T1=212.88	SAT (TO) f=123 TT=2400.11 T1=45.32	SAT (TO) f=107 TT=2520.06 T1=40.54	OPT f=152 TT=149.33 T1=149.34	OPT f=152 TT=156.635 T1=156.63	OPT f=152 TT=148.537 T1=148.54	? (TO) TT=2400.21	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
StillLife/ StillLife-wastage-19_c18.xml	OPT f=190 TT=154.096 T1=154.1	No data	No data	No data	No data	SAT (TO) f=152 TT=2400.1 T1=95.36	SAT (TO) f=152 TT=2520.06 T1=100.42	SAT (TO) f=171 TT=2520.06 T1=1110.9	SAT (TO) f=169 TT=2520.06 T1=526.81	SAT (TO) f=166 TT=2520.13 T1=1904.75	SAT (TO) f=171 TT=2520.11 T1=1261.78	SAT (TO) f=57 TT=2519.6 T1=2225.65	No data	No data	SAT (TO) f=186 TT=2400.11 T1=1095.03	SAT (TO) f=186 TT=2519.91 T1=1099.84	No data	No data	SAT (TO) f=159 TT=2400.49 T1=170.08	SAT (TO) f=158 TT=2520.06 T1=2416.26	SAT (TO) f=158 TT=2520.07 T1=2444.79	SAT (TO) f=159 TT=2400.07 T1=469.92	SAT (TO) f=143 TT=2520.1 T1=161.48	OPT f=190 TT=181.395 T1=181.39	OPT f=190 TT=233.283 T1=233.26	OPT f=190 TT=154.096 T1=154.1	? TT=436.447	No data	No data	No data	No data
StillLife/ StillLife-wastage-21_c18.xml	OPT f=232 TT=194.302 T1=194.29	No data	No data	No data	No data	SAT (TO) f=184 TT=2400.03 T1=2017.8	SAT (TO) f=184 TT=2520.1 T1=2031	SAT (TO) f=196 TT=2520.09 T1=758.99	SAT (TO) f=196 TT=2520.33 T1=906.44	SAT (TO) f=197 TT=2520.11 T1=2327.33	SAT (TO) f=199 TT=2520.13 T1=625.45	SAT (TO) f=58 TT=2520.03 T1=1551.6	No data	No data	SAT (TO) f=223 TT=2400.06 T1=1447.97	SAT (TO) f=223 TT=2519.99 T1=1449.05	No data	No data	SAT (TO) f=198 TT=2400.07 T1=444.29	SAT (TO) f=193 TT=2520.11 T1=917.57	SAT (TO) f=185 TT=2520.09 T1=327.64	SAT (TO) f=197 TT=2400.07 T1=2282.45	SAT (TO) f=194 TT=2520.09 T1=206.06	OPT f=232 TT=196.345 T1=196.37	OPT f=232 TT=196.568 T1=196.62	OPT f=232 TT=194.302 T1=194.29	? (problem) TT=1899.25	No data	No data	No data	No data
StillLife/ StillLife-wastage-25_c18.xml	OPT f=326 TT=300.557 T1=300.58	No data	No data	No data	No data	SAT (TO) f=266 TT=2400.12 T1=2222.2	SAT (TO) f=266 TT=2520.09 T1=2235.41	SAT (TO) f=269 TT=2520.09 T1=110.81	SAT (TO) f=269 TT=2520.1 T1=60.52	SAT (TO) f=271 TT=2520.06 T1=497.33	SAT (TO) f=273 TT=2520.08 T1=1852.54	SAT (TO) f=54 TT=2520 T1=2339.03	No data	No data	SAT (TO) f=314 TT=2400.05 T1=2319.79	SAT (TO) f=314 TT=2519.65 T1=2334.87	No data	No data	SAT (TO) f=258 TT=2400.05 T1=62.98	SAT (TO) f=258 TT=2520.12 T1=1908.75	SAT (TO) f=264 TT=2520.08 T1=633.26	SAT (TO) f=231 TT=2400.08 T1=936.78	SAT (TO) f=247 TT=2520.03 T1=1546.25	OPT f=326 TT=498.418 T1=498.43	OPT f=326 TT=501.55 T1=501.52	OPT f=326 TT=300.557 T1=300.58	? (problem) TT=1956.21	No data	No data	No data	No data
StillLife/ StillLife-wastage-30_c18.xml	OPT f=466 TT=1279.53 T1=1279.63	No data	No data	No data	No data	SAT (TO) f=367 TT=2400.04 T1=204.01	SAT (TO) f=367 TT=2520.03 T1=207.63	SAT (TO) f=385 TT=2520.05 T1=477.3	SAT (TO) f=378 TT=2520.11 T1=497.02	SAT (TO) f=362 TT=2520.11 T1=1181.26	SAT (TO) f=365 TT=2520.15 T1=403.49	SAT (TO) f=71 TT=2520.07 T1=8.7	No data	No data	SAT (TO) f=424 TT=2400.01 T1=1891.79	SAT (TO) f=424 TT=2519.82 T1=1887.42	No data	No data	SAT (TO) f=380 TT=2400.12 T1=426.83	SAT (TO) f=369 TT=2520.09 T1=85.39	SAT (TO) f=385 TT=2520.12 T1=753.47	SAT (TO) f=365 TT=2400.11 T1=228.57	SAT (TO) f=302 TT=2520.08 T1=499.33	OPT f=466 TT=705.704 T1=705.66	OPT f=466 TT=937.926 T1=937.81	OPT f=466 TT=1279.53 T1=1279.63	? TT=2254.6	No data	No data	No data	No data
StripPacking/ StripPacking-1a_c18.xml	UNSAT TT=0.029	? (NS) TT=0.012	? (NS) TT=0.012	? (NS) TT=0.012	? (NS) TT=0.012	UNSAT TT=1.384	UNSAT TT=1.341	UNSAT TT=8.717	UNSAT TT=9.755	UNSAT TT=9.288	UNSAT TT=10.082	? (NS) TT=0.018	? (NS) TT=0.013	? (NS) TT=0.012	UNSAT TT=0.029	UNSAT TT=0.029	UNSAT TT=1.968	UNSAT TT=1.965	? (TO) TT=2400.08	? (TO) TT=2520.07	UNSAT TT=1.978	No data	No data	UNSAT TT=2.487	UNSAT TT=2.483	UNSAT TT=2.556	UNSAT TT=2.073	UNSAT TT=3.54	UNSAT TT=4.806	UNSAT TT=3.529	UNSAT TT=4.887
StripPacking/ StripPacking-1b_c18.xml	SAT TT=0.636	? (NS) TT=0.011	? (NS) TT=0.011	? (NS) TT=0.011	? (NS) TT=0.011	SAT TT=2.413	SAT TT=2.452	SAT TT=13.77	SAT TT=12.813	SAT TT=10.154	SAT TT=13.271	? (NS) TT=0.016	? (NS) TT=0.011	? (NS) TT=0.011	SAT TT=0.636	SAT TT=0.759	? (TO) TT=2400.1	SAT TT=2477.3	? (TO) TT=2400.09	? (TO) TT=2520.04	? (TO) TT=2520.06	No data	No data	SAT TT=1.944	SAT TT=1.954	SAT TT=1.948	SAT TT=7.349	SAT TT=3.338	SAT TT=4.695	SAT TT=3.331	SAT TT=4.532
StripPacking/ StripPacking-C1P1_c18.xml	SAT TT=2.878	? (NS) TT=0.014	? (NS) TT=0.014	? (NS) TT=0.014	? (NS) TT=0.014	SAT TT=2.878	SAT TT=2.942	SAT TT=47.114	SAT TT=508.904	SAT TT=14.275	SAT TT=134.567	? (NS) TT=0.022	? (NS) TT=0.014	? (NS) TT=0.014	SAT TT=80.881	SAT TT=285.169	? (TO) TT=2400.08	? (TO) TT=2520.03	? (TO) TT=2400.03	? (TO) TT=2520.08	? (TO) TT=2520.04	No data	No data	SAT TT=13.505	SAT TT=28.668	SAT TT=13.492	SAT TT=370.324	SAT TT=7.627	SAT TT=8.539	SAT TT=7.642	SAT TT=8.605
StripPacking/ StripPacking-C1P2_c18.xml	SAT TT=8.085	? (NS) TT=0.014	? (NS) TT=0.015	? (NS) TT=0.014	? (NS) TT=0.015	SAT TT=27.822	SAT TT=28.413	SAT TT=537.082	SAT TT=104.426	SAT TT=54.207	SAT TT=130.269	? (NS) TT=0.023	? (NS) TT=0.015	? (NS) TT=0.014	SAT TT=1070.69	SAT TT=37.862	? (TO) TT=2400.1	? (TO) TT=2520.04	? (TO) TT=2400.06	? (TO) TT=2520.11	? (TO) TT=2520.06	No data	No data	SAT TT=27.607	SAT TT=67.53	SAT TT=27.513	SAT TT=328.854	SAT TT=8.152	SAT TT=10.172	SAT TT=8.085	SAT TT=10.182
StripPacking/ StripPacking-C1P3_c18.xml	SAT TT=4.426	? (NS) TT=0.014	? (NS) TT=0.014	? (NS) TT=0.014	? (NS) TT=0.014	SAT TT=6.426	SAT TT=5.933	SAT TT=19.052	SAT TT=90.28	SAT TT=77.61	SAT TT=527.723	? (NS) TT=0.023	? (NS) TT=0.014	? (NS) TT=0.014	SAT TT=1687.26	SAT TT=43.357	? (TO) TT=2400.02	? (TO) TT=2520.11	? (TO) TT=2400.1	? (TO) TT=2520.06	? (TO) TT=2520.11	No data	No data	SAT TT=5.342	SAT TT=5.379	SAT TT=12.409	SAT TT=79.703	SAT TT=4.426	SAT TT=5.621	SAT TT=4.469	SAT TT=5.579
StripPacking/ StripPacking-C2P1_c18.xml	SAT TT=6.134	? (NS) TT=0.019	? (NS) TT=0.019	? (NS) TT=0.018	? (NS) TT=0.019	SAT TT=11.996	SAT TT=11.851	SAT TT=518.439	? (TO) TT=2520.06	SAT TT=47.36	SAT TT=23.508	? (NS) TT=0.033	? (NS) TT=0.02	? (NS) TT=0.019	SAT TT=1369.43	SAT TT=1092.17	? (TO) TT=2400.1	? (TO) TT=2520.02	? (TO) TT=2400.05	? (TO) TT=2520.03	? (TO) TT=2520.04	No data	No data	SAT TT=113.829	SAT TT=114.15	SAT TT=312.552	? (TO) TT=2400.15	SAT TT=6.196	SAT TT=10.037	SAT TT=6.134	SAT TT=10.311
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
StripPacking/ StripPacking-N1a_c18.xml	SAT TT=118.386	? (NS) TT=0.016	? (NS) TT=0.016	? (NS) TT=0.016	? (NS) TT=0.016	? (TO) TT=2400.12	? (TO) TT=2520.4	? (TO) TT=2520.06	? (TO) TT=2520.06	? (TO) TT=2520.05	? (TO) TT=2520.07	? (NS) TT=0.024	? (NS) TT=0.017	? (NS) TT=0.016	? (TO) TT=2400.03	? (TO) TT=2519.75	? (TO) TT=2400.07	? (TO) TT=2520.06	? (TO) TT=2400.09	? (TO) TT=2520.03	? (TO) TT=2520.02	No data	No data	? (TO) TT=2519.82	? (TO) TT=2520.01	SAT TT=118.386	? (TO) TT=2400.13	SAT TT=228.742	? (TO) TT=2520.09	SAT TT=228.4	? (TO) TT=2520.05
StripPacking/ StripPacking-N1b_c18.xml	SAT TT=158.344	? (NS) TT=0.016	? (NS) TT=0.016	? (NS) TT=0.017	? (NS) TT=0.016	? (TO) TT=2400.04	? (TO) TT=2520.07	? (TO) TT=2520.1	? (TO) TT=2520.13	? (TO) TT=2520.09	? (TO) TT=2520.09	? (NS) TT=0.024	? (NS) TT=0.017	? (NS) TT=0.016	? (TO) TT=2400.01	? (TO) TT=2520.04	? (TO) TT=2400.11	? (TO) TT=2520.09	? (TO) TT=2400.12	? (TO) TT=2520.02	? (TO) TT=2520.05	No data	No data	? (TO) TT=2519.93	SAT TT=260.949	SAT TT=259.988	? (TO) TT=2400.13	SAT TT=159.457	SAT TT=451.722	SAT TT=158.344	SAT TT=453.668
StripPacking/ StripPacking-N1c_c18.xml	SAT TT=72.018	? (NS) TT=0.016	? (NS) TT=0.016	? (NS) TT=0.017	? (NS) TT=0.017	? (TO) TT=2400.06	? (TO) TT=2520.04	? (TO) TT=2520.14	? (TO) TT=2520.14	? (TO) TT=2520.07	? (TO) TT=2520.04	? (NS) TT=0.024	? (NS) TT=0.017	? (NS) TT=0.016	? (TO) TT=2400.07	? (TO) TT=2520.06	? (TO) TT=2400.11	? (TO) TT=2520.12	? (TO) TT=2400.05	? (TO) TT=2520.1	? (TO) TT=2520.04	No data	No data	SAT TT=156.988	SAT TT=1369.44	SAT TT=72.018	? (TO) TT=2400.17	SAT TT=1101.16	SAT TT=864.998	SAT TT=1088.29	SAT TT=875.362
StripPacking/ StripPacking-T1a_c18.xml	SAT TT=749.317	? (NS) TT=0.016	? (NS) TT=0.016	? (NS) TT=0.017	? (NS) TT=0.016	? (TO) TT=2400.07	? (TO) TT=2520.03	? (TO) TT=2520.06	? (TO) TT=2520.09	? (TO) TT=2520.08	? (TO) TT=2520.07	? (NS) TT=0.024	? (NS) TT=0.017	? (NS) TT=0.016	? (TO) TT=2400.05	? (TO) TT=2519.95	? (TO) TT=2400.05	? (TO) TT=2520.12	? (TO) TT=2400.05	? (TO) TT=2520.11	? (TO) TT=2520.08	No data	No data	? (TO) TT=2520.06	? (TO) TT=2519.97	? (TO) TT=2519.84	? (TO) TT=2400.11	SAT TT=750.318	SAT TT=1792.43	SAT TT=749.317	SAT TT=1785.64
StripPacking/ StripPacking-T1c_c18.xml	SAT TT=5.741	? (NS) TT=0.016	? (NS) TT=0.016	? (NS) TT=0.016	? (NS) TT=0.016	SAT TT=552.911	SAT TT=539.584	? (TO) TT=2520.14	SAT TT=194.269	SAT TT=692.281	SAT TT=347.682	? (NS) TT=0.023	? (NS) TT=0.016	? (NS) TT=0.016	SAT TT=23.605	SAT TT=11.673	? (TO) TT=2400.02	? (TO) TT=2520.02	? (TO) TT=2400.02	? (TO) TT=2520.02	? (TO) TT=2520.04	No data	No data	SAT TT=6.23	SAT TT=18.763	SAT TT=18.782	? (TO) TT=2400.12	SAT TT=5.741	SAT TT=7.665	SAT TT=5.768	SAT TT=7.724
StripPacking/ StripPacking-T1d_c18.xml	SAT TT=16.113	? (NS) TT=0.016	? (NS) TT=0.016	? (NS) TT=0.016	? (NS) TT=0.016	? (TO) TT=2400.13	? (TO) TT=2520.13	? (TO) TT=2520.13	? (TO) TT=2520.09	? (TO) TT=2520.14	? (TO) TT=2520.15	? (NS) TT=0.023	? (NS) TT=0.017	? (NS) TT=0.016	SAT TT=1174.84	SAT TT=559.016	? (TO) TT=2400.03	? (TO) TT=2520.04	? (TO) TT=2400.11	? (TO) TT=2520.06	? (TO) TT=2520.06	No data	No data	SAT TT=23.503	SAT TT=256.9	SAT TT=23.376	? (TO) TT=2401.44	SAT TT=16.193	SAT TT=104.993	SAT TT=16.113	SAT TT=104.914
Subisomorphism/ Subisomorphism-g08-g43_c18.xml	SAT TT=1.213	SAT TT=1.233	SAT TT=1.213	SAT TT=3.543	SAT TT=3.54	SAT TT=5.648	SAT TT=5.425	SAT TT=8.774	SAT TT=11.097	SAT TT=15.226	SAT TT=16.187	SAT TT=16.485	SAT TT=3.539	SAT TT=3.542	SAT TT=27.887	SAT TT=29.589	SAT TT=7.341	SAT TT=7.362	SAT TT=13.315	SAT TT=26.219	SAT TT=14.27	No data	No data	SAT TT=5.999	SAT TT=6.03	SAT TT=5.996	SAT TT=133.444	SAT TT=56.255	SAT TT=56.329	SAT TT=56.563	SAT TT=56.678
Subisomorphism/ Subisomorphism-g10-g23_c18.xml	UNSAT TT=87.64	? TT=2520.02	? TT=2520.02	UNSAT TT=87.014	UNSAT TT=87.64	UNSAT TT=141.224	UNSAT TT=138.87	? (TO) TT=2520.09	? (TO) TT=2520.08	? (TO) TT=2520.11	? (TO) TT=2520.1	? (TO) TT=2519.67	UNSAT TT=87.675	UNSAT TT=88.812	? (TO) TT=2400.07	? (TO) TT=2519.56	? (TO) TT=2400.06	? (TO) TT=2520.09	? (TO) TT=2400.04	? (TO) TT=2520.03	? (TO) TT=2520.11	No data	No data	? (TO) TT=2519.99	? (TO) TT=2520.04	? (TO) TT=2520.04	? (TO) TT=2400.12	? (TO) TT=2520.09	? (TO) TT=2520.12	? (TO) TT=2520.04	? (TO) TT=2520.14
Subisomorphism/ Subisomorphism-g10-g27_c18.xml	SAT TT=134.841	? TT=2520.02	? TT=2520.02	? (TO) TT=2519.98	? (TO) TT=2519.98	SAT TT=135.276	SAT TT=134.841	SAT TT=1153.36	? (TO) TT=2520.08	SAT TT=2284.93	? (TO) TT=2520.04	? (TO) TT=2519.8	? (TO) TT=2519.95	? TT=2520.02	? (TO) TT=2400.02	SAT TT=579.661	? (TO) TT=2400.08	? (TO) TT=2520.07	? (TO) TT=2400.09	? (TO) TT=2520.03	? (TO) TT=2520.03	No data	No data	? (TO) TT=2520.07	? (TO) TT=2519.9	? (TO) TT=2520.04	? (TO) TT=2400.12	? (TO) TT=2520.13	? (TO) TT=2520.08	? (TO) TT=2520.08	? (TO) TT=2520.1
Subisomorphism/ Subisomorphism-g11-g30_c18.xml	UNSAT TT=192.522	UNSAT TT=287.106	UNSAT TT=285.595	UNSAT TT=403.927	UNSAT TT=404.093	UNSAT TT=501.15	UNSAT TT=502.542	UNSAT TT=1211.7	UNSAT TT=1874.23	UNSAT TT=1149.97	UNSAT TT=1954.26	UNSAT TT=192.522	UNSAT TT=404.351	UNSAT TT=404.588	UNSAT TT=938.502	UNSAT TT=808.778	? (TO) TT=2400.12	? (TO) TT=2520.05	? (TO) TT=2400.06	? (TO) TT=2520.11	? (TO) TT=2520.06	No data	No data	? (TO) TT=2520.04	? (TO) TT=2519.93	? (TO) TT=2520	? (TO) TT=2400.16	? (MO) TT=940.324	? (TO) TT=2520.14	? (MO) TT=892.646	? (TO) TT=2520.07
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
Subisomorphism/ Subisomorphism-g11-g44_c18.xml	UNSAT TT=443.373	UNSAT TT=1208.66	UNSAT TT=1183.38	UNSAT TT=1269.9	UNSAT TT=1247.15	? (TO) TT=2400.1	? (TO) TT=2520.08	? (TO) TT=2520.09	? (TO) TT=2520.04	UNSAT TT=2345.06	? (TO) TT=2520.05	UNSAT TT=443.373	UNSAT TT=1255.85	UNSAT TT=1255.04	UNSAT TT=1313.11	UNSAT TT=1431.39	? (TO) TT=2400.06	? (TO) TT=2520.07	? (TO) TT=2400.03	? (TO) TT=2520.04	? (TO) TT=2520.06	No data	No data	? (TO) TT=2520.1	? (TO) TT=2520.09	? (TO) TT=2519.87	? (TO) TT=2400.15	? (MO) TT=1221.74	? (TO) TT=2520.11	? (MO) TT=1201.41	? (TO) TT=2520.07
Subisomorphism/ Subisomorphism-g17-g37_c18.xml	SAT TT=89.884	SAT TT=2306.99	SAT TT=2194.48	? (TO) TT=2520.01	? (TO) TT=2520.01	SAT TT=1956.37	SAT TT=1920.3	? (TO) TT=2520.13	? (TO) TT=2520.09	? (TO) TT=2520.11	? (TO) TT=2520.1	? (TO) TT=2519.79	? (TO) TT=2519.91	? (TO) TT=2520.02	SAT TT=343.619	SAT TT=653.891	? (TO) TT=2400.05	? (TO) TT=2520.03	? (TO) TT=2400.05	SAT TT=89.884	? (TO) TT=2520.06	No data	No data	? (TO) TT=2519.86	? (TO) TT=2519.94	? (TO) TT=2519.99	? (TO) TT=2400.1	? (MO) TT=388.679	? (TO) TT=2520.05	? (MO) TT=343.352	? (TO) TT=2520.13
Subisomorphism/ Subisomorphism-g17-g42_c18.xml	UNSAT TT=44.148	UNSAT TT=44.625	UNSAT TT=44.148	UNSAT TT=126.499	UNSAT TT=122.996	UNSAT TT=251.373	UNSAT TT=251.224	? (TO) TT=2520.12	? (TO) TT=2520.05	? (TO) TT=2520.05	? (TO) TT=2520.06	? (TO) TT=2520.07	UNSAT TT=124.528	UNSAT TT=124.28	? (TO) TT=2400.02	? (TO) TT=2520.07	? (TO) TT=2400.06	? (TO) TT=2520.09	? (TO) TT=2400.1	? (TO) TT=2520.04	? (TO) TT=2520.05	No data	No data	? (TO) TT=2520.03	? (TO) TT=2520	? (TO) TT=2519.92	? (TO) TT=2400.11	? (MO) TT=990.956	UNSAT TT=2132.26	? (MO) TT=987.148	UNSAT TT=2130.4
Subisomorphism/ Subisomorphism-g19-g43_c18.xml		? (TO) TT=2519.93	? (TO) TT=2520.07	? (TO) TT=2519.99	? TT=2520.07	? (TO) TT=2400.12	? (TO) TT=2520.11	? (TO) TT=2520.15	? (TO) TT=2520.15	? (TO) TT=2520.13	? (TO) TT=2520.12	? (TO) TT=2520.06	? (TO) TT=2520.05	? (TO) TT=2520.05	? (TO) TT=2400.04	? (TO) TT=2519.94	? (TO) TT=2400.06	? (TO) TT=2520.07	? (TO) TT=2400.1	? (TO) TT=2520.04	? (TO) TT=2520.04	No data	No data	? (TO) TT=2519.98	? (TO) TT=2520.09	? (TO) TT=2519.91	? (TO) TT=2400.12	? (MO) TT=760.776	? (TO) TT=2520.14	? (MO) TT=772.766	? (TO) TT=2520.13
Subisomorphism/ Subisomorphism-g19-g45_c18.xml		? (TO) TT=2520.05	? (TO) TT=2519.95	? (TO) TT=2520	? (TO) TT=2520.01	? (TO) TT=2400.11	? (TO) TT=2520.08	? (TO) TT=2520.13	? (TO) TT=2520.14	? (TO) TT=2520.1	? (TO) TT=2520.06	? (TO) TT=2519.76	? (TO) TT=2520.02	? (TO) TT=2519.91	? (TO) TT=2400.02	? (TO) TT=2520.05	? (TO) TT=2400.07	? (TO) TT=2520.05	? (TO) TT=2400.09	? (TO) TT=2520.11	? (TO) TT=2520.09	No data	No data	? (TO) TT=2519.93	? (TO) TT=2519.74	? (TO) TT=2519.89	? (TO) TT=2400.18	? (MO) TT=377.826	? (TO) TT=2520.14	? (MO) TT=377.482	? (TO) TT=2520.09
Subisomorphism/ Subisomorphism-g29-g50_c18.xml		? (TO) TT=2520	? (TO) TT=2519.91	? (TO) TT=2519.96	? (TO) TT=2520.03	? (TO) TT=2400.08	? (TO) TT=2520.1	? (TO) TT=2520.05	? (TO) TT=2520.09	? (TO) TT=2520.13	? (TO) TT=2520.08	? (TO) TT=2519.99	? (TO) TT=2519.99	? (TO) TT=2519.96	? (TO) TT=2400.02	? (TO) TT=2519.79	? (TO) TT=2400.03	? (TO) TT=2520.12	? (TO) TT=2400.04	? (TO) TT=2520.03	? (TO) TT=2520.06	No data	No data	? (TO) TT=2520.1	? (TO) TT=2519.98	? (TO) TT=2520.04	? TT=331.349	? (MO) TT=658.545	? (TO) TT=2520.18	? (MO) TT=664.144	? (TO) TT=2520.17
Subisomorphism/ Subisomorphism-g34-g36_c18.xml	UNSAT TT=0.569	UNSAT TT=3.502	UNSAT TT=3.467	UNSAT TT=4.408	UNSAT TT=4.446	UNSAT TT=12.788	UNSAT TT=12.531	UNSAT TT=32.563	UNSAT TT=28.331	UNSAT TT=28.042	UNSAT TT=31.664	UNSAT TT=0.569	UNSAT TT=4.405	UNSAT TT=4.434	UNSAT TT=17.59	UNSAT TT=11.954	UNSAT TT=8.706	UNSAT TT=8.7	? (TO) TT=2400.11	? (TO) TT=2520.04	UNSAT TT=7.198	No data	No data	UNSAT TT=109.66	UNSAT TT=111.868	UNSAT TT=110.784	? (TO) TT=2400.19	? (MO) TT=48.525	UNSAT TT=55.628	? (MO) TT=49.077	UNSAT TT=56.354
SumColoring/ SumColoring-1-fullins-3_c18.xml	OPT f=24 TT=0.517 T1=0.06	No data	No data	No data	No data	OPT f=24 TT=10.955 T1=1.15	OPT f=24 TT=9.484 T1=1.71	OPT f=24 TT=17.642 T1=4.93	OPT f=24 TT=21.976 T1=3.77	OPT f=24 TT=13.866 T1=3.58	OPT f=24 TT=28.358 T1=14.75	OPT f=24 TT=0.517 T1=0.06	No data	No data	OPT f=24 TT=0.153 T1=0.04	OPT f=24 TT=0.152 T1=0.04	No data	No data	SAT (TO) f=24 TT=2400.11 T1=14.25	SAT (TO) f=24 TT=2520.07 T1=4.92	OPT f=24 TT=75.117 T1=20.61	OPT f=24 TT=94.995 T1=80.33	OPT f=24 TT=93.824 T1=79.66	OPT f=24 TT=4.921 T1=4.92	OPT f=24 TT=4.83 T1=4.82	OPT f=24 TT=4.826 T1=4.83	? (TO) TT=2400.23	No data	No data	No data	No data
SumColoring/ SumColoring-1-fullins-4_c18.xml	OPT f=73 TT=859.495 T1=31.03	No data	No data	No data	No data	SAT (TO) f=93 TT=2400.09 T1=563.52	SAT (TO) f=93 TT=2520.07 T1=557.52	SAT (TO) f=73 TT=2520.05 T1=47.84	SAT (TO) f=73 TT=2520.12 T1=31.03	SAT (TO) f=73 TT=2520.12 T1=19.84	SAT (TO) f=86 TT=2520.15 T1=1839.17	SAT (TO) f=84 TT=2519.98 T1=23.34	No data	No data	SAT (TO) f=87 TT=2400.06 T1=34.4	SAT (TO) f=87 TT=2519.74 T1=34.63	No data	No data	SAT (TO) f=86 TT=2400.04 T1=198.08	SAT (TO) f=294 TT=2520.11 T1=2491.07	SAT (TO) f=88 TT=2520.04 T1=450.32	SAT (TO) f=91 TT=2400.11 T1=26.78	SAT (TO) f=91 TT=2520.03 T1=22.17	OPT f=73 TT=810.444 T1=810.46	OPT f=73 TT=864.321 T1=864.32	OPT f=73 TT=859.495 T1=859.46	SAT (TO) f=139 TT=2400.37 T1=3.55	No data	No data	No data	No data
SumColoring/ SumColoring-1-fullins-5_c18.xml	SAT f=227 TT=2520.1 T1=1534.83	No data	No data	No data	No data	SAT (TO) f=227 TT=2400.09 T1=1520.24	SAT (TO) f=227 TT=2520.1 T1=1534.83	SAT (TO) f=659 TT=2520.05 T1=1991.09	SAT (TO) f=284 TT=2520.08 T1=1300.68	SAT (TO) f=368 TT=2520.13 T1=1953.19	SAT (TO) f=368 TT=2520.08 T1=224.55	SAT (TO) f=290 TT=2520.05 T1=10.74	No data	No data	SAT (TO) f=280 TT=2400.1 T1=0.57	SAT (TO) f=280 TT=2520.05 T1=0.56	No data	No data	SAT (TO) f=1320 TT=2400.05 T1=2307.77	SAT (TO) f=1985 TT=2520.13 T1=1262.21	SAT (TO) f=3836 TT=2520.04 T1=2068.14	SAT (TO) f=241 TT=2400.06 T1=1196.7	SAT (TO) f=314 TT=2520.09 T1=227.96	? (TO) TT=2519.7	? (TO) TT=2520.09	? (TO) TT=2519.94	? (problem) TT=1460.83	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
SumColoring/ SumColoring-dsjc-125-1_c18.xml	SAT f=229 TT=2520.12 T1=2409.13	No data	No data	No data	No data	SAT (TO) f=231 TT=2400.11 T1=353.48	SAT (TO) f=229 TT=2520.12 T1=2409.13	SAT (TO) f=397 TT=2520.05 T1=1170.22	SAT (TO) f=367 TT=2520.04 T1=1376.75	SAT (TO) f=359 TT=2520.06 T1=1413.41	SAT (TO) f=413 TT=2520.12 T1=125.12	SAT (TO) f=241 TT=2520.07 T1=4.37	No data	No data	SAT (TO) f=260 TT=2400.05 T1=2374.44	SAT (TO) f=260 TT=2520.08 T1=2373.05	No data	No data	SAT (TO) f=779 TT=2400.05 T1=694.06	SAT (TO) f=886 TT=2520.08 T1=2069.6	SAT (TO) f=490 TT=2520.11 T1=598.47	SAT (TO) f=215 TT=2400.1 T1=1762.35	SAT (TO) f=242 TT=2520.17 T1=328.86	? (TO) TT=2520.01	? (TO) TT=2520.08	? (TO) TT=2519.89	SAT (TO) f=247 TT=2400.34 T1=7.24	No data	No data	No data	No data
SumColoring/ SumColoring-dsjc-125-5_c18.xml	SAT f=1014 TT=2520.1 T1=2208.31	No data	No data	No data	No data	SAT (TO) f=1078 TT=2400.07 T1=2211.55	SAT (TO) f=1078 TT=2520.07 T1=2236.3	SAT (TO) f=1697 TT=2520.12 T1=911.01	SAT (TO) f=1658 TT=2520.07 T1=1477.25	SAT (TO) f=1571 TT=2520.08 T1=513.18	SAT (TO) f=1469 TT=2520.06 T1=1815.84	SAT (TO) f=1123 TT=2520.06 T1=82	No data	No data	SAT (TO) f=1060 TT=2400.08 T1=1768.05	SAT (TO) f=1060 TT=2519.99 T1=1804.7	No data	No data	SAT (TO) f=1708 TT=2400.03 T1=2020.99	SAT (TO) f=1508 TT=2520.11 T1=2280.83	SAT (TO) f=3042 TT=2520.06 T1=71	SAT (TO) f=1046 TT=2400.06 T1=1675.54	SAT (TO) f=1014 TT=2520.1 T1=2208.31	? (TO) TT=2519.86	? (TO) TT=2520.02	? (TO) TT=2520.07	SAT (TO) f=1337 TT=2400.37 T1=57.77	No data	No data	No data	No data
SumColoring/ SumColoring-dsjc-125-9_c18.xml	SAT f=2523 TT=2520.1 T1=1730.38	No data	No data	No data	No data	SAT (TO) f=2772 TT=2400.08 T1=562.46	SAT (TO) f=2772 TT=2520.05 T1=552.78	? (TO) TT=2522.13	? (TO) TT=2522.19	? (TO) TT=2522.15	? (TO) TT=2522.2	SAT (TO) f=2797 TT=2519.93 T1=36.99	No data	No data	SAT (TO) f=2652 TT=2400.02 T1=2269.52	SAT (TO) f=2652 TT=2519.95 T1=2258.01	No data	No data	SAT (TO) f=4668 TT=2400.07 T1=1987.19	SAT (TO) f=3951 TT=2520.09 T1=403.75	SAT (TO) f=3702 TT=2520.1 T1=579.06	SAT (TO) f=2583 TT=2400.09 T1=2096.31	SAT (TO) f=2523 TT=2520.1 T1=1730.38	? (TO) TT=2519.85	? (TO) TT=2520.1	? (TO) TT=2520.06	? TT=934.424	No data	No data	No data	No data
SumColoring/ SumColoring-dsjc-250-1_c18.xml	SAT f=863 TT=2520.1 T1=1619.08	No data	No data	No data	No data	SAT (TO) f=906 TT=2400.08 T1=1228.97	SAT (TO) f=906 TT=2520.08 T1=1267.48	SAT (TO) f=3290 TT=2520.08 T1=424.8	SAT (TO) f=3026 TT=2520.06 T1=6.88	SAT (TO) f=2807 TT=2520.11 T1=6.67	SAT (TO) f=2735 TT=2520.15 T1=308.59	SAT (TO) f=928 TT=2520.08 T1=81.89	No data	No data	SAT (TO) f=1012 TT=2400.07 T1=1102.67	SAT (TO) f=1012 TT=2520.04 T1=1116.65	No data	No data	SAT (TO) f=4441 TT=2400.08 T1=1511.44	SAT (TO) f=4508 TT=2520.06 T1=1093.17	SAT (TO) f=3598 TT=2520.13 T1=1206.86	SAT (TO) f=848 TT=2400.1 T1=2193.31	SAT (TO) f=863 TT=2520.1 T1=1619.08	? (TO) TT=2519.74	? (TO) TT=2520.06	? (TO) TT=2520.07	? TT=1473.35	No data	No data	No data	No data
SumColoring/ SumColoring-dsjc-250-5_c18.xml	SAT f=3754 TT=2520.13 T1=2181.49	No data	No data	No data	No data	SAT (TO) f=3958 TT=2400.05 T1=479.45	SAT (TO) f=3958 TT=2520.05 T1=471.71	? (TO) TT=2522.18	? (TO) TT=2522.13	? (TO) TT=2522.14	? (TO) TT=2522.14	SAT (TO) f=3949 TT=2520.04 T1=2.54	No data	No data	SAT (TO) f=4231 TT=2400.07 T1=1925.67	SAT (TO) f=4231 TT=2520.02 T1=1972.89	No data	No data	SAT (TO) f=11058 TT=2400.11 T1=2359.37	SAT (TO) f=8272 TT=2520.09 T1=2130.99	SAT (TO) f=8741 TT=2520.1 T1=1477.96	SAT (TO) f=3662 TT=2400.08 T1=2092.52	SAT (TO) f=3754 TT=2520.13 T1=2181.49	? (TO) TT=2519.9	? (TO) TT=2520.06	? (TO) TT=2520.08	? TT=1354.32	No data	No data	No data	No data
SumColoring/ SumColoring-dsjc-250-9_c18.xml	SAT f=9554 TT=2520.08 T1=2063.34	No data	No data	No data	No data	SAT (TO) f=12513 TT=2400.05 T1=912	SAT (TO) f=12513 TT=2520.05 T1=871.77	? (TO) TT=2522.2	? (TO) TT=2522.13	? (TO) TT=2522.1	? (TO) TT=2522.18	SAT (TO) f=10090 TT=2519.97 T1=137.43	No data	No data	SAT (TO) f=9631 TT=2400.01 T1=2095.76	SAT (TO) f=9631 TT=2519.9 T1=2090.45	No data	No data	SAT (TO) f=17692 TT=2400.12 T1=1857.91	SAT (TO) f=15479 TT=2520.12 T1=1832.35	SAT (TO) f=17096 TT=2520.09 T1=1708.32	SAT (TO) f=10073 TT=2400.03 T1=2244.28	SAT (TO) f=9554 TT=2520.08 T1=2063.34	? (TO) TT=2520.09	? (TO) TT=2519.95	? (TO) TT=2520.04	? TT=870.169	No data	No data	No data	No data
SumColoring/ SumColoring-myciel3_c18.xml	OPT f=10 TT=0.009 T1=0.01	No data	No data	No data	No data	OPT f=10 TT=1.028 T1=0.48	OPT f=10 TT=1.035 T1=0.49	OPT f=10 TT=6.7 T1=3.19	OPT f=10 TT=6.755 T1=3.32	OPT f=10 TT=6.596 T1=3.18	OPT f=10 TT=6.652 T1=3.21	OPT f=10 TT=0.009 T1=0.01	No data	No data	OPT f=10 TT=0.008 T1=0	OPT f=10 TT=0.008 T1=0	No data	No data	SAT (TO) f=10 TT=2400.05 T1=1.09	SAT (TO) f=10 TT=2520.07 T1=1.05	OPT f=10 TT=1.685 T1=1.05	OPT f=10 TT=1.792 T1=1.09	OPT f=10 TT=1.705 T1=1.05	OPT f=10 TT=0.109 T1=0.1	OPT f=10 TT=0.113 T1=0.11	OPT f=10 TT=0.113 T1=0.11	OPT f=10 TT=1.16 T1=0.4	No data	No data	No data	No data
SumColoring/ SumColoring-myciel4_c18.xml	OPT f=22 TT=0.237 T1=0.07	No data	No data	No data	No data	OPT f=22 TT=6.946 T1=1.44	OPT f=22 TT=6.289 T1=1.35	OPT f=22 TT=13.529 T1=3.94	OPT f=22 TT=16.514 T1=3.66	OPT f=22 TT=12.591 T1=3.92	OPT f=22 TT=18.399 T1=3.64	OPT f=22 TT=0.237 T1=0.07	No data	No data	OPT f=22 TT=0.106 T1=0.02	OPT f=22 TT=0.105 T1=0.02	No data	No data	SAT (TO) f=22 TT=2400.03 T1=2.2	SAT (TO) f=22 TT=2520.08 T1=3.44	OPT f=22 TT=12.235 T1=2.23	OPT f=22 TT=10.555 T1=5.55	OPT f=22 TT=10.169 T1=5.38	OPT f=22 TT=2.181 T1=2.18	OPT f=22 TT=2.368 T1=2.36	OPT f=22 TT=2.361 T1=2.36	OPT f=22 TT=1331.39 T1=724.89	No data	No data	No data	No data
SumColoring/ SumColoring-myciel5_c18.xml	OPT f=46 TT=292.877 T1=72.58	No data	No data	No data	No data	SAT (TO) f=50 TT=2400.12 T1=721.38	SAT (TO) f=50 TT=2520.04 T1=742.67	SAT (TO) f=46 TT=2520.11 T1=504.01	SAT (TO) f=46 TT=2520.03 T1=122.1	SAT (TO) f=46 TT=2520.13 T1=27.65	SAT (TO) f=47 TT=2520.06 T1=6.26	OPT f=46 TT=1551.81 T1=228.59	No data	No data	OPT f=46 TT=118.274 T1=3.29	OPT f=46 TT=118.999 T1=3.32	No data	No data	SAT (TO) f=47 TT=2400.09 T1=1703.23	SAT (TO) f=46 TT=2520.03 T1=2173.79	SAT (TO) f=47 TT=2520.1 T1=336.19	SAT (TO) f=46 TT=2400.07 T1=77.92	SAT (TO) f=46 TT=2520.05 T1=72.58	OPT f=46 TT=240.015 T1=240.02	OPT f=46 TT=292.61 T1=292.6	OPT f=46 TT=292.877 T1=292.87	SAT (TO) f=71 TT=2400.4 T1=0.81	No data	No data	No data	No data
SumColoring/ SumColoring-myciel6_c18.xml	SAT f=98 TT=2520.12 T1=78.57	No data	No data	No data	No data	SAT (TO) f=107 TT=2400.09 T1=2005.48	SAT (TO) f=105 TT=2520.03 T1=2368.89	SAT (TO) f=99 TT=2520.14 T1=198.89	SAT (TO) f=98 TT=2520.12 T1=78.57	SAT (TO) f=105 TT=2520.06 T1=23.98	SAT (TO) f=96 TT=2520.12 T1=934.25	SAT (TO) f=118 TT=2519.94 T1=368.74	No data	No data	SAT (TO) f=98 TT=2400.04 T1=295.96	SAT (TO) f=98 TT=2520.06 T1=300.33	No data	No data	SAT (TO) f=183 TT=2400.08 T1=674.75	SAT (TO) f=353 TT=2520.07 T1=102.63	SAT (TO) f=311 TT=2520.04 T1=205.68	SAT (TO) f=112 TT=2400.11 T1=93.52	SAT (TO) f=100 TT=2520.07 T1=97.71	? (TO) TT=2520.12	? (TO) TT=2520.02	? (TO) TT=2520.05	SAT (TO) f=160 TT=2400.39 T1=3.59	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
SumColoring/ SumColoring-myciel7_c18.xml	SAT f=214 TT=2520.11 T1=795.09	No data	No data	No data	No data	SAT (TO) f=214 TT=2400.03 T1=809.07	SAT (TO) f=214 TT=2520.11 T1=795.09	SAT (TO) f=434 TT=2520.15 T1=840.26	SAT (TO) f=548 TT=2520.07 T1=63.54	SAT (TO) f=441 TT=2520.06 T1=681.46	SAT (TO) f=435 TT=2520.1 T1=690.01	SAT (TO) f=251 TT=2519.82 T1=0.38	No data	No data	SAT (TO) f=340 TT=2400.08 T1=1745.33	SAT (TO) f=340 TT=2519.82 T1=1732.75	No data	No data	SAT (TO) f=604 TT=2400.04 T1=748.53	SAT (TO) f=900 TT=2520.11 T1=844.96	SAT (TO) f=756 TT=2520.11 T1=2466.09	SAT (TO) f=215 TT=2400.05 T1=276.07	SAT (TO) f=251 TT=2520.09 T1=587.55	? (TO) TT=2520.09	? (TO) TT=2520.05	? (TO) TT=2519.95	? TT=1363.61	No data	No data	No data	No data
Tal/ Tal-01_c18.xml	OPT f=6 TT=0.031 T1=0.03	No data	No data	No data	No data	OPT f=6 TT=1.526 T1=0.7	OPT f=6 TT=1.593 T1=0.71	OPT f=6 TT=9.794 T1=4.21	OPT f=6 TT=10.194 T1=4.22	OPT f=6 TT=9.405 T1=4.25	OPT f=6 TT=9.405 T1=4.19	OPT f=6 TT=0.031 T1=0.03	No data	No data	Wrong UNSAT TT=0.144	OPT f=6 TT=0.202 T1=0.19	No data	No data	SAT (TO) f=6 TT=2400.05 T1=1.29	SAT (TO) f=6 TT=2520.02 T1=1.24	OPT f=6 TT=2.153 T1=1.23	OPT f=6 TT=2.232 T1=1.29	OPT f=6 TT=2.169 T1=1.24	OPT f=6 TT=0.559 T1=0.55	OPT f=6 TT=0.582 T1=0.58	OPT f=6 TT=0.5 T1=0.5	OPT f=6 TT=180.414 T1=170.71	No data	No data	No data	No data
Tal/ Tal-02_c18.xml	OPT f=86 TT=3.308 T1=0.79	No data	No data	No data	No data	Wrong UNSAT TT=2.304	OPT f=86 TT=7.483 T1=2.06	OPT f=86 TT=23.368 T1=6.89	OPT f=86 TT=22.121 T1=5.75	OPT f=86 TT=21.386 T1=7.31	OPT f=86 TT=21.505 T1=6.14	OPT f=86 TT=3.308 T1=0.79	No data	No data	? (MO) TT=18.345	? (NS) TT=0.021	No data	No data	? (TO) TT=2400.09	SAT (TO) f=86 TT=2520.07 T1=1.61	OPT f=86 TT=3.974 T1=1.63	Wrong UNSAT TT=3.36	OPT f=86 TT=4.444 T1=1.55	OPT f=86 TT=20.071 T1=20.08	OPT f=86 TT=21.184 T1=21.18	OPT f=86 TT=22.704 T1=22.7	? (TO) TT=2400.23	No data	No data	No data	No data
Tal/ Tal-03_c18.xml	OPT f=96 TT=141.932 T1=7.93	No data	No data	No data	No data	Wrong UNSAT TT=2.442	OPT f=96 TT=248.867 T1=190.65	OPT f=96 TT=1485.8 T1=402.76	OPT f=96 TT=727.119 T1=19.05	OPT f=96 TT=1581.97 T1=34.99	OPT f=96 TT=655.182 T1=152.01	OPT f=96 TT=1294.31 T1=1183.12	No data	No data	? (MO) TT=17.815	? (NS) TT=0.026	No data	No data	? (TO) TT=2400.11	SAT (TO) f=96 TT=2520.12 T1=12.52	OPT f=96 TT=141.932 T1=12.53	Wrong UNSAT TT=3.693	SAT (TO) f=96 TT=2520.06 T1=7.93	OPT f=96 TT=149.507 T1=149.51	OPT f=96 TT=182.25 T1=182.22	OPT f=96 TT=211.213 T1=211.24	? (TO) TT=2400.15	No data	No data	No data	No data
Tal/ Tal-04_c18.xml	OPT f=142 TT=1754.65 T1=29.15	No data	No data	No data	No data	Wrong Opt. f=190 TT=7.126 T1=2.19	SAT (TO) f=142 TT=2520.08 T1=29.15	SAT (TO) f=142 TT=2520.06 T1=517.27	SAT (TO) f=142 TT=2520.12 T1=501.96	SAT (TO) f=142 TT=2520.08 T1=12.65	SAT (TO) f=142 TT=2520.08 T1=17.02	SAT (TO) f=142 TT=2520.09 T1=1600.42	No data	No data	? (NS) TT=19.246	? (MO) TT=25.71	No data	No data	Wrong UNSAT TT=3.846	SAT (TO) f=142 TT=2520.04 T1=239.79	OPT f=142 TT=1754.65 T1=57.62	Wrong UNSAT TT=3.812	SAT (TO) f=146 TT=2520.13 T1=52.37	? (TO) TT=2520.05	? (TO) TT=2519.99	? (TO) TT=2519.89	? (TO) TT=2400.18	No data	No data	No data	No data
Tal/ Tal-05_c18.xml	OPT f=16 TT=48.312 T1=3.35	No data	No data	No data	No data	Wrong UNSAT TT=2.725	OPT f=16 TT=319.626 T1=3.35	OPT f=16 TT=981.089 T1=133.01	OPT f=16 TT=438.679 T1=64.19	OPT f=16 TT=446.217 T1=9.57	OPT f=16 TT=454.988 T1=39.59	OPT f=16 TT=1555.1 T1=127.94	No data	No data	? (MO) TT=18.826	? (MO) TT=24.134	No data	No data	? (TO) TT=2400.09	SAT (TO) f=16 TT=2520.04 T1=7.72	OPT f=16 TT=48.312 T1=5.71	Wrong UNSAT TT=3.871	SAT (TO) f=16 TT=2520.04 T1=6.74	? (TO) TT=2519.9	? (TO) TT=2519.58	? (TO) TT=2519.94	? (TO) TT=2400.13	No data	No data	No data	No data
Tal/ Tal-06_c18.xml	SAT f=94 TT=2520.12 T1=36.86	No data	No data	No data	No data	Wrong UNSAT TT=2.897	SAT (TO) f=114 TT=2520.05 T1=18.03	SAT (TO) f=94 TT=2520.12 T1=36.86	SAT (TO) f=94 TT=2520.12 T1=52.28	SAT (TO) f=118 TT=2520.08 T1=60.15	SAT (TO) f=94 TT=2520.05 T1=76.88	SAT (TO) f=106 TT=2519.97 T1=21.97	No data	No data	? (NS) TT=15.392	? (NS) TT=0.039	No data	No data	Wrong UNSAT TT=3.783	SAT (TO) f=106 TT=2520.06 T1=135.46	SAT (TO) f=106 TT=2520.12 T1=12.54	Wrong UNSAT TT=4.026	SAT (TO) f=106 TT=2520.03 T1=405.45	? (TO) TT=2520.09	? (TO) TT=2520.06	? (TO) TT=2519.66	? (TO) TT=2400.13	No data	No data	No data	No data
Tal/ Tal-07_c18.xml	SAT f=126 TT=2520.1 T1=79.65	No data	No data	No data	No data	Wrong UNSAT TT=2.985	SAT (TO) f=166 TT=2520.1 T1=1420.29	SAT (TO) f=138 TT=2520.09 T1=162.68	SAT (TO) f=126 TT=2520.1 T1=79.65	SAT (TO) f=138 TT=2520.07 T1=27.12	SAT (TO) f=158 TT=2520.07 T1=2050.47	? (TO) TT=2520.08	No data	No data	? (NS) TT=15.321	? (NS) TT=0.046	No data	No data	Wrong UNSAT TT=4.098	SAT (TO) f=138 TT=2520.07 T1=915.68	SAT (TO) f=138 TT=2520.09 T1=474.9	Wrong UNSAT TT=4.054	SAT (TO) f=134 TT=2520.43 T1=19.21	? (TO) TT=2519.97	? (TO) TT=2519.56	? (TO) TT=2520.13	? (TO) TT=2400.18	No data	No data	No data	No data
Tal/ Tal-08_c18.xml	SAT f=178 TT=2520.09 T1=224.8	No data	No data	No data	No data	Wrong UNSAT TT=3.179	SAT (TO) f=224 TT=2520.1 T1=1993.56	? (TO) TT=2520.08	? (TO) TT=2520.07	SAT (TO) f=212 TT=2520.09 T1=179.84	SAT (TO) f=230 TT=2520.06 T1=1911.31	? (TO) TT=2520.07	No data	No data	? (NS) TT=14.742	? (NS) TT=0.048	No data	No data	Wrong UNSAT TT=4.353	SAT (TO) f=180 TT=2520.06 T1=1465.51	SAT (TO) f=178 TT=2520.11 T1=224.8	Wrong UNSAT TT=4.317	SAT (TO) f=178 TT=2520.09 T1=1480.32	? (TO) TT=2519.2	? (TO) TT=2519.68	? (TO) TT=2519.98	? (TO) TT=2400.2	No data	No data	No data	No data
Tal/ Tal-09_c18.xml	SAT f=86 TT=2520.04 T1=413.64	No data	No data	No data	No data	Wrong UNSAT TT=3.092	SAT (TO) f=130 TT=2520.06 T1=1709.7	SAT (TO) f=128 TT=2520.13 T1=73.35	SAT (TO) f=158 TT=2520.09 T1=1681.38	SAT (TO) f=116 TT=2520.12 T1=2285.89	SAT (TO) f=136 TT=2520.12 T1=1571.22	SAT (TO) f=206 TT=2520.04 T1=0.53	No data	No data	? (NS) TT=15.368	? (NS) TT=0.049	No data	No data	Wrong UNSAT TT=4.369	SAT (TO) f=154 TT=2520.06 T1=403.37	SAT (TO) f=114 TT=2520.09 T1=741.26	Wrong UNSAT TT=4.422	SAT (TO) f=86 TT=2520.04 T1=413.64	? (TO) TT=2519.94	? (TO) TT=2520.11	? (TO) TT=2519.97	? (TO) TT=2400.12	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
Tal/ Tal-10_c18.xml	SAT f=258 TT=2520.1 T1=147.29	No data	No data	No data	No data	Wrong UNSAT TT=3.734	? (TO) TT=2520.04	? (TO) TT=2520.13	? (TO) TT=2520.06	? (TO) TT=2520.06	? (TO) TT=2520.08	? (TO) TT=2519.91	No data	No data	? (NS) TT=15.65	? (NS) TT=0.062	No data	No data	Wrong UNSAT TT=4.59	SAT (TO) f=280 TT=2520.13 T1=320.17	SAT (TO) f=258 TT=2520.1 T1=147.29	Wrong UNSAT TT=4.592	? (TO) TT=2520.05	? (TO) TT=2519.84	? (exit code) TT=72.117	? (exit code) TT=65.099	? (TO) TT=2400.17	No data	No data	No data	No data
TemplateDesign/ TemplateDesign-m1-1_c18.xml	OPT f=2 TT=0.243 T1=0.24	No data	No data	No data	No data	OPT f=2 TT=1.185 T1=0.57	OPT f=2 TT=1.226 T1=0.58	OPT f=2 TT=7.882 T1=3.82	OPT f=2 TT=7.923 T1=3.88	OPT f=2 TT=7.686 T1=3.79	OPT f=2 TT=8.054 T1=3.84	OPT f=2 TT=15.961 T1=15.66	No data	No data	OPT f=2 TT=0.012 T1=0.01	OPT f=2 TT=0.011 T1=0.01	No data	No data	SAT (TO) f=2 TT=2400.1 T1=1.13	SAT (TO) f=2 TT=2520.09 T1=1.05	OPT f=2 TT=1.631 T1=1.06	OPT f=2 TT=1.703 T1=1.11	OPT f=2 TT=1.603 T1=1.04	OPT f=2 TT=0.248 T1=0.24	OPT f=2 TT=0.258 T1=0.25	OPT f=2 TT=0.243 T1=0.24	OPT f=2 TT=80.939 T1=50.95	No data	No data	No data	No data
TemplateDesign/ TemplateDesign-m1-2_c18.xml	OPT f=2 TT=0.654 T1=0.65	No data	No data	No data	No data	OPT f=2 TT=1.501 T1=0.64	OPT f=2 TT=1.512 T1=0.65	OPT f=2 TT=9.613 T1=4.26	OPT f=2 TT=9.466 T1=4.29	OPT f=2 TT=8.853 T1=4.09	OPT f=2 TT=8.782 T1=4.11	? (TO) TT=2519.79	No data	No data	OPT f=2 TT=0.019 T1=0.01	OPT f=2 TT=0.02 T1=0.01	No data	No data	SAT (TO) f=2 TT=2400.03 T1=1.17	SAT (TO) f=2 TT=2520.08 T1=1.09	OPT f=2 TT=1.724 T1=1.1	OPT f=2 TT=1.939 T1=1.19	OPT f=2 TT=1.794 T1=1.11	OPT f=2 TT=0.656 T1=0.65	OPT f=2 TT=0.671 T1=0.67	OPT f=2 TT=0.654 T1=0.65	? (TO) TT=2408.19	No data	No data	No data	No data
TemplateDesign/ TemplateDesign-m1-catfood_c18.xml	OPT f=2 TT=1.739 T1=0.73	No data	No data	No data	No data	OPT f=2 TT=1.815 T1=0.75	OPT f=2 TT=1.739 T1=0.73	OPT f=2 TT=11.79 T1=4.9	OPT f=2 TT=11.581 T1=4.9	OPT f=2 TT=12.615 T1=5.45	OPT f=2 TT=10.478 T1=4.58	? (TO) TT=2519.54	No data	No data	OPT f=2 TT=0.03 T1=0.03	OPT f=2 TT=0.031 T1=0.03	No data	No data	SAT (TO) f=2 TT=2400.04 T1=1.28	SAT (TO) f=2 TT=2520.12 T1=1.18	OPT f=2 TT=2.039 T1=1.18	OPT f=2 TT=13.289 T1=10.8	OPT f=2 TT=12.985 T1=10.7	OPT f=2 TT=3.504 T1=3.5	OPT f=2 TT=3.811 T1=3.81	OPT f=2 TT=3.426 T1=3.42	? (problem) TT=1525.28	No data	No data	No data	No data
TemplateDesign/ TemplateDesign-m1-herbs_c18.xml	OPT f=2 TT=97.654 T1=97.65	No data	No data	No data	No data	? (TO) TT=2400.06	? (TO) TT=2520.09	? (TO) TT=2520.11	? (TO) TT=2520.12	? (TO) TT=2520.1	? (TO) TT=2520.14	? (TO) TT=2519.79	No data	No data	? (TO) TT=2400.03	? (TO) TT=2520.03	No data	No data	? (TO) TT=2400.09	? (TO) TT=2520.03	? (TO) TT=2520.1	? (TO) TT=2400.06	? (TO) TT=2520.06	OPT f=2 TT=160.429 T1=160.42	OPT f=2 TT=98.548 T1=98.57	OPT f=2 TT=97.654 T1=97.65	? TT=1591.15	No data	No data	No data	No data
TemplateDesign/ TemplateDesign-m1-magazine_c18.xml		No data	No data	No data	No data	? (TO) TT=2400.04	? (TO) TT=2520.1	? (TO) TT=2520.14	? (TO) TT=2520.11	? (TO) TT=2520.11	? (TO) TT=2520.13	? (TO) TT=2519.81	No data	No data	? (TO) TT=2400.02	? (TO) TT=2520.03	No data	No data	? (TO) TT=2400.07	? (TO) TT=2520.03	? (TO) TT=2520.06	? (TO) TT=2400.09	? (TO) TT=2520.03	? (TO) TT=2519.84	? (TO) TT=2519.75	? (TO) TT=2520.08	? TT=1670.43	No data	No data	No data	No data
TemplateDesign/ TemplateDesign-m1s-1_c18.xml	OPT f=2 TT=1.211 T1=0.58	No data	No data	No data	No data	OPT f=2 TT=1.174 T1=0.57	OPT f=2 TT=1.211 T1=0.58	OPT f=2 TT=7.457 T1=3.66	OPT f=2 TT=7.435 T1=3.69	OPT f=2 TT=7.21 T1=3.62	OPT f=2 TT=7.365 T1=3.66	SAT (TO) f=2 TT=2519.56 T1=52.98	No data	No data	OPT f=2 TT=0.012 T1=0.01	OPT f=2 TT=0.012 T1=0.01	No data	No data	SAT (TO) f=2 TT=2400.06 T1=1.19	SAT (TO) f=2 TT=2520.11 T1=1.13	OPT f=2 TT=1.821 T1=1.13	OPT f=2 TT=1.942 T1=1.2	OPT f=2 TT=1.843 T1=1.12	OPT f=2 TT=1.181 T1=1.18	OPT f=2 TT=1.217 T1=1.22	OPT f=2 TT=1.594 T1=1.59	OPT f=2 TT=77.551 T1=22.9	No data	No data	No data	No data
TemplateDesign/ TemplateDesign-m1s-2_c18.xml	OPT f=2 TT=1.381 T1=0.63	No data	No data	No data	No data	OPT f=2 TT=1.384 T1=0.62	OPT f=2 TT=1.381 T1=0.63	OPT f=2 TT=11.225 T1=4.03	OPT f=2 TT=8.678 T1=4.08	OPT f=2 TT=8.238 T1=3.89	OPT f=2 TT=8.199 T1=3.91	SAT (TO) f=2 TT=2519.77 T1=51.11	No data	No data	OPT f=2 TT=0.023 T1=0.01	OPT f=2 TT=0.023 T1=0.01	No data	No data	SAT (TO) f=2 TT=2400.1 T1=1.27	SAT (TO) f=2 TT=2520.04 T1=1.18	OPT f=2 TT=1.986 T1=1.18	OPT f=2 TT=2.235 T1=1.29	OPT f=2 TT=2.091 T1=1.23	OPT f=2 TT=3.181 T1=3.18	OPT f=2 TT=3.063 T1=3.06	OPT f=2 TT=3.473 T1=3.47	? (problem) TT=1356.04	No data	No data	No data	No data
TemplateDesign/ TemplateDesign-m1s-catfood_c18.xml	OPT f=2 TT=1.63 T1=0.67	No data	No data	No data	No data	OPT f=2 TT=1.662 T1=0.69	OPT f=2 TT=1.63 T1=0.67	OPT f=2 TT=11.271 T1=5.15	OPT f=2 TT=13.486 T1=4.53	OPT f=2 TT=9.538 T1=4.29	OPT f=2 TT=9.313 T1=4.19	? (TO) TT=2519.81	No data	No data	OPT f=2 TT=0.046 T1=0.03	OPT f=2 TT=0.046 T1=0.03	No data	No data	SAT (TO) f=2 TT=2400.1 T1=1.38	SAT (TO) f=2 TT=2520.09 T1=1.27	OPT f=2 TT=2.407 T1=1.3	OPT f=2 TT=2.665 T1=1.41	OPT f=2 TT=2.388 T1=1.28	OPT f=2 TT=11.772 T1=11.77	OPT f=2 TT=10.548 T1=10.55	OPT f=2 TT=9.987 T1=9.98	? (problem) TT=1293.6	No data	No data	No data	No data
TemplateDesign/ TemplateDesign-m1s-herbs_c18.xml	OPT f=2 TT=163.517 T1=43.4	No data	No data	No data	No data	OPT f=2 TT=162.793 T1=68.5	OPT f=2 TT=163.517 T1=68.63	SAT (TO) f=2 TT=2520.08 T1=43.4	SAT (TO) f=3 TT=2520.14 T1=773.27	SAT (TO) f=4 TT=2520.08 T1=90.78	OPT f=2 TT=23.597 T1=8.68	? (TO) TT=2519.8	No data	No data	SAT (TO) f=2 TT=2400.01 T1=12.74	SAT (TO) f=2 TT=2519.97 T1=12.88	No data	No data	SAT (TO) f=9 TT=2400.05 T1=1734.96	SAT (TO) f=8 TT=2520.08 T1=1826.11	SAT (TO) f=13 TT=2520.1 T1=299.91	SAT f=2 TT=487.903 T1=449.71	SAT (TO) f=3 TT=2520.09 T1=335.16	? (TO) TT=2519.98	OPT f=2 TT=2264.21 T1=2264.26	? (TO) TT=2520.1	? TT=1764	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
TemplateDesign/ TemplateDesign-m1s-magazine_c18.xml	SAT f=5 TT=2520.06 T1=2065.31	No data	No data	No data	No data	SAT (TO) f=14 TT=2400.11 T1=287.9	SAT (TO) f=14 TT=2520.06 T1=272.5	? (TO) TT=2520.12	? (TO) TT=2520.2	? (TO) TT=2520.15	? (TO) TT=2520.12	? (TO) TT=2519.62	No data	No data	SAT (TO) f=5 TT=2400.1 T1=27.93	SAT (TO) f=5 TT=2519.92 T1=27.89	No data	No data	? (TO) TT=2400.05	SAT (TO) f=23 TT=2520.1 T1=758.48	? (TO) TT=2520.11	SAT (TO) f=6 TT=2400.08 T1=921.65	SAT (TO) f=5 TT=2520.06 T1=2065.31	? (TO) TT=2520.03	? (TO) TT=2519.79	? (TO) TT=2519.83	? (problem) TT=1590.54	No data	No data	No data	No data
TemplateDesign/ TemplateDesign-m2-1_c18.xml	OPT f=2 TT=1.097 T1=0.54	No data	No data	No data	No data	OPT f=2 TT=1.082 T1=0.53	OPT f=2 TT=1.097 T1=0.54	OPT f=2 TT=8.032 T1=3.84	OPT f=2 TT=7.841 T1=3.87	OPT f=2 TT=7.847 T1=4.68	OPT f=2 TT=7.877 T1=3.82	OPT f=2 TT=391.473 T1=391.48	No data	No data	OPT f=2 TT=0.008 T1=0	OPT f=2 TT=0.008 T1=0	No data	No data	SAT (TO) f=2 TT=2400.03 T1=1.07	SAT (TO) f=2 TT=2520.06 T1=1.02	OPT f=2 TT=1.567 T1=1.02	OPT f=2 TT=1.682 T1=1.1	OPT f=2 TT=1.575 T1=1.03	OPT f=2 TT=1.399 T1=1.39	OPT f=2 TT=1.382 T1=1.38	OPT f=2 TT=1.381 T1=1.38	? (problem) TT=1865.39	No data	No data	No data	No data
TemplateDesign/ TemplateDesign-m2-2_c18.xml	OPT f=2 TT=0.651 T1=0.56	No data	No data	No data	No data	OPT f=2 TT=1.223 T1=0.57	OPT f=2 TT=1.193 T1=0.56	OPT f=2 TT=9.337 T1=4.22	OPT f=2 TT=9.63 T1=4.28	OPT f=2 TT=8.664 T1=4.06	OPT f=2 TT=8.865 T1=4.11	? (TO) TT=2519.63	No data	No data	OPT f=2 TT=0.011 T1=0.01	OPT f=2 TT=0.01 T1=0.01	No data	No data	SAT (TO) f=2 TT=2400.06 T1=1.12	SAT (TO) f=2 TT=2520.09 T1=1.07	OPT f=2 TT=1.676 T1=1.16	OPT f=2 TT=1.957 T1=1.18	OPT f=2 TT=1.827 T1=1.11	OPT f=2 TT=0.679 T1=0.67	OPT f=2 TT=0.657 T1=0.65	OPT f=2 TT=0.651 T1=0.65	? (problem) TT=1422.03	No data	No data	No data	No data
TemplateDesign/ TemplateDesign-m2-catfood_c18.xml	OPT f=2 TT=1.345 T1=0.6	No data	No data	No data	No data	OPT f=2 TT=1.371 T1=0.6	OPT f=2 TT=1.345 T1=0.6	OPT f=2 TT=11.531 T1=4.86	OPT f=2 TT=10.793 T1=5.2	OPT f=2 TT=12.457 T1=5.8	OPT f=2 TT=9.824 T1=4.52	? (TO) TT=2519.81	No data	No data	OPT f=2 TT=0.014 T1=0.01	OPT f=2 TT=0.014 T1=0.01	No data	No data	SAT (TO) f=2 TT=2400.05 T1=14.74	SAT (TO) f=2 TT=2520.04 T1=13.86	OPT f=2 TT=16.621 T1=14.11	OPT f=2 TT=5.924 T1=4.27	OPT f=2 TT=5.921 T1=3.91	OPT f=2 TT=2.745 T1=2.74	OPT f=2 TT=2.791 T1=2.79	OPT f=2 TT=2.773 T1=2.77	? TT=1790.02	No data	No data	No data	No data
TemplateDesign/ TemplateDesign-m2-herbs_c18.xml	OPT f=2 TT=32.292 T1=14.11	No data	No data	No data	No data	OPT f=2 TT=429.241 T1=422.13	OPT f=2 TT=441.875 T1=434.08	OPT f=2 TT=32.292 T1=14.11	? (TO) TT=2520.09	? (TO) TT=2520.06	SAT (TO) f=13 TT=2520.12 T1=805.82	? (TO) TT=2519.37	No data	No data	? (TO) TT=2400.01	? (TO) TT=2519.93	No data	No data	? (TO) TT=2400.08	? (TO) TT=2520.1	? (TO) TT=2520.03	? (TO) TT=2400.1	? (TO) TT=2520.05	OPT f=2 TT=737.978 T1=737.98	OPT f=2 TT=539.617 T1=539.59	OPT f=2 TT=516.38 T1=516.35	? TT=1736.46	No data	No data	No data	No data
TemplateDesign/ TemplateDesign-m2-magazine_c18.xml		No data	No data	No data	No data	? (TO) TT=2400.12	? (TO) TT=2520.13	? (TO) TT=2520.12	? (TO) TT=2520.11	? (TO) TT=2520.13	? (TO) TT=2520.09	? (TO) TT=2519.77	No data	No data	? (TO) TT=2400.01	? (TO) TT=2520.05	No data	No data	? (TO) TT=2400.05	? (TO) TT=2520.1	? (TO) TT=2520.02	? (TO) TT=2400.04	? (TO) TT=2520.1	? (TO) TT=2519.93	? (TO) TT=2520.04	? (TO) TT=2519.91	? (problem) TT=1465.82	No data	No data	No data	No data
TravelingTournament/ TravelingTournament-a2-galaxy04_c18.xml	OPT f=517 TT=2.736 T1=0.84	No data	No data	No data	No data	OPT f=517 TT=4.049 T1=0.84	OPT f=517 TT=4.081 T1=0.84	OPT f=517 TT=16.333 T1=4.99	OPT f=517 TT=17.486 T1=4.91	OPT f=517 TT=15.925 T1=4.87	OPT f=517 TT=16.458 T1=6.19	? (NS) TT=0.005	No data	No data	OPT f=517 TT=0.222 T1=0.06	Signal TT=0.04	No data	No data	SAT (TO) f=517 TT=2400.04 T1=1.89	SAT (TO) f=517 TT=2520.11 T1=1.49	OPT f=517 TT=2.789 T1=1.49	OPT f=517 TT=2.827 T1=1.43	OPT f=517 TT=2.736 T1=1.35	OPT f=517 TT=4.804 T1=4.8	OPT f=517 TT=4.594 T1=4.59	OPT f=517 TT=4.609 T1=4.6	OPT f=517 TT=10.566 T1=4.81	No data	No data	No data	No data
TravelingTournament/ TravelingTournament-a2-galaxy06_c18.xml	SAT f=1582 TT=2520.1 T1=8.1	No data	No data	No data	No data	SAT (TO) f=1637 TT=2400.04 T1=1.94	SAT (TO) f=1637 TT=2520.08 T1=1.67	SAT (TO) f=1587 TT=2520.1 T1=198.28	SAT (TO) f=1591 TT=2520.05 T1=458.17	SAT (TO) f=1599 TT=2520.1 T1=900.49	SAT (TO) f=1592 TT=2520.05 T1=2034.2	? (NS) TT=0.005	No data	No data	SAT (TO) f=1604 TT=2400.07 T1=2301.24	Signal TT=0.143	No data	No data	SAT (TO) f=1588 TT=2400.06 T1=2024.84	SAT (TO) f=1583 TT=2520.1 T1=277.03	SAT (TO) f=1584 TT=2520.06 T1=2431.7	SAT (TO) f=1593 TT=2400.12 T1=2025.92	SAT (TO) f=1582 TT=2520.1 T1=8.1	? (TO) TT=2519.92	? (TO) TT=2520.06	? (TO) TT=2519.82	SAT (TO) f=1622 TT=2400.18 T1=1304.1	No data	No data	No data	No data
TravelingTournament/ TravelingTournament-a2-galaxy08_c18.xml	SAT f=3179 TT=2520.05 T1=374.68	No data	No data	No data	No data	SAT (TO) f=3246 TT=2400.05 T1=690.63	SAT (TO) f=3246 TT=2520.02 T1=683.81	SAT (TO) f=3179 TT=2520.05 T1=374.68	SAT (TO) f=3344 TT=2520.11 T1=1770.09	SAT (TO) f=3252 TT=2520.07 T1=46.62	SAT (TO) f=3193 TT=2520.09 T1=289.81	? (NS) TT=0.007	No data	No data	SAT (TO) f=3188 TT=2400.02 T1=641.28	Signal TT=0.381	No data	No data	SAT (TO) f=3272 TT=2400.07 T1=1081.86	SAT (TO) f=3288 TT=2520.1 T1=168.88	SAT (TO) f=3261 TT=2520.04 T1=158.45	SAT (TO) f=3242 TT=2400.12 T1=2238.53	SAT (TO) f=3343 TT=2520.05 T1=1908.42	? (TO) TT=2520.05	? (TO) TT=2520.09	? (TO) TT=2520.04	SAT (TO) f=3544 TT=2400.19 T1=2251.3	No data	No data	No data	No data
TravelingTournament/ TravelingTournament-a2-galaxy10_c18.xml	SAT f=6276 TT=2520.11 T1=2287.18	No data	No data	No data	No data	SAT (TO) f=6382 TT=2400.12 T1=78.82	SAT (TO) f=6362 TT=2520.09 T1=2502.53	SAT (TO) f=6456 TT=2520.05 T1=2024.04	SAT (TO) f=6558 TT=2520.06 T1=491.97	SAT (TO) f=6315 TT=2520.15 T1=106.87	SAT (TO) f=6514 TT=2520.11 T1=673.11	? (NS) TT=0.009	No data	No data	SAT (TO) f=6028 TT=2400.07 T1=1621.35	SAT (TO) f=6026 TT=2519.91 T1=1155.81	No data	No data	SAT (TO) f=6490 TT=2400.1 T1=1753.05	SAT (TO) f=6458 TT=2520.06 T1=176.2	SAT (TO) f=6363 TT=2520.09 T1=2354.23	SAT (TO) f=6636 TT=2400.1 T1=14.79	SAT (TO) f=6276 TT=2520.11 T1=2287.18	? (TO) TT=2519.87	? (TO) TT=2520.1	? (TO) TT=2520.05	? (TO) TT=2400.17	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
TravelingTournament/ TravelingTournament-a2-galaxy12_c18.xml	SAT f=10735 TT=2520.05 T1=1401.7	No data	No data	No data	No data	SAT (TO) f=10796 TT=2400.06 T1=1490.39	SAT (TO) f=10796 TT=2520.07 T1=1485.61	SAT (TO) f=10735 TT=2520.05 T1=1401.7	SAT (TO) f=11113 TT=2520.12 T1=276.02	SAT (TO) f=10821 TT=2520.07 T1=849.68	SAT (TO) f=10930 TT=2520.14 T1=418.57	? (NS) TT=0.012	No data	No data	SAT (TO) f=10418 TT=2400.11 T1=1760.81	Signal TT=1.618	No data	No data	SAT (TO) f=11026 TT=2400.13 T1=991.9	SAT (TO) f=11092 TT=2520.06 T1=238.03	SAT (TO) f=11042 TT=2520.11 T1=1059.25	SAT (TO) f=10787 TT=2400.12 T1=1376.32	SAT (TO) f=10992 TT=2520.06 T1=92.2	? (TO) TT=2519.91	? (TO) TT=2519.83	? (TO) TT=2519.9	? (TO) TT=2400.12	No data	No data	No data	No data
TravelingTournament/ TravelingTournament-a2-galaxy14_c18.xml	SAT f=16618 TT=2520.05 T1=599.09	No data	No data	No data	No data	SAT (TO) f=16618 TT=2400.03 T1=587.26	SAT (TO) f=16618 TT=2520.05 T1=599.09	SAT (TO) f=17157 TT=2520.11 T1=419.84	SAT (TO) f=17227 TT=2520.16 T1=2221.61	SAT (TO) f=17038 TT=2520.13 T1=237	SAT (TO) f=17284 TT=2520.17 T1=1066.74	? (NS) TT=0.016	No data	No data	SAT (TO) f=16607 TT=2400.07 T1=2299.41	Signal TT=2.708	No data	No data	SAT (TO) f=17214 TT=2400.1 T1=1857.59	SAT (TO) f=17406 TT=2520.11 T1=2368.25	SAT (TO) f=17424 TT=2520.11 T1=1181.76	SAT (TO) f=17658 TT=2400.13 T1=109.12	SAT (TO) f=17141 TT=2520.09 T1=1748.26	? (TO) TT=2520.07	? (TO) TT=2520.07	? (TO) TT=2520.03	? (TO) TT=2400.22	No data	No data	No data	No data
TravelingTournament/ TravelingTournament-a2-galaxy16_c18.xml	SAT f=23338 TT=2520.06 T1=364.76	No data	No data	No data	No data	SAT (TO) f=23338 TT=2400.1 T1=350.35	SAT (TO) f=23338 TT=2520.06 T1=364.76	SAT (TO) f=24073 TT=2520.1 T1=2132.08	SAT (TO) f=24497 TT=2520.13 T1=280.13	SAT (TO) f=24182 TT=2520.16 T1=420.29	SAT (TO) f=24404 TT=2520.14 T1=2029.07	? (NS) TT=0.021	No data	No data	SAT (TO) f=23492 TT=2400.02 T1=1793.42	Signal TT=4.555	No data	No data	? (TO) TT=2400.05	SAT (TO) f=24761 TT=2520.08 T1=1136.79	SAT (TO) f=24794 TT=2520.15 T1=1910.89	SAT (TO) f=24581 TT=2400.13 T1=45.76	SAT (TO) f=24581 TT=2520.09 T1=47.54	? (TO) TT=2520.04	? (TO) TT=2520.03	? (TO) TT=2520.07	? (TO) TT=2400.14	No data	No data	No data	No data
TravelingTournament/ TravelingTournament-a3-galaxy04_c18.xml	OPT f=416 TT=2.868 T1=1.23	No data	No data	No data	No data	OPT f=416 TT=6.369 T1=1.23	OPT f=416 TT=6.278 T1=1.23	OPT f=416 TT=17.63 T1=7.12	OPT f=416 TT=19.482 T1=7.3	OPT f=416 TT=18.05 T1=4.59	OPT f=416 TT=18.053 T1=6.23	? (NS) TT=0.005	No data	No data	OPT f=416 TT=0.28 T1=0.24	Signal TT=0.041	No data	No data	SAT (TO) f=416 TT=2400.08 T1=1.45	SAT (TO) f=416 TT=2520.03 T1=1.38	OPT f=416 TT=2.868 T1=1.35	OPT f=416 TT=3.001 T1=1.43	OPT f=416 TT=2.896 T1=1.37	OPT f=416 TT=5.152 T1=5.15	OPT f=416 TT=4.537 T1=4.54	OPT f=416 TT=4.491 T1=4.49	OPT f=416 TT=15.587 T1=5.48	No data	No data	No data	No data
TravelingTournament/ TravelingTournament-a3-galaxy06_c18.xml	SAT f=1401 TT=2520.06 T1=2172.96	No data	No data	No data	No data	SAT (TO) f=1414 TT=2400.07 T1=2234.35	SAT (TO) f=1414 TT=2520.07 T1=2234.55	SAT (TO) f=1442 TT=2520.13 T1=146.63	SAT (TO) f=1413 TT=2520.05 T1=133.84	SAT (TO) f=1407 TT=2520.07 T1=431.67	SAT (TO) f=1417 TT=2520.09 T1=51.25	? (NS) TT=0.005	No data	No data	SAT (TO) f=1397 TT=2400.09 T1=1110.34	Signal TT=0.148	No data	No data	SAT (TO) f=1419 TT=2400.09 T1=1285.56	SAT (TO) f=1407 TT=2520.07 T1=591.32	SAT (TO) f=1401 TT=2520.06 T1=2172.96	SAT (TO) f=1463 TT=2400.1 T1=76.57	SAT (TO) f=1403 TT=2520.02 T1=476.4	? (TO) TT=2520.05	? (TO) TT=2519.96	? (TO) TT=2520.06	SAT (TO) f=1504 TT=2400.17 T1=1794.91	No data	No data	No data	No data
TravelingTournament/ TravelingTournament-a3-galaxy08_c18.xml	SAT f=2888 TT=2520.05 T1=689.51	No data	No data	No data	No data	SAT (TO) f=2959 TT=2400.08 T1=559.96	SAT (TO) f=2959 TT=2520.03 T1=565.06	SAT (TO) f=2888 TT=2520.05 T1=689.51	SAT (TO) f=2922 TT=2520.06 T1=148.97	SAT (TO) f=2839 TT=2520.1 T1=1078.57	SAT (TO) f=2966 TT=2520.1 T1=1307.32	? (NS) TT=0.007	No data	No data	SAT (TO) f=2673 TT=2400.03 T1=2138.26	Signal TT=0.386	No data	No data	SAT (TO) f=2910 TT=2400.08 T1=1941.75	SAT (TO) f=3027 TT=2520.12 T1=1599.25	SAT (TO) f=3102 TT=2520.05 T1=165.74	SAT (TO) f=3131 TT=2400.12 T1=443.66	SAT (TO) f=3155 TT=2520.08 T1=1906.65	? (TO) TT=2519.75	? (TO) TT=2519.86	? (TO) TT=2520.04	SAT (TO) f=3294 TT=2400.25 T1=2275.56	No data	No data	No data	No data
TravelingTournament/ TravelingTournament-a3-galaxy10_c18.xml	SAT f=5686 TT=2520.03 T1=97.07	No data	No data	No data	No data	SAT (TO) f=5686 TT=2400.11 T1=95.18	SAT (TO) f=5686 TT=2520.03 T1=97.07	SAT (TO) f=5847 TT=2520.08 T1=847.17	SAT (TO) f=5696 TT=2520.07 T1=231.57	SAT (TO) f=5715 TT=2520.1 T1=66.84	SAT (TO) f=5821 TT=2520.14 T1=877.58	? (NS) TT=0.009	No data	No data	SAT (TO) f=5360 TT=2400.05 T1=2098.67	SAT (TO) f=5318 TT=2520.03 T1=1778.05	No data	No data	SAT (TO) f=5936 TT=2400.11 T1=610.35	SAT (TO) f=5939 TT=2520.05 T1=251.35	SAT (TO) f=5944 TT=2520.1 T1=1641.76	SAT (TO) f=6073 TT=2400.14 T1=504.82	SAT (TO) f=6050 TT=2520.06 T1=1388.12	? (TO) TT=2520.04	? (TO) TT=2520	? (TO) TT=2520.06	? (TO) TT=2405.32	No data	No data	No data	No data
TravelingTournament/ TravelingTournament-a3-galaxy12_c18.xml	SAT f=9651 TT=2520.09 T1=180	No data	No data	No data	No data	SAT (TO) f=9651 TT=2400.11 T1=179.41	SAT (TO) f=9651 TT=2520.09 T1=180	SAT (TO) f=9702 TT=2520.1 T1=76.78	SAT (TO) f=9709 TT=2520.08 T1=366.55	SAT (TO) f=9600 TT=2520.11 T1=588.39	SAT (TO) f=10056 TT=2520.08 T1=310.41	? (NS) TT=0.012	No data	No data	SAT (TO) f=9309 TT=2400.08 T1=1608.93	Signal TT=1.604	No data	No data	SAT (TO) f=10064 TT=2400.04 T1=1969.05	SAT (TO) f=9929 TT=2520.05 T1=136.74	SAT (TO) f=10102 TT=2520.1 T1=199.35	SAT (TO) f=10154 TT=2400.08 T1=1390.84	SAT (TO) f=10255 TT=2520.1 T1=1625.24	? (TO) TT=2520.05	? (TO) TT=2519.94	? (TO) TT=2519.89	? (TO) TT=2400.11	No data	No data	No data	No data
TravelingTournament/ TravelingTournament-a3-galaxy14_c18.xml	SAT f=14498 TT=2520.11 T1=988.34	No data	No data	No data	No data	SAT (TO) f=14498 TT=2400.07 T1=956.65	SAT (TO) f=14498 TT=2520.11 T1=988.34	SAT (TO) f=15390 TT=2520.12 T1=1481.51	SAT (TO) f=15581 TT=2520.09 T1=489.6	SAT (TO) f=15279 TT=2520.04 T1=244.6	SAT (TO) f=15572 TT=2520.07 T1=147.57	? (NS) TT=0.015	No data	No data	SAT (TO) f=14479 TT=2400.04 T1=1931.42	Signal TT=2.86	No data	No data	SAT (TO) f=15789 TT=2400.05 T1=1538.4	SAT (TO) f=16059 TT=2520.08 T1=5.7	SAT (TO) f=16011 TT=2520.04 T1=1231.56	SAT (TO) f=16327 TT=2400.08 T1=820.31	SAT (TO) f=16349 TT=2520.12 T1=786.01	? (TO) TT=2520.05	? (TO) TT=2520.04	? (TO) TT=2520.1	? (TO) TT=2400.14	No data	No data	No data	No data
TravelingTournament/ TravelingTournament-a3-galaxy16_c18.xml	SAT f=20855 TT=2520.12 T1=1242.34	No data	No data	No data	No data	SAT (TO) f=20855 TT=2400.08 T1=1243.87	SAT (TO) f=20855 TT=2520.12 T1=1242.34	SAT (TO) f=21617 TT=2520.11 T1=2433.21	SAT (TO) f=21340 TT=2520.15 T1=2325.39	SAT (TO) f=21482 TT=2520.1 T1=1158.86	SAT (TO) f=22056 TT=2520.12 T1=181.88	? (NS) TT=0.02	No data	No data	SAT (TO) f=20391 TT=2400.06 T1=1959.59	Signal TT=4.584	No data	No data	SAT (TO) f=22392 TT=2400.07 T1=253.29	SAT (TO) f=22051 TT=2520.06 T1=212.29	SAT (TO) f=22251 TT=2520.12 T1=2244.35	SAT (TO) f=23158 TT=2400.14 T1=1471.33	SAT (TO) f=23425 TT=2520.11 T1=1681.06	? (TO) TT=2519.93	? (TO) TT=2519.92	? (TO) TT=2519.93	? (TO) TT=2400.1	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
TravellingSalesman/ TravellingSalesman-10-20-00_c18.xml	OPT f=47 TT=1.946 T1=0.05	No data	No data	No data	No data	OPT f=47 TT=5.918 T1=0.68	OPT f=47 TT=6.726 T1=0.8	OPT f=47 TT=13.236 T1=3.76	OPT f=47 TT=13.034 T1=3.85	OPT f=47 TT=12.685 T1=3.75	OPT f=47 TT=12.23 T1=3.8	OPT f=47 TT=1.946 T1=0.05	No data	No data	OPT f=47 TT=0.614 T1=0.01	OPT f=47 TT=0.613 T1=0.01	No data	No data	SAT (TO) f=47 TT=2400.04 T1=1.25	SAT (TO) f=47 TT=2520.05 T1=1.23	OPT f=47 TT=3.214 T1=1.23	OPT f=47 TT=3.355 T1=1.21	OPT f=47 TT=3.274 T1=1.15	OPT f=47 TT=5.388 T1=5.38	OPT f=47 TT=5.689 T1=5.68	OPT f=47 TT=5.721 T1=5.72	OPT f=47 TT=544.446 T1=84.91	No data	No data	No data	No data
TravellingSalesman/ TravellingSalesman-100-50-00_c18.xml	SAT f=451 TT=2519.97 T1=38.15	No data	No data	No data	No data	SAT (TO) f=484 TT=2400.13 T1=101.57	SAT (TO) f=484 TT=2520.02 T1=101.11	SAT (TO) f=756 TT=2520.11 T1=628.69	SAT (TO) f=762 TT=2520.1 T1=196.07	SAT (TO) f=686 TT=2520.06 T1=227.4	SAT (TO) f=680 TT=2520.1 T1=75.33	SAT (TO) f=451 TT=2519.97 T1=38.15	No data	No data	SAT (TO) f=654 TT=2400.04 T1=2377.17	SAT (TO) f=654 TT=2519.9 T1=2358.69	No data	No data	SAT (TO) f=1606 TT=2400.12 T1=1949.75	SAT (TO) f=1465 TT=2520.1 T1=1320.97	SAT (TO) f=1435 TT=2520.05 T1=101.19	SAT (TO) f=574 TT=2400.11 T1=1330.46	SAT (TO) f=501 TT=2520.06 T1=1713.6	? (TO) TT=2519.98	? (TO) TT=2519.69	? (TO) TT=2520.1	? TT=423.118	No data	No data	No data	No data
TravellingSalesman/ TravellingSalesman-15-30-00_c18.xml	OPT f=93 TT=83.067 T1=0.62	No data	No data	No data	No data	SAT (TO) f=93 TT=2400.07 T1=0.63	SAT (TO) f=93 TT=2520.06 T1=0.62	OPT f=93 TT=1128.55 T1=4.89	SAT (TO) f=93 TT=2520.12 T1=5.06	OPT f=93 TT=1196.12 T1=5.6	SAT (TO) f=93 TT=2520.04 T1=4.3	SAT (TO) f=93 TT=2520.07 T1=6.05	No data	No data	OPT f=93 TT=848.043 T1=0.04	OPT f=93 TT=853.876 T1=0.04	No data	No data	SAT (TO) f=93 TT=2400.11 T1=46.27	SAT (TO) f=93 TT=2520.09 T1=18.71	SAT (TO) f=93 TT=2520.02 T1=14.91	SAT (TO) f=93 TT=2400.05 T1=1.64	SAT (TO) f=93 TT=2520.08 T1=1.63	OPT f=93 TT=49.445 T1=49.44	OPT f=93 TT=83.359 T1=83.38	OPT f=93 TT=83.067 T1=83.06	SAT (TO) f=122 TT=2400.13 T1=1932.95	No data	No data	No data	No data
TravellingSalesman/ TravellingSalesman-150-50-00_c18.xml	SAT f=537 TT=2520 T1=72.96	No data	No data	No data	No data	SAT (TO) f=608 TT=2400.07 T1=65.46	SAT (TO) f=608 TT=2520.06 T1=64.65	SAT (TO) f=1088 TT=2520.11 T1=1789.8	SAT (TO) f=1033 TT=2520.08 T1=77.86	SAT (TO) f=1027 TT=2520.1 T1=373.69	SAT (TO) f=1067 TT=2520.1 T1=249.12	SAT (TO) f=537 TT=2520 T1=72.96	No data	No data	SAT (TO) f=1144 TT=2400.03 T1=2386.7	SAT (TO) f=1143 TT=2519.98 T1=2449.83	No data	No data	SAT (TO) f=2184 TT=2400.03 T1=2210.45	SAT (TO) f=2274 TT=2520.11 T1=1684.22	SAT (TO) f=2709 TT=2520.12 T1=480.4	SAT (TO) f=742 TT=2400.06 T1=2113.59	SAT (TO) f=734 TT=2520.05 T1=2219.44	? (TO) TT=2519.86	? (TO) TT=2520	? (TO) TT=2520.05	? TT=1999.35	No data	No data	No data	No data
TravellingSalesman/ TravellingSalesman-20-30-00_c18.xml	OPT f=104 TT=1250.67 T1=4.83	No data	No data	No data	No data	SAT (TO) f=104 TT=2400.09 T1=132.72	SAT (TO) f=104 TT=2520.11 T1=132.64	SAT (TO) f=104 TT=2520.13 T1=25.34	SAT (TO) f=104 TT=2520.05 T1=39.98	SAT (TO) f=104 TT=2520.09 T1=33.68	SAT (TO) f=104 TT=2520.12 T1=30.17	SAT (TO) f=104 TT=2519.74 T1=1776.56	No data	No data	SAT (TO) f=104 TT=2400.04 T1=0.34	SAT (TO) f=104 TT=2519.71 T1=0.34	No data	No data	SAT (TO) f=108 TT=2400.03 T1=2173.37	SAT (TO) f=108 TT=2520.11 T1=72.14	SAT (TO) f=117 TT=2520.07 T1=1328.59	SAT (TO) f=113 TT=2400.03 T1=6.48	SAT (TO) f=104 TT=2520.12 T1=4.83	OPT f=104 TT=770.526 T1=770.45	OPT f=104 TT=1252.49 T1=1252.46	OPT f=104 TT=1250.67 T1=1250.62	SAT (TO) f=198 TT=2400.15 T1=2036.14	No data	No data	No data	No data
TravellingSalesman/ TravellingSalesman-25-30-00_c18.xml	SAT f=113 TT=2520.06 T1=10.34	No data	No data	No data	No data	SAT (TO) f=119 TT=2400.09 T1=746	SAT (TO) f=119 TT=2520.06 T1=741.55	SAT (TO) f=113 TT=2520.06 T1=13.86	SAT (TO) f=113 TT=2520.08 T1=668.67	SAT (TO) f=126 TT=2520.12 T1=44.79	SAT (TO) f=115 TT=2520.11 T1=207.99	SAT (TO) f=118 TT=2520.01 T1=52.08	No data	No data	SAT (TO) f=113 TT=2400.06 T1=1054.57	SAT (TO) f=113 TT=2520.07 T1=1054.32	No data	No data	SAT (TO) f=154 TT=2400.12 T1=1990.84	SAT (TO) f=151 TT=2520.12 T1=150.49	SAT (TO) f=144 TT=2520.06 T1=2383.63	SAT (TO) f=115 TT=2400.13 T1=17.36	SAT (TO) f=113 TT=2520.09 T1=10.34	? (TO) TT=2519.82	? (TO) TT=2519.9	? (TO) TT=2519.98	SAT (TO) f=253 TT=2400.27 T1=1703.43	No data	No data	No data	No data
TravellingSalesman/ TravellingSalesman-30-30-00_c18.xml	SAT f=127 TT=2520.06 T1=1306.04	No data	No data	No data	No data	SAT (TO) f=127 TT=2400.1 T1=1296.7	SAT (TO) f=127 TT=2520.06 T1=1306.04	SAT (TO) f=141 TT=2520.06 T1=1005.19	SAT (TO) f=145 TT=2520.09 T1=1491.85	SAT (TO) f=126 TT=2520.1 T1=304.28	SAT (TO) f=144 TT=2520.09 T1=309.21	SAT (TO) f=132 TT=2519.87 T1=6.74	No data	No data	SAT (TO) f=150 TT=2400.1 T1=1066.69	SAT (TO) f=150 TT=2519.82 T1=1063.81	No data	No data	SAT (TO) f=192 TT=2400.08 T1=21.06	SAT (TO) f=189 TT=2520.05 T1=1967.66	SAT (TO) f=210 TT=2520.06 T1=41.54	SAT (TO) f=145 TT=2400.07 T1=19.92	SAT (TO) f=141 TT=2520.03 T1=39.04	? (TO) TT=2519.87	? (TO) TT=2519.97	? (TO) TT=2519.77	SAT (TO) f=332 TT=2400.18 T1=2014.23	No data	No data	No data	No data
TravellingSalesman/ TravellingSalesman-35-30-00_c18.xml	SAT f=148 TT=2520.11 T1=78.07	No data	No data	No data	No data	SAT (TO) f=169 TT=2400.07 T1=780.62	SAT (TO) f=169 TT=2520.03 T1=779.11	SAT (TO) f=161 TT=2520.1 T1=184.85	SAT (TO) f=172 TT=2520.07 T1=1310.86	SAT (TO) f=181 TT=2520.08 T1=1617.46	SAT (TO) f=178 TT=2520.1 T1=401.42	SAT (TO) f=162 TT=2520 T1=201.68	No data	No data	SAT (TO) f=185 TT=2400.08 T1=2373.09	SAT (TO) f=185 TT=2519.93 T1=2363.13	No data	No data	SAT (TO) f=243 TT=2400.06 T1=587.09	SAT (TO) f=246 TT=2520.06 T1=214.99	SAT (TO) f=243 TT=2520.07 T1=179.22	SAT (TO) f=164 TT=2400.03 T1=21.37	SAT (TO) f=148 TT=2520.11 T1=78.07	? (TO) TT=2519.88	? (TO) TT=2519.66	? (TO) TT=2519.76	SAT (TO) f=420 TT=2400.25 T1=578.46	No data	No data	No data	No data
TravellingSalesman/ TravellingSalesman-40-30-00_c18.xml	SAT f=170 TT=2520.12 T1=22.78	No data	No data	No data	No data	SAT (TO) f=178 TT=2400.06 T1=1573.73	SAT (TO) f=178 TT=2520.03 T1=1612.77	SAT (TO) f=190 TT=2520.06 T1=137.2	SAT (TO) f=198 TT=2520.11 T1=1339.84	SAT (TO) f=196 TT=2520.12 T1=318.47	SAT (TO) f=224 TT=2520.12 T1=2470.01	SAT (TO) f=188 TT=2519.78 T1=1.08	No data	No data	SAT (TO) f=194 TT=2400.06 T1=2353.68	SAT (TO) f=194 TT=2519.91 T1=2375.6	No data	No data	SAT (TO) f=304 TT=2400.05 T1=2334.98	SAT (TO) f=272 TT=2520.08 T1=1548.63	SAT (TO) f=308 TT=2520.08 T1=52.1	SAT (TO) f=170 TT=2400.09 T1=147.97	SAT (TO) f=170 TT=2520.12 T1=22.78	? (TO) TT=2520.07	? (TO) TT=2519.99	? (TO) TT=2520.05	SAT (TO) f=432 TT=2400.31 T1=805.79	No data	No data	No data	No data
TravellingSalesman/ TravellingSalesman-45-30-00_c18.xml	SAT f=184 TT=2520.05 T1=1498.91	No data	No data	No data	No data	SAT (TO) f=203 TT=2400.03 T1=103.77	SAT (TO) f=203 TT=2520.07 T1=105.48	SAT (TO) f=215 TT=2520.14 T1=404.44	SAT (TO) f=235 TT=2520.15 T1=115.01	SAT (TO) f=216 TT=2520.12 T1=2085.47	SAT (TO) f=226 TT=2520.09 T1=845.4	SAT (TO) f=192 TT=2520.01 T1=751.98	No data	No data	SAT (TO) f=265 TT=2400.01 T1=1843.96	SAT (TO) f=265 TT=2520.07 T1=1817.88	No data	No data	SAT (TO) f=330 TT=2400.08 T1=30.93	SAT (TO) f=335 TT=2520.32 T1=610.82	SAT (TO) f=374 TT=2520.08 T1=193.87	SAT (TO) f=176 TT=2400.09 T1=51.9	SAT (TO) f=184 TT=2520.05 T1=1498.91	? (TO) TT=2519.85	? (TO) TT=2519.79	? (TO) TT=2519.93	SAT (TO) f=520 TT=2400.34 T1=59.2	No data	No data	No data	No data
Bench name	Best results	BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
TravellingSalesman/ TravellingSalesman-50-30-00_c18.xml	SAT f=190 TT=2519.91 T1=62.68	No data	No data	No data	No data	SAT (TO) f=210 TT=2400.09 T1=79.9	SAT (TO) f=210 TT=2520.11 T1=78.3	SAT (TO) f=254 TT=2520.11 T1=1101.92	SAT (TO) f=256 TT=2520.09 T1=124.68	SAT (TO) f=248 TT=2520.14 T1=1440.62	SAT (TO) f=248 TT=2520.07 T1=2439.99	SAT (TO) f=190 TT=2519.91 T1=62.68	No data	No data	SAT (TO) f=261 TT=2400.07 T1=1700.94	SAT (TO) f=261 TT=2519.92 T1=1693.62	No data	No data	SAT (TO) f=346 TT=2400.08 T1=83.28	SAT (TO) f=369 TT=2520.06 T1=1619.85	SAT (TO) f=358 TT=2520.09 T1=1506.56	SAT (TO) f=198 TT=2400.08 T1=201.09	SAT (TO) f=198 TT=2520.07 T1=2336.9	? (TO) TT=2520.02	? (TO) TT=2520	? (TO) TT=2519.96	SAT (TO) f=677 TT=2400.28 T1=1161.72	No data	No data	No data	No data
TravellingSalesman/ TravellingSalesman-75-50-00_c18.xml	SAT f=357 TT=2519.91 T1=0.59	No data	No data	No data	No data	SAT (TO) f=396 TT=2400.11 T1=798.58	SAT (TO) f=396 TT=2520.06 T1=818.13	SAT (TO) f=656 TT=2520.1 T1=1080.23	SAT (TO) f=581 TT=2520.08 T1=881.97	SAT (TO) f=539 TT=2520.04 T1=40.45	SAT (TO) f=627 TT=2520.14 T1=28.66	SAT (TO) f=357 TT=2519.91 T1=0.59	No data	No data	SAT (TO) f=679 TT=2400.1 T1=1850.69	SAT (TO) f=679 TT=2520.08 T1=1829.94	No data	No data	SAT (TO) f=1072 TT=2400.06 T1=631.02	SAT (TO) f=1130 TT=2520.07 T1=262.83	SAT (TO) f=1110 TT=2520.09 T1=299.81	SAT (TO) f=436 TT=2400.08 T1=621.26	SAT (TO) f=415 TT=2520.05 T1=505.99	? (TO) TT=2519.98	? (TO) TT=2520.02	? (TO) TT=2519.91	? (TO) TT=2400.15	No data	No data	No data	No data

Statistics		BTD 2018.06.11_3 (complete)	BTD 2018.07.27_3 (complete)	BTD_12 2018-06-11_12 (complete)	BTD_12 2018.07.27_12 (complete)	Choco-solver 4.0.7 seq (493a269) (complete)	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	Concrete 3.8 2018-06-13 (complete)	Concrete 3.8-SuperNG 2018-06-13 (complete)	cosoco 1.12 (complete)	macht 2018.06.11 (complete)	macht 2018.07.27 (complete)	Mistral-2.0 2018-06-15 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering 2018-07-02 (complete)	OscaR - Conflict Ordering 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-07-02 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-07-02 (complete)	OscaR - Hybrid 2018-08-14 (complete)	PicatSAT 2018-06-15 (complete)	PicatSAT 2018-08-02 (complete)	PicatSAT 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)	scop both+MapleCOMSPS (2018-06-13) (complete)	scop both+MapleCOMSPS (2018-07-31) (complete)	scop order+MapleCOMSPS (2018-06-13) (complete)	scop order+MapleCOMSPS (2018-07-31) (complete)
Number of times the solver is able to give the best known answer		0	76	0	77	115	269	240	223	95	86	191	0	66	115	116	81	82	0	92	222	0	132	137	137	270	169	123	141	123	146
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)		0	8	0	12	3	56	30	22	0	0	85	0	2	24	18	1	3	0	18	34	0	62	7	6	84	38	3	4	11	4