2018 XCSP3 competition: fast COP track: results by benchmark

Results by benchmark for category all categories

This page displays the results of the different solvers for each benchmark for category all categories

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell example	Meaning
Answer	Solver result
f=...	value of the objective function for the model reported by the solver
TT=...	Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out

Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

Abbreviation	Meaning
f=...	Value of the objective function
TO	Time Out
MO	Mem. Out (out of memory)

Meaning of the different colors:

Color	Meaning
text	the solver cannot handle this instance
text	the solver gave no answer
text	the solver could give an answer (SAT)
text	the solver gave a definitive answer (OPTIMUM FOUND or UNSAT)
text	the solver performed better than the other ones on that instance (complete solvers point of view)
text	the solver performed better than the other ones on that instance (incomplete solvers point of view)
text	the solver was ended by a signal or other problem
text	the solver gave an incomplete answer
text	the solver gave a wrong answer

For better readability, you may choose to hide some solvers:
Choco-solver 4.0.7b seq (e747e1e) (complete)
Concrete 3.9.2 (complete)
Concrete 3.9.2-SuperNG (complete)
cosoco 1.12 (complete)
Mistral-2.0 2018-08-01 (complete)
OscaR - Conflict Ordering with restarts 2018-08-14 (complete)
OscaR - Conflict Ordering with restarts 2018-08-17 (complete)
OscaR - Hybrid 2018-08-14 (complete)
Sat4j-CSP 2018-07-11 (complete)

Bench name	Best results on this instance	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
Auction/Auction-cnt/ Auction-cnt-decay100_c18.xml	SAT f=742697 TT=252.1 T1=222.68	SAT (TO) f=604812 TT=252.034 T1=225.02	SAT (TO) f=488065 TT=252.09 T1=87.53	SAT (TO) f=465551 TT=252.105 T1=34.97	SAT (TO) f=223947 TT=252.003 T1=251.66	SAT (TO) f=528021 TT=252.003 T1=127.8	SAT f=522786 TT=250.886 T1=24.66	SAT (TO) f=526634 TT=252.136 T1=179.53	SAT (TO) f=742697 TT=252.1 T1=222.68	? (TO) TT=252.175
Auction/Auction-cnt/ Auction-cnt-decay200_c18.xml	SAT f=1371551 TT=252.137 T1=216.45	SAT (TO) f=1190035 TT=252.077 T1=198.22	SAT (TO) f=936156 TT=252.09 T1=26.49	SAT (TO) f=880139 TT=252.07 T1=193.49	SAT (TO) f=375130 TT=252.003 T1=251.9	SAT (TO) f=985021 TT=252.011 T1=147.1	SAT f=817501 TT=251.928 T1=129.6	SAT f=787685 TT=251.088 T1=13.75	SAT (TO) f=1371551 TT=252.137 T1=216.45	SAT (TO) f=361678 TT=252.093 T1=99.77
Auction/Auction-cnt/ Auction-cnt-decay300_c18.xml	SAT f=1822443 TT=252.105 T1=134.72	SAT (TO) f=1589004 TT=252.026 T1=86.93	SAT (TO) f=1269671 TT=252.072 T1=226.46	SAT (TO) f=1280816 TT=252.143 T1=89.68	SAT (TO) f=421162 TT=252.026 T1=251.85	SAT (TO) f=1234520 TT=252.02 T1=122.75	SAT f=1120240 TT=251.431 T1=235.01	SAT (TO) f=1109036 TT=252.07 T1=116.51	SAT (TO) f=1822443 TT=252.105 T1=134.72	SAT (TO) f=648100 TT=252.134 T1=44.26
Auction/Auction-cnt/ Auction-cnt-decay400_c18.xml	SAT f=2265995 TT=252.099 T1=236.12	SAT (TO) f=2091019 TT=252.103 T1=199.75	SAT (TO) f=1589386 TT=252.124 T1=178.11	SAT (TO) f=1605517 TT=252.061 T1=9.43	SAT (TO) f=519292 TT=252.034 T1=251.97	SAT (TO) f=1483778 TT=252.014 T1=62.4	SAT f=1531454 TT=251.944 T1=239.43	SAT f=1503634 TT=251.071 T1=204.39	SAT (TO) f=2265995 TT=252.099 T1=236.12	SAT (TO) f=769415 TT=252.083 T1=35.45
Auction/Auction-cnt/ Auction-cnt-decay500_c18.xml	SAT f=2721013 TT=252.081 T1=230.73	SAT (TO) f=2474766 TT=252.075 T1=189.19	SAT (TO) f=1878485 TT=252.132 T1=184.39	SAT (TO) f=1865380 TT=252.149 T1=54.73	SAT (TO) f=610494 TT=251.998 T1=251.63	SAT (TO) f=1704167 TT=252.01 T1=61.78	SAT f=1826932 TT=252.042 T1=246.34	SAT (TO) f=1839263 TT=252.089 T1=200.49	SAT (TO) f=2721013 TT=252.081 T1=230.73	SAT (TO) f=818177 TT=252.175 T1=24.62
Auction/Auction-cnt/ Auction-cnt-example_c18.xml	OPT f=54 TT=0.004 T1=0	OPT f=54 TT=0.816 T1=0.44	OPT f=54 TT=5.955 T1=3.07	OPT f=54 TT=6.084 T1=3.09	OPT f=54 TT=0.004 T1=0	OPT f=54 TT=0.004 T1=0	SAT f=54 TT=251.246 T1=1	OPT f=54 TT=1.489 T1=1.23	OPT f=54 TT=1.473 T1=0.99	OPT f=54 TT=0.578 T1=0.34
Auction/Auction-cnt/ Auction-cnt-matching040_c18.xml	SAT f=12965081 TT=252.089 T1=100.38	SAT (TO) f=7828656 TT=252.066 T1=1.44	SAT (TO) f=9573738 TT=252.156 T1=125.73	SAT (TO) f=8233961 TT=252.063 T1=9.14	SAT (TO) f=3582894 TT=252.032 T1=233.42	SAT (TO) f=9704465 TT=251.987 T1=199.32	SAT f=7336175 TT=251.023 T1=24.87	SAT f=7765907 TT=251.268 T1=101.67	SAT (TO) f=12965081 TT=252.089 T1=100.38	SAT (TO) f=8923883 TT=252.126 T1=89.67
Auction/Auction-cnt/ Auction-cnt-matching080_c18.xml	SAT f=43279714 TT=252.118 T1=128.51	SAT (TO) f=30568459 TT=252.05 T1=1.39	SAT (TO) f=32505140 TT=252.091 T1=132.96	SAT (TO) f=32451413 TT=252.129 T1=22.05	SAT (TO) f=7017782 TT=251.995 T1=198.71	SAT (TO) f=21649649 TT=251.981 T1=200.42	SAT f=30448558 TT=250.894 T1=236.82	SAT f=29457707 TT=251.156 T1=28.18	SAT (TO) f=43279714 TT=252.118 T1=128.51	SAT (TO) f=24823333 TT=252.089 T1=161.88
Auction/Auction-sum/ Auction-sum-decay100_c18.xml	SAT f=753878 TT=252.037 T1=174.98	SAT (TO) f=590630 TT=252.024 T1=189.51	SAT (TO) f=536082 TT=252.209 T1=153.35	SAT (TO) f=422809 TT=252.573 T1=177.17	SAT (TO) f=269629 TT=251.962 T1=251.98	SAT (TO) f=558027 TT=251.973 T1=241.99	SAT f=527300 TT=250.747 T1=155.37	SAT f=514358 TT=251.034 T1=35.27	SAT (TO) f=753878 TT=252.037 T1=174.98	? (TO) TT=252.089
Auction/Auction-sum/ Auction-sum-decay200_c18.xml	SAT f=1435945 TT=252.062 T1=234.73	SAT (TO) f=1193486 TT=252.038 T1=190.13	SAT (TO) f=996057 TT=252.07 T1=94.3	SAT (TO) f=1001781 TT=252.15 T1=160.14	SAT (TO) f=350747 TT=252.003 T1=251.91	SAT (TO) f=1014918 TT=252.014 T1=196.55	SAT f=799824 TT=251.833 T1=9.58	SAT f=800255 TT=251.842 T1=9.85	SAT (TO) f=1435945 TT=252.062 T1=234.73	? (TO) TT=252.052
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
Auction/Auction-sum/ Auction-sum-decay300_c18.xml	SAT f=1964667 TT=252.102 T1=238.43	SAT (TO) f=1658038 TT=252.085 T1=69.82	SAT (TO) f=1504862 TT=252.091 T1=109.08	SAT (TO) f=1425801 TT=252.159 T1=185.83	SAT (TO) f=423605 TT=251.945 T1=252	SAT (TO) f=1221727 TT=251.998 T1=121.79	SAT f=1180732 TT=251.28 T1=179.04	SAT f=1115298 TT=251.069 T1=231.97	SAT (TO) f=1964667 TT=252.102 T1=238.43	SAT (TO) f=578422 TT=252.099 T1=121.2
Auction/Auction-sum/ Auction-sum-decay400_c18.xml	SAT f=2380118 TT=252.088 T1=237.97	SAT (TO) f=2178480 TT=252.107 T1=62.86	SAT (TO) f=1795857 TT=252.064 T1=88.35	SAT (TO) f=1775875 TT=252.123 T1=85.35	SAT (TO) f=490080 TT=252 T1=251.77	SAT (TO) f=1439188 TT=252.002 T1=116.56	SAT f=1444789 TT=250.95 T1=105.47	SAT (TO) f=1470150 TT=252.073 T1=202.72	SAT (TO) f=2380118 TT=252.088 T1=237.97	SAT (TO) f=699850 TT=252.145 T1=214.32
Auction/Auction-sum/ Auction-sum-decay500_c18.xml	SAT f=2899564 TT=252.105 T1=232.72	SAT (TO) f=2788055 TT=252.07 T1=207.21	SAT (TO) f=2266300 TT=252.101 T1=212.64	SAT (TO) f=2164068 TT=252.083 T1=108.46	SAT (TO) f=592439 TT=251.998 T1=251.94	SAT (TO) f=1647146 TT=251.977 T1=219.98	SAT (TO) f=1845351 TT=252.104 T1=160.11	SAT (TO) f=1807811 TT=252.056 T1=246.62	SAT (TO) f=2899564 TT=252.105 T1=232.72	SAT (TO) f=1170840 TT=252.126 T1=73.64
Auction/Auction-sum/ Auction-sum-example_c18.xml	OPT f=54 TT=0.005 T1=0	OPT f=54 TT=0.826 T1=0.44	OPT f=54 TT=6.082 T1=3.12	OPT f=54 TT=5.997 T1=3.14	OPT f=54 TT=0.005 T1=0	OPT f=54 TT=0.005 T1=0	SAT f=54 TT=250.857 T1=0.98	OPT f=54 TT=1.451 T1=0.98	OPT f=54 TT=1.455 T1=0.97	OPT f=54 TT=0.554 T1=0.34
Auction/Auction-sum/ Auction-sum-matching040_c18.xml	SAT f=12968875 TT=252.099 T1=210.62	SAT (TO) f=8382363 TT=252.096 T1=111.87	SAT (TO) f=8306093 TT=252.199 T1=130.33	SAT (TO) f=8881534 TT=252.082 T1=182.45	SAT (TO) f=3610393 TT=252.036 T1=130.95	SAT (TO) f=7198596 TT=252.003 T1=24.36	SAT f=7707304 TT=251.138 T1=51.36	SAT f=7538972 TT=250.986 T1=13.75	SAT (TO) f=12968875 TT=252.099 T1=210.62	? (TO) TT=252.125
Auction/Auction-sum/ Auction-sum-matching080_c18.xml	SAT f=45496100 TT=252.139 T1=194.97	SAT (TO) f=29379027 TT=252.108 T1=107.34	SAT (TO) f=35918870 TT=252.101 T1=113.53	SAT (TO) f=34430993 TT=252.387 T1=169.4	SAT (TO) f=7395397 TT=252.031 T1=226.21	SAT (TO) f=15872294 TT=252.001 T1=215.05	SAT f=29536850 TT=250.98 T1=238.47	SAT f=29911590 TT=250.917 T1=199.62	SAT (TO) f=45496100 TT=252.139 T1=194.97	SAT (TO) f=19633438 TT=252.098 T1=136.23
Bacp/Bacp-m1/ Bacp-m1-05_c18.xml	OPT f=9 TT=0.025 T1=0.01	OPT f=9 TT=1.754 T1=0.54	OPT f=9 TT=9.492 T1=3.56	OPT f=9 TT=9.115 T1=3.63	OPT f=9 TT=0.025 T1=0.01	OPT f=9 TT=0.025 T1=0.01	SAT f=9 TT=251.627 T1=1.14	OPT f=9 TT=2.02 T1=1.14	OPT f=9 TT=2.066 T1=1.14	OPT f=9 TT=2.334 T1=0.92
Bacp/Bacp-m1/ Bacp-m1-06_c18.xml	OPT f=10 TT=6.667 T1=0.54	OPT f=10 TT=10.019 T1=0.54	OPT f=10 TT=29.517 T1=3.72	OPT f=10 TT=28.117 T1=3.73	Signal TT=0.011	OPT f=10 TT=1.957 T1=0.01	SAT (TO) f=10 TT=252.063 T1=1.17	OPT f=10 TT=6.667 T1=1.17	SAT (TO) f=10 TT=252.091 T1=1.17	OPT f=10 TT=9.514 T1=1.55
Bacp/Bacp-m1/ Bacp-m1-07a_c18.xml	SAT f=12 TT=251.59 T1=0.03	SAT (TO) f=12 TT=252.036 T1=0.73	SAT (TO) f=12 TT=252.127 T1=3.93	SAT (TO) f=12 TT=252.05 T1=4.07	SAT (TO) f=12 TT=252.012 T1=0.03	SAT (TO) f=12 TT=251.962 T1=0.02	SAT f=12 TT=251.597 T1=1.24	SAT f=12 TT=251.59 T1=1.23	SAT (TO) f=12 TT=252.035 T1=1.26	SAT (TO) f=12 TT=252.113 T1=3.84
Bacp/Bacp-m1/ Bacp-m1-07b_c18.xml	SAT f=13 TT=250.725 T1=0.02	SAT (TO) f=13 TT=252.044 T1=0.66	SAT (TO) f=13 TT=252.063 T1=4.09	SAT (TO) f=13 TT=252.079 T1=4.28	SAT (TO) f=13 TT=252.023 T1=0.02	SAT (TO) f=13 TT=251.994 T1=0.01	SAT f=13 TT=250.923 T1=1.26	SAT f=13 TT=250.725 T1=1.27	SAT (TO) f=13 TT=252.024 T1=1.44	SAT (TO) f=13 TT=252.116 T1=6.97
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
Bacp/Bacp-m1/ Bacp-m1-08a_c18.xml	SAT f=17 TT=250.99 T1=0.04	SAT (TO) f=17 TT=252.093 T1=0.68	SAT (TO) f=17 TT=252.108 T1=4.15	SAT (TO) f=17 TT=252.068 T1=4.22	SAT (TO) f=17 TT=252.001 T1=0.04	SAT (TO) f=17 TT=251.99 T1=0.03	SAT f=17 TT=251.206 T1=1.7	SAT f=17 TT=250.99 T1=1.33	SAT (TO) f=17 TT=252.074 T1=2.55	SAT (TO) f=17 TT=252.081 T1=35.4
Bacp/Bacp-m1/ Bacp-m1-08b_c18.xml	SAT f=17 TT=251.321 T1=0.04	SAT (TO) f=17 TT=252.122 T1=0.66	SAT (TO) f=17 TT=252.114 T1=4.19	SAT (TO) f=17 TT=252.082 T1=4.26	SAT (TO) f=17 TT=252.007 T1=0.04	SAT (TO) f=17 TT=251.992 T1=0.06	SAT f=17 TT=251.321 T1=1.31	SAT f=17 TT=251.802 T1=1.35	SAT (TO) f=17 TT=252.14 T1=3.68	SAT (TO) f=17 TT=252.148 T1=17.05
Bacp/Bacp-m1/ Bacp-m1-10a-nodv_c18.xml	SAT f=14 TT=251.433 T1=0.05	SAT (TO) f=14 TT=252.032 T1=0.89	SAT (TO) f=14 TT=252.126 T1=4.19	SAT (TO) f=14 TT=252.072 T1=4.19	SAT (TO) f=14 TT=252.037 T1=0.05	SAT (TO) f=14 TT=251.998 T1=0.07	SAT f=14 TT=251.433 T1=1.81	SAT f=14 TT=251.6 T1=1.36	SAT (TO) f=14 TT=252.04 T1=2.04	SAT (TO) f=14 TT=252.117 T1=28.54
Bacp/Bacp-m1/ Bacp-m1-10a_c18.xml	SAT f=14 TT=251.844 T1=0.32	SAT (TO) f=14 TT=252.065 T1=0.66	SAT (TO) f=14 TT=252.133 T1=4.17	SAT (TO) f=14 TT=252.132 T1=4.14	SAT (TO) f=14 TT=252.006 T1=0.32	SAT (TO) f=14 TT=251.976 T1=0.07	SAT f=14 TT=251.844 T1=2.48	SAT f=14 TT=251.956 T1=2.34	SAT (TO) f=14 TT=252.14 T1=135.66	SAT (TO) f=14 TT=252.096 T1=28.53
Bacp/Bacp-m1/ Bacp-m1-10b_c18.xml	SAT f=14 TT=250.797 T1=0.03	SAT (TO) f=14 TT=252.046 T1=0.65	SAT (TO) f=14 TT=252.113 T1=4.2	SAT (TO) f=14 TT=252.127 T1=4.17	SAT (TO) f=14 TT=252.038 T1=0.03	SAT (TO) f=14 TT=251.998 T1=0.03	SAT f=14 TT=251.12 T1=1.31	SAT f=14 TT=250.797 T1=1.31	SAT (TO) f=14 TT=252.09 T1=1.42	SAT (TO) f=14 TT=252.072 T1=93.41
Bacp/Bacp-m1/ Bacp-m1-12a-nodv_c18.xml	SAT f=17 TT=251.599 T1=1.54	SAT (TO) f=17 TT=252.107 T1=8.15	SAT (TO) f=18 TT=252.143 T1=5.16	SAT (TO) f=18 TT=252.16 T1=5.18	SAT (TO) f=17 TT=252.038 T1=3.93	SAT (TO) f=17 TT=252.007 T1=0.18	SAT f=17 TT=251.599 T1=1.58	SAT f=17 TT=251.824 T1=1.54	SAT (TO) f=17 TT=252.107 T1=1.54	? (TO) TT=252.053
Bacp/Bacp-m1/ Bacp-m1-12a_c18.xml	SAT f=17 TT=251.57 T1=0.31	SAT (TO) f=17 TT=252.097 T1=8.24	SAT (TO) f=17 TT=252.088 T1=5.06	SAT (TO) f=17 TT=252.122 T1=5.38	SAT (TO) f=17 TT=252.007 T1=0.31	SAT (TO) f=17 TT=252.013 T1=0.18	SAT f=17 TT=251.59 T1=1.51	SAT f=17 TT=251.57 T1=1.53	? (TO) TT=252.09	? (TO) TT=252.091
Bacp/Bacp-m1/ Bacp-m1-12b_c18.xml	SAT f=17 TT=251.167 T1=0.05	SAT (TO) f=17 TT=252.062 T1=14.01	SAT (TO) f=17 TT=252.05 T1=4.85	SAT (TO) f=17 TT=252.109 T1=4.69	SAT (TO) f=17 TT=251.974 T1=0.05	SAT (TO) f=17 TT=251.998 T1=0.25	SAT f=17 TT=251.239 T1=1.47	SAT f=17 TT=251.167 T1=1.48	SAT (TO) f=17 TT=252.093 T1=1.47	? (TO) TT=252.128
Bacp/Bacp-m2/ Bacp-m2-05_c18.xml	OPT f=9 TT=0.025 T1=0.01	OPT f=9 TT=1.979 T1=0.56	OPT f=9 TT=11.636 T1=4.56	OPT f=9 TT=12.041 T1=4.59	OPT f=9 TT=0.025 T1=0.01	OPT f=9 TT=0.101 T1=0.01	SAT f=9 TT=251.802 T1=1.19	OPT f=9 TT=2.224 T1=1.19	OPT f=9 TT=2.255 T1=1.19	OPT f=9 TT=2.86 T1=0.81
Bacp/Bacp-m2/ Bacp-m2-06_c18.xml	OPT f=10 TT=3.27 T1=0.01	OPT f=10 TT=7.76 T1=0.6	OPT f=10 TT=45.317 T1=5.28	OPT f=10 TT=36.413 T1=4.73	OPT f=10 TT=3.27 T1=0.01	OPT f=10 TT=2.708 T1=0.01	SAT (TO) f=10 TT=252.078 T1=1.24	OPT f=10 TT=10.828 T1=1.24	SAT (TO) f=10 TT=252.096 T1=1.24	OPT f=10 TT=16.914 T1=1.66
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
Bacp/Bacp-m2/ Bacp-m2-07a_c18.xml	SAT f=12 TT=251.895 T1=0.02	SAT (TO) f=12 TT=252.116 T1=0.63	SAT (TO) f=12 TT=252.125 T1=6.09	SAT (TO) f=12 TT=252.056 T1=5.68	SAT (TO) f=12 TT=252.014 T1=0.02	SAT (TO) f=12 TT=251.789 T1=0.32	SAT f=12 TT=252.009 T1=1.67	SAT f=12 TT=251.895 T1=1.3	SAT (TO) f=12 TT=252.08 T1=1.73	SAT (TO) f=12 TT=252.079 T1=3.77
Bacp/Bacp-m2/ Bacp-m2-07b_c18.xml	SAT f=13 TT=252.006 T1=0.03	SAT (TO) f=13 TT=252.064 T1=1.76	SAT (TO) f=13 TT=252.116 T1=5.84	SAT (TO) f=13 TT=252.127 T1=6.59	SAT (TO) f=13 TT=252.006 T1=0.03	SAT (TO) f=13 TT=251.999 T1=0.02	SAT (TO) f=13 TT=252.058 T1=1.35	SAT f=13 TT=252.008 T1=1.36	SAT (TO) f=13 TT=252.064 T1=1.64	SAT (TO) f=13 TT=252.244 T1=5.85
Bacp/Bacp-m2/ Bacp-m2-08a_c18.xml	SAT f=17 TT=251.967 T1=0.27	SAT (TO) f=17 TT=252.079 T1=0.7	SAT (TO) f=17 TT=252.124 T1=7.74	SAT (TO) f=17 TT=252.127 T1=8.29	SAT (TO) f=17 TT=251.967 T1=0.27	SAT (TO) f=17 TT=251.982 T1=0.05	SAT (TO) f=17 TT=252.031 T1=1.44	SAT (TO) f=17 TT=252.033 T1=1.46	SAT (TO) f=17 TT=252.103 T1=3.27	SAT (TO) f=17 TT=252.137 T1=21.45
Bacp/Bacp-m2/ Bacp-m2-08b_c18.xml	SAT f=17 TT=251.783 T1=0.69	SAT (TO) f=17 TT=252.095 T1=0.69	SAT (TO) f=17 TT=252.124 T1=7.48	SAT (TO) f=17 TT=252.059 T1=7.82	SAT (TO) f=17 TT=252.008 T1=1.85	SAT (TO) f=17 TT=251.961 T1=0.06	SAT f=17 TT=251.783 T1=1.66	SAT (TO) f=17 TT=252.087 T1=1.83	SAT (TO) f=17 TT=252.067 T1=4.54	SAT (TO) f=17 TT=252.049 T1=26.14
Bacp/Bacp-m2/ Bacp-m2-10a-nodv_c18.xml	SAT f=14 TT=252.028 T1=0.12	SAT (TO) f=14 TT=252.116 T1=0.72	SAT (TO) f=14 TT=252.111 T1=6.9	SAT (TO) f=14 TT=252.154 T1=7.12	SAT (TO) f=14 TT=252.028 T1=0.12	SAT (TO) f=14 TT=251.997 T1=0.2	SAT (TO) f=14 TT=252.1 T1=1.55	SAT (TO) f=14 TT=252.067 T1=1.53	SAT (TO) f=14 TT=252.114 T1=3	SAT (TO) f=14 TT=252.092 T1=39.63
Bacp/Bacp-m2/ Bacp-m2-10a_c18.xml	SAT f=14 TT=252.069 T1=0.71	SAT (TO) f=14 TT=252.089 T1=0.71	SAT (TO) f=14 TT=252.081 T1=7.65	SAT (TO) f=14 TT=252.069 T1=6.99	? (TO) TT=251.99	SAT (TO) f=14 TT=252.012 T1=0.2	SAT (TO) f=14 TT=252.1 T1=2.69	SAT (TO) f=14 TT=252.104 T1=2.7	SAT (TO) f=14 TT=252.113 T1=207.1	SAT (TO) f=14 TT=252.128 T1=39.23
Bacp/Bacp-m2/ Bacp-m2-10b_c18.xml	SAT f=14 TT=251.711 T1=0.19	SAT (TO) f=14 TT=252.065 T1=0.71	SAT (TO) f=14 TT=252.08 T1=7.33	SAT (TO) f=14 TT=252.153 T1=7.5	SAT (TO) f=14 TT=252.008 T1=0.19	SAT (TO) f=14 TT=252.009 T1=3.66	SAT f=14 TT=251.968 T1=1.44	SAT f=14 TT=251.711 T1=1.46	SAT (TO) f=14 TT=252.025 T1=1.62	SAT (TO) f=14 TT=252.097 T1=122.27
Bacp/Bacp-m2/ Bacp-m2-12a-nodv_c18.xml	SAT f=17 TT=251.992 T1=0.73	SAT (TO) f=17 TT=252.031 T1=5.59	SAT (TO) f=17 TT=252.162 T1=11.14	SAT (TO) f=17 TT=252.147 T1=18.1	SAT (TO) f=17 TT=251.992 T1=0.73	SAT (TO) f=18 TT=251.99 T1=0.11	SAT (TO) f=18 TT=252.034 T1=1.71	SAT (TO) f=17 TT=252.059 T1=129.62	SAT (TO) f=17 TT=252.058 T1=5.52	SAT (TO) f=23 TT=252.077 T1=238.18
Bacp/Bacp-m2/ Bacp-m2-12a_c18.xml	SAT f=17 TT=252.042 T1=1.72	SAT (TO) f=17 TT=252.042 T1=5.72	SAT (TO) f=17 TT=252.077 T1=13	SAT (TO) f=17 TT=252.161 T1=11.32	? (TO) TT=251.991	SAT (TO) f=18 TT=252.004 T1=0.11	SAT (TO) f=17 TT=252.112 T1=1.72	SAT (TO) f=17 TT=252.059 T1=1.72	? (TO) TT=252.049	SAT (TO) f=23 TT=252.133 T1=240.36
Bacp/Bacp-m2/ Bacp-m2-12b_c18.xml	SAT f=17 TT=252.037 T1=0.07	SAT (TO) f=17 TT=252.044 T1=6.02	SAT (TO) f=17 TT=252.111 T1=11.27	SAT (TO) f=17 TT=252.174 T1=11.8	SAT (TO) f=17 TT=252.037 T1=0.07	SAT (TO) f=18 TT=252.016 T1=0.11	SAT (TO) f=17 TT=252.084 T1=1.7	SAT (TO) f=17 TT=252.046 T1=2.17	SAT (TO) f=17 TT=252.094 T1=1.67	? (TO) TT=252.088
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
CrosswordDesign/ CrosswordDesign-03-4-rom_c18.xml	OPT f=12 TT=0.053 T1=0.05	OPT f=12 TT=1.826 T1=0.76	OPT f=12 TT=9.628 T1=4.29	OPT f=12 TT=9.61 T1=4.31	OPT f=12 TT=0.053 T1=0.05	OPT f=12 TT=0.232 T1=0.21	SAT (TO) f=12 TT=252.07 T1=1.26	OPT f=12 TT=2.544 T1=1.26	OPT f=12 TT=2.586 T1=1.26	? (TO) TT=252.075
CrosswordDesign/ CrosswordDesign-04-4-rom_c18.xml	OPT f=24 TT=0.399 T1=0.32	OPT f=24 TT=3.539 T1=1.7	OPT f=24 TT=14.501 T1=5.7	OPT f=24 TT=14.638 T1=5.6	OPT f=24 TT=0.399 T1=0.32	OPT f=24 TT=13.38 T1=10.05	SAT (TO) f=24 TT=252.051 T1=1.67	OPT f=24 TT=4.463 T1=1.68	OPT f=24 TT=4.295 T1=1.61	? (TO) TT=253.949
CrosswordDesign/ CrosswordDesign-05-4-rom_c18.xml	OPT f=38 TT=23.681 T1=10.76	OPT f=38 TT=72.629 T1=16.56	OPT f=38 TT=168.369 T1=28.51	OPT f=38 TT=139.779 T1=84.45	OPT f=38 TT=23.681 T1=10.76	? (NS) TT=21.354	SAT (TO) f=38 TT=252.104 T1=43.08	SAT (TO) f=38 TT=252.115 T1=66.27	SAT (TO) f=38 TT=252.065 T1=87.57	? (TO) TT=255.438
CrosswordDesign/ CrosswordDesign-06-4-rom_c18.xml	SAT f=52 TT=252.043 T1=214.15	SAT (TO) f=52 TT=252.043 T1=214.15	SAT (TO) f=44 TT=252.129 T1=40.48	SAT (TO) f=44 TT=252.132 T1=127.17	SAT (TO) f=46 TT=252.006 T1=147.3	? (MO) TT=28.272	SAT (TO) f=40 TT=252.051 T1=66.91	SAT (TO) f=42 TT=252.055 T1=24.62	SAT (TO) f=41 TT=252.134 T1=149.89	? (TO) TT=260.179
CrosswordDesign/ CrosswordDesign-07-4-rom_c18.xml	SAT f=50 TT=252.121 T1=76.8	? TT=203.221	SAT (TO) f=48 TT=252.141 T1=222.73	SAT (TO) f=50 TT=252.121 T1=76.8	SAT (TO) f=41 TT=252.038 T1=194.9	? (NS) TT=15.625	? TT=77.557	? TT=76.692	? TT=69.829	? (TO) TT=252.345
CrosswordDesign/ CrosswordDesign-08-4-rom_c18.xml	SAT f=56 TT=252.402 T1=193.27	? TT=225.113	SAT (TO) f=47 TT=252.303 T1=173.24	SAT (TO) f=56 TT=252.402 T1=193.27	SAT (TO) f=4 TT=252.032 T1=59.73	? (NS) TT=16.349	? TT=70.692	? TT=64.425	? TT=66.034	? (TO) TT=252.298
CrosswordDesign/ CrosswordDesign-09-4-rom_c18.xml		? (TO) TT=252.081	? (TO) TT=253.159	? (TO) TT=254.078	? (TO) TT=251.989	? (NS) TT=1.855	? TT=76.234	? TT=71.177	? TT=80.97	? (TO) TT=252.393
CrosswordDesign/ CrosswordDesign-10-4-rom_c18.xml		? TT=201.326	? (TO) TT=255.941	? (TO) TT=256.505	? (TO) TT=251.953	? (NS) TT=2.496	? TT=70.838	? TT=68.193	? TT=72.911	? (TO) TT=252.342
CrosswordDesign/ CrosswordDesign-11-4-rom_c18.xml	SAT f=20 TT=252.011 T1=242.15	? (TO) TT=252.203	? (TO) TT=254.168	? (TO) TT=254.109	SAT (TO) f=20 TT=252.011 T1=242.15	? (NS) TT=3.115	? TT=75.592	? TT=72.665	? TT=73.475	? (TO) TT=252.27
CrosswordDesign/ CrosswordDesign-12-4-rom_c18.xml	SAT f=25 TT=252.016 T1=244.97	? (TO) TT=252.184	? (TO) TT=254.575	? (TO) TT=254.483	SAT (TO) f=25 TT=252.016 T1=244.97	? (NS) TT=3.59	? TT=74.3	? TT=82.814	? TT=74.4	? (TO) TT=252.271
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
CrosswordDesign/ CrosswordDesign-13-4-rom_c18.xml	SAT f=18 TT=251.99 T1=240.45	? (TO) TT=252.185	? (TO) TT=254.962	? (TO) TT=255.413	SAT (TO) f=18 TT=251.99 T1=240.45	? (NS) TT=4.047	? TT=72.244	? TT=70.13	? TT=72.038	? (TO) TT=252.292
CrosswordDesign/ CrosswordDesign-14-4-rom_c18.xml	SAT f=21 TT=252.029 T1=229.25	? (TO) TT=252.193	? (TO) TT=258.744	? (TO) TT=260.283	SAT (TO) f=21 TT=252.029 T1=229.25	? (NS) TT=4.34	? TT=72.173	? TT=68.987	? TT=75.45	? (TO) TT=252.386
CrosswordDesign/ CrosswordDesign-15-4-rom_c18.xml	SAT f=19 TT=252.056 T1=234.16	? TT=62.397	? (TO) TT=257.063	? (TO) TT=260.063	SAT (TO) f=19 TT=252.056 T1=234.16	? (NS) TT=4.621	? TT=77.383	? TT=82.679	? TT=79.612	? (TO) TT=252.342
Fapp/Fapp-m2s/ Fapp-m2s-01-0200_c18.xml	SAT f=35806253 TT=252.084 T1=28.7	SAT (TO) f=98309624 TT=252.065 T1=77.52	SAT (TO) f=35806253 TT=252.084 T1=28.7	SAT (TO) f=98990615 TT=252.206 T1=26.64	? TT=17.012	SAT (TO) f=100532549 TT=251.952 T1=181.81	? (TO) TT=252.11	? (TO) TT=252.05	SAT (TO) f=99729137 TT=252.073 T1=224.16	? (TO) TT=259.842
Fapp/Fapp-m2s/ Fapp-m2s-02-0250_c18.xml	SAT f=40498464 TT=252.17 T1=148.76	? (TO) TT=252.127	SAT (TO) f=40498464 TT=252.17 T1=148.76	SAT (TO) f=60605762 TT=252.121 T1=90.65	? TT=28.397	SAT (TO) f=225625522 TT=252.048 T1=232.38	? (TO) TT=252.095	? (TO) TT=252.133	SAT (TO) f=229148316 TT=252.142 T1=210.32	? (TO) TT=260.268
Fapp/Fapp-m2s/ Fapp-m2s-03-0300_c18.xml	SAT f=294996012 TT=252.162 T1=209.5	? (TO) TT=252.148	SAT (TO) f=294996012 TT=252.162 T1=209.5	SAT (TO) f=295057490 TT=252.224 T1=102.57	? (TO) TT=252.01	SAT (TO) f=470255330 TT=252.033 T1=151.18	? (TO) TT=252.1	? (TO) TT=252.144	SAT (TO) f=479835348 TT=252.126 T1=192.92	? (TO) TT=260.052
Fapp/Fapp-m2s/ Fapp-m2s-04-0300_c18.xml	SAT f=27270670 TT=252.074 T1=74.47	? (TO) TT=252.103	SAT (TO) f=27286280 TT=252.123 T1=90.24	SAT (TO) f=27270670 TT=252.074 T1=74.47	? (TO) TT=252.045	SAT (TO) f=272227144 TT=252.067 T1=241	? (TO) TT=252.102	? (TO) TT=252.088	SAT (TO) f=278945308 TT=252.147 T1=209.49	? (TO) TT=260.221
Fapp/Fapp-m2s/ Fapp-m2s-05-0350_c18.xml	SAT f=524900968 TT=252.272 T1=117.14	? (TO) TT=252.091	SAT (TO) f=525597929 TT=252.127 T1=107.59	SAT (TO) f=524900968 TT=252.272 T1=117.14	? (TO) TT=252.078	SAT (TO) f=533742907 TT=252.061 T1=99.95	? (TO) TT=252.071	? (TO) TT=252.092	? (TO) TT=252.12	? (TO) TT=259.972
Fapp/Fapp-m2s/ Fapp-m2s-06-0500_c18.xml		? (TO) TT=252.137	? (TO) TT=255.835	? (TO) TT=252.146	? (TO) TT=252.08	SAT (TO) f=1050121712 TT=252.084 T1=233.21	? (TO) TT=252.086	? (TO) TT=252.131	? (TO) TT=252.087	? (TO) TT=260.054
Fapp/Fapp-m2s/ Fapp-m2s-07-0600_c18.xml		? (TO) TT=252.127	? (TO) TT=260.064	? (TO) TT=254.128	? (TO) TT=252.156	Wrong UNSAT TT=167.644	? (TO) TT=252.145	? (TO) TT=252.071	? (TO) TT=259.803	? (TO) TT=260.246
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
Fapp/Fapp-m2s/ Fapp-m2s-08-0700_c18.xml		? (TO) TT=252.077	? (TO) TT=253.597	? (TO) TT=260.344	? (TO) TT=252.074	? (TO) TT=252.148	? (TO) TT=252.096	? (TO) TT=252.139	? (TO) TT=252.101	? (TO) TT=260.35
Fapp/Fapp-m2s/ Fapp-m2s-16-0260_c18.xml	SAT f=372140116 TT=252.068 T1=209.25	SAT (TO) f=372140116 TT=252.068 T1=209.25	SAT (TO) f=375507683 TT=252.179 T1=55.81	SAT (TO) f=376869124 TT=252.243 T1=57.82	? TT=37.805	SAT (TO) f=387393741 TT=251.993 T1=104.94	? (TO) TT=252.1	? (TO) TT=252.122	SAT (TO) f=395710634 TT=252.147 T1=207.14	? (TO) TT=257.856
Fapp/Fapp-m2s/ Fapp-m2s-17-0300_c18.xml	OPT f=130971634 TT=82.359 T1=53.12	OPT f=130971634 TT=242.71 T1=225.16	OPT f=130971634 TT=82.359 T1=53.12	OPT f=130971634 TT=102.594 T1=60.13	? (TO) TT=252.019	SAT (TO) f=376480188 TT=252.048 T1=139.01	? (TO) TT=252.127	? (TO) TT=252.086	SAT (TO) f=386181891 TT=252.157 T1=206.64	? (TO) TT=260.167
Fapp/Fapp-m2s/ Fapp-m2s-18-0350_c18.xml	OPT f=162507055 TT=68.795 T1=43.97	OPT f=162507055 TT=68.795 T1=55.28	OPT f=162507055 TT=74.657 T1=46.99	OPT f=162507055 TT=69.789 T1=43.97	? TT=34.242	SAT (TO) f=232823170 TT=252.025 T1=62.34	? (TO) TT=252.069	? (TO) TT=252.104	SAT (TO) f=235860764 TT=252.152 T1=216.23	? (TO) TT=259.979
Fapp/Fapp-m2s/ Fapp-m2s-20-0420_c18.xml	OPT f=506362597 TT=93.068 T1=63.46	OPT f=506362597 TT=183.764 T1=150.31	OPT f=506362597 TT=98.142 T1=68.88	OPT f=506362597 TT=93.068 T1=63.46	? (TO) TT=252.047	SAT (TO) f=579092427 TT=252.022 T1=102.59	? (TO) TT=252.046	? (TO) TT=252.081	? (TO) TT=252.077	? (TO) TT=260.142
Fapp/Fapp-m2s/ Fapp-m2s-21-0500_c18.xml	OPT f=63832030 TT=61.685 T1=38.83	OPT f=63832030 TT=162.011 T1=146.4	OPT f=63832030 TT=61.685 T1=38.83	OPT f=63832030 TT=62.795 T1=38.95	? TT=29.634	SAT (TO) f=182635488 TT=251.982 T1=114.92	? (TO) TT=252.11	? (TO) TT=252.098	SAT (TO) f=187514712 TT=252.148 T1=221.85	? (TO) TT=260.165
Fapp/Fapp-m2s/ Fapp-m2s-ex1_c18.xml	OPT f=523 TT=3.143 T1=1.12	OPT f=523 TT=3.143 T1=1.12	OPT f=523 TT=10 T1=4.17	OPT f=523 TT=10.035 T1=4.42	OPT f=523 TT=18.604 T1=11.61	OPT f=523 TT=0.081 T1=0.07	SAT (TO) f=523 TT=252.07 T1=7.42	OPT f=523 TT=12.999 T1=8.56	SAT (TO) f=523 TT=252.055 T1=2.37	SAT (TO) f=1127 TT=252.073 T1=198.08
Fapp/Fapp-m2s/ Fapp-m2s-ex2_c18.xml	OPT f=13871 TT=2.39 T1=0.89	OPT f=13871 TT=2.39 T1=0.89	OPT f=13871 TT=10.21 T1=4.36	OPT f=13871 TT=10.311 T1=4.42	OPT f=13871 TT=4.605 T1=4.34	OPT f=13871 TT=0.216 T1=0.21	SAT (TO) f=13871 TT=252.054 T1=3.98	OPT f=13871 TT=8.91 T1=4.04	OPT f=13871 TT=3.952 T1=1.72	SAT (TO) f=21587 TT=252.105 T1=80.32
Fapp/Fapp-m2s/ Fapp-m2s-test01-0150_c18.xml	SAT f=18292601 TT=252.062 T1=132.42	SAT (TO) f=18292601 TT=252.062 T1=132.42	SAT (TO) f=18346786 TT=252.152 T1=26.77	SAT (TO) f=50572912 TT=252.163 T1=27.32	SAT (TO) f=51087255 TT=251.986 T1=98.78	SAT (TO) f=51175216 TT=251.937 T1=190.74	? (TO) TT=252.093	SAT (TO) f=51575114 TT=252.111 T1=177.95	SAT (TO) f=51114875 TT=252.232 T1=228.7	? (TO) TT=260.046
Fapp/Fapp-m2s/ Fapp-m2s-test02-0250_c18.xml	SAT f=89722677 TT=252.15 T1=78.21	? (TO) TT=252.095	SAT (TO) f=89722677 TT=252.15 T1=78.21	SAT (TO) f=89790668 TT=252.157 T1=59.94	SAT (TO) f=145576714 TT=252.006 T1=211.81	SAT (TO) f=143278877 TT=252.059 T1=99.39	? (TO) TT=252.043	? (TO) TT=252.071	SAT (TO) f=145181596 TT=252.088 T1=220.12	? (TO) TT=260
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
Fapp/Fapp-m2s/ Fapp-m2s-test03-0400_c18.xml	SAT f=163903545 TT=252.14 T1=97.98	? (TO) TT=252.105	SAT (TO) f=164044993 TT=252.084 T1=87.52	SAT (TO) f=163903545 TT=252.14 T1=97.98	? TT=60.514	SAT (TO) f=456165615 TT=252.028 T1=224.68	? (TO) TT=252.136	? (TO) TT=252.109	? (TO) TT=252.258	? (TO) TT=257.7
GolombRuler/ GolombRuler-a3v18-07_c18.xml	OPT f=25 TT=0.041 T1=0.02	OPT f=25 TT=2.697 T1=0.79	OPT f=25 TT=7.597 T1=3.51	OPT f=25 TT=7.758 T1=3.54	OPT f=25 TT=0.041 T1=0.02	OPT f=25 TT=0.03 T1=0.01	SAT f=25 TT=251.63 T1=1.08	OPT f=25 TT=1.814 T1=1.09	OPT f=25 TT=1.889 T1=1.11	OPT f=25 TT=13.206 T1=4.28
GolombRuler/ GolombRuler-a3v18-08_c18.xml	OPT f=34 TT=0.294 T1=0.1	OPT f=34 TT=8.67 T1=2.54	OPT f=34 TT=9.466 T1=3.91	OPT f=34 TT=10.41 T1=4.17	OPT f=34 TT=0.294 T1=0.1	OPT f=34 TT=0.172 T1=0.03	SAT f=34 TT=251.726 T1=1.28	OPT f=34 TT=2.708 T1=1.28	OPT f=34 TT=2.312 T1=1.13	OPT f=34 TT=58.345 T1=12.85
GolombRuler/ GolombRuler-a3v18-09_c18.xml	OPT f=44 TT=1.832 T1=1.35	OPT f=44 TT=71.545 T1=26.55	OPT f=44 TT=17.938 T1=9.7	OPT f=44 TT=19.122 T1=6.56	OPT f=44 TT=1.832 T1=1.35	OPT f=44 TT=1.657 T1=0.05	SAT (TO) f=44 TT=252.082 T1=2.06	OPT f=44 TT=4.568 T1=2.05	OPT f=44 TT=4.249 T1=1.79	SAT (TO) f=44 TT=252.108 T1=128.58
GolombRuler/ GolombRuler-a3v18-10_c18.xml	OPT f=55 TT=7.438 T1=4.03	SAT (TO) f=59 TT=252.069 T1=20.44	OPT f=55 TT=74.321 T1=53.33	OPT f=55 TT=68.66 T1=22.94	OPT f=55 TT=14.979 T1=8.84	OPT f=55 TT=12.577 T1=0.43	SAT (TO) f=55 TT=252.122 T1=14.96	OPT f=55 TT=26.523 T1=6.95	OPT f=55 TT=7.438 T1=4.03	SAT (TO) f=64 TT=252.183 T1=232.26
GolombRuler/ GolombRuler-a3v18-11_c18.xml	SAT f=72 TT=249.995 T1=65.5	SAT (TO) f=78 TT=252.09 T1=149.04	SAT (TO) f=72 TT=252.12 T1=198.66	SAT (TO) f=72 TT=252.125 T1=66.94	SAT (TO) f=72 TT=251.948 T1=77.25	SAT (TO) f=72 TT=251.999 T1=7.35	SAT f=72 TT=249.995 T1=116.43	SAT f=72 TT=250.176 T1=65.5	SAT (TO) f=77 TT=252.088 T1=7.72	SAT (TO) f=95 TT=252.071 T1=243.96
GolombRuler/ GolombRuler-a3v18-12_c18.xml	SAT f=92 TT=250.493 T1=211.29	SAT (TO) f=104 TT=252.024 T1=196.83	SAT (TO) f=93 TT=252.091 T1=120.38	SAT (TO) f=96 TT=252.148 T1=143.91	SAT (TO) f=94 TT=251.988 T1=227.28	SAT (TO) f=91 TT=252.01 T1=242.39	SAT f=93 TT=250.613 T1=162.35	SAT f=92 TT=250.493 T1=211.29	SAT (TO) f=96 TT=252.078 T1=18.22	SAT (TO) f=124 TT=252.13 T1=213
GolombRuler/ GolombRuler-a3v18-13_c18.xml	SAT f=114 TT=250.599 T1=80.61	SAT (TO) f=133 TT=252.046 T1=65.63	SAT (TO) f=117 TT=252.084 T1=143.45	SAT (TO) f=121 TT=252.109 T1=83.63	SAT (TO) f=119 TT=252.007 T1=179.19	SAT (TO) f=110 TT=252.004 T1=21.48	SAT f=116 TT=250.661 T1=237.98	SAT f=114 TT=250.599 T1=80.61	SAT (TO) f=120 TT=252.086 T1=172.86	? (TO) TT=252.092
GolombRuler/ GolombRuler-a3v18-14_c18.xml	SAT f=140 TT=250.339 T1=170.28	SAT (TO) f=166 TT=252.061 T1=45.2	SAT (TO) f=140 TT=252.075 T1=170.28	SAT (TO) f=148 TT=252.071 T1=94.42	SAT (TO) f=146 TT=252.008 T1=233.98	SAT (TO) f=141 TT=252.003 T1=195.75	SAT f=148 TT=250.247 T1=144.48	SAT f=140 TT=250.339 T1=213.37	SAT (TO) f=148 TT=252.086 T1=238.53	? (TO) TT=252.083
GolombRuler/ GolombRuler-a3v18-15_c18.xml	SAT f=165 TT=252.079 T1=124.68	SAT (TO) f=200 TT=252.108 T1=111.58	SAT (TO) f=165 TT=252.079 T1=124.68	SAT (TO) f=173 TT=252.133 T1=15.82	SAT (TO) f=181 TT=252.039 T1=50.31	SAT (TO) f=171 TT=251.976 T1=44.75	SAT f=177 TT=250.213 T1=112.09	SAT f=171 TT=250.416 T1=125.64	SAT (TO) f=177 TT=252.106 T1=212.89	? (TO) TT=252.104
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
GolombRuler/ GolombRuler-a3v18-16_c18.xml	SAT f=205 TT=252.136 T1=26.8	SAT (TO) f=227 TT=252.031 T1=36.14	SAT (TO) f=205 TT=252.136 T1=26.8	SAT (TO) f=213 TT=252.099 T1=214.33	SAT (TO) f=218 TT=251.963 T1=198.41	SAT (TO) f=199 TT=252.009 T1=248.51	SAT f=210 TT=250.532 T1=94.09	SAT f=209 TT=250.522 T1=241.75	SAT (TO) f=221 TT=252.119 T1=198.36	? (TO) TT=252.154
GolombRuler/ GolombRuler-a3v18-nodv-09_c18.xml	OPT f=44 TT=11.641 T1=9.1	OPT f=44 TT=69.917 T1=26.28	OPT f=44 TT=110.868 T1=52.92	OPT f=44 TT=209.951 T1=11.36	OPT f=44 TT=11.641 T1=9.1	OPT f=44 TT=1.674 T1=0.05	SAT f=44 TT=249.879 T1=33.14	SAT f=44 TT=250.434 T1=215.75	SAT (TO) f=47 TT=252.033 T1=103.3	SAT (TO) f=44 TT=252.185 T1=128.11
GolombRuler/ GolombRuler-a3v18-nodv-10_c18.xml	OPT f=55 TT=174.462 T1=144.33	SAT (TO) f=59 TT=252.12 T1=20.6	SAT (TO) f=60 TT=252.142 T1=7.46	SAT (TO) f=61 TT=252.074 T1=14.11	OPT f=55 TT=174.462 T1=144.33	OPT f=55 TT=12.663 T1=0.43	SAT f=61 TT=250.133 T1=193.09	SAT f=61 TT=250.036 T1=124.48	SAT (TO) f=64 TT=252.061 T1=69.71	SAT (TO) f=64 TT=252.088 T1=234.31
GolombRuler/ GolombRuler-a3v18-nodv-11_c18.xml	SAT f=75 TT=252.13 T1=12.66	SAT (TO) f=78 TT=252.11 T1=147.06	SAT (TO) f=75 TT=252.13 T1=12.66	SAT (TO) f=80 TT=252.125 T1=38.83	SAT (TO) f=76 TT=252.021 T1=96.42	SAT (TO) f=72 TT=251.976 T1=7.26	SAT f=88 TT=250.243 T1=71.11	SAT f=85 TT=250.209 T1=26.05	SAT (TO) f=84 TT=252.064 T1=244.13	SAT (TO) f=95 TT=252.173 T1=243.58
GraphColoring/ GraphColoring-2-fullins-5_c18.xml	OPT f=6 TT=236.466 T1=1.97	SAT (TO) f=6 TT=252.114 T1=2.28	SAT (TO) f=6 TT=252.158 T1=32.49	SAT (TO) f=6 TT=252.082 T1=56.29	OPT f=6 TT=236.466 T1=1.97	SAT (TO) f=6 TT=251.982 T1=2.19	SAT (TO) f=6 TT=252.047 T1=50.46	SAT (TO) f=6 TT=252.09 T1=51.7	SAT (TO) f=240 TT=252.108 T1=235.2	? (TO) TT=260.202
GraphColoring/ GraphColoring-3-fullins-5_c18.xml	SAT f=7 TT=251.941 T1=5.65	SAT (TO) f=7 TT=252.133 T1=5.75	SAT (TO) f=1646 TT=252.104 T1=221.05	SAT (TO) f=1830 TT=252.145 T1=198.69	SAT (TO) f=7 TT=251.941 T1=5.65	SAT (TO) f=7 TT=251.992 T1=6.87	SAT (TO) f=1479 TT=252.111 T1=238.16	SAT (TO) f=1470 TT=252.057 T1=237.35	SAT (TO) f=1845 TT=252.712 T1=224.34	? (TO) TT=260.111
GraphColoring/ GraphColoring-4-fullins-4_c18.xml	OPT f=7 TT=18.72 T1=1.09	OPT f=7 TT=74.584 T1=1.46	OPT f=7 TT=120.831 T1=19.26	SAT (TO) f=7 TT=252.131 T1=26.05	OPT f=7 TT=18.72 T1=1.09	OPT f=7 TT=38.258 T1=1.16	SAT (TO) f=7 TT=252.138 T1=19.7	OPT f=7 TT=226.853 T1=20.31	SAT (TO) f=7 TT=252.127 T1=50.87	? (TO) TT=260.164
GraphColoring/ GraphColoring-4-fullins-5_c18.xml	SAT f=8 TT=251.918 T1=13.6	SAT (TO) f=8 TT=252.061 T1=23.78	? (TO) TT=254.21	? (TO) TT=254.16	SAT (TO) f=8 TT=251.918 T1=13.6	SAT (TO) f=8 TT=251.997 T1=18.99	SAT (TO) f=4113 TT=252.117 T1=213.93	SAT (TO) f=4114 TT=252.426 T1=205.72	SAT (TO) f=4138 TT=252.17 T1=52	? (TO) TT=252.736
GraphColoring/ GraphColoring-5-fullins-3_c18.xml	OPT f=7 TT=0.358 T1=0.13	OPT f=7 TT=6.039 T1=0.74	OPT f=7 TT=17.281 T1=4.93	OPT f=7 TT=18.328 T1=5.12	OPT f=7 TT=0.358 T1=0.13	OPT f=7 TT=0.499 T1=0.14	SAT (TO) f=7 TT=252.031 T1=1.71	OPT f=7 TT=17.94 T1=1.69	SAT (TO) f=7 TT=252.034 T1=2.01	SAT (TO) f=126 TT=252.094 T1=219.49
GraphColoring/ GraphColoring-5-fullins-4_c18.xml	OPT f=8 TT=207.316 T1=1.86	SAT (TO) f=8 TT=252.034 T1=2.47	SAT (TO) f=8 TT=252.096 T1=45.07	SAT (TO) f=8 TT=252.117 T1=82.98	OPT f=8 TT=207.316 T1=1.86	SAT (TO) f=8 TT=251.981 T1=2.08	SAT (TO) f=8 TT=252.126 T1=74.24	SAT (TO) f=8 TT=252.141 T1=81.05	SAT (TO) f=600 TT=252.134 T1=234.51	? (TO) TT=260.019
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
GraphColoring/ GraphColoring-qg-order100_c18.xml		? (TO) TT=252.124	? (TO) TT=259.061	? (TO) TT=259.914	? (MO) TT=69.863	? (TO) TT=252.084	? (TO) TT=257.218	? (TO) TT=255.315	? (TO) TT=259.108	? (TO) TT=254.991
GraphColoring/ GraphColoring-qg-order30_c18.xml	SAT f=29 TT=251.924 T1=3.16	SAT (TO) f=29 TT=252.156 T1=3.16	SAT (TO) f=83 TT=252.096 T1=233.09	SAT (TO) f=497 TT=252.155 T1=229.31	SAT (TO) f=29 TT=251.924 T1=4.44	SAT (TO) f=29 TT=251.958 T1=5.25	SAT (TO) f=29 TT=252.071 T1=143.23	SAT (TO) f=29 TT=252.129 T1=147.98	SAT (TO) f=297 TT=252.075 T1=235.46	? (TO) TT=259.909
GraphColoring/ GraphColoring-qg-order40_c18.xml	SAT f=39 TT=251.871 T1=9.28	SAT (TO) f=39 TT=252.109 T1=9.28	SAT (TO) f=1274 TT=252.056 T1=228.91	SAT (TO) f=1429 TT=252.131 T1=224.78	SAT (TO) f=39 TT=251.871 T1=10.71	SAT (TO) f=39 TT=252.029 T1=15.39	SAT (TO) f=1266 TT=252.124 T1=235.21	SAT (TO) f=1276 TT=252.117 T1=233.72	SAT (TO) f=1433 TT=252.301 T1=227.53	? (TO) TT=260.352
GraphColoring/ GraphColoring-qg-order60_c18.xml	SAT f=59 TT=252.101 T1=81.8	SAT (TO) f=59 TT=252.101 T1=223.24	SAT (TO) f=3571 TT=252.103 T1=187.82	SAT (TO) f=3578 TT=252.097 T1=178.44	SAT (TO) f=59 TT=252.249 T1=81.8	SAT (TO) f=59 TT=252.012 T1=141.91	? (TO) TT=253.995	? (TO) TT=253.018	? (TO) TT=255.873	? (TO) TT=260.162
GraphColoring/ GraphColoring-queen5-5_c18.xml	OPT f=4 TT=0.032 T1=0.03	OPT f=4 TT=1.254 T1=0.57	OPT f=4 TT=7.639 T1=3.61	OPT f=4 TT=7.141 T1=3.57	OPT f=4 TT=0.032 T1=0.03	OPT f=4 TT=0.031 T1=0.03	SAT f=4 TT=251.844 T1=1.15	OPT f=4 TT=1.993 T1=1.5	OPT f=4 TT=2.008 T1=1.16	OPT f=4 TT=3.08 T1=1.33
Knapsack/ Knapsack-30-100-00_c18.xml	OPT f=709 TT=0.751 T1=0.43	OPT f=709 TT=5.569 T1=1.29	OPT f=709 TT=16.206 T1=3.93	OPT f=709 TT=14.847 T1=3.49	OPT f=709 TT=0.751 T1=0.43	OPT f=709 TT=0.097 T1=0.01	SAT f=709 TT=251.493 T1=1.14	OPT f=709 TT=2.221 T1=1.43	OPT f=709 TT=2.231 T1=1.07	OPT f=709 TT=110.733 T1=22.02
Knapsack/ Knapsack-30-100-01_c18.xml	OPT f=871 TT=0.459 T1=0.35	OPT f=871 TT=3.516 T1=1.26	OPT f=871 TT=12.755 T1=3.88	OPT f=871 TT=12.19 T1=3.91	OPT f=871 TT=0.459 T1=0.35	OPT f=871 TT=0.07 T1=0	SAT f=871 TT=251.073 T1=1.22	OPT f=871 TT=2.186 T1=1.22	OPT f=871 TT=2.295 T1=1.24	OPT f=871 TT=88.833 T1=5.67
Knapsack/ Knapsack-40-150-00_c18.xml	OPT f=932 TT=6.64 T1=1.9	OPT f=932 TT=82.706 T1=45.64	OPT f=932 TT=140.147 T1=4.74	SAT (TO) f=932 TT=252.056 T1=3.85	OPT f=932 TT=27.028 T1=20.59	OPT f=932 TT=3.408 T1=0.01	SAT f=932 TT=251.374 T1=1.91	OPT f=932 TT=7.732 T1=1.9	OPT f=932 TT=6.64 T1=4.18	SAT (TO) f=897 TT=252.119 T1=234.59
Knapsack/ Knapsack-40-150-01_c18.xml	OPT f=1128 TT=4.09 T1=1.76	OPT f=1128 TT=38.866 T1=19.62	OPT f=1128 TT=86.665 T1=5.88	OPT f=1128 TT=117.04 T1=4.49	OPT f=1128 TT=19.092 T1=17.9	OPT f=1128 TT=1.9 T1=0.02	SAT f=1128 TT=251.718 T1=1.76	OPT f=1128 TT=4.289 T1=1.78	OPT f=1128 TT=4.09 T1=2.18	SAT (TO) f=1110 TT=252.167 T1=12.19
Knapsack/ Knapsack-50-200-00_c18.xml	OPT f=960 TT=34.981 T1=5.68	SAT (TO) f=960 TT=252.037 T1=101.29	SAT (TO) f=960 TT=252.15 T1=5.68	SAT (TO) f=960 TT=252.052 T1=12.67	OPT f=960 TT=59.825 T1=59.43	OPT f=960 TT=8.033 T1=0.44	SAT f=960 TT=251.815 T1=9.61	OPT f=960 TT=34.981 T1=13.66	SAT (TO) f=960 TT=252.035 T1=6.5	SAT (TO) f=842 TT=252.139 T1=228.7
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
Knapsack/ Knapsack-50-200-01_c18.xml	OPT f=1297 TT=28.697 T1=4.26	SAT (TO) f=1297 TT=252.062 T1=50.98	SAT (TO) f=1297 TT=252.091 T1=4.85	SAT (TO) f=1297 TT=252.136 T1=5.08	OPT f=1297 TT=84.436 T1=78.52	OPT f=1297 TT=6.188 T1=0.01	SAT f=1297 TT=251.809 T1=4.26	OPT f=1297 TT=28.697 T1=5.6	SAT (TO) f=1297 TT=252.097 T1=4.75	? (TO) TT=252.037
Knapsack/ Knapsack-60-250-00_c18.xml	SAT f=1404 TT=248.953 T1=9.69	SAT (TO) f=1332 TT=252.125 T1=2.56	SAT (TO) f=1404 TT=252.092 T1=53.35	SAT (TO) f=1404 TT=252.056 T1=9.69	SAT (TO) f=1238 TT=252.008 T1=133.31	OPT f=1404 TT=211.063 T1=0.06	SAT f=1404 TT=248.953 T1=42.26	SAT f=1404 TT=248.978 T1=143.89	SAT (TO) f=1393 TT=252.104 T1=3.31	? (TO) TT=252.071
Knapsack/ Knapsack-60-250-01_c18.xml	SAT f=1323 TT=248.906 T1=2.97	SAT (TO) f=1313 TT=252.058 T1=233.06	SAT (TO) f=1323 TT=252.086 T1=6.07	SAT (TO) f=1323 TT=252.107 T1=44.32	SAT (TO) f=1237 TT=251.957 T1=119.19	OPT f=1323 TT=157.546 T1=0.05	SAT f=1323 TT=248.94 T1=88.29	SAT f=1323 TT=248.906 T1=41.77	SAT (TO) f=1323 TT=252.049 T1=2.97	? (TO) TT=252.042
Knapsack/ Knapsack-70-300-00_c18.xml	SAT f=1425 TT=248.92 T1=46.37	SAT (TO) f=1318 TT=252.117 T1=22.47	SAT (TO) f=1422 TT=252.041 T1=19.71	SAT (TO) f=1425 TT=252.068 T1=93.35	SAT (TO) f=1189 TT=252.014 T1=191.78	SAT (TO) f=1425 TT=251.97 T1=0.11	SAT f=1425 TT=248.92 T1=133.49	SAT f=1425 TT=248.981 T1=165.97	SAT (TO) f=1425 TT=252.064 T1=46.37	? (TO) TT=252.096
Knapsack/ Knapsack-70-300-01_c18.xml	SAT f=1465 TT=248.995 T1=4.95	SAT (TO) f=1406 TT=252.112 T1=140.31	SAT (TO) f=1465 TT=252.076 T1=5.79	SAT (TO) f=1465 TT=252.09 T1=4.95	SAT (TO) f=1204 TT=252.002 T1=234.07	SAT (TO) f=1465 TT=252.001 T1=0.02	SAT f=1465 TT=249.078 T1=110.07	SAT f=1465 TT=248.995 T1=99.54	SAT (TO) f=1465 TT=252.07 T1=6.87	? (TO) TT=252.093
Knapsack/ Knapsack-80-350-00_c18.xml	SAT f=1564 TT=248.986 T1=13.4	SAT (TO) f=1389 TT=252.037 T1=151.97	SAT (TO) f=1562 TT=252.065 T1=123.4	SAT (TO) f=1564 TT=252.082 T1=13.4	SAT (TO) f=1215 TT=252.004 T1=174.55	SAT (TO) f=1564 TT=252.001 T1=0.03	SAT f=1564 TT=248.986 T1=241.92	SAT f=1561 TT=248.895 T1=18.24	SAT (TO) f=1564 TT=252.027 T1=26.11	? (TO) TT=252.112
Knapsack/ Knapsack-80-350-01_c18.xml	SAT f=1668 TT=252.067 T1=14.07	SAT (TO) f=1512 TT=252.037 T1=5.54	SAT (TO) f=1668 TT=252.148 T1=53.64	SAT (TO) f=1668 TT=252.076 T1=16.14	SAT (TO) f=1339 TT=251.994 T1=227.58	SAT (TO) f=1668 TT=251.972 T1=0.02	SAT f=1659 TT=248.969 T1=23.85	SAT f=1659 TT=249.044 T1=140.48	SAT (TO) f=1668 TT=252.067 T1=14.07	? (TO) TT=252.359
Knapsack/ Knapsack-90-400-00_c18.xml	SAT f=1551 TT=252.139 T1=71.88	SAT (TO) f=1366 TT=252.083 T1=218.29	SAT (TO) f=1551 TT=252.139 T1=71.88	SAT (TO) f=1505 TT=252.158 T1=146.63	SAT (TO) f=1019 TT=252.007 T1=22.04	SAT (TO) f=1551 TT=252.004 T1=0.05	SAT f=1519 TT=249.049 T1=95.84	SAT f=1497 TT=249.04 T1=44.51	SAT (TO) f=1547 TT=252.06 T1=62.94	? (TO) TT=252.098
Knapsack/ Knapsack-90-400-01_c18.xml	SAT f=1681 TT=248.932 T1=8.14	SAT (TO) f=1501 TT=252.089 T1=128.37	SAT (TO) f=1681 TT=252.104 T1=40.4	SAT (TO) f=1681 TT=252.129 T1=215.86	SAT (TO) f=1141 TT=252.037 T1=138.52	SAT (TO) f=1681 TT=252.014 T1=0.08	SAT f=1681 TT=248.932 T1=86.06	SAT f=1680 TT=248.972 T1=52.63	SAT (TO) f=1681 TT=252.061 T1=8.14	? (TO) TT=252.068
LowAutocorrelation/ LowAutocorrelation-015_c18.xml	OPT f=15 TT=2.694 T1=0.25	OPT f=15 TT=9.945 T1=1.62	OPT f=15 TT=27.205 T1=6.35	OPT f=15 TT=20.433 T1=5.94	OPT f=15 TT=7.893 T1=0.25	OPT f=15 TT=1.624 T1=0.56	SAT (TO) f=15 TT=252.049 T1=1.31	OPT f=15 TT=4.159 T1=1.32	OPT f=15 TT=4.128 T1=1.24	OPT f=15 TT=2.694 T1=0.84
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
LowAutocorrelation/ LowAutocorrelation-020_c18.xml	OPT f=26 TT=33.518 T1=2.24	SAT (TO) f=26 TT=252.112 T1=11.64	SAT (TO) f=26 TT=252.088 T1=62.26	SAT (TO) f=26 TT=252.107 T1=229.2	SAT (TO) f=34 TT=251.961 T1=2.72	OPT f=26 TT=36.565 T1=6.03	SAT (TO) f=26 TT=252.066 T1=3.25	OPT f=26 TT=78.632 T1=3.71	SAT (TO) f=26 TT=252.092 T1=2.24	OPT f=26 TT=33.518 T1=7.73
LowAutocorrelation/ LowAutocorrelation-021_c18.xml	OPT f=26 TT=59.209 T1=3.15	SAT (TO) f=34 TT=252.076 T1=9.79	SAT (TO) f=26 TT=252.145 T1=169.19	SAT (TO) f=34 TT=252.119 T1=141.67	SAT (TO) f=26 TT=252.007 T1=77.94	OPT f=26 TT=32.961 T1=7.19	SAT (TO) f=26 TT=252.053 T1=3.15	OPT f=26 TT=154.971 T1=24.34	SAT (TO) f=26 TT=252.059 T1=24.1	OPT f=26 TT=59.209 T1=23.77
LowAutocorrelation/ LowAutocorrelation-022_c18.xml	OPT f=39 TT=151.521 T1=1.78	SAT (TO) f=39 TT=252.085 T1=3.71	SAT (TO) f=39 TT=252.121 T1=47.53	SAT (TO) f=39 TT=252.111 T1=100.08	SAT (TO) f=71 TT=252.006 T1=199.75	SAT (TO) f=39 TT=251.996 T1=2.23	SAT (TO) f=39 TT=252.131 T1=15.21	SAT (TO) f=39 TT=252.1 T1=3.11	SAT (TO) f=39 TT=252.075 T1=2.24	OPT f=39 TT=151.521 T1=1.78
LowAutocorrelation/ LowAutocorrelation-023_c18.xml	SAT f=47 TT=252.011 T1=2.13	SAT (TO) f=51 TT=252.103 T1=243.36	SAT (TO) f=47 TT=252.063 T1=68.62	SAT (TO) f=51 TT=252.115 T1=26.08	SAT (TO) f=47 TT=252.011 T1=192.58	SAT (TO) f=47 TT=252.012 T1=18.43	SAT (TO) f=47 TT=252.049 T1=10.2	SAT (TO) f=47 TT=252.08 T1=4.46	SAT (TO) f=47 TT=252.034 T1=2.13	SAT (TO) f=47 TT=252.129 T1=19.02
LowAutocorrelation/ LowAutocorrelation-024_c18.xml	SAT f=36 TT=252.065 T1=14.86	SAT (TO) f=56 TT=252.055 T1=144.49	SAT (TO) f=36 TT=252.134 T1=158.61	SAT (TO) f=36 TT=252.093 T1=14.86	SAT (TO) f=56 TT=251.987 T1=72.63	OPT f=36 TT=209.089 T1=15.96	SAT (TO) f=36 TT=252.065 T1=216.27	SAT (TO) f=36 TT=252.108 T1=168.24	SAT (TO) f=44 TT=252.076 T1=6.86	SAT (TO) f=36 TT=252.137 T1=65.35
LowAutocorrelation/ LowAutocorrelation-025_c18.xml	SAT f=36 TT=252.034 T1=8.98	SAT (TO) f=44 TT=252.073 T1=183.5	SAT (TO) f=56 TT=252.104 T1=137.61	SAT (TO) f=36 TT=252.099 T1=8.98	SAT (TO) f=52 TT=251.996 T1=235.54	SAT (TO) f=44 TT=251.985 T1=130.77	SAT (TO) f=36 TT=252.034 T1=234.43	SAT (TO) f=44 TT=252.077 T1=69.54	SAT (TO) f=44 TT=252.036 T1=11.76	SAT (TO) f=36 TT=252.083 T1=239.19
LowAutocorrelation/ LowAutocorrelation-026_c18.xml	SAT f=45 TT=252.046 T1=130.96	SAT (TO) f=61 TT=252.066 T1=242.65	SAT (TO) f=61 TT=252.138 T1=6.62	SAT (TO) f=69 TT=252.143 T1=193.81	SAT (TO) f=77 TT=251.982 T1=37.7	SAT (TO) f=45 TT=252.002 T1=38.75	SAT (TO) f=45 TT=252.079 T1=170.39	SAT (TO) f=45 TT=252.046 T1=130.96	SAT (TO) f=53 TT=252.064 T1=5.05	SAT (TO) f=45 TT=252.15 T1=157.46
LowAutocorrelation/ LowAutocorrelation-027_c18.xml	SAT f=53 TT=252.106 T1=212.98	SAT (TO) f=77 TT=252.051 T1=68.05	SAT (TO) f=61 TT=252.135 T1=166.06	SAT (TO) f=69 TT=252.09 T1=25.19	SAT (TO) f=73 TT=252.002 T1=64.41	SAT (TO) f=65 TT=251.997 T1=55.09	SAT (TO) f=69 TT=252.103 T1=107.09	SAT (TO) f=53 TT=252.106 T1=212.98	SAT (TO) f=65 TT=252.085 T1=234.55	SAT (TO) f=65 TT=252.084 T1=151.9
LowAutocorrelation/ LowAutocorrelation-028_c18.xml	SAT f=66 TT=252.098 T1=108.75	SAT (TO) f=94 TT=252.093 T1=87.68	SAT (TO) f=82 TT=252.139 T1=108.78	SAT (TO) f=86 TT=252.089 T1=198.59	SAT (TO) f=170 TT=251.996 T1=0.9	SAT (TO) f=74 TT=251.988 T1=19.04	SAT (TO) f=70 TT=252.061 T1=53.13	SAT (TO) f=66 TT=252.118 T1=202.32	SAT (TO) f=66 TT=252.098 T1=108.75	SAT (TO) f=70 TT=252.07 T1=57.84
LowAutocorrelation/ LowAutocorrelation-030_c18.xml	SAT f=67 TT=252.092 T1=151.68	SAT (TO) f=115 TT=252.111 T1=26.42	SAT (TO) f=99 TT=252.107 T1=220.04	SAT (TO) f=99 TT=252.139 T1=39.35	SAT (TO) f=131 TT=252.037 T1=184.96	SAT (TO) f=75 TT=251.995 T1=208.87	SAT (TO) f=83 TT=252.106 T1=7.97	SAT (TO) f=67 TT=252.092 T1=151.68	SAT (TO) f=99 TT=252.073 T1=9.88	SAT (TO) f=83 TT=252.074 T1=133.83
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
LowAutocorrelation/ LowAutocorrelation-035_c18.xml	SAT f=117 TT=252.113 T1=138.51	SAT (TO) f=137 TT=252.105 T1=1.77	SAT (TO) f=117 TT=252.113 T1=138.51	SAT (TO) f=165 TT=252.053 T1=174.7	SAT (TO) f=281 TT=252.002 T1=207.1	SAT (TO) f=133 TT=251.999 T1=135.58	SAT (TO) f=129 TT=252.119 T1=21.82	SAT (TO) f=125 TT=252.056 T1=49.06	SAT (TO) f=169 TT=252.116 T1=18.42	SAT (TO) f=137 TT=252.115 T1=110.1
LowAutocorrelation/ LowAutocorrelation-040_c18.xml	SAT f=156 TT=252.102 T1=212.98	SAT (TO) f=224 TT=252.068 T1=40.16	SAT (TO) f=208 TT=252.072 T1=212.54	SAT (TO) f=220 TT=252.067 T1=58.72	SAT (TO) f=500 TT=252.033 T1=2.73	SAT (TO) f=180 TT=251.997 T1=203.33	SAT (TO) f=188 TT=252.051 T1=49.51	SAT (TO) f=156 TT=252.102 T1=212.98	SAT (TO) f=272 TT=252.034 T1=26.32	SAT (TO) f=204 TT=252.115 T1=43.8
LowAutocorrelation/ LowAutocorrelation-050_c18.xml	SAT f=329 TT=252.076 T1=186.52	SAT (TO) f=433 TT=252.074 T1=156.01	SAT (TO) f=377 TT=252.088 T1=130.44	SAT (TO) f=361 TT=252.107 T1=75.6	SAT (TO) f=673 TT=251.992 T1=35.14	SAT (TO) f=321 TT=252 T1=50.07	SAT (TO) f=345 TT=252.047 T1=233.75	SAT (TO) f=345 TT=252.126 T1=92.1	SAT (TO) f=329 TT=252.076 T1=186.52	SAT (TO) f=401 TT=252.062 T1=227.87
Mario/ Mario-easy-2_c18.xml	OPT f=628 TT=1.318 T1=0.6	OPT f=628 TT=1.318 T1=0.6	OPT f=628 TT=8.041 T1=3.9	OPT f=628 TT=7.895 T1=4.02	? (NS) TT=0.011	OPT f=628 TT=0.044 T1=0.04	SAT f=628 TT=251.698 T1=1.29	OPT f=628 TT=2.255 T1=1.26	OPT f=628 TT=2.34 T1=1.29	? (TO) TT=252.426
Mario/ Mario-easy-4_c18.xml	OPT f=545 TT=2.851 T1=1.42	OPT f=545 TT=4.518 T1=1.59	OPT f=545 TT=13.957 T1=6.25	OPT f=545 TT=17.152 T1=7.25	? (NS) TT=0.011	OPT f=545 TT=8.262 T1=8.07	SAT (TO) f=545 TT=252.11 T1=1.48	OPT f=545 TT=3.019 T1=1.49	OPT f=545 TT=2.851 T1=1.42	? (TO) TT=256.163
Mario/ Mario-easy-5_c18.xml	OPT f=445 TT=2.492 T1=0.98	OPT f=445 TT=2.713 T1=0.98	OPT f=445 TT=13.997 T1=4.63	OPT f=445 TT=14.254 T1=4.37	? (NS) TT=0.011	OPT f=445 TT=2.671 T1=0.03	SAT (TO) f=445 TT=252.048 T1=1.32	OPT f=445 TT=2.794 T1=1.31	OPT f=445 TT=2.492 T1=1.23	? (TO) TT=252.579
Mario/ Mario-n-medium-2_c18.xml	OPT f=1053 TT=169.378 T1=93.93	OPT f=1053 TT=169.378 T1=93.93	SAT (TO) f=990 TT=252.056 T1=18.28	SAT (TO) f=990 TT=252.09 T1=106.94	? (NS) TT=0.02	SAT (TO) f=1053 TT=252.024 T1=97.05	SAT f=990 TT=251.009 T1=139.49	SAT f=990 TT=251.131 T1=154.57	SAT (TO) f=886 TT=252.098 T1=21.04	? (TO) TT=253.05
Mario/ Mario-n-medium-3_c18.xml	OPT f=943 TT=83.269 T1=3.04	OPT f=943 TT=83.269 T1=59.07	SAT (TO) f=943 TT=252.148 T1=21.42	SAT (TO) f=943 TT=252.122 T1=173.12	? (NS) TT=0.02	SAT (TO) f=943 TT=252.025 T1=122.03	SAT (TO) f=943 TT=252.057 T1=3.04	OPT f=943 TT=105.947 T1=9.23	SAT (TO) f=943 TT=252.063 T1=11.23	? (TO) TT=252.31
Mario/ Mario-n-medium-4_c18.xml	OPT f=832 TT=124.685 T1=21.39	OPT f=832 TT=124.685 T1=97.78	SAT (TO) f=832 TT=252.079 T1=25.63	SAT (TO) f=832 TT=252.102 T1=193.39	? (NS) TT=0.021	SAT (TO) f=832 TT=251.972 T1=28.92	SAT (TO) f=832 TT=252.081 T1=37.61	OPT f=832 TT=171.596 T1=40.2	SAT (TO) f=832 TT=252.076 T1=21.39	? (TO) TT=252.29
Mario/ Mario-n-medium-5_c18.xml	OPT f=771 TT=87.232 T1=15.26	OPT f=771 TT=87.232 T1=34.07	SAT (TO) f=771 TT=252.101 T1=28.66	SAT (TO) f=771 TT=252.045 T1=163.14	? (NS) TT=0.021	SAT (TO) f=771 TT=252.01 T1=11.1	SAT (TO) f=771 TT=252.118 T1=30.17	OPT f=771 TT=113.776 T1=15.26	SAT (TO) f=769 TT=252.095 T1=8.39	? (TO) TT=252.551
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
Mario/ Mario-t-hard-1_c18.xml	OPT f=4783 TT=4.511 T1=1.89	OPT f=4783 TT=14.498 T1=11.2	OPT f=4783 TT=30.82 T1=19.15	OPT f=4783 TT=32.712 T1=18.81	? (NS) TT=0.087	OPT f=4783 TT=5.707 T1=5.71	SAT (TO) f=4783 TT=252.063 T1=2.55	OPT f=4783 TT=6.478 T1=2.59	OPT f=4783 TT=4.511 T1=1.89	? (TO) TT=260.059
Mario/ Mario-t-hard-2_c18.xml	OPT f=4850 TT=6.452 T1=2.68	OPT f=4850 TT=15.881 T1=12.65	OPT f=4850 TT=82.172 T1=67.42	OPT f=4850 TT=33.103 T1=18.62	? (NS) TT=0.087	OPT f=4850 TT=12.311 T1=12.31	SAT (TO) f=4850 TT=252.074 T1=2.75	OPT f=4850 TT=6.452 T1=2.68	OPT f=4850 TT=8.829 T1=4.9	? (TO) TT=260.25
Mario/ Mario-t-hard-5_c18.xml	OPT f=4482 TT=3.751 T1=1.61	OPT f=4482 TT=14.58 T1=11.43	OPT f=4482 TT=26.172 T1=13.66	OPT f=4482 TT=33.361 T1=18.71	? (NS) TT=0.086	OPT f=4482 TT=7.213 T1=7.2	SAT (TO) f=4482 TT=252.06 T1=1.64	OPT f=4482 TT=3.788 T1=1.63	OPT f=4482 TT=3.751 T1=1.61	? (TO) TT=260.343
NurseRostering/ NurseRostering-00_c18.xml	OPT f=1202 TT=5.061 T1=2.65	OPT f=1202 TT=23.922 T1=9.37	OPT f=1202 TT=57.609 T1=7.23	OPT f=1202 TT=56.544 T1=17.94	OPT f=1202 TT=11.539 T1=3.59	OPT f=1202 TT=4.994 T1=0.65	SAT (TO) f=1202 TT=252.046 T1=3.97	OPT f=1202 TT=18.115 T1=5.06	OPT f=1202 TT=14.145 T1=9.09	OPT f=1202 TT=5.061 T1=2.65
NurseRostering/ NurseRostering-01_c18.xml	OPT f=607 TT=5.872 T1=2.68	OPT f=607 TT=134.405 T1=9.91	OPT f=607 TT=126.136 T1=55.32	OPT f=607 TT=229.383 T1=7.05	OPT f=607 TT=20.766 T1=11.21	OPT f=607 TT=7.796 T1=0.45	SAT (TO) f=607 TT=252.058 T1=33.82	OPT f=607 TT=69.114 T1=34.65	SAT (TO) f=607 TT=252.121 T1=164.79	OPT f=607 TT=5.872 T1=2.68
NurseRostering/ NurseRostering-02_c18.xml	SAT f=1062 TT=252.126 T1=205.7	SAT (TO) f=1527 TT=252.083 T1=245.46	SAT (TO) f=1551 TT=252.127 T1=177.3	SAT (TO) f=1638 TT=252.107 T1=219.82	SAT (TO) f=1752 TT=252.004 T1=37.87	SAT (TO) f=1036 TT=252.018 T1=191.96	SAT (TO) f=1749 TT=252.051 T1=239.75	SAT (TO) f=2055 TT=252.119 T1=104.49	SAT (TO) f=1225 TT=252.124 T1=56.28	SAT (TO) f=1062 TT=252.126 T1=205.7
NurseRostering/ NurseRostering-03_c18.xml	SAT f=1502 TT=252.065 T1=72.11	SAT (TO) f=1502 TT=252.065 T1=72.11	SAT (TO) f=2170 TT=252.099 T1=83.88	SAT (TO) f=2368 TT=252.119 T1=207.79	SAT (TO) f=4453 TT=251.995 T1=0.19	SAT (TO) f=1333 TT=251.999 T1=193.73	SAT (TO) f=3780 TT=252.042 T1=155.4	SAT (TO) f=3582 TT=252.128 T1=40.37	SAT (TO) f=1606 TT=252.129 T1=202.33	SAT (TO) f=1661 TT=252.129 T1=243.39
NurseRostering/ NurseRostering-04_c18.xml	SAT f=2331 TT=252.104 T1=178.76	SAT (TO) f=2428 TT=252.114 T1=200.93	SAT (TO) f=3552 TT=252.08 T1=43.74	SAT (TO) f=3169 TT=252.066 T1=211.74	SAT (TO) f=2979 TT=251.98 T1=191.1	SAT (TO) f=2248 TT=251.955 T1=242.74	SAT (TO) f=4305 TT=252.099 T1=204.44	SAT (TO) f=4001 TT=252.043 T1=226.84	SAT (TO) f=2331 TT=252.104 T1=178.76	SAT (TO) f=2583 TT=252.266 T1=244.54
NurseRostering/ NurseRostering-05_c18.xml	SAT f=2564 TT=252.13 T1=237.99	SAT (TO) f=3032 TT=252.115 T1=121.53	SAT (TO) f=4183 TT=252.067 T1=186.86	SAT (TO) f=4990 TT=252.042 T1=43.24	SAT (TO) f=7851 TT=251.976 T1=206.01	SAT (TO) f=3515 TT=251.95 T1=227.16	SAT (TO) f=5646 TT=252.063 T1=198.21	? (TO) TT=252.088	SAT (TO) f=2564 TT=252.13 T1=237.99	SAT (TO) f=4256 TT=252.148 T1=241.54
NurseRostering/ NurseRostering-06_c18.xml	SAT f=4456 TT=252.074 T1=219	SAT (TO) f=4456 TT=252.074 T1=219	? (TO) TT=252.129	? (TO) TT=252.103	SAT (TO) f=6937 TT=252.008 T1=5.91	SAT (TO) f=5221 TT=251.996 T1=243.99	? (TO) TT=252.048	? (TO) TT=252.119	? (TO) TT=252.081	SAT (TO) f=7863 TT=252.156 T1=188.03
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
NurseRostering/ NurseRostering-07_c18.xml	SAT f=5922 TT=252.039 T1=12.42	SAT (TO) f=5922 TT=252.039 T1=12.42	? (TO) TT=252.105	? (TO) TT=252.097	SAT (TO) f=9193 TT=251.995 T1=44.06	SAT (TO) f=10856 TT=251.952 T1=125.54	? (TO) TT=252.024	? (TO) TT=252.117	? (TO) TT=252.062	SAT (TO) f=8644 TT=252.138 T1=195.75
NurseRostering/ NurseRostering-08_c18.xml	SAT f=10030 TT=252.09 T1=141.53	SAT (TO) f=10030 TT=252.09 T1=141.53	? (TO) TT=252.089	? (TO) TT=252.119	SAT (TO) f=17697 TT=251.984 T1=130.71	? (TO) TT=251.988	? (TO) TT=252.046	? (TO) TT=252.067	? (TO) TT=252.121	? (TO) TT=252.153
NurseRostering/ NurseRostering-09_c18.xml	SAT f=2850 TT=252.092 T1=144.52	SAT (TO) f=2850 TT=252.092 T1=144.52	SAT (TO) f=6458 TT=252.158 T1=159.48	SAT (TO) f=6134 TT=252.095 T1=149.13	SAT (TO) f=10554 TT=251.989 T1=211.1	SAT (TO) f=8823 TT=251.928 T1=228.6	SAT (TO) f=11020 TT=252.075 T1=235.36	? (TO) TT=252.041	? (TO) TT=252.119	? (TO) TT=252.136
NurseRostering/ NurseRostering-10_c18.xml	SAT f=11310 TT=252.088 T1=127.67	SAT (TO) f=11310 TT=252.088 T1=127.67	? (TO) TT=252.097	? (TO) TT=252.147	SAT (TO) f=25329 TT=251.964 T1=5.07	SAT (TO) f=22647 TT=251.954 T1=205.46	? (TO) TT=252.047	? (TO) TT=252.058	? (TO) TT=252.064	? (TO) TT=252.178
NurseRostering/ NurseRostering-11_c18.xml		? (TO) TT=252.107	? (TO) TT=252.121	? (TO) TT=252.081	? (TO) TT=251.982	? (TO) TT=251.97	? (TO) TT=252.042	? (TO) TT=252.098	? (TO) TT=252.039	? (TO) TT=252.126
NurseRostering/ NurseRostering-12_c18.xml	SAT f=56621 TT=251.971 T1=141.71	? (TO) TT=252.19	? (TO) TT=252.142	? (TO) TT=252.102	SAT (TO) f=56621 TT=251.971 T1=141.71	? (TO) TT=251.985	? (TO) TT=252.113	? (TO) TT=252.112	? (TO) TT=252.074	? (TO) TT=253.473
NurseRostering/ NurseRostering-13_c18.xml		? (TO) TT=252.086	? (TO) TT=252.159	? (TO) TT=252.071	? (TO) TT=251.988	? (TO) TT=251.992	? (TO) TT=252.076	? (TO) TT=252.107	? (TO) TT=252.084	? (TO) TT=259.985
NurseRostering/ NurseRostering-14_c18.xml		? (TO) TT=252.06	? (TO) TT=252.108	? (TO) TT=252.077	? (TO) TT=251.98	? (TO) TT=251.963	? (TO) TT=252.053	? (TO) TT=252.049	? (TO) TT=252.074	? (TO) TT=252.167
NurseRostering/ NurseRostering-15_c18.xml	SAT f=43817 TT=251.967 T1=77.01	? (TO) TT=252.097	? (TO) TT=252.117	? (TO) TT=252.087	SAT (TO) f=43817 TT=251.967 T1=77.01	? (TO) TT=251.924	? (TO) TT=252.082	? (TO) TT=252.046	? (TO) TT=252.07	? (TO) TT=252.194
NurseRostering/ NurseRostering-16_c18.xml		? (TO) TT=252.036	? (TO) TT=252.073	? (TO) TT=252.127	? (TO) TT=251.983	? (TO) TT=251.973	? (TO) TT=252.061	? (TO) TT=252.098	? (TO) TT=252.077	? (TO) TT=252.172
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
NurseRostering/ NurseRostering-17_c18.xml		? (TO) TT=252.121	? (TO) TT=252.106	? (TO) TT=252.103	? (TO) TT=251.98	? (TO) TT=251.859	? (TO) TT=252.086	? (TO) TT=252.086	? (TO) TT=252.14	? (TO) TT=252.208
NurseRostering/ NurseRostering-18_c18.xml		? (TO) TT=252.121	? (TO) TT=252.094	? (TO) TT=252.13	? (TO) TT=251.95	? (TO) TT=251.989	? (TO) TT=252.126	? (TO) TT=252.034	? (TO) TT=252.044	? (TO) TT=252.116
NurseRostering/ NurseRostering-19_c18.xml		? (TO) TT=252.074	? (TO) TT=252.101	? (TO) TT=252.071	? (TO) TT=251.967	? (TO) TT=251.964	? (TO) TT=252.109	? (TO) TT=252.108	? (TO) TT=252.097	? (TO) TT=252.181
NurseRostering/ NurseRostering-20_c18.xml		? (TO) TT=252.12	? (TO) TT=252.155	? (TO) TT=252.09	? (TO) TT=252.158	? (TO) TT=251.99	? (TO) TT=252.16	? (TO) TT=252.159	? (TO) TT=252.1	? (TO) TT=259.411
PeacableArmies/ PeacableArmies-m1-05_c18.xml	OPT f=4 TT=1.54 T1=0.13	OPT f=4 TT=1.918 T1=0.65	OPT f=4 TT=14.175 T1=5.3	OPT f=4 TT=15.616 T1=5.61	OPT f=4 TT=1.54 T1=0.13	OPT f=4 TT=0.268 T1=0.11	SAT f=4 TT=251.673 T1=1.27	OPT f=4 TT=2.53 T1=1.25	OPT f=4 TT=2.525 T1=1.3	OPT f=4 TT=2.107 T1=0.72
PeacableArmies/ PeacableArmies-m1-07_c18.xml	OPT f=7 TT=33.381 T1=1.26	OPT f=7 TT=33.381 T1=1.26	SAT (TO) f=7 TT=252.132 T1=12.8	SAT (TO) f=7 TT=252.133 T1=8.46	SAT (TO) f=7 TT=252.003 T1=9.32	OPT f=7 TT=216.592 T1=2.34	SAT (TO) f=7 TT=252.087 T1=1.47	OPT f=7 TT=162.221 T1=1.48	SAT (TO) f=7 TT=252.032 T1=4.45	SAT (TO) f=7 TT=252.215 T1=4.98
PeacableArmies/ PeacableArmies-m1-10_c18.xml	SAT f=13 TT=252.014 T1=12.26	SAT (TO) f=13 TT=252.027 T1=200.82	SAT (TO) f=12 TT=252.219 T1=95.73	SAT (TO) f=12 TT=252.086 T1=23.64	SAT (TO) f=11 TT=251.999 T1=202.06	SAT (TO) f=14 TT=252.014 T1=166.28	SAT (TO) f=13 TT=252.061 T1=54.68	SAT f=13 TT=252.014 T1=12.26	SAT (TO) f=13 TT=252.033 T1=48.21	? (TO) TT=252.05
PeacableArmies/ PeacableArmies-m1-15_c18.xml	SAT f=27 TT=252.102 T1=15.39	SAT (TO) f=19 TT=252.029 T1=46.55	SAT (TO) f=22 TT=252.124 T1=46.37	SAT (TO) f=24 TT=252.102 T1=69.58	SAT (TO) f=17 TT=251.958 T1=124.5	SAT (TO) f=24 TT=251.957 T1=47.59	SAT (TO) f=25 TT=252.086 T1=2.9	SAT (TO) f=25 TT=252.074 T1=2.91	SAT (TO) f=27 TT=252.102 T1=15.39	? (TO) TT=252.072
PeacableArmies/ PeacableArmies-m1-20_c18.xml	SAT f=38 TT=252.089 T1=36.05	SAT (TO) f=25 TT=252.029 T1=9.23	SAT (TO) f=36 TT=254.335 T1=163.98	SAT (TO) f=32 TT=252.213 T1=176.34	SAT (TO) f=25 TT=251.944 T1=183.01	SAT (TO) f=39 TT=251.896 T1=149.62	SAT (TO) f=35 TT=252.133 T1=4.88	SAT (TO) f=35 TT=252.086 T1=6.04	SAT (TO) f=38 TT=252.089 T1=36.05	? (TO) TT=252.114
PeacableArmies/ PeacableArmies-m1-30_c18.xml	SAT f=73 TT=252.302 T1=71.09	SAT (TO) f=43 TT=252.119 T1=207.06	? (TO) TT=254.151	? (TO) TT=254.135	SAT (TO) f=44 TT=252.102 T1=219.33	SAT (TO) f=71 TT=251.928 T1=223.98	SAT (TO) f=69 TT=252.11 T1=15.83	SAT (TO) f=68 TT=252.067 T1=8.3	SAT (TO) f=73 TT=252.302 T1=71.09	? (TO) TT=252.275
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
PeacableArmies/ PeacableArmies-m1-50_c18.xml	SAT f=174 TT=252.072 T1=112.27	SAT (TO) f=63 TT=252.101 T1=237.86	? (TO) TT=254.765	? (TO) TT=254.205	? (MO) TT=101.646	? (TO) TT=252.069	SAT (TO) f=165 TT=252.081 T1=110.36	SAT (TO) f=174 TT=252.072 T1=112.27	SAT (TO) f=169 TT=252.088 T1=217.26	? TT=46.532
PeacableArmies/ PeacableArmies-m2-05_c18.xml	OPT f=4 TT=0.506 T1=0.09	OPT f=4 TT=1.752 T1=0.69	OPT f=4 TT=44.484 T1=6.36	OPT f=4 TT=36.119 T1=7.37	OPT f=4 TT=0.506 T1=0.09	OPT f=4 TT=0.448 T1=0.07	SAT (TO) f=4 TT=252.1 T1=1.35	OPT f=4 TT=15.846 T1=1.33	SAT (TO) f=4 TT=252.077 T1=4.3	OPT f=4 TT=1.797 T1=0.59
PeacableArmies/ PeacableArmies-m2-07_c18.xml	OPT f=7 TT=22.279 T1=1.3	OPT f=7 TT=22.279 T1=7.87	? (TO) TT=252.089	? (TO) TT=252.129	SAT (TO) f=7 TT=252.012 T1=30.16	SAT (TO) f=7 TT=252.009 T1=0.28	SAT f=4 TT=251.533 T1=118.18	SAT f=6 TT=251.321 T1=94.45	SAT (TO) f=7 TT=252.13 T1=3.29	OPT f=7 TT=30.648 T1=1.3
PeacableArmies/ PeacableArmies-m2-10_c18.xml	SAT f=14 TT=252.149 T1=68.44	SAT (TO) f=10 TT=252.024 T1=15.01	? (TO) TT=252.131	? (TO) TT=252.074	SAT (TO) f=9 TT=252.001 T1=58.85	SAT (TO) f=13 TT=251.993 T1=59.15	? (TO) TT=252.082	? (TO) TT=252.122	SAT (TO) f=12 TT=252.061 T1=5.22	SAT (TO) f=14 TT=252.149 T1=68.44
PeacableArmies/ PeacableArmies-m2-15_c18.xml	SAT f=26 TT=252.083 T1=208.74	SAT (TO) f=11 TT=252.045 T1=67.68	? (TO) TT=252.083	? (TO) TT=252.072	SAT (TO) f=10 TT=251.99 T1=121.49	SAT (TO) f=24 TT=251.979 T1=13.1	? (TO) TT=252.094	? (TO) TT=252.041	SAT (TO) f=24 TT=252.125 T1=25.39	SAT (TO) f=26 TT=252.083 T1=208.74
PeacableArmies/ PeacableArmies-m2-20_c18.xml	SAT f=34 TT=252.145 T1=117.33	SAT (TO) f=10 TT=252.065 T1=31.87	? (TO) TT=252.151	? (TO) TT=252.069	SAT (TO) f=10 TT=252.016 T1=27.05	SAT (TO) f=37 TT=251.927 T1=90.3	? (TO) TT=252.131	? (TO) TT=252.034	SAT (TO) f=22 TT=252.14 T1=72.88	SAT (TO) f=34 TT=252.145 T1=117.33
PeacableArmies/ PeacableArmies-m2-30_c18.xml	SAT f=54 TT=252.087 T1=81.52	SAT (TO) f=10 TT=252.059 T1=133.87	? (TO) TT=254.063	? (TO) TT=254.084	SAT (TO) f=11 TT=252.038 T1=157.57	SAT (TO) f=70 TT=251.932 T1=136.77	? (TO) TT=252.083	SAT (TO) f=54 TT=252.087 T1=81.52	SAT (TO) f=15 TT=252.087 T1=31.48	SAT (TO) f=46 TT=252.142 T1=158.16
PeacableArmies/ PeacableArmies-m2-50_c18.xml	SAT f=96 TT=252.083 T1=208.15	SAT (TO) f=6 TT=252.084 T1=108.76	? (TO) TT=254.133	? (TO) TT=254.609	SAT (TO) f=8 TT=252.265 T1=144.74	SAT (TO) f=9 TT=252.094 T1=249.04	? (TO) TT=252.053	SAT (TO) f=96 TT=252.083 T1=208.15	SAT (TO) f=28 TT=252.114 T1=122.83	? (TO) TT=252.21
PizzaVoucher/ PizzaVoucher-10a_c18.xml	OPT f=210 TT=4.096 T1=1.02	OPT f=210 TT=11.714 T1=1.02	OPT f=210 TT=12.729 T1=4.84	OPT f=210 TT=21.981 T1=6.22	OPT f=210 TT=187.534 T1=100.18	OPT f=210 TT=20.219 T1=5.07	SAT (TO) f=210 TT=252.054 T1=43.11	SAT (TO) f=210 TT=252.082 T1=16.81	? (TO) TT=252.094	OPT f=210 TT=4.096 T1=1.67
PizzaVoucher/ PizzaVoucher-10b_c18.xml	OPT f=28 TT=9.326 T1=1.32	OPT f=28 TT=243.298 T1=8.66	OPT f=28 TT=56.174 T1=6.01	OPT f=28 TT=113.029 T1=8.01	SAT (TO) f=28 TT=252.006 T1=208.8	OPT f=28 TT=19.731 T1=2.54	SAT (TO) f=28 TT=252.076 T1=55.84	SAT (TO) f=28 TT=252.042 T1=5.93	SAT (TO) f=28 TT=252.101 T1=239.57	OPT f=28 TT=9.326 T1=1.32
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
PizzaVoucher/ PizzaVoucher-12a_c18.xml	OPT f=40 TT=7.256 T1=3.88	SAT (TO) f=40 TT=252.026 T1=126.77	SAT (TO) f=40 TT=252.119 T1=147.66	OPT f=40 TT=60.663 T1=37.06	SAT (TO) f=41 TT=252.004 T1=9.51	SAT (TO) f=40 TT=251.989 T1=88.08	SAT (TO) f=49 TT=252.088 T1=1.49	SAT (TO) f=42 TT=252.088 T1=10.11	? (TO) TT=252.127	OPT f=40 TT=7.256 T1=3.88
PizzaVoucher/ PizzaVoucher-12b_c18.xml	OPT f=40 TT=9.815 T1=5.86	SAT (TO) f=42 TT=252.076 T1=70.75	OPT f=40 TT=86.794 T1=48.38	SAT (TO) f=40 TT=252.133 T1=155.38	SAT (TO) f=41 TT=252.005 T1=34.54	SAT (TO) f=40 TT=251.986 T1=5.02	SAT (TO) f=43 TT=252.105 T1=65.34	SAT (TO) f=43 TT=252.124 T1=141.52	? (TO) TT=252.109	OPT f=40 TT=9.815 T1=5.86
PizzaVoucher/ PizzaVoucher-15a_c18.xml	SAT f=42 TT=252.115 T1=9.13	SAT (TO) f=44 TT=252.074 T1=138.98	SAT (TO) f=42 TT=252.115 T1=9.13	SAT (TO) f=43 TT=252.142 T1=111.71	SAT (TO) f=59 TT=252.041 T1=93.26	SAT (TO) f=42 TT=251.96 T1=9.89	SAT (TO) f=55 TT=252.109 T1=5.38	SAT (TO) f=55 TT=252.039 T1=5.71	? (TO) TT=252.068	SAT (TO) f=42 TT=252.384 T1=12.3
PizzaVoucher/ PizzaVoucher-15b_c18.xml	SAT f=42 TT=252.063 T1=10.44	SAT (TO) f=45 TT=252.09 T1=116.56	SAT (TO) f=47 TT=252.06 T1=84.69	SAT (TO) f=44 TT=252.077 T1=45.86	SAT (TO) f=49 TT=252.014 T1=38.58	SAT (TO) f=42 TT=252.014 T1=6.38	SAT (TO) f=64 TT=252.028 T1=19.84	SAT (TO) f=54 TT=252.061 T1=72.05	? (TO) TT=252.044	SAT (TO) f=42 TT=252.063 T1=10.44
PizzaVoucher/ PizzaVoucher-18a_c18.xml	SAT f=49 TT=252.503 T1=77.96	SAT (TO) f=57 TT=252.025 T1=160.08	SAT (TO) f=60 TT=252.085 T1=22.76	SAT (TO) f=55 TT=252.086 T1=62.92	SAT (TO) f=62 TT=252.038 T1=219.28	SAT (TO) f=49 TT=252.022 T1=47.95	? (TO) TT=252.04	? (TO) TT=252.073	? (TO) TT=252.181	SAT (TO) f=49 TT=252.503 T1=77.96
PizzaVoucher/ PizzaVoucher-18b_c18.xml	SAT f=49 TT=252.098 T1=42.84	SAT (TO) f=80 TT=252.057 T1=120.77	SAT (TO) f=51 TT=252.069 T1=5.88	SAT (TO) f=60 TT=252.052 T1=178.27	? (TO) TT=252.008	SAT (TO) f=50 TT=252.003 T1=29.73	SAT (TO) f=67 TT=252.043 T1=103.62	SAT (TO) f=62 TT=252.068 T1=156.21	? (TO) TT=252.089	SAT (TO) f=49 TT=252.098 T1=42.84
PizzaVoucher/ PizzaVoucher-22a_c18.xml	SAT f=61 TT=252.067 T1=69.63	? (TO) TT=252.029	SAT (TO) f=63 TT=252.055 T1=12.61	SAT (TO) f=74 TT=252.043 T1=62.98	? (TO) TT=251.985	SAT (TO) f=62 TT=251.981 T1=97.16	? (TO) TT=252.105	? (TO) TT=252.136	? (TO) TT=252.119	SAT (TO) f=61 TT=252.067 T1=69.63
PizzaVoucher/ PizzaVoucher-22b_c18.xml	SAT f=55 TT=252.107 T1=211	? (TO) TT=252.077	SAT (TO) f=80 TT=252.054 T1=74	SAT (TO) f=73 TT=252.118 T1=181.9	? (TO) TT=251.982	SAT (TO) f=60 TT=252.039 T1=46.07	? (TO) TT=252.073	? (TO) TT=252.104	? (TO) TT=252.092	SAT (TO) f=55 TT=252.107 T1=211
PseudoBoolean-opt/ Pb-circ10-3_c18.xml	SAT f=364 TT=252.031 T1=92.62	SAT (TO) f=364 TT=252.031 T1=92.62	? (TO) TT=254.081	? (TO) TT=254.125	SAT (TO) f=374 TT=251.979 T1=154.69	SAT (TO) f=354 TT=251.96 T1=129.79	? (TO) TT=252.041	? (TO) TT=252.12	? (TO) TT=252.097	? (TO) TT=252.155
PseudoBoolean-opt/ Pb-circ4-3_c18.xml	OPT f=20 TT=4.281 T1=0.09	OPT f=20 TT=6.024 T1=0.65	OPT f=20 TT=22.057 T1=5.69	OPT f=20 TT=20.24 T1=5.78	OPT f=20 TT=13.05 T1=0.09	OPT f=20 TT=3.228 T1=0.11	SAT (TO) f=20 TT=252.028 T1=1.73	OPT f=20 TT=4.383 T1=1.7	OPT f=20 TT=4.281 T1=1.73	OPT f=20 TT=4.685 T1=1.17
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
PseudoBoolean-opt/ Pb-circ6-3_c18.xml	SAT f=70 TT=251.98 T1=20.41	SAT (TO) f=70 TT=252.034 T1=20.41	SAT (TO) f=70 TT=252.126 T1=72.75	SAT (TO) f=76 TT=252.151 T1=82.47	SAT (TO) f=70 TT=251.98 T1=55.19	SAT (TO) f=66 TT=251.981 T1=46.45	SAT (TO) f=70 TT=252.082 T1=168.93	SAT (TO) f=72 TT=252.086 T1=169.6	SAT (TO) f=70 TT=252.1 T1=111.86	SAT (TO) f=70 TT=252.113 T1=127.27
PseudoBoolean-opt/ Pb-circ8-3_c18.xml	SAT f=178 TT=252.036 T1=36.01	SAT (TO) f=178 TT=252.036 T1=36.01	? (TO) TT=252.106	? (TO) TT=252.074	SAT (TO) f=180 TT=251.974 T1=229.53	SAT (TO) f=172 TT=252.003 T1=51.1	? (TO) TT=252.102	? (TO) TT=252.095	? (TO) TT=252.093	SAT (TO) f=186 TT=252.138 T1=165.36
PseudoBoolean-opt/ Pb-garden-100x100_c18.xml	SAT f=2675 TT=252.071 T1=20.65	SAT (TO) f=2864 TT=252.083 T1=11.55	? (TO) TT=260.177	? (TO) TT=260.216	SAT (TO) f=2675 TT=252.071 T1=20.65	SAT (TO) f=2757 TT=251.966 T1=199.73	SAT (TO) f=5268 TT=252.132 T1=160.48	SAT (TO) f=5269 TT=252.063 T1=213.96	SAT (TO) f=6231 TT=252.186 T1=165.9	SAT (TO) f=3475 TT=252.234 T1=108.86
PseudoBoolean-opt/ Pb-garden-15x15_c18.xml	SAT f=57 TT=252.027 T1=96.6	SAT (TO) f=63 TT=252.081 T1=218.28	SAT (TO) f=63 TT=252.071 T1=158.15	SAT (TO) f=67 TT=252.132 T1=137.35	SAT (TO) f=66 TT=252.013 T1=0.02	SAT (TO) f=62 TT=252.008 T1=20.88	SAT f=90 TT=249.504 T1=166.1	SAT f=91 TT=249.689 T1=231.59	SAT (TO) f=57 TT=252.027 T1=96.6	SAT (TO) f=78 TT=252.344 T1=9.89
PseudoBoolean-opt/ Pb-garden-4x4_c18.xml	OPT f=4 TT=0.006 T1=0	OPT f=4 TT=0.878 T1=0.45	OPT f=4 TT=7.105 T1=3.54	OPT f=4 TT=7.089 T1=3.55	OPT f=4 TT=0.006 T1=0	OPT f=4 TT=0.006 T1=0	SAT f=4 TT=251.16 T1=1	OPT f=4 TT=1.532 T1=1.01	OPT f=4 TT=1.56 T1=1.02	OPT f=4 TT=0.615 T1=0.35
PseudoBoolean-opt/ Pb-garden-7x7_c18.xml	OPT f=12 TT=28.356 T1=8.43	OPT f=12 TT=47.533 T1=12.53	SAT (TO) f=13 TT=252.114 T1=4.57	OPT f=12 TT=28.356 T1=10.51	SAT (TO) f=12 TT=251.983 T1=237.49	OPT f=12 TT=43.729 T1=0.14	SAT f=12 TT=251.109 T1=43.45	OPT f=12 TT=48.896 T1=8.43	SAT (TO) f=13 TT=252.102 T1=4.23	OPT f=12 TT=41.682 T1=23.5
PseudoBoolean-opt/ Pb-garden-9x9_c18.xml	SAT f=21 TT=252.076 T1=8.44	SAT (TO) f=22 TT=252.114 T1=3	SAT (TO) f=21 TT=252.076 T1=21.03	SAT (TO) f=21 TT=252.09 T1=8.44	SAT (TO) f=22 TT=252.008 T1=0.01	SAT (TO) f=21 TT=252.001 T1=0.11	SAT f=25 TT=249.264 T1=22.08	SAT f=24 TT=249.343 T1=105.76	SAT (TO) f=21 TT=252.081 T1=28.26	SAT (TO) f=23 TT=252.283 T1=0.46
PseudoBoolean-opt/ Pb-radar-10-10-45-050-100_c18.xml	SAT f=4 TT=252.056 T1=2.95	SAT (TO) f=4 TT=252.056 T1=2.95	SAT (TO) f=14 TT=252.073 T1=12.72	SAT (TO) f=16 TT=252.05 T1=58.8	SAT (TO) f=11 TT=252.007 T1=1.62	SAT (TO) f=11 TT=252.005 T1=218.77	SAT f=41 TT=249.955 T1=17	SAT f=38 TT=249.787 T1=32.72	SAT (TO) f=6 TT=252.04 T1=11.23	SAT (TO) f=13 TT=252.293 T1=0.6
PseudoBoolean-opt/ Pb-radar-10-20-45-050-100_c18.xml	SAT f=5 TT=252.053 T1=75.92	SAT (TO) f=13 TT=252.049 T1=11.47	SAT (TO) f=49 TT=252.082 T1=8.63	SAT (TO) f=37 TT=252.111 T1=8.79	SAT (TO) f=22 TT=251.98 T1=2.85	SAT (TO) f=18 TT=251.979 T1=0.1	SAT f=93 TT=250.643 T1=243.28	SAT f=105 TT=250.483 T1=22.96	SAT (TO) f=5 TT=252.053 T1=75.92	? (TO) TT=252.434
PseudoBoolean-opt/ Pb-radar-30-30-45-050-100_c18.xml	SAT f=50 TT=252.114 T1=4.32	SAT (TO) f=130 TT=252.07 T1=104.48	SAT (TO) f=179 TT=252.111 T1=178.27	? (TO) TT=254.56	SAT (TO) f=169 TT=252.007 T1=188.01	SAT (TO) f=171 TT=252.008 T1=98.22	SAT (TO) f=1317 TT=252.079 T1=244.22	SAT (TO) f=1303 TT=252.051 T1=235.48	SAT (TO) f=158 TT=252.096 T1=217.31	SAT (TO) f=50 TT=252.114 T1=4.32
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
PseudoBoolean-opt/ Pb-radar-30-70-45-050-100_c18.xml	SAT f=158 TT=252.13 T1=17.87	SAT (TO) f=586 TT=252.088 T1=157.22	? (TO) TT=260.356	? (TO) TT=261.635	SAT (TO) f=575 TT=252.012 T1=15.85	SAT (TO) f=445 TT=251.99 T1=57.57	SAT (TO) f=3342 TT=252.156 T1=227.3	SAT (TO) f=3353 TT=252.146 T1=242.2	SAT (TO) f=17006 TT=252.133 T1=32.77	SAT (TO) f=158 TT=252.13 T1=17.87
QuadraticAssignment/ QuadraticAssignment-esc128_c18.xml	SAT f=32 TT=252.042 T1=144.47	SAT (TO) f=32 TT=252.042 T1=144.47	SAT (TO) f=38 TT=252.058 T1=28.23	SAT (TO) f=48 TT=252.152 T1=47.86	SAT (TO) f=44 TT=252.003 T1=1.05	SAT (TO) f=32 TT=251.98 T1=108.96	SAT (TO) f=36 TT=252.056 T1=196.16	SAT (TO) f=36 TT=252.112 T1=225.63	SAT (TO) f=36 TT=252.099 T1=22.15	? (TO) TT=253.973
QuadraticAssignment/ QuadraticAssignment-esc16a_c18.xml	SAT f=34 TT=250.591 T1=1.42	SAT (TO) f=34 TT=252.05 T1=1.72	SAT (TO) f=38 TT=252.12 T1=5.72	SAT (TO) f=34 TT=252.079 T1=10.87	SAT (TO) f=34 TT=252.008 T1=1.42	SAT (TO) f=34 TT=252 T1=0.02	SAT f=34 TT=251.12 T1=4.85	SAT f=34 TT=250.591 T1=9.71	SAT (TO) f=34 TT=252.037 T1=3.15	SAT (TO) f=38 TT=252.135 T1=163.4
QuadraticAssignment/ QuadraticAssignment-esc16d_c18.xml	OPT f=8 TT=52.907 T1=0.03	OPT f=8 TT=56.891 T1=1.04	OPT f=8 TT=96.427 T1=3.7	OPT f=8 TT=52.907 T1=4.31	OPT f=8 TT=138.792 T1=0.03	OPT f=8 TT=53.039 T1=0.01	SAT f=8 TT=251.763 T1=1.99	OPT f=8 TT=207.433 T1=2	SAT (TO) f=8 TT=252.029 T1=2.46	OPT f=8 TT=157.52 T1=10.79
QuadraticAssignment/ QuadraticAssignment-esc32a_c18.xml	SAT f=69 TT=252.125 T1=86.5	SAT (TO) f=69 TT=252.125 T1=86.5	SAT (TO) f=113 TT=252.087 T1=102.09	SAT (TO) f=86 TT=252.097 T1=137.19	SAT (TO) f=96 TT=252.033 T1=35.77	SAT (TO) f=67 TT=252.021 T1=6.53	SAT f=87 TT=250.781 T1=49.83	SAT f=87 TT=250.674 T1=43.84	SAT (TO) f=71 TT=252.107 T1=113.56	SAT (TO) f=196 TT=252.221 T1=144.69
QuadraticAssignment/ QuadraticAssignment-esc32d_c18.xml	SAT f=104 TT=252.075 T1=8.09	SAT (TO) f=104 TT=252.075 T1=8.09	SAT (TO) f=105 TT=252.129 T1=11.63	SAT (TO) f=108 TT=252.084 T1=5.57	SAT (TO) f=110 TT=251.98 T1=24.77	SAT (TO) f=101 TT=252.005 T1=61.79	SAT f=115 TT=251.274 T1=22.88	SAT f=117 TT=251.358 T1=8.52	SAT (TO) f=105 TT=252.044 T1=19.37	SAT (TO) f=151 TT=252.153 T1=58.97
QuadraticAssignment/ QuadraticAssignment-esc64a_c18.xml	SAT f=58 TT=251.32 T1=0.67	SAT (TO) f=58 TT=252.056 T1=19.71	SAT (TO) f=58 TT=252.088 T1=75.92	SAT (TO) f=60 TT=252.105 T1=105.99	SAT (TO) f=58 TT=252.037 T1=0.67	SAT (TO) f=58 TT=251.994 T1=0.64	SAT f=59 TT=251.704 T1=32.64	SAT f=58 TT=251.32 T1=10.75	SAT (TO) f=58 TT=252.113 T1=16.91	? (TO) TT=252.089
QuadraticAssignment/ QuadraticAssignment-had12_c18.xml	SAT f=830 TT=252.036 T1=6.15	SAT (TO) f=845 TT=252.083 T1=8.81	SAT (TO) f=830 TT=252.036 T1=6.15	SAT (TO) f=830 TT=252.09 T1=101.28	SAT (TO) f=847 TT=252.004 T1=199.32	SAT (TO) f=826 TT=251.967 T1=3.8	SAT f=833 TT=250.804 T1=85.52	SAT f=835 TT=250.759 T1=125.56	SAT (TO) f=838 TT=252.097 T1=4.74	SAT (TO) f=882 TT=252.117 T1=47.18
QuadraticAssignment/ QuadraticAssignment-had14_c18.xml	SAT f=1372 TT=252.091 T1=21.64	SAT (TO) f=1409 TT=252.03 T1=129.43	SAT (TO) f=1395 TT=252.081 T1=23.01	SAT (TO) f=1386 TT=252.102 T1=119.81	SAT (TO) f=1385 TT=252.041 T1=204.03	SAT (TO) f=1362 TT=251.997 T1=22.17	SAT f=1393 TT=250.85 T1=224.16	SAT f=1378 TT=251.071 T1=54.47	SAT (TO) f=1372 TT=252.091 T1=21.64	SAT (TO) f=1477 TT=252.08 T1=34.43
QuadraticAssignment/ QuadraticAssignment-had16_c18.xml	SAT f=1860 TT=252.047 T1=7.91	SAT (TO) f=1960 TT=252.045 T1=187.41	SAT (TO) f=1887 TT=252.111 T1=188.19	SAT (TO) f=1883 TT=252.089 T1=137.84	SAT (TO) f=1938 TT=252.041 T1=86.45	SAT (TO) f=1861 TT=252.002 T1=87.41	SAT f=1913 TT=251.287 T1=244.57	SAT f=1931 TT=251.373 T1=160.81	SAT (TO) f=1860 TT=252.047 T1=7.91	SAT (TO) f=2035 TT=252.162 T1=80.83
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
QuadraticAssignment/ QuadraticAssignment-had18_c18.xml	SAT f=2679 TT=252.092 T1=160.71	SAT (TO) f=2774 TT=252.043 T1=147.6	SAT (TO) f=2761 TT=252.096 T1=35.18	SAT (TO) f=2763 TT=252.09 T1=55.47	SAT (TO) f=2787 TT=251.991 T1=145.92	SAT (TO) f=2683 TT=251.999 T1=230.37	SAT f=2759 TT=251.53 T1=11.01	SAT f=2733 TT=251.466 T1=50.35	SAT (TO) f=2679 TT=252.092 T1=160.71	SAT (TO) f=2915 TT=252.178 T1=45.3
QuadraticAssignment/ QuadraticAssignment-had20_c18.xml	SAT f=3476 TT=252.099 T1=115.29	SAT (TO) f=3628 TT=252.111 T1=82.51	SAT (TO) f=3527 TT=252.115 T1=56.99	SAT (TO) f=3685 TT=252.138 T1=233.87	SAT (TO) f=3701 TT=252.039 T1=118.15	SAT (TO) f=3535 TT=252.014 T1=216.52	SAT f=3655 TT=251.608 T1=220.98	SAT f=3660 TT=251.45 T1=220.46	SAT (TO) f=3476 TT=252.099 T1=115.29	SAT (TO) f=3811 TT=255.393 T1=20.19
QuadraticAssignment/ QuadraticAssignment-qap_c18.xml	OPT f=4776 TT=0.378 T1=0.36	OPT f=4776 TT=2.344 T1=0.71	OPT f=4776 TT=9.211 T1=3.77	OPT f=4776 TT=9.966 T1=4.21	OPT f=4776 TT=0.378 T1=0.36	OPT f=4776 TT=0.068 T1=0.03	SAT f=4776 TT=251.856 T1=1.45	OPT f=4776 TT=3.043 T1=1.48	OPT f=4776 TT=3.355 T1=1.4	SAT (TO) f=5201 TT=252.072 T1=245.14
QuadraticAssignment/ QuadraticAssignment-sko42_c18.xml	SAT f=8337 TT=252.045 T1=241.27	SAT (TO) f=9220 TT=252.074 T1=173.73	SAT (TO) f=9372 TT=252.058 T1=45.57	SAT (TO) f=9412 TT=252.089 T1=23.73	SAT (TO) f=9538 TT=251.999 T1=247.12	SAT (TO) f=8693 TT=251.982 T1=106.7	SAT (TO) f=9353 TT=252.115 T1=5.34	SAT (TO) f=9490 TT=252.05 T1=27.32	SAT (TO) f=8337 TT=252.045 T1=241.27	? (TO) TT=260.267
QuadraticAssignment/ QuadraticAssignment-sko49_c18.xml	SAT f=12296 TT=252.074 T1=229.86	SAT (TO) f=13374 TT=252.077 T1=120.93	SAT (TO) f=13541 TT=252.074 T1=123.58	SAT (TO) f=13528 TT=252.152 T1=60.26	SAT (TO) f=14055 TT=252.026 T1=234	SAT (TO) f=12921 TT=252.042 T1=223.35	SAT (TO) f=13882 TT=252.09 T1=13.36	SAT (TO) f=13657 TT=252.064 T1=27.85	SAT (TO) f=12296 TT=252.074 T1=229.86	? (TO) TT=259.98
QuadraticAssignment/ QuadraticAssignment-sko56_c18.xml	SAT f=18368 TT=252.114 T1=239.62	SAT (TO) f=19417 TT=252.107 T1=240.06	SAT (TO) f=19752 TT=252.112 T1=115.54	SAT (TO) f=19730 TT=252.158 T1=115.72	SAT (TO) f=20530 TT=251.978 T1=120.27	SAT (TO) f=19133 TT=251.965 T1=206.14	SAT (TO) f=20300 TT=252.054 T1=60.34	SAT (TO) f=20263 TT=252.086 T1=70.4	SAT (TO) f=18368 TT=252.114 T1=239.62	? (TO) TT=259.938
QuadraticAssignment/ QuadraticAssignment-sko64_c18.xml	SAT f=25919 TT=252.113 T1=162.92	? (TO) TT=252.073	? (TO) TT=252.082	? (TO) TT=252.15	SAT (TO) f=28926 TT=251.983 T1=76.79	SAT (TO) f=27265 TT=252.05 T1=242.06	SAT (TO) f=28222 TT=252.094 T1=228.61	SAT (TO) f=28267 TT=252.098 T1=15.33	SAT (TO) f=25919 TT=252.113 T1=162.92	? (TO) TT=260.256
QuadraticAssignment/ QuadraticAssignment-sko72_c18.xml	SAT f=36340 TT=252.14 T1=226.37	? (TO) TT=252.086	? (TO) TT=252.144	? (TO) TT=252.157	SAT (TO) f=39085 TT=251.985 T1=157.74	SAT (TO) f=37450 TT=252.059 T1=234.56	SAT (TO) f=38323 TT=252.127 T1=97.95	SAT (TO) f=38595 TT=252.075 T1=215.94	SAT (TO) f=36340 TT=252.14 T1=226.37	? (TO) TT=252.488
QuadraticAssignment/ QuadraticAssignment-sko81_c18.xml	SAT f=50747 TT=252.089 T1=225.83	? (TO) TT=252.111	? (TO) TT=252.474	? (TO) TT=253.149	SAT (TO) f=53545 TT=251.985 T1=138.95	SAT (TO) f=51501 TT=252.073 T1=251.09	SAT (TO) f=52860 TT=252.068 T1=82.09	SAT (TO) f=52637 TT=252.147 T1=87.56	SAT (TO) f=50747 TT=252.089 T1=225.83	? (TO) TT=253.746
QuadraticAssignment/ QuadraticAssignment-sko90_c18.xml	SAT f=64864 TT=252.148 T1=215.4	? (TO) TT=252.125	? (TO) TT=252.481	? (TO) TT=252.214	SAT (TO) f=67862 TT=251.991 T1=142.62	SAT (TO) f=64260 TT=252.216 T1=217.89	SAT (TO) f=67345 TT=252.104 T1=156.77	SAT (TO) f=67481 TT=252.162 T1=50.58	SAT (TO) f=64864 TT=252.148 T1=215.4	? (TO) TT=259.977
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
Rcpsp/ Rcpsp-j120-01-01_c18.xml	SAT f=105 TT=250.865 T1=161.02	SAT (TO) f=115 TT=252.051 T1=144.61	SAT (TO) f=113 TT=252.151 T1=78.68	SAT (TO) f=113 TT=252.112 T1=148.88	? (NS) TT=0.077	SAT (TO) f=109 TT=252.009 T1=30.15	SAT f=107 TT=251.752 T1=95.43	SAT f=105 TT=250.865 T1=161.02	SAT (TO) f=107 TT=252.062 T1=68.5	? (TO) TT=259.854
Rcpsp/ Rcpsp-j120-01-02_c18.xml	OPT f=109 TT=153.124 T1=99.33	SAT (TO) f=114 TT=252.067 T1=82.47	SAT (TO) f=111 TT=252.129 T1=183.34	SAT (TO) f=113 TT=252.051 T1=90.18	? (NS) TT=0.077	SAT (TO) f=111 TT=251.973 T1=74.95	SAT (TO) f=109 TT=252.06 T1=122.04	OPT f=109 TT=153.124 T1=99.33	SAT (TO) f=111 TT=252.079 T1=30.73	? (TO) TT=259.797
Rcpsp/ Rcpsp-j120-01-03_c18.xml	SAT f=126 TT=251.994 T1=194.74	SAT (TO) f=130 TT=252.054 T1=125.71	SAT (TO) f=129 TT=252.072 T1=115.68	SAT (TO) f=129 TT=252.076 T1=95.09	? (NS) TT=0.078	SAT (TO) f=126 TT=251.961 T1=176.95	SAT (TO) f=127 TT=252.073 T1=5.42	SAT f=126 TT=251.994 T1=194.74	SAT (TO) f=128 TT=252.032 T1=3.34	? (TO) TT=259.834
Rcpsp/ Rcpsp-j120-01-04_c18.xml	OPT f=97 TT=8.625 T1=3.06	SAT (TO) f=98 TT=252.124 T1=96.77	SAT (TO) f=97 TT=252.069 T1=8.13	SAT (TO) f=97 TT=252.136 T1=130.98	? (NS) TT=0.077	OPT f=97 TT=29.184 T1=13.78	SAT (TO) f=97 TT=252.048 T1=3.06	OPT f=97 TT=8.625 T1=4.05	SAT (TO) f=97 TT=252.07 T1=9.75	? (TO) TT=260.191
Rcpsp/ Rcpsp-j30-01-01_c18.xml	OPT f=43 TT=1.293 T1=0.6	OPT f=43 TT=1.293 T1=0.6	OPT f=43 TT=7.689 T1=3.71	OPT f=43 TT=8.075 T1=3.77	? (NS) TT=0.023	OPT f=43 TT=0.08 T1=0.07	SAT f=43 TT=251.79 T1=1.24	OPT f=43 TT=2.136 T1=1.23	OPT f=43 TT=2.291 T1=1.28	OPT f=43 TT=50.368 T1=41.41
Rcpsp/ Rcpsp-j30-01-02_c18.xml	OPT f=47 TT=1.193 T1=0.58	OPT f=47 TT=1.193 T1=0.58	OPT f=47 TT=7.627 T1=3.64	OPT f=47 TT=7.555 T1=3.63	? (NS) TT=0.023	OPT f=47 TT=0.08 T1=0.07	SAT f=47 TT=251.949 T1=1.21	OPT f=47 TT=2.042 T1=1.21	OPT f=47 TT=2.356 T1=1.3	OPT f=47 TT=66.921 T1=55.37
Rcpsp/ Rcpsp-j30-01-03_c18.xml	OPT f=47 TT=1.338 T1=0.62	OPT f=47 TT=1.338 T1=0.62	OPT f=47 TT=7.99 T1=3.81	OPT f=47 TT=8.362 T1=4.23	? (NS) TT=0.023	OPT f=47 TT=0.067 T1=0.06	SAT f=47 TT=251.855 T1=1.22	OPT f=47 TT=2.063 T1=1.21	OPT f=47 TT=2.194 T1=1.23	OPT f=47 TT=44.269 T1=35.25
Rcpsp/ Rcpsp-j30-01-04_c18.xml	OPT f=62 TT=2.036 T1=1.1	OPT f=62 TT=2.036 T1=1.1	OPT f=62 TT=7.737 T1=3.81	OPT f=62 TT=7.883 T1=3.7	? (NS) TT=0.023	OPT f=62 TT=0.093 T1=0.08	SAT f=62 TT=251.763 T1=1.46	OPT f=62 TT=2.211 T1=1.25	OPT f=62 TT=2.254 T1=1.29	OPT f=62 TT=205.407 T1=187.11
Rcpsp/ Rcpsp-j60-01-01_c18.xml	OPT f=77 TT=1.729 T1=0.72	OPT f=77 TT=1.729 T1=0.72	OPT f=77 TT=8.629 T1=4.04	OPT f=77 TT=9.372 T1=4.26	? (NS) TT=0.041	OPT f=77 TT=0.666 T1=0.64	SAT (TO) f=77 TT=252.095 T1=1.51	OPT f=77 TT=3.042 T1=1.51	OPT f=77 TT=3.837 T1=1.73	? (TO) TT=257.141
Rcpsp/ Rcpsp-j60-01-02_c18.xml	OPT f=68 TT=3.337 T1=1.39	OPT f=68 TT=3.828 T1=1.39	OPT f=68 TT=9.697 T1=4.5	OPT f=68 TT=9.413 T1=4.35	? (NS) TT=0.041	OPT f=68 TT=0.769 T1=0.67	SAT (TO) f=68 TT=252.096 T1=1.94	OPT f=68 TT=3.337 T1=1.62	OPT f=68 TT=3.824 T1=1.74	SAT (TO) f=343 TT=252.153 T1=113.07
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
Rcpsp/ Rcpsp-j60-01-03_c18.xml	OPT f=68 TT=1.764 T1=0.73	OPT f=68 TT=1.764 T1=0.73	OPT f=68 TT=8.907 T1=4.08	OPT f=68 TT=9.501 T1=4.2	? (NS) TT=0.04	OPT f=68 TT=0.547 T1=0.52	SAT (TO) f=68 TT=252.103 T1=1.48	OPT f=68 TT=2.91 T1=1.48	OPT f=68 TT=3.498 T1=2.02	SAT (TO) f=309 TT=252.165 T1=168.33
Rcpsp/ Rcpsp-j60-01-04_c18.xml	OPT f=91 TT=2.848 T1=1.33	OPT f=91 TT=3.784 T1=1.33	OPT f=91 TT=9.995 T1=4.37	OPT f=91 TT=9.032 T1=4.21	? (NS) TT=0.041	OPT f=91 TT=0.644 T1=0.55	SAT (TO) f=91 TT=252.035 T1=1.47	OPT f=91 TT=2.848 T1=1.47	OPT f=91 TT=3.441 T1=1.58	SAT (TO) f=322 TT=252.172 T1=115.3
Rcpsp/ Rcpsp-j90-01-01_c18.xml	OPT f=73 TT=6.11 T1=2.5	OPT f=73 TT=14.521 T1=7.52	OPT f=73 TT=17.066 T1=8.27	OPT f=73 TT=13.563 T1=5.59	? (NS) TT=0.062	OPT f=73 TT=5.42 T1=5.14	SAT (TO) f=73 TT=252.109 T1=2.5	OPT f=73 TT=7.504 T1=3.68	OPT f=73 TT=6.11 T1=2.59	? (TO) TT=259.906
Rcpsp/ Rcpsp-j90-01-02_c18.xml	OPT f=92 TT=3.026 T1=1.13	OPT f=92 TT=3.026 T1=1.13	OPT f=92 TT=11.003 T1=4.93	OPT f=92 TT=10.955 T1=5.21	? (NS) TT=0.059	OPT f=92 TT=2.24 T1=2.12	SAT (TO) f=92 TT=252.107 T1=1.88	OPT f=92 TT=4.167 T1=1.85	OPT f=92 TT=5.089 T1=2.12	? (TO) TT=259.949
Rcpsp/ Rcpsp-j90-01-03_c18.xml	OPT f=66 TT=5.363 T1=2.15	OPT f=66 TT=9.731 T1=3.4	OPT f=66 TT=14.438 T1=5.8	OPT f=66 TT=13.03 T1=5.11	? (NS) TT=0.059	OPT f=66 TT=4.495 T1=4.02	SAT (TO) f=66 TT=252.051 T1=2.25	OPT f=66 TT=5.384 T1=2.24	OPT f=66 TT=5.363 T1=2.15	? (TO) TT=260.15
Rcpsp/ Rcpsp-j90-01-04_c18.xml	OPT f=86 TT=6.482 T1=1	SAT (TO) f=86 TT=252.036 T1=1	OPT f=86 TT=52.159 T1=5.44	OPT f=86 TT=95.943 T1=5.01	? (NS) TT=0.059	OPT f=86 TT=16.681 T1=3.05	SAT (TO) f=86 TT=252.087 T1=2.52	OPT f=86 TT=7.81 T1=2.52	OPT f=86 TT=6.482 T1=2.35	? (TO) TT=260.014
Rlfap/Rlfap-opt/ Rlfap-graph-01-opt_c18.xml	SAT f=18 TT=252.141 T1=104.57	SAT (TO) f=20 TT=252.068 T1=104.66	SAT (TO) f=18 TT=252.141 T1=104.57	SAT (TO) f=20 TT=252.137 T1=45.71	SAT (TO) f=20 TT=251.997 T1=217.48	SAT (TO) f=30 TT=251.995 T1=252	SAT f=48 TT=250.731 T1=13.88	SAT f=48 TT=251.405 T1=13.87	SAT (TO) f=44 TT=252.046 T1=4.97	SAT (TO) f=44 TT=252.204 T1=33.82
Rlfap/Rlfap-opt/ Rlfap-graph-02-opt_c18.xml	SAT f=14 TT=251.998 T1=28.66	SAT (TO) f=18 TT=252.063 T1=191.42	SAT (TO) f=16 TT=252.146 T1=186.93	SAT (TO) f=16 TT=252.08 T1=184.3	SAT (TO) f=14 TT=251.998 T1=28.66	SAT (TO) f=32 TT=251.975 T1=252	SAT f=48 TT=251.657 T1=239.49	SAT f=48 TT=252.037 T1=82.62	SAT (TO) f=44 TT=252.082 T1=18.27	? (TO) TT=259.996
Rlfap/Rlfap-opt/ Rlfap-graph-03-opt_c18.xml	OPT f=380 TT=0.929 T1=0.92	OPT f=380 TT=5.898 T1=2.56	OPT f=380 TT=24.331 T1=10.77	OPT f=380 TT=26.524 T1=12.54	OPT f=380 TT=0.929 T1=0.92	OPT f=380 TT=0.97 T1=0.95	SAT (TO) f=792 TT=252.098 T1=181.48	OPT f=380 TT=77.907 T1=72.91	SAT (TO) f=778 TT=252.123 T1=47.72	SAT (TO) f=778 TT=252.165 T1=12.17
Rlfap/Rlfap-opt/ Rlfap-graph-04-opt_c18.xml	OPT f=394 TT=3.825 T1=2.65	OPT f=394 TT=7.962 T1=2.65	OPT f=394 TT=31.778 T1=17.12	OPT f=394 TT=38.378 T1=20.41	OPT f=394 TT=3.825 T1=3.82	OPT f=394 TT=2.814 T1=2.78	? (TO) TT=252.038	? (TO) TT=252.114	? (TO) TT=252.07	SAT (TO) f=792 TT=252.212 T1=21.04
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
Rlfap/Rlfap-opt/ Rlfap-graph-05-opt_c18.xml	SAT f=24148 TT=252.004 T1=58.97	SAT (TO) f=29741 TT=252.117 T1=91.26	SAT (TO) f=29172 TT=252.126 T1=12.06	SAT (TO) f=31051 TT=252.09 T1=12.88	SAT (TO) f=24148 TT=252.004 T1=58.97	SAT (TO) f=79346 TT=251.999 T1=249.27	SAT (TO) f=74620 TT=252.05 T1=222.17	SAT (TO) f=88483 TT=252.073 T1=124.28	SAT (TO) f=48854 TT=252.148 T1=231.08	SAT (TO) f=31154 TT=252.076 T1=2.31
Rlfap/Rlfap-opt/ Rlfap-graph-06-opt_c18.xml	SAT f=41530 TT=251.979 T1=62.63	SAT (TO) f=69941 TT=252.063 T1=121.32	SAT (TO) f=63708 TT=252.202 T1=30.16	SAT (TO) f=76341 TT=252.098 T1=44.57	SAT (TO) f=41530 TT=251.979 T1=62.63	SAT (TO) f=154268 TT=251.968 T1=225.81	? (TO) TT=252.126	? (TO) TT=252.033	SAT (TO) f=148644 TT=252.051 T1=238.32	SAT (TO) f=49463 TT=252.504 T1=4.87
Rlfap/Rlfap-opt/ Rlfap-graph-07-opt_c18.xml	SAT f=14479 TT=252.078 T1=174.24	SAT (TO) f=14479 TT=252.078 T1=174.24	SAT (TO) f=67335 TT=252.116 T1=213.23	SAT (TO) f=68255 TT=252.147 T1=17.34	SAT (TO) f=34880 TT=252.002 T1=23.33	SAT (TO) f=103986 TT=251.98 T1=148.66	SAT (TO) f=162097 TT=252.054 T1=212.04	SAT (TO) f=159504 TT=252.107 T1=207.93	SAT (TO) f=151308 TT=252.072 T1=236.82	SAT (TO) f=25976 TT=252.156 T1=4.49
Rlfap/Rlfap-opt/ Rlfap-graph-08-opt_c18.xml	SAT f=20 TT=251.994 T1=46.91	SAT (TO) f=24 TT=252.078 T1=175.9	SAT (TO) f=30 TT=252.107 T1=217.77	SAT (TO) f=34 TT=252.115 T1=205.36	SAT (TO) f=20 TT=251.994 T1=46.91	SAT (TO) f=48 TT=251.964 T1=252	? TT=251.726	? TT=251.792	SAT (TO) f=48 TT=252.111 T1=58.06	? (TO) TT=259.859
Rlfap/Rlfap-opt/ Rlfap-graph-09-opt_c18.xml	SAT f=22 TT=252.029 T1=48.73	? (TO) TT=252.115	SAT (TO) f=44 TT=252.128 T1=206.16	? (TO) TT=252.129	SAT (TO) f=22 TT=252.029 T1=48.73	? (TO) TT=251.884	? TT=251.893	? (TO) TT=252.12	SAT (TO) f=46 TT=252.042 T1=180.13	? (TO) TT=260.321
Rlfap/Rlfap-opt/ Rlfap-graph-10-opt_c18.xml	OPT f=394 TT=11.709 T1=5.35	OPT f=394 TT=11.709 T1=5.35	OPT f=394 TT=58.249 T1=39.83	OPT f=394 TT=59.33 T1=39.45	OPT f=394 TT=20.17 T1=20.16	OPT f=394 TT=5.914 T1=5.86	? (TO) TT=252.059	? (TO) TT=252.069	? (TO) TT=252.092	SAT (TO) f=792 TT=252.298 T1=108.58
Rlfap/Rlfap-opt/ Rlfap-graph-11-opt_c18.xml	SAT f=101994 TT=252.138 T1=8.26	? (TO) TT=252.113	SAT (TO) f=126767 TT=252.128 T1=54.15	SAT (TO) f=107313 TT=257.833 T1=82.18	SAT (TO) f=165690 TT=252.017 T1=131.47	SAT (TO) f=311517 TT=251.938 T1=178.95	? (TO) TT=252.066	? (TO) TT=252.069	? (TO) TT=252.095	SAT (TO) f=101994 TT=252.138 T1=8.26
Rlfap/Rlfap-opt/ Rlfap-graph-12-opt_c18.xml	SAT f=21762 TT=252.145 T1=9.27	SAT (TO) f=83493 TT=252.045 T1=14	SAT (TO) f=138065 TT=252.088 T1=38.39	? (TO) TT=255.894	SAT (TO) f=82780 TT=251.97 T1=116.69	SAT (TO) f=259756 TT=251.889 T1=166.46	? (TO) TT=252.034	? (TO) TT=252.092	SAT (TO) f=413604 TT=252.102 T1=235.35	SAT (TO) f=21762 TT=252.145 T1=9.27
Rlfap/Rlfap-opt/ Rlfap-graph-13-opt_c18.xml	SAT f=135769 TT=252.173 T1=9.44	? (TO) TT=252.092	SAT (TO) f=159930 TT=252.122 T1=104.39	? (TO) TT=252.757	SAT (TO) f=261388 TT=252.065 T1=57.9	SAT (TO) f=509310 TT=251.955 T1=149.84	? (TO) TT=252.115	? (TO) TT=252.139	? (TO) TT=252.057	SAT (TO) f=135769 TT=252.173 T1=9.44
Rlfap/Rlfap-opt/ Rlfap-graph-14-opt_c18.xml	SAT f=14 TT=251.996 T1=37.67	? (TO) TT=252.128	? (TO) TT=252.173	? (TO) TT=252.304	SAT (TO) f=14 TT=251.996 T1=37.67	? (TO) TT=251.953	? TT=251.881	? TT=251.698	SAT (TO) f=44 TT=252.086 T1=155.7	? (TO) TT=258.024
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
Rlfap/Rlfap-opt/ Rlfap-scen-01-opt_c18.xml	SAT f=16 TT=252.017 T1=45.72	? (TO) TT=252.108	? (TO) TT=252.243	? (TO) TT=252.154	SAT (TO) f=16 TT=252.017 T1=45.72	? (TO) TT=251.982	? (TO) TT=252.035	? TT=251.811	SAT (TO) f=48 TT=252.032 T1=64.59	? (TO) TT=259.158
Rlfap/Rlfap-opt/ Rlfap-scen-02-opt_c18.xml	SAT f=14 TT=252.001 T1=4.62	SAT (TO) f=14 TT=252.055 T1=30.8	SAT (TO) f=16 TT=252.162 T1=39.99	SAT (TO) f=14 TT=252.169 T1=43.94	SAT (TO) f=14 TT=252.001 T1=4.62	SAT (TO) f=28 TT=251.982 T1=252	SAT f=42 TT=251.573 T1=28.24	SAT f=44 TT=251.458 T1=4.3	SAT (TO) f=40 TT=252.097 T1=4.47	SAT (TO) f=38 TT=252.238 T1=37.79
Rlfap/Rlfap-opt/ Rlfap-scen-03-opt_c18.xml	SAT f=16 TT=251.992 T1=46.99	SAT (TO) f=18 TT=252.048 T1=164.05	SAT (TO) f=20 TT=252.068 T1=160.44	SAT (TO) f=18 TT=252.166 T1=211.78	SAT (TO) f=16 TT=251.992 T1=46.99	SAT (TO) f=34 TT=251.958 T1=252	SAT (TO) f=46 TT=252.049 T1=102.32	SAT (TO) f=46 TT=252.105 T1=103.92	? (TO) TT=252.228	? (TO) TT=260.06
Rlfap/Rlfap-opt/ Rlfap-scen-04-opt_c18.xml	OPT f=46 TT=2.146 T1=2.13	OPT f=46 TT=26.095 T1=18.91	OPT f=46 TT=30.27 T1=13.31	OPT f=46 TT=30.135 T1=13.08	OPT f=46 TT=2.146 T1=2.13	SAT (TO) f=46 TT=251.973 T1=252	? TT=251.886	? TT=251.699	? (TO) TT=252.064	? (TO) TT=258.49
Rlfap/Rlfap-opt/ Rlfap-scen-05-opt_c18.xml	OPT f=792 TT=1.474 T1=1.47	OPT f=792 TT=4.677 T1=1.58	OPT f=792 TT=21.753 T1=8.27	OPT f=792 TT=22.303 T1=8.3	OPT f=792 TT=1.474 T1=1.47	OPT f=792 TT=1.497 T1=1.47	SAT (TO) f=792 TT=252.112 T1=23.97	OPT f=792 TT=18.277 T1=13.23	OPT f=792 TT=11.664 T1=6.19	OPT f=792 TT=38.66 T1=5.41
Rlfap/Rlfap-opt/ Rlfap-scen-06-opt_c18.xml	SAT f=15000 TT=252.026 T1=46.73	SAT (TO) f=34635 TT=252.055 T1=147.95	SAT (TO) f=37723 TT=252.195 T1=14.15	SAT (TO) f=31750 TT=252.076 T1=20.14	SAT (TO) f=15000 TT=252.026 T1=46.73	SAT (TO) f=38738 TT=251.976 T1=251.43	? (TO) TT=252.087	? (TO) TT=252.071	SAT (TO) f=56773 TT=252.098 T1=226.14	SAT (TO) f=19782 TT=252.095 T1=100.17
Rlfap/Rlfap-opt/ Rlfap-scen-07-opt_c18.xml	SAT f=29026880 TT=252.16 T1=9.67	? (TO) TT=253.622	SAT (TO) f=43544007 TT=252.098 T1=65.06	SAT (TO) f=60195808 TT=252.108 T1=82.68	SAT (TO) f=50839875 TT=251.963 T1=95.69	SAT (TO) f=124872301 TT=252.202 T1=243.9	? (TO) TT=252.119	? (TO) TT=252.092	? (TO) TT=252.148	SAT (TO) f=29026880 TT=252.16 T1=9.67
Rlfap/Rlfap-opt/ Rlfap-scen-08-opt_c18.xml	SAT f=2505 TT=251.999 T1=62.5	? (TO) TT=252.106	? (TO) TT=252.158	? (TO) TT=256.255	SAT (TO) f=2505 TT=251.999 T1=62.5	SAT (TO) f=4454 TT=251.97 T1=250.22	? (TO) TT=252.094	? (TO) TT=252.065	? (TO) TT=252.054	SAT (TO) f=2538 TT=252.152 T1=16.98
Rlfap/Rlfap-opt/ Rlfap-scen-09-opt_c18.xml	SAT f=19049 TT=252.18 T1=8.42	SAT (TO) f=37373 TT=252.043 T1=10.59	SAT (TO) f=82209 TT=252.133 T1=35.13	SAT (TO) f=91591 TT=252.079 T1=37.67	SAT (TO) f=36799 TT=251.992 T1=226.2	SAT (TO) f=80808 TT=251.941 T1=240.15	SAT (TO) f=417128 TT=252.126 T1=242.98	? (TO) TT=252.047	SAT (TO) f=365470 TT=252.084 T1=228.86	SAT (TO) f=19049 TT=252.18 T1=8.42
Rlfap/Rlfap-opt/ Rlfap-scen-10-opt_c18.xml	SAT f=31727 TT=252.11 T1=228.23	SAT (TO) f=158233 TT=252.113 T1=63.42	SAT (TO) f=2616361 TT=252.185 T1=160.87	? (TO) TT=255.108	SAT (TO) f=555211 TT=252 T1=90.49	SAT (TO) f=390744 TT=251.934 T1=240.64	SAT (TO) f=9067350 TT=252.111 T1=225.8	SAT (TO) f=10728361 TT=252.091 T1=239.17	SAT (TO) f=10671575 TT=252.111 T1=231.17	SAT (TO) f=31727 TT=252.11 T1=228.23
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
Rlfap/Rlfap-opt/ Rlfap-scen-11-opt_c18.xml	SAT f=22 TT=252.031 T1=49.11	? (TO) TT=252.119	? (TO) TT=252.179	? (TO) TT=252.172	SAT (TO) f=22 TT=252.031 T1=49.11	SAT (TO) f=42 TT=251.999 T1=252	? TT=251.929	? (TO) TT=252.05	? (TO) TT=252.089	? (TO) TT=260.337
SteelMillSlab/ SteelMillSlab-m1-2-0_c18.xml		? (TO) TT=252.035	? (TO) TT=252.118	? (TO) TT=254.016	? (NS) TT=0.022	? (NS) TT=0.053	? (TO) TT=252.081	? (TO) TT=252.069	? (TO) TT=252.105	? (TO) TT=259.97
SteelMillSlab/ SteelMillSlab-m1-2-2_c18.xml		? (TO) TT=252.045	? (TO) TT=252.159	? (TO) TT=252.46	? (NS) TT=0.02	? (NS) TT=0.053	? (TO) TT=252.113	? (TO) TT=252.032	? (TO) TT=252.1	? (TO) TT=259.992
SteelMillSlab/ SteelMillSlab-m1-3-0_c18.xml		? (TO) TT=252.129	? (TO) TT=252.156	? (TO) TT=252.516	? (NS) TT=0.02	? (NS) TT=0.053	? (TO) TT=252.058	? (TO) TT=252.117	? (TO) TT=252.107	? (TO) TT=260.124
SteelMillSlab/ SteelMillSlab-m1-3-2_c18.xml		? (TO) TT=252.174	? (TO) TT=252.079	? (TO) TT=252.173	? (NS) TT=0.02	? (NS) TT=0.053	? (TO) TT=252.069	? (TO) TT=252.085	? (TO) TT=252.077	? (TO) TT=260.163
SteelMillSlab/ SteelMillSlab-m1-4-0_c18.xml		? (TO) TT=252.108	? (TO) TT=252.068	? (TO) TT=252.117	? (NS) TT=0.019	? (NS) TT=0.052	? (TO) TT=252.064	? (TO) TT=252.076	? (TO) TT=252.045	? (TO) TT=259.965
SteelMillSlab/ SteelMillSlab-m1-simple_c18.xml	OPT f=0 TT=1.392 T1=0.63	OPT f=0 TT=1.392 T1=0.63	OPT f=0 TT=9.25 T1=4.19	OPT f=0 TT=9.628 T1=4.31	? (NS) TT=0.006	OPT f=0 TT=0.041 T1=0.04	SAT f=0 TT=251.414 T1=1.2	OPT f=0 TT=1.966 T1=1.2	OPT f=0 TT=1.967 T1=1.2	OPT f=0 TT=2.551 T1=1.11
SteelMillSlab/ SteelMillSlab-m2-2-0_c18.xml		? (TO) TT=252.07	? (TO) TT=252.132	? (TO) TT=252.154	? (TO) TT=252.304	? (TO) TT=251.837	? (TO) TT=252.043	? (TO) TT=252.137	? (TO) TT=252.079	? (TO) TT=259.895
SteelMillSlab/ SteelMillSlab-m2-2-2_c18.xml		? (TO) TT=252.055	? (TO) TT=252.118	? (TO) TT=252.131	? (TO) TT=252.29	? (TO) TT=252.005	? (TO) TT=252.038	? (TO) TT=252.052	? (TO) TT=252.033	? (TO) TT=260.184
SteelMillSlab/ SteelMillSlab-m2-3-0_c18.xml		? (TO) TT=252.068	? (TO) TT=252.171	? (TO) TT=252.091	? (TO) TT=252.329	? (TO) TT=251.871	? (TO) TT=252.06	? (TO) TT=252.109	? (TO) TT=252.098	? (TO) TT=257.506
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
SteelMillSlab/ SteelMillSlab-m2-3-2_c18.xml		? (TO) TT=252.106	? (TO) TT=252.18	? (TO) TT=252.066	? (TO) TT=252.268	? (TO) TT=251.931	? (TO) TT=252.091	? (TO) TT=252.092	? (TO) TT=252.043	? (TO) TT=260.107
SteelMillSlab/ SteelMillSlab-m2-4-0_c18.xml		? (TO) TT=252.093	? (TO) TT=252.118	? (TO) TT=252.128	? (TO) TT=252.268	? (TO) TT=252.046	? (TO) TT=252.045	? (TO) TT=252.108	? (TO) TT=252.073	? (TO) TT=259.182
SteelMillSlab/ SteelMillSlab-m2-simple_c18.xml	OPT f=0 TT=1.793 T1=0.76	OPT f=0 TT=1.793 T1=0.76	OPT f=0 TT=9.038 T1=4.2	OPT f=0 TT=9.449 T1=4.3	SAT f=8 TT=0.093 T1=0.09	OPT f=0 TT=0.098 T1=0.09	SAT f=0 TT=252.06 T1=1.35	OPT f=0 TT=2.596 T1=1.36	OPT f=0 TT=2.462 T1=1.34	OPT f=0 TT=2.835 T1=1.29
SteelMillSlab/ SteelMillSlab-m2s-2-0_c18.xml	SAT f=346 TT=252.143 T1=134.64	? (TO) TT=252.052	SAT (TO) f=346 TT=252.143 T1=134.64	? (TO) TT=260.206	? (TO) TT=252.266	SAT (TO) f=94 TT=251.876 T1=84.72	? (TO) TT=252.04	? (TO) TT=252.076	? (TO) TT=252.112	? (TO) TT=252.254
SteelMillSlab/ SteelMillSlab-m2s-2-2_c18.xml	SAT f=647 TT=252.127 T1=89.16	? (TO) TT=252.096	SAT (TO) f=647 TT=252.127 T1=89.16	? (TO) TT=255.32	? (TO) TT=252.346	SAT (TO) f=182 TT=251.864 T1=222.98	SAT (TO) f=1116 TT=252.162 T1=206.88	? (TO) TT=252.095	? (TO) TT=252.129	? (TO) TT=252.135
SteelMillSlab/ SteelMillSlab-m2s-3-0_c18.xml	SAT f=220 TT=252.151 T1=191.98	? (TO) TT=252.038	SAT (TO) f=220 TT=252.151 T1=191.98	? (TO) TT=255.46	? (TO) TT=252.303	SAT (TO) f=37 TT=251.881 T1=220.64	SAT (TO) f=377 TT=252.173 T1=238.12	SAT (TO) f=417 TT=252.128 T1=189.23	? (TO) TT=252.127	? (TO) TT=252.193
SteelMillSlab/ SteelMillSlab-m2s-3-2_c18.xml	SAT f=330 TT=252.509 T1=120.19	? (TO) TT=252.037	SAT (TO) f=372 TT=252.157 T1=144.81	SAT (TO) f=330 TT=252.509 T1=120.19	? (TO) TT=252.37	SAT (TO) f=73 TT=251.89 T1=224.75	SAT (TO) f=549 TT=252.178 T1=200.84	SAT (TO) f=515 TT=252.144 T1=228.02	? (TO) TT=252.104	? (TO) TT=252.209
SteelMillSlab/ SteelMillSlab-m2s-4-0_c18.xml	SAT f=203 TT=252.26 T1=113.87	? (TO) TT=252.102	SAT (TO) f=238 TT=252.15 T1=46.61	SAT (TO) f=203 TT=252.26 T1=113.87	? (TO) TT=252.367	SAT (TO) f=62 TT=251.85 T1=91.22	SAT (TO) f=314 TT=252.111 T1=196.44	SAT (TO) f=300 TT=252.146 T1=225.31	? (TO) TT=252.037	? (TO) TT=252.199
StillLife/ StillLife-wastage-08_c18.xml	OPT f=36 TT=3.638 T1=3.27	OPT f=36 TT=13.786 T1=10.23	OPT f=36 TT=42.977 T1=28.37	OPT f=36 TT=25.774 T1=13.81	OPT f=36 TT=3.638 T1=3.27	OPT f=36 TT=5.228 T1=5.14	SAT (TO) f=36 TT=252.097 T1=10.04	OPT f=36 TT=26.207 T1=22.11	SAT (TO) f=33 TT=252.036 T1=4.59	? (TO) TT=252.083
StillLife/ StillLife-wastage-09_c18.xml	OPT f=43 TT=102.766 T1=6.63	OPT f=43 TT=102.766 T1=21.15	OPT f=43 TT=230.983 T1=64.45	OPT f=43 TT=106.979 T1=15.81	OPT f=43 TT=153.016 T1=24.94	OPT f=43 TT=102.299 T1=15.11	SAT (TO) f=43 TT=252.043 T1=6.63	SAT (TO) f=43 TT=252.041 T1=125.79	SAT (TO) f=40 TT=252.127 T1=7.59	? (TO) TT=252.151
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
StillLife/ StillLife-wastage-10_c18.xml	SAT f=54 TT=252.035 T1=23.35	SAT (TO) f=53 TT=252.09 T1=157.93	SAT (TO) f=54 TT=252.1 T1=202.87	SAT (TO) f=51 TT=252.127 T1=122.25	SAT (TO) f=49 TT=251.98 T1=143.57	SAT (TO) f=54 TT=251.99 T1=122.94	SAT (TO) f=54 TT=252.035 T1=23.35	SAT (TO) f=54 TT=252.046 T1=162.94	SAT (TO) f=46 TT=252.124 T1=5.79	? (TO) TT=252.093
StillLife/ StillLife-wastage-11_c18.xml	SAT f=63 TT=252.064 T1=129.34	SAT (TO) f=56 TT=252.075 T1=160.83	SAT (TO) f=63 TT=252.064 T1=129.34	SAT (TO) f=61 TT=252.128 T1=164.75	SAT (TO) f=55 TT=251.983 T1=247.91	SAT (TO) f=64 TT=251.84 T1=192.88	SAT (TO) f=59 TT=252.068 T1=152.49	SAT (TO) f=54 TT=252.088 T1=4.82	SAT (TO) f=61 TT=252.079 T1=59.13	? (TO) TT=252.106
StillLife/ StillLife-wastage-12_c18.xml	SAT f=74 TT=252.054 T1=14.35	SAT (TO) f=70 TT=252.109 T1=95.03	SAT (TO) f=74 TT=252.123 T1=84.84	SAT (TO) f=73 TT=252.082 T1=85.66	SAT (TO) f=53 TT=251.982 T1=232.37	SAT (TO) f=76 TT=251.994 T1=167.2	SAT (TO) f=73 TT=252.12 T1=242.57	SAT (TO) f=68 TT=252.038 T1=11.6	SAT (TO) f=74 TT=252.054 T1=14.35	? (TO) TT=252.17
StillLife/ StillLife-wastage-13_c18.xml	SAT f=86 TT=252.128 T1=26.14	SAT (TO) f=75 TT=252.031 T1=22.65	SAT (TO) f=86 TT=252.128 T1=26.14	SAT (TO) f=83 TT=252.143 T1=141.95	SAT (TO) f=59 TT=252.012 T1=237.98	SAT (TO) f=89 TT=251.979 T1=159.4	SAT (TO) f=85 TT=252.064 T1=26.88	SAT (TO) f=82 TT=252.119 T1=6.95	SAT (TO) f=81 TT=252.073 T1=59.41	? (TO) TT=252.101
StillLife/ StillLife-wastage-14_c18.xml	SAT f=97 TT=252.15 T1=55.42	SAT (TO) f=88 TT=252.116 T1=22.6	SAT (TO) f=92 TT=252.151 T1=205.41	SAT (TO) f=97 TT=252.15 T1=55.42	SAT (TO) f=56 TT=252.012 T1=218.65	SAT (TO) f=100 TT=251.981 T1=187.07	SAT (TO) f=82 TT=252.13 T1=33.68	SAT (TO) f=86 TT=252.133 T1=234.6	SAT (TO) f=88 TT=252.11 T1=23.7	? (TO) TT=252.191
StillLife/ StillLife-wastage-15_c18.xml	SAT f=109 TT=252.096 T1=189.35	SAT (TO) f=98 TT=252.045 T1=198.64	SAT (TO) f=109 TT=252.096 T1=189.35	SAT (TO) f=107 TT=252.053 T1=89.28	SAT (TO) f=60 TT=251.976 T1=240.28	SAT (TO) f=115 TT=251.904 T1=227.22	SAT (TO) f=108 TT=252.07 T1=163.4	SAT (TO) f=106 TT=252.072 T1=55.84	SAT (TO) f=90 TT=252.077 T1=79.68	? (TO) TT=252.165
StillLife/ StillLife-wastage-17_c18.xml	SAT f=138 TT=252.145 T1=177.01	SAT (TO) f=119 TT=252.026 T1=156.02	SAT (TO) f=138 TT=252.145 T1=177.01	SAT (TO) f=134 TT=252.163 T1=105.76	SAT (TO) f=92 TT=252.012 T1=0.13	SAT (TO) f=145 TT=251.91 T1=238.46	SAT (TO) f=135 TT=252.073 T1=171.43	SAT (TO) f=129 TT=252.096 T1=19.23	SAT (TO) f=107 TT=252.034 T1=26.39	? (TO) TT=252.134
StillLife/ StillLife-wastage-19_c18.xml	SAT f=168 TT=252.144 T1=69.79	SAT (TO) f=152 TT=252.085 T1=98.25	SAT (TO) f=168 TT=252.144 T1=69.79	SAT (TO) f=159 TT=252.126 T1=214.28	SAT (TO) f=49 TT=252.011 T1=1.63	SAT (TO) f=178 TT=251.934 T1=112.75	SAT (TO) f=161 TT=252.121 T1=167.67	SAT (TO) f=152 TT=252.096 T1=246.57	SAT (TO) f=154 TT=252.077 T1=222.47	? (TO) TT=260.159
StillLife/ StillLife-wastage-21_c18.xml	SAT f=195 TT=252.101 T1=85.75	SAT (TO) f=171 TT=252.111 T1=33.39	SAT (TO) f=195 TT=252.101 T1=85.75	SAT (TO) f=194 TT=252.128 T1=124.35	SAT (TO) f=52 TT=251.976 T1=193.66	SAT (TO) f=216 TT=251.945 T1=212.87	SAT (TO) f=181 TT=252.441 T1=101.71	SAT (TO) f=180 TT=252.151 T1=18.12	SAT (TO) f=164 TT=252.112 T1=12.45	? (TO) TT=260.085
StillLife/ StillLife-wastage-25_c18.xml	SAT f=274 TT=252.122 T1=217.69	SAT (TO) f=248 TT=252.026 T1=29.84	SAT (TO) f=274 TT=252.122 T1=217.69	SAT (TO) f=271 TT=252.103 T1=40.74	SAT (TO) f=42 TT=251.979 T1=186.73	SAT (TO) f=288 TT=251.955 T1=208.02	SAT (TO) f=256 TT=252.13 T1=56.01	SAT (TO) f=254 TT=252.045 T1=70.11	SAT (TO) f=223 TT=252.063 T1=44.99	? (TO) TT=260.142
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
StillLife/ StillLife-wastage-30_c18.xml	SAT f=396 TT=252.181 T1=47.51	SAT (TO) f=367 TT=252.11 T1=210.11	SAT (TO) f=396 TT=252.181 T1=47.51	SAT (TO) f=371 TT=252.15 T1=39.78	SAT (TO) f=71 TT=251.993 T1=9.23	SAT (TO) f=387 TT=251.743 T1=209.95	SAT (TO) f=364 TT=252.062 T1=28.76	SAT (TO) f=367 TT=252.135 T1=100.77	SAT (TO) f=266 TT=252.125 T1=19.15	? (TO) TT=260.286
SumColoring/ SumColoring-1-fullins-3_c18.xml	OPT f=24 TT=0.517 T1=0.05	OPT f=24 TT=9.311 T1=1.02	OPT f=24 TT=13.289 T1=3.61	OPT f=24 TT=24.528 T1=5.24	OPT f=24 TT=0.517 T1=0.05	OPT f=24 TT=0.152 T1=0.04	SAT (TO) f=24 TT=252.103 T1=6.6	OPT f=24 TT=72.451 T1=6.89	SAT (TO) f=24 TT=252.096 T1=25.77	SAT (TO) f=27 TT=252.355 T1=0.75
SumColoring/ SumColoring-1-fullins-4_c18.xml	SAT f=73 TT=252.06 T1=24.99	SAT (TO) f=97 TT=252.057 T1=169.44	SAT (TO) f=73 TT=252.07 T1=24.99	SAT (TO) f=73 TT=252.06 T1=140.56	SAT (TO) f=84 TT=252.003 T1=23.23	SAT (TO) f=87 TT=252.006 T1=34.28	SAT f=109 TT=251.906 T1=239.62	SAT (TO) f=98 TT=252.063 T1=99.53	SAT (TO) f=91 TT=252.086 T1=25.59	SAT (TO) f=139 TT=252.3 T1=3.6
SumColoring/ SumColoring-1-fullins-5_c18.xml	SAT f=241 TT=252.092 T1=1.1	SAT (TO) f=241 TT=252.092 T1=1.1	SAT (TO) f=1067 TT=252.109 T1=130.06	SAT (TO) f=333 TT=252.126 T1=209.11	SAT (TO) f=290 TT=251.999 T1=12.02	SAT (TO) f=280 TT=251.982 T1=0.57	SAT (TO) f=11673 TT=252.102 T1=245.17	SAT (TO) f=6225 TT=252.108 T1=243.65	SAT (TO) f=272 TT=252.117 T1=73.53	? (TO) TT=260.282
SumColoring/ SumColoring-dsjc-125-1_c18.xml	SAT f=233 TT=252.035 T1=0.76	SAT (TO) f=233 TT=252.035 T1=0.76	SAT (TO) f=522 TT=252.143 T1=175.39	SAT (TO) f=641 TT=252.109 T1=150.35	SAT (TO) f=241 TT=252.007 T1=4.46	SAT (TO) f=268 TT=252.009 T1=223.3	SAT (TO) f=1485 TT=252.031 T1=242.19	SAT f=1003 TT=251.854 T1=225.27	SAT (TO) f=236 TT=252.123 T1=161.67	SAT (TO) f=247 TT=252.26 T1=7.59
SumColoring/ SumColoring-dsjc-125-5_c18.xml	SAT f=1060 TT=252.052 T1=144.17	SAT (TO) f=1086 TT=252.068 T1=34.88	SAT (TO) f=1412 TT=252.113 T1=150.04	SAT (TO) f=1814 TT=252.079 T1=124.88	SAT (TO) f=1123 TT=252.032 T1=81.82	SAT (TO) f=1090 TT=251.994 T1=154.19	SAT (TO) f=3039 TT=252.095 T1=228.02	SAT (TO) f=2566 TT=252.056 T1=185.47	SAT (TO) f=1060 TT=252.052 T1=144.17	SAT (TO) f=1337 TT=252.355 T1=58.81
SumColoring/ SumColoring-dsjc-125-9_c18.xml	SAT f=2639 TT=252.332 T1=221.67	SAT (TO) f=2787 TT=252.047 T1=210.48	? (TO) TT=254.199	? (TO) TT=254.2	SAT (TO) f=2797 TT=251.995 T1=36.12	SAT (TO) f=2720 TT=251.98 T1=164.17	SAT (TO) f=3925 TT=252.107 T1=192.73	SAT (TO) f=4563 TT=252.112 T1=190.13	SAT (TO) f=2639 TT=252.332 T1=221.67	? (TO) TT=255.477
SumColoring/ SumColoring-dsjc-250-1_c18.xml	SAT f=853 TT=252.09 T1=127.36	SAT (TO) f=925 TT=252.112 T1=158.94	SAT (TO) f=3308 TT=252.147 T1=23.49	SAT (TO) f=2951 TT=252.089 T1=5.94	SAT (TO) f=928 TT=251.998 T1=78.54	SAT (TO) f=1036 TT=251.967 T1=186.99	SAT (TO) f=13499 TT=252.056 T1=244.36	SAT (TO) f=13489 TT=252.058 T1=245.04	SAT (TO) f=853 TT=252.09 T1=127.36	? (TO) TT=259.877
SumColoring/ SumColoring-dsjc-250-5_c18.xml	SAT f=3949 TT=252 T1=2.56	SAT (TO) f=3979 TT=252.073 T1=155.89	? (TO) TT=254.096	? (TO) TT=254.131	SAT (TO) f=3949 TT=252 T1=2.56	SAT (TO) f=4270 TT=251.999 T1=240.37	SAT (TO) f=17293 TT=252.051 T1=239.96	SAT (TO) f=14757 TT=252.085 T1=162.38	SAT (TO) f=4004 TT=252.13 T1=236.88	? (TO) TT=252.674
SumColoring/ SumColoring-dsjc-250-9_c18.xml	SAT f=10090 TT=251.968 T1=122.87	SAT (TO) f=15018 TT=252.084 T1=68.23	? (TO) TT=254.186	? (TO) TT=254.182	SAT (TO) f=10090 TT=251.968 T1=122.87	SAT (TO) f=9746 TT=251.945 T1=208.57	SAT (TO) f=18397 TT=252.086 T1=181.07	SAT (TO) f=20429 TT=252.076 T1=170.85	SAT (TO) f=11681 TT=252.045 T1=239.72	? (TO) TT=260.041
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
SumColoring/ SumColoring-myciel3_c18.xml	OPT f=10 TT=0.009 T1=0.01	OPT f=10 TT=1.027 T1=0.49	OPT f=10 TT=6.512 T1=3.17	OPT f=10 TT=6.703 T1=3.21	OPT f=10 TT=0.009 T1=0.01	OPT f=10 TT=0.008 T1=0	SAT f=10 TT=251.713 T1=1.06	OPT f=10 TT=1.669 T1=1.05	OPT f=10 TT=1.762 T1=1.08	OPT f=10 TT=1.214 T1=0.41
SumColoring/ SumColoring-myciel4_c18.xml	OPT f=22 TT=0.242 T1=0.07	OPT f=22 TT=6.574 T1=1.4	OPT f=22 TT=14.27 T1=3.79	OPT f=22 TT=17.066 T1=4.18	OPT f=22 TT=0.242 T1=0.07	OPT f=22 TT=0.106 T1=0.02	SAT (TO) f=22 TT=252.045 T1=3.49	OPT f=22 TT=10.006 T1=3.65	OPT f=22 TT=10.299 T1=5.81	SAT (TO) f=23 TT=252.142 T1=231.25
SumColoring/ SumColoring-myciel5_c18.xml	SAT f=46 TT=251.958 T1=26.63	SAT (TO) f=51 TT=252.089 T1=14.07	SAT (TO) f=46 TT=252.136 T1=29.63	SAT (TO) f=50 TT=252.132 T1=4.57	SAT (TO) f=46 TT=251.958 T1=229.89	OPT f=46 TT=118.26 T1=3.29	SAT f=47 TT=251.026 T1=133.02	SAT f=51 TT=250.653 T1=132.71	SAT (TO) f=46 TT=252.051 T1=26.63	SAT (TO) f=71 TT=252.23 T1=0.81
SumColoring/ SumColoring-myciel6_c18.xml	SAT f=112 TT=252.061 T1=10.46	SAT (TO) f=122 TT=252.067 T1=112.04	SAT (TO) f=112 TT=252.061 T1=10.46	SAT (TO) f=122 TT=252.094 T1=115.07	SAT (TO) f=119 TT=251.971 T1=134.51	SAT (TO) f=99 TT=251.983 T1=191.89	SAT (TO) f=285 TT=252.112 T1=246.55	SAT (TO) f=133 TT=252.034 T1=169.91	SAT (TO) f=118 TT=252.129 T1=170.36	SAT (TO) f=160 TT=252.316 T1=3.63
SumColoring/ SumColoring-myciel7_c18.xml	SAT f=234 TT=252.072 T1=201.26	SAT (TO) f=267 TT=252.07 T1=9.52	SAT (TO) f=530 TT=252.099 T1=24.5	SAT (TO) f=780 TT=252.099 T1=132.83	SAT (TO) f=251 TT=252.035 T1=0.37	SAT (TO) f=375 TT=252.012 T1=232.37	SAT (TO) f=2755 TT=252.07 T1=200.28	SAT (TO) f=2345 TT=252.075 T1=224.29	SAT (TO) f=234 TT=252.072 T1=201.26	? (TO) TT=260.071
Tal/ Tal-01_c18.xml	OPT f=6 TT=0.031 T1=0.03	OPT f=6 TT=1.577 T1=0.71	OPT f=6 TT=9.79 T1=4.19	OPT f=6 TT=10.054 T1=4.26	OPT f=6 TT=0.031 T1=0.03	OPT f=6 TT=0.202 T1=0.2	SAT f=6 TT=251.685 T1=1.23	OPT f=6 TT=2.154 T1=1.22	OPT f=6 TT=2.137 T1=1.22	OPT f=6 TT=202.584 T1=192.53
Tal/ Tal-02_c18.xml	OPT f=86 TT=3.318 T1=0.79	OPT f=86 TT=7.254 T1=2.06	OPT f=86 TT=21.264 T1=5.81	OPT f=86 TT=22.166 T1=6.39	OPT f=86 TT=3.318 T1=0.79	? (NS) TT=0.022	SAT (TO) f=86 TT=252.044 T1=1.61	OPT f=86 TT=3.996 T1=1.64	OPT f=86 TT=4.485 T1=1.56	? (TO) TT=252.141
Tal/ Tal-03_c18.xml	OPT f=96 TT=98.604 T1=4.24	OPT f=96 TT=241.604 T1=184.96	SAT (TO) f=96 TT=252.146 T1=20.39	SAT (TO) f=96 TT=252.054 T1=14.72	SAT (TO) f=104 TT=252.006 T1=224.45	? (NS) TT=0.026	SAT (TO) f=96 TT=252.046 T1=4.24	OPT f=96 TT=98.604 T1=41.16	SAT (TO) f=104 TT=252.043 T1=4.79	? (TO) TT=252.147
Tal/ Tal-04_c18.xml	SAT f=142 TT=252.034 T1=28.46	SAT (TO) f=142 TT=252.058 T1=28.87	SAT (TO) f=146 TT=252.146 T1=21.7	SAT (TO) f=142 TT=252.06 T1=195.13	SAT (TO) f=146 TT=251.981 T1=141.02	? (MO) TT=19.973	SAT (TO) f=142 TT=252.034 T1=28.46	SAT (TO) f=142 TT=252.062 T1=40.41	SAT (TO) f=146 TT=252.104 T1=3.38	? (TO) TT=252.141
Tal/ Tal-05_c18.xml	OPT f=16 TT=69.854 T1=3.33	SAT (TO) f=16 TT=252.094 T1=3.33	SAT (TO) f=16 TT=252.118 T1=9.71	SAT (TO) f=16 TT=252.079 T1=53.39	SAT (TO) f=16 TT=251.98 T1=129.09	? (MO) TT=19.89	SAT (TO) f=16 TT=252.07 T1=11.2	OPT f=16 TT=69.854 T1=11.07	SAT (TO) f=16 TT=252.054 T1=6.03	? (TO) TT=252.11
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
Tal/ Tal-06_c18.xml	SAT f=94 TT=252.03 T1=155.14	SAT (TO) f=114 TT=252.091 T1=17.25	SAT (TO) f=94 TT=252.131 T1=155.14	SAT (TO) f=106 TT=252.052 T1=38.51	SAT (TO) f=106 TT=252.015 T1=22.24	? (NS) TT=0.036	SAT (TO) f=94 TT=252.03 T1=166.89	SAT (TO) f=94 TT=252.118 T1=203.71	SAT (TO) f=114 TT=252.06 T1=10.21	? (TO) TT=252.109
Tal/ Tal-07_c18.xml	SAT f=138 TT=252.061 T1=188.28	SAT (TO) f=210 TT=252.113 T1=20.85	SAT (TO) f=154 TT=252.065 T1=219.69	SAT (TO) f=190 TT=252.112 T1=82.53	? (TO) TT=252.01	? (NS) TT=0.042	SAT (TO) f=158 TT=252.084 T1=223.48	SAT (TO) f=138 TT=252.061 T1=188.28	SAT (TO) f=190 TT=252.135 T1=5.64	? (TO) TT=252.149
Tal/ Tal-08_c18.xml	SAT f=176 TT=252.046 T1=202.73	? (TO) TT=252.123	? (TO) TT=252.112	? (TO) TT=252.062	? (TO) TT=252.005	? (NS) TT=0.049	SAT (TO) f=194 TT=252.128 T1=167.44	SAT (TO) f=176 TT=252.046 T1=202.73	? (TO) TT=252.032	? (TO) TT=252.109
Tal/ Tal-09_c18.xml	SAT f=106 TT=252.043 T1=71.17	SAT (TO) f=158 TT=252.111 T1=134.54	SAT (TO) f=178 TT=252.134 T1=144.32	SAT (TO) f=208 TT=252.108 T1=60.78	SAT (TO) f=206 TT=251.994 T1=0.53	? (NS) TT=0.049	SAT (TO) f=138 TT=252.027 T1=30.96	SAT (TO) f=192 TT=252.053 T1=19.75	SAT (TO) f=106 TT=252.043 T1=71.17	? (TO) TT=252.114
Tal/ Tal-10_c18.xml	SAT f=276 TT=252.041 T1=56.85	? (TO) TT=252.042	? (TO) TT=252.149	? (TO) TT=252.077	? (TO) TT=252.001	? (NS) TT=0.061	SAT (TO) f=276 TT=252.041 T1=56.85	SAT (TO) f=288 TT=252.085 T1=157.75	? (TO) TT=252.066	? (TO) TT=254.002
TemplateDesign/ TemplateDesign-m1-1_c18.xml	OPT f=2 TT=1.221 T1=0.57	OPT f=2 TT=1.221 T1=0.57	OPT f=2 TT=7.803 T1=3.82	OPT f=2 TT=7.954 T1=3.83	OPT f=2 TT=16.016 T1=15.71	OPT f=2 TT=0.011 T1=0.01	SAT f=2 TT=251.42 T1=1.07	OPT f=2 TT=1.606 T1=1.05	OPT f=2 TT=1.601 T1=1.04	OPT f=2 TT=72.746 T1=48.8
TemplateDesign/ TemplateDesign-m1-2_c18.xml	OPT f=2 TT=1.504 T1=0.66	OPT f=2 TT=1.504 T1=0.66	OPT f=2 TT=9.288 T1=4.22	OPT f=2 TT=9.626 T1=4.3	? (TO) TT=252.007	OPT f=2 TT=0.019 T1=0.01	SAT f=2 TT=251.182 T1=1.09	OPT f=2 TT=1.691 T1=1.09	OPT f=2 TT=1.791 T1=1.11	? (TO) TT=260.321
TemplateDesign/ TemplateDesign-m1-catfood_c18.xml	OPT f=2 TT=1.768 T1=0.74	OPT f=2 TT=1.768 T1=0.74	OPT f=2 TT=10.325 T1=4.65	OPT f=2 TT=10.763 T1=4.8	? (TO) TT=251.979	OPT f=2 TT=0.03 T1=0.03	SAT f=2 TT=251.595 T1=1.18	OPT f=2 TT=2.014 T1=1.18	OPT f=2 TT=11.182 T1=9.33	? (TO) TT=260.19
TemplateDesign/ TemplateDesign-m1-herbs_c18.xml	OPT f=2 TT=61.037 T1=40.25	? (TO) TT=252.065	OPT f=2 TT=61.037 T1=40.25	? (TO) TT=252.105	? (TO) TT=251.961	? (TO) TT=251.964	? (TO) TT=252.034	? (TO) TT=252.103	? (TO) TT=252.101	? (TO) TT=259.87
TemplateDesign/ TemplateDesign-m1-magazine_c18.xml		? (TO) TT=252.052	? (TO) TT=252.138	? (TO) TT=252.237	? (TO) TT=251.998	? (TO) TT=251.992	? (TO) TT=252.072	? (TO) TT=252.077	? (TO) TT=252.095	? (TO) TT=260.247
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
TemplateDesign/ TemplateDesign-m1s-1_c18.xml	OPT f=2 TT=1.215 T1=0.58	OPT f=2 TT=1.215 T1=0.58	OPT f=2 TT=7.335 T1=3.67	OPT f=2 TT=7.4 T1=3.73	SAT (TO) f=2 TT=251.974 T1=53.25	OPT f=2 TT=0.012 T1=0.01	SAT f=2 TT=251.118 T1=1.12	OPT f=2 TT=1.809 T1=1.41	OPT f=2 TT=1.85 T1=1.13	OPT f=2 TT=78.047 T1=22.56
TemplateDesign/ TemplateDesign-m1s-2_c18.xml	OPT f=2 TT=1.42 T1=0.62	OPT f=2 TT=1.42 T1=0.62	OPT f=2 TT=8.851 T1=4.2	OPT f=2 TT=8.738 T1=4.04	SAT (TO) f=2 TT=252.01 T1=50.45	OPT f=2 TT=0.023 T1=0.01	SAT f=2 TT=251.099 T1=1.19	OPT f=2 TT=2.09 T1=1.2	OPT f=2 TT=2.099 T1=1.2	? (TO) TT=260.352
TemplateDesign/ TemplateDesign-m1s-catfood_c18.xml	OPT f=2 TT=1.551 T1=0.67	OPT f=2 TT=1.551 T1=0.67	OPT f=2 TT=10.415 T1=4.45	OPT f=2 TT=11.591 T1=4.41	? (TO) TT=252.039	OPT f=2 TT=0.047 T1=0.03	SAT f=2 TT=251.891 T1=1.27	OPT f=2 TT=2.346 T1=1.28	OPT f=2 TT=2.427 T1=1.27	? (TO) TT=260.082
TemplateDesign/ TemplateDesign-m1s-herbs_c18.xml	OPT f=2 TT=70.127 T1=29.61	OPT f=2 TT=162.908 T1=68.51	SAT (TO) f=2 TT=252.157 T1=129.54	OPT f=2 TT=70.127 T1=29.61	? (TO) TT=251.971	SAT (TO) f=2 TT=252.018 T1=13.13	? (TO) TT=252.033	? TT=251.872	SAT (TO) f=4 TT=252.122 T1=65.16	? (TO) TT=260.204
TemplateDesign/ TemplateDesign-m1s-magazine_c18.xml	SAT f=14 TT=252.15 T1=138.8	? (TO) TT=252.116	? (TO) TT=252.141	? (TO) TT=252.173	? (TO) TT=252.03	SAT (TO) f=5 TT=251.914 T1=28.38	? (TO) TT=252.056	? (TO) TT=252.075	SAT (TO) f=14 TT=252.15 T1=138.8	? (TO) TT=260.157
TemplateDesign/ TemplateDesign-m2-1_c18.xml	OPT f=2 TT=1.07 T1=0.53	OPT f=2 TT=1.07 T1=0.53	OPT f=2 TT=8.042 T1=3.87	OPT f=2 TT=7.98 T1=3.84	? (TO) TT=251.962	OPT f=2 TT=0.008 T1=0	SAT f=2 TT=250.506 T1=1.03	OPT f=2 TT=1.565 T1=1.02	OPT f=2 TT=1.581 T1=1.03	? (TO) TT=259.546
TemplateDesign/ TemplateDesign-m2-2_c18.xml	OPT f=2 TT=1.219 T1=0.57	OPT f=2 TT=1.219 T1=0.57	OPT f=2 TT=9.327 T1=4.24	OPT f=2 TT=9.377 T1=4.24	? (TO) TT=251.997	OPT f=2 TT=0.011 T1=0.01	SAT f=2 TT=251.408 T1=1.06	OPT f=2 TT=1.618 T1=1.06	OPT f=2 TT=1.835 T1=1.1	? (TO) TT=259.991
TemplateDesign/ TemplateDesign-m2-catfood_c18.xml	OPT f=2 TT=1.368 T1=0.61	OPT f=2 TT=1.368 T1=0.61	OPT f=2 TT=14.445 T1=6.97	OPT f=2 TT=11.712 T1=5.02	? (TO) TT=252.009	OPT f=2 TT=0.014 T1=0.01	SAT (TO) f=2 TT=252.104 T1=14.01	OPT f=2 TT=15.962 T1=13.97	OPT f=2 TT=6.625 T1=4.41	? (TO) TT=259.914
TemplateDesign/ TemplateDesign-m2-herbs_c18.xml	OPT f=2 TT=34.01 T1=14.58	SAT (TO) f=23 TT=252.021 T1=123.74	OPT f=2 TT=34.01 T1=14.58	OPT f=2 TT=34.033 T1=15.02	? (TO) TT=252.038	? (TO) TT=251.979	? TT=251.028	? TT=251.32	? (TO) TT=252.093	? (TO) TT=260.051
TemplateDesign/ TemplateDesign-m2-magazine_c18.xml		? (TO) TT=252.095	? (TO) TT=252.151	? (TO) TT=252.142	? (TO) TT=252	? (TO) TT=251.993	? TT=251.504	? TT=251.582	? (TO) TT=252.028	? (TO) TT=260.138
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
TravelingTournament/ TravelingTournament-a2-galaxy04_c18.xml	OPT f=517 TT=2.782 T1=0.84	OPT f=517 TT=4.115 T1=0.84	OPT f=517 TT=16.56 T1=4.5	SAT f=517 TT=15.323 T1=5.3	? (NS) TT=0.005	Signal TT=0.039	SAT (TO) f=517 TT=252.049 T1=1.48	OPT f=517 TT=2.791 T1=1.49	OPT f=517 TT=2.782 T1=1.37	OPT f=517 TT=11.235 T1=5.06
TravelingTournament/ TravelingTournament-a2-galaxy06_c18.xml	SAT f=1587 TT=252.084 T1=6.98	SAT (TO) f=1637 TT=252.041 T1=1.68	SAT (TO) f=1587 TT=252.084 T1=6.98	SAT (TO) f=1613 TT=252.135 T1=6.62	? (NS) TT=0.006	Signal TT=0.145	SAT (TO) f=1605 TT=252.103 T1=136.59	SAT (TO) f=1598 TT=252.073 T1=236.52	SAT (TO) f=1609 TT=252.078 T1=39.99	SAT (TO) f=1707 TT=252.164 T1=120.75
TravelingTournament/ TravelingTournament-a2-galaxy08_c18.xml	SAT f=3261 TT=252.062 T1=222.71	SAT (TO) f=3320 TT=252.047 T1=75.19	SAT (TO) f=3275 TT=252.138 T1=44.34	SAT (TO) f=3302 TT=252.147 T1=38.5	? (NS) TT=0.008	Signal TT=0.39	SAT (TO) f=3346 TT=252.05 T1=123.85	SAT (TO) f=3318 TT=252.07 T1=235.02	SAT (TO) f=3261 TT=252.062 T1=222.71	? (TO) TT=252.078
TravelingTournament/ TravelingTournament-a2-galaxy10_c18.xml	SAT f=6378 TT=252.083 T1=229.38	SAT (TO) f=6382 TT=252.117 T1=78.54	SAT (TO) f=6451 TT=252.052 T1=23.91	SAT (TO) f=6426 TT=252.133 T1=58.4	? (NS) TT=0.009	SAT (TO) f=6120 TT=251.976 T1=241.14	SAT (TO) f=6520 TT=252.083 T1=92.77	SAT (TO) f=6564 TT=252.051 T1=63.56	SAT (TO) f=6378 TT=252.083 T1=229.38	? (TO) TT=252.113
TravelingTournament/ TravelingTournament-a2-galaxy12_c18.xml	SAT f=10904 TT=252.089 T1=48.55	SAT (TO) f=10912 TT=252.118 T1=159.31	SAT (TO) f=10970 TT=252.05 T1=99.69	SAT (TO) f=10904 TT=252.089 T1=48.55	? (NS) TT=0.012	Signal TT=1.616	SAT (TO) f=11149 TT=252.094 T1=239.48	SAT (TO) f=11120 TT=252.135 T1=7.92	SAT (TO) f=11032 TT=252.073 T1=31.34	? (TO) TT=252.097
TravelingTournament/ TravelingTournament-a2-galaxy14_c18.xml	SAT f=17126 TT=252.078 T1=127.24	SAT (TO) f=17126 TT=252.078 T1=127.24	SAT (TO) f=17326 TT=252.11 T1=163.95	SAT (TO) f=17590 TT=252.666 T1=208.76	? (NS) TT=0.016	Signal TT=2.697	SAT (TO) f=17671 TT=252.054 T1=21.45	SAT (TO) f=17741 TT=252.05 T1=10.92	SAT (TO) f=17684 TT=252.057 T1=5.79	? (TO) TT=252.068
TravelingTournament/ TravelingTournament-a2-galaxy16_c18.xml	SAT f=23746 TT=252.047 T1=138.33	SAT (TO) f=23746 TT=252.047 T1=138.33	? (TO) TT=252.132	SAT (TO) f=25239 TT=252.106 T1=127.67	? (NS) TT=0.021	Signal TT=4.604	? (TO) TT=252.033	SAT (TO) f=25258 TT=252.139 T1=65.13	SAT (TO) f=25035 TT=252.054 T1=25.38	? (TO) TT=252.111
TravelingTournament/ TravelingTournament-a3-galaxy04_c18.xml	OPT f=416 TT=2.915 T1=1.22	OPT f=416 TT=6.411 T1=1.22	OPT f=416 TT=19.307 T1=5.17	OPT f=416 TT=18.784 T1=5.2	? (NS) TT=0.004	Signal TT=0.039	SAT (TO) f=416 TT=252.123 T1=1.37	OPT f=416 TT=2.922 T1=1.37	OPT f=416 TT=2.915 T1=1.37	OPT f=416 TT=15.391 T1=5.42
TravelingTournament/ TravelingTournament-a3-galaxy06_c18.xml	SAT f=1438 TT=252.066 T1=62.33	SAT (TO) f=1464 TT=252.115 T1=198.63	SAT (TO) f=1438 TT=252.066 T1=62.33	SAT (TO) f=1453 TT=252.12 T1=78.96	? (NS) TT=0.006	Signal TT=0.145	SAT (TO) f=1444 TT=252.123 T1=183.78	SAT (TO) f=1451 TT=252.073 T1=4.73	SAT (TO) f=1447 TT=252.045 T1=117.09	SAT (TO) f=1540 TT=252.165 T1=103
TravelingTournament/ TravelingTournament-a3-galaxy08_c18.xml	SAT f=2803 TT=252.13 T1=101.51	SAT (TO) f=3019 TT=252.115 T1=87.84	SAT (TO) f=2803 TT=252.13 T1=101.51	SAT (TO) f=2970 TT=252.093 T1=173.04	? (NS) TT=0.007	Signal TT=0.383	SAT (TO) f=3084 TT=252.107 T1=170.94	SAT (TO) f=3097 TT=252.119 T1=200.02	SAT (TO) f=3185 TT=252.052 T1=18.59	? (TO) TT=252.089
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
TravelingTournament/ TravelingTournament-a3-galaxy10_c18.xml	SAT f=5686 TT=252.12 T1=96.07	SAT (TO) f=5686 TT=252.12 T1=96.07	SAT (TO) f=5826 TT=252.14 T1=26.54	SAT (TO) f=6033 TT=252.152 T1=214.87	? (NS) TT=0.009	SAT (TO) f=5524 TT=251.975 T1=211.94	SAT (TO) f=5990 TT=252.04 T1=107.3	SAT (TO) f=5939 TT=252.101 T1=197.9	SAT (TO) f=6028 TT=252.085 T1=32.6	? (TO) TT=252.146
TravelingTournament/ TravelingTournament-a3-galaxy12_c18.xml	SAT f=9651 TT=252.179 T1=180.63	SAT (TO) f=9651 TT=252.179 T1=180.63	SAT (TO) f=9906 TT=252.137 T1=200.64	SAT (TO) f=9935 TT=252.052 T1=99.73	? (NS) TT=0.012	Signal TT=1.589	SAT (TO) f=10069 TT=252.138 T1=162.4	SAT (TO) f=10144 TT=252.098 T1=212.01	SAT (TO) f=10291 TT=252.117 T1=26.06	? (TO) TT=252.077
TravelingTournament/ TravelingTournament-a3-galaxy14_c18.xml	SAT f=14558 TT=252.059 T1=54.45	SAT (TO) f=14558 TT=252.059 T1=54.45	SAT (TO) f=15471 TT=252.107 T1=172.86	SAT (TO) f=15786 TT=252.112 T1=129.7	? (NS) TT=0.016	Signal TT=2.744	SAT (TO) f=16135 TT=252.104 T1=217.36	SAT (TO) f=15969 TT=252.044 T1=88.38	SAT (TO) f=16587 TT=252.101 T1=112.36	? (TO) TT=252.079
TravelingTournament/ TravelingTournament-a3-galaxy16_c18.xml	SAT f=21058 TT=252.115 T1=137.42	SAT (TO) f=21058 TT=252.115 T1=137.42	SAT (TO) f=22160 TT=252.114 T1=90.42	SAT (TO) f=22508 TT=252.098 T1=145.28	? (NS) TT=0.021	Signal TT=4.605	SAT (TO) f=22755 TT=252.079 T1=49.15	SAT (TO) f=22566 TT=252.049 T1=13.41	SAT (TO) f=23542 TT=252.047 T1=20.14	? (TO) TT=252.158
TravellingSalesman/ TravellingSalesman-10-20-00_c18.xml	OPT f=47 TT=1.926 T1=0.05	OPT f=47 TT=6.558 T1=0.69	OPT f=47 TT=12.657 T1=3.86	OPT f=47 TT=13.085 T1=4.02	OPT f=47 TT=1.926 T1=0.05	OPT f=47 TT=0.62 T1=0.01	SAT (TO) f=47 TT=252.11 T1=1.24	OPT f=47 TT=3.239 T1=1.23	OPT f=47 TT=3.319 T1=1.16	SAT (TO) f=47 TT=252.082 T1=85.94
TravellingSalesman/ TravellingSalesman-100-50-00_c18.xml	SAT f=451 TT=252.003 T1=38.22	SAT (TO) f=484 TT=252.03 T1=101.27	SAT (TO) f=858 TT=252.067 T1=112.91	SAT (TO) f=790 TT=252.062 T1=42.36	SAT (TO) f=451 TT=252.003 T1=38.22	SAT (TO) f=746 TT=251.944 T1=209.21	SAT f=1595 TT=251.879 T1=241.81	SAT f=1434 TT=251.89 T1=215.55	SAT (TO) f=563 TT=252.115 T1=243.31	? (TO) TT=260.223
TravellingSalesman/ TravellingSalesman-15-30-00_c18.xml	SAT f=93 TT=249.846 T1=0.64	SAT (TO) f=93 TT=252.03 T1=0.64	SAT (TO) f=93 TT=252.131 T1=4.63	SAT (TO) f=93 TT=252.102 T1=4.46	SAT (TO) f=93 TT=251.969 T1=6.05	SAT (TO) f=93 TT=252.006 T1=0.04	SAT f=93 TT=249.846 T1=18.26	SAT f=93 TT=250.014 T1=8.8	SAT (TO) f=93 TT=252.085 T1=1.65	SAT (TO) f=143 TT=252.132 T1=200.06
TravellingSalesman/ TravellingSalesman-150-50-00_c18.xml	SAT f=537 TT=252.034 T1=73.23	SAT (TO) f=608 TT=252.118 T1=60.49	SAT (TO) f=1428 TT=252.129 T1=91.76	SAT (TO) f=989 TT=252.143 T1=84.42	SAT (TO) f=537 TT=252.034 T1=73.23	SAT (TO) f=1240 TT=251.977 T1=245.37	SAT (TO) f=2640 TT=252.089 T1=79.3	SAT (TO) f=2900 TT=252.051 T1=163.86	SAT (TO) f=1008 TT=252.095 T1=240.5	? (TO) TT=260.193
TravellingSalesman/ TravellingSalesman-20-30-00_c18.xml	SAT f=104 TT=252.109 T1=9.29	SAT (TO) f=104 TT=252.109 T1=138.29	SAT (TO) f=104 TT=252.133 T1=9.29	SAT (TO) f=104 TT=252.126 T1=11.48	SAT (TO) f=108 TT=252.008 T1=187.26	SAT (TO) f=104 TT=252.003 T1=0.34	SAT f=119 TT=250.088 T1=64.55	SAT f=122 TT=250.087 T1=10.91	SAT (TO) f=116 TT=252.028 T1=16.85	SAT (TO) f=203 TT=252.15 T1=244.2
TravellingSalesman/ TravellingSalesman-25-30-00_c18.xml	SAT f=113 TT=252.104 T1=19.65	SAT (TO) f=121 TT=252.055 T1=218.54	SAT (TO) f=113 TT=252.123 T1=128.31	SAT (TO) f=115 TT=252.114 T1=68.99	SAT (TO) f=118 TT=252.037 T1=52.25	SAT (TO) f=129 TT=252.016 T1=18.1	SAT f=175 TT=250.432 T1=109.36	SAT f=191 TT=250.563 T1=245.94	SAT (TO) f=113 TT=252.104 T1=19.65	SAT (TO) f=284 TT=252.113 T1=78.7
Bench name	Best results	Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
TravellingSalesman/ TravellingSalesman-30-30-00_c18.xml	SAT f=126 TT=252.058 T1=61.64	SAT (TO) f=134 TT=252.126 T1=120.97	SAT (TO) f=142 TT=252.13 T1=174.6	SAT (TO) f=152 TT=252.124 T1=143.94	SAT (TO) f=132 TT=251.965 T1=6.74	SAT (TO) f=155 TT=252.001 T1=163.16	SAT f=199 TT=250.31 T1=8.57	SAT f=202 TT=250.583 T1=227.7	SAT (TO) f=126 TT=252.058 T1=61.64	SAT (TO) f=354 TT=252.164 T1=55.75
TravellingSalesman/ TravellingSalesman-35-30-00_c18.xml	SAT f=157 TT=252.05 T1=75.89	SAT (TO) f=173 TT=252.054 T1=68.55	SAT (TO) f=184 TT=252.102 T1=142.91	SAT (TO) f=194 TT=252.085 T1=228.33	SAT (TO) f=162 TT=252.008 T1=202.66	SAT (TO) f=217 TT=251.986 T1=82.61	SAT f=237 TT=250.885 T1=176.18	SAT f=245 TT=250.446 T1=12.91	SAT (TO) f=157 TT=252.05 T1=75.89	SAT (TO) f=430 TT=252.15 T1=138.29
TravellingSalesman/ TravellingSalesman-40-30-00_c18.xml	SAT f=171 TT=252.108 T1=86.24	SAT (TO) f=199 TT=252.082 T1=203.93	SAT (TO) f=221 TT=252.116 T1=82.27	SAT (TO) f=237 TT=252.067 T1=50.43	SAT (TO) f=188 TT=251.992 T1=1.08	SAT (TO) f=234 TT=252.005 T1=208.63	SAT f=291 TT=250.847 T1=64.48	SAT f=350 TT=250.862 T1=145.46	SAT (TO) f=171 TT=252.108 T1=86.24	SAT (TO) f=529 TT=252.212 T1=81
TravellingSalesman/ TravellingSalesman-45-30-00_c18.xml	SAT f=174 TT=252.072 T1=209.96	SAT (TO) f=203 TT=252.061 T1=104.72	SAT (TO) f=247 TT=252.088 T1=126.68	SAT (TO) f=223 TT=252.115 T1=237.99	SAT (TO) f=194 TT=251.959 T1=4.53	SAT (TO) f=272 TT=251.994 T1=222.63	SAT f=325 TT=251.044 T1=122.98	SAT f=349 TT=251.283 T1=220.46	SAT (TO) f=174 TT=252.072 T1=209.96	SAT (TO) f=520 TT=252.215 T1=56.64
TravellingSalesman/ TravellingSalesman-50-30-00_c18.xml	SAT f=190 TT=252.013 T1=63.5	SAT (TO) f=210 TT=252.11 T1=80.66	SAT (TO) f=259 TT=252.127 T1=148.55	SAT (TO) f=262 TT=252.065 T1=229.65	SAT (TO) f=190 TT=252.013 T1=63.5	SAT (TO) f=273 TT=251.997 T1=42.01	SAT f=399 TT=251.064 T1=89.31	SAT f=375 TT=251.119 T1=75.05	SAT (TO) f=197 TT=252.067 T1=52.59	? (TO) TT=252.076
TravellingSalesman/ TravellingSalesman-75-50-00_c18.xml	SAT f=357 TT=252.005 T1=0.59	SAT (TO) f=408 TT=252.103 T1=23.27	SAT (TO) f=626 TT=252.08 T1=105.76	SAT (TO) f=647 TT=252.053 T1=110.47	SAT (TO) f=357 TT=252.005 T1=0.59	SAT (TO) f=731 TT=251.979 T1=213.73	SAT f=1237 TT=251.665 T1=167.16	SAT f=1011 TT=251.504 T1=49.32	SAT (TO) f=440 TT=252.078 T1=234.91	? (TO) TT=260.179

Statistics		Choco-solver 4.0.7b seq (e747e1e) (complete)	Concrete 3.9.2 (complete)	Concrete 3.9.2-SuperNG (complete)	cosoco 1.12 (complete)	Mistral-2.0 2018-08-01 (complete)	OscaR - Conflict Ordering with restarts 2018-08-14 (complete)	OscaR - Conflict Ordering with restarts 2018-08-17 (complete)	OscaR - Hybrid 2018-08-14 (complete)	Sat4j-CSP 2018-07-11 (complete)
Number of times the solver is able to give the best known answer		146	151	129	107	0	39	133	139	78
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)		64	46	27	74	0	25	48	69	35