MAX-CSP 2008 Competition: results by benchmark

Results by benchmark for category binary constraints in intension (2-ARY-INT), subcategory problems with patterns

This page displays the results of the different solvers for each benchmark for category binary constraints in intension (2-ARY-INT), subcategory problems with patterns

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell example	Meaning
Answer	Solver result
f=...	value of the objective function for the model reported by the solver
TT=...	Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out
T1=...	Time to get the best model (T1): this is the wall clock time (in seconds) ellapsed when the solver found its best model. This time is obtained from the 'o lines'. It is useful to compare both complete and incomplete solvers. It doesn't take into account the time to prove optimality of the solution.

Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

Abbreviation	Meaning
f=...	Value of the objective function
TO	Time Out
MO	Mem. Out (out of memory)

Meaning of the different colors:

Color	Meaning
text	the solver cannot handle this instance
text	the solver gave no answer
text	the solver could give an answer (SAT)
text	the solver gave a definitive answer (OPTIMUM FOUND or UNSAT)
text	the solver performed better than the other ones on that instance
text	the solver was ended by a signal
text	the solver gave an incomplete answer
text	the solver was disqualified in the category
text	the solver gave a wrong answer

For better readability, you may choose to hide some solvers:
AbsconMax 112 pc-d
AbsconMax 112 pc-w
Concrete + CSP4J - MCRW Engine 2008-05-30
Concrete + CSP4J - Tabu Engine 2008-05-30
Sugar v1.13+minisat
Sugar++ v1.13+minisat-inc

Bench name	Best results on this instance	AbsconMax 112 pc-d	AbsconMax 112 pc-w	Concrete + CSP4J - MCRW Engine 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Sugar v1.13+minisat	Sugar++ v1.13+minisat-inc
csp/graphColoring/ hos/ normalized-abb313GPIA-5.xml	MSAT TO f=2828 TT=3600.01	MSAT (TO) f=3931 TT=2007.57 T1=122.08	MSAT (TO) f=3931 TT=1774.91 T1=117	MSAT (TO) f=3029 TT=3600.04	MSAT (TO) f=2828 TT=3600.01	? TT=185.855	? TT=186.228
csp/graphColoring/ hos/ normalized-ash331GPIA-3.xml	MSAT TO f=177 TT=3600.09	MSAT (TO) f=222 TT=1798.84 T1=25.03	MSAT (TO) f=222 TT=1683.08 T1=24.63	MSAT (TO) f=177 TT=3600.09	MSAT (TO) f=214 TT=3600.02	MSAT (MO) f=816 TT=2675.65 T1=128.21	? (MO) TT=61.214
csp/graphColoring/ hos/ normalized-ash608GPIA-3.xml	MSAT TO f=322 TT=3600.03	MSAT (TO) f=426 TT=1811.22 T1=32.42	MSAT (TO) f=426 TT=1692.7 T1=30.55	MSAT (TO) f=322 TT=3600.03	MSAT (TO) f=380 TT=3600.01	? (MO) TT=166.436	? (MO) TT=168.748
csp/graphColoring/ hos/ normalized-ash958GPIA-3.xml	MSAT TO f=566 TT=3600.1	MSAT (TO) f=722 TT=1945.13 T1=41.48	MSAT (TO) f=722 TT=1707.75 T1=46.09	MSAT (TO) f=566 TT=3600.1	MSAT (TO) f=629 TT=3600.03	? TT=219.453	? TT=229.134
csp/graphColoring/ hos/ normalized-will199GPIA-6.xml	MSAT TO f=1 TT=3600.1	MSAT (TO) f=2 TT=1706.31 T1=20.36	MSAT (TO) f=2 TT=1701.65 T1=20.08	MSAT (TO) f=1 TT=3600.1	MSAT (TO) f=2 TT=3600.04	? (MO) TT=135.461	? (MO) TT=134.889
csp/graphColoring/insertion/ full-insertion/ normalized-1-fullins-3-3.xml	MOPT f=2 TT=1.52 T1=0.58	OPTIMUM f=2 TT=35.91 T1=0.58 CHECKS=287304 NODES=2681	OPTIMUM f=2 TT=38.939 T1=0.64 CHECKS=412448 NODES=2543	MSAT (TO) f=2 TT=3600.01	MSAT (TO) f=2 TT=3600.05	OPTIMUM f=2 TT=1.614 T1=1.47	OPTIMUM f=2 TT=1.52 T1=1.36
csp/graphColoring/insertion/ full-insertion/ normalized-1-fullins-5-5.xml	MSAT TO f=2 TT=1650.1 T1=1.34	MSAT (TO) f=2 TT=1670.8 T1=1.34	MSAT (TO) f=2 TT=1650.1 T1=1.36	MSAT (TO) f=2 TT=3600.06	MSAT (TO) f=2 TT=3600.01	MSAT (TO) f=2 TT=3600.13 T1=336.18	MSAT (TO) f=2 TT=3600.34 T1=43.22
csp/graphColoring/insertion/ full-insertion/ normalized-2-fullins-3-4.xml	MOPT f=1 TT=2.239 T1=0.62	OPTIMUM f=1 TT=493.17 T1=0.62 CHECKS=24447100 NODES=137974	OPTIMUM f=1 TT=157.779 T1=0.68 CHECKS=7763310 NODES=49935	MSAT (TO) f=1 TT=3600.08	MSAT (TO) f=1 TT=3600.03	OPTIMUM f=1 TT=2.477 T1=2.26	OPTIMUM f=1 TT=2.239 T1=2.02
csp/graphColoring/insertion/ full-insertion/ normalized-4-fullins-4-7.xml	MSAT TO f=1 TT=1657.69 T1=3.14	MSAT (TO) f=1 TT=1906.32 T1=3.53	MSAT (TO) f=1 TT=1657.69 T1=3.14	MSAT (TO) f=1 TT=3600.08	MSAT (TO) f=1 TT=3600.05	? (MO) TT=115.371	? (MO) TT=111.735
csp/graphColoring/insertion/ full-insertion/ normalized-5-fullins-3-6.xml	MOPT f=3 TT=9.822 T1=0.88	MSAT (TO) f=3 TT=1664.9 T1=0.97	MSAT (TO) f=3 TT=1632.94 T1=0.88	MSAT (TO) f=3 TT=3600.04	MSAT (TO) f=3 TT=3600.05	OPTIMUM f=3 TT=17.368 T1=13.22	OPTIMUM f=3 TT=9.822 T1=5.45
Bench name	Best results on this instance	AbsconMax 112 pc-d	AbsconMax 112 pc-w	Concrete + CSP4J - MCRW Engine 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Sugar v1.13+minisat	Sugar++ v1.13+minisat-inc
csp/graphColoring/insertion/ k-insertion/ normalized-1-insertions-4-3.xml	MOPT f=6 TT=935.497 T1=0.64	MSAT (TO) f=6 TT=1643.66 T1=0.64	MSAT (TO) f=6 TT=1634.26 T1=0.71	MSAT (TO) f=6 TT=3600.04	MSAT (TO) f=6 TT=3600.07	OPTIMUM f=6 TT=935.497 T1=935.28	OPTIMUM f=6 TT=1033.14 T1=1032.9
csp/graphColoring/insertion/ k-insertion/ normalized-2-insertions-3-3.xml	MOPT f=1 TT=1.435 T1=0.48	OPTIMUM f=1 TT=37.27 T1=0.58 CHECKS=466336 NODES=15688	OPTIMUM f=1 TT=45.945 T1=0.48 CHECKS=932054 NODES=14793	MSAT (TO) f=1 TT=3600.08	MSAT (TO) f=1 TT=3600.03	OPTIMUM f=1 TT=1.435 T1=1.23	OPTIMUM f=1 TT=1.445 T1=1.22
csp/graphColoring/insertion/ k-insertion/ normalized-3-insertions-4-3.xml	MSAT TO f=10 TT=1640.33 T1=5.19	MSAT (TO) f=10 TT=1654.76 T1=5.62	MSAT (TO) f=10 TT=1640.33 T1=5.19	MSAT (TO) f=10 TT=3600.1	MSAT (TO) f=10 TT=3600.02	MSAT (TO) f=13 TT=3600.06 T1=567.55	MSAT (TO) f=17 TT=3600.08 T1=318.32
csp/graphColoring/insertion/ k-insertion/ normalized-3-insertions-5-3.xml	MSAT TO f=100 TT=3600.06	MSAT (TO) f=112 TT=1734.34 T1=13.12	MSAT (TO) f=112 TT=1711.98 T1=14.03	MSAT (TO) f=100 TT=3600.06	MSAT (TO) f=111 TT=3600.05	? TT=199.057	? TT=293.808
csp/graphColoring/insertion/ k-insertion/ normalized-4-insertions-4-3.xml	MSAT TO f=12 TT=1658.93 T1=6.54	MSAT (TO) f=12 TT=1690.9 T1=6.54	MSAT (TO) f=12 TT=1658.93 T1=7.02	MSAT (TO) f=12 TT=3600.04	MSAT (TO) f=12 TT=3600.02	MSAT (TO) f=30 TT=3600.07 T1=108.73	MSAT (TO) f=28 TT=3600.22 T1=107.19
csp/graphColoring/leighton/ leighton-15/ normalized-lei450-15b-08.xml	MSAT TO f=340 TT=3600.1	MSAT (TO) f=414 TT=1675.89 T1=41.29	MSAT (TO) f=414 TT=1682.94 T1=41.01	MSAT (TO) f=377 TT=3600.04	MSAT (TO) f=340 TT=3600.1	? (MO) TT=176.686	? (MO) TT=176.269
csp/graphColoring/leighton/ leighton-15/ normalized-lei450-15b-10.xml	MSAT TO f=185 TT=3600.09	MSAT (TO) f=241 TT=1720.72 T1=100.24	MSAT (TO) f=241 TT=1722.17 T1=102.5	MSAT (TO) f=207 TT=3600.05	MSAT (TO) f=185 TT=3600.09	? (MO) TT=169.486	? (MO) TT=178.224
csp/graphColoring/leighton/ leighton-15/ normalized-lei450-15c-13.xml	MSAT TO f=183 TT=3600.1	MSAT (TO) f=395 TT=1710.68 T1=92.54	MSAT (TO) f=395 TT=1715.63 T1=92.37	MSAT (TO) f=183 TT=3600.1	MSAT (TO) f=285 TT=3600.01	? TT=241.212	? TT=253.45
csp/graphColoring/leighton/ leighton-15/ normalized-lei450-15d-05.xml	MSAT TO f=2043 TT=3600.06	MSAT (TO) f=2264 TT=1702.02 T1=67.83	MSAT (TO) f=2264 TT=1718.63 T1=77.4	MSAT (TO) f=2043 TT=3600.06	MSAT (TO) f=2093 TT=3600.04	? TT=218.313	? TT=572.508
csp/graphColoring/leighton/ leighton-15/ normalized-lei450-15d-08.xml	MSAT TO f=892 TT=3600.05	MSAT (TO) f=1114 TT=1707 T1=73.94	MSAT (TO) f=1114 TT=1725.8 T1=73.87	MSAT (TO) f=892 TT=3600.05	MSAT (TO) f=927 TT=3600.08	? TT=215.201	? TT=221.21
Bench name	Best results on this instance	AbsconMax 112 pc-d	AbsconMax 112 pc-w	Concrete + CSP4J - MCRW Engine 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Sugar v1.13+minisat	Sugar++ v1.13+minisat-inc
csp/graphColoring/leighton/ leighton-25/ normalized-lei450-25a-15.xml	MSAT TO f=104 TT=3600.07	MSAT (TO) f=145 TT=1679.68 T1=88.69	MSAT (TO) f=145 TT=1695.88 T1=88.55	MSAT (TO) f=119 TT=3600.07	MSAT (TO) f=104 TT=3600.07	? (MO) TT=192.686	? (MO) TT=193.387
csp/graphColoring/leighton/ leighton-25/ normalized-lei450-25a-23.xml	MSAT TO f=13 TT=3600.05	MSAT (TO) f=23 TT=1651.98 T1=10.24	MSAT (TO) f=23 TT=1636.98 T1=10.81	MSAT (TO) f=13 TT=3600.05	MSAT (TO) f=13 TT=3600.06	? (MO) TT=198.132	? (MO) TT=188.238
csp/graphColoring/leighton/ leighton-25/ normalized-lei450-25b-15.xml	MSAT TO f=109 TT=3600.01	MSAT (TO) f=148 TT=1702.58 T1=65.3	MSAT (TO) f=148 TT=1713.69 T1=69.77	MSAT (TO) f=126 TT=3600	MSAT (TO) f=109 TT=3600.01	? (MO) TT=179.584	? (MO) TT=174.709
csp/graphColoring/leighton/ leighton-25/ normalized-lei450-25b-24.xml	MSAT TO f=6 TT=3600.01	MSAT (TO) f=9 TT=1644.26 T1=12.58	MSAT (TO) f=9 TT=1640.59 T1=12.41	MSAT (TO) f=6 TT=3600.01	MSAT (TO) f=6 TT=3600.06	? (MO) TT=180.605	? (MO) TT=188.198
csp/graphColoring/leighton/ leighton-25/ normalized-lei450-25c-05.xml	MSAT TO f=2408 TT=3600.02	MSAT (TO) f=2569 TT=1761.46 T1=144.39	MSAT (TO) f=2569 TT=1757.59 T1=132.85	MSAT (TO) f=2412 TT=3600.04	MSAT (TO) f=2408 TT=3600.02	? TT=167.749	? TT=174.454
csp/graphColoring/leighton/ leighton-5/ normalized-lei450-05a-04.xml	MSAT TO f=393 TT=3600.08	MSAT (TO) f=444 TT=1690.83 T1=26.14	MSAT (TO) f=444 TT=1681.31 T1=24.85	MSAT (TO) f=393 TT=3600.08	MSAT (TO) f=401 TT=3600.08	? (MO) TT=94.43	? (MO) TT=96.393
csp/graphColoring/ mug/ normalized-mug100-1-3.xml	MOPT f=1 TT=2.174 T1=0.64	MSAT (TO) f=1 TT=1749.94 T1=0.7	MSAT (TO) f=1 TT=1707.26 T1=0.64	MSAT (TO) f=1 TT=3600	MSAT (TO) f=1 TT=3600.04	OPTIMUM f=1 TT=2.174 T1=1.96	OPTIMUM f=1 TT=2.452 T1=2
csp/graphColoring/ mug/ normalized-mug100-25-3.xml	MOPT f=1 TT=2.045 T1=0.64	MSAT (TO) f=1 TT=1713.9 T1=0.67	MSAT (TO) f=1 TT=1766.35 T1=0.64	MSAT (TO) f=1 TT=3600.06	MSAT (TO) f=1 TT=3600.01	OPTIMUM f=1 TT=2.21 T1=2	OPTIMUM f=1 TT=2.045 T1=1.85
csp/graphColoring/ mug/ normalized-mug88-1-3.xml	MOPT f=1 TT=1.879 T1=0.59	MSAT (TO) f=1 TT=1720.47 T1=0.59	MSAT (TO) f=1 TT=1727.56 T1=0.68	MSAT (TO) f=1 TT=3600.07	MSAT (TO) f=1 TT=3600.04	OPTIMUM f=1 TT=2.007 T1=1.88	OPTIMUM f=1 TT=1.879 T1=1.69
csp/graphColoring/ mug/ normalized-mug88-25-3.xml	MOPT f=1 TT=1.857 T1=0.67	MSAT (TO) f=1 TT=1724.15 T1=0.68	MSAT (TO) f=1 TT=1721.03 T1=0.67	MSAT (TO) f=1 TT=3600.03	MSAT (TO) f=1 TT=3600.07	OPTIMUM f=1 TT=2.026 T1=1.84	OPTIMUM f=1 TT=1.857 T1=1.67
Bench name	Best results on this instance	AbsconMax 112 pc-d	AbsconMax 112 pc-w	Concrete + CSP4J - MCRW Engine 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Sugar v1.13+minisat	Sugar++ v1.13+minisat-inc
csp/graphColoring/ myciel/ normalized-myciel4-4.xml	MOPT f=1 TT=1.439 T1=0.55	OPTIMUM f=1 TT=35.341 T1=0.55 CHECKS=312660 NODES=6244	OPTIMUM f=1 TT=41.836 T1=0.57 CHECKS=586229 NODES=5261	MSAT (TO) f=1 TT=3600.02	MSAT (TO) f=1 TT=3600.05	OPTIMUM f=1 TT=1.439 T1=1.23	OPTIMUM f=1 TT=4.047 T1=3.82
csp/graphColoring/ myciel/ normalized-myciel5-4.xml	MOPT f=4 TT=736.879 T1=0.6	MSAT (TO) f=4 TT=1632.26 T1=0.79	MSAT (TO) f=4 TT=1624.61 T1=0.6	MSAT (TO) f=4 TT=3600.01	MSAT (TO) f=4 TT=3600.03	OPTIMUM f=4 TT=736.879 T1=736.75	OPTIMUM f=4 TT=903.786 T1=903.71
csp/graphColoring/ myciel/ normalized-myciel5-5.xml	MOPT f=1 TT=93.888 T1=0.58	MSAT (TO) f=1 TT=1648.38 T1=0.63	MSAT (TO) f=1 TT=1652.28 T1=0.58	MSAT (TO) f=1 TT=3600.08	MSAT (TO) f=1 TT=3600.02	OPTIMUM f=1 TT=93.888 T1=2.71	OPTIMUM f=1 TT=435.02 T1=2.13
csp/graphColoring/ myciel/ normalized-myciel7-4.xml	MSAT TO f=56 TT=1662.61 T1=25.06	MSAT (TO) f=56 TT=1667.8 T1=25.06	MSAT (TO) f=56 TT=1662.61 T1=25.77	MSAT (TO) f=56 TT=3600.08	MSAT (TO) f=56 TT=3600.03	MSAT (TO) f=200 TT=3600.08 T1=70.71	MSAT (TO) f=224 TT=3600.24 T1=24.17
csp/graphColoring/ myciel/ normalized-myciel7-5.xml	MSAT TO f=16 TT=1645.12 T1=16.91	MSAT (TO) f=16 TT=1645.12 T1=17.9	MSAT (TO) f=16 TT=1667.44 T1=16.91	MSAT (TO) f=16 TT=3600.07	MSAT (TO) f=16 TT=3600.1	MSAT (TO) f=42 TT=3600.05 T1=468.71	MSAT (TO) f=50 TT=3600.15 T1=990.95
csp/graphColoring/ register/fpsol/ normalized-fpsol2-i-1-55.xml	MSAT TO f=10 TT=1686.28 T1=61.45	MSAT (TO) f=10 TT=1686.28 T1=65.53	MSAT (TO) f=10 TT=1704.13 T1=61.45	MSAT (TO) f=10 TT=3600.09	MSAT (TO) f=15 TT=3600	? TT=219.793	? TT=212.651
csp/graphColoring/ register/fpsol/ normalized-fpsol2-i-2-10.xml	MSAT TO f=103 TT=3600	MSAT (TO) f=115 TT=1665.58 T1=11.96	MSAT (TO) f=115 TT=1658.75 T1=11.19	MSAT (TO) f=103 TT=3600	MSAT (TO) f=116 TT=3600.06	? TT=183.93	? TT=198.09
csp/graphColoring/ register/fpsol/ normalized-fpsol2-i-2-15.xml	MSAT TO f=44 TT=3600.06	MSAT (TO) f=47 TT=1634.45 T1=16.29	MSAT (TO) f=47 TT=1667.35 T1=17.35	MSAT (TO) f=44 TT=3600.06	MSAT (TO) f=64 TT=3600.01	? TT=203.605	? TT=188.678
csp/graphColoring/ register/fpsol/ normalized-fpsol2-i-2-27.xml	MSAT TO f=5 TT=1674.19 T1=89.04	MSAT (TO) f=5 TT=1684.8 T1=89.04	MSAT (TO) f=5 TT=1674.19 T1=91.12	MSAT (TO) f=5 TT=3600.01	MSAT (TO) f=24 TT=3600.04	? TT=186.181	? TT=199.154
csp/graphColoring/ register/fpsol/ normalized-fpsol2-i-3-27.xml	MSAT TO f=5 TT=1672.91 T1=56.65	MSAT (TO) f=5 TT=1699.49 T1=56.65	MSAT (TO) f=5 TT=1672.91 T1=56.7	MSAT (TO) f=5 TT=3600.02	MSAT (TO) f=26 TT=3600.03	? TT=174.55	? TT=190.319
Bench name	Best results on this instance	AbsconMax 112 pc-d	AbsconMax 112 pc-w	Concrete + CSP4J - MCRW Engine 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Sugar v1.13+minisat	Sugar++ v1.13+minisat-inc
csp/graphColoring/ register/inithx/ normalized-inithx-i-1-10.xml	MSAT TO f=307 TT=3600	MSAT (TO) f=406 TT=1709.61 T1=27.72	MSAT (TO) f=406 TT=1678.72 T1=27.79	MSAT (TO) f=307 TT=3600	MSAT (TO) f=341 TT=3600	? TT=177.515	? TT=174.692
csp/graphColoring/ register/inithx/ normalized-inithx-i-1-45.xml	MSAT TO f=9 TT=1781.18 T1=1781.12	MSAT (TO) f=14 TT=1738.01 T1=120.06	MSAT (TO) f=9 TT=1781.18 T1=1781.12	MSAT (TO) f=9 TT=3600.05	MSAT (TO) f=30 TT=3600.07	? TT=167.823	? TT=157.422
csp/graphColoring/ register/inithx/ normalized-inithx-i-2-15.xml	MSAT TO f=47 TT=3600.02	MSAT (TO) f=50 TT=1712.78 T1=54.85	MSAT (TO) f=50 TT=1714.1 T1=51.45	MSAT (TO) f=47 TT=3600.02	MSAT (TO) f=128 TT=3600	? TT=196.107	? TT=194.252
csp/graphColoring/ register/inithx/ normalized-inithx-i-2-20.xml	MSAT TO f=20 TT=3600	MSAT (TO) f=37 TT=1684.28 T1=70.77	MSAT (TO) f=21 TT=1781.89 T1=1781.84	MSAT (TO) f=20 TT=3600	MSAT (TO) f=91 TT=3600.07	? TT=196.206	? TT=183.92
csp/graphColoring/ register/inithx/ normalized-inithx-i-3-28.xml	MSAT TO f=5 TT=1705.38 T1=1705.33	MSAT (TO) f=19 TT=1707.15 T1=80.47	MSAT (TO) f=5 TT=1705.38 T1=1705.33	MSAT (TO) f=6 TT=3600.01	MSAT (TO) f=54 TT=3600.08	? TT=197.508	? TT=194.102
csp/graphColoring/ register/mulsol/ normalized-mulsol-i-1-05.xml	MSAT TO f=389 TT=3600.02	MSAT (TO) f=420 TT=1622.41 T1=2.02	MSAT (TO) f=420 TT=1639.86 T1=2.19	MSAT (TO) f=389 TT=3600.02	MSAT (TO) f=389 TT=3600.06	MSAT (TO) f=661 TT=3600.3 T1=116.73	No Cert. TT=68.267
csp/graphColoring/ register/mulsol/ normalized-mulsol-i-1-25.xml	MSAT TO f=33 TT=3600.08	MSAT (TO) f=34 TT=1648.81 T1=34.38	MSAT (TO) f=34 TT=1690.57 T1=34.4	MSAT (TO) f=33 TT=3600.08	MSAT (TO) f=34 TT=3600.02	MSAT (TO) f=58 TT=3600.36 T1=220.06	MSAT (MO) f=244 TT=50.334 T1=49.61
csp/graphColoring/ register/mulsol/ normalized-mulsol-i-2-05.xml	MSAT TO f=223 TT=3600	MSAT (TO) f=254 TT=1666.07 T1=21.18	MSAT (TO) f=254 TT=1653.93 T1=19.86	MSAT (TO) f=223 TT=3600	MSAT (TO) f=223 TT=3600.08	MSAT (TO) f=697 TT=3600.32 T1=111.29	MSAT (TO) f=517 TT=3600.41 T1=79.79
csp/graphColoring/ register/mulsol/ normalized-mulsol-i-2-20.xml	MSAT TO f=22 TT=3568.04 T1=3567.91	MSAT (TO) f=23 TT=1643.28 T1=5.45	MSAT (TO) f=22 TT=3568.04 T1=3567.91	MSAT (TO) f=22 TT=3600.05	MSAT (TO) f=23 TT=3600.05	MSAT (TO) f=37 TT=3600.28 T1=348.6	MSAT (MO) f=307 TT=53.979 T1=50.98
csp/graphColoring/ register/mulsol/ normalized-mulsol-i-5-10.xml	MSAT TO f=77 TT=3600.05	MSAT (TO) f=84 TT=1640.48 T1=5.67	MSAT (TO) f=84 TT=1650.91 T1=6.05	MSAT (TO) f=77 TT=3600.05	MSAT (TO) f=77 TT=3600.07	MSAT (TO) f=167 TT=3600.28 T1=776.3	MSAT (MO) f=616 TT=52.689 T1=50.35
Bench name	Best results on this instance	AbsconMax 112 pc-d	AbsconMax 112 pc-w	Concrete + CSP4J - MCRW Engine 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Sugar v1.13+minisat	Sugar++ v1.13+minisat-inc
csp/graphColoring/ register/zeroin/ normalized-zeroin-i-1-45.xml	MSAT TO f=4 TT=1636.76 T1=7.63	MSAT (TO) f=4 TT=1643.49 T1=7.63	MSAT (TO) f=4 TT=1636.76 T1=7.66	MSAT (TO) f=4 TT=3600.08	MSAT (TO) f=7 TT=3600.08	MSAT (TO) f=7 TT=3600.26 T1=3003.81	No Cert. TT=57.727
csp/graphColoring/ register/zeroin/ normalized-zeroin-i-2-28.xml	MSAT TO f=2 TT=1646.92 T1=2.85	MSAT (TO) f=2 TT=1646.98 T1=2.85	MSAT (TO) f=2 TT=1646.92 T1=3.14	MSAT (TO) f=2 TT=3600.08	MSAT (TO) f=8 TT=3600.09	MSAT (TO) f=4 TT=3600.24 T1=581.8	MSAT (TO) f=3 TT=3600.31 T1=49.31
csp/graphColoring/ school/ normalized-school1-13.xml	MSAT TO f=126 TT=3600.01	MSAT (TO) f=159 TT=1688.65 T1=25.59	MSAT (TO) f=159 TT=1678.1 T1=26.57	MSAT (TO) f=126 TT=3600.01	MSAT (TO) f=126 TT=3600.05	? TT=170.977	? TT=176.968
csp/graphColoring/ school/ normalized-school1-nsh-12.xml	MSAT TO f=177 TT=3600.08	MSAT (TO) f=243 TT=1673.68 T1=26.99	MSAT (TO) f=243 TT=1674.98 T1=28.51	MSAT (TO) f=177 TT=3600.08	MSAT (TO) f=177 TT=3600.09	? TT=208.663	? TT=201.168
csp/graphColoring/ school/ normalized-school1-nsh-13.xml	MSAT TO f=80 TT=3600.01	MSAT (TO) f=101 TT=1675.14 T1=37.3	MSAT (TO) f=101 TT=1702.25 T1=35.17	MSAT (TO) f=80 TT=3600.03	MSAT (TO) f=80 TT=3600.01	? TT=204.962	? TT=202.635
csp/graphColoring/ sgb/book/ normalized-anna-9.xml	MOPT f=2 TT=22.813 T1=0.81	MSAT (TO) f=2 TT=1628.02 T1=0.81	MSAT (TO) f=2 TT=1637.66 T1=0.87	MSAT (TO) f=2 TT=3600.06	MSAT (TO) f=2 TT=3600.02	OPTIMUM f=2 TT=24.803 T1=4.5	OPTIMUM f=2 TT=22.813 T1=3.62
csp/graphColoring/ sgb/book/ normalized-david-5.xml	MSAT TO f=20 TT=3600	MSAT (TO) f=21 TT=1613.85 T1=1.28	MSAT (TO) f=21 TT=1627.11 T1=1.36	MSAT (TO) f=20 TT=3600	MSAT (TO) f=20 TT=3600	MSAT (TO) f=35 TT=3600.04 T1=2.1	MSAT (TO) f=25 TT=3600.1 T1=199.04
csp/graphColoring/ sgb/book/ normalized-homer-12.xml	MOPT f=1 TT=163.587 T1=1.17	MSAT (TO) f=1 TT=1614.23 T1=1.17	MSAT (TO) f=1 TT=1735.3 T1=1.27	MSAT (TO) f=1 TT=3600.03	MSAT (TO) f=1 TT=3600.09	OPTIMUM f=1 TT=170.487 T1=62.09	OPTIMUM f=1 TT=163.587 T1=12.89
csp/graphColoring/ sgb/book/ normalized-huck-10.xml	MOPT f=1 TT=5.681 T1=0.67	MSAT (TO) f=1 TT=1631.26 T1=0.77	MSAT (TO) f=1 TT=1627.01 T1=0.67	MSAT (TO) f=1 TT=3600.03	MSAT (TO) f=1 TT=3600.08	OPTIMUM f=1 TT=5.681 T1=3.11	OPTIMUM f=1 TT=5.716 T1=2.63
csp/graphColoring/ sgb/book/ normalized-jean-5.xml	MSAT TO f=14 TT=1621.42 T1=2.54	MSAT (TO) f=14 TT=1645.87 T1=2.54	MSAT (TO) f=14 TT=1621.42 T1=2.55	MSAT (TO) f=14 TT=3600.1	MSAT (TO) f=14 TT=3600	MSAT (TO) f=16 TT=3600.07 T1=2.24	MSAT (TO) f=19 TT=3600.1 T1=1.7
Bench name	Best results on this instance	AbsconMax 112 pc-d	AbsconMax 112 pc-w	Concrete + CSP4J - MCRW Engine 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Sugar v1.13+minisat	Sugar++ v1.13+minisat-inc
csp/graphColoring/sgb/games/ normalized-games120-8.xml	MOPT f=2 TT=5.095 T1=0.84	MSAT (TO) f=2 TT=1631.37 T1=0.84	MSAT (TO) f=2 TT=1655.18 T1=0.88	MSAT (TO) f=2 TT=3600.03	MSAT (TO) f=2 TT=3600.03	OPTIMUM f=2 TT=9.558 T1=8.31	OPTIMUM f=2 TT=5.095 T1=4.72
csp/graphColoring/ sgb/miles/ normalized-miles1500-30.xml	MSAT TO f=58 TT=3600.01	MSAT (TO) f=68 TT=1624.14 T1=1.83	MSAT (TO) f=68 TT=1644.95 T1=2.07	MSAT (TO) f=61 TT=3600.01	MSAT (TO) f=58 TT=3600.01	? (MO) TT=77.25	? (MO) TT=81.259
csp/graphColoring/ sgb/miles/ normalized-miles500-10.xml	MSAT TO f=18 TT=3600.08	MSAT (TO) f=21 TT=1618.47 T1=3.18	MSAT (TO) f=21 TT=1632.42 T1=2.94	MSAT (TO) f=19 TT=3600.03	MSAT (TO) f=18 TT=3600.08	MSAT (TO) f=34 TT=3600.05 T1=22.81	MSAT (TO) f=31 TT=3600.1 T1=220.87
csp/graphColoring/ sgb/miles/ normalized-miles500-15.xml	MSAT TO f=5 TT=1608.2 T1=1.36	MSAT (TO) f=5 TT=1608.2 T1=1.47	MSAT (TO) f=5 TT=1618.87 T1=1.36	MSAT (TO) f=5 TT=3600.03	MSAT (TO) f=5 TT=3600.06	MSAT (TO) f=6 TT=3600.03 T1=275.02	MSAT (TO) f=5 TT=3600.15 T1=1782.87
csp/graphColoring/ sgb/miles/ normalized-miles750-15.xml	MSAT TO f=28 TT=3600.02	MSAT (TO) f=33 TT=1645.07 T1=32.38	MSAT (TO) f=33 TT=1659.5 T1=32.26	MSAT (TO) f=29 TT=3600.01	MSAT (TO) f=28 TT=3600.02	MSAT (TO) f=62 TT=3600.09 T1=39.59	MSAT (TO) f=60 TT=3600.13 T1=20.34
csp/graphColoring/ sgb/miles/ normalized-miles750-29.xml	MSAT TO f=2 TT=1616.38 T1=1.35	MSAT (TO) f=2 TT=1624.85 T1=1.35	MSAT (TO) f=2 TT=1616.38 T1=1.47	MSAT (TO) f=2 TT=3600.02	MSAT (TO) f=2 TT=3600.04	MSAT (TO) f=2 TT=3600.04 T1=92.91	MSAT (TO) f=2 TT=3600.2 T1=22.39
csp/graphColoring/ sgb/queen/ normalized-queen12-12-11.xml	MSAT TO f=31 TT=3600.09	MSAT (TO) f=34 TT=1630.33 T1=13.91	MSAT (TO) f=34 TT=1627.5 T1=14.17	MSAT (TO) f=32 TT=3600.09	MSAT (TO) f=31 TT=3600.09	MSAT (MO) f=120 TT=3442.62 T1=59.33	MSAT (MO) f=97 TT=2944.87 T1=24.41
csp/graphColoring/ sgb/queen/ normalized-queen8-12-10.xml	MSAT TO f=16 TT=1618.64 T1=3.46	MSAT (TO) f=16 TT=1618.64 T1=3.46	MSAT (TO) f=16 TT=1639.77 T1=3.52	MSAT (TO) f=16 TT=3600.05	MSAT (TO) f=16 TT=3600.08	MSAT (TO) f=33 TT=3600.01 T1=625.43	MSAT (TO) f=51 TT=3600.19 T1=265.9
csp/haystacks/ normalized-haystacks-05.xml	MOPT f=1 TT=1.211 T1=1.07	OPTIMUM f=1 TT=1070.06 T1=1.08 CHECKS=91602700 NODES=1297980	OPTIMUM f=1 TT=846.722 T1=1.13 CHECKS=54165700 NODES=642818	MSAT (TO) f=1 TT=3600.06	MSAT (TO) f=1 TT=3600.09	OPTIMUM f=1 TT=1.211 T1=1.07	OPTIMUM f=1 TT=1.23 T1=1.08
csp/haystacks/ normalized-haystacks-08.xml	MOPT f=1 TT=2.722 T1=2.33	MSAT (TO) f=1 TT=1661.96 T1=4.59	MSAT (TO) f=1 TT=1650.67 T1=4.96	MSAT (TO) f=1 TT=3600.02	MSAT (TO) f=1 TT=3600.1	OPTIMUM f=1 TT=3.089 T1=2.71	OPTIMUM f=1 TT=2.722 T1=2.33
Bench name	Best results on this instance	AbsconMax 112 pc-d	AbsconMax 112 pc-w	Concrete + CSP4J - MCRW Engine 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Sugar v1.13+minisat	Sugar++ v1.13+minisat-inc
csp/haystacks/ normalized-haystacks-10.xml	MOPT f=1 TT=6.646 T1=3.69	MSAT (TO) f=1 TT=1658.8 T1=11.66	MSAT (TO) f=1 TT=1650.87 T1=11.76	MSAT (TO) f=1 TT=3600.06	MSAT (TO) f=1 TT=3600.01	OPTIMUM f=1 TT=8.474 T1=5.88	OPTIMUM f=1 TT=6.646 T1=3.69
csp/haystacks/ normalized-haystacks-11.xml	MOPT f=1 TT=15.76 T1=2.35	MSAT (TO) f=1 TT=1642.39 T1=2.56	MSAT (TO) f=1 TT=1766.57 T1=2.35	MSAT (TO) f=1 TT=3600.08	MSAT (TO) f=1 TT=3600.02	OPTIMUM f=1 TT=17.67 T1=8.06	OPTIMUM f=1 TT=15.76 T1=4.58
csp/haystacks/ normalized-haystacks-12.xml	MOPT f=1 TT=70.381 T1=4.45	MSAT (TO) f=1 TT=1646.57 T1=4.45	MSAT (TO) f=1 TT=1657.11 T1=4.53	MSAT (TO) f=1 TT=3600.05	MSAT (TO) f=1 TT=3600.08	OPTIMUM f=1 TT=70.381 T1=13.16	OPTIMUM f=1 TT=87.328 T1=5.26
csp/haystacks/ normalized-haystacks-15.xml	MSAT TO f=1 TT=1639.09 T1=1.34	MSAT (TO) f=1 TT=1650.02 T1=1.34	MSAT (TO) f=1 TT=1639.09 T1=1.37	MSAT (TO) f=1 TT=3600.04	MSAT (TO) f=1 TT=3600.06	MSAT (TO) f=1 TT=3600.06 T1=74.36	MSAT (TO) f=1 TT=3600.24 T1=17
csp/haystacks/ normalized-haystacks-16.xml	MSAT TO f=1 TT=1661.73 T1=12.28	MSAT (TO) f=1 TT=1662.6 T1=12.35	MSAT (TO) f=1 TT=1661.73 T1=12.28	MSAT (TO) f=1 TT=3600.04	MSAT (TO) f=1 TT=3600.07	MSAT (TO) f=1 TT=3600.06 T1=119.84	MSAT (TO) f=1 TT=3600.2 T1=42.63
csp/haystacks/ normalized-haystacks-20.xml	MSAT TO f=1 TT=1677.64 T1=23.49	MSAT (TO) f=1 TT=1690.57 T1=23.49	MSAT (TO) f=1 TT=1677.64 T1=25.04	MSAT (TO) f=1 TT=3600.03	MSAT (TO) f=1 TT=3600	MSAT (TO) f=1 TT=3600.27 T1=949.79	No Cert. TT=129.979
csp/haystacks/ normalized-haystacks-29.xml	MSAT TO f=1 TT=3600.02	MSAT (TO) f=2 TT=1659.91 T1=7.49	MSAT (TO) f=2 TT=1721.24 T1=6.75	MSAT (TO) f=1 TT=3600.03	MSAT (TO) f=1 TT=3600.02	? TT=233.223	? TT=247.919
csp/haystacks/ normalized-haystacks-30.xml	MSAT TO f=1 TT=1691.9 T1=28.29	MSAT (TO) f=1 TT=1691.9 T1=28.51	MSAT (TO) f=1 TT=1810.18 T1=28.29	MSAT (TO) f=1 TT=3600.07	MSAT (TO) f=1 TT=3600.01	? TT=177.04	? TT=183.826
csp/haystacks/ normalized-haystacks-35.xml	MSAT TO f=1 TT=1731.63 T1=22.53	MSAT (TO) f=1 TT=1770.9 T1=22.77	MSAT (TO) f=1 TT=1731.63 T1=22.53	MSAT (TO) f=1 TT=3600.02	MSAT (TO) f=2 TT=3600.03	? TT=241.426	? TT=213.555
csp/haystacks/ normalized-haystacks-36.xml	MSAT TO f=1 TT=3600.03	MSAT (TO) f=2 TT=1727.45 T1=26.29	MSAT (TO) f=2 TT=1752.62 T1=24.22	MSAT (TO) f=1 TT=3600.03	MSAT (TO) f=2 TT=3600.04	? TT=221.702	? TT=207.488
Bench name	Best results on this instance	AbsconMax 112 pc-d	AbsconMax 112 pc-w	Concrete + CSP4J - MCRW Engine 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Sugar v1.13+minisat	Sugar++ v1.13+minisat-inc
csp/haystacks/ normalized-haystacks-43.xml	MSAT TO f=1 TT=3600.02	? TT=90.091	? TT=98.047	MSAT (TO) f=1 TT=3600.02	MSAT (TO) f=2 TT=3600.04	? TT=230.704	? TT=224.265
csp/js-taillard-15/ normalized-js-taillard-15-95-2.xml	MOPT f=2 TT=1098.43 T1=1098.14	? TT=165.866	? TT=165.749	MSAT (TO) f=18 TT=3600.01	MSAT (TO) f=22 TT=3600.03	OPTIMUM f=2 TT=1506.77 T1=1506.58	OPTIMUM f=2 TT=1098.43 T1=1098.14
csp/js-taillard-15/ normalized-js-taillard-15-95-4.xml	MSAT TO f=2 TT=3600.16 T1=304.62	? TT=188.418	? TT=188.621	MSAT (TO) f=17 TT=3600.07	MSAT (TO) f=22 TT=3600.09	MSAT (TO) f=2 TT=3600.16 T1=304.62	MSAT (TO) f=2 TT=3600.29 T1=712.07
csp/js-taillard-20/ normalized-js-taillard-20-95-7.xml	MSAT TO f=37 TT=3600.09	? TT=131.296	? TT=123.843	MSAT (TO) f=42 TT=3600.08	MSAT (TO) f=37 TT=3600.09	? (MO) TT=78.67	? (MO) TT=85.708
csp/os-gp/ normalized-gp10-01-1092.xml	MOPT f=1 TT=17.798 T1=16.82	? TT=84.444	? TT=87.821	MSAT (TO) f=8 TT=3600.09	MSAT (TO) f=10 TT=3600.07	OPTIMUM f=1 TT=48.585 T1=43.18	OPTIMUM f=1 TT=17.798 T1=16.82
csp/os-gp/ normalized-gp10-02-1096.xml	MOPT f=1 TT=14.577 T1=13.95	? TT=73.275	? TT=74.891	MSAT (TO) f=7 TT=3600.05	MSAT (TO) f=11 TT=3600	OPTIMUM f=1 TT=45.645 T1=42.02	OPTIMUM f=1 TT=14.577 T1=13.95
csp/os-gp/ normalized-gp10-03-1080.xml	MOPT f=1 TT=21.037 T1=15.89	? TT=98.459	? TT=95.007	MSAT (TO) f=8 TT=3600.08	MSAT (TO) f=11 TT=3600.08	OPTIMUM f=1 TT=51.569 T1=43.51	OPTIMUM f=1 TT=21.037 T1=15.89
csp/os-gp/ normalized-gp10-04-1076.xml	MOPT f=1 TT=16.887 T1=15.91	? TT=111.292	? TT=105.801	MSAT (TO) f=8 TT=3600.02	MSAT (TO) f=11 TT=3600.08	OPTIMUM f=1 TT=46.365 T1=42.24	OPTIMUM f=1 TT=16.887 T1=15.91
csp/os-gp/ normalized-gp10-05-1070.xml	MOPT f=1 TT=27.825 T1=18.08	? TT=76.84	? TT=80.296	MSAT (TO) f=9 TT=3600.01	MSAT (TO) f=12 TT=3600.04	OPTIMUM f=1 TT=67.201 T1=53.08	OPTIMUM f=1 TT=27.825 T1=18.08
csp/os-gp/ normalized-gp10-06-1070.xml	MOPT f=1 TT=16.719 T1=15.65	? TT=77.407	? TT=81.241	MSAT (TO) f=7 TT=3600.04	MSAT (TO) f=11 TT=3600.08	OPTIMUM f=1 TT=51.194 T1=46.04	OPTIMUM f=1 TT=16.719 T1=15.65
Bench name	Best results on this instance	AbsconMax 112 pc-d	AbsconMax 112 pc-w	Concrete + CSP4J - MCRW Engine 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Sugar v1.13+minisat	Sugar++ v1.13+minisat-inc
csp/os-gp/ normalized-gp10-07-1078.xml	MOPT f=1 TT=17.835 T1=17.41	? TT=114.973	? TT=115.004	MSAT (TO) f=8 TT=3600.09	MSAT (TO) f=11 TT=3600.07	OPTIMUM f=1 TT=45.617 T1=41.43	OPTIMUM f=1 TT=17.835 T1=17.41
csp/os-gp/ normalized-gp10-08-1092.xml	MOPT f=1 TT=17.356 T1=16.04	? TT=80.731	? TT=77.314	MSAT (TO) f=7 TT=3600.02	MSAT (TO) f=10 TT=3600.03	OPTIMUM f=1 TT=45.893 T1=41.48	OPTIMUM f=1 TT=17.356 T1=16.04
csp/os-gp/ normalized-gp10-09-1111.xml	MOPT f=1 TT=14.631 T1=13.97	? TT=98.964	? TT=94.239	MSAT (TO) f=6 TT=3600.06	MSAT (TO) f=11 TT=3600.07	OPTIMUM f=1 TT=45.144 T1=41.37	OPTIMUM f=1 TT=14.631 T1=13.97
csp/os-gp/ normalized-gp10-10-1091.xml	MOPT f=1 TT=16.095 T1=14.64	? TT=88.96	? TT=85.297	MSAT (TO) f=7 TT=3600.04	MSAT (TO) f=11 TT=3600.04	OPTIMUM f=1 TT=45.62 T1=41.11	OPTIMUM f=1 TT=16.095 T1=14.64
csp/os-taillard-10/ normalized-os-taillard-10-95-1.xml	MOPT f=4 TT=210.432 T1=192.33	? TT=92.94	? TT=91.691	MSAT (TO) f=8 TT=3600.07	MSAT (TO) f=9 TT=3600.01	OPTIMUM f=4 TT=1045.37 T1=628.98	OPTIMUM f=4 TT=210.432 T1=192.33
csp/os-taillard-10/ normalized-os-taillard-10-95-3.xml	MOPT f=4 TT=2152.9 T1=2152.55	? TT=89.26	? TT=60.445	MSAT (TO) f=9 TT=3600.08	MSAT (TO) f=9 TT=3600.03	MSAT (TO) f=5 TT=3600.03 T1=1782.57	OPTIMUM f=4 TT=2152.9 T1=2152.55
csp/os-taillard-10/ normalized-os-taillard-10-95-8.xml	MOPT f=3 TT=167.157 T1=142.53	? TT=82.446	? TT=78.977	MSAT (TO) f=12 TT=3600.04	MSAT (TO) f=11 TT=3600.08	OPTIMUM f=3 TT=778.149 T1=346.91	OPTIMUM f=3 TT=167.157 T1=142.53
csp/os-taillard-15/ normalized-os-taillard-15-95-2.xml	MSAT TO f=15 TT=3600.08	? TT=122.641	? TT=115.812	MSAT (TO) f=19 TT=3600.01	MSAT (TO) f=15 TT=3600.08	MSAT (TO) f=18 TT=3600.23 T1=333.05	No Cert. TT=50.932
csp/os-taillard-15/ normalized-os-taillard-15-95-9.xml	MSAT TO f=12 TT=3600.35 T1=399.33	? TT=198.683	? TT=199.271	MSAT (TO) f=18 TT=3600.1	MSAT (TO) f=18 TT=3600.01	MSAT (TO) f=12 TT=3600.35 T1=399.33	No Cert. TT=53.194
csp/os-taillard-20/ normalized-os-taillard-20-95-0.xml	MSAT TO f=26 TT=3600.02	? TT=160.111	? TT=151.312	MSAT (TO) f=33 TT=3600.07	MSAT (TO) f=26 TT=3600.02	? TT=172.472	? TT=172.451
Bench name	Best results on this instance	AbsconMax 112 pc-d	AbsconMax 112 pc-w	Concrete + CSP4J - MCRW Engine 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Sugar v1.13+minisat	Sugar++ v1.13+minisat-inc
csp/os-taillard-20/ normalized-os-taillard-20-95-2.xml	MSAT TO f=24 TT=3600.06	? TT=189.454	? TT=189.406	MSAT (TO) f=32 TT=3600.01	MSAT (TO) f=24 TT=3600.06	? TT=171.329	? TT=177.88
csp/os-taillard-20/ normalized-os-taillard-20-95-6.xml	MSAT TO f=25 TT=3600.02	? TT=97.729	? TT=102.909	MSAT (TO) f=32 TT=3600.09	MSAT (TO) f=25 TT=3600.02	? TT=176.818	? TT=169.137
csp/os-taillard-4/ normalized-os-taillard-4-95-4.xml	MOPT f=2 TT=1.299 T1=1.13	MSAT (TO) f=2 TT=1601.72 T1=1.37	MSAT (TO) f=2 TT=1592.83 T1=1.22	MSAT (TO) f=2 TT=3600.04	MSAT (TO) f=2 TT=3600.02	OPTIMUM f=2 TT=1.418 T1=1.26	OPTIMUM f=2 TT=1.299 T1=1.13
csp/os-taillard-4/ normalized-os-taillard-4-95-5.xml	MOPT f=3 TT=1.164 T1=0.96	MSAT (TO) f=3 TT=1600.35 T1=0.96	MSAT (TO) f=3 TT=1605.42 T1=1.07	MSAT (TO) f=3 TT=3600.06	MSAT (TO) f=3 TT=3600.07	OPTIMUM f=3 TT=1.367 T1=1.15	OPTIMUM f=3 TT=1.164 T1=1
csp/os-taillard-4/ normalized-os-taillard-4-95-8.xml	MOPT f=3 TT=1.423 T1=1.21	MSAT (TO) f=3 TT=1598.08 T1=2.67	MSAT (TO) f=3 TT=1602.14 T1=2.95	MSAT (TO) f=3 TT=3600.08	MSAT (TO) f=3 TT=3600.06	OPTIMUM f=3 TT=1.423 T1=1.21	OPTIMUM f=3 TT=1.524 T1=1.35
csp/os-taillard-5/ normalized-os-taillard-5-95-5.xml	MOPT f=3 TT=2.634 T1=2.45	MSAT (TO) f=4 TT=1622.42 T1=8.19	MSAT (TO) f=4 TT=1589.88 T1=8.89	MSAT (TO) f=4 TT=3600.09	MSAT (TO) f=4 TT=3600.07	OPTIMUM f=3 TT=2.923 T1=2.48	OPTIMUM f=3 TT=2.634 T1=2.45
csp/os-taillard-5/ normalized-os-taillard-5-95-8.xml	MOPT f=5 TT=2.159 T1=1.52	MSAT (TO) f=5 TT=1603.16 T1=4.43	MSAT (TO) f=5 TT=1605.44 T1=4.09	MSAT (TO) f=5 TT=3600.01	MSAT (TO) f=5 TT=3600.05	OPTIMUM f=5 TT=3.047 T1=2.35	OPTIMUM f=5 TT=2.159 T1=1.52
csp/os-taillard-5/ normalized-os-taillard-5-95-9.xml	MOPT f=4 TT=2.126 T1=1.95	MSAT (TO) f=4 TT=1617.11 T1=3.54	MSAT (TO) f=4 TT=1599.26 T1=3.45	MSAT (TO) f=4 TT=3600.01	MSAT (TO) f=4 TT=3600.02	OPTIMUM f=4 TT=3.201 T1=3.03	OPTIMUM f=4 TT=2.126 T1=1.95
csp/os-taillard-7/ normalized-os-taillard-7-95-4.xml	MOPT f=3 TT=8.857 T1=8.64	MSAT (TO) f=6 TT=1599.45 T1=15.13	MSAT (TO) f=6 TT=1600.58 T1=13.86	MSAT (TO) f=6 TT=3600.08	MSAT (TO) f=6 TT=3600.09	OPTIMUM f=3 TT=13.43 T1=13.1	OPTIMUM f=3 TT=8.857 T1=8.64
csp/os-taillard-7/ normalized-os-taillard-7-95-7.xml	MOPT f=4 TT=7.419 T1=6.89	MSAT (TO) f=5 TT=1620.87 T1=42.77	MSAT (TO) f=5 TT=1619.42 T1=39.95	MSAT (TO) f=5 TT=3600.07	MSAT (TO) f=6 TT=3600.01	OPTIMUM f=4 TT=15.065 T1=9.62	OPTIMUM f=4 TT=7.419 T1=6.89
Bench name	Best results on this instance	AbsconMax 112 pc-d	AbsconMax 112 pc-w	Concrete + CSP4J - MCRW Engine 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Sugar v1.13+minisat	Sugar++ v1.13+minisat-inc
csp/os-taillard-7/ normalized-os-taillard-7-95-8.xml	MOPT f=3 TT=7.928 T1=7.69	MSAT (TO) f=5 TT=1655.02 T1=38.12	MSAT (TO) f=5 TT=1671.45 T1=45.85	MSAT (TO) f=6 TT=3600.08	MSAT (TO) f=6 TT=3600.05	OPTIMUM f=3 TT=20.158 T1=19.83	OPTIMUM f=3 TT=7.928 T1=7.69
csp/os-taillard-7/ normalized-os-taillard-7-95-9.xml	MOPT f=3 TT=13.742 T1=13.52	MSAT (TO) f=6 TT=1623 T1=19.5	MSAT (TO) f=6 TT=1576.93 T1=18.4	MSAT (TO) f=6 TT=3600.08	MSAT (TO) f=7 TT=3600.02	OPTIMUM f=3 TT=18.505 T1=18.18	OPTIMUM f=3 TT=13.742 T1=13.52
csp/super-js/super-js-taillard-15/ normalized-super-js-taillard-15-22.xml	MSAT TO f=51 TT=3600.03	? TT=306.456	? TT=327.093	MSAT (TO) f=51 TT=3600.03	MSAT (TO) f=85 TT=3600	? (MO) TT=119.508	? (MO) TT=113.873
csp/super-js/super-js-taillard-15/ normalized-super-js-taillard-15-23.xml	MSAT TO f=48 TT=3600.08	? TT=207.608	? TT=232.194	MSAT (TO) f=48 TT=3600.08	MSAT (TO) f=82 TT=3600.04	? (MO) TT=113.024	? (MO) TT=116.238
csp/super-js/super-js-taillard-15/ normalized-super-js-taillard-15-30.xml	MSAT TO f=51 TT=3600.02	? TT=445.473	? TT=447.505	MSAT (TO) f=51 TT=3600.02	MSAT (TO) f=91 TT=3600.04	? (MO) TT=334.809	? (MO) TT=122.076
csp/super-js/super-js-taillard-20-15/ normalized-super-js-taillard-20-15-24.xml	MSAT TO f=89 TT=3600.07	? TT=708.489	? TT=693.797	MSAT (TO) f=89 TT=3600.07	MSAT (TO) f=141 TT=3600.08	? TT=189.931	? TT=190.403
csp/super-os/super-os-taillard-10/ normalized-super-os-taillard-10-24.xml	MSAT TO f=33 TT=3600.03	? TT=149.672	? TT=141.502	MSAT (TO) f=33 TT=3600.03	MSAT (TO) f=36 TT=3600.06	MSAT (TO) f=85 TT=3600.21 T1=160.21	MSAT (TO) f=120 TT=3600.33 T1=49.77
csp/super-os/super-os-taillard-15/ normalized-super-os-taillard-15-27.xml	MSAT TO f=56 TT=3600.03	? TT=290.6	? TT=276.361	MSAT (TO) f=56 TT=3600.03	MSAT (TO) f=58 TT=3600.09	? TT=189.717	? TT=194.978
csp/super-os/super-os-taillard-20/ normalized-super-os-taillard-20-27.xml	MSAT TO f=90 TT=3600.07	? TT=154.219	? TT=157.916	MSAT (TO) f=90 TT=3600.07	MSAT (TO) f=118 TT=3600.08	? TT=227.641	? TT=233.794
csp/super-os/super-os-taillard-20/ normalized-super-os-taillard-20-29.xml	MSAT TO f=94 TT=3600.03	? TT=158.444	? TT=171.015	MSAT (TO) f=94 TT=3600.03	MSAT (TO) f=115 TT=3600.02	? TT=258.421	? TT=246.549
Bench name	Best results on this instance	AbsconMax 112 pc-d	AbsconMax 112 pc-w	Concrete + CSP4J - MCRW Engine 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Sugar v1.13+minisat	Sugar++ v1.13+minisat-inc
csp/super-os/super-os-taillard-4/ normalized-super-os-taillard-4-5.xml	MOPT f=3 TT=13.353 T1=3.02	MSAT (TO) f=3 TT=1604.27 T1=3.02	MSAT (TO) f=3 TT=1581.49 T1=3.35	MSAT (TO) f=3 TT=3600.09	MSAT (TO) f=3 TT=3600.07	OPTIMUM f=3 TT=16.269 T1=16.1	OPTIMUM f=3 TT=13.353 T1=13.24
csp/super-os/super-os-taillard-4/ normalized-super-os-taillard-4-8.xml	MOPT f=3 TT=9.747 T1=1.56	MSAT (TO) f=3 TT=1608.58 T1=1.56	MSAT (TO) f=3 TT=1609.31 T1=1.67	MSAT (TO) f=3 TT=3600.09	MSAT (TO) f=3 TT=3600.05	OPTIMUM f=3 TT=9.747 T1=9.54	OPTIMUM f=3 TT=13.393 T1=13.2
csp/super-os/super-os-taillard-5/ normalized-super-os-taillard-5-1.xml	MOPT f=3 TT=411.296 T1=411.02	MSAT (TO) f=6 TT=1597.51 T1=8.23	MSAT (TO) f=6 TT=1611.68 T1=8.39	MSAT (TO) f=3 TT=3600.06	MSAT (TO) f=6 TT=3600.02	OPTIMUM f=3 TT=428.101 T1=427.88	OPTIMUM f=3 TT=411.296 T1=411.02
csp/super-os/super-os-taillard-5/ normalized-super-os-taillard-5-21.xml	MSAT TO f=11 TT=3600.04	MSAT (TO) f=12 TT=1613.36 T1=31.31	MSAT (TO) f=12 TT=1640.1 T1=28.55	MSAT (TO) f=11 TT=3600.04	MSAT (TO) f=12 TT=3600.01	MSAT (TO) f=16 TT=3600.1 T1=48.39	MSAT (TO) f=14 TT=3600.12 T1=11.08
csp/super-os/super-os-taillard-5/ normalized-super-os-taillard-5-4.xml	MSAT TO f=3 TT=3600.09 T1=2503.36	MSAT (TO) f=7 TT=1600.46 T1=8.68	MSAT (TO) f=7 TT=1601.87 T1=8.95	MSAT (TO) f=9 TT=3600.07	MSAT (TO) f=12 TT=3600.09	MSAT (TO) f=4 TT=3600.06 T1=534.46	MSAT (TO) f=3 TT=3600.09 T1=2503.36
csp/super-os/super-os-taillard-5/ normalized-super-os-taillard-5-5.xml	MSAT TO f=3 TT=3600.02 T1=2917.04	MSAT (TO) f=11 TT=1602.96 T1=10.78	MSAT (TO) f=11 TT=1615.08 T1=9.83	MSAT (TO) f=6 TT=3600.07	MSAT (TO) f=9 TT=3600.02	MSAT (TO) f=3 TT=3600.02 T1=2917.04	MSAT (TO) f=3 TT=3600.1 T1=3591.6
csp/super-os/super-os-taillard-5/ normalized-super-os-taillard-5-7.xml	MOPT f=3 TT=645.598 T1=645.39	MSAT (TO) f=6 TT=1596.08 T1=3.22	MSAT (TO) f=6 TT=1625.03 T1=3.54	MSAT (TO) f=3 TT=3600.02	MSAT (TO) f=9 TT=3600.02	OPTIMUM f=3 TT=645.598 T1=645.39	OPTIMUM f=3 TT=702.724 T1=702.46
csp/super-os/super-os-taillard-7/ normalized-super-os-taillard-7-2.xml	MSAT TO f=5 TT=3600.08 T1=1040.23	MSAT (TO) f=13 TT=1679.33 T1=43.87	MSAT (TO) f=13 TT=1633.18 T1=38.14	MSAT (TO) f=15 TT=3600.08	MSAT (TO) f=21 TT=3600.02	MSAT (TO) f=5 TT=3600.08 T1=1040.23	MSAT (TO) f=5 TT=3600.2 T1=1123.98
csp/super-os/super-os-taillard-7/ normalized-super-os-taillard-7-30.xml	MSAT TO f=16 TT=3600.18 T1=3462.2	MSAT (TO) f=18 TT=1650.07 T1=42.36	MSAT (TO) f=18 TT=1659.66 T1=45.14	MSAT (TO) f=22 TT=3600	MSAT (TO) f=25 TT=3600.06	MSAT (TO) f=44 TT=3600.03 T1=48.19	MSAT (TO) f=16 TT=3600.18 T1=3462.2
csp/super-queens/ normalized-super-queens-11.xml	MOPT f=3 TT=0.944 T1=0.55	OPTIMUM f=3 TT=31.026 T1=0.68 CHECKS=4156 NODES=45	OPTIMUM f=3 TT=30.891 T1=0.55 CHECKS=5915 NODES=63	MSAT (TO) f=3 TT=3600.03	MSAT (TO) f=3 TT=3600.05	OPTIMUM f=3 TT=0.946 T1=0.82	OPTIMUM f=3 TT=0.944 T1=0.8
Bench name	Best results on this instance	AbsconMax 112 pc-d	AbsconMax 112 pc-w	Concrete + CSP4J - MCRW Engine 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Sugar v1.13+minisat	Sugar++ v1.13+minisat-inc
csp/super-queens/ normalized-super-queens-13.xml	MOPT f=4 TT=34.546 T1=0.62	MSAT (TO) f=4 TT=1591.9 T1=0.62	OPTIMUM f=4 TT=638.256 T1=0.67 CHECKS=33327100 NODES=75122	MSAT (TO) f=4 TT=3600.1	MSAT (TO) f=4 TT=3600.01	OPTIMUM f=4 TT=34.546 T1=9.45	OPTIMUM f=4 TT=45.132 T1=6.33
csp/super-queens/ normalized-super-queens-1.xml	MOPT f=4 TT=3088.74 T1=0.78	MSAT (TO) f=4 TT=1609.14 T1=0.78	MSAT (TO) f=4 TT=1601.3 T1=0.88	MSAT (TO) f=4 TT=3600.08	MSAT (TO) f=4 TT=3600.1	OPTIMUM f=4 TT=3088.74 T1=86.01	OPTIMUM f=4 TT=3564.83 T1=3564.64
csp/super-queens/ normalized-super-queens-2.xml	MSAT TO f=11 TT=1696.76 T1=97.29	MSAT (TO) f=11 TT=1714.73 T1=97.29	MSAT (TO) f=11 TT=1696.76 T1=104.87	MSAT (TO) f=12 TT=3600	MSAT (TO) f=20 TT=3600.06	? TT=199.651	? TT=198.342
csp/super-queens/ normalized-super-queens-3.xml	MSAT TO f=4 TT=1594.97 T1=1.15	MSAT (TO) f=4 TT=1598.14 T1=1.15	MSAT (TO) f=4 TT=1594.97 T1=1.19	MSAT (TO) f=4 TT=3600.03	MSAT (TO) f=4 TT=3600.01	MSAT (TO) f=6 TT=3600.06 T1=4.1	MSAT (TO) f=6 TT=3600.08 T1=3.56
csp/super-queens/ normalized-super-queens-4.xml	MSAT TO f=13 TT=3600.03	MSAT (TO) f=17 TT=1754.12 T1=162.35	MSAT (TO) f=17 TT=1742.28 T1=161.93	MSAT (TO) f=13 TT=3600.03	MSAT (TO) f=29 TT=3600.09	? TT=286.988	? TT=279.495
csp/super-queens/ normalized-super-queens-5.xml	MSAT TO f=3 TT=3600.03	MSAT (TO) f=4 TT=1598.08 T1=2.85	MSAT (TO) f=4 TT=1603.85 T1=3.05	MSAT (TO) f=3 TT=3600.09	MSAT (TO) f=3 TT=3600.03	MSAT (TO) f=5 TT=3600.04 T1=223.42	MSAT (TO) f=4 TT=3600.12 T1=1972.87
csp/super-queens/ normalized-super-queens-6.xml	MSAT TO f=16 TT=3600.08	MSAT (TO) f=68 TT=1686.98 T1=1686.93	MSAT (TO) f=86 TT=1733.27 T1=34.1	MSAT (TO) f=16 TT=3600.08	MSAT (TO) f=34 TT=3600.06	? TT=202.593	? TT=191.837
csp/super-queens/ normalized-super-queens-7.xml	MSAT TO f=22 TT=3600.06	? TT=114.449	? TT=120.974	MSAT (TO) f=22 TT=3600.06	MSAT (TO) f=41 TT=3600.1	? TT=188.897	? TT=192.826
csp/super-queens/ normalized-super-queens-8.xml	MSAT TO f=4 TT=1600.71 T1=2.87	MSAT (TO) f=4 TT=1600.71 T1=3.26	MSAT (TO) f=4 TT=1617.93 T1=2.87	MSAT (TO) f=4 TT=3600.04	MSAT (TO) f=4 TT=3600.05	MSAT (TO) f=9 TT=3600.02 T1=274.75	MSAT (TO) f=9 TT=3600.07 T1=113.11

Some statistics...

	AbsconMax 112 pc-d	AbsconMax 112 pc-w	Concrete + CSP4J - MCRW Engine 2008-05-30	Concrete + CSP4J - Tabu Engine 2008-05-30	Sugar v1.13+minisat	Sugar++ v1.13+minisat-inc
Number of times the solver is able to give the best known answer	29	33	64	42	60	62
Number of times the solver is able to give the best known answer from an incomplete solver point of view (i.e. without considering optimality proof)	54	57	97	73	60	62
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)	6	20	40	26	18	43
Number of times the solver is the best solver from an incomplete solver point of view (i.e. best known answer and best T1 time)	28	22	0	0	7	27