PB'07 evaluation: results by benchmark

Results by benchmark for category optimisation, small integers, linear constraints (OPT-SMALLINT-LIN), subcategory Handmade, subsubcategory MPS to OPB

This page displays the results of the different solvers for each benchmark for category optimisation, small integers, linear constraints (OPT-SMALLINT-LIN), subcategory Handmade, subsubcategory MPS to OPB

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell example	Meaning
Answer	Solver result
f=...	value of the objective function for the model reported by the solver
TT=...	Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out
T1=...	Time to get the best model (T1): this is the wall clock time (in seconds) ellapsed when the solver found its best model. This time is obtained from the 'o lines'. It is useful to compare both complete and incomplete solvers. It doesn't take into account the time to prove optimality of the solution.

Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

Abbreviation	Meaning
f=...	Value of the objective function
TO	Time Out
MO	Mem. Out (out of memory)

Meaning of the different colors:

Color	Meaning
text	the solver cannot handle this instance
text	the solver gave no answer
text	the solver could give an answer (SAT)
text	the solver gave a definitive answer (OPTIMUM FOUND or UNSAT)
text	the solver performed better than the other ones on that instance (complete solvers point of view)
text	the solver performed better than the other ones on that instance (incomplete solvers point of view)
text	the solver was ended by a signal or other problem
text	the solver gave an incomplete answer
text	the solver gave a wrong answer

For better readability, you may choose to hide some solvers:
absconPseudo 102
bsolo 3.0.16
bsolo 3.0.17
glpPB 0.2
minisat+ 1.14
oree 0.1.2 alpha
PB-clasp 2007-03-23
PB-clasp 2007-04-10
PBS4 2007-03-23
PBS4_v2 2007-03-23
Pueblo 1.4
sat4jPseudoCP 2007-03-23
sat4jPseudoCPClause 2007-03-23
SAT4JPseudoResolution 2007-03-23
wildcat-rnp 2007-03-21
wildcat-skc 2007-03-21

Bench name	Best results on this instance	absconPseudo 102	bsolo 3.0.16	bsolo 3.0.17	glpPB 0.2	minisat+ 1.14	oree 0.1.2 alpha	PB-clasp 2007-03-23	PB-clasp 2007-04-10	PBS4 2007-03-23	PBS4_v2 2007-03-23	Pueblo 1.4	sat4jPseudoCP 2007-03-23	sat4jPseudoCPClause 2007-03-23	SAT4JPseudoResolution 2007-03-23	wildcat-rnp 2007-03-21	wildcat-skc 2007-03-21
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib2003/ normalized-mps-v2-20-10-10teams.opb	SAT f=904 TT=1800.1 T1=307.07	? (TO) TT=1800.06	SAT (TO) f=916 TT=1800.08 T1=315.69	SAT (TO) f=904 TT=1800.1 T1=307.07	? (TO) TT=1800.07	SAT (TO) f=950 TT=1800.14 T1=1615.81	? (TO) TT=1800.37	SAT (TO) f=968 TT=1802.02 T1=332.71	SAT (TO) f=926 TT=1801.93 T1=116.18	No Cert. TT=1800.09	SAT (TO) f=940 TT=1800.09 T1=1189.8	SAT f=938 TT=1782.74 T1=561.37	SAT (TO) f=962 TT=1800.24 T1=1150.08	? (TO) TT=1800.25	SAT (TO) f=928 TT=1800.29 T1=2.05	SAT (TO) f=930 TT=1800.03 T1=419.65	SAT (TO) f=912 TT=1800.05 T1=461.65
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib2003/ normalized-mps-v2-20-10-fast0507.opb	SAT f=248 TT=1800.16 T1=4.52	SAT (TO) f=248 TT=1800.16 T1=4.52	SAT (TO) f=308 TT=1800.24 T1=1205.75	SAT (TO) f=309 TT=1800.36 T1=190.03	? (TO) TT=1800.07	Signal TT=6.691	SAT (TO) f=354 TT=1800.55 T1=125.34	SAT (TO) f=60927 TT=1802.11 T1=190.02	SAT (TO) f=340 TT=1802 T1=363.86	No Cert. TT=1800.04	SAT (TO) f=122399 TT=1800.1 T1=1757.77	SAT f=318 TT=1782.85 T1=121.35	SAT (TO) f=259 TT=1804.13 T1=1235.72	SAT (TO) f=259 TT=1804.05 T1=1178.7	SAT (TO) f=259 TT=1804.06 T1=15.88	SAT (TO) f=22487 TT=1801.48 T1=1797.68	SAT (TO) f=7155 TT=1800.16 T1=1788.31
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib2003/ normalized-mps-v2-20-10-manna81.opb	SAT f=-11560 TT=1800.09 T1=69.46	SAT (TO) f=-10374 TT=1800.19 T1=2.02	SAT (MO) f=-11409 TT=241.426 T1=13.2	SAT (MO) f=-11409 TT=243.245 T1=13.2	? (TO) TT=1800.02	SAT (TO) f=-8204 TT=1800.14 T1=1.29	SAT (MO) f=-10535 TT=903.741 T1=823.48	? (TO) TT=1800.07	? (TO) TT=1800.07	No Cert. TT=1800.08	SAT (TO) f=-11136 TT=1800.13 T1=1800.04	SAT f=-219 TT=1782.91 T1=1782.85	SAT (TO) f=-10912 TT=1800.52 T1=1684	SAT (TO) f=-10912 TT=1800.66 T1=1680.57	SAT (TO) f=-11411 TT=1800.52 T1=92.39	SAT (TO) f=-10857 TT=1800.07 T1=1201.54	SAT (TO) f=-11560 TT=1800.09 T1=69.46
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib2003/ normalized-mps-v2-20-10-mzzv11.opb	SAT f=-10639 TT=1800.64 T1=1606.83	? (TO) TT=1800.19	SAT (TO) f=-7987 TT=1800.1 T1=1774.46	SAT (TO) f=-7985 TT=1800.11 T1=1754.31	? (TO) TT=1800.04	SAT (TO) f=-3614 TT=1800.13 T1=1423.29	? (TO) TT=1800.08	? (TO) TT=1800.1	? (TO) TT=1801.83	No Cert. TT=1800.12	SAT (TO) f=-5479 TT=1800.12 T1=1800.09	SAT f=-4984 TT=1782.75 T1=1782.72	SAT (TO) f=-7444 TT=1800.56 T1=1435	SAT (TO) f=-9114 TT=1800.61 T1=392.74	SAT (TO) f=-10639 TT=1800.64 T1=1606.83	? (TO) TT=1800.07	? (TO) TT=1800.09
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib2003/ normalized-mps-v2-20-10-mzzv42z.opb	OPT f=-10270 TT=1022.91 T1=1021.26	? (TO) TT=1800.24	Wrong Cert. TT=1800.15	SAT (TO) f=-9230 TT=1800.11 T1=1789.64	? (TO) TT=1800.08	SAT (TO) f=-3209 TT=1800.14 T1=1109.14	? (TO) TT=1800.17	? (TO) TT=1800.15	? (TO) TT=1800.09	No Cert. TT=1800.05	SAT (TO) f=-4748 TT=1800.06 T1=1800.03	SAT f=-8180 TT=1782.75 T1=1782.71	SAT (TO) f=-7434 TT=1800.56 T1=1556.75	SAT (TO) f=-7746 TT=1800.59 T1=1265.24	OPT f=-10270 TT=1022.91 T1=1021.26	? (TO) TT=1800.05	? (TO) TT=1800.08
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib2003/ normalized-mps-v2-20-10-net12.opb	OPT f=214 TT=13.687 T1=4.09	? TT=7.9	OPT f=214 TT=28.283 T1=4.49	OPT f=214 TT=13.687 T1=4.09	? (TO) TT=1800.11	? (TO) TT=1800.13	? (TO) TT=1800.15	? (MO) TT=53.315	SAT (TO) f=296 TT=1802.01 T1=1596.84	OPT f=214 TT=66.777 T1=63.66	OPT f=214 TT=66.83 T1=63.68	OPT f=214 TT=38.929 T1=26.68	SAT (TO) f=275 TT=1800.78 T1=690.34	SAT (TO) f=337 TT=1800.82 T1=22.87	OPT f=214 TT=42.365 T1=39.46	? (TO) TT=1800.12	? (TO) TT=1800.15
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib2003/ normalized-mps-v2-20-10-opt1217.opb	SAT f=-16384 TT=1800.06 T1=128.46	? (TO) TT=1800.08	SAT f=-16383 TT=1800.07 T1=322.79	SAT (TO) f=-16383 TT=1800.1 T1=323.79	? (TO) TT=1800.03	SAT (TO) f=-4096 TT=1800.11 T1=483.63	? (TO) TT=1800.21	? (TO) TT=1800.24	? (TO) TT=1800.12	No Cert. TT=1800.09	SAT (TO) f=-14336 TT=1800.07 T1=1800.04	SAT f=-14336 TT=1782.74 T1=1782.73	SAT (TO) f=-16384 TT=1800.2 T1=591.03	SAT (TO) f=-8 TT=1800.32 T1=356.79	SAT (TO) f=0 TT=1800.19 T1=1.03	SAT (TO) f=-16384 TT=1800.09 T1=479.49	SAT (TO) f=-16384 TT=1800.06 T1=128.46
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib2003/ normalized-mps-v2-20-10-p2756.opb	SAT f=3695 TT=1800.38 T1=1789.11	? (TO) TT=1800.14	SAT f=5289 TT=1800.07 T1=214.32	SAT (TO) f=5407 TT=1800.15 T1=657.99	? (TO) TT=1800.15	SAT (TO) f=64559 TT=1800.08 T1=952.44	SAT (TO) f=89885 TT=1801.67 T1=333.1	? (MO) TT=27.1	SAT (TO) f=4140 TT=1802.06 T1=1599.4	No Cert. TT=1800.02	SAT (TO) f=5944 TT=1800.05 T1=1786.43	SAT f=17727 TT=1782.73 T1=2.59	SAT (TO) f=3793 TT=1800.26 T1=1168.8	SAT (TO) f=3695 TT=1800.38 T1=1789.11	SAT (TO) f=3842 TT=1800.58 T1=400.64	SAT (TO) f=53140 TT=1800.04 T1=1187.21	SAT (TO) f=32951 TT=1800.1 T1=347.58
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib2003/ normalized-mps-v2-20-10-protfold.opb	SAT f=-24 TT=1800.05 T1=385.98	? (TO) TT=1800.11	SAT (TO) f=-23 TT=1800.13 T1=1251.55	SAT (TO) f=-23 TT=1800.18 T1=1245.53	? (TO) TT=1800.1	SAT (TO) f=-24 TT=1800.06 T1=385.98	? (TO) TT=1800.35	? (TO) TT=1800.07	? (TO) TT=1800.08	No Cert. TT=1800.08	SAT (TO) f=-24 TT=1800.05 T1=1800.02	SAT f=-19 TT=1782.74 T1=1782.69	SAT (TO) f=-17 TT=1800.33 T1=38.96	SAT (TO) f=-18 TT=1800.31 T1=105.51	SAT (TO) f=-23 TT=1800.81 T1=60.4	SAT (TO) f=-19 TT=1800.03 T1=455.94	SAT (TO) f=-24 TT=1800.05 T1=609.83
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib2003/ normalized-mps-v2-20-10-seymour.opb	SAT f=425 TT=1800.09 T1=458.95	SAT (TO) f=466 TT=1800.06 T1=0.72	SAT (TO) f=436 TT=1800.07 T1=46.47	SAT (TO) f=435 TT=1800.09 T1=586.16	? (TO) TT=1800.05	SAT (TO) f=466 TT=1800.09 T1=0.29	SAT (TO) f=490 TT=1800.41 T1=1.19	SAT (TO) f=474 TT=1801.98 T1=1765.66	SAT (TO) f=481 TT=1801.99 T1=0.69	No Cert. TT=1800.08	SAT (TO) f=504 TT=1800.07 T1=480.08	SAT f=492 TT=1782.75 T1=107.96	SAT (TO) f=487 TT=1800.26 T1=1679.6	SAT (TO) f=488 TT=1800.36 T1=1278.62	SAT (TO) f=490 TT=1800.33 T1=1.52	SAT (TO) f=425 TT=1800.09 T1=458.95	SAT (TO) f=426 TT=1800.03 T1=948.64
Bench name	Best results on this instance	absconPseudo 102	bsolo 3.0.16	bsolo 3.0.17	glpPB 0.2	minisat+ 1.14	oree 0.1.2 alpha	PB-clasp 2007-03-23	PB-clasp 2007-04-10	PBS4 2007-03-23	PBS4_v2 2007-03-23	Pueblo 1.4	sat4jPseudoCP 2007-03-23	sat4jPseudoCPClause 2007-03-23	SAT4JPseudoResolution 2007-03-23	wildcat-rnp 2007-03-21	wildcat-skc 2007-03-21
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib3/ normalized-mps-v2-20-10-mitre.opb	SAT f=7680 TT=1800.78 T1=34.93	? (TO) TT=1800.15	? (TO) TT=1800.1	? (TO) TT=1800.09	? (TO) TT=1800.04	SAT (TO) f=9264 TT=1800.13 T1=1502.87	? (TO) TT=1800.3	? (TO) TT=1800.1	? (TO) TT=1800.13	? (TO) TT=1800.03	? (TO) TT=1800.09	SAT f=8008 TT=1782.79 T1=1782.71	SAT (TO) f=9064 TT=1800.71 T1=263.9	SAT (TO) f=10132 TT=1800.69 T1=16.95	SAT (TO) f=7680 TT=1800.78 T1=34.93	? (TO) TT=1800.09	? (TO) TT=1800.05
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-bm23.opb	OPT f=34 TT=0.086 T1=0	OPT f=34 TT=0.751 T1=0.47	OPT f=34 TT=1.306 T1=1	OPT f=34 TT=1.255 T1=1.19	OPT f=34 TT=0.094 T1=0.07	OPT f=34 TT=6.294 T1=5.28	OPT f=34 TT=56.443 T1=7.19	? (MO) TT=40.15	OPT f=34 TT=0.19 T1=0.05	OPT f=34 TT=0.808 T1=0.69	OPT f=34 TT=0.813 T1=0.69	OPT f=34 TT=0.086 T1=0	OPT f=34 TT=9.325 T1=7.48	OPT f=34 TT=418.654 T1=377.87	OPT f=34 TT=1.37 T1=1.17	SAT (TO) f=34 TT=1800.05 T1=0	SAT (TO) f=34 TT=1800.11 T1=0
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-cracpb1.opb	OPT f=22199 TT=1.036 T1=0.88	? (TO) TT=1800.08	OPT f=22199 TT=1.652 T1=1.59	OPT f=22199 TT=1.036 T1=0.88	OPT f=22199 TT=1.255 T1=1.19	SAT (TO) f=63282 TT=1800.1 T1=806.6	? (TO) TT=1800.4	? (MO) TT=33.46	SAT (TO) f=43373 TT=1802.06 T1=1546.54	No Cert. TT=1800.13	SAT (TO) f=36972 TT=1800.1 T1=1739.27	SAT f=118468 TT=1782.74 T1=1650.28	SAT (TO) f=55310 TT=1800.25 T1=28.68	SAT (TO) f=54756 TT=1800.24 T1=35.5	SAT (TO) f=50797 TT=1800.39 T1=1192.28	SAT (TO) f=22199 TT=1800.04 T1=13.69	SAT (TO) f=22199 TT=1800.1 T1=8.19
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-diamond.opb	UNSAT TT=0	UNSAT TT=0.416	UNSAT TT=0.005	UNSAT TT=0.006	UNSAT TT=0.006	UNSAT TT=0.001	UNSAT TT=0.001	? (TO) TT=1800.11	? (TO) TT=1800.1	UNSAT TT=0	UNSAT TT=0.002	UNSAT TT=0.002	UNSAT TT=0.245	UNSAT TT=0.262	UNSAT TT=0.241	? (TO) TT=1800.07	? (TO) TT=1800.08
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-l152lav.opb	OPT f=4722 TT=74.223 T1=74.16	? (TO) TT=1800.13	? (TO) TT=1800.23	? (TO) TT=1800.25	OPT f=4722 TT=74.223 T1=74.16	? (TO) TT=1800.14	? (TO) TT=1801.2	? (TO) TT=1802.04	? (TO) TT=1802.07	? (TO) TT=1800.07	? (TO) TT=1800.03	? (TO) TT=1800.05	? (TO) TT=1800.19	SAT (TO) f=5336 TT=1800.36 T1=1382.87	? (TO) TT=1800.22	? (TO) TT=1800.11	? (TO) TT=1800.1
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-lp4l.opb	OPT f=2967 TT=3.027 T1=0.89	? (TO) TT=1800.13	OPT f=2967 TT=543.746 T1=0.99	OPT f=2967 TT=5.894 T1=0.89	OPT f=2967 TT=3.027 T1=2.98	SAT (TO) f=3907 TT=1800.16 T1=1099.35	? (TO) TT=1800.69	SAT (MO) f=4706 TT=40.23 T1=2.26	SAT (TO) f=3696 TT=1802.07 T1=637.64	No Cert. TT=1800.13	SAT (TO) f=3819 TT=1800.1 T1=750.92	SAT f=4368 TT=1782.73 T1=469.87	SAT (TO) f=4267 TT=1800.35 T1=106.35	SAT (TO) f=4244 TT=1800.25 T1=131.97	SAT (TO) f=3964 TT=1800.38 T1=316.34	SAT (TO) f=3552 TT=1800.02 T1=580.93	SAT (TO) f=3207 TT=1800.1 T1=519.76
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-lseu.opb	OPT f=1120 TT=8.647 T1=8.58	OPT f=1120 TT=428.835 T1=15.4	OPT f=1120 TT=88.659 T1=40.35	OPT f=1120 TT=40.75 T1=12.69	OPT f=1120 TT=8.647 T1=8.58	OPT f=1120 TT=173.251 T1=147.24	SAT (TO) f=2741 TT=1800.19 T1=1797.52	? (MO) TT=42.34	OPT f=1120 TT=252.428 T1=137.87	No Cert. TT=1800.04	SAT (TO) f=1226 TT=1800.07 T1=428.68	OPT f=1120 TT=454.566 T1=384.98	SAT (TO) f=1149 TT=1800.18 T1=1271.29	SAT (TO) f=1431 TT=1800.15 T1=399.54	OPT f=1120 TT=371.508 T1=110.81	SAT (TO) f=1120 TT=1800.01 T1=17.78	SAT (TO) f=1120 TT=1800.07 T1=335.51
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-mod008.opb	OPT f=307 TT=4.741 T1=0	SAT (TO) f=307 TT=1800.07 T1=0.8	OPT f=307 TT=29.725 T1=0	OPT f=307 TT=17.476 T1=0	OPT f=307 TT=4.741 T1=4.68	SAT (TO) f=360 TT=1800.06 T1=814.97	SAT (TO) f=393 TT=1801.36 T1=716.39	? (MO) TT=22.41	SAT (TO) f=310 TT=1802.07 T1=449.49	No Cert. TT=1800.09	SAT (TO) f=377 TT=1800.07 T1=900.62	SAT f=602 TT=1782.74 T1=0.01	SAT (TO) f=307 TT=1800.21 T1=324.41	SAT (TO) f=359 TT=1800.25 T1=0.42	SAT (TO) f=317 TT=1800.26 T1=215.37	SAT (TO) f=307 TT=1800.09 T1=694.15	SAT (TO) f=313 TT=1800.11 T1=1185.86
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-mod010.opb	OPT f=6548 TT=2.237 T1=2.19	? (TO) TT=1800.16	? (TO) TT=1800.23	? (TO) TT=1800.31	OPT f=6548 TT=2.237 T1=2.19	? (TO) TT=1800.1	? (TO) TT=1800.42	? (TO) TT=1802	? (TO) TT=1801.96	? (TO) TT=1800.05	? (TO) TT=1800.08	? (TO) TT=1800.07	? (TO) TT=1800.2	SAT (TO) f=7726 TT=1800.33 T1=1395.95	? (TO) TT=1800.24	? (TO) TT=1800.03	SAT (TO) f=7562 TT=1800.1 T1=1692.91
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-p0033.opb	OPT f=3089 TT=0.017 T1=0	OPT f=3089 TT=0.807 T1=0.44	OPT f=3089 TT=0.176 T1=0	OPT f=3089 TT=0.167 T1=0	OPT f=3089 TT=0.09 T1=0.03	OPT f=3089 TT=0.535 T1=0.39	OPT f=3089 TT=0.871 T1=0.19	SAT (MO) f=4318 TT=46.09 T1=11.07	OPT f=3089 TT=0.139 T1=0.07	OPT f=3089 TT=0.017 T1=0	OPT f=3089 TT=0.019 T1=0	OPT f=3089 TT=0.026 T1=0	OPT f=3089 TT=3.232 T1=1.06	OPT f=3089 TT=3.028 T1=2.05	OPT f=3089 TT=0.719 T1=0.54	SAT (TO) f=3089 TT=1800.03 T1=15.09	SAT (TO) f=3089 TT=1800.06 T1=12.69
Bench name	Best results on this instance	absconPseudo 102	bsolo 3.0.16	bsolo 3.0.17	glpPB 0.2	minisat+ 1.14	oree 0.1.2 alpha	PB-clasp 2007-03-23	PB-clasp 2007-04-10	PBS4 2007-03-23	PBS4_v2 2007-03-23	Pueblo 1.4	sat4jPseudoCP 2007-03-23	sat4jPseudoCPClause 2007-03-23	SAT4JPseudoResolution 2007-03-23	wildcat-rnp 2007-03-21	wildcat-skc 2007-03-21
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-p0040.opb	OPT f=62027 TT=0.017 T1=0	OPT f=62027 TT=7.749 T1=0.34	OPT f=62027 TT=0.152 T1=0.09	OPT f=62027 TT=0.063 T1=0	OPT f=62027 TT=0.017 T1=0	OPT f=62027 TT=67.067 T1=63.96	SAT (TO) f=62371 TT=1800.18 T1=1663.58	SAT (MO) f=65392 TT=40.02 T1=0.01	OPT f=62027 TT=12.036 T1=5.19	OPT f=62027 TT=765.318 T1=531.54	OPT f=62027 TT=765.48 T1=531.6	OPT f=62027 TT=9.098 T1=8.99	OPT f=62027 TT=1.949 T1=0.33	OPT f=62027 TT=2.378 T1=0.33	OPT f=62027 TT=50.07 T1=0.23	SAT (TO) f=62027 TT=1800.06 T1=29.59	SAT (TO) f=62027 TT=1800.09 T1=22.79
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-p0201.opb	OPT f=1523 TT=2.647 T1=0.5	SAT (TO) f=1523 TT=1800.08 T1=0.5	OPT f=1523 TT=79.264 T1=10.29	OPT f=1523 TT=8.74 T1=2.99	OPT f=1523 TT=2.647 T1=2.59	SAT (TO) f=1571 TT=1800.03 T1=1703.25	SAT (TO) f=1866 TT=1800.09 T1=1769.83	? (MO) TT=36.73	SAT (TO) f=1523 TT=1802.07 T1=434.77	No Cert. TT=1800.03	SAT (TO) f=1575 TT=1800.04 T1=571.6	SAT f=1581 TT=1782.73 T1=556.09	SAT (TO) f=1555 TT=1800.14 T1=1299.02	SAT (TO) f=1587 TT=1800.25 T1=492.01	SAT (TO) f=1523 TT=1800.21 T1=410	SAT (TO) f=1547 TT=1800.08 T1=1104.42	SAT (TO) f=1555 TT=1800.02 T1=1351.49
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-p0548.opb	SAT f=23777 TT=1800.21 T1=124.36	? (TO) TT=1800.12	? TT=1800.01	? (TO) TT=1800.06	? (TO) TT=1800.05	SAT (TO) f=45401 TT=1800.08 T1=782.41	SAT (TO) f=26472 TT=1800.24 T1=35.78	? (MO) TT=35.3	SAT (TO) f=27628 TT=1800.08 T1=948.42	No Cert. TT=1800.07	SAT (TO) f=34180 TT=1800.07 T1=1765.75	? (TO) TT=1800.07	SAT (TO) f=23777 TT=1800.21 T1=124.36	SAT (TO) f=42089 TT=1800.27 T1=1468.8	SAT (TO) f=31456 TT=1800.7 T1=1119.36	? (TO) TT=1800.09	SAT (TO) f=27312 TT=1800.02 T1=1683.83
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-sentoy.opb	OPT f=-7772 TT=0.131 T1=0.09	OPT f=-7772 TT=5.883 T1=0.49	OPT f=-7772 TT=5.662 T1=3.49	OPT f=-7772 TT=2.235 T1=1.39	OPT f=-7772 TT=0.131 T1=0.09	SAT (TO) f=-6013 TT=1800.1 T1=1762.88	SAT (TO) f=-6812 TT=1800.05 T1=1581.57	? (TO) TT=1800.09	? (TO) TT=1800.13	No Cert. TT=1800.05	SAT (TO) f=-7551 TT=1800.04 T1=1800.02	SAT f=-3883 TT=1782.73 T1=1782.63	OPT f=-7772 TT=58.567 T1=19.35	SAT (TO) f=-7675 TT=1800.15 T1=0.49	OPT f=-7772 TT=17.144 T1=9.22	SAT (TO) f=-7772 TT=1800.11 T1=1457.88	SAT (TO) f=-7772 TT=1800.07 T1=279.91
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-stein45.opb	OPT f=30 TT=23.669 T1=0	OPT f=30 TT=240.496 T1=1.12	OPT f=30 TT=261.079 T1=1.1	OPT f=30 TT=164.781 T1=0.99	OPT f=30 TT=94.824 T1=94.76	OPT f=30 TT=23.669 T1=0.39	SAT (TO) f=30 TT=1800.14 T1=20.58	OPT f=30 TT=91.615 T1=0.95	OPT f=30 TT=64.795 T1=0.06	No Cert. TT=1800.03	SAT (TO) f=30 TT=1800.04 T1=0	OPT f=30 TT=91.751 T1=0	SAT (TO) f=30 TT=1800.15 T1=6.2	SAT (TO) f=30 TT=1800.32 T1=256.24	SAT (TO) f=30 TT=1800.18 T1=0.65	SAT (TO) f=30 TT=1800.09 T1=0	SAT (TO) f=30 TT=1800.06 T1=4.29
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/plato.asu.edu/pub/ milp/normalized-mps-v2-20-10-neos10.opb	OPT f=-1135 TT=11.422 T1=10.61	? TT=7.763	OPT f=-1135 TT=54.671 T1=27.28	OPT f=-1135 TT=27.395 T1=27.29	? (TO) TT=1800.04	OPT f=-1135 TT=222.291 T1=211.39	SAT (TO) f=-1132 TT=1800.57 T1=1398.15	? (TO) TT=1800.14	? (TO) TT=1800.06	OPT f=-1135 TT=11.478 T1=11.39	OPT f=-1135 TT=11.422 T1=11.38	OPT f=-1135 TT=17.805 T1=17.68	SAT (TO) f=-1100 TT=1800.5 T1=1697.75	SAT (TO) f=-1101 TT=1800.57 T1=1449.3	OPT f=-1135 TT=12.467 T1=10.61	? (TO) TT=1800.14	SAT (TO) f=-456 TT=1800.17 T1=1502.4
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/plato.asu.edu/pub/ milp/normalized-mps-v2-20-10-neos16.opb	SAT f=114 TT=1782.73 T1=20.28	? (TO) TT=1800.07	SAT f=119 TT=1800.01 T1=1385.28	SAT (TO) f=119 TT=1800.08 T1=1343.55	? (TO) TT=1800.07	SAT (TO) f=115 TT=1800.12 T1=1201.52	SAT (TO) f=128 TT=1800.08 T1=103.56	? (MO) TT=42.19	SAT (TO) f=114 TT=1802.06 T1=330.07	No Cert. TT=1800.08	SAT (TO) f=114 TT=1800.07 T1=20.28	SAT f=114 TT=1782.73 T1=189.14	SAT (TO) f=119 TT=1800.33 T1=52.18	SAT (TO) f=116 TT=1800.24 T1=86.46	SAT (TO) f=114 TT=1800.17 T1=814.67	? (TO) TT=1800.02	? (TO) TT=1800.04
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/plato.asu.edu/pub/ milp/normalized-mps-v2-20-10-neos18.opb	OPT f=16 TT=0.315 T1=0.08	? (TO) TT=1800.15	OPT f=16 TT=1.461 T1=0.09	OPT f=16 TT=1.409 T1=0.08	? (TO) TT=1800.06	OPT f=16 TT=0.315 T1=0.19	SAT (TO) f=17 TT=1800.29 T1=214.94	OPT f=16 TT=6.837 T1=6.01	OPT f=16 TT=5.236 T1=4.43	OPT f=16 TT=2.259 T1=0.19	OPT f=16 TT=2.233 T1=0.19	OPT f=16 TT=0.655 T1=0.19	OPT f=16 TT=13.501 T1=7.12	OPT f=16 TT=13.903 T1=10.05	OPT f=16 TT=3.19 T1=2.73	SAT (TO) f=16 TT=1800.04 T1=4.39	SAT (TO) f=16 TT=1800.08 T1=2.89
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/plato.asu.edu/pub/ milp/normalized-mps-v2-20-10-neos19.opb	SAT f=-13 TT=1800.75 T1=1800.02	? (MO) TT=16.662	SAT (TO) f=-2 TT=1800.65 T1=208.2	SAT (TO) f=-2 TT=1800.62 T1=206.74	? (TO) TT=1800.11	? (MO) TT=320.601	? (TO) TT=1800.56	? (TO) TT=1802.85	? (TO) TT=1802.23	No Cert. TT=1800.15	SAT (TO) f=-13 TT=1800.75 T1=1800.02	SAT f=-3 TT=1783.17 T1=1782.65	? (TO) TT=1800.2	? (TO) TT=1800.28	? (TO) TT=1800.21	? (MO) TT=18.872	? (MO) TT=17.961
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/plato.asu.edu/pub/milp/ normalized-mps-v2-20-10-neos1.opb	OPT f=19 TT=28.035 T1=18.59	? (TO) TT=1800.08	SAT f=26 TT=1800.05 T1=17.98	SAT (TO) f=26 TT=1800.09 T1=17.59	? (TO) TT=1800.08	OPT f=19 TT=28.035 T1=18.59	SAT (TO) f=63 TT=1800.11 T1=89.57	SAT (TO) f=79 TT=1802 T1=1691.75	SAT (TO) f=27 TT=1802.15 T1=10.66	No Cert. TT=1800.13	SAT (TO) f=37 TT=1800.07 T1=1061.45	SAT f=26 TT=1782.74 T1=4.3	SAT (TO) f=24 TT=1800.26 T1=4.22	SAT (TO) f=24 TT=1800.42 T1=4.18	SAT (TO) f=24 TT=1800.6 T1=2.45	SAT (TO) f=19 TT=1800.03 T1=101.66	SAT (TO) f=19 TT=1800.11 T1=438.24
Bench name	Best results on this instance	absconPseudo 102	bsolo 3.0.16	bsolo 3.0.17	glpPB 0.2	minisat+ 1.14	oree 0.1.2 alpha	PB-clasp 2007-03-23	PB-clasp 2007-04-10	PBS4 2007-03-23	PBS4_v2 2007-03-23	Pueblo 1.4	sat4jPseudoCP 2007-03-23	sat4jPseudoCPClause 2007-03-23	SAT4JPseudoResolution 2007-03-23	wildcat-rnp 2007-03-21	wildcat-skc 2007-03-21
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/plato.asu.edu/pub/milp/ normalized-mps-v2-20-10-neos5.opb	SAT f=15363 TT=1800.08 T1=1370.68	SAT (TO) f=21504 TT=1800.17 T1=0.48	SAT f=15780 TT=1800.01 T1=108.62	SAT (TO) f=15746 TT=1800.09 T1=1668.41	? (TO) TT=1800.1	SAT (TO) f=15872 TT=1800.11 T1=151.64	SAT (TO) f=21504 TT=1800.33 T1=1.99	? (MO) TT=31.74	SAT (TO) f=16384 TT=1802.06 T1=210.57	No Cert. TT=1800.08	SAT (TO) f=17408 TT=1800.05 T1=224.13	SAT f=17408 TT=1782.73 T1=439.88	SAT (TO) f=17859 TT=1800.27 T1=26.85	SAT (TO) f=20447 TT=1800.22 T1=2.54	SAT (TO) f=17644 TT=1800.27 T1=1768.65	SAT (TO) f=15363 TT=1800.08 T1=1370.68	SAT (TO) f=16384 TT=1800.03 T1=1.1
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/plato.asu.edu/pub/milp/ normalized-mps-v2-20-10-neos8.opb	OPT f=-3719 TT=4.485 T1=4.39	? TT=8.408	OPT f=-3719 TT=24.641 T1=24.52	OPT f=-3719 TT=24.588 T1=24.49	? (TO) TT=1800.08	OPT f=-3719 TT=270.271 T1=268.61	OPT f=-3719 TT=28.316 T1=16.97	? (TO) TT=1800.09	? (TO) TT=1800.08	OPT f=-3719 TT=6.695 T1=6.6	OPT f=-3719 TT=6.74 T1=6.69	OPT f=-3719 TT=4.485 T1=4.39	OPT f=-3719 TT=292.141 T1=216.19	OPT f=-3719 TT=1178.6 T1=97.29	OPT f=-3719 TT=8.837 T1=8.02	? (TO) TT=1800.18	SAT (TO) f=-2507 TT=1800.18 T1=1134.19

Some statistics...

	absconPseudo 102	bsolo 3.0.17	glpPB 0.2	minisat+ 1.14	oree 0.1.2 alpha	PB-clasp 2007-03-23	PB-clasp 2007-04-10	PBS4 2007-03-23	PBS4_v2 2007-03-23	Pueblo 1.4	sat4jPseudoCP 2007-03-23	sat4jPseudoCPClause 2007-03-23	SAT4JPseudoResolution 2007-03-23	wildcat-rnp 2007-03-21	wildcat-skc 2007-03-21
Number of times the solver is able to give the best known answer	8	16	13	11	4	2	7	8	11	11	9	7	14	3	3
Number of times the solver is able to give the best known answer from an incomplete solver point of view (i.e. without considering optimality proof)	10	16	13	11	5	2	8	8	12	11	11	8	16	13	12
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)	1	3	8	4	0	0	0	2	3	3	2	1	3	3	3
Number of times the solver is the best solver from an incomplete solver point of view (i.e. best known answer and best T1 time)	2	8	5	2	0	0	0	1	4	4	1	1	4	4	3