PB'16 competition: satisfaction and optimization track: results by benchmark

Results by benchmark for category optimisation, small integers, linear constraints (OPT-SMALLINT-LIN), subcategory Handmade, subsubcategory MPS to OPB

This page displays the results of the different solvers for each benchmark for category optimisation, small integers, linear constraints (OPT-SMALLINT-LIN), subcategory Handmade, subsubcategory MPS to OPB

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell example	Meaning
Answer	Solver result
f=...	value of the objective function for the model reported by the solver
TT=...	Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out

Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

Abbreviation	Meaning
f=...	Value of the objective function
TO	Time Out
MO	Mem. Out (out of memory)

Meaning of the different colors:

Color	Meaning
text	the solver cannot handle this instance
text	the solver gave no answer
text	the solver could give an answer (SAT)
text	the solver gave a definitive answer (OPTIMUM FOUND or UNSAT)
text	the solver performed better than the other ones on that instance (complete solvers point of view)
text	the solver performed better than the other ones on that instance (incomplete solvers point of view)
text	the solver was ended by a signal or other problem
text	the solver gave an incomplete answer
text	the solver gave a wrong answer

For better readability, you may choose to hide some solvers:
cdcl-cuttingplanes OPT binary search 2016-05-01 (complete)
cdcl-cuttingplanes OPT linear search 2016-05-01 (complete)
minisatp 2012-10-02 git-d91742b (complete)
NaPS 1.02 (complete)
Open-WBO PB16 (complete)
Open-WBO-LSU PB16 (complete)
Sat4j PB 2.3.6 Res+CP PB16 (complete)
Sat4j PB 2.3.6 Resolution PB16 (complete)
toysat 2016-05-02 (complete)

Bench name	Best results on this instance	cdcl-cuttingplanes OPT binary search 2016-05-01 (complete)	cdcl-cuttingplanes OPT linear search 2016-05-01 (complete)	minisatp 2012-10-02 git-d91742b (complete)	NaPS 1.02 (complete)	Open-WBO PB16 (complete)	Open-WBO-LSU PB16 (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib2003/ normalized-mps-v2-20-10-10teams.opb	SAT f=914 TT=1800.03 T1=1800.3	? (TO) TT=1800.06	? (TO) TT=1800.02	SAT (TO) f=926 TT=1800.01 T1=1800.3	SAT (TO) f=972 TT=1800.11 T1=1800.4	SAT (TO) f=914 TT=1800.03 T1=1800.3	SAT (TO) f=922 TT=1800.07 T1=1800.4	SAT (TO) f=916 TT=1800.87 T1=47.63	SAT (TO) f=916 TT=1800.09 T1=128.78	? (TO) TT=1800.02
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib2003/ normalized-mps-v2-20-10-fast0507.opb	SAT f=251 TT=1800.09 T1=1800.4	? (TO) TT=1800.04	? (TO) TT=1800.03	SAT (TO) f=349 TT=1800.12 T1=1800.5	SAT (TO) f=349 TT=1800.1 T1=1800.4	SAT (TO) f=251 TT=1800.09 T1=1800.4	SAT (TO) f=294 TT=1800.06 T1=1800.4	SAT (TO) f=259 TT=1800.05 T1=80.62	SAT (TO) f=259 TT=1800.09 T1=1.37	? (TO) TT=1800.15
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib2003/ normalized-mps-v2-20-10-manna81.opb	SAT f=-11615 TT=1800.1 T1=1800.4	? (TO) TT=1800.1	? (TO) TT=1800.04	SAT (TO) f=-11615 TT=1800.1 T1=1800.4	SAT (TO) f=-10793 TT=1800.06 T1=1800.4	SAT (TO) f=-9792 TT=1800.08 T1=1800.4	SAT (TO) f=-11306 TT=1800.31 T1=1800.4	SAT (TO) f=-11521 TT=1800.12 T1=69.78	SAT (TO) f=-10910 TT=1800.06 T1=20.82	? (TO) TT=1800.04
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib2003/ normalized-mps-v2-20-10-mzzv11.opb	OPT f=-10859 TT=217.128 T1=217.11	? (TO) TT=1800.12	? (TO) TT=1800.02	SAT (TO) f=-9909 TT=1800.08 T1=1800.4	OPT f=-10859 TT=217.128 T1=217.11	OPT f=-10859 TT=250.064 T1=250.09	? TT=179.48	SAT (TO) f=-10319 TT=1800.05 T1=136.39	OPT f=-10859 TT=336.933 T1=333.31	? (TO) TT=1800.12
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib2003/ normalized-mps-v2-20-10-mzzv42z.opb	OPT f=-10270 TT=53.71 T1=53.66	? (TO) TT=1800.1	? (TO) TT=1800.02	OPT f=-10270 TT=456.118 T1=456.2	OPT f=-10270 TT=53.71 T1=53.66	OPT f=-10270 TT=170.322 T1=170.3	? TT=185.58	SAT (TO) f=-10171 TT=1800 T1=177.74	OPT f=-10270 TT=355.05 T1=351.42	OPT f=-10270 TT=65.911 T1=65.84
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib2003/ normalized-mps-v2-20-10-net12.opb	OPT f=214 TT=2.877 T1=2.84	OPT f=214 TT=4.988 T1=4.94	OPT f=214 TT=2.877 T1=2.84	? (TO) TT=1800.05	OPT f=214 TT=954.538 T1=953.92	? (MO) TT=61.054	? (MO) TT=61.599	OPT f=214 TT=17.072 T1=8.32	OPT f=214 TT=7.473 T1=4.58	OPT f=214 TT=40.068 T1=23.42
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib2003/ normalized-mps-v2-20-10-opt1217.opb	SAT f=-16384 TT=1800.75 T1=79.38	? (TO) TT=1800.01	? (TO) TT=1800.02	SAT (TO) f=-12288 TT=1800.02 T1=1800.3	SAT (TO) f=0 TT=1800.02 T1=1800.3	SAT (TO) f=0 TT=1800.31 T1=1800.3	? (TO) TT=1800.21	SAT (TO) f=-16384 TT=1800.75 T1=79.38	? (TO) TT=1800.05	? (TO) TT=1800.03
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib2003/ normalized-mps-v2-20-10-p2756.opb	SAT f=3671 TT=1800.66 T1=156.69	? (TO) TT=1800.01	? (TO) TT=1800.02	SAT (TO) f=151108 TT=1800.05 T1=1800.4	SAT (TO) f=3743 TT=1800.03 T1=1800.4	SAT (TO) f=9372 TT=1800.09 T1=1800.4	? TT=74.482	SAT (TO) f=3671 TT=1800.66 T1=156.69	SAT (TO) f=3719 TT=1800.95 T1=585.32	? (TO) TT=1800.05
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib2003/ normalized-mps-v2-20-10-protfold.opb	SAT f=-25 TT=1800.01 T1=1642.99	? (TO) TT=1800.1	? (TO) TT=1800.02	SAT (TO) f=-24 TT=1800.1 T1=1800.4	SAT (TO) f=-14 TT=1800.08 T1=1800.4	SAT (TO) f=-14 TT=1800.03 T1=1800.3	SAT (TO) f=-25 TT=1800.01 T1=1800.3	SAT (TO) f=-21 TT=1800.67 T1=463.44	SAT (TO) f=-25 TT=1800.8 T1=1642.99	? (TO) TT=1800.02
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib2003/ normalized-mps-v2-20-10-seymour.opb	SAT f=448 TT=1800.01 T1=1800.3	? (TO) TT=1800.01	? (TO) TT=1800.03	SAT (TO) f=457 TT=1800.02 T1=1800.3	SAT (TO) f=466 TT=1800.02 T1=1800.3	SAT (TO) f=466 TT=1800.02 T1=1800.3	SAT (TO) f=448 TT=1800.01 T1=1800.3	SAT (TO) f=477 TT=1800.04 T1=230.48	SAT (TO) f=490 TT=1800.06 T1=0.37	? (TO) TT=1800.05
Bench name	Best results	cdcl-cuttingplanes OPT binary search 2016-05-01 (complete)	cdcl-cuttingplanes OPT linear search 2016-05-01 (complete)	minisatp 2012-10-02 git-d91742b (complete)	NaPS 1.02 (complete)	Open-WBO PB16 (complete)	Open-WBO-LSU PB16 (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib3/ normalized-mps-v2-20-10-mitre.opb	SAT f=7680 TT=1800.15 T1=6.29	? (TO) TT=1800.01	? (TO) TT=1800.01	SAT (TO) f=9073 TT=1800.08 T1=1800.4	SAT (TO) f=9331 TT=1800.2 T1=1800.4	? TT=61.417	? TT=62.36	SAT (TO) f=7680 TT=1800.15 T1=6.29	SAT (TO) f=7941 TT=1800.04 T1=1.96	? (TO) TT=1800.01
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-bm23.opb	OPT f=34 TT=0.171 T1=0.17	OPT f=34 TT=0.294 T1=0.29	OPT f=34 TT=0.171 T1=0.17	OPT f=34 TT=1.184 T1=1.18	OPT f=34 TT=0.486 T1=0.48	OPT f=34 TT=0.527 T1=0.55	OPT f=34 TT=0.773 T1=0.78	OPT f=34 TT=1.305 T1=1.64	OPT f=34 TT=0.644 T1=0.23	OPT f=34 TT=0.986 T1=0.95
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-cracpb1.opb	SAT f=22199 TT=1800.09 T1=1800.4	? (TO) TT=1800.01	? (TO) TT=1800.02	SAT (TO) f=30188 TT=1800.1 T1=1800.4	SAT (TO) f=44407 TT=1800.09 T1=1800.4	SAT (TO) f=22199 TT=1800.09 T1=1800.4	? TT=76.441	SAT (TO) f=44331 TT=1800.12 T1=10.79	? (TO) TT=1800.9	? (TO) TT=1800.05
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-diamond.opb	UNSAT TT=0	UNSAT TT=0.021	UNSAT TT=0.012	UNSAT TT=0.001	UNSAT TT=0.001	UNSAT TT=0	UNSAT TT=0	UNSAT TT=0.165	UNSAT TT=0.154	UNSAT TT=0.016
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-l152lav.opb	SAT f=5556 TT=1800.47 T1=1800.3	? (TO) TT=1800.09	? (TO) TT=1800.1	? (TO) TT=1800.1	SAT (TO) f=5614 TT=1800.08 T1=1800.4	SAT (TO) f=5561 TT=1800.01 T1=1800.3	SAT (TO) f=5556 TT=1800.47 T1=1800.3	? (TO) TT=1800.07	? (TO) TT=1800.8	? (TO) TT=1800.02
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-lp4l.opb	SAT f=3666 TT=1800.12 T1=1800.4	? (TO) TT=1800.02	? (TO) TT=1800.01	SAT (TO) f=3666 TT=1800.12 T1=1800.4	SAT (TO) f=4551 TT=1800.09 T1=1800.4	SAT (TO) f=4001 TT=1800.08 T1=1800.4	SAT (TO) f=3899 TT=1800.08 T1=1800.4	SAT (TO) f=3792 TT=1800.77 T1=82.11	SAT (TO) f=3958 TT=1800.01 T1=1314.86	? (TO) TT=1800.08
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-lseu.opb	OPT f=1120 TT=34.554 T1=8.14	OPT f=1120 TT=85.011 T1=85.02	OPT f=1120 TT=234.073 T1=234.11	OPT f=1120 TT=37.752 T1=37.75	OPT f=1120 TT=34.554 T1=34.55	OPT f=1120 TT=607.395 T1=607.51	OPT f=1120 TT=275.005 T1=275	OPT f=1120 TT=53.026 T1=10.67	OPT f=1120 TT=49.597 T1=8.14	OPT f=1120 TT=113.609 T1=113.57
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-mod008.opb	OPT f=307 TT=1187.4 T1=0.33	? (TO) TT=1800.02	? (TO) TT=1800.02	SAT (TO) f=321 TT=1800.05 T1=1800.4	SAT (TO) f=409 TT=1800.08 T1=1800.4	? TT=99.756	? TT=98.893	OPT f=307 TT=1187.4 T1=1.2	SAT (TO) f=307 TT=1800.9 T1=0.33	? (TO) TT=1800.03
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-mod010.opb	SAT f=7984 TT=1800.16 T1=714.05	? (TO) TT=1800.07	? (TO) TT=1800.02	? (TO) TT=1800.11	? (TO) TT=1800.1	? (TO) TT=1800.1	? (TO) TT=1800.02	SAT (TO) f=7984 TT=1800.16 T1=714.05	? (TO) TT=1800.57	? (TO) TT=1800.03
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-p0033.opb	OPT f=3089 TT=0.016 T1=0.01	OPT f=3089 TT=0.048 T1=0.05	OPT f=3089 TT=0.016 T1=0.01	OPT f=3089 TT=0.271 T1=0.27	OPT f=3089 TT=0.207 T1=0.21	OPT f=3089 TT=0.033 T1=0.03	OPT f=3089 TT=0.608 T1=0.6	OPT f=3089 TT=0.723 T1=1.15	OPT f=3089 TT=0.404 T1=0.15	OPT f=3089 TT=0.154 T1=0.15
Bench name	Best results	cdcl-cuttingplanes OPT binary search 2016-05-01 (complete)	cdcl-cuttingplanes OPT linear search 2016-05-01 (complete)	minisatp 2012-10-02 git-d91742b (complete)	NaPS 1.02 (complete)	Open-WBO PB16 (complete)	Open-WBO-LSU PB16 (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-p0040.opb	OPT f=62027 TT=0.015 T1=0.01	OPT f=62027 TT=0.282 T1=0.28	OPT f=62027 TT=0.075 T1=0.07	OPT f=62027 TT=51.82 T1=51.82	OPT f=62027 TT=1.306 T1=1.3	OPT f=62027 TT=0.015 T1=0.01	? TT=128.93	OPT f=62027 TT=0.815 T1=1.83	OPT f=62027 TT=13.782 T1=0.13	OPT f=62027 TT=19.186 T1=10.37
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-p0201.opb	OPT f=1523 TT=436.099 T1=0.33	? (TO) TT=1800.02	? (TO) TT=1800.02	OPT f=1523 TT=544.158 T1=544.24	OPT f=1523 TT=436.099 T1=436.16	OPT f=1523 TT=612.866 T1=612.96	SAT (TO) f=1563 TT=1800.04 T1=1800.4	SAT (TO) f=1523 TT=1800.64 T1=3.72	SAT (TO) f=1523 TT=1800.09 T1=0.33	? (TO) TT=1800.06
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-p0548.opb	SAT f=10087 TT=1800.65 T1=3.22	? (TO) TT=1800.02	? (TO) TT=1800.02	SAT (TO) f=44199 TT=1800.02 T1=1800.3	SAT (TO) f=15217 TT=1800.1 T1=1800.4	SAT (TO) f=59642 TT=1800.07 T1=1800.4	? TT=85.08	SAT (TO) f=10087 TT=1800.65 T1=3.22	? (TO) TT=1800.06	? (TO) TT=1800.05
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-sentoy.opb	OPT f=-7772 TT=2.588 T1=0.54	? (TO) TT=1800.11	? (TO) TT=1800.01	SAT (TO) f=-7502 TT=1800.01 T1=1800.3	OPT f=-7772 TT=69.279 T1=69.27	? TT=74.237	? TT=74.968	OPT f=-7772 TT=4.562 T1=3.7	OPT f=-7772 TT=2.588 T1=0.54	OPT f=-7772 TT=12.606 T1=7.57
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-stein15.opb	OPT f=9 TT=0.003 T1=0	OPT f=9 TT=0.023 T1=0.02	OPT f=9 TT=0.013 T1=0.01	OPT f=9 TT=0.004 T1=0	OPT f=9 TT=0.006 T1=0	OPT f=9 TT=0.003 T1=0	OPT f=9 TT=0.003 T1=0	OPT f=9 TT=0.374 T1=0.63	OPT f=9 TT=0.263 T1=0.12	OPT f=9 TT=0.049 T1=0.02
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-stein27.opb	OPT f=18 TT=0.046 T1=0.04	OPT f=18 TT=0.329 T1=0.32	OPT f=18 TT=0.285 T1=0.28	OPT f=18 TT=0.165 T1=0.16	OPT f=18 TT=0.139 T1=0.13	OPT f=18 TT=0.046 T1=0.04	OPT f=18 TT=0.14 T1=0.14	OPT f=18 TT=5.502 T1=0.65	OPT f=18 TT=2.134 T1=0.13	OPT f=18 TT=2.517 T1=2.46
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-stein45.opb	OPT f=30 TT=7.583 T1=1.21	OPT f=30 TT=265.117 T1=265.15	OPT f=30 TT=235.005 T1=235.04	OPT f=30 TT=10.397 T1=10.39	OPT f=30 TT=7.583 T1=7.58	OPT f=30 TT=35.292 T1=35.29	OPT f=30 TT=9.199 T1=9.2	OPT f=30 TT=1723.22 T1=2.18	OPT f=30 TT=582.647 T1=1.21	OPT f=30 TT=420.124 T1=420.13
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/MIPLIB/miplib/ normalized-mps-v2-20-10-stein9.opb	OPT f=5 TT=0 T1=0	OPT f=5 TT=0.021 T1=0.02	OPT f=5 TT=0.012 T1=0.01	OPT f=5 TT=0.001 T1=0	OPT f=5 TT=0.001 T1=0	OPT f=5 TT=0 T1=0	OPT f=5 TT=0.001 T1=0	OPT f=5 TT=0.163 T1=0.62	OPT f=5 TT=0.164 T1=0.11	OPT f=5 TT=0.02 T1=0.01
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/plato.asu.edu/pub/ milp/normalized-mps-v2-20-10-neos10.opb	OPT f=-1135 TT=4.334 T1=2.36	OPT f=-1135 TT=18.457 T1=18.44	OPT f=-1135 TT=13.896 T1=13.89	OPT f=-1135 TT=79.025 T1=79.01	OPT f=-1135 TT=28.36 T1=28.32	OPT f=-1135 TT=92.54 T1=92.53	OPT f=-1135 TT=80.941 T1=80.78	OPT f=-1135 TT=104.858 T1=109.88	OPT f=-1135 TT=4.334 T1=2.36	OPT f=-1135 TT=19.078 T1=19.02
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/plato.asu.edu/pub/ milp/normalized-mps-v2-20-10-neos16.opb	SAT f=114 TT=1800.01 T1=40.32	? (TO) TT=1800.02	? (TO) TT=1800.02	SAT (TO) f=114 TT=1800.02 T1=1800.3	SAT (TO) f=114 TT=1800.02 T1=1800.3	SAT (TO) f=122 TT=1800.01 T1=1800.3	? (TO) TT=1800.02	SAT (TO) f=114 TT=1800.18 T1=40.32	SAT (TO) f=114 TT=1800.01 T1=370.93	? (TO) TT=1800.08
Bench name	Best results	cdcl-cuttingplanes OPT binary search 2016-05-01 (complete)	cdcl-cuttingplanes OPT linear search 2016-05-01 (complete)	minisatp 2012-10-02 git-d91742b (complete)	NaPS 1.02 (complete)	Open-WBO PB16 (complete)	Open-WBO-LSU PB16 (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/plato.asu.edu/pub/ milp/normalized-mps-v2-20-10-neos18.opb	OPT f=16 TT=0.043 T1=0.04	OPT f=16 TT=0.182 T1=0.18	OPT f=16 TT=0.131 T1=0.13	OPT f=16 TT=0.115 T1=0.11	OPT f=16 TT=0.106 T1=0.1	OPT f=16 TT=0.043 T1=0.04	OPT f=16 TT=0.047 T1=0.04	OPT f=16 TT=3.59 T1=4.46	OPT f=16 TT=1.507 T1=0.51	OPT f=16 TT=0.617 T1=0.24
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/plato.asu.edu/pub/ milp/normalized-mps-v2-20-10-neos19.opb		? (exit code) TT=1.354	? (exit code) TT=0.006	? (TO) TT=1800.31	? (TO) TT=1800.13	? TT=38.082	? TT=38.136	? (TO) TT=1800.19	? (TO) TT=1800.08	? (TO) TT=1800.27
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/plato.asu.edu/pub/milp/ normalized-mps-v2-20-10-neos1.opb	OPT f=19 TT=3.841 T1=3.83	? (TO) TT=1800.01	? (TO) TT=1800.02	OPT f=19 TT=3.841 T1=3.83	OPT f=19 TT=8.465 T1=8.46	OPT f=19 TT=5.774 T1=5.77	OPT f=19 TT=6.279 T1=6.27	SAT (TO) f=22 TT=1800.8 T1=117.48	SAT (TO) f=22 TT=1800.81 T1=1122.57	? (TO) TT=1800.05
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/plato.asu.edu/pub/milp/ normalized-mps-v2-20-10-neos5.opb	OPT f=15360 TT=465.143 T1=465.21	? (TO) TT=1800.02	? (TO) TT=1800.02	SAT (TO) f=15488 TT=1800.02 T1=1800.3	OPT f=15360 TT=465.143 T1=465.21	SAT (TO) f=16409 TT=1800.03 T1=1800.4	SAT (TO) f=16288 TT=1800.04 T1=1800.4	SAT (TO) f=16430 TT=1800.66 T1=707.18	SAT (TO) f=19713 TT=1800.02 T1=1062.77	? (TO) TT=1800.1
normalized-PB06/OPT-SMALLINT/ mps-v2-20-10/plato.asu.edu/pub/milp/ normalized-mps-v2-20-10-neos8.opb	OPT f=-3719 TT=0.899 T1=0.89	OPT f=-3719 TT=0.899 T1=0.89	OPT f=-3719 TT=3.575 T1=3.56	OPT f=-3719 TT=120.375 T1=120.36	OPT f=-3719 TT=27.438 T1=27.39	OPT f=-3719 TT=31.353 T1=31.32	OPT f=-3719 TT=49.49 T1=49.28	OPT f=-3719 TT=6.751 T1=10.13	OPT f=-3719 TT=3.255 T1=1.82	OPT f=-3719 TT=14.989 T1=13.61

Statistics		cdcl-cuttingplanes OPT binary search 2016-05-01 (complete)	cdcl-cuttingplanes OPT linear search 2016-05-01 (complete)	minisatp 2012-10-02 git-d91742b (complete)	NaPS 1.02 (complete)	Open-WBO PB16 (complete)	Open-WBO-LSU PB16 (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
Number of times the solver is able to give the best known answer		13	13	18	20	19	15	21	18	15
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)		1	3	4	7	8	5	7	4	0