PB'16 competition: satisfaction and optimization track: results by benchmark

Results by benchmark for category optimisation, big integers, non linear constraints (OPT-BIGINT-NLC)

This page displays the results of the different solvers for each benchmark for category optimisation, big integers, non linear constraints (OPT-BIGINT-NLC)

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell example	Meaning
Answer	Solver result
f=...	value of the objective function for the model reported by the solver
TT=...	Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out

Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

Abbreviation	Meaning
f=...	Value of the objective function
TO	Time Out
MO	Mem. Out (out of memory)

Meaning of the different colors:

Color	Meaning
text	the solver cannot handle this instance
text	the solver gave no answer
text	the solver could give an answer (SAT)
text	the solver gave a definitive answer (OPTIMUM FOUND or UNSAT)
text	the solver performed better than the other ones on that instance (complete solvers point of view)
text	the solver performed better than the other ones on that instance (incomplete solvers point of view)
text	the solver was ended by a signal or other problem
text	the solver gave an incomplete answer
text	the solver gave a wrong answer

For better readability, you may choose to hide some solvers:
minisatp 2012-10-02 git-d91742b (complete)
Sat4j PB 2.3.6 Res+CP PB16 (complete)
Sat4j PB 2.3.6 Resolution PB16 (complete)
toysat 2016-05-02 (complete)

Bench name	Best results on this instance	minisatp 2012-10-02 git-d91742b (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/opb/ normalized-autocorr_bern40-40.opb	SAT f=-15954 TT=1800.1 T1=1800.4	SAT (TO) f=-15954 TT=1800.1 T1=1800.4	SAT (TO) f=-14742 TT=1800.38 T1=922.55	SAT (TO) f=-14832 TT=1800.03 T1=1752.22	? (TO) TT=1800.09
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/opb/ normalized-autocorr_bern45-23.opb	SAT f=-21016 TT=1800.11 T1=1800.4	SAT (TO) f=-21016 TT=1800.11 T1=1800.4	SAT (TO) f=-12385 TT=1800.06 T1=908.06	SAT (TO) f=-13535 TT=1800.01 T1=1767.62	? (TO) TT=1800.1
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/opb/ normalized-autocorr_bern45-34.opb	SAT f=-18063 TT=1800.1 T1=1800.4	SAT (TO) f=-18063 TT=1800.1 T1=1800.4	SAT (TO) f=-13187 TT=1800.23 T1=918.12	SAT (TO) f=-14590 TT=1800.07 T1=1765.84	? (TO) TT=1800.01
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/opb/ normalized-autocorr_bern45-45.opb	SAT f=-26484 TT=1800.1 T1=1800.5	SAT (TO) f=-26484 TT=1800.1 T1=1800.5	SAT (TO) f=-23412 TT=1800.24 T1=952.67	SAT (TO) f=-23731 TT=1800.02 T1=1433.19	? (TO) TT=1800.01
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/opb/ normalized-autocorr_bern50-25.opb	SAT f=-30412 TT=1800.03 T1=1800.4	SAT (TO) f=-30412 TT=1800.03 T1=1800.4	SAT (TO) f=-14895 TT=1800.36 T1=881.13	SAT (TO) f=-16683 TT=1800.6 T1=1766.04	? (TO) TT=1800.02
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/opb/ normalized-autocorr_bern50-38.opb	SAT f=-25180 TT=1800.06 T1=1800.4	SAT (TO) f=-25180 TT=1800.06 T1=1800.4	SAT (TO) f=-18054 TT=1800.07 T1=921.85	SAT (TO) f=-19845 TT=1800.05 T1=1530.02	? (TO) TT=1800.05
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/opb/ normalized-autocorr_bern50-50.opb	SAT f=-19950 TT=1800.1 T1=1800.4	SAT (TO) f=-19950 TT=1800.1 T1=1800.4	SAT (TO) f=-16901 TT=1800.04 T1=831.14	SAT (TO) f=-17401 TT=1800.98 T1=1776.41	? (TO) TT=1800.01
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/opb/ normalized-autocorr_bern55-28.opb	SAT f=-46719 TT=1800.04 T1=1800.4	SAT (TO) f=-46719 TT=1800.04 T1=1800.4	SAT (TO) f=-19193 TT=1800.27 T1=932.83	SAT (TO) f=-21686 TT=1800.03 T1=1432.69	? (TO) TT=1800.01
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/opb/ normalized-autocorr_bern55-41.opb	SAT f=-82163 TT=1800.08 T1=1800.4	SAT (TO) f=-82163 TT=1800.08 T1=1800.4	SAT (TO) f=-45635 TT=1800.37 T1=946.38	SAT (TO) f=-51399 TT=1800.08 T1=1701.66	? (TO) TT=1800.1
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/opb/ normalized-autocorr_bern55-55.opb	SAT f=-56529 TT=1800.09 T1=1800.4	SAT (TO) f=-56529 TT=1800.09 T1=1800.4	SAT (TO) f=-43356 TT=1800.33 T1=993.06	SAT (TO) f=-46930 TT=1800.76 T1=1163.21	? (TO) TT=1800.11
Bench name	Best results	minisatp 2012-10-02 git-d91742b (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/opb/ normalized-autocorr_bern60-30.opb	SAT f=-31856 TT=1800.09 T1=1800.4	SAT (TO) f=-31856 TT=1800.09 T1=1800.4	SAT (TO) f=-11331 TT=1800.06 T1=821.05	SAT (TO) f=-12650 TT=1800.01 T1=1727.46	? (TO) TT=1800.01
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/opb/ normalized-autocorr_bern60-45.opb	SAT f=-114288 TT=1800.09 T1=1800.4	SAT (TO) f=-114288 TT=1800.09 T1=1800.4	SAT (TO) f=-49825 TT=1800.32 T1=961.21	SAT (TO) f=-64542 TT=1800.46 T1=1573.18	? (TO) TT=1800.09
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/opb/ normalized-autocorr_bern60-60.opb	SAT f=-83949 TT=1800.09 T1=1800.4	SAT (TO) f=-83949 TT=1800.09 T1=1800.4	SAT (TO) f=-54902 TT=1800.17 T1=802.86	SAT (TO) f=-63175 TT=1800.07 T1=1574.36	? (TO) TT=1800.11
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/ opb/normalized-cardqp_inlp.opb	SAT f=816688461479 TT=1800.66 T1=423.69	SAT (TO) f=1138165245729 TT=1800.02 T1=1800.3	SAT (TO) f=816688461479 TT=1800.66 T1=423.69	SAT (TO) f=826410679589 TT=1800.08 T1=108.31	? (MO) TT=228.989
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/ minlplib2-pb-0.1.0/opb/normalized-crossdock_15x8.opb	SAT f=16087 TT=1800.09 T1=1800.4	SAT (TO) f=16087 TT=1800.09 T1=1800.4	SAT (TO) f=16178 TT=1801.59 T1=723.23	SAT (TO) f=16202 TT=1800.03 T1=0.53	? (TO) TT=1800.09
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/ opb/normalized-faclay60.opb	SAT f=1681658 TT=1800.05 T1=1131.63	SAT (TO) f=2001841 TT=1800.14 T1=1800.5	SAT (TO) f=1682418 TT=1800.05 T1=1606.14	SAT (TO) f=1681658 TT=1800.05 T1=1131.63	? (TO) TT=1800.11
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/ opb/normalized-faclay70.opb	SAT f=1906945 TT=1055.92 T1=202.79	SAT (TO) f=2223662 TT=1800.12 T1=1800.5	SAT (TO) f=1907167 TT=1800.43 T1=1659.19	SAT f=1906945 TT=1055.92 T1=202.79	? (TO) TT=1800.1
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/ opb/normalized-faclay75.opb	SAT f=2669211 TT=610.042 T1=3.16	SAT (TO) f=3145596 TT=1800.15 T1=1800.5	SAT (TO) f=2669211 TT=1800.58 T1=10.47	SAT f=2669211 TT=610.042 T1=3.16	? (TO) TT=1800.1
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/ opb/normalized-faclay80.opb	SAT f=4789911 TT=646.217 T1=590.85	SAT (TO) f=5780747 TT=1800.15 T1=1800.5	SAT (TO) f=4790849 TT=1800.12 T1=1613.73	SAT f=4789911 TT=646.217 T1=590.85	? (TO) TT=1800.05
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/ opb/normalized-graphpart_2g-0044-1601.opb	OPT f=-954077 TT=0.683 T1=0.31	OPT f=-954077 TT=328.888 T1=328.94	OPT f=-954077 TT=2.184 T1=3.66	OPT f=-954077 TT=0.683 T1=0.31	OPT f=-954077 TT=1.483 T1=0.43
Bench name	Best results	minisatp 2012-10-02 git-d91742b (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/ opb/normalized-graphpart_2g-0055-0062.opb	OPT f=-1484348 TT=10.942 T1=1.38	OPT f=-1484348 TT=788.316 T1=788.44	OPT f=-1484348 TT=25.649 T1=12.7	OPT f=-1484348 TT=10.942 T1=1.38	OPT f=-1484348 TT=131.03 T1=3.11
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/ opb/normalized-graphpart_2g-0066-0066.opb	OPT f=-2865560 TT=1035.79 T1=1035.95	OPT f=-2865560 TT=1035.79 T1=1035.95	SAT (TO) f=-2610196 TT=1800.06 T1=6.2	SAT (TO) f=-2496488 TT=1800.02 T1=0.2	? (TO) TT=1800.01
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/ opb/normalized-graphpart_2g-0077-0077.opb	SAT f=-3229106 TT=1800.01 T1=225.99	SAT (TO) f=-1800958 TT=1800.02 T1=1800.3	SAT (TO) f=-2996225 TT=1800.62 T1=1.69	SAT (TO) f=-3229106 TT=1800.01 T1=225.99	? (TO) TT=1800.04
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/ opb/normalized-graphpart_2g-0088-0088.opb	SAT f=-5170311 TT=1800.02 T1=0.4	SAT (TO) f=-3663064 TT=1800.02 T1=1800.3	SAT (TO) f=-5170311 TT=1800.6 T1=2.77	SAT (TO) f=-5170311 TT=1800.02 T1=0.4	? (TO) TT=1800.1
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/ opb/normalized-graphpart_2g-0099-9211.opb	SAT f=-4287109 TT=1800.69 T1=1.04	SAT (TO) f=-2331750 TT=1800.03 T1=1800.3	SAT (TO) f=-4287109 TT=1800.72 T1=7.77	SAT (TO) f=-4287109 TT=1800.69 T1=1.04	? (TO) TT=1800.06
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/ opb/normalized-graphpart_2g-1010-0824.opb	SAT f=-6046909 TT=1800.04 T1=2.5	SAT (TO) f=-1834425 TT=1800.02 T1=1800.3	SAT (TO) f=-5994097 TT=1800.57 T1=11.8	SAT (TO) f=-6046909 TT=1800.04 T1=2.5	? (TO) TT=1800.05
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/ opb/normalized-graphpart_3g-0234-0234.opb	OPT f=-1952753 TT=137.243 T1=66.65	SAT (TO) f=-1850578 TT=1800.01 T1=1800.3	OPT f=-1952753 TT=389.506 T1=151.26	OPT f=-1952753 TT=137.243 T1=66.65	? (TO) TT=1800.02
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/ opb/normalized-graphpart_3g-0244-0244.opb	SAT f=-2521044 TT=1800.02 T1=12.64	SAT (TO) f=-1615633 TT=1800.02 T1=1800.3	SAT (TO) f=-2414467 TT=1800.18 T1=889.28	SAT (TO) f=-2521044 TT=1800.02 T1=12.64	? (TO) TT=1800.04
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/ opb/normalized-graphpart_3g-0333-0333.opb	SAT f=-1882389 TT=1800 T1=247.15	SAT (TO) f=-1348640 TT=1800.02 T1=1800.3	SAT (TO) f=-1882389 TT=1800.52 T1=493.05	SAT (TO) f=-1882389 TT=1800 T1=247.15	? (TO) TT=1800.13
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/ opb/normalized-graphpart_3g-0334-0334.opb	SAT f=-2953024 TT=1800.64 T1=3.2	SAT (TO) f=-1812983 TT=1800.02 T1=1800.3	SAT (TO) f=-2953024 TT=1800.64 T1=3.2	SAT (TO) f=-2863952 TT=1800.03 T1=0.21	? (TO) TT=1800.01
Bench name	Best results	minisatp 2012-10-02 git-d91742b (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/ opb/normalized-graphpart_3g-0344-0344.opb	SAT f=-4947487 TT=1800.12 T1=4.25	SAT (TO) f=-2413925 TT=1800.02 T1=1800.3	SAT (TO) f=-4947487 TT=1800.12 T1=4.25	SAT (TO) f=-4576984 TT=1800.07 T1=0.48	? (TO) TT=1800.11
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/ opb/normalized-graphpart_3g-0444-0444.opb	SAT f=-6194804 TT=1800.04 T1=0.22	SAT (TO) f=-3692302 TT=1800.03 T1=1800.3	SAT (TO) f=-6194804 TT=1800.62 T1=0.73	SAT (TO) f=-6194804 TT=1800.04 T1=0.22	? (TO) TT=1800.11
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/ opb/normalized-pb302035.opb	SAT f=4211459 TT=1800.61 T1=7.91	SAT (TO) f=8751206 TT=1800.04 T1=1800.3	SAT (TO) f=4215155 TT=1800.04 T1=16.34	SAT (TO) f=4211459 TT=1800.61 T1=7.91	? (TO) TT=1800.11
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/ opb/normalized-pb302055.opb	SAT f=4372559 TT=1800.56 T1=7.3	SAT (TO) f=8755941 TT=1800.12 T1=1800.4	SAT (TO) f=4372559 TT=1800.56 T1=632.35	SAT (TO) f=4372559 TT=1800.66 T1=7.3	? (TO) TT=1800.06
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/ opb/normalized-pb302075.opb	SAT f=4628361 TT=1800.08 T1=726.63	SAT (TO) f=7848561 TT=1800.09 T1=1800.4	SAT (TO) f=4644489 TT=1800.71 T1=12.5	SAT (TO) f=4628361 TT=1800.08 T1=726.63	? (TO) TT=1800.14
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/ opb/normalized-pb302095.opb	SAT f=6656449 TT=1800.76 T1=11.63	? (TO) TT=1800.02	SAT (TO) f=6656449 TT=1800.76 T1=11.63	SAT (TO) f=7155970 TT=1800.07 T1=1205.95	? (TO) TT=1800.16
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/ opb/normalized-pb351535.opb	SAT f=5590266 TT=1800.48 T1=6.03	SAT (TO) f=10400405 TT=1800.11 T1=1800.4	SAT (TO) f=5599936 TT=1800.61 T1=980.2	SAT (TO) f=5590266 TT=1800.48 T1=6.03	? (TO) TT=1800.23
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/ opb/normalized-pb351555.opb	SAT f=5239782 TT=1800.02 T1=2.68	SAT (TO) f=10755911 TT=1800.03 T1=1800.3	SAT (TO) f=5239782 TT=1800.02 T1=5.57	SAT (TO) f=5239782 TT=1800.74 T1=2.68	? (TO) TT=1800.09
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/ opb/normalized-pb351575.opb	SAT f=6785589 TT=1800.7 T1=648.78	SAT (TO) f=11462323 TT=1800.03 T1=1800.3	SAT (TO) f=6785589 TT=1800.7 T1=648.78	SAT (TO) f=6792554 TT=1800.68 T1=7.03	? (TO) TT=1800.06
PB15eval/normalized-PB15eval/ OPT-BIGINT-NLC/minlplib2-pb-0.1.0/ opb/normalized-pb351595.opb	SAT f=7328208 TT=1800.55 T1=3.93	SAT (TO) f=11884420 TT=1800.04 T1=1800.3	SAT (TO) f=7328208 TT=1800.55 T1=3.93	SAT (TO) f=7405964 TT=1800.07 T1=50.48	? (TO) TT=1800.21
Bench name	Best results	minisatp 2012-10-02 git-d91742b (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
PB15eval/normalized-PB15eval/OPT-BIGINT-NLC/ minlplib2-pb-0.1.0/opb/normalized-qap.opb	SAT f=210318 TT=1800.6 T1=264.93	SAT (TO) f=228216 TT=1800.02 T1=1800.3	SAT (TO) f=210318 TT=1800.6 T1=264.93	SAT (TO) f=212944 TT=1800.03 T1=1538.58	? (TO) TT=1800.08

Statistics		minisatp 2012-10-02 git-d91742b (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
Number of times the solver is able to give the best known answer		17	17	19	2
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)		15	13	19	0