PB'10 competition: satisfaction and optimization track: results by benchmark

Results by benchmark for category no optimisation, small integers, linear constraints (DEC-SMALLINT-LIN), subcategory Handmade, subsubcategory Graph Problems

This page displays the results of the different solvers for each benchmark for category no optimisation, small integers, linear constraints (DEC-SMALLINT-LIN), subcategory Handmade, subsubcategory Graph Problems

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell example	Meaning
Answer	Solver result
f=...	value of the objective function for the model reported by the solver
TT=...	Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out

Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

Abbreviation	Meaning
f=...	Value of the objective function
TO	Time Out
MO	Mem. Out (out of memory)

Meaning of the different colors:

Color	Meaning
text	the solver cannot handle this instance
text	the solver gave no answer
text	the solver could give an answer (SAT)
text	the solver gave a definitive answer (OPTIMUM FOUND or UNSAT)
text	the solver performed better than the other ones on that instance (complete solvers point of view)
text	the solver performed better than the other ones on that instance (incomplete solvers point of view)
text	the solver was ended by a signal or other problem
text	the solver gave an incomplete answer
text	the solver gave a wrong answer

For better readability, you may choose to hide some solvers:
borg-pb 10.05.30 (complete)
bsolo 3.2 Card (complete)
bsolo 3.2 Cl (complete)
pb_cplex 2010-06-29 (complete)
PB/CT 0.1 (complete)
PB/CT 0.1 fixed (complete)
PBPASSolver 2010-06-13 (complete)
PB-wave alpha 2 (incomplete)
SAT4J PB CuttingPlanes 2.2.0 2010-05-26 (complete)
SAT4J PB RES // CP 2.2.0 2010-05-31 (complete)
SAT4J PB Resolution 2.2.0 2010-05-26 (complete)
SCIPclp SCIP 1.2.1.2 with Clp 1.11.1 (Release Version) as LP solver (complete)
SCIPnone SCIP 1.2.1.2 without any LP solver (complete)
SCIPspx SCIP 1.2.1.2 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)
SCIPspx SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)
wbo 1.4a (complete)

Bench name	Best results on this instance	borg-pb 10.05.30 (complete)	bsolo 3.2 Card (complete)	bsolo 3.2 Cl (complete)	pb_cplex 2010-06-29 (complete)	PB/CT 0.1 (complete)	PB/CT 0.1 fixed (complete)	PBPASSolver 2010-06-13 (complete)	PB-wave alpha 2 (incomplete)	SAT4J PB CuttingPlanes 2.2.0 2010-05-26 (complete)	SAT4J PB RES // CP 2.2.0 2010-05-31 (complete)	SAT4J PB Resolution 2.2.0 2010-05-26 (complete)	SCIPclp SCIP 1.2.1.2 with Clp 1.11.1 (Release Version) as LP solver (complete)	SCIPnone SCIP 1.2.1.2 without any LP solver (complete)	SCIPspx SCIP 1.2.1.2 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)	SCIPspx SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)	wbo 1.4a (complete)
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB06/liu/dbst/ normalized-dbst_v30_e350_d15_mw10_1.opb.PB06.opb	SAT TT=1.555 T1=1.55	SAT TT=1.942 T1=2.25	SAT TT=58.275 T1=58.3	SAT TT=58.243 T1=58.27	SAT TT=132.802 T1=132.78	SAT TT=307.754 T1=304.36	SAT TT=321.641 T1=318.24	? (MO) TT=12.68	? (exit code) TT=0.964	SAT TT=13.12 T1=9.2	SAT TT=10.869 T1=8.06	SAT TT=8.304 T1=6.49	? (TO) TT=1802.13	? (TO) TT=1800.9	? (TO) TT=1802.13	? TT=1790.89	SAT TT=1.555 T1=1.55
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB06/liu/dbst/ normalized-dbst_v30_e350_d15_mw10_2.opb.PB06.opb	SAT TT=1.553 T1=1.55	SAT TT=1.836 T1=2.17	SAT TT=58.215 T1=58.22	SAT TT=58.414 T1=58.44	SAT TT=179.5 T1=181.8	SAT TT=322.308 T1=318.9	SAT TT=311.098 T1=307.69	? (MO) TT=13.18	? (exit code) TT=0.963	SAT TT=11.62 T1=8.67	SAT TT=13.076 T1=8.86	SAT TT=8.571 T1=6.69	? (TO) TT=1802.09	SAT TT=348.255 T1=347.68	? (TO) TT=1802.12	? (TO) TT=1802.14	SAT TT=1.553 T1=1.55
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB06/liu/dbst/ normalized-dbst_v30_e350_d15_mw10_3.opb.PB06.opb	SAT TT=1.561 T1=1.55	SAT TT=1.869 T1=2.22	SAT TT=58.201 T1=58.22	SAT TT=58.228 T1=58.23	SAT TT=39.765 T1=39.72	SAT TT=237.272 T1=233.87	SAT TT=335.033 T1=331.62	? (MO) TT=12.59	? (exit code) TT=0.969	SAT TT=12.598 T1=8.86	SAT TT=13.581 T1=9.92	SAT TT=8.811 T1=6.83	? (TO) TT=1802.1	SAT TT=811.934 T1=811.47	? (TO) TT=1802.14	? TT=1795.95	SAT TT=1.561 T1=1.55
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB06/liu/dbst/ normalized-dbst_v30_e350_d15_mw10_8.opb.PB06.opb	SAT TT=1.558 T1=1.55	SAT TT=1.869 T1=2.19	SAT TT=57.699 T1=57.73	SAT TT=57.702 T1=57.71	SAT TT=323.014 T1=288.31	SAT TT=298.183 T1=294.79	SAT TT=245.653 T1=242.23	? (MO) TT=13.09	? (exit code) TT=0.955	SAT TT=12.009 T1=8.37	SAT TT=11.212 T1=8.59	SAT TT=8.461 T1=6.87	? (TO) TT=1802.12	SAT TT=36.7 T1=36.05	? (TO) TT=1802.12	? (TO) TT=1802.17	SAT TT=1.558 T1=1.55
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB06/liu/dbst/ normalized-dbst_v30_e350_d15_mw10_9.opb.PB06.opb	SAT TT=1.564 T1=1.56	SAT TT=1.957 T1=2.3	SAT TT=57.463 T1=57.5	SAT TT=58.676 T1=58.7	SAT TT=142.841 T1=142.83	SAT TT=305.761 T1=302.35	SAT TT=314.682 T1=311.3	? (MO) TT=12.77	? (exit code) TT=0.974	SAT TT=13.006 T1=8.7	SAT TT=12.217 T1=9.09	SAT TT=16.689 T1=6.54	? (TO) TT=1802.19	SAT TT=301.647 T1=301.05	? (TO) TT=1802.11	? TT=1797.27	SAT TT=1.564 T1=1.56
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB06/liu/dbst/ normalized-dbst_v40_e600_d20_mw10_0.opb.PB06.opb	SAT TT=2.824 T1=3.1	SAT TT=2.824 T1=3.1	SAT TT=185.933 T1=185.99	SAT TT=185.928 T1=185.98	? (MO) TT=68.86	? (MO) TT=32.957	? (MO) TT=28.064	? (MO) TT=12.98	? (exit code) TT=0.004	SAT TT=20.101 T1=15.78	SAT TT=22.416 T1=17.97	SAT TT=17.966 T1=14.63	? (MO) TT=51.812	SAT TT=1603.84 T1=1602.04	? (MO) TT=43.683	? (MO) TT=35.286	SAT TT=4.912 T1=4.9
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB06/liu/dbst/ normalized-dbst_v40_e600_d20_mw10_2.opb.PB06.opb	SAT TT=2.812 T1=2.78	SAT TT=2.812 T1=2.78	SAT TT=185.87 T1=185.89	SAT TT=185.905 T1=185.95	? (MO) TT=71.48	? (MO) TT=33.375	? (MO) TT=28.067	? (MO) TT=13.09	? (exit code) TT=0.004	SAT TT=20.794 T1=16.44	SAT TT=25.998 T1=20.14	SAT TT=18.26 T1=14.75	? (MO) TT=51.217	? (TO) TT=1802.08	? (exit code) TT=41.207	? (MO) TT=35.023	SAT TT=4.909 T1=4.9
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB06/liu/dbst/ normalized-dbst_v40_e600_d20_mw10_5.opb.PB06.opb	SAT TT=2.97 T1=3.6	SAT TT=2.97 T1=3.6	SAT TT=185.893 T1=185.95	SAT TT=186 T1=186.05	? (MO) TT=61.37	? (MO) TT=32.87	? (MO) TT=27.773	? (MO) TT=13.08	? (exit code) TT=0.003	SAT TT=20.948 T1=16.81	SAT TT=24.844 T1=20.11	SAT TT=17.793 T1=14.31	? (MO) TT=51.699	? (TO) TT=1802.06	? (MO) TT=44.538	? (MO) TT=37.445	SAT TT=4.89 T1=4.88
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB06/liu/dbst/ normalized-dbst_v40_e600_d20_mw10_6.opb.PB06.opb	SAT TT=2.894 T1=4.11	SAT TT=2.894 T1=4.11	SAT TT=185.913 T1=185.96	SAT TT=185.922 T1=185.98	? (MO) TT=60.45	? (MO) TT=32.956	? (MO) TT=27.875	? (MO) TT=12.59	? (exit code) TT=0.003	SAT TT=19.953 T1=16.02	SAT TT=23.267 T1=18.99	SAT TT=17.816 T1=14.53	? (MO) TT=52.133	SAT TT=167.98 T1=165.96	? (MO) TT=43.554	? (MO) TT=36.929	SAT TT=4.959 T1=4.95
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB06/liu/dbst/ normalized-dbst_v40_e600_d20_mw10_9.opb.PB06.opb	SAT TT=2.807 T1=2.95	SAT TT=2.807 T1=2.95	SAT TT=185.932 T1=185.96	SAT TT=185.884 T1=185.94	? (MO) TT=68.97	? (MO) TT=32.976	? (MO) TT=27.95	? (MO) TT=13.29	? (exit code) TT=0.003	SAT TT=19.938 T1=15.64	SAT TT=23.075 T1=18.1	SAT TT=17.41 T1=14.15	? (MO) TT=52.548	SAT TT=168.801 T1=166.72	? (MO) TT=43.673	? (MO) TT=37.017	SAT TT=4.887 T1=4.89
Bench name	Best results	borg-pb 10.05.30 (complete)	bsolo 3.2 Card (complete)	bsolo 3.2 Cl (complete)	pb_cplex 2010-06-29 (complete)	PB/CT 0.1 (complete)	PB/CT 0.1 fixed (complete)	PBPASSolver 2010-06-13 (complete)	PB-wave alpha 2 (incomplete)	SAT4J PB CuttingPlanes 2.2.0 2010-05-26 (complete)	SAT4J PB RES // CP 2.2.0 2010-05-31 (complete)	SAT4J PB Resolution 2.2.0 2010-05-26 (complete)	SCIPclp SCIP 1.2.1.2 with Clp 1.11.1 (Release Version) as LP solver (complete)	SCIPnone SCIP 1.2.1.2 without any LP solver (complete)	SCIPspx SCIP 1.2.1.2 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)	SCIPspx SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)	wbo 1.4a (complete)
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB06/liu/dbst/ normalized-dbst_v50_e1000_d25_mw10_1.opb.PB06.opb	SAT TT=5.11 T1=5.44	SAT TT=5.11 T1=5.44	SAT TT=196.915 T1=196.91	SAT TT=196.894 T1=196.89	? (exit code) TT=94.632	? (MO) TT=68.75	? (MO) TT=57.167	? (MO) TT=12.74	? (exit code) TT=0.004	SAT TT=41.673 T1=36.27	SAT TT=51.766 T1=42.64	SAT TT=37.479 T1=33.08	? (MO) TT=48.656	? (MO) TT=49.497	? (MO) TT=49.333	? (MO) TT=42.064	SAT TT=12.146 T1=12.13
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB06/liu/dbst/ normalized-dbst_v50_e1000_d25_mw10_3.opb.PB06.opb	SAT TT=5.098 T1=5.32	SAT TT=5.098 T1=5.32	SAT TT=196.935 T1=196.95	SAT TT=196.768 T1=196.77	? (exit code) TT=93.609	? (MO) TT=68.861	? (MO) TT=57.042	? (MO) TT=12.82	? (exit code) TT=0.003	SAT TT=41.076 T1=35.1	SAT TT=53.749 T1=44.41	SAT TT=38.897 T1=34.57	? (MO) TT=54.392	? (MO) TT=49.434	? (MO) TT=50.19	? (MO) TT=44.958	SAT TT=12.844 T1=12.83
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB06/liu/dbst/ normalized-dbst_v50_e1000_d25_mw10_5.opb.PB06.opb	SAT TT=5.064 T1=5.54	SAT TT=5.064 T1=5.54	SAT TT=196.998 T1=197.02	SAT TT=196.974 T1=196.98	? (exit code) TT=89.139	? (MO) TT=68.959	? (MO) TT=56.983	? (MO) TT=13.58	? (exit code) TT=0.003	SAT TT=44.452 T1=37.9	SAT TT=53.227 T1=44.5	SAT TT=37.141 T1=33.28	? (MO) TT=48.457	? (MO) TT=49.477	? (MO) TT=48.811	? (MO) TT=44.81	SAT TT=12.834 T1=12.82
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB06/liu/dbst/ normalized-dbst_v50_e1000_d25_mw10_6.opb.PB06.opb	SAT TT=5.112 T1=5.47	SAT TT=5.112 T1=5.47	SAT TT=196.805 T1=196.81	SAT TT=196.818 T1=196.84	? (exit code) TT=89.263	? (MO) TT=69.369	? (MO) TT=56.977	? (MO) TT=12.84	? (exit code) TT=0.004	SAT TT=41.595 T1=35.33	SAT TT=55.322 T1=45.82	SAT TT=36.878 T1=31.78	? (MO) TT=49.481	? (MO) TT=46.187	? (MO) TT=49.411	? (MO) TT=45.161	SAT TT=12.85 T1=12.84
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB06/liu/dbst/ normalized-dbst_v50_e1000_d25_mw10_9.opb.PB06.opb	SAT TT=4.724 T1=4.77	SAT TT=4.724 T1=4.77	SAT TT=197.279 T1=197.28	SAT TT=196.899 T1=196.93	? (exit code) TT=89.26	? (MO) TT=68.857	? (MO) TT=57.058	? (MO) TT=13.3	? (exit code) TT=0.003	SAT TT=37.318 T1=32.19	SAT TT=50.505 T1=41.29	SAT TT=39.525 T1=34.79	? (MO) TT=50.618	? (MO) TT=49.382	? (MO) TT=49.948	? (MO) TT=45.242	SAT TT=12.883 T1=12.87

Statistics		borg-pb 10.05.30 (complete)	bsolo 3.2 Card (complete)	bsolo 3.2 Cl (complete)	pb_cplex 2010-06-29 (complete)	PB/CT 0.1 (complete)	PB/CT 0.1 fixed (complete)	PBPASSolver 2010-06-13 (complete)	PB-wave alpha 2 (incomplete)	SAT4J PB CuttingPlanes 2.2.0 2010-05-26 (complete)	SAT4J PB RES // CP 2.2.0 2010-05-31 (complete)	SAT4J PB Resolution 2.2.0 2010-05-26 (complete)	SCIPclp SCIP 1.2.1.2 with Clp 1.11.1 (Release Version) as LP solver (complete)	SCIPnone SCIP 1.2.1.2 without any LP solver (complete)	SCIPspx SCIP 1.2.1.2 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)	SCIPspx SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)	wbo 1.4a (complete)
Number of times the solver is able to give the best known answer		15	15	15	5	0	5	0	0	15	15	15	0	7	0	0	15
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)		10	0	0	0	0	0	0	0	0	0	0	0	0	0	0	5