PB'11 competition: satisfaction and optimization track: results by benchmark

Results by benchmark for category no optimisation, small integers, linear constraints (DEC-SMALLINT-LIN), subcategory Industrial, subsubcategory UCLID Benchmarks

This page displays the results of the different solvers for each benchmark for category no optimisation, small integers, linear constraints (DEC-SMALLINT-LIN), subcategory Industrial, subsubcategory UCLID Benchmarks

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell example	Meaning
Answer	Solver result
f=...	value of the objective function for the model reported by the solver
TT=...	Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out

Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

Abbreviation	Meaning
f=...	Value of the objective function
TO	Time Out
MO	Mem. Out (out of memory)

Meaning of the different colors:

Color	Meaning
text	the solver cannot handle this instance
text	the solver gave no answer
text	the solver could give an answer (SAT)
text	the solver gave a definitive answer (OPTIMUM FOUND or UNSAT)
text	the solver performed better than the other ones on that instance (complete solvers point of view)
text	the solver performed better than the other ones on that instance (incomplete solvers point of view)
text	the solver was ended by a signal or other problem
text	the solver gave an incomplete answer
text	the solver gave a wrong answer

For better readability, you may choose to hide some solvers:
borg pb-dec-11.04.03 (complete)
bsolo 3.2 (complete)
clasp 2.0-R4191 (complete)
MinisatID 2.4.8 [DEPRECATED] (complete)
MinisatID 2.4.8-gmp [DEPRECATED] (complete)
MinisatID 2.5.2 (fixed) (complete)
MinisatID 2.5.2-gmp (fixed) (complete)
Sat4j CuttingPlanes 2.3.0 (complete)
Sat4j Res//CP 2.3.0 (complete)
Sat4j Resolution 2.3.0 (complete)
SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED] (complete)
SCIP spx 2 2011-06-10 (fixed) (complete)
SCIP spx E SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED] (complete)
SCIP spx E_2 2011-06-10 (fixed) (complete)
wbo 1.6 (complete)

Bench name	Best results on this instance	borg pb-dec-11.04.03 (complete)	bsolo 3.2 (complete)	clasp 2.0-R4191 (complete)	MinisatID 2.4.8 [DEPRECATED] (complete)	MinisatID 2.4.8-gmp [DEPRECATED] (complete)	MinisatID 2.5.2 (fixed) (complete)	MinisatID 2.5.2-gmp (fixed) (complete)	Sat4j CuttingPlanes 2.3.0 (complete)	Sat4j Res//CP 2.3.0 (complete)	Sat4j Resolution 2.3.0 (complete)	SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED] (complete)	SCIP spx 2 2011-06-10 (fixed) (complete)	SCIP spx E SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED] (complete)	SCIP spx E_2 2011-06-10 (fixed) (complete)	wbo 1.6 (complete)
normalized-PB06/SATUNSAT-SMALLINT/ web/ uclid_pb_benchmarks/normalized-22s.smv.opb	UNSAT TT=0.073	UNSAT TT=0.81	UNSAT TT=0.455	UNSAT TT=0.096	UNSAT TT=294.905	UNSAT TT=366.55	UNSAT TT=8.793	UNSAT TT=46.38	UNSAT TT=1.363	UNSAT TT=1.85	UNSAT TT=1.341	UNSAT TT=101.337	UNSAT TT=100.068	UNSAT TT=101.534	UNSAT TT=99.993	UNSAT TT=0.073
normalized-PB06/SATUNSAT-SMALLINT/ web/ uclid_pb_benchmarks/normalized-25s.smv.opb	UNSAT TT=0.035	UNSAT TT=0.754	UNSAT TT=0.212	UNSAT TT=0.035	UNSAT TT=137.048	UNSAT TT=160.733	UNSAT TT=1.529	UNSAT TT=7.369	UNSAT TT=1.172	UNSAT TT=1.486	UNSAT TT=1.33	UNSAT TT=49.243	UNSAT TT=44.173	UNSAT TT=44.64	UNSAT TT=44.16	UNSAT TT=0.053
normalized-PB06/SATUNSAT-SMALLINT/ web/ uclid_pb_benchmarks/normalized-37s.smv.opb	UNSAT TT=0.011	UNSAT TT=0.524	UNSAT TT=0.035	UNSAT TT=0.011	UNSAT TT=3.542	UNSAT TT=7.12	UNSAT TT=0.256	UNSAT TT=1.29	UNSAT TT=0.922	UNSAT TT=0.898	UNSAT TT=0.733	UNSAT TT=11.965	UNSAT TT=11.927	UNSAT TT=11.935	UNSAT TT=11.941	UNSAT TT=0.013
normalized-PB06/SATUNSAT-SMALLINT/ web/ uclid_pb_benchmarks/normalized-43s.smv.opb	UNSAT TT=0.108	UNSAT TT=0.734	UNSAT TT=0.364	UNSAT TT=0.146	UNSAT TT=112.068	UNSAT TT=156.106	UNSAT TT=1.766	UNSAT TT=8.801	UNSAT TT=16.786	UNSAT TT=2.55	UNSAT TT=1.326	UNSAT TT=51.431	UNSAT TT=48.712	UNSAT TT=48.303	UNSAT TT=48.669	UNSAT TT=0.108
normalized-PB06/SATUNSAT-SMALLINT/ web/ uclid_pb_benchmarks/normalized-44s.smv.opb	UNSAT TT=0.007	UNSAT TT=0.481	UNSAT TT=0.011	UNSAT TT=0.008	UNSAT TT=0.069	UNSAT TT=0.393	UNSAT TT=0.039	UNSAT TT=0.169	UNSAT TT=1.039	UNSAT TT=0.824	UNSAT TT=0.54	UNSAT TT=1.26	UNSAT TT=1.273	UNSAT TT=1.256	UNSAT TT=1.278	UNSAT TT=0.007
normalized-PB06/SATUNSAT-SMALLINT/ web/ uclid_pb_benchmarks/normalized-46s.smv.opb	UNSAT TT=0.023	UNSAT TT=0.533	UNSAT TT=0.099	UNSAT TT=0.023	UNSAT TT=3.165	UNSAT TT=6.81	UNSAT TT=0.282	UNSAT TT=1.305	UNSAT TT=2.39	UNSAT TT=1.491	UNSAT TT=0.918	UNSAT TT=16.952	UNSAT TT=17.095	UNSAT TT=16.876	UNSAT TT=17.132	UNSAT TT=0.024
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-floppy1-2.ucl.opb	UNSAT TT=0.007	UNSAT TT=0.985	UNSAT TT=0.014	UNSAT TT=0.008	UNSAT TT=0.117	UNSAT TT=0.895	UNSAT TT=0.038	UNSAT TT=0.26	UNSAT TT=2.396	UNSAT TT=1.089	UNSAT TT=0.419	UNSAT TT=0.019	UNSAT TT=0.02	UNSAT TT=0.021	UNSAT TT=0.02	UNSAT TT=0.007
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-floppy1-3.ucl.opb	UNSAT TT=0.001	UNSAT TT=0.678	UNSAT TT=0.001	UNSAT TT=0.003	UNSAT TT=0.005	UNSAT TT=0.014	UNSAT TT=0.006	UNSAT TT=0.014	UNSAT TT=0.24	UNSAT TT=0.265	UNSAT TT=0.244	UNSAT TT=0.016	UNSAT TT=0.015	UNSAT TT=0.016	UNSAT TT=0.015	UNSAT TT=0.001
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-floppy1-4.ucl.opb	UNSAT TT=0.001	UNSAT TT=0.479	UNSAT TT=0.001	UNSAT TT=0.002	UNSAT TT=0.004	UNSAT TT=0.011	UNSAT TT=0.004	UNSAT TT=0.011	UNSAT TT=0.223	UNSAT TT=0.241	UNSAT TT=0.224	UNSAT TT=0.015	UNSAT TT=0.013	UNSAT TT=0.015	UNSAT TT=0.014	UNSAT TT=0.002
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-floppy1-6.ucl.opb	UNSAT TT=0.013	UNSAT TT=0.52	UNSAT TT=0.065	UNSAT TT=0.016	UNSAT TT=0.09	UNSAT TT=0.651	UNSAT TT=0.068	UNSAT TT=0.441	UNSAT TT=0.481	UNSAT TT=0.824	UNSAT TT=0.739	UNSAT TT=0.09	UNSAT TT=0.088	UNSAT TT=0.086	UNSAT TT=0.083	UNSAT TT=0.013
Bench name	Best results	borg pb-dec-11.04.03 (complete)	bsolo 3.2 (complete)	clasp 2.0-R4191 (complete)	MinisatID 2.4.8 [DEPRECATED] (complete)	MinisatID 2.4.8-gmp [DEPRECATED] (complete)	MinisatID 2.5.2 (fixed) (complete)	MinisatID 2.5.2-gmp (fixed) (complete)	Sat4j CuttingPlanes 2.3.0 (complete)	Sat4j Res//CP 2.3.0 (complete)	Sat4j Resolution 2.3.0 (complete)	SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED] (complete)	SCIP spx 2 2011-06-10 (fixed) (complete)	SCIP spx E SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED] (complete)	SCIP spx E_2 2011-06-10 (fixed) (complete)	wbo 1.6 (complete)
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-floppy1-7.ucl.opb	UNSAT TT=0.015	UNSAT TT=1.432	UNSAT TT=0.026	UNSAT TT=0.015	UNSAT TT=0.076	UNSAT TT=0.242	UNSAT TT=0.083	UNSAT TT=0.239	UNSAT TT=0.829	UNSAT TT=1.209	UNSAT TT=0.808	UNSAT TT=0.262	UNSAT TT=0.223	UNSAT TT=0.233	UNSAT TT=0.219	UNSAT TT=0.027
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-floppy1-8.ucl.opb	UNSAT TT=0.01	UNSAT TT=1.554	UNSAT TT=0.025	UNSAT TT=0.01	UNSAT TT=0.092	UNSAT TT=0.263	UNSAT TT=0.093	UNSAT TT=0.262	UNSAT TT=0.941	UNSAT TT=1.285	UNSAT TT=0.906	UNSAT TT=0.246	UNSAT TT=0.213	UNSAT TT=0.221	UNSAT TT=0.211	UNSAT TT=0.023
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-tlan2.ucl.opb	UNSAT TT=0.023	UNSAT TT=1.871	UNSAT TT=0.089	UNSAT TT=0.023	UNSAT TT=0.077	UNSAT TT=0.322	UNSAT TT=0.08	UNSAT TT=0.323	UNSAT TT=0.712	UNSAT TT=1.373	UNSAT TT=1.205	UNSAT TT=0.141	UNSAT TT=0.151	UNSAT TT=0.14	UNSAT TT=0.137	UNSAT TT=0.034
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-tlan3.ucl.opb	UNSAT TT=0.067	UNSAT TT=1.724	UNSAT TT=0.184	UNSAT TT=0.067	UNSAT TT=0.838	UNSAT TT=2.226	UNSAT TT=0.902	UNSAT TT=2.204	UNSAT TT=1.707	UNSAT TT=2.203	UNSAT TT=1.619	UNSAT TT=1.078	UNSAT TT=1.011	UNSAT TT=1.012	UNSAT TT=0.953	UNSAT TT=0.16
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-cache-ibm-q-full.all.ucl.opb	UNSAT TT=850.051	? (MO) TT=1640.75	UNSAT TT=850.051	? (TO) TT=1800.08	? (TO) TT=1800.07	? (TO) TT=1800.1	? (TO) TT=1800.07	? (TO) TT=1800.09	? (TO) TT=1800.21	? (TO) TT=1800.4	? (TO) TT=1800.15	? (TO) TT=1801.32	? (TO) TT=1800.13	? (TO) TT=1802.12	? (TO) TT=1800.1	UNSAT TT=1073.59
normalized-PB06/SATUNSAT-SMALLINT/web/ uclid_pb_benchmarks/ normalized-cache-ibm-q-unbounded.Ic22arity.ucl.opb	UNSAT TT=5.562	UNSAT TT=10.995	UNSAT TT=204.752	UNSAT TT=5.562	? (TO) TT=1800.09	? (TO) TT=1800.1	? (TO) TT=1800.07	? (TO) TT=1800.11	UNSAT TT=177.096	UNSAT TT=131.259	UNSAT TT=55.848	? (TO) TT=1802.12	? (TO) TT=1800.09	? (TO) TT=1802.11	? TT=1800.03	UNSAT TT=15.77
normalized-PB06/SATUNSAT-SMALLINT/web/ uclid_pb_benchmarks/ normalized-cache-ibm-q-unbounded.Icl2arity.ucl.opb	UNSAT TT=0.58	UNSAT TT=2.112	UNSAT TT=17.102	UNSAT TT=0.58	? (TO) TT=1800.06	? (TO) TT=1800.13	UNSAT TT=333.202	UNSAT TT=1401.64	UNSAT TT=14.049	UNSAT TT=10.603	UNSAT TT=4.493	? (TO) TT=1800.29	? TT=1797.31	? (TO) TT=1800.32	? TT=1797.35	UNSAT TT=0.59
normalized-PB06/SATUNSAT-SMALLINT/web/ uclid_pb_benchmarks/ normalized-cache-ibm-q-unbounded.Ih1arity.ucl.opb	UNSAT TT=836.829	? (MO) TT=569.23	UNSAT TT=836.829	UNSAT TT=1357.15	? (TO) TT=1800.11	? (TO) TT=1800.13	? (TO) TT=1800.06	? (TO) TT=1800.11	? (TO) TT=1800.29	? (TO) TT=1800.35	? (TO) TT=1800.22	? (TO) TT=1802.18	? (TO) TT=1800.13	? (TO) TT=1802.18	? (TO) TT=1802.12	UNSAT TT=1327.47
normalized-PB06/SATUNSAT-SMALLINT/web/ uclid_pb_benchmarks/ normalized-cache-ibm-q-unbounded.Ih2arity.ucl.opb	SAT TT=0.386 T1=0.36	SAT TT=4.527 T1=4.57	SAT TT=185.076 T1=185.06	SAT TT=0.386 T1=0.36	? (TO) TT=1800.09	? (TO) TT=1800.09	? (TO) TT=1800.09	? (TO) TT=1800.06	SAT TT=4.065 T1=2.8	SAT TT=5.224 T1=2.8	SAT TT=3.302 T1=2.25	? (TO) TT=1802.13	? (TO) TT=1800.1	? (TO) TT=1802.1	? (TO) TT=1802.1	SAT TT=1.229 T1=1.2
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-cache.inv10.ucl.opb	UNSAT TT=0.074	UNSAT TT=0.948	UNSAT TT=0.088	UNSAT TT=0.074	UNSAT TT=840.219	UNSAT TT=977.919	UNSAT TT=8.318	UNSAT TT=40.095	UNSAT TT=1.126	UNSAT TT=1.401	UNSAT TT=1.181	UNSAT TT=92.646	UNSAT TT=80.691	UNSAT TT=81.833	UNSAT TT=80.237	UNSAT TT=0.079
Bench name	Best results	borg pb-dec-11.04.03 (complete)	bsolo 3.2 (complete)	clasp 2.0-R4191 (complete)	MinisatID 2.4.8 [DEPRECATED] (complete)	MinisatID 2.4.8-gmp [DEPRECATED] (complete)	MinisatID 2.5.2 (fixed) (complete)	MinisatID 2.5.2-gmp (fixed) (complete)	Sat4j CuttingPlanes 2.3.0 (complete)	Sat4j Res//CP 2.3.0 (complete)	Sat4j Resolution 2.3.0 (complete)	SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED] (complete)	SCIP spx 2 2011-06-10 (fixed) (complete)	SCIP spx E SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED] (complete)	SCIP spx E_2 2011-06-10 (fixed) (complete)	wbo 1.6 (complete)
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-cache.inv12.ucl.opb	UNSAT TT=1.09	UNSAT TT=2.877	UNSAT TT=44.214	UNSAT TT=1.09	? (TO) TT=1800.07	? (TO) TT=1800.06	? (TO) TT=1800.06	? (TO) TT=1800.07	UNSAT TT=1051.13	UNSAT TT=32	UNSAT TT=13.124	? (TO) TT=1800.1	? TT=1797.38	? (TO) TT=1800.09	? TT=1797.33	UNSAT TT=5.042
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-cache.inv14.ucl.opb	UNSAT TT=357.773	? (MO) TT=1693	UNSAT TT=1039.79	UNSAT TT=357.773	? (TO) TT=1800.09	? (TO) TT=1800.11	? (TO) TT=1800.05	? (TO) TT=1800.08	? (TO) TT=1800.25	? (TO) TT=1800.27	? (TO) TT=1800.11	? (TO) TT=1801.66	? TT=1798.86	? (TO) TT=1800.8	? (TO) TT=1800.09	UNSAT TT=1054.99
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-cache.inv8.ucl.opb	UNSAT TT=0.007	UNSAT TT=0.568	UNSAT TT=0.015	UNSAT TT=0.007	UNSAT TT=0.116	UNSAT TT=0.359	UNSAT TT=0.153	UNSAT TT=0.631	UNSAT TT=0.61	UNSAT TT=0.846	UNSAT TT=0.633	UNSAT TT=0.17	UNSAT TT=0.172	UNSAT TT=0.162	UNSAT TT=0.164	UNSAT TT=0.013
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-dlx1c.rwmem1.ucl.opb	UNSAT TT=0.145	UNSAT TT=1	UNSAT TT=1.601	UNSAT TT=0.145	? (TO) TT=1800.05	? (TO) TT=1800.05	UNSAT TT=30.733	UNSAT TT=146.026	UNSAT TT=1.994	UNSAT TT=2.295	UNSAT TT=1.664	UNSAT TT=438.837	UNSAT TT=390.559	UNSAT TT=436.681	UNSAT TT=426.459	UNSAT TT=0.179
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-dlx1c.rwmem.ucl.opb	UNSAT TT=0.134	UNSAT TT=1.029	UNSAT TT=2.951	UNSAT TT=0.134	? (TO) TT=1800.06	? (TO) TT=1800.08	UNSAT TT=34.1	UNSAT TT=159.333	UNSAT TT=2.409	UNSAT TT=3.087	UNSAT TT=1.777	UNSAT TT=734.481	UNSAT TT=728.753	UNSAT TT=734.791	UNSAT TT=726.238	UNSAT TT=0.244
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-dlx1c.ucl.opb	UNSAT TT=0.081	UNSAT TT=0.815	UNSAT TT=1.076	UNSAT TT=0.081	? (TO) TT=1800.05	? (TO) TT=1800.05	UNSAT TT=6.969	UNSAT TT=35.469	UNSAT TT=2.055	UNSAT TT=2.328	UNSAT TT=1.59	UNSAT TT=1176.17	UNSAT TT=1162.93	UNSAT TT=1176.91	UNSAT TT=1268.71	UNSAT TT=0.103
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-elf.rf10.ucl.opb	UNSAT TT=1.172	UNSAT TT=4.347	UNSAT TT=59.157	UNSAT TT=1.172	? (TO) TT=1800.06	? (TO) TT=1800.07	? (TO) TT=1800.07	? (TO) TT=1800.05	UNSAT TT=11.272	UNSAT TT=12.153	UNSAT TT=5.732	? (TO) TT=1800.61	? TT=1797.55	? (TO) TT=1800.61	? TT=1798.39	UNSAT TT=1.917
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-elf.rf6.ucl.opb	UNSAT TT=0.001	UNSAT TT=0.508	UNSAT TT=0.001	UNSAT TT=0.002	UNSAT TT=0.002	UNSAT TT=0.007	UNSAT TT=0.003	UNSAT TT=0.007	UNSAT TT=0.172	UNSAT TT=0.183	UNSAT TT=0.175	UNSAT TT=0.015	UNSAT TT=0.014	UNSAT TT=0.016	UNSAT TT=0.015	UNSAT TT=0.001
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-elf.rf7.ucl.opb	UNSAT TT=0.009	UNSAT TT=0.509	UNSAT TT=0.046	UNSAT TT=0.009	UNSAT TT=2.104	UNSAT TT=4.211	UNSAT TT=0.21	UNSAT TT=0.943	UNSAT TT=0.66	UNSAT TT=0.877	UNSAT TT=0.597	UNSAT TT=0.234	UNSAT TT=0.262	UNSAT TT=0.232	UNSAT TT=0.254	UNSAT TT=0.012
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-elf.rf8.ucl.opb	UNSAT TT=0.045	UNSAT TT=0.753	UNSAT TT=0.527	UNSAT TT=0.045	UNSAT TT=1389.35	UNSAT TT=1629.12	UNSAT TT=5.666	UNSAT TT=26.733	UNSAT TT=1.43	UNSAT TT=1.502	UNSAT TT=1.193	UNSAT TT=71.244	UNSAT TT=70.582	UNSAT TT=71.382	UNSAT TT=70.565	UNSAT TT=0.065
Bench name	Best results	borg pb-dec-11.04.03 (complete)	bsolo 3.2 (complete)	clasp 2.0-R4191 (complete)	MinisatID 2.4.8 [DEPRECATED] (complete)	MinisatID 2.4.8-gmp [DEPRECATED] (complete)	MinisatID 2.5.2 (fixed) (complete)	MinisatID 2.5.2-gmp (fixed) (complete)	Sat4j CuttingPlanes 2.3.0 (complete)	Sat4j Res//CP 2.3.0 (complete)	Sat4j Resolution 2.3.0 (complete)	SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED] (complete)	SCIP spx 2 2011-06-10 (fixed) (complete)	SCIP spx E SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED] (complete)	SCIP spx E_2 2011-06-10 (fixed) (complete)	wbo 1.6 (complete)
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-elf.rf9.ucl.opb	UNSAT TT=0.23	UNSAT TT=1.303	UNSAT TT=5.671	UNSAT TT=0.23	? (TO) TT=1800.09	? (TO) TT=1800.07	UNSAT TT=833.35	? (TO) TT=1800.07	UNSAT TT=2.778	UNSAT TT=2.867	UNSAT TT=1.879	UNSAT TT=447.638	UNSAT TT=441.013	UNSAT TT=450.27	UNSAT TT=440.372	UNSAT TT=0.313
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.burch_dill.2.accl.ucl.opb	UNSAT TT=0.411	UNSAT TT=1.074	UNSAT TT=2.49	UNSAT TT=0.411	? (TO) TT=1800.08	? (TO) TT=1800.05	UNSAT TT=4.083	UNSAT TT=20.679	UNSAT TT=205.902	UNSAT TT=5.909	UNSAT TT=2.585	UNSAT TT=290.625	UNSAT TT=346.255	UNSAT TT=289.635	UNSAT TT=313.329	UNSAT TT=0.529
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.burch_dill.3.accl.ucl.opb	UNSAT TT=1.073	UNSAT TT=1.935	UNSAT TT=7.738	UNSAT TT=1.073	? (TO) TT=1800.08	? (TO) TT=1800.05	UNSAT TT=33.126	UNSAT TT=165.627	UNSAT TT=596.966	UNSAT TT=11.574	UNSAT TT=5.477	? (TO) TT=1800.09	? TT=1797.13	? (TO) TT=1800.21	? TT=1797.12	UNSAT TT=1.559
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.burch_dill.4.accl.ucl.opb	UNSAT TT=0.056	UNSAT TT=0.68	UNSAT TT=0.217	UNSAT TT=0.056	? (TO) TT=1800.06	? (TO) TT=1800.07	UNSAT TT=16.053	UNSAT TT=79.179	UNSAT TT=1.528	UNSAT TT=1.575	UNSAT TT=1.57	UNSAT TT=41.905	UNSAT TT=41.875	UNSAT TT=41.762	UNSAT TT=41.877	UNSAT TT=0.112
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.burch_dill.6.accl.ucl.opb	UNSAT TT=1637.97	? (MO) TT=581.53	? TT=1798.05	UNSAT TT=1637.97	? (TO) TT=1800.06	? (TO) TT=1800.13	? (TO) TT=1800.07	? (TO) TT=1800.06	? (TO) TT=1800.24	? (TO) TT=1800.33	? (TO) TT=1800.21	? (TO) TT=1800.06	? TT=1797.31	? (TO) TT=1800.09	? TT=1797.33	? (TO) TT=1800.15
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.burch_dill.8.accl.ucl.opb	UNSAT TT=87.162	? (MO) TT=577.9	? TT=1798.04	UNSAT TT=87.162	? (TO) TT=1800.08	? (TO) TT=1800.07	? (TO) TT=1800.04	? (TO) TT=1800.07	? (TO) TT=1800.31	UNSAT TT=845.149	UNSAT TT=354.414	? (TO) TT=1800.39	? TT=1797.32	? (TO) TT=1800.09	? TT=1797.29	UNSAT TT=1230.29
normalized-PB06/SATUNSAT-SMALLINT/web/ uclid_pb_benchmarks/ normalized-ooo.ex.br.mem.RobRegValid_bar.ucl.opb	UNSAT TT=22.452	UNSAT TT=123.953	UNSAT TT=96.16	UNSAT TT=22.452	? (TO) TT=1800.08	? (TO) TT=1800.07	? (TO) TT=1800.07	? (TO) TT=1800.09	? (TO) TT=1800.3	UNSAT TT=93.895	UNSAT TT=37.777	? (TO) TT=1800.77	? TT=1798.69	? (TO) TT=1800.65	? TT=1797.54	UNSAT TT=25.887
normalized-PB06/SATUNSAT-SMALLINT/web/uclid_pb_benchmarks/ normalized-ooo.ex.br.mem.Src1Valid_Src1ValidBar.ucl.opb	UNSAT TT=438.709	? (MO) TT=1647.12	? TT=1798.06	? (TO) TT=1800.06	? (TO) TT=1800.08	? (TO) TT=1800.13	? (TO) TT=1800.07	? (TO) TT=1800.09	? (TO) TT=1800.34	? (TO) TT=1800.3	UNSAT TT=698.413	? (TO) TT=1802.12	? (TO) TT=1800.09	? (TO) TT=1802.1	? (TO) TT=1801.63	UNSAT TT=438.709
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.ex.mem.LdValue.ucl.opb		? (MO) TT=1399.81	? TT=1798.1	? (TO) TT=1800.06	? (TO) TT=1800.07	? (TO) TT=1800.08	? (TO) TT=1800.08	? (TO) TT=1800.09	? (TO) TT=1800.25	? (TO) TT=1800.38	? (TO) TT=1800.24	? (TO) TT=1802.1	? (TO) TT=1800.1	? (TO) TT=1802.07	? (TO) TT=1802.09	? (TO) TT=1800.12
normalized-PB06/SATUNSAT-SMALLINT/web/uclid_pb_benchmarks/ normalized-ooo.ex.mem.LsqHdStrong.ucl.opb	UNSAT TT=1056.77	? (MO) TT=1071.53	? TT=1798.03	? (TO) TT=1800.1	? (TO) TT=1800.15	? (TO) TT=1800.12	? (TO) TT=1800.08	? (TO) TT=1800.11	? (TO) TT=1800.29	? (TO) TT=1800.39	? (TO) TT=1800.23	? (TO) TT=1801.9	? TT=1798.79	? (TO) TT=1801.9	? TT=1798.93	UNSAT TT=1056.77
Bench name	Best results	borg pb-dec-11.04.03 (complete)	bsolo 3.2 (complete)	clasp 2.0-R4191 (complete)	MinisatID 2.4.8 [DEPRECATED] (complete)	MinisatID 2.4.8-gmp [DEPRECATED] (complete)	MinisatID 2.5.2 (fixed) (complete)	MinisatID 2.5.2-gmp (fixed) (complete)	Sat4j CuttingPlanes 2.3.0 (complete)	Sat4j Res//CP 2.3.0 (complete)	Sat4j Resolution 2.3.0 (complete)	SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED] (complete)	SCIP spx 2 2011-06-10 (fixed) (complete)	SCIP spx E SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED] (complete)	SCIP spx E_2 2011-06-10 (fixed) (complete)	wbo 1.6 (complete)
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.rf10.ucl.opb	UNSAT TT=2.832	UNSAT TT=5.795	UNSAT TT=74.994	UNSAT TT=2.832	? (TO) TT=1800.07	? (TO) TT=1800.07	? (TO) TT=1800.07	? (TO) TT=1800.06	UNSAT TT=540.195	UNSAT TT=32.773	UNSAT TT=12.829	? (TO) TT=1800.14	? TT=1797.28	? (TO) TT=1800.37	? TT=1797.28	UNSAT TT=4.567
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.rf6.ucl.opb	UNSAT TT=0.014	UNSAT TT=0.517	UNSAT TT=0.098	UNSAT TT=0.014	UNSAT TT=17.62	UNSAT TT=30.373	UNSAT TT=0.537	UNSAT TT=2.256	UNSAT TT=0.979	UNSAT TT=1.117	UNSAT TT=0.783	UNSAT TT=17.608	UNSAT TT=17.535	UNSAT TT=17.477	UNSAT TT=19.442	UNSAT TT=0.02
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.rf7.ucl.opb	UNSAT TT=0.039	UNSAT TT=0.632	UNSAT TT=0.494	UNSAT TT=0.039	UNSAT TT=1563.98	? (TO) TT=1800.06	UNSAT TT=2.058	UNSAT TT=10.894	UNSAT TT=2.433	UNSAT TT=2.093	UNSAT TT=1.493	UNSAT TT=45.587	UNSAT TT=45.137	UNSAT TT=45.461	UNSAT TT=45.157	UNSAT TT=0.072
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.rf8.ucl.opb	UNSAT TT=0.114	UNSAT TT=0.91	UNSAT TT=1.782	UNSAT TT=0.114	? (TO) TT=1800.04	? (TO) TT=1800.06	UNSAT TT=15.339	UNSAT TT=74.524	UNSAT TT=4.226	UNSAT TT=3.195	UNSAT TT=1.756	UNSAT TT=395.366	UNSAT TT=389.997	UNSAT TT=394.81	UNSAT TT=389.036	UNSAT TT=0.304
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.rf9.ucl.opb	UNSAT TT=0.414	UNSAT TT=1.899	UNSAT TT=8.218	UNSAT TT=0.414	? (TO) TT=1800.06	? (TO) TT=1800.09	UNSAT TT=219.023	UNSAT TT=883.022	UNSAT TT=12.818	UNSAT TT=11.64	UNSAT TT=5.148	? (TO) TT=1800.62	? TT=1797.15	? (TO) TT=1800.56	? TT=1797.21	UNSAT TT=0.842
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.tag10.ucl.opb	UNSAT TT=0.397	UNSAT TT=2.267	UNSAT TT=4.46	UNSAT TT=0.397	? (TO) TT=1800.07	? (TO) TT=1800.08	UNSAT TT=49.982	UNSAT TT=227.11	? (TO) TT=1800.21	UNSAT TT=22.852	UNSAT TT=10.461	UNSAT TT=347.924	UNSAT TT=316.189	UNSAT TT=327.307	UNSAT TT=351.165	UNSAT TT=2.497
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.tag12.ucl.opb	UNSAT TT=0.759	UNSAT TT=3.885	UNSAT TT=35.497	UNSAT TT=0.759	? (TO) TT=1800.07	? (TO) TT=1800.05	UNSAT TT=822.732	? (TO) TT=1800.06	? (TO) TT=1800.27	UNSAT TT=237.731	UNSAT TT=91.392	? (TO) TT=1800.33	? TT=1797.24	? (TO) TT=1800.05	? TT=1797.26	UNSAT TT=5.833
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.tag14.ucl.opb	UNSAT TT=8.833	UNSAT TT=13.013	UNSAT TT=714.177	UNSAT TT=8.833	? (TO) TT=1800.08	? (TO) TT=1800.09	? (TO) TT=1800.06	? (TO) TT=1800.09	? (TO) TT=1800.3	UNSAT TT=666.315	UNSAT TT=244.084	? (TO) TT=1800.49	? TT=1799.94	? (TO) TT=1800.49	? TT=1799.59	UNSAT TT=97.194
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.tag8.ucl.opb	UNSAT TT=0.062	UNSAT TT=0.828	UNSAT TT=2.259	UNSAT TT=0.062	UNSAT TT=1199.17	UNSAT TT=1699.92	UNSAT TT=2.201	UNSAT TT=10.265	UNSAT TT=223.554	UNSAT TT=6.165	UNSAT TT=2.995	UNSAT TT=47	UNSAT TT=46.506	UNSAT TT=47.072	UNSAT TT=51.694	UNSAT TT=0.623
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.unbounded.all.ucl.opb	UNSAT TT=94.659	? (MO) TT=1423.18	UNSAT TT=650.771	UNSAT TT=94.659	? (TO) TT=1800.08	? (TO) TT=1800.11	? (TO) TT=1800.07	? (TO) TT=1800.07	? (TO) TT=1800.27	UNSAT TT=1166.94	UNSAT TT=440.884	? (TO) TT=1800.97	? TT=1798.41	? (TO) TT=1800.99	? TT=1797.95	UNSAT TT=178.715

Statistics		borg pb-dec-11.04.03 (complete)	bsolo 3.2 (complete)	clasp 2.0-R4191 (complete)	MinisatID 2.4.8 [DEPRECATED] (complete)	MinisatID 2.4.8-gmp [DEPRECATED] (complete)	MinisatID 2.5.2 (fixed) (complete)	MinisatID 2.5.2-gmp (fixed) (complete)	Sat4j CuttingPlanes 2.3.0 (complete)	Sat4j Res//CP 2.3.0 (complete)	Sat4j Resolution 2.3.0 (complete)	SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED] (complete)	SCIP spx 2 2011-06-10 (fixed) (complete)	SCIP spx E SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED] (complete)	SCIP spx E_2 2011-06-10 (fixed) (complete)	wbo 1.6 (complete)
Number of times the solver is able to give the best known answer		41	45	46	0	0	34	32	37	43	44	30	30	30	30	48
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)		0	5	37	0	0	0	0	0	0	0	0	0	0	0	9