PB'10 competition: satisfaction and optimization track: results by benchmark

Results by benchmark for category no optimisation, small integers, linear constraints (DEC-SMALLINT-LIN), subcategory Industrial, subsubcategory UCLID Benchmarks

This page displays the results of the different solvers for each benchmark for category no optimisation, small integers, linear constraints (DEC-SMALLINT-LIN), subcategory Industrial, subsubcategory UCLID Benchmarks

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell example	Meaning
Answer	Solver result
f=...	value of the objective function for the model reported by the solver
TT=...	Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out

Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

Abbreviation	Meaning
f=...	Value of the objective function
TO	Time Out
MO	Mem. Out (out of memory)

Meaning of the different colors:

Color	Meaning
text	the solver cannot handle this instance
text	the solver gave no answer
text	the solver could give an answer (SAT)
text	the solver gave a definitive answer (OPTIMUM FOUND or UNSAT)
text	the solver performed better than the other ones on that instance (complete solvers point of view)
text	the solver performed better than the other ones on that instance (incomplete solvers point of view)
text	the solver was ended by a signal or other problem
text	the solver gave an incomplete answer
text	the solver gave a wrong answer

For better readability, you may choose to hide some solvers:
borg-pb 10.05.30 (complete)
bsolo 3.2 Card (complete)
bsolo 3.2 Cl (complete)
pb_cplex 2010-06-29 (complete)
PB/CT 0.1 (complete)
PB/CT 0.1 fixed (complete)
PBPASSolver 2010-06-13 (complete)
PB-wave alpha 2 (incomplete)
SAT4J PB CuttingPlanes 2.2.0 2010-05-26 (complete)
SAT4J PB RES // CP 2.2.0 2010-05-31 (complete)
SAT4J PB Resolution 2.2.0 2010-05-26 (complete)
SCIPclp SCIP 1.2.1.2 with Clp 1.11.1 (Release Version) as LP solver (complete)
SCIPnone SCIP 1.2.1.2 without any LP solver (complete)
SCIPspx SCIP 1.2.1.2 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)
SCIPspx SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)
wbo 1.4a (complete)

Bench name	Best results on this instance	borg-pb 10.05.30 (complete)	bsolo 3.2 Card (complete)	bsolo 3.2 Cl (complete)	pb_cplex 2010-06-29 (complete)	PB/CT 0.1 (complete)	PB/CT 0.1 fixed (complete)	PBPASSolver 2010-06-13 (complete)	PB-wave alpha 2 (incomplete)	SAT4J PB CuttingPlanes 2.2.0 2010-05-26 (complete)	SAT4J PB RES // CP 2.2.0 2010-05-31 (complete)	SAT4J PB Resolution 2.2.0 2010-05-26 (complete)	SCIPclp SCIP 1.2.1.2 with Clp 1.11.1 (Release Version) as LP solver (complete)	SCIPnone SCIP 1.2.1.2 without any LP solver (complete)	SCIPspx SCIP 1.2.1.2 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)	SCIPspx SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)	wbo 1.4a (complete)
normalized-PB06/SATUNSAT-SMALLINT/ web/ uclid_pb_benchmarks/normalized-22s.smv.opb	UNSAT TT=0.153	UNSAT TT=1.525	UNSAT TT=0.982	UNSAT TT=0.985	UNSAT TT=2.787	UNSAT TT=2.352	UNSAT TT=2.389	? (TO) TT=1800.11	? (TO) TT=1800.08	UNSAT TT=3.321	UNSAT TT=4.148	UNSAT TT=3.704	UNSAT TT=682.725	UNSAT TT=12.279	UNSAT TT=524.268	UNSAT TT=1231.27	UNSAT TT=0.153
normalized-PB06/SATUNSAT-SMALLINT/ web/ uclid_pb_benchmarks/normalized-25s.smv.opb	UNSAT TT=0.098	UNSAT TT=1.377	UNSAT TT=0.463	UNSAT TT=0.465	UNSAT TT=1.786	UNSAT TT=1.903	UNSAT TT=1.893	? (TO) TT=1800.02	? (TO) TT=1800.07	UNSAT TT=2.9	UNSAT TT=4.59	UNSAT TT=3.281	UNSAT TT=3	UNSAT TT=29.407	UNSAT TT=3.965	UNSAT TT=3.632	UNSAT TT=0.098
normalized-PB06/SATUNSAT-SMALLINT/ web/ uclid_pb_benchmarks/normalized-37s.smv.opb	UNSAT TT=0.024	UNSAT TT=1.381	UNSAT TT=0.072	UNSAT TT=0.07	UNSAT TT=4.301	UNSAT TT=0.257	UNSAT TT=0.241	UNSAT TT=50.215	? (TO) TT=1800.11	UNSAT TT=2.03	UNSAT TT=1.989	UNSAT TT=1.678	UNSAT TT=0.769	UNSAT TT=0.845	UNSAT TT=0.828	UNSAT TT=0.795	UNSAT TT=0.024
normalized-PB06/SATUNSAT-SMALLINT/ web/ uclid_pb_benchmarks/normalized-43s.smv.opb	UNSAT TT=0.203	UNSAT TT=1.767	UNSAT TT=0.746	UNSAT TT=0.601	? (TO) TT=1800.15	UNSAT TT=3.935	UNSAT TT=3.429	? (TO) TT=1800.04	? (TO) TT=1800.04	UNSAT TT=43.858	UNSAT TT=8.116	UNSAT TT=3.966	UNSAT TT=119.101	UNSAT TT=31.881	UNSAT TT=147.102	UNSAT TT=209.16	UNSAT TT=0.203
normalized-PB06/SATUNSAT-SMALLINT/ web/ uclid_pb_benchmarks/normalized-44s.smv.opb	UNSAT TT=0.019	UNSAT TT=1.363	UNSAT TT=0.024	UNSAT TT=0.028	UNSAT TT=0.959	UNSAT TT=0.307	UNSAT TT=0.294	Wrong Cert. TT=402.904	? (TO) TT=1800.09	UNSAT TT=2.572	UNSAT TT=1.979	UNSAT TT=1.676	UNSAT TT=6.55	UNSAT TT=0.287	UNSAT TT=10.641	UNSAT TT=5.417	UNSAT TT=0.019
normalized-PB06/SATUNSAT-SMALLINT/ web/ uclid_pb_benchmarks/normalized-46s.smv.opb	UNSAT TT=0.056	UNSAT TT=1.403	UNSAT TT=0.198	UNSAT TT=0.188	UNSAT TT=22.044	UNSAT TT=0.752	UNSAT TT=0.736	? (TO) TT=1800.09	? (TO) TT=1800.11	UNSAT TT=5.67	UNSAT TT=4.882	UNSAT TT=2.769	UNSAT TT=46.311	UNSAT TT=1.525	? (TO) TT=1800.09	UNSAT TT=53.234	UNSAT TT=0.056
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-floppy1-2.ucl.opb	UNSAT TT=0.014	UNSAT TT=1.282	UNSAT TT=0.023	UNSAT TT=0.014	UNSAT TT=0.022	UNSAT TT=0.185	UNSAT TT=0.181	? (TO) TT=1800.05	? (TO) TT=1800.09	UNSAT TT=6.025	UNSAT TT=2.519	UNSAT TT=1.219	UNSAT TT=0.062	UNSAT TT=0.06	UNSAT TT=0.06	UNSAT TT=0.052	UNSAT TT=0.023
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-floppy1-3.ucl.opb	UNSAT TT=0.004	UNSAT TT=1.35	UNSAT TT=0.004	UNSAT TT=0.004	UNSAT TT=0.018	Wrong Cert. TT=0.122	UNSAT TT=0.008	UNSAT TT=1.146	? (TO) TT=1800.02	UNSAT TT=0.499	UNSAT TT=0.587	UNSAT TT=0.501	UNSAT TT=0.037	UNSAT TT=0.036	UNSAT TT=0.037	UNSAT TT=0.037	UNSAT TT=0.004
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-floppy1-4.ucl.opb	UNSAT TT=0.003	UNSAT TT=1.326	UNSAT TT=0.003	UNSAT TT=0.003	UNSAT TT=0.017	Wrong Cert. TT=0.095	UNSAT TT=0.006	UNSAT TT=1.03	? (TO) TT=1800.07	UNSAT TT=0.458	UNSAT TT=0.534	UNSAT TT=0.445	UNSAT TT=0.034	UNSAT TT=0.033	UNSAT TT=0.034	UNSAT TT=0.035	UNSAT TT=0.003
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-floppy1-6.ucl.opb	UNSAT TT=0.036	UNSAT TT=1.38	UNSAT TT=0.119	UNSAT TT=0.082	UNSAT TT=0.072	UNSAT TT=0.793	UNSAT TT=0.823	? (TO) TT=1800.04	? (TO) TT=1800.05	UNSAT TT=0.98	UNSAT TT=2.305	UNSAT TT=2.336	UNSAT TT=0.528	UNSAT TT=0.546	UNSAT TT=0.477	UNSAT TT=0.439	UNSAT TT=0.036
Bench name	Best results	borg-pb 10.05.30 (complete)	bsolo 3.2 Card (complete)	bsolo 3.2 Cl (complete)	pb_cplex 2010-06-29 (complete)	PB/CT 0.1 (complete)	PB/CT 0.1 fixed (complete)	PBPASSolver 2010-06-13 (complete)	PB-wave alpha 2 (incomplete)	SAT4J PB CuttingPlanes 2.2.0 2010-05-26 (complete)	SAT4J PB RES // CP 2.2.0 2010-05-31 (complete)	SAT4J PB Resolution 2.2.0 2010-05-26 (complete)	SCIPclp SCIP 1.2.1.2 with Clp 1.11.1 (Release Version) as LP solver (complete)	SCIPnone SCIP 1.2.1.2 without any LP solver (complete)	SCIPspx SCIP 1.2.1.2 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)	SCIPspx SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)	wbo 1.4a (complete)
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-floppy1-7.ucl.opb	UNSAT TT=0.044	UNSAT TT=1.28	UNSAT TT=0.063	UNSAT TT=0.064	UNSAT TT=0.39	UNSAT TT=1.911	UNSAT TT=1.825	? (TO) TT=1800.08	? (TO) TT=1800.05	UNSAT TT=1.985	UNSAT TT=2.93	UNSAT TT=2.075	UNSAT TT=0.959	UNSAT TT=1.028	UNSAT TT=0.96	UNSAT TT=0.649	UNSAT TT=0.044
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-floppy1-8.ucl.opb	UNSAT TT=0.054	UNSAT TT=1.37	UNSAT TT=0.063	UNSAT TT=0.063	UNSAT TT=0.307	UNSAT TT=2.495	UNSAT TT=2.383	? (TO) TT=1800.03	? (TO) TT=1800.03	UNSAT TT=2.023	UNSAT TT=3.098	UNSAT TT=1.839	UNSAT TT=1.111	UNSAT TT=1.124	UNSAT TT=1.124	UNSAT TT=0.722	UNSAT TT=0.054
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-tlan2.ucl.opb	UNSAT TT=0.074	UNSAT TT=1.413	UNSAT TT=0.182	UNSAT TT=0.16	UNSAT TT=0.158	UNSAT TT=1.473	UNSAT TT=1.417	? (TO) TT=1800.03	? (TO) TT=1800.12	UNSAT TT=1.584	UNSAT TT=2.978	UNSAT TT=3.803	UNSAT TT=0.655	UNSAT TT=0.655	UNSAT TT=0.65	UNSAT TT=0.419	UNSAT TT=0.074
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-tlan3.ucl.opb	UNSAT TT=0.367	UNSAT TT=1.494	UNSAT TT=0.481	UNSAT TT=0.478	UNSAT TT=1.616	Wrong Cert. TT=18.924	UNSAT TT=0.915	? (MO) TT=13.57	? (TO) TT=1800.04	UNSAT TT=4.493	UNSAT TT=5.538	UNSAT TT=4.852	UNSAT TT=2.728	UNSAT TT=2.741	UNSAT TT=2.76	UNSAT TT=2.523	UNSAT TT=0.367
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-cache-ibm-q-full.all.ucl.opb		? (TO) TT=1800.03	? TT=1798.08	? TT=1798.05	? TT=1798.91	UNSAT TT=1222.92	? (TO) TT=1800.17	? (MO) TT=12.98	? (TO) TT=1800.06	? (TO) TT=1800.2	? (TO) TT=1800.41	? (TO) TT=1800.18	? (TO) TT=1802.12	? (TO) TT=1802.22	? (TO) TT=1802.17	? TT=1791.05	? (TO) TT=1800.06
normalized-PB06/SATUNSAT-SMALLINT/web/ uclid_pb_benchmarks/ normalized-cache-ibm-q-unbounded.Ic22arity.ucl.opb	UNSAT TT=53.527	UNSAT TT=53.527	UNSAT TT=219.04	UNSAT TT=494.26	? TT=1799.15	UNSAT TT=157.056	UNSAT TT=189.767	? (MO) TT=13.6	? (TO) TT=1800.07	UNSAT TT=514.877	UNSAT TT=310.533	UNSAT TT=137.352	? (TO) TT=1802.03	? (TO) TT=1801.89	? (TO) TT=1802	? TT=1791.94	UNSAT TT=55.013
normalized-PB06/SATUNSAT-SMALLINT/web/ uclid_pb_benchmarks/ normalized-cache-ibm-q-unbounded.Icl2arity.ucl.opb	UNSAT TT=2.84	UNSAT TT=2.84	UNSAT TT=59.106	UNSAT TT=65.604	? (TO) TT=1800.18	UNSAT TT=15.638	UNSAT TT=20.849	? (MO) TT=13.51	? (TO) TT=1800.08	UNSAT TT=29.048	UNSAT TT=27.835	UNSAT TT=13.803	? (TO) TT=1800.61	UNSAT TT=1195.56	? (TO) TT=1802.08	? TT=1789.93	UNSAT TT=4.488
normalized-PB06/SATUNSAT-SMALLINT/web/ uclid_pb_benchmarks/ normalized-cache-ibm-q-unbounded.Ih1arity.ucl.opb		? (TO) TT=1800.01	? TT=1798.14	? TT=1798.21	? TT=1799.69	? (MO) TT=25.234	? (MO) TT=36.489	? (MO) TT=13.3	? (exit code) TT=0.004	? (TO) TT=1800.26	? (TO) TT=1800.28	? (TO) TT=1800.15	? (MO) TT=52.413	? (TO) TT=1802.17	? (MO) TT=54.39	? (MO) TT=42.227	? (TO) TT=1800.09
normalized-PB06/SATUNSAT-SMALLINT/web/ uclid_pb_benchmarks/ normalized-cache-ibm-q-unbounded.Ih2arity.ucl.opb	SAT TT=1.9 T1=1.87	SAT TT=6.944 T1=7.23	SAT TT=181.971 T1=181.98	SAT TT=199.528 T1=199.57	? TT=1799.07	SAT TT=29.871 T1=24.22	SAT TT=27.053 T1=21.38	? (MO) TT=13.6	? (TO) TT=1800.11	SAT TT=10.4 T1=7.14	SAT TT=14.8 T1=9.38	SAT TT=9.67 T1=7	? (TO) TT=1802.16	? (TO) TT=1801.53	? (TO) TT=1802.14	? TT=1791.72	SAT TT=1.9 T1=1.87
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-cache.inv10.ucl.opb	UNSAT TT=0.119	UNSAT TT=1.442	UNSAT TT=0.219	UNSAT TT=0.219	UNSAT TT=3.607	UNSAT TT=1.411	UNSAT TT=1.383	UNSAT TT=33.885	? (TO) TT=1800.09	UNSAT TT=2.759	UNSAT TT=4.116	UNSAT TT=3.019	UNSAT TT=713.678	UNSAT TT=3.004	UNSAT TT=523.247	UNSAT TT=800.988	UNSAT TT=0.119
Bench name	Best results	borg-pb 10.05.30 (complete)	bsolo 3.2 Card (complete)	bsolo 3.2 Cl (complete)	pb_cplex 2010-06-29 (complete)	PB/CT 0.1 (complete)	PB/CT 0.1 fixed (complete)	PBPASSolver 2010-06-13 (complete)	PB-wave alpha 2 (incomplete)	SAT4J PB CuttingPlanes 2.2.0 2010-05-26 (complete)	SAT4J PB RES // CP 2.2.0 2010-05-31 (complete)	SAT4J PB Resolution 2.2.0 2010-05-26 (complete)	SCIPclp SCIP 1.2.1.2 with Clp 1.11.1 (Release Version) as LP solver (complete)	SCIPnone SCIP 1.2.1.2 without any LP solver (complete)	SCIPspx SCIP 1.2.1.2 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)	SCIPspx SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)	wbo 1.4a (complete)
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-cache.inv12.ucl.opb	UNSAT TT=9.104	UNSAT TT=9.104	UNSAT TT=146.516	UNSAT TT=115.368	? (TO) TT=1800.16	UNSAT TT=24.906	UNSAT TT=24.162	? (MO) TT=13.39	? (TO) TT=1800.03	? (TO) TT=1800.2	UNSAT TT=47.652	UNSAT TT=25.174	? (TO) TT=1801.27	UNSAT TT=229.102	? (TO) TT=1802.07	? TT=1797.25	UNSAT TT=19.234
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-cache.inv14.ucl.opb		? (TO) TT=1800.05	? TT=1798.07	? TT=1798.13	? TT=1798.98	UNSAT TT=442.88	? (TO) TT=1800.09	? (MO) TT=13.73	? (TO) TT=1800.07	? (TO) TT=1800.15	? (TO) TT=1800.24	? (TO) TT=1800.19	? (TO) TT=1802.15	? (MO) TT=1314.27	? (TO) TT=1802.1	? TT=1791.06	? (TO) TT=1800.1
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-cache.inv8.ucl.opb	UNSAT TT=0.029	UNSAT TT=1.387	UNSAT TT=0.037	UNSAT TT=0.037	UNSAT TT=0.094	UNSAT TT=0.373	UNSAT TT=0.088	UNSAT TT=5.616	? (TO) TT=1800.09	UNSAT TT=1.341	UNSAT TT=1.991	UNSAT TT=1.358	UNSAT TT=0.467	UNSAT TT=0.469	UNSAT TT=0.476	UNSAT TT=0.421	UNSAT TT=0.029
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-dlx1c.rwmem1.ucl.opb	UNSAT TT=0.442	UNSAT TT=1.696	UNSAT TT=4.089	UNSAT TT=3.715	? (TO) TT=1800.09	UNSAT TT=1.925	UNSAT TT=1.846	UNSAT TT=438.836	? (TO) TT=1800.03	UNSAT TT=5.385	UNSAT TT=6.049	UNSAT TT=3.924	? (TO) TT=1800.28	UNSAT TT=128.344	? (TO) TT=1801.34	? TT=1789.54	UNSAT TT=0.442
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-dlx1c.rwmem.ucl.opb	UNSAT TT=0.48	UNSAT TT=1.777	UNSAT TT=7.421	UNSAT TT=7.143	? (TO) TT=1800.01	UNSAT TT=2.13	UNSAT TT=2.049	UNSAT TT=623.978	? (TO) TT=1800.06	UNSAT TT=6.058	UNSAT TT=7.217	UNSAT TT=4.806	? (TO) TT=1802.06	UNSAT TT=147.134	? (TO) TT=1800.63	? TT=1789.47	UNSAT TT=0.48
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-dlx1c.ucl.opb	UNSAT TT=0.25	UNSAT TT=1.578	UNSAT TT=2.605	UNSAT TT=2.379	? (TO) TT=1800.04	UNSAT TT=1.561	UNSAT TT=1.494	UNSAT TT=447.325	? (TO) TT=1800.12	UNSAT TT=4.899	UNSAT TT=5.935	UNSAT TT=3.637	? (TO) TT=1801.31	UNSAT TT=61.792	UNSAT TT=1625.87	UNSAT TT=1380.15	UNSAT TT=0.25
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-elf.rf10.ucl.opb	UNSAT TT=5.518	UNSAT TT=6.098	UNSAT TT=167.248	UNSAT TT=161.893	? TT=1799.37	UNSAT TT=15.358	UNSAT TT=14.261	? (MO) TT=13.1	? (TO) TT=1800.06	UNSAT TT=24.71	UNSAT TT=22.211	UNSAT TT=12.459	? (TO) TT=1802.15	UNSAT TT=567.86	? (TO) TT=1802.11	? TT=1795.38	UNSAT TT=5.518
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-elf.rf6.ucl.opb	UNSAT TT=0.002	UNSAT TT=1.342	UNSAT TT=0.002	UNSAT TT=0.002	UNSAT TT=0.017	UNSAT TT=0.012	UNSAT TT=0.012	UNSAT TT=0.134	? (TO) TT=1800.06	UNSAT TT=0.348	UNSAT TT=0.374	UNSAT TT=0.333	UNSAT TT=0.039	UNSAT TT=0.039	UNSAT TT=0.039	UNSAT TT=0.036	UNSAT TT=0.002
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-elf.rf7.ucl.opb	UNSAT TT=0.02	UNSAT TT=1.327	UNSAT TT=0.096	UNSAT TT=0.095	UNSAT TT=0.582	UNSAT TT=0.3	UNSAT TT=0.272	UNSAT TT=4.586	? (TO) TT=1800.02	UNSAT TT=1.477	UNSAT TT=1.612	UNSAT TT=1.29	UNSAT TT=0.584	UNSAT TT=0.374	UNSAT TT=0.657	UNSAT TT=0.63	UNSAT TT=0.02
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-elf.rf8.ucl.opb	UNSAT TT=0.106	UNSAT TT=1.418	UNSAT TT=1.198	UNSAT TT=1.202	UNSAT TT=52.924	UNSAT TT=1.45	UNSAT TT=1.386	UNSAT TT=27.195	? (TO) TT=1800.09	UNSAT TT=4.081	UNSAT TT=3.511	UNSAT TT=3.002	UNSAT TT=359.653	UNSAT TT=3.422	UNSAT TT=415.227	UNSAT TT=403.213	UNSAT TT=0.106
Bench name	Best results	borg-pb 10.05.30 (complete)	bsolo 3.2 Card (complete)	bsolo 3.2 Cl (complete)	pb_cplex 2010-06-29 (complete)	PB/CT 0.1 (complete)	PB/CT 0.1 fixed (complete)	PBPASSolver 2010-06-13 (complete)	PB-wave alpha 2 (incomplete)	SAT4J PB CuttingPlanes 2.2.0 2010-05-26 (complete)	SAT4J PB RES // CP 2.2.0 2010-05-31 (complete)	SAT4J PB Resolution 2.2.0 2010-05-26 (complete)	SCIPclp SCIP 1.2.1.2 with Clp 1.11.1 (Release Version) as LP solver (complete)	SCIPnone SCIP 1.2.1.2 without any LP solver (complete)	SCIPspx SCIP 1.2.1.2 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)	SCIPspx SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)	wbo 1.4a (complete)
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-elf.rf9.ucl.opb	UNSAT TT=0.714	UNSAT TT=1.838	UNSAT TT=18.04	UNSAT TT=17.875	? (TO) TT=1800.1	UNSAT TT=5.039	UNSAT TT=4.843	? (MO) TT=13.2	? (TO) TT=1800.07	UNSAT TT=6.57	UNSAT TT=5.529	UNSAT TT=4.072	? (TO) TT=1802.13	UNSAT TT=37.587	? (TO) TT=1802.12	? TT=1789.93	UNSAT TT=0.714
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.burch_dill.2.accl.ucl.opb	UNSAT TT=1.371	UNSAT TT=2.429	UNSAT TT=5.176	UNSAT TT=4.76	? (TO) TT=1800.08	UNSAT TT=6.966	UNSAT TT=6.553	? (TO) TT=1800.03	? (TO) TT=1800.04	UNSAT TT=606.653	UNSAT TT=17.534	UNSAT TT=8.008	UNSAT TT=1240.82	UNSAT TT=1123.66	UNSAT TT=1218.45	? TT=1789.76	UNSAT TT=1.371
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.burch_dill.3.accl.ucl.opb	UNSAT TT=5.11	UNSAT TT=5.41	UNSAT TT=20.837	UNSAT TT=18.031	? (TO) TT=1800.13	UNSAT TT=21.168	UNSAT TT=25.327	? (TO) TT=1800.02	? (TO) TT=1800.07	UNSAT TT=1738.71	UNSAT TT=35.15	UNSAT TT=13.418	? (TO) TT=1800.19	? (TO) TT=1801.64	? (TO) TT=1800.34	? TT=1789.64	UNSAT TT=5.11
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.burch_dill.4.accl.ucl.opb	UNSAT TT=0.197	UNSAT TT=1.578	UNSAT TT=0.456	UNSAT TT=0.412	? (TO) TT=1800.1	UNSAT TT=1.201	UNSAT TT=1.158	? (TO) TT=1800.05	? (TO) TT=1800.08	UNSAT TT=3.701	UNSAT TT=4.667	UNSAT TT=3.211	UNSAT TT=316.517	UNSAT TT=131.248	UNSAT TT=319.632	UNSAT TT=306.207	UNSAT TT=0.197
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.burch_dill.6.accl.ucl.opb		? (TO) TT=1800.07	? TT=1798.09	? TT=1798.1	? (TO) TT=1800.18	? (TO) TT=1800.1	? (TO) TT=1800.06	? (MO) TT=13.39	? (TO) TT=1800.05	? (TO) TT=1800.26	? (TO) TT=1800.77	? (TO) TT=1800.19	? (TO) TT=1802.1	? (TO) TT=1801.4	? (TO) TT=1802.06	? TT=1794.31	? (TO) TT=1800.09
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.burch_dill.8.accl.ucl.opb	UNSAT TT=1219.55	? (TO) TT=1800.07	? TT=1798.05	? TT=1798.09	? (TO) TT=1800.05	? (TO) TT=1800.13	UNSAT TT=1449.55	? (MO) TT=13.2	? (TO) TT=1800.08	? (TO) TT=1800.25	? (TO) TT=1800.38	UNSAT TT=1219.55	? (TO) TT=1802.11	? (TO) TT=1801.43	? (TO) TT=1802.06	? TT=1794.54	? (TO) TT=1800.1
normalized-PB06/SATUNSAT-SMALLINT/web/ uclid_pb_benchmarks/ normalized-ooo.ex.br.mem.RobRegValid_bar.ucl.opb	UNSAT TT=44.148	UNSAT TT=114.371	UNSAT TT=299.058	UNSAT TT=374.543	? TT=1798.67	UNSAT TT=295.466	UNSAT TT=359.462	? (MO) TT=12.69	? (TO) TT=1800.04	? (TO) TT=1800.18	UNSAT TT=221.425	UNSAT TT=97.6	? (TO) TT=1802.06	? (TO) TT=1801.26	? (TO) TT=1802.14	? TT=1790.55	UNSAT TT=44.148
normalized-PB06/SATUNSAT-SMALLINT/web/uclid_pb_benchmarks/ normalized-ooo.ex.br.mem.Src1Valid_Src1ValidBar.ucl.opb	UNSAT TT=786.704	? (TO) TT=1800.07	? TT=1798.09	? TT=1798.04	? TT=1799.1	? (TO) TT=1800.21	? (TO) TT=1800.2	? (MO) TT=13.39	? (TO) TT=1800.02	? (TO) TT=1800.04	? (TO) TT=1800.54	? (TO) TT=1800.16	? (TO) TT=1802.19	? (TO) TT=1801.98	? (TO) TT=1802.16	? TT=1792.11	UNSAT TT=786.704
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.ex.mem.LdValue.ucl.opb	UNSAT TT=1799.77	? (TO) TT=1800.06	? TT=1798.07	? TT=1798.07	? TT=1799.16	UNSAT TT=1740.53	UNSAT TT=1799.77	? (MO) TT=13.29	? (TO) TT=1800.12	? (TO) TT=1800.32	? (TO) TT=1800.48	? (TO) TT=1800.27	? (TO) TT=1802.14	? (TO) TT=1801.47	? (TO) TT=1802.09	? TT=1791.99	? (TO) TT=1800.11
normalized-PB06/SATUNSAT-SMALLINT/web/uclid_pb_benchmarks/ normalized-ooo.ex.mem.LsqHdStrong.ucl.opb		? (TO) TT=1800.11	? TT=1798.43	? TT=1798.27	? TT=1799.94	? (MO) TT=25.801	? (MO) TT=34.304	? (MO) TT=13.2	? (exit code) TT=0.003	? (TO) TT=1800.25	? (TO) TT=1800.5	? (TO) TT=1800.2	? (MO) TT=68.731	? (TO) TT=1802.21	? (MO) TT=133.023	? (MO) TT=139.227	? (TO) TT=1800.09
Bench name	Best results	borg-pb 10.05.30 (complete)	bsolo 3.2 Card (complete)	bsolo 3.2 Cl (complete)	pb_cplex 2010-06-29 (complete)	PB/CT 0.1 (complete)	PB/CT 0.1 fixed (complete)	PBPASSolver 2010-06-13 (complete)	PB-wave alpha 2 (incomplete)	SAT4J PB CuttingPlanes 2.2.0 2010-05-26 (complete)	SAT4J PB RES // CP 2.2.0 2010-05-31 (complete)	SAT4J PB Resolution 2.2.0 2010-05-26 (complete)	SCIPclp SCIP 1.2.1.2 with Clp 1.11.1 (Release Version) as LP solver (complete)	SCIPnone SCIP 1.2.1.2 without any LP solver (complete)	SCIPspx SCIP 1.2.1.2 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)	SCIPspx SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)	wbo 1.4a (complete)
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.rf10.ucl.opb	UNSAT TT=22.634	UNSAT TT=25.599	UNSAT TT=229.464	UNSAT TT=135.2	? (TO) TT=1800.1	UNSAT TT=23.849	UNSAT TT=23.17	? (MO) TT=13.2	? (TO) TT=1800.09	UNSAT TT=1364.41	UNSAT TT=83.966	UNSAT TT=40.968	? (TO) TT=1802.04	? (TO) TT=1801.64	? (TO) TT=1802.1	? TT=1795.61	UNSAT TT=22.634
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.rf6.ucl.opb	UNSAT TT=0.041	UNSAT TT=1.37	UNSAT TT=0.199	UNSAT TT=0.244	UNSAT TT=30.831	UNSAT TT=0.382	UNSAT TT=0.366	UNSAT TT=32.649	? (TO) TT=1800.08	UNSAT TT=2.26	UNSAT TT=2.904	UNSAT TT=2.11	UNSAT TT=1.12	UNSAT TT=1.012	UNSAT TT=1.247	UNSAT TT=1.314	UNSAT TT=0.041
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.rf7.ucl.opb	UNSAT TT=0.122	UNSAT TT=1.435	UNSAT TT=1.041	UNSAT TT=1.262	? (TO) TT=1800.04	UNSAT TT=0.889	UNSAT TT=0.859	UNSAT TT=1114.25	? (TO) TT=1800.05	UNSAT TT=6.259	UNSAT TT=5.836	UNSAT TT=4.842	UNSAT TT=284.46	UNSAT TT=8.151	UNSAT TT=4.981	UNSAT TT=4.862	UNSAT TT=0.122
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.rf8.ucl.opb	UNSAT TT=0.758	UNSAT TT=1.674	UNSAT TT=4.607	UNSAT TT=7.548	? (TO) TT=1800.07	UNSAT TT=1.928	UNSAT TT=1.798	UNSAT TT=1491.95	? (TO) TT=1800.11	UNSAT TT=9.397	UNSAT TT=8.621	UNSAT TT=4.906	? (TO) TT=1802.06	UNSAT TT=40.664	? (TO) TT=1800.76	? TT=1790.15	UNSAT TT=0.758
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.rf9.ucl.opb	UNSAT TT=2.474	UNSAT TT=3.736	UNSAT TT=25.377	UNSAT TT=27.92	? (TO) TT=1800.08	UNSAT TT=5.466	UNSAT TT=5.284	? (MO) TT=46.85	? (TO) TT=1800.03	UNSAT TT=25.852	UNSAT TT=17.317	UNSAT TT=12.472	? (TO) TT=1802.04	UNSAT TT=382.596	? (TO) TT=1800.94	? TT=1793.88	UNSAT TT=2.474
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.tag10.ucl.opb	UNSAT TT=5.032	UNSAT TT=5.032	UNSAT TT=13.223	UNSAT TT=59.364	? (TO) TT=1800.13	UNSAT TT=8.336	UNSAT TT=8.005	? (TO) TT=1800.1	? (TO) TT=1800.1	? (TO) TT=1800.14	UNSAT TT=76.261	UNSAT TT=39.781	? (TO) TT=1802.06	? (TO) TT=1800.72	? (TO) TT=1800.75	? TT=1791.21	UNSAT TT=24.108
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.tag12.ucl.opb	UNSAT TT=30.418	UNSAT TT=30.418	UNSAT TT=115.058	UNSAT TT=474.026	? (TO) TT=1800.15	UNSAT TT=27.757	UNSAT TT=59.056	? (MO) TT=12.84	? (TO) TT=1800.06	? (TO) TT=1800.17	UNSAT TT=305.247	UNSAT TT=152.164	? (TO) TT=1802.13	? (TO) TT=1801.23	? (TO) TT=1802.05	? TT=1792.31	UNSAT TT=127.143
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.tag14.ucl.opb	UNSAT TT=52.8	UNSAT TT=52.8	UNSAT TT=460.271	UNSAT TT=1619.02	? TT=1799.11	UNSAT TT=150.175	UNSAT TT=128.38	? (MO) TT=13.69	? (TO) TT=1800.1	? (TO) TT=1800.22	UNSAT TT=1648.88	UNSAT TT=767.276	? (TO) TT=1802.12	? (TO) TT=1801.67	? (TO) TT=1802.14	? TT=1790.71	UNSAT TT=433.181
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.tag8.ucl.opb	UNSAT TT=0.564	UNSAT TT=2.085	UNSAT TT=4.973	UNSAT TT=4.573	? (TO) TT=1800.17	UNSAT TT=1.977	UNSAT TT=1.999	UNSAT TT=1333.97	? (TO) TT=1800.04	UNSAT TT=745.94	UNSAT TT=16.794	UNSAT TT=7.819	UNSAT TT=316.9	UNSAT TT=225.442	? (TO) TT=1800.37	UNSAT TT=186.48	UNSAT TT=0.564
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.unbounded.all.ucl.opb	UNSAT TT=663.935	? (TO) TT=1800.11	UNSAT TT=1593.46	? TT=1798.04	? TT=1799.08	UNSAT TT=1045.06	UNSAT TT=1125.42	? (MO) TT=13.96	? (TO) TT=1800.06	? (TO) TT=1800.19	? (TO) TT=1800.52	? (TO) TT=1800.27	? (TO) TT=1802.14	? (TO) TT=1802.02	? (TO) TT=1802.19	? TT=1791.86	UNSAT TT=663.935

Statistics		borg-pb 10.05.30 (complete)	bsolo 3.2 Card (complete)	bsolo 3.2 Cl (complete)	pb_cplex 2010-06-29 (complete)	PB/CT 0.1 (complete)	PB/CT 0.1 fixed (complete)	PBPASSolver 2010-06-13 (complete)	PB-wave alpha 2 (incomplete)	SAT4J PB CuttingPlanes 2.2.0 2010-05-26 (complete)	SAT4J PB RES // CP 2.2.0 2010-05-31 (complete)	SAT4J PB Resolution 2.2.0 2010-05-26 (complete)	SCIPclp SCIP 1.2.1.2 with Clp 1.11.1 (Release Version) as LP solver (complete)	SCIPnone SCIP 1.2.1.2 without any LP solver (complete)	SCIPspx SCIP 1.2.1.2 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)	SCIPspx SCIP 1.2.1.3 with SoPlex 1.4.2 (CVS Version 30.5.2010) as LP solver (complete)	wbo 1.4a (complete)
Number of times the solver is able to give the best known answer		41	42	41	19	0	44	0	0	36	41	42	24	33	23	24	43
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)		6	2	3	0	0	1	0	0	0	0	1	0	0	0	0	36