PB'16 competition: satisfaction and optimization track: results by benchmark

Results by benchmark for category no optimisation, small integers, linear constraints (DEC-SMALLINT-LIN), subcategory Industrial

This page displays the results of the different solvers for each benchmark for category no optimisation, small integers, linear constraints (DEC-SMALLINT-LIN), subcategory Industrial

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell example	Meaning
Answer	Solver result
f=...	value of the objective function for the model reported by the solver
TT=...	Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out

Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

Abbreviation	Meaning
f=...	Value of the objective function
TO	Time Out
MO	Mem. Out (out of memory)

Meaning of the different colors:

Color	Meaning
text	the solver cannot handle this instance
text	the solver gave no answer
text	the solver could give an answer (SAT)
text	the solver gave a definitive answer (OPTIMUM FOUND or UNSAT)
text	the solver performed better than the other ones on that instance (complete solvers point of view)
text	the solver performed better than the other ones on that instance (incomplete solvers point of view)
text	the solver was ended by a signal or other problem
text	the solver gave an incomplete answer
text	the solver gave a wrong answer

For better readability, you may choose to hide some solvers:
cdcl-cuttingplanes DEC 2016-05-01 (complete)
minisatp 2012-10-02 git-d91742b (complete)
NaPS 1.02 (complete)
Open-WBO PB16 (complete)
Open-WBO-LSU PB16 (complete)
Sat4j PB 2.3.6 Res+CP PB16 (complete)
Sat4j PB 2.3.6 Resolution PB16 (complete)
toysat 2016-05-02 (complete)

Bench name	Best results on this instance	cdcl-cuttingplanes DEC 2016-05-01 (complete)	minisatp 2012-10-02 git-d91742b (complete)	NaPS 1.02 (complete)	Open-WBO PB16 (complete)	Open-WBO-LSU PB16 (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl10_11_pb.cnf.cr.opb	UNSAT TT=0.01	UNSAT TT=0.01	UNSAT TT=4.987	UNSAT TT=0.535	UNSAT TT=1.121	UNSAT TT=1.119	UNSAT TT=0.293	UNSAT TT=86.215	UNSAT TT=505.555
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl10_15_pb.cnf.cr.opb	UNSAT TT=0.011	UNSAT TT=0.011	UNSAT TT=3.527	UNSAT TT=0.486	UNSAT TT=1.143	UNSAT TT=1.145	UNSAT TT=0.224	UNSAT TT=101.469	? (TO) TT=1800.04
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl10_20_pb.cnf.cr.opb	UNSAT TT=0.012	UNSAT TT=0.012	UNSAT TT=6.86	UNSAT TT=0.591	UNSAT TT=0.897	UNSAT TT=0.904	UNSAT TT=0.264	UNSAT TT=758.719	? (TO) TT=1800.1
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl15_16_pb.cnf.cr.opb	UNSAT TT=0.012	UNSAT TT=0.012	? (TO) TT=1800.01	UNSAT TT=1523.41	UNSAT TT=691.089	UNSAT TT=689.081	UNSAT TT=0.374	? (TO) TT=1800.01	? (TO) TT=1800.04
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl15_20_pb.cnf.cr.opb	UNSAT TT=0.011	UNSAT TT=0.011	? (TO) TT=1800.02	UNSAT TT=522.197	UNSAT TT=415.873	UNSAT TT=417.81	UNSAT TT=0.283	? (TO) TT=1800.05	? (TO) TT=1800.07
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl15_25_pb.cnf.cr.opb	UNSAT TT=0.012	UNSAT TT=0.012	? (TO) TT=1800.02	UNSAT TT=551.04	UNSAT TT=332.548	UNSAT TT=331.712	UNSAT TT=0.374	? (TO) TT=1800.09	? (TO) TT=1800.05
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl20_21_pb.cnf.cr.opb	UNSAT TT=0.012	UNSAT TT=0.012	? (TO) TT=1800.02	? (TO) TT=1800.06	? (TO) TT=1800.02	? (TO) TT=1800.02	UNSAT TT=0.566	? (TO) TT=1800.07	? (TO) TT=1800.1
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl20_25_pb.cnf.cr.opb	UNSAT TT=0.013	UNSAT TT=0.013	? (TO) TT=1800.02	? (TO) TT=1800.03	? (TO) TT=1800.02	? (TO) TT=1800.02	UNSAT TT=0.416	? (TO) TT=1800.66	? (TO) TT=1800.11
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl20_30_pb.cnf.cr.opb	UNSAT TT=0.013	UNSAT TT=0.013	? (TO) TT=1800.02	? (TO) TT=1800.02	? (TO) TT=1800.02	? (TO) TT=1800.02	UNSAT TT=0.443	? (TO) TT=1800.04	? (TO) TT=1800.05
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl30_31_pb.cnf.cr.opb	UNSAT TT=0.016	UNSAT TT=0.016	? (TO) TT=1800.02	? (TO) TT=1800.02	? (TO) TT=1800.01	? (TO) TT=1800.03	UNSAT TT=1.035	? (TO) TT=1800.62	? (TO) TT=1800.04
Bench name	Best results	cdcl-cuttingplanes DEC 2016-05-01 (complete)	minisatp 2012-10-02 git-d91742b (complete)	NaPS 1.02 (complete)	Open-WBO PB16 (complete)	Open-WBO-LSU PB16 (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl30_35_pb.cnf.cr.opb	UNSAT TT=0.017	UNSAT TT=0.017	? (TO) TT=1800.01	? (TO) TT=1800.11	? (TO) TT=1800.1	? (TO) TT=1800.02	UNSAT TT=0.623	? (TO) TT=1800.92	? (TO) TT=1800.07
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl30_40_pb.cnf.cr.opb	UNSAT TT=0.018	UNSAT TT=0.018	? (TO) TT=1800.03	? (TO) TT=1800.11	? (TO) TT=1800.09	? (TO) TT=1800.1	UNSAT TT=0.635	? (TO) TT=1800.01	? (TO) TT=1800.09
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl35_36_pb.cnf.cr.opb	UNSAT TT=0.018	UNSAT TT=0.018	? (TO) TT=1800.03	? (TO) TT=1800.03	? (TO) TT=1800.1	? (TO) TT=1800.11	UNSAT TT=1.504	? (TO) TT=1800.94	? (TO) TT=1800.1
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl35_40_pb.cnf.cr.opb	UNSAT TT=0.021	UNSAT TT=0.021	? (TO) TT=1800.02	? (TO) TT=1800.08	? (TO) TT=1800.02	? (TO) TT=1800.03	UNSAT TT=0.824	? (TO) TT=1800.84	? (TO) TT=1800.07
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl35_45_pb.cnf.cr.opb	UNSAT TT=0.021	UNSAT TT=0.021	? (TO) TT=1800.02	? (TO) TT=1800.02	? (TO) TT=1800.08	? (TO) TT=1800.02	UNSAT TT=0.883	? (TO) TT=1800.04	? (TO) TT=1800.06
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl40_41_pb.cnf.cr.opb	UNSAT TT=0.021	UNSAT TT=0.021	? (TO) TT=1800.11	? (TO) TT=1800.05	? (TO) TT=1800.02	? (TO) TT=1800.01	UNSAT TT=1.563	? (TO) TT=1800.03	? (TO) TT=1800.1
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl40_45_pb.cnf.cr.opb	UNSAT TT=0.023	UNSAT TT=0.023	? (TO) TT=1800.02	? (TO) TT=1800.03	? (TO) TT=1800.01	? (TO) TT=1800.1	UNSAT TT=0.963	? (TO) TT=1800.02	? (TO) TT=1800.09
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl40_50_pb.cnf.cr.opb	UNSAT TT=0.025	UNSAT TT=0.025	? (TO) TT=1800.09	? (TO) TT=1800.11	? (TO) TT=1800.02	? (TO) TT=1800.1	UNSAT TT=0.994	? (TO) TT=1800.72	? (TO) TT=1800.01
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl50_51_pb.cnf.cr.opb	UNSAT TT=0.029	UNSAT TT=0.029	? (TO) TT=1800.1	? (TO) TT=1800.11	? (TO) TT=1800.09	? (TO) TT=1800.02	UNSAT TT=1.973	? (TO) TT=1800.03	? (TO) TT=1800.04
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl50_55_pb.cnf.cr.opb	UNSAT TT=0.032	UNSAT TT=0.032	? (TO) TT=1800.1	? (TO) TT=1800.1	? (TO) TT=1800.01	? (TO) TT=1800.11	UNSAT TT=1.123	? (TO) TT=1800.07	? (TO) TT=1800.11
Bench name	Best results	cdcl-cuttingplanes DEC 2016-05-01 (complete)	minisatp 2012-10-02 git-d91742b (complete)	NaPS 1.02 (complete)	Open-WBO PB16 (complete)	Open-WBO-LSU PB16 (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl50_60_pb.cnf.cr.opb	UNSAT TT=0.034	UNSAT TT=0.034	? (TO) TT=1800.1	? (TO) TT=1800.11	? (TO) TT=1800.01	? (TO) TT=1800.07	UNSAT TT=1.164	? (TO) TT=1800.06	? (TO) TT=1800.05
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga10_10_sat_pb.cnf.cr.opb	SAT TT=0.007 T1=0	SAT TT=0.01 T1=0.01	SAT TT=0.99 T1=0.98	SAT TT=0.017 T1=0.01	SAT TT=0.007 T1=0	SAT TT=0.007 T1=0	SAT TT=0.564 T1=0.67	SAT TT=0.404 T1=0.24	SAT TT=0.054 T1=0.05
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga10_8_sat_pb.cnf.cr.opb	SAT TT=0.006 T1=0	SAT TT=0.011 T1=0.01	SAT TT=0.008 T1=0	SAT TT=0.011 T1=0.01	SAT TT=0.006 T1=0	SAT TT=0.006 T1=0	SAT TT=0.883 T1=0.68	SAT TT=0.423 T1=0.23	SAT TT=0.032 T1=0.03
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga10_9_sat_pb.cnf.cr.opb	SAT TT=0.007 T1=0	SAT TT=0.011 T1=0.01	SAT TT=0.021 T1=0.02	SAT TT=0.013 T1=0.01	SAT TT=0.007 T1=0	SAT TT=0.008 T1=0	SAT TT=0.895 T1=0.65	SAT TT=0.413 T1=0.22	SAT TT=0.028 T1=0.02
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga11_10_sat_pb.cnf.cr.opb	SAT TT=0.009 T1=0.01	SAT TT=0.01 T1=0.01	SAT TT=0.011 T1=0.01	SAT TT=0.014 T1=0.01	SAT TT=0.009 T1=0.01	SAT TT=0.009 T1=0.01	SAT TT=0.683 T1=0.66	SAT TT=0.434 T1=0.25	SAT TT=0.034 T1=0.03
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga11_11_sat_pb.cnf.cr.opb	SAT TT=0.007 T1=0	SAT TT=0.013 T1=0.01	SAT TT=0.027 T1=0.02	SAT TT=0.013 T1=0.01	SAT TT=0.009 T1=0	SAT TT=0.007 T1=0.01	SAT TT=0.943 T1=0.67	SAT TT=0.432 T1=0.25	SAT TT=0.035 T1=0.03
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga11_9_sat_pb.cnf.cr.opb	SAT TT=0.006 T1=0	SAT TT=0.013 T1=0.01	SAT TT=0.009 T1=0	SAT TT=0.026 T1=0.02	SAT TT=0.006 T1=0	SAT TT=0.006 T1=0	SAT TT=1.255 T1=0.67	SAT TT=0.423 T1=0.24	SAT TT=0.038 T1=0.03
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga12_10_sat_pb.cnf.cr.opb	SAT TT=0.011 T1=0.01	SAT TT=0.011 T1=0.01	SAT TT=0.014 T1=0.01	SAT TT=0.016 T1=0.01	SAT TT=0.021 T1=0.02	SAT TT=0.02 T1=0.02	SAT TT=1.176 T1=0.67	SAT TT=0.484 T1=0.27	SAT TT=0.043 T1=0.04
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga12_11_sat_pb.cnf.cr.opb	SAT TT=0.011 T1=0.01	SAT TT=0.011 T1=0.01	SAT TT=0.011 T1=0.01	SAT TT=0.019 T1=0.02	SAT TT=0.012 T1=0.01	SAT TT=0.011 T1=0.01	SAT TT=1.286 T1=0.68	SAT TT=0.534 T1=0.3	SAT TT=0.097 T1=0.09
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga12_12_sat_pb.cnf.cr.opb	SAT TT=0.012 T1=0.01	SAT TT=0.012 T1=0.01	SAT TT=0.18 T1=0.17	SAT TT=0.031 T1=0.03	SAT TT=0.024 T1=0.02	SAT TT=0.027 T1=0.02	SAT TT=1.434 T1=0.68	SAT TT=0.483 T1=0.28	SAT TT=0.065 T1=0.06
Bench name	Best results	cdcl-cuttingplanes DEC 2016-05-01 (complete)	minisatp 2012-10-02 git-d91742b (complete)	NaPS 1.02 (complete)	Open-WBO PB16 (complete)	Open-WBO-LSU PB16 (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga13_11_sat_pb.cnf.cr.opb	SAT TT=0.007 T1=0	SAT TT=0.011 T1=0.01	SAT TT=0.012 T1=0.01	SAT TT=0.016 T1=0.01	SAT TT=0.007 T1=0	SAT TT=0.007 T1=0	SAT TT=1.192 T1=0.68	SAT TT=0.506 T1=0.27	SAT TT=0.036 T1=0.03
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga13_12_sat_pb.cnf.cr.opb	SAT TT=0.011 T1=0.01	SAT TT=0.011 T1=0.01	SAT TT=0.016 T1=0.01	SAT TT=0.02 T1=0.02	SAT TT=0.028 T1=0.02	SAT TT=0.028 T1=0.03	SAT TT=0.925 T1=0.69	SAT TT=0.524 T1=0.27	SAT TT=0.124 T1=0.12
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga13_13_sat_pb.cnf.cr.opb	SAT TT=0.011 T1=0.01	SAT TT=0.011 T1=0.01	SAT TT=0.026 T1=0.02	SAT TT=0.021 T1=0.02	SAT TT=0.024 T1=0.02	SAT TT=0.024 T1=0.02	SAT TT=1.445 T1=0.83	SAT TT=0.495 T1=0.28	SAT TT=0.046 T1=0.04
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga14_12_sat_pb.cnf.cr.opb	SAT TT=0.012 T1=0.01	SAT TT=0.012 T1=0.01	SAT TT=0.016 T1=0.01	SAT TT=0.025 T1=0.02	SAT TT=0.021 T1=0.02	SAT TT=0.02 T1=0.02	SAT TT=1.443 T1=0.68	SAT TT=0.543 T1=0.31	SAT TT=0.076 T1=0.07
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga14_13_sat_pb.cnf.cr.opb	SAT TT=0.005 T1=0	SAT TT=0.011 T1=0.01	SAT TT=0.02 T1=0.02	SAT TT=0.05 T1=0.05	SAT TT=0.005 T1=0.01	SAT TT=0.006 T1=0	SAT TT=1.543 T1=0.69	SAT TT=0.543 T1=0.33	SAT TT=0.083 T1=0.08
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga14_14_sat_pb.cnf.cr.opb	SAT TT=0.012 T1=0.01	SAT TT=0.012 T1=0.01	SAT TT=0.017 T1=0.02	SAT TT=0.025 T1=0.02	SAT TT=0.012 T1=0.02	SAT TT=0.012 T1=0.01	SAT TT=1.065 T1=0.71	SAT TT=0.495 T1=0.27	SAT TT=0.086 T1=0.08
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga15_13_sat_pb.cnf.cr.opb	SAT TT=0.012 T1=0.01	SAT TT=0.012 T1=0.01	SAT TT=0.02 T1=0.01	SAT TT=0.024 T1=0.02	SAT TT=0.012 T1=0.01	SAT TT=0.012 T1=0.01	SAT TT=1.526 T1=0.7	SAT TT=0.603 T1=0.36	SAT TT=0.128 T1=0.12
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga15_14_sat_pb.cnf.cr.opb	SAT TT=0.014 T1=0.01	SAT TT=0.014 T1=0.01	SAT TT=0.016 T1=0.01	SAT TT=0.022 T1=0.02	SAT TT=0.024 T1=0.02	SAT TT=0.024 T1=0.02	SAT TT=1.344 T1=0.72	SAT TT=0.583 T1=0.33	SAT TT=0.111 T1=0.1
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga15_15_sat_pb.cnf.cr.opb	SAT TT=0.013 T1=0.01	SAT TT=0.013 T1=0.01	SAT TT=0.018 T1=0.01	SAT TT=0.024 T1=0.02	SAT TT=0.017 T1=0.01	SAT TT=0.017 T1=0.01	SAT TT=1.553 T1=0.7	SAT TT=0.604 T1=0.34	SAT TT=0.096 T1=0.09
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga20_18_sat_pb.cnf.cr.opb	SAT TT=0.015 T1=0.01	SAT TT=0.015 T1=0.01	SAT TT=0.029 T1=0.02	SAT TT=0.041 T1=0.04	SAT TT=0.037 T1=0.03	SAT TT=0.036 T1=0.03	SAT TT=1.764 T1=0.73	SAT TT=0.664 T1=0.38	SAT TT=0.147 T1=0.13
Bench name	Best results	cdcl-cuttingplanes DEC 2016-05-01 (complete)	minisatp 2012-10-02 git-d91742b (complete)	NaPS 1.02 (complete)	Open-WBO PB16 (complete)	Open-WBO-LSU PB16 (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga20_19_sat_pb.cnf.cr.opb	SAT TT=0.015 T1=0.01	SAT TT=0.015 T1=0.01	SAT TT=0.033 T1=0.03	SAT TT=0.041 T1=0.04	SAT TT=0.043 T1=0.04	SAT TT=0.042 T1=0.04	SAT TT=1.784 T1=0.75	SAT TT=0.823 T1=0.75	SAT TT=0.142 T1=0.13
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga20_20_sat_pb.cnf.cr.opb	SAT TT=0.014 T1=0.01	SAT TT=0.014 T1=0.01	SAT TT=0.035 T1=0.03	SAT TT=0.045 T1=0.04	SAT TT=0.588 T1=0.68	SAT TT=0.589 T1=0.59	SAT TT=1.753 T1=0.75	SAT TT=0.704 T1=0.38	SAT TT=0.242 T1=0.23
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga25_23_sat_pb.cnf.cr.opb	SAT TT=0.018 T1=0.01	SAT TT=0.018 T1=0.01	SAT TT=0.07 T1=0.06	SAT TT=0.062 T1=0.06	SAT TT=0.127 T1=0.12	SAT TT=0.128 T1=0.12	SAT TT=3.623 T1=1.27	SAT TT=0.973 T1=0.63	SAT TT=0.302 T1=0.28
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga25_24_sat_pb.cnf.cr.opb	SAT TT=0.018 T1=0.01	SAT TT=0.018 T1=0.01	SAT TT=0.07 T1=0.06	SAT TT=0.071 T1=0.07	SAT TT=0.294 T1=0.29	SAT TT=0.3 T1=0.3	SAT TT=3.433 T1=1.28	SAT TT=1.055 T1=0.66	SAT TT=0.427 T1=0.4
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga25_25_sat_pb.cnf.cr.opb	SAT TT=0.018 T1=0.01	SAT TT=0.018 T1=0.01	SAT TT=0.079 T1=0.07	SAT TT=0.08 T1=0.08	SAT TT=0.054 T1=0.05	SAT TT=0.055 T1=0.05	SAT TT=3.742 T1=1.28	SAT TT=1.174 T1=0.69	SAT TT=0.74 T1=0.7
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga30_28_sat_pb.cnf.cr.opb	SAT TT=0.024 T1=0.02	SAT TT=0.024 T1=0.02	SAT TT=0.101 T1=0.1	SAT TT=0.119 T1=0.11	SAT TT=0.985 T1=0.98	SAT TT=0.989 T1=0.99	SAT TT=2.873 T1=1.31	SAT TT=1.123 T1=0.61	SAT TT=0.646 T1=0.6
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga30_29_sat_pb.cnf.cr.opb	SAT TT=0.023 T1=0.02	SAT TT=0.023 T1=0.02	SAT TT=0.101 T1=0.1	SAT TT=0.105 T1=0.1	SAT TT=0.223 T1=0.22	SAT TT=0.224 T1=0.22	SAT TT=2.563 T1=0.8	SAT TT=1.073 T1=0.61	SAT TT=0.637 T1=0.59
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga30_30_sat_pb.cnf.cr.opb	SAT TT=0.023 T1=0.02	SAT TT=0.023 T1=0.02	SAT TT=0.104 T1=0.1	SAT TT=0.141 T1=0.14	SAT TT=0.522 T1=0.52	SAT TT=0.495 T1=0.49	SAT TT=2.995 T1=1.32	SAT TT=1.133 T1=0.66	SAT TT=0.71 T1=0.65
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga35_33_sat_pb.cnf.cr.opb	SAT TT=0.032 T1=0.03	SAT TT=0.032 T1=0.03	SAT TT=0.162 T1=0.16	SAT TT=0.202 T1=0.2	SAT TT=1.135 T1=1.13	SAT TT=1.136 T1=1.13	SAT TT=3.203 T1=1.34	SAT TT=1.153 T1=0.56	SAT TT=0.949 T1=0.88
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga35_34_sat_pb.cnf.cr.opb	SAT TT=0.033 T1=0.03	SAT TT=0.033 T1=0.03	SAT TT=0.213 T1=0.21	SAT TT=0.188 T1=0.18	SAT TT=1.85 T1=1.85	SAT TT=1.817 T1=1.81	SAT TT=7.105 T1=2.86	SAT TT=1.154 T1=0.57	SAT TT=2.07 T1=2.01
Bench name	Best results	cdcl-cuttingplanes DEC 2016-05-01 (complete)	minisatp 2012-10-02 git-d91742b (complete)	NaPS 1.02 (complete)	Open-WBO PB16 (complete)	Open-WBO-LSU PB16 (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga35_35_sat_pb.cnf.cr.opb	SAT TT=0.03 T1=0.03	SAT TT=0.03 T1=0.03	SAT TT=0.202 T1=0.2	SAT TT=0.199 T1=0.2	SAT TT=1.842 T1=1.84	SAT TT=1.828 T1=1.82	SAT TT=6.002 T1=2.36	SAT TT=1.143 T1=0.56	SAT TT=1.09 T1=1.03
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga40_38_sat_pb.cnf.cr.opb	SAT TT=0.042 T1=0.04	SAT TT=0.042 T1=0.04	SAT TT=0.425 T1=0.42	SAT TT=0.226 T1=0.22	SAT TT=0.572 T1=0.57	SAT TT=0.573 T1=0.57	SAT TT=7.334 T1=2.88	SAT TT=1.655 T1=1	SAT TT=6.158 T1=6.1
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga40_39_sat_pb.cnf.cr.opb	SAT TT=0.039 T1=0.03	SAT TT=0.039 T1=0.03	SAT TT=0.288 T1=0.28	SAT TT=0.23 T1=0.23	SAT TT=2.09 T1=2.09	SAT TT=2.169 T1=2.16	SAT TT=7.464 T1=3.4	SAT TT=1.913 T1=1.18	SAT TT=0.984 T1=0.92
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga40_40_sat_pb.cnf.cr.opb	SAT TT=0.04 T1=0.03	SAT TT=0.04 T1=0.03	SAT TT=0.235 T1=0.23	SAT TT=0.241 T1=0.24	SAT TT=5.177 T1=5.17	SAT TT=5.232 T1=5.23	SAT TT=5.882 T1=2.37	SAT TT=1.513 T1=0.88	SAT TT=8.668 T1=8.6
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga45_43_sat_pb.cnf.cr.opb	SAT TT=0.053 T1=0.05	SAT TT=0.053 T1=0.05	SAT TT=0.384 T1=0.38	SAT TT=0.306 T1=0.3	SAT TT=11.305 T1=11.3	SAT TT=11.246 T1=11.24	SAT TT=8.833 T1=3.47	SAT TT=4.322 T1=3.51	SAT TT=3.14 T1=3.08
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga45_44_sat_pb.cnf.cr.opb	SAT TT=0.054 T1=0.05	SAT TT=0.054 T1=0.05	SAT TT=0.374 T1=0.37	SAT TT=0.345 T1=0.34	SAT TT=4.576 T1=4.57	SAT TT=4.533 T1=4.53	SAT TT=7.901 T1=3.42	SAT TT=1.863 T1=1.11	SAT TT=1.079 T1=1.35
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga45_45_sat_pb.cnf.cr.opb	SAT TT=0.05 T1=0.05	SAT TT=0.05 T1=0.05	SAT TT=0.414 T1=0.41	SAT TT=0.326 T1=0.32	SAT TT=19.136 T1=19.14	SAT TT=19.192 T1=19.19	SAT TT=8.922 T1=3.92	SAT TT=2.975 T1=2.21	SAT TT=4.924 T1=4.86
normalized-PB06/SATUNSAT-SMALLINT/ web/ uclid_pb_benchmarks/normalized-22s.smv.opb	UNSAT TT=0.088	UNSAT TT=0.088	UNSAT TT=0.529	UNSAT TT=7.772	UNSAT TT=2.306	UNSAT TT=2.311	UNSAT TT=2.084	UNSAT TT=1.203	UNSAT TT=0.986
normalized-PB06/SATUNSAT-SMALLINT/ web/ uclid_pb_benchmarks/normalized-25s.smv.opb	UNSAT TT=0.069	UNSAT TT=0.069	UNSAT TT=0.426	UNSAT TT=5.827	UNSAT TT=2.249	UNSAT TT=2.248	UNSAT TT=1.994	UNSAT TT=1.044	UNSAT TT=0.481
normalized-PB06/SATUNSAT-SMALLINT/ web/ uclid_pb_benchmarks/normalized-37s.smv.opb	UNSAT TT=0.029	UNSAT TT=0.029	UNSAT TT=0.067	UNSAT TT=0.242	UNSAT TT=0.041	UNSAT TT=0.041	UNSAT TT=1.023	UNSAT TT=0.764	UNSAT TT=0.199
Bench name	Best results	cdcl-cuttingplanes DEC 2016-05-01 (complete)	minisatp 2012-10-02 git-d91742b (complete)	NaPS 1.02 (complete)	Open-WBO PB16 (complete)	Open-WBO-LSU PB16 (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
normalized-PB06/SATUNSAT-SMALLINT/ web/ uclid_pb_benchmarks/normalized-43s.smv.opb	UNSAT TT=0.129	UNSAT TT=0.129	UNSAT TT=0.291	UNSAT TT=3.93	UNSAT TT=1.716	UNSAT TT=1.706	UNSAT TT=3.203	UNSAT TT=1.296	UNSAT TT=1.115
normalized-PB06/SATUNSAT-SMALLINT/ web/ uclid_pb_benchmarks/normalized-44s.smv.opb	UNSAT TT=0.016	UNSAT TT=0.016	UNSAT TT=0.037	UNSAT TT=0.401	UNSAT TT=0.057	UNSAT TT=0.059	UNSAT TT=1.044	UNSAT TT=0.564	UNSAT TT=0.09
normalized-PB06/SATUNSAT-SMALLINT/ web/ uclid_pb_benchmarks/normalized-46s.smv.opb	UNSAT TT=0.04	UNSAT TT=0.04	UNSAT TT=0.122	UNSAT TT=1.179	UNSAT TT=0.191	UNSAT TT=0.191	UNSAT TT=1.744	UNSAT TT=0.935	UNSAT TT=0.318
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-floppy1-2.ucl.opb	UNSAT TT=0.009	UNSAT TT=0.01	UNSAT TT=0.009	UNSAT TT=0.158	UNSAT TT=0.083	UNSAT TT=0.084	UNSAT TT=1.114	UNSAT TT=0.443	UNSAT TT=0.058
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-floppy1-3.ucl.opb	UNSAT TT=0.003	UNSAT TT=0.01	UNSAT TT=0.009	UNSAT TT=0.003	UNSAT TT=0.012	UNSAT TT=0.014	UNSAT TT=0.275	UNSAT TT=0.253	UNSAT TT=0.023
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-floppy1-4.ucl.opb	UNSAT TT=0.003	UNSAT TT=0.011	UNSAT TT=0.007	UNSAT TT=0.003	UNSAT TT=0.033	UNSAT TT=0.035	UNSAT TT=0.264	UNSAT TT=0.235	UNSAT TT=0.02
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-floppy1-6.ucl.opb	UNSAT TT=0.015	UNSAT TT=0.015	UNSAT TT=0.043	UNSAT TT=1.173	UNSAT TT=0.387	UNSAT TT=0.389	UNSAT TT=0.875	UNSAT TT=0.685	UNSAT TT=0.11
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-floppy1-7.ucl.opb	UNSAT TT=0.035	UNSAT TT=0.035	UNSAT TT=0.302	UNSAT TT=48.71	UNSAT TT=9.768	UNSAT TT=9.83	UNSAT TT=1.164	UNSAT TT=0.794	UNSAT TT=0.18
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-floppy1-8.ucl.opb	UNSAT TT=0.04	UNSAT TT=0.04	UNSAT TT=0.179	UNSAT TT=0.208	UNSAT TT=12.153	UNSAT TT=12.302	UNSAT TT=1.374	UNSAT TT=0.923	UNSAT TT=0.217
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-tlan2.ucl.opb	UNSAT TT=0.031	UNSAT TT=0.031	UNSAT TT=0.179	UNSAT TT=8.422	UNSAT TT=1.157	UNSAT TT=1.16	UNSAT TT=1.295	UNSAT TT=1.224	UNSAT TT=0.216
Bench name	Best results	cdcl-cuttingplanes DEC 2016-05-01 (complete)	minisatp 2012-10-02 git-d91742b (complete)	NaPS 1.02 (complete)	Open-WBO PB16 (complete)	Open-WBO-LSU PB16 (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-tlan3.ucl.opb	UNSAT TT=0.164	UNSAT TT=0.164	UNSAT TT=1.219	UNSAT TT=0.515	UNSAT TT=6.67	UNSAT TT=6.722	UNSAT TT=2.954	UNSAT TT=1.803	UNSAT TT=1.152
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-cache-ibm-q-full.all.ucl.opb	UNSAT TT=53.252	? (TO) TT=1800.03	UNSAT TT=53.252	UNSAT TT=225.322	UNSAT TT=222.064	UNSAT TT=207.56	? (TO) TT=1800.01	UNSAT TT=812.361	? (TO) TT=1800.07
normalized-PB06/SATUNSAT-SMALLINT/web/ uclid_pb_benchmarks/ normalized-cache-ibm-q-unbounded.Ic22arity.ucl.opb	UNSAT TT=5.698	UNSAT TT=6.882	UNSAT TT=5.698	UNSAT TT=50.062	UNSAT TT=15.846	UNSAT TT=15.848	UNSAT TT=55.506	UNSAT TT=38.149	UNSAT TT=132.829
normalized-PB06/SATUNSAT-SMALLINT/web/ uclid_pb_benchmarks/ normalized-cache-ibm-q-unbounded.Icl2arity.ucl.opb	UNSAT TT=0.835	UNSAT TT=0.835	UNSAT TT=1.574	UNSAT TT=11.461	UNSAT TT=2.307	UNSAT TT=2.291	UNSAT TT=7.553	UNSAT TT=2.794	UNSAT TT=14.455
normalized-PB06/SATUNSAT-SMALLINT/web/ uclid_pb_benchmarks/ normalized-cache-ibm-q-unbounded.Ih1arity.ucl.opb	UNSAT TT=55.624	? (TO) TT=1800.01	UNSAT TT=55.624	UNSAT TT=376.761	UNSAT TT=403.441	UNSAT TT=392.94	? (TO) TT=1800.11	UNSAT TT=923.7	? (TO) TT=1800.06
normalized-PB06/SATUNSAT-SMALLINT/web/ uclid_pb_benchmarks/ normalized-cache-ibm-q-unbounded.Ih2arity.ucl.opb	SAT TT=0.923 T1=0.9	SAT TT=0.923 T1=0.9	SAT TT=4.799 T1=4.76	SAT TT=95.685 T1=95.6	SAT TT=5.735 T1=5.66	SAT TT=5.732 T1=5.65	SAT TT=7.083 T1=3.4	SAT TT=3.832 T1=2.59	SAT TT=8.571 T1=8.44
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-cache.inv10.ucl.opb	UNSAT TT=0.104	UNSAT TT=0.104	UNSAT TT=0.418	UNSAT TT=0.425	UNSAT TT=0.11	UNSAT TT=0.11	UNSAT TT=2.713	UNSAT TT=1.133	UNSAT TT=0.663
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-cache.inv12.ucl.opb	UNSAT TT=1.61	UNSAT TT=25.798	UNSAT TT=1.719	UNSAT TT=1.61	UNSAT TT=2.206	UNSAT TT=2.191	UNSAT TT=26.101	UNSAT TT=10.591	UNSAT TT=61.824
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-cache.inv14.ucl.opb	UNSAT TT=99.91	? (TO) TT=1800.02	UNSAT TT=122.439	UNSAT TT=99.91	UNSAT TT=143.793	UNSAT TT=141.303	? (TO) TT=1800.84	UNSAT TT=1604.29	? (TO) TT=1800.05
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-cache.inv8.ucl.opb	UNSAT TT=0.029	UNSAT TT=0.033	UNSAT TT=0.084	UNSAT TT=0.077	UNSAT TT=0.029	UNSAT TT=0.029	UNSAT TT=1.555	UNSAT TT=0.645	UNSAT TT=0.133
Bench name	Best results	cdcl-cuttingplanes DEC 2016-05-01 (complete)	minisatp 2012-10-02 git-d91742b (complete)	NaPS 1.02 (complete)	Open-WBO PB16 (complete)	Open-WBO-LSU PB16 (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-dlx1c.rwmem1.ucl.opb	UNSAT TT=0.29	UNSAT TT=0.291	UNSAT TT=0.461	UNSAT TT=1.531	UNSAT TT=0.29	UNSAT TT=0.291	UNSAT TT=4.312	UNSAT TT=1.937	UNSAT TT=2.251
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-dlx1c.rwmem.ucl.opb	UNSAT TT=0.276	UNSAT TT=0.276	UNSAT TT=0.543	UNSAT TT=1.566	UNSAT TT=0.444	UNSAT TT=0.441	UNSAT TT=3.462	UNSAT TT=1.684	UNSAT TT=2.678
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-dlx1c.ucl.opb	UNSAT TT=0.14	UNSAT TT=0.14	UNSAT TT=0.373	UNSAT TT=1.363	UNSAT TT=0.262	UNSAT TT=0.263	UNSAT TT=2.714	UNSAT TT=1.494	UNSAT TT=0.949
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-elf.rf10.ucl.opb	UNSAT TT=3.067	UNSAT TT=3.067	UNSAT TT=3.413	UNSAT TT=14.798	UNSAT TT=3.292	UNSAT TT=3.305	UNSAT TT=11.702	UNSAT TT=4.693	UNSAT TT=23.679
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-elf.rf6.ucl.opb	UNSAT TT=0.001	UNSAT TT=0.01	UNSAT TT=0.003	UNSAT TT=0.004	UNSAT TT=0.001	UNSAT TT=0.001	UNSAT TT=0.185	UNSAT TT=0.172	UNSAT TT=0.017
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-elf.rf7.ucl.opb	UNSAT TT=0.024	UNSAT TT=0.024	UNSAT TT=0.072	UNSAT TT=0.206	UNSAT TT=0.029	UNSAT TT=0.034	UNSAT TT=0.994	UNSAT TT=0.613	UNSAT TT=0.119
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-elf.rf8.ucl.opb	UNSAT TT=0.088	UNSAT TT=0.088	UNSAT TT=0.344	UNSAT TT=1.457	UNSAT TT=0.18	UNSAT TT=0.182	UNSAT TT=2.234	UNSAT TT=1.175	UNSAT TT=0.538
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-elf.rf9.ucl.opb	UNSAT TT=0.447	UNSAT TT=0.447	UNSAT TT=1.198	UNSAT TT=5.6	UNSAT TT=0.64	UNSAT TT=0.646	UNSAT TT=3.632	UNSAT TT=2.055	UNSAT TT=3.278
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.burch_dill.2.accl.ucl.opb	UNSAT TT=0.291	UNSAT TT=0.77	UNSAT TT=0.291	UNSAT TT=1.27	UNSAT TT=1.189	UNSAT TT=1.186	UNSAT TT=7.29	UNSAT TT=2.454	UNSAT TT=6.455
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.burch_dill.3.accl.ucl.opb	UNSAT TT=0.865	UNSAT TT=3.52	UNSAT TT=0.865	UNSAT TT=3.383	UNSAT TT=3.921	UNSAT TT=3.988	UNSAT TT=11.761	UNSAT TT=4.402	UNSAT TT=20.333
Bench name	Best results	cdcl-cuttingplanes DEC 2016-05-01 (complete)	minisatp 2012-10-02 git-d91742b (complete)	NaPS 1.02 (complete)	Open-WBO PB16 (complete)	Open-WBO-LSU PB16 (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.burch_dill.4.accl.ucl.opb	UNSAT TT=0.103	UNSAT TT=0.103	UNSAT TT=0.268	UNSAT TT=2.285	UNSAT TT=0.359	UNSAT TT=0.36	UNSAT TT=3.504	UNSAT TT=1.634	UNSAT TT=0.565
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.burch_dill.6.accl.ucl.opb	UNSAT TT=477.303	? (TO) TT=1800.1	UNSAT TT=489.47	UNSAT TT=477.303	UNSAT TT=923.75	UNSAT TT=959.686	? (TO) TT=1800.05	UNSAT TT=583.905	? (TO) TT=1800.07
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.burch_dill.8.accl.ucl.opb	UNSAT TT=61.157	? (TO) TT=1800.02	UNSAT TT=61.157	UNSAT TT=122.528	UNSAT TT=216.423	UNSAT TT=190.479	UNSAT TT=463.753	UNSAT TT=193.13	? (TO) TT=1800.08
normalized-PB06/SATUNSAT-SMALLINT/web/ uclid_pb_benchmarks/ normalized-ooo.ex.br.mem.RobRegValid_bar.ucl.opb	UNSAT TT=6.301	UNSAT TT=179.854	UNSAT TT=6.301	UNSAT TT=169.183	UNSAT TT=45.136	UNSAT TT=45.39	UNSAT TT=77.371	UNSAT TT=22.162	UNSAT TT=131.046
normalized-PB06/SATUNSAT-SMALLINT/web/uclid_pb_benchmarks/ normalized-ooo.ex.br.mem.Src1Valid_Src1ValidBar.ucl.opb	UNSAT TT=40.521	? (TO) TT=1800.02	UNSAT TT=40.521	UNSAT TT=1449.56	UNSAT TT=472.589	UNSAT TT=495.587	UNSAT TT=786.752	UNSAT TT=354.14	? (TO) TT=1800.1
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.ex.mem.LdValue.ucl.opb	UNSAT TT=34.559	? (TO) TT=1800.02	UNSAT TT=34.559	? (TO) TT=1800.05	UNSAT TT=305.186	UNSAT TT=344.729	UNSAT TT=607.091	UNSAT TT=294.009	? (TO) TT=1800.05
normalized-PB06/SATUNSAT-SMALLINT/web/uclid_pb_benchmarks/ normalized-ooo.ex.mem.LsqHdStrong.ucl.opb	UNSAT TT=62.995	? (TO) TT=1800.11	UNSAT TT=62.995	? (TO) TT=1800.06	UNSAT TT=1321.3	UNSAT TT=1298.24	? (TO) TT=1800.18	? (TO) TT=1800.77	? (TO) TT=1800.04
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.rf10.ucl.opb	UNSAT TT=2.392	UNSAT TT=10.574	UNSAT TT=2.392	UNSAT TT=4.463	UNSAT TT=5.828	UNSAT TT=5.917	UNSAT TT=31.559	UNSAT TT=16.241	UNSAT TT=348.415
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.rf6.ucl.opb	UNSAT TT=0.034	UNSAT TT=0.034	UNSAT TT=0.096	UNSAT TT=0.107	UNSAT TT=0.035	UNSAT TT=0.035	UNSAT TT=1.483	UNSAT TT=0.834	UNSAT TT=0.245
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.rf7.ucl.opb	UNSAT TT=0.096	UNSAT TT=0.096	UNSAT TT=0.203	UNSAT TT=0.661	UNSAT TT=0.103	UNSAT TT=0.106	UNSAT TT=3.943	UNSAT TT=1.164	UNSAT TT=0.954
Bench name	Best results	cdcl-cuttingplanes DEC 2016-05-01 (complete)	minisatp 2012-10-02 git-d91742b (complete)	NaPS 1.02 (complete)	Open-WBO PB16 (complete)	Open-WBO-LSU PB16 (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.rf8.ucl.opb	UNSAT TT=0.232	UNSAT TT=0.334	UNSAT TT=0.39	UNSAT TT=1.019	UNSAT TT=0.233	UNSAT TT=0.232	UNSAT TT=4.004	UNSAT TT=1.704	UNSAT TT=3.458
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.rf9.ucl.opb	UNSAT TT=0.762	UNSAT TT=1.42	UNSAT TT=0.762	UNSAT TT=1.687	UNSAT TT=0.951	UNSAT TT=0.962	UNSAT TT=13.552	UNSAT TT=4.892	UNSAT TT=23.041
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.tag10.ucl.opb	UNSAT TT=0.504	UNSAT TT=0.663	UNSAT TT=0.894	UNSAT TT=1.874	UNSAT TT=0.504	UNSAT TT=0.504	UNSAT TT=21.661	UNSAT TT=6.303	UNSAT TT=45.578
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.tag12.ucl.opb	UNSAT TT=1.582	UNSAT TT=4.577	UNSAT TT=3.571	UNSAT TT=3.159	UNSAT TT=1.643	UNSAT TT=1.582	UNSAT TT=103.977	UNSAT TT=62.41	UNSAT TT=183.174
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.tag14.ucl.opb	UNSAT TT=4.165	UNSAT TT=12.741	UNSAT TT=22.036	UNSAT TT=8.87	UNSAT TT=4.165	UNSAT TT=4.215	UNSAT TT=425.25	UNSAT TT=110.41	UNSAT TT=489.829
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.tag8.ucl.opb	UNSAT TT=0.205	UNSAT TT=0.567	UNSAT TT=0.205	UNSAT TT=0.678	UNSAT TT=0.346	UNSAT TT=0.372	UNSAT TT=6.705	UNSAT TT=2.423	UNSAT TT=4.307
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.unbounded.all.ucl.opb	UNSAT TT=31.401	? (TO) TT=1800.02	UNSAT TT=31.401	UNSAT TT=845.324	UNSAT TT=160.95	UNSAT TT=160.755	UNSAT TT=622.92	UNSAT TT=214.432	UNSAT TT=1675.13

Statistics		cdcl-cuttingplanes DEC 2016-05-01 (complete)	minisatp 2012-10-02 git-d91742b (complete)	NaPS 1.02 (complete)	Open-WBO PB16 (complete)	Open-WBO-LSU PB16 (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
Number of times the solver is able to give the best known answer		98	89	90	92	92	102	88	79
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)		72	16	5	12	11	0	0	0