PB'06 evaluation: results by benchmark

Results by benchmark for category no optimisation, small integers (SATUNSAT-SMALLINT), subcategory Industrial

This page displays the results of the different solvers for each benchmark for category no optimisation, small integers (SATUNSAT-SMALLINT), subcategory Industrial

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell example	Meaning
Answer	Solver result
f=...	value of the objective function for the model reported by the solver
TT=...	Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out
T1=...	Time to get the best model (T1): this is the wall clock time (in seconds) ellapsed when the solver found its best model. This time is obtained from the 'o lines'. It is useful to compare both complete and incomplete solvers. It doesn't take into account the time to prove optimality of the solution.

Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

Abbreviation	Meaning
f=...	Value of the objective function
TO	Time Out
MO	Mem. Out (out of memory)

Meaning of the different colors:

Color	Meaning
text	the solver cannot handle this instance
text	the solver gave no answer
text	the solver could give an answer (SAT)
text	the solver gave a definitive answer (OPTIMUM FOUND or UNSAT)
text	the solver performed better than the other ones on that instance (complete solvers point of view)
text	the solver performed better than the other ones on that instance (incomplete solvers point of view)
text	the solver was ended by a signal or other problem
text	the solver gave an incomplete answer
text	the solver gave a wrong answer

For better readability, you may choose to hide some solvers:
absconPseudo 1
bsolo 2006/05
glpPB 0.2
minisat+ 1.14
PBS 4.1L
PB-smodels 1.28
PB-smodels 1.31
Pueblo 1.3
Pueblo 1.4
SAT4JPSEUDO 2006.2
SAT4JPSEUDO 2006.2 Heuristics
wildcat-rnp 0.8.0
wildcat-skc 0.8.0

Bench name	Best results on this instance	absconPseudo 1	bsolo 2006/05	glpPB 0.2	minisat+ 1.14	PBS 4.1L	PB-smodels 1.28	PB-smodels 1.31	Pueblo 1.3	Pueblo 1.4	SAT4JPSEUDO 2006.2	SAT4JPSEUDO 2006.2 Heuristics	wildcat-rnp 0.8.0	wildcat-skc 0.8.0
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl10_11_pb.cnf.cr.opb	UNSAT TT=0.002	? (TO) TT=1800.49	UNSAT TT=53.961	UNSAT TT=0.008	UNSAT TT=29.807	UNSAT TT=0.002	? (TO) TT=1800.75	? (TO) TT=1800.67	UNSAT TT=886.094	UNSAT TT=0.007	UNSAT TT=0.394	UNSAT TT=0.375	? (TO) TT=1800.61	? (TO) TT=1800.5
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl10_15_pb.cnf.cr.opb	UNSAT TT=0.004	? (TO) TT=1800.27	UNSAT TT=105.05	UNSAT TT=0.007	UNSAT TT=138.541	UNSAT TT=0.004	? (TO) TT=1800.5	? (TO) TT=1800.67	? (TO) TT=1800.7	UNSAT TT=0.008	UNSAT TT=0.4	UNSAT TT=0.397	? (TO) TT=1800.79	? (TO) TT=1800.7
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl10_20_pb.cnf.cr.opb	UNSAT TT=0.003	? (TO) TT=1800.34	UNSAT TT=154.963	UNSAT TT=0.009	UNSAT TT=120.545	UNSAT TT=0.003	? (TO) TT=1800.6	? (TO) TT=1800.54	? (TO) TT=1800.69	UNSAT TT=0.008	UNSAT TT=0.423	UNSAT TT=0.434	? (TO) TT=1800.47	? (TO) TT=1800.7
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl15_16_pb.cnf.cr.opb	UNSAT TT=0.004	? (TO) TT=1800.67	? TT=1800.09	UNSAT TT=0.011	? (TO) TT=1800.53	UNSAT TT=0.004	? (TO) TT=1800.63	? (TO) TT=1800.77	? (TO) TT=1800.55	UNSAT TT=0.011	UNSAT TT=0.647	UNSAT TT=0.638	? (TO) TT=1800.7	? (TO) TT=1800.65
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl15_20_pb.cnf.cr.opb	UNSAT TT=0.004	? (TO) TT=1800.32	? TT=1800.1	UNSAT TT=0.011	? (TO) TT=1800.67	UNSAT TT=0.004	? (TO) TT=1800.52	? (TO) TT=1800.66	? (TO) TT=1800.66	UNSAT TT=0.014	UNSAT TT=0.543	UNSAT TT=0.551	? (TO) TT=1800.64	? (TO) TT=1800.71
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl15_25_pb.cnf.cr.opb	UNSAT TT=0.006	? (TO) TT=1800.37	? TT=1800.11	UNSAT TT=0.015	? (TO) TT=1800.7	UNSAT TT=0.006	? (TO) TT=1800.8	? (TO) TT=1800.77	? TT=1784.95	UNSAT TT=0.013	UNSAT TT=0.627	UNSAT TT=0.622	? (TO) TT=1800.65	? (TO) TT=1800.51
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl20_21_pb.cnf.cr.opb	UNSAT TT=0.006	? (TO) TT=1800.53	? TT=1800.04	UNSAT TT=0.019	? (TO) TT=1800.73	UNSAT TT=0.006	? (TO) TT=1800.61	? (TO) TT=1800.74	? (TO) TT=1800.54	UNSAT TT=0.016	UNSAT TT=0.793	UNSAT TT=0.856	? (TO) TT=1800.75	? (TO) TT=1800.64
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl20_25_pb.cnf.cr.opb	UNSAT TT=0.008	? (TO) TT=1800.33	? TT=1800.02	UNSAT TT=0.019	? (TO) TT=1800.71	UNSAT TT=0.008	? (TO) TT=1800.62	? (TO) TT=1800.72	? (TO) TT=1800.55	UNSAT TT=0.024	UNSAT TT=0.659	UNSAT TT=0.678	? (TO) TT=1800.51	? (TO) TT=1800.55
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl20_30_pb.cnf.cr.opb	UNSAT TT=0.009	? (TO) TT=1800.54	? TT=1800.02	UNSAT TT=0.022	? (TO) TT=1800.56	UNSAT TT=0.009	? (TO) TT=1800.74	? (TO) TT=1800.72	? (TO) TT=1800.71	UNSAT TT=0.03	UNSAT TT=0.728	UNSAT TT=0.72	? (TO) TT=1800.7	? (TO) TT=1800.86
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl30_31_pb.cnf.cr.opb	UNSAT TT=0.015	? (TO) TT=1800.49	? TT=1800.01	UNSAT TT=0.04	? (TO) TT=1800.48	UNSAT TT=0.015	? (TO) TT=1800.8	? (TO) TT=1800.75	? (TO) TT=1800.71	UNSAT TT=0.054	UNSAT TT=1.139	UNSAT TT=1.138	? (TO) TT=1800.59	? (TO) TT=1800.45
Bench name	Best results on this instance	absconPseudo 1	bsolo 2006/05	glpPB 0.2	minisat+ 1.14	PBS 4.1L	PB-smodels 1.28	PB-smodels 1.31	Pueblo 1.3	Pueblo 1.4	SAT4JPSEUDO 2006.2	SAT4JPSEUDO 2006.2 Heuristics	wildcat-rnp 0.8.0	wildcat-skc 0.8.0
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl30_35_pb.cnf.cr.opb	UNSAT TT=0.015	? (TO) TT=1800.54	? TT=1800.01	UNSAT TT=0.041	? (TO) TT=1800.75	UNSAT TT=0.015	? (TO) TT=1800.75	? (TO) TT=1800.82	? (TO) TT=1800.57	UNSAT TT=0.088	UNSAT TT=0.872	UNSAT TT=0.884	? (TO) TT=1800.48	? (TO) TT=1800.53
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl30_40_pb.cnf.cr.opb	UNSAT TT=0.019	? (TO) TT=1800.4	? TT=1800.01	UNSAT TT=0.046	? (TO) TT=1800.5	UNSAT TT=0.019	? (TO) TT=1800.77	? (TO) TT=1800.62	? (TO) TT=1800.59	UNSAT TT=0.099	UNSAT TT=0.916	UNSAT TT=0.919	? (TO) TT=1800.63	? (TO) TT=1800.69
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl35_36_pb.cnf.cr.opb	UNSAT TT=0.019	? (TO) TT=1800.55	? TT=1800.01	UNSAT TT=0.053	? (TO) TT=1800.7	UNSAT TT=0.019	? (TO) TT=1800.69	? (TO) TT=1800.71	? (TO) TT=1800.57	UNSAT TT=0.106	UNSAT TT=1.236	UNSAT TT=1.226	? (TO) TT=1800.65	? (TO) TT=1800.66
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl35_40_pb.cnf.cr.opb	UNSAT TT=0.023	? (TO) TT=1800.56	? TT=1800.01	UNSAT TT=0.057	? (TO) TT=1800.63	UNSAT TT=0.023	? (TO) TT=1800.61	? (TO) TT=1800.74	? (TO) TT=1800.63	UNSAT TT=0.14	UNSAT TT=1.007	UNSAT TT=1.072	? (TO) TT=1800.67	? (TO) TT=1800.6
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl35_45_pb.cnf.cr.opb	UNSAT TT=0.026	? (TO) TT=1800.72	? TT=1800.01	UNSAT TT=0.066	? (TO) TT=1800.62	UNSAT TT=0.026	? (TO) TT=1800.76	? (TO) TT=1800.65	? (TO) TT=1800.5	UNSAT TT=0.176	UNSAT TT=1.088	UNSAT TT=1.056	? (TO) TT=1800.51	? (TO) TT=1800.7
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl40_41_pb.cnf.cr.opb	UNSAT TT=0.026	? (TO) TT=1800.67	? TT=1800	UNSAT TT=0.082	? (TO) TT=1800.73	UNSAT TT=0.026	? (TO) TT=1800.51	? (TO) TT=1800.69	? (TO) TT=1800.69	UNSAT TT=0.181	UNSAT TT=1.477	UNSAT TT=1.485	? (TO) TT=1800.72	? (TO) TT=1800.75
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl40_45_pb.cnf.cr.opb	UNSAT TT=0.03	? (TO) TT=1800.7	? TT=1800.01	UNSAT TT=0.077	? (TO) TT=1800.71	UNSAT TT=0.03	? (TO) TT=1800.79	? (TO) TT=1800.69	? (TO) TT=1800.56	UNSAT TT=0.202	UNSAT TT=1.209	UNSAT TT=1.116	? (TO) TT=1800.7	? (TO) TT=1800.74
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl40_50_pb.cnf.cr.opb	UNSAT TT=0.031	? (TO) TT=1800.75	? TT=1800.01	UNSAT TT=0.087	? (TO) TT=1800.71	UNSAT TT=0.031	? (TO) TT=1800.74	? (TO) TT=1800.63	? (TO) TT=1800.7	UNSAT TT=0.268	UNSAT TT=1.232	UNSAT TT=1.205	? (TO) TT=1800.55	? (TO) TT=1800.66
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl50_51_pb.cnf.cr.opb	UNSAT TT=0.041	? (TO) TT=1800.78	? TT=1800	UNSAT TT=0.119	? (TO) TT=1800.6	UNSAT TT=0.041	? (TO) TT=1800.64	? (TO) TT=1800.6	? (TO) TT=1800.65	UNSAT TT=0.459	UNSAT TT=2.214	UNSAT TT=2.554	? (TO) TT=1800.55	? (TO) TT=1800.65
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl50_55_pb.cnf.cr.opb	UNSAT TT=0.045	? (TO) TT=1800.61	? TT=1800.01	UNSAT TT=0.131	? (TO) TT=1800.84	UNSAT TT=0.045	? (TO) TT=1800.62	? (TO) TT=1800.78	? (TO) TT=1800.69	UNSAT TT=0.55	UNSAT TT=1.628	UNSAT TT=1.557	? (TO) TT=1800.63	? (TO) TT=1800.53
Bench name	Best results on this instance	absconPseudo 1	bsolo 2006/05	glpPB 0.2	minisat+ 1.14	PBS 4.1L	PB-smodels 1.28	PB-smodels 1.31	Pueblo 1.3	Pueblo 1.4	SAT4JPSEUDO 2006.2	SAT4JPSEUDO 2006.2 Heuristics	wildcat-rnp 0.8.0	wildcat-skc 0.8.0
normalized-PB06/SATUNSAT-SMALLINT/submitted-PB05/ aloul/FPGA_SAT05/normalized-chnl50_60_pb.cnf.cr.opb	UNSAT TT=0.049	? (TO) TT=1800.9	? TT=1800.01	UNSAT TT=0.141	? (TO) TT=1800.55	UNSAT TT=0.049	? (TO) TT=1800.77	? (TO) TT=1800.75	? (TO) TT=1800.66	UNSAT TT=0.722	UNSAT TT=1.643	UNSAT TT=2.16	? (TO) TT=1800.68	? (TO) TT=1800.71
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga10_10_sat_pb.cnf.cr.opb	SAT TT=0.003 T1=0	? (TO) TT=1800.34	SAT TT=334.914 T1=335	SAT TT=0.289 T1=0.29	SAT TT=0.018 T1=0.01	SAT TT=1.156 T1=1.16	SAT TT=0.063 T1=0.05	SAT TT=0.051 T1=0.04	SAT TT=0.013 T1=0.01	SAT TT=0.007 T1=0.01	SAT TT=0.907 T1=0.58	SAT TT=0.96 T1=0.58	SAT TT=0.003 T1=0	SAT TT=0.003 T1=0
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga10_8_sat_pb.cnf.cr.opb	SAT TT=0.002 T1=0	? (TO) TT=1800.28	SAT TT=78.919 T1=78.94	SAT TT=5.044 T1=5.05	SAT TT=0.015 T1=0.01	SAT TT=0.663 T1=0.66	SAT TT=2.433 T1=2.45	SAT TT=0.053 T1=0.05	SAT TT=0.014 T1=0.01	SAT TT=0.011 T1=0.01	SAT TT=23.849 T1=22.52	SAT TT=23.992 T1=22.72	SAT TT=0.002 T1=0	SAT TT=0.003 T1=0
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga10_9_sat_pb.cnf.cr.opb	SAT TT=0.002 T1=0	? (TO) TT=1800.27	SAT TT=12.555 T1=12.56	SAT TT=111.623 T1=111.65	SAT TT=0.016 T1=0.01	SAT TT=0.925 T1=0.92	SAT TT=0.523 T1=0.52	SAT TT=0.057 T1=0.05	SAT TT=0.016 T1=0.01	SAT TT=0.006 T1=0.01	SAT TT=2.192 T1=1.34	SAT TT=2.167 T1=1.34	SAT TT=0.003 T1=0	SAT TT=0.002 T1=0
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga11_10_sat_pb.cnf.cr.opb	SAT TT=0.003 T1=0	? (TO) TT=1800.24	SAT TT=824.58 T1=824.87	? (TO) TT=1800.65	SAT TT=0.018 T1=0.02	SAT TT=1.226 T1=1.22	SAT TT=2.473 T1=2.46	SAT TT=0.063 T1=0.05	SAT TT=0.018 T1=0.02	SAT TT=0.007 T1=0.01	SAT TT=0.69 T1=0.47	SAT TT=0.687 T1=0.49	SAT TT=0.003 T1=0	SAT TT=0.005 T1=0
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga11_11_sat_pb.cnf.cr.opb	SAT TT=0.003 T1=0	? (TO) TT=1800.43	SAT TT=11.032 T1=11.07	SAT TT=861.152 T1=861.4	SAT TT=0.021 T1=0.02	SAT TT=1.522 T1=1.52	SAT TT=0.166 T1=0.15	SAT TT=0.065 T1=0.05	SAT TT=0.052 T1=0.05	SAT TT=0.01 T1=0.01	SAT TT=0.701 T1=0.47	SAT TT=0.728 T1=0.52	SAT TT=0.003 T1=0	SAT TT=0.003 T1=0
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga11_9_sat_pb.cnf.cr.opb	SAT TT=0.002 T1=0	? (TO) TT=1800.29	? TT=1800.09	SAT TT=1731.02 T1=1731.51	SAT TT=0.019 T1=0.02	SAT TT=1.022 T1=1.02	SAT TT=211.613 T1=211.67	SAT TT=0.814 T1=0.81	SAT TT=0.07 T1=0.07	SAT TT=0.011 T1=0.01	SAT TT=1058.22 T1=1048.48	SAT TT=1058.91 T1=1048.87	SAT TT=0.002 T1=0	SAT TT=0.003 T1=0
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga12_10_sat_pb.cnf.cr.opb	SAT TT=0.003 T1=0	? (TO) TT=1800.29	? TT=1800.08	? (TO) TT=1800.48	SAT TT=0.02 T1=0.02	SAT TT=1.603 T1=1.6	SAT TT=373.796 T1=373.89	SAT TT=0.329 T1=0.32	SAT TT=0.039 T1=0.04	SAT TT=0.02 T1=0.02	SAT TT=26.158 T1=24.59	SAT TT=28.1 T1=24.78	SAT TT=0.004 T1=0	SAT TT=0.003 T1=0
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga12_11_sat_pb.cnf.cr.opb	SAT TT=0.003 T1=0	? (TO) TT=1800.45	? TT=1800.06	? (TO) TT=1800.72	SAT TT=0.023 T1=0.02	SAT TT=1.923 T1=1.92	SAT TT=0.818 T1=0.8	SAT TT=0.117 T1=0.12	SAT TT=0.033 T1=0.03	SAT TT=0.011 T1=0.01	SAT TT=2.189 T1=1.41	SAT TT=2.435 T1=1.48	SAT TT=0.003 T1=0	SAT TT=0.004 T1=0
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga12_12_sat_pb.cnf.cr.opb	SAT TT=0.003 T1=0	? (TO) TT=1800.42	SAT TT=206.888 T1=207.16	? (TO) TT=1800.51	SAT TT=0.025 T1=0.02	SAT TT=2.093 T1=2.09	SAT TT=8.837 T1=8.82	SAT TT=0.086 T1=0.07	SAT TT=0.075 T1=0.07	SAT TT=0.011 T1=0.01	SAT TT=1.041 T1=0.64	SAT TT=1.029 T1=0.66	SAT TT=0.003 T1=0	SAT TT=0.003 T1=0
Bench name	Best results on this instance	absconPseudo 1	bsolo 2006/05	glpPB 0.2	minisat+ 1.14	PBS 4.1L	PB-smodels 1.28	PB-smodels 1.31	Pueblo 1.3	Pueblo 1.4	SAT4JPSEUDO 2006.2	SAT4JPSEUDO 2006.2 Heuristics	wildcat-rnp 0.8.0	wildcat-skc 0.8.0
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga13_11_sat_pb.cnf.cr.opb	SAT TT=0.003 T1=0	? (TO) TT=1800.84	? TT=1800.1	? (TO) TT=1800.59	SAT TT=0.023 T1=0.02	SAT TT=2.068 T1=2.06	SAT TT=95.043 T1=95.05	SAT TT=1.91 T1=1.89	SAT TT=0.05 T1=0.05	SAT TT=0.009 T1=0.01	SAT TT=82.91 T1=81.89	SAT TT=82.255 T1=80.13	SAT TT=0.005 T1=0	SAT TT=0.003 T1=0
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga13_12_sat_pb.cnf.cr.opb	SAT TT=0.003 T1=0	? (TO) TT=1800.23	? TT=1800.07	? (TO) TT=1800.61	SAT TT=0.028 T1=0.03	SAT TT=1.707 T1=1.71	SAT TT=22.581 T1=22.58	SAT TT=7.961 T1=7.95	SAT TT=0.061 T1=0.06	SAT TT=0.028 T1=0.03	SAT TT=0.887 T1=0.55	SAT TT=0.915 T1=0.54	SAT TT=0.005 T1=0	SAT TT=0.003 T1=0
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga13_13_sat_pb.cnf.cr.opb	SAT TT=0.004 T1=0	? (TO) TT=1800.37	? TT=1800.1	? (TO) TT=1800.52	SAT TT=0.071 T1=0.07	SAT TT=1.989 T1=1.99	SAT TT=9.424 T1=9.41	SAT TT=0.081 T1=0.11	SAT TT=0.063 T1=0.06	SAT TT=0.012 T1=0.01	SAT TT=1.032 T1=0.66	SAT TT=1.012 T1=0.65	SAT TT=0.005 T1=0	SAT TT=0.004 T1=0
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga14_12_sat_pb.cnf.cr.opb	SAT TT=0.004 T1=0	? (TO) TT=1800.32	? TT=1800.09	? (TO) TT=1800.65	SAT TT=0.027 T1=0.02	SAT TT=2.305 T1=2.3	SAT TT=736.962 T1=737.23	SAT TT=0.987 T1=0.97	SAT TT=0.437 T1=0.44	SAT TT=0.012 T1=0.01	? (TO) TT=1800.82	? (TO) TT=1801.16	SAT TT=0.004 T1=0	SAT TT=0.004 T1=0
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga14_13_sat_pb.cnf.cr.opb	SAT TT=0.004 T1=0	? (TO) TT=1800.41	? TT=1800.09	? (TO) TT=1800.81	SAT TT=0.042 T1=0.04	SAT TT=1.998 T1=2	SAT TT=31.35 T1=31.34	SAT TT=0.926 T1=0.91	SAT TT=0.11 T1=0.11	SAT TT=0.024 T1=0.02	SAT TT=19.734 T1=18.15	SAT TT=20.247 T1=18.66	SAT TT=0.004 T1=0	SAT TT=0.005 T1=0
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga14_14_sat_pb.cnf.cr.opb	SAT TT=0.005 T1=0	? (TO) TT=1800.2	? TT=1800.16	? (TO) TT=1800.71	SAT TT=0.034 T1=0.03	SAT TT=2.168 T1=2.17	SAT TT=31.164 T1=31.16	SAT TT=0.377 T1=0.36	SAT TT=0.102 T1=0.1	SAT TT=0.017 T1=0.01	SAT TT=0.975 T1=0.62	SAT TT=1.053 T1=0.71	SAT TT=0.006 T1=0	SAT TT=0.005 T1=0
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga15_13_sat_pb.cnf.cr.opb	SAT TT=0.004 T1=0	? (TO) TT=1800.29	? TT=1800.09	? (TO) TT=1800.67	SAT TT=0.03 T1=0.03	SAT TT=1.659 T1=1.65	? (TO) TT=1800.74	SAT TT=69.582 T1=69.59	SAT TT=0.154 T1=0.15	SAT TT=0.015 T1=0.01	? (TO) TT=1800.82	? (TO) TT=1800.58	SAT TT=0.004 T1=0	SAT TT=0.005 T1=0
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga15_14_sat_pb.cnf.cr.opb	SAT TT=0.006 T1=0	? (TO) TT=1800.45	? TT=1800.1	? (TO) TT=1800.59	SAT TT=0.04 T1=0.04	SAT TT=2.798 T1=2.8	? (TO) TT=1800.61	SAT TT=16.217 T1=16.2	SAT TT=0.208 T1=0.21	SAT TT=0.03 T1=0.03	SAT TT=0.998 T1=0.65	SAT TT=1.056 T1=0.69	SAT TT=0.006 T1=0	SAT TT=0.006 T1=0
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga15_15_sat_pb.cnf.cr.opb	SAT TT=0.005 T1=0	? (TO) TT=1800.76	? TT=1800.09	? (TO) TT=1800.66	SAT TT=0.058 T1=0.06	SAT TT=3.717 T1=3.71	SAT TT=1466.35 T1=1466.89	SAT TT=0.222 T1=0.26	SAT TT=0.162 T1=0.16	SAT TT=0.019 T1=0.02	SAT TT=1.023 T1=0.71	SAT TT=1.028 T1=0.7	SAT TT=0.006 T1=0	SAT TT=0.005 T1=0
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga20_18_sat_pb.cnf.cr.opb	SAT TT=0.009 T1=0.01	? (TO) TT=1800.48	? TT=1800.03	? (TO) TT=1800.67	SAT TT=0.186 T1=0.18	SAT TT=5.133 T1=5.13	? (TO) TT=1800.69	? (TO) TT=1800.64	SAT TT=1.57 T1=1.57	SAT TT=0.038 T1=0.04	? (TO) TT=1800.3	? (TO) TT=1800.42	SAT TT=0.01 T1=0.01	SAT TT=0.009 T1=0.01
Bench name	Best results on this instance	absconPseudo 1	bsolo 2006/05	glpPB 0.2	minisat+ 1.14	PBS 4.1L	PB-smodels 1.28	PB-smodels 1.31	Pueblo 1.3	Pueblo 1.4	SAT4JPSEUDO 2006.2	SAT4JPSEUDO 2006.2 Heuristics	wildcat-rnp 0.8.0	wildcat-skc 0.8.0
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga20_19_sat_pb.cnf.cr.opb	SAT TT=0.009 T1=0.01	? (TO) TT=1800.69	? TT=1800.04	? (TO) TT=1800.63	SAT TT=0.268 T1=0.27	SAT TT=7.267 T1=7.27	? (TO) TT=1800.56	SAT TT=129.852 T1=129.87	SAT TT=1.254 T1=1.25	SAT TT=0.044 T1=0.04	? (TO) TT=1800.26	? (TO) TT=1800.73	SAT TT=0.009 T1=0.01	SAT TT=0.012 T1=0.01
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga20_20_sat_pb.cnf.cr.opb	SAT TT=0.01 T1=0.01	? (TO) TT=1800.2	? TT=1800.03	? (TO) TT=1800.65	SAT TT=0.114 T1=0.11	SAT TT=16.318 T1=16.32	? (TO) TT=1800.88	SAT TT=265.804 T1=265.84	SAT TT=4.806 T1=4.8	SAT TT=0.039 T1=0.04	SAT TT=1.911 T1=1.17	SAT TT=1.848 T1=1.14	SAT TT=0.011 T1=0.01	SAT TT=0.01 T1=0.01
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga25_23_sat_pb.cnf.cr.opb	SAT TT=0.016 T1=0.01	? (TO) TT=1800.98	? TT=1800.01	? (TO) TT=1800.64	SAT TT=13.647 T1=13.65	SAT TT=12.783 T1=12.78	? (TO) TT=1800.84	? (TO) TT=1800.82	SAT TT=9.463 T1=9.46	SAT TT=0.06 T1=0.06	? (TO) TT=1800.7	? (TO) TT=1801	SAT TT=0.016 T1=0.01	SAT TT=0.016 T1=0.01
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga25_24_sat_pb.cnf.cr.opb	SAT TT=0.016 T1=0.01	? (TO) TT=1800.67	? TT=1800.01	? (TO) TT=1800.55	SAT TT=0.377 T1=0.38	SAT TT=30.719 T1=30.72	? (TO) TT=1800.67	? (TO) TT=1800.6	SAT TT=8.604 T1=8.6	SAT TT=0.421 T1=0.42	SAT TT=952.319 T1=937.34	SAT TT=927.059 T1=912.33	SAT TT=0.017 T1=0.01	SAT TT=0.016 T1=0.01
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga25_25_sat_pb.cnf.cr.opb	SAT TT=0.016 T1=0.01	? (TO) TT=1800.57	? TT=1800.02	? (TO) TT=1800.72	SAT TT=0.301 T1=0.3	SAT TT=54.665 T1=54.67	? (TO) TT=1800.68	? (TO) TT=1800.83	SAT TT=2.364 T1=2.36	SAT TT=0.092 T1=0.09	SAT TT=2.668 T1=1.76	SAT TT=2.67 T1=1.73	SAT TT=0.017 T1=0.01	SAT TT=0.016 T1=0.01
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga30_28_sat_pb.cnf.cr.opb	SAT TT=0.022 T1=0.02	? (TO) TT=1800.76	? TT=1800.01	? (TO) TT=1800.71	SAT TT=10.224 T1=10.23	SAT TT=767.672 T1=767.86	? (TO) TT=1800.76	? (TO) TT=1800.61	SAT TT=1103.59 T1=1103.91	SAT TT=0.232 T1=0.23	? (TO) TT=1801.05	? (TO) TT=1800.61	SAT TT=0.025 T1=0.02	SAT TT=0.022 T1=0.02
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga30_29_sat_pb.cnf.cr.opb	SAT TT=0.022 T1=0.02	? (TO) TT=1800.64	? TT=1800.01	? (TO) TT=1800.66	SAT TT=0.245 T1=0.24	SAT TT=857.033 T1=857.29	? (TO) TT=1800.8	? (TO) TT=1800.85	SAT TT=275.098 T1=275.17	SAT TT=0.186 T1=0.18	? (TO) TT=1800.44	? (TO) TT=1800.49	SAT TT=0.029 T1=0.02	SAT TT=0.022 T1=0.02
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga30_30_sat_pb.cnf.cr.opb	SAT TT=0.022 T1=0.02	? (TO) TT=1800.77	? TT=1800.01	? (TO) TT=1800.69	SAT TT=1.606 T1=1.6	? (TO) TT=1800.26	? (TO) TT=1800.66	? (TO) TT=1800.6	SAT TT=511.569 T1=511.69	SAT TT=0.231 T1=0.23	SAT TT=3.225 T1=2.39	SAT TT=3.426 T1=2.53	SAT TT=0.028 T1=0.03	SAT TT=0.022 T1=0.02
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga35_33_sat_pb.cnf.cr.opb	SAT TT=0.034 T1=0.03	? (TO) TT=1800.65	? TT=1800.01	? (TO) TT=1800.67	? (TO) TT=1800.66	? (TO) TT=1800.54	? (TO) TT=1800.95	? (TO) TT=1800.88	? (TO) TT=1800.44	SAT TT=0.31 T1=0.31	? (TO) TT=1800.38	? (TO) TT=1800.87	SAT TT=0.039 T1=0.03	SAT TT=0.034 T1=0.03
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga35_34_sat_pb.cnf.cr.opb	SAT TT=0.042 T1=0.04	? (TO) TT=1800.76	? TT=1800.01	? (TO) TT=1800.75	SAT TT=120.807 T1=120.84	? (TO) TT=1800.59	? (TO) TT=1800.8	? (TO) TT=1800.75	? (TO) TT=1800.47	SAT TT=0.381 T1=0.38	SAT TT=1528.96 T1=1498.29	SAT TT=1503.4 T1=1473.62	SAT TT=0.042 T1=0.04	SAT TT=0.042 T1=0.04
Bench name	Best results on this instance	absconPseudo 1	bsolo 2006/05	glpPB 0.2	minisat+ 1.14	PBS 4.1L	PB-smodels 1.28	PB-smodels 1.31	Pueblo 1.3	Pueblo 1.4	SAT4JPSEUDO 2006.2	SAT4JPSEUDO 2006.2 Heuristics	wildcat-rnp 0.8.0	wildcat-skc 0.8.0
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga35_35_sat_pb.cnf.cr.opb	SAT TT=0.039 T1=0.03	? (TO) TT=1800.68	? TT=1800.01	? (TO) TT=1800.5	SAT TT=66.879 T1=66.89	? (TO) TT=1800.83	? (TO) TT=1800.85	? (TO) TT=1800.64	? (TO) TT=1800.59	SAT TT=0.4 T1=0.4	? (TO) TT=1801.05	? (TO) TT=1800.46	SAT TT=0.048 T1=0.04	SAT TT=0.039 T1=0.03
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga40_38_sat_pb.cnf.cr.opb	SAT TT=0.049 T1=0.04	? (TO) TT=1800.8	? TT=1800	? (TO) TT=1800.66	? (TO) TT=1800.52	? (TO) TT=1800.52	? (TO) TT=1800.87	? (TO) TT=1800.91	? (TO) TT=1800.49	SAT TT=2.773 T1=2.77	? (TO) TT=1800.4	? (TO) TT=1800.22	SAT TT=0.056 T1=0.05	SAT TT=0.049 T1=0.04
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga40_39_sat_pb.cnf.cr.opb	SAT TT=0.052 T1=0.05	? (TO) TT=1800.84	? TT=1800.01	? (TO) TT=1800.66	SAT TT=240.234 T1=240.31	? (TO) TT=1800.72	? (TO) TT=1800.98	? (TO) TT=1800.84	? (TO) TT=1800.67	SAT TT=0.842 T1=0.84	? (TO) TT=1800.57	? (TO) TT=1800.68	SAT TT=0.065 T1=0.06	SAT TT=0.052 T1=0.05
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga40_40_sat_pb.cnf.cr.opb	SAT TT=0.059 T1=0.05	? (TO) TT=1800.76	? TT=1800.01	? (TO) TT=1800.64	SAT TT=575.925 T1=576.07	? (TO) TT=1800.71	? (TO) TT=1800.9	? (TO) TT=1800.89	? (TO) TT=1800.56	SAT TT=0.751 T1=0.75	SAT TT=6.99 T1=5.92	SAT TT=7.31 T1=6.34	SAT TT=0.065 T1=0.06	SAT TT=0.059 T1=0.05
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga45_43_sat_pb.cnf.cr.opb	SAT TT=0.075 T1=0.07	? (TO) TT=1800.35	? TT=1800	? (TO) TT=1800.66	? (TO) TT=1800.78	? (TO) TT=1800.72	? (TO) TT=1800.99	? (TO) TT=1800.9	? (TO) TT=1800.53	SAT TT=2.272 T1=2.27	? (TO) TT=1800.7	? (TO) TT=1800.26	SAT TT=0.088 T1=0.08	SAT TT=0.075 T1=0.07
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga45_44_sat_pb.cnf.cr.opb	SAT TT=0.071 T1=0.06	? (TO) TT=1800.91	? TT=1800.01	? (TO) TT=1800.71	? (TO) TT=1800.58	? (TO) TT=1800.6	? (TO) TT=1800.1	? (TO) TT=1801.01	? (TO) TT=1800.51	SAT TT=1.49 T1=1.49	? (TO) TT=1800.71	? (TO) TT=1800.85	SAT TT=0.092 T1=0.09	SAT TT=0.071 T1=0.06
normalized-PB06/SATUNSAT-SMALLINT/ submitted-PB05/aloul/FPGA_SAT05/ normalized-fpga45_45_sat_pb.cnf.cr.opb	SAT TT=0.077 T1=0.07	? (TO) TT=1800.3	? TT=1800.01	? (TO) TT=1800.59	SAT TT=1.156 T1=1.15	? (TO) TT=1800.65	? (TO) TT=1800	? (TO) TT=1800.89	? (TO) TT=1800.56	SAT TT=1.16 T1=1.15	? (TO) TT=1800.39	? (TO) TT=1801.17	SAT TT=0.094 T1=0.09	SAT TT=0.077 T1=0.07
normalized-PB06/SATUNSAT-SMALLINT/ web/ uclid_pb_benchmarks/normalized-22s.smv.opb	UNSAT TT=0.326	? (TO) TT=1800.95	UNSAT TT=10.123	? (TO) TT=1800.79	UNSAT TT=1.352	UNSAT TT=0.326	UNSAT TT=2.531	UNSAT TT=8.518	UNSAT TT=0.461	UNSAT TT=0.443	UNSAT TT=5.091	UNSAT TT=4.944	? (TO) TT=1800.59	? TT=1064.05
normalized-PB06/SATUNSAT-SMALLINT/ web/ uclid_pb_benchmarks/normalized-25s.smv.opb	UNSAT TT=0.194	? (TO) TT=1800.6	UNSAT TT=4.774	? (TO) TT=1800.67	UNSAT TT=1.025	UNSAT TT=0.194	UNSAT TT=3.199	UNSAT TT=27.072	UNSAT TT=0.331	UNSAT TT=0.329	UNSAT TT=4.041	UNSAT TT=3.617	? (TO) TT=1800.64	? TT=699.894
normalized-PB06/SATUNSAT-SMALLINT/ web/ uclid_pb_benchmarks/normalized-37s.smv.opb	UNSAT TT=0.054	? (TO) TT=1800.62	UNSAT TT=1.328	? (TO) TT=1800.7	UNSAT TT=0.141	UNSAT TT=0.056	UNSAT TT=1.362	UNSAT TT=1.876	UNSAT TT=0.054	UNSAT TT=0.056	UNSAT TT=1.924	UNSAT TT=1.899	? (TO) TT=1800.68	? TT=761.177
Bench name	Best results on this instance	absconPseudo 1	bsolo 2006/05	glpPB 0.2	minisat+ 1.14	PBS 4.1L	PB-smodels 1.28	PB-smodels 1.31	Pueblo 1.3	Pueblo 1.4	SAT4JPSEUDO 2006.2	SAT4JPSEUDO 2006.2 Heuristics	wildcat-rnp 0.8.0	wildcat-skc 0.8.0
normalized-PB06/SATUNSAT-SMALLINT/ web/ uclid_pb_benchmarks/normalized-43s.smv.opb	UNSAT TT=0.366	? (TO) TT=1801.04	UNSAT TT=5.956	? (TO) TT=1800.63	UNSAT TT=0.933	UNSAT TT=0.366	? (TO) TT=1800.14	? (TO) TT=1800.98	UNSAT TT=0.556	UNSAT TT=0.394	UNSAT TT=192.439	UNSAT TT=193.514	? (TO) TT=1800.66	? TT=749.202
normalized-PB06/SATUNSAT-SMALLINT/ web/ uclid_pb_benchmarks/normalized-44s.smv.opb	UNSAT TT=0.026	? (TO) TT=1800.5	UNSAT TT=0.575	? (TO) TT=1800.75	UNSAT TT=0.087	UNSAT TT=0.029	? (TO) TT=1800.82	? (TO) TT=1800.78	UNSAT TT=0.042	UNSAT TT=0.026	UNSAT TT=2.084	UNSAT TT=2.097	? (TO) TT=1800.71	? TT=757.711
normalized-PB06/SATUNSAT-SMALLINT/ web/ uclid_pb_benchmarks/normalized-46s.smv.opb	UNSAT TT=0.084	? (TO) TT=1800.58	UNSAT TT=2.089	? (TO) TT=1800.52	UNSAT TT=0.302	UNSAT TT=0.084	? (TO) TT=1800.86	? (TO) TT=1800.85	UNSAT TT=0.129	UNSAT TT=0.119	UNSAT TT=8.308	UNSAT TT=7.947	? (TO) TT=1800.62	? TT=742.444
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-floppy1-2.ucl.opb	UNSAT TT=0.013	UNSAT TT=1.42	UNSAT TT=0.552	UNSAT TT=0.013	UNSAT TT=0.023	UNSAT TT=0.042	UNSAT TT=0.067	UNSAT TT=0.095	UNSAT TT=0.03	UNSAT TT=0.02	UNSAT TT=1.035	UNSAT TT=1.367	? (TO) TT=1800.65	? TT=774.132
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-floppy1-3.ucl.opb	UNSAT TT=0.005	UNSAT TT=0.994	UNSAT TT=0.019	UNSAT TT=0.02	UNSAT TT=0.021	UNSAT TT=0.005	UNSAT TT=0.074	UNSAT TT=0.078	UNSAT TT=0.008	UNSAT TT=0.01	UNSAT TT=0.539	UNSAT TT=0.558	? (TO) TT=1800.72	? TT=685.723
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-floppy1-4.ucl.opb	UNSAT TT=0.005	UNSAT TT=1.036	UNSAT TT=0.017	UNSAT TT=0.013	UNSAT TT=0.016	UNSAT TT=0.005	UNSAT TT=0.064	UNSAT TT=0.067	UNSAT TT=0.008	UNSAT TT=0.007	UNSAT TT=0.498	UNSAT TT=0.518	? (TO) TT=1800.7	? TT=753.537
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-floppy1-6.ucl.opb	UNSAT TT=0.053	UNSAT TT=2.046	UNSAT TT=0.625	UNSAT TT=0.102	UNSAT TT=0.086	UNSAT TT=0.112	UNSAT TT=0.217	UNSAT TT=76.668	UNSAT TT=0.074	UNSAT TT=0.053	UNSAT TT=1.08	UNSAT TT=1.05	? (TO) TT=1800.61	? TT=815.662
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-floppy1-7.ucl.opb	UNSAT TT=0.098	UNSAT TT=2.056	Signal TT=634.528	UNSAT TT=1.227	UNSAT TT=0.548	UNSAT TT=0.098	UNSAT TT=0.744	UNSAT TT=1.924	UNSAT TT=0.123	UNSAT TT=0.124	UNSAT TT=1.747	UNSAT TT=1.847	? (TO) TT=1800.64	? (TO) TT=1800.55
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-floppy1-8.ucl.opb	UNSAT TT=0.119	UNSAT TT=2.985	UNSAT TT=0.171	UNSAT TT=1.973	UNSAT TT=0.411	UNSAT TT=0.119	UNSAT TT=0.973	UNSAT TT=0.86	UNSAT TT=0.157	UNSAT TT=0.16	UNSAT TT=1.976	UNSAT TT=1.982	? (TO) TT=1800.69	? TT=1602.83
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-tlan2.ucl.opb	UNSAT TT=0.115	? (TO) TT=1800.25	UNSAT TT=2.972	UNSAT TT=0.945	UNSAT TT=0.369	UNSAT TT=0.115	UNSAT TT=1.015	? (TO) TT=1800.78	UNSAT TT=0.169	UNSAT TT=0.138	UNSAT TT=1.496	UNSAT TT=1.479	? (TO) TT=1800.48	? TT=692.514
Bench name	Best results on this instance	absconPseudo 1	bsolo 2006/05	glpPB 0.2	minisat+ 1.14	PBS 4.1L	PB-smodels 1.28	PB-smodels 1.31	Pueblo 1.3	Pueblo 1.4	SAT4JPSEUDO 2006.2	SAT4JPSEUDO 2006.2 Heuristics	wildcat-rnp 0.8.0	wildcat-skc 0.8.0
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-blast-tlan3.ucl.opb	UNSAT TT=0.957	UNSAT TT=8.064	UNSAT TT=1.749	UNSAT TT=195.102	UNSAT TT=3.018	UNSAT TT=0.957	UNSAT TT=7.985	UNSAT TT=7.382	UNSAT TT=2.82	UNSAT TT=2.793	UNSAT TT=22.247	UNSAT TT=21.32	? (TO) TT=1800.7	? TT=803.422
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-cache-ibm-q-full.all.ucl.opb	UNSAT TT=1167.75	? TT=18.487	? TT=1800.04	? (TO) TT=1800.75	UNSAT TT=1167.75	? (TO) TT=1800.65	? (TO) TT=1800.94	? (TO) TT=1800.05	? (TO) TT=1800.57	? (TO) TT=1800.67	? (TO) TT=1800.29	? (TO) TT=1800.79	? (TO) TT=1800.5	? (TO) TT=1800.72
normalized-PB06/SATUNSAT-SMALLINT/web/ uclid_pb_benchmarks/ normalized-cache-ibm-q-unbounded.Ic22arity.ucl.opb	UNSAT TT=20.558	? TT=19.69	UNSAT TT=1148.6	? (TO) TT=1800.54	UNSAT TT=20.558	UNSAT TT=69.938	? (TO) TT=1800.46	? (TO) TT=1800.95	UNSAT TT=119.989	UNSAT TT=85.834	? (TO) TT=1800.39	? (TO) TT=1800.35	? (TO) TT=1800.58	? (TO) TT=1800.74
normalized-PB06/SATUNSAT-SMALLINT/web/ uclid_pb_benchmarks/ normalized-cache-ibm-q-unbounded.Icl2arity.ucl.opb	UNSAT TT=3.504	? (TO) TT=1800.8	UNSAT TT=210.383	? (TO) TT=1800.77	UNSAT TT=3.836	UNSAT TT=3.504	? (TO) TT=1800.38	? (TO) TT=1800.93	UNSAT TT=6.63	UNSAT TT=6.515	? (TO) TT=1800.79	? (TO) TT=1800.94	? (TO) TT=1800.72	? TT=788.431
normalized-PB06/SATUNSAT-SMALLINT/web/ uclid_pb_benchmarks/ normalized-cache-ibm-q-unbounded.Ih1arity.ucl.opb	UNSAT TT=796.385	? TT=33.138	? TT=1800.31	? (TO) TT=1800.79	UNSAT TT=796.385	? (TO) TT=1800.8	? (TO) TT=1800.74	? (TO) TT=1800.04	? (TO) TT=1800.72	? (TO) TT=1800.64	? (TO) TT=1800.56	? (TO) TT=1800.15	? (TO) TT=1800.76	? (TO) TT=1800.58
normalized-PB06/SATUNSAT-SMALLINT/web/ uclid_pb_benchmarks/ normalized-cache-ibm-q-unbounded.Ih2arity.ucl.opb	SAT TT=3.18 T1=3.13	? TT=19.264	SAT TT=26.92 T1=26.84	? (TO) TT=1800.75	SAT TT=10.392 T1=10.45	SAT TT=3.18 T1=3.13	? (TO) TT=1800.37	? (TO) TT=1800.97	SAT TT=46.223 T1=46.21	SAT TT=42.788 T1=42.77	SAT TT=469.686 T1=466.03	SAT TT=472.85 T1=469.28	? (TO) TT=1800.65	? (TO) TT=1800.51
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-cache.inv10.ucl.opb	UNSAT TT=0.275	UNSAT TT=117.584	UNSAT TT=0.909	? (TO) TT=1800.58	UNSAT TT=0.907	UNSAT TT=0.275	UNSAT TT=3.109	UNSAT TT=41.211	UNSAT TT=0.727	UNSAT TT=0.739	UNSAT TT=7.053	UNSAT TT=6.831	? (TO) TT=1800.54	? TT=1164.46
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-cache.inv12.ucl.opb	UNSAT TT=9.068	? (TO) TT=1800.79	UNSAT TT=594.786	? (TO) TT=1800.69	UNSAT TT=11.263	UNSAT TT=9.068	? (TO) TT=1800.34	? (TO) TT=1800.79	UNSAT TT=28.387	UNSAT TT=35.638	? (TO) TT=1800.14	? (TO) TT=1800.08	? (TO) TT=1800.42	? TT=1365.38
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-cache.inv14.ucl.opb		? (TO) TT=1800.47	? TT=1800.31	? (TO) TT=1800.72	? (TO) TT=1800.57	? (TO) TT=1800.68	? (TO) TT=1800.32	? (TO) TT=1800.6	? (TO) TT=1800.74	? (TO) TT=1800.7	? (TO) TT=1800.28	? (TO) TT=1800.57	? (TO) TT=1800.7	? TT=1799.23
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-cache.inv8.ucl.opb	UNSAT TT=0.065	UNSAT TT=2.061	UNSAT TT=0.129	UNSAT TT=1.313	UNSAT TT=0.223	UNSAT TT=0.065	UNSAT TT=0.733	UNSAT TT=0.702	UNSAT TT=0.107	UNSAT TT=0.107	UNSAT TT=1.503	UNSAT TT=1.676	? (TO) TT=1800.7	? TT=928.835
Bench name	Best results on this instance	absconPseudo 1	bsolo 2006/05	glpPB 0.2	minisat+ 1.14	PBS 4.1L	PB-smodels 1.28	PB-smodels 1.31	Pueblo 1.3	Pueblo 1.4	SAT4JPSEUDO 2006.2	SAT4JPSEUDO 2006.2 Heuristics	wildcat-rnp 0.8.0	wildcat-skc 0.8.0
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-dlx1c.rwmem1.ucl.opb	UNSAT TT=0.61	? (TO) TT=1800.27	UNSAT TT=16.623	? (TO) TT=1800.72	UNSAT TT=1.171	UNSAT TT=0.61	? (TO) TT=1800.54	? (TO) TT=1800.21	UNSAT TT=0.892	UNSAT TT=0.992	UNSAT TT=60.095	UNSAT TT=64.477	? (TO) TT=1800.45	? TT=1237.71
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-dlx1c.rwmem.ucl.opb	UNSAT TT=0.857	? (TO) TT=1800.56	UNSAT TT=30.035	? (TO) TT=1800.68	UNSAT TT=1.329	UNSAT TT=0.857	? (TO) TT=1800.54	? (TO) TT=1800.42	UNSAT TT=1.059	UNSAT TT=1.117	UNSAT TT=112.489	UNSAT TT=106.551	? (TO) TT=1800.65	? TT=1303.51
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-dlx1c.ucl.opb	UNSAT TT=0.363	? (TO) TT=1801.01	UNSAT TT=11.298	? (TO) TT=1800.7	UNSAT TT=0.973	UNSAT TT=0.363	? (TO) TT=1800.28	? (TO) TT=1800.19	UNSAT TT=0.531	UNSAT TT=0.557	UNSAT TT=44.18	UNSAT TT=41.963	? (TO) TT=1800.48	? TT=1173.67
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-elf.rf10.ucl.opb	UNSAT TT=9.756	? (TO) TT=1800.27	UNSAT TT=176.457	? (TO) TT=1800.68	UNSAT TT=9.756	UNSAT TT=10.39	? (TO) TT=1800.69	? (TO) TT=1800.63	UNSAT TT=52.553	UNSAT TT=57.259	? (TO) TT=1800.98	? (TO) TT=1800.16	? (TO) TT=1800.58	? (TO) TT=1800.54
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-elf.rf6.ucl.opb	UNSAT TT=0.002	UNSAT TT=0.993	UNSAT TT=0.009	UNSAT TT=0.018	UNSAT TT=0.009	UNSAT TT=0.002	UNSAT TT=0.047	UNSAT TT=0.044	UNSAT TT=0.004	UNSAT TT=0.005	UNSAT TT=0.358	UNSAT TT=0.356	? TT=1422.07	? TT=793.033
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-elf.rf7.ucl.opb	UNSAT TT=0.034	? (TO) TT=1800.85	UNSAT TT=0.162	? (TO) TT=1800.81	UNSAT TT=0.15	UNSAT TT=0.034	UNSAT TT=0.439	UNSAT TT=0.989	UNSAT TT=0.055	UNSAT TT=0.055	UNSAT TT=1.685	UNSAT TT=1.634	? (TO) TT=1800.73	? TT=886.207
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-elf.rf8.ucl.opb	UNSAT TT=0.211	? (TO) TT=1800.16	UNSAT TT=3.716	? (TO) TT=1800.52	UNSAT TT=0.741	UNSAT TT=0.211	UNSAT TT=4.339	UNSAT TT=33.257	UNSAT TT=0.438	UNSAT TT=0.459	UNSAT TT=9.25	UNSAT TT=8.508	? (TO) TT=1800.56	? TT=1156.46
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-elf.rf9.ucl.opb	UNSAT TT=1.162	? (TO) TT=1800.75	UNSAT TT=29.85	? (TO) TT=1800.58	UNSAT TT=2.752	UNSAT TT=1.162	UNSAT TT=147.745	UNSAT TT=1571.42	UNSAT TT=3.13	UNSAT TT=3.327	UNSAT TT=172.869	UNSAT TT=234.292	? (TO) TT=1800.65	? TT=1575.02
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.burch_dill.2.accl.ucl.opb	UNSAT TT=1.266	? (TO) TT=1800.07	UNSAT TT=29.644	? (TO) TT=1800.72	UNSAT TT=1.363	UNSAT TT=1.266	? (TO) TT=1801.01	? (TO) TT=1800.92	UNSAT TT=1.419	UNSAT TT=1.431	? (TO) TT=1800.17	? (TO) TT=1800.26	? (TO) TT=1800.7	? TT=724.376
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.burch_dill.3.accl.ucl.opb	UNSAT TT=3.416	? (TO) TT=1800.28	UNSAT TT=169.004	? (TO) TT=1800.73	UNSAT TT=3.416	UNSAT TT=4.608	? (TO) TT=1800.26	? (TO) TT=1800.14	UNSAT TT=4.45	UNSAT TT=9.3	? (TO) TT=1800.22	? (TO) TT=1800.13	? (TO) TT=1800.71	? TT=1156.14
Bench name	Best results on this instance	absconPseudo 1	bsolo 2006/05	glpPB 0.2	minisat+ 1.14	PBS 4.1L	PB-smodels 1.28	PB-smodels 1.31	Pueblo 1.3	Pueblo 1.4	SAT4JPSEUDO 2006.2	SAT4JPSEUDO 2006.2 Heuristics	wildcat-rnp 0.8.0	wildcat-skc 0.8.0
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.burch_dill.4.accl.ucl.opb	UNSAT TT=0.248	? (TO) TT=1800.18	UNSAT TT=14.282	? (TO) TT=1800.76	UNSAT TT=0.572	UNSAT TT=0.283	? (TO) TT=1801.04	? (TO) TT=1800.05	UNSAT TT=0.248	UNSAT TT=0.287	UNSAT TT=8.816	UNSAT TT=7.189	? (TO) TT=1800.7	? (TO) TT=1800.71
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.burch_dill.6.accl.ucl.opb		? (TO) TT=1800.48	? TT=1800.04	? (TO) TT=1800.54	? (TO) TT=1800.67	? (TO) TT=1800.64	? (TO) TT=1800.25	? (TO) TT=1800.88	? (TO) TT=1800.48	? (TO) TT=1800.52	? (TO) TT=1800.39	? (TO) TT=1800.98	? (TO) TT=1800.74	? (TO) TT=1800.65
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.burch_dill.8.accl.ucl.opb	UNSAT TT=304.332	? (TO) TT=1800.56	? TT=1800.04	? (TO) TT=1800.59	? (TO) TT=1800.55	UNSAT TT=304.332	? (TO) TT=1800.17	? (TO) TT=1800.64	UNSAT TT=1538.47	? (TO) TT=1800.73	? (TO) TT=1801.02	? (TO) TT=1800.29	? (TO) TT=1800.69	? (TO) TT=1800.57
normalized-PB06/SATUNSAT-SMALLINT/web/ uclid_pb_benchmarks/ normalized-ooo.ex.br.mem.RobRegValid_bar.ucl.opb	UNSAT TT=90.698	? TT=15.942	UNSAT TT=1380.64	? (TO) TT=1800.66	UNSAT TT=90.698	UNSAT TT=199.124	? (TO) TT=1800.29	? (TO) TT=1800.54	UNSAT TT=179.282	UNSAT TT=91.124	? (TO) TT=1800.38	? (TO) TT=1800.18	? (TO) TT=1800.57	? (TO) TT=1800.7
normalized-PB06/SATUNSAT-SMALLINT/web/uclid_pb_benchmarks/ normalized-ooo.ex.br.mem.Src1Valid_Src1ValidBar.ucl.opb	UNSAT TT=1503.32	? TT=21.62	? TT=1800.07	? (TO) TT=1800.76	UNSAT TT=1503.32	? (TO) TT=1800.58	? (TO) TT=1800.3	? (TO) TT=1800.93	? (TO) TT=1800.75	? (TO) TT=1800.76	? (TO) TT=1800.46	? (TO) TT=1800.56	? (TO) TT=1800.73	? (TO) TT=1800.77
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.ex.mem.LdValue.ucl.opb	UNSAT TT=1360.12	? TT=19.916	? TT=1800.12	? (TO) TT=1800.75	UNSAT TT=1360.12	UNSAT TT=1732.67	? (TO) TT=1800.73	? (TO) TT=1800.36	? (TO) TT=1800.74	? (TO) TT=1800.6	? (TO) TT=1800.61	? (TO) TT=1800.23	? (TO) TT=1800.66	? (TO) TT=1800.46
normalized-PB06/SATUNSAT-SMALLINT/web/uclid_pb_benchmarks/ normalized-ooo.ex.mem.LsqHdStrong.ucl.opb	UNSAT TT=1574.92	? TT=28.825	? TT=1800.1	? (TO) TT=1800.74	UNSAT TT=1574.92	? (TO) TT=1800.61	? (TO) TT=1800.44	? (TO) TT=1800.38	? (TO) TT=1800.72	? (TO) TT=1800.59	? (TO) TT=1800.36	? (TO) TT=1800.3	? (TO) TT=1800.74	? (TO) TT=1800.72
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.rf10.ucl.opb	UNSAT TT=12.17	? (TO) TT=1800.75	? TT=1800.06	? (TO) TT=1800.68	UNSAT TT=12.17	UNSAT TT=28.243	? (TO) TT=1800.64	? (TO) TT=1800.26	UNSAT TT=61.599	UNSAT TT=70.444	? (TO) TT=1800.35	? (TO) TT=1800.35	? (TO) TT=1800.71	? TT=1314.05
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.rf6.ucl.opb	UNSAT TT=0.057	? (TO) TT=1800.77	UNSAT TT=1.336	? (TO) TT=1800.52	UNSAT TT=0.209	UNSAT TT=0.057	? (TO) TT=1800.7	? (TO) TT=1800.8	UNSAT TT=0.105	UNSAT TT=0.094	UNSAT TT=3.155	UNSAT TT=3.126	? (TO) TT=1800.64	? TT=919.691
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.rf7.ucl.opb	UNSAT TT=0.2	? (TO) TT=1800.58	UNSAT TT=9.181	? (TO) TT=1800.68	UNSAT TT=0.466	UNSAT TT=0.2	? (TO) TT=1800.04	? (TO) TT=1800.11	UNSAT TT=0.35	UNSAT TT=0.394	UNSAT TT=21.303	UNSAT TT=21.44	? (TO) TT=1800.44	? TT=958.703
Bench name	Best results on this instance	absconPseudo 1	bsolo 2006/05	glpPB 0.2	minisat+ 1.14	PBS 4.1L	PB-smodels 1.28	PB-smodels 1.31	Pueblo 1.3	Pueblo 1.4	SAT4JPSEUDO 2006.2	SAT4JPSEUDO 2006.2 Heuristics	wildcat-rnp 0.8.0	wildcat-skc 0.8.0
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.rf8.ucl.opb	UNSAT TT=0.723	? (TO) TT=1800.06	UNSAT TT=31.279	? (TO) TT=1800.73	UNSAT TT=1.031	UNSAT TT=0.723	? (TO) TT=1800.51	? (TO) TT=1800.37	UNSAT TT=1.607	UNSAT TT=1.398	UNSAT TT=129.211	UNSAT TT=130.209	? (TO) TT=1800.66	? TT=1019.86
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.rf9.ucl.opb	UNSAT TT=2.784	? (TO) TT=1800.24	UNSAT TT=154.342	? (TO) TT=1800.71	UNSAT TT=2.784	UNSAT TT=3.337	? (TO) TT=1800.94	? (TO) TT=1800.67	UNSAT TT=6.609	UNSAT TT=9.147	UNSAT TT=1221.96	UNSAT TT=1312.4	? (TO) TT=1800.51	? TT=1146.43
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.tag10.ucl.opb	UNSAT TT=2.062	? (TO) TT=1800.12	UNSAT TT=209.591	? (TO) TT=1800.59	UNSAT TT=3.764	UNSAT TT=2.062	? (TO) TT=1800.56	? (TO) TT=1800.4	UNSAT TT=15.644	UNSAT TT=22.861	? (TO) TT=1800.26	? (TO) TT=1800.34	? (TO) TT=1800.69	? TT=1177.04
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.tag12.ucl.opb	UNSAT TT=12.177	? (TO) TT=1800.77	UNSAT TT=1272.6	? (TO) TT=1800.67	UNSAT TT=21.874	UNSAT TT=12.177	? (TO) TT=1800.74	? (TO) TT=1800.37	UNSAT TT=58.679	UNSAT TT=51.92	? (TO) TT=1800.44	? (TO) TT=1800.35	? (TO) TT=1800.58	? TT=1434.95
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.tag14.ucl.opb	UNSAT TT=46.487	? (TO) TT=1801.07	? TT=1800.04	? (TO) TT=1800.59	UNSAT TT=47.889	UNSAT TT=46.487	? (TO) TT=1800.04	? (TO) TT=1800.09	UNSAT TT=157.787	UNSAT TT=759.793	? (TO) TT=1800.32	? (TO) TT=1801.06	? (TO) TT=1800.51	? TT=1697.91
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.tag8.ucl.opb	UNSAT TT=0.498	? (TO) TT=1801.01	UNSAT TT=68.444	? (TO) TT=1800.57	UNSAT TT=1.021	UNSAT TT=0.498	? (TO) TT=1800.11	? (TO) TT=1800.05	UNSAT TT=1.617	UNSAT TT=3.009	UNSAT TT=227.866	UNSAT TT=221.152	? (TO) TT=1800.67	? TT=945.267
normalized-PB06/SATUNSAT-SMALLINT/ web/uclid_pb_benchmarks/ normalized-ooo.unbounded.all.ucl.opb	UNSAT TT=431.497	? TT=18.881	? TT=1800.05	? (TO) TT=1800.61	UNSAT TT=431.497	UNSAT TT=1270.99	? (TO) TT=1800.43	? (TO) TT=1800.05	UNSAT TT=991.858	UNSAT TT=779.327	? (TO) TT=1800.47	? (TO) TT=1800.72	? (TO) TT=1800.7	? (TO) TT=1800.56

Some statistics...

	absconPseudo 1	bsolo 2006/05	glpPB 0.2	minisat+ 1.14	PBS 4.1L	PB-smodels 1.28	PB-smodels 1.31	Pueblo 1.3	Pueblo 1.4	SAT4JPSEUDO 2006.2	SAT4JPSEUDO 2006.2 Heuristics	wildcat-rnp 0.8.0	wildcat-skc 0.8.0
Number of times the solver is able to give the best known answer	10	47	36	82	91	33	36	71	99	73	73	36	36
Number of times the solver is able to give the best known answer from an incomplete solver point of view (i.e. without considering optimality proof)	10	47	36	82	91	33	36	71	99	73	73	36	36
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)	0	0	1	12	52	0	0	2	2	0	0	13	27
Number of times the solver is the best solver from an incomplete solver point of view (i.e. best known answer and best T1 time)	0	0	0	0	1	0	0	0	0	0	0	28	36