PB'16 competition: satisfaction and optimization track: results by benchmark

Results by benchmark for category no optimisation, small integers, non linear constraints (DEC-SMALLINT-NLC)

This page displays the results of the different solvers for each benchmark for category no optimisation, small integers, non linear constraints (DEC-SMALLINT-NLC)

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell example	Meaning
Answer	Solver result
f=...	value of the objective function for the model reported by the solver
TT=...	Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out

Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

Abbreviation	Meaning
f=...	Value of the objective function
TO	Time Out
MO	Mem. Out (out of memory)

Meaning of the different colors:

Color	Meaning
text	the solver cannot handle this instance
text	the solver gave no answer
text	the solver could give an answer (SAT)
text	the solver gave a definitive answer (OPTIMUM FOUND or UNSAT)
text	the solver performed better than the other ones on that instance (complete solvers point of view)
text	the solver performed better than the other ones on that instance (incomplete solvers point of view)
text	the solver was ended by a signal or other problem
text	the solver gave an incomplete answer
text	the solver gave a wrong answer

For better readability, you may choose to hide some solvers:
minisatp 2012-10-02 git-d91742b (complete)
Sat4j PB 2.3.6 Res+CP PB16 (complete)
Sat4j PB 2.3.6 Resolution PB16 (complete)
toysat 2016-05-02 (complete)

Bench name	Best results on this instance	minisatp 2012-10-02 git-d91742b (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_100_10_1_15.opb	SAT TT=0.062 T1=0.06	SAT TT=0.123 T1=0.12	SAT TT=4.163 T1=1.32	? (TO) TT=1800.29	SAT TT=0.062 T1=0.06
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_100_10_1_40.opb		? (TO) TT=1800.02	? (TO) TT=1800.15	? (TO) TT=1800.54	? (TO) TT=1800.01
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_100_10_2_15.opb	SAT TT=0.062 T1=0.05	SAT TT=0.136 T1=0.13	SAT TT=3.331 T1=1.33	? (TO) TT=1800.04	SAT TT=0.062 T1=0.05
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_100_10_2_40.opb		? (TO) TT=1800.02	? (TO) TT=1800.05	? (TO) TT=1800.02	? (TO) TT=1800.07
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_100_10_3_15.opb	SAT TT=0.06 T1=0.05	SAT TT=0.122 T1=0.12	SAT TT=3.395 T1=1.34	? (TO) TT=1800.07	SAT TT=0.06 T1=0.05
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_100_10_3_40.opb		? (TO) TT=1800.02	? (TO) TT=1800.74	? (TO) TT=1800.03	? (TO) TT=1800.05
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_100_10_4_15.opb	SAT TT=0.062 T1=0.05	SAT TT=0.13 T1=0.13	SAT TT=3.272 T1=1.32	SAT TT=611.51 T1=608.63	SAT TT=0.062 T1=0.05
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_100_10_4_40.opb		? (TO) TT=1800.02	? (TO) TT=1800.11	? (TO) TT=1800.87	? (TO) TT=1800.03
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_100_10_5_15.opb	SAT TT=0.062 T1=0.05	SAT TT=0.122 T1=0.12	SAT TT=3.064 T1=1.32	? (TO) TT=1800.56	SAT TT=0.062 T1=0.05
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_100_10_5_40.opb		? (TO) TT=1800.02	? (TO) TT=1800.02	? (TO) TT=1800.04	? (TO) TT=1800.02
Bench name	Best results	minisatp 2012-10-02 git-d91742b (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_100_25_1_15.opb	SAT TT=1224.98 T1=1225.19	SAT TT=1224.98 T1=1225.19	? (TO) TT=1800.1	? (TO) TT=1800.07	? (TO) TT=1800.05
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_100_25_1_40.opb		? (TO) TT=1800.02	? (TO) TT=1800.07	? (TO) TT=1800.61	? (TO) TT=1800.09
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_100_25_2_15.opb	SAT TT=932.568 T1=932.71	SAT TT=932.568 T1=932.71	? (TO) TT=1800.15	? (TO) TT=1800	? (TO) TT=1800.03
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_100_25_2_40.opb		? (TO) TT=1800.02	? (TO) TT=1800.64	? (TO) TT=1800.01	? (TO) TT=1800.09
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_100_25_3_15.opb		? (TO) TT=1800.02	? (TO) TT=1800.01	? (TO) TT=1800.72	? (TO) TT=1800.07
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_100_25_3_40.opb		? (TO) TT=1800.01	? (TO) TT=1800.12	? (TO) TT=1800.79	? (TO) TT=1800.06
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_100_25_4_15.opb		? (TO) TT=1800.02	? (TO) TT=1800.04	? (TO) TT=1800.72	? (TO) TT=1800.07
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_100_25_4_40.opb		? (TO) TT=1800.02	? (TO) TT=1800.02	? (TO) TT=1800.82	? (TO) TT=1800.04
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_100_25_5_15.opb		? (TO) TT=1800.02	? (TO) TT=1800.73	? (TO) TT=1800.62	? (TO) TT=1800.09
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_100_25_5_40.opb		? (TO) TT=1800.02	? (TO) TT=1800.83	? (TO) TT=1800.74	? (TO) TT=1800.1
Bench name	Best results	minisatp 2012-10-02 git-d91742b (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/ dbsg/normalized-dbsg_10_4_1_5.opb	UNSAT TT=0.008	UNSAT TT=0.008	UNSAT TT=0.604	UNSAT TT=0.304	UNSAT TT=0.025
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/ dbsg/normalized-dbsg_10_4_1_8.opb	UNSAT TT=0.007	UNSAT TT=0.007	UNSAT TT=0.223	UNSAT TT=0.175	UNSAT TT=0.02
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/ dbsg/normalized-dbsg_10_4_2_5.opb	UNSAT TT=0.008	UNSAT TT=0.008	UNSAT TT=0.533	UNSAT TT=0.254	UNSAT TT=0.027
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/ dbsg/normalized-dbsg_10_4_2_8.opb	UNSAT TT=0.007	UNSAT TT=0.007	UNSAT TT=0.214	UNSAT TT=0.175	UNSAT TT=0.016
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/ dbsg/normalized-dbsg_10_4_3_5.opb	UNSAT TT=0.009	UNSAT TT=0.009	UNSAT TT=0.595	UNSAT TT=0.303	UNSAT TT=0.024
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/ dbsg/normalized-dbsg_10_4_3_8.opb	UNSAT TT=0.007	UNSAT TT=0.007	UNSAT TT=0.184	UNSAT TT=0.174	UNSAT TT=0.019
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/ dbsg/normalized-dbsg_10_4_4_5.opb	UNSAT TT=0.007	UNSAT TT=0.007	UNSAT TT=0.613	UNSAT TT=0.312	UNSAT TT=0.027
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/ dbsg/normalized-dbsg_10_4_4_8.opb	UNSAT TT=0.007	UNSAT TT=0.007	UNSAT TT=0.174	UNSAT TT=0.172	UNSAT TT=0.016
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/ dbsg/normalized-dbsg_10_4_5_5.opb	UNSAT TT=0.008	UNSAT TT=0.008	UNSAT TT=0.592	UNSAT TT=0.334	UNSAT TT=0.029
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/ dbsg/normalized-dbsg_10_4_5_8.opb	UNSAT TT=0.007	UNSAT TT=0.007	UNSAT TT=0.203	UNSAT TT=0.175	UNSAT TT=0.019
Bench name	Best results	minisatp 2012-10-02 git-d91742b (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_200_10_1_15.opb	SAT TT=0.126 T1=0.12	SAT TT=0.246 T1=0.24	SAT TT=1.774 T1=0.93	? (TO) TT=1800.03	SAT TT=0.126 T1=0.12
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_200_10_1_40.opb	SAT TT=0.251 T1=0.24	SAT TT=0.553 T1=0.55	SAT TT=94.26 T1=45.95	? (TO) TT=1800.04	SAT TT=0.251 T1=0.24
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_200_10_2_15.opb	SAT TT=0.116 T1=0.11	SAT TT=0.313 T1=0.31	SAT TT=1.733 T1=0.93	? (TO) TT=1800.04	SAT TT=0.116 T1=0.11
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_200_10_2_40.opb	SAT TT=0.248 T1=0.37	SAT TT=0.373 T1=0.37	SAT TT=12.762 T1=5.44	? (TO) TT=1800.05	SAT TT=0.248 T1=0.43
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_200_10_3_15.opb	SAT TT=0.11 T1=0.11	SAT TT=0.346 T1=0.34	SAT TT=1.704 T1=0.95	? (TO) TT=1800.29	SAT TT=0.11 T1=0.11
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_200_10_3_40.opb	SAT TT=0.246 T1=0.23	SAT TT=0.376 T1=0.37	SAT TT=16.061 T1=6.93	? (TO) TT=1800.02	SAT TT=0.246 T1=0.23
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_200_10_4_15.opb	SAT TT=0.118 T1=0.11	SAT TT=0.293 T1=0.29	SAT TT=1.805 T1=0.92	? (TO) TT=1800.05	SAT TT=0.118 T1=0.11
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_200_10_4_40.opb	SAT TT=0.31 T1=0.29	SAT TT=0.412 T1=0.41	SAT TT=8.322 T1=3.44	? (TO) TT=1800	SAT TT=0.31 T1=0.29
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_200_10_5_15.opb	SAT TT=0.12 T1=0.11	SAT TT=0.263 T1=0.26	SAT TT=1.814 T1=0.94	? (TO) TT=1800.79	SAT TT=0.12 T1=0.11
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_200_10_5_40.opb	SAT TT=0.214 T1=0.2	SAT TT=0.383 T1=0.38	SAT TT=7.892 T1=2.93	? (TO) TT=1800.92	SAT TT=0.214 T1=0.2
Bench name	Best results	minisatp 2012-10-02 git-d91742b (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_200_25_1_15.opb	SAT TT=0.237 T1=0.22	SAT TT=0.677 T1=0.67	SAT TT=7.923 T1=2.69	? (TO) TT=1800.08	SAT TT=0.237 T1=0.22
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_200_25_1_40.opb		? (TO) TT=1800.08	? (TO) TT=1800.02	? (TO) TT=1800	? (TO) TT=1800.02
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_200_25_2_15.opb	SAT TT=0.233 T1=0.22	SAT TT=0.631 T1=0.62	SAT TT=8.003 T1=2.69	? (TO) TT=1800.08	SAT TT=0.233 T1=0.22
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_200_25_2_40.opb		? (TO) TT=1800.02	? (TO) TT=1800	? (TO) TT=1800.09	? (TO) TT=1800.05
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_200_25_3_15.opb	SAT TT=0.233 T1=0.22	SAT TT=0.72 T1=0.71	SAT TT=8.763 T1=3.18	? (TO) TT=1800.02	SAT TT=0.233 T1=0.22
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_200_25_3_40.opb		? (TO) TT=1800.02	? (TO) TT=1800.12	? (TO) TT=1800.08	? (TO) TT=1800.07
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_200_25_4_15.opb	SAT TT=0.243 T1=0.23	SAT TT=0.629 T1=0.62	SAT TT=8.333 T1=2.68	? (TO) TT=1800.01	SAT TT=0.243 T1=0.23
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_200_25_4_40.opb		? (TO) TT=1800.02	? (TO) TT=1800.14	? (TO) TT=1800.75	? (TO) TT=1800.07
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_200_25_5_15.opb	SAT TT=0.236 T1=0.23	SAT TT=0.736 T1=0.73	SAT TT=9.733 T1=3.69	? (TO) TT=1800.08	SAT TT=0.236 T1=0.23
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_200_25_5_40.opb		? (TO) TT=1800.11	? (TO) TT=1800.08	? (TO) TT=1800.09	? (TO) TT=1800.04
Bench name	Best results	minisatp 2012-10-02 git-d91742b (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_10_1_15.opb	SAT TT=0.262 T1=0.25	SAT TT=2.403 T1=2.4	SAT TT=4.874 T1=1.75	? (TO) TT=1800.09	SAT TT=0.262 T1=0.25
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_10_1_80.opb	SAT TT=0.259 T1=0.24	SAT TT=1.694 T1=1.68	SAT TT=18.481 T1=7.72	? (TO) TT=1800	SAT TT=0.259 T1=0.24
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_10_2_15.opb	SAT TT=0.264 T1=0.46	SAT TT=1.587 T1=1.58	SAT TT=4.762 T1=1.76	? (TO) TT=1800.77	SAT TT=0.264 T1=0.46
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_10_2_80.opb	SAT TT=0.261 T1=0.25	SAT TT=2.124 T1=2.11	SAT TT=19.072 T1=8.23	? (TO) TT=1800.05	SAT TT=0.261 T1=0.25
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_10_3_15.opb	SAT TT=0.257 T1=0.24	SAT TT=0.916 T1=0.91	SAT TT=4.673 T1=1.72	? (TO) TT=1800.7	SAT TT=0.257 T1=0.24
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_10_3_80.opb	SAT TT=0.268 T1=0.26	SAT TT=1.866 T1=1.86	SAT TT=19.611 T1=8.26	? (TO) TT=1800.05	SAT TT=0.268 T1=0.26
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_10_4_15.opb	SAT TT=0.256 T1=0.24	SAT TT=1.403 T1=1.39	SAT TT=4.65 T1=1.72	? (TO) TT=1800.04	SAT TT=0.256 T1=0.24
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_10_4_80.opb	SAT TT=0.256 T1=0.24	SAT TT=1.526 T1=1.52	SAT TT=20.63 T1=8.73	? (TO) TT=1800.88	SAT TT=0.256 T1=0.24
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_10_5_15.opb	SAT TT=0.258 T1=0.24	SAT TT=1.4 T1=1.39	SAT TT=4.642 T1=1.72	? (TO) TT=1800.65	SAT TT=0.258 T1=0.24
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_10_5_80.opb	SAT TT=0.258 T1=0.24	SAT TT=4.09 T1=4.08	SAT TT=17.401 T1=7.26	? (TO) TT=1800.05	SAT TT=0.258 T1=0.24
Bench name	Best results	minisatp 2012-10-02 git-d91742b (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_25_1_15.opb	SAT TT=0.605 T1=0.73	SAT TT=3.289 T1=3.28	SAT TT=5.435 T1=2.15	? (TO) TT=1800.05	SAT TT=0.605 T1=0.73
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_25_1_80.opb		? (TO) TT=1800.08	? (TO) TT=1800.06	? (TO) TT=1800.67	? (TO) TT=1800.04
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_25_2_15.opb	SAT TT=0.592 T1=0.57	SAT TT=2.577 T1=2.56	SAT TT=4.454 T1=1.62	? (TO) TT=1800.06	SAT TT=0.592 T1=0.57
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_25_2_80.opb		? (TO) TT=1800.1	? (TO) TT=1800.18	? (TO) TT=1800.08	? (TO) TT=1800.11
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_25_3_15.opb	SAT TT=0.59 T1=0.58	SAT TT=4.489 T1=4.48	SAT TT=5.013 T1=1.64	? (TO) TT=1800.69	SAT TT=0.59 T1=0.58
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_25_3_80.opb		? (TO) TT=1800.07	? (TO) TT=1800.08	? (TO) TT=1800.77	? (TO) TT=1800.08
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_25_4_15.opb	SAT TT=0.601 T1=0.58	SAT TT=2.818 T1=2.81	SAT TT=5.373 T1=2.12	? (TO) TT=1800.07	SAT TT=0.601 T1=0.58
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_25_4_80.opb		? (TO) TT=1800.1	? (TO) TT=1800.73	? (TO) TT=1800.43	? (TO) TT=1800.12
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_25_5_15.opb	SAT TT=0.591 T1=0.57	SAT TT=2.016 T1=2	SAT TT=5.071 T1=2.13	? (TO) TT=1800.04	SAT TT=0.591 T1=0.57
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_25_5_80.opb		? (TO) TT=1800.11	? (TO) TT=1800.66	? (TO) TT=1800.02	? (TO) TT=1800.02
Bench name	Best results	minisatp 2012-10-02 git-d91742b (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_60_1_15.opb	SAT TT=1.39 T1=1.36	SAT TT=5.426 T1=5.4	SAT TT=5.532 T1=2.01	? (TO) TT=1800.04	SAT TT=1.39 T1=1.36
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_60_1_80.opb		? (TO) TT=1800.08	? (TO) TT=1800.07	? (TO) TT=1800.48	? (TO) TT=1800.02
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_60_2_15.opb	SAT TT=1.474 T1=1.44	SAT TT=6.715 T1=6.77	SAT TT=6.022 T1=2.11	? (TO) TT=1802.35	SAT TT=1.474 T1=1.44
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_60_2_80.opb		? (TO) TT=1800.05	? (TO) TT=1800.13	? (TO) TT=1800.07	? (TO) TT=1800.13
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_60_3_15.opb	SAT TT=1.43 T1=1.39	SAT TT=6.815 T1=6.79	SAT TT=5.243 T1=1.99	? (TO) TT=1800.08	SAT TT=1.43 T1=1.39
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_60_3_80.opb		? (TO) TT=1800.1	? (TO) TT=1800.14	? (TO) TT=1801	? (TO) TT=1800.09
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_60_4_15.opb	SAT TT=1.455 T1=1.42	SAT TT=7.226 T1=7.2	SAT TT=5.293 T1=2	? (TO) TT=1800.04	SAT TT=1.455 T1=1.42
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_60_4_80.opb		? (TO) TT=1800.07	? (TO) TT=1800.08	? (TO) TT=1800.61	? (TO) TT=1800.1
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_60_5_15.opb	SAT TT=1.416 T1=1.39	SAT TT=10.92 T1=10.9	SAT TT=5.233 T1=1.99	? (TO) TT=1800.08	SAT TT=1.416 T1=1.39
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_500_60_5_80.opb		? (TO) TT=1800.01	? (TO) TT=1800.02	? (TO) TT=1800.51	? (TO) TT=1800.12
Bench name	Best results	minisatp 2012-10-02 git-d91742b (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_50_10_1_15.opb		? (TO) TT=1800.02	? (TO) TT=1800.91	? (TO) TT=1800.08	? (TO) TT=1800.06
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_50_10_1_5.opb	SAT TT=0.035 T1=0.03	SAT TT=0.079 T1=0.07	SAT TT=0.523 T1=0.76	SAT TT=0.984 T1=0.63	SAT TT=0.035 T1=0.03
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_50_10_2_15.opb		? (TO) TT=1800.02	? (TO) TT=1800.01	? (TO) TT=1800.61	? (TO) TT=1800.01
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_50_10_2_5.opb	SAT TT=0.037 T1=0.03	SAT TT=0.068 T1=0.06	SAT TT=0.463 T1=0.75	SAT TT=0.804 T1=0.46	SAT TT=0.037 T1=0.03
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_50_10_3_15.opb		? (TO) TT=1800.02	? (TO) TT=1800.95	? (TO) TT=1800.06	? (TO) TT=1800.1
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_50_10_3_5.opb	SAT TT=0.033 T1=0.03	SAT TT=0.07 T1=0.07	SAT TT=0.483 T1=0.75	SAT TT=0.775 T1=0.48	SAT TT=0.033 T1=0.03
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_50_10_4_15.opb		? (TO) TT=1800.02	? (TO) TT=1800.02	? (TO) TT=1800.02	? (TO) TT=1800.14
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_50_10_4_5.opb	SAT TT=0.036 T1=0.03	SAT TT=0.074 T1=0.07	SAT TT=0.504 T1=0.75	SAT TT=0.884 T1=0.55	SAT TT=0.036 T1=0.03
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_50_10_5_15.opb		? (TO) TT=1800.02	? (TO) TT=1800.67	? (TO) TT=1800.03	? (TO) TT=1800.1
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_50_10_5_5.opb	SAT TT=0.034 T1=0.03	SAT TT=0.075 T1=0.07	SAT TT=0.574 T1=0.76	SAT TT=0.956 T1=0.6	SAT TT=0.034 T1=0.03
Bench name	Best results	minisatp 2012-10-02 git-d91742b (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_50_25_1_15.opb	UNSAT TT=6.53	UNSAT TT=6.53	UNSAT TT=34.018	UNSAT TT=31.84	UNSAT TT=180.077
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_50_25_1_5.opb	SAT TT=0.165 T1=0.16	SAT TT=0.165 T1=0.16	SAT TT=2.326 T1=0.85	SAT TT=0.892 T1=0.49	SAT TT=0.254 T1=0.23
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_50_25_2_15.opb	UNSAT TT=6.032	UNSAT TT=6.032	UNSAT TT=24.63	UNSAT TT=36.939	UNSAT TT=289.04
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_50_25_2_5.opb	SAT TT=0.061 T1=0.06	SAT TT=0.159 T1=0.16	SAT TT=2.123 T1=0.83	SAT TT=0.985 T1=0.58	SAT TT=0.061 T1=0.06
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_50_25_3_15.opb	UNSAT TT=6.563	UNSAT TT=6.563	UNSAT TT=54.716	UNSAT TT=37.849	UNSAT TT=243.302
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_50_25_3_5.opb	SAT TT=0.146 T1=0.14	SAT TT=0.146 T1=0.14	SAT TT=2.183 T1=0.85	SAT TT=0.944 T1=0.52	SAT TT=0.913 T1=0.86
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_50_25_4_15.opb	UNSAT TT=6.89	UNSAT TT=6.89	UNSAT TT=71.465	UNSAT TT=41.838	UNSAT TT=229.035
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_50_25_4_5.opb	SAT TT=0.162 T1=0.16	SAT TT=0.162 T1=0.16	SAT TT=2.255 T1=0.85	SAT TT=0.874 T1=0.46	SAT TT=0.443 T1=0.41
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_50_25_5_15.opb	UNSAT TT=7.127	UNSAT TT=7.127	UNSAT TT=40.048	UNSAT TT=37	UNSAT TT=338.858
normalized-PB07/SATUNSAT-SMALLINT-NLC/ submittedPB07/manquinho/dbsg/ normalized-dbsg_50_25_5_5.opb	SAT TT=0.159 T1=0.16	SAT TT=0.159 T1=0.16	SAT TT=2.154 T1=0.82	SAT TT=0.944 T1=0.57	SAT TT=0.385 T1=0.36

Statistics		minisatp 2012-10-02 git-d91742b (complete)	Sat4j PB 2.3.6 Res+CP PB16 (complete)	Sat4j PB 2.3.6 Resolution PB16 (complete)	toysat 2016-05-02 (complete)
Number of times the solver is able to give the best known answer		67	65	26	65
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)		21	0	0	46