PB'11 competition: satisfaction and optimization track: results by benchmark

Results by benchmark for category optimisation, small integers, linear constraints (OPT-SMALLINT-LIN), subcategory Handmade, subsubcategory Queens Problems

This page displays the results of the different solvers for each benchmark for category optimisation, small integers, linear constraints (OPT-SMALLINT-LIN), subcategory Handmade, subsubcategory Queens Problems

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell example	Meaning
Answer	Solver result
f=...	value of the objective function for the model reported by the solver
TT=...	Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out

Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

Abbreviation	Meaning
f=...	Value of the objective function
TO	Time Out
MO	Mem. Out (out of memory)

Meaning of the different colors:

Color	Meaning
text	the solver cannot handle this instance
text	the solver gave no answer
text	the solver could give an answer (SAT)
text	the solver gave a definitive answer (OPTIMUM FOUND or UNSAT)
text	the solver performed better than the other ones on that instance (complete solvers point of view)
text	the solver performed better than the other ones on that instance (incomplete solvers point of view)
text	the solver was ended by a signal or other problem
text	the solver gave an incomplete answer
text	the solver gave a wrong answer

For better readability, you may choose to hide some solvers:
borg pb-opt-11.04.03 (complete)
bsolo 3.2 (complete)
clasp 2.0-R4191 (complete)
MinisatID 2.4.8 [DEPRECATED] (complete)
MinisatID 2.4.8-gmp [DEPRECATED] (complete)
MinisatID 2.5.2 (fixed) (complete)
MinisatID 2.5.2-gmp (fixed) (complete)
pwbo 1.1 (complete)
Sat4j CuttingPlanes 2.3.0 (complete)
Sat4j Res//CP 2.3.0 (complete)
Sat4j Resolution 2.3.0 (complete)
SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED] (complete)
SCIP spx 2 2011-06-10 (fixed) (complete)
SCIP spx E SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED] (complete)
SCIP spx E_2 2011-06-10 (fixed) (complete)
wbo 1.6 (complete)

Bench name	Best results on this instance	borg pb-opt-11.04.03 (complete)	bsolo 3.2 (complete)	clasp 2.0-R4191 (complete)	MinisatID 2.4.8 [DEPRECATED] (complete)	MinisatID 2.4.8-gmp [DEPRECATED] (complete)	MinisatID 2.5.2 (fixed) (complete)	MinisatID 2.5.2-gmp (fixed) (complete)	pwbo 1.1 (complete)	Sat4j CuttingPlanes 2.3.0 (complete)	Sat4j Res//CP 2.3.0 (complete)	Sat4j Resolution 2.3.0 (complete)	SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED] (complete)	SCIP spx 2 2011-06-10 (fixed) (complete)	SCIP spx E SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED] (complete)	SCIP spx E_2 2011-06-10 (fixed) (complete)	wbo 1.6 (complete)
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_0.opb.PB06.opb	SAT f=886 TT=1800.25 T1=3.36	Wrong Opt. f=5624 TT=1418.3 T1=1413.18	SAT f=5661 TT=1798.1 T1=710.56	? (TO) TT=1800.09	? (TO) TT=1800.13	? (TO) TT=1800.12	? (TO) f=6369 TT=1800.1 T1=1800	? (TO) f=6369 TT=1800.12 T1=1800	SAT (TO) f=5275 TT=1800.31 T1=900.71	? (TO) TT=1800.42	SAT (TO) f=886 TT=1800.49 T1=4.13	SAT (TO) f=886 TT=1800.25 T1=3.36	? (TO) TT=1802.13	? TT=1799.65	? (TO) TT=1802.13	? TT=1799.54	? (TO) TT=1800.28
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_10.opb.PB06.opb	SAT f=840 TT=1800.23 T1=3.4	Wrong Opt. f=5670 TT=1145.39 T1=1139.49	SAT f=5518 TT=1798.07 T1=1481.37	? (TO) TT=1800.07	? (TO) TT=1800.09	? (TO) TT=1800.14	? (TO) f=6108 TT=1800.09 T1=1800	? (TO) f=6108 TT=1800.12 T1=1800	SAT (TO) f=5270 TT=1800.45 T1=900.92	? (TO) TT=1800.39	SAT (TO) f=840 TT=1800.4 T1=4.14	SAT (TO) f=840 TT=1800.23 T1=3.4	? (TO) TT=1800.17	? TT=1799.66	? (TO) TT=1802.11	? TT=1799.46	? (TO) TT=1800.31
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_11.opb.PB06.opb	SAT f=761 TT=1800.27 T1=3.37	Wrong Opt. f=6252 TT=773.938 T1=771.26	SAT f=5402 TT=1798.1 T1=413.74	? (TO) TT=1800.09	? (TO) TT=1800.09	? (TO) TT=1800.14	? (TO) f=5622 TT=1800.12 T1=1800.1	? (TO) f=5622 TT=1800.11 T1=1800	SAT (TO) f=5206 TT=1800.4 T1=900.71	? (TO) TT=1800.46	SAT (TO) f=761 TT=1800.55 T1=4.16	SAT (TO) f=761 TT=1800.27 T1=3.37	? (TO) TT=1802.13	? TT=1799.64	? (TO) TT=1802.12	? TT=1799.89	? (TO) TT=1800.31
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_12.opb.PB06.opb	SAT f=488 TT=1800.29 T1=3.31	Wrong Opt. f=6046 TT=1108.89 T1=1103.73	SAT f=5670 TT=1798.24 T1=1152.3	? (TO) TT=1800.08	? (TO) TT=1800.09	? (TO) TT=1800.1	? (TO) f=6526 TT=1800.1 T1=1800	? (TO) f=6526 TT=1800.13 T1=1800	SAT (TO) f=5417 TT=1800.46 T1=900.81	? (TO) TT=1800.4	SAT (TO) f=488 TT=1800.69 T1=4.13	SAT (TO) f=488 TT=1800.29 T1=3.31	? (TO) TT=1802.14	? TT=1799.86	? (TO) TT=1802.13	? TT=1799.53	? (TO) TT=1800.28
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_13.opb.PB06.opb	SAT f=1188 TT=1800.33 T1=4.07	Wrong Opt. f=6378 TT=1111.6 T1=1106.17	SAT f=5745 TT=1798.08 T1=1420.44	? (TO) TT=1800.08	? (TO) TT=1800.13	? (TO) TT=1800.13	? (TO) f=5858 TT=1800.1 T1=1800	? (TO) f=5858 TT=1800.13 T1=1800	SAT (TO) f=5399 TT=1800.36 T1=900.81	? (TO) TT=1800.39	SAT (TO) f=1188 TT=1800.53 T1=5.36	SAT (TO) f=1188 TT=1800.33 T1=4.07	? (TO) TT=1802.13	? TT=1799.49	? (TO) TT=1802.11	? TT=1799.5	? (TO) TT=1800.28
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_14.opb.PB06.opb	SAT f=768 TT=1800.23 T1=3.65	Wrong Opt. f=6726 TT=1094.39 T1=1089.76	SAT f=5800 TT=1798.1 T1=1106.88	? (TO) TT=1800.08	? (TO) TT=1800.11	? (TO) TT=1800.12	? (TO) f=6663 TT=1800.09 T1=1800	? (TO) f=6663 TT=1800.12 T1=1800	SAT (TO) f=5378 TT=1800.47 T1=900.81	? (TO) TT=1800.45	SAT (TO) f=768 TT=1800.51 T1=4.98	SAT (TO) f=768 TT=1800.23 T1=3.65	? (TO) TT=1802.11	? TT=1799.46	? (TO) TT=1802.11	? TT=1799.38	? (TO) TT=1800.27
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_1.opb.PB06.opb	SAT f=712 TT=1800.25 T1=3.35	Wrong Opt. f=6144 TT=1083.29 T1=1078.47	SAT f=5583 TT=1798.07 T1=388.79	? (TO) TT=1800.1	? (TO) TT=1800.12	? (TO) TT=1800.12	? (TO) f=6988 TT=1800.09 T1=1800	? (TO) f=6988 TT=1800.1 T1=1800	SAT (TO) f=5237 TT=1800.39 T1=900.71	? (TO) TT=1800.42	SAT (TO) f=712 TT=1800.6 T1=4.2	SAT (TO) f=712 TT=1800.25 T1=3.35	? (TO) TT=1802.16	? TT=1799.47	? (TO) TT=1802.11	? TT=1799.38	? (TO) TT=1800.29
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_2.opb.PB06.opb	SAT f=840 TT=1800.22 T1=3.62	Wrong Opt. f=5798 TT=940.224 T1=936.52	SAT f=5740 TT=1798.13 T1=302.39	? (TO) TT=1800.12	? (TO) TT=1800.11	? (TO) TT=1800.14	? (TO) f=5922 TT=1800.06 T1=1800	? (TO) f=5922 TT=1800.12 T1=1800	SAT (TO) f=5267 TT=1800.4 T1=900.81	? (TO) TT=1800.38	SAT (TO) f=840 TT=1800.49 T1=4.8	SAT (TO) f=840 TT=1800.22 T1=3.62	? (TO) TT=1802.12	? TT=1799.44	? (TO) TT=1802.08	? TT=1799.64	? (TO) TT=1800.28
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_3.opb.PB06.opb	SAT f=1077 TT=1800.28 T1=4.24	Wrong Opt. f=6278 TT=973.012 T1=969.51	SAT f=5830 TT=1798.05 T1=1140.89	? (TO) TT=1800.12	? (TO) TT=1800.1	? (TO) TT=1800.12	? (TO) f=6248 TT=1800.11 T1=1800	? (TO) f=6248 TT=1800.11 T1=1800	SAT (TO) f=5425 TT=1800.3 T1=900.81	? (TO) TT=1800.45	SAT (TO) f=1077 TT=1800.47 T1=6.46	SAT (TO) f=1077 TT=1800.28 T1=4.24	? (TO) TT=1802.13	? TT=1799.5	? (TO) TT=1802.12	? TT=1799.39	? (TO) TT=1800.3
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_4.opb.PB06.opb	SAT f=836 TT=1800.24 T1=3.33	Wrong Opt. f=6227 TT=918.539 T1=915.43	SAT f=5623 TT=1798.1 T1=983.19	? (TO) TT=1800.09	? (TO) TT=1800.08	? (TO) TT=1800.09	? (TO) f=6599 TT=1800.08 T1=1800	? (TO) f=6599 TT=1800.12 T1=1800	SAT (TO) f=5272 TT=1800.32 T1=900.81	? (TO) TT=1800.41	SAT (TO) f=836 TT=1800.4 T1=4.15	SAT (TO) f=836 TT=1800.24 T1=3.33	? (TO) TT=1802.12	? TT=1799.46	? (TO) TT=1802.11	? TT=1799.41	? (TO) TT=1800.21
Bench name	Best results	borg pb-opt-11.04.03 (complete)	bsolo 3.2 (complete)	clasp 2.0-R4191 (complete)	MinisatID 2.4.8 [DEPRECATED] (complete)	MinisatID 2.4.8-gmp [DEPRECATED] (complete)	MinisatID 2.5.2 (fixed) (complete)	MinisatID 2.5.2-gmp (fixed) (complete)	pwbo 1.1 (complete)	Sat4j CuttingPlanes 2.3.0 (complete)	Sat4j Res//CP 2.3.0 (complete)	Sat4j Resolution 2.3.0 (complete)	SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED] (complete)	SCIP spx 2 2011-06-10 (fixed) (complete)	SCIP spx E SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED] (complete)	SCIP spx E_2 2011-06-10 (fixed) (complete)	wbo 1.6 (complete)
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_5.opb.PB06.opb	SAT f=891 TT=1800.36 T1=3.37	Wrong Opt. f=5956 TT=964.083 T1=959.97	SAT f=5464 TT=1798.07 T1=1413.1	? (TO) TT=1800.08	? (TO) TT=1800.08	? (TO) TT=1800.1	? (TO) f=6725 TT=1800.1 T1=1800	? (TO) f=6725 TT=1800.11 T1=1800	SAT (TO) f=5044 TT=1800.46 T1=901.01	? (TO) TT=1800.5	SAT (TO) f=891 TT=1800.56 T1=4.49	SAT (TO) f=891 TT=1800.36 T1=3.37	? (TO) TT=1802.09	? (TO) TT=1800.09	? (TO) TT=1802.1	? (TO) TT=1802.12	? (TO) TT=1800.23
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_6.opb.PB06.opb	SAT f=977 TT=1800.31 T1=4.09	Wrong Opt. f=5821 TT=1059.3 T1=1054.76	SAT f=5484 TT=1798.06 T1=1470.3	? (TO) TT=1800.07	? (TO) TT=1800.11	? (TO) TT=1800.14	? (TO) f=5531 TT=1800.1 T1=1800	? (TO) f=5531 TT=1800.12 T1=1800	SAT (TO) f=5338 TT=1800.35 T1=396.39	? (TO) TT=1800.44	SAT (TO) f=977 TT=1800.69 T1=5.32	SAT (TO) f=977 TT=1800.31 T1=4.09	? (TO) TT=1802.09	? TT=1799.73	? (TO) TT=1802.12	? TT=1799.38	? (TO) TT=1800.24
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_7.opb.PB06.opb	SAT f=944 TT=1800.32 T1=3.34	Wrong Opt. f=5632 TT=994.235 T1=990.9	SAT f=5521 TT=1798.13 T1=334.92	? (TO) TT=1800.08	? (TO) TT=1800.08	? (TO) TT=1800.11	? (TO) f=6144 TT=1800.1 T1=1800	? (TO) f=6144 TT=1800.09 T1=1800	SAT (TO) f=5106 TT=1800.36 T1=900.71	? (TO) TT=1800.4	SAT (TO) f=944 TT=1800.72 T1=4.18	SAT (TO) f=944 TT=1800.32 T1=3.34	? (TO) TT=1802.14	? TT=1799.74	? (TO) TT=1802.13	? TT=1799.62	? (TO) TT=1800.19
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_8.opb.PB06.opb	SAT f=744 TT=1800.29 T1=3.87	Wrong Opt. f=6228 TT=1041.52 T1=1037.26	SAT f=5679 TT=1798.05 T1=1650.5	? (TO) TT=1800.06	? (TO) TT=1800.12	? (TO) TT=1800.13	? (TO) f=6576 TT=1800.11 T1=1800	? (TO) f=6576 TT=1800.12 T1=1800	SAT (TO) f=5345 TT=1800.39 T1=900.81	? (TO) TT=1800.38	SAT (TO) f=744 TT=1800.56 T1=5.38	SAT (TO) f=744 TT=1800.29 T1=3.87	? (TO) TT=1802.14	? TT=1799.52	? (TO) TT=1802.12	? TT=1799.39	? (TO) TT=1800.25
normalized-PB06/OPT-SMALLINT/ submitted-PB06/liu/wnq/ normalized-wnq_n100_mw99_9.opb.PB06.opb	SAT f=736 TT=1800.31 T1=3.36	Wrong Opt. f=5784 TT=1250.73 T1=1245.26	SAT f=5730 TT=1798.08 T1=1580.97	? (TO) TT=1800.09	? (TO) TT=1800.11	? (TO) TT=1800.16	? (TO) f=6388 TT=1800.08 T1=1800	? (TO) f=6388 TT=1800.11 T1=1800	SAT (TO) f=5421 TT=1800.37 T1=900.91	? (TO) TT=1800.38	SAT (TO) f=736 TT=1800.45 T1=4.13	SAT (TO) f=736 TT=1800.31 T1=3.36	? (TO) TT=1802.13	? TT=1799.46	? (TO) TT=1802.18	? TT=1799.77	? (TO) TT=1800.3

Statistics		borg pb-opt-11.04.03 (complete)	bsolo 3.2 (complete)	clasp 2.0-R4191 (complete)	MinisatID 2.4.8 [DEPRECATED] (complete)	MinisatID 2.4.8-gmp [DEPRECATED] (complete)	MinisatID 2.5.2 (fixed) (complete)	MinisatID 2.5.2-gmp (fixed) (complete)	pwbo 1.1 (complete)	Sat4j CuttingPlanes 2.3.0 (complete)	Sat4j Res//CP 2.3.0 (complete)	Sat4j Resolution 2.3.0 (complete)	SCIP spx SCIP 2.0.1.4. with SoPlex 1.5.0.4 [DEPRECATED] (complete)	SCIP spx 2 2011-06-10 (fixed) (complete)	SCIP spx E SCIP 2.0.1.4b with SoPlex 1.5.0.4 [DEPRECATED] (complete)	SCIP spx E_2 2011-06-10 (fixed) (complete)	wbo 1.6 (complete)
Number of times the solver is able to give the best known answer		0	0	0	0	0	0	0	0	0	15	15	0	0	0	0	0
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)		0	0	0	0	0	0	0	0	0	15	15	0	0	0	0	0