PB'25 competition: ranking of solvers
All selected instances (including those not supported by some solvers)
max WC time for any solver=3605 s

Ranking on final answers (complete solver point of view)

This section presents the ranking of the different solvers. Solvers are ranked on the number of solved instances, i.e. where a final answer was given. Ties are broken by considering the minimum total solution time.

Solvers which gave a wrong answer in a category are excluded from the ranking in that category.

The Virtual Best Solver (VBS) is a theoretical construction which returns the best answer provided by one of the submitted solver. It corresponds to the 'Best result' column in the bench per bench view of the results. It can be considered as a meta solver which uses a perfect oracle to guess instantaneously which of the submitted solver will be the most efficient to solve the instance. Another way to look at it is to consider this VBS as a solver which would run all other solvers in parallel, bringing together all the solvers strenghs. This VBS is essentially the same notion as State Of The Art (SOTA) solver defined in "Evaluating general purpose automated theorem proving systems", Artificial Intelligence, Volume 131, Issues 1-2, September 2001, Pages 39-54 Geoff Sutcliffe, Christian Suttner. However, SOTA is a more general notion: the VBS can be seen as the least powerful SOTA solver.

Category "decision problem, linear constraints, no UNSAT certificate" (DEC-LIN): SAT+UNSAT answers

Rank	Solver	Version	Number of solved instances	Detail	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 502
Virtual Best Solver (VBS)			468	165 SAT, 303 UNSAT	93%	100%	63669.52	0.37	136.05	-	-	-	-
1	OR-Tools CP-SAT 20 cores	9.14	425	150 SAT, 275 UNSAT	85%	91%	467883.10	3.96	1100.90	25415.04	0.47	59.80	18.41
2	SCIP-NaPS	1.00a2	406	147 SAT, 259 UNSAT	81%	87%	109347.52	11.31	269.33	109707.57	11.72	270.22	1.00
3	Hybrid-CASHWMaxSATDisjCadS+SynLSCD	2025-07-23	404	140 SAT, 264 UNSAT	80%	86%	202979.63	10.46	502.42	203572.36	10.55	503.89	1.00
4	LSIPExact	S1	402	131 SAT, 271 UNSAT	80%	86%	135327.90	12.58	336.64	80201.93	11.63	199.51	1.69
5	Hybrid-CASHWMaxSATDisjComS+SynLSCD	2025-07-23	392	132 SAT, 260 UNSAT	78%	84%	149545.86	16.37	381.49	149996.31	16.46	382.64	1.00
6	LSIPExact	S2	389	132 SAT, 257 UNSAT	77%	83%	174411.85	15.03	448.36	98019.03	13.98	251.98	1.78
7	roundingsat+pbsuma	63f22ad	388	126 SAT, 262 UNSAT	77%	83%	36710.69	1.22	94.62	36813.39	1.24	94.88	1.00
8	roundingsat+pbsuma-log	63f22ad	387	128 SAT, 1 UNSAT, 258 UNSATC	77%	83%	37847.81	1.48	97.80	38084.62	1.58	98.41	0.99
9	UWrMaxSat-SCIP	2025-06-04	384	121 SAT, 263 UNSAT	76%	82%	55863.83	8.98	145.48	56034.81	9.01	145.92	1.00
10	roundingsat-ls	63f22ad	380	127 SAT, 253 UNSAT	76%	81%	30277.51	0.81	79.68	30365.19	0.83	79.91	1.00
11	roundingsat-ls-log	63f22ad	380	127 SAT, 253 UNSATC	76%	81%	32211.58	0.96	84.77	32321.49	0.98	85.06	1.00
12	SynLSCD-PS	2025-07-23	378	128 SAT, 250 UNSAT	75%	81%	48909.08	9.52	129.39	49079.16	10.00	129.84	1.00
13	roundingsat	63f22ad	373	114 SAT, 259 UNSAT	74%	80%	51417.59	1.26	137.85	51557.26	1.27	138.22	1.00
14	Exact_no_dombreaking	2025-06-16	373	122 SAT, 251 UNSAT	74%	80%	175674.80	359.84	470.98	176132.75	360.70	472.21	1.00
15	LSIPExact 20 cores	S1	373	131 SAT, 242 UNSAT	74%	80%	291003.18	12.28	780.17	37122.53	10.90	99.52	7.84
16	roundingsat-log	63f22ad	371	113 SAT, 258 UNSATC	74%	79%	46517.07	1.22	125.38	46661.11	1.33	125.77	1.00
17	SynLSCD	2025-07-23	371	124 SAT, 247 UNSAT	74%	79%	49107.26	11.15	132.36	49276.83	11.28	132.82	1.00
18	Exact	2025-06-02	367	119 SAT, 248 UNSAT	73%	78%	174064.90	359.73	474.29	174525.37	360.63	475.55	1.00
19	Exact_proof	2025-06-02	365	116 SAT, 15 UNSAT, 234 UNSATC	73%	78%	182802.87	360.38	500.83	183501.42	361.14	502.74	1.00
20	Exact_no_db_no_ls	2025-06-19	363	110 SAT, 253 UNSAT	72%	78%	41320.24	1.24	113.83	41437.72	1.24	114.15	1.00
21	OR-Tools CP-SAT	9.14	357	128 SAT, 229 UNSAT	71%	76%	39876.72	1.38	111.70	20957.21	1.01	58.70	1.90
22	LSIPExact 20 cores	S2	357	131 SAT, 226 UNSAT	71%	76%	244597.68	16.39	685.15	35850.73	11.95	100.42	6.82
23	NaPS	naps-1.03a6	323	139 SAT, 184 UNSAT	64%	69%	55977.17	4.60	173.30	56137.69	4.64	173.80	1.00
24	pb_picat	2025-07-11	319	140 SAT, 179 UNSAT	64%	68%	56859.54	8.14	178.24	57024.39	8.18	178.76	1.00
25	CASHWMaxSATDisjCad-S	2025-06-01	318	137 SAT, 181 UNSAT	63%	68%	43698.73	3.03	137.42	43828.80	3.05	137.83	1.00
26	UWrMaxSat	2025-06-04	318	137 SAT, 181 UNSAT	63%	68%	50443.33	3.28	158.63	50594.41	3.29	159.10	1.00
27	AI448PBSolver	2025-05-31	311	95 SAT, 216 UNSAT	62%	66%	65488.67	0.74	210.57	65724.00	0.76	211.33	1.00
28	CASHWMaxSATDisjCom-S	2025-06-01	308	124 SAT, 184 UNSAT	61%	66%	51168.61	3.56	166.13	51313.42	3.57	166.60	1.00
29	SCIP	2025-06-13	294	82 SAT, 212 UNSAT	59%	63%	51800.13	1.01	176.19	51982.00	1.02	176.81	1.00
30	SCIP	2024-07-07	291	81 SAT, 210 UNSAT	58%	62%	33655.88	1.01	115.66	33760.48	1.02	116.02	1.00
31	Sat4j Resolution	2025-06-06	260	103 SAT, 157 UNSAT	52%	56%	53433.25	4.47	205.51	53072.44	2.91	204.12	1.01
32	Sat4j Resolution VeriPB	2025-06-06	260	103 SAT, 157 UNSATC	52%	56%	57524.90	5.84	221.25	57010.03	3.78	219.27	1.01
33	Sat4j CP VeriPB	2025-06-06	243	68 SAT, 175 UNSATC	48%	52%	35255.56	5.44	145.08	34315.64	2.89	141.22	1.03
34	Sat4j CP	2025-06-06	182	54 SAT, 128 UNSAT	36%	39%	28985.26	2.39	159.26	28434.61	1.35	156.23	1.02
35	PRINTEMPS	2025-05-28	104	104 SAT	21%	22%	31342.17	9.12	301.37	24401.95	5.59	234.63	1.28
36	PRINTEMPS 20 cores	2025-05-28	104	104 SAT	21%	22%	135130.29	28.77	1299.33	22819.58	3.54	219.42	5.92

Category "decision problem, linear constraints, no UNSAT certificate" (DEC-LIN): SAT answers

Rank	Solver	Version	Number of solved instances	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 502
Virtual Best Solver (VBS)			165	33%	100%	39277.73	0.82	238.05	-	-	-	-
1	OR-Tools CP-SAT 20 cores	9.14	150	30%	91%	257147.09	10.59	1714.31	14590.83	2.61	97.27	17.62
2	SCIP-NaPS	1.00a2	147	29%	89%	66237.22	201.19	450.59	66448.13	201.81	452.03	1.00
3	Hybrid-CASHWMaxSATDisjCadS+SynLSCD	2025-07-23	140	28%	85%	21117.54	8.25	150.84	21190.10	8.33	151.36	1.00
4	pb_picat	2025-07-11	140	28%	85%	27819.21	10.28	198.71	27901.78	10.33	199.30	1.00
5	NaPS	naps-1.03a6	139	28%	84%	33780.44	12.61	243.02	33875.03	12.65	243.71	1.00
6	CASHWMaxSATDisjCad-S	2025-06-01	137	27%	83%	19545.75	8.17	142.67	19603.74	8.20	143.09	1.00
7	UWrMaxSat	2025-06-04	137	27%	83%	27617.40	8.11	201.59	27702.85	8.14	202.21	1.00
8	Hybrid-CASHWMaxSATDisjComS+SynLSCD	2025-07-23	132	26%	80%	30351.68	20.61	229.94	30463.78	20.85	230.79	1.00
9	LSIPExact	S2	132	26%	80%	43121.25	9.72	326.68	24570.38	9.75	186.14	1.76
10	LSIPExact	S1	131	26%	79%	32280.64	9.62	246.42	18890.53	9.66	144.20	1.71
11	LSIPExact 20 cores	S2	131	26%	79%	117057.42	9.52	893.57	16989.28	9.56	129.69	6.89
12	LSIPExact 20 cores	S1	131	26%	79%	117419.07	9.98	896.33	17179.91	10.02	131.14	6.83
13	roundingsat+pbsuma-log	63f22ad	128	25%	78%	11730.79	4.67	91.65	11833.29	4.71	92.45	0.99
14	SynLSCD-PS	2025-07-23	128	25%	78%	15293.76	3.98	119.48	15360.55	4.07	120.00	1.00
15	OR-Tools CP-SAT	9.14	128	25%	78%	20875.27	5.27	163.09	11304.99	4.16	88.32	1.85
16	roundingsat-ls	63f22ad	127	25%	77%	8700.10	2.31	68.50	8726.67	2.33	68.71	1.00
17	roundingsat-ls-log	63f22ad	127	25%	77%	8996.31	2.36	70.84	9027.07	2.39	71.08	1.00
18	roundingsat+pbsuma	63f22ad	126	25%	76%	12085.21	4.12	95.91	12121.09	4.15	96.20	1.00
19	SynLSCD	2025-07-23	124	25%	75%	14107.52	2.90	113.77	14166.81	2.92	114.25	1.00
20	CASHWMaxSATDisjCom-S	2025-06-01	124	25%	75%	26778.42	12.53	215.95	26856.27	12.56	216.58	1.00
21	Exact_no_dombreaking	2025-06-16	122	24%	74%	55681.73	360.70	456.41	55835.74	361.91	457.67	1.00
22	UWrMaxSat-SCIP	2025-06-04	121	24%	73%	5568.12	6.08	46.02	5584.43	6.09	46.15	1.00
23	Exact	2025-06-02	119	24%	72%	55409.85	359.94	465.63	55566.79	360.94	466.95	1.00
24	Exact_proof	2025-06-02	116	23%	70%	61528.65	366.93	530.42	61783.26	367.71	532.61	1.00
25	roundingsat	63f22ad	114	23%	69%	18720.62	5.29	164.22	18772.96	5.31	164.68	1.00
26	roundingsat-log	63f22ad	113	23%	68%	15749.73	5.58	139.38	15804.89	5.63	139.87	1.00
27	Exact_no_db_no_ls	2025-06-19	110	22%	67%	12087.51	5.85	109.89	12123.47	5.88	110.21	1.00
28	PRINTEMPS	2025-05-28	104	21%	63%	31342.17	9.12	301.37	24401.95	5.59	234.63	1.28
29	PRINTEMPS 20 cores	2025-05-28	104	21%	63%	135130.29	28.77	1299.33	22819.58	3.54	219.42	5.92
30	Sat4j Resolution	2025-06-06	103	21%	62%	15092.61	4.50	146.53	14921.88	3.07	144.87	1.01
31	Sat4j Resolution VeriPB	2025-06-06	103	21%	62%	17092.42	6.81	165.95	16805.42	4.34	163.16	1.02
32	AI448PBSolver	2025-05-31	95	19%	58%	17621.47	0.82	185.49	17680.51	0.87	186.11	1.00
33	SCIP	2025-06-13	82	16%	50%	27211.29	8.17	331.85	27310.60	8.20	333.06	1.00
34	SCIP	2024-07-07	81	16%	49%	15196.49	7.42	187.61	15240.91	7.47	188.16	1.00
35	Sat4j CP VeriPB	2025-06-06	68	14%	41%	13085.54	14.25	192.43	12765.79	10.24	187.73	1.03
36	Sat4j CP	2025-06-06	54	11%	33%	13680.86	7.31	253.35	13310.40	4.77	246.49	1.03

Category "decision problem, linear constraints, no UNSAT certificate" (DEC-LIN): UNSAT answers

Rank	Solver	Version	Number of solved instances	Detail	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 502
Virtual Best Solver (VBS)			303	303 UNSAT	60%	100%	24391.79	0.25	80.50	-	-	-	-
1	OR-Tools CP-SAT 20 cores	9.14	275	275 UNSAT	55%	91%	210736.01	2.55	766.31	10824.21	0.29	39.36	19.47
2	LSIPExact	S1	271	271 UNSAT	54%	89%	103047.26	15.17	380.25	61311.40	14.95	226.24	1.68
3	Hybrid-CASHWMaxSATDisjCadS+SynLSCD	2025-07-23	264	264 UNSAT	53%	87%	181862.09	16.57	688.87	182382.26	16.67	690.84	1.00
4	UWrMaxSat-SCIP	2025-06-04	263	263 UNSAT	52%	87%	50295.72	16.84	191.24	50450.39	16.93	191.83	1.00
5	roundingsat+pbsuma	63f22ad	262	262 UNSAT	52%	86%	24625.48	0.64	93.99	24692.29	0.66	94.25	1.00
6	Hybrid-CASHWMaxSATDisjComS+SynLSCD	2025-07-23	260	260 UNSAT	52%	86%	119194.18	15.52	458.44	119532.52	15.64	459.74	1.00
7	roundingsat+pbsuma-log	63f22ad	259	1 UNSAT, 258 UNSATC	52%	85%	26117.01	0.93	100.84	26251.33	0.96	101.36	0.99
8	roundingsat	63f22ad	259	259 UNSAT	52%	85%	32696.97	0.93	126.24	32784.29	0.96	126.58	1.00
9	SCIP-NaPS	1.00a2	259	259 UNSAT	52%	85%	43110.30	2.20	166.45	43259.45	2.25	167.02	1.00
10	roundingsat-log	63f22ad	258	258 UNSATC	51%	85%	30767.35	0.66	119.25	30856.22	0.69	119.60	1.00
11	LSIPExact	S2	257	257 UNSAT	51%	85%	131290.60	25.31	510.86	73448.65	21.43	285.79	1.79
12	roundingsat-ls	63f22ad	253	253 UNSAT	50%	83%	21577.41	0.56	85.29	21638.53	0.57	85.53	1.00
13	roundingsat-ls-log	63f22ad	253	253 UNSATC	50%	83%	23215.26	0.70	91.76	23294.42	0.73	92.07	1.00
14	Exact_no_db_no_ls	2025-06-19	253	253 UNSAT	50%	83%	29232.73	0.77	115.54	29314.25	0.77	115.87	1.00
15	Exact_no_dombreaking	2025-06-16	251	251 UNSAT	50%	83%	119993.07	359.59	478.06	120297.01	360.46	479.27	1.00
16	SynLSCD-PS	2025-07-23	250	250 UNSAT	50%	83%	33615.32	13.14	134.46	33718.61	13.23	134.87	1.00
17	Exact_proof	2025-06-02	249	15 UNSAT, 234 UNSATC	50%	82%	121274.22	359.77	487.05	121718.16	360.50	488.83	1.00
18	Exact	2025-06-02	248	248 UNSAT	49%	82%	118655.05	359.57	478.45	118958.59	360.44	479.67	1.00
19	SynLSCD	2025-07-23	247	247 UNSAT	49%	82%	34999.74	15.20	141.70	35110.01	15.26	142.15	1.00
20	LSIPExact 20 cores	S1	242	242 UNSAT	48%	80%	173584.11	16.11	717.29	19942.62	11.96	82.41	8.70
21	OR-Tools CP-SAT	9.14	229	229 UNSAT	46%	76%	19001.46	1.11	82.98	9652.21	0.70	42.15	1.97
22	LSIPExact 20 cores	S2	226	226 UNSAT	45%	75%	127540.26	17.37	564.34	18861.45	14.18	83.46	6.76
23	AI448PBSolver	2025-05-31	216	216 UNSAT	43%	71%	47867.20	0.70	221.61	48043.49	0.74	222.42	1.00
24	SCIP	2025-06-13	212	212 UNSAT	42%	70%	24588.84	0.57	115.99	24671.40	0.57	116.37	1.00
25	SCIP	2024-07-07	210	210 UNSAT	42%	69%	18459.39	0.51	87.90	18519.56	0.52	88.19	1.00
26	NaPS	naps-1.03a6	184	184 UNSAT	37%	61%	22196.73	2.94	120.63	22262.66	2.97	120.99	1.00
27	CASHWMaxSATDisjCom-S	2025-06-01	184	184 UNSAT	37%	61%	24390.19	1.78	132.56	24457.15	1.84	132.92	1.00
28	UWrMaxSat	2025-06-04	181	181 UNSAT	36%	60%	22825.93	1.12	126.11	22891.56	1.15	126.47	1.00
29	CASHWMaxSATDisjCad-S	2025-06-01	181	181 UNSAT	36%	60%	24152.98	1.72	133.44	24225.06	1.72	133.84	1.00
30	pb_picat	2025-07-11	179	179 UNSAT	36%	59%	29040.33	4.39	162.24	29122.62	4.45	162.70	1.00
31	Sat4j CP VeriPB	2025-06-06	175	175 UNSATC	35%	58%	22170.01	2.47	126.69	21549.85	1.35	123.14	1.03
32	Sat4j Resolution	2025-06-06	157	157 UNSAT	31%	52%	38340.64	3.97	244.21	38150.56	2.52	243.00	1.00
33	Sat4j Resolution VeriPB	2025-06-06	157	157 UNSATC	31%	52%	40432.47	5.50	257.53	40204.61	3.09	256.08	1.01
34	Sat4j CP	2025-06-06	128	128 UNSAT	25%	42%	15304.41	1.36	119.57	15124.21	0.84	118.16	1.01
35	PRINTEMPS	2025-05-28	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
36	PRINTEMPS 20 cores	2025-05-28	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-

Category "decision problem, non-linear constraints, no UNSAT certificate" (DEC-NLC): SAT+UNSAT answers

Rank	Solver	Version	Number of solved instances	Detail	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 10
Virtual Best Solver (VBS)			9	8 SAT, 1 UNSAT	90%	100%	932.22	0.12	103.58	-	-	-	-
1	pb_picat	2025-07-11	9	8 SAT, 1 UNSAT	90%	100%	992.62	0.95	110.29	995.55	0.96	110.62	1.00
2	SCIP	2025-06-13	9	8 SAT, 1 UNSAT	90%	100%	1546.55	0.17	171.84	1552.53	0.17	172.50	1.00
3	SCIP	2024-07-07	9	8 SAT, 1 UNSAT	90%	100%	1984.37	0.19	220.49	1992.48	0.36	221.39	1.00
4	Exact_no_db_no_ls	2025-06-19	8	8 SAT	80%	89%	1.04	0.12	0.13	1.07	0.13	0.13	0.97
5	PRINTEMPS	2025-05-28	8	8 SAT	80%	89%	3.64	0.37	0.46	3.07	0.31	0.38	1.19
6	OR-Tools CP-SAT	9.14	8	8 SAT	80%	89%	6.24	0.61	0.78	6.22	0.61	0.78	1.00
7	PRINTEMPS 20 cores	2025-05-28	8	8 SAT	80%	89%	6.95	0.82	0.87	2.15	0.21	0.27	3.24
8	OR-Tools CP-SAT 20 cores	9.14	8	8 SAT	80%	89%	8.02	1.06	1.00	5.60	0.58	0.70	1.43
9	Sat4j CP	2025-06-06	8	8 SAT	80%	89%	15.80	0.96	1.98	9.85	0.56	1.23	1.60
10	Exact	2025-06-02	8	8 SAT	80%	89%	2874.01	359.27	359.25	2881.20	360.14	360.15	1.00
11	Exact_no_dombreaking	2025-06-16	8	8 SAT	80%	89%	2874.24	359.31	359.28	2881.81	360.22	360.23	1.00
12	Sat4j Resolution	2025-06-06	4	4 SAT	40%	44%	8287.82	2546.20	2071.96	8254.40	2531.56	2063.60	1.00

Category "decision problem, non-linear constraints, no UNSAT certificate" (DEC-NLC): SAT answers

Rank	Solver	Version	Number of solved instances	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 10
Virtual Best Solver (VBS)			8	80%	100%	1.00	0.12	0.12	-	-	-	-
1	Exact_no_db_no_ls	2025-06-19	8	80%	100%	1.04	0.12	0.13	1.07	0.13	0.13	0.97
2	SCIP	2025-06-13	8	80%	100%	1.56	0.15	0.20	1.67	0.16	0.21	0.94
3	SCIP	2024-07-07	8	80%	100%	2.12	0.17	0.26	3.07	0.35	0.38	0.69
4	PRINTEMPS	2025-05-28	8	80%	100%	3.64	0.37	0.46	3.07	0.31	0.38	1.19
5	OR-Tools CP-SAT	9.14	8	80%	100%	6.24	0.61	0.78	6.22	0.61	0.78	1.00
6	PRINTEMPS 20 cores	2025-05-28	8	80%	100%	6.95	0.82	0.87	2.15	0.21	0.27	3.24
7	OR-Tools CP-SAT 20 cores	9.14	8	80%	100%	8.02	1.06	1.00	5.60	0.58	0.70	1.43
8	pb_picat	2025-07-11	8	80%	100%	8.46	0.93	1.06	8.53	0.94	1.07	0.99
9	Sat4j CP	2025-06-06	8	80%	100%	15.80	0.96	1.98	9.85	0.56	1.23	1.60
10	Exact	2025-06-02	8	80%	100%	2874.01	359.27	359.25	2881.20	360.14	360.15	1.00
11	Exact_no_dombreaking	2025-06-16	8	80%	100%	2874.24	359.31	359.28	2881.81	360.22	360.23	1.00
12	Sat4j Resolution	2025-06-06	4	40%	50%	8287.82	2546.20	2071.96	8254.40	2531.56	2063.60	1.00

Category "decision problem, non-linear constraints, no UNSAT certificate" (DEC-NLC): UNSAT answers

Rank	Solver	Version	Number of solved instances	Detail	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 10
Virtual Best Solver (VBS)			1	1 UNSAT	10%	100%	931.22	931.22	931.22	-	-	-	-
1	pb_picat	2025-07-11	1	1 UNSAT	10%	100%	984.16	984.16	984.16	987.02	987.02	987.02	1.00
2	SCIP	2025-06-13	1	1 UNSAT	10%	100%	1544.99	1544.99	1544.99	1550.86	1550.86	1550.86	1.00
3	SCIP	2024-07-07	1	1 UNSAT	10%	100%	1982.25	1982.25	1982.25	1989.41	1989.41	1989.41	1.00
4	Exact	2025-06-02	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
5	Exact_no_db_no_ls	2025-06-19	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
6	Exact_no_dombreaking	2025-06-16	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
7	OR-Tools CP-SAT	9.14	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
8	OR-Tools CP-SAT 20 cores	9.14	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
9	PRINTEMPS	2025-05-28	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
10	PRINTEMPS 20 cores	2025-05-28	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
11	Sat4j CP	2025-06-06	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
12	Sat4j Resolution	2025-06-06	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-

Category "optimization problem, linear constraints, no OPT/UNSAT certificate" (OPT-LIN): OPT+UNSAT answers

Rank	Solver	Version	Number of solved instances	Detail	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 555
Virtual Best Solver (VBS)			439	415 OPT, 24 UNSAT	79%	100%	336925.97	0.91	767.49	-	-	-	-
1	OR-Tools CP-SAT 20 cores	9.14	403	381 OPT, 22 UNSAT	73%	92%	961306.27	12.13	2385.38	51613.41	1.61	128.07	18.63
2	UWrMaxSat-SCIP	2025-06-04	357	338 OPT, 19 UNSAT	64%	81%	80696.93	4.46	226.04	80941.64	4.47	226.73	1.00
3	Hybrid-CASHWMaxSATDisjCom+ExactPRS9	2025-06-02	357	337 OPT, 20 UNSAT	64%	81%	166449.66	5.93	466.25	166958.96	6.01	467.67	1.00
4	Hybrid-CASHWMaxSATDisjCad+ExactPRS	2025-06-01	356	336 OPT, 20 UNSAT	64%	81%	164105.09	8.26	460.97	164614.32	8.32	462.40	1.00
5	Hybrid-CASHWMaxSATDisjCom+ExactPRS	2025-06-01	354	334 OPT, 20 UNSAT	64%	81%	169074.34	5.56	477.61	169590.45	5.61	479.07	1.00
6	LSIPExact 20 cores	S1	351	331 OPT, 20 UNSAT	63%	80%	592836.13	12.56	1688.99	53990.07	10.65	153.82	10.98
7	SCIP-NaPS	1.00a2	350	329 OPT, 21 UNSAT	63%	80%	93141.51	5.63	266.12	93453.07	5.71	267.01	1.00
8	ExactPR-S	2025-06-01	347	327 OPT, 20 UNSAT	63%	79%	78216.11	12.37	225.41	78466.32	12.42	226.13	1.00
9	mixed-bag-2024	2025-07-10	337	318 OPT, 19 UNSAT	61%	77%	45539.12	3.14	135.13	45683.45	3.16	135.56	1.00
10	roundingsat+pbsuma-opt-log	0f876b3	329	1 OPT, 310 OPTC, 18 UNSATC	59%	75%	63136.75	4.03	191.91	64040.80	4.22	194.65	0.99
11	roundingsat+pbsuma-opt	63f22ad	326	308 OPT, 18 UNSAT	59%	74%	59739.21	2.44	183.25	59910.79	2.46	183.78	1.00
12	roundingsat-ls+pbsuma-log	0f876b3	323	305 OPTC, 18 UNSATC	58%	74%	61527.55	3.94	190.49	62221.56	4.06	192.64	0.99
13	roundingsat-ls+pbsuma	63f22ad	317	299 OPT, 18 UNSAT	57%	72%	57649.19	2.98	181.86	57813.41	3.00	182.38	1.00
14	LSIPExact	S1	315	296 OPT, 19 UNSAT	57%	72%	72758.66	10.71	230.98	47730.10	10.47	151.52	1.52
15	Exact_no_dombreaking	2025-06-16	311	292 OPT, 19 UNSAT	56%	71%	152810.25	361.88	491.35	153232.48	363.22	492.71	1.00
16	ExactPR	2025-06-01	309	290 OPT, 19 UNSAT	56%	70%	50181.16	62.37	162.40	50329.75	62.59	162.88	1.00
17	Exact	2025-06-02	309	290 OPT, 19 UNSAT	56%	70%	147846.96	361.71	478.47	148257.06	362.67	479.80	1.00
18	SCIP	2024-07-07	308	290 OPT, 18 UNSAT	55%	70%	64459.09	1.86	209.28	64662.94	1.90	209.94	1.00
19	Exact_no_db_no_ls	2025-06-19	303	284 OPT, 19 UNSAT	55%	69%	52495.54	2.45	173.25	52649.61	2.46	173.76	1.00
20	OR-Tools CP-SAT	9.14	301	283 OPT, 18 UNSAT	54%	69%	60987.32	1.58	202.62	31562.13	1.17	104.86	1.93
21	SCIP	2025-06-13	300	282 OPT, 18 UNSAT	54%	68%	60686.56	1.69	202.29	60876.02	1.71	202.92	1.00
22	IPBHS-GUROBI-SYM	2025-06-09	293	283 OPT, 10 UNSAT	53%	67%	56172.09	1.60	191.71	56312.23	1.62	192.19	1.00
23	IPBHS-GUROBI	2025-06-09	291	281 OPT, 10 UNSAT	52%	66%	56061.56	1.30	192.65	56165.46	1.31	193.01	1.00
24	WMaxCDCL-SCIP	2025-06-21	290	276 OPT, 14 UNSAT	52%	66%	57735.38	6.22	199.09	57930.13	6.28	199.76	1.00
25	Exact_proof	2025-06-02	284	50 OPT, 214 OPTC, 20 UNSATC	51%	65%	154986.18	361.82	545.73	156382.14	363.18	550.64	0.99
26	UWrMaxSat	2025-06-04	280	263 OPT, 17 UNSAT	50%	64%	49872.42	1.60	178.12	50024.30	1.62	178.66	1.00
27	LSIPExact	S2	280	262 OPT, 18 UNSAT	50%	64%	152387.06	14.06	544.24	94130.52	14.03	336.18	1.62
28	CASHWMaxSATDisjCad-S	2025-06-01	278	261 OPT, 17 UNSAT	50%	63%	168992.20	18.48	607.89	169505.88	18.54	609.73	1.00
29	CASHWMaxSATDisjCom-S	2025-06-01	277	260 OPT, 17 UNSAT	50%	63%	178133.91	18.34	643.08	178687.93	18.41	645.08	1.00
30	LSIPExact 20 cores	S2	273	255 OPT, 18 UNSAT	49%	62%	228652.63	14.50	837.56	62411.76	12.88	228.61	3.66
31	IPBHS-SCIP	2025-06-09	271	261 OPT, 10 UNSAT	49%	62%	50172.97	2.99	185.14	50317.04	3.01	185.67	1.00
32	pb-oll-rs-2024	2025-05-06	257	240 OPT, 17 UNSAT	46%	59%	44295.99	1.72	172.36	44420.68	1.73	172.84	1.00
33	NaPS	naps-1.03a6	244	227 OPT, 17 UNSAT	44%	56%	40031.78	7.07	164.06	40155.57	7.13	164.57	1.00
34	pb_picat	2025-07-11	224	206 OPT, 18 UNSAT	40%	51%	54624.47	14.86	243.86	54784.80	14.92	244.57	1.00
35	roundingsat-ihs	63f22ad	200	183 OPT, 17 UNSAT	36%	46%	39184.12	2.27	195.92	39294.52	2.30	196.47	1.00
36	roundingsat-ihs-log	63f22ad	200	1 OPT, 182 OPTC, 17 UNSATC	36%	46%	43256.83	3.00	216.28	43465.12	3.05	217.33	1.00
37	Sat4j Resolution	2025-06-06	150	133 OPT, 17 UNSAT	27%	34%	25718.43	3.22	171.46	25479.66	1.83	169.86	1.01
38	Sat4j Resolution VeriPB	2025-06-06	149	6 OPT, 127 OPTC, 16 UNSATC	27%	34%	25528.47	3.33	171.33	25494.74	1.81	171.11	1.00
39	toulbar2	2025-06-13	145	132 OPT, 13 UNSAT	26%	33%	16154.14	0.57	111.41	16202.32	0.57	111.74	1.00
40	Sat4j CP VeriPB	2025-06-06	135	121 OPTC, 14 UNSATC	24%	31%	18969.13	4.06	140.51	18542.07	2.21	137.35	1.02
41	Sat4j CP	2025-06-06	117	103 OPT, 14 UNSAT	21%	27%	12664.17	1.93	108.24	12315.66	1.14	105.26	1.03
42	Hybrid-NuPBODeepOptS-ExactPRnols	2025-06-02	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
43	NuPBO-DeepOpt+	2025-06-02	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
44	NuPBO-DeepOptS+	2025-06-02	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
45	PRINTEMPS	2025-05-28	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
46	PRINTEMPS 20 cores	2025-05-28	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-

Category "optimization problem, linear constraints, no OPT/UNSAT certificate" (OPT-LIN): OPT answers

Rank	Solver	Version	Number of solved instances	Detail	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 555
Virtual Best Solver (VBS)			415	415 OPT	75%	100%	288540.57	1.08	695.28	-	-	-	-
1	OR-Tools CP-SAT 20 cores	9.14	381	381 OPT	69%	92%	901342.73	13.84	2365.73	48581.46	2.10	127.51	18.55
2	UWrMaxSat-SCIP	2025-06-04	338	338 OPT	61%	81%	78396.68	5.48	231.94	78635.53	5.49	232.65	1.00
3	Hybrid-CASHWMaxSATDisjCom+ExactPRS9	2025-06-02	337	337 OPT	61%	81%	161394.96	7.32	478.92	161889.03	7.37	480.38	1.00
4	Hybrid-CASHWMaxSATDisjCad+ExactPRS	2025-06-01	336	336 OPT	61%	81%	159926.24	8.99	475.97	160422.36	9.07	477.45	1.00
5	Hybrid-CASHWMaxSATDisjCom+ExactPRS	2025-06-01	334	334 OPT	60%	80%	164794.10	6.41	493.40	165295.93	6.46	494.90	1.00
6	LSIPExact 20 cores	S1	331	331 OPT	60%	80%	571469.82	12.74	1726.49	52256.96	10.67	157.88	10.94
7	SCIP-NaPS	1.00a2	329	329 OPT	59%	79%	91925.75	7.43	279.41	92232.91	7.50	280.34	1.00
8	ExactPR-S	2025-06-01	327	327 OPT	59%	79%	76740.40	14.22	234.68	76986.11	14.27	235.43	1.00
9	mixed-bag-2024	2025-07-10	318	318 OPT	57%	77%	45464.48	3.97	142.97	45608.33	4.02	143.42	1.00
10	roundingsat+pbsuma-opt-log	0f876b3	311	1 OPT, 310 OPTC	56%	75%	63099.17	5.91	202.89	64000.68	6.02	205.79	0.99
11	roundingsat+pbsuma-opt	63f22ad	308	308 OPT	55%	74%	59709.73	4.22	193.86	59880.82	4.26	194.42	1.00
12	roundingsat-ls+pbsuma-log	0f876b3	305	305 OPTC	55%	73%	61484.21	6.05	201.59	62176.99	6.12	203.86	0.99
13	roundingsat-ls+pbsuma	63f22ad	299	299 OPT	54%	72%	57611.15	3.96	192.68	57774.73	3.99	193.23	1.00
14	LSIPExact	S1	296	296 OPT	53%	71%	68842.80	10.83	232.58	45577.76	10.51	153.98	1.51
15	Exact_no_dombreaking	2025-06-16	292	292 OPT	53%	70%	149497.55	362.50	511.98	149909.53	363.59	513.39	1.00
16	ExactPR	2025-06-01	290	290 OPT	52%	70%	49747.57	63.15	171.54	49894.36	63.32	172.05	1.00
17	SCIP	2024-07-07	290	290 OPT	52%	70%	61810.29	2.05	213.14	62006.19	2.06	213.81	1.00
18	Exact	2025-06-02	290	290 OPT	52%	70%	144529.88	362.24	498.38	144929.62	363.36	499.76	1.00
19	Exact_no_db_no_ls	2025-06-19	284	284 OPT	51%	68%	52427.38	3.38	184.60	52581.11	3.40	185.14	1.00
20	IPBHS-GUROBI-SYM	2025-06-09	283	283 OPT	51%	68%	54427.97	1.79	192.32	54562.85	1.82	192.80	1.00
21	OR-Tools CP-SAT	9.14	283	283 OPT	51%	68%	60943.72	2.25	215.35	31536.56	1.33	111.44	1.93
22	SCIP	2025-06-13	282	282 OPT	51%	68%	58015.28	1.81	205.73	58196.75	1.82	206.37	1.00
23	IPBHS-GUROBI	2025-06-09	281	281 OPT	51%	68%	54308.06	1.39	193.27	54407.72	1.42	193.62	1.00
24	WMaxCDCL-SCIP	2025-06-21	276	276 OPT	50%	67%	57683.95	8.66	209.00	57877.85	8.73	209.70	1.00
25	Exact_proof	2025-06-02	264	50 OPT, 214 OPTC	48%	64%	145143.43	362.82	549.79	146359.05	363.95	554.39	0.99
26	UWrMaxSat	2025-06-04	263	263 OPT	47%	63%	49693.07	1.70	188.95	49844.26	1.71	189.52	1.00
27	LSIPExact	S2	262	262 OPT	47%	63%	149908.36	15.65	572.17	92702.06	14.57	353.82	1.62
28	IPBHS-SCIP	2025-06-09	261	261 OPT	47%	63%	48440.38	3.07	185.60	48579.34	3.09	186.13	1.00
29	CASHWMaxSATDisjCad-S	2025-06-01	261	261 OPT	47%	63%	166023.42	25.39	636.11	166527.80	25.48	638.04	1.00
30	CASHWMaxSATDisjCom-S	2025-06-01	260	260 OPT	47%	63%	174200.86	21.81	670.00	174741.96	21.95	672.08	1.00
31	LSIPExact 20 cores	S2	255	255 OPT	46%	61%	224224.57	16.93	879.31	61769.79	13.41	242.23	3.63
32	pb-oll-rs-2024	2025-05-06	240	240 OPT	43%	58%	42815.69	2.10	178.40	42936.42	2.11	178.90	1.00
33	NaPS	naps-1.03a6	227	227 OPT	41%	55%	39195.67	8.42	172.67	39316.57	8.50	173.20	1.00
34	pb_picat	2025-07-11	206	206 OPT	37%	50%	52529.27	17.05	255.00	52683.54	17.10	255.75	1.00
35	roundingsat-ihs	63f22ad	183	183 OPT	33%	44%	39143.80	2.95	213.90	39253.79	2.97	214.50	1.00
36	roundingsat-ihs-log	63f22ad	183	1 OPT, 182 OPTC	33%	44%	43210.61	3.38	236.12	43417.14	3.42	237.25	1.00
37	Sat4j Resolution	2025-06-06	133	133 OPT	24%	32%	21982.51	3.65	165.28	21759.63	2.12	163.61	1.01
38	Sat4j Resolution VeriPB	2025-06-06	133	6 OPT, 127 OPTC	24%	32%	25332.71	4.20	190.47	25305.20	2.21	190.26	1.00
39	toulbar2	2025-06-13	132	132 OPT	24%	32%	16002.01	0.62	121.23	16049.69	0.63	121.59	1.00
40	Sat4j CP VeriPB	2025-06-06	121	121 OPTC	22%	29%	18452.24	5.93	152.50	18044.77	3.15	149.13	1.02
41	Sat4j CP	2025-06-06	103	103 OPT	19%	25%	12636.22	2.37	122.68	12298.40	1.38	119.40	1.03
42	Hybrid-NuPBODeepOptS-ExactPRnols	2025-06-02	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
43	NuPBO-DeepOpt+	2025-06-02	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
44	NuPBO-DeepOptS+	2025-06-02	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
45	PRINTEMPS	2025-05-28	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
46	PRINTEMPS 20 cores	2025-05-28	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-

Category "optimization problem, linear constraints, no OPT/UNSAT certificate" (OPT-LIN): UNSAT answers

Rank	Solver	Version	Number of solved instances	Detail	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 555
Virtual Best Solver (VBS)			24	24 UNSAT	4%	100%	48385.40	0.12	2016.06	-	-	-	-
1	OR-Tools CP-SAT 20 cores	9.14	22	22 UNSAT	4%	92%	59963.55	0.18	2725.62	3031.95	0.18	137.82	19.78
2	SCIP-NaPS	1.00a2	21	21 UNSAT	4%	88%	1215.76	0.96	57.89	1220.17	1.06	58.10	1.00
3	ExactPR-S	2025-06-01	20	20 UNSAT	4%	83%	1475.71	0.19	73.79	1480.21	0.21	74.01	1.00
4	Hybrid-CASHWMaxSATDisjCad+ExactPRS	2025-06-01	20	20 UNSAT	4%	83%	4178.85	0.58	208.94	4191.96	0.62	209.60	1.00
5	Hybrid-CASHWMaxSATDisjCom+ExactPRS	2025-06-01	20	20 UNSAT	4%	83%	4280.24	0.54	214.01	4294.52	0.58	214.73	1.00
6	Hybrid-CASHWMaxSATDisjCom+ExactPRS9	2025-06-02	20	20 UNSAT	4%	83%	5054.69	0.21	252.73	5069.94	0.24	253.50	1.00
7	Exact_proof	2025-06-02	20	20 UNSATC	4%	83%	9842.76	359.33	492.14	10023.09	360.29	501.15	0.98
8	LSIPExact 20 cores	S1	20	20 UNSAT	4%	83%	21366.31	10.17	1068.32	1733.12	10.24	86.66	12.33
9	Exact_no_db_no_ls	2025-06-19	19	19 UNSAT	3%	79%	68.16	0.12	3.59	68.50	0.13	3.61	0.99
10	mixed-bag-2024	2025-07-10	19	19 UNSAT	3%	79%	74.63	0.30	3.93	75.12	0.31	3.95	0.99
11	ExactPR	2025-06-01	19	19 UNSAT	3%	79%	433.59	0.26	22.82	435.39	0.30	22.92	1.00
12	UWrMaxSat-SCIP	2025-06-04	19	19 UNSAT	3%	79%	2300.25	0.21	121.07	2306.12	0.21	121.37	1.00
13	Exact_no_dombreaking	2025-06-16	19	19 UNSAT	3%	79%	3312.70	0.40	174.35	3322.95	0.45	174.89	1.00
14	Exact	2025-06-02	19	19 UNSAT	3%	79%	3317.08	0.18	174.58	3327.44	0.18	175.13	1.00
15	LSIPExact	S1	19	19 UNSAT	3%	79%	3915.85	10.16	206.10	2152.34	10.22	113.28	1.82
16	roundingsat+pbsuma-opt	63f22ad	18	18 UNSAT	3%	75%	29.48	0.16	1.64	29.97	0.18	1.67	0.98
17	roundingsat+pbsuma-opt-log	0f876b3	18	18 UNSATC	3%	75%	37.57	0.45	2.09	40.12	0.56	2.23	0.94
18	roundingsat-ls+pbsuma	63f22ad	18	18 UNSAT	3%	75%	38.04	0.40	2.11	38.68	0.44	2.15	0.98
19	roundingsat-ls+pbsuma-log	0f876b3	18	18 UNSATC	3%	75%	43.34	0.20	2.41	44.57	0.23	2.48	0.97
20	OR-Tools CP-SAT	9.14	18	18 UNSAT	3%	75%	43.59	0.20	2.42	25.57	0.20	1.42	1.70
21	pb_picat	2025-07-11	18	18 UNSAT	3%	75%	2095.20	0.74	116.40	2101.26	0.75	116.74	1.00
22	LSIPExact	S2	18	18 UNSAT	3%	75%	2478.70	10.23	137.71	1428.46	10.28	79.36	1.74
23	SCIP	2024-07-07	18	18 UNSAT	3%	75%	2648.80	0.33	147.16	2656.76	0.33	147.60	1.00
24	SCIP	2025-06-13	18	18 UNSAT	3%	75%	2671.28	0.28	148.40	2679.27	0.28	148.85	1.00
25	LSIPExact 20 cores	S2	18	18 UNSAT	3%	75%	4428.06	10.15	246.00	641.97	10.24	35.67	6.90
26	roundingsat-ihs	63f22ad	17	17 UNSAT	3%	71%	40.32	0.09	2.37	40.73	0.11	2.40	0.99
27	roundingsat-ihs-log	63f22ad	17	17 UNSATC	3%	71%	46.22	0.16	2.72	47.98	0.17	2.82	0.96
28	UWrMaxSat	2025-06-04	17	17 UNSAT	3%	71%	179.35	0.19	10.55	180.03	0.19	10.59	1.00
29	NaPS	naps-1.03a6	17	17 UNSAT	3%	71%	836.11	0.20	49.18	839.00	0.22	49.35	1.00
30	pb-oll-rs-2024	2025-05-06	17	17 UNSAT	3%	71%	1480.30	0.10	87.08	1484.26	0.11	87.31	1.00
31	CASHWMaxSATDisjCad-S	2025-06-01	17	17 UNSAT	3%	71%	2968.78	0.19	174.63	2978.07	0.19	175.18	1.00
32	Sat4j Resolution	2025-06-06	17	17 UNSAT	3%	71%	3735.92	0.81	219.76	3720.03	0.51	218.83	1.00
33	CASHWMaxSATDisjCom-S	2025-06-01	17	17 UNSAT	3%	71%	3933.05	0.19	231.36	3945.97	0.20	232.12	1.00
34	Sat4j Resolution VeriPB	2025-06-06	16	16 UNSATC	3%	67%	195.76	0.81	12.23	189.53	0.47	11.85	1.03
35	Sat4j CP	2025-06-06	14	14 UNSAT	3%	58%	27.95	1.54	2.00	17.26	0.91	1.23	1.62
36	WMaxCDCL-SCIP	2025-06-21	14	14 UNSAT	3%	58%	51.43	0.12	3.67	52.28	0.16	3.73	0.98
37	Sat4j CP VeriPB	2025-06-06	14	14 UNSATC	3%	58%	516.89	2.07	36.92	497.30	1.15	35.52	1.04
38	toulbar2	2025-06-13	13	13 UNSAT	2%	54%	152.14	0.46	11.70	152.63	0.47	11.74	1.00
39	IPBHS-SCIP	2025-06-09	10	10 UNSAT	2%	42%	1732.59	0.22	173.26	1737.70	0.23	173.77	1.00
40	IPBHS-GUROBI-SYM	2025-06-09	10	10 UNSAT	2%	42%	1744.12	0.50	174.41	1749.38	0.54	174.94	1.00
41	IPBHS-GUROBI	2025-06-09	10	10 UNSAT	2%	42%	1753.50	0.30	175.35	1757.74	0.31	175.77	1.00
42	Hybrid-NuPBODeepOptS-ExactPRnols	2025-06-02	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
43	NuPBO-DeepOpt+	2025-06-02	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
44	NuPBO-DeepOptS+	2025-06-02	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
45	PRINTEMPS	2025-05-28	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
46	PRINTEMPS 20 cores	2025-05-28	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-

Category "optimization problem, non-linear constraints, no OPT/UNSAT certificate" (OPT-NLC): OPT+UNSAT answers

Rank	Solver	Version	Number of solved instances	Detail	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 57
Virtual Best Solver (VBS)			33	33 OPT	58%	100%	9594.67	0.86	290.75	-	-	-	-
1	SCIP	2024-07-07	32	32 OPT	56%	97%	10773.48	5.13	336.67	10805.86	5.16	337.68	1.00
2	SCIP	2025-06-13	31	31 OPT	54%	94%	8871.56	4.32	286.18	8898.78	4.34	287.06	1.00
3	OR-Tools CP-SAT 20 cores	9.14	30	30 OPT	53%	91%	184693.90	1.43	6156.46	9314.28	0.28	310.48	19.83
4	WMaxCDCL-SCIP	2025-06-21	29	29 OPT	51%	88%	5422.50	3.47	186.98	5440.36	3.51	187.60	1.00
5	OR-Tools CP-SAT	9.14	25	25 OPT	44%	76%	1181.11	0.64	47.24	596.09	0.38	23.84	1.98
6	pb_picat	2025-07-11	22	22 OPT	39%	67%	7072.24	0.05	321.47	7092.10	0.05	322.37	1.00
7	Exact	2025-06-02	21	21 OPT	37%	64%	10964.75	359.36	522.13	10991.18	360.04	523.39	1.00
8	Exact_no_db_no_ls	2025-06-19	20	20 OPT	35%	61%	1146.34	0.04	57.32	1149.22	0.04	57.46	1.00
9	Exact_no_dombreaking	2025-06-16	20	20 OPT	35%	61%	8567.79	359.33	428.39	8589.71	360.04	429.49	1.00
10	Sat4j Resolution	2025-06-06	15	15 OPT	26%	45%	200.74	2.16	13.38	184.41	1.14	12.29	1.09
11	Sat4j CP	2025-06-06	9	9 OPT	16%	27%	195.46	0.45	21.72	187.16	0.32	20.80	1.04
12	PRINTEMPS	2025-05-28	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-

Category "optimization problem, non-linear constraints, no OPT/UNSAT certificate" (OPT-NLC): OPT answers

Rank	Solver	Version	Number of solved instances	Detail	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 57
Virtual Best Solver (VBS)			33	33 OPT	58%	100%	9594.67	0.86	290.75	-	-	-	-
1	SCIP	2024-07-07	32	32 OPT	56%	97%	10773.48	5.13	336.67	10805.86	5.16	337.68	1.00
2	SCIP	2025-06-13	31	31 OPT	54%	94%	8871.56	4.32	286.18	8898.78	4.34	287.06	1.00
3	OR-Tools CP-SAT 20 cores	9.14	30	30 OPT	53%	91%	184693.90	1.43	6156.46	9314.28	0.28	310.48	19.83
4	WMaxCDCL-SCIP	2025-06-21	29	29 OPT	51%	88%	5422.50	3.47	186.98	5440.36	3.51	187.60	1.00
5	OR-Tools CP-SAT	9.14	25	25 OPT	44%	76%	1181.11	0.64	47.24	596.09	0.38	23.84	1.98
6	pb_picat	2025-07-11	22	22 OPT	39%	67%	7072.24	0.05	321.47	7092.10	0.05	322.37	1.00
7	Exact	2025-06-02	21	21 OPT	37%	64%	10964.75	359.36	522.13	10991.18	360.04	523.39	1.00
8	Exact_no_db_no_ls	2025-06-19	20	20 OPT	35%	61%	1146.34	0.04	57.32	1149.22	0.04	57.46	1.00
9	Exact_no_dombreaking	2025-06-16	20	20 OPT	35%	61%	8567.79	359.33	428.39	8589.71	360.04	429.49	1.00
10	Sat4j Resolution	2025-06-06	15	15 OPT	26%	45%	200.74	2.16	13.38	184.41	1.14	12.29	1.09
11	Sat4j CP	2025-06-06	9	9 OPT	16%	27%	195.46	0.45	21.72	187.16	0.32	20.80	1.04
12	PRINTEMPS	2025-05-28	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-

Category "optimization problem, non-linear constraints, no OPT/UNSAT certificate" (OPT-NLC): UNSAT answers

Rank	Solver	Version	Number of solved instances	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 57
Virtual Best Solver (VBS)			0	0%	-	0.00	0.00	-	-	-	-	-
1	Exact	2025-06-02	0	0%	-	0.00	0.00	-	0.00	0.00	-	-
2	Exact_no_db_no_ls	2025-06-19	0	0%	-	0.00	0.00	-	0.00	0.00	-	-
3	Exact_no_dombreaking	2025-06-16	0	0%	-	0.00	0.00	-	0.00	0.00	-	-
4	OR-Tools CP-SAT	9.14	0	0%	-	0.00	0.00	-	0.00	0.00	-	-
5	OR-Tools CP-SAT 20 cores	9.14	0	0%	-	0.00	0.00	-	0.00	0.00	-	-
6	PRINTEMPS	2025-05-28	0	0%	-	0.00	0.00	-	0.00	0.00	-	-
7	Sat4j CP	2025-06-06	0	0%	-	0.00	0.00	-	0.00	0.00	-	-
8	Sat4j Resolution	2025-06-06	0	0%	-	0.00	0.00	-	0.00	0.00	-	-
9	SCIP	2024-07-07	0	0%	-	0.00	0.00	-	0.00	0.00	-	-
10	SCIP	2025-06-13	0	0%	-	0.00	0.00	-	0.00	0.00	-	-
11	WMaxCDCL-SCIP	2025-06-21	0	0%	-	0.00	0.00	-	0.00	0.00	-	-
12	pb_picat	2025-07-11	0	0%	-	0.00	0.00	-	0.00	0.00	-	-

Ranking on the number of best answers found (incomplete solver point of view)

Number of best answers found with no further time limit

This section presents the ranking of the different solvers. Solvers are ranked on the number of instances for which they were able to find the best solution obtained by one the solvers within the global time limit. Ties are broken using the cumulated T1 time.

Solvers which gave a wrong answer in a category are excluded from the ranking in that category.

Category "optimization problem, linear constraints, no OPT/UNSAT certificate" (OPT-LIN): answers

Rank	Solver	Version	Number of best solutions found	% of all instances	% of VBS	Cumulated time
Total number of instances in the category: 555
Virtual Best Solver (VBS)			518	93%	100%
1	OR-Tools CP-SAT 20 cores	9.14	403	73%	78%	83399.43
2	Hybrid-CASHWMaxSATDisjCom+ExactPRS9	2025-06-02	391	70%	75%	289209.5
3	Hybrid-CASHWMaxSATDisjCad+ExactPRS	2025-06-01	386	70%	75%	272483.01
4	Hybrid-CASHWMaxSATDisjCom+ExactPRS	2025-06-01	381	69%	74%	266668.6
5	UWrMaxSat-SCIP	2025-06-04	357	64%	69%	73018.62
6	LSIPExact 20 cores	S1	351	63%	68%	73599.48
7	SCIP-NaPS	1.00a2	350	63%	68%	93303.48
8	ExactPR-S	2025-06-01	347	63%	67%	76901.67
9	mixed-bag-2024	2025-07-10	337	61%	65%	45662.51
10	OR-Tools CP-SAT	9.14	330	59%	64%	26167.27
11	roundingsat+pbsuma-opt-log	0f876b3	329	59%	64%	63665.6
12	roundingsat+pbsuma-opt	63f22ad	326	59%	63%	59902.41
13	roundingsat-ls+pbsuma-log	0f876b3	323	58%	62%	61921.46
14	LSIPExact	S1	319	57%	62%	57294.07
15	roundingsat-ls+pbsuma	63f22ad	317	57%	61%	57802.06
16	SCIP	2024-07-07	314	57%	61%	57887.22
17	Exact_no_dombreaking	2025-06-16	311	56%	60%	134052.08
18	Exact	2025-06-02	309	56%	60%	138513.87
19	ExactPR	2025-06-01	309	56%	60%	35870.77
20	SCIP	2025-06-13	307	55%	59%	57094.89
21	Exact_no_db_no_ls	2025-06-19	303	55%	58%	39195.08
22	IPBHS-GUROBI-SYM	2025-06-09	293	53%	57%	56287.89
23	LSIPExact	S2	291	52%	56%	115503.52
24	IPBHS-GUROBI	2025-06-09	291	52%	56%	56146.65
25	WMaxCDCL-SCIP	2025-06-21	290	52%	56%	57923.28
26	Exact_proof	2025-06-02	284	51%	55%	144857.81
27	UWrMaxSat	2025-06-04	280	50%	54%	36940.95
28	CASHWMaxSATDisjCad-S	2025-06-01	278	50%	54%	148603.36
29	CASHWMaxSATDisjCom-S	2025-06-01	277	50%	53%	155030.27
30	LSIPExact 20 cores	S2	273	49%	53%	66181.54
31	IPBHS-SCIP	2025-06-09	271	49%	52%	50293.65
32	pb-oll-rs-2024	2025-05-06	257	46%	50%	44411.64
33	NaPS	naps-1.03a6	244	44%	47%	40122.19
34	pb_picat	2025-07-11	224	40%	43%	38375.6
35	roundingsat-ihs-log	63f22ad	200	36%	39%	43397.85
36	roundingsat-ihs	63f22ad	200	36%	39%	39290.6
37	Sat4j Resolution	2025-06-06	157	28%	30%	20463.94
38	Sat4j Resolution VeriPB	2025-06-06	150	27%	29%	16968.38
39	toulbar2	2025-06-13	145	26%	28%	5041.31
40	Sat4j CP VeriPB	2025-06-06	137	25%	26%	9999.98
41	Sat4j CP	2025-06-06	120	22%	23%	9620.12
42	Hybrid-NuPBODeepOptS-ExactPRnols	2025-06-02	34	6%	7%	122282.06
43	PRINTEMPS	2025-05-28	11	2%	2%	7127.53

Category "optimization problem, non-linear constraints, no OPT/UNSAT certificate" (OPT-NLC): answers

Rank	Solver	Version	Number of best solutions found	% of all instances	% of VBS	Cumulated time
Total number of instances in the category: 57
Virtual Best Solver (VBS)			57	100%	100%
1	SCIP	2024-07-07	35	61%	61%	6012.28
2	SCIP	2025-06-13	34	60%	60%	7125.9
3	OR-Tools CP-SAT	9.14	32	56%	56%	1310.83
4	OR-Tools CP-SAT 20 cores	9.14	30	53%	53%	4.66
5	WMaxCDCL-SCIP	2025-06-21	29	51%	51%	5439.86
6	pb_picat	2025-07-11	22	39%	39%	2932.47
7	Exact	2025-06-02	21	37%	37%	9171.52
8	Exact_no_dombreaking	2025-06-16	20	35%	35%	7837.07
9	Exact_no_db_no_ls	2025-06-19	20	35%	35%	328.84
10	Sat4j Resolution	2025-06-06	15	26%	26%	26.16
11	Sat4j CP	2025-06-06	9	16%	16%	183.67
12	PRINTEMPS	2025-05-28	7	12%	12%	1150.73

Number of best answers found within 2 times the VBS time

This section presents the ranking of the different solvers. Solvers are ranked on the number of instances for which they were able to find the best solution obtained by one the solvers within 2 times the VBS time. Ties are broken using the cumulated T1 time.

Solvers which gave a wrong answer in a category are excluded from the ranking in that category.

Category "optimization problem, linear constraints, no OPT/UNSAT certificate" (OPT-LIN): answers

Rank	Solver	Version	Number of best solutions found	% of all instances	% of VBS	Cumulated time
Total number of instances in the category: 555
Virtual Best Solver (VBS)			518	93%	100%
1	OR-Tools CP-SAT 20 cores	9.14	206	37%	40%	9790.72
2	UWrMaxSat-SCIP	2025-06-04	120	22%	23%	2287.99
3	UWrMaxSat	2025-06-04	109	20%	21%	1239.08
4	Exact_no_db_no_ls	2025-06-19	105	19%	20%	1484.94
5	SCIP	2025-06-13	65	12%	13%	2525.84
6	OR-Tools CP-SAT	9.14	60	11%	12%	4400.48
7	Hybrid-CASHWMaxSATDisjCom+ExactPRS9	2025-06-02	57	10%	11%	21750.52
8	pb-oll-rs-2024	2025-05-06	54	10%	10%	1409.12
9	Hybrid-CASHWMaxSATDisjCom+ExactPRS	2025-06-01	44	8%	8%	3756.17
10	IPBHS-GUROBI	2025-06-09	42	8%	8%	1187.16
11	IPBHS-GUROBI-SYM	2025-06-09	38	7%	7%	1300.27
12	Hybrid-CASHWMaxSATDisjCad+ExactPRS	2025-06-01	35	6%	7%	7450.55
13	SCIP	2024-07-07	31	6%	6%	2458.83
14	CASHWMaxSATDisjCad-S	2025-06-01	31	6%	6%	1107.55
15	CASHWMaxSATDisjCom-S	2025-06-01	24	4%	5%	56.25
16	LSIPExact 20 cores	S1	23	4%	4%	6811.46
17	roundingsat+pbsuma-opt	63f22ad	21	4%	4%	1717.43
18	mixed-bag-2024	2025-07-10	21	4%	4%	112.03
19	Sat4j Resolution	2025-06-06	15	3%	3%	179.09
20	WMaxCDCL-SCIP	2025-06-21	12	2%	2%	575.51
21	Sat4j Resolution VeriPB	2025-06-06	11	2%	2%	130.11
22	roundingsat+pbsuma-opt-log	0f876b3	10	2%	2%	1358.94
23	NaPS	naps-1.03a6	10	2%	2%	538.74
24	LSIPExact	S1	8	1%	2%	3146.86
25	Sat4j CP	2025-06-06	8	1%	2%	15.98
26	toulbar2	2025-06-13	7	1%	1%	0.2
27	Hybrid-NuPBODeepOptS-ExactPRnols	2025-06-02	6	1%	1%	21583.34
28	IPBHS-SCIP	2025-06-09	6	1%	1%	998.51
29	roundingsat-ls+pbsuma	63f22ad	6	1%	1%	312.79
30	pb_picat	2025-07-11	4	1%	1%	2886.51
31	roundingsat-ihs	63f22ad	4	1%	1%	771.95
32	PRINTEMPS	2025-05-28	4	1%	1%	51.52
33	SCIP-NaPS	1.00a2	4	1%	1%	20.01
34	ExactPR	2025-06-01	3	1%	1%	1013.58
35	roundingsat-ihs-log	63f22ad	3	1%	1%	886.48
36	ExactPR-S	2025-06-01	3	1%	1%	781.43
37	roundingsat-ls+pbsuma-log	0f876b3	3	1%	1%	225.19
38	Exact_no_dombreaking	2025-06-16	2	0%	0%	1704.34
39	LSIPExact 20 cores	S2	2	0%	0%	158.99
40	LSIPExact	S2	1	0%	0%	1997.34
41	Exact	2025-06-02	1	0%	0%	903.43

Category "optimization problem, non-linear constraints, no OPT/UNSAT certificate" (OPT-NLC): answers

Rank	Solver	Version	Number of best solutions found	% of all instances	% of VBS	Cumulated time
Total number of instances in the category: 57
Virtual Best Solver (VBS)			57	100%	100%
1	OR-Tools CP-SAT 20 cores	9.14	27	47%	47%	4.26
2	Exact_no_db_no_ls	2025-06-19	12	21%	21%	0.14
3	OR-Tools CP-SAT	9.14	8	14%	14%	284.65
4	PRINTEMPS	2025-05-28	6	11%	11%	945.74
5	pb_picat	2025-07-11	6	11%	11%	0.21
6	SCIP	2025-06-13	5	9%	9%	87.93
7	SCIP	2024-07-07	3	5%	5%	1112.15
8	WMaxCDCL-SCIP	2025-06-21	1	2%	2%	116.79

Number of best answers found within 4 times the VBS time

This section presents the ranking of the different solvers. Solvers are ranked on the number of instances for which they were able to find the best solution obtained by one the solvers within 4 times the VBS time. Ties are broken using the cumulated T1 time.

Solvers which gave a wrong answer in a category are excluded from the ranking in that category.

Category "optimization problem, linear constraints, no OPT/UNSAT certificate" (OPT-LIN): answers

Rank	Solver	Version	Number of best solutions found	% of all instances	% of VBS	Cumulated time
Total number of instances in the category: 555
Virtual Best Solver (VBS)			518	93%	100%
1	OR-Tools CP-SAT 20 cores	9.14	276	50%	53%	10739.12
2	UWrMaxSat-SCIP	2025-06-04	152	27%	29%	9448.5
3	UWrMaxSat	2025-06-04	148	27%	29%	8222.49
4	Exact_no_db_no_ls	2025-06-19	133	24%	26%	3022.62
5	OR-Tools CP-SAT	9.14	128	23%	25%	6668.67
6	SCIP	2025-06-13	100	18%	19%	6429.19
7	IPBHS-GUROBI	2025-06-09	91	16%	18%	2339.9
8	Hybrid-CASHWMaxSATDisjCom+ExactPRS9	2025-06-02	80	14%	15%	27928.25
9	Hybrid-CASHWMaxSATDisjCom+ExactPRS	2025-06-01	78	14%	15%	9341.03
10	SCIP	2024-07-07	73	13%	14%	5479.9
11	IPBHS-GUROBI-SYM	2025-06-09	72	13%	14%	2421.93
12	pb-oll-rs-2024	2025-05-06	71	13%	14%	1947.04
13	roundingsat+pbsuma-opt	63f22ad	70	13%	14%	4107.03
14	Hybrid-CASHWMaxSATDisjCad+ExactPRS	2025-06-01	61	11%	12%	13387.04
15	mixed-bag-2024	2025-07-10	53	10%	10%	2423.87
16	CASHWMaxSATDisjCad-S	2025-06-01	44	8%	8%	1261.64
17	roundingsat-ls+pbsuma	63f22ad	43	8%	8%	3938.25
18	roundingsat+pbsuma-opt-log	0f876b3	41	7%	8%	4523.8
19	CASHWMaxSATDisjCom-S	2025-06-01	37	7%	7%	109.1
20	LSIPExact 20 cores	S1	32	6%	6%	8813.21
21	NaPS	naps-1.03a6	31	6%	6%	5608.2
22	IPBHS-SCIP	2025-06-09	28	5%	5%	2140.06
23	Sat4j Resolution	2025-06-06	25	5%	5%	1230.5
24	roundingsat-ls+pbsuma-log	0f876b3	24	4%	5%	4246.05
25	roundingsat-ihs	63f22ad	21	4%	4%	1390.24
26	ExactPR-S	2025-06-01	20	4%	4%	1017.62
27	WMaxCDCL-SCIP	2025-06-21	20	4%	4%	699.57
28	Sat4j Resolution VeriPB	2025-06-06	20	4%	4%	141.6
29	LSIPExact	S1	19	3%	4%	7257.1
30	roundingsat-ihs-log	63f22ad	18	3%	3%	1567.21
31	pb_picat	2025-07-11	17	3%	3%	2964.72
32	SCIP-NaPS	1.00a2	15	3%	3%	5474
33	Sat4j CP	2025-06-06	12	2%	2%	23.2
34	toulbar2	2025-06-13	12	2%	2%	0.74
35	ExactPR	2025-06-01	9	2%	2%	2862.07
36	Hybrid-NuPBODeepOptS-ExactPRnols	2025-06-02	8	1%	2%	28775.61
37	PRINTEMPS	2025-05-28	4	1%	1%	51.52
38	Exact_no_dombreaking	2025-06-16	3	1%	1%	2232.88
39	Exact_proof	2025-06-02	2	0%	0%	2212.93
40	LSIPExact	S2	2	0%	0%	2019.9
41	LSIPExact 20 cores	S2	2	0%	0%	158.99
42	Exact	2025-06-02	1	0%	0%	903.43

Category "optimization problem, non-linear constraints, no OPT/UNSAT certificate" (OPT-NLC): answers

Rank	Solver	Version	Number of best solutions found	% of all instances	% of VBS	Cumulated time
Total number of instances in the category: 57
Virtual Best Solver (VBS)			57	100%	100%
1	OR-Tools CP-SAT 20 cores	9.14	30	53%	53%	4.66
2	Exact_no_db_no_ls	2025-06-19	16	28%	28%	2.76
3	OR-Tools CP-SAT	9.14	11	19%	19%	285.68
4	pb_picat	2025-07-11	7	12%	12%	0.24
5	PRINTEMPS	2025-05-28	6	11%	11%	945.74
6	SCIP	2025-06-13	5	9%	9%	87.93
7	SCIP	2024-07-07	3	5%	5%	1112.15
8	WMaxCDCL-SCIP	2025-06-21	1	2%	2%	116.79
9	Sat4j Resolution	2025-06-06	1	2%	2%	0.86

Number of best answers found within 8 times the VBS time

This section presents the ranking of the different solvers. Solvers are ranked on the number of instances for which they were able to find the best solution obtained by one the solvers within 8 times the VBS time. Ties are broken using the cumulated T1 time.

Solvers which gave a wrong answer in a category are excluded from the ranking in that category.

Category "optimization problem, linear constraints, no OPT/UNSAT certificate" (OPT-LIN): answers

Rank	Solver	Version	Number of best solutions found	% of all instances	% of VBS	Cumulated time
Total number of instances in the category: 555
Virtual Best Solver (VBS)			518	93%	100%
1	OR-Tools CP-SAT 20 cores	9.14	311	56%	60%	13806.38
2	OR-Tools CP-SAT	9.14	190	34%	37%	9572.49
3	UWrMaxSat-SCIP	2025-06-04	172	31%	33%	15327.6
4	UWrMaxSat	2025-06-04	167	30%	32%	13378.01
5	Exact_no_db_no_ls	2025-06-19	157	28%	30%	7657.46
6	SCIP	2025-06-13	132	24%	25%	7755.45
7	IPBHS-GUROBI	2025-06-09	113	20%	22%	4649.85
8	IPBHS-GUROBI-SYM	2025-06-09	109	20%	21%	4747.89
9	roundingsat+pbsuma-opt	63f22ad	108	19%	21%	5735.42
10	SCIP	2024-07-07	105	19%	20%	7740.03
11	Hybrid-CASHWMaxSATDisjCom+ExactPRS9	2025-06-02	101	18%	19%	29862.13
12	pb-oll-rs-2024	2025-05-06	99	18%	19%	6446.81
13	Hybrid-CASHWMaxSATDisjCom+ExactPRS	2025-06-01	96	17%	19%	9463.13
14	Hybrid-CASHWMaxSATDisjCad+ExactPRS	2025-06-01	94	17%	18%	13628.23
15	roundingsat-ls+pbsuma	63f22ad	91	16%	18%	5153.42
16	roundingsat+pbsuma-opt-log	0f876b3	79	14%	15%	5797.1
17	mixed-bag-2024	2025-07-10	78	14%	15%	3833.88
18	roundingsat-ls+pbsuma-log	0f876b3	77	14%	15%	5463.77
19	CASHWMaxSATDisjCad-S	2025-06-01	60	11%	12%	5859.04
20	CASHWMaxSATDisjCom-S	2025-06-01	60	11%	12%	740.61
21	IPBHS-SCIP	2025-06-09	56	10%	11%	5688.28
22	NaPS	naps-1.03a6	54	10%	10%	6237.19
23	ExactPR-S	2025-06-01	45	8%	9%	3564.96
24	LSIPExact 20 cores	S1	43	8%	8%	10617.57
25	Sat4j Resolution	2025-06-06	38	7%	7%	1255.18
26	SCIP-NaPS	1.00a2	37	7%	7%	11186.63
27	WMaxCDCL-SCIP	2025-06-21	37	7%	7%	3729.59
28	roundingsat-ihs	63f22ad	33	6%	6%	2459.63
29	Sat4j Resolution VeriPB	2025-06-06	32	6%	6%	1625.83
30	LSIPExact	S1	30	5%	6%	8538.29
31	roundingsat-ihs-log	63f22ad	30	5%	6%	2693.06
32	pb_picat	2025-07-11	29	5%	6%	6352.96
33	toulbar2	2025-06-13	21	4%	4%	5.03
34	Sat4j CP	2025-06-06	19	3%	4%	51.12
35	ExactPR	2025-06-01	18	3%	3%	4396.47
36	Exact_no_dombreaking	2025-06-16	9	2%	2%	5750.59
37	Hybrid-NuPBODeepOptS-ExactPRnols	2025-06-02	8	1%	2%	28775.61
38	Exact	2025-06-02	8	1%	2%	5815.78
39	Exact_proof	2025-06-02	6	1%	1%	4330.71
40	Sat4j CP VeriPB	2025-06-06	5	1%	1%	17.23
41	PRINTEMPS	2025-05-28	4	1%	1%	51.52
42	LSIPExact	S2	2	0%	0%	2019.9
43	LSIPExact 20 cores	S2	2	0%	0%	158.99

Category "optimization problem, non-linear constraints, no OPT/UNSAT certificate" (OPT-NLC): answers

Rank	Solver	Version	Number of best solutions found	% of all instances	% of VBS	Cumulated time
Total number of instances in the category: 57
Virtual Best Solver (VBS)			57	100%	100%
1	OR-Tools CP-SAT 20 cores	9.14	30	53%	53%	4.66
2	Exact_no_db_no_ls	2025-06-19	18	32%	32%	6.16
3	OR-Tools CP-SAT	9.14	14	25%	25%	290.26
4	pb_picat	2025-07-11	7	12%	12%	0.24
5	PRINTEMPS	2025-05-28	6	11%	11%	945.74
6	SCIP	2025-06-13	6	11%	11%	91.75
7	SCIP	2024-07-07	3	5%	5%	1112.15
8	WMaxCDCL-SCIP	2025-06-21	2	4%	4%	120.28
9	Sat4j CP	2025-06-06	1	2%	2%	1.1
10	Sat4j Resolution	2025-06-06	1	2%	2%	0.86

PB'25 competition: ranking of solvers All selected instances (including those not supported by some solvers) max WC time for any solver=3605 s

Ranking on final answers (complete solver point of view)

Category "decision problem, linear constraints, no UNSAT certificate" (DEC-LIN): SAT+UNSAT answers

Category "decision problem, linear constraints, no UNSAT certificate" (DEC-LIN): SAT answers

Category "decision problem, linear constraints, no UNSAT certificate" (DEC-LIN): UNSAT answers

Category "decision problem, non-linear constraints, no UNSAT certificate" (DEC-NLC): SAT+UNSAT answers

Category "decision problem, non-linear constraints, no UNSAT certificate" (DEC-NLC): SAT answers

Category "decision problem, non-linear constraints, no UNSAT certificate" (DEC-NLC): UNSAT answers

Category "optimization problem, linear constraints, no OPT/UNSAT certificate" (OPT-LIN): OPT+UNSAT answers

Category "optimization problem, linear constraints, no OPT/UNSAT certificate" (OPT-LIN): OPT answers

Category "optimization problem, linear constraints, no OPT/UNSAT certificate" (OPT-LIN): UNSAT answers

Category "optimization problem, non-linear constraints, no OPT/UNSAT certificate" (OPT-NLC): OPT+UNSAT answers

Category "optimization problem, non-linear constraints, no OPT/UNSAT certificate" (OPT-NLC): OPT answers

Category "optimization problem, non-linear constraints, no OPT/UNSAT certificate" (OPT-NLC): UNSAT answers

Ranking on the number of best answers found (incomplete solver point of view)

Number of best answers found with no further time limit

Category "optimization problem, linear constraints, no OPT/UNSAT certificate" (OPT-LIN): answers

Category "optimization problem, non-linear constraints, no OPT/UNSAT certificate" (OPT-NLC): answers

Number of best answers found within 2 times the VBS time

Category "optimization problem, linear constraints, no OPT/UNSAT certificate" (OPT-LIN): answers

Category "optimization problem, non-linear constraints, no OPT/UNSAT certificate" (OPT-NLC): answers

Number of best answers found within 4 times the VBS time

Category "optimization problem, linear constraints, no OPT/UNSAT certificate" (OPT-LIN): answers

Category "optimization problem, non-linear constraints, no OPT/UNSAT certificate" (OPT-NLC): answers

Number of best answers found within 8 times the VBS time

Category "optimization problem, linear constraints, no OPT/UNSAT certificate" (OPT-LIN): answers

Category "optimization problem, non-linear constraints, no OPT/UNSAT certificate" (OPT-NLC): answers

PB'25 competition: ranking of solvers
All selected instances (including those not supported by some solvers)
max WC time for any solver=3605 s