PB'25 competition: ranking of solvers
Ranking on the subset of instances supported by all solvers
max WC time for any solver=3605 s

Ranking on final answers (complete solver point of view)

This section presents the ranking of the different solvers. Solvers are ranked on the number of solved instances, i.e. where a final answer was given. Ties are broken by considering the minimum total solution time.

Solvers which gave a wrong answer in a category are excluded from the ranking in that category.

The Virtual Best Solver (VBS) is a theoretical construction which returns the best answer provided by one of the submitted solver. It corresponds to the 'Best result' column in the bench per bench view of the results. It can be considered as a meta solver which uses a perfect oracle to guess instantaneously which of the submitted solver will be the most efficient to solve the instance. Another way to look at it is to consider this VBS as a solver which would run all other solvers in parallel, bringing together all the solvers strenghs. This VBS is essentially the same notion as State Of The Art (SOTA) solver defined in "Evaluating general purpose automated theorem proving systems", Artificial Intelligence, Volume 131, Issues 1-2, September 2001, Pages 39-54 Geoff Sutcliffe, Christian Suttner. However, SOTA is a more general notion: the VBS can be seen as the least powerful SOTA solver.

Category "decision problem, linear constraints, no UNSAT certificate" (DEC-LIN): SAT+UNSAT answers

Rank	Solver	Version	Number of solved instances	Detail	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 472
Virtual Best Solver (VBS)			439	165 SAT, 274 UNSAT	93%	100%	62311.91	0.37	141.94	-	-	-	-
1	OR-Tools CP-SAT 20 cores	9.14	406	150 SAT, 256 UNSAT	86%	92%	464213.87	3.96	1143.38	25228.37	0.47	62.14	18.40
2	Hybrid-CASHWMaxSATDisjCadS+SynLSCD	2025-07-23	390	140 SAT, 250 UNSAT	83%	89%	199908.08	10.46	512.58	200490.95	10.55	514.08	1.00
3	SCIP-NaPS	1.00a2	389	147 SAT, 242 UNSAT	82%	89%	106816.50	11.31	274.59	107169.22	11.72	275.50	1.00
4	LSIPExact	S1	389	131 SAT, 258 UNSAT	82%	89%	134396.70	12.58	345.49	79269.71	11.63	203.78	1.70
5	Hybrid-CASHWMaxSATDisjComS+SynLSCD	2025-07-23	379	132 SAT, 247 UNSAT	80%	86%	147830.52	16.37	390.05	148275.26	16.46	391.23	1.00
6	LSIPExact	S2	376	132 SAT, 244 UNSAT	80%	86%	173461.01	15.03	461.33	97067.10	13.98	258.16	1.79
7	roundingsat+pbsuma	63f22ad	375	126 SAT, 249 UNSAT	79%	85%	36187.25	1.22	96.50	36288.28	1.24	96.77	1.00
8	roundingsat+pbsuma-log	63f22ad	374	128 SAT, 1 UNSAT, 245 UNSATC	79%	85%	37244.62	1.48	99.58	37476.76	1.58	100.21	0.99
9	roundingsat-ls	63f22ad	367	127 SAT, 240 UNSAT	78%	84%	29759.43	0.81	81.09	29845.47	0.83	81.32	1.00
10	roundingsat-ls-log	63f22ad	367	127 SAT, 240 UNSATC	78%	84%	31641.34	0.96	86.22	31748.15	0.98	86.51	1.00
11	UWrMaxSat-SCIP	2025-06-04	367	121 SAT, 246 UNSAT	78%	84%	52551.51	8.98	143.19	52712.28	9.01	143.63	1.00
12	SynLSCD-PS	2025-07-23	366	128 SAT, 238 UNSAT	78%	83%	48578.94	9.52	132.73	48745.72	10.00	133.19	1.00
13	Exact_no_dombreaking	2025-06-16	360	122 SAT, 238 UNSAT	76%	82%	171292.47	359.84	475.81	171739.51	360.70	477.05	1.00
14	LSIPExact 20 cores	S1	360	131 SAT, 229 UNSAT	76%	82%	290002.37	12.28	805.56	36125.02	10.90	100.35	8.03
15	roundingsat	63f22ad	359	114 SAT, 245 UNSAT	76%	82%	47400.89	1.26	132.04	47528.32	1.27	132.39	1.00
16	SynLSCD	2025-07-23	358	124 SAT, 234 UNSAT	76%	82%	45569.40	11.15	127.29	45728.45	11.28	127.73	1.00
17	roundingsat-log	63f22ad	358	113 SAT, 245 UNSATC	76%	82%	45940.76	1.22	128.33	46082.00	1.33	128.72	1.00
18	Exact	2025-06-02	355	119 SAT, 236 UNSAT	75%	81%	170560.81	359.73	480.45	171012.96	360.63	481.73	1.00
19	Exact_proof	2025-06-02	352	116 SAT, 15 UNSAT, 221 UNSATC	75%	80%	177714.47	360.38	504.87	178397.65	361.14	506.81	1.00
20	Exact_no_db_no_ls	2025-06-19	351	110 SAT, 241 UNSAT	74%	80%	41046.95	1.24	116.94	41163.38	1.24	117.27	1.00
21	LSIPExact 20 cores	S2	344	131 SAT, 213 UNSAT	73%	78%	243594.68	16.39	708.12	34851.04	11.95	101.31	6.99
22	OR-Tools CP-SAT	9.14	339	128 SAT, 211 UNSAT	72%	77%	39560.58	1.38	116.70	20796.84	1.01	61.35	1.90
23	pb_picat	2025-07-11	308	140 SAT, 168 UNSAT	65%	70%	54645.49	8.14	177.42	54806.07	8.18	177.94	1.00
24	NaPS	naps-1.03a6	306	139 SAT, 167 UNSAT	65%	70%	53847.24	4.60	175.97	54001.08	4.64	176.47	1.00
25	CASHWMaxSATDisjCad-S	2025-06-01	305	137 SAT, 168 UNSAT	65%	69%	42447.15	3.03	139.17	42573.24	3.05	139.58	1.00
26	UWrMaxSat	2025-06-04	305	137 SAT, 168 UNSAT	65%	69%	49119.51	3.28	161.05	49267.05	3.29	161.53	1.00
27	AI448PBSolver	2025-05-31	297	95 SAT, 202 UNSAT	63%	68%	56898.63	0.74	191.58	57107.70	0.76	192.28	1.00
28	CASHWMaxSATDisjCom-S	2025-06-01	295	124 SAT, 171 UNSAT	63%	67%	48873.80	3.56	165.67	49012.19	3.57	166.14	1.00
29	SCIP	2025-06-13	286	82 SAT, 204 UNSAT	61%	65%	51474.80	1.01	179.98	51655.63	1.02	180.61	1.00
30	SCIP	2024-07-07	281	81 SAT, 200 UNSAT	60%	64%	33542.49	1.01	119.37	33646.67	1.02	119.74	1.00
31	Sat4j Resolution	2025-06-06	249	103 SAT, 146 UNSAT	53%	57%	53302.48	4.47	214.07	52950.95	2.91	212.65	1.01
32	Sat4j Resolution VeriPB	2025-06-06	249	103 SAT, 146 UNSATC	53%	57%	57372.39	5.84	230.41	56875.94	3.78	228.42	1.01
33	Sat4j CP VeriPB	2025-06-06	228	68 SAT, 160 UNSATC	48%	52%	35237.03	5.44	154.55	34304.82	2.89	150.46	1.03
34	Sat4j CP	2025-06-06	161	54 SAT, 107 UNSAT	34%	37%	28156.61	2.39	174.89	27633.01	1.35	171.63	1.02
35	PRINTEMPS	2025-05-28	104	104 SAT	22%	24%	31342.17	9.12	301.37	24401.95	5.59	234.63	1.28
36	PRINTEMPS 20 cores	2025-05-28	104	104 SAT	22%	24%	135130.29	28.77	1299.33	22819.58	3.54	219.42	5.92

Category "decision problem, linear constraints, no UNSAT certificate" (DEC-LIN): SAT answers

Rank	Solver	Version	Number of solved instances	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 472
Virtual Best Solver (VBS)			165	35%	100%	39277.73	0.82	238.05	-	-	-	-
1	OR-Tools CP-SAT 20 cores	9.14	150	32%	91%	257147.09	10.59	1714.31	14590.83	2.61	97.27	17.62
2	SCIP-NaPS	1.00a2	147	31%	89%	66237.22	201.19	450.59	66448.13	201.81	452.03	1.00
3	Hybrid-CASHWMaxSATDisjCadS+SynLSCD	2025-07-23	140	30%	85%	21117.54	8.25	150.84	21190.10	8.33	151.36	1.00
4	pb_picat	2025-07-11	140	30%	85%	27819.21	10.28	198.71	27901.78	10.33	199.30	1.00
5	NaPS	naps-1.03a6	139	29%	84%	33780.44	12.61	243.02	33875.03	12.65	243.71	1.00
6	CASHWMaxSATDisjCad-S	2025-06-01	137	29%	83%	19545.75	8.17	142.67	19603.74	8.20	143.09	1.00
7	UWrMaxSat	2025-06-04	137	29%	83%	27617.40	8.11	201.59	27702.85	8.14	202.21	1.00
8	Hybrid-CASHWMaxSATDisjComS+SynLSCD	2025-07-23	132	28%	80%	30351.68	20.61	229.94	30463.78	20.85	230.79	1.00
9	LSIPExact	S2	132	28%	80%	43121.25	9.72	326.68	24570.38	9.75	186.14	1.76
10	LSIPExact	S1	131	28%	79%	32280.64	9.62	246.42	18890.53	9.66	144.20	1.71
11	LSIPExact 20 cores	S2	131	28%	79%	117057.42	9.52	893.57	16989.28	9.56	129.69	6.89
12	LSIPExact 20 cores	S1	131	28%	79%	117419.07	9.98	896.33	17179.91	10.02	131.14	6.83
13	roundingsat+pbsuma-log	63f22ad	128	27%	78%	11730.79	4.67	91.65	11833.29	4.71	92.45	0.99
14	SynLSCD-PS	2025-07-23	128	27%	78%	15293.76	3.98	119.48	15360.55	4.07	120.00	1.00
15	OR-Tools CP-SAT	9.14	128	27%	78%	20875.27	5.27	163.09	11304.99	4.16	88.32	1.85
16	roundingsat-ls	63f22ad	127	27%	77%	8700.10	2.31	68.50	8726.67	2.33	68.71	1.00
17	roundingsat-ls-log	63f22ad	127	27%	77%	8996.31	2.36	70.84	9027.07	2.39	71.08	1.00
18	roundingsat+pbsuma	63f22ad	126	27%	76%	12085.21	4.12	95.91	12121.09	4.15	96.20	1.00
19	SynLSCD	2025-07-23	124	26%	75%	14107.52	2.90	113.77	14166.81	2.92	114.25	1.00
20	CASHWMaxSATDisjCom-S	2025-06-01	124	26%	75%	26778.42	12.53	215.95	26856.27	12.56	216.58	1.00
21	Exact_no_dombreaking	2025-06-16	122	26%	74%	55681.73	360.70	456.41	55835.74	361.91	457.67	1.00
22	UWrMaxSat-SCIP	2025-06-04	121	26%	73%	5568.12	6.08	46.02	5584.43	6.09	46.15	1.00
23	Exact	2025-06-02	119	25%	72%	55409.85	359.94	465.63	55566.79	360.94	466.95	1.00
24	Exact_proof	2025-06-02	116	25%	70%	61528.65	366.93	530.42	61783.26	367.71	532.61	1.00
25	roundingsat	63f22ad	114	24%	69%	18720.62	5.29	164.22	18772.96	5.31	164.68	1.00
26	roundingsat-log	63f22ad	113	24%	68%	15749.73	5.58	139.38	15804.89	5.63	139.87	1.00
27	Exact_no_db_no_ls	2025-06-19	110	23%	67%	12087.51	5.85	109.89	12123.47	5.88	110.21	1.00
28	PRINTEMPS	2025-05-28	104	22%	63%	31342.17	9.12	301.37	24401.95	5.59	234.63	1.28
29	PRINTEMPS 20 cores	2025-05-28	104	22%	63%	135130.29	28.77	1299.33	22819.58	3.54	219.42	5.92
30	Sat4j Resolution	2025-06-06	103	22%	62%	15092.61	4.50	146.53	14921.88	3.07	144.87	1.01
31	Sat4j Resolution VeriPB	2025-06-06	103	22%	62%	17092.42	6.81	165.95	16805.42	4.34	163.16	1.02
32	AI448PBSolver	2025-05-31	95	20%	58%	17621.47	0.82	185.49	17680.51	0.87	186.11	1.00
33	SCIP	2025-06-13	82	17%	50%	27211.29	8.17	331.85	27310.60	8.20	333.06	1.00
34	SCIP	2024-07-07	81	17%	49%	15196.49	7.42	187.61	15240.91	7.47	188.16	1.00
35	Sat4j CP VeriPB	2025-06-06	68	14%	41%	13085.54	14.25	192.43	12765.79	10.24	187.73	1.03
36	Sat4j CP	2025-06-06	54	11%	33%	13680.86	7.31	253.35	13310.40	4.77	246.49	1.03

Category "decision problem, linear constraints, no UNSAT certificate" (DEC-LIN): UNSAT answers

Rank	Solver	Version	Number of solved instances	Detail	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 472
Virtual Best Solver (VBS)			274	274 UNSAT	58%	100%	23034.18	0.25	84.07	-	-	-	-
1	LSIPExact	S1	258	258 UNSAT	55%	94%	102116.07	15.17	395.80	60379.18	14.95	234.03	1.69
2	OR-Tools CP-SAT 20 cores	9.14	256	256 UNSAT	54%	93%	207066.78	2.55	808.85	10637.54	0.29	41.55	19.47
3	Hybrid-CASHWMaxSATDisjCadS+SynLSCD	2025-07-23	250	250 UNSAT	53%	91%	178790.54	16.57	715.16	179300.85	16.67	717.20	1.00
4	roundingsat+pbsuma	63f22ad	249	249 UNSAT	53%	91%	24102.04	0.64	96.80	24167.19	0.66	97.06	1.00
5	Hybrid-CASHWMaxSATDisjComS+SynLSCD	2025-07-23	247	247 UNSAT	52%	90%	117478.83	15.52	475.62	117811.48	15.64	476.97	1.00
6	roundingsat+pbsuma-log	63f22ad	246	1 UNSAT, 245 UNSATC	52%	90%	25513.82	0.93	103.71	25643.47	0.96	104.24	0.99
7	UWrMaxSat-SCIP	2025-06-04	246	246 UNSAT	52%	90%	46983.39	16.84	190.99	47127.86	16.93	191.58	1.00
8	roundingsat	63f22ad	245	245 UNSAT	52%	89%	28680.26	0.93	117.06	28755.35	0.96	117.37	1.00
9	roundingsat-log	63f22ad	245	245 UNSATC	52%	89%	30191.04	0.66	123.23	30277.11	0.69	123.58	1.00
10	LSIPExact	S2	244	244 UNSAT	52%	89%	130339.77	25.31	534.18	72496.72	21.43	297.12	1.80
11	SCIP-NaPS	1.00a2	242	242 UNSAT	51%	88%	40579.28	2.20	167.68	40721.09	2.25	168.27	1.00
12	Exact_no_db_no_ls	2025-06-19	241	241 UNSAT	51%	88%	28959.44	0.77	120.16	29039.91	0.77	120.50	1.00
13	roundingsat-ls	63f22ad	240	240 UNSAT	51%	88%	21059.33	0.56	87.75	21118.81	0.57	88.00	1.00
14	roundingsat-ls-log	63f22ad	240	240 UNSATC	51%	88%	22645.03	0.70	94.35	22721.08	0.73	94.67	1.00
15	SynLSCD-PS	2025-07-23	238	238 UNSAT	50%	87%	33285.18	13.14	139.85	33385.17	13.23	140.27	1.00
16	Exact_no_dombreaking	2025-06-16	238	238 UNSAT	50%	87%	115610.74	359.59	485.76	115903.77	360.46	486.99	1.00
17	Exact	2025-06-02	236	236 UNSAT	50%	86%	115150.95	359.57	487.93	115446.17	360.44	489.18	1.00
18	Exact_proof	2025-06-02	236	15 UNSAT, 221 UNSATC	50%	86%	116185.82	359.77	492.31	116614.39	360.50	494.13	1.00
19	SynLSCD	2025-07-23	234	234 UNSAT	50%	85%	31461.88	15.20	134.45	31561.63	15.26	134.88	1.00
20	LSIPExact 20 cores	S1	229	229 UNSAT	49%	84%	172583.30	16.11	753.64	18945.11	11.96	82.73	9.11
21	LSIPExact 20 cores	S2	213	213 UNSAT	45%	78%	126537.26	17.37	594.07	17861.75	14.18	83.86	7.08
22	OR-Tools CP-SAT	9.14	211	211 UNSAT	45%	77%	18685.31	1.11	88.56	9491.85	0.70	44.99	1.97
23	SCIP	2025-06-13	204	204 UNSAT	43%	74%	24263.51	0.57	118.94	24345.03	0.57	119.34	1.00
24	AI448PBSolver	2025-05-31	202	202 UNSAT	43%	74%	39277.16	0.70	194.44	39427.19	0.74	195.18	1.00
25	SCIP	2024-07-07	200	200 UNSAT	42%	73%	18346.00	0.51	91.73	18405.75	0.52	92.03	1.00
26	CASHWMaxSATDisjCom-S	2025-06-01	171	171 UNSAT	36%	62%	22095.38	1.78	129.21	22155.92	1.84	129.57	1.00
27	UWrMaxSat	2025-06-04	168	168 UNSAT	36%	61%	21502.11	1.12	127.99	21564.21	1.15	128.36	1.00
28	CASHWMaxSATDisjCad-S	2025-06-01	168	168 UNSAT	36%	61%	22901.40	1.72	136.32	22969.51	1.72	136.72	1.00
29	pb_picat	2025-07-11	168	168 UNSAT	36%	61%	26826.28	4.39	159.68	26904.30	4.45	160.14	1.00
30	NaPS	naps-1.03a6	167	167 UNSAT	35%	61%	20066.80	2.94	120.16	20126.05	2.97	120.52	1.00
31	Sat4j CP VeriPB	2025-06-06	160	160 UNSATC	34%	58%	22151.49	2.47	138.45	21539.03	1.35	134.62	1.03
32	Sat4j Resolution	2025-06-06	146	146 UNSAT	31%	53%	38209.87	3.97	261.71	38029.07	2.52	260.47	1.00
33	Sat4j Resolution VeriPB	2025-06-06	146	146 UNSATC	31%	53%	40279.96	5.50	275.89	40070.52	3.09	274.46	1.01
34	Sat4j CP	2025-06-06	107	107 UNSAT	23%	39%	14475.75	1.36	135.29	14322.62	0.84	133.86	1.01
35	PRINTEMPS	2025-05-28	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
36	PRINTEMPS 20 cores	2025-05-28	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-

Category "decision problem, non-linear constraints, no UNSAT certificate" (DEC-NLC): SAT+UNSAT answers

Rank	Solver	Version	Number of solved instances	Detail	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 10
Virtual Best Solver (VBS)			9	8 SAT, 1 UNSAT	90%	100%	932.22	0.12	103.58	-	-	-	-
1	pb_picat	2025-07-11	9	8 SAT, 1 UNSAT	90%	100%	992.62	0.95	110.29	995.55	0.96	110.62	1.00
2	SCIP	2025-06-13	9	8 SAT, 1 UNSAT	90%	100%	1546.55	0.17	171.84	1552.53	0.17	172.50	1.00
3	SCIP	2024-07-07	9	8 SAT, 1 UNSAT	90%	100%	1984.37	0.19	220.49	1992.48	0.36	221.39	1.00
4	Exact_no_db_no_ls	2025-06-19	8	8 SAT	80%	89%	1.04	0.12	0.13	1.07	0.13	0.13	0.97
5	PRINTEMPS	2025-05-28	8	8 SAT	80%	89%	3.64	0.37	0.46	3.07	0.31	0.38	1.19
6	OR-Tools CP-SAT	9.14	8	8 SAT	80%	89%	6.24	0.61	0.78	6.22	0.61	0.78	1.00
7	PRINTEMPS 20 cores	2025-05-28	8	8 SAT	80%	89%	6.95	0.82	0.87	2.15	0.21	0.27	3.24
8	OR-Tools CP-SAT 20 cores	9.14	8	8 SAT	80%	89%	8.02	1.06	1.00	5.60	0.58	0.70	1.43
9	Sat4j CP	2025-06-06	8	8 SAT	80%	89%	15.80	0.96	1.98	9.85	0.56	1.23	1.60
10	Exact	2025-06-02	8	8 SAT	80%	89%	2874.01	359.27	359.25	2881.20	360.14	360.15	1.00
11	Exact_no_dombreaking	2025-06-16	8	8 SAT	80%	89%	2874.24	359.31	359.28	2881.81	360.22	360.23	1.00
12	Sat4j Resolution	2025-06-06	4	4 SAT	40%	44%	8287.82	2546.20	2071.96	8254.40	2531.56	2063.60	1.00

Category "decision problem, non-linear constraints, no UNSAT certificate" (DEC-NLC): SAT answers

Rank	Solver	Version	Number of solved instances	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 10
Virtual Best Solver (VBS)			8	80%	100%	1.00	0.12	0.12	-	-	-	-
1	Exact_no_db_no_ls	2025-06-19	8	80%	100%	1.04	0.12	0.13	1.07	0.13	0.13	0.97
2	SCIP	2025-06-13	8	80%	100%	1.56	0.15	0.20	1.67	0.16	0.21	0.94
3	SCIP	2024-07-07	8	80%	100%	2.12	0.17	0.26	3.07	0.35	0.38	0.69
4	PRINTEMPS	2025-05-28	8	80%	100%	3.64	0.37	0.46	3.07	0.31	0.38	1.19
5	OR-Tools CP-SAT	9.14	8	80%	100%	6.24	0.61	0.78	6.22	0.61	0.78	1.00
6	PRINTEMPS 20 cores	2025-05-28	8	80%	100%	6.95	0.82	0.87	2.15	0.21	0.27	3.24
7	OR-Tools CP-SAT 20 cores	9.14	8	80%	100%	8.02	1.06	1.00	5.60	0.58	0.70	1.43
8	pb_picat	2025-07-11	8	80%	100%	8.46	0.93	1.06	8.53	0.94	1.07	0.99
9	Sat4j CP	2025-06-06	8	80%	100%	15.80	0.96	1.98	9.85	0.56	1.23	1.60
10	Exact	2025-06-02	8	80%	100%	2874.01	359.27	359.25	2881.20	360.14	360.15	1.00
11	Exact_no_dombreaking	2025-06-16	8	80%	100%	2874.24	359.31	359.28	2881.81	360.22	360.23	1.00
12	Sat4j Resolution	2025-06-06	4	40%	50%	8287.82	2546.20	2071.96	8254.40	2531.56	2063.60	1.00

Category "decision problem, non-linear constraints, no UNSAT certificate" (DEC-NLC): UNSAT answers

Rank	Solver	Version	Number of solved instances	Detail	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 10
Virtual Best Solver (VBS)			1	1 UNSAT	10%	100%	931.22	931.22	931.22	-	-	-	-
1	pb_picat	2025-07-11	1	1 UNSAT	10%	100%	984.16	984.16	984.16	987.02	987.02	987.02	1.00
2	SCIP	2025-06-13	1	1 UNSAT	10%	100%	1544.99	1544.99	1544.99	1550.86	1550.86	1550.86	1.00
3	SCIP	2024-07-07	1	1 UNSAT	10%	100%	1982.25	1982.25	1982.25	1989.41	1989.41	1989.41	1.00
4	Exact	2025-06-02	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
5	Exact_no_db_no_ls	2025-06-19	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
6	Exact_no_dombreaking	2025-06-16	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
7	OR-Tools CP-SAT	9.14	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
8	OR-Tools CP-SAT 20 cores	9.14	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
9	PRINTEMPS	2025-05-28	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
10	PRINTEMPS 20 cores	2025-05-28	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
11	Sat4j CP	2025-06-06	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
12	Sat4j Resolution	2025-06-06	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-

Category "optimization problem, linear constraints, no OPT/UNSAT certificate" (OPT-LIN): OPT+UNSAT answers

Rank	Solver	Version	Number of solved instances	Detail	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 420
Virtual Best Solver (VBS)			340	334 OPT, 6 UNSAT	81%	100%	285801.48	0.91	840.59	-	-	-	-
1	OR-Tools CP-SAT 20 cores	9.14	312	307 OPT, 5 UNSAT	74%	92%	700533.71	12.13	2245.30	37850.31	1.61	121.32	18.51
2	Hybrid-CASHWMaxSATDisjCom+ExactPRS9	2025-06-02	289	286 OPT, 3 UNSAT	69%	85%	137831.30	5.93	476.92	138252.73	6.01	478.38	1.00
3	Hybrid-CASHWMaxSATDisjCad+ExactPRS	2025-06-01	287	284 OPT, 3 UNSAT	68%	84%	136593.78	8.26	475.94	137019.23	8.32	477.42	1.00
4	Hybrid-CASHWMaxSATDisjCom+ExactPRS	2025-06-01	286	283 OPT, 3 UNSAT	68%	84%	144323.34	5.56	504.63	144762.86	5.61	506.16	1.00
5	UWrMaxSat-SCIP	2025-06-04	284	282 OPT, 2 UNSAT	68%	84%	63635.33	4.46	224.07	63825.32	4.47	224.74	1.00
6	ExactPR-S	2025-06-01	280	277 OPT, 3 UNSAT	67%	82%	64570.58	12.37	230.61	64781.76	12.42	231.36	1.00
7	SCIP-NaPS	1.00a2	280	277 OPT, 3 UNSAT	67%	82%	81921.64	5.63	292.58	82195.31	5.71	293.55	1.00
8	LSIPExact 20 cores	S1	278	274 OPT, 4 UNSAT	66%	82%	500749.62	12.56	1801.26	41361.02	10.65	148.78	12.11
9	mixed-bag-2024	2025-07-10	274	272 OPT, 2 UNSAT	65%	81%	43257.06	3.14	157.87	43393.05	3.16	158.37	1.00
10	roundingsat+pbsuma-opt-log	0f876b3	270	1 OPT, 267 OPTC, 2 UNSATC	64%	79%	50372.78	4.03	186.57	51085.11	4.22	189.20	0.99
11	IPBHS-GUROBI-SYM	2025-06-09	269	267 OPT, 2 UNSAT	64%	79%	49963.54	1.60	185.74	50089.50	1.62	186.21	1.00
12	IPBHS-GUROBI	2025-06-09	268	266 OPT, 2 UNSAT	64%	79%	52158.07	1.30	194.62	52254.41	1.31	194.98	1.00
13	roundingsat+pbsuma-opt	63f22ad	266	264 OPT, 2 UNSAT	63%	78%	49456.00	2.44	185.92	49598.46	2.46	186.46	1.00
14	roundingsat-ls+pbsuma-log	0f876b3	262	260 OPTC, 2 UNSATC	62%	77%	50603.85	3.94	193.14	51155.05	4.06	195.25	0.99
15	roundingsat-ls+pbsuma	63f22ad	258	256 OPT, 2 UNSAT	61%	76%	48825.19	2.98	189.24	48963.30	3.00	189.78	1.00
16	LSIPExact	S1	254	251 OPT, 3 UNSAT	60%	75%	61922.74	10.71	243.79	40908.41	10.47	161.06	1.51
17	IPBHS-SCIP	2025-06-09	249	247 OPT, 2 UNSAT	59%	73%	45711.97	2.99	183.58	45844.29	3.01	184.11	1.00
18	Exact_no_dombreaking	2025-06-16	245	243 OPT, 2 UNSAT	58%	72%	128178.09	361.88	523.18	128531.03	363.22	524.62	1.00
19	Exact	2025-06-02	244	242 OPT, 2 UNSAT	58%	72%	124067.07	361.71	508.47	124408.71	362.67	509.87	1.00
20	Exact_no_db_no_ls	2025-06-19	243	241 OPT, 2 UNSAT	58%	71%	46298.68	2.45	190.53	46435.21	2.46	191.09	1.00
21	ExactPR	2025-06-01	242	240 OPT, 2 UNSAT	58%	71%	43763.86	62.37	180.84	43891.18	62.59	181.37	1.00
22	WMaxCDCL-SCIP	2025-06-21	242	240 OPT, 2 UNSAT	58%	71%	47626.47	6.22	196.80	47783.09	6.28	197.45	1.00
23	SCIP	2024-07-07	236	232 OPT, 4 UNSAT	56%	69%	33749.44	1.86	143.01	33858.16	1.90	143.47	1.00
24	SCIP	2025-06-13	235	231 OPT, 4 UNSAT	56%	69%	28694.25	1.69	122.10	28788.10	1.71	122.50	1.00
25	UWrMaxSat	2025-06-04	227	225 OPT, 2 UNSAT	54%	67%	46192.06	1.60	203.49	46331.95	1.62	204.11	1.00
26	OR-Tools CP-SAT	9.14	224	222 OPT, 2 UNSAT	53%	66%	43766.53	1.58	195.39	22574.08	1.17	100.78	1.94
27	CASHWMaxSATDisjCad-S	2025-06-01	224	222 OPT, 2 UNSAT	53%	66%	144835.02	18.48	646.58	145276.44	18.54	648.56	1.00
28	Exact_proof	2025-06-02	222	38 OPT, 182 OPTC, 2 UNSATC	53%	65%	122234.62	361.82	550.61	123090.48	363.18	554.46	0.99
29	LSIPExact	S2	221	219 OPT, 2 UNSAT	53%	65%	144428.34	14.06	653.52	88437.62	14.03	400.17	1.63
30	CASHWMaxSATDisjCom-S	2025-06-01	221	219 OPT, 2 UNSAT	53%	65%	151307.62	18.34	684.65	151772.68	18.41	686.75	1.00
31	LSIPExact 20 cores	S2	209	207 OPT, 2 UNSAT	50%	61%	206576.47	14.50	988.40	51568.42	12.88	246.74	4.01
32	pb-oll-rs-2024	2025-05-06	208	206 OPT, 2 UNSAT	50%	61%	37472.41	1.72	180.16	37577.86	1.73	180.66	1.00
33	NaPS	naps-1.03a6	192	190 OPT, 2 UNSAT	46%	56%	29919.49	7.07	155.83	30012.53	7.13	156.32	1.00
34	pb_picat	2025-07-11	179	176 OPT, 3 UNSAT	43%	53%	46112.92	14.86	257.61	46248.71	14.92	258.37	1.00
35	roundingsat-ihs	63f22ad	162	160 OPT, 2 UNSAT	39%	48%	28516.56	2.27	176.03	28597.43	2.30	176.53	1.00
36	roundingsat-ihs-log	63f22ad	162	1 OPT, 159 OPTC, 2 UNSATC	39%	48%	31657.89	3.00	195.42	31807.35	3.05	196.34	1.00
37	toulbar2	2025-06-13	114	113 OPT, 1 UNSAT	27%	34%	13814.76	0.57	121.18	13855.89	0.57	121.54	1.00
38	Sat4j Resolution	2025-06-06	112	110 OPT, 2 UNSAT	27%	33%	19944.84	3.22	178.08	19768.50	1.83	176.50	1.01
39	Sat4j Resolution VeriPB	2025-06-06	112	5 OPT, 105 OPTC, 2 UNSATC	27%	33%	22425.90	3.33	200.23	22463.11	1.81	200.56	1.00
40	Sat4j CP VeriPB	2025-06-06	109	108 OPTC, 1 UNSATC	26%	32%	14490.64	4.06	132.94	14131.72	2.21	129.65	1.03
41	Sat4j CP	2025-06-06	87	87 OPT	21%	26%	12002.02	1.93	137.95	11701.00	1.14	134.49	1.03
42	Hybrid-NuPBODeepOptS-ExactPRnols	2025-06-02	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
43	NuPBO-DeepOpt+	2025-06-02	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
44	NuPBO-DeepOptS+	2025-06-02	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
45	PRINTEMPS	2025-05-28	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
46	PRINTEMPS 20 cores	2025-05-28	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-

Category "optimization problem, linear constraints, no OPT/UNSAT certificate" (OPT-LIN): OPT answers

Rank	Solver	Version	Number of solved instances	Detail	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 420
Virtual Best Solver (VBS)			334	334 OPT	80%	100%	237419.28	1.08	710.84	-	-	-	-
1	OR-Tools CP-SAT 20 cores	9.14	307	307 OPT	73%	92%	653549.47	13.84	2128.83	35477.36	2.10	115.56	18.42
2	Hybrid-CASHWMaxSATDisjCom+ExactPRS9	2025-06-02	286	286 OPT	68%	86%	136501.97	7.32	477.28	136919.19	7.37	478.74	1.00
3	Hybrid-CASHWMaxSATDisjCad+ExactPRS	2025-06-01	284	284 OPT	68%	85%	135047.70	8.99	475.52	135468.62	9.07	477.00	1.00
4	Hybrid-CASHWMaxSATDisjCom+ExactPRS	2025-06-01	283	283 OPT	67%	85%	142674.08	6.41	504.15	143108.02	6.46	505.68	1.00
5	UWrMaxSat-SCIP	2025-06-04	282	282 OPT	67%	84%	63474.89	5.48	225.09	63664.42	5.49	225.76	1.00
6	ExactPR-S	2025-06-01	277	277 OPT	66%	83%	64372.47	14.22	232.39	64582.75	14.27	233.15	1.00
7	SCIP-NaPS	1.00a2	277	277 OPT	66%	83%	81742.07	7.43	295.10	82015.01	7.50	296.08	1.00
8	LSIPExact 20 cores	S1	274	274 OPT	65%	82%	483667.04	12.74	1765.21	40192.88	10.67	146.69	12.03
9	mixed-bag-2024	2025-07-10	272	272 OPT	65%	81%	43203.04	3.97	158.83	43338.83	4.02	159.33	1.00
10	roundingsat+pbsuma-opt-log	0f876b3	268	1 OPT, 267 OPTC	64%	80%	50347.63	5.91	187.86	51059.46	6.02	190.52	0.99
11	IPBHS-GUROBI-SYM	2025-06-09	267	267 OPT	64%	80%	48221.94	1.79	180.61	48343.12	1.82	181.06	1.00
12	IPBHS-GUROBI	2025-06-09	266	266 OPT	63%	80%	50406.91	1.39	189.50	50499.15	1.42	189.85	1.00
13	roundingsat+pbsuma-opt	63f22ad	264	264 OPT	63%	79%	49434.69	4.22	187.25	49577.00	4.26	187.79	1.00
14	roundingsat-ls+pbsuma-log	0f876b3	260	260 OPTC	62%	78%	50574.04	6.05	194.52	51124.89	6.12	196.63	0.99
15	roundingsat-ls+pbsuma	63f22ad	256	256 OPT	61%	77%	48799.33	3.96	190.62	48937.34	3.99	191.16	1.00
16	LSIPExact	S1	251	251 OPT	60%	75%	61828.83	10.83	246.33	40845.00	10.51	162.73	1.51
17	IPBHS-SCIP	2025-06-09	247	247 OPT	59%	74%	43980.97	3.07	178.06	44108.22	3.09	178.58	1.00
18	Exact_no_dombreaking	2025-06-16	243	243 OPT	58%	73%	127442.91	362.50	524.46	127793.85	363.59	525.90	1.00
19	Exact	2025-06-02	242	242 OPT	58%	72%	123325.51	362.24	509.61	123665.34	363.36	511.01	1.00
20	Exact_no_db_no_ls	2025-06-19	241	241 OPT	57%	72%	46273.78	3.38	192.01	46410.25	3.40	192.57	1.00
21	ExactPR	2025-06-01	240	240 OPT	57%	72%	43610.82	63.15	181.71	43737.75	63.32	182.24	1.00
22	WMaxCDCL-SCIP	2025-06-21	240	240 OPT	57%	72%	47577.04	8.66	198.24	47733.37	8.73	198.89	1.00
23	SCIP	2024-07-07	232	232 OPT	55%	69%	31115.37	2.05	134.12	31216.30	2.06	134.55	1.00
24	SCIP	2025-06-13	231	231 OPT	55%	69%	26037.68	1.81	112.72	26123.59	1.82	113.09	1.00
25	UWrMaxSat	2025-06-04	225	225 OPT	54%	67%	46036.59	1.70	204.61	46175.92	1.71	205.23	1.00
26	OR-Tools CP-SAT	9.14	222	222 OPT	53%	66%	43729.19	2.25	196.98	22554.41	1.33	101.60	1.94
27	CASHWMaxSATDisjCad-S	2025-06-01	222	222 OPT	53%	66%	144628.46	25.39	651.48	145069.32	25.48	653.47	1.00
28	Exact_proof	2025-06-02	220	38 OPT, 182 OPTC	52%	66%	121493.22	362.82	552.24	122347.30	363.95	556.12	0.99
29	LSIPExact	S2	219	219 OPT	52%	66%	144341.70	15.65	659.09	88362.57	14.57	403.48	1.63
30	CASHWMaxSATDisjCom-S	2025-06-01	219	219 OPT	52%	66%	149210.70	21.81	681.33	149668.86	21.95	683.42	1.00
31	LSIPExact 20 cores	S2	207	207 OPT	49%	62%	206440.00	16.93	997.29	51499.52	13.41	248.79	4.01
32	pb-oll-rs-2024	2025-05-06	206	206 OPT	49%	62%	35995.14	2.10	174.73	36096.81	2.11	175.23	1.00
33	NaPS	naps-1.03a6	190	190 OPT	45%	57%	29135.37	8.42	153.34	29225.94	8.50	153.82	1.00
34	pb_picat	2025-07-11	176	176 OPT	42%	53%	44120.85	17.05	250.69	44251.02	17.10	251.43	1.00
35	roundingsat-ihs	63f22ad	160	160 OPT	38%	48%	28483.07	2.95	178.02	28563.78	2.97	178.52	1.00
36	roundingsat-ihs-log	63f22ad	160	1 OPT, 159 OPTC	38%	48%	31620.10	3.38	197.63	31768.18	3.42	198.55	1.00
37	toulbar2	2025-06-13	113	113 OPT	27%	34%	13673.39	0.62	121.00	13714.18	0.63	121.36	1.00
38	Sat4j Resolution	2025-06-06	110	110 OPT	26%	33%	19772.10	3.65	179.75	19602.18	2.12	178.20	1.01
39	Sat4j Resolution VeriPB	2025-06-06	110	5 OPT, 105 OPTC	26%	33%	22246.61	4.20	202.24	22284.29	2.21	202.58	1.00
40	Sat4j CP VeriPB	2025-06-06	108	108 OPTC	26%	32%	14008.08	5.93	129.70	13654.90	3.15	126.43	1.03
41	Sat4j CP	2025-06-06	87	87 OPT	21%	26%	12002.02	2.37	137.95	11701.00	1.38	134.49	1.03
42	Hybrid-NuPBODeepOptS-ExactPRnols	2025-06-02	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
43	NuPBO-DeepOpt+	2025-06-02	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
44	NuPBO-DeepOptS+	2025-06-02	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
45	PRINTEMPS	2025-05-28	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
46	PRINTEMPS 20 cores	2025-05-28	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-

Category "optimization problem, linear constraints, no OPT/UNSAT certificate" (OPT-LIN): UNSAT answers

Rank	Solver	Version	Number of solved instances	Detail	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 420
Virtual Best Solver (VBS)			6	6 UNSAT	1%	100%	48382.20	0.12	8063.70	-	-	-	-
1	OR-Tools CP-SAT 20 cores	9.14	5	5 UNSAT	1%	83%	46984.24	0.18	9396.85	2372.95	0.18	474.59	19.80
2	SCIP	2024-07-07	4	4 UNSAT	1%	67%	2634.07	0.33	658.52	2641.86	0.33	660.47	1.00
3	SCIP	2025-06-13	4	4 UNSAT	1%	67%	2656.57	0.28	664.14	2664.51	0.28	666.13	1.00
4	LSIPExact 20 cores	S1	4	4 UNSAT	1%	67%	17082.59	10.17	4270.65	1168.14	10.24	292.04	14.62
5	LSIPExact	S1	3	3 UNSAT	1%	50%	93.91	10.16	31.30	63.40	10.22	21.13	1.48
6	SCIP-NaPS	1.00a2	3	3 UNSAT	1%	50%	179.57	0.96	59.86	180.30	1.06	60.10	1.00
7	ExactPR-S	2025-06-01	3	3 UNSAT	1%	50%	198.11	0.19	66.04	199.01	0.21	66.34	1.00
8	Hybrid-CASHWMaxSATDisjCom+ExactPRS9	2025-06-02	3	3 UNSAT	1%	50%	1329.33	0.21	443.11	1333.55	0.24	444.52	1.00
9	Hybrid-CASHWMaxSATDisjCad+ExactPRS	2025-06-01	3	3 UNSAT	1%	50%	1546.08	0.58	515.36	1550.61	0.62	516.87	1.00
10	Hybrid-CASHWMaxSATDisjCom+ExactPRS	2025-06-01	3	3 UNSAT	1%	50%	1649.26	0.54	549.75	1654.84	0.58	551.61	1.00
11	pb_picat	2025-07-11	3	3 UNSAT	1%	50%	1992.07	0.74	664.02	1997.69	0.75	665.90	1.00
12	roundingsat+pbsuma-opt	63f22ad	2	2 UNSAT	0%	33%	21.32	0.16	10.66	21.46	0.18	10.73	0.99
13	Exact_no_db_no_ls	2025-06-19	2	2 UNSAT	0%	33%	24.90	0.12	12.45	24.97	0.13	12.48	1.00
14	roundingsat+pbsuma-opt-log	0f876b3	2	2 UNSATC	0%	33%	25.16	0.45	12.58	25.65	0.56	12.82	0.98
15	roundingsat-ls+pbsuma	63f22ad	2	2 UNSAT	0%	33%	25.85	0.40	12.93	25.96	0.44	12.98	1.00
16	roundingsat-ls+pbsuma-log	0f876b3	2	2 UNSATC	0%	33%	29.81	0.20	14.91	30.15	0.23	15.08	0.99
17	roundingsat-ihs	63f22ad	2	2 UNSAT	0%	33%	33.49	0.09	16.75	33.66	0.11	16.83	1.00
18	OR-Tools CP-SAT	9.14	2	2 UNSAT	0%	33%	37.35	0.20	18.67	19.66	0.20	9.83	1.90
19	roundingsat-ihs-log	63f22ad	2	2 UNSATC	0%	33%	37.78	0.16	18.89	39.17	0.17	19.58	0.96
20	WMaxCDCL-SCIP	2025-06-21	2	2 UNSAT	0%	33%	49.43	0.12	24.72	49.72	0.16	24.86	0.99
21	mixed-bag-2024	2025-07-10	2	2 UNSAT	0%	33%	54.02	0.30	27.01	54.22	0.31	27.11	1.00
22	LSIPExact	S2	2	2 UNSAT	0%	33%	86.64	10.23	43.32	75.05	10.28	37.52	1.15
23	LSIPExact 20 cores	S2	2	2 UNSAT	0%	33%	136.46	10.15	68.23	68.90	10.24	34.45	1.98
24	ExactPR	2025-06-01	2	2 UNSAT	0%	33%	153.05	0.26	76.52	153.42	0.30	76.71	1.00
25	UWrMaxSat	2025-06-04	2	2 UNSAT	0%	33%	155.46	0.19	77.73	156.03	0.19	78.02	1.00
26	UWrMaxSat-SCIP	2025-06-04	2	2 UNSAT	0%	33%	160.44	0.21	80.22	160.90	0.21	80.45	1.00
27	Sat4j Resolution	2025-06-06	2	2 UNSAT	0%	33%	172.73	0.81	86.37	166.32	0.51	83.16	1.04
28	Sat4j Resolution VeriPB	2025-06-06	2	2 UNSATC	0%	33%	179.29	0.81	89.64	178.82	0.47	89.41	1.00
29	CASHWMaxSATDisjCad-S	2025-06-01	2	2 UNSAT	0%	33%	206.55	0.19	103.28	207.11	0.19	103.56	1.00
30	Exact_no_dombreaking	2025-06-16	2	2 UNSAT	0%	33%	735.18	0.40	367.59	737.19	0.45	368.59	1.00
31	Exact_proof	2025-06-02	2	2 UNSATC	0%	33%	741.40	359.33	370.70	743.18	360.29	371.59	1.00
32	Exact	2025-06-02	2	2 UNSAT	0%	33%	741.57	0.18	370.78	743.38	0.18	371.69	1.00
33	NaPS	naps-1.03a6	2	2 UNSAT	0%	33%	784.12	0.20	392.06	786.59	0.22	393.29	1.00
34	pb-oll-rs-2024	2025-05-06	2	2 UNSAT	0%	33%	1477.27	0.10	738.64	1481.05	0.11	740.53	1.00
35	IPBHS-SCIP	2025-06-09	2	2 UNSAT	0%	33%	1731.01	0.22	865.50	1736.07	0.23	868.03	1.00
36	IPBHS-GUROBI-SYM	2025-06-09	2	2 UNSAT	0%	33%	1741.60	0.50	870.80	1746.38	0.54	873.19	1.00
37	IPBHS-GUROBI	2025-06-09	2	2 UNSAT	0%	33%	1751.16	0.30	875.58	1755.26	0.31	877.63	1.00
38	CASHWMaxSATDisjCom-S	2025-06-01	2	2 UNSAT	0%	33%	2096.92	0.19	1048.46	2103.83	0.20	1051.91	1.00
39	toulbar2	2025-06-13	1	1 UNSAT	0%	17%	141.36	0.46	141.36	141.71	0.47	141.71	1.00
40	Sat4j CP VeriPB	2025-06-06	1	1 UNSATC	0%	17%	482.56	2.07	482.56	476.83	1.15	476.83	1.01
41	Hybrid-NuPBODeepOptS-ExactPRnols	2025-06-02	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
42	NuPBO-DeepOpt+	2025-06-02	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
43	NuPBO-DeepOptS+	2025-06-02	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
44	PRINTEMPS	2025-05-28	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
45	PRINTEMPS 20 cores	2025-05-28	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-
46	Sat4j CP	2025-06-06	0		0%	0%	0.00	1.54	-	0.00	0.91	-	-

Category "optimization problem, non-linear constraints, no OPT/UNSAT certificate" (OPT-NLC): OPT+UNSAT answers

Rank	Solver	Version	Number of solved instances	Detail	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 56
Virtual Best Solver (VBS)			33	33 OPT	59%	100%	9594.67	0.86	290.75	-	-	-	-
1	SCIP	2024-07-07	32	32 OPT	57%	97%	10773.48	5.13	336.67	10805.86	5.16	337.68	1.00
2	SCIP	2025-06-13	31	31 OPT	55%	94%	8871.56	4.32	286.18	8898.78	4.34	287.06	1.00
3	OR-Tools CP-SAT 20 cores	9.14	30	30 OPT	54%	91%	184693.90	1.43	6156.46	9314.28	0.28	310.48	19.83
4	WMaxCDCL-SCIP	2025-06-21	29	29 OPT	52%	88%	5422.50	3.47	186.98	5440.36	3.51	187.60	1.00
5	OR-Tools CP-SAT	9.14	25	25 OPT	45%	76%	1181.11	0.64	47.24	596.09	0.38	23.84	1.98
6	pb_picat	2025-07-11	22	22 OPT	39%	67%	7072.24	0.05	321.47	7092.10	0.05	322.37	1.00
7	Exact	2025-06-02	21	21 OPT	38%	64%	10964.75	359.36	522.13	10991.18	360.04	523.39	1.00
8	Exact_no_db_no_ls	2025-06-19	20	20 OPT	36%	61%	1146.34	0.04	57.32	1149.22	0.04	57.46	1.00
9	Exact_no_dombreaking	2025-06-16	20	20 OPT	36%	61%	8567.79	359.33	428.39	8589.71	360.04	429.49	1.00
10	Sat4j Resolution	2025-06-06	15	15 OPT	27%	45%	200.74	2.16	13.38	184.41	1.14	12.29	1.09
11	Sat4j CP	2025-06-06	9	9 OPT	16%	27%	195.46	0.45	21.72	187.16	0.32	20.80	1.04
12	PRINTEMPS	2025-05-28	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-

Category "optimization problem, non-linear constraints, no OPT/UNSAT certificate" (OPT-NLC): OPT answers

Rank	Solver	Version	Number of solved instances	Detail	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 56
Virtual Best Solver (VBS)			33	33 OPT	59%	100%	9594.67	0.86	290.75	-	-	-	-
1	SCIP	2024-07-07	32	32 OPT	57%	97%	10773.48	5.13	336.67	10805.86	5.16	337.68	1.00
2	SCIP	2025-06-13	31	31 OPT	55%	94%	8871.56	4.32	286.18	8898.78	4.34	287.06	1.00
3	OR-Tools CP-SAT 20 cores	9.14	30	30 OPT	54%	91%	184693.90	1.43	6156.46	9314.28	0.28	310.48	19.83
4	WMaxCDCL-SCIP	2025-06-21	29	29 OPT	52%	88%	5422.50	3.47	186.98	5440.36	3.51	187.60	1.00
5	OR-Tools CP-SAT	9.14	25	25 OPT	45%	76%	1181.11	0.64	47.24	596.09	0.38	23.84	1.98
6	pb_picat	2025-07-11	22	22 OPT	39%	67%	7072.24	0.05	321.47	7092.10	0.05	322.37	1.00
7	Exact	2025-06-02	21	21 OPT	38%	64%	10964.75	359.36	522.13	10991.18	360.04	523.39	1.00
8	Exact_no_db_no_ls	2025-06-19	20	20 OPT	36%	61%	1146.34	0.04	57.32	1149.22	0.04	57.46	1.00
9	Exact_no_dombreaking	2025-06-16	20	20 OPT	36%	61%	8567.79	359.33	428.39	8589.71	360.04	429.49	1.00
10	Sat4j Resolution	2025-06-06	15	15 OPT	27%	45%	200.74	2.16	13.38	184.41	1.14	12.29	1.09
11	Sat4j CP	2025-06-06	9	9 OPT	16%	27%	195.46	0.45	21.72	187.16	0.32	20.80	1.04
12	PRINTEMPS	2025-05-28	0		0%	0%	0.00	0.00	-	0.00	0.00	-	-

Category "optimization problem, non-linear constraints, no OPT/UNSAT certificate" (OPT-NLC): UNSAT answers

Rank	Solver	Version	Number of solved instances	% of all instances	% of VBS	Cumulated CPU time on solved instances	Median CPU time	Average CPU time per solved instance	Cumulated WC time on solved instances	Median WC time	Average WC time per solved instance	CPU/WC
Total number of instances in the category: 56
Virtual Best Solver (VBS)			0	0%	-	0.00	0.00	-	-	-	-	-
1	Exact	2025-06-02	0	0%	-	0.00	0.00	-	0.00	0.00	-	-
2	Exact_no_db_no_ls	2025-06-19	0	0%	-	0.00	0.00	-	0.00	0.00	-	-
3	Exact_no_dombreaking	2025-06-16	0	0%	-	0.00	0.00	-	0.00	0.00	-	-
4	OR-Tools CP-SAT	9.14	0	0%	-	0.00	0.00	-	0.00	0.00	-	-
5	OR-Tools CP-SAT 20 cores	9.14	0	0%	-	0.00	0.00	-	0.00	0.00	-	-
6	PRINTEMPS	2025-05-28	0	0%	-	0.00	0.00	-	0.00	0.00	-	-
7	Sat4j CP	2025-06-06	0	0%	-	0.00	0.00	-	0.00	0.00	-	-
8	Sat4j Resolution	2025-06-06	0	0%	-	0.00	0.00	-	0.00	0.00	-	-
9	SCIP	2024-07-07	0	0%	-	0.00	0.00	-	0.00	0.00	-	-
10	SCIP	2025-06-13	0	0%	-	0.00	0.00	-	0.00	0.00	-	-
11	WMaxCDCL-SCIP	2025-06-21	0	0%	-	0.00	0.00	-	0.00	0.00	-	-
12	pb_picat	2025-07-11	0	0%	-	0.00	0.00	-	0.00	0.00	-	-

Ranking on the number of best answers found (incomplete solver point of view)

Number of best answers found with no further time limit

This section presents the ranking of the different solvers. Solvers are ranked on the number of instances for which they were able to find the best solution obtained by one the solvers within the global time limit. Ties are broken using the cumulated T1 time.

Solvers which gave a wrong answer in a category are excluded from the ranking in that category.

Category "optimization problem, linear constraints, no OPT/UNSAT certificate" (OPT-LIN): answers

Rank	Solver	Version	Number of best solutions found	% of all instances	% of VBS	Cumulated time
Total number of instances in the category: 555
Virtual Best Solver (VBS)			518	93%	100%
1	OR-Tools CP-SAT 20 cores	9.14	403	73%	78%	83399.43
2	Hybrid-CASHWMaxSATDisjCom+ExactPRS9	2025-06-02	391	70%	75%	289209.5
3	Hybrid-CASHWMaxSATDisjCad+ExactPRS	2025-06-01	386	70%	75%	272483.01
4	Hybrid-CASHWMaxSATDisjCom+ExactPRS	2025-06-01	381	69%	74%	266668.6
5	UWrMaxSat-SCIP	2025-06-04	357	64%	69%	73018.62
6	LSIPExact 20 cores	S1	351	63%	68%	73599.48
7	SCIP-NaPS	1.00a2	350	63%	68%	93303.48
8	ExactPR-S	2025-06-01	347	63%	67%	76901.67
9	mixed-bag-2024	2025-07-10	337	61%	65%	45662.51
10	OR-Tools CP-SAT	9.14	330	59%	64%	26167.27
11	roundingsat+pbsuma-opt-log	0f876b3	329	59%	64%	63665.6
12	roundingsat+pbsuma-opt	63f22ad	326	59%	63%	59902.41
13	roundingsat-ls+pbsuma-log	0f876b3	323	58%	62%	61921.46
14	LSIPExact	S1	319	57%	62%	57294.07
15	roundingsat-ls+pbsuma	63f22ad	317	57%	61%	57802.06
16	SCIP	2024-07-07	314	57%	61%	57887.22
17	Exact_no_dombreaking	2025-06-16	311	56%	60%	134052.08
18	Exact	2025-06-02	309	56%	60%	138513.87
19	ExactPR	2025-06-01	309	56%	60%	35870.77
20	SCIP	2025-06-13	307	55%	59%	57094.89
21	Exact_no_db_no_ls	2025-06-19	303	55%	58%	39195.08
22	IPBHS-GUROBI-SYM	2025-06-09	293	53%	57%	56287.89
23	LSIPExact	S2	291	52%	56%	115503.52
24	IPBHS-GUROBI	2025-06-09	291	52%	56%	56146.65
25	WMaxCDCL-SCIP	2025-06-21	290	52%	56%	57923.28
26	Exact_proof	2025-06-02	284	51%	55%	144857.81
27	UWrMaxSat	2025-06-04	280	50%	54%	36940.95
28	CASHWMaxSATDisjCad-S	2025-06-01	278	50%	54%	148603.36
29	CASHWMaxSATDisjCom-S	2025-06-01	277	50%	53%	155030.27
30	LSIPExact 20 cores	S2	273	49%	53%	66181.54
31	IPBHS-SCIP	2025-06-09	271	49%	52%	50293.65
32	pb-oll-rs-2024	2025-05-06	257	46%	50%	44411.64
33	NaPS	naps-1.03a6	244	44%	47%	40122.19
34	pb_picat	2025-07-11	224	40%	43%	38375.6
35	roundingsat-ihs-log	63f22ad	200	36%	39%	43397.85
36	roundingsat-ihs	63f22ad	200	36%	39%	39290.6
37	Sat4j Resolution	2025-06-06	157	28%	30%	20463.94
38	Sat4j Resolution VeriPB	2025-06-06	150	27%	29%	16968.38
39	toulbar2	2025-06-13	145	26%	28%	5041.31
40	Sat4j CP VeriPB	2025-06-06	137	25%	26%	9999.98
41	Sat4j CP	2025-06-06	120	22%	23%	9620.12
42	Hybrid-NuPBODeepOptS-ExactPRnols	2025-06-02	34	6%	7%	122282.06
43	PRINTEMPS	2025-05-28	11	2%	2%	7127.53

Category "optimization problem, non-linear constraints, no OPT/UNSAT certificate" (OPT-NLC): answers

Rank	Solver	Version	Number of best solutions found	% of all instances	% of VBS	Cumulated time
Total number of instances in the category: 57
Virtual Best Solver (VBS)			57	100%	100%
1	SCIP	2024-07-07	35	61%	61%	6012.28
2	SCIP	2025-06-13	34	60%	60%	7125.9
3	OR-Tools CP-SAT	9.14	32	56%	56%	1310.83
4	OR-Tools CP-SAT 20 cores	9.14	30	53%	53%	4.66
5	WMaxCDCL-SCIP	2025-06-21	29	51%	51%	5439.86
6	pb_picat	2025-07-11	22	39%	39%	2932.47
7	Exact	2025-06-02	21	37%	37%	9171.52
8	Exact_no_dombreaking	2025-06-16	20	35%	35%	7837.07
9	Exact_no_db_no_ls	2025-06-19	20	35%	35%	328.84
10	Sat4j Resolution	2025-06-06	15	26%	26%	26.16
11	Sat4j CP	2025-06-06	9	16%	16%	183.67
12	PRINTEMPS	2025-05-28	7	12%	12%	1150.73

Number of best answers found within 2 times the VBS time

This section presents the ranking of the different solvers. Solvers are ranked on the number of instances for which they were able to find the best solution obtained by one the solvers within 2 times the VBS time. Ties are broken using the cumulated T1 time.

Solvers which gave a wrong answer in a category are excluded from the ranking in that category.

Category "optimization problem, linear constraints, no OPT/UNSAT certificate" (OPT-LIN): answers

Rank	Solver	Version	Number of best solutions found	% of all instances	% of VBS	Cumulated time
Total number of instances in the category: 555
Virtual Best Solver (VBS)			518	93%	100%
1	OR-Tools CP-SAT 20 cores	9.14	206	37%	40%	9790.72
2	UWrMaxSat-SCIP	2025-06-04	120	22%	23%	2287.99
3	UWrMaxSat	2025-06-04	109	20%	21%	1239.08
4	Exact_no_db_no_ls	2025-06-19	105	19%	20%	1484.94
5	SCIP	2025-06-13	65	12%	13%	2525.84
6	OR-Tools CP-SAT	9.14	60	11%	12%	4400.48
7	Hybrid-CASHWMaxSATDisjCom+ExactPRS9	2025-06-02	57	10%	11%	21750.52
8	pb-oll-rs-2024	2025-05-06	54	10%	10%	1409.12
9	Hybrid-CASHWMaxSATDisjCom+ExactPRS	2025-06-01	44	8%	8%	3756.17
10	IPBHS-GUROBI	2025-06-09	42	8%	8%	1187.16
11	IPBHS-GUROBI-SYM	2025-06-09	38	7%	7%	1300.27
12	Hybrid-CASHWMaxSATDisjCad+ExactPRS	2025-06-01	35	6%	7%	7450.55
13	SCIP	2024-07-07	31	6%	6%	2458.83
14	CASHWMaxSATDisjCad-S	2025-06-01	31	6%	6%	1107.55
15	CASHWMaxSATDisjCom-S	2025-06-01	24	4%	5%	56.25
16	LSIPExact 20 cores	S1	23	4%	4%	6811.46
17	roundingsat+pbsuma-opt	63f22ad	21	4%	4%	1717.43
18	mixed-bag-2024	2025-07-10	21	4%	4%	112.03
19	Sat4j Resolution	2025-06-06	15	3%	3%	179.09
20	WMaxCDCL-SCIP	2025-06-21	12	2%	2%	575.51
21	Sat4j Resolution VeriPB	2025-06-06	11	2%	2%	130.11
22	roundingsat+pbsuma-opt-log	0f876b3	10	2%	2%	1358.94
23	NaPS	naps-1.03a6	10	2%	2%	538.74
24	LSIPExact	S1	8	1%	2%	3146.86
25	Sat4j CP	2025-06-06	8	1%	2%	15.98
26	toulbar2	2025-06-13	7	1%	1%	0.2
27	Hybrid-NuPBODeepOptS-ExactPRnols	2025-06-02	6	1%	1%	21583.34
28	IPBHS-SCIP	2025-06-09	6	1%	1%	998.51
29	roundingsat-ls+pbsuma	63f22ad	6	1%	1%	312.79
30	pb_picat	2025-07-11	4	1%	1%	2886.51
31	roundingsat-ihs	63f22ad	4	1%	1%	771.95
32	PRINTEMPS	2025-05-28	4	1%	1%	51.52
33	SCIP-NaPS	1.00a2	4	1%	1%	20.01
34	ExactPR	2025-06-01	3	1%	1%	1013.58
35	roundingsat-ihs-log	63f22ad	3	1%	1%	886.48
36	ExactPR-S	2025-06-01	3	1%	1%	781.43
37	roundingsat-ls+pbsuma-log	0f876b3	3	1%	1%	225.19
38	Exact_no_dombreaking	2025-06-16	2	0%	0%	1704.34
39	LSIPExact 20 cores	S2	2	0%	0%	158.99
40	LSIPExact	S2	1	0%	0%	1997.34
41	Exact	2025-06-02	1	0%	0%	903.43

Category "optimization problem, non-linear constraints, no OPT/UNSAT certificate" (OPT-NLC): answers

Rank	Solver	Version	Number of best solutions found	% of all instances	% of VBS	Cumulated time
Total number of instances in the category: 57
Virtual Best Solver (VBS)			57	100%	100%
1	OR-Tools CP-SAT 20 cores	9.14	27	47%	47%	4.26
2	Exact_no_db_no_ls	2025-06-19	12	21%	21%	0.14
3	OR-Tools CP-SAT	9.14	8	14%	14%	284.65
4	PRINTEMPS	2025-05-28	6	11%	11%	945.74
5	pb_picat	2025-07-11	6	11%	11%	0.21
6	SCIP	2025-06-13	5	9%	9%	87.93
7	SCIP	2024-07-07	3	5%	5%	1112.15
8	WMaxCDCL-SCIP	2025-06-21	1	2%	2%	116.79

Number of best answers found within 4 times the VBS time

This section presents the ranking of the different solvers. Solvers are ranked on the number of instances for which they were able to find the best solution obtained by one the solvers within 4 times the VBS time. Ties are broken using the cumulated T1 time.

Solvers which gave a wrong answer in a category are excluded from the ranking in that category.

Category "optimization problem, linear constraints, no OPT/UNSAT certificate" (OPT-LIN): answers

Rank	Solver	Version	Number of best solutions found	% of all instances	% of VBS	Cumulated time
Total number of instances in the category: 555
Virtual Best Solver (VBS)			518	93%	100%
1	OR-Tools CP-SAT 20 cores	9.14	276	50%	53%	10739.12
2	UWrMaxSat-SCIP	2025-06-04	152	27%	29%	9448.5
3	UWrMaxSat	2025-06-04	148	27%	29%	8222.49
4	Exact_no_db_no_ls	2025-06-19	133	24%	26%	3022.62
5	OR-Tools CP-SAT	9.14	128	23%	25%	6668.67
6	SCIP	2025-06-13	100	18%	19%	6429.19
7	IPBHS-GUROBI	2025-06-09	91	16%	18%	2339.9
8	Hybrid-CASHWMaxSATDisjCom+ExactPRS9	2025-06-02	80	14%	15%	27928.25
9	Hybrid-CASHWMaxSATDisjCom+ExactPRS	2025-06-01	78	14%	15%	9341.03
10	SCIP	2024-07-07	73	13%	14%	5479.9
11	IPBHS-GUROBI-SYM	2025-06-09	72	13%	14%	2421.93
12	pb-oll-rs-2024	2025-05-06	71	13%	14%	1947.04
13	roundingsat+pbsuma-opt	63f22ad	70	13%	14%	4107.03
14	Hybrid-CASHWMaxSATDisjCad+ExactPRS	2025-06-01	61	11%	12%	13387.04
15	mixed-bag-2024	2025-07-10	53	10%	10%	2423.87
16	CASHWMaxSATDisjCad-S	2025-06-01	44	8%	8%	1261.64
17	roundingsat-ls+pbsuma	63f22ad	43	8%	8%	3938.25
18	roundingsat+pbsuma-opt-log	0f876b3	41	7%	8%	4523.8
19	CASHWMaxSATDisjCom-S	2025-06-01	37	7%	7%	109.1
20	LSIPExact 20 cores	S1	32	6%	6%	8813.21
21	NaPS	naps-1.03a6	31	6%	6%	5608.2
22	IPBHS-SCIP	2025-06-09	28	5%	5%	2140.06
23	Sat4j Resolution	2025-06-06	25	5%	5%	1230.5
24	roundingsat-ls+pbsuma-log	0f876b3	24	4%	5%	4246.05
25	roundingsat-ihs	63f22ad	21	4%	4%	1390.24
26	ExactPR-S	2025-06-01	20	4%	4%	1017.62
27	WMaxCDCL-SCIP	2025-06-21	20	4%	4%	699.57
28	Sat4j Resolution VeriPB	2025-06-06	20	4%	4%	141.6
29	LSIPExact	S1	19	3%	4%	7257.1
30	roundingsat-ihs-log	63f22ad	18	3%	3%	1567.21
31	pb_picat	2025-07-11	17	3%	3%	2964.72
32	SCIP-NaPS	1.00a2	15	3%	3%	5474
33	Sat4j CP	2025-06-06	12	2%	2%	23.2
34	toulbar2	2025-06-13	12	2%	2%	0.74
35	ExactPR	2025-06-01	9	2%	2%	2862.07
36	Hybrid-NuPBODeepOptS-ExactPRnols	2025-06-02	8	1%	2%	28775.61
37	PRINTEMPS	2025-05-28	4	1%	1%	51.52
38	Exact_no_dombreaking	2025-06-16	3	1%	1%	2232.88
39	Exact_proof	2025-06-02	2	0%	0%	2212.93
40	LSIPExact	S2	2	0%	0%	2019.9
41	LSIPExact 20 cores	S2	2	0%	0%	158.99
42	Exact	2025-06-02	1	0%	0%	903.43

Category "optimization problem, non-linear constraints, no OPT/UNSAT certificate" (OPT-NLC): answers

Rank	Solver	Version	Number of best solutions found	% of all instances	% of VBS	Cumulated time
Total number of instances in the category: 57
Virtual Best Solver (VBS)			57	100%	100%
1	OR-Tools CP-SAT 20 cores	9.14	30	53%	53%	4.66
2	Exact_no_db_no_ls	2025-06-19	16	28%	28%	2.76
3	OR-Tools CP-SAT	9.14	11	19%	19%	285.68
4	pb_picat	2025-07-11	7	12%	12%	0.24
5	PRINTEMPS	2025-05-28	6	11%	11%	945.74
6	SCIP	2025-06-13	5	9%	9%	87.93
7	SCIP	2024-07-07	3	5%	5%	1112.15
8	WMaxCDCL-SCIP	2025-06-21	1	2%	2%	116.79
9	Sat4j Resolution	2025-06-06	1	2%	2%	0.86

Number of best answers found within 8 times the VBS time

This section presents the ranking of the different solvers. Solvers are ranked on the number of instances for which they were able to find the best solution obtained by one the solvers within 8 times the VBS time. Ties are broken using the cumulated T1 time.

Solvers which gave a wrong answer in a category are excluded from the ranking in that category.

Category "optimization problem, linear constraints, no OPT/UNSAT certificate" (OPT-LIN): answers

Rank	Solver	Version	Number of best solutions found	% of all instances	% of VBS	Cumulated time
Total number of instances in the category: 555
Virtual Best Solver (VBS)			518	93%	100%
1	OR-Tools CP-SAT 20 cores	9.14	311	56%	60%	13806.38
2	OR-Tools CP-SAT	9.14	190	34%	37%	9572.49
3	UWrMaxSat-SCIP	2025-06-04	172	31%	33%	15327.6
4	UWrMaxSat	2025-06-04	167	30%	32%	13378.01
5	Exact_no_db_no_ls	2025-06-19	157	28%	30%	7657.46
6	SCIP	2025-06-13	132	24%	25%	7755.45
7	IPBHS-GUROBI	2025-06-09	113	20%	22%	4649.85
8	IPBHS-GUROBI-SYM	2025-06-09	109	20%	21%	4747.89
9	roundingsat+pbsuma-opt	63f22ad	108	19%	21%	5735.42
10	SCIP	2024-07-07	105	19%	20%	7740.03
11	Hybrid-CASHWMaxSATDisjCom+ExactPRS9	2025-06-02	101	18%	19%	29862.13
12	pb-oll-rs-2024	2025-05-06	99	18%	19%	6446.81
13	Hybrid-CASHWMaxSATDisjCom+ExactPRS	2025-06-01	96	17%	19%	9463.13
14	Hybrid-CASHWMaxSATDisjCad+ExactPRS	2025-06-01	94	17%	18%	13628.23
15	roundingsat-ls+pbsuma	63f22ad	91	16%	18%	5153.42
16	roundingsat+pbsuma-opt-log	0f876b3	79	14%	15%	5797.1
17	mixed-bag-2024	2025-07-10	78	14%	15%	3833.88
18	roundingsat-ls+pbsuma-log	0f876b3	77	14%	15%	5463.77
19	CASHWMaxSATDisjCad-S	2025-06-01	60	11%	12%	5859.04
20	CASHWMaxSATDisjCom-S	2025-06-01	60	11%	12%	740.61
21	IPBHS-SCIP	2025-06-09	56	10%	11%	5688.28
22	NaPS	naps-1.03a6	54	10%	10%	6237.19
23	ExactPR-S	2025-06-01	45	8%	9%	3564.96
24	LSIPExact 20 cores	S1	43	8%	8%	10617.57
25	Sat4j Resolution	2025-06-06	38	7%	7%	1255.18
26	SCIP-NaPS	1.00a2	37	7%	7%	11186.63
27	WMaxCDCL-SCIP	2025-06-21	37	7%	7%	3729.59
28	roundingsat-ihs	63f22ad	33	6%	6%	2459.63
29	Sat4j Resolution VeriPB	2025-06-06	32	6%	6%	1625.83
30	LSIPExact	S1	30	5%	6%	8538.29
31	roundingsat-ihs-log	63f22ad	30	5%	6%	2693.06
32	pb_picat	2025-07-11	29	5%	6%	6352.96
33	toulbar2	2025-06-13	21	4%	4%	5.03
34	Sat4j CP	2025-06-06	19	3%	4%	51.12
35	ExactPR	2025-06-01	18	3%	3%	4396.47
36	Exact_no_dombreaking	2025-06-16	9	2%	2%	5750.59
37	Hybrid-NuPBODeepOptS-ExactPRnols	2025-06-02	8	1%	2%	28775.61
38	Exact	2025-06-02	8	1%	2%	5815.78
39	Exact_proof	2025-06-02	6	1%	1%	4330.71
40	Sat4j CP VeriPB	2025-06-06	5	1%	1%	17.23
41	PRINTEMPS	2025-05-28	4	1%	1%	51.52
42	LSIPExact	S2	2	0%	0%	2019.9
43	LSIPExact 20 cores	S2	2	0%	0%	158.99

Category "optimization problem, non-linear constraints, no OPT/UNSAT certificate" (OPT-NLC): answers

Rank	Solver	Version	Number of best solutions found	% of all instances	% of VBS	Cumulated time
Total number of instances in the category: 57
Virtual Best Solver (VBS)			57	100%	100%
1	OR-Tools CP-SAT 20 cores	9.14	30	53%	53%	4.66
2	Exact_no_db_no_ls	2025-06-19	18	32%	32%	6.16
3	OR-Tools CP-SAT	9.14	14	25%	25%	290.26
4	pb_picat	2025-07-11	7	12%	12%	0.24
5	PRINTEMPS	2025-05-28	6	11%	11%	945.74
6	SCIP	2025-06-13	6	11%	11%	91.75
7	SCIP	2024-07-07	3	5%	5%	1112.15
8	WMaxCDCL-SCIP	2025-06-21	2	4%	4%	120.28
9	Sat4j CP	2025-06-06	1	2%	2%	1.1
10	Sat4j Resolution	2025-06-06	1	2%	2%	0.86

PB'25 competition: ranking of solvers Ranking on the subset of instances supported by all solvers max WC time for any solver=3605 s

Ranking on final answers (complete solver point of view)

Category "decision problem, linear constraints, no UNSAT certificate" (DEC-LIN): SAT+UNSAT answers

Category "decision problem, linear constraints, no UNSAT certificate" (DEC-LIN): SAT answers

Category "decision problem, linear constraints, no UNSAT certificate" (DEC-LIN): UNSAT answers

Category "decision problem, non-linear constraints, no UNSAT certificate" (DEC-NLC): SAT+UNSAT answers

Category "decision problem, non-linear constraints, no UNSAT certificate" (DEC-NLC): SAT answers

Category "decision problem, non-linear constraints, no UNSAT certificate" (DEC-NLC): UNSAT answers

Category "optimization problem, linear constraints, no OPT/UNSAT certificate" (OPT-LIN): OPT+UNSAT answers

Category "optimization problem, linear constraints, no OPT/UNSAT certificate" (OPT-LIN): OPT answers

Category "optimization problem, linear constraints, no OPT/UNSAT certificate" (OPT-LIN): UNSAT answers

Category "optimization problem, non-linear constraints, no OPT/UNSAT certificate" (OPT-NLC): OPT+UNSAT answers

Category "optimization problem, non-linear constraints, no OPT/UNSAT certificate" (OPT-NLC): OPT answers

Category "optimization problem, non-linear constraints, no OPT/UNSAT certificate" (OPT-NLC): UNSAT answers

Ranking on the number of best answers found (incomplete solver point of view)

Number of best answers found with no further time limit

Category "optimization problem, linear constraints, no OPT/UNSAT certificate" (OPT-LIN): answers

Category "optimization problem, non-linear constraints, no OPT/UNSAT certificate" (OPT-NLC): answers

Number of best answers found within 2 times the VBS time

Category "optimization problem, linear constraints, no OPT/UNSAT certificate" (OPT-LIN): answers

Category "optimization problem, non-linear constraints, no OPT/UNSAT certificate" (OPT-NLC): answers

Number of best answers found within 4 times the VBS time

Category "optimization problem, linear constraints, no OPT/UNSAT certificate" (OPT-LIN): answers

Category "optimization problem, non-linear constraints, no OPT/UNSAT certificate" (OPT-NLC): answers

Number of best answers found within 8 times the VBS time

Category "optimization problem, linear constraints, no OPT/UNSAT certificate" (OPT-LIN): answers

Category "optimization problem, non-linear constraints, no OPT/UNSAT certificate" (OPT-NLC): answers

PB'25 competition: ranking of solvers
Ranking on the subset of instances supported by all solvers
max WC time for any solver=3605 s