PB'24 competition: WBO track: solvers page (tabular format, summary)

Some explanations

Category "WBO, both soft and hard constraints, linear constraints" (PARTIAL-LIN)

Solver Name#benchsSolvedUNSATMOPTSATMSATUNKNOWNIncomplete AnswersBad Answers
Exact
2024-07-05
(complete)
208155
(74.52%)
0
(0%)
155
(74.52%)
0
(0%)
45
(21.63%)
8
(3.85%)
0
(0%)
0
(0%)
Exact_no_SoPlex
2024-07-05
(complete)
208156
(75%)
0
(0%)
156
(75%)
0
(0%)
46
(22.12%)
6
(2.88%)
0
(0%)
0
(0%)
FiberSCIP
2024-07-02
(complete)
208155
(74.52%)
0
(0%)
155
(74.52%)
0
(0%)
16
(7.69%)
37
(17.79%)
0
(0%)
0
(0%)
FiberSCIP
2024-07-30
(complete)
208155
(74.52%)
0
(0%)
155
(74.52%)
0
(0%)
16
(7.69%)
37
(17.79%)
0
(0%)
0
(0%)
FiberSCIP
20 cores, 2024-07-30
(complete)
208160
(76.92%)
0
(0%)
160
(76.92%)
0
(0%)
0
(0%)
48
(23.08%)
0
(0%)
0
(0%)
NaPS
1.03
(complete)
208139
(66.83%)
0
(0%)
139
(66.83%)
0
(0%)
42
(20.19%)
18
(8.65%)
0
(0%)
9
(4.33%)
NaPS
1.03a1
(complete)
208139
(66.83%)
0
(0%)
139
(66.83%)
0
(0%)
42
(20.19%)
18
(8.65%)
0
(0%)
9
(4.33%)
NaPS
1.03a2
(complete)
208139
(66.83%)
0
(0%)
139
(66.83%)
0
(0%)
43
(20.67%)
26
(12.5%)
0
(0%)
0
(0%)
NaPS-GM
1.03
(complete)
208139
(66.83%)
0
(0%)
139
(66.83%)
0
(0%)
45
(21.63%)
12
(5.77%)
0
(0%)
12
(5.77%)
NaPS-GM
1.03a1
(complete)
208138
(66.35%)
0
(0%)
138
(66.35%)
0
(0%)
46
(22.12%)
12
(5.77%)
0
(0%)
12
(5.77%)
NaPS-GM
1.03a2
(complete)
208138
(66.35%)
0
(0%)
138
(66.35%)
0
(0%)
46
(22.12%)
24
(11.54%)
0
(0%)
0
(0%)
NaPS-PB16
1.02b5
(complete)
208147
(70.67%)
0
(0%)
147
(70.67%)
0
(0%)
36
(17.31%)
25
(12.02%)
0
(0%)
0
(0%)
Sat4j Cutting Planes 2024
2024-07-12
(complete)
20878
(37.5%)
0
(0%)
78
(37.5%)
0
(0%)
44
(21.15%)
86
(41.35%)
0
(0%)
0
(0%)
Sat4j Partial RoundingSAT 2024
2024-07-12
(complete)
208108
(51.92%)
0
(0%)
108
(51.92%)
0
(0%)
75
(36.06%)
25
(12.02%)
0
(0%)
0
(0%)
Sat4j Resolution Default
2024-07-07
(complete)
208135
(64.9%)
0
(0%)
135
(64.9%)
0
(0%)
58
(27.88%)
15
(7.21%)
0
(0%)
0
(0%)
SCIP
2024-07-07
(complete)
208156
(75%)
0
(0%)
156
(75%)
0
(0%)
19
(9.13%)
33
(15.87%)
0
(0%)
0
(0%)

Solver NameProgress
Exact
2024-07-05
(complete)
done
208
MOPT
155
MSAT
45
?
8
Exact_no_SoPlex
2024-07-05
(complete)
done
208
MOPT
156
MSAT
46
?
6
FiberSCIP
2024-07-02
(complete)
done
208
MOPT
155
MSAT
16
?
37
FiberSCIP
2024-07-30
(complete)
done
208
MOPT
155
MSAT
16
?
37
FiberSCIP
20 cores, 2024-07-30
(complete)
done
208
MOPT
160
?
48
NaPS
1.03
(complete)
done
208
MOPT
139
MSAT
42
?
18
ERR
9
NaPS
1.03a1
(complete)
done
208
MOPT
139
MSAT
42
?
18
ERR
9
NaPS
1.03a2
(complete)
done
208
MOPT
139
MSAT
43
?
26
NaPS-GM
1.03
(complete)
done
208
MOPT
139
MSAT
45
?
12
ERR
12
NaPS-GM
1.03a1
(complete)
done
208
MOPT
138
MSAT
46
?
12
ERR
12
NaPS-GM
1.03a2
(complete)
done
208
MOPT
138
MSAT
46
?
24
NaPS-PB16
1.02b5
(complete)
done
208
MOPT
147
MSAT
36
?
25
Sat4j Cutting Planes 2024
2024-07-12
(complete)
done
208
MOPT
78
MSAT
44
?
86
Sat4j Partial RoundingSAT 2024
2024-07-12
(complete)
done
208
MOPT
108
MSAT
75
?
25
Sat4j Resolution Default
2024-07-07
(complete)
done
208
MOPT
135
MSAT
58
?
15
SCIP
2024-07-07
(complete)
done
208
MOPT
156
MSAT
19
?
33

Category "WBO, only soft constraints, linear constraints" (SOFT-LIN)

Solver Name#benchsSolvedUNSATMOPTSATMSATUNKNOWNIncomplete AnswersBad Answers
Exact
2024-07-05
(complete)
6046
(76.67%)
0
(0%)
46
(76.67%)
0
(0%)
14
(23.33%)
0
(0%)
0
(0%)
0
(0%)
Exact_no_SoPlex
2024-07-05
(complete)
6044
(73.33%)
0
(0%)
44
(73.33%)
0
(0%)
16
(26.67%)
0
(0%)
0
(0%)
0
(0%)
FiberSCIP
2024-07-02
(complete)
6054
(90%)
0
(0%)
54
(90%)
0
(0%)
5
(8.33%)
1
(1.67%)
0
(0%)
0
(0%)
FiberSCIP
2024-07-30
(complete)
6054
(90%)
0
(0%)
54
(90%)
0
(0%)
6
(10%)
0
(0%)
0
(0%)
0
(0%)
FiberSCIP
20 cores, 2024-07-30
(complete)
6055
(91.67%)
0
(0%)
55
(91.67%)
0
(0%)
0
(0%)
5
(8.33%)
0
(0%)
0
(0%)
NaPS
1.03
(complete)
6038
(63.33%)
0
(0%)
38
(63.33%)
0
(0%)
19
(31.67%)
3
(5%)
0
(0%)
0
(0%)
NaPS
1.03a1
(complete)
6038
(63.33%)
0
(0%)
38
(63.33%)
0
(0%)
19
(31.67%)
3
(5%)
0
(0%)
0
(0%)
NaPS
1.03a2
(complete)
6038
(63.33%)
0
(0%)
38
(63.33%)
0
(0%)
19
(31.67%)
3
(5%)
0
(0%)
0
(0%)
NaPS-GM
1.03
(complete)
6037
(61.67%)
0
(0%)
37
(61.67%)
0
(0%)
18
(30%)
5
(8.33%)
0
(0%)
0
(0%)
NaPS-GM
1.03a1
(complete)
6037
(61.67%)
0
(0%)
37
(61.67%)
0
(0%)
18
(30%)
5
(8.33%)
0
(0%)
0
(0%)
NaPS-GM
1.03a2
(complete)
6037
(61.67%)
0
(0%)
37
(61.67%)
0
(0%)
18
(30%)
5
(8.33%)
0
(0%)
0
(0%)
NaPS-PB16
1.02b5
(complete)
6038
(63.33%)
0
(0%)
38
(63.33%)
0
(0%)
21
(35%)
1
(1.67%)
0
(0%)
0
(0%)
Sat4j Cutting Planes 2024
2024-07-12
(complete)
6027
(45%)
0
(0%)
27
(45%)
0
(0%)
1
(1.67%)
32
(53.33%)
0
(0%)
0
(0%)
Sat4j Partial RoundingSAT 2024
2024-07-12
(complete)
6047
(78.33%)
0
(0%)
47
(78.33%)
0
(0%)
13
(21.67%)
0
(0%)
0
(0%)
0
(0%)
Sat4j Resolution Default
2024-07-07
(complete)
6041
(68.33%)
0
(0%)
41
(68.33%)
0
(0%)
17
(28.33%)
2
(3.33%)
0
(0%)
0
(0%)
SCIP
2024-07-07
(complete)
6053
(88.33%)
0
(0%)
53
(88.33%)
0
(0%)
6
(10%)
1
(1.67%)
0
(0%)
0
(0%)

Solver NameProgress
Exact
2024-07-05
(complete)
done
60
MOPT
46
MSAT
14
Exact_no_SoPlex
2024-07-05
(complete)
done
60
MOPT
44
MSAT
16
FiberSCIP
2024-07-02
(complete)
done
60
MOPT
54
MSAT
5
?
1
FiberSCIP
2024-07-30
(complete)
done
60
MOPT
54
MSAT
6
FiberSCIP
20 cores, 2024-07-30
(complete)
done
60
MOPT
55
?
5
NaPS
1.03
(complete)
done
60
MOPT
38
MSAT
19
?
3
NaPS
1.03a1
(complete)
done
60
MOPT
38
MSAT
19
?
3
NaPS
1.03a2
(complete)
done
60
MOPT
38
MSAT
19
?
3
NaPS-GM
1.03
(complete)
done
60
MOPT
37
MSAT
18
?
5
NaPS-GM
1.03a1
(complete)
done
60
MOPT
37
MSAT
18
?
5
NaPS-GM
1.03a2
(complete)
done
60
MOPT
37
MSAT
18
?
5
NaPS-PB16
1.02b5
(complete)
done
60
MOPT
38
MSAT
21
?
1
Sat4j Cutting Planes 2024
2024-07-12
(complete)
done
60
MOPT
27
MSAT
1
?
32
Sat4j Partial RoundingSAT 2024
2024-07-12
(complete)
done
60
MOPT
47
MSAT
13
Sat4j Resolution Default
2024-07-07
(complete)
done
60
MOPT
41
MSAT
17
?
2
SCIP
2024-07-07
(complete)
done
60
MOPT
53
MSAT
6
?
1

All categories for solvers registered in all categories

Solver Name#benchsSolvedUNSATMOPTSATMSATUNKNOWNIncomplete AnswersBad Answers
Exact
2024-07-05
(complete)
268201
(75%)
0
(0%)
201
(75%)
0
(0%)
59
(22.01%)
8
(2.99%)
0
(0%)
0
(0%)
Exact_no_SoPlex
2024-07-05
(complete)
268200
(74.63%)
0
(0%)
200
(74.63%)
0
(0%)
62
(23.13%)
6
(2.24%)
0
(0%)
0
(0%)
FiberSCIP
2024-07-02
(complete)
268209
(77.99%)
0
(0%)
209
(77.99%)
0
(0%)
21
(7.84%)
38
(14.18%)
0
(0%)
0
(0%)
FiberSCIP
2024-07-30
(complete)
268209
(77.99%)
0
(0%)
209
(77.99%)
0
(0%)
22
(8.21%)
37
(13.81%)
0
(0%)
0
(0%)
FiberSCIP
20 cores, 2024-07-30
(complete)
268215
(80.22%)
0
(0%)
215
(80.22%)
0
(0%)
0
(0%)
53
(19.78%)
0
(0%)
0
(0%)
NaPS
1.03
(complete)
268177
(66.04%)
0
(0%)
177
(66.04%)
0
(0%)
61
(22.76%)
21
(7.84%)
0
(0%)
9
(3.36%)
NaPS
1.03a1
(complete)
268177
(66.04%)
0
(0%)
177
(66.04%)
0
(0%)
61
(22.76%)
21
(7.84%)
0
(0%)
9
(3.36%)
NaPS
1.03a2
(complete)
268177
(66.04%)
0
(0%)
177
(66.04%)
0
(0%)
62
(23.13%)
29
(10.82%)
0
(0%)
0
(0%)
NaPS-GM
1.03
(complete)
268176
(65.67%)
0
(0%)
176
(65.67%)
0
(0%)
63
(23.51%)
17
(6.34%)
0
(0%)
12
(4.48%)
NaPS-GM
1.03a1
(complete)
268175
(65.3%)
0
(0%)
175
(65.3%)
0
(0%)
64
(23.88%)
17
(6.34%)
0
(0%)
12
(4.48%)
NaPS-GM
1.03a2
(complete)
268175
(65.3%)
0
(0%)
175
(65.3%)
0
(0%)
64
(23.88%)
29
(10.82%)
0
(0%)
0
(0%)
NaPS-PB16
1.02b5
(complete)
268185
(69.03%)
0
(0%)
185
(69.03%)
0
(0%)
57
(21.27%)
26
(9.7%)
0
(0%)
0
(0%)
Sat4j Cutting Planes 2024
2024-07-12
(complete)
268105
(39.18%)
0
(0%)
105
(39.18%)
0
(0%)
45
(16.79%)
118
(44.03%)
0
(0%)
0
(0%)
Sat4j Partial RoundingSAT 2024
2024-07-12
(complete)
268155
(57.84%)
0
(0%)
155
(57.84%)
0
(0%)
88
(32.84%)
25
(9.33%)
0
(0%)
0
(0%)
Sat4j Resolution Default
2024-07-07
(complete)
268176
(65.67%)
0
(0%)
176
(65.67%)
0
(0%)
75
(27.99%)
17
(6.34%)
0
(0%)
0
(0%)
SCIP
2024-07-07
(complete)
268209
(77.99%)
0
(0%)
209
(77.99%)
0
(0%)
25
(9.33%)
34
(12.69%)
0
(0%)
0
(0%)

Solver NameProgress
Exact
2024-07-05
(complete)
done
268
MOPT
201
MSAT
59
?
8
Exact_no_SoPlex
2024-07-05
(complete)
done
268
MOPT
200
MSAT
62
?
6
FiberSCIP
2024-07-02
(complete)
done
268
MOPT
209
MSAT
21
?
38
FiberSCIP
2024-07-30
(complete)
done
268
MOPT
209
MSAT
22
?
37
FiberSCIP
20 cores, 2024-07-30
(complete)
done
268
MOPT
215
?
53
NaPS
1.03
(complete)
done
268
MOPT
177
MSAT
61
?
21
ERR
9
NaPS
1.03a1
(complete)
done
268
MOPT
177
MSAT
61
?
21
ERR
9
NaPS
1.03a2
(complete)
done
268
MOPT
177
MSAT
62
?
29
NaPS-GM
1.03
(complete)
done
268
MOPT
176
MSAT
63
?
17
ERR
12
NaPS-GM
1.03a1
(complete)
done
268
MOPT
175
MSAT
64
?
17
ERR
12
NaPS-GM
1.03a2
(complete)
done
268
MOPT
175
MSAT
64
?
29
NaPS-PB16
1.02b5
(complete)
done
268
MOPT
185
MSAT
57
?
26
Sat4j Cutting Planes 2024
2024-07-12
(complete)
done
268
MOPT
105
MSAT
45
?
118
Sat4j Partial RoundingSAT 2024
2024-07-12
(complete)
done
268
MOPT
155
MSAT
88
?
25
Sat4j Resolution Default
2024-07-07
(complete)
done
268
MOPT
176
MSAT
75
?
17
SCIP
2024-07-07
(complete)
done
268
MOPT
209
MSAT
25
?
34

All categories and all solvers (regardless of their ability to support all categories)

This table summarizes the results in all categories for all solvers. Note however that some solvers may not have support for some categories. Therefore, the solvers may have run on a different number of benchmarks and this table should not be used to compare them.

Solver Name#benchsSolvedUNSATMOPTSATMSATUNKNOWNIncomplete AnswersBad Answers
Exact
2024-07-05
(complete)
268201
(75%)
0
(0%)
201
(75%)
0
(0%)
59
(22.01%)
8
(2.99%)
0
(0%)
0
(0%)
Exact_no_SoPlex
2024-07-05
(complete)
268200
(74.63%)
0
(0%)
200
(74.63%)
0
(0%)
62
(23.13%)
6
(2.24%)
0
(0%)
0
(0%)
FiberSCIP
2024-07-02
(complete)
268209
(77.99%)
0
(0%)
209
(77.99%)
0
(0%)
21
(7.84%)
38
(14.18%)
0
(0%)
0
(0%)
FiberSCIP
2024-07-30
(complete)
268209
(77.99%)
0
(0%)
209
(77.99%)
0
(0%)
22
(8.21%)
37
(13.81%)
0
(0%)
0
(0%)
FiberSCIP
20 cores, 2024-07-30
(complete)
268215
(80.22%)
0
(0%)
215
(80.22%)
0
(0%)
0
(0%)
53
(19.78%)
0
(0%)
0
(0%)
NaPS
1.03
(complete)
268177
(66.04%)
0
(0%)
177
(66.04%)
0
(0%)
61
(22.76%)
21
(7.84%)
0
(0%)
9
(3.36%)
NaPS
1.03a1
(complete)
268177
(66.04%)
0
(0%)
177
(66.04%)
0
(0%)
61
(22.76%)
21
(7.84%)
0
(0%)
9
(3.36%)
NaPS
1.03a2
(complete)
268177
(66.04%)
0
(0%)
177
(66.04%)
0
(0%)
62
(23.13%)
29
(10.82%)
0
(0%)
0
(0%)
NaPS-GM
1.03
(complete)
268176
(65.67%)
0
(0%)
176
(65.67%)
0
(0%)
63
(23.51%)
17
(6.34%)
0
(0%)
12
(4.48%)
NaPS-GM
1.03a1
(complete)
268175
(65.3%)
0
(0%)
175
(65.3%)
0
(0%)
64
(23.88%)
17
(6.34%)
0
(0%)
12
(4.48%)
NaPS-GM
1.03a2
(complete)
268175
(65.3%)
0
(0%)
175
(65.3%)
0
(0%)
64
(23.88%)
29
(10.82%)
0
(0%)
0
(0%)
NaPS-PB16
1.02b5
(complete)
268185
(69.03%)
0
(0%)
185
(69.03%)
0
(0%)
57
(21.27%)
26
(9.7%)
0
(0%)
0
(0%)
Sat4j Cutting Planes 2024
2024-07-12
(complete)
268105
(39.18%)
0
(0%)
105
(39.18%)
0
(0%)
45
(16.79%)
118
(44.03%)
0
(0%)
0
(0%)
Sat4j Partial RoundingSAT 2024
2024-07-12
(complete)
268155
(57.84%)
0
(0%)
155
(57.84%)
0
(0%)
88
(32.84%)
25
(9.33%)
0
(0%)
0
(0%)
Sat4j Resolution Default
2024-07-07
(complete)
268176
(65.67%)
0
(0%)
176
(65.67%)
0
(0%)
75
(27.99%)
17
(6.34%)
0
(0%)
0
(0%)
SCIP
2024-07-07
(complete)
268209
(77.99%)
0
(0%)
209
(77.99%)
0
(0%)
25
(9.33%)
34
(12.69%)
0
(0%)
0
(0%)

Solver NameProgress
Exact
2024-07-05
(complete)
done
268
MOPT
201
MSAT
59
?
8
Exact_no_SoPlex
2024-07-05
(complete)
done
268
MOPT
200
MSAT
62
?
6
FiberSCIP
2024-07-02
(complete)
done
268
MOPT
209
MSAT
21
?
38
FiberSCIP
2024-07-30
(complete)
done
268
MOPT
209
MSAT
22
?
37
FiberSCIP
20 cores, 2024-07-30
(complete)
done
268
MOPT
215
?
53
NaPS
1.03
(complete)
done
268
MOPT
177
MSAT
61
?
21
ERR
9
NaPS
1.03a1
(complete)
done
268
MOPT
177
MSAT
61
?
21
ERR
9
NaPS
1.03a2
(complete)
done
268
MOPT
177
MSAT
62
?
29
NaPS-GM
1.03
(complete)
done
268
MOPT
176
MSAT
63
?
17
ERR
12
NaPS-GM
1.03a1
(complete)
done
268
MOPT
175
MSAT
64
?
17
ERR
12
NaPS-GM
1.03a2
(complete)
done
268
MOPT
175
MSAT
64
?
29
NaPS-PB16
1.02b5
(complete)
done
268
MOPT
185
MSAT
57
?
26
Sat4j Cutting Planes 2024
2024-07-12
(complete)
done
268
MOPT
105
MSAT
45
?
118
Sat4j Partial RoundingSAT 2024
2024-07-12
(complete)
done
268
MOPT
155
MSAT
88
?
25
Sat4j Resolution Default
2024-07-07
(complete)
done
268
MOPT
176
MSAT
75
?
17
SCIP
2024-07-07
(complete)
done
268
MOPT
209
MSAT
25
?
34