MAX-CSP 2006 Competition: results by benchmark

Results by benchmark for category n-ary constraints in extension (N-ARY-EXT), subcategory real world problems

This page displays the results of the different solvers for each benchmark for category n-ary constraints in extension (N-ARY-EXT), subcategory real world problems

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell exampleMeaning
AnswerSolver result
f=...value of the objective function for the model reported by the solver
TT=...Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out
Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

AbbreviationMeaning
f=...Value of the objective function
TOTime Out
MOMem. Out (out of memory)

Meaning of the different colors:

ColorMeaning
textthe solver cannot handle this instance
textthe solver gave no answer
textthe solver could give an answer (SAT)
textthe solver gave a definitive answer (OPTIMUM FOUND or UNSAT)
textthe solver performed better than the other ones on that instance
textthe solver was ended by a signal
textthe solver gave an incomplete answer
textthe solver was disqualified in the category
textthe solver gave a wrong answer

For better readability, you may choose to hide some solvers:
AbsconMax 109 EPFC
AbsconMax 109 PFC
aolibdvo 2007-01-17
aolibpvo 2007-01-17
CSP4J - MaxCSP 2006-12-19
toolbar 2007-01-12
Toolbar_BTD 2007-01-12
Toolbar_MaxSat 2007-01-19
Toulbar2 2007-01-12

Bench nameBest results
on this
instance		AbsconMax
109 EPFC		AbsconMax
109 PFC		aolibdvo
2007-01-17		aolibpvo
2007-01-17		CSP4J - MaxCSP
2006-12-19		toolbar
2007-01-12		Toolbar_BTD
2007-01-12		Toolbar_MaxSat
2007-01-19		Toulbar2
2007-01-12
MaxCSP/
pedigree/
connell_ext.xml		 MOPT
f=14
TT=0.009

 OPTIMUM
f=14
TT=0.663
T1=0.51
 OPTIMUM
f=14
TT=0.69
T1=0.54
 OPTIMUM
f=14
TT=0.009
T1=0.01
 OPTIMUM
f=14
TT=0.009
T1=0.01
 MSAT (TO)
f=14
TT=2400.06
T1=0.33
 OPTIMUM
f=14
TT=1.513

 OPTIMUM
f=14
TT=1.674

 OPTIMUM
f=14
TT=0.722

 OPTIMUM
f=14
TT=1.517
MaxCSP/
pedigree/
eye_ext.xml		 MOPT
f=52
TT=0.127

 OPTIMUM
f=52
TT=0.733
T1=0.59
 OPTIMUM
f=52
TT=0.781
T1=0.6
 OPTIMUM
f=52
TT=0.127
T1=0.12
 OPTIMUM
f=52
TT=0.112
T1=0.1
 ? (problem)

TT=0.642

NODES=0
 OPTIMUM
f=52
TT=18.073

 OPTIMUM
f=52
TT=18.305

 ? (MO)

TT=1.07

 OPTIMUM
f=52
TT=17.863
MaxCSP/pedigree/
pedck1000_ext.xml		 MOPT
f=1651
TT=32.828

 MSAT (TO)
f=1651
TT=2400.11
T1=2.29
 MSAT (TO)
f=1651
TT=2400.07
T1=2.28
 ? (TO)

TT=2400.12

 ? (TO)

TT=2400.05

 ? (problem)

TT=1.302

NODES=0
 ? (TO)

TT=2400

 ? (TO)

TT=2400.01

 ? (MO)

TT=2.11

 OPTIMUM
f=1651
TT=32.828
MaxCSP/
pedigree/
pedck350_ext.xml		 MOPT
f=576
TT=5.035

 OPTIMUM
f=576
TT=69.917
T1=1
 OPTIMUM
f=576
TT=46.993
T1=1
 OPTIMUM
f=576
TT=5.035
T1=5.03
 OPTIMUM
f=576
TT=4.968
T1=4.98
 ? (problem)

TT=0.958

NODES=0
 OPTIMUM
f=576
TT=62.157

 OPTIMUM
f=576
TT=64.034

 ? (MO)

TT=1.06

 OPTIMUM
f=576
TT=60.991
MaxCSP/pedigree/
pedck350l2_ext.xml		 MOPT
f=577
TT=1.272

 OPTIMUM
f=577
TT=1.282
T1=1.03
 OPTIMUM
f=577
TT=1.272
T1=0.97
 OPTIMUM
f=577
TT=1.864
T1=1.83
 Wrong Opt.
f=575
TT=115.261
T1=4.11
 ? (problem)

TT=0.925

NODES=0
 OPTIMUM
f=577
TT=82.201

 OPTIMUM
f=577
TT=78.449

 ? (MO)

TT=1.08

 OPTIMUM
f=577
TT=76.765
MaxCSP/pedigree/
pedck60-L12_ext.xml		 MOPT
f=102
TT=3.078

 OPTIMUM
f=102
TT=31.943
T1=0.6
 OPTIMUM
f=102
TT=35.808
T1=0.73
 OPTIMUM
f=102
TT=3.078
T1=3.1
 OPTIMUM
f=102
TT=0.452
T1=0.45
 ? (problem)

TT=0.464

NODES=0
 OPTIMUM
f=102
TT=11.04

 OPTIMUM
f=102
TT=11.517

 ? (TO)

TT=2400

 OPTIMUM
f=102
TT=11.403
MaxCSP/pedigree/
pedck60-L1_ext.xml		 MOPT
f=106
TT=0.138

 OPTIMUM
f=106
TT=30.845
T1=0.67
 OPTIMUM
f=106
TT=30.889
T1=0.63
 OPTIMUM
f=106
TT=0.138
T1=0.14
 OPTIMUM
f=106
TT=0.12
T1=0.12
 ? (problem)

TT=0.529

NODES=0
 OPTIMUM
f=106
TT=13.821

 OPTIMUM
f=106
TT=13.421

 OPTIMUM
f=106
TT=1953.78

 OPTIMUM
f=106
TT=14.126
MaxCSP/pedigree/
pedck60-L2_ext.xml		 MOPT
f=104
TT=0.123

 OPTIMUM
f=104
TT=31.287
T1=0.63
 OPTIMUM
f=104
TT=30.953
T1=0.59
 OPTIMUM
f=104
TT=0.123
T1=0.12
 OPTIMUM
f=104
TT=0.09
T1=0.09
 ? (problem)

TT=0.469

NODES=0
 OPTIMUM
f=104
TT=12.323

 OPTIMUM
f=104
TT=12.264

 ? (TO)

TT=2400.04

 OPTIMUM
f=104
TT=11.741
MaxCSP/
pedigree/
sheep4nr_ext.xml		 MOPT
f=11190
TT=678.828

 MSAT (TO)
f=11189
TT=2400.12
T1=307.13
 MSAT (TO)
f=11189
TT=2400.08
T1=308.98
 ? (MO)

TT=75.354

 ? (MO)

TT=75.575

 ? (problem)

TT=35.325

NODES=0
 ? (MO)

TT=116.77

 ? (MO)

TT=28.48

 ? (MO)

TT=0.03

 OPTIMUM
f=11190
TT=678.828
MaxCSP/pedigree/
sheep4r-4-2_ext.xml		 MOPT
f=2829
TT=37.098

 OPTIMUM
f=2829
TT=104.901
T1=97.92
 OPTIMUM
f=2829
TT=106.54
T1=99.23
 ? (TO)

TT=2400.19

 ? (TO)

TT=2400.14

 ? (problem)

TT=3.875

NODES=0
 OPTIMUM
f=2829
TT=56.979

 ? (TO)

TT=2400.07

 ? (exit code)

TT=3.495

 OPTIMUM
f=2829
TT=37.098
Bench nameBest results
on this
instance		AbsconMax
109 EPFC		AbsconMax
109 PFC		aolibdvo
2007-01-17		aolibpvo
2007-01-17		CSP4J - MaxCSP
2006-12-19		toolbar
2007-01-12		Toolbar_BTD
2007-01-12		Toolbar_MaxSat
2007-01-19		Toulbar2
2007-01-12
MaxCSP/pedigree/
sheep4r-4-3_ext.xml		 MOPT
f=3634
TT=38.064

 OPTIMUM
f=3634
TT=41.464
T1=39.2
 OPTIMUM
f=3634
TT=39.91
T1=37.86
 ? (MO)

TT=8.695

 ? (MO)

TT=8.642

 ? (problem)

TT=5.137

NODES=0
 ? (MO)

TT=37.75

 ? (TO)

TT=2400.04

 ? (MO)

TT=0.031

 OPTIMUM
f=3634
TT=38.064
MaxCSP/
pedigree/
sheep4r_ext.xml		 MOPT
f=11105
TT=679.678

 MSAT (TO)
f=11101
TT=2400.1
T1=292.6
 MSAT (TO)
f=11101
TT=2400.14
T1=291.51
 ? (MO)

TT=74.979

 ? (MO)

TT=75.206

 ? (problem)

TT=40.185

NODES=0
 ? (MO)

TT=121.05

 ? (MO)

TT=27

 ? (MO)

TT=0.033

 OPTIMUM
f=11105
TT=679.678
MaxCSP/
pedigree/
simple_ext.xml		 MOPT
f=5
TT=0.004

 OPTIMUM
f=5
TT=0.656
T1=0.52
 OPTIMUM
f=5
TT=0.687
T1=0.57
 OPTIMUM
f=5
TT=0.004
T1=0
 OPTIMUM
f=5
TT=0.004
T1=0
 ? (problem)

TT=0.33

NODES=0
 OPTIMUM
f=5
TT=1.433

 OPTIMUM
f=5
TT=1.486

 OPTIMUM
f=5
TT=0.85

 OPTIMUM
f=5
TT=1.338
MaxCSP/pedigree/
wijsmanguo_ext.xml		 MOPT
f=67
TT=0.829

 OPTIMUM
f=67
TT=0.966
T1=0.67
 OPTIMUM
f=67
TT=0.829
T1=0.65
 OPTIMUM
f=67
TT=1.238
T1=1.23
 OPTIMUM
f=67
TT=5.507
T1=5.5
 ? (problem)

TT=1.339

NODES=0
 OPTIMUM
f=67
TT=25.469

 OPTIMUM
f=67
TT=24.243

 ? (exit code)

TT=1.47

 OPTIMUM
f=67
TT=24.5
MaxCSP/spot5/
spot5-28_ext.xml		 MSAT TO
f=5046
TT=2400.09

 MSAT (TO)
f=5035
TT=2400.16
T1=418.91
 MSAT (TO)
f=5037
TT=2400.1
T1=741
 ? (TO)

TT=2400.09

 ? (TO)

TT=2400.06

 ? (problem)

TT=3.372

NODES=0
 MSAT (TO)
f=5046
TT=2400.09

 ? (TO)

TT=2400.01

 ? (TO)

TT=2400.02

 ? (TO)

TT=2400.1
MaxCSP/spot5/
spot5-404_ext.xml		 MOPT
f=645
TT=1806.98

 MSAT (TO)
f=642
TT=2400.1
T1=42.21
 MSAT (TO)
f=643
TT=2400.09
T1=39.79
 OPTIMUM
f=645
TT=1806.98
T1=1816.61
 OPTIMUM
f=645
TT=41.578
T1=41.88
 ? (problem)

TT=0.57

NODES=0
 MSAT (TO)
f=645
TT=2400.11

 ? (TO)

TT=2400.05

 ? (TO)

TT=2400.06

 MSAT (TO)
f=645
TT=2400.14
MaxCSP/spot5/
spot5-408_ext.xml		 MSAT TO
f=2097
TT=2400.04

 MSAT (TO)
f=2092
TT=2400.02
T1=46.16
 MSAT (TO)
f=2090
TT=2400.12
T1=15.79
 ? (TO)

TT=2400.06

 ? (TO)

TT=2400.05

 ? (problem)

TT=0.972

NODES=0
 MSAT (TO)
f=2097
TT=2400.11

 ? (TO)

TT=2400

 ? (TO)

TT=2400.08

 MSAT (TO)
f=2097
TT=2400.04
MaxCSP/spot5/
spot5-412_ext.xml		 MSAT TO
f=4134
TT=2400.05

 MSAT (TO)
f=4123
TT=2400.06
T1=33.27
 MSAT (TO)
f=4126
TT=2400.11
T1=33.77
 ? (TO)

TT=2400.09

 ? (TO)

TT=2400.11

 ? (problem)

TT=2.492

NODES=0
 MSAT (TO)
f=4134
TT=2400.05

 ? (TO)

TT=2400.08

 ? (TO)

TT=2400.09

 MSAT (TO)
f=4134
TT=2400.05
MaxCSP/spot5/
spot5-414_ext.xml		 MSAT TO
f=9847
TT=2400.06

 MSAT (TO)
f=9831
TT=2400.14
T1=44.64
 MSAT (TO)
f=9832
TT=2400.14
T1=40.04
 ? (TO)

TT=2400.11

 ? (TO)

TT=2400.11

 ? (problem)

TT=13.838

NODES=0
 ? (TO)

TT=2400.13

 ? (TO)

TT=2400.05

 ? (TO)

TT=2400.04

 MSAT (TO)
f=9847
TT=2400.06
MaxCSP/spot5/
spot5-42_ext.xml		 MSAT TO
f=1284
TT=2400.09

 MSAT (TO)
f=1276
TT=2400.11
T1=14.82
 MSAT (TO)
f=1276
TT=2400.09
T1=13.77
 ? (TO)

TT=2400.08

 ? (TO)

TT=2400.04

 ? (problem)

TT=0.775

NODES=0
 MSAT (TO)
f=1284
TT=2400.14

 ? (TO)

TT=2400.03

 ? (TO)

TT=2400.03

 MSAT (TO)
f=1284
TT=2400.09
Bench nameBest results
on this
instance		AbsconMax
109 EPFC		AbsconMax
109 PFC		aolibdvo
2007-01-17		aolibpvo
2007-01-17		CSP4J - MaxCSP
2006-12-19		toolbar
2007-01-12		Toolbar_BTD
2007-01-12		Toolbar_MaxSat
2007-01-19		Toulbar2
2007-01-12
MaxCSP/spot5/
spot5-503_ext.xml		 MOPT
f=566
TT=14.794

 MSAT (TO)
f=563
TT=2400.07
T1=28.46
 MSAT (TO)
f=563
TT=2400.14
T1=29.02
 ? (TO)

TT=2400.1

 OPTIMUM
f=566
TT=137.079
T1=138.1
 ? (problem)

TT=0.57

NODES=0
 MSAT (TO)
f=566
TT=2400.12

 OPTIMUM
f=566
TT=14.794

 ? (TO)

TT=2400.07

 MSAT (TO)
f=566
TT=2400.04
MaxCSP/spot5/
spot5-505_ext.xml		 MSAT TO
f=2091
TT=2400.11

 MSAT (TO)
f=2081
TT=2400.15
T1=978.45
 MSAT (TO)
f=2082
TT=2400.07
T1=1007.57
 ? (TO)

TT=2400.09

 ? (TO)

TT=2400.02

 ? (problem)

TT=1.038

NODES=0
 MSAT (TO)
f=2091
TT=2400.11

 ? (TO)

TT=2400.02

 ? (TO)

TT=2400.04

 MSAT (TO)
f=2091
TT=2400.11
MaxCSP/spot5/
spot5-507_ext.xml		 MSAT TO
f=5519
TT=2400.05

 MSAT (TO)
f=5508
TT=2400.12
T1=34.72
 MSAT (TO)
f=5509
TT=2400.06
T1=1105.89
 ? (TO)

TT=2400.1

 ? (TO)

TT=2400.08

 ? (problem)

TT=4.028

NODES=0
 MSAT (TO)
f=5519
TT=2400.05

 ? (TO)

TT=2400.05

 ? (TO)

TT=2400.07

 MSAT (TO)
f=5519
TT=2400.06
MaxCSP/spot5/
spot5-509_ext.xml		 MSAT TO
f=8379
TT=2400.05

 MSAT (TO)
f=8359
TT=2400.11
T1=1.37
 MSAT (TO)
f=8359
TT=2400.05
T1=1.35
 ? (TO)

TT=2400.05

 ? (TO)

TT=2400.04

 ? (problem)

TT=9.591

NODES=0
 MSAT (TO)
f=8379
TT=2400.06

 ? (TO)

TT=2400.01

 ? (TO)

TT=2400.06

 MSAT (TO)
f=8379
TT=2400.05
MaxCSP/spot5/
spot5-54_ext.xml		 MOPT
f=249
TT=0.271

 MSAT (TO)
f=249
TT=2400.08
T1=3.34
 MSAT (TO)
f=249
TT=2400.1
T1=3.32
 OPTIMUM
f=249
TT=0.271
T1=0.27
 OPTIMUM
f=249
TT=1.525
T1=1.53
 ? (problem)

TT=0.445

NODES=0
 OPTIMUM
f=249
TT=5.615

 OPTIMUM
f=249
TT=4.44

 ? (TO)

TT=2400.11

 OPTIMUM
f=249
TT=6.917
MaxCSP/spot5/
spot5-5_ext.xml		 MSAT TO
f=5413
TT=2400.11

 MSAT (TO)
f=5393
TT=2400.08
T1=38.61
 MSAT (TO)
f=5395
TT=2400.13
T1=708.86
 ? (TO)

TT=2400.05

 ? (TO)

TT=2400.09

 ? (problem)

TT=3.958

NODES=0
 MSAT (TO)
f=5413
TT=2400.11

 ? (TO)

TT=2400.09

 ? (TO)

TT=2400

 MSAT (TO)
f=5413
TT=2400.12

Some statistics...

AbsconMax
109 EPFC		AbsconMax
109 PFC		aolibdvo
2007-01-17		aolibpvo
2007-01-17		CSP4J - MaxCSP
2006-12-19		toolbar
2007-01-12		Toolbar_BTD
2007-01-12		Toolbar_MaxSat
2007-01-19		Toulbar2
2007-01-12
Number of times the solver is able to give the best known answer111111001911323
Number of times the solver is the best solver from a complete solver point of view (i.e. best known answer and best TT time)0290081013