2019 XCSP3 competition: main track (CSP and COP, sequential and parallel solvers): results by benchmark

Results by benchmark for category decision problem (CSP)

This page displays the results of the different solvers for each benchmark for category decision problem (CSP)

REMINDER

Keep in mind that the 'Best result' columns only provide the best result given by one of the solvers. This 'Best result' may be wrong in case of an UNSATISFIABLE or OPTIMUM FOUND answer (because there's no efficient way to check these answers).

Description of a cell contents:

Cell exampleMeaning
AnswerSolver result
f=...value of the objective function for the model reported by the solver
TT=...Total Time (TT): this is the CPU time (in seconds) used by the solver until termination. This time is only meaningful for complete solvers because incomplete solvers will always run until they time out
Remember that CPU time and wall clock time are two very different notions. The CPU time represents the time during which the instructions of the solver were executed by the processor. The wall clock time represents how much time ellapsed on the clock. For a same event, the CPU time may be either smaller or greater than the wall clock time depending on the number of threads of execution and the number of processors.

Meaning of some abbreviations:

AbbreviationMeaning
f=...Value of the objective function
TOTime Out
MOMem. Out (out of memory)

Meaning of the different colors:

ColorMeaning
textthe solver cannot handle this instance
textthe solver gave no answer
textthe solver could give an answer (SAT)
textthe solver gave a definitive answer (OPTIMUM FOUND or UNSAT)
textthe solver performed better than the other ones on that instance (complete solvers point of view)
textthe solver performed better than the other ones on that instance (incomplete solvers point of view)
textthe solver was ended by a signal or other problem
textthe solver gave an incomplete answer
textthe solver gave a wrong answer

For better readability, you may choose to hide some solvers:
AbsCon 2019-07-23 (complete)
BTD 19.07.01 (complete)
choco-solver 2019-06-14 (complete)
choco-solver 2019-06-14 parallel (complete)
choco-solver 2019-09-16 (complete)
choco-solver 2019-09-16 parallel (complete)
choco-solver 2019-09-20 (complete)
choco-solver 2019-09-20 parallel (complete)
choco-solver 2019-09-24 (complete)
choco-solver 2019-09-24 parallel (complete)
Concrete 3.10 (complete)
Concrete 3.12.2 (complete)
Concrete 3.12.2 (complete)
Concrete 3.12.3 (complete)
cosoco 2 (complete)
cosoco 2.0 (complete)
cosoco 2.0 (complete)
cosoco 2.0 parallel (complete)
cosoco 2.O parallel (complete)
Fun-sCOP hybrid+CryptoMiniSat (2019-06-15) (complete)
Fun-sCOP hybrid+ManyGlucose (2019-06-15) (complete)
Fun-sCOP hybrid+ManyGlucose (2019-09-22) (complete)
Fun-sCOP order+GlueMiniSat (2019-06-15) (complete)
Fun-sCOP order+ManyGlucose (2019-06-15) (complete)
Fun-sCOP order+ManyGlucose (2019-09-22) (complete)
PicatSAT 2019-09-12 (complete)

Bench nameBest results
on this
instance
AbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
AllInterval/
AllInterval-m1-s1/
AllInterval-035.xml
SAT
TT=0.031

SAT
TT=3.653

SAT
TT=0.031

SAT
TT=1.604

SAT
TT=2.325

SAT
TT=1.511

SAT
TT=2.344

SAT
TT=1.524

SAT
TT=2.069

SAT
TT=69.122

SAT
TT=2.23

SAT
TT=81.215

SAT
TT=519.439

? (TO)
TT=2520.08

SAT
TT=9.81

SAT
TT=14.919

SAT
TT=14.877

SAT
TT=14.874

SAT
TT=0.153

SAT
TT=0.133

SAT
TT=11.694

SAT
TT=16.513

SAT
TT=16.204

SAT
TT=14.149

SAT
TT=16.557

SAT
TT=16.325

SAT
TT=228.644

AllInterval/
AllInterval-m1-s1/
AllInterval-040.xml
SAT
TT=0.045

SAT
TT=3.771

SAT
TT=0.045

SAT
TT=1.689

SAT
TT=2.585

SAT
TT=1.753

SAT
TT=2.574

SAT
TT=1.579

SAT
TT=2.203

SAT
TT=46.751

SAT
TT=2.036

SAT
TT=127.241

? (TO)
TT=2520.1

SAT
TT=93.539

SAT
TT=189.841

SAT
TT=0.348

SAT
TT=0.349

SAT
TT=0.348

SAT
TT=0.194

SAT
TT=0.192

SAT
TT=14.883

SAT
TT=25.122

SAT
TT=23.845

SAT
TT=14.602

SAT
TT=24.214

SAT
TT=23.517

SAT
TT=290.328

Bibd/Bibd-sc-open/
Bibd-sc-85-085-36-36-15.xml



? (TO)
TT=2520.04

? (TO)
TT=2519.91

? (TO)
TT=2520.42

? (TO)
TT=19402.4

? (TO)
TT=2520.66

? (TO)
TT=19340.5

? (TO)
TT=2520.74

? (TO)
TT=19380.1

? (TO)
TT=2520.05

? (TO)
TT=19271.9

? (TO)
TT=2520.16

? (TO)
TT=2520.14

? (TO)
TT=2520.44

? (TO)
TT=2520.2

? (MO)
TT=140.442

? (MO)
TT=119.782

? (MO)
TT=120.215

? (MO)
TT=48.47

? (MO)
TT=49.634

? (TO)
TT=2520.44

? (TO)
TT=4488.1

? (TO)
TT=4525.37

? (TO)
TT=2520.28

? (TO)
TT=6325.04

? (TO)
TT=6125.75

? (TO)
TT=2520.07

Bibd/Bibd-sc-stab1/
Bibd-sc-25-05-01.xml
SAT
TT=21.175

? (TO)
TT=2520.1

? (TO)
TT=2519.89

? (TO)
TT=2520.63

? (TO)
TT=20073.8

SAT
TT=1666.8

SAT
TT=1367.89

SAT
TT=1680.89

SAT
TT=1274.01

SAT
TT=379.265

SAT
TT=1596.31

? (TO)
TT=2520.11

? (TO)
TT=2520.1

? (TO)
TT=2520.09

? (TO)
TT=2520.12

? (TO)
TT=2519.75

? (TO)
TT=2520.08

? (TO)
TT=2519.73

? (TO)
TT=20036.7

? (TO)
TT=20037.8

SAT
TT=45.027

SAT
TT=39.221

SAT
TT=40.793

SAT
TT=43.3

SAT
TT=38.891

SAT
TT=38.062

SAT
TT=21.175

Bibd/Bibd-sc-stab1/
Bibd-sc-25-09-03.xml
SAT
TT=137.506

? (TO)
TT=2520.11

? (TO)
TT=2519.77

SAT
TT=137.506

SAT
TT=294.378

SAT
TT=260.369

SAT
TT=438.017

SAT
TT=211.421

SAT
TT=430.111

? (TO)
TT=2520.11

SAT
TT=446.248

? (TO)
TT=2520.1

? (TO)
TT=2520.11

? (TO)
TT=2520.09

? (TO)
TT=2520.07

? (TO)
TT=2520.02

? (TO)
TT=2519.96

? (TO)
TT=2519.65

? (TO)
TT=20046.3

? (TO)
TT=20047

SAT
TT=1689.17

? (TO)
TT=18675.2

? (TO)
TT=18853.6

? (TO)
TT=2520.14

SAT
TT=16513.1

SAT
TT=16599.1

SAT
TT=515.664

Bibd/Bibd-sc-stab2/
Bibd-sc-21-06-02.xml



? (TO)
TT=2520.12

? (TO)
TT=2519.99

? (TO)
TT=2520.59

? (TO)
TT=20073.8

? (TO)
TT=2520.62

? (TO)
TT=20088.6

? (TO)
TT=2520.67

? (TO)
TT=20077

? (TO)
TT=2520.04

? (TO)
TT=20086.1

? (TO)
TT=2520.13

? (TO)
TT=2520.14

? (TO)
TT=2520.12

? (TO)
TT=2520.05

? (TO)
TT=2520.04

? (TO)
TT=2519.91

? (TO)
TT=2519.89

? (TO)
TT=20064.8

? (TO)
TT=20062.2

? (TO)
TT=2520.14

? (TO)
TT=18339.6

? (TO)
TT=18319.9

? (TO)
TT=2520.14

? (TO)
TT=18065.5

? (TO)
TT=17944.3

? (TO)
TT=2520

Bibd/Bibd-sum-lex/
Bibd-sum-09-120-40-03-10.xml
SAT
TT=34.124

? (TO)
TT=2520.06

? (TO)
TT=2519.93

SAT
TT=1795.65

SAT
TT=4039.16

SAT
TT=1366.78

SAT
TT=25.136

SAT
TT=1402.42

SAT
TT=37.73

? (TO)
TT=2520.03

SAT
TT=34.124

? (TO)
TT=2520.09

? (TO)
TT=2520.08

? (TO)
TT=2520.14

? (TO)
TT=2520.14

? (MO)
TT=2300.04

? (MO)
TT=1992.96

? (MO)
TT=2038.52

? (TO)
TT=2520.88

? (TO)
TT=2520.59

SAT
TT=38.633

SAT
TT=53.218

SAT
TT=51.854

SAT
TT=48.753

SAT
TT=55.223

SAT
TT=55.262

SAT
TT=70.406

Bibd/Bibd-sum-open/
Bibd-sum-22-033-12-08-04.xml



? (TO)
TT=2520.03

? (TO)
TT=2519.99

? (TO)
TT=2520.57

? (TO)
TT=20092.8

? (TO)
TT=2520.53

? (TO)
TT=20096.6

? (TO)
TT=2520.67

? (TO)
TT=20096.2

? (TO)
TT=2520.08

? (TO)
TT=20095.4

? (TO)
TT=2520.08

? (TO)
TT=2520.05

? (TO)
TT=2520.12

? (TO)
TT=2520.14

? (MO)
TT=1391.02

? (MO)
TT=1389.9

? (MO)
TT=1386.57

? (TO)
TT=2521.1

? (TO)
TT=2520.73

? (TO)
TT=2520.14

? (TO)
TT=19475.9

? (TO)
TT=19374.7

? (TO)
TT=2520.09

? (TO)
TT=18713.8

? (TO)
TT=18650.1

? (TO)
TT=2519.8

Bibd/Bibd-sum-open/
Bibd-sum-40-052-13-10-03.xml



? (TO)
TT=2520.02

? (TO)
TT=2519.9

? (TO)
TT=2520.42

? (TO)
TT=20060.5

? (TO)
TT=2520.34

? (TO)
TT=19934.2

? (TO)
TT=2520.74

? (TO)
TT=19933.3

? (TO)
TT=2520.03

? (TO)
TT=19936.6

? (TO)
TT=2520.14

? (TO)
TT=2520.09

? (TO)
TT=2520.13

? (TO)
TT=2520.14

? (MO)
TT=1669.37

? (MO)
TT=1666.14

? (MO)
TT=1659.26

? (TO)
TT=2520.95

? (TO)
TT=2520.83

? (TO)
TT=2520.11

? (TO)
TT=18285.7

? (TO)
TT=18087.7

? (TO)
TT=2520.11

? (TO)
TT=17595.3

? (TO)
TT=17487.4

? (TO)
TT=2519.54

Bibd/Bibd-sum-open/
Bibd-sum-57-076-20-15-05.xml



? (TO)
TT=2520.06

? (TO)
TT=2519.93

? (TO)
TT=2520.35

? (TO)
TT=20047.8

? (TO)
TT=2520.53

? (TO)
TT=20041.1

? (TO)
TT=2520.74

? (TO)
TT=20043.3

? (TO)
TT=2520.09

? (TO)
TT=20039.4

? (TO)
TT=2520.06

? (TO)
TT=2520.12

? (TO)
TT=2520.14

? (TO)
TT=2520.16

? (MO)
TT=2065.37

? (MO)
TT=1766.87

? (MO)
TT=1786.62

? (TO)
TT=2520.85

? (TO)
TT=2521

? (TO)
TT=2520.21

? (TO)
TT=17223.3

? (TO)
TT=17215.9

? (TO)
TT=2520.18

? (TO)
TT=18621.1

? (TO)
TT=18015.3

? (TO)
TT=2519.86

Bench nameBest resultsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
Bibd/Bibd-sum-open/
Bibd-sum-85-085-36-36-15.xml



? (TO)
TT=2520.02

? (TO)
TT=2519.88

? (TO)
TT=2520.73

? (TO)
TT=20009.7

? (TO)
TT=2520.51

? (TO)
TT=19994.2

? (TO)
TT=2520.43

? (TO)
TT=19996.2

? (TO)
TT=2520.09

? (TO)
TT=20000.1

? (TO)
TT=2520.09

? (TO)
TT=2520.16

? (TO)
TT=2520.8

? (TO)
TT=2520.23

? (MO)
TT=2323.2

? (MO)
TT=2032.29

? (MO)
TT=2038.5

? (TO)
TT=2520.67

? (TO)
TT=2520.6

? (TO)
TT=2520.29

? (TO)
TT=4654.85

? (TO)
TT=4645.94

? (TO)
TT=2520.35

? (TO)
TT=6040.58

? (TO)
TT=6261.07

? (TO)
TT=2519.51

Bibd/Bibd-sum-stab1/
Bibd-sum-25-05-01.xml
SAT
TT=21.199

SAT
TT=353.876

? (TO)
TT=2519.57

? (TO)
TT=2520.32

SAT
TT=5698.28

SAT
TT=234.585

SAT
TT=546.432

SAT
TT=243.264

SAT
TT=545.734

SAT
TT=63.736

SAT
TT=539.82

? (TO)
TT=2520.04

? (TO)
TT=2520.14

? (TO)
TT=2520.1

? (TO)
TT=2520.04

? (MO)
TT=1436.17

? (MO)
TT=1432.35

? (MO)
TT=1276.41

? (TO)
TT=2520.94

? (TO)
TT=2520.79

SAT
TT=24.881

SAT
TT=34.396

SAT
TT=34.47

SAT
TT=26.684

SAT
TT=28.545

SAT
TT=27.717

SAT
TT=21.199

Blackhole/
Blackhole-m1-s13/
Blackhole-13-3-04.xml
SAT
TT=0.02

SAT
TT=4.912

SAT
TT=0.02

SAT
TT=2.104

SAT
TT=4.025

SAT
TT=2.095

SAT
TT=3.562

SAT
TT=2.062

SAT
TT=3.796

SAT
TT=1.481

SAT
TT=3.437

SAT
TT=8.09

SAT
TT=7.418

SAT
TT=9.649

SAT
TT=7.65

? (TO)
TT=2520.04

? (TO)
TT=2519.57

? (TO)
TT=2519.94

? (TO)
TT=20091.4

? (TO)
TT=20092.9

SAT
TT=17.361

SAT
TT=17.845

SAT
TT=17.02

SAT
TT=16.927

SAT
TT=16.719

SAT
TT=16.73

SAT
TT=13.666

Blackhole/
Blackhole-xcsp2-s07/
Blackhole-4-07-1_X2.xml
UNSAT
TT=168.634

? (TO)
TT=2520.08

? (TO)
TT=2519.75

? (TO)
TT=2520.5

? (TO)
TT=20048

? (TO)
TT=2520.58

? (TO)
TT=20044.7

? (TO)
TT=2520.42

? (TO)
TT=20041.8

? (TO)
TT=2520.07

? (TO)
TT=20035.9

? (TO)
TT=2520.09

? (TO)
TT=2520.12

? (TO)
TT=2520.14

? (TO)
TT=2520.13

? (TO)
TT=2519.93

? (TO)
TT=2520.07

? (TO)
TT=2520.06

? (TO)
TT=20110.5

? (TO)
TT=20110.2

UNSAT
TT=168.634

UNSAT
TT=1015.34

UNSAT
TT=1014.24

UNSAT
TT=318.635

UNSAT
TT=1023.17

UNSAT
TT=1029.52

UNSAT
TT=377.818

Blackhole/
Blackhole-xcsp2-s07/
Blackhole-4-07-2_X2.xml
UNSAT
TT=165.879

? (TO)
TT=2520.09

?
TT=2520.03

? (TO)
TT=2520.59

? (TO)
TT=20043.9

? (TO)
TT=2520.68

? (TO)
TT=20045.5

? (TO)
TT=2520.43

? (TO)
TT=20040.8

? (TO)
TT=2520.12

? (TO)
TT=20046.5

? (TO)
TT=2520.11

? (TO)
TT=2520.07

? (TO)
TT=2520.13

? (TO)
TT=2520.07

? (TO)
TT=2519.47

? (TO)
TT=2520.04

? (TO)
TT=2520.02

? (TO)
TT=20107.5

? (TO)
TT=20107.9

UNSAT
TT=165.879

UNSAT
TT=944.572

UNSAT
TT=937.691

UNSAT
TT=403.59

UNSAT
TT=958.174

UNSAT
TT=943.171

UNSAT
TT=523.057

Blackhole/
Blackhole-xcsp2-s07/
Blackhole-4-07-3_X2.xml
UNSAT
TT=165.887

? (TO)
TT=2520.08

?
TT=2520.03

? (TO)
TT=2520.48

? (TO)
TT=20052.8

? (TO)
TT=2520.6

? (TO)
TT=20045.3

? (TO)
TT=2520.55

? (TO)
TT=20043.9

? (TO)
TT=2520.07

? (TO)
TT=20046.6

? (TO)
TT=2520.08

? (TO)
TT=2520.14

? (TO)
TT=2520.1

? (TO)
TT=2520.1

? (TO)
TT=2519.78

? (TO)
TT=2519.63

? (TO)
TT=2520.06

? (TO)
TT=20107.9

? (TO)
TT=20106.4

UNSAT
TT=165.887

UNSAT
TT=1001.34

UNSAT
TT=949.946

UNSAT
TT=445.11

UNSAT
TT=947.615

UNSAT
TT=955.609

UNSAT
TT=525.37

Blackhole/
Blackhole-xcsp2-s07/
Blackhole-4-07-7_X2.xml
UNSAT
TT=164.594

? (TO)
TT=2520.08

?
TT=2520.03

? (TO)
TT=2520.34

? (TO)
TT=20055.2

? (TO)
TT=2520.64

? (TO)
TT=20041.8

? (TO)
TT=2520.32

? (TO)
TT=20041.8

? (TO)
TT=2520.06

? (TO)
TT=20049.7

? (TO)
TT=2520.06

? (TO)
TT=2520.05

? (TO)
TT=2520.09

? (TO)
TT=2520.07

? (TO)
TT=2520.02

? (TO)
TT=2520.03

? (TO)
TT=2519.9

? (TO)
TT=20109.7

? (TO)
TT=20108.5

UNSAT
TT=164.594

UNSAT
TT=887.448

UNSAT
TT=877.408

UNSAT
TT=375.342

UNSAT
TT=874.681

UNSAT
TT=875.5

UNSAT
TT=516.57

Blackhole/
Blackhole-xcsp2-s07/
Blackhole-4-07-8_X2.xml
UNSAT
TT=166.47

? (TO)
TT=2520.05

? (TO)
TT=2519.94

? (TO)
TT=2520.66

? (TO)
TT=20053

? (TO)
TT=2520.56

? (TO)
TT=20038.8

? (TO)
TT=2520.31

? (TO)
TT=20044.7

? (TO)
TT=2520.15

? (TO)
TT=20043.1

? (TO)
TT=2520.06

? (TO)
TT=2520.07

? (TO)
TT=2520.12

? (TO)
TT=2520.11

? (TO)
TT=2519.94

? (TO)
TT=2519.91

? (TO)
TT=2520.05

? (TO)
TT=20107.7

? (TO)
TT=20107.7

UNSAT
TT=166.47

UNSAT
TT=937.958

UNSAT
TT=922.212

UNSAT
TT=419.472

UNSAT
TT=929.764

UNSAT
TT=934.114

UNSAT
TT=484.472

Blackhole/
Blackhole-xcsp2-s07/
Blackhole-4-07-9_X2.xml
UNSAT
TT=170.678

? (TO)
TT=2520.08

? (TO)
TT=2520.02

? (TO)
TT=2520.45

? (TO)
TT=20049.8

? (TO)
TT=2520.65

? (TO)
TT=20045.5

? (TO)
TT=2520.29

? (TO)
TT=20048.2

? (TO)
TT=2520.11

? (TO)
TT=20057.4

? (TO)
TT=2520.07

? (TO)
TT=2520.13

? (TO)
TT=2520.11

? (TO)
TT=2520.12

? (TO)
TT=2520.09

? (TO)
TT=2519.7

? (TO)
TT=2520.08

? (TO)
TT=20109.5

? (TO)
TT=20107.4

UNSAT
TT=170.678

UNSAT
TT=1073.54

UNSAT
TT=1068.38

UNSAT
TT=371.33

UNSAT
TT=1076.85

UNSAT
TT=1072.34

UNSAT
TT=580.943

Blackhole/
Blackhole-xcsp2-s07/
Blackhole-4-07h-0_X2.xml
UNSAT
TT=163.179

? (TO)
TT=2520.03

? (TO)
TT=2519.63

? (TO)
TT=2520.58

? (TO)
TT=20053.5

? (TO)
TT=2520.71

? (TO)
TT=20040.8

? (TO)
TT=2520.32

? (TO)
TT=20047

? (TO)
TT=2520.05

? (TO)
TT=20045.7

? (TO)
TT=2520.09

? (TO)
TT=2520.13

? (TO)
TT=2520.1

? (TO)
TT=2520.12

? (TO)
TT=2520.02

? (TO)
TT=2519.95

? (TO)
TT=2520.01

? (TO)
TT=20108.3

? (TO)
TT=20109.6

UNSAT
TT=163.179

UNSAT
TT=1177.89

UNSAT
TT=1186.69

UNSAT
TT=283.192

UNSAT
TT=1187.33

UNSAT
TT=1180.49

UNSAT
TT=513.726

Bench nameBest resultsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
Blackhole/
Blackhole-xcsp2-s07/
Blackhole-4-07h-1_X2.xml
UNSAT
TT=174.931

? (TO)
TT=2520.06

? (TO)
TT=2519.8

? (TO)
TT=2520.38

? (TO)
TT=20058.1

? (TO)
TT=2520.7

? (TO)
TT=20038.6

? (TO)
TT=2520.41

? (TO)
TT=20037.1

? (TO)
TT=2520.16

? (TO)
TT=20042.3

? (TO)
TT=2520.11

? (TO)
TT=2520.13

? (TO)
TT=2520.11

? (TO)
TT=2520.11

? (TO)
TT=2520.02

? (TO)
TT=2520.04

? (TO)
TT=2520.07

? (TO)
TT=20108.6

? (TO)
TT=20111.2

UNSAT
TT=174.931

UNSAT
TT=940.947

UNSAT
TT=951.052

UNSAT
TT=372.924

UNSAT
TT=987.085

UNSAT
TT=938.791

UNSAT
TT=548.384

Blackhole/
Blackhole-xcsp2-s07/
Blackhole-4-07h-4_X2.xml
UNSAT
TT=170.824

? (TO)
TT=2520.09

? (TO)
TT=2519.9

? (TO)
TT=2520.34

? (TO)
TT=20059.8

? (TO)
TT=2520.38

? (TO)
TT=20042

? (TO)
TT=2520.54

? (TO)
TT=20035.8

? (TO)
TT=2520.09

? (TO)
TT=20042.1

? (TO)
TT=2520.14

? (TO)
TT=2520.05

? (TO)
TT=2520.11

? (TO)
TT=2520.09

? (TO)
TT=2520.02

? (TO)
TT=2520.01

? (TO)
TT=2519.94

? (TO)
TT=20107.3

? (TO)
TT=20110.2

UNSAT
TT=170.824

UNSAT
TT=1175.27

UNSAT
TT=1195.35

UNSAT
TT=337.065

UNSAT
TT=1152.66

UNSAT
TT=1170.31

UNSAT
TT=487.832

Blackhole/
Blackhole-xcsp2-s07/
Blackhole-4-07h-5_X2.xml
UNSAT
TT=165.955

? (TO)
TT=2520.09

? (TO)
TT=2519.89

? (TO)
TT=2520.4

? (TO)
TT=20049.7

? (TO)
TT=2520.4

? (TO)
TT=20042.2

? (TO)
TT=2520.61

? (TO)
TT=20042.9

? (TO)
TT=2520.08

? (TO)
TT=20043

? (TO)
TT=2520.06

? (TO)
TT=2520.13

? (TO)
TT=2520.07

? (TO)
TT=2520.13

? (TO)
TT=2520.03

? (TO)
TT=2519.94

? (TO)
TT=2520.01

? (TO)
TT=20108.7

? (TO)
TT=20108

UNSAT
TT=165.955

UNSAT
TT=1114.2

UNSAT
TT=1159.08

UNSAT
TT=382.657

UNSAT
TT=1111.72

UNSAT
TT=1117.83

UNSAT
TT=465.142

Blackhole/
Blackhole-xcsp2-s07/
Blackhole-4-07h-6_X2.xml
UNSAT
TT=169.632

? (TO)
TT=2520.06

? (TO)
TT=2519.92

? (TO)
TT=2520.41

? (TO)
TT=20052.7

? (TO)
TT=2520.37

? (TO)
TT=20046.9

? (TO)
TT=2520.29

? (TO)
TT=20048.5

? (TO)
TT=2520.09

? (TO)
TT=20045.9

? (TO)
TT=2520.14

? (TO)
TT=2520.07

? (TO)
TT=2520.12

? (TO)
TT=2520.11

? (TO)
TT=2520.05

? (TO)
TT=2519.93

? (TO)
TT=2519.76

? (TO)
TT=20108.3

? (TO)
TT=20108.2

UNSAT
TT=169.632

UNSAT
TT=1157.39

UNSAT
TT=1125.17

UNSAT
TT=424.807

UNSAT
TT=1114.07

UNSAT
TT=1095.88

UNSAT
TT=503.638

Blackhole/
Blackhole-xcsp2-s07/
Blackhole-4-07h-7_X2.xml
UNSAT
TT=159.215

? (TO)
TT=2520.05

? (TO)
TT=2519.84

? (TO)
TT=2520.62

? (TO)
TT=20048.1

? (TO)
TT=2520.45

? (TO)
TT=20045

? (TO)
TT=2520.78

? (TO)
TT=20041.2

? (TO)
TT=2520.06

? (TO)
TT=20046.7

? (TO)
TT=2520.08

? (TO)
TT=2520.11

? (TO)
TT=2520.1

? (TO)
TT=2520.11

? (TO)
TT=2520.02

? (TO)
TT=2520.05

? (TO)
TT=2520.07

? (TO)
TT=20108.7

? (TO)
TT=20108.2

UNSAT
TT=159.215

UNSAT
TT=1036.39

UNSAT
TT=1044.1

UNSAT
TT=242.84

UNSAT
TT=1040.93

UNSAT
TT=1039.67

UNSAT
TT=540.243

Blackhole/
Blackhole-xcsp2-s13/
Blackhole-4-13-2_X2.xml



? (TO)
TT=2520.1

? (TO)
TT=2519.57

? (TO)
TT=2520.55

? (TO)
TT=20078.2

? (TO)
TT=2520.66

? (TO)
TT=20084.7

? (TO)
TT=2520.48

? (TO)
TT=20087.7

? (TO)
TT=2520.02

? (TO)
TT=20082

? (TO)
TT=2520.06

? (TO)
TT=2520.13

? (TO)
TT=2520.09

? (TO)
TT=2520.07

? (TO)
TT=2520.01

? (TO)
TT=2519.92

? (TO)
TT=2520.07

? (TO)
TT=20105.8

? (TO)
TT=20104.5

? (TO)
TT=2520.08

? (TO)
TT=17573.4

? (TO)
TT=17512.8

? (TO)
TT=2520.06

? (TO)
TT=17443.9

? (TO)
TT=17684.5

? (TO)
TT=2519.93

CarSequencing/
CarSequencing-m1-jcr/
CarSequencing-16-81.xml
SAT
TT=27.946

? (TO)
TT=2520.04

? (TO)
TT=2519.91

? (TO)
TT=2520.25

? (TO)
TT=20055.7

? (TO)
TT=2520.33

? (TO)
TT=20023

? (TO)
TT=2520.34

? (TO)
TT=20015.1

? (TO)
TT=2520.11

? (TO)
TT=20027

? (TO)
TT=2520.15

? (TO)
TT=2520.08

? (TO)
TT=2520.07

? (TO)
TT=2520.07

? (TO)
TT=2520.02

? (TO)
TT=2519.9

? (TO)
TT=2520.05

? (TO)
TT=20114

? (TO)
TT=20114.5

SAT
TT=614.615

SAT
TT=4202.02

SAT
TT=4102.93

SAT
TT=1306.56

SAT
TT=1850.78

SAT
TT=1857.83

SAT
TT=27.946

CarSequencing/
CarSequencing-m1-jcr/
CarSequencing-36-92.xml
UNSAT
TT=67.6

UNSAT
TT=1551.79

? (TO)
TT=2520.03

? (TO)
TT=2520.43

? (TO)
TT=20051

? (TO)
TT=2520.34

? (TO)
TT=20042.5

? (TO)
TT=2520.36

? (TO)
TT=20038.7

? (TO)
TT=2520.07

? (TO)
TT=20036.3

? (TO)
TT=2520.08

? (TO)
TT=2520.16

? (TO)
TT=2520.06

? (TO)
TT=2520.09

? (TO)
TT=2520.02

? (TO)
TT=2520

? (TO)
TT=2520.01

? (TO)
TT=20112.3

? (TO)
TT=20113.5

? (TO)
TT=2520.13

? (TO)
TT=19445.1

? (TO)
TT=19321.5

? (TO)
TT=2520.13

? (TO)
TT=18974

? (TO)
TT=18515.7

UNSAT
TT=67.6

CarSequencing/
CarSequencing-m1-jcr/
CarSequencing-60-01.xml
SAT
TT=3.471

SAT
TT=79.076

? (TO)
TT=2520.01

SAT
TT=62.77

SAT
TT=140.08

SAT
TT=51.395

SAT
TT=102.32

SAT
TT=49.976

SAT
TT=100.599

SAT
TT=429.175

SAT
TT=102.433

SAT
TT=100.841

SAT
TT=41.645

SAT
TT=711.076

SAT
TT=38.647

SAT
TT=10.696

SAT
TT=10.709

SAT
TT=10.943

SAT
TT=3.563

SAT
TT=3.471

SAT
TT=202.176

SAT
TT=74.824

SAT
TT=72.225

SAT
TT=42.918

SAT
TT=50.395

SAT
TT=54.511

SAT
TT=38.601

CarSequencing/
CarSequencing-m1-jcr/
CarSequencing-60-03.xml
SAT
TT=14.782

SAT
TT=53.676

? (TO)
TT=2519.93

SAT
TT=29.772

SAT
TT=59.383

SAT
TT=115.474

SAT
TT=223.55

SAT
TT=120.037

SAT
TT=224.398

SAT
TT=281.345

SAT
TT=109.973

SAT
TT=27.477

SAT
TT=31.611

SAT
TT=272.624

SAT
TT=29.932

? (TO)
TT=2520.01

? (TO)
TT=2519.87

? (TO)
TT=2519.9

SAT
TT=14.782

SAT
TT=14.84

SAT
TT=109.43

SAT
TT=216.229

SAT
TT=214.34

SAT
TT=44.883

SAT
TT=52.344

SAT
TT=54.195

SAT
TT=31.727

Bench nameBest resultsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
CarSequencing/
CarSequencing-m1-jcr/
CarSequencing-65-10.xml
SAT
TT=1.894

SAT
TT=15.932

? (TO)
TT=2520.07

SAT
TT=33.08

SAT
TT=72.615

SAT
TT=36.524

SAT
TT=77.162

SAT
TT=38.159

SAT
TT=72.9

SAT
TT=31.84

SAT
TT=71.209

SAT
TT=27.362

SAT
TT=27.203

SAT
TT=37.936

SAT
TT=25.256

SAT
TT=1.892

SAT
TT=1.894

SAT
TT=1.899

SAT
TT=24.364

SAT
TT=23.629

SAT
TT=46.386

SAT
TT=119.299

SAT
TT=117.837

SAT
TT=45.088

SAT
TT=53.743

SAT
TT=51.212

SAT
TT=35.932

CarSequencing/
CarSequencing-m1-jcr/
CarSequencing-70-05.xml
SAT
TT=31.723

SAT
TT=196.73

? (TO)
TT=2520.06

SAT
TT=49.909

SAT
TT=112.939

SAT
TT=53.437

SAT
TT=106.636

SAT
TT=63.651

SAT
TT=67.734

SAT
TT=267.56

SAT
TT=66.41

SAT
TT=31.723

SAT
TT=37.383

SAT
TT=2109.07

SAT
TT=33.021

SAT
TT=2147.1

SAT
TT=1878.35

SAT
TT=1885.24

SAT
TT=32.007

SAT
TT=32.224

SAT
TT=126.614

SAT
TT=183.747

SAT
TT=182.851

SAT
TT=45.652

SAT
TT=76.176

SAT
TT=72.07

SAT
TT=46.845

CarSequencing/
CarSequencing-m1-jcr/
CarSequencing-70-08.xml
SAT
TT=7.387

SAT
TT=27.248

? (TO)
TT=2519.92

SAT
TT=39.48

SAT
TT=88.881

SAT
TT=44.023

SAT
TT=90.744

SAT
TT=42.401

SAT
TT=87.969

SAT
TT=36.276

SAT
TT=81.464

SAT
TT=106.761

SAT
TT=554.535

SAT
TT=32.684

SAT
TT=25.007

SAT
TT=8.441

SAT
TT=7.387

SAT
TT=7.399

SAT
TT=26.58

SAT
TT=26.542

SAT
TT=221.543

SAT
TT=334.984

SAT
TT=336.121

SAT
TT=47.178

SAT
TT=57.732

SAT
TT=62.269

SAT
TT=45.377

CarSequencing/
CarSequencing-m1-jcr/
CarSequencing-75-03.xml
SAT
TT=6.327

SAT
TT=1374.88

? (TO)
TT=2520.08

SAT
TT=48.411

SAT
TT=98.536

SAT
TT=73.183

SAT
TT=20.049

SAT
TT=72.195

SAT
TT=19.303

? (TO)
TT=2520.03

SAT
TT=20.544

SAT
TT=46.888

SAT
TT=31.567

SAT
TT=1953.82

SAT
TT=34.52

SAT
TT=269.189

SAT
TT=236.738

SAT
TT=233.492

SAT
TT=6.845

SAT
TT=6.327

SAT
TT=108.678

SAT
TT=128.588

SAT
TT=133.053

SAT
TT=46.812

SAT
TT=107.514

SAT
TT=109.84

SAT
TT=57.843

CarSequencing/
CarSequencing-m1-jcr/
CarSequencing-75-09.xml
SAT
TT=15.545

SAT
TT=15.545

? (TO)
TT=2520.08

SAT
TT=63.168

SAT
TT=133.321

SAT
TT=53.714

SAT
TT=103.532

SAT
TT=57.346

SAT
TT=105.665

SAT
TT=750.65

SAT
TT=201.387

SAT
TT=261.522

SAT
TT=35.632

? (TO)
TT=2520.15

SAT
TT=59.695

SAT
TT=443.997

SAT
TT=444.943

SAT
TT=431.53

SAT
TT=44.596

SAT
TT=42.077

SAT
TT=283.568

SAT
TT=825.983

SAT
TT=807.422

SAT
TT=108.291

SAT
TT=104.032

SAT
TT=115.112

SAT
TT=47.306

CarSequencing/
CarSequencing-m1-jcr/
CarSequencing-80-07.xml
SAT
TT=53.429

? (TO)
TT=2520.09

? (TO)
TT=2520.09

SAT
TT=80.922

SAT
TT=184.466

SAT
TT=61.063

SAT
TT=121.828

SAT
TT=63.869

SAT
TT=124.658

? (TO)
TT=2520.08

SAT
TT=130.826

? (TO)
TT=2520.08

? (TO)
TT=2520.15

? (TO)
TT=2520.09

? (TO)
TT=2520.08

? (TO)
TT=2519.93

? (TO)
TT=2520.05

? (TO)
TT=2520.05

SAT
TT=972.235

SAT
TT=1055.32

SAT
TT=201.898

SAT
TT=706.279

SAT
TT=730.628

SAT
TT=67.076

SAT
TT=69.494

SAT
TT=69.635

SAT
TT=53.429

CarSequencing/
CarSequencing-m1-jcr/
CarSequencing-85-05.xml
SAT
TT=78.5

? (TO)
TT=2520.02

? (TO)
TT=2520.03

SAT
TT=416.836

SAT
TT=879.72

? (TO)
TT=2520.51

SAT
TT=7970.92

? (TO)
TT=2520.49

SAT
TT=8537.51

? (TO)
TT=2520.06

SAT
TT=3566.26

? (TO)
TT=2520.08

? (TO)
TT=2520.11

? (TO)
TT=2520.13

? (TO)
TT=2520.14

? (TO)
TT=2520.02

? (TO)
TT=2520.03

? (TO)
TT=2520.01

? (TO)
TT=20116.2

? (TO)
TT=20116.9

SAT
TT=1870.03

? (TO)
TT=18335.3

? (TO)
TT=18326.3

SAT
TT=1253.94

SAT
TT=1401.13

SAT
TT=1402.32

SAT
TT=78.5

CarSequencing/
CarSequencing-m1-jcr/
CarSequencing-90-07.xml
SAT
TT=78.807

SAT
TT=1284.44

? (TO)
TT=2520.11

? (TO)
TT=2520.38

? (TO)
TT=20059.2

? (TO)
TT=2520.54

? (TO)
TT=20020.7

? (TO)
TT=2520.67

? (TO)
TT=20025.4

? (TO)
TT=2520.08

? (TO)
TT=20022.9

? (TO)
TT=2520.07

? (TO)
TT=2520.13

? (TO)
TT=2520.08

? (TO)
TT=2520.12

? (TO)
TT=2520.02

? (TO)
TT=2520.02

? (TO)
TT=2520.01

? (TO)
TT=20117.6

? (TO)
TT=20115.5

? (TO)
TT=2520.15

? (TO)
TT=18166.2

? (TO)
TT=18568.1

? (TO)
TT=2520.17

? (TO)
TT=19021.8

? (TO)
TT=18805.2

SAT
TT=78.807

Crossword/
Crossword-m1c-lex-herald/
Crossword-m1c-lex-h2101.xml
SAT
TT=193.944

? (TO)
TT=2520.15

SAT
TT=193.944

? (TO)
TT=2520.3

SAT
TT=15043.6

? (TO)
TT=2520.39

SAT
TT=13166.1

? (TO)
TT=2520.54

SAT
TT=14035.9

? (TO)
TT=2520.1

SAT
TT=4104.95

? (TO)
TT=2520.14

? (TO)
TT=2520.12

? (TO)
TT=2520.07

? (TO)
TT=2520.12

SAT
TT=1935.96

SAT
TT=1944.71

SAT
TT=1944.55

? (TO)
TT=20107.1

SAT
TT=5528.81

? (TO)
TT=2520.1

? (TO)
TT=17018

? (TO)
TT=16987.4

? (TO)
TT=2520.14

? (TO)
TT=16980

? (TO)
TT=16991.5

? (TO)
TT=2520.12

Crossword/
Crossword-m1c-lex-herald/
Crossword-m1c-lex-h2307.xml
SAT
TT=0.988

SAT
TT=5.964

SAT
TT=2.494

SAT
TT=8.641

SAT
TT=17.689

SAT
TT=8.964

SAT
TT=15.22

SAT
TT=9.769

SAT
TT=11.574

SAT
TT=2.858

SAT
TT=8.298

SAT
TT=14.303

SAT
TT=15.478

SAT
TT=72.472

SAT
TT=16.732

SAT
TT=0.99

SAT
TT=0.988

SAT
TT=0.999

SAT
TT=7.714

SAT
TT=7.91

? (TO)
TT=2520.13

? (TO)
TT=17148.6

? (TO)
TT=17209.1

? (TO)
TT=2520.09

? (TO)
TT=17278.8

? (TO)
TT=17153.6

SAT
TT=794.255

Bench nameBest resultsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
Crossword/
Crossword-m1c-ogd2008-herald/
Crossword-m1c-ogd2008-h1506.xml
SAT
TT=0.985

SAT
TT=6.237

SAT
TT=0.985

SAT
TT=11.994

SAT
TT=30.467

SAT
TT=11.456

SAT
TT=31.264

SAT
TT=11.754

SAT
TT=29.022

SAT
TT=4.084

SAT
TT=30.828

SAT
TT=15.224

SAT
TT=15.429

SAT
TT=18.483

SAT
TT=14.707

SAT
TT=2.543

SAT
TT=2.544

SAT
TT=2.548

SAT
TT=6.001

SAT
TT=5.998

? (TO)
TT=2528.25

SAT
TT=1198.4

SAT
TT=1190.75

? (TO)
TT=2527.9

SAT
TT=1183.18

SAT
TT=1199.59

SAT
TT=404.925

Crossword/
Crossword-m1c-ogd2008-herald/
Crossword-m1c-ogd2008-h1509.xml
SAT
TT=0.427

SAT
TT=4.914

SAT
TT=0.427

SAT
TT=7.844

SAT
TT=14.044

SAT
TT=7.512

SAT
TT=13.974

SAT
TT=7.499

SAT
TT=14.758

SAT
TT=2.86

SAT
TT=14.023

SAT
TT=13.048

SAT
TT=11.948

SAT
TT=14.811

SAT
TT=10.909

SAT
TT=0.908

SAT
TT=0.907

SAT
TT=0.913

SAT
TT=2.511

SAT
TT=2.54

SAT
TT=577.503

SAT
TT=599.627

SAT
TT=584.373

SAT
TT=556.865

SAT
TT=567.859

SAT
TT=567.004

SAT
TT=60.341

Crossword/
Crossword-m1c-ogd2008-herald/
Crossword-m1c-ogd2008-h1905.xml
SAT
TT=0.588

SAT
TT=5.249

SAT
TT=0.588

SAT
TT=7.782

SAT
TT=21.687

SAT
TT=7.513

SAT
TT=22.086

SAT
TT=9.033

SAT
TT=23.977

SAT
TT=3

SAT
TT=22.384

SAT
TT=13.968

SAT
TT=13.368

SAT
TT=16.707

SAT
TT=13.263

SAT
TT=1.397

SAT
TT=1.397

SAT
TT=1.404

SAT
TT=3.576

SAT
TT=3.606

SAT
TT=1032.16

SAT
TT=827.449

SAT
TT=818.526

SAT
TT=945.982

SAT
TT=823.773

SAT
TT=808.783

SAT
TT=161.245

Crossword/
Crossword-m1c-ogd-herald/
Crossword-m1c-ogd-h1502.xml
SAT
TT=0.441

SAT
TT=4.954

SAT
TT=0.441

SAT
TT=6.031

SAT
TT=10.856

SAT
TT=5.935

SAT
TT=11.561

SAT
TT=6.875

SAT
TT=11.384

SAT
TT=2.703

SAT
TT=11.428

SAT
TT=12.07

SAT
TT=12.623

SAT
TT=13.515

SAT
TT=10.806

SAT
TT=1.353

SAT
TT=1.351

SAT
TT=1.358

SAT
TT=2.71

SAT
TT=2.824

SAT
TT=394.418

SAT
TT=392.167

SAT
TT=415.796

SAT
TT=355.411

SAT
TT=393.741

SAT
TT=391.116

SAT
TT=155.817

Crossword/
Crossword-m1c-ogd-herald/
Crossword-m1c-ogd-h2105.xml
SAT
TT=0.519

SAT
TT=5.182

SAT
TT=0.519

SAT
TT=6.804

SAT
TT=16.394

SAT
TT=6.952

SAT
TT=15.374

SAT
TT=7.149

SAT
TT=15.892

SAT
TT=3.297

SAT
TT=15.215

SAT
TT=12.408

SAT
TT=12.036

SAT
TT=17.966

SAT
TT=11.91

SAT
TT=0.739

SAT
TT=0.709

SAT
TT=0.734

SAT
TT=3.595

SAT
TT=3.578

? (TO)
TT=2520.19

SAT
TT=1038.88

SAT
TT=1021.15

SAT
TT=917.053

SAT
TT=1055.09

SAT
TT=1040.99

SAT
TT=82.258

Crossword/
Crossword-m1c-ogd-herald/
Crossword-m1c-ogd-h2107.xml
SAT
TT=0.643

SAT
TT=5.398

SAT
TT=0.643

SAT
TT=8.218

SAT
TT=22.006

SAT
TT=8.019

SAT
TT=21.756

SAT
TT=8.546

SAT
TT=24.848

SAT
TT=2.926

SAT
TT=22.706

SAT
TT=13.044

SAT
TT=13.509

SAT
TT=17.829

SAT
TT=13.016

SAT
TT=1.387

SAT
TT=1.389

SAT
TT=1.393

SAT
TT=4.691

SAT
TT=4.678

SAT
TT=1509.6

SAT
TT=950.754

SAT
TT=956.709

SAT
TT=998.986

SAT
TT=1023.7

SAT
TT=929.945

SAT
TT=323.651

Crossword/Crossword-m1c-ogd-vg/
Crossword-m1c-ogd-vg-14-15.xml
UNSAT
TT=4.057

UNSAT
TT=6.471

UNSAT
TT=4.057

UNSAT
TT=20.812

UNSAT
TT=31.174

UNSAT
TT=21.8

UNSAT
TT=33.661

UNSAT
TT=21.857

UNSAT
TT=34.144

UNSAT
TT=5.831

UNSAT
TT=36.339

UNSAT
TT=93.779

UNSAT
TT=90.621

UNSAT
TT=241.348

UNSAT
TT=151.712

UNSAT
TT=17.69

UNSAT
TT=18.553

UNSAT
TT=18.263

UNSAT
TT=139.666

UNSAT
TT=128.954

? (TO)
TT=2520.17

UNSAT
TT=9185.51

UNSAT
TT=9222.59

? (TO)
TT=2520.12

UNSAT
TT=9201.5

UNSAT
TT=9151.65

UNSAT
TT=1247.15

Crossword/Crossword-m1c-ogd-vg/
Crossword-m1c-ogd-vg-15-19.xml
UNSAT
TT=0.204

UNSAT
TT=4.057

UNSAT
TT=0.204

UNSAT
TT=3.356

UNSAT
TT=7.192

UNSAT
TT=3.383

UNSAT
TT=7.125

UNSAT
TT=3.375

UNSAT
TT=7.384

UNSAT
TT=1.879

UNSAT
TT=7.359

UNSAT
TT=9.545

UNSAT
TT=9.889

UNSAT
TT=9.413

UNSAT
TT=8.589

UNSAT
TT=0.292

UNSAT
TT=0.292

UNSAT
TT=0.303

UNSAT
TT=0.986

UNSAT
TT=0.991

UNSAT
TT=260.591

UNSAT
TT=286.133

UNSAT
TT=291.595

UNSAT
TT=270.138

UNSAT
TT=287.308

UNSAT
TT=289.702

UNSAT
TT=146.508

Crossword/Crossword-m1c-ogd-vg/
Crossword-m1c-ogd-vg-16-16.xml
UNSAT
TT=0.088

UNSAT
TT=3.793

UNSAT
TT=0.088

UNSAT
TT=2.633

UNSAT
TT=4.233

UNSAT
TT=2.402

UNSAT
TT=3.984

UNSAT
TT=2.478

UNSAT
TT=4.393

UNSAT
TT=1.53

UNSAT
TT=3.325

UNSAT
TT=9.247

UNSAT
TT=9.705

UNSAT
TT=9.696

UNSAT
TT=9.113

UNSAT
TT=0.108

UNSAT
TT=0.109

UNSAT
TT=0.109

UNSAT
TT=0.549

UNSAT
TT=0.548

UNSAT
TT=181.108

UNSAT
TT=224.745

UNSAT
TT=224.754

UNSAT
TT=197.764

UNSAT
TT=223.004

UNSAT
TT=224.923

UNSAT
TT=125.121

Crossword/Crossword-m1c-uk-vg/
Crossword-m1c-uk-vg-10-13.xml
UNSAT
TT=183.225

UNSAT
TT=183.225

UNSAT
TT=561.575

UNSAT
TT=1402.97

UNSAT
TT=2480.71

UNSAT
TT=1045.02

UNSAT
TT=2264.29

UNSAT
TT=1039.41

UNSAT
TT=2500.67

UNSAT
TT=448.91

UNSAT
TT=2631.96

? (TO)
TT=2520.09

? (TO)
TT=2520.05

? (TO)
TT=2520.13

? (TO)
TT=2520.06

? (TO)
TT=2520.05

? (TO)
TT=2519.97

? (TO)
TT=2519.84

? (TO)
TT=20113.9

? (TO)
TT=20117.8

? (TO)
TT=2520.09

? (TO)
TT=16740.3

? (TO)
TT=16902.7

? (TO)
TT=2520.15

? (TO)
TT=16438.6

? (TO)
TT=16775.5

? (TO)
TT=2519.75

Bench nameBest resultsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
Crossword/
Crossword-m1c-words-herald/
Crossword-m1c-words-h2303.xml
SAT
TT=4.324

SAT
TT=6.636

SAT
TT=45.287

SAT
TT=25.927

SAT
TT=15.136

SAT
TT=24.207

SAT
TT=14.735

SAT
TT=24.628

SAT
TT=15.554

SAT
TT=4.324

SAT
TT=12.334

SAT
TT=17.811

SAT
TT=16.933

SAT
TT=54.756

SAT
TT=20.496

SAT
TT=4.465

SAT
TT=4.432

SAT
TT=4.449

SAT
TT=22.58

SAT
TT=17.992

? (TO)
TT=2520.25

? (TO)
TT=15689.7

? (TO)
TT=15661.7

? (TO)
TT=2520.18

? (TO)
TT=15498.9

? (TO)
TT=15684.8

SAT
TT=677.981

Crossword/Crossword-m1c-words-vg/
Crossword-m1c-words-vg-7-7.xml
SAT
TT=0.094

SAT
TT=40.856

SAT
TT=0.094

SAT
TT=39.527

SAT
TT=79.024

SAT
TT=68.639

SAT
TT=126.056

SAT
TT=65.277

SAT
TT=122.86

SAT
TT=4.028

SAT
TT=15.836

SAT
TT=53.935

SAT
TT=48.482

SAT
TT=312.947

SAT
TT=55.173

SAT
TT=85.487

SAT
TT=85.265

SAT
TT=90.004

SAT
TT=36.349

SAT
TT=154.597

? (TO)
TT=2520.07

? (TO)
TT=19306.8

? (TO)
TT=19244.9

? (TO)
TT=2520.1

? (TO)
TT=19180.1

? (TO)
TT=19454.6

SAT
TT=316.931

Crossword/Crossword-m1c-words-vg/
Crossword-m1c-words-vg-8-10.xml
UNSAT
TT=10.046

UNSAT
TT=10.046

UNSAT
TT=14.469

UNSAT
TT=33.496

UNSAT
TT=69.538

UNSAT
TT=34.059

UNSAT
TT=69.334

UNSAT
TT=35.562

UNSAT
TT=71.85

UNSAT
TT=17.314

UNSAT
TT=87.164

UNSAT
TT=439.811

UNSAT
TT=307.002

UNSAT
TT=411.442

UNSAT
TT=207.147

UNSAT
TT=94.523

UNSAT
TT=95.306

UNSAT
TT=95.068

UNSAT
TT=117.385

UNSAT
TT=125.437

? (TO)
TT=2520.11

UNSAT
TT=7871.64

UNSAT
TT=7725.57

? (TO)
TT=2520.09

UNSAT
TT=7795.47

UNSAT
TT=8266.68

UNSAT
TT=208.54

DeBruijnSequence/
DeBruijnSequence-m1-s1/
DeBruijnSequence-02-09.xml
SAT
TT=2.882

SAT
TT=11.155

SAT
TT=1117.4

SAT
TT=32.937

SAT
TT=73.89

SAT
TT=31.138

SAT
TT=62.112

SAT
TT=32.973

SAT
TT=62.075

SAT
TT=20.323

SAT
TT=56.986

SAT
TT=41.839

SAT
TT=84.83

SAT
TT=199.724

SAT
TT=83.361

SAT
TT=2.87

SAT
TT=2.882

SAT
TT=3.003

SAT
TT=6.326

SAT
TT=6.412

? (MO)
TT=734.734

? (TO)
TT=745.335

SAT
TT=1714.88

? (MO)
TT=651.679

? (TO)
TT=700.449

SAT
TT=738.252

SAT
TT=1744.41

DeBruijnSequence/
DeBruijnSequence-m1-s1/
DeBruijnSequence-03-05.xml
SAT
TT=0.252

SAT
TT=6.595

SAT
TT=35.185

SAT
TT=9.271

SAT
TT=22.013

SAT
TT=10.24

SAT
TT=19.791

SAT
TT=10.157

SAT
TT=18.69

SAT
TT=8.315

SAT
TT=22.502

SAT
TT=21.229

SAT
TT=24.622

SAT
TT=34.803

SAT
TT=24.695

SAT
TT=0.253

SAT
TT=0.252

SAT
TT=0.255

SAT
TT=0.836

SAT
TT=0.851

SAT
TT=136.909

SAT
TT=171.275

SAT
TT=166.796

SAT
TT=100.615

SAT
TT=112.836

SAT
TT=115.628

SAT
TT=243.788

Domino/Domino-m1-s1/
Domino-9000-9000.xml
SAT
TT=1.392

SAT
TT=53.15

SAT
TT=187.833

? (TO)
TT=2520.54

SAT
TT=2370.44

? (TO)
TT=2520.7

SAT
TT=732.399

? (TO)
TT=2520.34

SAT
TT=846.297

? (TO)
TT=2520.1

SAT
TT=1004.86

SAT
TT=7.281

SAT
TT=7.433

SAT
TT=7.767

SAT
TT=7.551

SAT
TT=135.813

SAT
TT=128.109

SAT
TT=131.012

SAT
TT=38.761

SAT
TT=40.422

SAT
TT=114.155

SAT
TT=139.055

SAT
TT=140.2

? (TO)
TT=2528.1

? (MO)
TT=656.239

? (MO)
TT=652.304

SAT
TT=1.392

Dubois/
Dubois-m1-s1/
Dubois-028.xml
UNSAT
TT=0.008

? (TO)
TT=2520.08

UNSAT
TT=0.009

UNSAT
TT=550.152

UNSAT
TT=18.526

UNSAT
TT=142.428

UNSAT
TT=309.942

UNSAT
TT=125.202

UNSAT
TT=298.979

? (TO)
TT=2520.08

UNSAT
TT=286.146

? (TO)
TT=2520.1

? (TO)
TT=2520.14

? (TO)
TT=2520.05

? (TO)
TT=2520.11

? (TO)
TT=2520.02

? (TO)
TT=2520.05

? (TO)
TT=2520.01

? (TO)
TT=19842

? (TO)
TT=19839.4

UNSAT
TT=4.918

UNSAT
TT=5.011

UNSAT
TT=5.024

UNSAT
TT=4.893

UNSAT
TT=5.016

UNSAT
TT=5.002

UNSAT
TT=0.008

Dubois/
Dubois-m1-s1/
Dubois-050.xml
UNSAT
TT=0.009

? (TO)
TT=2520.11

UNSAT
TT=0.012

? (TO)
TT=2520.72

UNSAT
TT=26.668

? (TO)
TT=2520.58

? (TO)
TT=19930.3

? (TO)
TT=2520.67

? (TO)
TT=19955.1

? (TO)
TT=2520.11

? (TO)
TT=19949

? (TO)
TT=2520.06

? (TO)
TT=2520.08

? (TO)
TT=2520.07

? (TO)
TT=2520.07

? (TO)
TT=2520.02

? (TO)
TT=2519.36

? (TO)
TT=2520.06

? (TO)
TT=20015.5

? (TO)
TT=20010.3

UNSAT
TT=5.614

UNSAT
TT=5.988

UNSAT
TT=6.008

UNSAT
TT=5.755

UNSAT
TT=5.909

UNSAT
TT=5.846

UNSAT
TT=0.009

Fischer/Fischer-m1-s1/
Fischer-06-08-fair.xml
UNSAT
TT=13.516

? (problem)
TT=75.608

? (MO)
TT=31.741

? (TO)
TT=2520.38

? (TO)
TT=20014.1

? (TO)
TT=2520.61

? (TO)
TT=20010.1

? (TO)
TT=2520.35

? (TO)
TT=20013.5

? (TO)
TT=2520.08

? (TO)
TT=20009.3

UNSAT
TT=2512.91

? (TO)
TT=2520.12

? (TO)
TT=2520.14

UNSAT
TT=1816.35

Signal
TT=2.636

Signal
TT=2.603

Signal
TT=2.651

Signal
TT=18.392

Signal
TT=18.217

UNSAT
TT=69.458

UNSAT
TT=70.641

UNSAT
TT=71.016

? (MO)
TT=383.55

? (TO)
TT=473.716

UNSAT
TT=628.472

UNSAT
TT=13.516

Fischer/Fischer-m1-s1/
Fischer-07-07-fair.xml
UNSAT
TT=12.085

? (problem)
TT=99.375

? (MO)
TT=32.215

? (TO)
TT=2520.65

? (TO)
TT=20014.9

? (TO)
TT=2520.43

? (TO)
TT=20011.7

? (TO)
TT=2520.63

? (TO)
TT=20019.3

? (TO)
TT=2520.07

? (TO)
TT=20016.4

UNSAT
TT=2062.8

UNSAT
TT=1608.86

? (TO)
TT=2520.13

UNSAT
TT=1600.39

Signal
TT=2.697

Signal
TT=2.71

Signal
TT=2.734

Signal
TT=18.665

Signal
TT=18.83

UNSAT
TT=64.986

UNSAT
TT=53.213

UNSAT
TT=54.063

? (MO)
TT=381.953

? (MO)
TT=486.456

? (MO)
TT=470.182

UNSAT
TT=12.085

Bench nameBest resultsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
Fischer/Fischer-m1-s1/
Fischer-08-07-fair.xml
UNSAT
TT=13.97

? (problem)
TT=83.985

? (MO)
TT=31.737

? (TO)
TT=2520.37

? (TO)
TT=20006.9

? (TO)
TT=2520.54

? (TO)
TT=20008.6

? (TO)
TT=2520.44

? (TO)
TT=20002.8

? (TO)
TT=2520.05

? (TO)
TT=20013.7

? (TO)
TT=2520.06

? (TO)
TT=2520.13

? (TO)
TT=2520.13

? (TO)
TT=2520.14

Signal
TT=3.155

Signal
TT=3.152

Signal
TT=3.191

Signal
TT=20.82

Signal
TT=21.151

UNSAT
TT=83.26

UNSAT
TT=58.094

UNSAT
TT=60.701

? (MO)
TT=461.124

? (MO)
TT=544.041

? (MO)
TT=543.5

UNSAT
TT=13.97

Fischer/Fischer-m1-s1/
Fischer-09-06-fair.xml
UNSAT
TT=14.604

? (problem)
TT=67.984

? (MO)
TT=32.207

? (TO)
TT=2520.47

UNSAT
TT=14184.1

? (TO)
TT=2520.37

UNSAT
TT=16683.2

? (TO)
TT=2520.69

UNSAT
TT=7549.03

UNSAT
TT=1483.98

UNSAT
TT=8075.83

UNSAT
TT=1081.04

? (TO)
TT=2520.06

? (TO)
TT=2520.13

UNSAT
TT=1614.58

Signal
TT=3.027

Signal
TT=3.066

Signal
TT=3.065

Signal
TT=20.259

Signal
TT=20.26

UNSAT
TT=73.235

UNSAT
TT=49.843

UNSAT
TT=50.565

? (MO)
TT=431.978

? (MO)
TT=527.577

? (MO)
TT=529.368

UNSAT
TT=14.604

Fischer/Fischer-m1-s1/
Fischer-10-06-fair.xml
UNSAT
TT=15.417

? (problem)
TT=86.788

? (MO)
TT=31.686

? (TO)
TT=2520.39

UNSAT
TT=7991.9

? (TO)
TT=2520.32

? (TO)
TT=20004

? (TO)
TT=2520.72

? (TO)
TT=19983.9

? (TO)
TT=2520.05

? (TO)
TT=19997.7

? (TO)
TT=2520.14

? (TO)
TT=2520.06

? (TO)
TT=2520.14

? (TO)
TT=2520.1

Signal
TT=3.446

Signal
TT=3.425

Signal
TT=3.486

Signal
TT=22.351

Signal
TT=22.215

UNSAT
TT=77.328

UNSAT
TT=54.106

UNSAT
TT=54.612

? (MO)
TT=484.638

? (MO)
TT=558.804

? (MO)
TT=584.137

UNSAT
TT=15.417

Fischer/Fischer-m1-s1/
Fischer-11-05-fair.xml
UNSAT
TT=14.114

? (problem)
TT=82.285

? (MO)
TT=31.596

UNSAT
TT=2515.61

UNSAT
TT=1041.51

UNSAT
TT=2142.56

UNSAT
TT=3722.76

UNSAT
TT=2181.18

UNSAT
TT=3719.97

UNSAT
TT=704.607

UNSAT
TT=3477.41

UNSAT
TT=1316.9

UNSAT
TT=1163.78

? (TO)
TT=2520.18

UNSAT
TT=1183.9

Signal
TT=3.183

Signal
TT=3.221

Signal
TT=3.181

Signal
TT=21.551

Signal
TT=21.16

UNSAT
TT=59.659

UNSAT
TT=47.707

UNSAT
TT=48.091

? (MO)
TT=438.477

? (MO)
TT=553.679

? (MO)
TT=554.197

UNSAT
TT=14.114

GracefulGraph/
GracefulGraph-K05-P02_c18.xml
SAT
TT=0.285

SAT
TT=118.681

SAT
TT=31.434

SAT
TT=75.356

SAT
TT=174.681

SAT
TT=62.319

SAT
TT=128.597

SAT
TT=63.878

SAT
TT=142.787

SAT
TT=165.944

SAT
TT=25.068

SAT
TT=58.939

SAT
TT=16.272

SAT
TT=1043.25

SAT
TT=337.333

SAT
TT=18.53

SAT
TT=18.615

SAT
TT=18.505

SAT
TT=0.29

SAT
TT=0.285

SAT
TT=7.58

SAT
TT=8.153

SAT
TT=8.225

SAT
TT=6.943

SAT
TT=7.982

SAT
TT=8.167

SAT
TT=15.62

GracefulGraph/
GracefulGraph-m1-s1/
GracefulGraph-K03-P07.xml
SAT
TT=5.041

SAT
TT=73.257

SAT
TT=81.842

SAT
TT=9.664

SAT
TT=31.511

SAT
TT=10.651

SAT
TT=20.09

SAT
TT=11.973

SAT
TT=25.261

SAT
TT=7.381

SAT
TT=15.496

SAT
TT=11.997

SAT
TT=7.448

SAT
TT=41.125

SAT
TT=10.793

SAT
TT=19.662

SAT
TT=19.559

SAT
TT=19.649

SAT
TT=5.129

SAT
TT=5.041

SAT
TT=12.185

SAT
TT=17.593

SAT
TT=16.232

SAT
TT=10.359

SAT
TT=17.061

SAT
TT=17.295

SAT
TT=58.472

GracefulGraph/
GracefulGraph-m1-s1/
GracefulGraph-K05-P02.xml
SAT
TT=0.275

SAT
TT=110.359

SAT
TT=31.389

SAT
TT=75.096

SAT
TT=165.175

SAT
TT=63.424

SAT
TT=144.11

SAT
TT=62.033

SAT
TT=131.781

SAT
TT=164.74

SAT
TT=16.116

SAT
TT=39.02

SAT
TT=1002.44

? (TO)
TT=2520.05

SAT
TT=82.744

SAT
TT=18.6

SAT
TT=18.68

SAT
TT=18.674

SAT
TT=0.284

SAT
TT=0.275

SAT
TT=7.508

SAT
TT=8.117

SAT
TT=8.09

SAT
TT=7.054

SAT
TT=8.095

SAT
TT=7.984

SAT
TT=15.656

GracefulGraph/
GracefulGraph-m1-s1/
GracefulGraph-K09-P07.xml



? (TO)
TT=2520.01

? (TO)
TT=2520.03

? (TO)
TT=2520.38

? (TO)
TT=20048.7

? (TO)
TT=2520.54

? (TO)
TT=20017.2

? (TO)
TT=2520.52

? (TO)
TT=20020

? (TO)
TT=2520.06

? (TO)
TT=19855.4

? (TO)
TT=2520.14

? (TO)
TT=2520.16

?
TT=117.256

? (TO)
TT=2520.17

? (TO)
TT=2520.02

? (TO)
TT=2520.04

? (TO)
TT=2520.08

? (TO)
TT=20118.3

? (TO)
TT=20119.7

? (TO)
TT=2520.22

? (TO)
TT=9308.46

? (TO)
TT=8755.11

? (MO)
TT=336.068

? (TO)
TT=2539.23

? (TO)
TT=2540.41

? (TO)
TT=2519.95

GracefulGraph/
GracefulGraph-m1-s1/
GracefulGraph-K13-P08.xml



? (TO)
TT=2520.11

? (TO)
TT=2519.9

? (TO)
TT=2520.64

? (TO)
TT=19953.7

? (TO)
TT=2520.62

? (TO)
TT=20095.9

? (TO)
TT=2520.46

? (TO)
TT=20100.5

? (TO)
TT=2520.03

? (TO)
TT=20091.7

? (TO)
TT=2520.12

? (TO)
TT=2520.1

?
TT=25.664

? (TO)
TT=2520.15

? (TO)
TT=2519.91

? (TO)
TT=2519.61

? (TO)
TT=2520.01

? (TO)
TT=20115.4

? (TO)
TT=20113.9

? (MO)
TT=339.969

? (TO)
TT=396.25

? (TO)
TT=2590.09

?
TT=1337.81

?
TT=1465.78

?
TT=1498.76

? (TO)
TT=2520.06

Hanoi/
Hanoi-m1-s1/
Hanoi-07.xml
SAT
TT=0.777

SAT
TT=4.22

SAT
TT=1.778

SAT
TT=29.396

SAT
TT=57.593

SAT
TT=31.109

SAT
TT=56.762

SAT
TT=29.624

SAT
TT=60.343

SAT
TT=16.645

SAT
TT=55.013

SAT
TT=10.302

SAT
TT=10.119

SAT
TT=13.271

SAT
TT=11.357

SAT
TT=0.779

SAT
TT=0.777

SAT
TT=0.781

SAT
TT=5.777

SAT
TT=6.119

SAT
TT=159.939

SAT
TT=168.309

SAT
TT=167.682

SAT
TT=161.299

SAT
TT=169.23

SAT
TT=167.174

SAT
TT=5.322

Bench nameBest resultsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
Haystacks/
Haystacks-m1-s1/
Haystacks-39.xml



? (TO)
TT=2520.08

? (TO)
TT=2519.84

? (TO)
TT=2520.44

? (TO)
TT=19588

? (TO)
TT=2520.64

? (TO)
TT=19620.2

? (TO)
TT=2521.09

? (TO)
TT=19618.1

? (TO)
TT=2520.05

? (TO)
TT=19627.4

? (TO)
TT=2520.08

? (TO)
TT=2520.13

? (TO)
TT=2520.1

? (TO)
TT=2520.13

? (TO)
TT=2485.06

? (TO)
TT=2520.01

? (TO)
TT=2449.72

? (TO)
TT=19886

? (TO)
TT=19874.7

? (TO)
TT=2520.16

? (TO)
TT=18910.3

? (TO)
TT=19195.4

? (TO)
TT=2520.15

? (TO)
TT=19227.9

? (TO)
TT=19261.5

? (TO)
TT=2519.97

Knights/
Knights-m1-s1/
Knights-015-09.xml
UNSAT
TT=0.089

UNSAT
TT=6.258

UNSAT
TT=1.159

UNSAT
TT=3.476

UNSAT
TT=7.442

UNSAT
TT=3.358

UNSAT
TT=5.902

UNSAT
TT=3.165

UNSAT
TT=5.009

UNSAT
TT=2.324

UNSAT
TT=4.993

UNSAT
TT=8.846

UNSAT
TT=8.325

UNSAT
TT=8.586

UNSAT
TT=7.55

UNSAT
TT=0.091

UNSAT
TT=0.089

UNSAT
TT=0.089

UNSAT
TT=0.154

UNSAT
TT=0.151

UNSAT
TT=39.158

UNSAT
TT=54.807

UNSAT
TT=52.907

UNSAT
TT=18.795

UNSAT
TT=53.918

UNSAT
TT=56.655

UNSAT
TT=11.158

Knights/
Knights-m1-s1/
Knights-080-25.xml
UNSAT
TT=51.522

? (TO)
TT=2520.1

? (TO)
TT=2519.87

? (TO)
TT=2520.36

? (TO)
TT=19759.4

? (TO)
TT=2520.43

? (TO)
TT=19994.6

? (TO)
TT=2520.55

? (TO)
TT=19995.6

? (TO)
TT=2520.04

? (TO)
TT=19990.5

? (TO)
TT=2520.04

? (TO)
TT=2520.09

? (TO)
TT=2520.08

? (TO)
TT=2520.04

? (TO)
TT=2520.1

? (TO)
TT=2520.02

? (TO)
TT=2520.03

? (TO)
TT=20124.3

? (TO)
TT=20121.6

UNSAT
TT=51.522

UNSAT
TT=205.635

UNSAT
TT=205.959

? (TO)
TT=2520.11

? (TO)
TT=18363.5

? (TO)
TT=18135.8

UNSAT
TT=152.615

KnightTour/
KnightTour-ext-s1/
KnightTour-08-ext03.xml
SAT
TT=2.538

SAT
TT=4.662

SAT
TT=2.682

SAT
TT=2.538

SAT
TT=4.246

SAT
TT=2.601

SAT
TT=4.094

SAT
TT=2.612

SAT
TT=5.253

SAT
TT=2.882

SAT
TT=4.615

SAT
TT=12.366

SAT
TT=13.081

SAT
TT=19.061

SAT
TT=14.547

SAT
TT=220.128

SAT
TT=190.531

SAT
TT=189.329

SAT
TT=316.131

SAT
TT=289.397

SAT
TT=60.346

SAT
TT=554.119

SAT
TT=567.623

SAT
TT=59.161

SAT
TT=565.545

SAT
TT=552.241

SAT
TT=798.236

KnightTour/
KnightTour-ext-s1/
KnightTour-12-ext04.xml
SAT
TT=46.925

SAT
TT=274.582

? (TO)
TT=2520.03

? (TO)
TT=2520.67

SAT
TT=6475.94

SAT
TT=52.41

SAT
TT=101.049

SAT
TT=46.925

SAT
TT=105.319

? (TO)
TT=2520.05

SAT
TT=5262.69

? (TO)
TT=2520.13

? (TO)
TT=2520.07

? (TO)
TT=2520.1

? (TO)
TT=2520.08

? (TO)
TT=2520.1

? (TO)
TT=2519.82

? (TO)
TT=2520.08

? (TO)
TT=20115

? (TO)
TT=20116.4

? (TO)
TT=2520.09

? (TO)
TT=18884.7

? (TO)
TT=18889.3

? (TO)
TT=2520.16

? (TO)
TT=18782.9

? (TO)
TT=18892.9

? (TO)
TT=2519.95

KnightTour/
KnightTour-ext-s1/
KnightTour-15-ext06.xml
UNSAT
TT=17.359

? (problem)
TT=62.95

UNSAT
TT=72.916

?
TT=1474.95

?
TT=1444.5

?
TT=140.585

?
TT=371.929

?
TT=127.7

?
TT=4677.65

?
TT=118.042

?
TT=406.131

UNSAT
TT=69.283

UNSAT
TT=60.097

UNSAT
TT=67.426

UNSAT
TT=62.701

UNSAT
TT=19.91

UNSAT
TT=17.359

UNSAT
TT=17.41

UNSAT
TT=51.291

UNSAT
TT=50.348

UNSAT
TT=2381.93

UNSAT
TT=1317.65

UNSAT
TT=1307.97

UNSAT
TT=2392.3

UNSAT
TT=1294.49

UNSAT
TT=1352.86

? (TO)
TT=2519.82

KnightTour/
KnightTour-ext-s1/
KnightTour-17-ext02.xml
UNSAT
TT=0.031

UNSAT
TT=4.264

UNSAT
TT=0.233

UNSAT
TT=5.522

UNSAT
TT=15.168

UNSAT
TT=5.595

UNSAT
TT=11.072

UNSAT
TT=7.09

UNSAT
TT=11.273

UNSAT
TT=2.489

UNSAT
TT=11.826

UNSAT
TT=18.152

UNSAT
TT=17.815

UNSAT
TT=19.026

UNSAT
TT=17.097

UNSAT
TT=0.037

UNSAT
TT=0.031

UNSAT
TT=0.031

UNSAT
TT=0.416

UNSAT
TT=0.418

UNSAT
TT=38.588

UNSAT
TT=38.756

UNSAT
TT=38.167

UNSAT
TT=38.54

UNSAT
TT=38.16

UNSAT
TT=38.542

UNSAT
TT=1.262

KnightTour/
KnightTour-int-s1/
KnightTour-05-int.xml
UNSAT
TT=0.033

UNSAT
TT=3.361

UNSAT
TT=0.033

UNSAT
TT=1.684

UNSAT
TT=2.56

UNSAT
TT=1.697

UNSAT
TT=2.275

UNSAT
TT=1.664

UNSAT
TT=2.132

UNSAT
TT=1.243

UNSAT
TT=2.18

UNSAT
TT=7.101

UNSAT
TT=7.338

UNSAT
TT=6.96

UNSAT
TT=6.562

UNSAT
TT=0.069

UNSAT
TT=0.061

UNSAT
TT=0.062

UNSAT
TT=0.066

UNSAT
TT=0.065

UNSAT
TT=7.652

UNSAT
TT=23.163

UNSAT
TT=23.723

UNSAT
TT=8.308

UNSAT
TT=25.123

UNSAT
TT=23.048

UNSAT
TT=5.501

KnightTour/
KnightTour-int-s1/
KnightTour-08-int.xml
SAT
TT=8.591

SAT
TT=9.621

SAT
TT=154.63

SAT
TT=18.816

SAT
TT=41.107

SAT
TT=34.34

SAT
TT=64.829

SAT
TT=39.9

SAT
TT=62.126

SAT
TT=8.591

SAT
TT=39.428

SAT
TT=51.682

? (TO)
TT=2520.13

? (TO)
TT=2520.13

SAT
TT=71.33

SAT
TT=2318.51

SAT
TT=2284.83

SAT
TT=2283.4

SAT
TT=210.097

SAT
TT=516.435

SAT
TT=222.623

SAT
TT=1930.15

SAT
TT=1947.63

? (TO)
TT=2520.11

SAT
TT=1952.34

SAT
TT=1930.19

SAT
TT=80.063

KnightTour/
KnightTour-int-s1/
KnightTour-17-int.xml
UNSAT
TT=0.736

UNSAT
TT=8.222

UNSAT
TT=1.847

UNSAT
TT=41.013

UNSAT
TT=82.682

UNSAT
TT=40.752

UNSAT
TT=80.979

UNSAT
TT=45.167

UNSAT
TT=80.928

UNSAT
TT=12.442

UNSAT
TT=80.809

UNSAT
TT=29.975

UNSAT
TT=29.068

UNSAT
TT=30.677

UNSAT
TT=30.413

UNSAT
TT=0.744

UNSAT
TT=0.745

UNSAT
TT=0.736

UNSAT
TT=1.083

UNSAT
TT=1.132

? (TO)
TT=2520.35

? (TO)
TT=9427.53

? (TO)
TT=9615.46

? (TO)
TT=2520.19

? (TO)
TT=9387.07

? (TO)
TT=9423.45

? (TO)
TT=2519.87

Bench nameBest resultsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
Langford/
Langford-m1-k3/
Langford-3-14.xml
UNSAT
TT=45.4

UNSAT
TT=233.213

UNSAT
TT=1764.21

UNSAT
TT=187.471

UNSAT
TT=315.704

UNSAT
TT=194.722

UNSAT
TT=318.222

UNSAT
TT=201.655

UNSAT
TT=323.886

UNSAT
TT=45.4

UNSAT
TT=321.406

UNSAT
TT=266.4

UNSAT
TT=739.906

UNSAT
TT=1130.6

UNSAT
TT=623.751

UNSAT
TT=565.24

UNSAT
TT=549.748

UNSAT
TT=551.319

UNSAT
TT=4513.49

UNSAT
TT=4198.88

UNSAT
TT=163.051

UNSAT
TT=599.728

UNSAT
TT=596.78

UNSAT
TT=366.333

UNSAT
TT=602.58

UNSAT
TT=604.198

UNSAT
TT=169.69

Langford/
Langford-m1-k3/
Langford-3-30.xml



? (TO)
TT=2520.1

?
TT=2520.02

? (TO)
TT=2520.78

? (TO)
TT=20046.6

? (TO)
TT=2520.51

? (TO)
TT=20010.8

? (TO)
TT=2520.63

? (TO)
TT=20016.2

? (TO)
TT=2520.13

? (TO)
TT=20006.6

? (TO)
TT=2520.09

? (TO)
TT=2520.12

? (TO)
TT=2520.09

? (TO)
TT=2520.06

? (TO)
TT=2520.05

? (TO)
TT=2520.02

? (TO)
TT=2519.92

? (TO)
TT=20120.3

? (TO)
TT=20120.9

? (TO)
TT=2520.11

? (TO)
TT=19516.1

? (TO)
TT=19567.3

? (TO)
TT=2520.15

? (TO)
TT=19546.3

? (TO)
TT=19638.8

? (TO)
TT=2519.69

Langford/
Langford-m1-k3/
Langford-3-35.xml
SAT
TT=46.149

SAT
TT=112.438

? (TO)
TT=2519.97

SAT
TT=228.882

SAT
TT=466.965

SAT
TT=126.391

SAT
TT=257.603

SAT
TT=131.499

SAT
TT=255.074

SAT
TT=67.612

SAT
TT=152.412

SAT
TT=1321.56

SAT
TT=2010.75

? (TO)
TT=2520.12

SAT
TT=1078.42

? (TO)
TT=2519.82

? (TO)
TT=2519.83

? (TO)
TT=2519.91

? (TO)
TT=20119.8

SAT
TT=1917.01

SAT
TT=372.927

SAT
TT=731.001

SAT
TT=761.468

SAT
TT=46.149

SAT
TT=757.869

SAT
TT=759.803

SAT
TT=337.942

Langford/
Langford-m1-k3/
Langford-3-45.xml
SAT
TT=185.253

SAT
TT=433.86

? (TO)
TT=2519.61

SAT
TT=524.666

SAT
TT=1119.45

SAT
TT=182.519

SAT
TT=371.555

SAT
TT=185.253

SAT
TT=407.669

? (TO)
TT=2520.1

SAT
TT=231.877

? (TO)
TT=2520.08

? (TO)
TT=2520.09

? (TO)
TT=2520.07

? (TO)
TT=2520.11

? (TO)
TT=2520.03

? (TO)
TT=2520.02

? (TO)
TT=2520.08

? (TO)
TT=20119

? (TO)
TT=20114.6

SAT
TT=450.54

SAT
TT=2253.19

SAT
TT=2234.13

? (TO)
TT=2520.11

SAT
TT=2265.44

SAT
TT=2285.34

? (TO)
TT=2519.86

Langford/
Langford-m1-k4/
Langford-4-15.xml
UNSAT
TT=22.394

UNSAT
TT=161.114

UNSAT
TT=1872.92

UNSAT
TT=84.617

UNSAT
TT=123.97

UNSAT
TT=83.31

UNSAT
TT=127.86

UNSAT
TT=88.818

UNSAT
TT=140.578

UNSAT
TT=22.394

UNSAT
TT=119.171

UNSAT
TT=104.289

UNSAT
TT=246.188

UNSAT
TT=366.467

UNSAT
TT=203.632

UNSAT
TT=339.809

UNSAT
TT=332.798

UNSAT
TT=332.641

UNSAT
TT=2068.93

UNSAT
TT=1913.03

UNSAT
TT=54.56

UNSAT
TT=193.722

UNSAT
TT=196.008

UNSAT
TT=59.491

UNSAT
TT=194.353

UNSAT
TT=195.751

UNSAT
TT=40.57

Langford/
Langford-m1-k4/
Langford-4-16.xml
UNSAT
TT=85.547

UNSAT
TT=681.784

? (TO)
TT=2519.83

UNSAT
TT=323.639

UNSAT
TT=469.046

UNSAT
TT=336.253

UNSAT
TT=472.048

UNSAT
TT=357.206

UNSAT
TT=493.88

UNSAT
TT=86.993

UNSAT
TT=507.451

UNSAT
TT=331.139

UNSAT
TT=1018.23

UNSAT
TT=1688.68

UNSAT
TT=842.612

UNSAT
TT=1752.39

UNSAT
TT=1500.24

UNSAT
TT=1497.89

UNSAT
TT=13977.5

UNSAT
TT=11356.6

UNSAT
TT=172.898

UNSAT
TT=521.533

UNSAT
TT=531.108

UNSAT
TT=115.741

UNSAT
TT=522.385

UNSAT
TT=531.741

UNSAT
TT=85.547

Langford/
Langford-m2-s1/
LangfordBin-25.xml



? (TO)
TT=2520.09

?
TT=2520.02

? (TO)
TT=2520.27

? (TO)
TT=20057.4

? (TO)
TT=2520.32

? (TO)
TT=20058.7

? (TO)
TT=2520.59

? (TO)
TT=20048.9

? (TO)
TT=2520.08

? (TO)
TT=20059.5

? (TO)
TT=2520.1

? (TO)
TT=2520.04

? (TO)
TT=2520.13

? (TO)
TT=2520.07

? (TO)
TT=2519.9

? (TO)
TT=2519.76

? (TO)
TT=2520.04

? (TO)
TT=20083.6

? (TO)
TT=20035.5

? (TO)
TT=2520.08

? (TO)
TT=17086.9

? (TO)
TT=17050.4

? (TO)
TT=2520.1

? (TO)
TT=17213.6

? (TO)
TT=17080.5

? (TO)
TT=2519.92

LatinSquare/
LatinSquare-m1-gp/
qwh-o30-h374-01.xml
SAT
TT=8.472

SAT
TT=22.142

? (TO)
TT=2519.97

SAT
TT=152.175

SAT
TT=318.435

SAT
TT=7.63

SAT
TT=18.701

SAT
TT=8.472

SAT
TT=14.093

SAT
TT=113.018

SAT
TT=459.664

SAT
TT=47.828

SAT
TT=44

SAT
TT=38.967

SAT
TT=39.318

? (TO)
TT=2519.92

? (TO)
TT=2519.58

? (TO)
TT=2520.08

? (TO)
TT=20117.9

? (TO)
TT=20115.1

SAT
TT=82.314

SAT
TT=136.273

SAT
TT=132.542

SAT
TT=49.92

SAT
TT=133.067

SAT
TT=131.005

SAT
TT=12.182

LatinSquare/
LatinSquare-m1-gp/
qwh-o30-h374-02.xml
SAT
TT=10.202

SAT
TT=13.603

?
TT=2520.03

SAT
TT=374.188

SAT
TT=797.45

? (TO)
TT=2520.47

SAT
TT=1230.11

? (TO)
TT=2520.39

SAT
TT=1171.99

SAT
TT=775.063

SAT
TT=782.498

SAT
TT=10.202

SAT
TT=11.086

SAT
TT=86.598

SAT
TT=35.507

? (TO)
TT=2520.06

? (TO)
TT=2519.95

? (TO)
TT=2520.1

? (TO)
TT=20118.7

? (TO)
TT=20113.6

SAT
TT=42.883

SAT
TT=125.818

SAT
TT=123.315

SAT
TT=35.104

SAT
TT=120.131

SAT
TT=119.806

SAT
TT=13.417

LatinSquare/
LatinSquare-m1-gp/
qwh-o30-h374-05.xml
SAT
TT=10.488

SAT
TT=36.155

? (TO)
TT=2519.93

SAT
TT=161.792

SAT
TT=311.71

SAT
TT=1088.67

SAT
TT=2334.93

SAT
TT=1064.68

SAT
TT=2067.09

SAT
TT=331.32

SAT
TT=3068.55

SAT
TT=44.147

SAT
TT=287.633

SAT
TT=234.513

SAT
TT=90.58

? (TO)
TT=2520.05

? (TO)
TT=2520.08

? (TO)
TT=2520.04

? (TO)
TT=20113.2

? (TO)
TT=20116.3

SAT
TT=87.137

SAT
TT=160.333

SAT
TT=161.584

SAT
TT=119.809

SAT
TT=161.802

SAT
TT=166.721

SAT
TT=10.488

Bench nameBest resultsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
LatinSquare/
LatinSquare-m1-gp/
qwh-o30-h374-07.xml
SAT
TT=18.108

SAT
TT=156.374

? (TO)
TT=2520.01

? (TO)
TT=2520.52

? (TO)
TT=19951.3

? (TO)
TT=2520.49

SAT
TT=6226.53

? (TO)
TT=2520.48

SAT
TT=5848.35

SAT
TT=826.027

SAT
TT=7423.99

SAT
TT=54.984

SAT
TT=539.178

SAT
TT=234.697

SAT
TT=752.445

? (TO)
TT=2520

? (TO)
TT=2519.93

? (TO)
TT=2519.79

? (TO)
TT=20119.5

? (TO)
TT=20115.8

SAT
TT=116.904

SAT
TT=278.117

SAT
TT=280.933

SAT
TT=95.729

SAT
TT=281.565

SAT
TT=277.395

SAT
TT=18.108

LatinSquare/
LatinSquare-m1-gp/
qwh-o30-h374-08.xml
SAT
TT=9.154

SAT
TT=19.983

? (TO)
TT=2520.01

SAT
TT=432.567

SAT
TT=865.816

SAT
TT=241.071

SAT
TT=469.466

SAT
TT=231.137

SAT
TT=489.639

SAT
TT=1436.47

SAT
TT=1392.15

SAT
TT=26.247

SAT
TT=174.693

SAT
TT=54.85

SAT
TT=15.37

? (TO)
TT=2520.03

? (TO)
TT=2519.94

? (TO)
TT=2519.71

? (TO)
TT=20116.3

? (TO)
TT=20116.1

SAT
TT=60.311

SAT
TT=103.55

SAT
TT=113.984

SAT
TT=47.693

SAT
TT=103.462

SAT
TT=109.936

SAT
TT=9.154

LatinSquare/
LatinSquare-m1-gp/
qwh-o30-h374-11.xml
SAT
TT=18.79

SAT
TT=52.28

?
TT=2520.03

? (TO)
TT=2520.74

SAT
TT=10304.9

? (TO)
TT=2520.39

? (TO)
TT=20045.3

? (TO)
TT=2520.68

SAT
TT=13022.3

? (TO)
TT=2520.03

SAT
TT=17016.2

SAT
TT=212.34

SAT
TT=144.797

? (TO)
TT=2520.07

SAT
TT=18.79

? (TO)
TT=2520.06

? (TO)
TT=2519.93

? (TO)
TT=2520.03

? (TO)
TT=20115.3

? (TO)
TT=20116.3

SAT
TT=122.017

SAT
TT=228.609

SAT
TT=239.958

SAT
TT=139.992

SAT
TT=213.087

SAT
TT=235.787

SAT
TT=19.645

LatinSquare/
LatinSquare-m1-gp/
qwh-o30-h374-19.xml
SAT
TT=32.512

SAT
TT=432.688

?
TT=2520.03

SAT
TT=797.455

SAT
TT=1587.59

? (TO)
TT=2520.34

SAT
TT=6578.23

? (TO)
TT=2520.38

SAT
TT=6473.57

? (TO)
TT=2520.12

SAT
TT=16648.6

SAT
TT=237.719

SAT
TT=597.225

SAT
TT=1723.93

SAT
TT=223.166

? (TO)
TT=2519.44

? (TO)
TT=2520.02

? (TO)
TT=2520.04

? (TO)
TT=20116.3

? (TO)
TT=20116.9

SAT
TT=91.05

SAT
TT=217.651

SAT
TT=215.651

SAT
TT=134.119

SAT
TT=212.491

SAT
TT=215.996

SAT
TT=32.512

LatinSquare/
LatinSquare-m1-gp/
qwh-o30-h375-01.xml
SAT
TT=11.87

SAT
TT=65.798

? (TO)
TT=2519.73

SAT
TT=944.66

SAT
TT=1992.03

? (TO)
TT=2520.42

? (TO)
TT=20044.7

? (TO)
TT=2520.48

? (TO)
TT=20049.2

SAT
TT=2197.74

SAT
TT=18145.8

SAT
TT=343.25

SAT
TT=439.986

SAT
TT=936.29

SAT
TT=317.59

? (TO)
TT=2520.06

? (TO)
TT=2520.06

? (TO)
TT=2520.06

? (TO)
TT=20116.6

? (TO)
TT=20116.4

SAT
TT=96.36

SAT
TT=480.808

SAT
TT=490.469

SAT
TT=179.69

SAT
TT=480.426

SAT
TT=484.371

SAT
TT=11.87

LatinSquare/
LatinSquare-m1-gp/
qwh-o30-h375-06.xml
SAT
TT=18.363

SAT
TT=353.413

? (TO)
TT=2520.02

SAT
TT=341.737

SAT
TT=725.532

? (TO)
TT=2520.54

SAT
TT=15829.2

? (TO)
TT=2520.59

? (TO)
TT=20046.1

? (TO)
TT=2520.08

? (TO)
TT=20042.4

SAT
TT=120.684

SAT
TT=269.107

SAT
TT=263.908

SAT
TT=40.806

? (TO)
TT=2519.94

? (TO)
TT=2519.89

? (TO)
TT=2519.78

? (TO)
TT=20117.3

? (TO)
TT=20116.7

SAT
TT=95.349

SAT
TT=562.433

SAT
TT=562.259

SAT
TT=161.739

SAT
TT=560.611

SAT
TT=574.108

SAT
TT=18.363

LatinSquare/
LatinSquare-m1-gp/
qwh-o30-h375-07.xml
SAT
TT=9.084

SAT
TT=17.808

? (TO)
TT=2519.65

SAT
TT=1305.65

SAT
TT=2652.12

SAT
TT=1153.26

SAT
TT=2291.36

SAT
TT=1153.4

SAT
TT=2359.4

SAT
TT=1279.55

SAT
TT=282.4

SAT
TT=13.983

SAT
TT=83.953

SAT
TT=69.626

SAT
TT=89.647

? (TO)
TT=2519.98

? (TO)
TT=2520.03

? (TO)
TT=2519.88

? (TO)
TT=20116.2

? (TO)
TT=20119.6

SAT
TT=89.908

SAT
TT=126.039

SAT
TT=128.957

SAT
TT=87.5

SAT
TT=124.26

SAT
TT=127.008

SAT
TT=9.084

LatinSquare/
LatinSquare-m1-gp/
qwh-o30-h375-19.xml
SAT
TT=6.572

SAT
TT=6.572

?
TT=2520.02

SAT
TT=1162.95

SAT
TT=2374.81

SAT
TT=1433.17

SAT
TT=2737.28

SAT
TT=1371.61

SAT
TT=2680.55

SAT
TT=1100.63

SAT
TT=2918.63

SAT
TT=66.186

SAT
TT=506.89

SAT
TT=180.011

SAT
TT=201.681

? (TO)
TT=2519.81

? (TO)
TT=2520.08

? (TO)
TT=2520.07

? (TO)
TT=20116.2

? (TO)
TT=20120.1

SAT
TT=61.049

SAT
TT=225.632

SAT
TT=230.393

SAT
TT=60.262

SAT
TT=225.124

SAT
TT=236.259

SAT
TT=11.021

LatinSquare/
LatinSquare-m1-gp/
qwh-o30-h375-20.xml
SAT
TT=28.868

SAT
TT=28.868

? (TO)
TT=2520.02

? (TO)
TT=2520.53

? (TO)
TT=20072.4

? (TO)
TT=2520.37

? (TO)
TT=20046.7

? (TO)
TT=2520.45

? (TO)
TT=20051.9

? (TO)
TT=2520.13

? (TO)
TT=20051.2

SAT
TT=297.395

SAT
TT=2096.82

? (TO)
TT=2520.11

SAT
TT=345.901

? (TO)
TT=2520.08

? (TO)
TT=2520.01

? (TO)
TT=2520.01

? (TO)
TT=20118.5

? (TO)
TT=20118.6

SAT
TT=160.882

SAT
TT=964.132

SAT
TT=966.437

SAT
TT=153.855

SAT
TT=985.342

SAT
TT=1014.72

SAT
TT=41.201

LatinSquare/
LatinSquare-m1-gs/
qwh-o100-h10000.xml
SAT
TT=67.705

SAT
TT=133.527

? (TO)
TT=2519.58

SAT
TT=67.705

SAT
TT=145.146

SAT
TT=72.78

SAT
TT=152.858

SAT
TT=77.438

SAT
TT=153.518

? (TO)
TT=2520.11

SAT
TT=154.451

SAT
TT=1540.28

? (TO)
TT=2520.09

? (TO)
TT=2520.44

SAT
TT=1502.85

? (TO)
TT=2520.07

? (TO)
TT=2519.82

? (TO)
TT=2519.81

? (TO)
TT=20114.2

? (TO)
TT=20115.2

? (MO)
TT=1528.23

? (TO)
TT=1317.96

SAT
TT=1344.84

? (MO)
TT=1459.99

? (TO)
TT=1353.59

SAT
TT=1347.15

? (TO)
TT=2519.71

Bench nameBest resultsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
LatinSquare/
LatinSquare-xcsp2-qcp25/
qcp-25-264-08_X2.xml
SAT
TT=7.737

? (TO)
TT=2520.07

? (TO)
TT=2520.02

? (TO)
TT=2520.46

? (TO)
TT=20090.9

? (TO)
TT=2520.67

? (TO)
TT=20095.1

? (TO)
TT=2520.72

? (TO)
TT=20094.4

? (TO)
TT=2520.06

? (TO)
TT=20093.2

? (TO)
TT=2520.1

? (TO)
TT=2520.13

? (TO)
TT=2520.16

? (TO)
TT=2520.07

? (TO)
TT=2520.09

? (TO)
TT=2520.05

? (TO)
TT=2520.03

? (TO)
TT=20091.3

? (TO)
TT=20094.9

SAT
TT=25.728

SAT
TT=64.509

SAT
TT=63.528

SAT
TT=33.043

SAT
TT=62.649

SAT
TT=62.088

SAT
TT=7.737

LatinSquare/
LatinSquare-xcsp2-qcp25/
qcp-25-264-12_X2.xml
UNSAT
TT=3.287

? (TO)
TT=2520.08

? (TO)
TT=2519.93

? (exit code)
TT=-1

UNSAT
TT=120.252

? (TO)
TT=2520.52

UNSAT
TT=6113.15

? (TO)
TT=2520.35

UNSAT
TT=5798.52

UNSAT
TT=2059.93

UNSAT
TT=6330.17

? (TO)
TT=2520.04

UNSAT
TT=373.53

? (TO)
TT=2520.1

? (TO)
TT=2520.07

UNSAT
TT=97.022

UNSAT
TT=87.787

UNSAT
TT=114.456

UNSAT
TT=16.374

UNSAT
TT=23.787

UNSAT
TT=21.889

UNSAT
TT=24.121

UNSAT
TT=24.112

UNSAT
TT=21.711

UNSAT
TT=22.716

UNSAT
TT=22.118

UNSAT
TT=3.287

MagicHexagon/
MagicHexagon-m1-s1/
MagicHexagon-04-m018.xml
SAT
TT=2.55

SAT
TT=5.799

SAT
TT=9.819

SAT
TT=10.151

SAT
TT=15.162

SAT
TT=8.446

SAT
TT=25.166

SAT
TT=10.073

SAT
TT=20.403

SAT
TT=5.412

SAT
TT=19.492

SAT
TT=53.519

SAT
TT=79.659

SAT
TT=204.004

SAT
TT=167.081

SAT
TT=4.606

SAT
TT=4.61

SAT
TT=4.562

SAT
TT=2.915

SAT
TT=2.55

SAT
TT=46.152

SAT
TT=5348.73

SAT
TT=5357.66

SAT
TT=158.591

SAT
TT=125.527

SAT
TT=128.387

SAT
TT=140.299

MagicHexagon/
MagicHexagon-m1-s1/
MagicHexagon-06-0021.xml
SAT
TT=50.893

? (TO)
TT=2520.05

? (TO)
TT=2519.5

? (TO)
TT=2520.72

? (TO)
TT=20033.3

? (TO)
TT=2520.52

SAT
TT=49.385

? (TO)
TT=2520.62

SAT
TT=50.893

? (TO)
TT=2520.04

SAT
TT=53.072

? (TO)
TT=2520.15

? (TO)
TT=2520.13

? (TO)
TT=2520.05

? (TO)
TT=2520.08

? (TO)
TT=2519.86

? (TO)
TT=2520.07

? (TO)
TT=2519.8

? (TO)
TT=20113.2

? (TO)
TT=20113.4

? (TO)
TT=2520.13

? (TO)
TT=18318.4

? (TO)
TT=18085.7

? (TO)
TT=2520.15

? (TO)
TT=19129.3

? (TO)
TT=19103.7

? (TO)
TT=2519.65

MagicHexagon/
MagicHexagon-m1-s1/
MagicHexagon-06-m100.xml
SAT
TT=11498

? (TO)
TT=2520.05

? (TO)
TT=2519.84

? (TO)
TT=2520.57

? (TO)
TT=20004.6

? (TO)
TT=2520.66

? (TO)
TT=19991

? (TO)
TT=2520.41

? (TO)
TT=19977.9

? (TO)
TT=2520.13

? (TO)
TT=19979.6

? (TO)
TT=2520.12

? (TO)
TT=2520.11

? (TO)
TT=2520.05

? (TO)
TT=2520.15

? (TO)
TT=2519.46

? (TO)
TT=2520.04

? (TO)
TT=2519.85

SAT
TT=11498

SAT
TT=12169.8

? (TO)
TT=2520.07

? (TO)
TT=17887.3

? (TO)
TT=17866.5

? (TO)
TT=2520.11

? (TO)
TT=18676.8

? (TO)
TT=18668.3

? (TO)
TT=2519.82

MagicHexagon/
MagicHexagon-m1-s1/
MagicHexagon-07-0080.xml
UNSAT
TT=139.896

? (TO)
TT=2520.03

? (TO)
TT=2519.85

? (TO)
TT=2520.38

UNSAT
TT=6257.36

? (TO)
TT=2520.53

UNSAT
TT=6780.84

? (TO)
TT=2520.25

UNSAT
TT=6668.05

? (TO)
TT=2520.13

UNSAT
TT=6819.67

? (TO)
TT=2520.07

? (TO)
TT=2520.11

? (TO)
TT=2520.07

? (TO)
TT=2520.12

? (TO)
TT=2520.02

? (TO)
TT=2519.97

? (TO)
TT=2519.81

? (TO)
TT=20115

? (TO)
TT=20114

? (TO)
TT=2520.11

? (TO)
TT=15515.6

? (TO)
TT=15462.7

UNSAT
TT=139.896

UNSAT
TT=472.418

UNSAT
TT=495.395

UNSAT
TT=1158.69

MagicHexagon/
MagicHexagon-m1-s1/
MagicHexagon-08-m039.xml



? (TO)
TT=2520.04

? (TO)
TT=2520

? (TO)
TT=2520.55

? (TO)
TT=20002

? (TO)
TT=2520.71

? (TO)
TT=19837.5

? (TO)
TT=2520.51

? (TO)
TT=19845.3

? (TO)
TT=2520.18

? (TO)
TT=19858.2

? (TO)
TT=2520.11

? (TO)
TT=2520.16

? (TO)
TT=2520.17

? (TO)
TT=2520.11

? (TO)
TT=2520.02

? (TO)
TT=2520.03

? (TO)
TT=2520.08

? (TO)
TT=20114.4

? (TO)
TT=20119.6

? (TO)
TT=2520.13

? (TO)
TT=19147.9

? (TO)
TT=18712.6

? (MO)
TT=165.874

? (TO)
TT=2530.82

? (TO)
TT=2531.19

? (TO)
TT=2519.82

MagicHexagon/
MagicHexagon-m1-s1/
MagicHexagon-09-m057.xml



? (TO)
TT=2520.06

? (TO)
TT=2520.05

? (TO)
TT=2520.48

? (TO)
TT=19926.6

? (TO)
TT=2520.6

? (TO)
TT=19826.4

? (TO)
TT=2520.34

? (TO)
TT=19791.5

? (TO)
TT=2520.15

? (TO)
TT=19792.4

? (TO)
TT=2520.12

? (TO)
TT=2520.07

? (TO)
TT=2520.12

? (TO)
TT=2520.15

? (TO)
TT=2519.96

? (TO)
TT=2519.95

? (TO)
TT=2520.05

? (TO)
TT=20117.8

? (TO)
TT=20118.7

? (TO)
TT=2520.18

? (TO)
TT=18316

? (TO)
TT=18183.6

? (MO)
TT=374.722

? (TO)
TT=448.713

? (TO)
TT=2548.15

? (TO)
TT=2519.97

MagicSequence/
MagicSequence-card-s1/
MagicSequence-1000-ca.xml
SAT
TT=3.157

SAT
TT=93.941

SAT
TT=230.585

SAT
TT=6.582

SAT
TT=14.312

SAT
TT=11.608

SAT
TT=13.332

SAT
TT=9.586

SAT
TT=12.924

SAT
TT=3.157

SAT
TT=12.242

SAT
TT=205.493

SAT
TT=162.125

?
TT=59.187

SAT
TT=184.631

SAT
TT=102.292

SAT
TT=91.237

SAT
TT=64.94

SAT
TT=603.258

SAT
TT=581.14

SAT
TT=1772.37

SAT
TT=1820.98

SAT
TT=1823.35

SAT
TT=1437.31

SAT
TT=1435.45

SAT
TT=1445.09

SAT
TT=678.639

MagicSequence/
MagicSequence-card-s1/
MagicSequence-800-ca.xml
SAT
TT=2.786

SAT
TT=54.04

SAT
TT=126.651

SAT
TT=6.22

SAT
TT=12.36

SAT
TT=5.847

SAT
TT=11.059

SAT
TT=6.694

SAT
TT=17.683

SAT
TT=2.786

SAT
TT=8.996

SAT
TT=28.906

SAT
TT=137.307

?
TT=37.554

SAT
TT=49.024

SAT
TT=41.746

SAT
TT=46.632

SAT
TT=42.227

SAT
TT=224.076

SAT
TT=278.479

SAT
TT=880.126

SAT
TT=995.887

SAT
TT=988.048

SAT
TT=669.085

SAT
TT=850.498

SAT
TT=838.876

SAT
TT=535.786

Bench nameBest resultsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
MagicSequence/
MagicSequence-count-s1/
MagicSequence-1000-co.xml
SAT
TT=10.739

SAT
TT=95.127

SAT
TT=239.931

SAT
TT=37.582

SAT
TT=71.881

SAT
TT=74.995

SAT
TT=74.216

SAT
TT=72.694

SAT
TT=81.233

SAT
TT=10.739

SAT
TT=72.173

SAT
TT=97.715

SAT
TT=184.392

?
TT=51.788

SAT
TT=254.48

SAT
TT=91.192

SAT
TT=91.149

SAT
TT=91.012

SAT
TT=448.59

SAT
TT=333.803

SAT
TT=1765.21

SAT
TT=1775.46

SAT
TT=1791.27

SAT
TT=1334.46

SAT
TT=1444.01

SAT
TT=1414.31

SAT
TT=674.516

MagicSequence/
MagicSequence-count-s1/
MagicSequence-600-co.xml
SAT
TT=7.064

SAT
TT=20.785

SAT
TT=58.801

SAT
TT=17.255

SAT
TT=42.351

SAT
TT=25.312

SAT
TT=36.577

SAT
TT=21.378

SAT
TT=29.552

SAT
TT=7.064

SAT
TT=24.489

SAT
TT=44.243

SAT
TT=66.102

?
TT=21.885

SAT
TT=60.285

SAT
TT=14.687

SAT
TT=14.488

SAT
TT=14.306

SAT
TT=110.966

SAT
TT=36.066

SAT
TT=397.05

SAT
TT=456.381

SAT
TT=457.871

? (NS)
TT=1938.7

? (NS)
TT=12601.5

? (NS)
TT=13252.1

SAT
TT=392.773

MagicSquare/
MagicSquare-mdd-s1/
MagicSquare-9-mdd.xml
SAT
TT=23.367

SAT
TT=84.746

? (TO)
TT=2519.77

SAT
TT=161.423

SAT
TT=327.855

SAT
TT=106.59

SAT
TT=216.661

SAT
TT=108.46

SAT
TT=218.006

? (TO)
TT=2520.06

SAT
TT=253.485

SAT
TT=35.853

SAT
TT=32.841

SAT
TT=93.2

SAT
TT=25.26

SAT
TT=26.101

SAT
TT=23.367

SAT
TT=23.373

SAT
TT=31.328

SAT
TT=31.9

? (TO)
TT=2520.15

? (TO)
TT=15601.5

? (TO)
TT=15580.4

? (TO)
TT=2520.1

? (TO)
TT=15545

? (TO)
TT=15643.5

? (TO)
TT=2520.02

MagicSquare/
MagicSquare-sum-s1/
MagicSquare-12-sum.xml
SAT
TT=4.445

SAT
TT=139.782

?
TT=2520.01

SAT
TT=60.412

SAT
TT=124.549

SAT
TT=134.038

SAT
TT=249.379

SAT
TT=127.858

SAT
TT=251.614

? (TO)
TT=2520.12

SAT
TT=41.282

SAT
TT=20.818

SAT
TT=35.237

SAT
TT=46.445

SAT
TT=22.606

SAT
TT=5.142

SAT
TT=4.445

SAT
TT=4.45

SAT
TT=26.821

SAT
TT=25.775

? (TO)
TT=2520.14

? (TO)
TT=18247.2

? (TO)
TT=18088.6

? (TO)
TT=2520.27

? (TO)
TT=4687.57

SAT
TT=4464.38

? (TO)
TT=2519.87

MagicSquare/
MagicSquare-sum-s1/
MagicSquare-13-sum.xml
SAT
TT=16.612

? (TO)
TT=2520.12

?
TT=2520.02

SAT
TT=550.449

SAT
TT=1055.18

SAT
TT=113.146

SAT
TT=224.866

SAT
TT=108.313

SAT
TT=216.249

? (TO)
TT=2520.1

SAT
TT=116.866

SAT
TT=86.469

SAT
TT=98.163

SAT
TT=39.992

SAT
TT=23.386

SAT
TT=19.201

SAT
TT=16.612

SAT
TT=16.65

SAT
TT=105.069

SAT
TT=102.14

SAT
TT=1139.4

? (TO)
TT=18343.9

? (TO)
TT=18526

? (TO)
TT=2520.48

? (TO)
TT=2531.3

? (TO)
TT=2534.09

? (TO)
TT=2519.7

MaxCSP/MaxCSP-kbtree-s9-5/
kbtree-9-5-3-5-80-33.xml
UNSAT
TT=0.006

UNSAT
TT=3.474

UNSAT
TT=0.01

UNSAT
TT=1.789

UNSAT
TT=2.448

UNSAT
TT=1.797

UNSAT
TT=2.386

UNSAT
TT=1.77

UNSAT
TT=2.413

UNSAT
TT=1.152

UNSAT
TT=2.403

UNSAT
TT=5.933

UNSAT
TT=5.959

UNSAT
TT=6.545

UNSAT
TT=5.321

UNSAT
TT=0.014

UNSAT
TT=0.013

UNSAT
TT=0.013

UNSAT
TT=0.019

UNSAT
TT=0.019

UNSAT
TT=2.109

UNSAT
TT=2.178

UNSAT
TT=2.205

UNSAT
TT=2.088

UNSAT
TT=2.2

UNSAT
TT=2.156

UNSAT
TT=0.006

MaxCSP/MaxCSP-maxclique-s1/
MaxCSP-brock-800-2.xml
UNSAT
TT=1.221

UNSAT
TT=12.766

UNSAT
TT=2.214

UNSAT
TT=18.795

UNSAT
TT=43.924

UNSAT
TT=18.067

UNSAT
TT=42.813

UNSAT
TT=18.453

UNSAT
TT=42.085

UNSAT
TT=7.247

UNSAT
TT=44.36

UNSAT
TT=25.952

UNSAT
TT=25.03

UNSAT
TT=25.248

UNSAT
TT=24.738

UNSAT
TT=1.888

UNSAT
TT=1.872

UNSAT
TT=1.868

UNSAT
TT=4.316

UNSAT
TT=4.373

UNSAT
TT=20.59

UNSAT
TT=19.72

UNSAT
TT=20.781

UNSAT
TT=20.234

UNSAT
TT=20.262

UNSAT
TT=21.076

UNSAT
TT=1.221

Mixed/
Mixed-xcsp2-cril/
cril-2_X2.xml
UNSAT
TT=0.103

UNSAT
TT=8.218

UNSAT
TT=190.35

UNSAT
TT=4.262

UNSAT
TT=7.102

UNSAT
TT=4.193

UNSAT
TT=6.947

UNSAT
TT=4.339

UNSAT
TT=7.391

UNSAT
TT=2.498

UNSAT
TT=6.998

UNSAT
TT=122.641

UNSAT
TT=28.39

UNSAT
TT=186.591

UNSAT
TT=114.052

UNSAT
TT=0.104

UNSAT
TT=0.103

UNSAT
TT=0.105

UNSAT
TT=0.186

UNSAT
TT=0.168

UNSAT
TT=15.233

UNSAT
TT=17.398

UNSAT
TT=17.572

UNSAT
TT=15.697

UNSAT
TT=17.605

UNSAT
TT=17.125

UNSAT
TT=0.14

Nonogram/
Nonogram-table-s1/
Nonogram-020-table.xml
SAT
TT=3.701

SAT
TT=11.781

SAT
TT=3.701

SAT
TT=27.442

SAT
TT=65.502

SAT
TT=27.289

SAT
TT=65.205

SAT
TT=27.518

SAT
TT=67.605

SAT
TT=7.198

SAT
TT=63.981

SAT
TT=20.448

SAT
TT=20.711

SAT
TT=20.678

SAT
TT=19.644

SAT
TT=19.738

SAT
TT=19.774

SAT
TT=19.687

SAT
TT=26.405

SAT
TT=26.624

SAT
TT=128.484

SAT
TT=126.395

SAT
TT=160.568

SAT
TT=129.804

SAT
TT=131.276

SAT
TT=131.342

SAT
TT=15.567

Nonogram/
Nonogram-table-s1/
Nonogram-122-table.xml
SAT
TT=32.326

SAT
TT=72.907

SAT
TT=32.326

?
TT=721.737

?
TT=3132.67

?
TT=784.847

?
TT=3249.78

?
TT=761.62

?
TT=3261.3

SAT
TT=65.102

?
TT=3054.21

SAT
TT=112.421

SAT
TT=111.199

SAT
TT=114.535

SAT
TT=112.774

SAT
TT=199.4

SAT
TT=172.87

SAT
TT=173.021

SAT
TT=219.553

SAT
TT=229.26

SAT
TT=618.291

SAT
TT=642.244

SAT
TT=656.967

SAT
TT=621.315

SAT
TT=653.44

SAT
TT=641.77

SAT
TT=159.167

Bench nameBest resultsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
NumberPartitioning/
NumberPartitioning-m1-s1/
NumberPartitioning-064.xml
SAT
TT=5.184

SAT
TT=5.184

SAT
TT=76.355

SAT
TT=18.046

SAT
TT=32.919

SAT
TT=18.841

SAT
TT=31.706

SAT
TT=19.13

SAT
TT=31.436

SAT
TT=6.873

SAT
TT=30.979

SAT
TT=20.905

SAT
TT=19.991

SAT
TT=15.131

SAT
TT=14.656

? (TO)
TT=2519.93

? (TO)
TT=2519.89

? (TO)
TT=2520.01

SAT
TT=38.641

SAT
TT=36.882

SAT
TT=153.781

SAT
TT=122.137

SAT
TT=121.222

? (TO)
TT=2528.05

? (TO)
TT=17646.9

? (TO)
TT=17557.4

SAT
TT=13.372

NumberPartitioning/
NumberPartitioning-m1-s1/
NumberPartitioning-116.xml
SAT
TT=23.713

SAT
TT=23.713

SAT
TT=520.022

SAT
TT=161.187

SAT
TT=336.459

SAT
TT=380.912

SAT
TT=318.665

SAT
TT=406.212

SAT
TT=303.717

? (TO)
TT=2520.1

SAT
TT=322.72

SAT
TT=268.208

SAT
TT=36.712

SAT
TT=727.017

SAT
TT=31.3

? (TO)
TT=2520.03

? (TO)
TT=2520.04

? (TO)
TT=2520.05

? (TO)
TT=20096.2

? (TO)
TT=20096.8

SAT
TT=1114.82

? (TO)
TT=16994.5

? (TO)
TT=16865.4

? (TO)
TT=2528.09

? (NS)
TT=3518.96

? (NS)
TT=3644.44

SAT
TT=26.203

NumberPartitioning/
NumberPartitioning-m1-s1/
NumberPartitioning-128.xml
SAT
TT=25.859

SAT
TT=25.859

SAT
TT=129.069

SAT
TT=260.84

SAT
TT=544.244

SAT
TT=218.711

SAT
TT=457.39

SAT
TT=217.375

SAT
TT=459.403

SAT
TT=55.387

SAT
TT=447.68

SAT
TT=144.49

SAT
TT=1840.94

SAT
TT=905.384

? (TO)
TT=2520.17

? (TO)
TT=2520.03

? (TO)
TT=2520.03

? (TO)
TT=2520.05

? (TO)
TT=20082.8

? (TO)
TT=20079.6

SAT
TT=1413.47

? (TO)
TT=16705.6

? (TO)
TT=16686.9

? (NS)
TT=1711.86

? (NS)
TT=3646.5

? (NS)
TT=3705.59

SAT
TT=48.154

NumberPartitioning/
NumberPartitioning-m1-s1/
NumberPartitioning-144.xml
SAT
TT=31.334

SAT
TT=31.334

SAT
TT=155.088

SAT
TT=41.351

SAT
TT=94.683

SAT
TT=155.023

SAT
TT=297.822

SAT
TT=145.133

SAT
TT=37.611

? (TO)
TT=2520.11

SAT
TT=36.065

SAT
TT=74.84

SAT
TT=1151.25

SAT
TT=41.534

? (TO)
TT=2520.28

? (TO)
TT=2520.02

? (TO)
TT=2520.04

? (TO)
TT=2520.04

? (TO)
TT=20084.2

? (TO)
TT=20081.9

? (TO)
TT=2520.12

? (TO)
TT=17496.8

? (TO)
TT=16473.8

? (NS)
TT=1507.34

? (NS)
TT=3509.61

? (NS)
TT=3554.11

SAT
TT=151.011

NumberPartitioning/
NumberPartitioning-m1-s1/
NumberPartitioning-152.xml
SAT
TT=6.685

SAT
TT=33.745

SAT
TT=1318.71

SAT
TT=22.797

SAT
TT=51.733

SAT
TT=17.576

SAT
TT=33.838

SAT
TT=17.304

SAT
TT=43.329

SAT
TT=6.685

SAT
TT=41.926

SAT
TT=18.887

? (TO)
TT=2520.11

? (TO)
TT=2520.11

SAT
TT=724.426

? (TO)
TT=2520.02

? (TO)
TT=2519.94

? (TO)
TT=2520.04

? (TO)
TT=20082.5

? (TO)
TT=20085.6

? (TO)
TT=2520.19

? (TO)
TT=15498.8

? (TO)
TT=15499.5

? (NS)
TT=1506.1

? (NS)
TT=3552.53

? (NS)
TT=3580.77

SAT
TT=773.418

NumberPartitioning/
NumberPartitioning-m1-s1/
NumberPartitioning-164.xml
SAT
TT=118.733

SAT
TT=203.099

? (TO)
TT=2520.02

SAT
TT=118.733

SAT
TT=349.156

SAT
TT=1399.99

SAT
TT=2756.68

SAT
TT=1448.49

SAT
TT=2964.65

? (TO)
TT=2520.1

SAT
TT=3086.45

? (TO)
TT=2520.13

? (TO)
TT=2520.08

? (TO)
TT=2520.13

? (TO)
TT=2520.17

? (TO)
TT=2520.03

? (TO)
TT=2520.04

? (TO)
TT=2520.05

? (TO)
TT=20090.4

? (TO)
TT=20085.1

? (TO)
TT=2520.12

? (TO)
TT=15186.6

? (TO)
TT=15243.7

? (NS)
TT=1509.56

? (NS)
TT=17248.9

? (NS)
TT=14186.2

SAT
TT=383.355

NumberPartitioning/
NumberPartitioning-m1-s1/
NumberPartitioning-184.xml
SAT
TT=114.434

SAT
TT=920.903

? (TO)
TT=2520.04

SAT
TT=114.434

SAT
TT=291.57

SAT
TT=248.872

SAT
TT=494.79

SAT
TT=245.691

SAT
TT=511.56

? (TO)
TT=2520.06

SAT
TT=514.442

? (TO)
TT=2520.07

? (TO)
TT=2520.09

? (TO)
TT=2520.1

? (TO)
TT=2520.12

? (TO)
TT=2520.02

? (TO)
TT=2519.81

? (TO)
TT=2519.83

? (TO)
TT=20088.5

? (TO)
TT=20092.1

? (TO)
TT=2520.24

? (TO)
TT=8785.67

? (TO)
TT=8991.19

? (NS)
TT=1552.39

? (NS)
TT=3175.95

? (NS)
TT=3239.91

SAT
TT=1339.77

PigeonsPlus/
PigeonsPlus-m1-s1/
pigeonsPlus-08-08.xml
UNSAT
TT=29.432

UNSAT
TT=49.738

UNSAT
TT=29.432

?
TT=290.637

?
TT=1297.48

?
TT=320.353

?
TT=1174.92

?
TT=325.35

?
TT=1170.54

UNSAT
TT=52.471

?
TT=1237.92

UNSAT
TT=41.142

UNSAT
TT=37.84

UNSAT
TT=39.823

UNSAT
TT=36.762

? (TO)
TT=2519.96

? (TO)
TT=2520.06

? (TO)
TT=2519.95

UNSAT
TT=14162.4

UNSAT
TT=13897.3

UNSAT
TT=88.371

UNSAT
TT=95.855

UNSAT
TT=95.79

UNSAT
TT=90.393

UNSAT
TT=96.667

UNSAT
TT=95.636

? (TO)
TT=2519.99

PigeonsPlus/
PigeonsPlus-m1-s1/
pigeonsPlus-10-05.xml
UNSAT
TT=9.061

UNSAT
TT=16.585

UNSAT
TT=20.99

UNSAT
TT=55.55

UNSAT
TT=118.445

UNSAT
TT=52.297

UNSAT
TT=114.152

UNSAT
TT=56.217

UNSAT
TT=126.696

UNSAT
TT=26.903

UNSAT
TT=118.53

UNSAT
TT=45.566

UNSAT
TT=43.171

UNSAT
TT=122.286

UNSAT
TT=43.53

UNSAT
TT=1761.26

UNSAT
TT=1758.18

UNSAT
TT=1762.22

UNSAT
TT=10442.8

UNSAT
TT=8899.35

UNSAT
TT=9.061

UNSAT
TT=10.67

UNSAT
TT=10.836

UNSAT
TT=9.834

UNSAT
TT=10.499

UNSAT
TT=10.537

UNSAT
TT=18.49

Primes/Primes-m1-p10/
Primes-10-40-2-7.xml
SAT
TT=0.265

SAT
TT=5.599

SAT
TT=0.265

SAT
TT=8.199

SAT
TT=20.174

SAT
TT=14.78

SAT
TT=33.369

SAT
TT=12.799

SAT
TT=28.037

SAT
TT=6.076

SAT
TT=23.629

SAT
TT=9.009

SAT
TT=10.054

SAT
TT=12.402

SAT
TT=21.349

SAT
TT=0.56

SAT
TT=0.484

SAT
TT=0.484

SAT
TT=17.238

SAT
TT=1.57

SAT
TT=154.778

SAT
TT=2469.71

SAT
TT=2506.39

SAT
TT=440.384

SAT
TT=90.2

SAT
TT=89.174

SAT
TT=625.825

Bench nameBest resultsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
Primes/Primes-m1-p10/
Primes-10-40-3-5.xml
SAT
TT=33.443

SAT
TT=83.828

SAT
TT=245.042

SAT
TT=716.862

SAT
TT=1527.78

SAT
TT=264.524

SAT
TT=532.663

SAT
TT=242.825

SAT
TT=540.463

? (TO)
TT=2520.02

SAT
TT=987.785

SAT
TT=40.877

SAT
TT=376.357

SAT
TT=583.729

SAT
TT=33.443

? (TO)
TT=2520.04

? (TO)
TT=2519.82

? (TO)
TT=2520.05

SAT
TT=1925.76

SAT
TT=1326.99

? (TO)
TT=2520.1

? (TO)
TT=19737.6

? (TO)
TT=19705.8

SAT
TT=1969.11

SAT
TT=2979.15

SAT
TT=3039.61

SAT
TT=2278.57

Primes/Primes-m1-p10/
Primes-10-60-3-5.xml
SAT
TT=0.304

SAT
TT=4.102

SAT
TT=0.304

SAT
TT=4.607

SAT
TT=6.001

SAT
TT=3.055

SAT
TT=4.308

SAT
TT=3.171

SAT
TT=3.973

SAT
TT=1.508

SAT
TT=4.13

SAT
TT=8.267

SAT
TT=11.124

SAT
TT=13.998

SAT
TT=10.782

SAT
TT=0.335

SAT
TT=0.331

SAT
TT=0.332

SAT
TT=0.955

SAT
TT=0.557

SAT
TT=34.446

SAT
TT=113.446

SAT
TT=114.919

SAT
TT=52.548

SAT
TT=202.764

SAT
TT=199.334

SAT
TT=38.849

Primes/Primes-m1-p10/
Primes-10-60-3-7.xml
SAT
TT=71.738

SAT
TT=71.738

SAT
TT=2169.22

SAT
TT=496.108

SAT
TT=993.578

SAT
TT=1331.87

SAT
TT=2719.11

SAT
TT=1389.13

SAT
TT=2505.68

SAT
TT=521.06

SAT
TT=3385.98

SAT
TT=74.623

SAT
TT=599.218

? (TO)
TT=2520.13

SAT
TT=1580.27

? (TO)
TT=2520.04

? (TO)
TT=2520.07

? (TO)
TT=2519.89

SAT
TT=5044.01

SAT
TT=16615.5

SAT
TT=378.24

SAT
TT=689.031

SAT
TT=691.307

SAT
TT=457.297

SAT
TT=664.831

SAT
TT=707.931

SAT
TT=159.185

Primes/Primes-m1-p15/
Primes-15-20-2-7.xml
SAT
TT=0.008

SAT
TT=3.25

SAT
TT=0.009

SAT
TT=1.25

SAT
TT=1.486

SAT
TT=1.318

SAT
TT=1.388

SAT
TT=1.196

SAT
TT=1.475

SAT
TT=0.987

SAT
TT=1.478

SAT
TT=5.44

SAT
TT=5.606

SAT
TT=5.646

SAT
TT=4.765

SAT
TT=0.008

SAT
TT=0.009

SAT
TT=0.008

SAT
TT=0.017

SAT
TT=0.017

SAT
TT=19.167

SAT
TT=72.579

SAT
TT=73.39

SAT
TT=211.402

SAT
TT=359.751

SAT
TT=330.759

SAT
TT=6.064

Primes/Primes-m1-p15/
Primes-15-40-3-5.xml
SAT
TT=208.669

SAT
TT=406.692

?
TT=2520.01

? (TO)
TT=2520.39

? (TO)
TT=17580.8

SAT
TT=2036.83

SAT
TT=4173.77

SAT
TT=2024.3

SAT
TT=4235.66

? (TO)
TT=2520.11

SAT
TT=1322.8

SAT
TT=208.669

SAT
TT=355.635

? (TO)
TT=2520.16

? (TO)
TT=2520.08

? (TO)
TT=2519.93

? (TO)
TT=2519.57

? (TO)
TT=2520.05

? (TO)
TT=20001.7

? (TO)
TT=19996.9

? (TO)
TT=2520.12

? (TO)
TT=19187.5

? (TO)
TT=19116

? (TO)
TT=2520.44

? (TO)
TT=2609.8

? (TO)
TT=2605.67

SAT
TT=2444.64

Primes/Primes-m1-p15/
Primes-15-80-3-7.xml
SAT
TT=0.067

SAT
TT=3.516

SAT
TT=0.067

SAT
TT=1.888

SAT
TT=2.999

SAT
TT=2.025

SAT
TT=3.022

SAT
TT=1.897

SAT
TT=2.71

SAT
TT=1.249

SAT
TT=2.737

SAT
TT=6.518

SAT
TT=6.687

SAT
TT=7.405

SAT
TT=6.232

SAT
TT=0.546

SAT
TT=0.471

SAT
TT=0.473

SAT
TT=0.179

SAT
TT=0.197

SAT
TT=34.546

SAT
TT=152.645

SAT
TT=153.579

? (MO)
TT=678.766

? (TO)
TT=1191.75

SAT
TT=1230.8

SAT
TT=22.722

Primes/Primes-m1-p20/
Primes-20-20-3-3.xml
SAT
TT=0.008

SAT
TT=3.225

SAT
TT=0.01

SAT
TT=1.341

SAT
TT=1.662

SAT
TT=1.345

SAT
TT=1.645

SAT
TT=1.37

SAT
TT=1.738

SAT
TT=1.05

SAT
TT=1.661

SAT
TT=5.266

SAT
TT=5.319

SAT
TT=5.547

SAT
TT=4.58

SAT
TT=0.007

SAT
TT=0.008

SAT
TT=0.008

SAT
TT=0.018

SAT
TT=0.016

SAT
TT=10.734

SAT
TT=11.777

SAT
TT=11.671

SAT
TT=103.783

SAT
TT=153.459

SAT
TT=159.97

SAT
TT=5.47

Primes/Primes-m1-p20/
Primes-20-40-2-7.xml
SAT
TT=1.738

SAT
TT=11.409

SAT
TT=1.738

SAT
TT=47.009

SAT
TT=78.975

SAT
TT=80.054

SAT
TT=164.743

SAT
TT=86.635

SAT
TT=22.913

SAT
TT=15.103

SAT
TT=167.827

SAT
TT=46.498

SAT
TT=24.981

SAT
TT=215.299

SAT
TT=10.252

? (TO)
TT=2519.96

? (TO)
TT=2520

? (TO)
TT=2519.76

SAT
TT=243.854

SAT
TT=1127.48

? (TO)
TT=2520.12

? (TO)
TT=18701.6

? (TO)
TT=19084.9

? (TO)
TT=2528.15

?
TT=4852.11

?
TT=4800.11

SAT
TT=1522

Primes/Primes-m1-p20/
Primes-20-60-3-5.xml
SAT
TT=27.468

SAT
TT=83.422

SAT
TT=502.306

SAT
TT=134.615

SAT
TT=265.779

SAT
TT=125.397

SAT
TT=249.952

SAT
TT=114.541

SAT
TT=212.107

SAT
TT=27.468

SAT
TT=74.533

SAT
TT=199.276

SAT
TT=122.74

SAT
TT=669.44

SAT
TT=254.499

? (TO)
TT=2519.69

? (TO)
TT=2519.92

? (TO)
TT=2520.09

SAT
TT=3714.57

SAT
TT=2258.55

? (TO)
TT=2520.12

SAT
TT=16748.2

SAT
TT=16737.5

? (MO)
TT=1927.58

? (MO)
TT=4056.28

? (MO)
TT=3804.87

? (TO)
TT=2519.95

Primes/Primes-m1-p25/
Primes-25-20-3-1.xml
SAT
TT=0.011

SAT
TT=3.247

SAT
TT=0.011

SAT
TT=1.796

SAT
TT=2.662

SAT
TT=1.8

SAT
TT=2.658

SAT
TT=1.764

SAT
TT=2.588

SAT
TT=1.132

SAT
TT=2.635

SAT
TT=5.392

SAT
TT=5.55

SAT
TT=5.843

SAT
TT=4.872

SAT
TT=0.016

SAT
TT=0.016

SAT
TT=0.016

SAT
TT=0.02

SAT
TT=0.024

SAT
TT=10.103

SAT
TT=7.754

SAT
TT=7.818

SAT
TT=7.874

SAT
TT=7.717

SAT
TT=7.749

SAT
TT=4.252

Bench nameBest resultsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
Primes/Primes-m1-p25/
Primes-25-40-2-7.xml
SAT
TT=4.619

SAT
TT=5.081

SAT
TT=4.619

SAT
TT=6.238

SAT
TT=12.659

SAT
TT=6.075

SAT
TT=16.315

SAT
TT=5.685

SAT
TT=15.927

SAT
TT=4.673

SAT
TT=14.421

SAT
TT=22.663

SAT
TT=13.97

SAT
TT=10.537

SAT
TT=15.822

SAT
TT=269.197

SAT
TT=262.489

SAT
TT=262.831

SAT
TT=16.732

SAT
TT=19.795

? (TO)
TT=2520.2

? (TO)
TT=18663.8

? (TO)
TT=18506.6

? (NS)
TT=2352.21

? (TO)
TT=17804.3

? (TO)
TT=17821.7

? (TO)
TT=2519.7

Primes/Primes-m1-p25/
Primes-25-80-2-7.xml
SAT
TT=0.012

SAT
TT=3.366

SAT
TT=0.012

SAT
TT=2.42

SAT
TT=4.563

SAT
TT=2.752

SAT
TT=4.247

SAT
TT=2.485

SAT
TT=4.893

SAT
TT=1.387

SAT
TT=4.508

SAT
TT=6.062

SAT
TT=6.269

SAT
TT=6.356

SAT
TT=5.505

SAT
TT=0.015

SAT
TT=0.015

SAT
TT=0.015

SAT
TT=0.1

SAT
TT=0.091

SAT
TT=18.122

SAT
TT=42.84

SAT
TT=42.354

? (NS)
TT=2154.84

? (NS)
TT=11049.4

? (NS)
TT=11141.2

SAT
TT=9.131

Primes/Primes-m1-p30/
Primes-30-80-3-7.xml
SAT
TT=0.351

SAT
TT=4.192

SAT
TT=0.351

SAT
TT=3.116

SAT
TT=6.655

SAT
TT=2.32

SAT
TT=6.423

SAT
TT=2.369

SAT
TT=4.904

SAT
TT=1.309

SAT
TT=4.885

SAT
TT=8.319

SAT
TT=9.557

SAT
TT=10.706

SAT
TT=7.331

SAT
TT=1.249

SAT
TT=1.242

SAT
TT=1.239

SAT
TT=0.405

SAT
TT=0.405

SAT
TT=270.283

SAT
TT=816.824

SAT
TT=816.222

? (TO)
TT=2528.38

? (TO)
TT=18606.8

? (TO)
TT=18596.1

SAT
TT=115.895

PropStress/
PropStress-m1-s1/
PropStress-0150.xml
UNSAT
TT=19.599

? (TO)
TT=2520.06

UNSAT
TT=121.921

UNSAT
TT=398.757

UNSAT
TT=706.12

UNSAT
TT=341.169

UNSAT
TT=727.763

UNSAT
TT=320.278

UNSAT
TT=1141.15

UNSAT
TT=44.18

UNSAT
TT=1063.77

UNSAT
TT=145.645

UNSAT
TT=211.634

UNSAT
TT=242.287

UNSAT
TT=213.258

UNSAT
TT=759.706

UNSAT
TT=660.148

UNSAT
TT=660.93

UNSAT
TT=5786.04

UNSAT
TT=5714.61

UNSAT
TT=19.599

UNSAT
TT=27.74

UNSAT
TT=26.726

? (MO)
TT=544.709

UNSAT
TT=671.899

UNSAT
TT=699.706

UNSAT
TT=45.868

PropStress/
PropStress-m1-s1/
PropStress-0400.xml
UNSAT
TT=59.25

? (problem)
TT=46.844

? (MO)
TT=222.751

? (TO)
TT=2520.57

? (TO)
TT=19874.1

? (TO)
TT=2520.47

? (TO)
TT=19926.2

? (TO)
TT=2520.59

? (TO)
TT=19923.9

UNSAT
TT=1916.84

? (TO)
TT=19930.1

? (TO)
TT=2522.12

? (TO)
TT=2522.07

? (TO)
TT=2522.05

? (TO)
TT=2522.56

? (TO)
TT=2520.1

? (TO)
TT=2520.09

? (TO)
TT=2520.04

? (TO)
TT=19798.8

? (TO)
TT=19788.1

UNSAT
TT=59.25

UNSAT
TT=92.129

UNSAT
TT=92.889

? (TO)
TT=2528.2

?
TT=1673.21

?
TT=1499.73

UNSAT
TT=451.594

PropStress/
PropStress-m1-s1/
PropStress-0450.xml
UNSAT
TT=73.796

? (problem)
TT=51.389

? (MO)
TT=350.212

? (TO)
TT=2520.72

? (TO)
TT=19822.7

? (TO)
TT=2520.61

? (TO)
TT=19876.8

? (TO)
TT=2520.56

? (TO)
TT=19900.4

? (TO)
TT=2520.09

? (TO)
TT=19878.7

? (TO)
TT=2522.1

? (TO)
TT=2522.09

? (TO)
TT=2522.1

? (TO)
TT=2522.76

? (TO)
TT=2519.8

? (TO)
TT=2519.6

? (TO)
TT=2519.91

? (TO)
TT=19701.7

? (TO)
TT=19697.1

UNSAT
TT=73.796

UNSAT
TT=97.445

UNSAT
TT=104.326

? (TO)
TT=2528.13

? (TO)
TT=16098.1

? (TO)
TT=16213.6

UNSAT
TT=504.461

PseudoBoolean/
PseudoBoolean-dec-armies/
Pb-army11-17ls.xml
SAT
TT=18.403

? (TO)
TT=2520.07

? (TO)
TT=2519.92

SAT
TT=561.508

SAT
TT=1319.89

SAT
TT=396.328

SAT
TT=876.561

SAT
TT=393.141

SAT
TT=812.565

? (TO)
TT=2520.12

SAT
TT=892.335

? (TO)
TT=2520.08

? (TO)
TT=2520.06

? (TO)
TT=2520.07

? (TO)
TT=2520.06

SAT
TT=2319.35

SAT
TT=2269.52

SAT
TT=2358.17

SAT
TT=65.391

SAT
TT=59.66

SAT
TT=136.055

SAT
TT=71.445

SAT
TT=77.41

SAT
TT=31.318

SAT
TT=365.712

SAT
TT=366.602

SAT
TT=18.403

PseudoBoolean/
PseudoBoolean-dec-armies/
Pb-army12-21ls.xml
SAT
TT=470.531

? (TO)
TT=2520.04

? (TO)
TT=2519.83

SAT
TT=2067.35

SAT
TT=4591.52

? (TO)
TT=2520.52

? (TO)
TT=20051.5

? (TO)
TT=2520.62

? (TO)
TT=20059.8

? (TO)
TT=2520.13

? (TO)
TT=20059.7

? (TO)
TT=2520.11

? (TO)
TT=2520.13

? (TO)
TT=2520.11

? (TO)
TT=2520.06

? (TO)
TT=2520.02

? (TO)
TT=2519.51

? (TO)
TT=2520.08

? (TO)
TT=20113.6

? (TO)
TT=20112

? (TO)
TT=2520.05

? (TO)
TT=18270.2

? (TO)
TT=18506.9

? (TO)
TT=2520.07

? (TO)
TT=18979.5

? (TO)
TT=18570

SAT
TT=470.531

PseudoBoolean/
PseudoBoolean-dec-armies/
Pb-army13-24ls.xml
SAT
TT=34.667

? (TO)
TT=2520.11

? (TO)
TT=2519.93

SAT
TT=582.049

SAT
TT=1296.61

SAT
TT=923.816

SAT
TT=2054.45

SAT
TT=878.217

SAT
TT=1899.74

? (TO)
TT=2520.11

SAT
TT=1872.01

? (TO)
TT=2520.08

? (TO)
TT=2520.1

? (TO)
TT=2520.12

? (TO)
TT=2520.1

? (TO)
TT=2519.96

? (TO)
TT=2520.06

? (TO)
TT=2519.93

SAT
TT=345.394

SAT
TT=356.177

SAT
TT=1030

SAT
TT=7169.61

SAT
TT=7162.04

? (TO)
TT=2520.06

SAT
TT=3882.86

SAT
TT=3926.08

SAT
TT=34.667

PseudoBoolean/
PseudoBoolean-dec-fpga/
Pb-chnl10-11.xml
UNSAT
TT=56.547

UNSAT
TT=348.256

? (TO)
TT=2519.92

UNSAT
TT=963.684

UNSAT
TT=2065.63

UNSAT
TT=1032.45

UNSAT
TT=2374.59

UNSAT
TT=1030.35

UNSAT
TT=2362.87

? (TO)
TT=2520.04

UNSAT
TT=2450.33

UNSAT
TT=1539.87

UNSAT
TT=1849.28

? (TO)
TT=2520.08

UNSAT
TT=2182.4

UNSAT
TT=2431.26

UNSAT
TT=2121.5

UNSAT
TT=2113.71

UNSAT
TT=7007.34

UNSAT
TT=7047.5

UNSAT
TT=95.613

UNSAT
TT=388.471

UNSAT
TT=383.095

UNSAT
TT=120.546

UNSAT
TT=227.792

UNSAT
TT=228.737

UNSAT
TT=56.547

Bench nameBest resultsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
PseudoBoolean/
PseudoBoolean-dec-fpga/
Pb-chnl10-15.xml
UNSAT
TT=62.423

UNSAT
TT=691.693

?
TT=2520.01

UNSAT
TT=1273.7

UNSAT
TT=2586.22

UNSAT
TT=1296.99

UNSAT
TT=3217.87

UNSAT
TT=1322.4

UNSAT
TT=2815.78

? (TO)
TT=2520.12

UNSAT
TT=3195.02

UNSAT
TT=2299.34

? (TO)
TT=2520.13

? (TO)
TT=2520.11

? (TO)
TT=2520.05

? (TO)
TT=2520.11

? (TO)
TT=2520.06

? (TO)
TT=2520.05

UNSAT
TT=13869

UNSAT
TT=14536.4

UNSAT
TT=85.031

UNSAT
TT=134.477

UNSAT
TT=134.314

UNSAT
TT=62.423

UNSAT
TT=254.943

UNSAT
TT=252.742

UNSAT
TT=97.795

PseudoBoolean/
PseudoBoolean-dec-fpga/
Pb-fpga45-44-sat.xml
SAT
TT=6.711

SAT
TT=6.711

? (TO)
TT=2520.02

? (TO)
TT=2520.36

? (TO)
TT=20075.6

? (TO)
TT=2520.31

? (TO)
TT=20038.2

? (TO)
TT=2520.61

? (TO)
TT=20028.8

? (TO)
TT=2520.11

? (TO)
TT=20042.7

? (TO)
TT=2520.09

? (TO)
TT=2520.13

? (TO)
TT=2520.16

? (TO)
TT=2520.11

? (TO)
TT=2520.02

? (TO)
TT=2519.96

? (TO)
TT=2519.19

SAT
TT=475.826

SAT
TT=516.807

SAT
TT=219.013

SAT
TT=14.352

SAT
TT=14.424

SAT
TT=23.702

SAT
TT=26.319

SAT
TT=26.889

SAT
TT=135.858

PseudoBoolean/
PseudoBoolean-dec-pigeon/
Pb-pigeon-card-015-010.xml
UNSAT
TT=43.037

UNSAT
TT=637.056

?
TT=2520.01

UNSAT
TT=1269.39

UNSAT
TT=2637.84

UNSAT
TT=1306.58

UNSAT
TT=3226.55

UNSAT
TT=1189.16

UNSAT
TT=2584.36

? (TO)
TT=2520.11

UNSAT
TT=2702.55

UNSAT
TT=1667.37

UNSAT
TT=1266.86

? (TO)
TT=2520.05

UNSAT
TT=1484.53

? (TO)
TT=2519.83

? (TO)
TT=2520.07

? (TO)
TT=2519.71

UNSAT
TT=12260.6

UNSAT
TT=11581

UNSAT
TT=43.037

UNSAT
TT=149.788

UNSAT
TT=152.756

UNSAT
TT=54.018

UNSAT
TT=217.936

UNSAT
TT=219.955

UNSAT
TT=107.132

PseudoBoolean/
PseudoBoolean-dec-pigeon/
Pb-pigeon-clau-011-010.xml
UNSAT
TT=51.188

UNSAT
TT=490.602

UNSAT
TT=1400.43

UNSAT
TT=1795.6

UNSAT
TT=2653.67

UNSAT
TT=1727.9

UNSAT
TT=2304.45

UNSAT
TT=1687.31

UNSAT
TT=2556.34

? (TO)
TT=2520.06

UNSAT
TT=2375.27

UNSAT
TT=1018.67

UNSAT
TT=1784.53

? (TO)
TT=2520.11

UNSAT
TT=1472.24

UNSAT
TT=2487.41

? (TO)
TT=2519.73

UNSAT
TT=2460.58

UNSAT
TT=5895.13

UNSAT
TT=5981.47

UNSAT
TT=51.188

UNSAT
TT=395.115

UNSAT
TT=394.571

UNSAT
TT=52.38

UNSAT
TT=389.552

UNSAT
TT=392.523

UNSAT
TT=64.115

PseudoBoolean/
PseudoBoolean-dec-pigeon/
Pb-pigeon-clau-015-010.xml
UNSAT
TT=92.811

UNSAT
TT=621.108

?
TT=2520.02

? (TO)
TT=2520.47

UNSAT
TT=8501.54

? (TO)
TT=2520.41

UNSAT
TT=7618.76

? (TO)
TT=2520.63

UNSAT
TT=7332.93

? (TO)
TT=2520.09

UNSAT
TT=7586.58

UNSAT
TT=1989.68

? (TO)
TT=2520.06

? (TO)
TT=2520.07

? (TO)
TT=2520.06

? (TO)
TT=2519.75

? (TO)
TT=2519.72

? (TO)
TT=2519.81

UNSAT
TT=10804.1

UNSAT
TT=10904.8

UNSAT
TT=92.811

UNSAT
TT=672.753

UNSAT
TT=677.814

UNSAT
TT=261.727

UNSAT
TT=665.802

UNSAT
TT=670.743

UNSAT
TT=129.17

PseudoBoolean/
PseudoBoolean-dec-queen/
Pb-queen-0975965.xml
UNSAT
TT=7.267

UNSAT
TT=24.808

UNSAT
TT=26.31

UNSAT
TT=8.88

UNSAT
TT=16.547

UNSAT
TT=8.259

UNSAT
TT=15.587

UNSAT
TT=7.922

UNSAT
TT=15.286

UNSAT
TT=7.267

UNSAT
TT=14.679

UNSAT
TT=252.393

UNSAT
TT=324.826

UNSAT
TT=1556.2

UNSAT
TT=327.861

UNSAT
TT=191.596

UNSAT
TT=190.947

UNSAT
TT=192.401

UNSAT
TT=457.951

UNSAT
TT=496.408

UNSAT
TT=27.378

UNSAT
TT=60.909

UNSAT
TT=60.9

UNSAT
TT=17.783

UNSAT
TT=37.353

UNSAT
TT=36.866

UNSAT
TT=20.663

PseudoBoolean/
PseudoBoolean-dec-spba/
Pb-BeauxArts-K67.xml
SAT
TT=91.385

? (TO)
TT=2520.1

? (TO)
TT=2519.95

? (TO)
TT=2520.42

? (TO)
TT=20042.5

? (TO)
TT=2520.58

? (TO)
TT=20041.8

? (TO)
TT=2520.38

? (TO)
TT=20025.6

? (TO)
TT=2520.12

? (TO)
TT=20028.7

? (TO)
TT=2520.13

? (TO)
TT=2520.1

? (TO)
TT=2520.11

? (TO)
TT=2520.16

? (TO)
TT=2520.03

? (TO)
TT=2519.92

? (TO)
TT=2519.77

? (TO)
TT=20109.8

? (TO)
TT=20109

SAT
TT=91.385

SAT
TT=1182.18

SAT
TT=1189.42

SAT
TT=159.884

SAT
TT=382.778

SAT
TT=382.102

SAT
TT=166.767

PseudoBoolean/
PseudoBoolean-dec-spng/
Pb-NG-K146.xml



? (TO)
TT=2520.05

? (TO)
TT=2519.66

? (TO)
TT=2520.7

? (TO)
TT=20074.7

? (TO)
TT=2520.39

? (TO)
TT=20078.5

? (TO)
TT=2520.38

? (TO)
TT=20075

? (TO)
TT=2520.08

? (TO)
TT=20079.6

? (TO)
TT=2520.11

? (TO)
TT=2520.04

? (TO)
TT=2520.13

? (TO)
TT=2520.16

? (TO)
TT=2520.08

? (TO)
TT=2520.05

? (TO)
TT=2519.84

? (TO)
TT=20070.7

? (TO)
TT=20075.6

? (TO)
TT=2520.14

? (TO)
TT=18711.3

? (TO)
TT=18626.8

? (TO)
TT=2520.14

? (TO)
TT=18748

? (TO)
TT=18805.5

? (TO)
TT=2519.81

PseudoBoolean/
PseudoBoolean-dec-sptate/
Pb-Tate-K097.xml
SAT
TT=64.44

? (TO)
TT=2520.04

? (TO)
TT=2519.62

? (TO)
TT=2520.69

? (TO)
TT=20083.2

? (TO)
TT=2520.66

? (TO)
TT=20088.5

? (TO)
TT=2520.66

? (TO)
TT=20093

? (TO)
TT=2520.09

? (TO)
TT=20090.8

? (TO)
TT=2520.06

? (TO)
TT=2520.12

? (TO)
TT=2520.13

? (TO)
TT=2520.06

? (TO)
TT=2520.07

? (TO)
TT=2519.53

? (TO)
TT=2519.97

? (TO)
TT=20105.7

? (TO)
TT=20105.2

SAT
TT=64.44

SAT
TT=10186.4

SAT
TT=10177.6

SAT
TT=355.343

SAT
TT=2815.27

SAT
TT=2861.4

SAT
TT=121.404

PseudoBoolean/
PseudoBoolean-dec-sptate/
Pb-Tate-K100.xml
SAT
TT=49.688

? (TO)
TT=2520.09

? (TO)
TT=2520.01

? (TO)
TT=2520.44

? (TO)
TT=20085.6

? (TO)
TT=2520.65

? (TO)
TT=20086.8

? (TO)
TT=2520.38

? (TO)
TT=20092.4

? (TO)
TT=2520.1

? (TO)
TT=20091.2

? (TO)
TT=2520.07

? (TO)
TT=2520.12

? (TO)
TT=2520.18

? (TO)
TT=2520.13

? (TO)
TT=2520.07

? (TO)
TT=2519.88

? (TO)
TT=2520.08

? (TO)
TT=20105.9

? (TO)
TT=20102

SAT
TT=49.688

SAT
TT=4773.06

SAT
TT=5001.98

SAT
TT=626.956

SAT
TT=1213.5

SAT
TT=1236.71

SAT
TT=119.957

Bench nameBest resultsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
PseudoBoolean/
PseudoBoolean-dec-sptate/
Pb-Tate-K106.xml
SAT
TT=53.714

? (TO)
TT=2520.02

? (TO)
TT=2520.06

? (TO)
TT=2520.55

? (TO)
TT=20081.6

? (TO)
TT=2520.7

? (TO)
TT=20091.6

? (TO)
TT=2520.61

? (TO)
TT=20089.4

? (TO)
TT=2520.04

? (TO)
TT=20094

? (TO)
TT=2520.16

? (TO)
TT=2520.11

? (TO)
TT=2520.06

? (TO)
TT=2520.1

? (TO)
TT=2520

? (TO)
TT=2519.9

? (TO)
TT=2519.97

? (TO)
TT=20103.7

? (TO)
TT=20098.9

SAT
TT=53.714

SAT
TT=1552.99

SAT
TT=1540.55

SAT
TT=367.185

SAT
TT=1094.22

SAT
TT=1089.89

SAT
TT=101.038

QRandom/QRandom-bdd-15-21-2/
bdd-15-21-2-2713-79-06.xml
UNSAT
TT=21.668

UNSAT
TT=32.237

UNSAT
TT=21.668

UNSAT
TT=151.83

UNSAT
TT=556.936

UNSAT
TT=157.393

UNSAT
TT=573.458

UNSAT
TT=171.338

UNSAT
TT=555.287

UNSAT
TT=30.684

UNSAT
TT=624.577

UNSAT
TT=197.271

UNSAT
TT=226.417

UNSAT
TT=669.239

UNSAT
TT=285.739

UNSAT
TT=196.113

UNSAT
TT=202.026

UNSAT
TT=196.681

UNSAT
TT=849.058

UNSAT
TT=811.99

? (TO)
TT=2520.29

? (TO)
TT=12666.9

? (TO)
TT=12742.2

? (TO)
TT=2528.07

? (TO)
TT=12632.4

? (TO)
TT=12856.5

? (TO)
TT=2519.56

QRandom/QRandom-bdd-15-21-2/
bdd-15-21-2-2713-79-15.xml
UNSAT
TT=20.956

UNSAT
TT=32.259

UNSAT
TT=20.956

UNSAT
TT=153.294

UNSAT
TT=544.73

UNSAT
TT=156.11

UNSAT
TT=710.527

UNSAT
TT=187.24

UNSAT
TT=544.591

UNSAT
TT=28.963

UNSAT
TT=548.691

UNSAT
TT=219.608

UNSAT
TT=233.913

UNSAT
TT=547.383

UNSAT
TT=255.008

UNSAT
TT=198.924

UNSAT
TT=201.265

UNSAT
TT=199.415

UNSAT
TT=777.708

UNSAT
TT=775.97

? (TO)
TT=2528.19

? (TO)
TT=12687

? (TO)
TT=12694.8

? (TO)
TT=2528.1

? (TO)
TT=12713.3

? (TO)
TT=12771.6

? (TO)
TT=2519.84

QRandom/QRandom-bdd-18-21-2/
bdd-18-21-2-133-78-06.xml
UNSAT
TT=8.613

UNSAT
TT=13.846

UNSAT
TT=8.613

UNSAT
TT=50.927

UNSAT
TT=166.495

UNSAT
TT=53.628

UNSAT
TT=163.761

UNSAT
TT=57.373

UNSAT
TT=165.607

UNSAT
TT=14.291

UNSAT
TT=167.292

UNSAT
TT=141.541

UNSAT
TT=232.826

UNSAT
TT=381.38

UNSAT
TT=212.271

UNSAT
TT=129.348

UNSAT
TT=128.003

UNSAT
TT=130.325

UNSAT
TT=1400.29

UNSAT
TT=1476.77

? (TO)
TT=2527.99

? (NS)
TT=10540.1

? (NS)
TT=10367.9

? (TO)
TT=2528.55

? (NS)
TT=9923.72

? (NS)
TT=9788.6

? (TO)
TT=2520.05

QRandom/QRandom-bdd-18-21-2/
bdd-18-21-2-133-78-21.xml
UNSAT
TT=7.583

UNSAT
TT=13.855

UNSAT
TT=7.583

UNSAT
TT=51.604

UNSAT
TT=198.496

UNSAT
TT=52.572

UNSAT
TT=165.734

UNSAT
TT=66.878

UNSAT
TT=163.771

UNSAT
TT=15.598

UNSAT
TT=167.172

UNSAT
TT=215.94

UNSAT
TT=170.12

UNSAT
TT=353.827

UNSAT
TT=231.426

UNSAT
TT=128.685

UNSAT
TT=136.743

UNSAT
TT=131.2

UNSAT
TT=1583.22

UNSAT
TT=1601.75

? (TO)
TT=2528.19

? (NS)
TT=9834.99

? (NS)
TT=9838.6

? (TO)
TT=2528.31

? (NS)
TT=10454.8

? (NS)
TT=10152.7

? (TO)
TT=2519.9

QRandom/QRandom-bdd-18-21-2/
bdd-18-21-2-133-78-29.xml
UNSAT
TT=9.389

UNSAT
TT=13.274

UNSAT
TT=9.389

UNSAT
TT=52.551

UNSAT
TT=166.04

UNSAT
TT=52.926

UNSAT
TT=191.989

UNSAT
TT=56.07

UNSAT
TT=167.524

UNSAT
TT=17.438

UNSAT
TT=161.605

UNSAT
TT=188.316

UNSAT
TT=204.575

UNSAT
TT=300.281

UNSAT
TT=214.292

UNSAT
TT=141.896

UNSAT
TT=143.404

UNSAT
TT=141.183

UNSAT
TT=1588.32

UNSAT
TT=1598.52

? (TO)
TT=2528.32

? (NS)
TT=10637.1

? (NS)
TT=10208.6

? (TO)
TT=2528.43

? (NS)
TT=9923.05

? (NS)
TT=10066

? (TO)
TT=2519.87

QRandom/QRandom-bdd-18-21-2/
bdd-18-21-2-133-78-33.xml
UNSAT
TT=9.298

UNSAT
TT=14.048

UNSAT
TT=9.298

UNSAT
TT=54.519

UNSAT
TT=171.004

UNSAT
TT=55.713

UNSAT
TT=168.167

UNSAT
TT=57.42

UNSAT
TT=168.914

UNSAT
TT=15.922

UNSAT
TT=170.482

UNSAT
TT=207.518

UNSAT
TT=184.552

UNSAT
TT=468.22

UNSAT
TT=231.575

UNSAT
TT=123.701

UNSAT
TT=122.573

UNSAT
TT=123.832

UNSAT
TT=1587.98

UNSAT
TT=1639.74

? (TO)
TT=2528.08

? (NS)
TT=10885.8

? (NS)
TT=11207.1

? (TO)
TT=2528.25

? (NS)
TT=10084.6

? (NS)
TT=10048.7

? (TO)
TT=2519.94

QRandom/QRandom-geometric-rw/
geometric-50-20-d4-75-08.xml
SAT
TT=0.402

SAT
TT=10.962

SAT
TT=2.137

SAT
TT=13.313

SAT
TT=27.323

SAT
TT=11.864

SAT
TT=24.776

SAT
TT=12.256

SAT
TT=19.464

SAT
TT=5.9

SAT
TT=16.611

SAT
TT=11.567

SAT
TT=30.726

SAT
TT=20.241

SAT
TT=20.239

SAT
TT=1.435

SAT
TT=1.438

SAT
TT=1.437

SAT
TT=0.402

SAT
TT=0.424

SAT
TT=17.582

SAT
TT=19.194

SAT
TT=17.689

SAT
TT=17.013

SAT
TT=19.646

SAT
TT=19.802

SAT
TT=10.135

QRandom/QRandom-geometric-rw/
geometric-50-20-d4-75-76.xml
SAT
TT=0.043

SAT
TT=4.271

SAT
TT=0.043

SAT
TT=3.135

SAT
TT=4.658

SAT
TT=3.184

SAT
TT=5.182

SAT
TT=3.082

SAT
TT=4.941

SAT
TT=1.867

SAT
TT=4.71

SAT
TT=8.369

SAT
TT=8.439

SAT
TT=8.903

SAT
TT=7.476

SAT
TT=0.105

SAT
TT=0.105

SAT
TT=0.104

SAT
TT=0.143

SAT
TT=0.163

SAT
TT=16.274

SAT
TT=17.92

SAT
TT=16.891

SAT
TT=17.555

SAT
TT=17.371

SAT
TT=19.406

SAT
TT=0.251

QRandom/QRandom-geometric-rw/
geometric-50-20-d4-75-94.xml
UNSAT
TT=48.648

UNSAT
TT=86.862

UNSAT
TT=741.446

UNSAT
TT=473.911

UNSAT
TT=500.581

UNSAT
TT=493.981

UNSAT
TT=543.013

UNSAT
TT=523.079

UNSAT
TT=500.466

UNSAT
TT=163.225

UNSAT
TT=517.468

UNSAT
TT=437.33

UNSAT
TT=410.554

UNSAT
TT=927.076

UNSAT
TT=355.463

UNSAT
TT=48.549

UNSAT
TT=48.971

UNSAT
TT=48.648

UNSAT
TT=276.16

UNSAT
TT=240.499

UNSAT
TT=330.937

UNSAT
TT=299.781

UNSAT
TT=299.135

UNSAT
TT=140.458

UNSAT
TT=300.092

UNSAT
TT=300.703

UNSAT
TT=217.695

Bench nameBest resultsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
QRandom/
QRandom-mdd-7-25-5/
mdd-7-25-5-56-01.xml
SAT
TT=56.575

SAT
TT=471.519

SAT
TT=101.818

SAT
TT=277.543

SAT
TT=593.593

SAT
TT=70.444

SAT
TT=169.523

SAT
TT=74.545

SAT
TT=162.496

SAT
TT=150.44

SAT
TT=260.993

SAT
TT=164.797

SAT
TT=45.095

SAT
TT=56.575

SAT
TT=201.708

SAT
TT=1918.11

SAT
TT=1922.23

SAT
TT=1920.94

SAT
TT=2201.6

SAT
TT=2029.71

SAT
TT=753.806

SAT
TT=2021

SAT
TT=2037.71

SAT
TT=408.197

SAT
TT=1986.04

SAT
TT=2088.21

SAT
TT=862.572

QRandom/
QRandom-mdd-7-25-5/
mdd-7-25-5-56-09.xml
SAT
TT=120.316

SAT
TT=267.56

SAT
TT=373.744

SAT
TT=344.274

SAT
TT=685.432

SAT
TT=324.466

SAT
TT=683.777

SAT
TT=321.723

SAT
TT=670.554

SAT
TT=120.316

SAT
TT=1429.33

SAT
TT=1325.84

SAT
TT=696.214

SAT
TT=601.336

SAT
TT=246.599

? (TO)
TT=2519.95

? (TO)
TT=2519.9

? (TO)
TT=2520.04

? (TO)
TT=19997.2

? (TO)
TT=19997.6

SAT
TT=2198.73

SAT
TT=1730.16

SAT
TT=1775.28

SAT
TT=1911.96

SAT
TT=1752.95

SAT
TT=1775.77

SAT
TT=519.65

QRandom/
QRandom-mdd-7-25-5/
mdd-7-25-5-56-17.xml
SAT
TT=22.871

SAT
TT=91.855

SAT
TT=251.983

SAT
TT=222.325

SAT
TT=512.149

SAT
TT=942.348

SAT
TT=1287.03

SAT
TT=998.711

SAT
TT=1428.46

SAT
TT=164.275

SAT
TT=149.103

SAT
TT=22.871

SAT
TT=124.34

SAT
TT=714.923

SAT
TT=434.274

? (TO)
TT=2520.09

? (TO)
TT=2519.85

? (TO)
TT=2520.03

SAT
TT=5883.47

SAT
TT=6314.29

SAT
TT=815.415

SAT
TT=2871.2

SAT
TT=2882.58

SAT
TT=292.326

SAT
TT=2880.71

SAT
TT=2914.86

SAT
TT=382.659

QRandom/QRandom-mdd-7-25-5-p7/
mdd-7-25-5-ps05-psh07-4.xml
UNSAT
TT=84.877

UNSAT
TT=85.531

UNSAT
TT=98.574

UNSAT
TT=411.196

UNSAT
TT=867.259

UNSAT
TT=450.903

UNSAT
TT=740.667

UNSAT
TT=397.936

UNSAT
TT=775.432

UNSAT
TT=84.877

UNSAT
TT=774.835

UNSAT
TT=168.232

UNSAT
TT=136.932

UNSAT
TT=253.661

UNSAT
TT=140.999

? (TO)
TT=2520.07

? (TO)
TT=2519.77

? (TO)
TT=2520.09

? (TO)
TT=19997.3

? (TO)
TT=19998.2

UNSAT
TT=163.644

UNSAT
TT=364.905

UNSAT
TT=373.848

UNSAT
TT=136.984

UNSAT
TT=368.599

UNSAT
TT=367.085

UNSAT
TT=1125.85

QRandom/QRandom-mdd-7-25-5-p7/
mdd-7-25-5-ps05-psh07-9.xml
UNSAT
TT=42.401

UNSAT
TT=43.536

UNSAT
TT=42.401

UNSAT
TT=253.65

UNSAT
TT=543.887

UNSAT
TT=240.212

UNSAT
TT=522.405

UNSAT
TT=243.029

UNSAT
TT=507.761

UNSAT
TT=54.964

UNSAT
TT=516.057

UNSAT
TT=83.506

UNSAT
TT=78.6

UNSAT
TT=153.641

UNSAT
TT=85.236

? (TO)
TT=2520.03

? (TO)
TT=2519.82

? (TO)
TT=2519.9

UNSAT
TT=15956.5

UNSAT
TT=16203

UNSAT
TT=126.211

UNSAT
TT=280.864

UNSAT
TT=270.609

UNSAT
TT=103.372

UNSAT
TT=266.205

UNSAT
TT=269.009

UNSAT
TT=1119.96

QRandom/QRandom-reg2ext-s1/
reg-s20-p03-c20-d10-n10-l5-64.xml
UNSAT
TT=3.29

UNSAT
TT=6.245

UNSAT
TT=3.29

UNSAT
TT=25.444

UNSAT
TT=42.101

UNSAT
TT=28.546

UNSAT
TT=43.388

UNSAT
TT=26.751

UNSAT
TT=41.325

UNSAT
TT=7.662

UNSAT
TT=42.658

UNSAT
TT=25.134

UNSAT
TT=22.272

UNSAT
TT=35.464

UNSAT
TT=22.208

UNSAT
TT=9.754

UNSAT
TT=9.676

UNSAT
TT=9.731

UNSAT
TT=22.803

UNSAT
TT=21.205

UNSAT
TT=32.071

UNSAT
TT=62.957

UNSAT
TT=62.58

UNSAT
TT=27.362

UNSAT
TT=62.512

UNSAT
TT=61.987

UNSAT
TT=10.168

QuasiGroups/
QuasiGroups-elt-qg3/QuasiGroup-3-12.xml
SAT
TT=10.042

SAT
TT=2506.59

SAT
TT=1603.48

SAT
TT=10.042

SAT
TT=22.611

SAT
TT=65.72

SAT
TT=129.137

SAT
TT=69.714

SAT
TT=133.797

SAT
TT=722.606

SAT
TT=2518.24

SAT
TT=1659.54

SAT
TT=416.137

SAT
TT=1877.98

SAT
TT=1221.96

? (TO)
TT=2519.93

? (TO)
TT=2519.9

? (TO)
TT=2519.82

? (TO)
TT=20117.5

SAT
TT=717.683

SAT
TT=270.441

SAT
TT=206.173

SAT
TT=211.356

SAT
TT=372.373

SAT
TT=205.477

SAT
TT=210.22

SAT
TT=346.065

QuasiGroups/
QuasiGroups-elt-qg3/QuasiGroup-3-40.xml



? (TO)
TT=2520.13

? (TO)
TT=2520.02

? (TO)
TT=2520.63

? (TO)
TT=20000.9

? (TO)
TT=2520.69

? (TO)
TT=20002.5

? (TO)
TT=2520.72

? (TO)
TT=20008.4

? (TO)
TT=2520.04

? (TO)
TT=20004.9

? (TO)
TT=2528.08

? (TO)
TT=2522.23

? (TO)
TT=2523.22

? (TO)
TT=2523.33

? (TO)
TT=2520.1

? (TO)
TT=2520.05

? (TO)
TT=2519.82

? (TO)
TT=19996.6

? (TO)
TT=19973.4

? (TO)
TT=2520.31

? (TO)
TT=6585.17

? (TO)
TT=6220.75

? (TO)
TT=2520.27

? (TO)
TT=6578.05

? (TO)
TT=6463.42

? (TO)
TT=2519.72

QuasiGroups/
QuasiGroups-elt-qg7/QuasiGroup-7-39.xml



? (TO)
TT=2520.11

? (TO)
TT=2519.8

? (TO)
TT=2520.52

? (TO)
TT=20068.1

? (TO)
TT=2520.33

? (TO)
TT=19988.6

? (TO)
TT=2520.57

? (TO)
TT=19961.9

? (TO)
TT=2520.1

? (TO)
TT=19950.6

? (TO)
TT=2520.07

? (TO)
TT=2520.1

? (TO)
TT=2520.13

? (TO)
TT=2520.15

? (TO)
TT=2520.02

? (TO)
TT=2520.02

? (TO)
TT=2520.04

? (TO)
TT=20105.2

? (TO)
TT=20107.3

? (TO)
TT=2520.4

? (TO)
TT=14248.8

? (TO)
TT=13433.1

? (TO)
TT=2520.18

? (TO)
TT=13297.1

? (TO)
TT=14282

? (TO)
TT=2519.83

QueensKnights/
QueensKnights-m1-s1/
QueensKnights-050-05-mul.xml
UNSAT
TT=13.972

UNSAT
TT=46.418

UNSAT
TT=188.233

? (TO)
TT=2520.32

? (TO)
TT=19160.7

? (TO)
TT=2520.53

? (TO)
TT=19310

? (TO)
TT=2520.51

? (TO)
TT=19272.7

? (TO)
TT=2520.08

? (TO)
TT=19268.9

? (TO)
TT=2520.11

? (TO)
TT=2520.15

? (TO)
TT=2520.05

? (TO)
TT=2520.06

UNSAT
TT=342.365

UNSAT
TT=341.974

UNSAT
TT=345.113

UNSAT
TT=4115.61

UNSAT
TT=3755.75

UNSAT
TT=13.972

UNSAT
TT=25.92

UNSAT
TT=26.754

UNSAT
TT=14.716

UNSAT
TT=26.566

UNSAT
TT=25.608

UNSAT
TT=183.645

Bench nameBest resultsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
QueensKnights/
QueensKnights-m1-s1/
QueensKnights-080-05-add.xml
UNSAT
TT=25.09

UNSAT
TT=241.197

UNSAT
TT=820.724

UNSAT
TT=2152.36

UNSAT
TT=2037.76

? (TO)
TT=2520.51

UNSAT
TT=4587.82

? (TO)
TT=2520.89

UNSAT
TT=8681.14

UNSAT
TT=1198.85

UNSAT
TT=4083.96

? (TO)
TT=2520.09

? (TO)
TT=2520.09

? (TO)
TT=2520.07

? (TO)
TT=2520.04

UNSAT
TT=2157.56

UNSAT
TT=1846.85

UNSAT
TT=1841.43

UNSAT
TT=2461.33

UNSAT
TT=2434.89

UNSAT
TT=25.09

UNSAT
TT=30.234

UNSAT
TT=29.684

UNSAT
TT=27.268

UNSAT
TT=60.626

UNSAT
TT=60.593

UNSAT
TT=585.492

Queens/
Queens-m1-s1/
Queens-0150-m1.xml
SAT
TT=12.209

SAT
TT=12.209

SAT
TT=58.223

SAT
TT=446.149

SAT
TT=951.069

SAT
TT=456.256

SAT
TT=944.252

SAT
TT=444.668

SAT
TT=1049.71

SAT
TT=94.052

SAT
TT=903.206

SAT
TT=153.693

SAT
TT=347.609

SAT
TT=527.055

SAT
TT=117.443

SAT
TT=116.518

SAT
TT=120.553

SAT
TT=117.176

SAT
TT=129.634

SAT
TT=130.819

SAT
TT=80.845

SAT
TT=99.764

SAT
TT=102.793

SAT
TT=66.834

SAT
TT=101.662

SAT
TT=102.795

SAT
TT=482.25

Queens/
Queens-m1-s1/
Queens-0200-m1.xml
SAT
TT=18.614

SAT
TT=18.614

SAT
TT=273.665

SAT
TT=1321.57

SAT
TT=2649.09

SAT
TT=1131.91

SAT
TT=2366.24

SAT
TT=1148.44

SAT
TT=2345.07

SAT
TT=322.567

SAT
TT=2138.54

SAT
TT=320.714

SAT
TT=854.365

SAT
TT=928.281

SAT
TT=1025.14

SAT
TT=379.325

SAT
TT=366.097

SAT
TT=378.846

SAT
TT=395.589

SAT
TT=397.934

SAT
TT=179.639

SAT
TT=184.21

SAT
TT=185.402

SAT
TT=126.646

SAT
TT=182.533

SAT
TT=180.516

SAT
TT=757.211

Random/Random-B-2-26-26/
rand-2-26-26-325-155-9.xml
SAT
TT=13.497

SAT
TT=1243.02

SAT
TT=714.847

SAT
TT=2476.87

SAT
TT=1941.6

SAT
TT=1518.2

SAT
TT=1968.6

SAT
TT=1469.14

SAT
TT=1984.05

? (TO)
TT=2520.13

SAT
TT=526.937

SAT
TT=79.187

SAT
TT=307.804

? (TO)
TT=2520.12

? (TO)
TT=2520.12

SAT
TT=15.48

SAT
TT=13.497

SAT
TT=13.501

SAT
TT=79.29

SAT
TT=85.224

SAT
TT=275.89

? (TO)
TT=19385

? (TO)
TT=19317.9

SAT
TT=113.705

? (TO)
TT=19429.2

? (TO)
TT=19426.5

? (TO)
TT=2519.86

Random/Random-D-t020/
rand-2-40-11-414-020-79.xml
UNSAT
TT=31.319

UNSAT
TT=45.024

UNSAT
TT=176.533

UNSAT
TT=145.475

UNSAT
TT=176.473

UNSAT
TT=148.828

UNSAT
TT=180.21

UNSAT
TT=145.765

UNSAT
TT=192.673

UNSAT
TT=61.186

UNSAT
TT=182.34

UNSAT
TT=223.742

UNSAT
TT=259.726

UNSAT
TT=485.724

UNSAT
TT=217.671

UNSAT
TT=35.77

UNSAT
TT=31.493

UNSAT
TT=31.319

UNSAT
TT=139.549

UNSAT
TT=127.141

UNSAT
TT=81.116

UNSAT
TT=92.919

UNSAT
TT=93.322

UNSAT
TT=40.402

UNSAT
TT=93.276

UNSAT
TT=93.859

UNSAT
TT=36.388

Random/Random-D-t035/
rand-2-40-16-250-035-86.xml
SAT
TT=1.66

SAT
TT=13.712

SAT
TT=137.986

SAT
TT=30.658

SAT
TT=57.844

SAT
TT=28.416

SAT
TT=51.056

SAT
TT=27.868

SAT
TT=52.493

SAT
TT=25.97

SAT
TT=76.173

SAT
TT=16.14

SAT
TT=12.439

SAT
TT=17.787

SAT
TT=65.343

SAT
TT=1.894

SAT
TT=1.66

SAT
TT=1.674

SAT
TT=6.716

SAT
TT=29.712

SAT
TT=48.917

SAT
TT=39.732

SAT
TT=38.838

SAT
TT=13.509

SAT
TT=39.13

SAT
TT=39.759

SAT
TT=29.31

Random/Random-D-t050/
rand-2-40-25-180-050-34.xml
SAT
TT=1.271

SAT
TT=6.814

SAT
TT=27.042

SAT
TT=7.64

SAT
TT=12.49

SAT
TT=6.844

SAT
TT=11.823

SAT
TT=8.496

SAT
TT=12.463

SAT
TT=7.703

SAT
TT=11.853

SAT
TT=24.521

SAT
TT=14.442

SAT
TT=25.954

SAT
TT=15.59

SAT
TT=1.467

SAT
TT=1.275

SAT
TT=1.271

SAT
TT=10.247

SAT
TT=7.729

SAT
TT=104.891

SAT
TT=50.675

SAT
TT=50.464

SAT
TT=25.961

SAT
TT=50.392

SAT
TT=51.277

SAT
TT=17.672

Random/Random-D-t090/
rand-2-40-180-84-090-90.xml
UNSAT
TT=0.915

UNSAT
TT=5.627

UNSAT
TT=1.675

UNSAT
TT=16.026

UNSAT
TT=40.186

UNSAT
TT=16.06

UNSAT
TT=30.387

UNSAT
TT=16.807

UNSAT
TT=28.645

UNSAT
TT=5.936

UNSAT
TT=27.463

UNSAT
TT=14.273

UNSAT
TT=13.618

UNSAT
TT=17.4

UNSAT
TT=12.29

UNSAT
TT=0.93

UNSAT
TT=0.915

UNSAT
TT=0.915

UNSAT
TT=1.459

UNSAT
TT=1.398

UNSAT
TT=114.515

UNSAT
TT=159.21

UNSAT
TT=168.791

UNSAT
TT=90.321

UNSAT
TT=156.432

UNSAT
TT=158.166

UNSAT
TT=89.981

Random/Random-m1-5-12-12/
rand-5-12-12-200-12442-22.xml
UNSAT
TT=3.794

UNSAT
TT=12.109

UNSAT
TT=3.794

UNSAT
TT=42.43

UNSAT
TT=110.663

UNSAT
TT=41.738

UNSAT
TT=108.18

UNSAT
TT=50.835

UNSAT
TT=111.586

UNSAT
TT=10.929

UNSAT
TT=111.816

UNSAT
TT=61.077

UNSAT
TT=56.517

UNSAT
TT=92.537

UNSAT
TT=55.126

UNSAT
TT=55.357

UNSAT
TT=55.541

UNSAT
TT=55.314

UNSAT
TT=131.44

UNSAT
TT=130.366

UNSAT
TT=2018.28

UNSAT
TT=1352.41

UNSAT
TT=1344.77

UNSAT
TT=1982.59

UNSAT
TT=1467.39

UNSAT
TT=1464.76

UNSAT
TT=1158.61

Random/Random-m1-5-12-12t/
rand-5-12-12-200-t85-6.xml
UNSAT
TT=14.794

UNSAT
TT=40.649

UNSAT
TT=14.794

UNSAT
TT=162.467

UNSAT
TT=487.569

UNSAT
TT=157.74

UNSAT
TT=461.428

UNSAT
TT=170.504

UNSAT
TT=479.659

UNSAT
TT=38.783

UNSAT
TT=479.454

UNSAT
TT=145.889

UNSAT
TT=143.274

UNSAT
TT=345.061

UNSAT
TT=154.809

UNSAT
TT=365.626

UNSAT
TT=369.12

UNSAT
TT=366.365

UNSAT
TT=1037.64

UNSAT
TT=1001.28

? (TO)
TT=2527.89

UNSAT
TT=5771.44

UNSAT
TT=5776.81

? (TO)
TT=2521.96

UNSAT
TT=5726.26

UNSAT
TT=5873.33

? (TO)
TT=2519.77

Bench nameBest resultsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
Random/Random-m1-5-12-12t/
rand-5-12-12-200-t85-8.xml
UNSAT
TT=12.716

UNSAT
TT=37.421

UNSAT
TT=12.716

UNSAT
TT=123.586

UNSAT
TT=381.754

UNSAT
TT=109.554

UNSAT
TT=358.081

UNSAT
TT=131.002

UNSAT
TT=388.873

UNSAT
TT=35.02

UNSAT
TT=399.129

UNSAT
TT=110.894

UNSAT
TT=113.311

UNSAT
TT=248.507

UNSAT
TT=121.291

UNSAT
TT=249.887

UNSAT
TT=268.775

UNSAT
TT=250.429

UNSAT
TT=822.427

UNSAT
TT=837.095

? (TO)
TT=2527.83

UNSAT
TT=5426.3

UNSAT
TT=5380.26

? (TO)
TT=2528

UNSAT
TT=5806.39

UNSAT
TT=5337.38

? (TO)
TT=2519.9

Random/
Random-m1-5-8X-05c/
rand-5-8X-05c-44.xml
UNSAT
TT=83.873

UNSAT
TT=93.121

UNSAT
TT=83.873

UNSAT
TT=461.509

UNSAT
TT=1112.21

UNSAT
TT=539.149

UNSAT
TT=1052.68

UNSAT
TT=628.515

UNSAT
TT=1194.93

UNSAT
TT=121.082

UNSAT
TT=1121.29

UNSAT
TT=922.273

UNSAT
TT=791.866

UNSAT
TT=1827.27

UNSAT
TT=947.17

? (TO)
TT=2520.03

? (TO)
TT=2520.11

? (TO)
TT=2519.87

UNSAT
TT=14489.7

UNSAT
TT=13019.6

? (TO)
TT=2520.15

? (TO)
TT=15677.9

? (TO)
TT=15724.1

? (TO)
TT=2520.13

? (TO)
TT=15829.5

? (TO)
TT=15655

UNSAT
TT=228.336

Random/Random-m1-7-40-8t/
rand-7-40-8-40-15000-9.xml
UNSAT
TT=11.065

UNSAT
TT=13.584

UNSAT
TT=11.065

UNSAT
TT=66.549

UNSAT
TT=137.091

UNSAT
TT=67.29

UNSAT
TT=142.538

UNSAT
TT=65.535

UNSAT
TT=143.891

UNSAT
TT=21.963

UNSAT
TT=129.106

UNSAT
TT=265.367

UNSAT
TT=294.517

UNSAT
TT=793.299

UNSAT
TT=315.895

UNSAT
TT=312.515

UNSAT
TT=334.408

UNSAT
TT=315.279

UNSAT
TT=1754.43

UNSAT
TT=1415.66

? (TO)
TT=2520.13

UNSAT
TT=4109.93

UNSAT
TT=4166.52

? (TO)
TT=2520.16

UNSAT
TT=4115.97

UNSAT
TT=4194.81

UNSAT
TT=592.167

Random/Random-m1-7-40-8t/
rand-7-40-8-40-22500-0.xml
UNSAT
TT=64.945

UNSAT
TT=64.945

UNSAT
TT=93.348

UNSAT
TT=375.322

UNSAT
TT=805.523

UNSAT
TT=372.336

UNSAT
TT=712.777

UNSAT
TT=397.334

UNSAT
TT=783.576

UNSAT
TT=92.135

UNSAT
TT=994.076

? (TO)
TT=2520.09

? (TO)
TT=2520.13

? (TO)
TT=2520.16

? (TO)
TT=2520.09

? (TO)
TT=2520.05

? (TO)
TT=2520.05

? (TO)
TT=2520.08

? (TO)
TT=20071.1

UNSAT
TT=16739.6

? (TO)
TT=2520.15

? (TO)
TT=17443.8

? (TO)
TT=17449.2

? (TO)
TT=2520.17

? (TO)
TT=17047.3

? (TO)
TT=17515.2

UNSAT
TT=2050.37

Random/Random-RB-2-50-23f/
rand-2-50-23-587-230f-45.xml
SAT
TT=17.333

? (TO)
TT=2520.03

? (TO)
TT=2519.56

SAT
TT=1987.87

SAT
TT=4075.74

SAT
TT=1947.7

SAT
TT=4074.74

SAT
TT=1972.8

SAT
TT=3940.79

? (TO)
TT=2520.05

SAT
TT=4263.04

? (TO)
TT=2520.06

? (TO)
TT=2520.11

SAT
TT=871.059

? (TO)
TT=2520.11

SAT
TT=19.53

SAT
TT=17.333

SAT
TT=17.353

SAT
TT=6645.13

SAT
TT=6488.85

? (TO)
TT=2520.11

SAT
TT=10786.2

SAT
TT=11100.9

? (TO)
TT=2520.07

SAT
TT=10726.5

SAT
TT=10796.4

SAT
TT=729.404

Random/Random-RB-3-20-20/
rand-3-20-20-60-632-45.xml
UNSAT
TT=10.455

UNSAT
TT=12.62

UNSAT
TT=10.455

UNSAT
TT=49.348

UNSAT
TT=100.774

UNSAT
TT=46.925

UNSAT
TT=90.153

UNSAT
TT=46.55

UNSAT
TT=90.544

UNSAT
TT=14.341

UNSAT
TT=101.415

UNSAT
TT=88.665

UNSAT
TT=93.009

UNSAT
TT=205.063

UNSAT
TT=86.243

UNSAT
TT=128.466

UNSAT
TT=133.15

UNSAT
TT=132.909

UNSAT
TT=921.667

UNSAT
TT=990.983

UNSAT
TT=376.191

UNSAT
TT=687.573

UNSAT
TT=692.003

UNSAT
TT=247.014

UNSAT
TT=686.154

UNSAT
TT=689.895

UNSAT
TT=29.812

Random/Random-RB-3-24-24f/
rand-3-24-24-76-632f-49.xml
SAT
TT=410.687

SAT
TT=1100.05

SAT
TT=919.009

SAT
TT=1819.97

SAT
TT=3755.38

SAT
TT=437.851

SAT
TT=890.181

SAT
TT=456.222

SAT
TT=891.179

SAT
TT=413.805

SAT
TT=2784.4

? (TO)
TT=2520.15

? (TO)
TT=2520.06

? (TO)
TT=2520.11

? (TO)
TT=2520.11

? (TO)
TT=2519.62

? (TO)
TT=2520.07

? (TO)
TT=2519.75

SAT
TT=5652.92

SAT
TT=5690.4

? (TO)
TT=2520.1

SAT
TT=14934.8

SAT
TT=15092.4

? (TO)
TT=2520.14

SAT
TT=15529

SAT
TT=15084.2

SAT
TT=410.687

Random/Random-RB-3-24-24/
rand-3-24-24-76-632-16.xml
SAT
TT=68.022

SAT
TT=68.022

SAT
TT=105.329

SAT
TT=737.613

SAT
TT=1341.94

SAT
TT=696.769

SAT
TT=1307.58

SAT
TT=687.242

SAT
TT=1348.93

SAT
TT=187.067

SAT
TT=1815.78

SAT
TT=2467.25

SAT
TT=1726.38

? (TO)
TT=2520.1

SAT
TT=2052.57

? (TO)
TT=2520.04

? (TO)
TT=2520.07

? (TO)
TT=2519.96

SAT
TT=17908.5

SAT
TT=10883.1

? (TO)
TT=2520.07

SAT
TT=12093.2

SAT
TT=12309.6

? (TO)
TT=2520.09

SAT
TT=12096.4

SAT
TT=12348.1

SAT
TT=1419.65

RectPacking/
RectPacking-m1-perf/
RectPacking-perfect-012.xml
SAT
TT=18.042

? (TO)
TT=2520.02

? (NS)
TT=0.01

? (TO)
TT=2520.56

? (TO)
TT=20029

? (TO)
TT=2520.41

? (TO)
TT=19994.9

? (TO)
TT=2520.19

? (TO)
TT=19995.4

? (TO)
TT=2520.05

? (TO)
TT=19996.5

? (TO)
TT=2520.09

? (TO)
TT=2520.14

? (TO)
TT=2520.04

? (TO)
TT=2520.09

? (exit code)
TT=0.012

? (exit code)
TT=0.012

? (exit code)
TT=0.012

? (exit code)
TT=0.014

? (exit code)
TT=0.014

SAT
TT=63.389

SAT
TT=180.57

SAT
TT=184.631

SAT
TT=18.042

SAT
TT=182.381

SAT
TT=186.263

SAT
TT=214.025

RectPacking/
RectPacking-m1-perf/
RectPacking-perfect-023.xml
SAT
TT=88.546

? (TO)
TT=2520.07

? (NS)
TT=0.01

? (TO)
TT=2520.68

? (TO)
TT=20003.2

? (TO)
TT=2520.39

? (TO)
TT=19892.4

? (TO)
TT=2520.64

? (TO)
TT=19888

? (TO)
TT=2520.06

? (TO)
TT=19902.7

? (TO)
TT=2520.05

? (TO)
TT=2520.13

? (TO)
TT=2520.08

? (TO)
TT=2520.12

? (exit code)
TT=0.012

? (exit code)
TT=0.012

? (exit code)
TT=0.012

? (exit code)
TT=0.014

? (exit code)
TT=0.013

SAT
TT=293.061

SAT
TT=2704.26

SAT
TT=2607.95

SAT
TT=493.738

SAT
TT=2580.54

SAT
TT=2659.71

SAT
TT=88.546

Bench nameBest resultsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
RectPacking/
RectPacking-m1-perf/
RectPacking-perfect-029.xml
SAT
TT=242.093

? (TO)
TT=2520.05

? (NS)
TT=0.01

? (TO)
TT=2520.54

? (TO)
TT=19991.7

? (TO)
TT=2520.5

? (TO)
TT=19912.3

? (TO)
TT=2520.56

? (TO)
TT=19913.5

? (TO)
TT=2520.12

? (TO)
TT=19887.1

? (TO)
TT=2520.09

? (TO)
TT=2520.08

? (TO)
TT=2520.08

? (TO)
TT=2520.13

? (exit code)
TT=0.011

? (exit code)
TT=0.012

? (exit code)
TT=0.012

? (exit code)
TT=0.014

? (exit code)
TT=0.016

SAT
TT=1478.18

SAT
TT=242.093

SAT
TT=245.585

SAT
TT=492.524

SAT
TT=244.654

SAT
TT=244.861

SAT
TT=422.224

RectPacking/
RectPacking-m1-perf/
RectPacking-perfect-033.xml
SAT
TT=36.497

? (TO)
TT=2520.03

? (NS)
TT=0.011

SAT
TT=2056.89

SAT
TT=4682.48

SAT
TT=1878.88

SAT
TT=4401

SAT
TT=1973.14

SAT
TT=3686.79

SAT
TT=432.244

SAT
TT=5421.62

? (TO)
TT=2520.14

? (TO)
TT=2520.1

? (TO)
TT=2520.08

? (TO)
TT=2520.11

? (MO)
TT=175.142

? (MO)
TT=174.934

? (MO)
TT=175.97

?
TT=360.741

?
TT=363.824

SAT
TT=44.414

SAT
TT=147.096

SAT
TT=152.527

SAT
TT=49.152

SAT
TT=156.865

SAT
TT=151.207

SAT
TT=36.497

RectPacking/
RectPacking-m1-perf/
RectPacking-perfect-097.xml
SAT
TT=53.043

? (TO)
TT=2520.02

? (NS)
TT=0.011

? (TO)
TT=2520.35

? (TO)
TT=19964.1

? (TO)
TT=2520.33

? (TO)
TT=19793.2

? (TO)
TT=2520.52

? (TO)
TT=19755.3

? (TO)
TT=2520.11

? (TO)
TT=19950.1

? (TO)
TT=2520.04

? (TO)
TT=2520.08

? (TO)
TT=2520.13

? (TO)
TT=2520.11

? (MO)
TT=175.041

? (MO)
TT=173.536

? (MO)
TT=171.074

?
TT=405.543

?
TT=350.927

SAT
TT=53.043

SAT
TT=171.552

SAT
TT=171.666

SAT
TT=57.975

SAT
TT=173.821

SAT
TT=172.048

SAT
TT=81.138

RectPacking/
RectPacking-m1-perf/
RectPacking-perfect-126.xml
SAT
TT=45.919

? (TO)
TT=2520.05

? (NS)
TT=0.012

? (TO)
TT=2520.68

? (TO)
TT=19891.6

? (TO)
TT=2520.23

? (TO)
TT=19857

? (TO)
TT=2520.31

? (TO)
TT=19818

? (TO)
TT=2520.09

? (TO)
TT=19826.5

? (TO)
TT=2520.07

? (TO)
TT=2520.05

? (TO)
TT=2520.14

? (TO)
TT=2520.07

? (MO)
TT=171.477

? (MO)
TT=179.598

? (MO)
TT=171.652

?
TT=351.018

?
TT=366.298

SAT
TT=47.075

SAT
TT=315.315

SAT
TT=311.689

SAT
TT=55.972

SAT
TT=310.942

SAT
TT=318.439

SAT
TT=45.919

RectPacking/
RectPacking-m1-perf/
RectPacking-perfect-142.xml
SAT
TT=46.335

? (TO)
TT=2520.11

? (NS)
TT=0.013

? (TO)
TT=2520.51

? (TO)
TT=19884.6

? (TO)
TT=2520.63

? (TO)
TT=19789.1

? (TO)
TT=2520.64

? (TO)
TT=19811.7

? (TO)
TT=2520.03

? (TO)
TT=19815.3

? (TO)
TT=2520.1

? (TO)
TT=2520.05

? (TO)
TT=2520.1

? (TO)
TT=2520.06

? (MO)
TT=199.915

? (MO)
TT=175.379

? (MO)
TT=172.671

?
TT=359.333

?
TT=352.798

SAT
TT=123.498

SAT
TT=96.335

SAT
TT=96.595

SAT
TT=172.031

SAT
TT=96.17

SAT
TT=97.951

SAT
TT=46.335

RectPacking/
RectPacking-m1-perf/
RectPacking-perfect-152.xml
SAT
TT=69.07

? (TO)
TT=2520.08

? (NS)
TT=0.013

? (TO)
TT=2520.66

? (TO)
TT=19938.5

? (TO)
TT=2520.53

? (TO)
TT=19793.3

? (TO)
TT=2520.74

? (TO)
TT=19782.7

? (TO)
TT=2520.13

? (TO)
TT=19714.6

? (TO)
TT=2520.06

? (TO)
TT=2520.09

? (TO)
TT=2520.09

? (TO)
TT=2520.11

? (exit code)
TT=0.012

? (exit code)
TT=0.012

? (exit code)
TT=0.013

? (exit code)
TT=0.016

? (exit code)
TT=0.015

SAT
TT=1041.24

SAT
TT=1151.2

SAT
TT=1226.29

SAT
TT=279.437

SAT
TT=1177.68

SAT
TT=1161.88

SAT
TT=69.07

RectPacking/
RectPacking-m1-perf/
RectPacking-perfect-154.xml
SAT
TT=369.492

? (TO)
TT=2520.06

? (NS)
TT=0.013

? (TO)
TT=2520.48

? (TO)
TT=19916.1

? (TO)
TT=2520.51

? (TO)
TT=19783.9

? (TO)
TT=2520.61

? (TO)
TT=19798.9

? (TO)
TT=2520.09

? (TO)
TT=19782.6

? (TO)
TT=2520.05

? (TO)
TT=2520.06

? (TO)
TT=2520.12

? (TO)
TT=2520.08

? (MO)
TT=175.776

? (MO)
TT=178.736

? (MO)
TT=175.828

?
TT=354.705

?
TT=358.686

? (TO)
TT=2520.11

SAT
TT=2854.27

SAT
TT=2787.99

SAT
TT=431

SAT
TT=2797.63

SAT
TT=2808.08

SAT
TT=369.492

RectPacking/
RectPacking-m1-perf/
RectPacking-perfect-160.xml
SAT
TT=146.729

? (TO)
TT=2520.05

? (NS)
TT=0.013

? (TO)
TT=2520.49

? (TO)
TT=19968.5

? (TO)
TT=2520.35

? (TO)
TT=19815.5

? (TO)
TT=2520.58

? (TO)
TT=19761.2

? (TO)
TT=2520.12

? (TO)
TT=19723.7

? (TO)
TT=2520.14

? (TO)
TT=2520.08

? (TO)
TT=2520.11

? (TO)
TT=2520.13

? (MO)
TT=176.632

? (MO)
TT=201.543

? (MO)
TT=184.607

?
TT=362.334

?
TT=361.171

SAT
TT=245.808

SAT
TT=1486.89

SAT
TT=1509.03

SAT
TT=477.486

SAT
TT=1459.92

SAT
TT=1523.67

SAT
TT=146.729

RectPacking/
RectPacking-m1-perf/
RectPacking-perfect-183.xml
SAT
TT=17.706

? (TO)
TT=2520.02

? (NS)
TT=0.013

? (TO)
TT=2520.61

SAT
TT=5586.24

? (TO)
TT=2520.71

SAT
TT=4969.61

? (TO)
TT=2520.41

SAT
TT=9332.57

SAT
TT=693.178

SAT
TT=4604.62

SAT
TT=90.109

? (TO)
TT=2520.08

? (TO)
TT=2520.09

SAT
TT=797.925

? (exit code)
TT=0.012

? (exit code)
TT=0.012

? (exit code)
TT=0.012

? (exit code)
TT=0.016

? (exit code)
TT=0.018

SAT
TT=20.881

SAT
TT=94.909

SAT
TT=94.908

SAT
TT=19.279

SAT
TT=94.204

SAT
TT=94.355

SAT
TT=17.706

RectPacking/
RectPacking-m1-perf/
RectPacking-perfect-187.xml



? (TO)
TT=2520.11

? (NS)
TT=0.013

? (TO)
TT=2520.64

? (TO)
TT=19976.9

? (TO)
TT=2520.37

? (TO)
TT=19875.2

? (TO)
TT=2520.39

? (TO)
TT=19836.3

? (TO)
TT=2520.08

? (TO)
TT=19663.2

? (TO)
TT=2520.12

? (TO)
TT=2520.08

? (TO)
TT=2520.1

? (TO)
TT=2520.13

? (exit code)
TT=0.013

? (exit code)
TT=0.012

? (exit code)
TT=0.014

? (exit code)
TT=0.016

? (exit code)
TT=0.016

? (TO)
TT=2520.08

? (TO)
TT=19791.3

? (TO)
TT=19824.6

? (TO)
TT=2520.13

? (TO)
TT=19761.3

? (TO)
TT=19787.2

? (TO)
TT=2519.8

Bench nameBest resultsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
RectPacking/
RectPacking-m1-perf/
RectPacking-perfect-194.xml



? (TO)
TT=2520.03

? (NS)
TT=0.013

? (TO)
TT=2520.73

? (TO)
TT=19978.1

? (TO)
TT=2520.32

? (TO)
TT=19779

? (TO)
TT=2520.44

? (TO)
TT=19796.5

? (TO)
TT=2520.04

? (TO)
TT=19833

? (TO)
TT=2520.1

? (TO)
TT=2520.13

? (TO)
TT=2520.13

? (TO)
TT=2520.11

? (MO)
TT=177.196

? (MO)
TT=180.377

? (MO)
TT=179.481

?
TT=373.985

?
TT=354.124

? (TO)
TT=2520.12

? (TO)
TT=19732.1

? (TO)
TT=19659.5

? (TO)
TT=2520.1

? (TO)
TT=19730.5

? (TO)
TT=19715.9

? (TO)
TT=2519.95

RectPacking/
RectPacking-m1-perf/
RectPacking-perfect-199.xml
SAT
TT=17.439

? (TO)
TT=2520.08

? (NS)
TT=0.014

? (TO)
TT=2520.57

? (TO)
TT=19927.6

? (TO)
TT=2520.42

? (TO)
TT=19792.3

? (TO)
TT=2520.44

? (TO)
TT=19785.4

? (TO)
TT=2520.13

? (TO)
TT=19806.9

? (TO)
TT=2520.11

? (TO)
TT=2520.07

? (TO)
TT=2520.05

? (TO)
TT=2520.13

? (MO)
TT=172.099

? (MO)
TT=174.16

? (MO)
TT=173.212

?
TT=365.258

?
TT=349.177

SAT
TT=48.367

SAT
TT=145.772

SAT
TT=145.659

SAT
TT=37.842

SAT
TT=145.979

SAT
TT=146.521

SAT
TT=17.439

RectPacking/
RectPacking-m1-perf/
RectPacking-perfect-203.xml
SAT
TT=141.487

? (TO)
TT=2520.11

? (NS)
TT=0.014

? (TO)
TT=2520.55

? (TO)
TT=20008.7

? (TO)
TT=2520.29

? (TO)
TT=19766.1

? (TO)
TT=2520.61

? (TO)
TT=19838.4

? (TO)
TT=2520.08

? (TO)
TT=19836.9

? (TO)
TT=2520.06

? (TO)
TT=2520.07

? (TO)
TT=2520.04

? (TO)
TT=2520.07

? (MO)
TT=180.694

? (MO)
TT=180.519

? (MO)
TT=180.841

?
TT=361.399

?
TT=416.087

SAT
TT=435.502

SAT
TT=930.242

SAT
TT=943.071

SAT
TT=205.19

SAT
TT=941.647

SAT
TT=943.165

SAT
TT=141.487

Rlfap_mini/
Rlfap-dec-scens11_mini/
Rlfap-ext-scen-11-f02_c18.xml
UNSAT
TT=39.172

UNSAT
TT=267.204

? (TO)
TT=2519.92

? (TO)
TT=2520.51

UNSAT
TT=13573.1

? (TO)
TT=2520.34

UNSAT
TT=13949.2

? (TO)
TT=2520.37

UNSAT
TT=13974.4

UNSAT
TT=1433.96

UNSAT
TT=12715.1

UNSAT
TT=1660.84

UNSAT
TT=2268.08

? (TO)
TT=2520.08

UNSAT
TT=2050.11

UNSAT
TT=300.596

UNSAT
TT=327.899

UNSAT
TT=297.166

UNSAT
TT=2071.25

UNSAT
TT=1920.78

UNSAT
TT=39.172

UNSAT
TT=46.464

UNSAT
TT=47.663

UNSAT
TT=40.27

UNSAT
TT=48.826

UNSAT
TT=44.649

UNSAT
TT=177.102

Rlfap/Rlfap-dec-scens11/
Rlfap-scen-11-f05_c18.xml
UNSAT
TT=11.014

UNSAT
TT=19.173

UNSAT
TT=15.231

UNSAT
TT=578.309

UNSAT
TT=1063.5

UNSAT
TT=526.098

UNSAT
TT=1068.37

UNSAT
TT=522.881

UNSAT
TT=929.838

UNSAT
TT=90.303

UNSAT
TT=1280.2

UNSAT
TT=339.252

UNSAT
TT=377.936

? (TO)
TT=2520.16

UNSAT
TT=402.196

UNSAT
TT=10.975

UNSAT
TT=11.014

UNSAT
TT=11.032

UNSAT
TT=20.199

UNSAT
TT=22.931

UNSAT
TT=17.216

UNSAT
TT=21.95

UNSAT
TT=22.124

UNSAT
TT=17.402

UNSAT
TT=22.813

UNSAT
TT=21.444

UNSAT
TT=63.434

Rlfap/Rlfap-m1-graphsMod/
Rlfap-graph-08-f10.xml
SAT
TT=3.795

SAT
TT=9.434

SAT
TT=3.795

? (TO)
TT=2520.43

SAT
TT=211.312

? (TO)
TT=2520.42

SAT
TT=1734.6

? (TO)
TT=2520.65

SAT
TT=696.909

SAT
TT=101.196

SAT
TT=1541.76

SAT
TT=28.72

SAT
TT=29.535

SAT
TT=71.518

SAT
TT=30.455

SAT
TT=4.745

SAT
TT=3.982

SAT
TT=3.994

SAT
TT=5.137

SAT
TT=4.824

SAT
TT=16.024

SAT
TT=18.782

SAT
TT=18.782

SAT
TT=15.658

SAT
TT=18.292

SAT
TT=18.903

SAT
TT=141.655

Rlfap/Rlfap-m1-graphs/
Rlfap-graph-10.xml
SAT
TT=1.842

SAT
TT=9.827

SAT
TT=1.842

SAT
TT=23.346

SAT
TT=63.143

SAT
TT=26.289

SAT
TT=48.517

SAT
TT=25.595

SAT
TT=33.05

SAT
TT=10.029

SAT
TT=36.763

SAT
TT=27.623

SAT
TT=27.578

SAT
TT=90.121

SAT
TT=27.146

SAT
TT=6.315

SAT
TT=5.535

SAT
TT=5.675

SAT
TT=8.614

SAT
TT=7.493

SAT
TT=16.891

SAT
TT=18.985

SAT
TT=19.202

SAT
TT=16.943

SAT
TT=18.567

SAT
TT=20.519

SAT
TT=158.833

Rlfap/Rlfap-m1-scens11/
Rlfap-scen-11-f04.xml
UNSAT
TT=16.961

UNSAT
TT=63.036

UNSAT
TT=52.233

UNSAT
TT=2212.98

UNSAT
TT=4631.15

UNSAT
TT=2118.27

UNSAT
TT=4074.1

UNSAT
TT=2091.45

UNSAT
TT=4199.9

UNSAT
TT=297.204

UNSAT
TT=4076.92

UNSAT
TT=1311.07

UNSAT
TT=1489.64

? (TO)
TT=2520.13

UNSAT
TT=1484.95

UNSAT
TT=31.444

UNSAT
TT=28.006

UNSAT
TT=27.718

UNSAT
TT=161.859

UNSAT
TT=130.735

UNSAT
TT=17.281

UNSAT
TT=22.252

UNSAT
TT=22.41

UNSAT
TT=16.961

UNSAT
TT=22.671

UNSAT
TT=21.874

UNSAT
TT=50.173

Rlfap/
Rlfap-m1-scens/
Rlfap-scen-01.xml
SAT
TT=1.65

SAT
TT=9.32

SAT
TT=1.65

SAT
TT=14.344

SAT
TT=23.166

SAT
TT=13.75

SAT
TT=21.834

SAT
TT=12.139

SAT
TT=22.127

SAT
TT=11.753

SAT
TT=22.452

SAT
TT=39.73

SAT
TT=35.086

SAT
TT=66.442

SAT
TT=34.929

SAT
TT=8.415

SAT
TT=7.329

SAT
TT=7.338

SAT
TT=8.603

SAT
TT=8.547

SAT
TT=19.419

SAT
TT=23.333

SAT
TT=22.248

SAT
TT=19.364

SAT
TT=22.094

SAT
TT=22.104

SAT
TT=83.95

RoomMate/RoomMate-int-s1/
RoomMate-sr0600-int.xml
UNSAT
TT=118.465

UNSAT
TT=671.018

? (TO)
TT=2519.79

?
TT=1183.46

? (TO)
TT=17667.8

?
TT=1174.24

?
TT=3029.47

?
TT=1260.06

?
TT=1594.19

?
TT=1245.52

?
TT=1091.1

? (TO)
TT=2528.08

? (TO)
TT=2528.1

? (TO)
TT=2527.18

? (TO)
TT=2527.57

? (MO)
TT=188.148

? (MO)
TT=192.922

? (exit code)
TT=192.578

? (MO)
TT=114.527

? (MO)
TT=111.87

UNSAT
TT=341.548

UNSAT
TT=446.496

UNSAT
TT=415.207

UNSAT
TT=339.202

UNSAT
TT=394.452

UNSAT
TT=399.422

UNSAT
TT=118.465

Bench nameBest resultsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
RoomMate/RoomMate-int-s1/
RoomMate-sr0700d-int.xml
SAT
TT=176.644

? (problem)
TT=718.553

? (TO)
TT=2519.29

?
TT=565.544

? (TO)
TT=17649.3

?
TT=591.36

?
TT=1724.08

?
TT=631.331

? (TO)
TT=17773

?
TT=558.915

?
TT=1879.32

? (TO)
TT=2528.35

? (TO)
TT=2527.15

? (TO)
TT=2526.94

? (TO)
TT=2528.51

? (MO)
TT=194.111

? (MO)
TT=195.621

? (MO)
TT=171.369

? (MO)
TT=101.601

? (MO)
TT=101.304

SAT
TT=328.723

SAT
TT=385.235

SAT
TT=379.496

SAT
TT=372.629

SAT
TT=378.285

SAT
TT=383.773

SAT
TT=176.644

RoomMate/RoomMate-int-s1/
RoomMate-sr1000b-int.xml
UNSAT
TT=492.922

? (problem)
TT=657.178

? (TO)
TT=2519.72

?
TT=1183.45

?
TT=2530.85

?
TT=1167.78

?
TT=15662.8

?
TT=652.641

?
TT=2387.06

?
TT=1183.42

?
TT=16350.5

? (TO)
TT=2528.05

? (TO)
TT=2528.09

? (TO)
TT=2527.21

? (TO)
TT=2528.73

? (MO)
TT=142.191

? (MO)
TT=146.579

? (MO)
TT=141.87

? (MO)
TT=86.443

? (MO)
TT=87.04

UNSAT
TT=1031.35

UNSAT
TT=1039.25

UNSAT
TT=1089.18

UNSAT
TT=991.161

UNSAT
TT=1237.8

UNSAT
TT=1171.38

UNSAT
TT=492.922

RoomMate/RoomMate-int-s1/
RoomMate-sr1000c-int.xml
UNSAT
TT=495.64

? (problem)
TT=632.983

? (TO)
TT=2519.17

?
TT=1200.26

? (TO)
TT=17906.3

?
TT=1234.5

?
TT=2157.33

?
TT=1420

? (TO)
TT=17795.7

?
TT=1168.58

?
TT=2088.88

? (TO)
TT=2528.3

? (TO)
TT=2527.99

? (TO)
TT=2528.3

? (TO)
TT=2528.52

? (MO)
TT=140.222

? (MO)
TT=142.858

? (MO)
TT=144

? (MO)
TT=86.414

? (MO)
TT=87.343

UNSAT
TT=1479.4

UNSAT
TT=1688.26

UNSAT
TT=1778.37

UNSAT
TT=1561

UNSAT
TT=1844.03

UNSAT
TT=1743.67

UNSAT
TT=495.64

RoomMate/RoomMate-int-s1/
RoomMate-sr1000e-int.xml
UNSAT
TT=490.68

? (problem)
TT=633.483

? (TO)
TT=2519.6

?
TT=1189.67

?
TT=2330.73

?
TT=1273.21

? (TO)
TT=17868

?
TT=1624.47

?
TT=2303.43

?
TT=1206.12

?
TT=2238.43

? (TO)
TT=2528.22

? (TO)
TT=2528.07

? (TO)
TT=2528.25

? (TO)
TT=2528.43

? (MO)
TT=141.67

? (MO)
TT=161.232

? (MO)
TT=142.01

? (MO)
TT=87.347

? (MO)
TT=87.354

UNSAT
TT=995.208

UNSAT
TT=1159.82

UNSAT
TT=1088.7

UNSAT
TT=1000.3

UNSAT
TT=1106.42

UNSAT
TT=1026.9

UNSAT
TT=490.68

RoomMate/RoomMate-int-s1/
RoomMate-sr1000-int.xml
UNSAT
TT=492.618

? (problem)
TT=706.348

? (TO)
TT=2519.67

?
TT=1178.72

? (TO)
TT=17842

?
TT=1172.42

?
TT=3361.79

?
TT=1286.88

? (TO)
TT=17902.1

?
TT=1257.17

? (TO)
TT=18060.3

? (TO)
TT=2527.33

? (TO)
TT=2528.3

? (TO)
TT=2526.93

? (TO)
TT=2528.32

? (MO)
TT=141.493

? (MO)
TT=142.29

? (MO)
TT=144.494

? (MO)
TT=86.503

? (MO)
TT=88.205

UNSAT
TT=1248.94

UNSAT
TT=1392.34

UNSAT
TT=1308.07

UNSAT
TT=1233.45

UNSAT
TT=1350.4

UNSAT
TT=1516.87

UNSAT
TT=492.618

Sat/
Sat-m1-various/
hole-10.xml
UNSAT
TT=51.609

UNSAT
TT=915.491

UNSAT
TT=1710.27

? (TO)
TT=2520.36

UNSAT
TT=7825.67

? (TO)
TT=2520.38

UNSAT
TT=8468.07

? (TO)
TT=2520.63

UNSAT
TT=8924.3

? (TO)
TT=2520.12

UNSAT
TT=9518.83

UNSAT
TT=2276.13

? (TO)
TT=2520.11

? (TO)
TT=2520.14

? (TO)
TT=2520.1

UNSAT
TT=1388.75

UNSAT
TT=1392.87

UNSAT
TT=1394.9

UNSAT
TT=4086.14

UNSAT
TT=3413.41

UNSAT
TT=51.609

UNSAT
TT=394.03

UNSAT
TT=423.44

UNSAT
TT=53.012

UNSAT
TT=391.671

UNSAT
TT=390.03

UNSAT
TT=63.921

Sat/Sat-m1-various/
pret-150-40.xml
UNSAT
TT=0.011

? (TO)
TT=2520.02

UNSAT
TT=21.166

? (TO)
TT=2520.59

UNSAT
TT=258.436

? (TO)
TT=2520.62

? (TO)
TT=19844.8

? (TO)
TT=2520.55

? (TO)
TT=19954.4

? (TO)
TT=2520.08

? (TO)
TT=19984.3

? (TO)
TT=2520.09

? (TO)
TT=2520.08

? (TO)
TT=2520.07

? (TO)
TT=2520.11

? (TO)
TT=2520.05

? (TO)
TT=2519.94

? (TO)
TT=2520.05

? (TO)
TT=20035.6

? (TO)
TT=20026.9

UNSAT
TT=5.652

UNSAT
TT=5.893

UNSAT
TT=5.947

UNSAT
TT=5.586

UNSAT
TT=6.497

UNSAT
TT=5.815

UNSAT
TT=0.011

Scheduling/
Scheduling-jsdec-sadeh/
Sadeh-enddr1-7.xml
SAT
TT=0.022

SAT
TT=3.997

SAT
TT=0.022

SAT
TT=1.833

SAT
TT=3.014

SAT
TT=1.748

SAT
TT=3.427

SAT
TT=1.829

SAT
TT=2.753

SAT
TT=1.39

SAT
TT=2.699

SAT
TT=10.344

SAT
TT=10.254

SAT
TT=11.413

SAT
TT=13.613

SAT
TT=0.052

SAT
TT=0.044

SAT
TT=0.048

SAT
TT=0.095

SAT
TT=0.096

SAT
TT=5.933

SAT
TT=6.06

SAT
TT=6.04

SAT
TT=5.955

SAT
TT=6.028

SAT
TT=6.263

SAT
TT=2.197

SchurrLemma/
SchurrLemma-mod-s1/
SchurrLemma-mod-100-9.xml
UNSAT
TT=8.768

UNSAT
TT=700.863

UNSAT
TT=245.888

UNSAT
TT=1178.11

UNSAT
TT=2587.22

UNSAT
TT=1299.41

UNSAT
TT=2736.18

UNSAT
TT=1250.03

UNSAT
TT=2575.16

UNSAT
TT=297.802

UNSAT
TT=2734.93

UNSAT
TT=8.768

UNSAT
TT=9.393

UNSAT
TT=9.325

UNSAT
TT=8.887

UNSAT
TT=530.536

UNSAT
TT=536.037

UNSAT
TT=526.303

UNSAT
TT=2624.27

UNSAT
TT=2775.84

UNSAT
TT=23.902

UNSAT
TT=20.513

UNSAT
TT=19.724

UNSAT
TT=15.804

UNSAT
TT=19.693

UNSAT
TT=19.939

UNSAT
TT=35.831

SocialGolfers/
SocialGolfers-cp-s1/
SocialGolfers-5-3-4-cp.xml
SAT
TT=0.123

SAT
TT=4.715

SAT
TT=0.141

SAT
TT=4.131

SAT
TT=7.157

SAT
TT=4.166

SAT
TT=6.704

SAT
TT=4.329

SAT
TT=7.497

SAT
TT=2.261

SAT
TT=6.173

SAT
TT=10.911

SAT
TT=9.905

SAT
TT=10.717

SAT
TT=9.275

SAT
TT=0.298

SAT
TT=0.308

SAT
TT=0.297

SAT
TT=0.35

SAT
TT=0.358

SAT
TT=7.684

SAT
TT=8.276

SAT
TT=8.236

SAT
TT=7.582

SAT
TT=8.177

SAT
TT=7.894

SAT
TT=0.123

Bench nameBest resultsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
SocialGolfers/
SocialGolfers-cp-s1/
SocialGolfers-5-5-9-cp.xml
UNSAT
TT=15.753

? (TO)
TT=2520.1

? (TO)
TT=2520.01

? (TO)
TT=2521.84

? (TO)
TT=20021.3

? (TO)
TT=2521.54

? (TO)
TT=20026.4

? (TO)
TT=2520.53

? (TO)
TT=20026.5

? (TO)
TT=2520.1

? (TO)
TT=20020.7

? (TO)
TT=2520.13

? (TO)
TT=2520.1

? (TO)
TT=2520.13

? (TO)
TT=2520.06

? (TO)
TT=2520.06

? (TO)
TT=2520.02

? (TO)
TT=2519.74

? (TO)
TT=19990.7

? (TO)
TT=19986.7

UNSAT
TT=20.988

UNSAT
TT=26.208

UNSAT
TT=26.31

UNSAT
TT=22.25

UNSAT
TT=26.492

UNSAT
TT=26.059

UNSAT
TT=15.753

SocialGolfers/
SocialGolfers-cp-s1/
SocialGolfers-6-5-5-cp.xml
SAT
TT=117.039

? (TO)
TT=2520.03

? (TO)
TT=2519.95

? (TO)
TT=2520.69

SAT
TT=19469.4

? (TO)
TT=2520.56

? (TO)
TT=20041.2

? (TO)
TT=2520.66

? (TO)
TT=20043.5

? (TO)
TT=2520.04

? (TO)
TT=20042.9

? (TO)
TT=2520.14

? (TO)
TT=2520.07

? (TO)
TT=2520.03

? (TO)
TT=2520.06

? (TO)
TT=2519.85

? (TO)
TT=2519.92

? (TO)
TT=2520.01

? (TO)
TT=20082

? (TO)
TT=20083.2

SAT
TT=117.039

SAT
TT=7643.91

SAT
TT=7898.54

SAT
TT=1382.22

SAT
TT=7049.31

SAT
TT=7114.06

SAT
TT=235.74

SocialGolfers/
SocialGolfers-cp-s1/
SocialGolfers-8-4-8-cp.xml



? (TO)
TT=2520.1

? (TO)
TT=2520.01

? (TO)
TT=2528.15

?
TT=18216.7

? (TO)
TT=2528.11

UNSAT
TT=4318.45

? (TO)
TT=2528.69

?
TT=14354.5

? (TO)
TT=2520.1

? (TO)
TT=19215.2

? (TO)
TT=2520.09

? (TO)
TT=2520.11

? (TO)
TT=2520.07

? (TO)
TT=2520.11

? (TO)
TT=2519.96

? (TO)
TT=2519.99

? (TO)
TT=2520.03

? (TO)
TT=19997.4

? (TO)
TT=19994.8

? (TO)
TT=2520.07

? (TO)
TT=19452.1

? (TO)
TT=19369.4

? (TO)
TT=2520.07

? (TO)
TT=19002.5

? (TO)
TT=19013

? (TO)
TT=2519.94

SocialGolfers/
SocialGolfers-cp-s1/
SocialGolfers-8-5-6-cp.xml
SAT
TT=40.322

? (TO)
TT=2520.05

? (TO)
TT=2519.96

? (TO)
TT=2520.68

SAT
TT=11261.7

? (TO)
TT=2520.42

SAT
TT=11521.3

? (TO)
TT=2520.71

SAT
TT=13794.2

? (TO)
TT=2520.09

SAT
TT=12050.1

SAT
TT=1334.26

SAT
TT=431.039

? (TO)
TT=2520.14

? (TO)
TT=2520.13

SAT
TT=274.311

SAT
TT=197.811

SAT
TT=243.705

SAT
TT=1482.45

SAT
TT=2089.17

SAT
TT=285.695

? (TO)
TT=19462.8

? (TO)
TT=19183.8

SAT
TT=795.664

? (TO)
TT=19384.5

? (TO)
TT=19560.8

SAT
TT=40.322

SocialGolfers/
SocialGolfers-cp-s1/
SocialGolfers-8-9-9-cp.xml
UNSAT
TT=27.215

UNSAT
TT=27.215

UNSAT
TT=178.678

? (TO)
TT=2528.15

? (TO)
TT=19262.9

? (TO)
TT=2528

? (TO)
TT=19285.2

?
TT=1077.67

? (TO)
TT=19230.8

? (TO)
TT=2528.41

?
TT=3582.7

UNSAT
TT=169.376

UNSAT
TT=150.623

? (TO)
TT=2520.21

UNSAT
TT=123.939

UNSAT
TT=1220.12

UNSAT
TT=1211.76

UNSAT
TT=1202.74

UNSAT
TT=537.097

UNSAT
TT=562.016

UNSAT
TT=57.094

UNSAT
TT=55.143

UNSAT
TT=52.477

UNSAT
TT=75.733

UNSAT
TT=50.741

UNSAT
TT=51.148

UNSAT
TT=63.328

SocialGolfers/
SocialGolfers-cp-s1/
SocialGolfers-9-7-4-cp.xml
SAT
TT=16.867

? (TO)
TT=2520.08

? (TO)
TT=2520.01

?
TT=1306.54

?
TT=15226.4

? (TO)
TT=2528.17

?
TT=18773.3

?
TT=1386.08

? (TO)
TT=19183.2

? (TO)
TT=2520.1

? (TO)
TT=19048.6

SAT
TT=39.355

? (TO)
TT=2520.13

SAT
TT=205.745

SAT
TT=32.096

SAT
TT=143.505

SAT
TT=141.734

SAT
TT=116.634

SAT
TT=54.504

SAT
TT=51.297

SAT
TT=26.438

SAT
TT=31.885

SAT
TT=32.219

SAT
TT=26.471

SAT
TT=38.694

SAT
TT=36.724

SAT
TT=16.867

SocialGolfers/
SocialGolfers-cp-s1/
SocialGolfers-9-8-4-cp.xml
SAT
TT=138.482

? (TO)
TT=2520.08

? (TO)
TT=2519.67

?
TT=1157.62

?
TT=7479.66

? (TO)
TT=2528.11

? (TO)
TT=19238.4

?
TT=1207.91

?
TT=15863.9

? (TO)
TT=2520.06

?
TT=15111.5

? (TO)
TT=2520.17

SAT
TT=1483.49

? (TO)
TT=2520.09

? (TO)
TT=2520.12

? (TO)
TT=2516

? (TO)
TT=2519.91

? (TO)
TT=2519.82

SAT
TT=2512.14

SAT
TT=15352

SAT
TT=138.482

SAT
TT=4196.9

SAT
TT=4223.34

SAT
TT=2148.82

SAT
TT=5106.46

SAT
TT=5199.83

SAT
TT=488.315

SportsScheduling/
SportsScheduling-m1-s1/
SportsScheduling-14.xml
SAT
TT=167.509

SAT
TT=189.757

? (TO)
TT=2519.81

? (TO)
TT=2520.46

? (TO)
TT=20057

SAT
TT=688.387

SAT
TT=1377.01

SAT
TT=691.296

SAT
TT=1479.39

? (TO)
TT=2520.08

? (TO)
TT=19982.1

SAT
TT=175.861

SAT
TT=255.245

SAT
TT=1521.78

SAT
TT=221.619

? (TO)
TT=2520.04

? (TO)
TT=2520.1

? (TO)
TT=2520.06

? (TO)
TT=20105.6

? (TO)
TT=20101.9

SAT
TT=167.509

SAT
TT=8244.69

SAT
TT=8289.09

SAT
TT=594.669

SAT
TT=2121.54

SAT
TT=2129.28

SAT
TT=318.415

StripPacking/
StripPacking-m1-series1/
StripPacking-C3P2.xml
SAT
TT=226.379

? (TO)
TT=2520.08

? (NS)
TT=0.02

? (TO)
TT=2520.52

SAT
TT=11843.3

? (TO)
TT=2520.48

SAT
TT=8051.62

? (TO)
TT=2520.63

SAT
TT=9134.67

? (TO)
TT=2520.03

? (TO)
TT=20032.7

? (TO)
TT=2520.14

? (TO)
TT=2520.11

? (TO)
TT=2520.09

? (TO)
TT=2520.06

? (TO)
TT=2520.04

? (TO)
TT=2520.07

? (TO)
TT=2519.83

? (TO)
TT=20122.7

? (TO)
TT=20119.2

SAT
TT=226.379

SAT
TT=965.432

SAT
TT=979.357

? (TO)
TT=2520.11

SAT
TT=966.937

SAT
TT=977.075

? (TO)
TT=2519.96

StripPacking/
StripPacking-m1-seriesN/
StripPacking-N1c.xml
SAT
TT=122.854

? (TO)
TT=2520.05

? (NS)
TT=0.015

? (TO)
TT=2520.47

? (TO)
TT=20041.8

? (TO)
TT=2520.59

? (TO)
TT=20015.5

? (TO)
TT=2520.5

? (TO)
TT=20014.4

? (TO)
TT=2520.07

? (TO)
TT=20031.8

? (TO)
TT=2520.11

? (TO)
TT=2520.11

? (TO)
TT=2520.14

? (TO)
TT=2520.05

? (MO)
TT=426.778

? (MO)
TT=440.295

? (MO)
TT=430.31

?
TT=862.523

?
TT=848.375

SAT
TT=346.38

SAT
TT=126.926

SAT
TT=126.491

SAT
TT=652.589

SAT
TT=122.854

SAT
TT=125.304

SAT
TT=148.904

Bench nameBest resultsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
StripPacking/
StripPacking-m1-seriesN/
StripPacking-N6d.xml



? (TO)
TT=2520.06

? (NS)
TT=0.06

? (TO)
TT=2520.36

? (TO)
TT=19788.5

? (TO)
TT=2520.76

? (TO)
TT=19410.7

? (TO)
TT=2520.5

? (TO)
TT=19412.4

? (TO)
TT=2520.11

? (TO)
TT=19385.5

? (TO)
TT=2520.06

? (TO)
TT=2520.1

?
TT=27.738

? (TO)
TT=2520.18

? (MO)
TT=392.784

? (MO)
TT=390.89

? (MO)
TT=390.648

? (MO)
TT=755.419

? (MO)
TT=746.926

? (TO)
TT=2520.18

? (TO)
TT=18683.8

? (TO)
TT=18568.9

? (TO)
TT=2520.13

? (TO)
TT=18748.4

? (TO)
TT=18580.3

? (TO)
TT=2519.83

StripPacking/
StripPacking-m1-seriesT/
StripPacking-T1e.xml
SAT
TT=123.921

? (TO)
TT=2520.04

? (NS)
TT=0.015

? (TO)
TT=2520.44

? (TO)
TT=20023

? (TO)
TT=2520.58

? (TO)
TT=19985.9

? (TO)
TT=2520.58

? (TO)
TT=19993.9

? (TO)
TT=2520.06

? (TO)
TT=19992.6

? (TO)
TT=2520.13

? (TO)
TT=2520.07

? (TO)
TT=2520.07

? (TO)
TT=2520.12

?
TT=577.384

?
TT=503.484

?
TT=503.842

?
TT=978.012

?
TT=992.886

SAT
TT=137.622

SAT
TT=125.185

SAT
TT=123.921

SAT
TT=135.708

SAT
TT=125.564

SAT
TT=124.184

SAT
TT=300.309

StripPacking/
StripPacking-m1-seriesT/
StripPacking-T5c.xml



? (TO)
TT=2520.11

? (NS)
TT=0.047

? (TO)
TT=2520.53

? (TO)
TT=19866.2

? (TO)
TT=2520.69

? (TO)
TT=19698.1

? (TO)
TT=2520.77

? (TO)
TT=19669.9

? (TO)
TT=2520.1

? (TO)
TT=19680.8

? (TO)
TT=2520.12

? (TO)
TT=2520.1

? (TO)
TT=2520.11

? (TO)
TT=2520.7

? (MO)
TT=513.819

? (MO)
TT=440.004

? (MO)
TT=459.067

?
TT=882.597

?
TT=1002.49

? (TO)
TT=2520.16

? (TO)
TT=18893.7

? (TO)
TT=18808.6

? (TO)
TT=2520.11

? (TO)
TT=18582.8

? (TO)
TT=18855.9

? (TO)
TT=2519.96

Subisomorphism/
Subisomorphism-m1-LV/
Subisomorphism-g07-g20.xml
UNSAT
TT=7.742

UNSAT
TT=8.504

UNSAT
TT=7.742

UNSAT
TT=107.576

UNSAT
TT=230.402

UNSAT
TT=101.991

UNSAT
TT=204.568

UNSAT
TT=106.867

UNSAT
TT=209.161

UNSAT
TT=53.77

UNSAT
TT=199.677

UNSAT
TT=826.497

UNSAT
TT=898.575

UNSAT
TT=1754.48

UNSAT
TT=850.852

UNSAT
TT=113.901

UNSAT
TT=113.368

UNSAT
TT=114.247

UNSAT
TT=829.681

UNSAT
TT=969.423

? (TO)
TT=2520.1

UNSAT
TT=18722.6

? (TO)
TT=19124.2

? (TO)
TT=2520.1

UNSAT
TT=18717.3

UNSAT
TT=18823.2

? (TO)
TT=2519.98

Subisomorphism/
Subisomorphism-m1-SF/
Subisomorphism-A-20.xml
SAT
TT=0.042

SAT
TT=4.52

SAT
TT=0.419

SAT
TT=5.026

SAT
TT=11.236

SAT
TT=4.93

SAT
TT=9.362

SAT
TT=5.111

SAT
TT=9.016

SAT
TT=2.636

SAT
TT=9.836

SAT
TT=12.782

SAT
TT=13.82

SAT
TT=14.279

SAT
TT=14.332

SAT
TT=0.042

SAT
TT=0.042

SAT
TT=0.042

SAT
TT=0.307

SAT
TT=0.329

SAT
TT=178.047

SAT
TT=215.379

SAT
TT=216.322

SAT
TT=189.745

SAT
TT=210.41

SAT
TT=222.233

SAT
TT=3.226

Subisomorphism/
Subisomorphism-m1-SF/
Subisomorphism-B-08.xml
SAT
TT=0.531

SAT
TT=8.571

SAT
TT=8.581

SAT
TT=90.897

SAT
TT=194.13

SAT
TT=71.048

SAT
TT=158.508

SAT
TT=71.563

SAT
TT=164.983

SAT
TT=29.31

SAT
TT=148.403

SAT
TT=33.727

SAT
TT=35.924

SAT
TT=54.023

SAT
TT=39.585

SAT
TT=0.538

SAT
TT=0.531

SAT
TT=0.531

SAT
TT=4.941

SAT
TT=4.522

? (TO)
TT=2528.1

? (TO)
TT=2226.35

SAT
TT=2221.21

? (TO)
TT=2528.11

? (TO)
TT=2154.82

SAT
TT=2269.94

SAT
TT=46.715

Subisomorphism/
Subisomorphism-m1-si2-m4D/
Subisomorphism-si2-m4Dr6-m1296-04.xml
SAT
TT=0.735

SAT
TT=6.415

SAT
TT=16.264

SAT
TT=35.135

SAT
TT=91.954

SAT
TT=34.968

SAT
TT=80.034

SAT
TT=32.807

SAT
TT=79.795

SAT
TT=50.785

SAT
TT=78.908

SAT
TT=33.387

SAT
TT=39.562

SAT
TT=46.037

SAT
TT=33.367

SAT
TT=0.737

SAT
TT=0.735

SAT
TT=0.737

SAT
TT=11.772

SAT
TT=12.733

? (TO)
TT=2520.75

SAT
TT=1814.57

SAT
TT=1820

? (TO)
TT=2522.28

SAT
TT=1837.22

SAT
TT=1830.36

SAT
TT=130.105

Subisomorphism/
Subisomorphism-m1-si2-m4D/
Subisomorphism-si2-m4Dr6-m256-00.xml
SAT
TT=0.036

SAT
TT=4.094

SAT
TT=0.073

SAT
TT=2.472

SAT
TT=4.851

SAT
TT=2.569

SAT
TT=4.663

SAT
TT=2.643

SAT
TT=4.363

SAT
TT=2.109

SAT
TT=4.756

SAT
TT=11.502

SAT
TT=10.255

SAT
TT=11.343

SAT
TT=9.874

SAT
TT=0.037

SAT
TT=0.036

SAT
TT=0.037

SAT
TT=0.134

SAT
TT=0.172

SAT
TT=50.759

SAT
TT=55.682

SAT
TT=56.251

SAT
TT=52.998

SAT
TT=55.91

SAT
TT=55.802

SAT
TT=22.247

Subisomorphism/
Subisomorphism-m1-si2-rand/
Subisomorphism-si2-r001-m400-05.xml
SAT
TT=9.501

SAT
TT=9.501

SAT
TT=9.916

SAT
TT=107.939

SAT
TT=45.928

SAT
TT=54.913

SAT
TT=41.907

SAT
TT=53.836

SAT
TT=48.398

SAT
TT=22.501

SAT
TT=77.921

SAT
TT=25.11

SAT
TT=14.958

SAT
TT=36.396

SAT
TT=25.34

SAT
TT=91.304

SAT
TT=79.393

SAT
TT=79.88

SAT
TT=59.504

SAT
TT=11.578

SAT
TT=480.903

SAT
TT=830.974

SAT
TT=839.038

SAT
TT=284.617

SAT
TT=819.834

SAT
TT=828.776

SAT
TT=140.796

Subisomorphism/
Subisomorphism-m1-si4-bvg/
Subisomorphism-si4-b03m-m400-04.xml
SAT
TT=0.029

SAT
TT=4.409

SAT
TT=0.972

SAT
TT=3.52

SAT
TT=7.241

SAT
TT=3.899

SAT
TT=6.745

SAT
TT=3.494

SAT
TT=7.129

SAT
TT=1.991

SAT
TT=7.043

SAT
TT=8.401

SAT
TT=8.893

SAT
TT=10.116

SAT
TT=7.994

SAT
TT=0.03

SAT
TT=0.03

SAT
TT=0.029

SAT
TT=0.57

SAT
TT=0.562

SAT
TT=135.434

SAT
TT=158.227

SAT
TT=158.977

SAT
TT=138.349

SAT
TT=155.201

SAT
TT=153.84

SAT
TT=1.742

Bench nameBest resultsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
Subisomorphism/
Subisomorphism-m1-si4-m4D/
Subisomorphism-si4-m4D-m625-05.xml
SAT
TT=0.108

SAT
TT=5.521

SAT
TT=2.041

SAT
TT=10.037

SAT
TT=26.678

SAT
TT=9.757

SAT
TT=26.569

SAT
TT=10.268

SAT
TT=26.339

SAT
TT=4.811

SAT
TT=26.605

SAT
TT=20.226

SAT
TT=18.669

SAT
TT=21.416

SAT
TT=19.76

SAT
TT=0.124

SAT
TT=0.108

SAT
TT=0.108

SAT
TT=1.568

SAT
TT=1.702

SAT
TT=893.799

SAT
TT=893.112

SAT
TT=901.893

SAT
TT=863.093

SAT
TT=878.815

SAT
TT=871.497

SAT
TT=72.667

Subisomorphism/
Subisomorphism-m1-si4-m4D/
Subisomorphism-si4-m4Dr4-m1296-05.xml
SAT
TT=1.679

SAT
TT=12.634

SAT
TT=77.177

SAT
TT=79.646

SAT
TT=198.079

SAT
TT=115.536

SAT
TT=195.89

SAT
TT=101.281

SAT
TT=177.969

SAT
TT=159.641

SAT
TT=197.998

SAT
TT=70.966

SAT
TT=72.727

SAT
TT=94.978

SAT
TT=65.001

SAT
TT=1.916

SAT
TT=1.682

SAT
TT=1.679

SAT
TT=13.274

SAT
TT=12.922

? (TO)
TT=2527.92

? (TO)
TT=8556.97

? (TO)
TT=8492.52

? (TO)
TT=2528.17

? (TO)
TT=8495.66

? (TO)
TT=8845

SAT
TT=241.382

Subisomorphism/
Subisomorphism-m1-si4-m4D/
Subisomorphism-si4-m4Dr4-m256-08.xml
SAT
TT=0.019

SAT
TT=3.887

SAT
TT=0.185

SAT
TT=2.882

SAT
TT=5.338

SAT
TT=3.175

SAT
TT=5.393

SAT
TT=3.063

SAT
TT=5.29

SAT
TT=1.793

SAT
TT=4.878

SAT
TT=9.048

SAT
TT=9.845

SAT
TT=9.945

SAT
TT=8.806

SAT
TT=0.021

SAT
TT=0.019

SAT
TT=0.02

SAT
TT=0.202

SAT
TT=0.191

SAT
TT=109.921

SAT
TT=127.723

SAT
TT=129.43

SAT
TT=119.415

SAT
TT=126.471

SAT
TT=126.142

SAT
TT=1.239

Subisomorphism/
Subisomorphism-m1-si4-rand/
Subisomorphism-si4-r001-m600-08.xml
SAT
TT=8.7

SAT
TT=94.91

SAT
TT=83.027

SAT
TT=1106.46

SAT
TT=2065.72

SAT
TT=484.839

SAT
TT=972.851

SAT
TT=506.545

SAT
TT=983.936

SAT
TT=109.912

SAT
TT=908.622

SAT
TT=222.527

SAT
TT=153.268

SAT
TT=151.868

SAT
TT=208.252

SAT
TT=8.813

SAT
TT=8.7

SAT
TT=8.745

SAT
TT=10.377

SAT
TT=18.507

? (TO)
TT=2528.46

SAT
TT=1760.49

SAT
TT=1743.39

? (TO)
TT=2528.03

SAT
TT=1730.73

SAT
TT=1713.93

SAT
TT=102.834

Subisomorphism/
Subisomorphism-m1-si6-bvg/
Subisomorphism-si6-b06-m200-07.xml
SAT
TT=0.03

SAT
TT=4.272

SAT
TT=0.296

SAT
TT=3.848

SAT
TT=8.196

SAT
TT=4.159

SAT
TT=7.978

SAT
TT=4.005

SAT
TT=7.753

SAT
TT=2.01

SAT
TT=7.663

SAT
TT=10.583

SAT
TT=11.496

SAT
TT=12.107

SAT
TT=10.144

SAT
TT=0.031

SAT
TT=0.03

SAT
TT=0.03

SAT
TT=0.153

SAT
TT=0.151

SAT
TT=138.284

SAT
TT=168.598

SAT
TT=166.008

SAT
TT=150.977

SAT
TT=164.553

SAT
TT=163.239

SAT
TT=42.967

Subisomorphism/
Subisomorphism-m1-si6-bvg/
Subisomorphism-si6-b09-m200-08.xml
SAT
TT=0.034

SAT
TT=4.066

SAT
TT=0.435

SAT
TT=4.352

SAT
TT=9.214

SAT
TT=4.957

SAT
TT=10.485

SAT
TT=4.842

SAT
TT=9.335

SAT
TT=2.181

SAT
TT=9.289

SAT
TT=13.168

SAT
TT=14.342

SAT
TT=13.193

SAT
TT=11.949

SAT
TT=0.034

SAT
TT=0.034

SAT
TT=0.034

SAT
TT=0.159

SAT
TT=0.159

SAT
TT=237.897

SAT
TT=283.658

SAT
TT=285.974

SAT
TT=258.289

SAT
TT=280.624

SAT
TT=285.648

SAT
TT=64.78

Subisomorphism/
Subisomorphism-m1-si6-m4D/
Subisomorphism-si6-m4Dr2-m1296-05.xml
SAT
TT=0.887

SAT
TT=18.136

SAT
TT=101.686

SAT
TT=96.007

SAT
TT=310.505

SAT
TT=92.428

SAT
TT=283.135

SAT
TT=95.627

SAT
TT=253.893

SAT
TT=22.805

SAT
TT=247.934

SAT
TT=99.181

SAT
TT=99.384

SAT
TT=105.633

SAT
TT=98.207

SAT
TT=1.011

SAT
TT=0.887

SAT
TT=0.887

SAT
TT=15.653

SAT
TT=16.562

? (TO)
TT=2528.11

? (TO)
TT=8071.77

? (TO)
TT=9080.33

? (TO)
TT=2528.25

? (TO)
TT=8280.01

? (TO)
TT=9160.59

SAT
TT=41.998

Subisomorphism/
Subisomorphism-m1-si6-m4D/
Subisomorphism-si6-m4Dr4-m1296-04.xml
SAT
TT=0.309

SAT
TT=10.676

SAT
TT=95.782

SAT
TT=62.766

SAT
TT=256.19

SAT
TT=56.534

SAT
TT=233.943

SAT
TT=64.295

SAT
TT=235.936

SAT
TT=14.675

SAT
TT=195.107

SAT
TT=100.188

SAT
TT=89.191

SAT
TT=103.439

SAT
TT=94.267

SAT
TT=0.355

SAT
TT=0.312

SAT
TT=0.309

SAT
TT=9.012

SAT
TT=9.431

? (TO)
TT=2527.85

? (TO)
TT=8954.97

? (TO)
TT=8104.98

? (TO)
TT=2528.15

? (TO)
TT=9339.04

? (TO)
TT=9137.18

SAT
TT=45.382

Sudoku/Sudoku-table-s1/
Sudoku-s10c-table.xml
SAT
TT=3.545

SAT
TT=8.375

SAT
TT=3.545

SAT
TT=49.691

SAT
TT=149.047

SAT
TT=46.134

SAT
TT=143.376

SAT
TT=44.439

SAT
TT=160.686

SAT
TT=10.763

SAT
TT=143.324

SAT
TT=15.944

SAT
TT=16.291

SAT
TT=16.234

SAT
TT=15.389

SAT
TT=9.404

SAT
TT=9.361

SAT
TT=9.366

SAT
TT=21.261

SAT
TT=21.248

SAT
TT=45.399

?
TT=48.092

?
TT=47.545

SAT
TT=45.417

?
TT=47.625

?
TT=49.009

SAT
TT=842.471

SuperSolutions/
SuperSolutions-Taillard-js15/
SuperTaillard-js-15-02.xml
SAT
TT=422.969

? (TO)
TT=2520.03

? (TO)
TT=2519.57

?
TT=1560.73

?
TT=9599.85

?
TT=1694.16

?
TT=10043.6

? (TO)
TT=2528.92

?
TT=10018.3

?
TT=2208.54

?
TT=10628.5

? (TO)
TT=2520.18

? (TO)
TT=2520.04

? (TO)
TT=2520.07

? (TO)
TT=2520.11

? (TO)
TT=2519.83

? (TO)
TT=2519.91

? (TO)
TT=2519.92

? (TO)
TT=20076

? (TO)
TT=20077.1

SAT
TT=603.144

SAT
TT=649.004

SAT
TT=671.027

SAT
TT=736.705

SAT
TT=678.33

SAT
TT=665.715

SAT
TT=422.969

Bench nameBest resultsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
SuperSolutions/
SuperSolutions-Taillard-js15/
SuperTaillard-js-15-08.xml
UNSAT
TT=257.025

? (TO)
TT=2520.12

? (TO)
TT=2519.94

?
TT=1981.96

?
TT=7912.82

?
TT=1838.76

?
TT=8443.01

?
TT=2042.99

?
TT=7216.81

?
TT=2062.67

?
TT=15995.1

? (TO)
TT=2520.15

? (TO)
TT=2520.05

? (TO)
TT=2520.15

? (TO)
TT=2520.07

? (TO)
TT=2519.99

? (TO)
TT=2519.8

? (TO)
TT=2520.04

? (TO)
TT=20078.4

? (TO)
TT=20076.6

UNSAT
TT=742.904

UNSAT
TT=1831.32

UNSAT
TT=1759.63

UNSAT
TT=646.889

UNSAT
TT=1817.77

UNSAT
TT=1768.12

UNSAT
TT=257.025

SuperSolutions/
SuperSolutions-Taillard-js15/
SuperTaillard-js-15-13.xml
SAT
TT=110.493

? (TO)
TT=2520.02

? (TO)
TT=2519.9

?
TT=2095.43

?
TT=10220.7

?
TT=1911.71

?
TT=9460.34

?
TT=2139.09

?
TT=13356.2

?
TT=2255.58

? (TO)
TT=16700

? (TO)
TT=2520.12

? (TO)
TT=2520.06

? (TO)
TT=2520.16

? (TO)
TT=2520.13

? (TO)
TT=2520

? (TO)
TT=2519.96

? (TO)
TT=2520.1

? (TO)
TT=20077.1

? (TO)
TT=20076.5

SAT
TT=220.534

SAT
TT=145.343

SAT
TT=140.947

SAT
TT=123.069

SAT
TT=140.715

SAT
TT=141.053

SAT
TT=110.493

SuperSolutions/
SuperSolutions-Taillard-js15/
SuperTaillard-js-15-16.xml
SAT
TT=51.895

? (TO)
TT=2520.1

? (TO)
TT=2519.8

?
TT=1850.24

?
TT=7466.55

?
TT=1879.5

?
TT=12396.4

? (TO)
TT=2528.52

? (TO)
TT=16549.2

? (TO)
TT=2528.3

?
TT=13604.4

? (TO)
TT=2520.14

? (TO)
TT=2520.07

? (TO)
TT=2520.06

? (TO)
TT=2520.08

? (TO)
TT=2520.11

? (TO)
TT=2519.78

? (TO)
TT=2519.9

? (TO)
TT=20077.1

? (TO)
TT=20077.1

SAT
TT=383.77

SAT
TT=283.525

SAT
TT=274.9

SAT
TT=51.895

SAT
TT=278.277

SAT
TT=278.184

SAT
TT=108.812

SuperSolutions/
SuperSolutions-Taillard-js15/
SuperTaillard-js-15-18.xml
SAT
TT=52.371

? (TO)
TT=2520.11

? (TO)
TT=2520.08

?
TT=1812.2

?
TT=9192.85

?
TT=2173.71

?
TT=10448.3

? (TO)
TT=2528.21

?
TT=10646.7

? (TO)
TT=2527.21

?
TT=8336.73

? (TO)
TT=2520.16

? (TO)
TT=2520.12

? (TO)
TT=2520.17

? (TO)
TT=2520.12

? (TO)
TT=2519.99

? (TO)
TT=2520

? (TO)
TT=2519.72

? (TO)
TT=20078.2

? (TO)
TT=20078.4

SAT
TT=244.743

SAT
TT=422.197

SAT
TT=412.93

SAT
TT=52.371

SAT
TT=411.775

SAT
TT=412.767

SAT
TT=111.828

SuperSolutions/
SuperSolutions-Taillard-js15/
SuperTaillard-js-15-26.xml
UNSAT
TT=442.279

? (TO)
TT=2520.09

? (TO)
TT=2519.86

? (TO)
TT=2527.27

?
TT=8916.37

?
TT=1744.11

?
TT=9164.47

?
TT=2409.53

?
TT=7224.72

?
TT=2207.31

?
TT=7100.55

? (TO)
TT=2520.1

? (TO)
TT=2520.09

? (TO)
TT=2520.13

? (TO)
TT=2520.13

? (TO)
TT=2519.63

? (TO)
TT=2520.02

? (TO)
TT=2520.02

? (TO)
TT=20076.6

? (TO)
TT=20076.5

UNSAT
TT=541.839

UNSAT
TT=1443.79

UNSAT
TT=1427.06

UNSAT
TT=568.198

UNSAT
TT=1435.6

UNSAT
TT=1454.16

UNSAT
TT=442.279

SuperSolutions/
SuperSolutions-Taillard-js20/
SuperTaillard-js-20-10.xml



? (TO)
TT=2520.03

? (TO)
TT=2520.05

? (TO)
TT=2520.43

? (TO)
TT=20038.5

? (TO)
TT=2520.54

? (TO)
TT=20044.1

? (TO)
TT=2520.37

? (TO)
TT=20040.8

? (TO)
TT=2520.04

? (TO)
TT=20043.7

? (TO)
TT=2520.21

? (TO)
TT=2520.17

? (TO)
TT=2520.25

? (TO)
TT=2520.08

? (TO)
TT=2520

? (TO)
TT=2519.97

? (TO)
TT=2519.75

? (TO)
TT=20012.2

? (TO)
TT=20012.9

? (TO)
TT=2520.21

? (TO)
TT=9252.37

? (TO)
TT=9284.56

? (TO)
TT=2520.25

? (TO)
TT=9251.12

? (TO)
TT=9452.62

? (TO)
TT=2520.08

SuperSolutions/
SuperSolutions-Taillard-js20/
SuperTaillard-js-20-20.xml
SAT
TT=1437.98

? (TO)
TT=2520.11

? (TO)
TT=2520.01

? (TO)
TT=2520.68

? (TO)
TT=20043.4

? (TO)
TT=2520.63

? (TO)
TT=20047

? (TO)
TT=2520.59

? (TO)
TT=20044.4

? (TO)
TT=2520.03

? (TO)
TT=20043.9

? (TO)
TT=2520.15

? (TO)
TT=2520.16

? (TO)
TT=2520.33

? (TO)
TT=2520.08

? (TO)
TT=2519.98

? (TO)
TT=2519.98

? (TO)
TT=2519.87

? (TO)
TT=20010.8

? (TO)
TT=20011.4

? (TO)
TT=2520.2

SAT
TT=1548.03

SAT
TT=1437.98

SAT
TT=2323.69

SAT
TT=1480.51

SAT
TT=1462.6

SAT
TT=1466.07

SuperSolutions/
SuperSolutions-Taillard-os10/
SuperTaillard-os-10-01.xml
SAT
TT=41.282

? (TO)
TT=2520.04

? (TO)
TT=2520.04

?
TT=2316.02

?
TT=6858.43

? (TO)
TT=2528.07

?
TT=6448.46

?
TT=1824.67

?
TT=6383.46

?
TT=1809.05

?
TT=8890.53

? (TO)
TT=2520.08

? (TO)
TT=2520.16

? (TO)
TT=2520.1

? (TO)
TT=2520.19

? (TO)
TT=2519.91

? (TO)
TT=2519.92

? (TO)
TT=2519.81

? (TO)
TT=20101.4

? (TO)
TT=20100.7

SAT
TT=49.635

SAT
TT=92.907

SAT
TT=92.396

SAT
TT=62.991

SAT
TT=92.888

SAT
TT=88.68

SAT
TT=41.282

SuperSolutions/
SuperSolutions-Taillard-os10/
SuperTaillard-os-10-06.xml
UNSAT
TT=56.184

? (TO)
TT=2520.11

? (TO)
TT=2520.01

?
TT=2322.91

?
TT=8835.73

? (TO)
TT=2528.04

? (TO)
TT=15821.9

?
TT=2332.83

? (TO)
TT=15611.4

?
TT=2326.47

?
TT=7118.76

? (TO)
TT=2520.13

? (TO)
TT=2520.11

? (TO)
TT=2520.11

? (TO)
TT=2520.22

? (TO)
TT=2520.04

? (TO)
TT=2519.98

? (TO)
TT=2519.96

? (TO)
TT=20099.2

? (TO)
TT=20098

UNSAT
TT=61.277

UNSAT
TT=198.465

UNSAT
TT=193.893

UNSAT
TT=56.184

UNSAT
TT=196.617

UNSAT
TT=195.251

UNSAT
TT=148.279

SuperSolutions/
SuperSolutions-Taillard-os10/
SuperTaillard-os-10-12.xml
SAT
TT=24.916

? (TO)
TT=2520.11

? (TO)
TT=2519.89

? (TO)
TT=2527.21

?
TT=6276.79

? (TO)
TT=2527.44

?
TT=9257.49

? (TO)
TT=2527.31

?
TT=6612.47

?
TT=1776.96

?
TT=6595.67

SAT
TT=590.165

SAT
TT=1350.86

SAT
TT=1230.51

SAT
TT=556.693

SAT
TT=687.083

SAT
TT=664.688

SAT
TT=673.068

SAT
TT=7131.38

SAT
TT=7269.24

SAT
TT=24.916

SAT
TT=43.75

SAT
TT=44.406

SAT
TT=31.449

SAT
TT=41.981

SAT
TT=45.481

SAT
TT=37.53

Bench nameBest resultsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
SuperSolutions/
SuperSolutions-Taillard-os10/
SuperTaillard-os-10-21.xml
UNSAT
TT=45.669

? (TO)
TT=2520.03

? (TO)
TT=2519.94

?
TT=2386.17

? (TO)
TT=15715.7

?
TT=2394.04

?
TT=5291.9

? (TO)
TT=2528.53

?
TT=8041.02

? (TO)
TT=2528.27

?
TT=5714.38

? (TO)
TT=2520.14

? (TO)
TT=2520.16

? (TO)
TT=2520.14

? (TO)
TT=2520.18

? (TO)
TT=2520.01

? (TO)
TT=2520.1

? (TO)
TT=2520.01

? (TO)
TT=20098

? (TO)
TT=20101.3

UNSAT
TT=45.669

UNSAT
TT=152.74

UNSAT
TT=157.049

UNSAT
TT=140.386

UNSAT
TT=154.787

UNSAT
TT=155.935

UNSAT
TT=179.387

SuperSolutions/
SuperSolutions-Taillard-os10/
SuperTaillard-os-10-23.xml
UNSAT
TT=40.887

? (TO)
TT=2520.02

? (TO)
TT=2520

?
TT=1768.22

?
TT=6229.02

?
TT=2377.77

?
TT=5888.84

?
TT=1851.74

?
TT=6928.95

? (TO)
TT=2528.15

?
TT=7616.9

? (TO)
TT=2520.12

? (TO)
TT=2520.07

? (TO)
TT=2520.1

? (TO)
TT=2520.32

? (TO)
TT=2519.5

? (TO)
TT=2520

? (TO)
TT=2520.04

? (TO)
TT=20099.5

? (TO)
TT=20099.4

UNSAT
TT=40.887

UNSAT
TT=61.703

UNSAT
TT=59.367

UNSAT
TT=183.255

UNSAT
TT=60.433

UNSAT
TT=63.068

UNSAT
TT=107.715

SuperSolutions/
SuperSolutions-Taillard-os10/
SuperTaillard-os-10-26.xml
UNSAT
TT=15.591

? (TO)
TT=2520.03

? (TO)
TT=2520.01

? (TO)
TT=2527.48

? (TO)
TT=15946.6

?
TT=2435.67

?
TT=14551.1

?
TT=2396.4

?
TT=13530.4

?
TT=2433.9

?
TT=11218.3

? (TO)
TT=2520.14

? (TO)
TT=2520.17

? (TO)
TT=2520.11

? (TO)
TT=2520.23

? (TO)
TT=2519.7

? (TO)
TT=2519.63

? (TO)
TT=2519.77

? (TO)
TT=20102.6

? (TO)
TT=20099.7

UNSAT
TT=21.202

UNSAT
TT=38.541

UNSAT
TT=41.473

UNSAT
TT=27.551

UNSAT
TT=38.492

UNSAT
TT=38.599

UNSAT
TT=15.591

SuperSolutions/
SuperSolutions-Taillard-os20/
SuperTaillard-os-20-02.xml
SAT
TT=265.664

? (TO)
TT=2520.1

? (TO)
TT=2520.09

? (TO)
TT=2528.25

?
TT=7058.64

?
TT=1709.81

? (TO)
TT=16426

? (TO)
TT=2528.16

?
TT=7127.5

? (TO)
TT=2527.56

?
TT=6940.68

? (TO)
TT=2520.17

? (TO)
TT=2520.13

? (TO)
TT=2520.3

? (TO)
TT=2520.1

? (TO)
TT=2519.95

? (TO)
TT=2519.79

? (TO)
TT=2520.06

? (TO)
TT=19916.6

? (TO)
TT=19916

SAT
TT=466.377

SAT
TT=319.974

SAT
TT=275.889

SAT
TT=266.492

SAT
TT=266.326

SAT
TT=265.664

SAT
TT=1117.21

TravellingSalesman/
TravellingSalesman-xcsp2-s20a4/
TravellingSalesman-4-20-001-a4_X2.xml
SAT
TT=0.83

SAT
TT=6.369

SAT
TT=1.023

SAT
TT=10.41

SAT
TT=23.273

SAT
TT=9.938

SAT
TT=22.466

SAT
TT=10.348

SAT
TT=21.52

SAT
TT=3.92

SAT
TT=23.189

SAT
TT=10.469

SAT
TT=10.696

SAT
TT=11.704

SAT
TT=11.534

SAT
TT=0.833

SAT
TT=0.941

SAT
TT=0.83

SAT
TT=3.603

SAT
TT=6.58

SAT
TT=134.897

SAT
TT=142.118

SAT
TT=141.752

SAT
TT=138.026

SAT
TT=132.589

SAT
TT=134.825

SAT
TT=19.12

TravellingSalesman/
TravellingSalesman-xcsp2-s25/
TravellingSalesman-25-715_X2.xml
SAT
TT=3.291

SAT
TT=125.075

SAT
TT=168.046

SAT
TT=91.465

SAT
TT=183.468

SAT
TT=429.555

SAT
TT=689.312

SAT
TT=415.85

SAT
TT=665.982

SAT
TT=277.982

SAT
TT=710.571

SAT
TT=17.662

SAT
TT=41.132

SAT
TT=247.655

SAT
TT=37.963

SAT
TT=3.388

SAT
TT=3.291

SAT
TT=3.301

SAT
TT=271.515

SAT
TT=265.281

SAT
TT=237.921

SAT
TT=279.368

SAT
TT=292.884

SAT
TT=286.426

SAT
TT=280.706

SAT
TT=280.008

SAT
TT=163.391

XCSP19/ConsecutiveSquarePacking/
ConsecutiveSquarePacking-sp-15-035.xml
UNSAT
TT=4.886

? (TO)
TT=2520.1

UNSAT
TT=1523.36

? (TO)
TT=2528.15

?
TT=7652.53

?
TT=2333.38

? (TO)
TT=16313.5

? (TO)
TT=2527.52

?
TT=6134.86

? (TO)
TT=2520.16

?
TT=11576.1

? (TO)
TT=2520.05

? (TO)
TT=2520.06

? (TO)
TT=2520.14

? (TO)
TT=2520.06

? (TO)
TT=2520.04

? (TO)
TT=2519.94

? (TO)
TT=2520.08

? (TO)
TT=20121.3

? (TO)
TT=20123.6

UNSAT
TT=4.947

UNSAT
TT=5.168

UNSAT
TT=5.069

UNSAT
TT=4.886

UNSAT
TT=5.185

UNSAT
TT=5.209

UNSAT
TT=5.784

XCSP19/ConsecutiveSquarePacking/
ConsecutiveSquarePacking-sp-15-036.xml
SAT
TT=1.609

SAT
TT=13.073

SAT
TT=1.609

?
TT=1466.89

? (TO)
TT=16324.5

?
TT=2141.26

?
TT=6939.04

?
TT=1833.78

? (TO)
TT=16156.8

SAT
TT=147.224

?
TT=6850.57

SAT
TT=34.362

SAT
TT=10.703

SAT
TT=756.921

SAT
TT=10.992

SAT
TT=2.895

SAT
TT=2.864

SAT
TT=2.892

SAT
TT=10.407

SAT
TT=9.395

SAT
TT=5.114

SAT
TT=5.336

SAT
TT=5.569

SAT
TT=5.168

SAT
TT=5.397

SAT
TT=5.449

SAT
TT=6.349

XCSP19/ConsecutiveSquarePacking/
ConsecutiveSquarePacking-sp-17-042.xml
UNSAT
TT=9.939

? (TO)
TT=2520.02

? (TO)
TT=2519.93

? (TO)
TT=2528.14

?
TT=5795.11

? (TO)
TT=2527.7

?
TT=5703.45

? (TO)
TT=2528.14

?
TT=6549.44

? (TO)
TT=2520.13

? (TO)
TT=16332.6

? (TO)
TT=2520.1

? (TO)
TT=2520.15

? (TO)
TT=2520.07

? (TO)
TT=2520.08

? (TO)
TT=2519.75

? (TO)
TT=2520.05

? (TO)
TT=2519.73

? (TO)
TT=20121.9

? (TO)
TT=20124.3

UNSAT
TT=25.182

UNSAT
TT=37.097

UNSAT
TT=36.758

UNSAT
TT=9.939

UNSAT
TT=36.453

UNSAT
TT=36.991

UNSAT
TT=22.981

XCSP19/ConsecutiveSquarePacking/
ConsecutiveSquarePacking-sp-18-046.xml
UNSAT
TT=186.128

? (TO)
TT=2520.11

? (TO)
TT=2519.61

? (TO)
TT=2527.52

?
TT=11324.9

?
TT=2194.12

?
TT=9452.29

? (TO)
TT=2528.23

?
TT=6961.87

? (TO)
TT=2520.08

? (TO)
TT=15782.6

? (TO)
TT=2520.12

? (TO)
TT=2520.07

? (TO)
TT=2520.07

? (TO)
TT=2520.09

? (TO)
TT=2519.74

? (TO)
TT=2519.84

? (TO)
TT=2519.98

? (TO)
TT=20120.4

? (TO)
TT=20119.5

UNSAT
TT=186.128

UNSAT
TT=1232.03

UNSAT
TT=1241.61

UNSAT
TT=221.486

UNSAT
TT=1229.39

UNSAT
TT=1242.22

UNSAT
TT=206.274

Bench nameBest resultsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
XCSP19/ConsecutiveSquarePacking/
ConsecutiveSquarePacking-sp-18-047.xml
SAT
TT=4.713

SAT
TT=239.324

SAT
TT=4.713

? (TO)
TT=2528.16

?
TT=7356.31

? (TO)
TT=2527.62

? (TO)
TT=15856.2

? (TO)
TT=2527.65

?
TT=6599.04

SAT
TT=487.015

?
TT=8562.61

SAT
TT=984.122

SAT
TT=305.39

SAT
TT=343.101

? (TO)
TT=2520.07

? (TO)
TT=2519.49

? (TO)
TT=2519.92

? (TO)
TT=2520.06

SAT
TT=6921.07

SAT
TT=6818.8

SAT
TT=5.845

SAT
TT=6.186

SAT
TT=6.284

SAT
TT=5.791

SAT
TT=6.268

SAT
TT=6.342

SAT
TT=17.755

XCSP19/ConsecutiveSquarePacking/
ConsecutiveSquarePacking-sp-20-054.xml
SAT
TT=6.503

? (TO)
TT=2520.07

? (TO)
TT=2519.91

? (TO)
TT=2520.36

SAT
TT=13549.1

? (TO)
TT=2520.69

? (TO)
TT=18700.1

? (TO)
TT=2520.4

? (TO)
TT=18658.7

? (TO)
TT=2520.08

SAT
TT=2097.83

? (TO)
TT=2520.04

? (TO)
TT=2520.08

? (TO)
TT=2520.09

? (TO)
TT=2520.1

? (TO)
TT=2520.01

? (TO)
TT=2520.02

? (TO)
TT=2519.95

? (TO)
TT=20121.6

? (TO)
TT=20120.2

SAT
TT=6.503

SAT
TT=7.82

SAT
TT=7.673

SAT
TT=6.524

SAT
TT=7.67

SAT
TT=7.82

SAT
TT=27.865

XCSP19/ConsecutiveSquarePacking/
ConsecutiveSquarePacking-sp-24-071.xml
SAT
TT=8.031

? (TO)
TT=2520.08

? (TO)
TT=2520.07

SAT
TT=1370.73

SAT
TT=2886.59

? (TO)
TT=2520.32

SAT
TT=12483.7

? (TO)
TT=2520.39

SAT
TT=13070.1

? (TO)
TT=2520.05

SAT
TT=12022.6

? (TO)
TT=2520.09

? (TO)
TT=2520.14

? (TO)
TT=2520.14

? (TO)
TT=2520.07

? (TO)
TT=2520.04

? (TO)
TT=2520.06

? (TO)
TT=2520.05

? (TO)
TT=20119.7

? (TO)
TT=20120.1

SAT
TT=8.031

SAT
TT=9.609

SAT
TT=8.795

SAT
TT=8.874

SAT
TT=9.494

SAT
TT=8.866

SAT
TT=30.429

XCSP19/ConsecutiveSquarePacking/
ConsecutiveSquarePacking-sp-26-080.xml
SAT
TT=7.463

? (TO)
TT=2520.11

? (TO)
TT=2519.8

SAT
TT=188.926

SAT
TT=395.692

SAT
TT=478.737

SAT
TT=1023.97

SAT
TT=481.68

SAT
TT=979.992

? (TO)
TT=2520.09

SAT
TT=972.873

? (TO)
TT=2520.06

? (TO)
TT=2520.13

? (TO)
TT=2520.06

? (TO)
TT=2520.06

? (TO)
TT=2519.81

? (TO)
TT=2520.08

? (TO)
TT=2520.04

? (TO)
TT=20118.6

? (TO)
TT=20119.4

SAT
TT=7.669

SAT
TT=8.51

SAT
TT=8.457

SAT
TT=7.463

SAT
TT=8.699

SAT
TT=8.887

SAT
TT=90.018

XCSP19/ConsecutiveSquarePacking/
ConsecutiveSquarePacking-sp-27-083.xml



? (TO)
TT=2520.07

? (TO)
TT=2519.73

? (TO)
TT=2520.29

? (TO)
TT=20059.1

? (TO)
TT=2520.61

? (TO)
TT=20047.8

? (TO)
TT=2520.45

? (TO)
TT=20048.8

? (TO)
TT=2520.1

? (TO)
TT=20048

? (TO)
TT=2520.07

? (TO)
TT=2520.15

? (TO)
TT=2520.09

? (TO)
TT=2520.09

? (TO)
TT=2519.85

? (TO)
TT=2520.03

? (TO)
TT=2520.05

? (TO)
TT=20118.9

? (TO)
TT=20120.2

? (TO)
TT=2520.14

? (TO)
TT=19412.5

? (TO)
TT=19157.1

? (TO)
TT=2520.12

? (TO)
TT=19526

? (TO)
TT=19317.8

? (TO)
TT=2519.91

XCSP19/ConsecutiveSquarePacking/
ConsecutiveSquarePacking-sp-28-088.xml



? (TO)
TT=2520.03

? (TO)
TT=2519.71

? (TO)
TT=2520.64

? (TO)
TT=20059.4

? (TO)
TT=2520.59

? (TO)
TT=20048.8

? (TO)
TT=2520.73

? (TO)
TT=20047

? (TO)
TT=2520.12

? (TO)
TT=20046.8

? (TO)
TT=2520.07

? (TO)
TT=2520.08

? (TO)
TT=2520.12

? (TO)
TT=2520.13

? (TO)
TT=2519.92

? (TO)
TT=2520.06

? (TO)
TT=2520.1

? (TO)
TT=20122.5

? (TO)
TT=20118.9

? (TO)
TT=2520.06

? (TO)
TT=19436.9

? (TO)
TT=19475.7

? (TO)
TT=2520.12

? (TO)
TT=19516.7

? (TO)
TT=19526.6

? (TO)
TT=2519.83

XCSP19/ConsecutiveSquarePacking/
ConsecutiveSquarePacking-sp-29-092.xml



? (TO)
TT=2520.16

? (TO)
TT=2519.97

? (TO)
TT=2520.35

? (TO)
TT=20064.5

? (TO)
TT=2520.31

? (TO)
TT=20055.7

? (TO)
TT=2520.65

? (TO)
TT=20054.2

? (TO)
TT=2520.05

? (TO)
TT=20051.2

? (TO)
TT=2520.13

? (TO)
TT=2520.08

? (TO)
TT=2520.11

? (TO)
TT=2520.12

? (TO)
TT=2520.02

? (TO)
TT=2520.1

? (TO)
TT=2520.11

? (TO)
TT=20120.4

? (TO)
TT=20117.7

? (TO)
TT=2520.14

? (TO)
TT=19114.9

? (TO)
TT=19254.5

? (TO)
TT=2520.06

? (TO)
TT=18855.4

? (TO)
TT=19336.2

? (TO)
TT=2520.08

XCSP19/ConsecutiveSquarePacking/
ConsecutiveSquarePacking-sp-29-093.xml



? (TO)
TT=2520.08

? (TO)
TT=2519.95

? (TO)
TT=2520.69

? (TO)
TT=20060.1

? (TO)
TT=2520.43

? (TO)
TT=20047.7

? (TO)
TT=2520.66

? (TO)
TT=20051.4

? (TO)
TT=2520.05

? (TO)
TT=20048.3

? (TO)
TT=2520.14

? (TO)
TT=2520.09

? (TO)
TT=2520.08

? (TO)
TT=2520.12

? (TO)
TT=2519.23

? (TO)
TT=2520.03

? (TO)
TT=2520.02

? (TO)
TT=20118.3

? (TO)
TT=20121.5

? (TO)
TT=2520.05

? (TO)
TT=19474.1

? (TO)
TT=19461.8

? (TO)
TT=2520.06

? (TO)
TT=19317.8

? (TO)
TT=19419

? (TO)
TT=2520

XCSP19/ConsecutiveSquarePacking/
ConsecutiveSquarePacking-sp-30-098.xml
SAT
TT=153.956

? (TO)
TT=2520.11

? (TO)
TT=2519.82

? (TO)
TT=2520.69

? (TO)
TT=20058

? (TO)
TT=2520.48

? (TO)
TT=20041.8

? (TO)
TT=2520.73

? (TO)
TT=20048.2

? (TO)
TT=2520.09

? (TO)
TT=20041.3

? (TO)
TT=2520.12

? (TO)
TT=2520.12

? (TO)
TT=2520.06

? (TO)
TT=2520.07

? (TO)
TT=2520.03

? (TO)
TT=2519.81

? (TO)
TT=2519.82

? (TO)
TT=20116.2

? (TO)
TT=20119.2

SAT
TT=153.956

SAT
TT=2314.79

SAT
TT=2196.64

SAT
TT=970.752

SAT
TT=2203.92

SAT
TT=2201.61

SAT
TT=373.511

XCSP19/ConsecutiveSquarePacking/
ConsecutiveSquarePacking-sp-35-123.xml



? (TO)
TT=2520.05

? (TO)
TT=2520.08

? (TO)
TT=2520.37

? (TO)
TT=20064.5

? (TO)
TT=2520.63

? (TO)
TT=20037.4

? (TO)
TT=2520.41

? (TO)
TT=20071.6

? (TO)
TT=2520.05

? (TO)
TT=20065.1

? (TO)
TT=2520.14

? (TO)
TT=2520.1

? (TO)
TT=2520.09

? (TO)
TT=2520.1

? (TO)
TT=2519.96

? (TO)
TT=2520.01

? (TO)
TT=2520.02

? (TO)
TT=20116.1

? (TO)
TT=20115.9

? (TO)
TT=2520.05

? (TO)
TT=19408.8

? (TO)
TT=19435.3

? (TO)
TT=2520.12

? (TO)
TT=19419.4

? (TO)
TT=19183.4

? (TO)
TT=2519.69

Bench nameBest resultsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
XCSP19/
hcp/
graph162.xml
SAT
TT=3.996

? (exit code)
TT=4.521

? (exit code)
TT=0.167

SAT
TT=10.763

SAT
TT=20.17

SAT
TT=10.46

SAT
TT=20.568

SAT
TT=10.364

SAT
TT=21.134

SAT
TT=3.996

SAT
TT=19.655

? (TO)
TT=2520.15

Wrong Cert.
TT=11.488

?
TT=17.636

? (TO)
TT=2520.34

Signal
TT=0.531

? (NS)
TT=0.256

? (NS)
TT=0.257

? (NS)
TT=0.564

? (NS)
TT=0.567

? (NS)
TT=3.145

? (NS)
TT=3.028

? (NS)
TT=3.255

? (NS)
TT=3.111

? (NS)
TT=3.165

? (NS)
TT=3.198

SAT
TT=119.25

XCSP19/
hcp/
graph171.xml
SAT
TT=72.929

? (exit code)
TT=3.663

? (exit code)
TT=0.016

SAT
TT=74.45

SAT
TT=149.391

SAT
TT=112.257

SAT
TT=279.798

SAT
TT=114.772

SAT
TT=243.512

SAT
TT=148.855

SAT
TT=264.782

? (TO)
TT=2520.08

? (TO)
TT=2520.12

? (TO)
TT=2520.09

? (TO)
TT=2520.08

Signal
TT=0.03

? (NS)
TT=0.024

? (NS)
TT=0.024

? (NS)
TT=0.031

? (NS)
TT=0.033

? (NS)
TT=2.009

? (NS)
TT=1.961

? (NS)
TT=2.009

? (NS)
TT=2.021

? (NS)
TT=1.984

? (NS)
TT=2.024

SAT
TT=72.929

XCSP19/
hcp/
graph197.xml
SAT
TT=156.565

? (exit code)
TT=3.753

? (exit code)
TT=0.018

SAT
TT=451.178

SAT
TT=1023.83

SAT
TT=420.27

SAT
TT=915.925

SAT
TT=395.842

SAT
TT=929.393

? (TO)
TT=2520.08

SAT
TT=991.067

? (TO)
TT=2520.12

? (TO)
TT=2520.08

? (TO)
TT=2520.08

? (TO)
TT=2520.09

? (TO)
TT=2520.02

? (NS)
TT=0.028

? (NS)
TT=0.027

? (NS)
TT=0.037

? (NS)
TT=0.038

? (NS)
TT=2.083

? (NS)
TT=2.137

? (NS)
TT=2.125

? (NS)
TT=2.107

? (NS)
TT=2.101

? (NS)
TT=2.112

SAT
TT=156.565

XCSP19/
hcp/
graph223.xml
SAT
TT=178.186

? (exit code)
TT=3.896

? (exit code)
TT=0.021

? (TO)
TT=2520.65

? (TO)
TT=19949.2

? (TO)
TT=2520.57

? (TO)
TT=19991.2

? (TO)
TT=2520.46

? (TO)
TT=19994.6

? (TO)
TT=2520.11

? (TO)
TT=20002

? (TO)
TT=2520.08

? (TO)
TT=2520.09

? (TO)
TT=2520.1

? (TO)
TT=2520.05

Signal
TT=0.039

? (NS)
TT=0.032

? (NS)
TT=0.032

? (NS)
TT=0.043

? (NS)
TT=0.044

? (NS)
TT=2.146

? (NS)
TT=2.18

? (NS)
TT=2.161

? (NS)
TT=2.17

? (NS)
TT=2.178

? (NS)
TT=2.19

SAT
TT=178.186

XCSP19/
hcp/
graph237.xml
SAT
TT=228.804

? (exit code)
TT=3.988

? (exit code)
TT=0.021

SAT
TT=560.537

SAT
TT=1169.11

SAT
TT=564.361

SAT
TT=1285.87

SAT
TT=541.077

SAT
TT=1280.19

? (TO)
TT=2520.05

SAT
TT=1124.89

? (TO)
TT=2520.12

? (TO)
TT=2520.07

? (TO)
TT=2520.21

? (TO)
TT=2520.06

Signal
TT=0.045

? (NS)
TT=0.034

? (NS)
TT=0.034

? (NS)
TT=0.046

? (NS)
TT=0.046

? (NS)
TT=2.226

? (NS)
TT=2.203

? (NS)
TT=2.214

? (NS)
TT=2.193

? (NS)
TT=2.217

? (NS)
TT=2.244

SAT
TT=228.804

XCSP19/
hcp/
graph249.xml
SAT
TT=37.714

? (exit code)
TT=3.954

? (exit code)
TT=0.022

SAT
TT=45.801

SAT
TT=119.406

SAT
TT=37.233

SAT
TT=107.077

SAT
TT=37.714

SAT
TT=85.51

SAT
TT=428.361

SAT
TT=90.249

? (TO)
TT=2520.13

? (TO)
TT=2520.15

? (TO)
TT=2520.13

? (TO)
TT=2520.12

? (TO)
TT=2520.05

? (NS)
TT=0.035

? (NS)
TT=0.035

? (NS)
TT=0.047

? (NS)
TT=0.048

? (NS)
TT=2.209

? (NS)
TT=2.278

? (NS)
TT=2.312

? (NS)
TT=2.212

? (NS)
TT=2.236

? (NS)
TT=2.235

SAT
TT=153.399

XCSP19/
hcp/
graph252.xml
SAT
TT=338.702

? (exit code)
TT=3.975

? (exit code)
TT=0.022

SAT
TT=551.816

SAT
TT=1392.95

SAT
TT=888.983

SAT
TT=2132.74

SAT
TT=838.958

SAT
TT=2158.99

? (TO)
TT=2520.07

SAT
TT=2394.62

? (TO)
TT=2520.06

? (TO)
TT=2520.06

? (TO)
TT=2520.09

? (TO)
TT=2520.05

Signal
TT=0.044

? (NS)
TT=0.036

? (NS)
TT=0.036

? (NS)
TT=0.048

? (NS)
TT=0.048

? (NS)
TT=2.222

? (NS)
TT=2.259

? (NS)
TT=2.293

? (NS)
TT=2.255

? (NS)
TT=2.252

? (NS)
TT=2.275

SAT
TT=338.702

XCSP19/
hcp/
graph254.xml
SAT
TT=4.737

? (exit code)
TT=3.974

? (exit code)
TT=0.022

SAT
TT=27.845

SAT
TT=66.173

SAT
TT=18.506

SAT
TT=44.614

SAT
TT=21.088

SAT
TT=40.576

SAT
TT=4.737

SAT
TT=48.184

? (TO)
TT=2520.11

? (TO)
TT=2520.11

? (TO)
TT=2520.19

? (TO)
TT=2520.08

? (TO)
TT=2520.09

? (NS)
TT=0.036

? (NS)
TT=0.036

? (NS)
TT=0.048

? (NS)
TT=0.049

? (NS)
TT=2.202

? (NS)
TT=2.206

? (NS)
TT=2.268

? (NS)
TT=2.253

? (NS)
TT=2.25

? (NS)
TT=2.263

SAT
TT=166.233

XCSP19/
hcp/
graph255.xml
SAT
TT=163.368

? (exit code)
TT=3.993

? (exit code)
TT=0.023

? (TO)
TT=2520.42

? (TO)
TT=19937.1

? (TO)
TT=2520.4

? (TO)
TT=19988.5

? (TO)
TT=2520.44

? (TO)
TT=19998

? (TO)
TT=2520.04

? (TO)
TT=19999.9

? (TO)
TT=2520.08

? (TO)
TT=2520.07

? (TO)
TT=2520.12

? (TO)
TT=2520.08

Wrong UNSAT
TT=0.045

? (NS)
TT=0.037

? (NS)
TT=0.037

? (NS)
TT=0.05

? (NS)
TT=0.05

? (NS)
TT=2.262

? (NS)
TT=2.215

? (NS)
TT=2.284

? (NS)
TT=2.272

? (NS)
TT=2.338

? (NS)
TT=2.377

SAT
TT=163.368

XCSP19/
hcp/
graph48.xml
SAT
TT=1103.6

? (exit code)
TT=3.253

? (exit code)
TT=0.01

? (TO)
TT=2520.36

? (TO)
TT=20060.4

Wrong Cert.
TT=2306.85

Wrong Cert.
TT=4706.9

? (TO)
TT=2520.52

? (TO)
TT=19941.1

? (TO)
TT=2520.02

? (TO)
TT=19940.7

? (TO)
TT=2520.08

? (TO)
TT=2520.12

? (TO)
TT=2520.15

? (TO)
TT=2520.09

? (TO)
TT=2520.01

? (NS)
TT=0.011

? (NS)
TT=0.011

? (NS)
TT=0.015

? (NS)
TT=0.014

? (NS)
TT=1.616

? (NS)
TT=1.598

? (NS)
TT=1.61

? (NS)
TT=1.608

? (NS)
TT=1.591

? (NS)
TT=1.609

SAT
TT=1103.6




StatisticsAbsCon
2019-07-23
(complete)
BTD
19.07.01
(complete)
choco-solver
2019-06-14
(complete)
choco-solver
2019-06-14 parallel
(complete)
choco-solver
2019-09-16
(complete)
choco-solver
2019-09-16 parallel
(complete)
choco-solver
2019-09-20
(complete)
choco-solver
2019-09-20 parallel
(complete)
choco-solver
2019-09-24
(complete)
choco-solver
2019-09-24 parallel
(complete)
Concrete
3.10
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.2
(complete)
Concrete
3.12.3
(complete)
cosoco
2
(complete)
cosoco
2.0
(complete)
cosoco
2.0
(complete)
cosoco
2.0 parallel
(complete)
cosoco
2.O parallel
(complete)
Fun-sCOP
hybrid+CryptoMiniSat (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
hybrid+ManyGlucose (2019-09-22)
(complete)
Fun-sCOP
order+GlueMiniSat (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-06-15)
(complete)
Fun-sCOP
order+ManyGlucose (2019-09-22)
(complete)
PicatSAT
2019-09-12
(complete)
Number of times the solver is able to give the best known answer1681351681880016518514918516501331560126127147151209217219190205211246
Number of times the solver is the best solver from a complete solver point of view
(i.e. best known answer and best TT time)
1749600041151501202415364012111167