Trace number 4397737

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2.0? (MO) 1705.02 1707.15

General information on the benchmark

NameOpd/Opd-sum-small/
Opd-10-038-010.xml
MD5SUM8e449228b7b3cf049a5fccac2aa13fe6
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark2
Best CPU time to get the best result obtained on this benchmark4.37498
Satisfiable
(Un)Satisfiability was proved
Number of variables381
Number of constraints56
Number of domains2
Minimum domain size2
Maximum domain size37
Distribution of domain sizes[{"size":2,"count":380},{"size":37,"count":1}]
Minimum variable degree11
Maximum variable degree46
Distribution of variable degrees[{"degree":11,"count":380},{"degree":46,"count":1}]
Minimum constraint arity38
Maximum constraint arity380
Distribution of constraint arities[{"arity":38,"count":10},{"arity":77,"count":45},{"arity":380,"count":1}]
Number of extensional constraints0
Number of intensional constraints0
Distribution of constraint types[{"type":"lex","count":1},{"type":"sum","count":55}]
Optimization problemYES
Type of objectivemin VAR

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (1073 MiB free)
  memory of node 1: 16384 MiB (13818 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4397737-1568577494/watcher-4397737-1568577494 -o /tmp/evaluation-result-4397737-1568577494/solver-4397737-1568577494 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node104-1568568567-1514 --watchdog 2560 cosoco -lc -model -restarts -stick HOME/instance-4397737-1568577494.xml 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2500 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=2387, runsolver pid=2384
Current StackSize limit: 8192 KiB


[startup+0.100152 s]*
/proc/loadavg: 2.08 2.03 1.98 3/203 2392
/proc/meminfo: memFree=15244216/32770624 swapFree=66912904/67108860
[pid=2387] ppid=2384 vsize=28416 memory=4744 CPUtime=0.1 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) R 2384 2387 1482 0 -1 4202496 1390 0 1 0 10 0 0 0 20 0 1 0 434006544 29097984 1186 33554432000 4194304 6211092 140734410332848 140734410329384 140292094749669 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 7104 1186 845 493 0 409 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 28416 KiB
Current children cumulated memory: 4744 KiB

[startup+0.210077 s]*
/proc/loadavg: 2.08 2.03 1.98 3/203 2392
/proc/meminfo: memFree=15240276/32770624 swapFree=66912904/67108860
[pid=2387] ppid=2384 vsize=29380 memory=5800 CPUtime=0.2 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) R 2384 2387 1482 0 -1 4202496 1644 0 1 0 20 0 0 0 20 0 1 0 434006544 30085120 1450 33554432000 4194304 6211092 140734410332848 140734410329384 140292094749669 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 7345 1450 846 493 0 650 0
Current children cumulated CPU time: 0.2 s
Current children cumulated vsize: 29380 KiB
Current children cumulated memory: 5800 KiB

[startup+0.309811 s]*
/proc/loadavg: 2.08 2.03 1.98 3/203 2392
/proc/meminfo: memFree=15238832/32770624 swapFree=66912904/67108860
[pid=2387] ppid=2384 vsize=30344 memory=6592 CPUtime=0.3 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) R 2384 2387 1482 0 -1 4202496 1872 0 1 0 30 0 0 0 20 0 1 0 434006544 31072256 1648 33554432000 4194304 6211092 140734410332848 140734410311944 140292094749669 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 7586 1648 846 493 0 891 0
Current children cumulated CPU time: 0.3 s
Current children cumulated vsize: 30344 KiB
Current children cumulated memory: 6592 KiB

[startup+0.700238 s]
/proc/loadavg: 2.08 2.03 1.98 3/204 2393
/proc/meminfo: memFree=15232616/32770624 swapFree=66912904/67108860
[pid=2387] ppid=2384 vsize=33984 memory=10160 CPUtime=0.69 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) R 2384 2387 1482 0 -1 4202496 2756 0 1 0 69 0 0 0 20 0 1 0 434006544 34799616 2540 33554432000 4194304 6211092 140734410332848 140734410311944 140292094749669 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 8496 2540 846 493 0 1801 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 33984 KiB
Current children cumulated memory: 10160 KiB

[startup+1.50028 s]
/proc/loadavg: 2.08 2.03 1.98 3/204 2393
/proc/meminfo: memFree=15223316/32770624 swapFree=66912904/67108860
[pid=2387] ppid=2384 vsize=41456 memory=17700 CPUtime=1.49 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) R 2384 2387 1482 0 -1 4202496 4601 0 1 0 149 0 0 0 20 0 1 0 434006544 42450944 4425 33554432000 4194304 6211092 140734410332848 140734410311944 140292094749669 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 10364 4425 846 493 0 3669 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 41456 KiB
Current children cumulated memory: 17700 KiB

[startup+3.10962 s]
/proc/loadavg: 2.16 2.05 1.99 3/201 2393
/proc/meminfo: memFree=15282824/32770624 swapFree=66912924/67108860
[pid=2387] ppid=2384 vsize=56648 memory=32584 CPUtime=3.09 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) R 2384 2387 1482 0 -1 4202496 8355 0 1 0 309 0 0 0 20 0 1 0 434006544 58007552 8146 33554432000 4194304 6211092 140734410332848 140734410311944 140292095673174 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 14162 8146 846 493 0 7467 0
Current children cumulated CPU time: 3.09 s
Current children cumulated vsize: 56648 KiB
Current children cumulated memory: 32584 KiB

[startup+6.30027 s]
/proc/loadavg: 2.16 2.05 1.99 3/201 2393
/proc/meminfo: memFree=15232984/32770624 swapFree=66912924/67108860
[pid=2387] ppid=2384 vsize=85724 memory=61960 CPUtime=6.28 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) R 2384 2387 1482 0 -1 4202496 15755 0 1 0 627 1 0 0 20 0 1 0 434006544 87781376 15490 33554432000 4194304 6211092 140734410332848 140734410311944 140292094749669 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 21431 15490 846 493 0 14736 0
Current children cumulated CPU time: 6.28 s
Current children cumulated vsize: 85724 KiB
Current children cumulated memory: 61960 KiB

[startup+12.7003 s]
/proc/loadavg: 2.14 2.05 1.99 3/201 2393
/proc/meminfo: memFree=15133040/32770624 swapFree=66912924/67108860
[pid=2387] ppid=2384 vsize=145692 memory=120884 CPUtime=12.68 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) R 2384 2387 1482 0 -1 4202496 30556 0 1 0 1265 3 0 0 20 0 1 0 434006544 149188608 30221 33554432000 4194304 6211092 140734410332848 140734410311944 140292094749669 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 36423 30221 846 493 0 29728 0
Current children cumulated CPU time: 12.68 s
Current children cumulated vsize: 145692 KiB
Current children cumulated memory: 120884 KiB

[startup+25.5003 s]
/proc/loadavg: 2.11 2.04 1.99 3/201 2393
/proc/meminfo: memFree=14913728/32770624 swapFree=66912924/67108860
[pid=2387] ppid=2384 vsize=266500 memory=240536 CPUtime=25.48 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) R 2384 2387 1482 0 -1 4202496 60074 0 1 0 2541 7 0 0 20 0 1 0 434006544 272896000 60134 33554432000 4194304 6211092 140734410332848 140734410311944 140292094749669 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 66625 60134 846 493 0 59930 0
Current children cumulated CPU time: 25.48 s
Current children cumulated vsize: 266500 KiB
Current children cumulated memory: 240536 KiB

[startup+51.1094 s]
/proc/loadavg: 2.07 2.04 1.99 3/201 2406
/proc/meminfo: memFree=14477920/32770624 swapFree=66912924/67108860
[pid=2387] ppid=2384 vsize=499240 memory=474864 CPUtime=51.08 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) R 2384 2387 1482 0 -1 4202496 117166 0 1 0 5094 14 0 0 20 0 1 0 434006544 511221760 118716 33554432000 4194304 6211092 140734410332848 140734410311944 140292094749669 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 124810 118716 846 493 0 118115 0
Current children cumulated CPU time: 51.08 s
Current children cumulated vsize: 499240 KiB
Current children cumulated memory: 474864 KiB

[startup+102.3 s]
/proc/loadavg: 2.03 2.04 1.99 3/201 2411
/proc/meminfo: memFree=13617460/32770624 swapFree=66912924/67108860
[pid=2387] ppid=2384 vsize=977148 memory=946780 CPUtime=102.27 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) R 2384 2387 1482 0 -1 4202496 230571 0 1 0 10199 28 0 0 20 0 1 0 434006544 1000599552 236695 33554432000 4194304 6211092 140734410332848 140734410311944 140292094749669 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 244287 236695 846 493 0 237592 0
Current children cumulated CPU time: 102.27 s
Current children cumulated vsize: 977148 KiB
Current children cumulated memory: 946780 KiB

[startup+162.3 s]
/proc/loadavg: 2.57 2.20 2.05 3/202 2488
/proc/meminfo: memFree=28782572/32770624 swapFree=66913108/67108860
[pid=2387] ppid=2384 vsize=1529904 memory=1501404 CPUtime=162.26 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) R 2384 2387 1482 0 -1 4202496 363153 0 1 0 16179 47 0 0 20 0 1 0 434006544 1566621696 375351 33554432000 4194304 6211092 140734410332848 140734410311944 140292094749669 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 382476 375351 846 493 0 375781 0
Current children cumulated CPU time: 162.26 s
Current children cumulated vsize: 1529904 KiB
Current children cumulated memory: 1501404 KiB

[startup+222.3 s]
/proc/loadavg: 2.21 2.16 2.05 3/202 2494
/proc/meminfo: memFree=28229484/32770624 swapFree=66913108/67108860
[pid=2387] ppid=2384 vsize=2095772 memory=2054972 CPUtime=222.24 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) R 2384 2387 1482 0 -1 4202496 495912 0 1 0 22159 65 0 0 20 0 1 0 434006544 2146070528 513743 33554432000 4194304 6211092 140734410332848 140734410311944 140292094749669 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 523943 513743 846 493 0 517248 0
Current children cumulated CPU time: 222.24 s
Current children cumulated vsize: 2095772 KiB
Current children cumulated memory: 2054972 KiB

[startup+282.3 s]
/proc/loadavg: 2.08 2.13 2.05 3/202 2498
/proc/meminfo: memFree=27675208/32770624 swapFree=66913108/67108860
[pid=2387] ppid=2384 vsize=2680304 memory=2608584 CPUtime=282.23 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) R 2384 2387 1482 0 -1 4202496 628691 0 1 0 28140 83 0 0 20 0 1 0 434006544 2744631296 652146 33554432000 4194304 6211092 140734410332848 140734410311944 140292094749669 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 670076 652146 846 493 0 663381 0
Current children cumulated CPU time: 282.23 s
Current children cumulated vsize: 2680304 KiB
Current children cumulated memory: 2608584 KiB

[startup+342.3 s]
/proc/loadavg: 2.03 2.11 2.05 3/199 2502
/proc/meminfo: memFree=27121196/32770624 swapFree=66913108/67108860
[pid=2387] ppid=2384 vsize=3209528 memory=3162024 CPUtime=342.21 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) R 2384 2387 1482 0 -1 4202496 760965 0 1 0 34122 99 0 0 20 0 1 0 434006544 3286556672 790506 33554432000 4194304 6211092 140734410332848 140734410311944 140292094749669 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 802382 790506 846 493 0 795687 0
Current children cumulated CPU time: 342.21 s
Current children cumulated vsize: 3209528 KiB
Current children cumulated memory: 3162024 KiB

[startup+402.3 s]
/proc/loadavg: 2.01 2.09 2.05 3/199 2508
/proc/meminfo: memFree=26568672/32770624 swapFree=66913108/67108860
[pid=2387] ppid=2384 vsize=3737660 memory=3713616 CPUtime=402.2 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) R 2384 2387 1482 0 -1 4202496 893317 0 1 0 40101 119 0 0 20 0 1 0 434006544 3827363840 928404 33554432000 4194304 6211092 140734410332848 140734410311944 140292094749669 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 934415 928404 846 493 0 927720 0
Current children cumulated CPU time: 402.2 s
Current children cumulated vsize: 3737660 KiB
Current children cumulated memory: 3713616 KiB

[startup+462.3 s]
/proc/loadavg: 2.00 2.07 2.05 3/198 2635
/proc/meminfo: memFree=25789820/32770624 swapFree=66913108/67108860
[pid=2387] ppid=2384 vsize=4350144 memory=4267472 CPUtime=462.18 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) R 2384 2387 1482 0 -1 4202496 1025695 0 1 0 46081 137 0 0 20 0 1 0 434006544 4454547456 1066868 33554432000 4194304 6211092 140734410332848 140734410311944 140292094749669 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 1087536 1066868 846 493 0 1080841 0
Current children cumulated CPU time: 462.18 s
Current children cumulated vsize: 4350144 KiB
Current children cumulated memory: 4267472 KiB


################
# More data... #
################

Current children cumulated vsize: 9944988 KiB
Current children cumulated memory: 9797056 KiB

[startup+1122.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/198 2718
/proc/meminfo: memFree=19695084/32770624 swapFree=66913108/67108860
[pid=2387] ppid=2384 vsize=10472328 memory=10349708 CPUtime=1121.99 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) R 2384 2387 1482 0 -1 4202496 2480166 0 1 0 111795 404 0 0 20 0 1 0 434006544 10723663872 2587427 33554432000 4194304 6211092 140734410332848 140734410311944 140292094749669 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 2618082 2587427 846 493 0 2611387 0
Current children cumulated CPU time: 1121.99 s
Current children cumulated vsize: 10472328 KiB
Current children cumulated memory: 10349708 KiB

[startup+1182.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/198 2722
/proc/meminfo: memFree=19141220/32770624 swapFree=66913108/67108860
[pid=2387] ppid=2384 vsize=10999800 memory=10902588 CPUtime=1181.98 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) R 2384 2387 1482 0 -1 4202496 2612365 0 1 0 117763 435 0 0 20 0 1 0 434006544 11263795200 2725647 33554432000 4194304 6211092 140734410332848 140734410311944 140292094749669 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 2749950 2725647 846 493 0 2743255 0
Current children cumulated CPU time: 1181.98 s
Current children cumulated vsize: 10999800 KiB
Current children cumulated memory: 10902588 KiB

[startup+1242.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/198 2726
/proc/meminfo: memFree=18587072/32770624 swapFree=66913108/67108860
[pid=2387] ppid=2384 vsize=11527272 memory=11455256 CPUtime=1241.96 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) R 2384 2387 1482 0 -1 4202496 2744496 0 1 0 123729 467 0 0 20 0 1 0 434006544 11803926528 2863814 33554432000 4194304 6211092 140734410332848 140734410311944 140292094749669 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 2881818 2863814 846 493 0 2875123 0
Current children cumulated CPU time: 1241.96 s
Current children cumulated vsize: 11527272 KiB
Current children cumulated memory: 11455256 KiB

[startup+1302.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/198 2732
/proc/meminfo: memFree=18032960/32770624 swapFree=66913108/67108860
[pid=2387] ppid=2384 vsize=12059340 memory=12007864 CPUtime=1301.94 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) R 2384 2387 1482 0 -1 4202496 2876644 0 1 0 129696 498 0 0 20 0 1 0 434006544 12348764160 3001966 33554432000 4194304 6211092 140734410332848 140734410311944 140292094749669 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 3014835 3001966 846 493 0 3008140 0
Current children cumulated CPU time: 1301.94 s
Current children cumulated vsize: 12059340 KiB
Current children cumulated memory: 12007864 KiB

[startup+1362.3 s]
/proc/loadavg: 2.06 2.03 2.05 3/198 2743
/proc/meminfo: memFree=17480044/32770624 swapFree=66913108/67108860
[pid=2387] ppid=2384 vsize=12586680 memory=12559864 CPUtime=1361.93 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) R 2384 2387 1482 0 -1 4202496 3008631 0 1 0 135665 528 0 0 20 0 1 0 434006544 12888760320 3139966 33554432000 4194304 6211092 140734410332848 140734410311944 140292094749669 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 3146670 3139966 846 493 0 3139975 0
Current children cumulated CPU time: 1361.93 s
Current children cumulated vsize: 12586680 KiB
Current children cumulated memory: 12559864 KiB

[startup+1422.3 s]
/proc/loadavg: 2.02 2.02 2.05 3/198 2747
/proc/meminfo: memFree=16927712/32770624 swapFree=66913108/67108860
[pid=2387] ppid=2384 vsize=13394300 memory=13111568 CPUtime=1421.92 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) R 2384 2387 1482 0 -1 4202496 3141077 0 1 0 141632 560 0 0 20 0 1 0 434006544 13715763200 3277892 33554432000 4194304 6211092 140734410332848 140734410311944 140292094749669 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 3348575 3277892 846 493 0 3341880 0
Current children cumulated CPU time: 1421.92 s
Current children cumulated vsize: 13394300 KiB
Current children cumulated memory: 13111568 KiB

[startup+1482.3 s]
/proc/loadavg: 2.01 2.02 2.05 3/197 2753
/proc/meminfo: memFree=16373600/32770624 swapFree=66913120/67108860
[pid=2387] ppid=2384 vsize=13921376 memory=13665440 CPUtime=1481.9 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) R 2384 2387 1482 0 -1 4202496 3272960 0 1 0 147602 588 0 0 20 0 1 0 434006544 14255489024 3416360 33554432000 4194304 6211092 140734410332848 140734410311944 140292094749669 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 3480344 3416360 846 493 0 3473649 0
Current children cumulated CPU time: 1481.9 s
Current children cumulated vsize: 13921376 KiB
Current children cumulated memory: 13665440 KiB

[startup+1542.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/198 2758
/proc/meminfo: memFree=15815340/32770624 swapFree=66913120/67108860
[pid=2387] ppid=2384 vsize=14448584 memory=14216992 CPUtime=1541.88 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) R 2384 2387 1482 0 -1 4202496 3404877 0 1 0 153569 619 0 0 20 0 1 0 434006544 14795350016 3554248 33554432000 4194304 6211092 140734410332848 140734410311944 140292094749669 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 3612146 3554248 846 493 0 3605451 0
Current children cumulated CPU time: 1541.88 s
Current children cumulated vsize: 14448584 KiB
Current children cumulated memory: 14216992 KiB

[startup+1602.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/198 2762
/proc/meminfo: memFree=15263340/32770624 swapFree=66913120/67108860
[pid=2387] ppid=2384 vsize=14975528 memory=14768552 CPUtime=1601.86 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) R 2384 2387 1482 0 -1 4202496 3536733 0 1 0 159536 650 0 0 20 0 1 0 434006544 15334940672 3692138 33554432000 4194304 6211092 140734410332848 140734410311944 140292094749669 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 3743882 3692138 846 493 0 3737187 0
Current children cumulated CPU time: 1601.86 s
Current children cumulated vsize: 14975528 KiB
Current children cumulated memory: 14768552 KiB

[startup+1662.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/198 2768
/proc/meminfo: memFree=14710876/32770624 swapFree=66913120/67108860
[pid=2387] ppid=2384 vsize=15502472 memory=15320112 CPUtime=1661.85 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) R 2384 2387 1482 0 -1 4202496 3668593 0 1 0 165500 685 0 0 20 0 1 0 434006544 15874531328 3830028 33554432000 4194304 6211092 140734410332848 140734410311944 140292094749669 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 3875618 3830028 846 493 0 3868923 0
Current children cumulated CPU time: 1661.85 s
Current children cumulated vsize: 15502472 KiB
Current children cumulated memory: 15320112 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+1704.4 s]
/proc/loadavg: 2.00 2.01 2.05 3/198 2770
/proc/meminfo: memFree=14322392/32770624 swapFree=66913120/67108860
[pid=2387] ppid=2384 vsize=15872204 memory=15708288 CPUtime=1703.94 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) R 2384 2387 1482 0 -1 4202496 3761143 0 1 0 169688 706 0 0 20 0 1 0 434006544 16253136896 3927072 33554432000 4194304 6211092 140734410332848 140734410311944 140292094749669 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 3968051 3927072 846 493 0 3961356 0
Current children cumulated CPU time: 1703.94 s
Current children cumulated vsize: 15872204 KiB
Current children cumulated memory: 15708288 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+1707.15 s]
# the end of solver process 2387 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=1697.14 s, system=7.88016 s

Solver just ended.
Dumping a history of the last processes samples

[startup+1705.6 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 2771
/proc/meminfo: memFree=13990360/32770624 swapFree=66913120/67108860
[pid=2387] ppid=2384 vsize=15872204 memory=15708288 CPUtime=1704.13 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) R 2384 2387 1482 0 -1 512 3761148 0 1 0 169703 710 0 0 20 0 1 0 434006544 16253136896 3927072 33554432000 4194304 6211092 140734410332848 140734410311944 5189768 0 16384 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 3968051 3927072 846 493 0 3961356 0
Current children cumulated CPU time: 1704.13 s
Current children cumulated vsize: 15872204 KiB
Current children cumulated memory: 15708288 KiB

[startup+1706 s]*
/proc/loadavg: 2.00 2.01 2.05 4/199 2771
/proc/meminfo: memFree=13464088/32770624 swapFree=66913120/67108860
[pid=2387] ppid=2384 vsize=15872204 memory=15708288 CPUtime=1704.19 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) S 2384 2387 1482 0 -1 512 3761148 0 1 0 169707 712 0 0 20 0 1 0 434006544 16253136896 3927072 33554432000 4194304 6211092 140734410332848 140734410311944 5189768 0 16384 4096 8405026 18446744071580971376 0 0 17 1 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 3968051 3927072 846 493 0 3961356 0
Current children cumulated CPU time: 1704.19 s
Current children cumulated vsize: 15872204 KiB
Current children cumulated memory: 15708288 KiB

[startup+1706.4 s]
/proc/loadavg: 2.00 2.01 2.05 4/199 2771
/proc/meminfo: memFree=13464088/32770624 swapFree=66913120/67108860
[pid=2387] ppid=2384 vsize=15872204 memory=15708288 CPUtime=1704.25 cores=1,3,5,7
/proc/2387/stat : 2387 (cosoco) S 2384 2387 1482 0 -1 512 3761148 0 1 0 169713 712 0 0 20 0 1 0 434006544 16253136896 3927072 33554432000 4194304 6211092 140734410332848 140734410311944 5189768 0 16384 4096 8405026 18446744071580971376 0 0 17 1 0 0 0 0 0 8312316 8317760 27136000 140734410339339 140734410339441 140734410339441 140734410342350 0
/proc/2387/statm: 3968051 3927072 846 493 0 3961356 0
Current children cumulated CPU time: 1704.25 s
Current children cumulated vsize: 15872204 KiB
Current children cumulated memory: 15708288 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1707.15
CPU time (s): 1705.02
CPU user time (s): 1697.14
CPU system time (s): 7.88016
CPU usage (%): 99.8749
Max. virtual memory (cumulated for all children) (KiB): 15872204
Max. memory (cumulated for all children) (KiB): 15708288

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1697.14
system time used= 7.88016
maximum resident set size= 15708532
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3761148
page faults= 1
swaps= 0
block input operations= 4880
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 133837
involuntary context switches= 1619


# summary of solver processes directly reported to runsolver:
#   pid: 2387
#   total CPU time (s): 1705.02
#   total CPU user time (s): 1697.14
#   total CPU system time (s): 7.88016

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 5.95955 second user time and 11.6897 second system time

The end

Launcher Data 

Begin job on node104 at 2019-09-15 20:47:56
IDJOB=4397737
IDBENCH=122994
IDSOLVER=2850
FILE ID=node104/4397737-1568577494
RUNJOBID= node104-1568568567-1514
PBS_JOBID= 21703336
Free space on /tmp= 59008 MiB

SOLVER NAME= cosoco 2.0
BENCH NAME= XCSP17/Opd/Opd-sum-small/Opd-10-038-010.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4397737-1568577494/watcher-4397737-1568577494 -o /tmp/evaluation-result-4397737-1568577494/solver-4397737-1568577494 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node104-1568568567-1514 --watchdog 2560  cosoco -lc -model -restarts -stick HOME/instance-4397737-1568577494.xml

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 8e449228b7b3cf049a5fccac2aa13fe6
RANDOM SEED=2079047598

node104.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        15250468 kB
MemAvailable:   16832928 kB
Buffers:          139996 kB
Cached:          1721060 kB
SwapCached:        95228 kB
Active:         15429624 kB
Inactive:        1451256 kB
Active(anon):   14742140 kB
Inactive(anon):   326428 kB
Active(file):     687484 kB
Inactive(file):  1124828 kB
Unevictable:      110868 kB
Mlocked:          110868 kB
SwapTotal:      67108860 kB
SwapFree:       66912896 kB
Dirty:              3168 kB
Writeback:             0 kB
AnonPages:      15037080 kB
Mapped:            45544 kB
Shmem:             40576 kB
Slab:             201200 kB
SReclaimable:     171968 kB
SUnreclaim:        29232 kB
KernelStack:        3504 kB
PageTables:        34756 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:   15460580 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     98304 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      124288 kB
DirectMap2M:    33421312 kB

Free space on /tmp at the end= 58488 MiB
End job on node104 at 2019-09-15 21:16:23