Trace number 4408351

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2.0? (MO) 2072.75 2074.7

General information on the benchmark

NameOpd/Opd-sum-large/
Opd-04-350-100.xml
MD5SUM05a37308653f1418d8a9d31e27700b02
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark9
Best CPU time to get the best result obtained on this benchmark10.1781
Satisfiable
(Un)Satisfiability was proved
Number of variables1401
Number of constraints11
Number of domains2
Minimum domain size2
Maximum domain size342
Distribution of domain sizes[{"size":2,"count":1400},{"size":342,"count":1}]
Minimum variable degree5
Maximum variable degree7
Distribution of variable degrees[{"degree":5,"count":1400},{"degree":7,"count":1}]
Minimum constraint arity350
Maximum constraint arity1400
Distribution of constraint arities[{"arity":350,"count":4},{"arity":701,"count":6},{"arity":1400,"count":1}]
Number of extensional constraints0
Number of intensional constraints0
Distribution of constraint types[{"type":"lex","count":1},{"type":"sum","count":10}]
Optimization problemYES
Type of objectivemin VAR

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (12271 MiB free)
  memory of node 1: 16384 MiB (15676 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4408351-1569790237/watcher-4408351-1569790237 -o /tmp/evaluation-result-4408351-1569790237/solver-4408351-1569790237 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node148-1569787355-25780 --watchdog 2580 cosoco -lc -model -restarts -stick HOME/instance-4408351-1569790237.xml 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2550 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
Current StackSize limit: 8192 KiB

solver pid=26245, runsolver pid=26242

[startup+0.100058 s]*
/proc/loadavg: 2.16 2.05 1.94 3/199 26246
/proc/meminfo: memFree=28616400/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=29220 memory=5636 CPUtime=0.1 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) R 26242 26245 25748 0 -1 4202496 1611 0 0 0 10 0 0 0 20 0 1 0 555185139 29921280 1409 33554432000 4194304 6211132 140730487974512 140730487971048 5451586 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 7305 1409 846 493 0 610 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 29220 KiB
Current children cumulated memory: 5636 KiB

[startup+0.209962 s]*
/proc/loadavg: 2.16 2.05 1.94 3/199 26246
/proc/meminfo: memFree=28615656/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=30012 memory=6428 CPUtime=0.21 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) R 26242 26245 25748 0 -1 4202496 1821 0 0 0 21 0 0 0 20 0 1 0 555185139 30732288 1607 33554432000 4194304 6211132 140730487974512 140730487971048 139850266085264 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 7503 1607 846 493 0 808 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 30012 KiB
Current children cumulated memory: 6428 KiB

[startup+0.309043 s]*
/proc/loadavg: 2.16 2.05 1.94 4/199 26246
/proc/meminfo: memFree=28614912/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=30804 memory=7220 CPUtime=0.31 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) R 26242 26245 25748 0 -1 4202496 2011 0 0 0 31 0 0 0 20 0 1 0 555185139 31543296 1805 33554432000 4194304 6211132 140730487974512 140730487833800 139850275109725 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 7701 1805 846 493 0 1006 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 30804 KiB
Current children cumulated memory: 7220 KiB

[startup+0.700273 s]
/proc/loadavg: 2.16 2.05 1.94 3/199 26246
/proc/meminfo: memFree=28612944/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=33708 memory=10124 CPUtime=0.69 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) R 26242 26245 25748 0 -1 4202496 2746 0 0 0 69 0 0 0 20 0 1 0 555185139 34516992 2531 33554432000 4194304 6211132 140730487974512 140730487833800 139850265164773 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 8427 2531 846 493 0 1732 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 33708 KiB
Current children cumulated memory: 10124 KiB

[startup+1.50029 s]
/proc/loadavg: 2.16 2.05 1.94 3/199 26246
/proc/meminfo: memFree=28609224/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=39912 memory=16460 CPUtime=1.49 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) R 26242 26245 25748 0 -1 4202496 4281 0 0 0 149 0 0 0 20 0 1 0 555185139 40869888 4115 33554432000 4194304 6211132 140730487974512 140730487833800 139850265164773 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 9978 4115 846 493 0 3283 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 39912 KiB
Current children cumulated memory: 16460 KiB

[startup+3.10935 s]
/proc/loadavg: 2.16 2.05 1.94 3/199 26253
/proc/meminfo: memFree=28592184/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=52252 memory=28640 CPUtime=3.1 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) R 26242 26245 25748 0 -1 4202496 7552 0 0 0 309 1 0 0 20 0 1 0 555185139 53506048 7160 33554432000 4194304 6211132 140730487974512 140730487833800 139850265164773 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 13063 7160 846 493 0 6368 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 52252 KiB
Current children cumulated memory: 28640 KiB

[startup+6.30027 s]
/proc/loadavg: 2.15 2.05 1.94 3/199 26253
/proc/meminfo: memFree=28569228/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=76664 memory=53212 CPUtime=6.29 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) R 26242 26245 25748 0 -1 4202496 13835 0 0 0 627 2 0 0 20 0 1 0 555185139 78503936 13303 33554432000 4194304 6211132 140730487974512 140730487833800 139850265164773 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 19166 13303 846 493 0 12471 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 76664 KiB
Current children cumulated memory: 53212 KiB

[startup+12.7003 s]
/proc/loadavg: 2.14 2.05 1.94 3/199 26253
/proc/meminfo: memFree=28523316/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=125820 memory=102040 CPUtime=12.69 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) R 26242 26245 25748 0 -1 4202496 26269 0 0 0 1266 3 0 0 20 0 1 0 555185139 128839680 25510 33554432000 4194304 6211132 140730487974512 140730487833800 139850265164773 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 31455 25510 846 493 0 24760 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 125820 KiB
Current children cumulated memory: 102040 KiB

[startup+25.5003 s]
/proc/loadavg: 2.11 2.04 1.94 3/200 26255
/proc/meminfo: memFree=28424044/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=223652 memory=200108 CPUtime=25.48 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) R 26242 26245 25748 0 -1 4202496 51044 0 0 0 2542 6 0 0 20 0 1 0 555185139 229019648 50027 33554432000 4194304 6211132 140730487974512 140730487833800 139850265164773 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 55913 50027 846 493 0 49218 0
Current children cumulated CPU time: 25.48 s
Current children cumulated vsize: 223652 KiB
Current children cumulated memory: 200108 KiB

[startup+51.1095 s]
/proc/loadavg: 2.07 2.04 1.94 3/199 26255
/proc/meminfo: memFree=28225392/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=419816 memory=395916 CPUtime=51.09 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) R 26242 26245 25748 0 -1 4202496 100133 0 0 0 5096 13 0 0 20 0 1 0 555185139 429891584 98979 33554432000 4194304 6211132 140730487974512 140730487833800 5446119 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 104954 98979 846 493 0 98259 0
Current children cumulated CPU time: 51.09 s
Current children cumulated vsize: 419816 KiB
Current children cumulated memory: 395916 KiB

[startup+102.3 s]
/proc/loadavg: 2.03 2.04 1.95 3/199 26259
/proc/meminfo: memFree=27834336/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=812144 memory=787576 CPUtime=102.28 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) R 26242 26245 25748 0 -1 4202496 198174 0 0 0 10199 29 0 0 20 0 1 0 555185139 831635456 196894 33554432000 4194304 6211132 140730487974512 140730487833800 139850265164773 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 203036 196894 846 493 0 196341 0
Current children cumulated CPU time: 102.28 s
Current children cumulated vsize: 812144 KiB
Current children cumulated memory: 787576 KiB

[startup+162.3 s]
/proc/loadavg: 2.01 2.03 1.95 3/199 26265
/proc/meminfo: memFree=27373884/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=1271420 memory=1246672 CPUtime=162.27 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) R 26242 26245 25748 0 -1 4202496 312916 0 0 0 16181 46 0 0 20 0 1 0 555185139 1301934080 311668 33554432000 4194304 6211132 140730487974512 140730487833800 139850265164773 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 317855 311668 846 493 0 311160 0
Current children cumulated CPU time: 162.27 s
Current children cumulated vsize: 1271420 KiB
Current children cumulated memory: 1246672 KiB

[startup+222.3 s]
/proc/loadavg: 2.00 2.03 1.95 3/199 26269
/proc/meminfo: memFree=26913384/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=1731644 memory=1705452 CPUtime=222.27 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) R 26242 26245 25748 0 -1 4202496 427648 0 0 0 22164 63 0 0 20 0 1 0 555185139 1773203456 426363 33554432000 4194304 6211132 140730487974512 140730487833800 139850265164773 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 432911 426363 846 493 0 426216 0
Current children cumulated CPU time: 222.27 s
Current children cumulated vsize: 1731644 KiB
Current children cumulated memory: 1705452 KiB

[startup+282.3 s]
/proc/loadavg: 2.00 2.02 1.95 3/199 26273
/proc/meminfo: memFree=26453484/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=2188760 memory=2164548 CPUtime=282.25 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) R 26242 26245 25748 0 -1 4202496 542395 0 0 0 28144 81 0 0 20 0 1 0 555185139 2241290240 541137 33554432000 4194304 6211132 140730487974512 140730487833800 139850265160510 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 547190 541137 846 493 0 540495 0
Current children cumulated CPU time: 282.25 s
Current children cumulated vsize: 2188760 KiB
Current children cumulated memory: 2164548 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 2.01 1.95 3/199 26280
/proc/meminfo: memFree=25992996/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=2650604 memory=2624320 CPUtime=342.25 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) R 26242 26245 25748 0 -1 4202496 656859 0 0 0 34128 97 0 0 20 0 1 0 555185139 2714218496 656080 33554432000 4194304 6211132 140730487974512 140730487833800 139850265164773 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 662651 656080 846 493 0 655956 0
Current children cumulated CPU time: 342.25 s
Current children cumulated vsize: 2650604 KiB
Current children cumulated memory: 2624320 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 2.01 1.95 3/199 26284
/proc/meminfo: memFree=25532328/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=3107720 memory=3083396 CPUtime=402.25 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) R 26242 26245 25748 0 -1 4202496 771245 0 0 0 40109 116 0 0 20 0 1 0 555185139 3182305280 770849 33554432000 4194304 6211132 140730487974512 140730487833800 139850265164773 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 776930 770849 846 493 0 770235 0
Current children cumulated CPU time: 402.25 s
Current children cumulated vsize: 3107720 KiB
Current children cumulated memory: 3083396 KiB

[startup+462.3 s]
/proc/loadavg: 2.00 2.01 1.95 3/199 26288
/proc/meminfo: memFree=25073904/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=3571864 memory=3541268 CPUtime=462.24 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) R 26242 26245 25748 0 -1 4202496 885939 0 0 0 46088 136 0 0 20 0 1 0 555185139 3657588736 885317 33554432000 4194304 6211132 140730487974512 140730487833800 139850265164773 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 892966 885317 846 493 0 886271 0
Current children cumulated CPU time: 462.24 s
Current children cumulated vsize: 3571864 KiB
Current children cumulated memory: 3541268 KiB


################
# More data... #
################

Current children cumulated vsize: 10906196 KiB
Current children cumulated memory: 10879564 KiB

[startup+1482.3 s]
/proc/loadavg: 2.03 2.11 2.03 3/200 26408
/proc/meminfo: memFree=17261760/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=11362520 memory=11337776 CPUtime=1482.15 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) R 26242 26245 25748 0 -1 4202496 2836811 0 0 0 147691 524 0 0 20 0 1 0 555185139 11635220480 2834444 33554432000 4194304 6211132 140730487974512 140730487833800 139850265164773 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 2840630 2834444 846 493 0 2833935 0
Current children cumulated CPU time: 1482.15 s
Current children cumulated vsize: 11362520 KiB
Current children cumulated memory: 11337776 KiB

[startup+1542.3 s]
/proc/loadavg: 2.01 2.09 2.03 3/199 26412
/proc/meminfo: memFree=16802728/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=11843580 memory=11795784 CPUtime=1542.14 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) R 26242 26245 25748 0 -1 4202496 2951892 0 0 0 153664 550 0 0 20 0 1 0 555185139 12127825920 2948946 33554432000 4194304 6211132 140730487974512 140730487833800 139850265164773 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 2960895 2948946 846 493 0 2954200 0
Current children cumulated CPU time: 1542.14 s
Current children cumulated vsize: 11843580 KiB
Current children cumulated memory: 11795784 KiB

[startup+1602.3 s]
/proc/loadavg: 2.00 2.07 2.03 3/199 26418
/proc/meminfo: memFree=16342772/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=12300168 memory=12255008 CPUtime=1602.14 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) R 26242 26245 25748 0 -1 4202496 3066791 0 0 0 159636 578 0 0 20 0 1 0 555185139 12595372032 3063752 33554432000 4194304 6211132 140730487974512 140730487833800 139850265164773 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 3075042 3063752 846 493 0 3068347 0
Current children cumulated CPU time: 1602.14 s
Current children cumulated vsize: 12300168 KiB
Current children cumulated memory: 12255008 KiB

[startup+1662.3 s]
/proc/loadavg: 2.00 2.06 2.03 3/199 26422
/proc/meminfo: memFree=15882972/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=12756624 memory=12713576 CPUtime=1662.14 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) R 26242 26245 25748 0 -1 4202496 3181632 0 0 0 165610 604 0 0 20 0 1 0 555185139 13062782976 3178394 33554432000 4194304 6211132 140730487974512 140730487833800 139850265164773 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 3189156 3178394 846 493 0 3182461 0
Current children cumulated CPU time: 1662.14 s
Current children cumulated vsize: 12756624 KiB
Current children cumulated memory: 12713576 KiB

[startup+1722.3 s]
/proc/loadavg: 2.00 2.05 2.03 3/199 26426
/proc/meminfo: memFree=15423600/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=13213080 memory=13172048 CPUtime=1722.13 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) R 26242 26245 25748 0 -1 4202496 3296334 0 0 0 171580 633 0 0 20 0 1 0 555185139 13530193920 3293012 33554432000 4194304 6211132 140730487974512 140730487833800 139850265164773 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 3303270 3293012 846 493 0 3296575 0
Current children cumulated CPU time: 1722.13 s
Current children cumulated vsize: 13213080 KiB
Current children cumulated memory: 13172048 KiB

[startup+1782.3 s]
/proc/loadavg: 2.00 2.04 2.03 3/199 26432
/proc/meminfo: memFree=14964032/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=13669404 memory=13630524 CPUtime=1782.12 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) R 26242 26245 25748 0 -1 4202496 3411026 0 0 0 177551 661 0 0 20 0 1 0 555185139 13997469696 3407631 33554432000 4194304 6211132 140730487974512 140730487833800 139850265164773 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 3417351 3407631 846 493 0 3410656 0
Current children cumulated CPU time: 1782.12 s
Current children cumulated vsize: 13669404 KiB
Current children cumulated memory: 13630524 KiB

[startup+1842.3 s]
/proc/loadavg: 2.00 2.03 2.03 3/200 26444
/proc/meminfo: memFree=14505072/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=14125728 memory=14088956 CPUtime=1842.12 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) R 26242 26245 25748 0 -1 4202496 3525728 0 0 0 183522 690 0 0 20 0 1 0 555185139 14464745472 3522239 33554432000 4194304 6211132 140730487974512 140730487833800 139850265164773 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 3531432 3522239 846 493 0 3524737 0
Current children cumulated CPU time: 1842.12 s
Current children cumulated vsize: 14125728 KiB
Current children cumulated memory: 14088956 KiB

[startup+1902.3 s]
/proc/loadavg: 2.00 2.03 2.03 3/200 26448
/proc/meminfo: memFree=14045420/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=14582184 memory=14547260 CPUtime=1902.11 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) R 26242 26245 25748 0 -1 4202496 3640545 0 0 0 189492 719 0 0 20 0 1 0 555185139 14932156416 3636815 33554432000 4194304 6211132 140730487974512 140730487833800 139850265164773 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 3645546 3636815 846 493 0 3638851 0
Current children cumulated CPU time: 1902.11 s
Current children cumulated vsize: 14582184 KiB
Current children cumulated memory: 14547260 KiB

[startup+1962.3 s]
/proc/loadavg: 2.00 2.02 2.03 3/200 26454
/proc/meminfo: memFree=13585956/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=15038376 memory=15005472 CPUtime=1962.11 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) R 26242 26245 25748 0 -1 4202496 3755196 0 0 0 195462 749 0 0 20 0 1 0 555185139 15399297024 3751368 33554432000 4194304 6211132 140730487974512 140730487833800 139850265164773 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 3759594 3751368 846 493 0 3752899 0
Current children cumulated CPU time: 1962.11 s
Current children cumulated vsize: 15038376 KiB
Current children cumulated memory: 15005472 KiB

[startup+2022.3 s]
/proc/loadavg: 2.00 2.02 2.03 3/200 26458
/proc/meminfo: memFree=13127648/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=15494436 memory=15463732 CPUtime=2022.1 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) R 26242 26245 25748 0 -1 4202496 3869925 0 0 0 201431 779 0 0 20 0 1 0 555185139 15866302464 3865933 33554432000 4194304 6211132 140730487974512 140730487833800 139850265164773 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 3873609 3865933 846 493 0 3866914 0
Current children cumulated CPU time: 2022.1 s
Current children cumulated vsize: 15494436 KiB
Current children cumulated memory: 15463732 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+2072 s]*
/proc/loadavg: 2.00 2.01 2.03 3/200 26462
/proc/meminfo: memFree=12746976/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=15872088 memory=15841176 CPUtime=2071.81 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) R 26242 26245 25748 0 -1 4202496 3964813 0 0 0 206376 805 0 0 20 0 1 0 555185139 16253018112 3960294 33554432000 4194304 6211132 140730487974512 140730487833800 139850265164773 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 3968022 3960294 846 493 0 3961327 0
Current children cumulated CPU time: 2071.81 s
Current children cumulated vsize: 15872088 KiB
Current children cumulated memory: 15841176 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+2074.7 s]
# the end of solver process 26245 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2063.98 s, system=8.77295 s

Solver just ended.
Dumping a history of the last processes samples

[startup+2072.8 s]
/proc/loadavg: 2.00 2.01 2.03 3/200 26462
/proc/meminfo: memFree=12746976/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=15872092 memory=15841176 CPUtime=2071.87 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) S 26242 26245 25748 0 -1 512 3964819 0 0 0 206381 806 0 0 20 0 1 0 555185139 16253022208 3960294 33554432000 4194304 6211132 140730487974512 140730487833800 5189800 0 16384 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 3968023 3960294 846 493 0 3961328 0
Current children cumulated CPU time: 2071.87 s
Current children cumulated vsize: 15872092 KiB
Current children cumulated memory: 15841176 KiB

[startup+2073.6 s]
/proc/loadavg: 2.00 2.01 2.03 3/201 26463
/proc/meminfo: memFree=12469432/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=15872092 memory=15841176 CPUtime=2071.99 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) S 26242 26245 25748 0 -1 512 3964819 0 0 0 206391 808 0 0 20 0 1 0 555185139 16253022208 3960294 33554432000 4194304 6211132 140730487974512 140730487833800 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 1 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 3968023 3960294 846 493 0 3961328 0
Current children cumulated CPU time: 2071.99 s
Current children cumulated vsize: 15872092 KiB
Current children cumulated memory: 15841176 KiB

[startup+2074 s]*
/proc/loadavg: 2.00 2.01 2.03 4/201 26463
/proc/meminfo: memFree=11939456/32770624 swapFree=67002732/67108860
[pid=26245] ppid=26242 vsize=15872092 memory=15841176 CPUtime=2072.05 cores=1,3,5,7
/proc/26245/stat : 26245 (cosoco) S 26242 26245 25748 0 -1 512 3964819 0 0 0 206397 808 0 0 20 0 1 0 555185139 16253022208 3960294 33554432000 4194304 6211132 140730487974512 140730487833800 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 1 0 0 0 0 0 8312316 8317760 11325440 140730487981067 140730487981169 140730487981169 140730487984078 0
/proc/26245/statm: 3968023 3960294 846 493 0 3961328 0
Current children cumulated CPU time: 2072.05 s
Current children cumulated vsize: 15872092 KiB
Current children cumulated memory: 15841176 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 2074.7
CPU time (s): 2072.75
CPU user time (s): 2063.98
CPU system time (s): 8.77295
CPU usage (%): 99.906
Max. virtual memory (cumulated for all children) (KiB): 15872092
Max. memory (cumulated for all children) (KiB): 15841176

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2063.98
system time used= 8.77295
maximum resident set size= 15841428
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3964819
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 100146
involuntary context switches= 1674


# summary of solver processes directly reported to runsolver:
#   pid: 26245
#   total CPU time (s): 2072.75
#   total CPU user time (s): 2063.98
#   total CPU system time (s): 8.77295

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 6.85905 second user time and 14.7591 second system time

The end

Launcher Data 

Begin job on node148 at 2019-09-29 22:49:59
IDJOB=4408351
IDBENCH=122932
IDSOLVER=2874
FILE ID=node148/4408351-1569790237
RUNJOBID= node148-1569787355-25780
PBS_JOBID= 21705817
Free space on /tmp= 62028 MiB

SOLVER NAME= cosoco 2.0
BENCH NAME= XCSP17/Opd/Opd-sum-large/Opd-04-350-100.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4408351-1569790237/watcher-4408351-1569790237 -o /tmp/evaluation-result-4408351-1569790237/solver-4408351-1569790237 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node148-1569787355-25780 --watchdog 2580  cosoco -lc -model -restarts -stick HOME/instance-4408351-1569790237.xml

TIME LIMIT= 2520 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 05a37308653f1418d8a9d31e27700b02
RANDOM SEED=789568141

node148.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.13
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.13
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.13
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.13
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        28618508 kB
MemAvailable:   31875872 kB
Buffers:          155480 kB
Cached:          3441348 kB
SwapCached:         3480 kB
Active:          2222720 kB
Inactive:        1380296 kB
Active(anon):      19584 kB
Inactive(anon):    67148 kB
Active(file):    2203136 kB
Inactive(file):  1313148 kB
Unevictable:      110976 kB
Mlocked:          110976 kB
SwapTotal:      67108860 kB
SwapFree:       67002732 kB
Dirty:              2808 kB
Writeback:             0 kB
AnonPages:         61480 kB
Mapped:            27824 kB
Shmem:             71796 kB
Slab:             142800 kB
SReclaimable:     113832 kB
SUnreclaim:        28968 kB
KernelStack:        3760 kB
PageTables:         5400 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     569556 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     14336 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      124288 kB
DirectMap2M:    33421312 kB

Free space on /tmp at the end= 61640 MiB
End job on node148 at 2019-09-29 23:24:33