Trace number 4390386

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2? (MO) 1519.8 1521.07

General information on the benchmark

NameSteelMillSlab/
SteelMillSlab-bench_11_1.xml
MD5SUM32630071a9cd02ab12e325390d6637b6
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkSAT TO
Best value of the objective obtained on this benchmark260
Best CPU time to get the best result obtained on this benchmark2400.12
Satisfiable
(Un)Satisfiability was proved
Number of variables12654
Number of constraints466
Number of domains4
Minimum domain size2
Maximum domain size111
Distribution of domain sizes[{"size":2,"count":12321},{"size":13,"count":111},{"size":50,"count":111},{"size":111,"count":111}]
Minimum variable degree2
Maximum variable degree114
Distribution of variable degrees[{"degree":2,"count":12432},{"degree":3,"count":111},{"degree":112,"count":78},{"degree":113,"count":24},{"degree":114,"count":9}]
Minimum constraint arity2
Maximum constraint arity112
Distribution of constraint arities[{"arity":2,"count":132},{"arity":111,"count":112},{"arity":112,"count":222}]
Number of extensional constraints0
Number of intensional constraints132
Distribution of constraint types[{"type":"intension","count":132},{"type":"sum","count":112},{"type":"element","count":111},{"type":"channel","count":111}]
Optimization problemYES
Type of objectivemin SUM

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (8534 MiB free)
  memory of node 1: 16384 MiB (11644 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4390386-1568354084/watcher-4390386-1568354084 -o /tmp/evaluation-result-4390386-1568354084/solver-4390386-1568354084 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node125-1568323660-10514 --watchdog 2560 cosoco -lc -model -restarts -stick HOME/instance-4390386-1568354084.xml 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2500 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
Current StackSize limit: 8192 KiB

solver pid=16654, runsolver pid=16651

[startup+0.100167 s]*
/proc/loadavg: 2.06 2.03 2.05 3/199 16655
/proc/meminfo: memFree=20650892/32770624 swapFree=66940188/67108860
[pid=16654] ppid=16651 vsize=39152 memory=15356 CPUtime=0.1 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) R 16651 16654 10482 0 -1 4202496 4013 0 0 0 10 0 0 0 20 0 1 0 411675248 40091648 3839 33554432000 4194304 6210612 140730107778256 140730107774792 139728541481957 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 9788 3839 809 493 0 3093 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 39152 KiB
Current children cumulated memory: 15356 KiB

[startup+0.209615 s]*
/proc/loadavg: 2.06 2.03 2.05 3/199 16655
/proc/meminfo: memFree=20647916/32770624 swapFree=66940188/67108860
[pid=16654] ppid=16651 vsize=40728 memory=16872 CPUtime=0.21 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) R 16651 16654 10482 0 -1 4202496 4437 0 0 0 21 0 0 0 20 0 1 0 411675248 41705472 4218 33554432000 4194304 6210612 140730107778256 140730107774792 139728541481957 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 10182 4218 819 493 0 3487 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 40728 KiB
Current children cumulated memory: 16872 KiB

[startup+0.308901 s]*
/proc/loadavg: 2.06 2.03 2.05 3/199 16655
/proc/meminfo: memFree=20645684/32770624 swapFree=66940188/67108860
[pid=16654] ppid=16651 vsize=41844 memory=17928 CPUtime=0.3 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) R 16651 16654 10482 0 -1 4202496 4694 0 0 0 30 0 0 0 20 0 1 0 411675248 42848256 4482 33554432000 4194304 6210612 140730107778256 140730107774792 139728541481957 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 10461 4482 819 493 0 3766 0
Current children cumulated CPU time: 0.3 s
Current children cumulated vsize: 41844 KiB
Current children cumulated memory: 17928 KiB

[startup+0.70029 s]
/proc/loadavg: 2.06 2.03 2.05 3/199 16655
/proc/meminfo: memFree=20641592/32770624 swapFree=66940188/67108860
[pid=16654] ppid=16651 vsize=45840 memory=21992 CPUtime=0.69 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) R 16651 16654 10482 0 -1 4202496 5691 0 0 0 69 0 0 0 20 0 1 0 411675248 46940160 5498 33554432000 4194304 6210612 140730107778256 140730107774792 5451322 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 11460 5498 819 493 0 4765 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 45840 KiB
Current children cumulated memory: 21992 KiB

[startup+1.50029 s]
/proc/loadavg: 1.89 2.00 2.03 3/199 16655
/proc/meminfo: memFree=20631548/32770624 swapFree=66940188/67108860
[pid=16654] ppid=16651 vsize=54208 memory=30316 CPUtime=1.49 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) R 16651 16654 10482 0 -1 4202496 7772 0 0 0 148 1 0 0 20 0 1 0 411675248 55508992 7579 33554432000 4194304 6210612 140730107778256 140730107751816 139728542406098 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 13552 7579 819 493 0 6857 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 54208 KiB
Current children cumulated memory: 30316 KiB

[startup+3.10934 s]
/proc/loadavg: 1.89 2.00 2.03 3/199 16655
/proc/meminfo: memFree=20590168/32770624 swapFree=66940188/67108860
[pid=16654] ppid=16651 vsize=71156 memory=47068 CPUtime=3.1 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) R 16651 16654 10482 0 -1 4202496 12128 0 0 0 309 1 0 0 20 0 1 0 411675248 72863744 11767 33554432000 4194304 6210612 140730107778256 140730107751816 139728541481957 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 17789 11767 819 493 0 11094 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 71156 KiB
Current children cumulated memory: 47068 KiB

[startup+6.30027 s]
/proc/loadavg: 1.90 2.00 2.03 3/200 16657
/proc/meminfo: memFree=20526864/32770624 swapFree=66940188/67108860
[pid=16654] ppid=16651 vsize=104964 memory=80256 CPUtime=6.29 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) R 16651 16654 10482 0 -1 4202496 20601 0 0 0 627 2 0 0 20 0 1 0 411675248 107483136 20064 33554432000 4194304 6210612 140730107778256 140730107751816 5452055 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 26241 20064 819 493 0 19546 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 104964 KiB
Current children cumulated memory: 80256 KiB

[startup+12.7003 s]
/proc/loadavg: 1.91 2.00 2.03 3/199 16657
/proc/meminfo: memFree=20402044/32770624 swapFree=66940188/67108860
[pid=16654] ppid=16651 vsize=170820 memory=146784 CPUtime=12.69 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) R 16651 16654 10482 0 -1 4202496 37428 0 0 0 1265 4 0 0 20 0 1 0 411675248 174919680 36696 33554432000 4194304 6210612 140730107778256 140730107751816 5446088 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 42705 36696 819 493 0 36010 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 170820 KiB
Current children cumulated memory: 146784 KiB

[startup+25.5003 s]
/proc/loadavg: 1.92 2.00 2.03 3/199 16657
/proc/meminfo: memFree=20129620/32770624 swapFree=66940188/67108860
[pid=16654] ppid=16651 vsize=305316 memory=280516 CPUtime=25.49 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) R 16651 16654 10482 0 -1 4202496 70643 0 0 0 2540 9 0 0 20 0 1 0 411675248 312643584 70129 33554432000 4194304 6210612 140730107778256 140730107751816 5455493 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 76329 70129 819 493 0 69634 0
Current children cumulated CPU time: 25.49 s
Current children cumulated vsize: 305316 KiB
Current children cumulated memory: 280516 KiB

[startup+51.1094 s]
/proc/loadavg: 2.01 2.01 2.04 3/200 16660
/proc/meminfo: memFree=19587784/32770624 swapFree=66940188/67108860
[pid=16654] ppid=16651 vsize=575792 memory=548344 CPUtime=51.1 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) R 16651 16654 10482 0 -1 4202496 136756 0 0 0 5094 16 0 0 20 0 1 0 411675248 589611008 137086 33554432000 4194304 6210612 140730107778256 140730107751816 139728542405842 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 143948 137086 819 493 0 137253 0
Current children cumulated CPU time: 51.1 s
Current children cumulated vsize: 575792 KiB
Current children cumulated memory: 548344 KiB

[startup+102.3 s]
/proc/loadavg: 2.06 2.03 2.05 3/199 16665
/proc/meminfo: memFree=18521032/32770624 swapFree=66940188/67108860
[pid=16654] ppid=16651 vsize=1119384 memory=1079668 CPUtime=102.29 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) R 16651 16654 10482 0 -1 4202496 266683 0 0 0 10198 31 0 0 20 0 1 0 411675248 1146249216 269917 33554432000 4194304 6210612 140730107778256 140730107751816 139728541480074 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 279846 269917 819 493 0 273151 0
Current children cumulated CPU time: 102.29 s
Current children cumulated vsize: 1119384 KiB
Current children cumulated memory: 1079668 KiB

[startup+162.3 s]
/proc/loadavg: 2.02 2.02 2.05 3/199 16669
/proc/meminfo: memFree=17268680/32770624 swapFree=66940188/67108860
[pid=16654] ppid=16651 vsize=1748680 memory=1704012 CPUtime=162.29 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) R 16651 16654 10482 0 -1 4202496 418729 0 0 0 16177 52 0 0 20 0 1 0 411675248 1790648320 426003 33554432000 4194304 6210612 140730107778256 140730107751816 139728542405645 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 437170 426003 819 493 0 430475 0
Current children cumulated CPU time: 162.29 s
Current children cumulated vsize: 1748680 KiB
Current children cumulated memory: 1704012 KiB

[startup+222.3 s]
/proc/loadavg: 2.01 2.02 2.05 3/199 16674
/proc/meminfo: memFree=16012972/32770624 swapFree=66940188/67108860
[pid=16654] ppid=16651 vsize=2353504 memory=2329276 CPUtime=222.29 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) R 16651 16654 10482 0 -1 4202496 569923 0 0 0 22158 71 0 0 20 0 1 0 411675248 2409988096 582319 33554432000 4194304 6210612 140730107778256 140730107751816 139728551426894 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 588376 582319 819 493 0 581681 0
Current children cumulated CPU time: 222.29 s
Current children cumulated vsize: 2353504 KiB
Current children cumulated memory: 2329276 KiB

[startup+282.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 16681
/proc/meminfo: memFree=14762920/32770624 swapFree=66940188/67108860
[pid=16654] ppid=16651 vsize=2994976 memory=2952672 CPUtime=282.29 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) R 16651 16654 10482 0 -1 4202496 721655 0 0 0 28140 89 0 0 20 0 1 0 411675248 3066855424 738168 33554432000 4194304 6210612 140730107778256 140730107751816 5456138 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 748744 738168 819 493 0 742049 0
Current children cumulated CPU time: 282.29 s
Current children cumulated vsize: 2994976 KiB
Current children cumulated memory: 2952672 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 16685
/proc/meminfo: memFree=13510308/32770624 swapFree=66940188/67108860
[pid=16654] ppid=16651 vsize=3655032 memory=3576416 CPUtime=342.29 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) R 16651 16654 10482 0 -1 4202496 873341 0 0 0 34119 110 0 0 20 0 1 0 411675248 3742752768 894104 33554432000 4194304 6210612 140730107778256 140730107751816 5451347 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 913758 894104 819 493 0 907063 0
Current children cumulated CPU time: 342.29 s
Current children cumulated vsize: 3655032 KiB
Current children cumulated memory: 3576416 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 16697
/proc/meminfo: memFree=12258800/32770624 swapFree=66940192/67108860
[pid=16654] ppid=16651 vsize=4259592 memory=4200752 CPUtime=402.29 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) R 16651 16654 10482 0 -1 4202496 1025317 0 0 0 40098 131 0 0 20 0 1 0 411675248 4361822208 1050188 33554432000 4194304 6210612 140730107778256 140730107751816 5442932 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 1064898 1050188 819 493 0 1058203 0
Current children cumulated CPU time: 402.29 s
Current children cumulated vsize: 4259592 KiB
Current children cumulated memory: 4200752 KiB

[startup+462.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/201 16726
/proc/meminfo: memFree=11003668/32770624 swapFree=66940192/67108860
[pid=16654] ppid=16651 vsize=4863888 memory=4825500 CPUtime=462.29 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) R 16651 16654 10482 0 -1 4202496 1176948 0 0 0 46074 155 0 0 20 0 1 0 411675248 4980621312 1206375 33554432000 4194304 6210612 140730107778256 140730107751816 5453752 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 1215972 1206375 819 493 0 1209277 0
Current children cumulated CPU time: 462.29 s
Current children cumulated vsize: 4863888 KiB
Current children cumulated memory: 4825500 KiB


################
# More data... #
################

Current children cumulated vsize: 10507656 KiB
Current children cumulated memory: 10437016 KiB

[startup+1062.3 s]
/proc/loadavg: 2.05 2.03 2.05 3/199 16779
/proc/meminfo: memFree=190556/32770624 swapFree=66940196/67108860
[pid=16654] ppid=16651 vsize=11111028 memory=11058820 CPUtime=1062.29 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) R 16651 16654 10482 0 -1 4202496 2695111 0 0 0 105812 417 0 0 20 0 1 0 411675248 11377692672 2764705 33554432000 4194304 6210612 140730107778256 140730107751816 139728542405846 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 2777757 2764705 819 493 0 2771062 0
Current children cumulated CPU time: 1062.29 s
Current children cumulated vsize: 11111028 KiB
Current children cumulated memory: 11058820 KiB

[startup+1122.3 s]
/proc/loadavg: 2.02 2.02 2.05 3/199 16783
/proc/meminfo: memFree=189432/32770624 swapFree=66940196/67108860
[pid=16654] ppid=16651 vsize=11714268 memory=11682404 CPUtime=1122.29 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) R 16651 16654 10482 0 -1 4202496 2846601 0 0 0 111778 451 0 0 20 0 1 0 411675248 11995410432 2920601 33554432000 4194304 6210612 140730107778256 140730107751816 139728551426880 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 2928567 2920601 819 493 0 2921872 0
Current children cumulated CPU time: 1122.29 s
Current children cumulated vsize: 11714268 KiB
Current children cumulated memory: 11682404 KiB

[startup+1182.3 s]
/proc/loadavg: 2.01 2.02 2.05 3/199 16789
/proc/meminfo: memFree=183056/32770624 swapFree=66940196/67108860
[pid=16654] ppid=16651 vsize=12503968 memory=12305236 CPUtime=1182.29 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) R 16651 16654 10482 0 -1 4202496 2999062 0 0 0 117743 486 0 0 20 0 1 0 411675248 12804063232 3076309 33554432000 4194304 6210612 140730107778256 140730107751816 139728541481957 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 3125992 3076309 819 493 0 3119297 0
Current children cumulated CPU time: 1182.29 s
Current children cumulated vsize: 12503968 KiB
Current children cumulated memory: 12305236 KiB

[startup+1242.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 16793
/proc/meminfo: memFree=203044/32770624 swapFree=66940196/67108860
[pid=16654] ppid=16651 vsize=13107208 memory=12927036 CPUtime=1242.3 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) R 16651 16654 10482 0 -1 4202496 3150560 0 0 0 123709 521 0 0 20 0 1 0 411675248 13421780992 3231759 33554432000 4194304 6210612 140730107778256 140730107751816 139728542405638 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 3276802 3231759 819 493 0 3270107 0
Current children cumulated CPU time: 1242.3 s
Current children cumulated vsize: 13107208 KiB
Current children cumulated memory: 12927036 KiB

[startup+1302.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 16797
/proc/meminfo: memFree=211848/32770624 swapFree=66940196/67108860
[pid=16654] ppid=16651 vsize=13710184 memory=13550468 CPUtime=1302.29 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) R 16651 16654 10482 0 -1 4202496 3302177 0 0 0 129676 553 0 0 20 0 1 0 411675248 14039228416 3387617 33554432000 4194304 6210612 140730107778256 140730107751816 5442785 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 3427546 3387617 819 493 0 3420851 0
Current children cumulated CPU time: 1302.29 s
Current children cumulated vsize: 13710184 KiB
Current children cumulated memory: 13550468 KiB

[startup+1362.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 16803
/proc/meminfo: memFree=204684/32770624 swapFree=66940320/67108860
[pid=16654] ppid=16651 vsize=14312632 memory=14171324 CPUtime=1362.3 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) R 16651 16654 10482 0 -1 4202496 3453641 0 0 0 135638 592 0 0 20 0 1 0 411675248 14656135168 3542831 33554432000 4194304 6210612 140730107778256 140730107751816 139728541481957 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 3578158 3542831 819 493 0 3571463 0
Current children cumulated CPU time: 1362.3 s
Current children cumulated vsize: 14312632 KiB
Current children cumulated memory: 14171324 KiB

[startup+1422.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 16807
/proc/meminfo: memFree=197656/32770624 swapFree=66759868/67108860
[pid=16654] ppid=16651 vsize=14913760 memory=14788568 CPUtime=1422.29 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) R 16651 16654 10482 0 -1 4202496 3605212 0 0 0 141594 635 0 0 20 0 1 0 411675248 15271690240 3645944 33554432000 4194304 6210612 140730107778256 140730107751816 139728542405842 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 3728440 3645944 192 493 0 3721745 0
Current children cumulated CPU time: 1422.29 s
Current children cumulated vsize: 14913760 KiB
Current children cumulated memory: 14788568 KiB

[startup+1482.3 s]
/proc/loadavg: 2.05 2.03 2.05 3/199 16811
/proc/meminfo: memFree=215716/32770624 swapFree=66173820/67108860
[pid=16654] ppid=16651 vsize=15513304 memory=15408480 CPUtime=1482.28 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) R 16651 16654 10482 0 -1 4202496 3756573 0 0 0 147549 679 0 0 20 0 1 0 411675248 15885623296 3654410 33554432000 4194304 6210612 140730107778256 140730107751816 5012594 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 3878326 3654410 192 493 0 3871631 0
Current children cumulated CPU time: 1482.28 s
Current children cumulated vsize: 15513304 KiB
Current children cumulated memory: 15408480 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+1518.21 s]
/proc/loadavg: 2.46 2.12 2.08 3/200 16880
/proc/meminfo: memFree=16347584/32770624 swapFree=65890380/67108860
[pid=16654] ppid=16651 vsize=15872740 memory=15780172 CPUtime=1518.2 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) R 16651 16654 10482 0 -1 4202496 3847140 0 0 0 151117 703 0 0 20 0 1 0 411675248 16253685760 3676453 33554432000 4194304 6210612 140730107778256 140730107751816 5454000 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 3968185 3676453 192 493 0 3961490 0
Current children cumulated CPU time: 1518.2 s
Current children cumulated vsize: 15872740 KiB
Current children cumulated memory: 15780172 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+1521.07 s]
# the end of solver process 16654 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=1511.2 s, system=8.60429 s

Solver just ended.
Dumping a history of the last processes samples

[startup+1518.5 s]
/proc/loadavg: 2.46 2.12 2.08 3/200 16880
/proc/meminfo: memFree=16347584/32770624 swapFree=65890380/67108860
[pid=16654] ppid=16651 vsize=15873004 memory=15780540 CPUtime=1518.24 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) D 16651 16654 10482 0 -1 512 3847761 0 83 0 151119 705 0 0 20 0 1 0 411675248 16253956096 3677176 33554432000 4194304 6210612 140730107778256 140730107751816 5188952 0 16384 4096 8405026 18446744071580419444 0 0 17 7 0 0 19 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 3968251 3677176 200 493 0 3961556 0
Current children cumulated CPU time: 1518.24 s
Current children cumulated vsize: 15873004 KiB
Current children cumulated memory: 15780540 KiB

[startup+1519.3 s]
/proc/loadavg: 2.46 2.12 2.08 2/201 16881
/proc/meminfo: memFree=16172404/32770624 swapFree=65890388/67108860
[pid=16654] ppid=16651 vsize=15873004 memory=15780540 CPUtime=1518.42 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) D 16651 16654 10482 0 -1 512 3863473 0 2411 0 151119 723 0 0 20 0 1 0 411675248 16253956096 3695194 33554432000 4194304 6210612 140730107778256 140730107751816 5188952 0 16384 4096 8405026 18446744071580419444 0 0 17 1 0 0 81 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 3968251 3695194 200 493 0 3961556 0
Current children cumulated CPU time: 1518.42 s
Current children cumulated vsize: 15873004 KiB
Current children cumulated memory: 15780540 KiB

[startup+1519.7 s]
/proc/loadavg: 2.46 2.12 2.08 2/201 16881
/proc/meminfo: memFree=16172404/32770624 swapFree=65890388/67108860
[pid=16654] ppid=16651 vsize=15873004 memory=15780540 CPUtime=1518.58 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) D 16651 16654 10482 0 -1 512 3871923 0 3664 0 151119 739 0 0 20 0 1 0 411675248 16253956096 3704896 33554432000 4194304 6210612 140730107778256 140730107751816 5188952 0 16384 4096 8405026 18446744071580419444 0 0 17 1 0 0 110 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 3968251 3704896 200 493 0 3961556 0
Current children cumulated CPU time: 1518.58 s
Current children cumulated vsize: 15873004 KiB
Current children cumulated memory: 15780540 KiB

[startup+1520.1 s]*
/proc/loadavg: 2.46 2.12 2.08 3/201 16881
/proc/meminfo: memFree=15887080/32770624 swapFree=65890388/67108860
[pid=16654] ppid=16651 vsize=15873004 memory=15780540 CPUtime=1518.87 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) D 16651 16654 10482 0 -1 512 3879804 0 4833 0 151119 768 0 0 20 0 1 0 411675248 16253956096 3713938 33554432000 4194304 6210612 140730107778256 140730107751816 5188952 0 16384 4096 8405026 18446744071580419444 0 0 17 1 0 0 143 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 3968251 3713938 200 493 0 3961556 0
Current children cumulated CPU time: 1518.87 s
Current children cumulated vsize: 15873004 KiB
Current children cumulated memory: 15780540 KiB

[startup+1520.21 s]
/proc/loadavg: 2.46 2.12 2.08 3/201 16881
/proc/meminfo: memFree=15887080/32770624 swapFree=65890388/67108860
[pid=16654] ppid=16651 vsize=15873004 memory=15780540 CPUtime=1518.87 cores=1,3,5,7
/proc/16654/stat : 16654 (cosoco) R 16651 16654 10482 0 -1 512 3882390 0 5217 0 151119 768 0 0 20 0 1 0 411675248 16253956096 3716908 33554432000 4194304 6210612 140730107778256 140730107751816 5188952 0 16384 4096 8405026 0 0 0 17 1 0 0 150 0 0 8308220 8313664 18026496 140730107786251 140730107786353 140730107786353 140730107789262 0
/proc/16654/statm: 3968251 3716908 200 493 0 3961556 0
Current children cumulated CPU time: 1518.87 s
Current children cumulated vsize: 15873004 KiB
Current children cumulated memory: 15780540 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1521.07
CPU time (s): 1519.8
CPU user time (s): 1511.2
CPU system time (s): 8.60429
CPU usage (%): 99.9167
Max. virtual memory (cumulated for all children) (KiB): 15873004
Max. memory (cumulated for all children) (KiB): 15780540

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1511.2
system time used= 8.60429
maximum resident set size= 14867788
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3882390
page faults= 5218
swaps= 0
block input operations= 338472
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 5379
involuntary context switches= 2734


# summary of solver processes directly reported to runsolver:
#   pid: 16654
#   total CPU time (s): 1519.8
#   total CPU user time (s): 1511.2
#   total CPU system time (s): 8.60429

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 5.20085 second user time and 10.4395 second system time

The end

Launcher Data 

Begin job on node125 at 2019-09-13 07:54:02
IDJOB=4390386
IDBENCH=139382
IDSOLVER=2831
FILE ID=node125/4390386-1568354084
RUNJOBID= node125-1568323660-10514
PBS_JOBID= 21701146
Free space on /tmp= 57376 MiB

SOLVER NAME= cosoco 2
BENCH NAME= XCSP17/SteelMillSlab/SteelMillSlab-bench_11_1.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4390386-1568354084/watcher-4390386-1568354084 -o /tmp/evaluation-result-4390386-1568354084/solver-4390386-1568354084 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node125-1568323660-10514 --watchdog 2560  cosoco -lc -model -restarts -stick HOME/instance-4390386-1568354084.xml

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 32630071a9cd02ab12e325390d6637b6
RANDOM SEED=504309533

node125.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.29
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.29
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.29
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.29
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        20664052 kB
MemAvailable:   31696120 kB
Buffers:          195076 kB
Cached:         11016836 kB
SwapCached:        66996 kB
Active:          1072716 kB
Inactive:       10310372 kB
Active(anon):     107900 kB
Inactive(anon):   103868 kB
Active(file):     964816 kB
Inactive(file): 10206504 kB
Unevictable:      108984 kB
Mlocked:          108984 kB
SwapTotal:      67108860 kB
SwapFree:       66940188 kB
Dirty:              2484 kB
Writeback:             0 kB
AnonPages:        244876 kB
Mapped:            45880 kB
Shmem:             32228 kB
Slab:             318556 kB
SReclaimable:     289208 kB
SUnreclaim:        29348 kB
KernelStack:        3648 kB
PageTables:         5216 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     628760 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     98304 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      114048 kB
DirectMap2M:    33431552 kB

Free space on /tmp at the end= 57216 MiB
End job on node125 at 2019-09-13 08:19:25