Trace number 4409749

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2.0 parallel? (TO) 2520.84 2523.23

General information on the benchmark

NameSteelMillSlab/
SteelMillSlab-bench_9_13.xml
MD5SUM3c8d95ae467c6bf4a4f8a8c9b5dc34f0
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkSAT TO
Best value of the objective obtained on this benchmark244
Best CPU time to get the best result obtained on this benchmark19937.7
Satisfiable
(Un)Satisfiability was proved
Number of variables12654
Number of constraints466
Number of domains4
Minimum domain size2
Maximum domain size111
Distribution of domain sizes[{"size":2,"count":12321},{"size":15,"count":111},{"size":50,"count":111},{"size":111,"count":111}]
Minimum variable degree2
Maximum variable degree114
Distribution of variable degrees[{"degree":2,"count":12432},{"degree":3,"count":111},{"degree":112,"count":78},{"degree":113,"count":24},{"degree":114,"count":9}]
Minimum constraint arity2
Maximum constraint arity112
Distribution of constraint arities[{"arity":2,"count":132},{"arity":111,"count":112},{"arity":112,"count":222}]
Number of extensional constraints0
Number of intensional constraints132
Distribution of constraint types[{"type":"intension","count":132},{"type":"sum","count":112},{"type":"element","count":111},{"type":"channel","count":111}]
Optimization problemYES
Type of objectivemin SUM

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (15319 MiB free)
  memory of node 1: 16384 MiB (15779 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4409749-1569838503/watcher-4409749-1569838503 -o /tmp/evaluation-result-4409749-1569838503/solver-4409749-1569838503 -C 20160 -W 2520 -M 31000 --bin-var /tmp/runsolver-exch-node123-1569830857-16206 --watchdog 2580 cosoco -model -nbcores=8 HOME/instance-4409749-1569838503.xml 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 20160 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 20190 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=17073, runsolver pid=17070
Current StackSize limit: 8192 KiB


[startup+0.100164 s]*
/proc/loadavg: 7.62 8.14 8.24 2/192 17074
/proc/meminfo: memFree=31809044/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=67748 memory=43620 CPUtime=0.1 cores=0-7
/proc/17073/stat : 17073 (cosoco) R 17070 17073 16179 0 -1 4202496 11098 0 0 0 9 1 0 0 20 0 1 0 560115057 69373952 10905 33554432000 4194304 6211132 140731242291968 140731242288504 140320973573093 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 16937 10905 754 493 0 10242 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 67748 KiB
Current children cumulated memory: 43620 KiB

[startup+0.209512 s]*
/proc/loadavg: 7.62 8.14 8.24 2/192 17074
/proc/meminfo: memFree=31780140/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=93348 memory=69492 CPUtime=0.21 cores=0-7
/proc/17073/stat : 17073 (cosoco) R 17070 17073 16179 0 -1 4202496 17537 0 0 0 19 2 0 0 20 0 1 0 560115057 95588352 17373 33554432000 4194304 6211132 140731242291968 140731242287848 140320973573093 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 23337 17373 809 493 0 16642 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 93348 KiB
Current children cumulated memory: 69492 KiB

[startup+0.309382 s]*
/proc/loadavg: 7.62 8.14 8.24 2/192 17074
/proc/meminfo: memFree=31778652/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=94664 memory=70812 CPUtime=0.31 cores=0-7
/proc/17073/stat : 17073 (cosoco) R 17070 17073 16179 0 -1 4202496 17902 0 0 0 29 2 0 0 20 0 1 0 560115057 96935936 17703 33554432000 4194304 6211132 140731242291968 140731242287848 140320973573093 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 23666 17703 819 493 0 16971 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 94664 KiB
Current children cumulated memory: 70812 KiB

[startup+0.700272 s]
/proc/loadavg: 7.62 8.14 8.24 2/192 17074
/proc/meminfo: memFree=31776748/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=98632 memory=74772 CPUtime=0.69 cores=0-7
/proc/17073/stat : 17073 (cosoco) R 17070 17073 16179 0 -1 4202496 18897 0 0 0 67 2 0 0 20 0 1 0 560115057 100999168 18693 33554432000 4194304 6211132 140731242291968 140731242287848 5013457 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 24691 18693 819 493 0 17996 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 98632 KiB
Current children cumulated memory: 74772 KiB

[startup+1.50027 s]
/proc/loadavg: 7.62 8.14 8.24 2/192 17074
/proc/meminfo: memFree=31771540/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=107216 memory=83096 CPUtime=1.49 cores=0-7
/proc/17073/stat : 17073 (cosoco) R 17070 17073 16179 0 -1 4202496 20978 0 0 0 147 2 0 0 20 0 1 0 560115057 109789184 20774 33554432000 4194304 6211132 140731242291968 140731242287848 140320973573093 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 26804 20774 819 493 0 20109 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 107216 KiB
Current children cumulated memory: 83096 KiB

[startup+3.10923 s]
/proc/loadavg: 7.62 8.14 8.24 2/192 17074
/proc/meminfo: memFree=31750628/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=124032 memory=99852 CPUtime=3.09 cores=0-7
/proc/17073/stat : 17073 (cosoco) R 17070 17073 16179 0 -1 4202496 25407 0 0 0 307 2 0 0 20 0 1 0 560115057 127008768 24963 33554432000 4194304 6211132 140731242291968 140731242265528 140320973573093 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 31008 24963 819 493 0 24313 0
Current children cumulated CPU time: 3.09 s
Current children cumulated vsize: 124032 KiB
Current children cumulated memory: 99852 KiB

[startup+6.30026 s]
/proc/loadavg: 7.09 8.02 8.20 2/192 17074
/proc/meminfo: memFree=31719136/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=157840 memory=133036 CPUtime=6.28 cores=0-7
/proc/17073/stat : 17073 (cosoco) R 17070 17073 16179 0 -1 4202496 33953 0 0 0 625 3 0 0 20 0 1 0 560115057 161628160 33259 33554432000 4194304 6211132 140731242291968 140731242265528 140320973573093 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 39460 33259 819 493 0 32765 0
Current children cumulated CPU time: 6.28 s
Current children cumulated vsize: 157840 KiB
Current children cumulated memory: 133036 KiB

[startup+12.7003 s]
/proc/loadavg: 6.60 7.90 8.17 2/192 17074
/proc/meminfo: memFree=31656588/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=223828 memory=199828 CPUtime=12.68 cores=0-7
/proc/17073/stat : 17073 (cosoco) R 17070 17073 16179 0 -1 4202496 50801 0 0 0 1262 6 0 0 20 0 1 0 560115057 229199872 49957 33554432000 4194304 6211132 140731242291968 140731242265528 5442980 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 55957 49957 819 493 0 49262 0
Current children cumulated CPU time: 12.68 s
Current children cumulated vsize: 223828 KiB
Current children cumulated memory: 199828 KiB

[startup+25.5003 s]
/proc/loadavg: 5.36 7.57 8.05 2/192 17075
/proc/meminfo: memFree=31520044/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=358588 memory=334376 CPUtime=25.48 cores=0-7
/proc/17073/stat : 17073 (cosoco) R 17070 17073 16179 0 -1 4202496 84265 0 0 0 2538 10 0 0 20 0 1 0 560115057 367194112 83594 33554432000 4194304 6211132 140731242291968 140731242265528 5446311 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 89647 83594 819 493 0 82952 0
Current children cumulated CPU time: 25.48 s
Current children cumulated vsize: 358588 KiB
Current children cumulated memory: 334376 KiB

[startup+51.1092 s]
/proc/loadavg: 3.87 7.04 7.86 2/191 17077
/proc/meminfo: memFree=31248364/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=629856 memory=601060 CPUtime=51.08 cores=0-7
/proc/17073/stat : 17073 (cosoco) R 17070 17073 16179 0 -1 4202496 149975 0 0 0 5089 19 0 0 20 0 1 0 560115057 644972544 150265 33554432000 4194304 6211132 140731242291968 140731242265528 5446959 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 157464 150265 819 493 0 150769 0
Current children cumulated CPU time: 51.08 s
Current children cumulated vsize: 629856 KiB
Current children cumulated memory: 601060 KiB

[startup+102.3 s]
/proc/loadavg: 2.25 6.11 7.50 2/191 17080
/proc/meminfo: memFree=30713440/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=1176616 memory=1139100 CPUtime=102.25 cores=0-7
/proc/17073/stat : 17073 (cosoco) R 17070 17073 16179 0 -1 4202496 280892 0 0 0 10189 36 0 0 20 0 1 0 560115057 1204854784 284775 33554432000 4194304 6211132 140731242291968 140731242265528 140320973573093 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 294154 284775 819 493 0 287459 0
Current children cumulated CPU time: 102.25 s
Current children cumulated vsize: 1176616 KiB
Current children cumulated memory: 1139100 KiB

[startup+162.3 s]
/proc/loadavg: 1.46 5.18 7.10 2/191 17083
/proc/meminfo: memFree=30083016/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=1810136 memory=1766376 CPUtime=162.22 cores=0-7
/proc/17073/stat : 17073 (cosoco) R 17070 17073 16179 0 -1 4202496 433675 0 0 0 16168 54 0 0 20 0 1 0 560115057 1853579264 441594 33554432000 4194304 6211132 140731242291968 140731242265528 5479676 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 452534 441594 819 493 0 445839 0
Current children cumulated CPU time: 162.22 s
Current children cumulated vsize: 1810136 KiB
Current children cumulated memory: 1766376 KiB

[startup+222.3 s]
/proc/loadavg: 1.17 4.42 6.71 2/191 17088
/proc/meminfo: memFree=29454200/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=2418920 memory=2393852 CPUtime=222.21 cores=0-7
/proc/17073/stat : 17073 (cosoco) R 17070 17073 16179 0 -1 4202496 586045 0 0 0 22148 73 0 0 20 0 1 0 560115057 2476974080 598463 33554432000 4194304 6211132 140731242291968 140731242265528 140320973573093 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 604730 598463 819 493 0 598035 0
Current children cumulated CPU time: 222.21 s
Current children cumulated vsize: 2418920 KiB
Current children cumulated memory: 2393852 KiB

[startup+282.3 s]
/proc/loadavg: 1.06 3.80 6.36 2/191 17091
/proc/meminfo: memFree=28824308/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=3064616 memory=3022960 CPUtime=282.19 cores=0-7
/proc/17073/stat : 17073 (cosoco) R 17070 17073 16179 0 -1 4202496 738776 0 0 0 28124 95 0 0 20 0 1 0 560115057 3138166784 755740 33554432000 4194304 6211132 140731242291968 140731242265528 140320973573093 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 766154 755740 819 493 0 759459 0
Current children cumulated CPU time: 282.19 s
Current children cumulated vsize: 3064616 KiB
Current children cumulated memory: 3022960 KiB

[startup+342.3 s]
/proc/loadavg: 1.02 3.29 6.02 2/191 17101
/proc/meminfo: memFree=28193748/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=3728632 memory=3652096 CPUtime=342.17 cores=0-7
/proc/17073/stat : 17073 (cosoco) R 17070 17073 16179 0 -1 4202496 891565 0 0 0 34103 114 0 0 20 0 1 0 560115057 3818119168 913024 33554432000 4194304 6211132 140731242291968 140731242265528 140320973573093 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 932158 913024 819 493 0 925463 0
Current children cumulated CPU time: 342.17 s
Current children cumulated vsize: 3728632 KiB
Current children cumulated memory: 3652096 KiB

[startup+402.3 s]
/proc/loadavg: 1.01 2.87 5.71 2/191 17106
/proc/meminfo: memFree=27564832/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=4337416 memory=4279292 CPUtime=402.15 cores=0-7
/proc/17073/stat : 17073 (cosoco) R 17070 17073 16179 0 -1 4202496 1043901 0 0 0 40079 136 0 0 20 0 1 0 560115057 4441513984 1069823 33554432000 4194304 6211132 140731242291968 140731242265528 140320973573093 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 1084354 1069823 819 493 0 1077659 0
Current children cumulated CPU time: 402.15 s
Current children cumulated vsize: 4337416 KiB
Current children cumulated memory: 4279292 KiB

[startup+462.3 s]
/proc/loadavg: 1.00 2.53 5.41 2/191 17109
/proc/meminfo: memFree=26934832/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=4946068 memory=4908348 CPUtime=462.12 cores=0-7
/proc/17073/stat : 17073 (cosoco) R 17070 17073 16179 0 -1 4202496 1196230 0 0 0 46054 158 0 0 20 0 1 0 560115057 5064773632 1227087 33554432000 4194304 6211132 140731242291968 140731242265528 140320973573093 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 1236517 1227087 819 493 0 1229822 0
Current children cumulated CPU time: 462.12 s
Current children cumulated vsize: 4946068 KiB
Current children cumulated memory: 4908348 KiB


################
# More data... #
################

Current children cumulated vsize: 20202988 KiB
Current children cumulated memory: 19960232 KiB

[startup+1962.3 s]
/proc/loadavg: 1.00 1.01 1.88 2/191 17218
/proc/meminfo: memFree=11223964/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=20809132 memory=20587056 CPUtime=1961.75 cores=0-7
/proc/17073/stat : 17073 (cosoco) R 17070 17073 16179 0 -1 4202496 5002699 0 0 0 195194 981 0 0 20 0 1 0 560115057 21308551168 5146764 33554432000 4194304 6211132 140731242291968 140731242265528 140320973573093 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 5202283 5146764 819 493 0 5195588 0
Current children cumulated CPU time: 1961.75 s
Current children cumulated vsize: 20809132 KiB
Current children cumulated memory: 20587056 KiB

[startup+2022.3 s]
/proc/loadavg: 1.00 1.01 1.82 2/191 17223
/proc/meminfo: memFree=10598104/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=21415276 memory=21211492 CPUtime=2021.74 cores=0-7
/proc/17073/stat : 17073 (cosoco) R 17070 17073 16179 0 -1 4202496 5154516 0 0 0 201146 1028 0 0 20 0 1 0 560115057 21929242624 5302873 33554432000 4194304 6211132 140731242291968 140731242265528 5440687 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 5353819 5302873 819 493 0 5347124 0
Current children cumulated CPU time: 2021.74 s
Current children cumulated vsize: 21415276 KiB
Current children cumulated memory: 21211492 KiB

[startup+2082.3 s]
/proc/loadavg: 1.00 1.01 1.77 2/191 17226
/proc/meminfo: memFree=9971036/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=22021156 memory=21837972 CPUtime=2081.73 cores=0-7
/proc/17073/stat : 17073 (cosoco) R 17070 17073 16179 0 -1 4202496 5306295 0 0 0 207100 1073 0 0 20 0 1 0 560115057 22549663744 5459493 33554432000 4194304 6211132 140731242291968 140731242265528 140320973573093 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 5505289 5459493 819 493 0 5498594 0
Current children cumulated CPU time: 2081.73 s
Current children cumulated vsize: 22021156 KiB
Current children cumulated memory: 21837972 KiB

[startup+2142.3 s]
/proc/loadavg: 1.00 1.01 1.72 2/192 17237
/proc/meminfo: memFree=9344808/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=22627168 memory=22462440 CPUtime=2141.71 cores=0-7
/proc/17073/stat : 17073 (cosoco) R 17070 17073 16179 0 -1 4202496 5458142 0 0 0 213050 1121 0 0 20 0 1 0 560115057 23170220032 5615610 33554432000 4194304 6211132 140731242291968 140731242265528 140320973573093 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 5656792 5615610 819 493 0 5650097 0
Current children cumulated CPU time: 2141.71 s
Current children cumulated vsize: 22627168 KiB
Current children cumulated memory: 22462440 KiB

[startup+2202.3 s]
/proc/loadavg: 1.00 1.01 1.68 2/192 17242
/proc/meminfo: memFree=8717364/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=23232916 memory=23088656 CPUtime=2201.69 cores=0-7
/proc/17073/stat : 17073 (cosoco) R 17070 17073 16179 0 -1 4202496 5609893 0 0 0 219000 1169 0 0 20 0 1 0 560115057 23790505984 5772164 33554432000 4194304 6211132 140731242291968 140731242265528 140320973573093 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 5808229 5772164 819 493 0 5801534 0
Current children cumulated CPU time: 2201.69 s
Current children cumulated vsize: 23232916 KiB
Current children cumulated memory: 23088656 KiB

[startup+2262.3 s]
/proc/loadavg: 1.00 1.01 1.64 2/192 17245
/proc/meminfo: memFree=8090408/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=23838268 memory=23714376 CPUtime=2261.68 cores=0-7
/proc/17073/stat : 17073 (cosoco) R 17070 17073 16179 0 -1 4202496 5761574 0 0 0 224948 1220 0 0 20 0 1 0 560115057 24410386432 5928594 33554432000 4194304 6211132 140731242291968 140731242265528 140320973573093 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 5959567 5928594 819 493 0 5952872 0
Current children cumulated CPU time: 2261.68 s
Current children cumulated vsize: 23838268 KiB
Current children cumulated memory: 23714376 KiB

[startup+2322.3 s]
/proc/loadavg: 1.00 1.01 1.59 2/192 17248
/proc/meminfo: memFree=7465404/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=24443884 memory=24338548 CPUtime=2321.67 cores=0-7
/proc/17073/stat : 17073 (cosoco) R 17070 17073 16179 0 -1 4202496 5913292 0 0 0 230900 1267 0 0 20 0 1 0 560115057 25030537216 6084637 33554432000 4194304 6211132 140731242291968 140731242265528 5451514 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 6110971 6084637 819 493 0 6104276 0
Current children cumulated CPU time: 2321.67 s
Current children cumulated vsize: 24443884 KiB
Current children cumulated memory: 24338548 KiB

[startup+2382.3 s]
/proc/loadavg: 1.00 1.01 1.56 2/192 17253
/proc/meminfo: memFree=6838072/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=25049368 memory=24964532 CPUtime=2381.65 cores=0-7
/proc/17073/stat : 17073 (cosoco) R 17070 17073 16179 0 -1 4202496 6065013 0 0 0 236847 1318 0 0 20 0 1 0 560115057 25650552832 6241133 33554432000 4194304 6211132 140731242291968 140731242265528 140320973573093 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 6262342 6241133 819 493 0 6255647 0
Current children cumulated CPU time: 2381.65 s
Current children cumulated vsize: 25049368 KiB
Current children cumulated memory: 24964532 KiB

[startup+2442.3 s]
/proc/loadavg: 1.00 1.01 1.52 2/191 17257
/proc/meminfo: memFree=6213620/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=25654588 memory=25588128 CPUtime=2441.63 cores=0-7
/proc/17073/stat : 17073 (cosoco) R 17070 17073 16179 0 -1 4202496 6216618 0 0 0 242794 1369 0 0 20 0 1 0 560115057 26270298112 6397032 33554432000 4194304 6211132 140731242291968 140731242265528 140320973573093 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 6413647 6397032 819 493 0 6406952 0
Current children cumulated CPU time: 2441.63 s
Current children cumulated vsize: 25654588 KiB
Current children cumulated memory: 25588128 KiB

[startup+2502.3 s]
/proc/loadavg: 1.00 1.01 1.49 2/191 17260
/proc/meminfo: memFree=5587176/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=26259808 memory=26213632 CPUtime=2501.63 cores=0-7
/proc/17073/stat : 17073 (cosoco) R 17070 17073 16179 0 -1 4202496 6368268 0 0 0 248740 1423 0 0 20 0 1 0 560115057 26890043392 6553408 33554432000 4194304 6211132 140731242291968 140731242265528 140320973573093 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 6564952 6553408 819 493 0 6558257 0
Current children cumulated CPU time: 2501.63 s
Current children cumulated vsize: 26259808 KiB
Current children cumulated memory: 26213632 KiB



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+2520 s]*
/proc/loadavg: 1.00 1.01 1.48 2/191 17262
/proc/meminfo: memFree=5399004/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=26438404 memory=26398556 CPUtime=2519.33 cores=0-7
/proc/17073/stat : 17073 (cosoco) R 17070 17073 16179 0 -1 4202496 6413062 0 0 0 250496 1437 0 0 20 0 1 0 560115057 27072925696 6599639 33554432000 4194304 6211132 140731242291968 140731242265528 140320973573093 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 6609601 6599639 819 493 0 6602906 0
Current children cumulated CPU time: 2519.33 s
Current children cumulated vsize: 26438404 KiB
Current children cumulated memory: 26398556 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+2523.23 s]
# the end of solver process 17073 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2505.25 s, system=15.5999 s

Solver just ended.
Dumping a history of the last processes samples

[startup+2520.8 s]
/proc/loadavg: 1.00 1.01 1.48 2/191 17262
/proc/meminfo: memFree=5399004/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=26438404 memory=26398556 CPUtime=2519.44 cores=0-7
/proc/17073/stat : 17073 (cosoco) S 17070 17073 16179 0 -1 512 6413068 0 0 0 250507 1437 0 0 20 0 1 0 560115057 27072925696 6599639 33554432000 4194304 6211132 140731242291968 140731242265528 5189582 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 6609601 6599639 819 493 0 6602906 0
Current children cumulated CPU time: 2519.44 s
Current children cumulated vsize: 26438404 KiB
Current children cumulated memory: 26398556 KiB

[startup+2521.6 s]
/proc/loadavg: 1.00 1.01 1.48 2/192 17263
/proc/meminfo: memFree=4862192/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=26438404 memory=26398556 CPUtime=2519.56 cores=0-7
/proc/17073/stat : 17073 (cosoco) S 17070 17073 16179 0 -1 512 6413068 0 0 0 250518 1438 0 0 20 0 1 0 560115057 27072925696 6599639 33554432000 4194304 6211132 140731242291968 140731242265528 5189582 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 6609601 6599639 819 493 0 6602906 0
Current children cumulated CPU time: 2519.56 s
Current children cumulated vsize: 26438404 KiB
Current children cumulated memory: 26398556 KiB

[startup+2522 s]*
/proc/loadavg: 1.00 1.01 1.48 3/192 17263
/proc/meminfo: memFree=4363952/32770624 swapFree=66939156/67108860
[pid=17073] ppid=17070 vsize=26438404 memory=26398556 CPUtime=2519.62 cores=0-7
/proc/17073/stat : 17073 (cosoco) S 17070 17073 16179 0 -1 512 6413068 0 0 0 250524 1438 0 0 20 0 1 0 560115057 27072925696 6599639 33554432000 4194304 6211132 140731242291968 140731242265528 5189582 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 0 0 0 8312316 8317760 30470144 140731242300433 140731242300525 140731242300525 140731242303438 0
/proc/17073/statm: 6609601 6599639 819 493 0 6602906 0
Current children cumulated CPU time: 2519.62 s
Current children cumulated vsize: 26438404 KiB
Current children cumulated memory: 26398556 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 2523.23
CPU time (s): 2520.84
CPU user time (s): 2505.25
CPU system time (s): 15.5999
CPU usage (%): 99.9056
Max. virtual memory (cumulated for all children) (KiB): 26438404
Max. memory (cumulated for all children) (KiB): 26398556

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2505.25
system time used= 15.5999
maximum resident set size= 26398700
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 6413068
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 128755
involuntary context switches= 1944


# summary of solver processes directly reported to runsolver:
#   pid: 17073
#   total CPU time (s): 2520.84
#   total CPU user time (s): 2505.25
#   total CPU system time (s): 15.5999

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 8.5526 second user time and 17.0075 second system time

The end

Launcher Data 

Begin job on node123 at 2019-09-30 12:14:23
IDJOB=4409749
IDBENCH=139552
IDSOLVER=2877
FILE ID=node123/4409749-1569838503
RUNJOBID= node123-1569830857-16206
PBS_JOBID= 21705947
Free space on /tmp= 62032 MiB

SOLVER NAME= cosoco 2.0 parallel
BENCH NAME= XCSP17/SteelMillSlab/SteelMillSlab-bench_9_13.xml
COMMAND LINE= cosoco -model -nbcores=NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4409749-1569838503/watcher-4409749-1569838503 -o /tmp/evaluation-result-4409749-1569838503/solver-4409749-1569838503 -C 20160 -W 2520 -M 31000 --bin-var /tmp/runsolver-exch-node123-1569830857-16206 --watchdog 2580  cosoco -model -nbcores=8 HOME/instance-4409749-1569838503.xml

TIME LIMIT= 20160 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 3c8d95ae467c6bf4a4f8a8c9b5dc34f0
RANDOM SEED=1300980860

node123.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        31845456 kB
MemAvailable:   31813896 kB
Buffers:           14424 kB
Cached:           268532 kB
SwapCached:        63940 kB
Active:            68168 kB
Inactive:         282968 kB
Active(anon):       8448 kB
Inactive(anon):   137260 kB
Active(file):      59720 kB
Inactive(file):   145708 kB
Unevictable:      109864 kB
Mlocked:          109864 kB
SwapTotal:      67108860 kB
SwapFree:       66939156 kB
Dirty:              2496 kB
Writeback:             0 kB
AnonPages:        114228 kB
Mapped:            26492 kB
Shmem:             69132 kB
Slab:             167584 kB
SReclaimable:     137868 kB
SUnreclaim:        29716 kB
KernelStack:        3376 kB
PageTables:         5392 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     540472 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     67584 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      105856 kB
DirectMap2M:    33439744 kB

Free space on /tmp at the end= 61532 MiB
End job on node123 at 2019-09-30 12:56:26