Trace number 4398470

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2.O parallel? (TO) 2520.77 2523.21

General information on the benchmark

NameSteelMillSlab/
SteelMillSlab-bench_16_2.xml
MD5SUM8e5b61791e512734038a023d36a5924c
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkSAT TO
Best value of the objective obtained on this benchmark69
Best CPU time to get the best result obtained on this benchmark20045.8
Satisfiable
(Un)Satisfiability was proved
Number of variables12654
Number of constraints466
Number of domains4
Minimum domain size2
Maximum domain size111
Distribution of domain sizes[{"size":2,"count":12321},{"size":10,"count":111},{"size":51,"count":111},{"size":111,"count":111}]
Minimum variable degree2
Maximum variable degree114
Distribution of variable degrees[{"degree":2,"count":12432},{"degree":3,"count":111},{"degree":112,"count":78},{"degree":113,"count":24},{"degree":114,"count":9}]
Minimum constraint arity2
Maximum constraint arity112
Distribution of constraint arities[{"arity":2,"count":132},{"arity":111,"count":112},{"arity":112,"count":222}]
Number of extensional constraints0
Number of intensional constraints132
Distribution of constraint types[{"type":"intension","count":132},{"type":"sum","count":112},{"type":"element","count":111},{"type":"channel","count":111}]
Optimization problemYES
Type of objectivemin SUM

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (15326 MiB free)
  memory of node 1: 16384 MiB (15859 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4398470-1568614101/watcher-4398470-1568614101 -o /tmp/evaluation-result-4398470-1568614101/solver-4398470-1568614101 -C 20160 -W 2520 -M 31000 --bin-var /tmp/runsolver-exch-node136-1568611536-8725 --watchdog 2580 cosoco -model -nbcores=8 HOME/instance-4398470-1568614101.xml 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 20160 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 20190 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
Current StackSize limit: 8192 KiB

solver pid=9026, runsolver pid=9023

[startup+0.100166 s]*
/proc/loadavg: 8.61 8.37 8.14 2/191 9027
/proc/meminfo: memFree=31898068/32770624 swapFree=67001848/67108860
[pid=9026] ppid=9023 vsize=68644 memory=44512 CPUtime=0.1 cores=0-7
/proc/9026/stat : 9026 (cosoco) R 9023 9026 8695 0 -1 4202496 11351 0 0 0 9 1 0 0 20 0 1 0 437658769 70291456 11128 33554432000 4194304 6211092 140730548735232 140730548731768 139975415573477 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 17194 11128 754 493 0 10499 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 68644 KiB
Current children cumulated memory: 44512 KiB

[startup+0.209427 s]*
/proc/loadavg: 8.61 8.37 8.14 2/191 9027
/proc/meminfo: memFree=31869052/32770624 swapFree=67001848/67108860
[pid=9026] ppid=9023 vsize=93320 memory=69328 CPUtime=0.21 cores=0-7
/proc/9026/stat : 9026 (cosoco) R 9023 9026 8695 0 -1 4202496 17547 0 0 0 19 2 0 0 20 0 1 0 437658769 95559680 17332 33554432000 4194304 6211092 140730548735232 140730548730632 139975415573477 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 23330 17332 809 493 0 16635 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 93320 KiB
Current children cumulated memory: 69328 KiB

[startup+0.309331 s]*
/proc/loadavg: 8.61 8.37 8.14 2/191 9027
/proc/meminfo: memFree=31867564/32770624 swapFree=67001848/67108860
[pid=9026] ppid=9023 vsize=94760 memory=70808 CPUtime=0.31 cores=0-7
/proc/9026/stat : 9026 (cosoco) R 9023 9026 8695 0 -1 4202496 17905 0 0 0 29 2 0 0 20 0 1 0 437658769 97034240 17702 33554432000 4194304 6211092 140730548735232 140730548730632 139975416499630 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 23690 17702 819 493 0 16995 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 94760 KiB
Current children cumulated memory: 70808 KiB

[startup+0.700272 s]
/proc/loadavg: 8.61 8.37 8.14 2/191 9027
/proc/meminfo: memFree=31865704/32770624 swapFree=67001848/67108860
[pid=9026] ppid=9023 vsize=98736 memory=74768 CPUtime=0.69 cores=0-7
/proc/9026/stat : 9026 (cosoco) R 9023 9026 8695 0 -1 4202496 18899 0 0 0 67 2 0 0 20 0 1 0 437658769 101105664 18692 33554432000 4194304 6211092 140730548735232 140730548708792 5454160 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 24684 18692 819 493 0 17989 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 98736 KiB
Current children cumulated memory: 74768 KiB

[startup+1.50028 s]
/proc/loadavg: 8.61 8.37 8.14 3/191 9027
/proc/meminfo: memFree=31860496/32770624 swapFree=67001848/67108860
[pid=9026] ppid=9023 vsize=107188 memory=83068 CPUtime=1.49 cores=0-7
/proc/9026/stat : 9026 (cosoco) R 9023 9026 8695 0 -1 4202496 20977 0 0 0 147 2 0 0 20 0 1 0 437658769 109760512 20767 33554432000 4194304 6211092 140730548735232 140730548708792 5452242 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 26797 20767 819 493 0 20102 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 107188 KiB
Current children cumulated memory: 83068 KiB

[startup+3.10922 s]
/proc/loadavg: 8.00 8.25 8.10 2/191 9027
/proc/meminfo: memFree=31840288/32770624 swapFree=67001848/67108860
[pid=9026] ppid=9023 vsize=124004 memory=99828 CPUtime=3.1 cores=0-7
/proc/9026/stat : 9026 (cosoco) R 9023 9026 8695 0 -1 4202496 25409 0 0 0 308 2 0 0 20 0 1 0 437658769 126980096 24957 33554432000 4194304 6211092 140730548735232 140730548708792 139975415573477 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 31001 24957 819 493 0 24306 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 124004 KiB
Current children cumulated memory: 99828 KiB

[startup+6.30027 s]
/proc/loadavg: 8.00 8.25 8.10 3/191 9027
/proc/meminfo: memFree=31809456/32770624 swapFree=67001848/67108860
[pid=9026] ppid=9023 vsize=157680 memory=133000 CPUtime=6.28 cores=0-7
/proc/9026/stat : 9026 (cosoco) R 9023 9026 8695 0 -1 4202496 33935 0 0 0 626 2 0 0 20 0 1 0 437658769 161464320 33250 33554432000 4194304 6211092 140730548735232 140730548708792 5013083 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 39420 33250 819 493 0 32725 0
Current children cumulated CPU time: 6.28 s
Current children cumulated vsize: 157680 KiB
Current children cumulated memory: 133000 KiB

[startup+12.7003 s]
/proc/loadavg: 7.44 8.13 8.06 2/191 9027
/proc/meminfo: memFree=31747184/32770624 swapFree=67001848/67108860
[pid=9026] ppid=9023 vsize=223536 memory=199536 CPUtime=12.69 cores=0-7
/proc/9026/stat : 9026 (cosoco) R 9023 9026 8695 0 -1 4202496 50809 0 0 0 1264 5 0 0 20 0 1 0 437658769 228900864 49884 33554432000 4194304 6211092 140730548735232 140730548708792 5451570 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 55884 49884 819 493 0 49189 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 223536 KiB
Current children cumulated memory: 199536 KiB

[startup+25.5003 s]
/proc/loadavg: 6.02 7.78 7.95 2/192 9029
/proc/meminfo: memFree=31612056/32770624 swapFree=67001848/67108860
[pid=9026] ppid=9023 vsize=357636 memory=333256 CPUtime=25.48 cores=0-7
/proc/9026/stat : 9026 (cosoco) R 9023 9026 8695 0 -1 4202496 84227 0 0 0 2539 9 0 0 20 0 1 0 437658769 366219264 83314 33554432000 4194304 6211092 140730548735232 140730548708792 139975415573477 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 89409 83314 819 493 0 82714 0
Current children cumulated CPU time: 25.48 s
Current children cumulated vsize: 357636 KiB
Current children cumulated memory: 333256 KiB

[startup+51.1092 s]
/proc/loadavg: 4.30 7.24 7.76 2/191 9030
/proc/meminfo: memFree=31342328/32770624 swapFree=67001848/67108860
[pid=9026] ppid=9023 vsize=628244 memory=599236 CPUtime=51.08 cores=0-7
/proc/9026/stat : 9026 (cosoco) R 9023 9026 8695 0 -1 4202496 150315 0 0 0 5089 19 0 0 20 0 1 0 437658769 643321856 149809 33554432000 4194304 6211092 140730548735232 140730548708792 139975415573477 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 157061 149809 819 493 0 150366 0
Current children cumulated CPU time: 51.08 s
Current children cumulated vsize: 628244 KiB
Current children cumulated memory: 599236 KiB

[startup+102.3 s]
/proc/loadavg: 2.43 6.28 7.41 2/191 9033
/proc/meminfo: memFree=30807656/32770624 swapFree=67001848/67108860
[pid=9026] ppid=9023 vsize=1172892 memory=1135112 CPUtime=102.25 cores=0-7
/proc/9026/stat : 9026 (cosoco) R 9023 9026 8695 0 -1 4202496 280702 0 0 0 10192 33 0 0 20 0 1 0 437658769 1201041408 283778 33554432000 4194304 6211092 140730548735232 140730548708792 5451618 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 293223 283778 819 493 0 286528 0
Current children cumulated CPU time: 102.25 s
Current children cumulated vsize: 1172892 KiB
Current children cumulated memory: 1135112 KiB

[startup+162.3 s]
/proc/loadavg: 1.53 5.32 7.01 2/193 9047
/proc/meminfo: memFree=30181416/32770624 swapFree=67001848/67108860
[pid=9026] ppid=9023 vsize=1803772 memory=1759732 CPUtime=162.23 cores=0-7
/proc/9026/stat : 9026 (cosoco) R 9023 9026 8695 0 -1 4202496 432918 0 0 0 16171 52 0 0 20 0 1 0 437658769 1847062528 439933 33554432000 4194304 6211092 140730548735232 140730548708792 5451477 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 450943 439933 819 493 0 444248 0
Current children cumulated CPU time: 162.23 s
Current children cumulated vsize: 1803772 KiB
Current children cumulated memory: 1759732 KiB

[startup+222.3 s]
/proc/loadavg: 1.19 4.53 6.63 2/193 9050
/proc/meminfo: memFree=29555372/32770624 swapFree=67001848/67108860
[pid=9026] ppid=9023 vsize=2410048 memory=2384816 CPUtime=222.21 cores=0-7
/proc/9026/stat : 9026 (cosoco) R 9023 9026 8695 0 -1 4202496 584718 0 0 0 22149 72 0 0 20 0 1 0 437658769 2467889152 596204 33554432000 4194304 6211092 140730548735232 140730548708792 139975416497362 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 602512 596204 819 493 0 595817 0
Current children cumulated CPU time: 222.21 s
Current children cumulated vsize: 2410048 KiB
Current children cumulated memory: 2384816 KiB

[startup+282.3 s]
/proc/loadavg: 1.07 3.89 6.28 2/193 9053
/proc/meminfo: memFree=28927872/32770624 swapFree=67001848/67108860
[pid=9026] ppid=9023 vsize=3053236 memory=3011312 CPUtime=282.19 cores=0-7
/proc/9026/stat : 9026 (cosoco) R 9023 9026 8695 0 -1 4202496 736908 0 0 0 28126 93 0 0 20 0 1 0 437658769 3126513664 752828 33554432000 4194304 6211092 140730548735232 140730548708792 5012962 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 763309 752828 819 493 0 756614 0
Current children cumulated CPU time: 282.19 s
Current children cumulated vsize: 3053236 KiB
Current children cumulated memory: 3011312 KiB

[startup+342.3 s]
/proc/loadavg: 1.03 3.36 5.95 2/193 9058
/proc/meminfo: memFree=28299320/32770624 swapFree=67001848/67108860
[pid=9026] ppid=9023 vsize=3715008 memory=3638300 CPUtime=342.17 cores=0-7
/proc/9026/stat : 9026 (cosoco) R 9023 9026 8695 0 -1 4202496 889249 0 0 0 34104 113 0 0 20 0 1 0 437658769 3804168192 909575 33554432000 4194304 6211092 140730548735232 140730548708792 5452226 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 928752 909575 819 493 0 922057 0
Current children cumulated CPU time: 342.17 s
Current children cumulated vsize: 3715008 KiB
Current children cumulated memory: 3638300 KiB

[startup+402.3 s]
/proc/loadavg: 1.01 2.93 5.64 2/193 9061
/proc/meminfo: memFree=27673572/32770624 swapFree=67001848/67108860
[pid=9026] ppid=9023 vsize=4321284 memory=4262828 CPUtime=402.15 cores=0-7
/proc/9026/stat : 9026 (cosoco) R 9023 9026 8695 0 -1 4202496 1041107 0 0 0 40081 134 0 0 20 0 1 0 437658769 4424994816 1065707 33554432000 4194304 6211092 140730548735232 140730548708792 139975416497366 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 1080321 1065707 819 493 0 1073626 0
Current children cumulated CPU time: 402.15 s
Current children cumulated vsize: 4321284 KiB
Current children cumulated memory: 4262828 KiB

[startup+462.3 s]
/proc/loadavg: 1.00 2.58 5.35 2/191 9064
/proc/meminfo: memFree=27046364/32770624 swapFree=67001848/67108860
[pid=9026] ppid=9023 vsize=4927560 memory=4889724 CPUtime=462.13 cores=0-7
/proc/9026/stat : 9026 (cosoco) R 9023 9026 8695 0 -1 4202496 1193054 0 0 0 46059 154 0 0 20 0 1 0 437658769 5045821440 1222431 33554432000 4194304 6211092 140730548735232 140730548708792 5451482 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 1231890 1222431 819 493 0 1225195 0
Current children cumulated CPU time: 462.13 s
Current children cumulated vsize: 4927560 KiB
Current children cumulated memory: 4889724 KiB


################
# More data... #
################

Current children cumulated vsize: 20123892 KiB
Current children cumulated memory: 19879028 KiB

[startup+1962.3 s]
/proc/loadavg: 1.00 1.02 1.87 2/192 9184
/proc/meminfo: memFree=11401388/32770624 swapFree=67001860/67108860
[pid=9026] ppid=9023 vsize=20727528 memory=20503044 CPUtime=1961.63 cores=0-7
/proc/9026/stat : 9026 (cosoco) R 9023 9026 8695 0 -1 4202496 4991800 0 0 0 195241 922 0 0 20 0 1 0 437658769 21224988672 5125761 33554432000 4194304 6211092 140730548735232 140730548708792 5479594 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 5181882 5125761 819 493 0 5175187 0
Current children cumulated CPU time: 1961.63 s
Current children cumulated vsize: 20727528 KiB
Current children cumulated memory: 20503044 KiB

[startup+2022.3 s]
/proc/loadavg: 1.00 1.01 1.82 2/192 9187
/proc/meminfo: memFree=10778584/32770624 swapFree=67001860/67108860
[pid=9026] ppid=9023 vsize=21331164 memory=21125312 CPUtime=2021.62 cores=0-7
/proc/9026/stat : 9026 (cosoco) R 9023 9026 8695 0 -1 4202496 5143507 0 0 0 201194 968 0 0 20 0 1 0 437658769 21843111936 5281328 33554432000 4194304 6211092 140730548735232 140730548708792 5451548 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 5332791 5281328 819 493 0 5326096 0
Current children cumulated CPU time: 2021.62 s
Current children cumulated vsize: 21331164 KiB
Current children cumulated memory: 21125312 KiB

[startup+2082.3 s]
/proc/loadavg: 1.00 1.01 1.77 2/192 9190
/proc/meminfo: memFree=10152288/32770624 swapFree=67001860/67108860
[pid=9026] ppid=9023 vsize=21934800 memory=21749336 CPUtime=2081.62 cores=0-7
/proc/9026/stat : 9026 (cosoco) R 9023 9026 8695 0 -1 4202496 5295060 0 0 0 207149 1013 0 0 20 0 1 0 437658769 22461235200 5437334 33554432000 4194304 6211092 140730548735232 140730548708792 139975416499648 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 5483700 5437334 819 493 0 5477005 0
Current children cumulated CPU time: 2081.62 s
Current children cumulated vsize: 21934800 KiB
Current children cumulated memory: 21749336 KiB

[startup+2142.3 s]
/proc/loadavg: 1.00 1.01 1.72 2/192 9195
/proc/meminfo: memFree=9528972/32770624 swapFree=67001860/67108860
[pid=9026] ppid=9023 vsize=22538172 memory=22371052 CPUtime=2141.6 cores=0-7
/proc/9026/stat : 9026 (cosoco) R 9023 9026 8695 0 -1 4202496 5446550 0 0 0 213104 1056 0 0 20 0 1 0 437658769 23079088128 5592763 33554432000 4194304 6211092 140730548735232 140730548708792 139975425518478 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 5634543 5592763 819 493 0 5627848 0
Current children cumulated CPU time: 2141.6 s
Current children cumulated vsize: 22538172 KiB
Current children cumulated memory: 22371052 KiB

[startup+2202.3 s]
/proc/loadavg: 1.00 1.01 1.67 2/192 9198
/proc/meminfo: memFree=8903164/32770624 swapFree=67001860/67108860
[pid=9026] ppid=9023 vsize=23141676 memory=22995092 CPUtime=2201.6 cores=0-7
/proc/9026/stat : 9026 (cosoco) R 9023 9026 8695 0 -1 4202496 5598192 0 0 0 219059 1101 0 0 20 0 1 0 437658769 23697076224 5748773 33554432000 4194304 6211092 140730548735232 140730548708792 5443096 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 5785419 5748773 819 493 0 5778724 0
Current children cumulated CPU time: 2201.6 s
Current children cumulated vsize: 23141676 KiB
Current children cumulated memory: 22995092 KiB

[startup+2262.3 s]
/proc/loadavg: 1.08 1.03 1.64 2/191 9201
/proc/meminfo: memFree=8280800/32770624 swapFree=67001860/67108860
[pid=9026] ppid=9023 vsize=23744652 memory=23616536 CPUtime=2261.59 cores=0-7
/proc/9026/stat : 9026 (cosoco) R 9023 9026 8695 0 -1 4202496 5749609 0 0 0 225009 1150 0 0 20 0 1 0 437658769 24314523648 5904134 33554432000 4194304 6211092 140730548735232 140730548708792 5452223 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 5936163 5904134 819 493 0 5929468 0
Current children cumulated CPU time: 2261.59 s
Current children cumulated vsize: 23744652 KiB
Current children cumulated memory: 23616536 KiB

[startup+2322.3 s]
/proc/loadavg: 1.03 1.03 1.60 2/191 9206
/proc/meminfo: memFree=7656032/32770624 swapFree=67001860/67108860
[pid=9026] ppid=9023 vsize=24347892 memory=24240320 CPUtime=2321.58 cores=0-7
/proc/9026/stat : 9026 (cosoco) R 9023 9026 8695 0 -1 4202496 5901194 0 0 0 230960 1198 0 0 20 0 1 0 437658769 24932241408 6060080 33554432000 4194304 6211092 140730548735232 140730548708792 5446643 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 6086973 6060080 819 493 0 6080278 0
Current children cumulated CPU time: 2321.58 s
Current children cumulated vsize: 24347892 KiB
Current children cumulated memory: 24240320 KiB

[startup+2382.3 s]
/proc/loadavg: 1.01 1.02 1.56 2/191 9209
/proc/meminfo: memFree=7033620/32770624 swapFree=67001860/67108860
[pid=9026] ppid=9023 vsize=24950868 memory=24861464 CPUtime=2381.56 cores=0-7
/proc/9026/stat : 9026 (cosoco) R 9023 9026 8695 0 -1 4202496 6052573 0 0 0 236905 1251 0 0 20 0 1 0 437658769 25549688832 6215366 33554432000 4194304 6211092 140730548735232 140730548708792 139975416497649 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 6237717 6215366 819 493 0 6231022 0
Current children cumulated CPU time: 2381.56 s
Current children cumulated vsize: 24950868 KiB
Current children cumulated memory: 24861464 KiB

[startup+2442.3 s]
/proc/loadavg: 1.00 1.01 1.53 2/191 9212
/proc/meminfo: memFree=6408200/32770624 swapFree=67001860/67108860
[pid=9026] ppid=9023 vsize=25553712 memory=25485020 CPUtime=2441.56 cores=0-7
/proc/9026/stat : 9026 (cosoco) R 9023 9026 8695 0 -1 4202496 6204075 0 0 0 242855 1301 0 0 20 0 1 0 437658769 26167001088 6371255 33554432000 4194304 6211092 140730548735232 140730548708792 139975416493987 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 6388428 6371255 819 493 0 6381733 0
Current children cumulated CPU time: 2441.56 s
Current children cumulated vsize: 25553712 KiB
Current children cumulated memory: 25485020 KiB

[startup+2502.3 s]
/proc/loadavg: 1.00 1.01 1.49 2/191 9217
/proc/meminfo: memFree=5785448/32770624 swapFree=67001860/67108860
[pid=9026] ppid=9023 vsize=26156688 memory=26106224 CPUtime=2501.55 cores=0-7
/proc/9026/stat : 9026 (cosoco) R 9023 9026 8695 0 -1 4202496 6355571 0 0 0 248802 1353 0 0 20 0 1 0 437658769 26784448512 6526556 33554432000 4194304 6211092 140730548735232 140730548708792 5441106 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 6539172 6526556 819 493 0 6532477 0
Current children cumulated CPU time: 2501.55 s
Current children cumulated vsize: 26156688 KiB
Current children cumulated memory: 26106224 KiB



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+2520 s]*
/proc/loadavg: 1.00 1.01 1.48 2/191 9217
/proc/meminfo: memFree=5597956/32770624 swapFree=67001860/67108860
[pid=9026] ppid=9023 vsize=26334624 memory=26290324 CPUtime=2519.26 cores=0-7
/proc/9026/stat : 9026 (cosoco) R 9023 9026 8695 0 -1 4202496 6400324 0 0 0 250557 1369 0 0 20 0 1 0 437658769 26966654976 6572581 33554432000 4194304 6211092 140730548735232 140730548708792 139975415573477 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 6583656 6572581 819 493 0 6576961 0
Current children cumulated CPU time: 2519.26 s
Current children cumulated vsize: 26334624 KiB
Current children cumulated memory: 26290324 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+2523.21 s]
# the end of solver process 9026 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2505.85 s, system=14.9199 s

Solver just ended.
Dumping a history of the last processes samples

[startup+2520.8 s]
/proc/loadavg: 1.00 1.01 1.48 2/191 9217
/proc/meminfo: memFree=5597956/32770624 swapFree=67001860/67108860
[pid=9026] ppid=9023 vsize=26334624 memory=26290324 CPUtime=2519.39 cores=0-7
/proc/9026/stat : 9026 (cosoco) S 9023 9026 8695 0 -1 512 6400330 0 0 0 250569 1370 0 0 20 0 1 0 437658769 26966654976 6572581 33554432000 4194304 6211092 140730548735232 140730548708792 5189550 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 6583656 6572581 819 493 0 6576961 0
Current children cumulated CPU time: 2519.39 s
Current children cumulated vsize: 26334624 KiB
Current children cumulated memory: 26290324 KiB

[startup+2521.6 s]
/proc/loadavg: 1.00 1.01 1.48 2/192 9218
/proc/meminfo: memFree=5055756/32770624 swapFree=67001860/67108860
[pid=9026] ppid=9023 vsize=26334624 memory=26290324 CPUtime=2519.49 cores=0-7
/proc/9026/stat : 9026 (cosoco) S 9023 9026 8695 0 -1 512 6400330 0 0 0 250579 1370 0 0 20 0 1 0 437658769 26966654976 6572581 33554432000 4194304 6211092 140730548735232 140730548708792 5189550 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 6583656 6572581 819 493 0 6576961 0
Current children cumulated CPU time: 2519.49 s
Current children cumulated vsize: 26334624 KiB
Current children cumulated memory: 26290324 KiB

[startup+2522 s]*
/proc/loadavg: 1.00 1.01 1.48 1/192 9218
/proc/meminfo: memFree=4608280/32770624 swapFree=67001860/67108860
[pid=9026] ppid=9023 vsize=26334624 memory=26290324 CPUtime=2519.55 cores=0-7
/proc/9026/stat : 9026 (cosoco) S 9023 9026 8695 0 -1 512 6400330 0 0 0 250584 1371 0 0 20 0 1 0 437658769 26966654976 6572581 33554432000 4194304 6211092 140730548735232 140730548708792 5189550 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 0 0 0 8312316 8317760 17199104 140730548741137 140730548741229 140730548741229 140730548744142 0
/proc/9026/statm: 6583656 6572581 819 493 0 6576961 0
Current children cumulated CPU time: 2519.55 s
Current children cumulated vsize: 26334624 KiB
Current children cumulated memory: 26290324 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 2523.21
CPU time (s): 2520.77
CPU user time (s): 2505.85
CPU system time (s): 14.9199
CPU usage (%): 99.903
Max. virtual memory (cumulated for all children) (KiB): 26334624
Max. memory (cumulated for all children) (KiB): 26290324

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2505.85
system time used= 14.9199
maximum resident set size= 26290528
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 6400330
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 122569
involuntary context switches= 10812


# summary of solver processes directly reported to runsolver:
#   pid: 9026
#   total CPU time (s): 2520.77
#   total CPU user time (s): 2505.85
#   total CPU system time (s): 14.9199

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 10.5364 second user time and 15.3108 second system time

The end

Launcher Data 

Begin job on node136 at 2019-09-16 08:07:37
IDJOB=4398470
IDBENCH=139548
IDSOLVER=2851
FILE ID=node136/4398470-1568614101
RUNJOBID= node136-1568611536-8725
PBS_JOBID= 21703455
Free space on /tmp= 61172 MiB

SOLVER NAME= cosoco 2.O parallel
BENCH NAME= XCSP17/SteelMillSlab/SteelMillSlab-bench_16_2.xml
COMMAND LINE= cosoco -model -nbcores=NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4398470-1568614101/watcher-4398470-1568614101 -o /tmp/evaluation-result-4398470-1568614101/solver-4398470-1568614101 -C 20160 -W 2520 -M 31000 --bin-var /tmp/runsolver-exch-node136-1568611536-8725 --watchdog 2580  cosoco -model -nbcores=8 HOME/instance-4398470-1568614101.xml

TIME LIMIT= 20160 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 8e5b61791e512734038a023d36a5924c
RANDOM SEED=674589147

node136.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.00
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.54
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.00
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.54
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.00
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.54
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.00
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.54
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        31935108 kB
MemAvailable:   31904100 kB
Buffers:           22908 kB
Cached:           244308 kB
SwapCached:         3456 kB
Active:            55056 kB
Inactive:         216788 kB
Active(anon):       4476 kB
Inactive(anon):    48800 kB
Active(file):      50580 kB
Inactive(file):   167988 kB
Unevictable:      110392 kB
Mlocked:          110392 kB
SwapTotal:      67108860 kB
SwapFree:       67001848 kB
Dirty:              2948 kB
Writeback:             0 kB
AnonPages:        113004 kB
Mapped:            46368 kB
Shmem:             40224 kB
Slab:             154576 kB
SReclaimable:     125840 kB
SUnreclaim:        28736 kB
KernelStack:        3440 kB
PageTables:         5392 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     507040 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     67584 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      120192 kB
DirectMap2M:    33425408 kB

Free space on /tmp at the end= 60696 MiB
End job on node136 at 2019-09-16 08:49:40