Trace number 4409746

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2.0 parallel? (TO) 2520.75 2523.26

General information on the benchmark

NameSteelMillSlab/
SteelMillSlab-bench_10_9.xml
MD5SUM3187efa1e901acb6cc373eb045baee33
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkSAT TO
Best value of the objective obtained on this benchmark142
Best CPU time to get the best result obtained on this benchmark2520.15
Satisfiable
(Un)Satisfiability was proved
Number of variables12654
Number of constraints466
Number of domains4
Minimum domain size2
Maximum domain size111
Distribution of domain sizes[{"size":2,"count":12321},{"size":12,"count":111},{"size":47,"count":111},{"size":111,"count":111}]
Minimum variable degree2
Maximum variable degree114
Distribution of variable degrees[{"degree":2,"count":12432},{"degree":3,"count":111},{"degree":112,"count":78},{"degree":113,"count":24},{"degree":114,"count":9}]
Minimum constraint arity2
Maximum constraint arity112
Distribution of constraint arities[{"arity":2,"count":132},{"arity":111,"count":112},{"arity":112,"count":222}]
Number of extensional constraints0
Number of intensional constraints132
Distribution of constraint types[{"type":"intension","count":132},{"type":"sum","count":112},{"type":"element","count":111},{"type":"channel","count":111}]
Optimization problemYES
Type of objectivemin SUM

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (14805 MiB free)
  memory of node 1: 16384 MiB (14592 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4409746-1569838549/watcher-4409746-1569838549 -o /tmp/evaluation-result-4409746-1569838549/solver-4409746-1569838549 -C 20160 -W 2520 -M 31000 --bin-var /tmp/runsolver-exch-node119-1569830904-6198 --watchdog 2580 cosoco -model -nbcores=8 HOME/instance-4409746-1569838549.xml 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 20160 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 20190 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=7068, runsolver pid=7065
Current StackSize limit: 8192 KiB


[startup+0.100159 s]*
/proc/loadavg: 8.14 8.26 8.24 2/191 7069
/proc/meminfo: memFree=30068464/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=65872 memory=41740 CPUtime=0.1 cores=0-7
/proc/7068/stat : 7068 (cosoco) R 7065 7068 6171 0 -1 4202496 10649 0 0 0 8 2 0 0 20 0 1 0 559769636 67452928 10435 33554432000 4194304 6211132 140732524689888 140732524686424 140193716222949 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 16468 10435 754 493 0 9773 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 65872 KiB
Current children cumulated memory: 41740 KiB

[startup+0.209234 s]*
/proc/loadavg: 8.14 8.26 8.24 2/191 7069
/proc/meminfo: memFree=30037464/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=93188 memory=69196 CPUtime=0.2 cores=0-7
/proc/7068/stat : 7068 (cosoco) R 7065 7068 6171 0 -1 4202496 17506 0 0 0 18 2 0 0 20 0 1 0 559769636 95424512 17299 33554432000 4194304 6211132 140732524689888 140732524686424 140193716222949 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 23297 17299 809 493 0 16602 0
Current children cumulated CPU time: 0.2 s
Current children cumulated vsize: 93188 KiB
Current children cumulated memory: 69196 KiB

[startup+0.308571 s]*
/proc/loadavg: 8.14 8.26 8.24 2/191 7069
/proc/meminfo: memFree=30035604/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=94628 memory=70804 CPUtime=0.31 cores=0-7
/proc/7068/stat : 7068 (cosoco) R 7065 7068 6171 0 -1 4202496 17884 0 0 0 28 3 0 0 20 0 1 0 559769636 96899072 17701 33554432000 4194304 6211132 140732524689888 140732524686424 5454208 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 23657 17701 819 493 0 16962 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 94628 KiB
Current children cumulated memory: 70804 KiB

[startup+0.700328 s]
/proc/loadavg: 8.14 8.26 8.24 2/191 7069
/proc/meminfo: memFree=30033944/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=98604 memory=74764 CPUtime=0.69 cores=0-7
/proc/7068/stat : 7068 (cosoco) R 7065 7068 6171 0 -1 4202496 18884 0 0 0 66 3 0 0 20 0 1 0 559769636 100970496 18691 33554432000 4194304 6211132 140732524689888 140732524663448 140193716222949 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 24651 18691 819 493 0 17956 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 98604 KiB
Current children cumulated memory: 74764 KiB

[startup+1.50033 s]
/proc/loadavg: 8.14 8.26 8.24 2/191 7069
/proc/meminfo: memFree=30028940/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=107056 memory=83064 CPUtime=1.49 cores=0-7
/proc/7068/stat : 7068 (cosoco) R 7065 7068 6171 0 -1 4202496 20964 0 0 0 146 3 0 0 20 0 1 0 559769636 109625344 20766 33554432000 4194304 6211132 140732524689888 140732524663448 140193716222949 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 26764 20766 819 493 0 20069 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 107056 KiB
Current children cumulated memory: 83064 KiB

[startup+3.10853 s]
/proc/loadavg: 7.57 8.14 8.21 2/191 7069
/proc/meminfo: memFree=30008484/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=124004 memory=99824 CPUtime=3.1 cores=0-7
/proc/7068/stat : 7068 (cosoco) R 7065 7068 6171 0 -1 4202496 25394 0 0 0 306 4 0 0 20 0 1 0 559769636 126980096 24956 33554432000 4194304 6211132 140732524689888 140732524663448 140193716222949 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 31001 24956 819 493 0 24306 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 124004 KiB
Current children cumulated memory: 99824 KiB

[startup+6.30033 s]
/proc/loadavg: 7.57 8.14 8.21 2/191 7069
/proc/meminfo: memFree=29977236/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=157812 memory=133004 CPUtime=6.28 cores=0-7
/proc/7068/stat : 7068 (cosoco) R 7065 7068 6171 0 -1 4202496 33946 0 0 0 623 5 0 0 20 0 1 0 559769636 161599488 33251 33554432000 4194304 6211132 140732524689888 140732524663448 140193716222949 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 39453 33251 819 493 0 32758 0
Current children cumulated CPU time: 6.28 s
Current children cumulated vsize: 157812 KiB
Current children cumulated memory: 133004 KiB

[startup+12.7003 s]
/proc/loadavg: 7.04 8.02 8.17 2/192 7071
/proc/meminfo: memFree=29913728/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=224064 memory=199796 CPUtime=12.68 cores=0-7
/proc/7068/stat : 7068 (cosoco) R 7065 7068 6171 0 -1 4202496 50808 0 0 0 1261 7 0 0 20 0 1 0 559769636 229441536 49949 33554432000 4194304 6211132 140732524689888 140732524663448 140193716222949 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 56016 49949 819 493 0 49321 0
Current children cumulated CPU time: 12.68 s
Current children cumulated vsize: 224064 KiB
Current children cumulated memory: 199796 KiB

[startup+25.5003 s]
/proc/loadavg: 5.70 7.67 8.05 2/191 7071
/proc/meminfo: memFree=29776384/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=358956 memory=334760 CPUtime=25.48 cores=0-7
/proc/7068/stat : 7068 (cosoco) R 7065 7068 6171 0 -1 4202496 84280 0 0 0 2536 12 0 0 20 0 1 0 559769636 367570944 83690 33554432000 4194304 6211132 140732524689888 140732524663448 140193716222949 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 89739 83690 819 493 0 83044 0
Current children cumulated CPU time: 25.48 s
Current children cumulated vsize: 358956 KiB
Current children cumulated memory: 334760 KiB

[startup+51.1086 s]
/proc/loadavg: 4.10 7.14 7.86 2/191 7072
/proc/meminfo: memFree=29503972/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=630620 memory=601992 CPUtime=51.08 cores=0-7
/proc/7068/stat : 7068 (cosoco) R 7065 7068 6171 0 -1 4202496 150022 0 0 0 5086 22 0 0 20 0 1 0 559769636 645754880 150498 33554432000 4194304 6211132 140732524689888 140732524663448 140193716222949 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 157655 150498 819 493 0 150960 0
Current children cumulated CPU time: 51.08 s
Current children cumulated vsize: 630620 KiB
Current children cumulated memory: 601992 KiB

[startup+102.3 s]
/proc/loadavg: 2.34 6.19 7.50 2/192 7076
/proc/meminfo: memFree=28970980/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=1177248 memory=1139596 CPUtime=102.25 cores=0-7
/proc/7068/stat : 7068 (cosoco) R 7065 7068 6171 0 -1 4202496 280909 0 0 0 10186 39 0 0 20 0 1 0 559769636 1205501952 284899 33554432000 4194304 6211132 140732524689888 140732524663448 140193716222949 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 294312 284899 819 493 0 287617 0
Current children cumulated CPU time: 102.25 s
Current children cumulated vsize: 1177248 KiB
Current children cumulated memory: 1139596 KiB

[startup+162.3 s]
/proc/loadavg: 1.49 5.25 7.10 2/191 7079
/proc/meminfo: memFree=28342228/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=1810768 memory=1767044 CPUtime=162.22 cores=0-7
/proc/7068/stat : 7068 (cosoco) R 7065 7068 6171 0 -1 4202496 433665 0 0 0 16160 62 0 0 20 0 1 0 559769636 1854226432 441761 33554432000 4194304 6211132 140732524689888 140732524663448 140193716222949 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 452692 441761 819 493 0 445997 0
Current children cumulated CPU time: 162.22 s
Current children cumulated vsize: 1810768 KiB
Current children cumulated memory: 1767044 KiB

[startup+222.3 s]
/proc/loadavg: 1.18 4.47 6.72 2/191 7082
/proc/meminfo: memFree=27713924/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=2419552 memory=2394692 CPUtime=222.2 cores=0-7
/proc/7068/stat : 7068 (cosoco) R 7065 7068 6171 0 -1 4202496 585990 0 0 0 22137 83 0 0 20 0 1 0 559769636 2477621248 598673 33554432000 4194304 6211132 140732524689888 140732524663448 140193716222949 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 604888 598673 819 493 0 598193 0
Current children cumulated CPU time: 222.2 s
Current children cumulated vsize: 2419552 KiB
Current children cumulated memory: 2394692 KiB

[startup+282.3 s]
/proc/loadavg: 1.29 3.89 6.37 2/192 7087
/proc/meminfo: memFree=27083392/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=3065380 memory=3023880 CPUtime=282.18 cores=0-7
/proc/7068/stat : 7068 (cosoco) R 7065 7068 6171 0 -1 4202496 738695 0 0 0 28112 106 0 0 20 0 1 0 559769636 3138949120 755970 33554432000 4194304 6211132 140732524689888 140732524663448 140193716222949 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 766345 755970 819 493 0 759650 0
Current children cumulated CPU time: 282.18 s
Current children cumulated vsize: 3065380 KiB
Current children cumulated memory: 3023880 KiB

[startup+342.3 s]
/proc/loadavg: 1.30 3.44 6.06 2/191 7097
/proc/meminfo: memFree=26452416/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=3729528 memory=3653048 CPUtime=342.16 cores=0-7
/proc/7068/stat : 7068 (cosoco) R 7065 7068 6171 0 -1 4202496 891428 0 0 0 34086 130 0 0 20 0 1 0 559769636 3819036672 913262 33554432000 4194304 6211132 140732524689888 140732524663448 140193716218686 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 932382 913262 819 493 0 925687 0
Current children cumulated CPU time: 342.16 s
Current children cumulated vsize: 3729528 KiB
Current children cumulated memory: 3653048 KiB

[startup+402.3 s]
/proc/loadavg: 1.11 2.99 5.75 2/191 7100
/proc/meminfo: memFree=25823980/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=4338048 memory=4279844 CPUtime=402.14 cores=0-7
/proc/7068/stat : 7068 (cosoco) R 7065 7068 6171 0 -1 4202496 1043609 0 0 0 40061 153 0 0 20 0 1 0 559769636 4442161152 1069961 33554432000 4194304 6211132 140732524689888 140732524663448 140193716222949 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 1084512 1069961 819 493 0 1077817 0
Current children cumulated CPU time: 402.14 s
Current children cumulated vsize: 4338048 KiB
Current children cumulated memory: 4279844 KiB

[startup+462.3 s]
/proc/loadavg: 1.04 2.63 5.45 2/192 7105
/proc/meminfo: memFree=25194328/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=4946568 memory=4908888 CPUtime=462.12 cores=0-7
/proc/7068/stat : 7068 (cosoco) R 7065 7068 6171 0 -1 4202496 1195793 0 0 0 46035 177 0 0 20 0 1 0 559769636 5065285632 1227222 33554432000 4194304 6211132 140732524689888 140732524663448 140193716222949 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 1236642 1227222 819 493 0 1229947 0
Current children cumulated CPU time: 462.12 s
Current children cumulated vsize: 4946568 KiB
Current children cumulated memory: 4908888 KiB


################
# More data... #
################

Current children cumulated vsize: 20199528 KiB
Current children cumulated memory: 19956840 KiB

[startup+1962.3 s]
/proc/loadavg: 1.00 1.01 1.88 2/191 7237
/proc/meminfo: memFree=9490556/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=20805408 memory=20583388 CPUtime=1961.64 cores=0-7
/proc/7068/stat : 7068 (cosoco) R 7065 7068 6171 0 -1 4202496 4995532 0 0 0 195110 1054 0 0 20 0 1 0 559769636 21304737792 5145847 33554432000 4194304 6211132 140732524689888 140732524663448 140193716222949 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 5201352 5145847 819 493 0 5194657 0
Current children cumulated CPU time: 1961.64 s
Current children cumulated vsize: 20805408 KiB
Current children cumulated memory: 20583388 KiB

[startup+2022.3 s]
/proc/loadavg: 1.00 1.01 1.83 2/191 7240
/proc/meminfo: memFree=8865016/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=21411024 memory=21207364 CPUtime=2021.62 cores=0-7
/proc/7068/stat : 7068 (cosoco) R 7065 7068 6171 0 -1 4202496 5147031 0 0 0 201056 1106 0 0 20 0 1 0 559769636 21924888576 5301841 33554432000 4194304 6211132 140732524689888 140732524663448 140193716222949 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 5352756 5301841 819 493 0 5346061 0
Current children cumulated CPU time: 2021.62 s
Current children cumulated vsize: 21411024 KiB
Current children cumulated memory: 21207364 KiB

[startup+2082.3 s]
/proc/loadavg: 1.00 1.01 1.77 2/192 7245
/proc/meminfo: memFree=8237708/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=22016640 memory=21833384 CPUtime=2081.61 cores=0-7
/proc/7068/stat : 7068 (cosoco) R 7065 7068 6171 0 -1 4202496 5298512 0 0 0 207004 1157 0 0 20 0 1 0 559769636 22545039360 5458346 33554432000 4194304 6211132 140732524689888 140732524663448 140193716222949 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 5504160 5458346 819 493 0 5497465 0
Current children cumulated CPU time: 2081.61 s
Current children cumulated vsize: 22016640 KiB
Current children cumulated memory: 21833384 KiB

[startup+2142.3 s]
/proc/loadavg: 1.00 1.01 1.73 2/191 7255
/proc/meminfo: memFree=7612096/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=22622124 memory=22457360 CPUtime=2141.59 cores=0-7
/proc/7068/stat : 7068 (cosoco) R 7065 7068 6171 0 -1 4202496 5449985 0 0 0 212947 1212 0 0 20 0 1 0 559769636 23165054976 5614340 33554432000 4194304 6211132 140732524689888 140732524663448 140193716222949 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 5655531 5614340 819 493 0 5648836 0
Current children cumulated CPU time: 2141.59 s
Current children cumulated vsize: 22622124 KiB
Current children cumulated memory: 22457360 KiB

[startup+2202.3 s]
/proc/loadavg: 1.00 1.01 1.68 2/191 7258
/proc/meminfo: memFree=6984608/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=23227476 memory=23083116 CPUtime=2201.57 cores=0-7
/proc/7068/stat : 7068 (cosoco) R 7065 7068 6171 0 -1 4202496 5601378 0 0 0 218891 1266 0 0 20 0 1 0 559769636 23784935424 5770779 33554432000 4194304 6211132 140732524689888 140732524663448 140193716218686 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 5806869 5770779 819 493 0 5800174 0
Current children cumulated CPU time: 2201.57 s
Current children cumulated vsize: 23227476 KiB
Current children cumulated memory: 23083116 KiB

[startup+2262.3 s]
/proc/loadavg: 1.00 1.01 1.64 2/192 7263
/proc/meminfo: memFree=6359548/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=23832828 memory=23707032 CPUtime=2261.56 cores=0-7
/proc/7068/stat : 7068 (cosoco) R 7065 7068 6171 0 -1 4202496 5752846 0 0 0 224835 1321 0 0 20 0 1 0 559769636 24404815872 5926758 33554432000 4194304 6211132 140732524689888 140732524663448 140193716222949 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 5958207 5926758 819 493 0 5951512 0
Current children cumulated CPU time: 2261.56 s
Current children cumulated vsize: 23832828 KiB
Current children cumulated memory: 23707032 KiB

[startup+2322.3 s]
/proc/loadavg: 1.00 1.01 1.60 2/191 7266
/proc/meminfo: memFree=5731968/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=24438180 memory=24332788 CPUtime=2321.54 cores=0-7
/proc/7068/stat : 7068 (cosoco) R 7065 7068 6171 0 -1 4202496 5904246 0 0 0 230778 1376 0 0 20 0 1 0 559769636 25024696320 6083197 33554432000 4194304 6211132 140732524689888 140732524663448 140193716222949 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 6109545 6083197 819 493 0 6102850 0
Current children cumulated CPU time: 2321.54 s
Current children cumulated vsize: 24438180 KiB
Current children cumulated memory: 24332788 KiB

[startup+2382.3 s]
/proc/loadavg: 1.00 1.01 1.56 2/191 7269
/proc/meminfo: memFree=5107864/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=25043136 memory=24958280 CPUtime=2381.52 cores=0-7
/proc/7068/stat : 7068 (cosoco) R 7065 7068 6171 0 -1 4202496 6055584 0 0 0 236720 1432 0 0 20 0 1 0 559769636 25644171264 6239570 33554432000 4194304 6211132 140732524689888 140732524663448 140193716222949 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 6260784 6239570 819 493 0 6254089 0
Current children cumulated CPU time: 2381.52 s
Current children cumulated vsize: 25043136 KiB
Current children cumulated memory: 24958280 KiB

[startup+2442.3 s]
/proc/loadavg: 1.00 1.01 1.53 2/192 7275
/proc/meminfo: memFree=4481292/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=25648224 memory=25581668 CPUtime=2441.51 cores=0-7
/proc/7068/stat : 7068 (cosoco) R 7065 7068 6171 0 -1 4202496 6206953 0 0 0 242663 1488 0 0 20 0 1 0 559769636 26263781376 6395417 33554432000 4194304 6211132 140732524689888 140732524663448 140193716222949 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 6412056 6395417 819 493 0 6405361 0
Current children cumulated CPU time: 2441.51 s
Current children cumulated vsize: 25648224 KiB
Current children cumulated memory: 25581668 KiB

[startup+2502.3 s]
/proc/loadavg: 1.00 1.01 1.49 2/191 7278
/proc/meminfo: memFree=3855024/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=26252916 memory=26207100 CPUtime=2501.49 cores=0-7
/proc/7068/stat : 7068 (cosoco) R 7065 7068 6171 0 -1 4202496 6358238 0 0 0 248600 1549 0 0 20 0 1 0 559769636 26882985984 6551775 33554432000 4194304 6211132 140732524689888 140732524663448 140193716222949 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 6563229 6551775 819 493 0 6556534 0
Current children cumulated CPU time: 2501.49 s
Current children cumulated vsize: 26252916 KiB
Current children cumulated memory: 26207100 KiB



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+2520 s]*
/proc/loadavg: 1.00 1.01 1.48 2/191 7279
/proc/meminfo: memFree=3669112/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=26431380 memory=26389592 CPUtime=2519.19 cores=0-7
/proc/7068/stat : 7068 (cosoco) R 7065 7068 6171 0 -1 4202496 6402867 0 0 0 250354 1565 0 0 20 0 1 0 559769636 27065733120 6597398 33554432000 4194304 6211132 140732524689888 140732524663448 140193716222949 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 6607845 6597398 819 493 0 6601150 0
Current children cumulated CPU time: 2519.19 s
Current children cumulated vsize: 26431380 KiB
Current children cumulated memory: 26389592 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+2523.26 s]
# the end of solver process 7068 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2503.84 s, system=16.913 s

Solver just ended.
Dumping a history of the last processes samples

[startup+2520.8 s]
/proc/loadavg: 1.00 1.01 1.48 2/191 7279
/proc/meminfo: memFree=3669112/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=26431380 memory=26389592 CPUtime=2519.32 cores=0-7
/proc/7068/stat : 7068 (cosoco) S 7065 7068 6171 0 -1 512 6402874 0 0 0 250366 1566 0 0 20 0 1 0 559769636 27065733120 6597398 33554432000 4194304 6211132 140732524689888 140732524663448 5189582 0 16384 4096 8405026 18446744071580971376 0 0 17 6 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 6607845 6597398 819 493 0 6601150 0
Current children cumulated CPU time: 2519.32 s
Current children cumulated vsize: 26431380 KiB
Current children cumulated memory: 26389592 KiB

[startup+2521.6 s]
/proc/loadavg: 1.00 1.01 1.48 2/192 7280
/proc/meminfo: memFree=3114800/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=26431380 memory=26389592 CPUtime=2519.44 cores=0-7
/proc/7068/stat : 7068 (cosoco) S 7065 7068 6171 0 -1 512 6402874 0 0 0 250378 1566 0 0 20 0 1 0 559769636 27065733120 6597398 33554432000 4194304 6211132 140732524689888 140732524663448 5189582 0 16384 4096 8405026 18446744071580971376 0 0 17 6 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 6607845 6597398 819 493 0 6601150 0
Current children cumulated CPU time: 2519.44 s
Current children cumulated vsize: 26431380 KiB
Current children cumulated memory: 26389592 KiB

[startup+2522 s]*
/proc/loadavg: 1.00 1.01 1.48 3/192 7280
/proc/meminfo: memFree=2578744/32770624 swapFree=66954180/67108860
[pid=7068] ppid=7065 vsize=26431380 memory=26389592 CPUtime=2519.49 cores=0-7
/proc/7068/stat : 7068 (cosoco) S 7065 7068 6171 0 -1 512 6402874 0 0 0 250383 1566 0 0 20 0 1 0 559769636 27065733120 6597398 33554432000 4194304 6211132 140732524689888 140732524663448 5189582 0 16384 4096 8405026 18446744071580971376 0 0 17 6 0 0 0 0 0 8312316 8317760 39407616 140732524692497 140732524692589 140732524692589 140732524695502 0
/proc/7068/statm: 6607845 6597398 819 493 0 6601150 0
Current children cumulated CPU time: 2519.49 s
Current children cumulated vsize: 26431380 KiB
Current children cumulated memory: 26389592 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 2523.26
CPU time (s): 2520.75
CPU user time (s): 2503.84
CPU system time (s): 16.913
CPU usage (%): 99.9008
Max. virtual memory (cumulated for all children) (KiB): 26431380
Max. memory (cumulated for all children) (KiB): 26389592

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2503.84
system time used= 16.913
maximum resident set size= 26389688
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 6402874
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 136358
involuntary context switches= 1685


# summary of solver processes directly reported to runsolver:
#   pid: 7068
#   total CPU time (s): 2520.75
#   total CPU user time (s): 2503.84
#   total CPU system time (s): 16.913

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 8.7783 second user time and 16.3622 second system time

The end

Launcher Data 

Begin job on node119 at 2019-09-30 12:15:08
IDJOB=4409746
IDBENCH=139585
IDSOLVER=2877
FILE ID=node119/4409746-1569838549
RUNJOBID= node119-1569830904-6198
PBS_JOBID= 21705950
Free space on /tmp= 62032 MiB

SOLVER NAME= cosoco 2.0 parallel
BENCH NAME= XCSP17/SteelMillSlab/SteelMillSlab-bench_10_9.xml
COMMAND LINE= cosoco -model -nbcores=NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4409746-1569838549/watcher-4409746-1569838549 -o /tmp/evaluation-result-4409746-1569838549/solver-4409746-1569838549 -C 20160 -W 2520 -M 31000 --bin-var /tmp/runsolver-exch-node119-1569830904-6198 --watchdog 2580  cosoco -model -nbcores=8 HOME/instance-4409746-1569838549.xml

TIME LIMIT= 20160 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 3187efa1e901acb6cc373eb045baee33
RANDOM SEED=623139124

node119.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.47
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.47
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.47
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.47
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        30102576 kB
MemAvailable:   31824688 kB
Buffers:          233888 kB
Cached:          1825924 kB
SwapCached:        46700 kB
Active:          1077712 kB
Inactive:        1034584 kB
Active(anon):      17696 kB
Inactive(anon):   116460 kB
Active(file):    1060016 kB
Inactive(file):   918124 kB
Unevictable:      107468 kB
Mlocked:          107468 kB
SwapTotal:      67108860 kB
SwapFree:       66954180 kB
Dirty:              2776 kB
Writeback:             0 kB
AnonPages:        116504 kB
Mapped:            26800 kB
Shmem:             73180 kB
Slab:             148312 kB
SReclaimable:     119612 kB
SUnreclaim:        28700 kB
KernelStack:        3360 kB
PageTables:         5436 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     539564 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     69632 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      126336 kB
DirectMap2M:    33419264 kB

Free space on /tmp at the end= 61504 MiB
End job on node119 at 2019-09-30 12:57:11