Trace number 4409725

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2.0 parallel? (TO) 2521.03 2523.24

General information on the benchmark

NameSteelMillSlab/
SteelMillSlab-bench_15_2.xml
MD5SUM5f96bf565600cbf50f8c2cf3fe757a5f
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkSAT TO
Best value of the objective obtained on this benchmark105
Best CPU time to get the best result obtained on this benchmark2520.58
Satisfiable
(Un)Satisfiability was proved
Number of variables12654
Number of constraints466
Number of domains4
Minimum domain size2
Maximum domain size111
Distribution of domain sizes[{"size":2,"count":12321},{"size":11,"count":111},{"size":51,"count":111},{"size":111,"count":111}]
Minimum variable degree2
Maximum variable degree114
Distribution of variable degrees[{"degree":2,"count":12432},{"degree":3,"count":111},{"degree":112,"count":78},{"degree":113,"count":24},{"degree":114,"count":9}]
Minimum constraint arity2
Maximum constraint arity112
Distribution of constraint arities[{"arity":2,"count":132},{"arity":111,"count":112},{"arity":112,"count":222}]
Number of extensional constraints0
Number of intensional constraints132
Distribution of constraint types[{"type":"intension","count":132},{"type":"sum","count":112},{"type":"element","count":111},{"type":"channel","count":111}]
Optimization problemYES
Type of objectivemin SUM

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (15148 MiB free)
  memory of node 1: 16384 MiB (15487 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4409725-1569837691/watcher-4409725-1569837691 -o /tmp/evaluation-result-4409725-1569837691/solver-4409725-1569837691 -C 20160 -W 2520 -M 31000 --bin-var /tmp/runsolver-exch-node133-1569837650-18029 --watchdog 2580 cosoco -model -nbcores=8 HOME/instance-4409725-1569837691.xml 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 20160 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 20190 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=18046, runsolver pid=18043
Current StackSize limit: 8192 KiB


[startup+0.100163 s]*
/proc/loadavg: 7.50 8.14 8.32 2/197 18047
/proc/meminfo: memFree=31337388/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=66956 memory=42964 CPUtime=0.09 cores=0-7
/proc/18046/stat : 18046 (cosoco) R 18043 18046 18002 0 -1 4202496 10916 0 0 0 9 0 0 0 20 0 1 0 560008530 68562944 10741 33554432000 4194304 6211132 140736698865728 140736698862264 140649757268965 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 16739 10741 754 493 0 10044 0
Current children cumulated CPU time: 0.09 s
Current children cumulated vsize: 66956 KiB
Current children cumulated memory: 42964 KiB

[startup+0.209526 s]*
/proc/loadavg: 7.50 8.14 8.32 2/197 18047
/proc/meminfo: memFree=31306884/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=93216 memory=69364 CPUtime=0.2 cores=0-7
/proc/18046/stat : 18046 (cosoco) R 18043 18046 18002 0 -1 4202496 17530 0 0 0 19 1 0 0 20 0 1 0 560008530 95453184 17341 33554432000 4194304 6211132 140736698865728 140736698862264 140649757268965 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 23304 17341 809 493 0 16609 0
Current children cumulated CPU time: 0.2 s
Current children cumulated vsize: 93216 KiB
Current children cumulated memory: 69364 KiB

[startup+0.309588 s]*
/proc/loadavg: 7.50 8.14 8.32 2/197 18047
/proc/meminfo: memFree=31305396/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=94664 memory=70712 CPUtime=0.3 cores=0-7
/proc/18046/stat : 18046 (cosoco) R 18043 18046 18002 0 -1 4202496 17899 0 0 0 29 1 0 0 20 0 1 0 560008530 96935936 17678 33554432000 4194304 6211132 140736698865728 140736698862264 140649757268965 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 23666 17678 819 493 0 16971 0
Current children cumulated CPU time: 0.3 s
Current children cumulated vsize: 94664 KiB
Current children cumulated memory: 70712 KiB

[startup+0.700272 s]
/proc/loadavg: 7.50 8.14 8.32 2/197 18047
/proc/meminfo: memFree=31303248/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=98632 memory=74672 CPUtime=0.69 cores=0-7
/proc/18046/stat : 18046 (cosoco) R 18043 18046 18002 0 -1 4202496 18894 0 0 0 67 2 0 0 20 0 1 0 560008530 100999168 18668 33554432000 4194304 6211132 140736698865728 140736698862264 140649757268965 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 24658 18668 819 493 0 17963 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 98632 KiB
Current children cumulated memory: 74672 KiB

[startup+1.50028 s]
/proc/loadavg: 7.50 8.14 8.32 2/197 18047
/proc/meminfo: memFree=31298140/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=107216 memory=83096 CPUtime=1.49 cores=0-7
/proc/18046/stat : 18046 (cosoco) R 18043 18046 18002 0 -1 4202496 20975 0 0 0 147 2 0 0 20 0 1 0 560008530 109789184 20774 33554432000 4194304 6211132 140736698865728 140736698839288 140649757268965 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 26804 20774 819 493 0 20109 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 107216 KiB
Current children cumulated memory: 83096 KiB

[startup+3.10939 s]
/proc/loadavg: 7.50 8.14 8.32 2/197 18048
/proc/meminfo: memFree=31277680/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=124032 memory=99856 CPUtime=3.1 cores=0-7
/proc/18046/stat : 18046 (cosoco) R 18043 18046 18002 0 -1 4202496 25401 0 0 0 307 3 0 0 20 0 1 0 560008530 127008768 24964 33554432000 4194304 6211132 140736698865728 140736698839288 140649757268965 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 31008 24964 819 493 0 24313 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 124032 KiB
Current children cumulated memory: 99856 KiB

[startup+6.30026 s]
/proc/loadavg: 7.14 8.05 8.29 2/197 18048
/proc/meminfo: memFree=31246524/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=157840 memory=133044 CPUtime=6.28 cores=0-7
/proc/18046/stat : 18046 (cosoco) R 18043 18046 18002 0 -1 4202496 33935 0 0 0 625 3 0 0 20 0 1 0 560008530 161628160 33261 33554432000 4194304 6211132 140736698865728 140736698839288 140649757268965 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 39460 33261 819 493 0 32765 0
Current children cumulated CPU time: 6.28 s
Current children cumulated vsize: 157840 KiB
Current children cumulated memory: 133044 KiB

[startup+12.7003 s]
/proc/loadavg: 6.65 7.93 8.25 2/197 18070
/proc/meminfo: memFree=31183432/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=223696 memory=199564 CPUtime=12.68 cores=0-7
/proc/18046/stat : 18046 (cosoco) R 18043 18046 18002 0 -1 4202496 50816 0 0 0 1263 5 0 0 20 0 1 0 560008530 229064704 49891 33554432000 4194304 6211132 140736698865728 140736698839288 140649757268965 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 55924 49891 819 493 0 49229 0
Current children cumulated CPU time: 12.68 s
Current children cumulated vsize: 223696 KiB
Current children cumulated memory: 199564 KiB

[startup+25.5003 s]
/proc/loadavg: 5.39 7.59 8.14 2/197 18070
/proc/meminfo: memFree=31047956/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=358192 memory=333240 CPUtime=25.48 cores=0-7
/proc/18046/stat : 18046 (cosoco) R 18043 18046 18002 0 -1 4202496 84133 0 0 0 2537 11 0 0 20 0 1 0 560008530 366788608 83310 33554432000 4194304 6211132 140736698865728 140736698839288 140649757268965 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 89548 83310 819 493 0 82853 0
Current children cumulated CPU time: 25.48 s
Current children cumulated vsize: 358192 KiB
Current children cumulated memory: 333240 KiB

[startup+51.1094 s]
/proc/loadavg: 3.90 7.06 7.95 2/196 18072
/proc/meminfo: memFree=30775616/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=629064 memory=601768 CPUtime=51.08 cores=0-7
/proc/18046/stat : 18046 (cosoco) R 18043 18046 18002 0 -1 4202496 150272 0 0 0 5089 19 0 0 20 0 1 0 560008530 644161536 150442 33554432000 4194304 6211132 140736698865728 140736698839288 140649757268965 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 157266 150442 819 493 0 150571 0
Current children cumulated CPU time: 51.08 s
Current children cumulated vsize: 629064 KiB
Current children cumulated memory: 601768 KiB

[startup+102.3 s]
/proc/loadavg: 2.26 6.13 7.58 2/196 18075
/proc/meminfo: memFree=30243376/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=1174900 memory=1135204 CPUtime=102.26 cores=0-7
/proc/18046/stat : 18046 (cosoco) R 18043 18046 18002 0 -1 4202496 280584 0 0 0 10189 37 0 0 20 0 1 0 560008530 1203097600 283801 33554432000 4194304 6211132 140736698865728 140736698839288 140649757268965 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 293725 283801 819 493 0 287030 0
Current children cumulated CPU time: 102.26 s
Current children cumulated vsize: 1174900 KiB
Current children cumulated memory: 1135204 KiB

[startup+162.3 s]
/proc/loadavg: 1.46 5.20 7.17 2/196 18078
/proc/meminfo: memFree=29615016/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=1807232 memory=1762936 CPUtime=162.25 cores=0-7
/proc/18046/stat : 18046 (cosoco) R 18043 18046 18002 0 -1 4202496 433468 0 0 0 16169 56 0 0 20 0 1 0 560008530 1850605568 440734 33554432000 4194304 6211132 140736698865728 140736698839288 140649757268965 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 451808 440734 819 493 0 445113 0
Current children cumulated CPU time: 162.25 s
Current children cumulated vsize: 1807232 KiB
Current children cumulated memory: 1762936 KiB

[startup+222.3 s]
/proc/loadavg: 1.23 4.46 6.80 2/196 18083
/proc/meminfo: memFree=28985760/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=2415092 memory=2391064 CPUtime=222.24 cores=0-7
/proc/18046/stat : 18046 (cosoco) R 18043 18046 18002 0 -1 4202496 585517 0 0 0 22148 76 0 0 20 0 1 0 560008530 2473054208 597766 33554432000 4194304 6211132 140736698865728 140736698839288 140649757268965 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 603773 597766 819 493 0 597078 0
Current children cumulated CPU time: 222.24 s
Current children cumulated vsize: 2415092 KiB
Current children cumulated memory: 2391064 KiB

[startup+282.3 s]
/proc/loadavg: 1.09 3.83 6.43 2/195 18086
/proc/meminfo: memFree=28358500/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=3059732 memory=3017412 CPUtime=282.23 cores=0-7
/proc/18046/stat : 18046 (cosoco) R 18043 18046 18002 0 -1 4202496 738101 0 0 0 28125 98 0 0 20 0 1 0 560008530 3133165568 754353 33554432000 4194304 6211132 140736698865728 140736698839288 140649757268965 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 764933 754353 819 493 0 758238 0
Current children cumulated CPU time: 282.23 s
Current children cumulated vsize: 3059732 KiB
Current children cumulated memory: 3017412 KiB

[startup+342.3 s]
/proc/loadavg: 1.03 3.31 6.09 2/195 18089
/proc/meminfo: memFree=27730856/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=3722692 memory=3644552 CPUtime=342.21 cores=0-7
/proc/18046/stat : 18046 (cosoco) R 18043 18046 18002 0 -1 4202496 890388 0 0 0 34102 119 0 0 20 0 1 0 560008530 3812036608 911138 33554432000 4194304 6211132 140736698865728 140736698839288 140649757268965 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 930673 911138 819 493 0 923978 0
Current children cumulated CPU time: 342.21 s
Current children cumulated vsize: 3722692 KiB
Current children cumulated memory: 3644552 KiB

[startup+402.3 s]
/proc/loadavg: 1.01 2.89 5.77 2/195 18094
/proc/meminfo: memFree=27099848/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=4330156 memory=4273388 CPUtime=402.2 cores=0-7
/proc/18046/stat : 18046 (cosoco) R 18043 18046 18002 0 -1 4202496 1042697 0 0 0 40082 138 0 0 20 0 1 0 560008530 4434079744 1068347 33554432000 4194304 6211132 140736698865728 140736698839288 140649757268965 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 1082539 1068347 819 493 0 1075844 0
Current children cumulated CPU time: 402.2 s
Current children cumulated vsize: 4330156 KiB
Current children cumulated memory: 4273388 KiB

[startup+462.3 s]
/proc/loadavg: 1.00 2.55 5.48 2/195 18097
/proc/meminfo: memFree=26473580/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=4937884 memory=4899700 CPUtime=462.19 cores=0-7
/proc/18046/stat : 18046 (cosoco) R 18043 18046 18002 0 -1 4202496 1194850 0 0 0 46058 161 0 0 20 0 1 0 560008530 5056393216 1224925 33554432000 4194304 6211132 140736698865728 140736698839288 140649757268965 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 1234471 1224925 819 493 0 1227776 0
Current children cumulated CPU time: 462.19 s
Current children cumulated vsize: 4937884 KiB
Current children cumulated memory: 4899700 KiB


################
# More data... #
################

Current children cumulated vsize: 20167216 KiB
Current children cumulated memory: 19923980 KiB

[startup+1962.3 s]
/proc/loadavg: 1.01 1.04 1.91 2/195 18214
/proc/meminfo: memFree=10790004/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=20772304 memory=20549540 CPUtime=1961.86 cores=0-7
/proc/18046/stat : 18046 (cosoco) R 18043 18046 18002 0 -1 4202496 4996219 0 0 0 195231 955 0 0 20 0 1 0 560008530 21270839296 5137385 33554432000 4194304 6211132 140736698865728 140736698839288 140649757268965 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 5193076 5137385 819 493 0 5186381 0
Current children cumulated CPU time: 1961.86 s
Current children cumulated vsize: 20772304 KiB
Current children cumulated memory: 20549540 KiB

[startup+2022.3 s]
/proc/loadavg: 1.00 1.03 1.85 2/195 18219
/proc/meminfo: memFree=10165428/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=21377392 memory=21173088 CPUtime=2021.86 cores=0-7
/proc/18046/stat : 18046 (cosoco) R 18043 18046 18002 0 -1 4202496 5147883 0 0 0 201184 1002 0 0 20 0 1 0 560008530 21890449408 5293272 33554432000 4194304 6211132 140736698865728 140736698839288 140649757268965 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 5344348 5293272 819 493 0 5337653 0
Current children cumulated CPU time: 2021.86 s
Current children cumulated vsize: 21377392 KiB
Current children cumulated memory: 21173088 KiB

[startup+2082.3 s]
/proc/loadavg: 1.00 1.03 1.80 2/195 18222
/proc/meminfo: memFree=9538464/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=21982348 memory=21798416 CPUtime=2081.84 cores=0-7
/proc/18046/stat : 18046 (cosoco) R 18043 18046 18002 0 -1 4202496 5299489 0 0 0 207134 1050 0 0 20 0 1 0 560008530 22509924352 5449604 33554432000 4194304 6211132 140736698865728 140736698839288 140649757268965 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 5495587 5449604 819 493 0 5488892 0
Current children cumulated CPU time: 2081.84 s
Current children cumulated vsize: 21982348 KiB
Current children cumulated memory: 21798416 KiB

[startup+2142.3 s]
/proc/loadavg: 1.00 1.02 1.75 2/195 18225
/proc/meminfo: memFree=8914892/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=22586908 memory=22421428 CPUtime=2141.83 cores=0-7
/proc/18046/stat : 18046 (cosoco) R 18043 18046 18002 0 -1 4202496 5451036 0 0 0 213083 1100 0 0 20 0 1 0 560008530 23128993792 5605357 33554432000 4194304 6211132 140736698865728 140736698839288 140649757268965 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 5646727 5605357 819 493 0 5640032 0
Current children cumulated CPU time: 2141.83 s
Current children cumulated vsize: 22586908 KiB
Current children cumulated memory: 22421428 KiB

[startup+2202.3 s]
/proc/loadavg: 1.00 1.02 1.70 2/194 18230
/proc/meminfo: memFree=8287664/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=23191600 memory=23046724 CPUtime=2201.81 cores=0-7
/proc/18046/stat : 18046 (cosoco) R 18043 18046 18002 0 -1 4202496 5602671 0 0 0 219035 1146 0 0 20 0 1 0 560008530 23748198400 5761681 33554432000 4194304 6211132 140736698865728 140736698839288 140649757268965 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 5797900 5761681 819 493 0 5791205 0
Current children cumulated CPU time: 2201.81 s
Current children cumulated vsize: 23191600 KiB
Current children cumulated memory: 23046724 KiB

[startup+2262.3 s]
/proc/loadavg: 1.00 1.01 1.66 2/195 18234
/proc/meminfo: memFree=7664512/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=23796028 memory=23669480 CPUtime=2261.8 cores=0-7
/proc/18046/stat : 18046 (cosoco) R 18043 18046 18002 0 -1 4202496 5754160 0 0 0 224985 1195 0 0 20 0 1 0 560008530 24367132672 5917370 33554432000 4194304 6211132 140736698865728 140736698839288 140649757268965 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 5949007 5917370 819 493 0 5942312 0
Current children cumulated CPU time: 2261.8 s
Current children cumulated vsize: 23796028 KiB
Current children cumulated memory: 23669480 KiB

[startup+2322.3 s]
/proc/loadavg: 1.00 1.01 1.61 2/196 18238
/proc/meminfo: memFree=7037832/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=24400852 memory=24294768 CPUtime=2321.79 cores=0-7
/proc/18046/stat : 18046 (cosoco) R 18043 18046 18002 0 -1 4202496 5905827 0 0 0 230938 1241 0 0 20 0 1 0 560008530 24986472448 6073692 33554432000 4194304 6211132 140736698865728 140736698839288 140649757268965 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 6100213 6073692 819 493 0 6093518 0
Current children cumulated CPU time: 2321.79 s
Current children cumulated vsize: 24400852 KiB
Current children cumulated memory: 24294768 KiB

[startup+2382.3 s]
/proc/loadavg: 1.11 1.04 1.58 2/196 18250
/proc/meminfo: memFree=6413564/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=25005016 memory=24917260 CPUtime=2381.77 cores=0-7
/proc/18046/stat : 18046 (cosoco) R 18043 18046 18002 0 -1 4202496 6057249 0 0 0 236885 1292 0 0 20 0 1 0 560008530 25605136384 6229315 33554432000 4194304 6211132 140736698865728 140736698839288 140649757268965 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 6251254 6229315 819 493 0 6244559 0
Current children cumulated CPU time: 2381.77 s
Current children cumulated vsize: 25005016 KiB
Current children cumulated memory: 24917260 KiB

[startup+2442.3 s]
/proc/loadavg: 1.04 1.04 1.55 2/196 18253
/proc/meminfo: memFree=5787716/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=25609312 memory=25542020 CPUtime=2441.76 cores=0-7
/proc/18046/stat : 18046 (cosoco) R 18043 18046 18002 0 -1 4202496 6208792 0 0 0 242835 1341 0 0 20 0 1 0 560008530 26223935488 6385505 33554432000 4194304 6211132 140736698865728 140736698839288 140649757268965 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 6402328 6385505 819 493 0 6395633 0
Current children cumulated CPU time: 2441.76 s
Current children cumulated vsize: 25609312 KiB
Current children cumulated memory: 25542020 KiB

[startup+2502.3 s]
/proc/loadavg: 1.01 1.03 1.51 2/196 18256
/proc/meminfo: memFree=5163896/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=26213344 memory=26164736 CPUtime=2501.74 cores=0-7
/proc/18046/stat : 18046 (cosoco) R 18043 18046 18002 0 -1 4202496 6360291 0 0 0 248779 1395 0 0 20 0 1 0 560008530 26842464256 6541184 33554432000 4194304 6211132 140736698865728 140736698839288 140649757268965 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 6553336 6541184 819 493 0 6546641 0
Current children cumulated CPU time: 2501.74 s
Current children cumulated vsize: 26213344 KiB
Current children cumulated memory: 26164736 KiB



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+2520 s]*
/proc/loadavg: 1.01 1.03 1.51 2/196 18259
/proc/meminfo: memFree=4975984/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=26391544 memory=26349028 CPUtime=2519.46 cores=0-7
/proc/18046/stat : 18046 (cosoco) R 18043 18046 18002 0 -1 4202496 6404920 0 0 0 250534 1412 0 0 20 0 1 0 560008530 27024941056 6587257 33554432000 4194304 6211132 140736698865728 140736698839288 140649757268965 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 6597886 6587257 819 493 0 6591191 0
Current children cumulated CPU time: 2519.46 s
Current children cumulated vsize: 26391544 KiB
Current children cumulated memory: 26349028 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+2523.24 s]
# the end of solver process 18046 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2505.63 s, system=15.4026 s

Solver just ended.
Dumping a history of the last processes samples

[startup+2520.8 s]
/proc/loadavg: 1.01 1.03 1.51 2/196 18259
/proc/meminfo: memFree=4975984/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=26391544 memory=26349028 CPUtime=2519.59 cores=0-7
/proc/18046/stat : 18046 (cosoco) R 18043 18046 18002 0 -1 512 6404926 0 0 0 250546 1413 0 0 20 0 1 0 560008530 27024941056 6587257 33554432000 4194304 6211132 140736698865728 140736698839288 5189582 0 16384 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 6597886 6587257 819 493 0 6591191 0
Current children cumulated CPU time: 2519.59 s
Current children cumulated vsize: 26391544 KiB
Current children cumulated memory: 26349028 KiB

[startup+2521.6 s]
/proc/loadavg: 1.01 1.03 1.50 2/197 18260
/proc/meminfo: memFree=4415736/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=26391544 memory=26349028 CPUtime=2519.74 cores=0-7
/proc/18046/stat : 18046 (cosoco) S 18043 18046 18002 0 -1 512 6404926 0 0 0 250557 1417 0 0 20 0 1 0 560008530 27024941056 6587257 33554432000 4194304 6211132 140736698865728 140736698839288 5189582 0 16384 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 6597886 6587257 819 493 0 6591191 0
Current children cumulated CPU time: 2519.74 s
Current children cumulated vsize: 26391544 KiB
Current children cumulated memory: 26349028 KiB

[startup+2522 s]*
/proc/loadavg: 1.01 1.03 1.50 2/197 18260
/proc/meminfo: memFree=3863872/32770624 swapFree=66967820/67108860
[pid=18046] ppid=18043 vsize=26391544 memory=26349028 CPUtime=2519.8 cores=0-7
/proc/18046/stat : 18046 (cosoco) S 18043 18046 18002 0 -1 512 6404926 0 0 0 250563 1417 0 0 20 0 1 0 560008530 27024941056 6587257 33554432000 4194304 6211132 140736698865728 140736698839288 5189582 0 16384 4096 8405026 18446744071580971376 0 0 17 1 0 0 0 0 0 8312316 8317760 26107904 140736698872849 140736698872941 140736698872941 140736698875854 0
/proc/18046/statm: 6597886 6587257 819 493 0 6591191 0
Current children cumulated CPU time: 2519.8 s
Current children cumulated vsize: 26391544 KiB
Current children cumulated memory: 26349028 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 2523.24
CPU time (s): 2521.03
CPU user time (s): 2505.63
CPU system time (s): 15.4026
CPU usage (%): 99.9127
Max. virtual memory (cumulated for all children) (KiB): 26391544
Max. memory (cumulated for all children) (KiB): 26349028

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2505.63
system time used= 15.4026
maximum resident set size= 26349208
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 6404926
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 138670
involuntary context switches= 3793


# summary of solver processes directly reported to runsolver:
#   pid: 18046
#   total CPU time (s): 2521.03
#   total CPU user time (s): 2505.63
#   total CPU system time (s): 15.4026

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 8.7206 second user time and 17.2337 second system time

The end

Launcher Data 

Begin job on node133 at 2019-09-30 12:00:50
IDJOB=4409725
IDBENCH=139378
IDSOLVER=2877
FILE ID=node133/4409725-1569837691
RUNJOBID= node133-1569837650-18029
PBS_JOBID= 21705959
Free space on /tmp= 62032 MiB

SOLVER NAME= cosoco 2.0 parallel
BENCH NAME= XCSP17/SteelMillSlab/SteelMillSlab-bench_15_2.xml
COMMAND LINE= cosoco -model -nbcores=NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4409725-1569837691/watcher-4409725-1569837691 -o /tmp/evaluation-result-4409725-1569837691/solver-4409725-1569837691 -C 20160 -W 2520 -M 31000 --bin-var /tmp/runsolver-exch-node133-1569837650-18029 --watchdog 2580  cosoco -model -nbcores=8 HOME/instance-4409725-1569837691.xml

TIME LIMIT= 20160 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 5f96bf565600cbf50f8c2cf3fe757a5f
RANDOM SEED=818692590

node133.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        31372924 kB
MemAvailable:   31886056 kB
Buffers:           74908 kB
Cached:           737200 kB
SwapCached:        22164 kB
Active:           310772 kB
Inactive:         526756 kB
Active(anon):       5720 kB
Inactive(anon):    64744 kB
Active(file):     305052 kB
Inactive(file):   462012 kB
Unevictable:      108080 kB
Mlocked:          108080 kB
SwapTotal:      67108860 kB
SwapFree:       66967820 kB
Dirty:              2764 kB
Writeback:             0 kB
AnonPages:        113536 kB
Mapped:            25700 kB
Shmem:             36312 kB
Slab:             153040 kB
SReclaimable:     123808 kB
SUnreclaim:        29232 kB
KernelStack:        3376 kB
PageTables:         4768 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     538356 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     69632 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      122240 kB
DirectMap2M:    33423360 kB

Free space on /tmp at the end= 61492 MiB
End job on node133 at 2019-09-30 12:42:53