Trace number 4398490

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2.O parallel? (TO) 2520.59 2523.22

General information on the benchmark

NameSteelMillSlab/
SteelMillSlab-bench_9_11.xml
MD5SUMb3f80a69476de872363c069f96d9d14d
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkSAT TO
Best value of the objective obtained on this benchmark361
Best CPU time to get the best result obtained on this benchmark19756.9
Satisfiable
(Un)Satisfiability was proved
Number of variables12654
Number of constraints466
Number of domains4
Minimum domain size2
Maximum domain size111
Distribution of domain sizes[{"size":2,"count":12321},{"size":15,"count":111},{"size":51,"count":111},{"size":111,"count":111}]
Minimum variable degree2
Maximum variable degree114
Distribution of variable degrees[{"degree":2,"count":12432},{"degree":3,"count":111},{"degree":112,"count":78},{"degree":113,"count":24},{"degree":114,"count":9}]
Minimum constraint arity2
Maximum constraint arity112
Distribution of constraint arities[{"arity":2,"count":132},{"arity":111,"count":112},{"arity":112,"count":222}]
Number of extensional constraints0
Number of intensional constraints132
Distribution of constraint types[{"type":"intension","count":132},{"type":"sum","count":112},{"type":"element","count":111},{"type":"channel","count":111}]
Optimization problemYES
Type of objectivemin SUM

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (15414 MiB free)
  memory of node 1: 16384 MiB (15656 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4398490-1568614326/watcher-4398490-1568614326 -o /tmp/evaluation-result-4398490-1568614326/solver-4398490-1568614326 -C 20160 -W 2520 -M 31000 --bin-var /tmp/runsolver-exch-node132-1568611120-31382 --watchdog 2580 cosoco -model -nbcores=8 HOME/instance-4398490-1568614326.xml 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 20160 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 20190 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
Current StackSize limit: 8192 KiB

solver pid=31811, runsolver pid=31808

[startup+0.100074 s]*
/proc/loadavg: 5.17 4.84 6.28 2/197 31812
/proc/meminfo: memFree=31802836/32770624 swapFree=67011028/67108860
[pid=31811] ppid=31808 vsize=42408 memory=18448 CPUtime=0.05 cores=0-7
/proc/31811/stat : 31811 (cosoco) R 31808 31811 31355 0 -1 4202496 4805 0 1 0 5 0 0 0 20 0 1 0 437355276 43425792 4612 33554432000 4194304 6211092 140735779217760 140735779214296 140321171471333 0 0 4096 8405026 0 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 10602 4612 754 493 0 3907 0
Current children cumulated CPU time: 0.05 s
Current children cumulated vsize: 42408 KiB
Current children cumulated memory: 18448 KiB

[startup+0.209658 s]*
/proc/loadavg: 5.17 4.84 6.28 2/197 31812
/proc/meminfo: memFree=31747780/32770624 swapFree=67011028/67108860
[pid=31811] ppid=31808 vsize=92312 memory=68344 CPUtime=0.16 cores=0-7
/proc/31811/stat : 31811 (cosoco) R 31808 31811 31355 0 -1 4202496 17298 0 1 0 14 2 0 0 20 0 1 0 437355276 94527488 17086 33554432000 4194304 6211092 140735779217760 140735779214296 140321171471333 0 0 4096 8405026 0 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 23078 17086 808 493 0 16383 0
Current children cumulated CPU time: 0.16 s
Current children cumulated vsize: 92312 KiB
Current children cumulated memory: 68344 KiB

[startup+0.309691 s]*
/proc/loadavg: 5.17 4.84 6.28 2/197 31812
/proc/meminfo: memFree=31745920/32770624 swapFree=67011028/67108860
[pid=31811] ppid=31808 vsize=94156 memory=70336 CPUtime=0.26 cores=0-7
/proc/31811/stat : 31811 (cosoco) R 31808 31811 31355 0 -1 4202496 17783 0 1 0 24 2 0 0 20 0 1 0 437355276 96415744 17584 33554432000 4194304 6211092 140735779217760 140735779214296 140321171471333 0 0 4096 8405026 0 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 23539 17584 819 493 0 16844 0
Current children cumulated CPU time: 0.26 s
Current children cumulated vsize: 94156 KiB
Current children cumulated memory: 70336 KiB

[startup+0.700273 s]
/proc/loadavg: 5.17 4.84 6.28 2/197 31812
/proc/meminfo: memFree=31743688/32770624 swapFree=67011028/67108860
[pid=31811] ppid=31808 vsize=98264 memory=74296 CPUtime=0.64 cores=0-7
/proc/31811/stat : 31811 (cosoco) R 31808 31811 31355 0 -1 4202496 18772 0 1 0 62 2 0 0 20 0 1 0 437355276 100622336 18574 33554432000 4194304 6211092 140735779217760 140735779214296 140321171471333 0 0 4096 8405026 0 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 24566 18574 819 493 0 17871 0
Current children cumulated CPU time: 0.64 s
Current children cumulated vsize: 98264 KiB
Current children cumulated memory: 74296 KiB

[startup+1.50027 s]
/proc/loadavg: 5.17 4.84 6.28 2/197 31812
/proc/meminfo: memFree=31738372/32770624 swapFree=67011028/67108860
[pid=31811] ppid=31808 vsize=106584 memory=82600 CPUtime=1.44 cores=0-7
/proc/31811/stat : 31811 (cosoco) R 31808 31811 31355 0 -1 4202496 20842 0 1 0 142 2 0 0 20 0 1 0 437355276 109142016 20650 33554432000 4194304 6211092 140735779217760 140735779191320 140321171471333 0 0 4096 8405026 0 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 26646 20650 819 493 0 19951 0
Current children cumulated CPU time: 1.44 s
Current children cumulated vsize: 106584 KiB
Current children cumulated memory: 82600 KiB

[startup+3.10957 s]
/proc/loadavg: 4.83 4.78 6.25 2/197 31812
/proc/meminfo: memFree=31718256/32770624 swapFree=67011028/67108860
[pid=31811] ppid=31808 vsize=123532 memory=99356 CPUtime=3.06 cores=0-7
/proc/31811/stat : 31811 (cosoco) R 31808 31811 31355 0 -1 4202496 25256 0 1 0 303 3 0 0 20 0 1 0 437355276 126496768 24839 33554432000 4194304 6211092 140735779217760 140735779191320 140321171471333 0 0 4096 8405026 0 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 30883 24839 819 493 0 24188 0
Current children cumulated CPU time: 3.06 s
Current children cumulated vsize: 123532 KiB
Current children cumulated memory: 99356 KiB

[startup+6.30026 s]
/proc/loadavg: 4.83 4.78 6.25 2/197 31812
/proc/meminfo: memFree=31687040/32770624 swapFree=67011028/67108860
[pid=31811] ppid=31808 vsize=157076 memory=132284 CPUtime=6.24 cores=0-7
/proc/31811/stat : 31811 (cosoco) R 31808 31811 31355 0 -1 4202496 33766 0 1 0 621 3 0 0 20 0 1 0 437355276 160845824 33071 33554432000 4194304 6211092 140735779217760 140735779191320 140321171471333 0 0 4096 8405026 0 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 39269 33071 819 493 0 32574 0
Current children cumulated CPU time: 6.24 s
Current children cumulated vsize: 157076 KiB
Current children cumulated memory: 132284 KiB

[startup+12.7003 s]
/proc/loadavg: 4.24 4.66 6.19 2/198 31814
/proc/meminfo: memFree=31624620/32770624 swapFree=67011028/67108860
[pid=31811] ppid=31808 vsize=222668 memory=198540 CPUtime=12.64 cores=0-7
/proc/31811/stat : 31811 (cosoco) R 31808 31811 31355 0 -1 4202496 50578 0 1 0 1258 6 0 0 20 0 1 0 437355276 228012032 49635 33554432000 4194304 6211092 140735779217760 140735779191320 140321171471333 0 0 4096 8405026 0 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 55667 49635 819 493 0 48972 0
Current children cumulated CPU time: 12.64 s
Current children cumulated vsize: 222668 KiB
Current children cumulated memory: 198540 KiB

[startup+25.5003 s]
/proc/loadavg: 3.74 4.54 6.14 2/198 31815
/proc/meminfo: memFree=31491400/32770624 swapFree=67011028/67108860
[pid=31811] ppid=31808 vsize=356504 memory=331484 CPUtime=25.44 cores=0-7
/proc/31811/stat : 31811 (cosoco) R 31808 31811 31355 0 -1 4202496 83778 0 1 0 2533 11 0 0 20 0 1 0 437355276 365060096 82871 33554432000 4194304 6211092 140735779217760 140735779191320 140321171471333 0 0 4096 8405026 0 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 89126 82871 819 493 0 82431 0
Current children cumulated CPU time: 25.44 s
Current children cumulated vsize: 356504 KiB
Current children cumulated memory: 331484 KiB

[startup+51.1097 s]
/proc/loadavg: 2.81 4.25 6.00 2/199 31817
/proc/meminfo: memFree=31220924/32770624 swapFree=67011028/67108860
[pid=31811] ppid=31808 vsize=626452 memory=598788 CPUtime=51.03 cores=0-7
/proc/31811/stat : 31811 (cosoco) R 31808 31811 31355 0 -1 4202496 149747 0 1 0 5084 19 0 0 20 0 1 0 437355276 641486848 149697 33554432000 4194304 6211092 140735779217760 140735779191320 140321171471333 0 0 4096 8405026 0 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 156613 149697 819 493 0 149918 0
Current children cumulated CPU time: 51.03 s
Current children cumulated vsize: 626452 KiB
Current children cumulated memory: 598788 KiB

[startup+102.3 s]
/proc/loadavg: 1.72 3.70 5.71 2/200 31821
/proc/meminfo: memFree=30689828/32770624 swapFree=67011028/67108860
[pid=31811] ppid=31808 vsize=1170836 memory=1131012 CPUtime=102.2 cores=0-7
/proc/31811/stat : 31811 (cosoco) R 31808 31811 31355 0 -1 4202496 279692 0 1 0 10184 36 0 0 20 0 1 0 437355276 1198936064 282753 33554432000 4194304 6211092 140735779217760 140735779191320 140321171471333 0 0 4096 8405026 0 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 292709 282753 819 493 0 286014 0
Current children cumulated CPU time: 102.2 s
Current children cumulated vsize: 1170836 KiB
Current children cumulated memory: 1131012 KiB

[startup+162.3 s]
/proc/loadavg: 1.27 3.21 5.42 2/200 31825
/proc/meminfo: memFree=30062352/32770624 swapFree=67011028/67108860
[pid=31811] ppid=31808 vsize=1802112 memory=1757640 CPUtime=162.17 cores=0-7
/proc/31811/stat : 31811 (cosoco) R 31808 31811 31355 0 -1 4202496 432269 0 1 0 16163 54 0 0 20 0 1 0 437355276 1845362688 439410 33554432000 4194304 6211092 140735779217760 140735779191320 140321171471333 0 0 4096 8405026 0 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 450528 439410 819 493 0 443833 0
Current children cumulated CPU time: 162.17 s
Current children cumulated vsize: 1802112 KiB
Current children cumulated memory: 1757640 KiB

[startup+222.3 s]
/proc/loadavg: 1.17 2.83 5.15 2/200 31828
/proc/meminfo: memFree=29434464/32770624 swapFree=67011028/67108860
[pid=31811] ppid=31808 vsize=2408784 memory=2384664 CPUtime=222.15 cores=0-7
/proc/31811/stat : 31811 (cosoco) R 31808 31811 31355 0 -1 4202496 583961 0 1 0 22141 74 0 0 20 0 1 0 437355276 2466594816 596166 33554432000 4194304 6211092 140735779217760 140735779191320 140321171471333 0 0 4096 8405026 0 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 602196 596166 819 493 0 595501 0
Current children cumulated CPU time: 222.15 s
Current children cumulated vsize: 2408784 KiB
Current children cumulated memory: 2384664 KiB

[startup+282.3 s]
/proc/loadavg: 1.06 2.49 4.89 2/201 31833
/proc/meminfo: memFree=28808464/32770624 swapFree=67011028/67108860
[pid=31811] ppid=31808 vsize=3052236 memory=3009744 CPUtime=282.12 cores=0-7
/proc/31811/stat : 31811 (cosoco) R 31808 31811 31355 0 -1 4202496 736176 0 1 0 28119 93 0 0 20 0 1 0 437355276 3125489664 752436 33554432000 4194304 6211092 140735779217760 140735779191320 140321171467070 0 0 4096 8405026 0 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 763059 752436 819 493 0 756364 0
Current children cumulated CPU time: 282.12 s
Current children cumulated vsize: 3052236 KiB
Current children cumulated memory: 3009744 KiB

[startup+342.3 s]
/proc/loadavg: 1.02 2.22 4.64 2/197 31836
/proc/meminfo: memFree=28181300/32770624 swapFree=67011028/67108860
[pid=31811] ppid=31808 vsize=3714008 memory=3635688 CPUtime=342.09 cores=0-7
/proc/31811/stat : 31811 (cosoco) R 31808 31811 31355 0 -1 4202496 888078 0 1 0 34097 112 0 0 20 0 1 0 437355276 3803144192 908922 33554432000 4194304 6211092 140735779217760 140735779191320 140321171471333 0 0 4096 8405026 0 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 928502 908922 819 493 0 921807 0
Current children cumulated CPU time: 342.09 s
Current children cumulated vsize: 3714008 KiB
Current children cumulated memory: 3635688 KiB

[startup+402.3 s]
/proc/loadavg: 1.01 2.00 4.42 2/198 31840
/proc/meminfo: memFree=27553868/32770624 swapFree=67011028/67108860
[pid=31811] ppid=31808 vsize=4320548 memory=4263680 CPUtime=402.08 cores=0-7
/proc/31811/stat : 31811 (cosoco) R 31808 31811 31355 0 -1 4202496 1040040 0 1 0 40074 134 0 0 20 0 1 0 437355276 4424241152 1065920 33554432000 4194304 6211092 140735779217760 140735779191320 140321171471333 0 0 4096 8405026 0 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 1080137 1065920 819 493 0 1073442 0
Current children cumulated CPU time: 402.08 s
Current children cumulated vsize: 4320548 KiB
Current children cumulated memory: 4263680 KiB

[startup+462.3 s]
/proc/loadavg: 1.00 1.82 4.20 2/197 31845
/proc/meminfo: memFree=26925652/32770624 swapFree=67011028/67108860
[pid=31811] ppid=31808 vsize=4926824 memory=4888532 CPUtime=462.06 cores=0-7
/proc/31811/stat : 31811 (cosoco) R 31808 31811 31355 0 -1 4202496 1191694 0 1 0 46053 153 0 0 20 0 1 0 437355276 5045067776 1222133 33554432000 4194304 6211092 140735779217760 140735779191320 140321171471333 0 0 4096 8405026 0 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 1231706 1222133 819 493 0 1225011 0
Current children cumulated CPU time: 462.06 s
Current children cumulated vsize: 4926824 KiB
Current children cumulated memory: 4888532 KiB


################
# More data... #
################

Current children cumulated vsize: 20131340 KiB
Current children cumulated memory: 19887900 KiB

[startup+1962.3 s]
/proc/loadavg: 1.01 1.05 1.69 2/196 31961
/proc/meminfo: memFree=11272964/32770624 swapFree=67011048/67108860
[pid=31811] ppid=31808 vsize=20735504 memory=20510680 CPUtime=1961.57 cores=0-7
/proc/31811/stat : 31811 (cosoco) R 31808 31811 31355 0 -1 4202496 4980242 0 1 0 195228 929 0 0 20 0 1 0 437355276 21233156096 5127670 33554432000 4194304 6211092 140735779217760 140735779191320 140321171471333 0 0 4096 8405026 0 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 5183876 5127670 819 493 0 5177181 0
Current children cumulated CPU time: 1961.57 s
Current children cumulated vsize: 20735504 KiB
Current children cumulated memory: 20510680 KiB

[startup+2022.3 s]
/proc/loadavg: 1.01 1.04 1.65 2/196 31964
/proc/meminfo: memFree=10646820/32770624 swapFree=67011048/67108860
[pid=31811] ppid=31808 vsize=21339536 memory=21134976 CPUtime=2021.55 cores=0-7
/proc/31811/stat : 31811 (cosoco) R 31808 31811 31355 0 -1 4202496 5131347 0 1 0 201183 972 0 0 20 0 1 0 437355276 21851684864 5283744 33554432000 4194304 6211092 140735779217760 140735779191320 140321171471333 0 0 4096 8405026 0 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 5334884 5283744 819 493 0 5328189 0
Current children cumulated CPU time: 2021.55 s
Current children cumulated vsize: 21339536 KiB
Current children cumulated memory: 21134976 KiB

[startup+2082.3 s]
/proc/loadavg: 1.00 1.03 1.60 2/198 31970
/proc/meminfo: memFree=10023028/32770624 swapFree=67011048/67108860
[pid=31811] ppid=31808 vsize=21943436 memory=21757492 CPUtime=2081.52 cores=0-7
/proc/31811/stat : 31811 (cosoco) R 31808 31811 31355 0 -1 4202496 5282466 0 1 0 207134 1018 0 0 20 0 1 0 437355276 22470078464 5439373 33554432000 4194304 6211092 140735779217760 140735779191320 140321171471333 0 0 4096 8405026 0 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 5485859 5439373 819 493 0 5479164 0
Current children cumulated CPU time: 2081.52 s
Current children cumulated vsize: 21943436 KiB
Current children cumulated memory: 21757492 KiB

[startup+2142.3 s]
/proc/loadavg: 1.00 1.03 1.57 2/197 31973
/proc/meminfo: memFree=9397860/32770624 swapFree=67011048/67108860
[pid=31811] ppid=31808 vsize=22547336 memory=22381700 CPUtime=2141.49 cores=0-7
/proc/31811/stat : 31811 (cosoco) R 31808 31811 31355 0 -1 4202496 5433584 0 1 0 213090 1059 0 0 20 0 1 0 437355276 23088472064 5595425 33554432000 4194304 6211092 140735779217760 140735779191320 140321171471333 0 0 4096 8405026 0 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 5636834 5595425 819 493 0 5630139 0
Current children cumulated CPU time: 2141.49 s
Current children cumulated vsize: 22547336 KiB
Current children cumulated memory: 22381700 KiB

[startup+2202.3 s]
/proc/loadavg: 1.00 1.02 1.53 2/197 31977
/proc/meminfo: memFree=8774476/32770624 swapFree=67011048/67108860
[pid=31811] ppid=31808 vsize=23150972 memory=23003688 CPUtime=2201.48 cores=0-7
/proc/31811/stat : 31811 (cosoco) R 31808 31811 31355 0 -1 4202496 5584598 0 1 0 219039 1109 0 0 20 0 1 0 437355276 23706595328 5750922 33554432000 4194304 6211092 140735779217760 140735779191320 140321171471333 0 0 4096 8405026 0 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 5787743 5750922 819 493 0 5781048 0
Current children cumulated CPU time: 2201.48 s
Current children cumulated vsize: 23150972 KiB
Current children cumulated memory: 23003688 KiB

[startup+2262.3 s]
/proc/loadavg: 1.00 1.02 1.50 2/198 31982
/proc/meminfo: memFree=8148472/32770624 swapFree=67011048/67108860
[pid=31811] ppid=31808 vsize=23754476 memory=23627896 CPUtime=2261.45 cores=0-7
/proc/31811/stat : 31811 (cosoco) R 31808 31811 31355 0 -1 4202496 5735632 0 1 0 224989 1156 0 0 20 0 1 0 437355276 24324583424 5906974 33554432000 4194304 6211092 140735779217760 140735779191320 140321171471333 0 0 4096 8405026 0 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 5938619 5906974 819 493 0 5931924 0
Current children cumulated CPU time: 2261.45 s
Current children cumulated vsize: 23754476 KiB
Current children cumulated memory: 23627896 KiB

[startup+2322.3 s]
/proc/loadavg: 1.00 1.01 1.47 2/197 31985
/proc/meminfo: memFree=7525880/32770624 swapFree=67011048/67108860
[pid=31811] ppid=31808 vsize=24358112 memory=24249884 CPUtime=2321.44 cores=0-7
/proc/31811/stat : 31811 (cosoco) R 31808 31811 31355 0 -1 4202496 5886644 0 1 0 230941 1203 0 0 20 0 1 0 437355276 24942706688 6062471 33554432000 4194304 6211092 140735779217760 140735779191320 140321171471333 0 0 4096 8405026 0 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 6089528 6062471 819 493 0 6082833 0
Current children cumulated CPU time: 2321.44 s
Current children cumulated vsize: 24358112 KiB
Current children cumulated memory: 24249884 KiB

[startup+2382.3 s]
/proc/loadavg: 1.00 1.01 1.44 2/196 31993
/proc/meminfo: memFree=6900708/32770624 swapFree=67011048/67108860
[pid=31811] ppid=31808 vsize=24961352 memory=24873564 CPUtime=2381.42 cores=0-7
/proc/31811/stat : 31811 (cosoco) R 31808 31811 31355 0 -1 4202496 6037608 0 1 0 236887 1255 0 0 20 0 1 0 437355276 25560424448 6218391 33554432000 4194304 6211092 140735779217760 140735779191320 140321171471333 0 0 4096 8405026 0 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 6240338 6218391 819 493 0 6233643 0
Current children cumulated CPU time: 2381.42 s
Current children cumulated vsize: 24961352 KiB
Current children cumulated memory: 24873564 KiB

[startup+2442.3 s]
/proc/loadavg: 1.00 1.01 1.41 2/197 31998
/proc/meminfo: memFree=6277324/32770624 swapFree=67011048/67108860
[pid=31811] ppid=31808 vsize=25564724 memory=25495464 CPUtime=2441.39 cores=0-7
/proc/31811/stat : 31811 (cosoco) R 31808 31811 31355 0 -1 4202496 6188573 0 1 0 242835 1304 0 0 20 0 1 0 437355276 26178277376 6373866 33554432000 4194304 6211092 140735779217760 140735779191320 140321171471333 0 0 4096 8405026 0 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 6391181 6373866 819 493 0 6384486 0
Current children cumulated CPU time: 2441.39 s
Current children cumulated vsize: 25564724 KiB
Current children cumulated memory: 25495464 KiB

[startup+2502.3 s]
/proc/loadavg: 1.00 1.01 1.39 2/195 32001
/proc/meminfo: memFree=5652760/32770624 swapFree=67011048/67108860
[pid=31811] ppid=31808 vsize=26167964 memory=26118968 CPUtime=2501.37 cores=0-7
/proc/31811/stat : 31811 (cosoco) R 31808 31811 31355 0 -1 4202496 6339506 0 1 0 248782 1355 0 0 20 0 1 0 437355276 26795995136 6529742 33554432000 4194304 6211092 140735779217760 140735779191320 140321171471333 0 0 4096 8405026 0 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 6541991 6529742 819 493 0 6535296 0
Current children cumulated CPU time: 2501.37 s
Current children cumulated vsize: 26167964 KiB
Current children cumulated memory: 26118968 KiB



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+2520 s]*
/proc/loadavg: 1.00 1.01 1.38 2/197 32004
/proc/meminfo: memFree=5467192/32770624 swapFree=67011048/67108860
[pid=31811] ppid=31808 vsize=26346164 memory=26301344 CPUtime=2519.08 cores=0-7
/proc/31811/stat : 31811 (cosoco) R 31808 31811 31355 0 -1 4202496 6384094 0 1 0 250538 1370 0 0 20 0 1 0 437355276 26978471936 6575336 33554432000 4194304 6211092 140735779217760 140735779191320 140321171471333 0 0 4096 8405026 0 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 6586541 6575336 819 493 0 6579846 0
Current children cumulated CPU time: 2519.08 s
Current children cumulated vsize: 26346164 KiB
Current children cumulated memory: 26301344 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+2523.22 s]
# the end of solver process 31811 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2505.66 s, system=14.9367 s

Solver just ended.
Dumping a history of the last processes samples

[startup+2520.8 s]
/proc/loadavg: 1.00 1.01 1.38 2/197 32004
/proc/meminfo: memFree=5467192/32770624 swapFree=67011048/67108860
[pid=31811] ppid=31808 vsize=26346164 memory=26301344 CPUtime=2519.18 cores=0-7
/proc/31811/stat : 31811 (cosoco) S 31808 31811 31355 0 -1 512 6384099 0 1 0 250547 1371 0 0 20 0 1 0 437355276 26978471936 6575336 33554432000 4194304 6211092 140735779217760 140735779191320 5189550 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 6586541 6575336 819 493 0 6579846 0
Current children cumulated CPU time: 2519.18 s
Current children cumulated vsize: 26346164 KiB
Current children cumulated memory: 26301344 KiB

[startup+2521.6 s]
/proc/loadavg: 1.00 1.01 1.38 2/198 32005
/proc/meminfo: memFree=5041732/32770624 swapFree=67011048/67108860
[pid=31811] ppid=31808 vsize=26346164 memory=26301344 CPUtime=2519.31 cores=0-7
/proc/31811/stat : 31811 (cosoco) S 31808 31811 31355 0 -1 512 6384099 0 1 0 250559 1372 0 0 20 0 1 0 437355276 26978471936 6575336 33554432000 4194304 6211092 140735779217760 140735779191320 5189550 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 6586541 6575336 819 493 0 6579846 0
Current children cumulated CPU time: 2519.31 s
Current children cumulated vsize: 26346164 KiB
Current children cumulated memory: 26301344 KiB

[startup+2522 s]*
/proc/loadavg: 1.00 1.01 1.38 2/198 32005
/proc/meminfo: memFree=4505880/32770624 swapFree=67011048/67108860
[pid=31811] ppid=31808 vsize=26346164 memory=26301344 CPUtime=2519.37 cores=0-7
/proc/31811/stat : 31811 (cosoco) S 31808 31811 31355 0 -1 512 6384099 0 1 0 250565 1372 0 0 20 0 1 0 437355276 26978471936 6575336 33554432000 4194304 6211092 140735779217760 140735779191320 5189550 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 2 0 0 8312316 8317760 26636288 140735779222545 140735779222637 140735779222637 140735779225550 0
/proc/31811/statm: 6586541 6575336 819 493 0 6579846 0
Current children cumulated CPU time: 2519.37 s
Current children cumulated vsize: 26346164 KiB
Current children cumulated memory: 26301344 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 2523.22
CPU time (s): 2520.59
CPU user time (s): 2505.66
CPU system time (s): 14.9367
CPU usage (%): 99.8958
Max. virtual memory (cumulated for all children) (KiB): 26346164
Max. memory (cumulated for all children) (KiB): 26301344

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2505.66
system time used= 14.9367
maximum resident set size= 26301496
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 6384099
page faults= 1
swaps= 0
block input operations= 9984
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 121902
involuntary context switches= 2080


# summary of solver processes directly reported to runsolver:
#   pid: 31811
#   total CPU time (s): 2520.59
#   total CPU user time (s): 2505.66
#   total CPU system time (s): 14.9367

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 9.64546 second user time and 16.7136 second system time

The end

Launcher Data 

Begin job on node132 at 2019-09-16 08:11:16
IDJOB=4398490
IDBENCH=139658
IDSOLVER=2851
FILE ID=node132/4398490-1568614326
RUNJOBID= node132-1568611120-31382
PBS_JOBID= 21703453
Free space on /tmp= 61624 MiB

SOLVER NAME= cosoco 2.O parallel
BENCH NAME= XCSP17/SteelMillSlab/SteelMillSlab-bench_9_11.xml
COMMAND LINE= cosoco -model -nbcores=NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4398490-1568614326/watcher-4398490-1568614326 -o /tmp/evaluation-result-4398490-1568614326/solver-4398490-1568614326 -C 20160 -W 2520 -M 31000 --bin-var /tmp/runsolver-exch-node132-1568611120-31382 --watchdog 2580  cosoco -model -nbcores=8 HOME/instance-4398490-1568614326.xml

TIME LIMIT= 20160 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= b3f80a69476de872363c069f96d9d14d
RANDOM SEED=1960905207

node132.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.95
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.95
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.95
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.95
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        31818128 kB
MemAvailable:   31919880 kB
Buffers:           64480 kB
Cached:           323804 kB
SwapCached:         5952 kB
Active:           341932 kB
Inactive:          52240 kB
Active(anon):        588 kB
Inactive(anon):    35208 kB
Active(file):     341344 kB
Inactive(file):    17032 kB
Unevictable:      109612 kB
Mlocked:          109612 kB
SwapTotal:      67108860 kB
SwapFree:       67011028 kB
Dirty:              2724 kB
Writeback:             0 kB
AnonPages:        112440 kB
Mapped:            34784 kB
Shmem:             20788 kB
Slab:             148384 kB
SReclaimable:     118420 kB
SUnreclaim:        29964 kB
KernelStack:        3456 kB
PageTables:         5236 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     475636 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     67584 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      136576 kB
DirectMap2M:    33409024 kB

Free space on /tmp at the end= 61152 MiB
End job on node132 at 2019-09-16 08:53:19