Trace number 4409735

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2.0 parallel? (TO) 2520.64 2523.24

General information on the benchmark

NameSteelMillSlab/
SteelMillSlab-bench_13_6.xml
MD5SUM0b4cdd9f2ac3a0985a9efe7e14eeda02
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkSAT TO
Best value of the objective obtained on this benchmark268
Best CPU time to get the best result obtained on this benchmark20083.5
Satisfiable
(Un)Satisfiability was proved
Number of variables12654
Number of constraints466
Number of domains4
Minimum domain size2
Maximum domain size111
Distribution of domain sizes[{"size":2,"count":12321},{"size":12,"count":111},{"size":49,"count":111},{"size":111,"count":111}]
Minimum variable degree2
Maximum variable degree114
Distribution of variable degrees[{"degree":2,"count":12432},{"degree":3,"count":111},{"degree":112,"count":78},{"degree":113,"count":24},{"degree":114,"count":9}]
Minimum constraint arity2
Maximum constraint arity112
Distribution of constraint arities[{"arity":2,"count":132},{"arity":111,"count":112},{"arity":112,"count":222}]
Number of extensional constraints0
Number of intensional constraints132
Distribution of constraint types[{"type":"intension","count":132},{"type":"sum","count":112},{"type":"element","count":111},{"type":"channel","count":111}]
Optimization problemYES
Type of objectivemin SUM

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (15257 MiB free)
  memory of node 1: 16384 MiB (15540 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4409735-1569838515/watcher-4409735-1569838515 -o /tmp/evaluation-result-4409735-1569838515/solver-4409735-1569838515 -C 20160 -W 2520 -M 31000 --bin-var /tmp/runsolver-exch-node104-1569826684-2698 --watchdog 2580 cosoco -model -nbcores=8 HOME/instance-4409735-1569838515.xml 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 20160 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 20190 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
Current StackSize limit: 8192 KiB

solver pid=3584, runsolver pid=3581

[startup+0.100155 s]*
/proc/loadavg: 8.19 8.27 8.27 2/192 3585
/proc/meminfo: memFree=31502372/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=66796 memory=42800 CPUtime=0.09 cores=0-7
/proc/3584/stat : 3584 (cosoco) R 3581 3584 2671 0 -1 4202496 10866 0 0 0 8 1 0 0 20 0 1 0 560107774 68399104 10700 33554432000 4194304 6211132 140728482529872 140728482526408 140132532698085 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 16699 10700 754 493 0 10004 0
Current children cumulated CPU time: 0.09 s
Current children cumulated vsize: 66796 KiB
Current children cumulated memory: 42800 KiB

[startup+0.209045 s]*
/proc/loadavg: 8.19 8.27 8.27 2/192 3585
/proc/meminfo: memFree=31472612/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=93188 memory=69200 CPUtime=0.21 cores=0-7
/proc/3584/stat : 3584 (cosoco) R 3581 3584 2671 0 -1 4202496 17518 0 0 0 19 2 0 0 20 0 1 0 560107774 95424512 17300 33554432000 4194304 6211132 140728482529872 140728482526408 5734848 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 23297 17300 809 493 0 16602 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 93188 KiB
Current children cumulated memory: 69200 KiB

[startup+0.308794 s]*
/proc/loadavg: 8.19 8.27 8.27 2/192 3585
/proc/meminfo: memFree=31470752/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=94628 memory=70808 CPUtime=0.3 cores=0-7
/proc/3584/stat : 3584 (cosoco) R 3581 3584 2671 0 -1 4202496 17890 0 0 0 28 2 0 0 20 0 1 0 560107774 96899072 17702 33554432000 4194304 6211132 140728482529872 140728482526408 140132532698085 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 23657 17702 819 493 0 16962 0
Current children cumulated CPU time: 0.3 s
Current children cumulated vsize: 94628 KiB
Current children cumulated memory: 70808 KiB

[startup+0.700325 s]
/proc/loadavg: 8.19 8.27 8.27 2/192 3585
/proc/meminfo: memFree=31468768/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=98604 memory=74768 CPUtime=0.69 cores=0-7
/proc/3584/stat : 3584 (cosoco) R 3581 3584 2671 0 -1 4202496 18886 0 0 0 67 2 0 0 20 0 1 0 560107774 100970496 18692 33554432000 4194304 6211132 140728482529872 140728482526408 140132532698085 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 24651 18692 819 493 0 17956 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 98604 KiB
Current children cumulated memory: 74768 KiB

[startup+1.50033 s]
/proc/loadavg: 8.19 8.27 8.27 2/192 3585
/proc/meminfo: memFree=31463520/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=107056 memory=83068 CPUtime=1.49 cores=0-7
/proc/3584/stat : 3584 (cosoco) R 3581 3584 2671 0 -1 4202496 20958 0 0 0 147 2 0 0 20 0 1 0 560107774 109625344 20767 33554432000 4194304 6211132 140728482529872 140728482503432 140132532698085 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 26764 20767 819 493 0 20069 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 107056 KiB
Current children cumulated memory: 83068 KiB

[startup+3.10881 s]
/proc/loadavg: 7.61 8.15 8.23 2/192 3585
/proc/meminfo: memFree=31443384/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=124004 memory=99828 CPUtime=3.1 cores=0-7
/proc/3584/stat : 3584 (cosoco) R 3581 3584 2671 0 -1 4202496 25375 0 0 0 307 3 0 0 20 0 1 0 560107774 126980096 24957 33554432000 4194304 6211132 140728482529872 140728482503432 5012962 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 31001 24957 819 493 0 24306 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 124004 KiB
Current children cumulated memory: 99828 KiB

[startup+6.30035 s]
/proc/loadavg: 7.61 8.15 8.23 2/192 3585
/proc/meminfo: memFree=31411568/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=157548 memory=132748 CPUtime=6.29 cores=0-7
/proc/3584/stat : 3584 (cosoco) R 3581 3584 2671 0 -1 4202496 33885 0 0 0 625 4 0 0 20 0 1 0 560107774 161329152 33187 33554432000 4194304 6211132 140728482529872 140728482503432 140132532698085 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 39387 33187 819 493 0 32692 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 157548 KiB
Current children cumulated memory: 132748 KiB

[startup+12.7003 s]
/proc/loadavg: 7.08 8.03 8.19 2/192 3585
/proc/meminfo: memFree=31349108/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=223272 memory=199272 CPUtime=12.69 cores=0-7
/proc/3584/stat : 3584 (cosoco) R 3581 3584 2671 0 -1 4202496 50736 0 0 0 1262 7 0 0 20 0 1 0 560107774 228630528 49818 33554432000 4194304 6211132 140728482529872 140728482503432 140132532698085 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 55818 49818 819 493 0 49123 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 223272 KiB
Current children cumulated memory: 199272 KiB

[startup+25.5003 s]
/proc/loadavg: 5.74 7.69 8.08 2/190 3585
/proc/meminfo: memFree=31214536/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=357240 memory=331752 CPUtime=25.48 cores=0-7
/proc/3584/stat : 3584 (cosoco) R 3581 3584 2671 0 -1 4202496 84397 0 0 0 2536 12 0 0 20 0 1 0 560107774 365813760 82938 33554432000 4194304 6211132 140728482529872 140728482503432 140132532698085 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 89310 82938 819 493 0 82615 0
Current children cumulated CPU time: 25.48 s
Current children cumulated vsize: 357240 KiB
Current children cumulated memory: 331752 KiB

[startup+51.1086 s]
/proc/loadavg: 4.12 7.15 7.89 2/190 3588
/proc/meminfo: memFree=30942112/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=627056 memory=599440 CPUtime=51.08 cores=0-7
/proc/3584/stat : 3584 (cosoco) R 3581 3584 2671 0 -1 4202496 150377 0 0 0 5088 20 0 0 20 0 1 0 560107774 642105344 149860 33554432000 4194304 6211132 140728482529872 140728482503432 140132532698085 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 156764 149860 819 493 0 150069 0
Current children cumulated CPU time: 51.08 s
Current children cumulated vsize: 627056 KiB
Current children cumulated memory: 599440 KiB

[startup+102.3 s]
/proc/loadavg: 2.35 6.20 7.53 2/190 3591
/proc/meminfo: memFree=30412056/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=1169988 memory=1130164 CPUtime=102.24 cores=0-7
/proc/3584/stat : 3584 (cosoco) R 3581 3584 2671 0 -1 4202496 279991 0 0 0 10188 36 0 0 20 0 1 0 560107774 1198067712 282541 33554432000 4194304 6211132 140728482529872 140728482503432 140132532698085 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 292497 282541 819 493 0 285802 0
Current children cumulated CPU time: 102.24 s
Current children cumulated vsize: 1169988 KiB
Current children cumulated memory: 1130164 KiB

[startup+162.3 s]
/proc/loadavg: 1.50 5.26 7.12 2/190 3594
/proc/meminfo: memFree=29787232/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=1799020 memory=1754524 CPUtime=162.21 cores=0-7
/proc/3584/stat : 3584 (cosoco) R 3581 3584 2671 0 -1 4202496 432092 0 0 0 16167 54 0 0 20 0 1 0 560107774 1842196480 438631 33554432000 4194304 6211132 140728482529872 140728482503432 140132532698085 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 449755 438631 819 493 0 443060 0
Current children cumulated CPU time: 162.21 s
Current children cumulated vsize: 1799020 KiB
Current children cumulated memory: 1754524 KiB

[startup+222.3 s]
/proc/loadavg: 1.18 4.48 6.73 2/190 3599
/proc/meminfo: memFree=29160500/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=2403448 memory=2379280 CPUtime=222.19 cores=0-7
/proc/3584/stat : 3584 (cosoco) R 3581 3584 2671 0 -1 4202496 583318 0 0 0 22144 75 0 0 20 0 1 0 560107774 2461130752 594820 33554432000 4194304 6211132 140728482529872 140728482503432 140132532698085 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 600862 594820 819 493 0 594167 0
Current children cumulated CPU time: 222.19 s
Current children cumulated vsize: 2403448 KiB
Current children cumulated memory: 2379280 KiB

[startup+282.3 s]
/proc/loadavg: 1.07 3.85 6.38 2/190 3602
/proc/meminfo: memFree=28536944/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=3044656 memory=3002144 CPUtime=282.15 cores=0-7
/proc/3584/stat : 3584 (cosoco) R 3581 3584 2671 0 -1 4202496 735094 0 0 0 28120 95 0 0 20 0 1 0 560107774 3117727744 750536 33554432000 4194304 6211132 140728482529872 140728482503432 140132532698085 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 761164 750536 819 493 0 754469 0
Current children cumulated CPU time: 282.15 s
Current children cumulated vsize: 3044656 KiB
Current children cumulated memory: 3002144 KiB

[startup+342.3 s]
/proc/loadavg: 1.02 3.33 6.04 2/190 3612
/proc/meminfo: memFree=27910760/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=3704316 memory=3625448 CPUtime=342.13 cores=0-7
/proc/3584/stat : 3584 (cosoco) R 3581 3584 2671 0 -1 4202496 886577 0 0 0 34097 116 0 0 20 0 1 0 560107774 3793219584 906362 33554432000 4194304 6211132 140728482529872 140728482503432 140132532698085 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 926079 906362 819 493 0 919384 0
Current children cumulated CPU time: 342.13 s
Current children cumulated vsize: 3704316 KiB
Current children cumulated memory: 3625448 KiB

[startup+402.3 s]
/proc/loadavg: 1.01 2.91 5.72 2/190 3617
/proc/meminfo: memFree=27285084/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=4308480 memory=4249460 CPUtime=402.1 cores=0-7
/proc/3584/stat : 3584 (cosoco) R 3581 3584 2671 0 -1 4202496 1038236 0 0 0 40073 137 0 0 20 0 1 0 560107774 4411883520 1062365 33554432000 4194304 6211132 140728482529872 140728482503432 140132532698085 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 1077120 1062365 819 493 0 1070425 0
Current children cumulated CPU time: 402.1 s
Current children cumulated vsize: 4308480 KiB
Current children cumulated memory: 4249460 KiB

[startup+462.3 s]
/proc/loadavg: 1.00 2.56 5.43 2/190 3620
/proc/meminfo: memFree=26659972/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=4912512 memory=4874000 CPUtime=462.08 cores=0-7
/proc/3584/stat : 3584 (cosoco) R 3581 3584 2671 0 -1 4202496 1189656 0 0 0 46045 163 0 0 20 0 1 0 560107774 5030412288 1218500 33554432000 4194304 6211132 140728482529872 140728482503432 140132532698085 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 1228128 1218500 819 493 0 1221433 0
Current children cumulated CPU time: 462.08 s
Current children cumulated vsize: 4912512 KiB
Current children cumulated memory: 4874000 KiB


################
# More data... #
################

Current children cumulated vsize: 20057100 KiB
Current children cumulated memory: 19809612 KiB

[startup+1962.3 s]
/proc/loadavg: 1.00 1.01 1.88 2/190 3731
/proc/meminfo: memFree=11072496/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=20658624 memory=20431628 CPUtime=1961.57 cores=0-7
/proc/3584/stat : 3584 (cosoco) R 3581 3584 2671 0 -1 4202496 4974826 0 0 0 195185 972 0 0 20 0 1 0 560107774 21154430976 5107907 33554432000 4194304 6211132 140728482529872 140728482503432 140132532698085 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 5164656 5107907 819 493 0 5157961 0
Current children cumulated CPU time: 1961.57 s
Current children cumulated vsize: 20658624 KiB
Current children cumulated memory: 20431628 KiB

[startup+2022.3 s]
/proc/loadavg: 1.00 1.01 1.83 2/190 3736
/proc/meminfo: memFree=10452440/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=21259884 memory=21051268 CPUtime=2021.55 cores=0-7
/proc/3584/stat : 3584 (cosoco) R 3581 3584 2671 0 -1 4202496 5125780 0 0 0 201134 1021 0 0 20 0 1 0 560107774 21770121216 5262817 33554432000 4194304 6211132 140728482529872 140728482503432 140132532698085 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 5314971 5262817 819 493 0 5308276 0
Current children cumulated CPU time: 2021.55 s
Current children cumulated vsize: 21259884 KiB
Current children cumulated memory: 21051268 KiB

[startup+2082.3 s]
/proc/loadavg: 1.00 1.01 1.77 2/190 3739
/proc/meminfo: memFree=9829164/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=21861012 memory=21672952 CPUtime=2081.53 cores=0-7
/proc/3584/stat : 3584 (cosoco) R 3581 3584 2671 0 -1 4202496 5276728 0 0 0 207084 1069 0 0 20 0 1 0 560107774 22385676288 5418238 33554432000 4194304 6211132 140728482529872 140728482503432 140132532698085 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 5465286 5418238 819 493 0 5458591 0
Current children cumulated CPU time: 2081.53 s
Current children cumulated vsize: 21861012 KiB
Current children cumulated memory: 21672952 KiB

[startup+2142.3 s]
/proc/loadavg: 1.00 1.01 1.73 2/190 3749
/proc/meminfo: memFree=9208100/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=22462140 memory=22292592 CPUtime=2141.51 cores=0-7
/proc/3584/stat : 3584 (cosoco) R 3581 3584 2671 0 -1 4202496 5427649 0 0 0 213035 1116 0 0 20 0 1 0 560107774 23001231360 5573148 33554432000 4194304 6211132 140728482529872 140728482503432 140132532698085 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 5615535 5573148 819 493 0 5608840 0
Current children cumulated CPU time: 2141.51 s
Current children cumulated vsize: 22462140 KiB
Current children cumulated memory: 22292592 KiB

[startup+2202.3 s]
/proc/loadavg: 1.00 1.01 1.68 2/190 3754
/proc/meminfo: memFree=8585388/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=23063268 memory=22914036 CPUtime=2201.48 cores=0-7
/proc/3584/stat : 3584 (cosoco) R 3581 3584 2671 0 -1 4202496 5578682 0 0 0 218984 1164 0 0 20 0 1 0 560107774 23616786432 5728509 33554432000 4194304 6211132 140728482529872 140728482503432 140132532698085 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 5765817 5728509 819 493 0 5759122 0
Current children cumulated CPU time: 2201.48 s
Current children cumulated vsize: 23063268 KiB
Current children cumulated memory: 22914036 KiB

[startup+2262.3 s]
/proc/loadavg: 1.00 1.01 1.64 2/190 3757
/proc/meminfo: memFree=7965500/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=23664000 memory=23533144 CPUtime=2261.46 cores=0-7
/proc/3584/stat : 3584 (cosoco) R 3581 3584 2671 0 -1 4202496 5729502 0 0 0 224930 1216 0 0 20 0 1 0 560107774 24231936000 5883286 33554432000 4194304 6211132 140728482529872 140728482503432 140132532698085 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 5916000 5883286 819 493 0 5909305 0
Current children cumulated CPU time: 2261.46 s
Current children cumulated vsize: 23664000 KiB
Current children cumulated memory: 23533144 KiB

[startup+2322.3 s]
/proc/loadavg: 1.00 1.01 1.60 2/190 3760
/proc/meminfo: memFree=7342416/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=24264864 memory=24154592 CPUtime=2321.45 cores=0-7
/proc/3584/stat : 3584 (cosoco) R 3581 3584 2671 0 -1 4202496 5880492 0 0 0 230878 1267 0 0 20 0 1 0 560107774 24847220736 6038648 33554432000 4194304 6211132 140728482529872 140728482503432 140132532698085 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 6066216 6038648 819 493 0 6059521 0
Current children cumulated CPU time: 2321.45 s
Current children cumulated vsize: 24264864 KiB
Current children cumulated memory: 24154592 KiB

[startup+2382.3 s]
/proc/loadavg: 1.00 1.01 1.56 2/190 3765
/proc/meminfo: memFree=6722612/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=24865332 memory=24773392 CPUtime=2381.43 cores=0-7
/proc/3584/stat : 3584 (cosoco) R 3581 3584 2671 0 -1 4202496 6031266 0 0 0 236824 1319 0 0 20 0 1 0 560107774 25462099968 6193348 33554432000 4194304 6211132 140728482529872 140728482503432 140132532698085 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 6216333 6193348 819 493 0 6209638 0
Current children cumulated CPU time: 2381.43 s
Current children cumulated vsize: 24865332 KiB
Current children cumulated memory: 24773392 KiB

[startup+2442.3 s]
/proc/loadavg: 1.00 1.01 1.53 2/190 3768
/proc/meminfo: memFree=6100180/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=25465800 memory=25394352 CPUtime=2441.41 cores=0-7
/proc/3584/stat : 3584 (cosoco) R 3581 3584 2671 0 -1 4202496 6182144 0 0 0 242767 1374 0 0 20 0 1 0 560107774 26076979200 6348588 33554432000 4194304 6211132 140728482529872 140728482503432 5453928 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 6366450 6348588 819 493 0 6359755 0
Current children cumulated CPU time: 2441.41 s
Current children cumulated vsize: 25465800 KiB
Current children cumulated memory: 25394352 KiB

[startup+2502.3 s]
/proc/loadavg: 1.00 1.01 1.49 2/190 3771
/proc/meminfo: memFree=5480508/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=26066532 memory=26013492 CPUtime=2501.38 cores=0-7
/proc/3584/stat : 3584 (cosoco) R 3581 3584 2671 0 -1 4202496 6333074 0 0 0 248711 1427 0 0 20 0 1 0 560107774 26692128768 6503373 33554432000 4194304 6211132 140728482529872 140728482503432 140132532698085 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 6516633 6503373 819 493 0 6509938 0
Current children cumulated CPU time: 2501.38 s
Current children cumulated vsize: 26066532 KiB
Current children cumulated memory: 26013492 KiB



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+2520 s]*
/proc/loadavg: 1.00 1.01 1.48 2/190 3773
/proc/meminfo: memFree=5293132/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=26243676 memory=26196812 CPUtime=2519.09 cores=0-7
/proc/3584/stat : 3584 (cosoco) R 3581 3584 2671 0 -1 4202496 6377627 0 0 0 250465 1444 0 0 20 0 1 0 560107774 26873524224 6549203 33554432000 4194304 6211132 140728482529872 140728482503432 140132532698085 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 6560919 6549203 819 493 0 6554224 0
Current children cumulated CPU time: 2519.09 s
Current children cumulated vsize: 26243676 KiB
Current children cumulated memory: 26196812 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+2523.24 s]
# the end of solver process 3584 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2504.96 s, system=15.6799 s

Solver just ended.
Dumping a history of the last processes samples

[startup+2520.8 s]
/proc/loadavg: 1.00 1.01 1.48 2/190 3773
/proc/meminfo: memFree=5293132/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=26243676 memory=26196812 CPUtime=2519.21 cores=0-7
/proc/3584/stat : 3584 (cosoco) S 3581 3584 2671 0 -1 512 6377635 0 0 0 250477 1444 0 0 20 0 1 0 560107774 26873524224 6549203 33554432000 4194304 6211132 140728482529872 140728482503432 5189582 0 16384 4096 8405026 18446744071580971376 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 6560919 6549203 819 493 0 6554224 0
Current children cumulated CPU time: 2519.21 s
Current children cumulated vsize: 26243676 KiB
Current children cumulated memory: 26196812 KiB

[startup+2521.6 s]
/proc/loadavg: 1.00 1.01 1.48 3/191 3774
/proc/meminfo: memFree=4744152/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=26243676 memory=26196812 CPUtime=2519.34 cores=0-7
/proc/3584/stat : 3584 (cosoco) S 3581 3584 2671 0 -1 512 6377635 0 0 0 250490 1444 0 0 20 0 1 0 560107774 26873524224 6549203 33554432000 4194304 6211132 140728482529872 140728482503432 5189582 0 16384 4096 8405026 18446744071580971376 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 6560919 6549203 819 493 0 6554224 0
Current children cumulated CPU time: 2519.34 s
Current children cumulated vsize: 26243676 KiB
Current children cumulated memory: 26196812 KiB

[startup+2522 s]*
/proc/loadavg: 1.00 1.01 1.48 2/191 3774
/proc/meminfo: memFree=4178800/32770624 swapFree=66986684/67108860
[pid=3584] ppid=3581 vsize=26243676 memory=26196812 CPUtime=2519.41 cores=0-7
/proc/3584/stat : 3584 (cosoco) R 3581 3584 2671 0 -1 512 6377635 0 0 0 250496 1445 0 0 20 0 1 0 560107774 26873524224 6549203 33554432000 4194304 6211132 140728482529872 140728482503432 5189582 0 16384 4096 8405026 18446744071580971376 0 0 17 2 0 0 0 0 0 8312316 8317760 31563776 140728482538513 140728482538605 140728482538605 140728482541518 0
/proc/3584/statm: 6560919 6549203 819 493 0 6554224 0
Current children cumulated CPU time: 2519.41 s
Current children cumulated vsize: 26243676 KiB
Current children cumulated memory: 26196812 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 2523.24
CPU time (s): 2520.64
CPU user time (s): 2504.96
CPU system time (s): 15.6799
CPU usage (%): 99.8972
Max. virtual memory (cumulated for all children) (KiB): 26243676
Max. memory (cumulated for all children) (KiB): 26196812

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2504.96
system time used= 15.6799
maximum resident set size= 26197008
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 6377635
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 139831
involuntary context switches= 1316


# summary of solver processes directly reported to runsolver:
#   pid: 3584
#   total CPU time (s): 2520.64
#   total CPU user time (s): 2504.96
#   total CPU system time (s): 15.6799

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 8.34535 second user time and 17.1105 second system time

The end

Launcher Data 

Begin job on node104 at 2019-09-30 11:04:48
IDJOB=4409735
IDBENCH=139563
IDSOLVER=2877
FILE ID=node104/4409735-1569838515
RUNJOBID= node104-1569826684-2698
PBS_JOBID= 21705948
Free space on /tmp= 62032 MiB

SOLVER NAME= cosoco 2.0 parallel
BENCH NAME= XCSP17/SteelMillSlab/SteelMillSlab-bench_13_6.xml
COMMAND LINE= cosoco -model -nbcores=NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4409735-1569838515/watcher-4409735-1569838515 -o /tmp/evaluation-result-4409735-1569838515/solver-4409735-1569838515 -C 20160 -W 2520 -M 31000 --bin-var /tmp/runsolver-exch-node104-1569826684-2698 --watchdog 2580  cosoco -model -nbcores=8 HOME/instance-4409735-1569838515.xml

TIME LIMIT= 20160 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 0b4cdd9f2ac3a0985a9efe7e14eeda02
RANDOM SEED=1255418676

node104.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        31537224 kB
MemAvailable:   31837764 kB
Buffers:          217128 kB
Cached:           411500 kB
SwapCached:        22760 kB
Active:           312640 kB
Inactive:         343112 kB
Active(anon):      18724 kB
Inactive(anon):    91164 kB
Active(file):     293916 kB
Inactive(file):   251948 kB
Unevictable:      109656 kB
Mlocked:          109656 kB
SwapTotal:      67108860 kB
SwapFree:       66986684 kB
Dirty:              2744 kB
Writeback:            36 kB
AnonPages:        110428 kB
Mapped:            26032 kB
Shmem:             73632 kB
Slab:             170484 kB
SReclaimable:     141020 kB
SUnreclaim:        29464 kB
KernelStack:        3376 kB
PageTables:         5204 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     532196 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     63488 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      124288 kB
DirectMap2M:    33421312 kB

Free space on /tmp at the end= 61488 MiB
End job on node104 at 2019-09-30 11:46:51