Trace number 4408470

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2.0? (MO) 1520.82 1523.09

General information on the benchmark

NameSteelMillSlab/
SteelMillSlab-bench_9_11.xml
MD5SUMb3f80a69476de872363c069f96d9d14d
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkSAT TO
Best value of the objective obtained on this benchmark361
Best CPU time to get the best result obtained on this benchmark19756.9
Satisfiable
(Un)Satisfiability was proved
Number of variables12654
Number of constraints466
Number of domains4
Minimum domain size2
Maximum domain size111
Distribution of domain sizes[{"size":2,"count":12321},{"size":15,"count":111},{"size":51,"count":111},{"size":111,"count":111}]
Minimum variable degree2
Maximum variable degree114
Distribution of variable degrees[{"degree":2,"count":12432},{"degree":3,"count":111},{"degree":112,"count":78},{"degree":113,"count":24},{"degree":114,"count":9}]
Minimum constraint arity2
Maximum constraint arity112
Distribution of constraint arities[{"arity":2,"count":132},{"arity":111,"count":112},{"arity":112,"count":222}]
Number of extensional constraints0
Number of intensional constraints132
Distribution of constraint types[{"type":"intension","count":132},{"type":"sum","count":112},{"type":"element","count":111},{"type":"channel","count":111}]
Optimization problemYES
Type of objectivemin SUM

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (14860 MiB free)
  memory of node 1: 16384 MiB (14625 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4408470-1569812603/watcher-4408470-1569812603 -o /tmp/evaluation-result-4408470-1569812603/solver-4408470-1569812603 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node119-1569787352-30039 --watchdog 2580 cosoco -lc -model -restarts -stick HOME/instance-4408470-1569812603.xml 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2550 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=2158, runsolver pid=2155
Current StackSize limit: 8192 KiB


[startup+0.100207 s]*
/proc/loadavg: 2.01 2.02 2.05 3/200 2159
/proc/meminfo: memFree=30182576/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=39028 memory=15104 CPUtime=0.1 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) R 2155 2158 30008 0 -1 4202496 3986 0 0 0 10 0 0 0 20 0 1 0 557175044 39964672 3776 33554432000 4194304 6211132 140725689530480 140725689527016 5539508 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 9757 3776 809 493 0 3062 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 39028 KiB
Current children cumulated memory: 15104 KiB

[startup+0.209682 s]*
/proc/loadavg: 2.01 2.02 2.05 3/200 2159
/proc/meminfo: memFree=30180716/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=40736 memory=17008 CPUtime=0.2 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) R 2155 2158 30008 0 -1 4202496 4421 0 0 0 20 0 0 0 20 0 1 0 557175044 41713664 4252 33554432000 4194304 6211132 140725689530480 140725689527016 5013083 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 10184 4252 819 493 0 3489 0
Current children cumulated CPU time: 0.2 s
Current children cumulated vsize: 40736 KiB
Current children cumulated memory: 17008 KiB

[startup+0.30916 s]*
/proc/loadavg: 2.01 2.02 2.05 3/200 2159
/proc/meminfo: memFree=30179268/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=41720 memory=17800 CPUtime=0.3 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) R 2155 2158 30008 0 -1 4202496 4678 0 0 0 30 0 0 0 20 0 1 0 557175044 42721280 4450 33554432000 4194304 6211132 140725689530480 140725689504040 140015384214501 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 10430 4450 819 493 0 3735 0
Current children cumulated CPU time: 0.3 s
Current children cumulated vsize: 41720 KiB
Current children cumulated memory: 17800 KiB

[startup+0.700353 s]
/proc/loadavg: 2.01 2.02 2.05 3/200 2160
/proc/meminfo: memFree=30177408/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=45716 memory=21996 CPUtime=0.69 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) R 2155 2158 30008 0 -1 4202496 5670 0 0 0 68 1 0 0 20 0 1 0 557175044 46813184 5499 33554432000 4194304 6211132 140725689530480 140725689504040 140015385138419 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 11429 5499 819 493 0 4734 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 45716 KiB
Current children cumulated memory: 21996 KiB

[startup+1.50035 s]
/proc/loadavg: 2.01 2.02 2.05 4/200 2160
/proc/meminfo: memFree=30172324/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=54084 memory=30056 CPUtime=1.49 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) R 2155 2158 30008 0 -1 4202496 7736 0 0 0 148 1 0 0 20 0 1 0 557175044 55382016 7514 33554432000 4194304 6211132 140725689530480 140725689504040 5446657 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 13521 7514 819 493 0 6826 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 54084 KiB
Current children cumulated memory: 30056 KiB

[startup+3.10976 s]
/proc/loadavg: 2.01 2.02 2.05 4/200 2160
/proc/meminfo: memFree=30151748/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=70900 memory=46808 CPUtime=3.1 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) R 2155 2158 30008 0 -1 4202496 12062 0 0 0 309 1 0 0 20 0 1 0 557175044 72601600 11702 33554432000 4194304 6211132 140725689530480 140725689504040 140015384214501 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 17725 11702 819 493 0 11030 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 70900 KiB
Current children cumulated memory: 46808 KiB

[startup+6.30033 s]
/proc/loadavg: 2.01 2.02 2.05 4/200 2160
/proc/meminfo: memFree=30120800/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=104444 memory=79736 CPUtime=6.29 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) R 2155 2158 30008 0 -1 4202496 20476 0 0 0 627 2 0 0 20 0 1 0 557175044 106950656 19934 33554432000 4194304 6211132 140725689530480 140725689504040 140015385138178 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 26111 19934 819 493 0 19416 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 104444 KiB
Current children cumulated memory: 79736 KiB

[startup+12.7003 s]
/proc/loadavg: 2.01 2.02 2.05 4/200 2160
/proc/meminfo: memFree=30058620/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=169904 memory=145732 CPUtime=12.69 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) R 2155 2158 30008 0 -1 4202496 37169 0 0 0 1265 4 0 0 20 0 1 0 557175044 173981696 36433 33554432000 4194304 6211132 140725689530480 140725689504040 5456338 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 42476 36433 819 493 0 35781 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 169904 KiB
Current children cumulated memory: 145732 KiB

[startup+25.5004 s]
/proc/loadavg: 2.01 2.02 2.05 4/200 2160
/proc/meminfo: memFree=29924564/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=303476 memory=278212 CPUtime=25.49 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) R 2155 2158 30008 0 -1 4202496 70068 0 0 0 2540 9 0 0 20 0 1 0 557175044 310759424 69553 33554432000 4194304 6211132 140725689530480 140725689504040 5012996 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 75869 69553 819 493 0 69174 0
Current children cumulated CPU time: 25.49 s
Current children cumulated vsize: 303476 KiB
Current children cumulated memory: 278212 KiB

[startup+51.1097 s]
/proc/loadavg: 2.01 2.02 2.05 4/200 2162
/proc/meminfo: memFree=29653824/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=571840 memory=543860 CPUtime=51.09 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) R 2155 2158 30008 0 -1 4202496 135114 0 0 0 5092 17 0 0 20 0 1 0 557175044 585564160 135965 33554432000 4194304 6211132 140725689530480 140725689504040 140015384214501 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 142960 135965 819 493 0 136265 0
Current children cumulated CPU time: 51.09 s
Current children cumulated vsize: 571840 KiB
Current children cumulated memory: 543860 KiB

[startup+102.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 2352
/proc/meminfo: memFree=29085520/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=1110812 memory=1070692 CPUtime=102.28 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) R 2155 2158 30008 0 -1 4202496 263972 0 0 0 10192 36 0 0 20 0 1 0 557175044 1137471488 267673 33554432000 4194304 6211132 140725689530480 140725689504040 5013011 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 277703 267673 819 493 0 271008 0
Current children cumulated CPU time: 102.28 s
Current children cumulated vsize: 1110812 KiB
Current children cumulated memory: 1070692 KiB

[startup+162.3 s]
/proc/loadavg: 2.05 2.03 2.05 3/200 2356
/proc/meminfo: memFree=28459504/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=1739052 memory=1693992 CPUtime=162.26 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) R 2155 2158 30008 0 -1 4202496 415666 0 0 0 16169 57 0 0 20 0 1 0 557175044 1780789248 423498 33554432000 4194304 6211132 140725689530480 140725689504040 5442996 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 434763 423498 819 493 0 428068 0
Current children cumulated CPU time: 162.26 s
Current children cumulated vsize: 1739052 KiB
Current children cumulated memory: 1693992 KiB

[startup+222.3 s]
/proc/loadavg: 2.02 2.02 2.05 3/199 2362
/proc/meminfo: memFree=27834016/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=2342952 memory=2318200 CPUtime=222.23 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) R 2155 2158 30008 0 -1 4202496 566638 0 0 0 22146 77 0 0 20 0 1 0 557175044 2399182848 579550 33554432000 4194304 6211132 140725689530480 140725689504040 140015385138424 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 585738 579550 819 493 0 579043 0
Current children cumulated CPU time: 222.23 s
Current children cumulated vsize: 2342952 KiB
Current children cumulated memory: 2318200 KiB

[startup+282.3 s]
/proc/loadavg: 2.01 2.02 2.05 3/199 2366
/proc/meminfo: memFree=27210552/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=2983764 memory=2940664 CPUtime=282.21 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) R 2155 2158 30008 0 -1 4202496 718219 0 0 0 28126 95 0 0 20 0 1 0 557175044 3055374336 735166 33554432000 4194304 6211132 140725689530480 140725689504040 140015385138178 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 745941 735166 819 493 0 739246 0
Current children cumulated CPU time: 282.21 s
Current children cumulated vsize: 2983764 KiB
Current children cumulated memory: 2940664 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 2370
/proc/meminfo: memFree=26585648/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=3642896 memory=3563892 CPUtime=342.18 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) R 2155 2158 30008 0 -1 4202496 869481 0 0 0 34100 118 0 0 20 0 1 0 557175044 3730325504 890973 33554432000 4194304 6211132 140725689530480 140725689504040 5454208 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 910724 890973 819 493 0 904029 0
Current children cumulated CPU time: 342.18 s
Current children cumulated vsize: 3642896 KiB
Current children cumulated memory: 3563892 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 2376
/proc/meminfo: memFree=25960712/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=4246664 memory=4187232 CPUtime=402.15 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) R 2155 2158 30008 0 -1 4202496 1020808 0 0 0 40073 142 0 0 20 0 1 0 557175044 4348583936 1046808 33554432000 4194304 6211132 140725689530480 140725689504040 140015385138662 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 1061666 1046808 819 493 0 1054971 0
Current children cumulated CPU time: 402.15 s
Current children cumulated vsize: 4246664 KiB
Current children cumulated memory: 4187232 KiB

[startup+462.3 s]
/proc/loadavg: 2.29 2.09 2.07 3/201 2389
/proc/meminfo: memFree=25334132/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=4850300 memory=4811400 CPUtime=462.13 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) R 2155 2158 30008 0 -1 4202496 1171870 0 0 0 46048 165 0 0 20 0 1 0 557175044 4966707200 1202850 33554432000 4194304 6211132 140725689530480 140725689504040 5451553 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 1212575 1202850 819 493 0 1205880 0
Current children cumulated CPU time: 462.13 s
Current children cumulated vsize: 4850300 KiB
Current children cumulated memory: 4811400 KiB


################
# More data... #
################

Current children cumulated vsize: 9883436 KiB
Current children cumulated memory: 9791724 KiB

[startup+1002.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 2444
/proc/meminfo: memFree=19720536/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=10486148 memory=10414956 CPUtime=1001.92 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) R 2155 2158 30008 0 -1 4202496 2531583 0 0 0 99799 393 0 0 20 0 1 0 557175044 10737815552 2603739 33554432000 4194304 6211132 140725689530480 140725689504040 140015385138182 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 2621537 2603739 819 493 0 2614842 0
Current children cumulated CPU time: 1001.92 s
Current children cumulated vsize: 10486148 KiB
Current children cumulated memory: 10414956 KiB

[startup+1062.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 2456
/proc/meminfo: memFree=19098728/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=11088860 memory=11036100 CPUtime=1061.9 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) R 2155 2158 30008 0 -1 4202496 2682539 0 0 0 105770 420 0 0 20 0 1 0 557175044 11354992640 2759025 33554432000 4194304 6211132 140725689530480 140725689504040 5443128 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 2772215 2759025 819 493 0 2765520 0
Current children cumulated CPU time: 1061.9 s
Current children cumulated vsize: 11088860 KiB
Current children cumulated memory: 11036100 KiB

[startup+1122.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 2462
/proc/meminfo: memFree=18474048/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=11691440 memory=11659288 CPUtime=1121.88 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) R 2155 2158 30008 0 -1 4202496 2833455 0 0 0 111741 447 0 0 20 0 1 0 557175044 11972034560 2914822 33554432000 4194304 6211132 140725689530480 140725689504040 140015384214501 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 2922860 2914822 819 493 0 2916165 0
Current children cumulated CPU time: 1121.88 s
Current children cumulated vsize: 11691440 KiB
Current children cumulated memory: 11659288 KiB

[startup+1182.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 2466
/proc/meminfo: memFree=17850448/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=12480612 memory=12281252 CPUtime=1181.86 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) R 2155 2158 30008 0 -1 4202496 2984609 0 0 0 117709 477 0 0 20 0 1 0 557175044 12780146688 3070313 33554432000 4194304 6211132 140725689530480 140725689504040 5479626 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 3120153 3070313 819 493 0 3113458 0
Current children cumulated CPU time: 1181.86 s
Current children cumulated vsize: 12480612 KiB
Current children cumulated memory: 12281252 KiB

[startup+1242.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 2470
/proc/meminfo: memFree=17228804/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=13082796 memory=12902132 CPUtime=1241.84 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) R 2155 2158 30008 0 -1 4202496 3135429 0 0 0 123678 506 0 0 20 0 1 0 557175044 13396783104 3225533 33554432000 4194304 6211132 140725689530480 140725689504040 140015384214501 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 3270699 3225533 819 493 0 3264004 0
Current children cumulated CPU time: 1241.84 s
Current children cumulated vsize: 13082796 KiB
Current children cumulated memory: 12902132 KiB

[startup+1302.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 2476
/proc/meminfo: memFree=16605232/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=13684980 memory=13524564 CPUtime=1301.81 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) R 2155 2158 30008 0 -1 4202496 3286217 0 0 0 129644 537 0 0 20 0 1 0 557175044 14013419520 3381141 33554432000 4194304 6211132 140725689530480 140725689504040 5013073 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 3421245 3381141 819 493 0 3414550 0
Current children cumulated CPU time: 1301.81 s
Current children cumulated vsize: 13684980 KiB
Current children cumulated memory: 13524564 KiB

[startup+1362.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 2481
/proc/meminfo: memFree=15980816/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=14287296 memory=14145408 CPUtime=1361.79 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) R 2155 2158 30008 0 -1 4202496 3437072 0 0 0 135614 565 0 0 20 0 1 0 557175044 14630191104 3536352 33554432000 4194304 6211132 140725689530480 140725689504040 140015384214501 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 3571824 3536352 819 493 0 3565129 0
Current children cumulated CPU time: 1361.79 s
Current children cumulated vsize: 14287296 KiB
Current children cumulated memory: 14145408 KiB

[startup+1422.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 2485
/proc/meminfo: memFree=15358656/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=14889876 memory=14768436 CPUtime=1421.77 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) R 2155 2158 30008 0 -1 4202496 3588022 0 0 0 141579 598 0 0 20 0 1 0 557175044 15247233024 3692109 33554432000 4194304 6211132 140725689530480 140725689504040 4982624 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 3722469 3692109 819 493 0 3715774 0
Current children cumulated CPU time: 1421.77 s
Current children cumulated vsize: 14889876 KiB
Current children cumulated memory: 14768436 KiB

[startup+1482.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 2491
/proc/meminfo: memFree=14734076/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=15491532 memory=15388744 CPUtime=1481.74 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) R 2155 2158 30008 0 -1 4202496 3738737 0 0 0 147539 635 0 0 20 0 1 0 557175044 15863328768 3847186 33554432000 4194304 6211132 140725689530480 140725689504040 140015384214501 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 3872883 3847186 819 493 0 3866188 0
Current children cumulated CPU time: 1481.74 s
Current children cumulated vsize: 15491532 KiB
Current children cumulated memory: 15388744 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+1520.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 2494
/proc/meminfo: memFree=14341928/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=15872484 memory=15781716 CPUtime=1519.73 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) R 2155 2158 30008 0 -1 4202496 3834161 0 0 0 151312 661 0 0 20 0 1 0 557175044 16253423616 3945429 33554432000 4194304 6211132 140725689530480 140725689504040 5454200 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 3968121 3945429 819 493 0 3961426 0
Current children cumulated CPU time: 1519.73 s
Current children cumulated vsize: 15872484 KiB
Current children cumulated memory: 15781716 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+1523.09 s]
# the end of solver process 2158 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=1513.4 s, system=7.42173 s

Solver just ended.
Dumping a history of the last processes samples

[startup+1521.6 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 2495
/proc/meminfo: memFree=13982808/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=15872484 memory=15781716 CPUtime=1519.93 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) S 2155 2158 30008 0 -1 512 3834167 0 0 0 151330 663 0 0 20 0 1 0 557175044 16253423616 3945429 33554432000 4194304 6211132 140725689530480 140725689504040 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 3968121 3945429 819 493 0 3961426 0
Current children cumulated CPU time: 1519.93 s
Current children cumulated vsize: 15872484 KiB
Current children cumulated memory: 15781716 KiB

[startup+1522 s]*
/proc/loadavg: 2.00 2.01 2.05 3/200 2495
/proc/meminfo: memFree=13480608/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=15872484 memory=15781716 CPUtime=1519.99 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) S 2155 2158 30008 0 -1 512 3834167 0 0 0 151336 663 0 0 20 0 1 0 557175044 16253423616 3945429 33554432000 4194304 6211132 140725689530480 140725689504040 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 3968121 3945429 819 493 0 3961426 0
Current children cumulated CPU time: 1519.99 s
Current children cumulated vsize: 15872484 KiB
Current children cumulated memory: 15781716 KiB

[startup+1522.2 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 2495
/proc/meminfo: memFree=13480608/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=15872484 memory=15781716 CPUtime=1520.01 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) S 2155 2158 30008 0 -1 512 3834167 0 0 0 151338 663 0 0 20 0 1 0 557175044 16253423616 3945429 33554432000 4194304 6211132 140725689530480 140725689504040 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 3968121 3945429 819 493 0 3961426 0
Current children cumulated CPU time: 1520.01 s
Current children cumulated vsize: 15872484 KiB
Current children cumulated memory: 15781716 KiB

[startup+1522.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 2495
/proc/meminfo: memFree=13480608/32770624 swapFree=66954180/67108860
[pid=2158] ppid=2155 vsize=15872484 memory=15781716 CPUtime=1520.02 cores=0,2,4,6
/proc/2158/stat : 2158 (cosoco) R 2155 2158 30008 0 -1 512 3834167 0 0 0 151339 663 0 0 20 0 1 0 557175044 16253423616 3945429 33554432000 4194304 6211132 140725689530480 140725689504040 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8312316 8317760 11476992 140725689533451 140725689533553 140725689533553 140725689536462 0
/proc/2158/statm: 3968121 3945429 819 493 0 3961426 0
Current children cumulated CPU time: 1520.02 s
Current children cumulated vsize: 15872484 KiB
Current children cumulated memory: 15781716 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1523.09
CPU time (s): 1520.82
CPU user time (s): 1513.4
CPU system time (s): 7.42173
CPU usage (%): 99.8512
Max. virtual memory (cumulated for all children) (KiB): 15872484
Max. memory (cumulated for all children) (KiB): 15781716

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1513.4
system time used= 7.42173
maximum resident set size= 15781968
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3834167
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 124148
involuntary context switches= 1903


# summary of solver processes directly reported to runsolver:
#   pid: 2158
#   total CPU time (s): 1520.82
#   total CPU user time (s): 1513.4
#   total CPU system time (s): 7.42173

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 5.4393 second user time and 10.7181 second system time

The end

Launcher Data 

Begin job on node119 at 2019-09-30 05:02:42
IDJOB=4408470
IDBENCH=139658
IDSOLVER=2874
FILE ID=node119/4408470-1569812603
RUNJOBID= node119-1569787352-30039
PBS_JOBID= 21705833
Free space on /tmp= 62028 MiB

SOLVER NAME= cosoco 2.0
BENCH NAME= XCSP17/SteelMillSlab/SteelMillSlab-bench_9_11.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4408470-1569812603/watcher-4408470-1569812603 -o /tmp/evaluation-result-4408470-1569812603/solver-4408470-1569812603 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node119-1569787352-30039 --watchdog 2580  cosoco -lc -model -restarts -stick HOME/instance-4408470-1569812603.xml

TIME LIMIT= 2520 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= b3f80a69476de872363c069f96d9d14d
RANDOM SEED=1667953480

node119.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.47
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.47
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.47
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.47
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        30193980 kB
MemAvailable:   31821260 kB
Buffers:          218460 kB
Cached:          1747188 kB
SwapCached:        46700 kB
Active:          1081932 kB
Inactive:         941792 kB
Active(anon):      22392 kB
Inactive(anon):   117712 kB
Active(file):    1059540 kB
Inactive(file):   824080 kB
Unevictable:      107588 kB
Mlocked:          107588 kB
SwapTotal:      67108860 kB
SwapFree:       66954180 kB
Dirty:              2548 kB
Writeback:             0 kB
AnonPages:        120756 kB
Mapped:            28160 kB
Shmem:             73180 kB
Slab:             148116 kB
SReclaimable:     118992 kB
SUnreclaim:        29124 kB
KernelStack:        3584 kB
PageTables:         5140 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     575204 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     71680 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      126336 kB
DirectMap2M:    33419264 kB

Free space on /tmp at the end= 61544 MiB
End job on node119 at 2019-09-30 05:28:05