Trace number 3380685

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 40001.7 5009.8

General information on the benchmark

NameCRAFTED/rbsat/random/unforced/
rbsat-v2100c215164g9.cnf
MD5SUMe033e9c916da557514299cce9ca2468f
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2100
Number of clauses215164
Sum of the clauses size432288
Maximum clause length30
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2215094
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 570

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...486977/instance-3380685-1305486977.cnf
0.99/1.03	c Solving...
40000.77/5009.01	
40000.77/5009.01	*** INTERRUPTED! ***
40000.77/5009.01	*** Shutting down threads...
40001.54/5009.79	s UNKNOWN
40001.54/5009.79	
40001.54/5009.79	c INTERRUPTED   : 1
40001.54/5009.79	c Models        : 0+    
40001.54/5009.79	c Time          : 5009.027s (Solving: 5008.00s 1st Model: 0.00s Unsat: 0.00s)
40001.54/5009.79	c CPU Time      : 40000.870s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3380685-1305486977/watcher-3380685-1305486977 -o /tmp/evaluation-result-3380685-1305486977/solver-3380685-1305486977 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3380685-1305486977.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.72 7.69 7.88 2/354 4758
/proc/meminfo: memFree=14626684/32951124 swapFree=67111348/67111528
[pid=4758] ppid=4756 vsize=21764 CPUtime=0 cores=0-7
/proc/4758/stat : 4758 (claspmt) R 4756 4758 4732 0 -1 4202496 748 0 0 0 0 0 0 0 25 0 1 0 641113561 22286336 663 33554432000 4194304 4964800 140734686723968 18446744073709551615 233509958341 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/4758/statm: 5441 664 386 189 0 315 0

[startup+0.0796101 s]
/proc/loadavg: 6.72 7.69 7.88 2/354 4758
/proc/meminfo: memFree=14626684/32951124 swapFree=67111348/67111528
[pid=4758] ppid=4756 vsize=36076 CPUtime=0.07 cores=0-7
/proc/4758/stat : 4758 (claspmt) R 4756 4758 4732 0 -1 4202496 5216 0 0 0 7 0 0 0 25 0 1 0 641113561 36941824 3962 33554432000 4194304 4964800 140734686723968 18446744073709551615 4607644 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/4758/statm: 9019 3962 399 189 0 3893 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 36076

[startup+0.100605 s]
/proc/loadavg: 6.72 7.69 7.88 2/354 4758
/proc/meminfo: memFree=14626684/32951124 swapFree=67111348/67111528
[pid=4758] ppid=4756 vsize=36076 CPUtime=0.09 cores=0-7
/proc/4758/stat : 4758 (claspmt) R 4756 4758 4732 0 -1 4202496 5216 0 0 0 9 0 0 0 25 0 1 0 641113561 36941824 3962 33554432000 4194304 4964800 140734686723968 18446744073709551615 4607708 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/4758/statm: 9019 3962 399 189 0 3893 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 36076

[startup+0.30057 s]
/proc/loadavg: 6.72 7.69 7.88 2/354 4758
/proc/meminfo: memFree=14626684/32951124 swapFree=67111348/67111528
[pid=4758] ppid=4756 vsize=36076 CPUtime=0.29 cores=0-7
/proc/4758/stat : 4758 (claspmt) R 4756 4758 4732 0 -1 4202496 5216 0 0 0 29 0 0 0 25 0 1 0 641113561 36941824 3962 33554432000 4194304 4964800 140734686723968 18446744073709551615 4607585 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/4758/statm: 9019 3962 399 189 0 3893 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 36076

[startup+0.700497 s]
/proc/loadavg: 6.72 7.69 7.88 2/354 4758
/proc/meminfo: memFree=14626684/32951124 swapFree=67111348/67111528
[pid=4758] ppid=4756 vsize=36736 CPUtime=0.69 cores=0-7
/proc/4758/stat : 4758 (claspmt) R 4756 4758 4732 0 -1 4202496 5394 0 0 0 69 0 0 0 25 0 1 0 641113561 37617664 4140 33554432000 4194304 4964800 140734686723968 18446744073709551615 4600372 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/4758/statm: 9184 4140 400 189 0 4058 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 36736

[startup+1.50032 s]
/proc/loadavg: 6.72 7.69 7.88 2/355 4759
/proc/meminfo: memFree=14610692/32951124 swapFree=67111348/67111528
[pid=4758] ppid=4756 vsize=395644 CPUtime=3.61 cores=0-7
/proc/4758/stat : 4758 (claspmt) R 4756 4758 4732 0 -1 4202496 10988 0 0 0 357 4 0 0 17 0 8 0 641113561 405139456 9291 33554432000 4194304 4964800 140734686723968 18446744073709551615 4627211 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/4758/statm: 98911 9291 464 189 0 93260 0
Current children cumulated CPU time (s) 3.61
Current children cumulated vsize (KiB) 395644

[startup+3.10103 s]
/proc/loadavg: 6.72 7.69 7.88 9/362 4766
/proc/meminfo: memFree=14572072/32951124 swapFree=67111348/67111528
[pid=4758] ppid=4756 vsize=399116 CPUtime=16.39 cores=0-7
/proc/4758/stat : 4758 (claspmt) R 4756 4758 4732 0 -1 4202496 20485 0 0 0 1630 9 0 0 19 0 8 0 641113561 408694784 18788 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/4758/statm: 99779 18788 464 189 0 94128 0
[pid=4758/tid=4760] ppid=4756 vsize=399116 CPUtime=1.89 cores=0-7
/proc/4758/task/4760/stat : 4760 (claspmt) R 4756 4758 4732 0 -1 4202560 1512 0 0 0 188 1 0 0 18 0 8 0 641113664 408694784 18788 33554432000 4194304 4964800 140734686723968 18446744073709551615 4320966 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4758/tid=4761] ppid=4756 vsize=399116 CPUtime=1.9 cores=0-7
/proc/4758/task/4761/stat : 4761 (claspmt) R 4756 4758 4732 0 -1 4202560 1796 0 0 0 189 1 0 0 18 0 8 0 641113664 408694784 18789 33554432000 4194304 4964800 140734686723968 18446744073709551615 4627290 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4758/tid=4762] ppid=4756 vsize=399116 CPUtime=1.9 cores=0-7
/proc/4758/task/4762/stat : 4762 (claspmt) R 4756 4758 4732 0 -1 4202560 1634 0 0 0 189 1 0 0 18 0 8 0 641113664 408694784 18789 33554432000 4194304 4964800 140734686723968 18446744073709551615 4362964 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4758/tid=4763] ppid=4756 vsize=399116 CPUtime=1.87 cores=0-7
/proc/4758/task/4763/stat : 4763 (claspmt) R 4756 4758 4732 0 -1 4202560 1881 0 0 0 186 1 0 0 25 0 8 0 641113664 408694784 18789 33554432000 4194304 4964800 140734686723968 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4758/tid=4764] ppid=4756 vsize=399116 CPUtime=2.05 cores=0-7
/proc/4758/task/4764/stat : 4764 (claspmt) R 4756 4758 4732 0 -1 4202560 955 0 0 0 205 0 0 0 21 0 8 0 641113664 408694784 18789 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4758/tid=4765] ppid=4756 vsize=399116 CPUtime=1.91 cores=0-7
/proc/4758/task/4765/stat : 4765 (claspmt) R 4756 4758 4732 0 -1 4202560 1892 0 0 0 190 1 0 0 18 0 8 0 641113664 408694784 18789 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670343 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4758/tid=4766] ppid=4756 vsize=399116 CPUtime=1.91 cores=0-7
/proc/4758/task/4766/stat : 4766 (claspmt) R 4756 4758 4732 0 -1 4202560 3466 0 0 0 190 1 0 0 18 0 8 0 641113665 408694784 18790 33554432000 4194304 4964800 140734686723968 18446744073709551615 233509958341 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 16.39
Current children cumulated vsize (KiB) 399116

[startup+6.30046 s]
/proc/loadavg: 6.82 7.70 7.88 9/362 4766
/proc/meminfo: memFree=14514664/32951124 swapFree=67111348/67111528
[pid=4758] ppid=4756 vsize=408492 CPUtime=41.94 cores=0-7
/proc/4758/stat : 4758 (claspmt) R 4756 4758 4732 0 -1 4202496 34485 0 0 0 4177 17 0 0 25 0 8 0 641113561 418295808 32788 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/4758/statm: 102123 32789 466 189 0 96472 0
[pid=4758/tid=4760] ppid=4756 vsize=408492 CPUtime=5.04 cores=0-7
/proc/4758/task/4760/stat : 4760 (claspmt) R 4756 4758 4732 0 -1 4202560 3249 0 0 0 503 1 0 0 24 0 8 0 641113664 418295808 32789 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670357 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4758/tid=4761] ppid=4756 vsize=408492 CPUtime=5.09 cores=0-7
/proc/4758/task/4761/stat : 4761 (claspmt) R 4756 4758 4732 0 -1 4202560 3562 0 0 0 507 2 0 0 25 0 8 0 641113664 418295808 32789 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4758/tid=4762] ppid=4756 vsize=408492 CPUtime=5.1 cores=0-7
/proc/4758/task/4762/stat : 4762 (claspmt) R 4756 4758 4732 0 -1 4202560 2967 0 0 0 508 2 0 0 25 0 8 0 641113664 418295808 32789 33554432000 4194304 4964800 140734686723968 18446744073709551615 233509958689 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4758/tid=4763] ppid=4756 vsize=408492 CPUtime=5.07 cores=0-7
/proc/4758/task/4763/stat : 4763 (claspmt) R 4756 4758 4732 0 -1 4202560 3545 0 0 0 505 2 0 0 25 0 8 0 641113664 418295808 32789 33554432000 4194304 4964800 140734686723968 18446744073709551615 4320848 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4758/tid=4764] ppid=4756 vsize=408492 CPUtime=5.25 cores=0-7
/proc/4758/task/4764/stat : 4764 (claspmt) R 4756 4758 4732 0 -1 4202560 1759 0 0 0 525 0 0 0 19 0 8 0 641113664 418295808 32789 33554432000 4194304 4964800 140734686723968 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4758/tid=4765] ppid=4756 vsize=408492 CPUtime=5.1 cores=0-7
/proc/4758/task/4765/stat : 4765 (claspmt) R 4756 4758 4732 0 -1 4202560 3172 0 0 0 509 1 0 0 25 0 8 0 641113664 418295808 32789 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4758/tid=4766] ppid=4756 vsize=408492 CPUtime=5.1 cores=0-7
/proc/4758/task/4766/stat : 4766 (claspmt) R 4756 4758 4732 0 -1 4202560 7363 0 0 0 507 3 0 0 25 0 8 0 641113665 418295808 32789 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 41.94
Current children cumulated vsize (KiB) 408492

[startup+12.7003 s]
/proc/loadavg: 6.91 7.70 7.88 9/362 4766
/proc/meminfo: memFree=14430924/32951124 swapFree=67111348/67111528
[pid=4758] ppid=4756 vsize=417820 CPUtime=93.05 cores=0-7
/proc/4758/stat : 4758 (claspmt) R 4756 4758 4732 0 -1 4202496 52095 0 0 0 9276 29 0 0 18 0 8 0 641113561 427847680 50398 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/4758/statm: 104455 50398 466 189 0 98804 0
[pid=4758/tid=4760] ppid=4756 vsize=417820 CPUtime=11.37 cores=0-7
/proc/4758/task/4760/stat : 4760 (claspmt) R 4756 4758 4732 0 -1 4202560 4960 0 0 0 1134 3 0 0 25 0 8 0 641113664 427847680 50398 33554432000 4194304 4964800 140734686723968 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4758/tid=4761] ppid=4756 vsize=417820 CPUtime=11.5 cores=0-7
/proc/4758/task/4761/stat : 4761 (claspmt) R 4756 4758 4732 0 -1 4202560 5210 0 0 0 1146 4 0 0 25 0 8 0 641113664 427847680 50398 33554432000 4194304 4964800 140734686723968 18446744073709551615 233509957808 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4758/tid=4762] ppid=4756 vsize=417820 CPUtime=11.5 cores=0-7
/proc/4758/task/4762/stat : 4762 (claspmt) R 4756 4758 4732 0 -1 4202560 4427 0 0 0 1148 2 0 0 25 0 8 0 641113664 427847680 50398 33554432000 4194304 4964800 140734686723968 18446744073709551615 4627272 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4758/tid=4763] ppid=4756 vsize=417820 CPUtime=11.46 cores=0-7
/proc/4758/task/4763/stat : 4763 (claspmt) R 4756 4758 4732 0 -1 4202560 5062 0 0 0 1143 3 0 0 25 0 8 0 641113664 427847680 50398 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670323 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4758/tid=4764] ppid=4756 vsize=417820 CPUtime=11.65 cores=0-7
/proc/4758/task/4764/stat : 4764 (claspmt) R 4756 4758 4732 0 -1 4202560 3916 0 0 0 1163 2 0 0 25 0 8 0 641113664 427847680 50398 33554432000 4194304 4964800 140734686723968 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4758/tid=4765] ppid=4756 vsize=417820 CPUtime=11.5 cores=0-7
/proc/4758/task/4765/stat : 4765 (claspmt) R 4756 4758 4732 0 -1 4202560 4911 0 0 0 1148 2 0 0 25 0 8 0 641113664 427847680 50398 33554432000 4194304 4964800 140734686723968 18446744073709551615 4660139 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4758/tid=4766] ppid=4756 vsize=417820 CPUtime=11.5 cores=0-7
/proc/4758/task/4766/stat : 4766 (claspmt) R 4756 4758 4732 0 -1 4202560 13093 0 0 0 1144 6 0 0 25 0 8 0 641113665 427847680 50398 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 93.05
Current children cumulated vsize (KiB) 417820

[startup+25.501 s]
/proc/loadavg: 7.15 7.72 7.88 9/362 4767
/proc/meminfo: memFree=14385456/32951124 swapFree=67111348/67111528
[pid=4758] ppid=4756 vsize=431924 CPUtime=195.27 cores=0-7
/proc/4758/stat : 4758 (claspmt) R 4756 4758 4732 0 -1 4202496 64020 0 0 0 19490 37 0 0 25 0 8 0 641113561 442290176 62323 33554432000 4194304 4964800 140734686723968 18446744073709551615 4668998 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/4758/statm: 107981 62323 466 189 0 102330 0
[pid=4758/tid=4760] ppid=4756 vsize=431924 CPUtime=24 cores=0-7
/proc/4758/task/4760/stat : 4760 (claspmt) R 4756 4758 4732 0 -1 4202560 6213 0 0 0 2396 4 0 0 20 0 8 0 641113664 442290176 62323 33554432000 4194304 4964800 140734686723968 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4758/tid=4761] ppid=4756 vsize=431924 CPUtime=24.3 cores=0-7
/proc/4758/task/4761/stat : 4761 (claspmt) R 4756 4758 4732 0 -1 4202560 6278 0 0 0 2425 5 0 0 25 0 8 0 641113664 442290176 62323 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670295 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4758/tid=4762] ppid=4756 vsize=431924 CPUtime=24.3 cores=0-7
/proc/4758/task/4762/stat : 4762 (claspmt) R 4756 4758 4732 0 -1 4202560 4936 0 0 0 2428 2 0 0 25 0 8 0 641113664 442290176 62323 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4758/tid=4763] ppid=4756 vsize=431924 CPUtime=24.26 cores=0-7
/proc/4758/task/4763/stat : 4763 (claspmt) R 4756 4758 4732 0 -1 4202560 6497 0 0 0 2422 4 0 0 25 0 8 0 641113664 442290176 62323 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4758/tid=4764] ppid=4756 vsize=431924 CPUtime=24.44 cores=0-7
/proc/4758/task/4764/stat : 4764 (claspmt) R 4756 4758 4732 0 -1 4202560 4829 0 0 0 2442 2 0 0 25 0 8 0 641113664 442290176 62323 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4758/tid=4765] ppid=4756 vsize=431924 CPUtime=24.3 cores=0-7
/proc/4758/task/4765/stat : 4765 (claspmt) R 4756 4758 4732 0 -1 4202560 5643 0 0 0 2427 3 0 0 25 0 8 0 641113664 442290176 62323 33554432000 4194304 4964800 140734686723968 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4758/tid=4766] ppid=4756 vsize=431924 CPUtime=24.3 cores=0-7
/proc/4758/task/4766/stat : 4766 (claspmt) R 4756 4758 4732 0 -1 4202560 17930 0 0 0 2422 8 0 0 25 0 8 0 641113665 442290176 62323 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670499 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 195.27
Current children cumulated vsize (KiB) 431924

[startup+51.1004 s]
/proc/loadavg: 7.44 7.74 7.88 9/362 4767
/proc/meminfo: memFree=14274548/32951124 swapFree=67111348/67111528
[pid=4758] ppid=4756 vsize=443740 CPUtime=399.73 cores=0-7
/proc/4758/stat : 4758 (claspmt) R 4756 4758 4732 0 -1 4202496 90626 0 0 0 39913 60 0 0 25 0 8 0 641113561 454389760 88929 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/4758/statm: 110935 88929 466 189 0 105284 0
[pid=4758/tid=4760] ppid=4756 vsize=443740 CPUtime=49.28 cores=0-7
/proc/4758/task/4760/stat : 4760 (claspmt) R 4756 4758 4732 0 -1 4202560 9070 0 0 0 4922 6 0 0 25 0 8 0 641113664 454389760 88929 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4758/tid=4761] ppid=4756 vsize=443740 CPUtime=49.89 cores=0-7
/proc/4758/task/4761/stat : 4761 (claspmt) R 4756 4758 4732 0 -1 4202560 9149 0 0 0 4983 6 0 0 25 0 8 0 641113664 454389760 88929 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670332 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4758/tid=4762] ppid=4756 vsize=443740 CPUtime=49.89 cores=0-7
/proc/4758/task/4762/stat : 4762 (claspmt) R 4756 4758 4732 0 -1 4202560 7357 0 0 0 4985 4 0 0 25 0 8 0 641113664 454389760 88929 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4758/tid=4763] ppid=4756 vsize=443740 CPUtime=49.85 cores=0-7
/proc/4758/task/4763/stat : 4763 (claspmt) R 4756 4758 4732 0 -1 4202560 9599 0 0 0 4977 8 0 0 25 0 8 0 641113664 454389760 88929 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670361 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4758/tid=4764] ppid=4756 vsize=443740 CPUtime=50.05 cores=0-7
/proc/4758/task/4764/stat : 4764 (claspmt) R 4756 4758 4732 0 -1 4202560 7057 0 0 0 5001 4 0 0 25 0 8 0 641113664 454389760 88929 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4758/tid=4765] ppid=4756 vsize=443740 CPUtime=49.89 cores=0-7
/proc/4758/task/4765/stat : 4765 (claspmt) R 4756 4758 4732 0 -1 4202560 8059 0 0 0 4984 5 0 0 25 0 8 0 641113664 454389760 88929 33554432000 4194304 4964800 140734686723968 18446744073709551615 4627257 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4758/tid=4766] ppid=4756 vsize=443740 CPUtime=49.9 cores=0-7
/proc/4758/task/4766/stat : 4766 (claspmt) R 4756 4758 4732 0 -1 4202560 25571 0 0 0 4976 14 0 0 25 0 8 0 641113665 454389760 88929 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 399.73
Current children cumulated vsize (KiB) 443740

[startup+102.314 s]
/proc/loadavg: 7.80 7.79 7.89 9/362 4769
/proc/meminfo: memFree=14179060/32951124 swapFree=67111348/67111528
[pid=4758] ppid=4756 vsize=788228 CPUtime=808.77 cores=0-7
/proc/4758/stat : 4758 (claspmt) R 4756 4758 4732 0 -1 4202496 114568 0 0 0 80796 81 0 0 25 0 8 0 641113561 807145472 112797 33554432000 4194304 4964800 140734686723968 18446744073709551615 233509958689 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/4758/statm: 197057 112797 466 189 0 191406 0
[pid=4758/tid=4760] ppid=4756 vsize=788228 CPUtime=99.87 cores=0-7
/proc/4758/task/4760/stat : 4760 (claspmt) R 4756 4758 4732 0 -1 4202560 11046 0 0 0 9978 9 0 0 25 0 8 0 641113664 807145472 112797 33554432000 4194304 4964800 140734686723968 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4758/tid=4761] ppid=4756 vsize=788228 CPUtime=101.09 cores=0-7
/proc/4758/task/4761/stat : 4761 (claspmt) R 4756 4758 4732 0 -1 4202560 10464 0 0 0 10102 7 0 0 25 0 8 0 641113664 807145472 112797 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4758/tid=4762] ppid=4756 vsize=788228 CPUtime=101.11 cores=0-7
/proc/4758/task/4762/stat : 4762 (claspmt) R 4756 4758 4732 0 -1 4202560 8883 0 0 0 10106 5 0 0 25 0 8 0 641113664 807145472 112797 33554432000 4194304 4964800 140734686723968 18446744073709551615 4627309 0 0 4096 18946 0 0 0 -1 2 0 0 0

################
# More data... #
################

[pid=4758] ppid=4756 vsize=2169272 CPUtime=37710.8 cores=0-7
/proc/4758/stat : 4758 (claspmt) R 4756 4758 4732 0 -1 4202496 483617 0 0 0 3770500 577 0 0 25 0 8 0 641113561 2221334528 476784 33554432000 4194304 4964800 140734686723968 18446744073709551615 4627254 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/4758/statm: 542318 476784 466 189 0 536667 0
[pid=4758/tid=4760] ppid=4756 vsize=2169272 CPUtime=4677.07 cores=0-7
/proc/4758/task/4760/stat : 4760 (claspmt) R 4756 4758 4732 0 -1 4202560 40513 0 0 0 467656 51 0 0 25 0 8 0 641113664 2221334528 476784 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4758/tid=4761] ppid=4756 vsize=2169272 CPUtime=4720.22 cores=0-7
/proc/4758/task/4761/stat : 4761 (claspmt) R 4756 4758 4732 0 -1 4202560 32218 0 0 0 471996 26 0 0 25 0 8 0 641113664 2221334528 476784 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4758/tid=4762] ppid=4756 vsize=2169272 CPUtime=4721.03 cores=0-7
/proc/4758/task/4762/stat : 4762 (claspmt) R 4756 4758 4732 0 -1 4202560 23897 0 0 0 472074 29 0 0 25 0 8 0 641113664 2221334528 476784 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4758/tid=4763] ppid=4756 vsize=2169272 CPUtime=4719.39 cores=0-7
/proc/4758/task/4763/stat : 4763 (claspmt) R 4756 4758 4732 0 -1 4202560 28845 0 0 0 471903 36 0 0 25 0 8 0 641113664 2221334528 476784 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4758/tid=4764] ppid=4756 vsize=2169272 CPUtime=4721.22 cores=0-7
/proc/4758/task/4764/stat : 4764 (claspmt) R 4756 4758 4732 0 -1 4202560 27646 0 0 0 472096 26 0 0 25 0 8 0 641113664 2221334528 476784 33554432000 4194304 4964800 140734686723968 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4758/tid=4765] ppid=4756 vsize=2169272 CPUtime=4708.81 cores=0-7
/proc/4758/task/4765/stat : 4765 (claspmt) R 4756 4758 4732 0 -1 4202560 25049 0 0 0 470844 37 0 0 25 0 8 0 641113664 2221334528 476784 33554432000 4194304 4964800 140734686723968 18446744073709551615 4320928 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4758/tid=4766] ppid=4756 vsize=2169272 CPUtime=4720.95 cores=0-7
/proc/4758/task/4766/stat : 4766 (claspmt) R 4756 4758 4732 0 -1 4202560 237191 0 0 0 471830 265 0 0 25 0 8 0 641113665 2221334528 476784 33554432000 4194304 4964800 140734686723968 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 37710.8
Current children cumulated vsize (KiB) 2169272

[startup+4782.3 s]
/proc/loadavg: 8.10 8.07 8.02 9/362 4910
/proc/meminfo: memFree=12684688/32951124 swapFree=67111348/67111528
[pid=4758] ppid=4756 vsize=2169272 CPUtime=38189.9 cores=0-7
/proc/4758/stat : 4758 (claspmt) R 4756 4758 4732 0 -1 4202496 492460 0 0 0 3818403 590 0 0 25 0 8 0 641113561 2221334528 485458 33554432000 4194304 4964800 140734686723968 18446744073709551615 4321023 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/4758/statm: 542318 485458 466 189 0 536667 0
[pid=4758/tid=4760] ppid=4756 vsize=2169272 CPUtime=4737.06 cores=0-7
/proc/4758/task/4760/stat : 4760 (claspmt) R 4756 4758 4732 0 -1 4202560 40515 0 0 0 473655 51 0 0 25 0 8 0 641113664 2221334528 485458 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4758/tid=4761] ppid=4756 vsize=2169272 CPUtime=4780.19 cores=0-7
/proc/4758/task/4761/stat : 4761 (claspmt) R 4756 4758 4732 0 -1 4202560 32227 0 0 0 477993 26 0 0 25 0 8 0 641113664 2221334528 485458 33554432000 4194304 4964800 140734686723968 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4758/tid=4762] ppid=4756 vsize=2169272 CPUtime=4781.01 cores=0-7
/proc/4758/task/4762/stat : 4762 (claspmt) R 4756 4758 4732 0 -1 4202560 23899 0 0 0 478072 29 0 0 25 0 8 0 641113664 2221334528 485458 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4758/tid=4763] ppid=4756 vsize=2169272 CPUtime=4779.38 cores=0-7
/proc/4758/task/4763/stat : 4763 (claspmt) R 4756 4758 4732 0 -1 4202560 28944 0 0 0 477902 36 0 0 25 0 8 0 641113664 2221334528 485458 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4758/tid=4764] ppid=4756 vsize=2169272 CPUtime=4781.21 cores=0-7
/proc/4758/task/4764/stat : 4764 (claspmt) R 4756 4758 4732 0 -1 4202560 27894 0 0 0 478094 27 0 0 18 0 8 0 641113664 2221334528 485458 33554432000 4194304 4964800 140734686723968 18446744073709551615 233509995138 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4758/tid=4765] ppid=4756 vsize=2169272 CPUtime=4768.09 cores=0-7
/proc/4758/task/4765/stat : 4765 (claspmt) R 4756 4758 4732 0 -1 4202560 25598 0 0 0 476772 37 0 0 25 0 8 0 641113664 2221334528 485458 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670313 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4758/tid=4766] ppid=4756 vsize=2169272 CPUtime=4780.93 cores=0-7
/proc/4758/task/4766/stat : 4766 (claspmt) R 4756 4758 4732 0 -1 4202560 242796 0 0 0 477821 272 0 0 25 0 8 0 641113665 2221334528 485458 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 38189.9
Current children cumulated vsize (KiB) 2169272

[startup+4842.3 s]
/proc/loadavg: 8.09 8.07 8.02 9/362 4911
/proc/meminfo: memFree=12682432/32951124 swapFree=67111348/67111528
[pid=4758] ppid=4756 vsize=2169272 CPUtime=38669.2 cores=0-7
/proc/4758/stat : 4758 (claspmt) R 4756 4758 4732 0 -1 4202496 493005 0 0 0 3866326 592 0 0 25 0 8 0 641113561 2221334528 485988 33554432000 4194304 4964800 140734686723968 18446744073709551615 4320947 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/4758/statm: 542318 485988 466 189 0 536667 0
[pid=4758/tid=4760] ppid=4756 vsize=2169272 CPUtime=4797.06 cores=0-7
/proc/4758/task/4760/stat : 4760 (claspmt) R 4756 4758 4732 0 -1 4202560 40515 0 0 0 479655 51 0 0 25 0 8 0 641113664 2221334528 485988 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4758/tid=4761] ppid=4756 vsize=2169272 CPUtime=4840.18 cores=0-7
/proc/4758/task/4761/stat : 4761 (claspmt) R 4756 4758 4732 0 -1 4202560 32487 0 0 0 483992 26 0 0 25 0 8 0 641113664 2221334528 485988 33554432000 4194304 4964800 140734686723968 18446744073709551615 4627606 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4758/tid=4762] ppid=4756 vsize=2169272 CPUtime=4841.01 cores=0-7
/proc/4758/task/4762/stat : 4762 (claspmt) R 4756 4758 4732 0 -1 4202560 23900 0 0 0 484072 29 0 0 25 0 8 0 641113664 2221334528 485988 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4758/tid=4763] ppid=4756 vsize=2169272 CPUtime=4839.35 cores=0-7
/proc/4758/task/4763/stat : 4763 (claspmt) R 4756 4758 4732 0 -1 4202560 28944 0 0 0 483899 36 0 0 25 0 8 0 641113664 2221334528 485988 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4758/tid=4764] ppid=4756 vsize=2169272 CPUtime=4841.19 cores=0-7
/proc/4758/task/4764/stat : 4764 (claspmt) R 4756 4758 4732 0 -1 4202560 28169 0 0 0 484092 27 0 0 25 0 8 0 641113664 2221334528 485988 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4758/tid=4765] ppid=4756 vsize=2169272 CPUtime=4827.37 cores=0-7
/proc/4758/task/4765/stat : 4765 (claspmt) R 4756 4758 4732 0 -1 4202560 25598 0 0 0 482700 37 0 0 25 0 8 0 641113664 2221334528 485988 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4758/tid=4766] ppid=4756 vsize=2169272 CPUtime=4840.92 cores=0-7
/proc/4758/task/4766/stat : 4766 (claspmt) R 4756 4758 4732 0 -1 4202560 242797 0 0 0 483820 272 0 0 25 0 8 0 641113665 2221334528 485988 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 38669.2
Current children cumulated vsize (KiB) 2169272

[startup+4902.3 s]
/proc/loadavg: 8.08 8.07 8.02 9/362 4913
/proc/meminfo: memFree=12681776/32951124 swapFree=67111348/67111528
[pid=4758] ppid=4756 vsize=2169272 CPUtime=39148.5 cores=0-7
/proc/4758/stat : 4758 (claspmt) R 4756 4758 4732 0 -1 4202496 493140 0 0 0 3914255 592 0 0 25 0 8 0 641113561 2221334528 486049 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/4758/statm: 542318 486049 466 189 0 536667 0
[pid=4758/tid=4760] ppid=4756 vsize=2169272 CPUtime=4857.06 cores=0-7
/proc/4758/task/4760/stat : 4760 (claspmt) R 4756 4758 4732 0 -1 4202560 40516 0 0 0 485655 51 0 0 25 0 8 0 641113664 2221334528 486049 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4758/tid=4761] ppid=4756 vsize=2169272 CPUtime=4900.18 cores=0-7
/proc/4758/task/4761/stat : 4761 (claspmt) R 4756 4758 4732 0 -1 4202560 32601 0 0 0 489992 26 0 0 25 0 8 0 641113664 2221334528 486049 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4758/tid=4762] ppid=4756 vsize=2169272 CPUtime=4901.01 cores=0-7
/proc/4758/task/4762/stat : 4762 (claspmt) R 4756 4758 4732 0 -1 4202560 23900 0 0 0 490072 29 0 0 25 0 8 0 641113664 2221334528 486049 33554432000 4194304 4964800 140734686723968 18446744073709551615 4659391 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4758/tid=4763] ppid=4756 vsize=2169272 CPUtime=4899.35 cores=0-7
/proc/4758/task/4763/stat : 4763 (claspmt) R 4756 4758 4732 0 -1 4202560 28958 0 0 0 489899 36 0 0 25 0 8 0 641113664 2221334528 486049 33554432000 4194304 4964800 140734686723968 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4758/tid=4764] ppid=4756 vsize=2169272 CPUtime=4901.2 cores=0-7
/proc/4758/task/4764/stat : 4764 (claspmt) R 4756 4758 4732 0 -1 4202560 28169 0 0 0 490093 27 0 0 25 0 8 0 641113664 2221334528 486049 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670310 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4758/tid=4765] ppid=4756 vsize=2169272 CPUtime=4886.67 cores=0-7
/proc/4758/task/4765/stat : 4765 (claspmt) R 4756 4758 4732 0 -1 4202560 25599 0 0 0 488630 37 0 0 25 0 8 0 641113664 2221334528 486049 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4758/tid=4766] ppid=4756 vsize=2169272 CPUtime=4900.91 cores=0-7
/proc/4758/task/4766/stat : 4766 (claspmt) R 4756 4758 4732 0 -1 4202560 242799 0 0 0 489819 272 0 0 25 0 8 0 641113665 2221334528 486049 33554432000 4194304 4964800 140734686723968 18446744073709551615 4320947 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 39148.5
Current children cumulated vsize (KiB) 2169272

[startup+4962.3 s]
/proc/loadavg: 8.08 8.07 8.02 9/362 4914
/proc/meminfo: memFree=12681632/32951124 swapFree=67111348/67111528
[pid=4758] ppid=4756 vsize=2169272 CPUtime=39627.8 cores=0-7
/proc/4758/stat : 4758 (claspmt) R 4756 4758 4732 0 -1 4202496 493161 0 0 0 3962182 593 0 0 25 0 8 0 641113561 2221334528 486070 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670299 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/4758/statm: 542318 486070 466 189 0 536667 0
[pid=4758/tid=4760] ppid=4756 vsize=2169272 CPUtime=4917.06 cores=0-7
/proc/4758/task/4760/stat : 4760 (claspmt) R 4756 4758 4732 0 -1 4202560 40516 0 0 0 491655 51 0 0 25 0 8 0 641113664 2221334528 486070 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4758/tid=4761] ppid=4756 vsize=2169272 CPUtime=4960.16 cores=0-7
/proc/4758/task/4761/stat : 4761 (claspmt) R 4756 4758 4732 0 -1 4202560 32618 0 0 0 495990 26 0 0 25 0 8 0 641113664 2221334528 486070 33554432000 4194304 4964800 140734686723968 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4758/tid=4762] ppid=4756 vsize=2169272 CPUtime=4961.01 cores=0-7
/proc/4758/task/4762/stat : 4762 (claspmt) R 4756 4758 4732 0 -1 4202560 23901 0 0 0 496072 29 0 0 25 0 8 0 641113664 2221334528 486070 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670499 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4758/tid=4763] ppid=4756 vsize=2169272 CPUtime=4959.35 cores=0-7
/proc/4758/task/4763/stat : 4763 (claspmt) R 4756 4758 4732 0 -1 4202560 28961 0 0 0 495899 36 0 0 25 0 8 0 641113664 2221334528 486070 33554432000 4194304 4964800 140734686723968 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4758/tid=4764] ppid=4756 vsize=2169272 CPUtime=4961.2 cores=0-7
/proc/4758/task/4764/stat : 4764 (claspmt) R 4756 4758 4732 0 -1 4202560 28169 0 0 0 496093 27 0 0 25 0 8 0 641113664 2221334528 486070 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4758/tid=4765] ppid=4756 vsize=2169272 CPUtime=4945.96 cores=0-7
/proc/4758/task/4765/stat : 4765 (claspmt) R 4756 4758 4732 0 -1 4202560 25599 0 0 0 494559 37 0 0 25 0 8 0 641113664 2221334528 486070 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4758/tid=4766] ppid=4756 vsize=2169272 CPUtime=4960.92 cores=0-7
/proc/4758/task/4766/stat : 4766 (claspmt) R 4756 4758 4732 0 -1 4202560 242799 0 0 0 495819 273 0 0 25 0 8 0 641113665 2221334528 486070 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670499 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 39627.8
Current children cumulated vsize (KiB) 2169272



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5009 s]
/proc/loadavg: 8.14 8.08 8.02 9/362 4916
/proc/meminfo: memFree=12680732/32951124 swapFree=67111348/67111528
[pid=4758] ppid=4756 vsize=2169272 CPUtime=40000.8 cores=0-7
/proc/4758/stat : 4758 (claspmt) R 4756 4758 4732 0 -1 4202496 493668 0 0 0 3999482 595 0 0 25 0 8 0 641113561 2221334528 486124 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/4758/statm: 542318 486124 466 189 0 536667 0
[pid=4758/tid=4760] ppid=4756 vsize=2169272 CPUtime=4963.7 cores=0-7
/proc/4758/task/4760/stat : 4760 (claspmt) R 4756 4758 4732 0 -1 4202560 41003 0 0 0 496318 52 0 0 25 0 8 0 641113664 2221334528 486124 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670310 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=4758/tid=4761] ppid=4756 vsize=2169272 CPUtime=5006.88 cores=0-7
/proc/4758/task/4761/stat : 4761 (claspmt) R 4756 4758 4732 0 -1 4202560 32618 0 0 0 500661 27 0 0 25 0 8 0 641113664 2221334528 486124 33554432000 4194304 4964800 140734686723968 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=4758/tid=4762] ppid=4756 vsize=2169272 CPUtime=5007.72 cores=0-7
/proc/4758/task/4762/stat : 4762 (claspmt) R 4756 4758 4732 0 -1 4202560 23901 0 0 0 500743 29 0 0 25 0 8 0 641113664 2221334528 486124 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=4758/tid=4763] ppid=4756 vsize=2169272 CPUtime=5006.06 cores=0-7
/proc/4758/task/4763/stat : 4763 (claspmt) R 4756 4758 4732 0 -1 4202560 28961 0 0 0 500570 36 0 0 25 0 8 0 641113664 2221334528 486124 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=4758/tid=4764] ppid=4756 vsize=2169272 CPUtime=5007.92 cores=0-7
/proc/4758/task/4764/stat : 4764 (claspmt) R 4756 4758 4732 0 -1 4202560 28169 0 0 0 500764 28 0 0 25 0 8 0 641113664 2221334528 486124 33554432000 4194304 4964800 140734686723968 18446744073709551615 4320886 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=4758/tid=4765] ppid=4756 vsize=2169272 CPUtime=4992.12 cores=0-7
/proc/4758/task/4765/stat : 4765 (claspmt) R 4756 4758 4732 0 -1 4202560 25599 0 0 0 499174 38 0 0 25 0 8 0 641113664 2221334528 486124 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=4758/tid=4766] ppid=4756 vsize=2169272 CPUtime=5007.63 cores=0-7
/proc/4758/task/4766/stat : 4766 (claspmt) R 4756 4758 4732 0 -1 4202560 242818 0 0 0 500490 273 0 0 25 0 8 0 641113665 2221334528 486124 33554432000 4194304 4964800 140734686723968 18446744073709551615 4670295 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 40000.8
Current children cumulated vsize (KiB) 2169272

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5009.41 s]
/proc/loadavg: 8.14 8.08 8.02 2/356 4917
/proc/meminfo: memFree=12688220/32951124 swapFree=67111348/67111528
[pid=4758] ppid=4756 vsize=2158064 CPUtime=40001.3 cores=0-7
/proc/4758/stat : 4758 (claspmt) R 4756 4758 4732 0 -1 4202496 493678 0 0 0 3999531 595 0 0 25 0 1 0 641113561 2209857536 481470 33554432000 4194304 4964800 140734686723968 18446744073709551615 233509952793 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/4758/statm: 539516 481470 476 189 0 533865 0
Current children cumulated CPU time (s) 40001.3
Current children cumulated vsize (KiB) 2158064

[startup+5009.6 s]
/proc/loadavg: 8.14 8.08 8.02 2/356 4917
/proc/meminfo: memFree=12688220/32951124 swapFree=67111348/67111528
[pid=4758] ppid=4756 vsize=2158064 CPUtime=40001.4 cores=0-7
/proc/4758/stat : 4758 (claspmt) R 4756 4758 4732 0 -1 4202496 493678 0 0 0 3999549 595 0 0 25 0 1 0 641113561 2209857536 481465 33554432000 4194304 4964800 140734686723968 18446744073709551615 233509952715 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/4758/statm: 539516 481465 476 189 0 533865 0
Current children cumulated CPU time (s) 40001.4
Current children cumulated vsize (KiB) 2158064

[startup+5009.7 s]
/proc/loadavg: 8.14 8.08 8.02 2/356 4917
/proc/meminfo: memFree=12688220/32951124 swapFree=67111348/67111528
[pid=4758] ppid=4756 vsize=1988332 CPUtime=40001.5 cores=0-7
/proc/4758/stat : 4758 (claspmt) R 4756 4758 4732 0 -1 4202496 493678 0 0 0 3999558 596 0 0 25 0 1 0 641113561 2036051968 435930 33554432000 4194304 4964800 140734686723968 18446744073709551615 233510340967 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/4758/statm: 497083 435390 476 189 0 491432 0
Current children cumulated CPU time (s) 40001.5
Current children cumulated vsize (KiB) 1988332

Child status: 0
Real time (s): 5009.8
CPU time (s): 40001.7
CPU user time (s): 39995.6
CPU system time (s): 6.06508
CPU usage (%): 798.468
Max. virtual memory (cumulated for all children) (KiB): 2169272

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39995.6
system time used= 6.06508
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 493679
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 74673
involuntary context switches= 190495

runsolver used 16.3885 second user time and 42.8335 second system time

The end

Launcher Data 

Begin job on node145 at 2011-05-15 21:16:17
IDJOB=3380685
IDBENCH=71294
IDSOLVER=1865
FILE ID=node145/3380685-1305486977
RUNJOBID= node145-1305486977-4746
PBS_JOBID= 13325380
Free space on /tmp= 73664 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT09/CRAFTED/rbsat/random/unforced/rbsat-v2100c215164g9.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3380685-1305486977/watcher-3380685-1305486977 -o /tmp/evaluation-result-3380685-1305486977/solver-3380685-1305486977 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3380685-1305486977.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= e033e9c916da557514299cce9ca2468f
RANDOM SEED=530712817

node145.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.57
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      14626840 kB
Buffers:        325348 kB
Cached:        8561004 kB
SwapCached:          0 kB
Active:       11147904 kB
Inactive:      6807492 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      14626840 kB
SwapTotal:    67111528 kB
SwapFree:     67111348 kB
Dirty:            3624 kB
Writeback:           0 kB
AnonPages:     9069092 kB
Mapped:          27980 kB
Slab:           258912 kB
PageTables:      42804 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 83664964 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73664 MiB
End job on node145 at 2011-05-15 22:39:48