Trace number 3373537

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 4800.29 1203.35

General information on the benchmark

NameAPPLICATIONS/bitverif/countbitsrotate/
countbitsrotate032.cnf
MD5SUM8d76aa747359ea8ccf15726792949452
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables8527
Number of clauses25484
Sum of the clauses size59460
Maximum clause length3
Minimum clause length1
Number of clauses of size 12
Number of clauses of size 216988
Number of clauses of size 38494
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...418404/instance-3373537-1305418404.cnf
0.00/0.04	c Solving...
4800.22/1203.31	
4800.22/1203.31	*** INTERRUPTED! ***
4800.22/1203.31	*** Shutting down threads...
4800.22/1203.34	s UNKNOWN
4800.22/1203.34	
4800.22/1203.34	c INTERRUPTED   : 1
4800.22/1203.34	c Models        : 0+    
4800.22/1203.34	c Time          : 1203.315s (Solving: 1203.27s 1st Model: 0.00s Unsat: 0.00s)
4800.22/1203.34	c CPU Time      : 4800.250s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3373537-1305418404/watcher-3373537-1305418404 -o /tmp/evaluation-result-3373537-1305418404/solver-3373537-1305418404 -C 4800 -W 1300 -M 15500 HOME/claspmt -t4 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3373537-1305418404.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.93 6.05 7.26 2/319 25456
/proc/meminfo: memFree=25271864/32951124 swapFree=44485764/67111528
[pid=25456] ppid=25454 vsize=24148 CPUtime=0 cores=1,3,5,7
/proc/25456/stat : 25456 (claspmt) R 25454 25456 24676 0 -1 4202496 1511 0 0 0 0 0 0 0 18 0 1 0 653440827 24727552 1256 33554432000 4194304 4964800 140735582009376 18446744073709551615 244446641931 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/25456/statm: 6037 1256 398 189 0 911 0

[startup+0.575003 s]
/proc/loadavg: 3.93 6.05 7.26 2/319 25456
/proc/meminfo: memFree=25271864/32951124 swapFree=44485764/67111528
[pid=25456] ppid=25454 vsize=174812 CPUtime=2.14 cores=1,3,5,7
/proc/25456/stat : 25456 (claspmt) R 25454 25456 24676 0 -1 4202496 3579 0 0 0 214 0 0 0 19 0 4 0 653440827 179007488 2989 33554432000 4194304 4964800 140735582009376 18446744073709551615 4668773 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25456/statm: 43703 2989 458 189 0 38052 0
Current children cumulated CPU time (s) 2.14
Current children cumulated vsize (KiB) 174812

[startup+0.600936 s]
/proc/loadavg: 3.93 6.05 7.26 2/319 25456
/proc/meminfo: memFree=25271864/32951124 swapFree=44485764/67111528
[pid=25456] ppid=25454 vsize=174960 CPUtime=2.24 cores=1,3,5,7
/proc/25456/stat : 25456 (claspmt) R 25454 25456 24676 0 -1 4202496 3610 0 0 0 224 0 0 0 19 0 4 0 653440827 179159040 3020 33554432000 4194304 4964800 140735582009376 18446744073709551615 4362287 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25456/statm: 43740 3020 458 189 0 38089 0
Current children cumulated CPU time (s) 2.24
Current children cumulated vsize (KiB) 174960

[startup+0.700918 s]
/proc/loadavg: 3.93 6.05 7.26 2/319 25456
/proc/meminfo: memFree=25271864/32951124 swapFree=44485764/67111528
[pid=25456] ppid=25454 vsize=174960 CPUtime=2.64 cores=1,3,5,7
/proc/25456/stat : 25456 (claspmt) R 25454 25456 24676 0 -1 4202496 3701 0 0 0 264 0 0 0 19 0 4 0 653440827 179159040 3111 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670299 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25456/statm: 43740 3111 458 189 0 38089 0
Current children cumulated CPU time (s) 2.64
Current children cumulated vsize (KiB) 174960

[startup+0.800898 s]
/proc/loadavg: 3.93 6.05 7.26 2/319 25456
/proc/meminfo: memFree=25271864/32951124 swapFree=44485764/67111528
[pid=25456] ppid=25454 vsize=175092 CPUtime=3.04 cores=1,3,5,7
/proc/25456/stat : 25456 (claspmt) R 25454 25456 24676 0 -1 4202496 3783 0 0 0 304 0 0 0 20 0 4 0 653440827 179294208 3193 33554432000 4194304 4964800 140735582009376 18446744073709551615 4668709 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25456/statm: 43773 3193 458 189 0 38122 0
Current children cumulated CPU time (s) 3.04
Current children cumulated vsize (KiB) 175092

[startup+1.50077 s]
/proc/loadavg: 4.01 6.03 7.25 6/328 25466
/proc/meminfo: memFree=25084924/32951124 swapFree=44485764/67111528
[pid=25456] ppid=25454 vsize=175916 CPUtime=5.88 cores=1,3,5,7
/proc/25456/stat : 25456 (claspmt) R 25454 25456 24676 0 -1 4202496 4338 0 0 0 587 1 0 0 18 0 4 0 653440827 180137984 3748 33554432000 4194304 4964800 140735582009376 18446744073709551615 4669280 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25456/statm: 43979 3749 458 189 0 38328 0
[pid=25456/tid=25464] ppid=25454 vsize=175916 CPUtime=1.46 cores=1,3,5,7
/proc/25456/task/25464/stat : 25464 (claspmt) R 25454 25456 24676 0 -1 4202560 592 0 0 0 146 0 0 0 20 0 4 0 653440831 180137984 3749 33554432000 4194304 4964800 140735582009376 18446744073709551615 4660187 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25456/tid=25465] ppid=25454 vsize=175916 CPUtime=1.44 cores=1,3,5,7
/proc/25456/task/25465/stat : 25465 (claspmt) R 25454 25456 24676 0 -1 4202560 792 0 0 0 144 0 0 0 20 0 4 0 653440831 180137984 3749 33554432000 4194304 4964800 140735582009376 18446744073709551615 4660155 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25456/tid=25466] ppid=25454 vsize=175916 CPUtime=1.46 cores=1,3,5,7
/proc/25456/task/25466/stat : 25466 (claspmt) R 25454 25456 24676 0 -1 4202560 517 0 0 0 146 0 0 0 23 0 4 0 653440831 180137984 3749 33554432000 4194304 4964800 140735582009376 18446744073709551615 4683657 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 5.88
Current children cumulated vsize (KiB) 175916

[startup+3.10047 s]
/proc/loadavg: 4.01 6.03 7.25 6/328 25466
/proc/meminfo: memFree=25066924/32951124 swapFree=44485764/67111528
[pid=25456] ppid=25454 vsize=176312 CPUtime=12.22 cores=1,3,5,7
/proc/25456/stat : 25456 (claspmt) R 25454 25456 24676 0 -1 4202496 4852 0 0 0 1221 1 0 0 21 0 4 0 653440827 180543488 4262 33554432000 4194304 4964800 140735582009376 18446744073709551615 4668725 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25456/statm: 44078 4262 458 189 0 38427 0
[pid=25456/tid=25464] ppid=25454 vsize=176312 CPUtime=3.04 cores=1,3,5,7
/proc/25456/task/25464/stat : 25464 (claspmt) R 25454 25456 24676 0 -1 4202560 765 0 0 0 304 0 0 0 25 0 4 0 653440831 180543488 4262 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670129 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25456/tid=25465] ppid=25454 vsize=176312 CPUtime=3.02 cores=1,3,5,7
/proc/25456/task/25465/stat : 25465 (claspmt) R 25454 25456 24676 0 -1 4202560 1015 0 0 0 302 0 0 0 25 0 4 0 653440831 180543488 4262 33554432000 4194304 4964800 140735582009376 18446744073709551615 4378538 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25456/tid=25466] ppid=25454 vsize=176312 CPUtime=3.04 cores=1,3,5,7
/proc/25456/task/25466/stat : 25466 (claspmt) R 25454 25456 24676 0 -1 4202560 564 0 0 0 304 0 0 0 25 0 4 0 653440831 180543488 4262 33554432000 4194304 4964800 140735582009376 18446744073709551615 4668728 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 12.22
Current children cumulated vsize (KiB) 176312

[startup+6.30088 s]
/proc/loadavg: 4.33 6.06 7.25 9/331 25469
/proc/meminfo: memFree=24989780/32951124 swapFree=44485764/67111528
[pid=25456] ppid=25454 vsize=176444 CPUtime=24.99 cores=1,3,5,7
/proc/25456/stat : 25456 (claspmt) R 25454 25456 24676 0 -1 4202496 5414 0 0 0 2497 2 0 0 25 0 4 0 653440827 180678656 4824 33554432000 4194304 4964800 140735582009376 18446744073709551615 4668937 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25456/statm: 44111 4824 458 189 0 38460 0
[pid=25456/tid=25464] ppid=25454 vsize=176444 CPUtime=6.24 cores=1,3,5,7
/proc/25456/task/25464/stat : 25464 (claspmt) R 25454 25456 24676 0 -1 4202560 940 0 0 0 624 0 0 0 25 0 4 0 653440831 180678656 4824 33554432000 4194304 4964800 140735582009376 18446744073709551615 4660155 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25456/tid=25465] ppid=25454 vsize=176444 CPUtime=6.19 cores=1,3,5,7
/proc/25456/task/25465/stat : 25465 (claspmt) R 25454 25456 24676 0 -1 4202560 1332 0 0 0 618 1 0 0 25 0 4 0 653440831 180678656 4824 33554432000 4194304 4964800 140735582009376 18446744073709551615 4669136 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25456/tid=25466] ppid=25454 vsize=176444 CPUtime=6.25 cores=1,3,5,7
/proc/25456/task/25466/stat : 25466 (claspmt) R 25454 25456 24676 0 -1 4202560 598 0 0 0 625 0 0 0 25 0 4 0 653440831 180678656 4824 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 24.99
Current children cumulated vsize (KiB) 176444

[startup+12.7007 s]
/proc/loadavg: 4.63 6.09 7.26 9/331 25469
/proc/meminfo: memFree=24973944/32951124 swapFree=44485764/67111528
[pid=25456] ppid=25454 vsize=178408 CPUtime=50.52 cores=1,3,5,7
/proc/25456/stat : 25456 (claspmt) R 25454 25456 24676 0 -1 4202496 6328 0 0 0 5049 3 0 0 25 0 4 0 653440827 182689792 5738 33554432000 4194304 4964800 140735582009376 18446744073709551615 4660495 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25456/statm: 44602 5738 458 189 0 38951 0
[pid=25456/tid=25464] ppid=25454 vsize=178408 CPUtime=12.64 cores=1,3,5,7
/proc/25456/task/25464/stat : 25464 (claspmt) R 25454 25456 24676 0 -1 4202560 1084 0 0 0 1264 0 0 0 25 0 4 0 653440831 182689792 5738 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25456/tid=25465] ppid=25454 vsize=178408 CPUtime=12.52 cores=1,3,5,7
/proc/25456/task/25465/stat : 25465 (claspmt) R 25454 25456 24676 0 -1 4202560 1712 0 0 0 1251 1 0 0 25 0 4 0 653440831 182689792 5738 33554432000 4194304 4964800 140735582009376 18446744073709551615 4668698 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25456/tid=25466] ppid=25454 vsize=178408 CPUtime=12.65 cores=1,3,5,7
/proc/25456/task/25466/stat : 25466 (claspmt) R 25454 25456 24676 0 -1 4202560 678 0 0 0 1265 0 0 0 25 0 4 0 653440831 182689792 5738 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670160 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 50.52
Current children cumulated vsize (KiB) 178408

[startup+25.5003 s]
/proc/loadavg: 5.14 6.15 7.27 9/331 25469
/proc/meminfo: memFree=24958188/32951124 swapFree=44485764/67111528
[pid=25456] ppid=25454 vsize=179888 CPUtime=101.6 cores=1,3,5,7
/proc/25456/stat : 25456 (claspmt) R 25454 25456 24676 0 -1 4202496 7385 0 0 0 10156 4 0 0 25 0 4 0 653440827 184205312 6795 33554432000 4194304 4964800 140735582009376 18446744073709551615 4668588 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25456/statm: 44972 6795 460 189 0 39321 0
[pid=25456/tid=25464] ppid=25454 vsize=179888 CPUtime=25.44 cores=1,3,5,7
/proc/25456/task/25464/stat : 25464 (claspmt) R 25454 25456 24676 0 -1 4202560 1386 0 0 0 2544 0 0 0 25 0 4 0 653440831 184205312 6795 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25456/tid=25465] ppid=25454 vsize=179888 CPUtime=25.2 cores=1,3,5,7
/proc/25456/task/25465/stat : 25465 (claspmt) R 25454 25456 24676 0 -1 4202560 1996 0 0 0 2519 1 0 0 25 0 4 0 653440831 184205312 6795 33554432000 4194304 4964800 140735582009376 18446744073709551615 4668934 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25456/tid=25466] ppid=25454 vsize=179888 CPUtime=25.44 cores=1,3,5,7
/proc/25456/task/25466/stat : 25466 (claspmt) R 25454 25456 24676 0 -1 4202560 763 0 0 0 2544 0 0 0 25 0 4 0 653440831 184205312 6795 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670321 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 101.6
Current children cumulated vsize (KiB) 179888

[startup+51.1125 s]
/proc/loadavg: 6.39 6.36 7.30 9/331 25470
/proc/meminfo: memFree=24929728/32951124 swapFree=44485764/67111528
[pid=25456] ppid=25454 vsize=183364 CPUtime=203.79 cores=1,3,5,7
/proc/25456/stat : 25456 (claspmt) R 25454 25456 24676 0 -1 4202496 10780 0 0 0 20372 7 0 0 25 0 4 0 653440827 187764736 10190 33554432000 4194304 4964800 140735582009376 18446744073709551615 4669093 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25456/statm: 45841 10190 460 189 0 40190 0
[pid=25456/tid=25464] ppid=25454 vsize=183364 CPUtime=51.06 cores=1,3,5,7
/proc/25456/task/25464/stat : 25464 (claspmt) R 25454 25456 24676 0 -1 4202560 2122 0 0 0 5105 1 0 0 25 0 4 0 653440831 187764736 10190 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25456/tid=25465] ppid=25454 vsize=183364 CPUtime=50.54 cores=1,3,5,7
/proc/25456/task/25465/stat : 25465 (claspmt) R 25454 25456 24676 0 -1 4202560 3152 0 0 0 5052 2 0 0 25 0 4 0 653440831 187764736 10190 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25456/tid=25466] ppid=25454 vsize=183364 CPUtime=51.06 cores=1,3,5,7
/proc/25456/task/25466/stat : 25466 (claspmt) R 25454 25456 24676 0 -1 4202560 1237 0 0 0 5106 0 0 0 25 0 4 0 653440831 187764736 10190 33554432000 4194304 4964800 140735582009376 18446744073709551615 4684040 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 203.79
Current children cumulated vsize (KiB) 183364

[startup+102.301 s]
/proc/loadavg: 7.43 6.64 7.34 9/331 25471
/proc/meminfo: memFree=24880108/32951124 swapFree=44485764/67111528
[pid=25456] ppid=25454 vsize=188420 CPUtime=408 cores=1,3,5,7
/proc/25456/stat : 25456 (claspmt) R 25454 25456 24676 0 -1 4202496 14489 0 0 0 40790 10 0 0 25 0 4 0 653440827 192942080 13778 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25456/statm: 47105 13778 460 189 0 41454 0
[pid=25456/tid=25464] ppid=25454 vsize=188420 CPUtime=102.24 cores=1,3,5,7
/proc/25456/task/25464/stat : 25464 (claspmt) R 25454 25456 24676 0 -1 4202560 2976 0 0 0 10222 2 0 0 25 0 4 0 653440831 192942080 13778 33554432000 4194304 4964800 140735582009376 18446744073709551615 4668773 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25456/tid=25465] ppid=25454 vsize=188420 CPUtime=101.2 cores=1,3,5,7
/proc/25456/task/25465/stat : 25465 (claspmt) R 25454 25456 24676 0 -1 4202560 4687 0 0 0 10116 4 0 0 25 0 4 0 653440831 192942080 13778 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25456/tid=25466] ppid=25454 vsize=188420 CPUtime=102.24 cores=1,3,5,7
/proc/25456/task/25466/stat : 25466 (claspmt) R 25454 25456 24676 0 -1 4202560 1902 0 0 0 10224 0 0 0 25 0 4 0 653440831 192942080 13778 33554432000 4194304 4964800 140735582009376 18446744073709551615 4668794 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 408
Current children cumulated vsize (KiB) 188420

[startup+162.301 s]
/proc/loadavg: 7.91 6.92 7.39 9/331 25473
/proc/meminfo: memFree=24817456/32951124 swapFree=44485764/67111528
[pid=25456] ppid=25454 vsize=194908 CPUtime=647.35 cores=1,3,5,7
/proc/25456/stat : 25456 (claspmt) R 25454 25456 24676 0 -1 4202496 17299 0 0 0 64723 12 0 0 25 0 4 0 653440827 199585792 16588 33554432000 4194304 4964800 140735582009376 18446744073709551615 4668689 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25456/statm: 48727 16588 460 189 0 43076 0
[pid=25456/tid=25464] ppid=25454 vsize=194908 CPUtime=162.25 cores=1,3,5,7
/proc/25456/task/25464/stat : 25464 (claspmt) R 25454 25456 24676 0 -1 4202560 3790 0 0 0 16222 3 0 0 25 0 4 0 653440831 199585792 16588 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25456/tid=25465] ppid=25454 vsize=194908 CPUtime=160.55 cores=1,3,5,7
/proc/25456/task/25465/stat : 25465 (claspmt) R 25454 25456 24676 0 -1 4202560 5815 0 0 0 16050 5 0 0 25 0 4 0 653440831 199585792 16588 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670313 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25456/tid=25466] ppid=25454 vsize=194908 CPUtime=162.24 cores=1,3,5,7
/proc/25456/task/25466/stat : 25466 (claspmt) R 25454 25456 24676 0 -1 4202560 2271 0 0 0 16224 0 0 0 25 0 4 0 653440831 199585792 16588 33554432000 4194304 4964800 140735582009376 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 647.35
Current children cumulated vsize (KiB) 194908

[startup+222.301 s]
/proc/loadavg: 8.04 7.13 7.43 9/331 25474
/proc/meminfo: memFree=24764852/32951124 swapFree=44485764/67111528
[pid=25456] ppid=25454 vsize=200788 CPUtime=886.72 cores=1,3,5,7
/proc/25456/stat : 25456 (claspmt) R 25454 25456 24676 0 -1 4202496 19173 0 0 0 88658 14 0 0 25 0 4 0 653440827 205606912 18366 33554432000 4194304 4964800 140735582009376 18446744073709551615 4362492 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25456/statm: 50197 18366 460 189 0 44546 0
[pid=25456/tid=25464] ppid=25454 vsize=200788 CPUtime=222.25 cores=1,3,5,7
/proc/25456/task/25464/stat : 25464 (claspmt) R 25454 25456 24676 0 -1 4202560 4366 0 0 0 22222 3 0 0 25 0 4 0 653440831 205606912 18366 33554432000 4194304 4964800 140735582009376 18446744073709551615 4362191 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25456/tid=25465] ppid=25454 vsize=200788 CPUtime=219.9 cores=1,3,5,7
/proc/25456/task/25465/stat : 25465 (claspmt) R 25454 25456 24676 0 -1 4202560 6388 0 0 0 21985 5 0 0 25 0 4 0 653440831 205606912 18366 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25456/tid=25466] ppid=25454 vsize=200788 CPUtime=222.25 cores=1,3,5,7
/proc/25456/task/25466/stat : 25466 (claspmt) R 25454 25456 24676 0 -1 4202560 2618 0 0 0 22224 1 0 0 25 0 4 0 653440831 205606912 18366 33554432000 4194304 4964800 140735582009376 18446744073709551615 4669280 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 886.72
Current children cumulated vsize (KiB) 200788

[startup+282.3 s]

################
# More data... #
################

[pid=25456] ppid=25454 vsize=202748 CPUtime=2562.23 cores=1,3,5,7
/proc/25456/stat : 25456 (claspmt) R 25454 25456 24676 0 -1 4202496 23398 0 0 0 256203 20 0 0 25 0 4 0 653440827 207613952 22355 33554432000 4194304 4964800 140735582009376 18446744073709551615 4668712 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25456/statm: 50687 22355 460 189 0 45036 0
[pid=25456/tid=25464] ppid=25454 vsize=202748 CPUtime=642.24 cores=1,3,5,7
/proc/25456/task/25464/stat : 25464 (claspmt) R 25454 25456 24676 0 -1 4202560 5961 0 0 0 64220 4 0 0 25 0 4 0 653440831 207613952 22355 33554432000 4194304 4964800 140735582009376 18446744073709551615 4669276 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25456/tid=25465] ppid=25454 vsize=202748 CPUtime=635.4 cores=1,3,5,7
/proc/25456/task/25465/stat : 25465 (claspmt) R 25454 25456 24676 0 -1 4202560 7492 0 0 0 63533 7 0 0 25 0 4 0 653440831 207613952 22355 33554432000 4194304 4964800 140735582009376 18446744073709551615 4668715 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25456/tid=25466] ppid=25454 vsize=202748 CPUtime=642.26 cores=1,3,5,7
/proc/25456/task/25466/stat : 25466 (claspmt) R 25454 25456 24676 0 -1 4202560 3494 0 0 0 64224 2 0 0 25 0 4 0 653440831 207613952 22355 33554432000 4194304 4964800 140735582009376 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2562.23
Current children cumulated vsize (KiB) 202748

[startup+702.313 s]
/proc/loadavg: 8.17 7.91 7.69 9/331 25487
/proc/meminfo: memFree=24277048/32951124 swapFree=44485764/67111528
[pid=25456] ppid=25454 vsize=203596 CPUtime=2801.62 cores=1,3,5,7
/proc/25456/stat : 25456 (claspmt) R 25454 25456 24676 0 -1 4202496 23698 0 0 0 280142 20 0 0 25 0 4 0 653440827 208482304 22641 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25456/statm: 50899 22641 460 189 0 45248 0
[pid=25456/tid=25464] ppid=25454 vsize=203596 CPUtime=702.25 cores=1,3,5,7
/proc/25456/task/25464/stat : 25464 (claspmt) R 25454 25456 24676 0 -1 4202560 6182 0 0 0 70221 4 0 0 25 0 4 0 653440831 208482304 22641 33554432000 4194304 4964800 140735582009376 18446744073709551615 4668987 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25456/tid=25465] ppid=25454 vsize=203596 CPUtime=694.76 cores=1,3,5,7
/proc/25456/task/25465/stat : 25465 (claspmt) R 25454 25456 24676 0 -1 4202560 7492 0 0 0 69469 7 0 0 25 0 4 0 653440831 208482304 22641 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25456/tid=25466] ppid=25454 vsize=203596 CPUtime=702.27 cores=1,3,5,7
/proc/25456/task/25466/stat : 25466 (claspmt) R 25454 25456 24676 0 -1 4202560 3502 0 0 0 70225 2 0 0 25 0 4 0 653440831 208482304 22641 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2801.62
Current children cumulated vsize (KiB) 203596

[startup+762.301 s]
/proc/loadavg: 8.16 7.96 7.71 9/331 25489
/proc/meminfo: memFree=24227796/32951124 swapFree=44485764/67111528
[pid=25456] ppid=25454 vsize=205000 CPUtime=3040.94 cores=1,3,5,7
/proc/25456/stat : 25456 (claspmt) R 25454 25456 24676 0 -1 4202496 24167 0 0 0 304073 21 0 0 25 0 4 0 653440827 209920000 23097 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25456/statm: 51250 23097 460 189 0 45599 0
[pid=25456/tid=25464] ppid=25454 vsize=205000 CPUtime=762.24 cores=1,3,5,7
/proc/25456/task/25464/stat : 25464 (claspmt) R 25454 25456 24676 0 -1 4202560 6505 0 0 0 76220 4 0 0 25 0 4 0 653440831 209920000 23097 33554432000 4194304 4964800 140735582009376 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25456/tid=25465] ppid=25454 vsize=205000 CPUtime=754.11 cores=1,3,5,7
/proc/25456/task/25465/stat : 25465 (claspmt) R 25454 25456 24676 0 -1 4202560 7596 0 0 0 75403 8 0 0 25 0 4 0 653440831 209920000 23097 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25456/tid=25466] ppid=25454 vsize=205000 CPUtime=762.26 cores=1,3,5,7
/proc/25456/task/25466/stat : 25466 (claspmt) R 25454 25456 24676 0 -1 4202560 3530 0 0 0 76224 2 0 0 25 0 4 0 653440831 209920000 23097 33554432000 4194304 4964800 140735582009376 18446744073709551615 4669011 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3040.94
Current children cumulated vsize (KiB) 205000

[startup+822.3 s]
/proc/loadavg: 8.10 7.97 7.73 9/331 25490
/proc/meminfo: memFree=24170976/32951124 swapFree=44485764/67111528
[pid=25456] ppid=25454 vsize=205000 CPUtime=3280.3 cores=1,3,5,7
/proc/25456/stat : 25456 (claspmt) R 25454 25456 24676 0 -1 4202496 24783 0 0 0 328008 22 0 0 25 0 4 0 653440827 209920000 23653 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670299 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25456/statm: 51250 23653 460 189 0 45599 0
[pid=25456/tid=25464] ppid=25454 vsize=205000 CPUtime=822.24 cores=1,3,5,7
/proc/25456/task/25464/stat : 25464 (claspmt) R 25454 25456 24676 0 -1 4202560 6680 0 0 0 82220 4 0 0 25 0 4 0 653440831 209920000 23653 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25456/tid=25465] ppid=25454 vsize=205000 CPUtime=813.46 cores=1,3,5,7
/proc/25456/task/25465/stat : 25465 (claspmt) R 25454 25456 24676 0 -1 4202560 7870 0 0 0 81338 8 0 0 25 0 4 0 653440831 209920000 23653 33554432000 4194304 4964800 140735582009376 18446744073709551615 4668770 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25456/tid=25466] ppid=25454 vsize=205000 CPUtime=822.27 cores=1,3,5,7
/proc/25456/task/25466/stat : 25466 (claspmt) R 25454 25456 24676 0 -1 4202560 3577 0 0 0 82224 3 0 0 25 0 4 0 653440831 209920000 23653 33554432000 4194304 4964800 140735582009376 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3280.3
Current children cumulated vsize (KiB) 205000

[startup+882.3 s]
/proc/loadavg: 8.03 7.97 7.75 9/331 25492
/proc/meminfo: memFree=24117008/32951124 swapFree=44485764/67111528
[pid=25456] ppid=25454 vsize=205496 CPUtime=3519.66 cores=1,3,5,7
/proc/25456/stat : 25456 (claspmt) R 25454 25456 24676 0 -1 4202496 25113 0 0 0 351943 23 0 0 25 0 4 0 653440827 210427904 23957 33554432000 4194304 4964800 140735582009376 18446744073709551615 4321079 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25456/statm: 51374 23957 460 189 0 45723 0
[pid=25456/tid=25464] ppid=25454 vsize=205496 CPUtime=882.24 cores=1,3,5,7
/proc/25456/task/25464/stat : 25464 (claspmt) R 25454 25456 24676 0 -1 4202560 6851 0 0 0 88219 5 0 0 25 0 4 0 653440831 210427904 23957 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25456/tid=25465] ppid=25454 vsize=205496 CPUtime=872.81 cores=1,3,5,7
/proc/25456/task/25465/stat : 25465 (claspmt) R 25454 25456 24676 0 -1 4202560 7870 0 0 0 87273 8 0 0 25 0 4 0 653440831 210427904 23957 33554432000 4194304 4964800 140735582009376 18446744073709551615 4668734 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25456/tid=25466] ppid=25454 vsize=205496 CPUtime=882.27 cores=1,3,5,7
/proc/25456/task/25466/stat : 25466 (claspmt) R 25454 25456 24676 0 -1 4202560 3578 0 0 0 88224 3 0 0 25 0 4 0 653440831 210427904 23957 33554432000 4194304 4964800 140735582009376 18446744073709551615 4668709 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3519.66
Current children cumulated vsize (KiB) 205496

[startup+942.301 s]
/proc/loadavg: 8.09 7.99 7.76 9/331 25494
/proc/meminfo: memFree=24077912/32951124 swapFree=44485764/67111528
[pid=25456] ppid=25454 vsize=206056 CPUtime=3759.01 cores=1,3,5,7
/proc/25456/stat : 25456 (claspmt) R 25454 25456 24676 0 -1 4202496 25277 0 0 0 375878 23 0 0 25 0 4 0 653440827 211001344 24121 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670299 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25456/statm: 51514 24121 460 189 0 45863 0
[pid=25456/tid=25464] ppid=25454 vsize=206056 CPUtime=942.24 cores=1,3,5,7
/proc/25456/task/25464/stat : 25464 (claspmt) R 25454 25456 24676 0 -1 4202560 6926 0 0 0 94219 5 0 0 25 0 4 0 653440831 211001344 24121 33554432000 4194304 4964800 140735582009376 18446744073709551615 4660303 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25456/tid=25465] ppid=25454 vsize=206056 CPUtime=932.16 cores=1,3,5,7
/proc/25456/task/25465/stat : 25465 (claspmt) R 25454 25456 24676 0 -1 4202560 7873 0 0 0 93208 8 0 0 25 0 4 0 653440831 211001344 24121 33554432000 4194304 4964800 140735582009376 18446744073709551615 4342144 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25456/tid=25466] ppid=25454 vsize=206056 CPUtime=942.27 cores=1,3,5,7
/proc/25456/task/25466/stat : 25466 (claspmt) R 25454 25456 24676 0 -1 4202560 3578 0 0 0 94224 3 0 0 25 0 4 0 653440831 211001344 24121 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3759.01
Current children cumulated vsize (KiB) 206056

[startup+1002.3 s]
/proc/loadavg: 8.16 8.02 7.79 9/331 25495
/proc/meminfo: memFree=24047264/32951124 swapFree=44485764/67111528
[pid=25456] ppid=25454 vsize=206256 CPUtime=3998.37 cores=1,3,5,7
/proc/25456/stat : 25456 (claspmt) R 25454 25456 24676 0 -1 4202496 25462 0 0 0 399814 23 0 0 25 0 4 0 653440827 211206144 24278 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25456/statm: 51564 24278 460 189 0 45913 0
[pid=25456/tid=25464] ppid=25454 vsize=206256 CPUtime=1002.24 cores=1,3,5,7
/proc/25456/task/25464/stat : 25464 (claspmt) R 25454 25456 24676 0 -1 4202560 7020 0 0 0 100219 5 0 0 25 0 4 0 653440831 211206144 24278 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25456/tid=25465] ppid=25454 vsize=206256 CPUtime=991.52 cores=1,3,5,7
/proc/25456/task/25465/stat : 25465 (claspmt) R 25454 25456 24676 0 -1 4202560 7888 0 0 0 99144 8 0 0 25 0 4 0 653440831 211206144 24278 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25456/tid=25466] ppid=25454 vsize=206256 CPUtime=1002.27 cores=1,3,5,7
/proc/25456/task/25466/stat : 25466 (claspmt) R 25454 25456 24676 0 -1 4202560 3629 0 0 0 100224 3 0 0 25 0 4 0 653440831 211206144 24278 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3998.37
Current children cumulated vsize (KiB) 206256

[startup+1062.3 s]
/proc/loadavg: 8.11 8.03 7.80 9/331 25497
/proc/meminfo: memFree=23984248/32951124 swapFree=44485764/67111528
[pid=25456] ppid=25454 vsize=206900 CPUtime=4237.72 cores=1,3,5,7
/proc/25456/stat : 25456 (claspmt) R 25454 25456 24676 0 -1 4202496 26146 0 0 0 423748 24 0 0 25 0 4 0 653440827 211865600 24948 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25456/statm: 51725 24948 460 189 0 46074 0
[pid=25456/tid=25464] ppid=25454 vsize=206900 CPUtime=1062.24 cores=1,3,5,7
/proc/25456/task/25464/stat : 25464 (claspmt) R 25454 25456 24676 0 -1 4202560 7346 0 0 0 106219 5 0 0 25 0 4 0 653440831 211865600 24948 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670194 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25456/tid=25465] ppid=25454 vsize=206900 CPUtime=1050.87 cores=1,3,5,7
/proc/25456/task/25465/stat : 25465 (claspmt) R 25454 25456 24676 0 -1 4202560 7926 0 0 0 105078 9 0 0 25 0 4 0 653440831 211865600 24948 33554432000 4194304 4964800 140735582009376 18446744073709551615 4669276 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25456/tid=25466] ppid=25454 vsize=206900 CPUtime=1062.27 cores=1,3,5,7
/proc/25456/task/25466/stat : 25466 (claspmt) R 25454 25456 24676 0 -1 4202560 3739 0 0 0 106224 3 0 0 25 0 4 0 653440831 211865600 24948 33554432000 4194304 4964800 140735582009376 18446744073709551615 4321082 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4237.72
Current children cumulated vsize (KiB) 206900

[startup+1122.3 s]
/proc/loadavg: 8.15 8.05 7.82 9/331 25498
/proc/meminfo: memFree=23910440/32951124 swapFree=44485764/67111528
[pid=25456] ppid=25454 vsize=206900 CPUtime=4477.07 cores=1,3,5,7
/proc/25456/stat : 25456 (claspmt) R 25454 25456 24676 0 -1 4202496 26357 0 0 0 447683 24 0 0 25 0 4 0 653440827 211865600 25145 33554432000 4194304 4964800 140735582009376 18446744073709551615 4653426 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25456/statm: 51725 25145 460 189 0 46074 0
[pid=25456/tid=25464] ppid=25454 vsize=206900 CPUtime=1122.24 cores=1,3,5,7
/proc/25456/task/25464/stat : 25464 (claspmt) R 25454 25456 24676 0 -1 4202560 7426 0 0 0 112219 5 0 0 25 0 4 0 653440831 211865600 25145 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25456/tid=25465] ppid=25454 vsize=206900 CPUtime=1110.22 cores=1,3,5,7
/proc/25456/task/25465/stat : 25465 (claspmt) R 25454 25456 24676 0 -1 4202560 7950 0 0 0 111013 9 0 0 25 0 4 0 653440831 211865600 25145 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25456/tid=25466] ppid=25454 vsize=206900 CPUtime=1122.27 cores=1,3,5,7
/proc/25456/task/25466/stat : 25466 (claspmt) R 25454 25456 24676 0 -1 4202560 3754 0 0 0 112224 3 0 0 25 0 4 0 653440831 211865600 25145 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670534 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4477.07
Current children cumulated vsize (KiB) 206900

[startup+1182.3 s]
/proc/loadavg: 8.16 8.07 7.84 9/331 25500
/proc/meminfo: memFree=23852876/32951124 swapFree=44485764/67111528
[pid=25456] ppid=25454 vsize=207112 CPUtime=4716.42 cores=1,3,5,7
/proc/25456/stat : 25456 (claspmt) R 25454 25456 24676 0 -1 4202496 26556 0 0 0 471618 24 0 0 25 0 4 0 653440827 212082688 25330 33554432000 4194304 4964800 140735582009376 18446744073709551615 4669131 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25456/statm: 51778 25330 460 189 0 46127 0
[pid=25456/tid=25464] ppid=25454 vsize=207112 CPUtime=1182.24 cores=1,3,5,7
/proc/25456/task/25464/stat : 25464 (claspmt) R 25454 25456 24676 0 -1 4202560 7576 0 0 0 118219 5 0 0 25 0 4 0 653440831 212082688 25330 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25456/tid=25465] ppid=25454 vsize=207112 CPUtime=1169.57 cores=1,3,5,7
/proc/25456/task/25465/stat : 25465 (claspmt) R 25454 25456 24676 0 -1 4202560 7974 0 0 0 116948 9 0 0 25 0 4 0 653440831 212082688 25330 33554432000 4194304 4964800 140735582009376 18446744073709551615 4361250 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25456/tid=25466] ppid=25454 vsize=207112 CPUtime=1182.27 cores=1,3,5,7
/proc/25456/task/25466/stat : 25466 (claspmt) R 25454 25456 24676 0 -1 4202560 3779 0 0 0 118224 3 0 0 25 0 4 0 653440831 212082688 25330 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670264 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4716.42
Current children cumulated vsize (KiB) 207112



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.3 s]
/proc/loadavg: 8.19 8.08 7.85 9/331 25501
/proc/meminfo: memFree=23843936/32951124 swapFree=44485764/67111528
[pid=25456] ppid=25454 vsize=207320 CPUtime=4800.22 cores=1,3,5,7
/proc/25456/stat : 25456 (claspmt) R 25454 25456 24676 0 -1 4202496 26651 0 0 0 479998 24 0 0 25 0 4 0 653440827 212295680 25425 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670335 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/25456/statm: 51830 25425 460 189 0 46179 0
[pid=25456/tid=25464] ppid=25454 vsize=207320 CPUtime=1203.25 cores=1,3,5,7
/proc/25456/task/25464/stat : 25464 (claspmt) R 25454 25456 24676 0 -1 4202560 7671 0 0 0 120320 5 0 0 25 0 4 0 653440831 212295680 25425 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25456/tid=25465] ppid=25454 vsize=207320 CPUtime=1190.34 cores=1,3,5,7
/proc/25456/task/25465/stat : 25465 (claspmt) R 25454 25456 24676 0 -1 4202560 7974 0 0 0 119025 9 0 0 25 0 4 0 653440831 212295680 25425 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25456/tid=25466] ppid=25454 vsize=207320 CPUtime=1203.28 cores=1,3,5,7
/proc/25456/task/25466/stat : 25466 (claspmt) R 25454 25456 24676 0 -1 4202560 3779 0 0 0 120325 3 0 0 25 0 4 0 653440831 212295680 25425 33554432000 4194304 4964800 140735582009376 18446744073709551615 4670194 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.22
Current children cumulated vsize (KiB) 207320

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child status: 0
Real time (s): 1203.35
CPU time (s): 4800.29
CPU user time (s): 4800.04
CPU system time (s): 0.254961
CPU usage (%): 398.91
Max. virtual memory (cumulated for all children) (KiB): 207320

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4800.04
system time used= 0.254961
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 26664
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3898
involuntary context switches= 21698

runsolver used 3.59145 second user time and 9.36758 second system time

The end

Launcher Data 

Begin job on node105 at 2011-05-15 02:13:24
IDJOB=3373537
IDBENCH=70775
IDSOLVER=1853
FILE ID=node105/3373537-1305418404
RUNJOBID= node105-1305418403-25413
PBS_JOBID= 13324538
Free space on /tmp= 73576 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT09/APPLICATIONS/bitverif/countbitsrotate/countbitsrotate032.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3373537-1305418404/watcher-3373537-1305418404 -o /tmp/evaluation-result-3373537-1305418404/solver-3373537-1305418404 -C 4800 -W 1300 -M 15500  HOME/claspmt -t4 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3373537-1305418404.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 8d76aa747359ea8ccf15726792949452
RANDOM SEED=280364740

node105.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5410.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      25272144 kB
Buffers:        281388 kB
Cached:        5929016 kB
SwapCached:      39656 kB
Active:         579688 kB
Inactive:      6839168 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      25272144 kB
SwapTotal:    67111528 kB
SwapFree:     44485764 kB
Dirty:           14144 kB
Writeback:           0 kB
AnonPages:     1176168 kB
Mapped:          10200 kB
Slab:           128824 kB
PageTables:      66348 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 65318304 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73576 MiB
End job on node105 at 2011-05-15 02:33:29