Trace number 3381380

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 40001.7 5012.22

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m47-q841.5-n45-p30-s84954709.cnf
MD5SUM556c58061360a1b195d9b2f28888af77
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark0.134978
Satisfiable
(Un)Satisfiability was proved
Number of variables2115
Number of clauses356340
Sum of the clauses size714705
Maximum clause length47
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2356295
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 545

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...514412/instance-3381380-1305514412.cnf
2.20/2.26	c Solving...
40000.72/5011.32	
40000.72/5011.32	*** INTERRUPTED! ***
40000.72/5011.32	*** Shutting down threads...
40001.68/5012.22	s UNKNOWN
40001.68/5012.22	
40001.68/5012.22	c INTERRUPTED   : 1
40001.68/5012.22	c Models        : 0+    
40001.68/5012.22	c Time          : 5011.339s (Solving: 5009.08s 1st Model: 0.00s Unsat: 0.00s)
40001.68/5012.22	c CPU Time      : 40000.820s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3381380-1305514412/watcher-3381380-1305514412 -o /tmp/evaluation-result-3381380-1305514412/solver-3381380-1305514412 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3381380-1305514412.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.93 7.82 7.92 2/398 13034
/proc/meminfo: memFree=15771816/32951124 swapFree=67111348/67111528
[pid=13034] ppid=13032 vsize=22892 CPUtime=0 cores=0-7
/proc/13034/stat : 13034 (claspmt) R 13032 13034 13008 0 -1 4202496 1123 0 0 0 0 0 0 0 25 0 1 0 643857317 23441408 947 33554432000 4194304 4964800 140733683555088 18446744073709551615 255106429637 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/13034/statm: 5844 953 386 189 0 718 0

[startup+0.0498581 s]
/proc/loadavg: 6.93 7.82 7.92 2/398 13034
/proc/meminfo: memFree=15771816/32951124 swapFree=67111348/67111528
[pid=13034] ppid=13032 vsize=30412 CPUtime=0.04 cores=0-7
/proc/13034/stat : 13034 (claspmt) R 13032 13034 13008 0 -1 4202496 3883 0 0 0 4 0 0 0 25 0 1 0 643857317 31141888 2647 33554432000 4194304 4964800 140733683555088 18446744073709551615 255106429637 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/13034/statm: 7603 2649 386 189 0 2477 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 30412

[startup+0.100864 s]
/proc/loadavg: 6.93 7.82 7.92 2/398 13034
/proc/meminfo: memFree=15771816/32951124 swapFree=67111348/67111528
[pid=13034] ppid=13032 vsize=40260 CPUtime=0.09 cores=0-7
/proc/13034/stat : 13034 (claspmt) R 13032 13034 13008 0 -1 4202496 6943 0 0 0 9 0 0 0 25 0 1 0 643857317 41226240 5079 33554432000 4194304 4964800 140733683555088 18446744073709551615 255106429637 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/13034/statm: 10065 5080 397 189 0 4939 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 40260

[startup+0.300817 s]
/proc/loadavg: 6.93 7.82 7.92 2/398 13034
/proc/meminfo: memFree=15771816/32951124 swapFree=67111348/67111528
[pid=13034] ppid=13032 vsize=44316 CPUtime=0.29 cores=0-7
/proc/13034/stat : 13034 (claspmt) R 13032 13034 13008 0 -1 4202496 7841 0 0 0 29 0 0 0 25 0 1 0 643857317 45379584 5977 33554432000 4194304 4964800 140733683555088 18446744073709551615 4607562 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/13034/statm: 11079 5977 399 189 0 5953 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 44316

[startup+0.700745 s]
/proc/loadavg: 6.93 7.82 7.92 2/398 13034
/proc/meminfo: memFree=15771816/32951124 swapFree=67111348/67111528
[pid=13034] ppid=13032 vsize=44316 CPUtime=0.69 cores=0-7
/proc/13034/stat : 13034 (claspmt) R 13032 13034 13008 0 -1 4202496 7841 0 0 0 69 0 0 0 25 0 1 0 643857317 45379584 5977 33554432000 4194304 4964800 140733683555088 18446744073709551615 4607590 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/13034/statm: 11079 5977 399 189 0 5953 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 44316

[startup+1.50061 s]
/proc/loadavg: 6.93 7.82 7.92 2/399 13035
/proc/meminfo: memFree=15749116/32951124 swapFree=67111348/67111528
[pid=13034] ppid=13032 vsize=44580 CPUtime=1.48 cores=0-7
/proc/13034/stat : 13034 (claspmt) R 13032 13034 13008 0 -1 4202496 7923 0 0 0 148 0 0 0 25 0 1 0 643857317 45649920 6059 33554432000 4194304 4964800 140733683555088 18446744073709551615 4600434 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/13034/statm: 11145 6059 400 189 0 6019 0
Current children cumulated CPU time (s) 1.48
Current children cumulated vsize (KiB) 44580

[startup+3.10029 s]
/proc/loadavg: 6.46 7.70 7.89 2/399 13035
/proc/meminfo: memFree=15746140/32951124 swapFree=67111348/67111528
[pid=13034] ppid=13032 vsize=402888 CPUtime=7.28 cores=0-7
/proc/13034/stat : 13034 (claspmt) R 13032 13034 13008 0 -1 4202496 19623 0 0 0 720 8 0 0 18 0 8 0 643857317 412557312 17044 33554432000 4194304 4964800 140733683555088 18446744073709551615 4332478 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/13034/statm: 100722 17044 464 189 0 95071 0
Current children cumulated CPU time (s) 7.28
Current children cumulated vsize (KiB) 402888

[startup+6.30073 s]
/proc/loadavg: 6.46 7.70 7.89 9/406 13042
/proc/meminfo: memFree=15616280/32951124 swapFree=67111348/67111528
[pid=13034] ppid=13032 vsize=543348 CPUtime=32.79 cores=0-7
/proc/13034/stat : 13034 (claspmt) R 13032 13034 13008 0 -1 4202496 48106 0 0 0 3256 23 0 0 25 0 8 0 643857317 556388352 45527 33554432000 4194304 4964800 140733683555088 18446744073709551615 4320981 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/13034/statm: 135837 45529 464 189 0 130186 0
[pid=13034/tid=13036] ppid=13032 vsize=543348 CPUtime=3.75 cores=0-7
/proc/13034/task/13036/stat : 13036 (claspmt) R 13032 13034 13008 0 -1 4202560 4144 0 0 0 373 2 0 0 25 0 8 0 643857544 556388352 45529 33554432000 4194304 4964800 140733683555088 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=13034/tid=13037] ppid=13032 vsize=543348 CPUtime=3.71 cores=0-7
/proc/13034/task/13037/stat : 13037 (claspmt) R 13032 13034 13008 0 -1 4202560 2889 0 0 0 370 1 0 0 25 0 8 0 643857544 556388352 45529 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=13034/tid=13038] ppid=13032 vsize=543348 CPUtime=3.81 cores=0-7
/proc/13034/task/13038/stat : 13038 (claspmt) R 13032 13034 13008 0 -1 4202560 4290 0 0 0 378 3 0 0 21 0 8 0 643857544 556388352 45530 33554432000 4194304 4964800 140733683555088 18446744073709551615 4627259 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=13034/tid=13039] ppid=13032 vsize=543348 CPUtime=3.79 cores=0-7
/proc/13034/task/13039/stat : 13039 (claspmt) R 13032 13034 13008 0 -1 4202560 5439 0 0 0 376 3 0 0 25 0 8 0 643857544 556388352 45530 33554432000 4194304 4964800 140733683555088 18446744073709551615 4627259 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=13034/tid=13040] ppid=13032 vsize=543348 CPUtime=4.01 cores=0-7
/proc/13034/task/13040/stat : 13040 (claspmt) R 13032 13034 13008 0 -1 4202560 5589 0 0 0 399 2 0 0 25 0 8 0 643857544 556388352 45530 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=13034/tid=13041] ppid=13032 vsize=543348 CPUtime=3.79 cores=0-7
/proc/13034/task/13041/stat : 13041 (claspmt) R 13032 13034 13008 0 -1 4202560 5255 0 0 0 376 3 0 0 21 0 8 0 643857544 556388352 45530 33554432000 4194304 4964800 140733683555088 18446744073709551615 4342848 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=13034/tid=13042] ppid=13032 vsize=543348 CPUtime=3.8 cores=0-7
/proc/13034/task/13042/stat : 13042 (claspmt) R 13032 13034 13008 0 -1 4202560 6854 0 0 0 378 2 0 0 20 0 8 0 643857545 556388352 45530 33554432000 4194304 4964800 140733683555088 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 32.79
Current children cumulated vsize (KiB) 543348

[startup+12.7006 s]
/proc/loadavg: 6.69 7.71 7.89 9/406 13042
/proc/meminfo: memFree=15426860/32951124 swapFree=67111348/67111528
[pid=13034] ppid=13032 vsize=561988 CPUtime=83.82 cores=0-7
/proc/13034/stat : 13034 (claspmt) R 13032 13034 13008 0 -1 4202496 92553 0 0 0 8333 49 0 0 25 0 8 0 643857317 575475712 89974 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670335 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/13034/statm: 140497 89974 466 189 0 134846 0
[pid=13034/tid=13036] ppid=13032 vsize=561988 CPUtime=10.09 cores=0-7
/proc/13034/task/13036/stat : 13036 (claspmt) R 13032 13034 13008 0 -1 4202560 10739 0 0 0 1003 6 0 0 25 0 8 0 643857544 575475712 89974 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=13034/tid=13037] ppid=13032 vsize=561988 CPUtime=10.11 cores=0-7
/proc/13034/task/13037/stat : 13037 (claspmt) R 13032 13034 13008 0 -1 4202560 4649 0 0 0 1009 2 0 0 25 0 8 0 643857544 575475712 89975 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=13034/tid=13038] ppid=13032 vsize=561988 CPUtime=10.18 cores=0-7
/proc/13034/task/13038/stat : 13038 (claspmt) R 13032 13034 13008 0 -1 4202560 8375 0 0 0 1013 5 0 0 25 0 8 0 643857544 575475712 89975 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=13034/tid=13039] ppid=13032 vsize=561988 CPUtime=10.18 cores=0-7
/proc/13034/task/13039/stat : 13039 (claspmt) R 13032 13034 13008 0 -1 4202560 9207 0 0 0 1014 4 0 0 25 0 8 0 643857544 575475712 89975 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=13034/tid=13040] ppid=13032 vsize=561988 CPUtime=10.36 cores=0-7
/proc/13034/task/13040/stat : 13040 (claspmt) R 13032 13034 13008 0 -1 4202560 11759 0 0 0 1030 6 0 0 25 0 8 0 643857544 575475712 89975 33554432000 4194304 4964800 140733683555088 18446744073709551615 4320865 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=13034/tid=13041] ppid=13032 vsize=561988 CPUtime=10.17 cores=0-7
/proc/13034/task/13041/stat : 13041 (claspmt) R 13032 13034 13008 0 -1 4202560 11160 0 0 0 1011 6 0 0 25 0 8 0 643857544 575475712 89976 33554432000 4194304 4964800 140733683555088 18446744073709551615 4325409 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=13034/tid=13042] ppid=13032 vsize=561988 CPUtime=10.19 cores=0-7
/proc/13034/task/13042/stat : 13042 (claspmt) R 13032 13034 13008 0 -1 4202560 15899 0 0 0 1013 6 0 0 25 0 8 0 643857545 575475712 89976 33554432000 4194304 4964800 140733683555088 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 83.82
Current children cumulated vsize (KiB) 561988

[startup+25.5004 s]
/proc/loadavg: 6.89 7.72 7.89 9/406 13042
/proc/meminfo: memFree=15222896/32951124 swapFree=67111348/67111528
[pid=13034] ppid=13032 vsize=918380 CPUtime=186.01 cores=0-7
/proc/13034/stat : 13034 (claspmt) R 13032 13034 13008 0 -1 4202496 141452 0 0 0 18523 78 0 0 25 0 8 0 643857317 940421120 138873 33554432000 4194304 4964800 140733683555088 18446744073709551615 4320951 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/13034/statm: 229595 138873 466 189 0 223944 0
[pid=13034/tid=13036] ppid=13032 vsize=918380 CPUtime=22.89 cores=0-7
/proc/13034/task/13036/stat : 13036 (claspmt) R 13032 13034 13008 0 -1 4202560 19105 0 0 0 2278 11 0 0 25 0 8 0 643857544 940421120 138873 33554432000 4194304 4964800 140733683555088 18446744073709551615 4669131 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=13034/tid=13037] ppid=13032 vsize=918380 CPUtime=22.76 cores=0-7
/proc/13034/task/13037/stat : 13037 (claspmt) R 13032 13034 13008 0 -1 4202560 6477 0 0 0 2273 3 0 0 25 0 8 0 643857544 940421120 138873 33554432000 4194304 4964800 140733683555088 18446744073709551615 4320966 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=13034/tid=13038] ppid=13032 vsize=918380 CPUtime=22.98 cores=0-7
/proc/13034/task/13038/stat : 13038 (claspmt) R 13032 13034 13008 0 -1 4202560 12640 0 0 0 2291 7 0 0 25 0 8 0 643857544 940421120 138873 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670295 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=13034/tid=13039] ppid=13032 vsize=918380 CPUtime=22.98 cores=0-7
/proc/13034/task/13039/stat : 13039 (claspmt) R 13032 13034 13008 0 -1 4202560 11812 0 0 0 2293 5 0 0 25 0 8 0 643857544 940421120 138873 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670499 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=13034/tid=13040] ppid=13032 vsize=918380 CPUtime=23.13 cores=0-7
/proc/13034/task/13040/stat : 13040 (claspmt) R 13032 13034 13008 0 -1 4202560 21354 0 0 0 2301 12 0 0 25 0 8 0 643857544 940421120 138874 33554432000 4194304 4964800 140733683555088 18446744073709551615 255106429637 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=13034/tid=13041] ppid=13032 vsize=918380 CPUtime=22.97 cores=0-7
/proc/13034/task/13041/stat : 13041 (claspmt) R 13032 13034 13008 0 -1 4202560 14919 0 0 0 2289 8 0 0 25 0 8 0 643857544 940421120 138874 33554432000 4194304 4964800 140733683555088 18446744073709551615 4342939 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=13034/tid=13042] ppid=13032 vsize=918380 CPUtime=22.99 cores=0-7
/proc/13034/task/13042/stat : 13042 (claspmt) R 13032 13034 13008 0 -1 4202560 26540 0 0 0 2284 15 0 0 25 0 8 0 643857545 940421120 138874 33554432000 4194304 4964800 140733683555088 18446744073709551615 4320944 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 186.01
Current children cumulated vsize (KiB) 918380

[startup+51.1009 s]
/proc/loadavg: 7.34 7.76 7.90 9/406 13043
/proc/meminfo: memFree=15015024/32951124 swapFree=67111348/67111528
[pid=13034] ppid=13032 vsize=1068420 CPUtime=390.41 cores=0-7
/proc/13034/stat : 13034 (claspmt) R 13032 13034 13008 0 -1 4202496 192130 0 0 0 38927 114 0 0 25 0 8 0 643857317 1094062080 189546 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670299 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/13034/statm: 267105 189546 466 189 0 261454 0
[pid=13034/tid=13036] ppid=13032 vsize=1068420 CPUtime=48.47 cores=0-7
/proc/13034/task/13036/stat : 13036 (claspmt) R 13032 13034 13008 0 -1 4202560 27057 0 0 0 4832 15 0 0 25 0 8 0 643857544 1094062080 189546 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670194 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=13034/tid=13037] ppid=13032 vsize=1068420 CPUtime=48.01 cores=0-7
/proc/13034/task/13037/stat : 13037 (claspmt) R 13032 13034 13008 0 -1 4202560 7944 0 0 0 4797 4 0 0 25 0 8 0 643857544 1094062080 189546 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=13034/tid=13038] ppid=13032 vsize=1068420 CPUtime=48.58 cores=0-7
/proc/13034/task/13038/stat : 13038 (claspmt) R 13032 13034 13008 0 -1 4202560 17749 0 0 0 4847 11 0 0 25 0 8 0 643857544 1094062080 189546 33554432000 4194304 4964800 140733683555088 18446744073709551615 4653558 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=13034/tid=13039] ppid=13032 vsize=1068420 CPUtime=48.57 cores=0-7
/proc/13034/task/13039/stat : 13039 (claspmt) R 13032 13034 13008 0 -1 4202560 16326 0 0 0 4848 9 0 0 25 0 8 0 643857544 1094062080 189546 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=13034/tid=13040] ppid=13032 vsize=1068420 CPUtime=48.72 cores=0-7
/proc/13034/task/13040/stat : 13040 (claspmt) R 13032 13034 13008 0 -1 4202560 24880 0 0 0 4857 15 0 0 25 0 8 0 643857544 1094062080 189546 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=13034/tid=13041] ppid=13032 vsize=1068420 CPUtime=48.55 cores=0-7
/proc/13034/task/13041/stat : 13041 (claspmt) R 13032 13034 13008 0 -1 4202560 20387 0 0 0 4843 12 0 0 25 0 8 0 643857544 1094062080 189546 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=13034/tid=13042] ppid=13032 vsize=1068420 CPUtime=48.59 cores=0-7
/proc/13034/task/13042/stat : 13042 (claspmt) R 13032 13034 13008 0 -1 4202560 37725 0 0 0 4835 24 0 0 25 0 8 0 643857545 1094062080 189546 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 390.41
Current children cumulated vsize (KiB) 1068420

[startup+102.316 s]
/proc/loadavg: 7.81 7.81 7.91 9/406 13044
/proc/meminfo: memFree=14804712/32951124 swapFree=67111348/67111528
[pid=13034] ppid=13032 vsize=1211168 CPUtime=799.39 cores=0-7
/proc/13034/stat : 13034 (claspmt) R 13032 13034 13008 0 -1 4202496 244720 0 0 0 79783 156 0 0 25 0 8 0 643857317 1240236032 242067 33554432000 4194304 4964800 140733683555088 18446744073709551615 255106425834 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/13034/statm: 302792 242067 466 189 0 297141 0
[pid=13034/tid=13036] ppid=13032 vsize=1211168 CPUtime=99.67 cores=0-7
/proc/13034/task/13036/stat : 13036 (claspmt) R 13032 13034 13008 0 -1 4202560 37708 0 0 0 9942 25 0 0 25 0 8 0 643857544 1240236032 242067 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=13034/tid=13037] ppid=13032 vsize=1211168 CPUtime=98.53 cores=0-7
/proc/13034/task/13037/stat : 13037 (claspmt) R 13032 13034 13008 0 -1 4202560 9100 0 0 0 9848 5 0 0 25 0 8 0 643857544 1240236032 242067 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=13034/tid=13038] ppid=13032 vsize=1211168 CPUtime=99.79 cores=0-7
/proc/13034/task/13038/stat : 13038 (claspmt) R 13032 13034 13008 0 -1 4202560 25095 0 0 0 9962 17 0 0 25 0 8 0 643857544 1240236032 242067 33554432000 4194304 4964800 140733683555088 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=13034/tid=13039] ppid=13032 vsize=1211168 CPUtime=99.79 cores=0-7
/proc/13034/task/13039/stat : 13039 (claspmt) R 13032 13034 13008 0 -1 4202560 16877 0 0 0 9969 10 0 0 25 0 8 0 643857544 1240236032 242067 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=13034/tid=13040] ppid=13032 vsize=1211168 CPUtime=99.95 cores=0-7
/proc/13034/task/13040/stat : 13040 (claspmt) R 13032 13034 13008 0 -1 4202560 26302 0 0 0 9978 17 0 0 25 0 8 0 643857544 1240236032 242067 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=13034/tid=13041] ppid=13032 vsize=1211168 CPUtime=99.76 cores=0-7
/proc/13034/task/13041/stat : 13041 (claspmt) R 13032 13034 13008 0 -1 4202560 24969 0 0 0 9960 16 0 0 25 0 8 0 643857544 1240236032 242067 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=13034/tid=13042] ppid=13032 vsize=1211168 CPUtime=99.79 cores=0-7
/proc/13034/task/13042/stat : 13042 (claspmt) R 13032 13034 13008 0 -1 4202560 51498 0 0 0 9944 35 0 0 25 0 8 0 643857545 1240236032 242067 33554432000 4194304 4964800 140733683555088 18446744073709551615 4627259 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 799.39
Current children cumulated vsize (KiB) 1211168

[startup+162.301 s]
/proc/loadavg: 8.00 7.86 7.91 9/406 13046
/proc/meminfo: memFree=14698184/32951124 swapFree=67111348/67111528

################
# More data... #
################

/proc/13034/task/13041/stat : 13041 (claspmt) R 13032 13034 13008 0 -1 4202560 145686 0 0 0 465771 164 0 0 25 0 8 0 643857544 4926001152 1123539 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=13034/tid=13042] ppid=13032 vsize=4810548 CPUtime=4635.83 cores=0-7
/proc/13034/task/13042/stat : 13042 (claspmt) R 13032 13034 13008 0 -1 4202560 261308 0 0 0 463272 311 0 0 25 0 8 0 643857545 4926001152 1123539 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 37213.5
Current children cumulated vsize (KiB) 4810548

[startup+4722.32 s]
/proc/loadavg: 8.07 8.07 8.00 9/406 13189
/proc/meminfo: memFree=11240788/32951124 swapFree=67111348/67111528
[pid=13034] ppid=13032 vsize=4810548 CPUtime=37692.7 cores=0-7
/proc/13034/stat : 13034 (claspmt) R 13032 13034 13008 0 -1 4202496 1142774 0 0 0 3768042 1230 0 0 25 0 8 0 643857317 4926001152 1131022 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/13034/statm: 1202637 1131022 466 189 0 1196986 0
[pid=13034/tid=13036] ppid=13032 vsize=4810548 CPUtime=4712.31 cores=0-7
/proc/13034/task/13036/stat : 13036 (claspmt) R 13032 13034 13008 0 -1 4202560 131782 0 0 0 471105 126 0 0 25 0 8 0 643857544 4926001152 1131022 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=13034/tid=13037] ppid=13032 vsize=4810548 CPUtime=4717.83 cores=0-7
/proc/13034/task/13037/stat : 13037 (claspmt) R 13032 13034 13008 0 -1 4202560 57159 0 0 0 471747 36 0 0 25 0 8 0 643857544 4926001152 1131022 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=13034/tid=13038] ppid=13032 vsize=4810548 CPUtime=4719.52 cores=0-7
/proc/13034/task/13038/stat : 13038 (claspmt) R 13032 13034 13008 0 -1 4202560 130425 0 0 0 471800 152 0 0 25 0 8 0 643857544 4926001152 1131022 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=13034/tid=13039] ppid=13032 vsize=4810548 CPUtime=4692.4 cores=0-7
/proc/13034/task/13039/stat : 13039 (claspmt) R 13032 13034 13008 0 -1 4202560 72095 0 0 0 469148 92 0 0 25 0 8 0 643857544 4926001152 1131022 33554432000 4194304 4964800 140733683555088 18446744073709551615 4675467 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=13034/tid=13040] ppid=13032 vsize=4810548 CPUtime=4714.25 cores=0-7
/proc/13034/task/13040/stat : 13040 (claspmt) R 13032 13034 13008 0 -1 4202560 104715 0 0 0 471298 127 0 0 25 0 8 0 643857544 4926001152 1131022 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=13034/tid=13041] ppid=13032 vsize=4810548 CPUtime=4719.35 cores=0-7
/proc/13034/task/13041/stat : 13041 (claspmt) R 13032 13034 13008 0 -1 4202560 145721 0 0 0 471770 165 0 0 25 0 8 0 643857544 4926001152 1131022 33554432000 4194304 4964800 140733683555088 18446744073709551615 4658520 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=13034/tid=13042] ppid=13032 vsize=4810548 CPUtime=4695.84 cores=0-7
/proc/13034/task/13042/stat : 13042 (claspmt) R 13032 13034 13008 0 -1 4202560 265203 0 0 0 469267 317 0 0 25 0 8 0 643857545 4926001152 1131022 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670310 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 37692.7
Current children cumulated vsize (KiB) 4810548

[startup+4782.3 s]
/proc/loadavg: 8.11 8.08 8.01 9/406 13190
/proc/meminfo: memFree=11237928/32951124 swapFree=67111348/67111528
[pid=13034] ppid=13032 vsize=4810548 CPUtime=38171.8 cores=0-7
/proc/13034/stat : 13034 (claspmt) R 13032 13034 13008 0 -1 4202496 1143638 0 0 0 3815948 1231 0 0 25 0 8 0 643857317 4926001152 1131886 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670499 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/13034/statm: 1202637 1131886 466 189 0 1196986 0
[pid=13034/tid=13036] ppid=13032 vsize=4810548 CPUtime=4771.5 cores=0-7
/proc/13034/task/13036/stat : 13036 (claspmt) R 13032 13034 13008 0 -1 4202560 132187 0 0 0 477023 127 0 0 25 0 8 0 643857544 4926001152 1131886 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=13034/tid=13037] ppid=13032 vsize=4810548 CPUtime=4777.83 cores=0-7
/proc/13034/task/13037/stat : 13037 (claspmt) R 13032 13034 13008 0 -1 4202560 57595 0 0 0 477746 37 0 0 25 0 8 0 643857544 4926001152 1131886 33554432000 4194304 4964800 140733683555088 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=13034/tid=13038] ppid=13032 vsize=4810548 CPUtime=4779.5 cores=0-7
/proc/13034/task/13038/stat : 13038 (claspmt) R 13032 13034 13008 0 -1 4202560 130441 0 0 0 477798 152 0 0 25 0 8 0 643857544 4926001152 1131886 33554432000 4194304 4964800 140733683555088 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=13034/tid=13039] ppid=13032 vsize=4810548 CPUtime=4752.39 cores=0-7
/proc/13034/task/13039/stat : 13039 (claspmt) R 13032 13034 13008 0 -1 4202560 72095 0 0 0 475147 92 0 0 25 0 8 0 643857544 4926001152 1131886 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=13034/tid=13040] ppid=13032 vsize=4810548 CPUtime=4774.23 cores=0-7
/proc/13034/task/13040/stat : 13040 (claspmt) R 13032 13034 13008 0 -1 4202560 104722 0 0 0 477296 127 0 0 25 0 8 0 643857544 4926001152 1131886 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=13034/tid=13041] ppid=13032 vsize=4810548 CPUtime=4779.34 cores=0-7
/proc/13034/task/13041/stat : 13041 (claspmt) R 13032 13034 13008 0 -1 4202560 145721 0 0 0 477769 165 0 0 25 0 8 0 643857544 4926001152 1131886 33554432000 4194304 4964800 140733683555088 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=13034/tid=13042] ppid=13032 vsize=4810548 CPUtime=4755.82 cores=0-7
/proc/13034/task/13042/stat : 13042 (claspmt) R 13032 13034 13008 0 -1 4202560 265203 0 0 0 475265 317 0 0 25 0 8 0 643857545 4926001152 1131886 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38171.8
Current children cumulated vsize (KiB) 4810548

[startup+4842.3 s]
/proc/loadavg: 8.18 8.11 8.02 9/406 13192
/proc/meminfo: memFree=11235440/32951124 swapFree=67111348/67111528
[pid=13034] ppid=13032 vsize=4810548 CPUtime=38651 cores=0-7
/proc/13034/stat : 13034 (claspmt) R 13032 13034 13008 0 -1 4202496 1144249 0 0 0 3863865 1231 0 0 25 0 8 0 643857317 4926001152 1132497 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/13034/statm: 1202637 1132497 466 189 0 1196986 0
[pid=13034/tid=13036] ppid=13032 vsize=4810548 CPUtime=4830.67 cores=0-7
/proc/13034/task/13036/stat : 13036 (claspmt) R 13032 13034 13008 0 -1 4202560 132277 0 0 0 482940 127 0 0 25 0 8 0 643857544 4926001152 1132497 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=13034/tid=13037] ppid=13032 vsize=4810548 CPUtime=4837.83 cores=0-7
/proc/13034/task/13037/stat : 13037 (claspmt) R 13032 13034 13008 0 -1 4202560 57700 0 0 0 483746 37 0 0 25 0 8 0 643857544 4926001152 1132497 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=13034/tid=13038] ppid=13032 vsize=4810548 CPUtime=4839.51 cores=0-7
/proc/13034/task/13038/stat : 13038 (claspmt) R 13032 13034 13008 0 -1 4202560 130507 0 0 0 483799 152 0 0 25 0 8 0 643857544 4926001152 1132497 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=13034/tid=13039] ppid=13032 vsize=4810548 CPUtime=4812.39 cores=0-7
/proc/13034/task/13039/stat : 13039 (claspmt) R 13032 13034 13008 0 -1 4202560 72095 0 0 0 481147 92 0 0 25 0 8 0 643857544 4926001152 1132497 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=13034/tid=13040] ppid=13032 vsize=4810548 CPUtime=4834.24 cores=0-7
/proc/13034/task/13040/stat : 13040 (claspmt) R 13032 13034 13008 0 -1 4202560 104722 0 0 0 483297 127 0 0 25 0 8 0 643857544 4926001152 1132497 33554432000 4194304 4964800 140733683555088 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=13034/tid=13041] ppid=13032 vsize=4810548 CPUtime=4839.34 cores=0-7
/proc/13034/task/13041/stat : 13041 (claspmt) R 13032 13034 13008 0 -1 4202560 146066 0 0 0 483769 165 0 0 25 0 8 0 643857544 4926001152 1132497 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=13034/tid=13042] ppid=13032 vsize=4810548 CPUtime=4815.82 cores=0-7
/proc/13034/task/13042/stat : 13042 (claspmt) R 13032 13034 13008 0 -1 4202560 265208 0 0 0 481265 317 0 0 25 0 8 0 643857545 4926001152 1132497 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670273 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38651
Current children cumulated vsize (KiB) 4810548

[startup+4902.3 s]
/proc/loadavg: 8.06 8.08 8.01 9/406 13193
/proc/meminfo: memFree=11227620/32951124 swapFree=67111348/67111528
[pid=13034] ppid=13032 vsize=4814244 CPUtime=39130.1 cores=0-7
/proc/13034/stat : 13034 (claspmt) R 13032 13034 13008 0 -1 4202496 1146258 0 0 0 3911782 1232 0 0 25 0 8 0 643857317 4929785856 1134506 33554432000 4194304 4964800 140733683555088 18446744073709551615 4321079 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/13034/statm: 1203561 1134506 466 189 0 1197910 0
[pid=13034/tid=13036] ppid=13032 vsize=4814244 CPUtime=4889.86 cores=0-7
/proc/13034/task/13036/stat : 13036 (claspmt) R 13032 13034 13008 0 -1 4202560 132371 0 0 0 488859 127 0 0 25 0 8 0 643857544 4929785856 1134506 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=13034/tid=13037] ppid=13032 vsize=4814244 CPUtime=4897.83 cores=0-7
/proc/13034/task/13037/stat : 13037 (claspmt) R 13032 13034 13008 0 -1 4202560 58660 0 0 0 489746 37 0 0 25 0 8 0 643857544 4929785856 1134506 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=13034/tid=13038] ppid=13032 vsize=4814244 CPUtime=4899.5 cores=0-7
/proc/13034/task/13038/stat : 13038 (claspmt) R 13032 13034 13008 0 -1 4202560 130507 0 0 0 489798 152 0 0 25 0 8 0 643857544 4929785856 1134506 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=13034/tid=13039] ppid=13032 vsize=4814244 CPUtime=4872.39 cores=0-7
/proc/13034/task/13039/stat : 13039 (claspmt) R 13032 13034 13008 0 -1 4202560 72095 0 0 0 487147 92 0 0 25 0 8 0 643857544 4929785856 1134506 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670534 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=13034/tid=13040] ppid=13032 vsize=4814244 CPUtime=4894.24 cores=0-7
/proc/13034/task/13040/stat : 13040 (claspmt) R 13032 13034 13008 0 -1 4202560 104722 0 0 0 489297 127 0 0 25 0 8 0 643857544 4929785856 1134506 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=13034/tid=13041] ppid=13032 vsize=4814244 CPUtime=4899.34 cores=0-7
/proc/13034/task/13041/stat : 13041 (claspmt) R 13032 13034 13008 0 -1 4202560 147021 0 0 0 489769 165 0 0 25 0 8 0 643857544 4929785856 1134506 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670340 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=13034/tid=13042] ppid=13032 vsize=4814244 CPUtime=4875.82 cores=0-7
/proc/13034/task/13042/stat : 13042 (claspmt) R 13032 13034 13008 0 -1 4202560 265208 0 0 0 487265 317 0 0 25 0 8 0 643857545 4929785856 1134506 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670496 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39130.1
Current children cumulated vsize (KiB) 4814244

[startup+4962.3 s]
/proc/loadavg: 8.06 8.08 8.01 9/406 13195
/proc/meminfo: memFree=11097156/32951124 swapFree=67111348/67111528
[pid=13034] ppid=13032 vsize=4941484 CPUtime=39609.3 cores=0-7
/proc/13034/stat : 13034 (claspmt) R 13032 13034 13008 0 -1 4202496 1178949 0 0 0 3959667 1264 0 0 25 0 8 0 643857317 5060079616 1167197 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670499 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/13034/statm: 1235371 1167197 466 189 0 1229720 0
[pid=13034/tid=13036] ppid=13032 vsize=4941484 CPUtime=4949.04 cores=0-7
/proc/13034/task/13036/stat : 13036 (claspmt) R 13032 13034 13008 0 -1 4202560 136786 0 0 0 494772 132 0 0 25 0 8 0 643857544 5060079616 1167197 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=13034/tid=13037] ppid=13032 vsize=4941484 CPUtime=4957.83 cores=0-7
/proc/13034/task/13037/stat : 13037 (claspmt) R 13032 13034 13008 0 -1 4202560 59772 0 0 0 495746 37 0 0 25 0 8 0 643857544 5060079616 1167197 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=13034/tid=13038] ppid=13032 vsize=4941484 CPUtime=4959.5 cores=0-7
/proc/13034/task/13038/stat : 13038 (claspmt) R 13032 13034 13008 0 -1 4202560 130517 0 0 0 495798 152 0 0 25 0 8 0 643857544 5060079616 1167197 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=13034/tid=13039] ppid=13032 vsize=4941484 CPUtime=4932.39 cores=0-7
/proc/13034/task/13039/stat : 13039 (claspmt) R 13032 13034 13008 0 -1 4202560 76433 0 0 0 493139 100 0 0 25 0 8 0 643857544 5060079616 1167197 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=13034/tid=13040] ppid=13032 vsize=4941484 CPUtime=4954.24 cores=0-7
/proc/13034/task/13040/stat : 13040 (claspmt) R 13032 13034 13008 0 -1 4202560 106199 0 0 0 495295 129 0 0 25 0 8 0 643857544 5060079616 1167197 33554432000 4194304 4964800 140733683555088 18446744073709551615 4321029 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=13034/tid=13041] ppid=13032 vsize=4941484 CPUtime=4959.34 cores=0-7
/proc/13034/task/13041/stat : 13041 (claspmt) R 13032 13034 13008 0 -1 4202560 152394 0 0 0 495763 171 0 0 25 0 8 0 643857544 5060079616 1167197 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=13034/tid=13042] ppid=13032 vsize=4941484 CPUtime=4935.81 cores=0-7
/proc/13034/task/13042/stat : 13042 (claspmt) R 13032 13034 13008 0 -1 4202560 281174 0 0 0 493253 328 0 0 25 0 8 0 643857545 5060079616 1167197 33554432000 4194304 4964800 140733683555088 18446744073709551615 255106429637 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39609.3
Current children cumulated vsize (KiB) 4941484



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5011.3 s]
/proc/loadavg: 8.02 8.07 8.01 9/406 13196
/proc/meminfo: memFree=10988904/32951124 swapFree=67111348/67111528
[pid=13034] ppid=13032 vsize=5052876 CPUtime=40000.7 cores=0-7
/proc/13034/stat : 13034 (claspmt) R 13032 13034 13008 0 -1 4202496 1205787 0 0 0 3998776 1296 0 0 25 0 8 0 643857317 5174145024 1194035 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/13034/statm: 1263219 1194035 466 189 0 1257568 0
[pid=13034/tid=13036] ppid=13032 vsize=5052876 CPUtime=4997.38 cores=0-7
/proc/13034/task/13036/stat : 13036 (claspmt) R 13032 13034 13008 0 -1 4202560 139377 0 0 0 499601 137 0 0 25 0 8 0 643857544 5174145024 1194035 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=13034/tid=13037] ppid=13032 vsize=5052876 CPUtime=5006.84 cores=0-7
/proc/13034/task/13037/stat : 13037 (claspmt) R 13032 13034 13008 0 -1 4202560 59820 0 0 0 500647 37 0 0 25 0 8 0 643857544 5174145024 1194035 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=13034/tid=13038] ppid=13032 vsize=5052876 CPUtime=5008.52 cores=0-7
/proc/13034/task/13038/stat : 13038 (claspmt) R 13032 13034 13008 0 -1 4202560 130517 0 0 0 500700 152 0 0 25 0 8 0 643857544 5174145024 1194035 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670310 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=13034/tid=13039] ppid=13032 vsize=5052876 CPUtime=4981.41 cores=0-7
/proc/13034/task/13039/stat : 13039 (claspmt) R 13032 13034 13008 0 -1 4202560 82122 0 0 0 498031 110 0 0 25 0 8 0 643857544 5174145024 1194035 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=13034/tid=13040] ppid=13032 vsize=5052876 CPUtime=5003.26 cores=0-7
/proc/13034/task/13040/stat : 13040 (claspmt) R 13032 13034 13008 0 -1 4202560 109863 0 0 0 500192 134 0 0 25 0 8 0 643857544 5174145024 1194035 33554432000 4194304 4964800 140733683555088 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=13034/tid=13041] ppid=13032 vsize=5052876 CPUtime=5008.35 cores=0-7
/proc/13034/task/13041/stat : 13041 (claspmt) R 13032 13034 13008 0 -1 4202560 154999 0 0 0 500659 176 0 0 25 0 8 0 643857544 5174145024 1194035 33554432000 4194304 4964800 140733683555088 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=13034/tid=13042] ppid=13032 vsize=5052876 CPUtime=4984.82 cores=0-7
/proc/13034/task/13042/stat : 13042 (claspmt) R 13032 13034 13008 0 -1 4202560 293171 0 0 0 498147 335 0 0 25 0 8 0 643857545 5174145024 1194035 33554432000 4194304 4964800 140733683555088 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 40000.7
Current children cumulated vsize (KiB) 5052876

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5011.8 s]
/proc/loadavg: 8.02 8.07 8.01 2/400 13197
/proc/meminfo: memFree=10988952/32951124 swapFree=67111348/67111528
[pid=13034] ppid=13032 vsize=5015300 CPUtime=40001.3 cores=0-7
/proc/13034/stat : 13034 (claspmt) R 13032 13034 13008 0 -1 4202496 1205797 0 0 0 3998831 1296 0 0 25 0 1 0 643857317 5135667200 1178609 33554432000 4194304 4964800 140733683555088 18446744073709551615 255106424172 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/13034/statm: 1253825 1178609 476 189 0 1248174 0
Current children cumulated CPU time (s) 40001.3
Current children cumulated vsize (KiB) 5015300

[startup+5012.2 s]
/proc/loadavg: 8.02 8.07 8.01 2/400 13197
/proc/meminfo: memFree=10988952/32951124 swapFree=67111348/67111528
[pid=13034] ppid=13032 vsize=847164 CPUtime=40001.7 cores=0-7
/proc/13034/stat : 13034 (claspmt) R 13032 13034 13008 0 -1 4202496 1205797 0 0 0 3998853 1315 0 0 25 0 1 0 643857317 867495936 102600 33554432000 4194304 4964800 140733683555088 18446744073709551615 255106812039 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/13034/statm: 211791 102600 476 189 0 206140 0
Current children cumulated CPU time (s) 40001.7
Current children cumulated vsize (KiB) 847164

Child status: 0
Real time (s): 5012.22
CPU time (s): 40001.7
CPU user time (s): 39988.5
CPU system time (s): 13.171
CPU usage (%): 798.083
Max. virtual memory (cumulated for all children) (KiB): 5052876

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39988.5
system time used= 13.171
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1205798
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 79151
involuntary context switches= 188114

runsolver used 18.9841 second user time and 49.3715 second system time

The end

Launcher Data 

Begin job on node144 at 2011-05-16 04:53:32
IDJOB=3381380
IDBENCH=88216
IDSOLVER=1865
FILE ID=node144/3381380-1305514412
RUNJOBID= node144-1305514412-13022
PBS_JOBID= 13325560
Free space on /tmp= 73332 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m47-q841.5-n45-p30-s84954709.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3381380-1305514412/watcher-3381380-1305514412 -o /tmp/evaluation-result-3381380-1305514412/solver-3381380-1305514412 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3381380-1305514412.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 556c58061360a1b195d9b2f28888af77
RANDOM SEED=843822294

node144.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.16
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.850
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      15772352 kB
Buffers:        325236 kB
Cached:        3857588 kB
SwapCached:          0 kB
Active:       13990892 kB
Inactive:      2823296 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      15772352 kB
SwapTotal:    67111528 kB
SwapFree:     67111348 kB
Dirty:            5548 kB
Writeback:           0 kB
AnonPages:    12631696 kB
Mapped:          35404 kB
Slab:           244900 kB
PageTables:      51012 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 113615812 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73332 MiB
End job on node144 at 2011-05-16 06:17:05