Trace number 3472429

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 160022 5037.61

General information on the benchmark

Nameapplication/jarvisalo/HWMCC10-combination-timeframe-expansion/
hwmcc10-timeframe-expansion-k50-pdtvisns3p00-tseitin.cnf
MD5SUMa8ad0173824ed9404a8c153da07c5f9c
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark35489.3
Satisfiable
(Un)Satisfiability was proved
Number of variables183325
Number of clauses546914
Sum of the clauses size1276130
Maximum clause length3
Minimum clause length1
Number of clauses of size 12
Number of clauses of size 2364608
Number of clauses of size 3182304
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...640027/instance-3472429-1307640027.cnf
1.99/2.02	c Solving...
160003.84/5009.45	
160003.84/5009.45	*** INTERRUPTED! ***
160003.84/5009.45	*** Shutting down threads...

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3472429-1307640027/watcher-3472429-1307640027 -o /tmp/evaluation-result-3472429-1307640027/solver-3472429-1307640027 -C 160000 -W 5100 -M 256000 HOME/claspmt -t32 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3472429-1307640027.cnf 

running on 32 cores: 0,4,8,12,16,20,24,28,1,5,9,13,17,21,25,29,2,6,10,14,18,22,26,30,3,7,11,15,19,23,27,31

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 160000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 160030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 262144000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 262195200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 19.87 27.36 30.27 1/412 30343
/proc/meminfo: memFree=262316232/264257624 swapFree=67111528/67111528
[pid=30343] ppid=30341 vsize=44152 CPUtime=0.02 cores=0-31
/proc/30343/stat : 30343 (claspmt) R 30341 30343 29615 0 -1 4202496 5008 0 0 0 2 0 0 0 23 0 1 0 29389704 45211648 4938 270582939648 4194304 4964800 140734424414512 18446744073709551615 4700412 0 0 4096 18946 0 0 0 17 8 0 0 0
/proc/30343/statm: 11038 4946 373 189 0 5912 0

[startup+0.111 s]
/proc/loadavg: 19.87 27.36 30.27 1/412 30343
/proc/meminfo: memFree=262316232/264257624 swapFree=67111528/67111528
[pid=30343] ppid=30341 vsize=64512 CPUtime=0.1 cores=0-31
/proc/30343/stat : 30343 (claspmt) R 30341 30343 29615 0 -1 4202496 13339 0 0 0 8 2 0 0 24 0 1 0 29389704 66060288 11093 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 17 8 0 0 0
/proc/30343/statm: 16128 11095 383 189 0 11002 0
Current children cumulated CPU time (s) 0.1
Current children cumulated vsize (KiB) 64512

[startup+0.201005 s]
/proc/loadavg: 19.87 27.36 30.27 1/412 30343
/proc/meminfo: memFree=262316232/264257624 swapFree=67111528/67111528
[pid=30343] ppid=30341 vsize=76304 CPUtime=0.19 cores=0-31
/proc/30343/stat : 30343 (claspmt) R 30341 30343 29615 0 -1 4202496 17314 0 0 0 16 3 0 0 24 0 1 0 29389704 78135296 14054 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 17 8 0 0 0
/proc/30343/statm: 19076 14057 383 189 0 13950 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 76304

[startup+0.301024 s]
/proc/loadavg: 19.87 27.36 30.27 1/412 30343
/proc/meminfo: memFree=262316232/264257624 swapFree=67111528/67111528
[pid=30343] ppid=30341 vsize=97276 CPUtime=0.29 cores=0-31
/proc/30343/stat : 30343 (claspmt) R 30341 30343 29615 0 -1 4202496 24941 0 0 0 25 4 0 0 25 0 1 0 29389704 99610624 18934 270582939648 4194304 4964800 140734424414512 18446744073709551615 4618166 0 0 4096 18946 0 0 0 17 8 0 0 0
/proc/30343/statm: 24319 18935 396 189 0 19193 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 97276

[startup+0.701021 s]
/proc/loadavg: 19.87 27.36 30.27 1/412 30343
/proc/meminfo: memFree=262316232/264257624 swapFree=67111528/67111528
[pid=30343] ppid=30341 vsize=115268 CPUtime=0.69 cores=0-31
/proc/30343/stat : 30343 (claspmt) R 30341 30343 29615 0 -1 4202496 29353 0 0 0 64 5 0 0 25 0 1 0 29389704 118034432 23346 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 17 8 0 0 0
/proc/30343/statm: 28817 23346 401 189 0 23691 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 115268

[startup+1.50105 s]
/proc/loadavg: 19.87 27.36 30.27 2/413 30344
/proc/meminfo: memFree=262215892/264257624 swapFree=67111528/67111528
[pid=30343] ppid=30341 vsize=127940 CPUtime=1.49 cores=0-31
/proc/30343/stat : 30343 (claspmt) R 30341 30343 29615 0 -1 4202496 32531 0 0 0 143 6 0 0 25 0 1 0 29389704 131010560 26524 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 17 8 0 0 0
/proc/30343/statm: 31985 26524 401 189 0 26859 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 127940

[startup+3.10304 s]
/proc/loadavg: 19.87 27.36 30.27 32/444 30375
/proc/meminfo: memFree=262082400/264257624 swapFree=67111528/67111528
[pid=30343] ppid=30341 vsize=1992264 CPUtime=19.6 cores=0-31
/proc/30343/stat : 30343 (claspmt) S 30341 30343 29615 0 -1 4202496 199843 0 0 0 307 1653 0 0 18 0 32 0 29389704 2040078336 146613 270582939648 4194304 4964800 140734424414512 18446744073709551615 258662772457 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/30343/statm: 498066 146616 433 189 0 492415 0
[pid=30343/tid=30345] ppid=30341 vsize=1992264 CPUtime=0.6 cores=0-31
/proc/30343/task/30345/stat : 30345 (claspmt) D 30341 30343 29615 0 -1 4202560 3986 0 0 0 2 58 0 0 18 0 32 0 29389906 2040078336 146621 270582939648 4194304 4964800 140734424414512 18446744073709551615 4367620 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30343/tid=30346] ppid=30341 vsize=1992264 CPUtime=0.59 cores=0-31
/proc/30343/task/30346/stat : 30346 (claspmt) D 30341 30343 29615 0 -1 4202560 4848 0 0 0 2 57 0 0 18 0 32 0 29389906 2040078336 146625 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=30343/tid=30347] ppid=30341 vsize=1992264 CPUtime=0.57 cores=0-31
/proc/30343/task/30347/stat : 30347 (claspmt) D 30341 30343 29615 0 -1 4202560 5893 0 0 0 4 53 0 0 17 0 32 0 29389906 2040078336 146626 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 19 0 0 0
[pid=30343/tid=30348] ppid=30341 vsize=1992264 CPUtime=0.53 cores=0-31
/proc/30343/task/30348/stat : 30348 (claspmt) D 30341 30343 29615 0 -1 4202560 5231 0 0 0 3 50 0 0 17 0 32 0 29389906 2040078336 146627 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 27 0 0 0
[pid=30343/tid=30349] ppid=30341 vsize=1992264 CPUtime=0.55 cores=0-31
/proc/30343/task/30349/stat : 30349 (claspmt) D 30341 30343 29615 0 -1 4202560 5077 0 0 0 2 53 0 0 17 0 32 0 29389906 2040078336 146630 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 20 0 0 0
[pid=30343/tid=30350] ppid=30341 vsize=1992264 CPUtime=0.57 cores=0-31
/proc/30343/task/30350/stat : 30350 (claspmt) S 30341 30343 29615 0 -1 4202560 5856 0 0 0 3 54 0 0 17 0 32 0 29389906 2040078336 146633 270582939648 4194304 4964800 140734424414512 18446744073709551615 258651060958 0 0 4096 18946 18446744071566923784 0 0 -1 18 0 0 0
[pid=30343/tid=30351] ppid=30341 vsize=1992264 CPUtime=0.58 cores=0-31
/proc/30343/task/30351/stat : 30351 (claspmt) D 30341 30343 29615 0 -1 4202560 5712 0 0 0 2 56 0 0 17 0 32 0 29389906 2040078336 146636 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 29 0 0 0
[pid=30343/tid=30352] ppid=30341 vsize=1992264 CPUtime=0.59 cores=0-31
/proc/30343/task/30352/stat : 30352 (claspmt) D 30341 30343 29615 0 -1 4202560 4735 0 0 0 3 56 0 0 17 0 32 0 29389907 2040078336 146643 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=30343/tid=30353] ppid=30341 vsize=1992264 CPUtime=0.52 cores=0-31
/proc/30343/task/30353/stat : 30353 (claspmt) S 30341 30343 29615 0 -1 4202560 5714 0 0 0 9 43 0 0 17 0 32 0 29389907 2040078336 146651 270582939648 4194304 4964800 140734424414512 18446744073709551615 258662772457 0 0 4096 18946 18446604572847685088 0 0 -1 7 0 0 0
[pid=30343/tid=30354] ppid=30341 vsize=1992264 CPUtime=0.57 cores=0-31
/proc/30343/task/30354/stat : 30354 (claspmt) D 30341 30343 29615 0 -1 4202560 4221 0 0 0 2 55 0 0 18 0 32 0 29389907 2040078336 146654 270582939648 4194304 4964800 140734424414512 18446744073709551615 4377943 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=30343/tid=30355] ppid=30341 vsize=1992264 CPUtime=0.52 cores=0-31
/proc/30343/task/30355/stat : 30355 (claspmt) D 30341 30343 29615 0 -1 4202560 3869 0 0 0 1 51 0 0 18 0 32 0 29389907 2040078336 146661 270582939648 4194304 4964800 140734424414512 18446744073709551615 4679106 0 0 4096 18946 0 0 0 -1 15 0 0 0
[pid=30343/tid=30356] ppid=30341 vsize=1992264 CPUtime=0.55 cores=0-31
/proc/30343/task/30356/stat : 30356 (claspmt) D 30341 30343 29615 0 -1 4202560 5565 0 0 0 3 52 0 0 17 0 32 0 29389907 2040078336 146667 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 8 0 0 0
[pid=30343/tid=30357] ppid=30341 vsize=1992264 CPUtime=0.59 cores=0-31
/proc/30343/task/30357/stat : 30357 (claspmt) D 30341 30343 29615 0 -1 4202560 4437 0 0 0 1 58 0 0 18 0 32 0 29389907 2040078336 146668 270582939648 4194304 4964800 140734424414512 18446744073709551615 4367551 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=30343/tid=30358] ppid=30341 vsize=1992264 CPUtime=0.56 cores=0-31
/proc/30343/task/30358/stat : 30358 (claspmt) D 30341 30343 29615 0 -1 4202560 5622 0 0 0 3 53 0 0 17 0 32 0 29389907 2040078336 146669 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650616986 0 0 4096 18946 66551808 0 0 -1 9 0 0 0
[pid=30343/tid=30359] ppid=30341 vsize=1992264 CPUtime=0.54 cores=0-31
/proc/30343/task/30359/stat : 30359 (claspmt) D 30341 30343 29615 0 -1 4202560 6143 0 0 0 3 51 0 0 17 0 32 0 29389907 2040078336 146673 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 30 0 0 0
[pid=30343/tid=30360] ppid=30341 vsize=1992264 CPUtime=0.58 cores=0-31
/proc/30343/task/30360/stat : 30360 (claspmt) D 30341 30343 29615 0 -1 4202560 5463 0 0 0 2 56 0 0 17 0 32 0 29389907 2040078336 146678 270582939648 4194304 4964800 140734424414512 18446744073709551615 4685574 0 0 4096 18946 0 0 0 -1 16 0 0 0
[pid=30343/tid=30361] ppid=30341 vsize=1992264 CPUtime=0.54 cores=0-31
/proc/30343/task/30361/stat : 30361 (claspmt) D 30341 30343 29615 0 -1 4202560 5438 0 0 0 3 51 0 0 17 0 32 0 29389907 2040078336 146684 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 24 0 0 0
[pid=30343/tid=30362] ppid=30341 vsize=1992264 CPUtime=0.56 cores=0-31
/proc/30343/task/30362/stat : 30362 (claspmt) D 30341 30343 29615 0 -1 4202560 5946 0 0 0 5 51 0 0 17 0 32 0 29389907 2040078336 146687 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30343/tid=30363] ppid=30341 vsize=1992264 CPUtime=0.59 cores=0-31
/proc/30343/task/30363/stat : 30363 (claspmt) R 30341 30343 29615 0 -1 4202560 5716 0 0 0 4 55 0 0 17 0 32 0 29389907 2040078336 146694 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 13 0 0 0
[pid=30343/tid=30364] ppid=30341 vsize=1992264 CPUtime=0.57 cores=0-31
/proc/30343/task/30364/stat : 30364 (claspmt) D 30341 30343 29615 0 -1 4202560 5368 0 0 0 3 54 0 0 17 0 32 0 29389907 2040078336 146701 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 10 0 0 0
[pid=30343/tid=30365] ppid=30341 vsize=1992264 CPUtime=0.48 cores=0-31
/proc/30343/task/30365/stat : 30365 (claspmt) D 30341 30343 29615 0 -1 4202560 5203 0 0 0 3 45 0 0 17 0 32 0 29389907 2040078336 146702 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 11 0 0 0
[pid=30343/tid=30366] ppid=30341 vsize=1992264 CPUtime=0.53 cores=0-31
/proc/30343/task/30366/stat : 30366 (claspmt) D 30341 30343 29615 0 -1 4202560 5348 0 0 0 2 51 0 0 17 0 32 0 29389907 2040078336 146705 270582939648 4194304 4964800 140734424414512 18446744073709551615 4685574 0 0 4096 18946 0 0 0 -1 21 0 0 0
[pid=30343/tid=30367] ppid=30341 vsize=1992264 CPUtime=0.56 cores=0-31
/proc/30343/task/30367/stat : 30367 (claspmt) D 30341 30343 29615 0 -1 4202560 5561 0 0 0 2 54 0 0 17 0 32 0 29389907 2040078336 146709 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 28 0 0 0
[pid=30343/tid=30368] ppid=30341 vsize=1992264 CPUtime=0.56 cores=0-31
/proc/30343/task/30368/stat : 30368 (claspmt) D 30341 30343 29615 0 -1 4202560 5437 0 0 0 3 53 0 0 17 0 32 0 29389907 2040078336 146716 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 26 0 0 0
[pid=30343/tid=30369] ppid=30341 vsize=1992264 CPUtime=0.57 cores=0-31
/proc/30343/task/30369/stat : 30369 (claspmt) R 30341 30343 29615 0 -1 4202560 5161 0 0 0 3 54 0 0 17 0 32 0 29389907 2040078336 146721 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 12 0 0 0
[pid=30343/tid=30370] ppid=30341 vsize=1992264 CPUtime=0.53 cores=0-31
/proc/30343/task/30370/stat : 30370 (claspmt) D 30341 30343 29615 0 -1 4202560 5924 0 0 0 3 50 0 0 17 0 32 0 29389907 2040078336 146726 270582939648 4194304 4964800 140734424414512 18446744073709551615 4685574 0 0 4096 18946 0 0 0 -1 22 0 0 0
[pid=30343/tid=30371] ppid=30341 vsize=1992264 CPUtime=0.56 cores=0-31
/proc/30343/task/30371/stat : 30371 (claspmt) D 30341 30343 29615 0 -1 4202560 5659 0 0 0 3 53 0 0 17 0 32 0 29389907 2040078336 146729 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650653361 0 0 4096 18946 0 0 0 -1 25 0 0 0
[pid=30343/tid=30372] ppid=30341 vsize=1992264 CPUtime=0.56 cores=0-31
/proc/30343/task/30372/stat : 30372 (claspmt) D 30341 30343 29615 0 -1 4202560 5054 0 0 0 3 53 0 0 17 0 32 0 29389907 2040078336 146745 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30343/tid=30373] ppid=30341 vsize=1992264 CPUtime=0.52 cores=0-31
/proc/30343/task/30373/stat : 30373 (claspmt) D 30341 30343 29615 0 -1 4202560 5941 0 0 0 5 47 0 0 17 0 32 0 29389908 2040078336 146745 270582939648 4194304 4964800 140734424414512 18446744073709551615 4685574 0 0 4096 18946 0 0 0 -1 23 0 0 0
[pid=30343/tid=30374] ppid=30341 vsize=1992264 CPUtime=0.55 cores=0-31
/proc/30343/task/30374/stat : 30374 (claspmt) D 30341 30343 29615 0 -1 4202560 5909 0 0 0 3 52 0 0 17 0 32 0 29389908 2040078336 146745 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 19 0 0 0
[pid=30343/tid=30375] ppid=30341 vsize=1992264 CPUtime=0.55 cores=0-31
/proc/30343/task/30375/stat : 30375 (claspmt) D 30341 30343 29615 0 -1 4202560 6114 0 0 0 3 52 0 0 17 0 32 0 29389908 2040078336 146748 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 17 0 0 0
Current children cumulated CPU time (s) 19.6
Current children cumulated vsize (KiB) 1992264

[startup+6.30209 s]
/proc/loadavg: 20.53 27.37 30.26 35/444 30375
/proc/meminfo: memFree=261536024/264257624 swapFree=67111528/67111528
[pid=30343] ppid=30341 vsize=2203568 CPUtime=85.9 cores=0-31
/proc/30343/stat : 30343 (claspmt) R 30341 30343 29615 0 -1 4202496 250873 0 0 0 6770 1820 0 0 20 0 32 0 29389704 2256453632 197397 270582939648 4194304 4964800 140734424414512 18446744073709551615 4668773 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/30343/statm: 550892 197397 461 189 0 545241 0
[pid=30343/tid=30345] ppid=30341 vsize=2203568 CPUtime=2.58 cores=0-31
/proc/30343/task/30345/stat : 30345 (claspmt) R 30341 30343 29615 0 -1 4202560 5656 0 0 0 194 64 0 0 18 0 32 0 29389906 2256453632 197398 270582939648 4194304 4964800 140734424414512 18446744073709551615 4370229 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30343/tid=30346] ppid=30341 vsize=2203568 CPUtime=2.6 cores=0-31
/proc/30343/task/30346/stat : 30346 (claspmt) R 30341 30343 29615 0 -1 4202560 7018 0 0 0 195 65 0 0 18 0 32 0 29389906 2256453632 197398 270582939648 4194304 4964800 140734424414512 18446744073709551615 4660384 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=30343/tid=30347] ppid=30341 vsize=2203568 CPUtime=2.55 cores=0-31
/proc/30343/task/30347/stat : 30347 (claspmt) R 30341 30343 29615 0 -1 4202560 8121 0 0 0 196 59 0 0 18 0 32 0 29389906 2256453632 197398 270582939648 4194304 4964800 140734424414512 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 15 0 0 0
[pid=30343/tid=30348] ppid=30341 vsize=2203568 CPUtime=2.47 cores=0-31
/proc/30343/task/30348/stat : 30348 (claspmt) R 30341 30343 29615 0 -1 4202560 6253 0 0 0 194 53 0 0 18 0 32 0 29389906 2256453632 197398 270582939648 4194304 4964800 140734424414512 18446744073709551615 4371527 0 0 4096 18946 0 0 0 -1 27 0 0 0
[pid=30343/tid=30349] ppid=30341 vsize=2203568 CPUtime=3.34 cores=0-31
/proc/30343/task/30349/stat : 30349 (claspmt) R 30341 30343 29615 0 -1 4202560 6699 0 0 0 276 58 0 0 21 0 32 0 29389906 2256453632 197398 270582939648 4194304 4964800 140734424414512 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 16 0 0 0
[pid=30343/tid=30350] ppid=30341 vsize=2203568 CPUtime=2.56 cores=0-31
/proc/30343/task/30350/stat : 30350 (claspmt) R 30341 30343 29615 0 -1 4202560 8090 0 0 0 195 61 0 0 18 0 32 0 29389906 2256453632 197398 270582939648 4194304 4964800 140734424414512 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 18 0 0 0
[pid=30343/tid=30351] ppid=30341 vsize=2203568 CPUtime=2.59 cores=0-31
/proc/30343/task/30351/stat : 30351 (claspmt) R 30341 30343 29615 0 -1 4202560 7178 0 0 0 197 62 0 0 18 0 32 0 29389906 2256453632 197398 270582939648 4194304 4964800 140734424414512 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 29 0 0 0
[pid=30343/tid=30352] ppid=30341 vsize=2203568 CPUtime=2.56 cores=0-31
/proc/30343/task/30352/stat : 30352 (claspmt) R 30341 30343 29615 0 -1 4202560 6228 0 0 0 194 62 0 0 18 0 32 0 29389907 2256453632 197398 270582939648 4194304 4964800 140734424414512 18446744073709551615 4370264 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=30343/tid=30353] ppid=30341 vsize=2203568 CPUtime=2.4 cores=0-31
/proc/30343/task/30353/stat : 30353 (claspmt) R 30341 30343 29615 0 -1 4202560 5967 0 0 0 196 44 0 0 18 0 32 0 29389907 2256453632 197398 270582939648 4194304 4964800 140734424414512 18446744073709551615 4370259 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30343/tid=30354] ppid=30341 vsize=2203568 CPUtime=2.6 cores=0-31
/proc/30343/task/30354/stat : 30354 (claspmt) R 30341 30343 29615 0 -1 4202560 7160 0 0 0 197 63 0 0 18 0 32 0 29389907 2256453632 197398 270582939648 4194304 4964800 140734424414512 18446744073709551615 4668984 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=30343/tid=30355] ppid=30341 vsize=2203568 CPUtime=2.56 cores=0-31
/proc/30343/task/30355/stat : 30355 (claspmt) R 30341 30343 29615 0 -1 4202560 7268 0 0 0 197 59 0 0 18 0 32 0 29389907 2256453632 197398 270582939648 4194304 4964800 140734424414512 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 19 0 0 0
[pid=30343/tid=30356] ppid=30341 vsize=2203568 CPUtime=2.47 cores=0-31
/proc/30343/task/30356/stat : 30356 (claspmt) R 30341 30343 29615 0 -1 4202560 6256 0 0 0 192 55 0 0 17 0 32 0 29389907 2256453632 197398 270582939648 4194304 4964800 140734424414512 18446744073709551615 4669011 0 0 4096 18946 0 0 0 -1 8 0 0 0
[pid=30343/tid=30357] ppid=30341 vsize=2203568 CPUtime=3.38 cores=0-31
/proc/30343/task/30357/stat : 30357 (claspmt) R 30341 30343 29615 0 -1 4202560 6059 0 0 0 275 63 0 0 22 0 32 0 29389907 2256453632 197399 270582939648 4194304 4964800 140734424414512 18446744073709551615 4670437 0 0 4096 18946 0 0 0 -1 14 0 0 0
[pid=30343/tid=30358] ppid=30341 vsize=2203568 CPUtime=2.56 cores=0-31
/proc/30343/task/30358/stat : 30358 (claspmt) R 30341 30343 29615 0 -1 4202560 7979 0 0 0 196 60 0 0 18 0 32 0 29389907 2256453632 197399 270582939648 4194304 4964800 140734424414512 18446744073709551615 4669509 0 0 4096 18946 0 0 0 -1 9 0 0 0
[pid=30343/tid=30359] ppid=30341 vsize=2203568 CPUtime=2.52 cores=0-31
/proc/30343/task/30359/stat : 30359 (claspmt) R 30341 30343 29615 0 -1 4202560 7184 0 0 0 197 55 0 0 18 0 32 0 29389907 2256453632 197399 270582939648 4194304 4964800 140734424414512 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 30 0 0 0
[pid=30343/tid=30360] ppid=30341 vsize=2203568 CPUtime=2.51 cores=0-31
/proc/30343/task/30360/stat : 30360 (claspmt) R 30341 30343 29615 0 -1 4202560 6301 0 0 0 192 59 0 0 18 0 32 0 29389907 2256453632 197400 270582939648 4194304 4964800 140734424414512 18446744073709551615 4669086 0 0 4096 18946 0 0 0 -1 28 0 0 0
[pid=30343/tid=30361] ppid=30341 vsize=2203568 CPUtime=2.46 cores=0-31
/proc/30343/task/30361/stat : 30361 (claspmt) R 30341 30343 29615 0 -1 4202560 6380 0 0 0 192 54 0 0 18 0 32 0 29389907 2256453632 197400 270582939648 4194304 4964800 140734424414512 18446744073709551615 4668748 0 0 4096 18946 0 0 0 -1 24 0 0 0
[pid=30343/tid=30362] ppid=30341 vsize=2203568 CPUtime=2.54 cores=0-31
/proc/30343/task/30362/stat : 30362 (claspmt) R 30341 30343 29615 0 -1 4202560 8136 0 0 0 197 57 0 0 18 0 32 0 29389907 2256453632 197400 270582939648 4194304 4964800 140734424414512 18446744073709551615 4379082 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30343/tid=30363] ppid=30341 vsize=2203568 CPUtime=2.59 cores=0-31
/proc/30343/task/30363/stat : 30363 (claspmt) R 30341 30343 29615 0 -1 4202560 8089 0 0 0 196 63 0 0 18 0 32 0 29389907 2256453632 197400 270582939648 4194304 4964800 140734424414512 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 13 0 0 0
[pid=30343/tid=30364] ppid=30341 vsize=2203568 CPUtime=2.5 cores=0-31
/proc/30343/task/30364/stat : 30364 (claspmt) R 30341 30343 29615 0 -1 4202560 6383 0 0 0 192 58 0 0 18 0 32 0 29389907 2256453632 197401 270582939648 4194304 4964800 140734424414512 18446744073709551615 4336761 0 0 4096 18946 0 0 0 -1 10 0 0 0
[pid=30343/tid=30365] ppid=30341 vsize=2203568 CPUtime=3.22 cores=0-31
/proc/30343/task/30365/stat : 30365 (claspmt) R 30341 30343 29615 0 -1 4202560 6343 0 0 0 272 50 0 0 21 0 32 0 29389907 2256453632 197401 270582939648 4194304 4964800 140734424414512 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 31 0 0 0
[pid=30343/tid=30366] ppid=30341 vsize=2203568 CPUtime=2.54 cores=0-31
/proc/30343/task/30366/stat : 30366 (claspmt) R 30341 30343 29615 0 -1 4202560 8006 0 0 0 197 57 0 0 18 0 32 0 29389907 2256453632 197402 270582939648 4194304 4964800 140734424414512 18446744073709551615 4362900 0 0 4096 18946 0 0 0 -1 21 0 0 0
[pid=30343/tid=30367] ppid=30341 vsize=2203568 CPUtime=2.57 cores=0-31
/proc/30343/task/30367/stat : 30367 (claspmt) R 30341 30343 29615 0 -1 4202560 7149 0 0 0 197 60 0 0 17 0 32 0 29389907 2256453632 197402 270582939648 4194304 4964800 140734424414512 18446744073709551615 4669008 0 0 4096 18946 0 0 0 -1 20 0 0 0
[pid=30343/tid=30368] ppid=30341 vsize=2203568 CPUtime=2.49 cores=0-31
/proc/30343/task/30368/stat : 30368 (claspmt) R 30341 30343 29615 0 -1 4202560 6364 0 0 0 192 57 0 0 18 0 32 0 29389907 2256453632 197402 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 26 0 0 0
[pid=30343/tid=30369] ppid=30341 vsize=2203568 CPUtime=2.52 cores=0-31
/proc/30343/task/30369/stat : 30369 (claspmt) R 30341 30343 29615 0 -1 4202560 6316 0 0 0 193 59 0 0 18 0 32 0 29389907 2256453632 197403 270582939648 4194304 4964800 140734424414512 18446744073709551615 4370236 0 0 4096 18946 0 0 0 -1 12 0 0 0

################
# More data... #
################

/proc/30343/task/30370/stat : 30370 (claspmt) R 30341 30343 29615 0 -1 4202560 65344 0 0 0 495473 275 0 0 25 0 32 0 29389907 7638917120 1607106 270582939648 4194304 4964800 140734424414512 18446744073709551615 4669471 0 0 4096 18946 0 0 0 -1 22 0 0 0
[pid=30343/tid=30371] ppid=30341 vsize=7459880 CPUtime=4957.51 cores=0-31
/proc/30343/task/30371/stat : 30371 (claspmt) R 30341 30343 29615 0 -1 4202560 23653 0 0 0 495633 118 0 0 25 0 32 0 29389907 7638917120 1607106 270582939648 4194304 4964800 140734424414512 18446744073709551615 4668689 0 0 4096 18946 0 0 0 -1 25 0 0 0
[pid=30343/tid=30372] ppid=30341 vsize=7459880 CPUtime=4957.45 cores=0-31
/proc/30343/task/30372/stat : 30372 (claspmt) R 30341 30343 29615 0 -1 4202560 60098 0 0 0 495471 274 0 0 25 0 32 0 29389907 7638917120 1607106 270582939648 4194304 4964800 140734424414512 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30343/tid=30373] ppid=30341 vsize=7459880 CPUtime=4956.81 cores=0-31
/proc/30343/task/30373/stat : 30373 (claspmt) R 30341 30343 29615 0 -1 4202560 60301 0 0 0 495418 263 0 0 25 0 32 0 29389908 7638917120 1607106 270582939648 4194304 4964800 140734424414512 18446744073709551615 4668689 0 0 4096 18946 0 0 0 -1 14 0 0 0
[pid=30343/tid=30374] ppid=30341 vsize=7459880 CPUtime=4934.29 cores=0-31
/proc/30343/task/30374/stat : 30374 (claspmt) R 30341 30343 29615 0 -1 4202560 24588 0 0 0 493305 124 0 0 25 0 32 0 29389908 7638917120 1607106 270582939648 4194304 4964800 140734424414512 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 23 0 0 0
[pid=30343/tid=30375] ppid=30341 vsize=7459880 CPUtime=4957.42 cores=0-31
/proc/30343/task/30375/stat : 30375 (claspmt) R 30341 30343 29615 0 -1 4202560 155862 0 0 0 495102 640 0 0 25 0 32 0 29389908 7638917120 1607106 270582939648 4194304 4964800 140734424414512 18446744073709551615 4342208 0 0 4096 18946 0 0 0 -1 20 0 0 0
Current children cumulated CPU time (s) 158498
Current children cumulated vsize (KiB) 7459880



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5009.4 s]
/proc/loadavg: 32.36 32.16 32.07 34/444 30522
/proc/meminfo: memFree=255861052/264257624 swapFree=67111528/67111528
[pid=30343] ppid=30341 vsize=7459880 CPUtime=160004 cores=0-31
/proc/30343/stat : 30343 (claspmt) R 30341 30343 29615 0 -1 4202496 1670470 0 0 0 15992900 7484 0 0 25 0 32 0 29389704 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4669471 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/30343/statm: 1864970 1613575 463 189 0 1859319 0
[pid=30343/tid=30345] ppid=30341 vsize=7459880 CPUtime=4997.64 cores=0-31
/proc/30343/task/30345/stat : 30345 (claspmt) R 30341 30343 29615 0 -1 4202560 16098 0 0 0 499664 100 0 0 25 0 32 0 29389906 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30343/tid=30346] ppid=30341 vsize=7459880 CPUtime=5004.67 cores=0-31
/proc/30343/task/30346/stat : 30346 (claspmt) R 30341 30343 29615 0 -1 4202560 57317 0 0 0 500207 260 0 0 25 0 32 0 29389906 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 17 0 0 0
[pid=30343/tid=30347] ppid=30341 vsize=7459880 CPUtime=4975.62 cores=0-31
/proc/30343/task/30347/stat : 30347 (claspmt) R 30341 30343 29615 0 -1 4202560 27560 0 0 0 497428 134 0 0 25 0 32 0 29389906 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4670349 0 0 4096 18946 0 0 0 -1 31 0 0 0
[pid=30343/tid=30348] ppid=30341 vsize=7459880 CPUtime=5000.46 cores=0-31
/proc/30343/task/30348/stat : 30348 (claspmt) R 30341 30343 29615 0 -1 4202560 53171 0 0 0 499789 257 0 0 25 0 32 0 29389906 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 15 0 0 0
[pid=30343/tid=30349] ppid=30341 vsize=7459880 CPUtime=5005.43 cores=0-31
/proc/30343/task/30349/stat : 30349 (claspmt) R 30341 30343 29615 0 -1 4202560 56821 0 0 0 500296 247 0 0 25 0 32 0 29389906 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 27 0 0 0
[pid=30343/tid=30350] ppid=30341 vsize=7459880 CPUtime=5004.65 cores=0-31
/proc/30343/task/30350/stat : 30350 (claspmt) R 30341 30343 29615 0 -1 4202560 23941 0 0 0 500346 119 0 0 25 0 32 0 29389906 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4684107 0 0 4096 18946 0 0 0 -1 18 0 0 0
[pid=30343/tid=30351] ppid=30341 vsize=7459880 CPUtime=4990.64 cores=0-31
/proc/30343/task/30351/stat : 30351 (claspmt) R 30341 30343 29615 0 -1 4202560 171834 0 0 0 498334 730 0 0 25 0 32 0 29389906 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4669493 0 0 4096 18946 0 0 0 -1 21 0 0 0
[pid=30343/tid=30352] ppid=30341 vsize=7459880 CPUtime=4993.15 cores=0-31
/proc/30343/task/30352/stat : 30352 (claspmt) R 30341 30343 29615 0 -1 4202560 17487 0 0 0 499204 111 0 0 25 0 32 0 29389907 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4669093 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30343/tid=30353] ppid=30341 vsize=7459880 CPUtime=4990.75 cores=0-31
/proc/30343/task/30353/stat : 30353 (claspmt) R 30341 30343 29615 0 -1 4202560 16775 0 0 0 498996 79 0 0 25 0 32 0 29389907 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=30343/tid=30354] ppid=30341 vsize=7459880 CPUtime=5004.71 cores=0-31
/proc/30343/task/30354/stat : 30354 (claspmt) R 30341 30343 29615 0 -1 4202560 51779 0 0 0 500239 232 0 0 25 0 32 0 29389907 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=30343/tid=30355] ppid=30341 vsize=7459880 CPUtime=5004.57 cores=0-31
/proc/30343/task/30355/stat : 30355 (claspmt) R 30341 30343 29615 0 -1 4202560 22893 0 0 0 500339 118 0 0 25 0 32 0 29389907 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4669479 0 0 4096 18946 0 0 0 -1 13 0 0 0
[pid=30343/tid=30356] ppid=30341 vsize=7459880 CPUtime=5004.54 cores=0-31
/proc/30343/task/30356/stat : 30356 (claspmt) R 30341 30343 29615 0 -1 4202560 44695 0 0 0 500253 201 0 0 25 0 32 0 29389907 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4669413 0 0 4096 18946 0 0 0 -1 8 0 0 0
[pid=30343/tid=30357] ppid=30341 vsize=7459880 CPUtime=4994.53 cores=0-31
/proc/30343/task/30357/stat : 30357 (claspmt) R 30341 30343 29615 0 -1 4202560 55119 0 0 0 499188 265 0 0 25 0 32 0 29389907 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4668734 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=30343/tid=30358] ppid=30341 vsize=7459880 CPUtime=5004.62 cores=0-31
/proc/30343/task/30358/stat : 30358 (claspmt) R 30341 30343 29615 0 -1 4202560 25049 0 0 0 500327 135 0 0 25 0 32 0 29389907 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4668987 0 0 4096 18946 0 0 0 -1 9 0 0 0
[pid=30343/tid=30359] ppid=30341 vsize=7459880 CPUtime=5004.55 cores=0-31
/proc/30343/task/30359/stat : 30359 (claspmt) R 30341 30343 29615 0 -1 4202560 146569 0 0 0 499812 643 0 0 25 0 32 0 29389907 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 30 0 0 0
[pid=30343/tid=30360] ppid=30341 vsize=7459880 CPUtime=5003.09 cores=0-31
/proc/30343/task/30360/stat : 30360 (claspmt) R 30341 30343 29615 0 -1 4202560 17869 0 0 0 500203 106 0 0 25 0 32 0 29389907 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4669509 0 0 4096 18946 0 0 0 -1 28 0 0 0
[pid=30343/tid=30361] ppid=30341 vsize=7459880 CPUtime=5004.35 cores=0-31
/proc/30343/task/30361/stat : 30361 (claspmt) R 30341 30343 29615 0 -1 4202560 19397 0 0 0 500336 99 0 0 25 0 32 0 29389907 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4669509 0 0 4096 18946 0 0 0 -1 24 0 0 0
[pid=30343/tid=30362] ppid=30341 vsize=7459880 CPUtime=5004.64 cores=0-31
/proc/30343/task/30362/stat : 30362 (claspmt) R 30341 30343 29615 0 -1 4202560 60506 0 0 0 500214 250 0 0 25 0 32 0 29389907 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4660495 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30343/tid=30363] ppid=30341 vsize=7459880 CPUtime=4986.01 cores=0-31
/proc/30343/task/30363/stat : 30363 (claspmt) R 30341 30343 29615 0 -1 4202560 23861 0 0 0 498477 124 0 0 25 0 32 0 29389907 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4669509 0 0 4096 18946 0 0 0 -1 19 0 0 0
[pid=30343/tid=30364] ppid=30341 vsize=7459880 CPUtime=5004.55 cores=0-31
/proc/30343/task/30364/stat : 30364 (claspmt) R 30341 30343 29615 0 -1 4202560 50063 0 0 0 500209 246 0 0 25 0 32 0 29389907 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4668715 0 0 4096 18946 0 0 0 -1 10 0 0 0
[pid=30343/tid=30365] ppid=30341 vsize=7459880 CPUtime=4994.87 cores=0-31
/proc/30343/task/30365/stat : 30365 (claspmt) R 30341 30343 29615 0 -1 4202560 60673 0 0 0 499203 284 0 0 25 0 32 0 29389907 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4668721 0 0 4096 18946 0 0 0 -1 11 0 0 0
[pid=30343/tid=30366] ppid=30341 vsize=7459880 CPUtime=5004.62 cores=0-31
/proc/30343/task/30366/stat : 30366 (claspmt) R 30341 30343 29615 0 -1 4202560 25191 0 0 0 500337 125 0 0 25 0 32 0 29389907 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4669509 0 0 4096 18946 0 0 0 -1 29 0 0 0
[pid=30343/tid=30367] ppid=30341 vsize=7459880 CPUtime=5004.57 cores=0-31
/proc/30343/task/30367/stat : 30367 (claspmt) R 30341 30343 29615 0 -1 4202560 153131 0 0 0 499866 591 0 0 25 0 32 0 29389907 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4669509 0 0 4096 18946 0 0 0 -1 16 0 0 0
[pid=30343/tid=30368] ppid=30341 vsize=7459880 CPUtime=5004.62 cores=0-31
/proc/30343/task/30368/stat : 30368 (claspmt) R 30341 30343 29615 0 -1 4202560 16931 0 0 0 500359 103 0 0 25 0 32 0 29389907 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4668822 0 0 4096 18946 0 0 0 -1 26 0 0 0
[pid=30343/tid=30369] ppid=30341 vsize=7459880 CPUtime=5001.93 cores=0-31
/proc/30343/task/30369/stat : 30369 (claspmt) R 30341 30343 29615 0 -1 4202560 18311 0 0 0 500047 146 0 0 25 0 32 0 29389907 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 12 0 0 0
[pid=30343/tid=30370] ppid=30341 vsize=7459880 CPUtime=5004.6 cores=0-31
/proc/30343/task/30370/stat : 30370 (claspmt) R 30341 30343 29615 0 -1 4202560 65697 0 0 0 500184 276 0 0 25 0 32 0 29389907 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4670371 0 0 4096 18946 0 0 0 -1 22 0 0 0
[pid=30343/tid=30371] ppid=30341 vsize=7459880 CPUtime=5004.64 cores=0-31
/proc/30343/task/30371/stat : 30371 (claspmt) R 30341 30343 29615 0 -1 4202560 23653 0 0 0 500346 118 0 0 25 0 32 0 29389907 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 25 0 0 0
[pid=30343/tid=30372] ppid=30341 vsize=7459880 CPUtime=5004.58 cores=0-31
/proc/30343/task/30372/stat : 30372 (claspmt) R 30341 30343 29615 0 -1 4202560 60265 0 0 0 500183 275 0 0 25 0 32 0 29389907 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4338788 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30343/tid=30373] ppid=30341 vsize=7459880 CPUtime=5003.94 cores=0-31
/proc/30343/task/30373/stat : 30373 (claspmt) R 30341 30343 29615 0 -1 4202560 60665 0 0 0 500130 264 0 0 25 0 32 0 29389908 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 14 0 0 0
[pid=30343/tid=30374] ppid=30341 vsize=7459880 CPUtime=4981.42 cores=0-31
/proc/30343/task/30374/stat : 30374 (claspmt) R 30341 30343 29615 0 -1 4202560 24682 0 0 0 498018 124 0 0 25 0 32 0 29389908 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4654549 0 0 4096 18946 0 0 0 -1 23 0 0 0
[pid=30343/tid=30375] ppid=30341 vsize=7459880 CPUtime=5004.54 cores=0-31
/proc/30343/task/30375/stat : 30375 (claspmt) R 30341 30343 29615 0 -1 4202560 157108 0 0 0 499809 645 0 0 25 0 32 0 29389908 7638917120 1613575 270582939648 4194304 4964800 140734424414512 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 20 0 0 0
Current children cumulated CPU time (s) 160004
Current children cumulated vsize (KiB) 7459880

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)

Sending SIGKILL to -30343

[startup+5022.3 s]
/proc/loadavg: 27.70 31.16 31.75 2/413 30525
/proc/meminfo: memFree=252976596/264257624 swapFree=67111528/67111528
[pid=30343] ppid=30341 vsize=7303604 CPUtime=160012 cores=0-31
/proc/30343/stat : 30343 (claspmt) D 30341 30343 29615 0 -1 4204032 1670481 0 0 0 15993082 8087 0 0 18 0 1 0 29389704 7478890496 1596535 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650612305 0 0 4096 18946 18446744071562155273 0 0 17 28 0 0 463
/proc/30343/statm: 1825901 1596535 474 189 0 1820250 0
Current children cumulated CPU time (s) 160012
Current children cumulated vsize (KiB) 7303604

Solver just ended. Dumping a history of the last processes samples

[startup+5030.3 s]
/proc/loadavg: 23.90 30.24 31.44 2/413 30525
/proc/meminfo: memFree=251101776/264257624 swapFree=67111528/67111528
[pid=30343] ppid=30341 vsize=7303604 CPUtime=160017 cores=0-31
/proc/30343/stat : 30343 (claspmt) R 30341 30343 29615 0 -1 4204032 1670481 0 0 0 15993082 8578 0 0 18 0 1 0 29389704 7478890496 1596535 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650612305 0 0 4096 18946 0 0 0 17 28 0 0 848
/proc/30343/statm: 1825901 1596535 474 189 0 1820250 0
Current children cumulated CPU time (s) 160017
Current children cumulated vsize (KiB) 7303604

[startup+5033.5 s]
/proc/loadavg: 23.90 30.24 31.44 1/413 30525
/proc/meminfo: memFree=250408612/264257624 swapFree=67111528/67111528
[pid=30343] ppid=30341 vsize=7303604 CPUtime=160019 cores=0-31
/proc/30343/stat : 30343 (claspmt) R 30341 30343 29615 0 -1 4204032 1670481 0 0 0 15993082 8803 0 0 18 0 1 0 29389704 7478890496 1596535 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650612305 0 0 4096 18946 0 0 0 17 8 0 0 940
/proc/30343/statm: 1825901 1596535 474 189 0 1820250 0
Current children cumulated CPU time (s) 160019
Current children cumulated vsize (KiB) 7303604

[startup+5035.1 s]
/proc/loadavg: 22.22 29.78 31.29 1/413 30525
/proc/meminfo: memFree=249749924/264257624 swapFree=67111528/67111528
[pid=30343] ppid=30341 vsize=7303604 CPUtime=160020 cores=0-31
/proc/30343/stat : 30343 (claspmt) D 30341 30343 29615 0 -1 4204032 1670481 0 0 0 15993082 8919 0 0 18 0 1 0 29389704 7478890496 1596535 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650612305 0 0 4096 18946 18446744071562155273 0 0 17 8 0 0 966
/proc/30343/statm: 1825901 1596535 474 189 0 1820250 0
Current children cumulated CPU time (s) 160020
Current children cumulated vsize (KiB) 7303604

[startup+5036.7 s]
/proc/loadavg: 22.22 29.78 31.29 1/413 30525
/proc/meminfo: memFree=249487072/264257624 swapFree=67111528/67111528
[pid=30343] ppid=30341 vsize=7303604 CPUtime=160021 cores=0-31
/proc/30343/stat : 30343 (claspmt) R 30341 30343 29615 0 -1 4204032 1670481 0 0 0 15993082 9028 0 0 18 0 1 0 29389704 7478890496 1596535 270582939648 4194304 4964800 140734424414512 18446744073709551615 258650612305 0 0 4096 18946 0 0 0 17 28 0 0 1034
/proc/30343/statm: 1825901 1596535 474 189 0 1820250 0
Current children cumulated CPU time (s) 160021
Current children cumulated vsize (KiB) 7303604

[startup+5037.1 s]
/proc/loadavg: 22.22 29.78 31.29 1/413 30525
/proc/meminfo: memFree=249224192/264257624 swapFree=67111528/67111528
[pid=30343] ppid=30341 vsize=0 CPUtime=160021 cores=0-31
/proc/30343/stat : 30343 (claspmt) R 30341 30343 29615 0 -1 4204036 1670481 0 0 0 15993082 9053 0 0 18 0 1 0 29389704 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 28 0 0 1048
/proc/30343/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 160021
Current children cumulated vsize (KiB) 0

[startup+5037.5 s]
/proc/loadavg: 22.22 29.78 31.29 1/413 30525
/proc/meminfo: memFree=249224192/264257624 swapFree=67111528/67111528
[pid=30343] ppid=30341 vsize=0 CPUtime=160022 cores=0-31
/proc/30343/stat : 30343 (claspmt) R 30341 30343 29615 0 -1 4204036 1670481 0 0 0 15993082 9093 0 0 18 0 1 0 29389704 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 28 0 0 1048
/proc/30343/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 160022
Current children cumulated vsize (KiB) 0

[startup+5037.6 s]
/proc/loadavg: 22.22 29.78 31.29 1/413 30525
/proc/meminfo: memFree=249224192/264257624 swapFree=67111528/67111528
[pid=30343] ppid=30341 vsize=0 CPUtime=160022 cores=0-31
/proc/30343/stat : 30343 (claspmt) R 30341 30343 29615 0 -1 4204036 1670481 0 0 0 15993082 9103 0 0 18 0 1 0 29389704 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 28 0 0 1048
/proc/30343/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 160022
Current children cumulated vsize (KiB) 0

Child ended because it received signal 24 (SIGXCPU)
Child dumped core
Real time (s): 5037.61
CPU time (s): 160022
CPU user time (s): 159931
CPU system time (s): 91.0402
CPU usage (%): 3176.55
Max. virtual memory (cumulated for all children) (KiB): 7459880

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 159931
system time used= 91.0402
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1670481
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 219654
involuntary context switches= 382779

runsolver used 28.5127 second user time and 113.675 second system time

The end

Launcher Data 

Begin job on node200 at 2011-06-09 19:20:28
IDJOB=3472429
IDBENCH=82481
IDSOLVER=1961
FILE ID=node200/3472429-1307640027
RUNJOBID= node200-1307639768-29629
PBS_JOBID= 13547780
Free space on /tmp= 61736 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT11/application/jarvisalo/HWMCC10-combination-timeframe-expansion/hwmcc10-timeframe-expansion-k50-pdtvisns3p00-tseitin.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3472429-1307640027/watcher-3472429-1307640027 -o /tmp/evaluation-result-3472429-1307640027/solver-3472429-1307640027 -C 160000 -W 5100 -M 256000  HOME/claspmt -t32 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3472429-1307640027.cnf

TIME LIMIT= 160000 seconds
MEMORY LIMIT= 256000 MiB
NBCORE= 32

MD5SUM BENCH= a8ad0173824ed9404a8c153da07c5f9c
RANDOM SEED=1570151441

node200.alineos.net Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 64
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 96
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 66
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 98
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.99
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 68
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 100
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.03
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 70
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 102
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 16
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 17
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 48
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 18
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 80
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 19
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 112
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.97
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 20
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 21
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 50
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.10
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 22
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 82
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.10
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 114
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 24
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 25
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 26
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 84
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 27
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 116
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 28
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 29
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 54
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 30
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 86
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 31
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 118
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     264257624 kB
MemFree:      262317016 kB
Buffers:        235484 kB
Cached:        1406672 kB
SwapCached:          0 kB
Active:         260616 kB
Inactive:      1444872 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     264257624 kB
LowFree:      262317016 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           10500 kB
Writeback:           0 kB
AnonPages:       63356 kB
Mapped:          14528 kB
Slab:           152148 kB
PageTables:       4344 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  199240340 kB
Committed_AS:   164804 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296372 kB
VmallocChunk: 34359440603 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 55492 MiB
End job on node200 at 2011-06-09 20:44:26