Trace number 3472344

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 20467.7 5102.46

General information on the benchmark

Namevelev-npe-1.0-9dlx-b71.cnf
MD5SUM75d91f0ba4398ccb5b452707c0e7fa27
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark264.451
Satisfiable
(Un)Satisfiability was proved
Number of variables889302
Number of clauses14582074
Sum of the clauses size42006170
Maximum clause length5391
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 212803954
Number of clauses of size 3913633
Number of clauses of size 418493
Number of clauses of size 536153
Number of clauses of size over 5809840

Solver Data 

0.00/0.01	c clasp version 2.0.0-RC2
0.00/0.01	c Reading from ...503181/instance-3472344-1307503181.cnf
6.62/6.64	c Solving...
20465.20/5100.02	
20465.20/5100.02	*** INTERRUPTED! ***
20465.20/5100.02	*** Shutting down threads...

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3472344-1307503181/watcher-3472344-1307503181 -o /tmp/evaluation-result-3472344-1307503181/solver-3472344-1307503181 -C 160000 -W 5100 -M 256000 HOME/claspmt -t32 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3472344-1307503181.cnf 

running on 32 cores: 0,4,8,12,16,20,24,28,1,5,9,13,17,21,25,29,2,6,10,14,18,22,26,30,3,7,11,15,19,23,27,31

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 160000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 160030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 262144000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 262195200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 23.29 30.08 31.37 1/413 19911
/proc/meminfo: memFree=262756436/264257624 swapFree=67111528/67111528
[pid=19911] ppid=19909 vsize=133784 CPUtime=0.02 cores=0-31
/proc/19911/stat : 19911 (claspmt) R 19909 19911 19883 0 -1 4202496 1648 0 0 0 2 0 0 0 21 0 1 0 15706312 136994816 1577 270582939648 4194304 4964800 140734124447728 18446744073709551615 4700412 0 0 4096 18946 0 0 0 17 25 0 0 0
/proc/19911/statm: 33446 1582 373 189 0 28320 0

[startup+0.097045 s]
/proc/loadavg: 23.29 30.08 31.37 1/413 19911
/proc/meminfo: memFree=262756436/264257624 swapFree=67111528/67111528
[pid=19911] ppid=19909 vsize=133784 CPUtime=0.09 cores=0-31
/proc/19911/stat : 19911 (claspmt) R 19909 19911 19883 0 -1 4202496 6397 0 0 0 8 1 0 0 21 0 1 0 15706312 136994816 6326 270582939648 4194304 4964800 140734124447728 18446744073709551615 4700412 0 0 4096 18946 0 0 0 17 25 0 0 0
/proc/19911/statm: 33446 6331 373 189 0 28320 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 133784

[startup+0.101045 s]
/proc/loadavg: 23.29 30.08 31.37 1/413 19911
/proc/meminfo: memFree=262756436/264257624 swapFree=67111528/67111528
[pid=19911] ppid=19909 vsize=133784 CPUtime=0.09 cores=0-31
/proc/19911/stat : 19911 (claspmt) R 19909 19911 19883 0 -1 4202496 6709 0 0 0 8 1 0 0 21 0 1 0 15706312 136994816 6638 270582939648 4194304 4964800 140734124447728 18446744073709551615 4700412 0 0 4096 18946 0 0 0 17 25 0 0 0
/proc/19911/statm: 33446 6642 373 189 0 28320 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 133784

[startup+0.301052 s]
/proc/loadavg: 23.29 30.08 31.37 1/413 19911
/proc/meminfo: memFree=262756436/264257624 swapFree=67111528/67111528
[pid=19911] ppid=19909 vsize=133784 CPUtime=0.29 cores=0-31
/proc/19911/stat : 19911 (claspmt) R 19909 19911 19883 0 -1 4202496 22431 0 0 0 23 6 0 0 22 0 1 0 15706312 136994816 22360 270582939648 4194304 4964800 140734124447728 18446744073709551615 4700412 0 0 4096 18946 0 0 0 17 25 0 0 0
/proc/19911/statm: 33446 22365 373 189 0 28320 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 133784

[startup+0.701069 s]
/proc/loadavg: 23.29 30.08 31.37 1/413 19911
/proc/meminfo: memFree=262756436/264257624 swapFree=67111528/67111528
[pid=19911] ppid=19909 vsize=201620 CPUtime=0.69 cores=0-31
/proc/19911/stat : 19911 (claspmt) R 19909 19911 19883 0 -1 4202496 51794 0 0 0 56 13 0 0 23 0 1 0 15706312 206458880 44743 270582939648 4194304 4964800 140734124447728 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 17 25 0 0 0
/proc/19911/statm: 50405 44743 387 189 0 45279 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 201620

[startup+1.50108 s]
/proc/loadavg: 23.29 30.08 31.37 2/414 19912
/proc/meminfo: memFree=262569276/264257624 swapFree=67111528/67111528
[pid=19911] ppid=19909 vsize=241192 CPUtime=1.49 cores=0-31
/proc/19911/stat : 19911 (claspmt) R 19909 19911 19883 0 -1 4202496 62130 0 0 0 131 18 0 0 25 0 1 0 15706312 246980608 54424 270582939648 4194304 4964800 140734124447728 18446744073709551615 258650955760 0 0 4096 18946 0 0 0 17 25 0 0 0
/proc/19911/statm: 60298 54424 387 189 0 55172 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 241192

[startup+3.10116 s]
/proc/loadavg: 23.29 30.08 31.37 2/414 19912
/proc/meminfo: memFree=262505912/264257624 swapFree=67111528/67111528
[pid=19911] ppid=19909 vsize=328412 CPUtime=3.09 cores=0-31
/proc/19911/stat : 19911 (claspmt) R 19909 19911 19883 0 -1 4202496 84674 0 0 0 279 30 0 0 25 0 1 0 15706312 336293888 76028 270582939648 4194304 4964800 140734124447728 18446744073709551615 258650955760 0 0 4096 18946 0 0 0 17 25 0 0 0
/proc/19911/statm: 82103 76028 387 189 0 76977 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 328412

[startup+6.30127 s]
/proc/loadavg: 21.51 29.60 31.20 2/414 19912
/proc/meminfo: memFree=262325308/264257624 swapFree=67111528/67111528
[pid=19911] ppid=19909 vsize=518652 CPUtime=6.29 cores=0-31
/proc/19911/stat : 19911 (claspmt) R 19909 19911 19883 0 -1 4202496 133575 0 0 0 579 50 0 0 25 0 1 0 15706312 531099648 122816 270582939648 4194304 4964800 140734124447728 18446744073709551615 258650955760 0 0 4096 18946 0 0 0 17 25 0 0 0
/proc/19911/statm: 129663 122816 387 189 0 124537 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 518652

[startup+12.7024 s]
/proc/loadavg: 22.27 29.62 31.20 2/445 19943
/proc/meminfo: memFree=259892960/264257624 swapFree=67111528/67111528
[pid=19911] ppid=19909 vsize=4383704 CPUtime=99.35 cores=0-31
/proc/19911/stat : 19911 (claspmt) S 19909 19911 19883 0 -1 4202496 1000266 0 0 0 1232 8703 0 0 18 0 32 0 15706312 4488912896 756562 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446744073709551615 0 0 17 25 0 0 0
/proc/19911/statm: 1095926 756566 428 189 0 1090275 0
[pid=19911/tid=19913] ppid=19909 vsize=4383704 CPUtime=3.01 cores=0-31
/proc/19911/task/19913/stat : 19913 (claspmt) D 19909 19911 19883 0 -1 4202560 27156 0 0 0 23 278 0 0 15 0 32 0 15706977 4488912896 756568 270582939648 4194304 4964800 140734124447728 18446744073709551615 4697753 0 0 4096 18946 0 0 0 -1 26 0 0 0
[pid=19911/tid=19914] ppid=19909 vsize=4383704 CPUtime=3.04 cores=0-31
/proc/19911/task/19914/stat : 19914 (claspmt) D 19909 19911 19883 0 -1 4202560 25442 0 0 0 16 288 0 0 17 0 32 0 15706977 4488912896 756570 270582939648 4194304 4964800 140734124447728 18446744073709551615 4697753 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=19911/tid=19915] ppid=19909 vsize=4383704 CPUtime=3 cores=0-31
/proc/19911/task/19915/stat : 19915 (claspmt) S 19909 19911 19883 0 -1 4202560 24134 0 0 0 9 291 0 0 17 0 32 0 15706977 4488912896 756571 270582939648 4194304 4964800 140734124447728 18446744073709551615 258651060958 0 0 4096 18946 18446604567811169760 0 0 -1 0 0 0 0
[pid=19911/tid=19916] ppid=19909 vsize=4383704 CPUtime=2.98 cores=0-31
/proc/19911/task/19916/stat : 19916 (claspmt) D 19909 19911 19883 0 -1 4202560 27464 0 0 0 22 276 0 0 16 0 32 0 15706977 4488912896 756574 270582939648 4194304 4964800 140734124447728 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 19 0 0 0
[pid=19911/tid=19917] ppid=19909 vsize=4383704 CPUtime=3.02 cores=0-31
/proc/19911/task/19917/stat : 19917 (claspmt) D 19909 19911 19883 0 -1 4202560 26174 0 0 0 18 284 0 0 15 0 32 0 15706977 4488912896 756576 270582939648 4194304 4964800 140734124447728 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=19911/tid=19918] ppid=19909 vsize=4383704 CPUtime=3.02 cores=0-31
/proc/19911/task/19918/stat : 19918 (claspmt) D 19909 19911 19883 0 -1 4202560 27043 0 0 0 20 282 0 0 16 0 32 0 15706977 4488912896 756577 270582939648 4194304 4964800 140734124447728 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=19911/tid=19919] ppid=19909 vsize=4383704 CPUtime=3.03 cores=0-31
/proc/19911/task/19919/stat : 19919 (claspmt) D 19909 19911 19883 0 -1 4202560 28556 0 0 0 18 285 0 0 16 0 32 0 15706977 4488912896 756582 270582939648 4194304 4964800 140734124447728 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=19911/tid=19920] ppid=19909 vsize=4383704 CPUtime=3.05 cores=0-31
/proc/19911/task/19920/stat : 19920 (claspmt) S 19909 19911 19883 0 -1 4202560 26899 0 0 0 26 279 0 0 15 0 32 0 15706977 4488912896 756589 270582939648 4194304 4964800 140734124447728 18446744073709551615 258651060958 0 0 4096 18946 18446744071566932624 0 0 -1 6 0 0 0
[pid=19911/tid=19921] ppid=19909 vsize=4383704 CPUtime=3.06 cores=0-31
/proc/19911/task/19921/stat : 19921 (claspmt) R 19909 19911 19883 0 -1 4202560 26598 0 0 0 18 288 0 0 16 0 32 0 15706977 4488912896 756596 270582939648 4194304 4964800 140734124447728 18446744073709551615 258651061007 0 0 4096 18946 0 0 0 -1 30 0 0 0
[pid=19911/tid=19922] ppid=19909 vsize=4383704 CPUtime=2.86 cores=0-31
/proc/19911/task/19922/stat : 19922 (claspmt) D 19909 19911 19883 0 -1 4202560 29046 0 0 0 20 266 0 0 16 0 32 0 15706977 4488912896 756596 270582939648 4194304 4964800 140734124447728 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 21 0 0 0
[pid=19911/tid=19923] ppid=19909 vsize=4383704 CPUtime=2.88 cores=0-31
/proc/19911/task/19923/stat : 19923 (claspmt) S 19909 19911 19883 0 -1 4202560 29000 0 0 0 16 272 0 0 16 0 32 0 15706978 4488912896 756601 270582939648 4194304 4964800 140734124447728 18446744073709551615 258651060958 0 0 4096 18946 18446744071566925064 0 0 -1 16 0 0 0
[pid=19911/tid=19924] ppid=19909 vsize=4383704 CPUtime=2.99 cores=0-31
/proc/19911/task/19924/stat : 19924 (claspmt) D 19909 19911 19883 0 -1 4202560 24973 0 0 0 16 283 0 0 16 0 32 0 15706978 4488912896 756603 270582939648 4194304 4964800 140734124447728 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 22 0 0 0
[pid=19911/tid=19925] ppid=19909 vsize=4383704 CPUtime=3.03 cores=0-31
/proc/19911/task/19925/stat : 19925 (claspmt) D 19909 19911 19883 0 -1 4202560 24095 0 0 0 17 286 0 0 16 0 32 0 15706978 4488912896 756604 270582939648 4194304 4964800 140734124447728 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 13 0 0 0
[pid=19911/tid=19926] ppid=19909 vsize=4383704 CPUtime=3 cores=0-31
/proc/19911/task/19926/stat : 19926 (claspmt) D 19909 19911 19883 0 -1 4202560 28114 0 0 0 18 282 0 0 16 0 32 0 15706978 4488912896 756605 270582939648 4194304 4964800 140734124447728 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 -1 14 0 0 0
[pid=19911/tid=19927] ppid=19909 vsize=4383704 CPUtime=3 cores=0-31
/proc/19911/task/19927/stat : 19927 (claspmt) D 19909 19911 19883 0 -1 4202560 25672 0 0 0 11 289 0 0 17 0 32 0 15706978 4488912896 756606 270582939648 4194304 4964800 140734124447728 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=19911/tid=19928] ppid=19909 vsize=4383704 CPUtime=2.99 cores=0-31
/proc/19911/task/19928/stat : 19928 (claspmt) D 19909 19911 19883 0 -1 4202560 26637 0 0 0 19 280 0 0 16 0 32 0 15706978 4488912896 756606 270582939648 4194304 4964800 140734124447728 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 -1 28 0 0 0
[pid=19911/tid=19929] ppid=19909 vsize=4383704 CPUtime=2.93 cores=0-31
/proc/19911/task/19929/stat : 19929 (claspmt) R 19909 19911 19883 0 -1 4202560 27808 0 0 0 25 268 0 0 15 0 32 0 15706978 4488912896 756616 270582939648 4194304 4964800 140734124447728 18446744073709551615 4697753 0 0 4096 18946 0 0 0 -1 8 0 0 0
[pid=19911/tid=19930] ppid=19909 vsize=4383704 CPUtime=2.97 cores=0-31
/proc/19911/task/19930/stat : 19930 (claspmt) D 19909 19911 19883 0 -1 4202560 28492 0 0 0 19 278 0 0 16 0 32 0 15706978 4488912896 756618 270582939648 4194304 4964800 140734124447728 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 18 0 0 0
[pid=19911/tid=19931] ppid=19909 vsize=4383704 CPUtime=2.98 cores=0-31
/proc/19911/task/19931/stat : 19931 (claspmt) D 19909 19911 19883 0 -1 4202560 27570 0 0 0 21 277 0 0 16 0 32 0 15706978 4488912896 756622 270582939648 4194304 4964800 140734124447728 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 12 0 0 0
[pid=19911/tid=19932] ppid=19909 vsize=4383704 CPUtime=3.03 cores=0-31
/proc/19911/task/19932/stat : 19932 (claspmt) D 19909 19911 19883 0 -1 4202560 26447 0 0 0 20 283 0 0 15 0 32 0 15706978 4488912896 756626 270582939648 4194304 4964800 140734124447728 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 17 0 0 0
[pid=19911/tid=19933] ppid=19909 vsize=4383704 CPUtime=2.96 cores=0-31
/proc/19911/task/19933/stat : 19933 (claspmt) D 19909 19911 19883 0 -1 4202560 27299 0 0 0 20 276 0 0 16 0 32 0 15706978 4488912896 756628 270582939648 4194304 4964800 140734124447728 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 10 0 0 0
[pid=19911/tid=19934] ppid=19909 vsize=4383704 CPUtime=2.94 cores=0-31
/proc/19911/task/19934/stat : 19934 (claspmt) D 19909 19911 19883 0 -1 4202560 29147 0 0 0 25 269 0 0 16 0 32 0 15706978 4488912896 756631 270582939648 4194304 4964800 140734124447728 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=19911/tid=19935] ppid=19909 vsize=4383704 CPUtime=2.94 cores=0-31
/proc/19911/task/19935/stat : 19935 (claspmt) S 19909 19911 19883 0 -1 4202560 29645 0 0 0 17 277 0 0 16 0 32 0 15706978 4488912896 756631 270582939648 4194304 4964800 140734124447728 18446744073709551615 258651060958 0 0 4096 18946 18446744071566928104 0 0 -1 24 0 0 0
[pid=19911/tid=19936] ppid=19909 vsize=4383704 CPUtime=2.96 cores=0-31
/proc/19911/task/19936/stat : 19936 (claspmt) S 19909 19911 19883 0 -1 4202560 26988 0 0 0 21 275 0 0 15 0 32 0 15706978 4488912896 756634 270582939648 4194304 4964800 140734124447728 18446744073709551615 258651060958 0 0 4096 18946 18446604566920207840 0 0 -1 25 0 0 0
[pid=19911/tid=19937] ppid=19909 vsize=4383704 CPUtime=2.99 cores=0-31
/proc/19911/task/19937/stat : 19937 (claspmt) D 19909 19911 19883 0 -1 4202560 26118 0 0 0 22 277 0 0 16 0 32 0 15706978 4488912896 756640 270582939648 4194304 4964800 140734124447728 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 20 0 0 0
[pid=19911/tid=19938] ppid=19909 vsize=4383704 CPUtime=2.91 cores=0-31
/proc/19911/task/19938/stat : 19938 (claspmt) R 19909 19911 19883 0 -1 4202560 28342 0 0 0 18 273 0 0 16 0 32 0 15706978 4488912896 756644 270582939648 4194304 4964800 140734124447728 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 9 0 0 0
[pid=19911/tid=19939] ppid=19909 vsize=4383704 CPUtime=3.01 cores=0-31
/proc/19911/task/19939/stat : 19939 (claspmt) S 19909 19911 19883 0 -1 4202560 27051 0 0 0 16 285 0 0 15 0 32 0 15706978 4488912896 756651 270582939648 4194304 4964800 140734124447728 18446744073709551615 258651060958 0 0 4096 18946 18446744071566923704 0 0 -1 29 0 0 0
[pid=19911/tid=19940] ppid=19909 vsize=4383704 CPUtime=2.99 cores=0-31
/proc/19911/task/19940/stat : 19940 (claspmt) S 19909 19911 19883 0 -1 4202560 27079 0 0 0 21 278 0 0 15 0 32 0 15706978 4488912896 756651 270582939648 4194304 4964800 140734124447728 18446744073709551615 258651060958 0 0 4096 18946 18446744071566929904 0 0 -1 27 0 0 0
[pid=19911/tid=19941] ppid=19909 vsize=4383704 CPUtime=2.98 cores=0-31
/proc/19911/task/19941/stat : 19941 (claspmt) S 19909 19911 19883 0 -1 4202560 26962 0 0 0 22 276 0 0 15 0 32 0 15706978 4488912896 756654 270582939648 4194304 4964800 140734124447728 18446744073709551615 258651060958 0 0 4096 18946 18446604575232695776 0 0 -1 25 0 0 0
[pid=19911/tid=19942] ppid=19909 vsize=4383704 CPUtime=2.89 cores=0-31
/proc/19911/task/19942/stat : 19942 (claspmt) D 19909 19911 19883 0 -1 4202560 30728 0 0 0 27 262 0 0 16 0 32 0 15706978 4488912896 756655 270582939648 4194304 4964800 140734124447728 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 15 0 0 0
[pid=19911/tid=19943] ppid=19909 vsize=4383704 CPUtime=2.92 cores=0-31
/proc/19911/task/19943/stat : 19943 (claspmt) D 19909 19911 19883 0 -1 4202560 29667 0 0 0 22 270 0 0 16 0 32 0 15706978 4488912896 756657 270582939648 4194304 4964800 140734124447728 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 -1 11 0 0 0
Current children cumulated CPU time (s) 99.35
Current children cumulated vsize (KiB) 4383704

[startup+25.5028 s]
/proc/loadavg: 22.60 29.33 31.08 13/445 19943
/proc/meminfo: memFree=256854352/264257624 swapFree=67111528/67111528
[pid=19911] ppid=19909 vsize=6787900 CPUtime=150.44 cores=0-31
/proc/19911/stat : 19911 (claspmt) S 19909 19911 19883 0 -1 4202496 1742117 0 0 0 3545 11499 0 0 18 0 32 0 15706312 6950809600 1498413 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446744073709551615 0 0 17 25 0 0 0
/proc/19911/statm: 1696975 1498432 430 189 0 1691324 0
[pid=19911/tid=19913] ppid=19909 vsize=6787900 CPUtime=4.46 cores=0-31
/proc/19911/task/19913/stat : 19913 (claspmt) S 19909 19911 19883 0 -1 4202560 49347 0 0 0 76 370 0 0 15 0 32 0 15706977 6950809600 1498435 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604568488173024 0 0 -1 10 0 0 0
[pid=19911/tid=19914] ppid=19909 vsize=6787900 CPUtime=4.73 cores=0-31
/proc/19911/task/19914/stat : 19914 (claspmt) R 19909 19911 19883 0 -1 4202560 48989 0 0 0 87 386 0 0 16 0 32 0 15706977 6950809600 1498440 270582939648 4194304 4964800 140734124447728 18446744073709551615 4697753 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=19911/tid=19915] ppid=19909 vsize=6787900 CPUtime=4.52 cores=0-31
/proc/19911/task/19915/stat : 19915 (claspmt) R 19909 19911 19883 0 -1 4202560 47426 0 0 0 80 372 0 0 15 0 32 0 15706977 6950809600 1498450 270582939648 4194304 4964800 140734124447728 18446744073709551615 4697753 0 0 4096 18946 0 0 0 -1 16 0 0 0
[pid=19911/tid=19916] ppid=19909 vsize=6787900 CPUtime=4.42 cores=0-31
/proc/19911/task/19916/stat : 19916 (claspmt) S 19909 19911 19883 0 -1 4202560 49104 0 0 0 79 363 0 0 15 0 32 0 15706977 6950809600 1498456 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604642166930912 0 0 -1 19 0 0 0
[pid=19911/tid=19917] ppid=19909 vsize=6787900 CPUtime=4.56 cores=0-31
/proc/19911/task/19917/stat : 19917 (claspmt) D 19909 19911 19883 0 -1 4202560 51369 0 0 0 79 377 0 0 16 0 32 0 15706977 6950809600 1498461 270582939648 4194304 4964800 140734124447728 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 29 0 0 0
[pid=19911/tid=19918] ppid=19909 vsize=6787900 CPUtime=4.72 cores=0-31
/proc/19911/task/19918/stat : 19918 (claspmt) D 19909 19911 19883 0 -1 4202560 51181 0 0 0 95 377 0 0 16 0 32 0 15706977 6950809600 1498469 270582939648 4194304 4964800 140734124447728 18446744073709551615 258650652667 0 0 4096 18946 0 0 0 -1 12 0 0 0
[pid=19911/tid=19919] ppid=19909 vsize=6787900 CPUtime=4.64 cores=0-31
/proc/19911/task/19919/stat : 19919 (claspmt) R 19909 19911 19883 0 -1 4202560 52587 0 0 0 92 372 0 0 16 0 32 0 15706977 6950809600 1498477 270582939648 4194304 4964800 140734124447728 18446744073709551615 4652185 0 0 4096 18946 0 0 0 -1 13 0 0 0
[pid=19911/tid=19920] ppid=19909 vsize=6787900 CPUtime=4.55 cores=0-31
/proc/19911/task/19920/stat : 19920 (claspmt) S 19909 19911 19883 0 -1 4202560 49636 0 0 0 85 370 0 0 16 0 32 0 15706977 6950809600 1498480 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604568146025952 0 0 -1 6 0 0 0
[pid=19911/tid=19921] ppid=19909 vsize=6787900 CPUtime=4.49 cores=0-31
/proc/19911/task/19921/stat : 19921 (claspmt) S 19909 19911 19883 0 -1 4202560 49128 0 0 0 73 376 0 0 16 0 32 0 15706977 6950809600 1498488 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604563573325280 0 0 -1 30 0 0 0
[pid=19911/tid=19922] ppid=19909 vsize=6787900 CPUtime=4.79 cores=0-31
/proc/19911/task/19922/stat : 19922 (claspmt) D 19909 19911 19883 0 -1 4202560 54756 0 0 0 126 353 0 0 16 0 32 0 15706977 6950809600 1498497 270582939648 4194304 4964800 140734124447728 18446744073709551615 4375558 0 0 4096 18946 0 0 0 -1 11 0 0 0
[pid=19911/tid=19923] ppid=19909 vsize=6787900 CPUtime=4.88 cores=0-31
/proc/19911/task/19923/stat : 19923 (claspmt) D 19909 19911 19883 0 -1 4202560 54628 0 0 0 127 361 0 0 16 0 32 0 15706978 6950809600 1498497 270582939648 4194304 4964800 140734124447728 18446744073709551615 4376558 0 0 4096 18946 0 0 0 -1 28 0 0 0
[pid=19911/tid=19924] ppid=19909 vsize=6787900 CPUtime=4.51 cores=0-31
/proc/19911/task/19924/stat : 19924 (claspmt) S 19909 19911 19883 0 -1 4202560 48824 0 0 0 77 374 0 0 16 0 32 0 15706978 6950809600 1498502 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604563046587872 0 0 -1 20 0 0 0
[pid=19911/tid=19925] ppid=19909 vsize=6787900 CPUtime=4.61 cores=0-31
/proc/19911/task/19925/stat : 19925 (claspmt) D 19909 19911 19883 0 -1 4202560 47845 0 0 0 83 378 0 0 16 0 32 0 15706978 6950809600 1498507 270582939648 4194304 4964800 140734124447728 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 -1 17 0 0 0
[pid=19911/tid=19926] ppid=19909 vsize=6787900 CPUtime=4.75 cores=0-31
/proc/19911/task/19926/stat : 19926 (claspmt) D 19909 19911 19883 0 -1 4202560 53001 0 0 0 101 374 0 0 16 0 32 0 15706978 6950809600 1498510 270582939648 4194304 4964800 140734124447728 18446744073709551615 4376558 0 0 4096 18946 0 0 0 -1 26 0 0 0
[pid=19911/tid=19927] ppid=19909 vsize=6787900 CPUtime=4.59 cores=0-31
/proc/19911/task/19927/stat : 19927 (claspmt) D 19909 19911 19883 0 -1 4202560 49111 0 0 0 79 380 0 0 15 0 32 0 15706978 6950809600 1498514 270582939648 4194304 4964800 140734124447728 18446744073709551615 4697753 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=19911/tid=19928] ppid=19909 vsize=6787900 CPUtime=4.39 cores=0-31
/proc/19911/task/19928/stat : 19928 (claspmt) S 19909 19911 19883 0 -1 4202560 49124 0 0 0 77 362 0 0 16 0 32 0 15706978 6950809600 1498514 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604567811169760 0 0 -1 16 0 0 0

################
# More data... #
################

[pid=19911/tid=19920] ppid=19909 vsize=7440476 CPUtime=4.55 cores=0-31
/proc/19911/task/19920/stat : 19920 (claspmt) S 19909 19911 19883 0 -1 4202560 49636 0 0 0 85 370 0 0 16 0 32 0 15706977 7619047424 1626194 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604568146025952 0 0 -1 6 0 0 0
[pid=19911/tid=19921] ppid=19909 vsize=7440476 CPUtime=4.49 cores=0-31
/proc/19911/task/19921/stat : 19921 (claspmt) S 19909 19911 19883 0 -1 4202560 49128 0 0 0 73 376 0 0 16 0 32 0 15706977 7619047424 1626194 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604563573325280 0 0 -1 30 0 0 0
[pid=19911/tid=19922] ppid=19909 vsize=7440476 CPUtime=5.71 cores=0-31
/proc/19911/task/19922/stat : 19922 (claspmt) S 19909 19911 19883 0 -1 4202560 62918 0 0 0 210 361 0 0 17 0 32 0 15706977 7619047424 1626194 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604565863316960 0 0 -1 11 0 0 0
[pid=19911/tid=19923] ppid=19909 vsize=7440476 CPUtime=5.81 cores=0-31
/proc/19911/task/19923/stat : 19923 (claspmt) S 19909 19911 19883 0 -1 4202560 62714 0 0 0 211 370 0 0 17 0 32 0 15706978 7619047424 1626194 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604574952308192 0 0 -1 28 0 0 0
[pid=19911/tid=19924] ppid=19909 vsize=7440476 CPUtime=4.51 cores=0-31
/proc/19911/task/19924/stat : 19924 (claspmt) S 19909 19911 19883 0 -1 4202560 48824 0 0 0 77 374 0 0 16 0 32 0 15706978 7619047424 1626194 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604563046587872 0 0 -1 20 0 0 0
[pid=19911/tid=19925] ppid=19909 vsize=7440476 CPUtime=5060.15 cores=0-31
/proc/19911/task/19925/stat : 19925 (claspmt) R 19909 19911 19883 0 -1 4202560 60978 0 0 0 505617 398 0 0 25 0 32 0 15706978 7619047424 1626194 270582939648 4194304 4964800 140734124447728 18446744073709551615 4342167 0 0 4096 18946 0 0 0 -1 15 0 0 0
[pid=19911/tid=19926] ppid=19909 vsize=7440476 CPUtime=5.86 cores=0-31
/proc/19911/task/19926/stat : 19926 (claspmt) S 19909 19911 19883 0 -1 4202560 58517 0 0 0 207 379 0 0 17 0 32 0 15706978 7619047424 1626194 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604559170526688 0 0 -1 26 0 0 0
[pid=19911/tid=19927] ppid=19909 vsize=7440476 CPUtime=6.73 cores=0-31
/proc/19911/task/19927/stat : 19927 (claspmt) S 19909 19911 19883 0 -1 4202560 54307 0 0 0 270 403 0 0 18 0 32 0 15706978 7619047424 1626194 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604573253369312 0 0 -1 15 0 0 0
[pid=19911/tid=19928] ppid=19909 vsize=7440476 CPUtime=4.39 cores=0-31
/proc/19911/task/19928/stat : 19928 (claspmt) S 19909 19911 19883 0 -1 4202560 49124 0 0 0 77 362 0 0 16 0 32 0 15706978 7619047424 1626194 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604567811169760 0 0 -1 16 0 0 0
[pid=19911/tid=19929] ppid=19909 vsize=7440476 CPUtime=4.42 cores=0-31
/proc/19911/task/19929/stat : 19929 (claspmt) S 19909 19911 19883 0 -1 4202560 49466 0 0 0 85 357 0 0 16 0 32 0 15706978 7619047424 1626194 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604568391540192 0 0 -1 4 0 0 0
[pid=19911/tid=19930] ppid=19909 vsize=7440476 CPUtime=5.86 cores=0-31
/proc/19911/task/19930/stat : 19930 (claspmt) S 19909 19911 19883 0 -1 4202560 63262 0 0 0 207 379 0 0 17 0 32 0 15706978 7619047424 1626194 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604559170534880 0 0 -1 22 0 0 0
[pid=19911/tid=19931] ppid=19909 vsize=7440476 CPUtime=5.94 cores=0-31
/proc/19911/task/19931/stat : 19931 (claspmt) S 19909 19911 19883 0 -1 4202560 62485 0 0 0 223 371 0 0 17 0 32 0 15706978 7619047424 1626194 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604568488181216 0 0 -1 7 0 0 0
[pid=19911/tid=19932] ppid=19909 vsize=7440476 CPUtime=4.47 cores=0-31
/proc/19911/task/19932/stat : 19932 (claspmt) S 19909 19911 19883 0 -1 4202560 49464 0 0 0 77 370 0 0 16 0 32 0 15706978 7619047424 1626194 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604575232695776 0 0 -1 5 0 0 0
[pid=19911/tid=19933] ppid=19909 vsize=7440476 CPUtime=5060.56 cores=0-31
/proc/19911/task/19933/stat : 19933 (claspmt) R 19909 19911 19883 0 -1 4202560 61360 0 0 0 505678 378 0 0 25 0 32 0 15706978 7619047424 1626194 270582939648 4194304 4964800 140734124447728 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 18 0 0 0
[pid=19911/tid=19934] ppid=19909 vsize=7440476 CPUtime=6 cores=0-31
/proc/19911/task/19934/stat : 19934 (claspmt) S 19909 19911 19883 0 -1 4202560 62621 0 0 0 228 372 0 0 17 0 32 0 15706978 7619047424 1626194 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604574978162144 0 0 -1 15 0 0 0
[pid=19911/tid=19935] ppid=19909 vsize=7440476 CPUtime=6.69 cores=0-31
/proc/19911/task/19935/stat : 19935 (claspmt) S 19909 19911 19883 0 -1 4202560 57170 0 0 0 286 383 0 0 18 0 32 0 15706978 7619047424 1626194 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604568488164832 0 0 -1 24 0 0 0
[pid=19911/tid=19936] ppid=19909 vsize=7440476 CPUtime=4.39 cores=0-31
/proc/19911/task/19936/stat : 19936 (claspmt) S 19909 19911 19883 0 -1 4202560 49291 0 0 0 75 364 0 0 16 0 32 0 15706978 7619047424 1626194 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604566920207840 0 0 -1 17 0 0 0
[pid=19911/tid=19937] ppid=19909 vsize=7440476 CPUtime=4.48 cores=0-31
/proc/19911/task/19937/stat : 19937 (claspmt) S 19909 19911 19883 0 -1 4202560 49325 0 0 0 80 368 0 0 16 0 32 0 15706978 7619047424 1626194 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604561145241056 0 0 -1 8 0 0 0
[pid=19911/tid=19938] ppid=19909 vsize=7440476 CPUtime=5.75 cores=0-31
/proc/19911/task/19938/stat : 19938 (claspmt) S 19909 19911 19883 0 -1 4202560 61127 0 0 0 208 367 0 0 17 0 32 0 15706978 7619047424 1626194 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604572111085024 0 0 -1 10 0 0 0
[pid=19911/tid=19939] ppid=19909 vsize=7440476 CPUtime=5.77 cores=0-31
/proc/19911/task/19939/stat : 19939 (claspmt) S 19909 19911 19883 0 -1 4202560 59105 0 0 0 202 375 0 0 17 0 32 0 15706978 7619047424 1626194 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604558245821920 0 0 -1 25 0 0 0
[pid=19911/tid=19940] ppid=19909 vsize=7440476 CPUtime=4.41 cores=0-31
/proc/19911/task/19940/stat : 19940 (claspmt) S 19909 19911 19883 0 -1 4202560 49431 0 0 0 74 367 0 0 15 0 32 0 15706978 7619047424 1626194 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604559857237472 0 0 -1 27 0 0 0
[pid=19911/tid=19941] ppid=19909 vsize=7440476 CPUtime=5060.25 cores=0-31
/proc/19911/task/19941/stat : 19941 (claspmt) R 19909 19911 19883 0 -1 4202560 62376 0 0 0 505640 385 0 0 25 0 32 0 15706978 7619047424 1626194 270582939648 4194304 4964800 140734124447728 18446744073709551615 4669574 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=19911/tid=19942] ppid=19909 vsize=7440476 CPUtime=5.74 cores=0-31
/proc/19911/task/19942/stat : 19942 (claspmt) S 19909 19911 19883 0 -1 4202560 63151 0 0 0 215 359 0 0 17 0 32 0 15706978 7619047424 1626194 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604561145257440 0 0 -1 7 0 0 0
[pid=19911/tid=19943] ppid=19909 vsize=7440476 CPUtime=6.7 cores=0-31
/proc/19911/task/19943/stat : 19943 (claspmt) S 19909 19911 19883 0 -1 4202560 56972 0 0 0 293 377 0 0 18 0 32 0 15706978 7619047424 1626194 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604563573317088 0 0 -1 20 0 0 0
Current children cumulated CPU time (s) 20394.3
Current children cumulated vsize (KiB) 7440476



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5100 s]
/proc/loadavg: 4.02 4.02 4.12 5/445 20092
/proc/meminfo: memFree=256241620/264257624 swapFree=67111528/67111528
[pid=19911] ppid=19909 vsize=7441500 CPUtime=20465.2 cores=0-31
/proc/19911/stat : 19911 (claspmt) S 19909 19911 19883 0 -1 4202496 1895473 0 0 0 2034815 11705 0 0 18 0 32 0 15706312 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446744073709551615 0 0 17 25 0 0 0
/proc/19911/statm: 1860375 1626411 434 189 0 1854724 0
[pid=19911/tid=19913] ppid=19909 vsize=7441500 CPUtime=4.46 cores=0-31
/proc/19911/task/19913/stat : 19913 (claspmt) S 19909 19911 19883 0 -1 4202560 49347 0 0 0 76 370 0 0 15 0 32 0 15706977 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604568488173024 0 0 -1 10 0 0 0
[pid=19911/tid=19914] ppid=19909 vsize=7441500 CPUtime=6.08 cores=0-31
/proc/19911/task/19914/stat : 19914 (claspmt) S 19909 19911 19883 0 -1 4202560 58826 0 0 0 215 393 0 0 17 0 32 0 15706977 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604561308974560 0 0 -1 26 0 0 0
[pid=19911/tid=19915] ppid=19909 vsize=7441500 CPUtime=5.93 cores=0-31
/proc/19911/task/19915/stat : 19915 (claspmt) S 19909 19911 19883 0 -1 4202560 56599 0 0 0 212 381 0 0 17 0 32 0 15706977 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604565783338464 0 0 -1 26 0 0 0
[pid=19911/tid=19916] ppid=19909 vsize=7441500 CPUtime=4.42 cores=0-31
/proc/19911/task/19916/stat : 19916 (claspmt) S 19909 19911 19883 0 -1 4202560 49104 0 0 0 79 363 0 0 15 0 32 0 15706977 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604642166930912 0 0 -1 19 0 0 0
[pid=19911/tid=19917] ppid=19909 vsize=7441500 CPUtime=5077.91 cores=0-31
/proc/19911/task/19917/stat : 19917 (claspmt) R 19909 19911 19883 0 -1 4202560 57599 0 0 0 507405 386 0 0 25 0 32 0 15706977 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 4668592 0 0 4096 18946 0 0 0 -1 29 0 0 0
[pid=19911/tid=19918] ppid=19909 vsize=7441500 CPUtime=5.99 cores=0-31
/proc/19911/task/19918/stat : 19918 (claspmt) S 19909 19911 19883 0 -1 4202560 60632 0 0 0 214 385 0 0 17 0 32 0 15706977 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604568391523808 0 0 -1 10 0 0 0
[pid=19911/tid=19919] ppid=19909 vsize=7441500 CPUtime=6.65 cores=0-31
/proc/19911/task/19919/stat : 19919 (claspmt) S 19909 19911 19883 0 -1 4202560 56514 0 0 0 275 390 0 0 18 0 32 0 15706977 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604574952316384 0 0 -1 11 0 0 0
[pid=19911/tid=19920] ppid=19909 vsize=7441500 CPUtime=4.55 cores=0-31
/proc/19911/task/19920/stat : 19920 (claspmt) S 19909 19911 19883 0 -1 4202560 49636 0 0 0 85 370 0 0 16 0 32 0 15706977 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604568146025952 0 0 -1 6 0 0 0
[pid=19911/tid=19921] ppid=19909 vsize=7441500 CPUtime=4.49 cores=0-31
/proc/19911/task/19921/stat : 19921 (claspmt) S 19909 19911 19883 0 -1 4202560 49128 0 0 0 73 376 0 0 16 0 32 0 15706977 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604563573325280 0 0 -1 30 0 0 0
[pid=19911/tid=19922] ppid=19909 vsize=7441500 CPUtime=5.71 cores=0-31
/proc/19911/task/19922/stat : 19922 (claspmt) S 19909 19911 19883 0 -1 4202560 62918 0 0 0 210 361 0 0 17 0 32 0 15706977 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604565863316960 0 0 -1 11 0 0 0
[pid=19911/tid=19923] ppid=19909 vsize=7441500 CPUtime=5.81 cores=0-31
/proc/19911/task/19923/stat : 19923 (claspmt) S 19909 19911 19883 0 -1 4202560 62714 0 0 0 211 370 0 0 17 0 32 0 15706978 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604574952308192 0 0 -1 28 0 0 0
[pid=19911/tid=19924] ppid=19909 vsize=7441500 CPUtime=4.51 cores=0-31
/proc/19911/task/19924/stat : 19924 (claspmt) S 19909 19911 19883 0 -1 4202560 48824 0 0 0 77 374 0 0 16 0 32 0 15706978 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604563046587872 0 0 -1 20 0 0 0
[pid=19911/tid=19925] ppid=19909 vsize=7441500 CPUtime=5077.87 cores=0-31
/proc/19911/task/19925/stat : 19925 (claspmt) R 19909 19911 19883 0 -1 4202560 61038 0 0 0 507389 398 0 0 25 0 32 0 15706978 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 4668900 0 0 4096 18946 0 0 0 -1 15 0 0 0
[pid=19911/tid=19926] ppid=19909 vsize=7441500 CPUtime=5.86 cores=0-31
/proc/19911/task/19926/stat : 19926 (claspmt) S 19909 19911 19883 0 -1 4202560 58517 0 0 0 207 379 0 0 17 0 32 0 15706978 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604559170526688 0 0 -1 26 0 0 0
[pid=19911/tid=19927] ppid=19909 vsize=7441500 CPUtime=6.73 cores=0-31
/proc/19911/task/19927/stat : 19927 (claspmt) S 19909 19911 19883 0 -1 4202560 54307 0 0 0 270 403 0 0 18 0 32 0 15706978 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604573253369312 0 0 -1 15 0 0 0
[pid=19911/tid=19928] ppid=19909 vsize=7441500 CPUtime=4.39 cores=0-31
/proc/19911/task/19928/stat : 19928 (claspmt) S 19909 19911 19883 0 -1 4202560 49124 0 0 0 77 362 0 0 16 0 32 0 15706978 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604567811169760 0 0 -1 16 0 0 0
[pid=19911/tid=19929] ppid=19909 vsize=7441500 CPUtime=4.42 cores=0-31
/proc/19911/task/19929/stat : 19929 (claspmt) S 19909 19911 19883 0 -1 4202560 49466 0 0 0 85 357 0 0 16 0 32 0 15706978 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604568391540192 0 0 -1 4 0 0 0
[pid=19911/tid=19930] ppid=19909 vsize=7441500 CPUtime=5.86 cores=0-31
/proc/19911/task/19930/stat : 19930 (claspmt) S 19909 19911 19883 0 -1 4202560 63262 0 0 0 207 379 0 0 17 0 32 0 15706978 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604559170534880 0 0 -1 22 0 0 0
[pid=19911/tid=19931] ppid=19909 vsize=7441500 CPUtime=5.94 cores=0-31
/proc/19911/task/19931/stat : 19931 (claspmt) S 19909 19911 19883 0 -1 4202560 62485 0 0 0 223 371 0 0 17 0 32 0 15706978 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604568488181216 0 0 -1 7 0 0 0
[pid=19911/tid=19932] ppid=19909 vsize=7441500 CPUtime=4.47 cores=0-31
/proc/19911/task/19932/stat : 19932 (claspmt) S 19909 19911 19883 0 -1 4202560 49464 0 0 0 77 370 0 0 16 0 32 0 15706978 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604575232695776 0 0 -1 5 0 0 0
[pid=19911/tid=19933] ppid=19909 vsize=7441500 CPUtime=5078.27 cores=0-31
/proc/19911/task/19933/stat : 19933 (claspmt) R 19909 19911 19883 0 -1 4202560 61420 0 0 0 507449 378 0 0 25 0 32 0 15706978 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 18 0 0 0
[pid=19911/tid=19934] ppid=19909 vsize=7441500 CPUtime=6 cores=0-31
/proc/19911/task/19934/stat : 19934 (claspmt) S 19909 19911 19883 0 -1 4202560 62621 0 0 0 228 372 0 0 17 0 32 0 15706978 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604574978162144 0 0 -1 15 0 0 0
[pid=19911/tid=19935] ppid=19909 vsize=7441500 CPUtime=6.69 cores=0-31
/proc/19911/task/19935/stat : 19935 (claspmt) S 19909 19911 19883 0 -1 4202560 57170 0 0 0 286 383 0 0 18 0 32 0 15706978 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604568488164832 0 0 -1 24 0 0 0
[pid=19911/tid=19936] ppid=19909 vsize=7441500 CPUtime=4.39 cores=0-31
/proc/19911/task/19936/stat : 19936 (claspmt) S 19909 19911 19883 0 -1 4202560 49291 0 0 0 75 364 0 0 16 0 32 0 15706978 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604566920207840 0 0 -1 17 0 0 0
[pid=19911/tid=19937] ppid=19909 vsize=7441500 CPUtime=4.48 cores=0-31
/proc/19911/task/19937/stat : 19937 (claspmt) S 19909 19911 19883 0 -1 4202560 49325 0 0 0 80 368 0 0 16 0 32 0 15706978 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604561145241056 0 0 -1 8 0 0 0
[pid=19911/tid=19938] ppid=19909 vsize=7441500 CPUtime=5.75 cores=0-31
/proc/19911/task/19938/stat : 19938 (claspmt) S 19909 19911 19883 0 -1 4202560 61127 0 0 0 208 367 0 0 17 0 32 0 15706978 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604572111085024 0 0 -1 10 0 0 0
[pid=19911/tid=19939] ppid=19909 vsize=7441500 CPUtime=5.77 cores=0-31
/proc/19911/task/19939/stat : 19939 (claspmt) S 19909 19911 19883 0 -1 4202560 59105 0 0 0 202 375 0 0 17 0 32 0 15706978 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604558245821920 0 0 -1 25 0 0 0
[pid=19911/tid=19940] ppid=19909 vsize=7441500 CPUtime=4.41 cores=0-31
/proc/19911/task/19940/stat : 19940 (claspmt) S 19909 19911 19883 0 -1 4202560 49431 0 0 0 74 367 0 0 15 0 32 0 15706978 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604559857237472 0 0 -1 27 0 0 0
[pid=19911/tid=19941] ppid=19909 vsize=7441500 CPUtime=5077.97 cores=0-31
/proc/19911/task/19941/stat : 19941 (claspmt) R 19909 19911 19883 0 -1 4202560 62384 0 0 0 507412 385 0 0 25 0 32 0 15706978 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=19911/tid=19942] ppid=19909 vsize=7441500 CPUtime=5.74 cores=0-31
/proc/19911/task/19942/stat : 19942 (claspmt) S 19909 19911 19883 0 -1 4202560 63151 0 0 0 215 359 0 0 17 0 32 0 15706978 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604561145257440 0 0 -1 7 0 0 0
[pid=19911/tid=19943] ppid=19909 vsize=7441500 CPUtime=6.7 cores=0-31
/proc/19911/task/19943/stat : 19943 (claspmt) S 19909 19911 19883 0 -1 4202560 56972 0 0 0 293 377 0 0 18 0 32 0 15706978 7620096000 1626411 270582939648 4194304 4964800 140734124447728 18446744073709551615 258662772457 0 0 4096 18946 18446604563573317088 0 0 -1 20 0 0 0
Current children cumulated CPU time (s) 20465.2
Current children cumulated vsize (KiB) 7441500

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)

Sending SIGKILL to -19911

Solver just ended. Dumping a history of the last processes samples

[startup+5100.6 s]
/proc/loadavg: 4.02 4.02 4.12 2/415 20093
/proc/meminfo: memFree=256409012/264257624 swapFree=67111528/67111528
[pid=19911] ppid=19909 vsize=7162184 CPUtime=20465.9 cores=0-31
/proc/19911/stat : 19911 (claspmt) R 19909 19911 19883 0 -1 4202496 1895485 0 0 0 2034880 11707 0 0 18 0 1 0 15706312 7334076416 1579310 270582939648 4194304 4964800 140734124447728 18446744073709551615 4628834 0 0 4096 18946 0 0 0 17 25 0 0 0
/proc/19911/statm: 1790546 1579310 445 189 0 1784895 0
Current children cumulated CPU time (s) 20465.9
Current children cumulated vsize (KiB) 7162184

[startup+5101.4 s]
/proc/loadavg: 4.02 4.02 4.12 2/415 20093
/proc/meminfo: memFree=256409012/264257624 swapFree=67111528/67111528
[pid=19911] ppid=19909 vsize=6946924 CPUtime=20466.6 cores=0-31
/proc/19911/stat : 19911 (claspmt) R 19909 19911 19883 0 -1 4202496 1895485 0 0 0 2034956 11708 0 0 19 0 1 0 15706312 7113650176 1536771 270582939648 4194304 4964800 140734124447728 18446744073709551615 4628834 0 0 4096 18946 0 0 0 17 25 0 0 0
/proc/19911/statm: 1736731 1536771 445 189 0 1731080 0
Current children cumulated CPU time (s) 20466.6
Current children cumulated vsize (KiB) 6946924

[startup+5102.2 s]
/proc/loadavg: 4.02 4.02 4.12 2/415 20095
/proc/meminfo: memFree=256662212/264257624 swapFree=67111528/67111528
[pid=19911] ppid=19909 vsize=0 CPUtime=20467.4 cores=0-31
/proc/19911/stat : 19911 (claspmt) R 19909 19911 19883 0 -1 4203524 1895485 0 0 0 2035018 11726 0 0 20 0 1 0 15706312 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 25 0 0 0
/proc/19911/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 20467.4
Current children cumulated vsize (KiB) 0

[startup+5102.4 s]
/proc/loadavg: 4.02 4.02 4.12 2/415 20095
/proc/meminfo: memFree=256662212/264257624 swapFree=67111528/67111528
[pid=19911] ppid=19909 vsize=0 CPUtime=20467.6 cores=0-31
/proc/19911/stat : 19911 (claspmt) R 19909 19911 19883 0 -1 4203524 1895485 0 0 0 2035018 11746 0 0 21 0 1 0 15706312 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 25 0 0 0
/proc/19911/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 20467.6
Current children cumulated vsize (KiB) 0

Child ended because it received signal 9 (SIGKILL)
Real time (s): 5102.46
CPU time (s): 20467.7
CPU user time (s): 20350.2
CPU system time (s): 117.517
CPU usage (%): 401.134
Max. virtual memory (cumulated for all children) (KiB): 7478340

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 20350.2
system time used= 117.517
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1895485
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2172346
involuntary context switches= 86193

runsolver used 26.9009 second user time and 112.261 second system time

The end

Launcher Data 

Begin job on node200 at 2011-06-08 05:19:54
IDJOB=3472344
IDBENCH=71686
IDSOLVER=1961
FILE ID=node200/3472344-1307503181
RUNJOBID= node200-1307503181-19897
PBS_JOBID= 13509204
Free space on /tmp= 61436 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT_RACE06/velev-npe-1.0-9dlx-b71.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3472344-1307503181/watcher-3472344-1307503181 -o /tmp/evaluation-result-3472344-1307503181/solver-3472344-1307503181 -C 160000 -W 5100 -M 256000  HOME/claspmt -t32 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3472344-1307503181.cnf

TIME LIMIT= 160000 seconds
MEMORY LIMIT= 256000 MiB
NBCORE= 32

MD5SUM BENCH= 75d91f0ba4398ccb5b452707c0e7fa27
RANDOM SEED=1833311348

node200.alineos.net Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 64
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 96
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 66
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 98
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.99
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 68
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 100
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.03
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 70
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 102
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 16
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 17
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 48
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 18
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 80
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 19
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 112
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.97
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 20
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 21
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 50
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.10
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 22
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 82
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.10
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 114
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 24
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 25
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 26
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 84
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 27
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 116
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 28
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 29
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 54
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 30
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 86
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 31
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 118
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     264257624 kB
MemFree:      262756840 kB
Buffers:        158796 kB
Cached:        1072140 kB
SwapCached:          0 kB
Active:         503032 kB
Inactive:       791336 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     264257624 kB
LowFree:      262756840 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:          317408 kB
Writeback:           0 kB
AnonPages:       63384 kB
Mapped:          14476 kB
Slab:           127408 kB
PageTables:       4452 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  199240340 kB
Committed_AS:   159444 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296372 kB
VmallocChunk: 34359440603 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 61432 MiB
End job on node200 at 2011-06-08 06:44:57