Trace number 3472509

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 153028 5102.73

General information on the benchmark

NameSAT03/industrial/schuppan/l2s/
dme-03-1-k247-unsat.shuffled-as.sat03-407.cnf
MD5SUM3fef216b3efe77c0f10ea0e0be664a94
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables261352
Number of clauses773077
Sum of the clauses size1985588
Maximum clause length248
Minimum clause length1
Number of clauses of size 11592
Number of clauses of size 2357488
Number of clauses of size 3413748
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 5249

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...809051/instance-3472509-1307809051.cnf
5.59/5.62	c Solving...
153024.61/5100.02	
153024.61/5100.02	*** INTERRUPTED! ***
153024.61/5100.02	*** Shutting down threads...

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 988)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3472509-1307809051/watcher-3472509-1307809051 -o /tmp/evaluation-result-3472509-1307809051/solver-3472509-1307809051 -C 160000 -W 5100 -M 256000 HOME/claspmt -t32 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3472509-1307809051.cnf 

running on 32 cores: 0,4,8,12,16,20,24,28,1,5,9,13,17,21,25,29,2,6,10,14,18,22,26,30,3,7,11,15,19,23,27,31

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 160000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 160030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 262144000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 262195200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 26.82 27.14 28.60 1/413 10815
/proc/meminfo: memFree=262029568/264257624 swapFree=67111528/67111528
[pid=10815] ppid=10813 vsize=54060 CPUtime=0.02 cores=0-31
/proc/10815/stat : 10815 (claspmt) R 10813 10815 10706 0 -1 4202496 4453 0 0 0 2 0 0 0 22 0 1 0 46292130 55357440 4383 270582939648 4194304 4964800 140736730161280 18446744073709551615 4700412 0 0 4096 18946 0 0 0 17 20 0 0 0
/proc/10815/statm: 13515 4390 373 189 0 8389 0

[startup+0.067261 s]
/proc/loadavg: 26.82 27.14 28.60 1/413 10815
/proc/meminfo: memFree=262029568/264257624 swapFree=67111528/67111528
[pid=10815] ppid=10813 vsize=69380 CPUtime=0.06 cores=0-31
/proc/10815/stat : 10815 (claspmt) R 10813 10815 10706 0 -1 4202496 10532 0 0 0 4 2 0 0 22 0 1 0 46292130 71045120 10081 270582939648 4194304 4964800 140736730161280 18446744073709551615 4679094 0 0 4096 18946 0 0 0 17 20 0 0 0
/proc/10815/statm: 17345 10092 375 189 0 12219 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 69380

[startup+0.101268 s]
/proc/loadavg: 26.82 27.14 28.60 1/413 10815
/proc/meminfo: memFree=262029568/264257624 swapFree=67111528/67111528
[pid=10815] ppid=10813 vsize=74388 CPUtime=0.09 cores=0-31
/proc/10815/stat : 10815 (claspmt) R 10813 10815 10706 0 -1 4202496 15771 0 0 0 6 3 0 0 22 0 1 0 46292130 76173312 13570 270582939648 4194304 4964800 140736730161280 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 17 20 0 0 0
/proc/10815/statm: 18597 13572 386 189 0 13471 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 74388

[startup+0.301269 s]
/proc/loadavg: 26.82 27.14 28.60 1/413 10815
/proc/meminfo: memFree=262029568/264257624 swapFree=67111528/67111528
[pid=10815] ppid=10813 vsize=99072 CPUtime=0.29 cores=0-31
/proc/10815/stat : 10815 (claspmt) R 10813 10815 10706 0 -1 4202496 24196 0 0 0 24 5 0 0 23 0 1 0 46292130 101449728 19415 270582939648 4194304 4964800 140736730161280 18446744073709551615 4599599 0 0 4096 18946 0 0 0 17 20 0 0 0
/proc/10815/statm: 24768 19416 386 189 0 19642 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 99072

[startup+0.700297 s]
/proc/loadavg: 26.82 27.14 28.60 1/413 10815
/proc/meminfo: memFree=262029568/264257624 swapFree=67111528/67111528
[pid=10815] ppid=10813 vsize=143580 CPUtime=0.69 cores=0-31
/proc/10815/stat : 10815 (claspmt) R 10813 10815 10706 0 -1 4202496 38727 0 0 0 60 9 0 0 25 0 1 0 46292130 147025920 29962 270582939648 4194304 4964800 140736730161280 18446744073709551615 4625076 0 0 4096 18946 0 0 0 17 20 0 0 0
/proc/10815/statm: 35895 29962 397 189 0 30769 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 143580

[startup+1.50031 s]
/proc/loadavg: 26.82 27.14 28.60 2/414 10816
/proc/meminfo: memFree=261905420/264257624 swapFree=67111528/67111528
[pid=10815] ppid=10813 vsize=152848 CPUtime=1.49 cores=0-31
/proc/10815/stat : 10815 (claspmt) R 10813 10815 10706 0 -1 4202496 40905 0 0 0 140 9 0 0 25 0 1 0 46292130 156516352 32140 270582939648 4194304 4964800 140736730161280 18446744073709551615 4607641 0 0 4096 18946 0 0 0 17 20 0 0 0
/proc/10815/statm: 38212 32140 400 189 0 33086 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 152848

[startup+3.10041 s]
/proc/loadavg: 26.82 27.14 28.60 2/414 10816
/proc/meminfo: memFree=261901320/264257624 swapFree=67111528/67111528
[pid=10815] ppid=10813 vsize=160240 CPUtime=3.08 cores=0-31
/proc/10815/stat : 10815 (claspmt) R 10813 10815 10706 0 -1 4202496 42775 0 0 0 299 9 0 0 25 0 1 0 46292130 164085760 34010 270582939648 4194304 4964800 140736730161280 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 17 20 0 0 0
/proc/10815/statm: 40060 34010 401 189 0 34934 0
Current children cumulated CPU time (s) 3.08
Current children cumulated vsize (KiB) 160240

[startup+6.30035 s]
/proc/loadavg: 24.75 26.70 28.45 2/414 10816
/proc/meminfo: memFree=261877800/264257624 swapFree=67111528/67111528
[pid=10815] ppid=10813 vsize=953996 CPUtime=24.11 cores=0-31
/proc/10815/stat : 10815 (claspmt) S 10813 10815 10706 0 -1 4202496 189214 0 0 0 615 1796 0 0 18 0 32 0 46292130 976891904 130702 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446744073709551615 0 0 17 10 0 0 0
/proc/10815/statm: 238499 130723 434 189 0 232848 0
Current children cumulated CPU time (s) 24.11
Current children cumulated vsize (KiB) 953996

[startup+12.7017 s]
/proc/loadavg: 23.09 26.33 28.32 5/445 10847
/proc/meminfo: memFree=261027312/264257624 swapFree=67111528/67111528
[pid=10815] ppid=10813 vsize=2171992 CPUtime=54.71 cores=0-31
/proc/10815/stat : 10815 (claspmt) S 10813 10815 10706 0 -1 4202496 327839 0 0 0 3007 2464 0 0 18 0 32 0 46292130 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446744073709551615 0 0 17 10 0 0 0
/proc/10815/statm: 542998 250843 440 189 0 537347 0
[pid=10815/tid=10817] ppid=10813 vsize=2171992 CPUtime=0.82 cores=0-31
/proc/10815/task/10817/stat : 10817 (claspmt) S 10813 10815 10706 0 -1 4202560 7847 0 0 0 8 74 0 0 18 0 32 0 46292692 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604636497984992 0 0 -1 25 0 0 0
[pid=10815/tid=10818] ppid=10813 vsize=2171992 CPUtime=0.97 cores=0-31
/proc/10815/task/10818/stat : 10818 (claspmt) S 10813 10815 10706 0 -1 4202560 10314 0 0 0 13 84 0 0 18 0 32 0 46292692 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604502485605856 0 0 -1 10 0 0 0
[pid=10815/tid=10819] ppid=10813 vsize=2171992 CPUtime=1 cores=0-31
/proc/10815/task/10819/stat : 10819 (claspmt) S 10813 10815 10706 0 -1 4202560 9339 0 0 0 14 86 0 0 18 0 32 0 46292692 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604641547173344 0 0 -1 3 0 0 0
[pid=10815/tid=10820] ppid=10813 vsize=2171992 CPUtime=0.85 cores=0-31
/proc/10815/task/10820/stat : 10820 (claspmt) S 10813 10815 10706 0 -1 4202560 8034 0 0 0 9 76 0 0 18 0 32 0 46292692 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604502194986464 0 0 -1 7 0 0 0
[pid=10815/tid=10821] ppid=10813 vsize=2171992 CPUtime=6.11 cores=0-31
/proc/10815/task/10821/stat : 10821 (claspmt) R 10813 10815 10706 0 -1 4202560 8773 0 0 0 534 77 0 0 25 0 32 0 46292692 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 4670212 0 0 4096 18946 0 0 0 -1 17 0 0 0
[pid=10815/tid=10822] ppid=10813 vsize=2171992 CPUtime=0.98 cores=0-31
/proc/10815/task/10822/stat : 10822 (claspmt) S 10813 10815 10706 0 -1 4202560 10491 0 0 0 14 84 0 0 18 0 32 0 46292692 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604483509951968 0 0 -1 26 0 0 0
[pid=10815/tid=10823] ppid=10813 vsize=2171992 CPUtime=1.05 cores=0-31
/proc/10815/task/10823/stat : 10823 (claspmt) S 10813 10815 10706 0 -1 4202560 9403 0 0 0 14 91 0 0 18 0 32 0 46292692 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604637471604192 0 0 -1 23 0 0 0
[pid=10815/tid=10824] ppid=10813 vsize=2171992 CPUtime=0.86 cores=0-31
/proc/10815/task/10824/stat : 10824 (claspmt) S 10813 10815 10706 0 -1 4202560 7960 0 0 0 8 78 0 0 18 0 32 0 46292692 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604506127203808 0 0 -1 9 0 0 0
[pid=10815/tid=10825] ppid=10813 vsize=2171992 CPUtime=0.88 cores=0-31
/proc/10815/task/10825/stat : 10825 (claspmt) S 10813 10815 10706 0 -1 4202560 7988 0 0 0 8 80 0 0 18 0 32 0 46292692 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604639675497952 0 0 -1 5 0 0 0
[pid=10815/tid=10826] ppid=10813 vsize=2171992 CPUtime=0.99 cores=0-31
/proc/10815/task/10826/stat : 10826 (claspmt) S 10813 10815 10706 0 -1 4202560 10417 0 0 0 14 85 0 0 18 0 32 0 46292692 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604483388898784 0 0 -1 22 0 0 0
[pid=10815/tid=10827] ppid=10813 vsize=2171992 CPUtime=0.97 cores=0-31
/proc/10815/task/10827/stat : 10827 (claspmt) S 10813 10815 10706 0 -1 4202560 9988 0 0 0 14 83 0 0 18 0 32 0 46292692 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604505883393504 0 0 -1 6 0 0 0
[pid=10815/tid=10828] ppid=10813 vsize=2171992 CPUtime=0.85 cores=0-31
/proc/10815/task/10828/stat : 10828 (claspmt) S 10813 10815 10706 0 -1 4202560 7629 0 0 0 9 76 0 0 17 0 32 0 46292692 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604500622867936 0 0 -1 18 0 0 0
[pid=10815/tid=10829] ppid=10813 vsize=2171992 CPUtime=6.02 cores=0-31
/proc/10815/task/10829/stat : 10829 (claspmt) R 10813 10815 10706 0 -1 4202560 8683 0 0 0 514 88 0 0 25 0 32 0 46292692 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 4668712 0 0 4096 18946 0 0 0 -1 23 0 0 0
[pid=10815/tid=10830] ppid=10813 vsize=2171992 CPUtime=0.96 cores=0-31
/proc/10815/task/10830/stat : 10830 (claspmt) S 10813 10815 10706 0 -1 4202560 10288 0 0 0 15 81 0 0 18 0 32 0 46292692 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604640406150624 0 0 -1 12 0 0 0
[pid=10815/tid=10831] ppid=10813 vsize=2171992 CPUtime=0.98 cores=0-31
/proc/10815/task/10831/stat : 10831 (claspmt) S 10813 10815 10706 0 -1 4202560 9101 0 0 0 17 81 0 0 18 0 32 0 46292692 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604497121754592 0 0 -1 20 0 0 0
[pid=10815/tid=10832] ppid=10813 vsize=2171992 CPUtime=0.88 cores=0-31
/proc/10815/task/10832/stat : 10832 (claspmt) S 10813 10815 10706 0 -1 4202560 7984 0 0 0 10 78 0 0 18 0 32 0 46292692 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604638949555680 0 0 -1 14 0 0 0
[pid=10815/tid=10833] ppid=10813 vsize=2171992 CPUtime=0.98 cores=0-31
/proc/10815/task/10833/stat : 10833 (claspmt) S 10813 10815 10706 0 -1 4202560 7797 0 0 0 10 88 0 0 18 0 32 0 46292692 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604483388890592 0 0 -1 0 0 0 0
[pid=10815/tid=10834] ppid=10813 vsize=2171992 CPUtime=0.63 cores=0-31
/proc/10815/task/10834/stat : 10834 (claspmt) S 10813 10815 10706 0 -1 4202560 10060 0 0 0 13 50 0 0 18 0 32 0 46292692 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604485262007776 0 0 -1 24 0 0 0
[pid=10815/tid=10835] ppid=10813 vsize=2171992 CPUtime=0.77 cores=0-31
/proc/10815/task/10835/stat : 10835 (claspmt) S 10813 10815 10706 0 -1 4202560 9366 0 0 0 14 63 0 0 18 0 32 0 46292693 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604562198420960 0 0 -1 11 0 0 0
[pid=10815/tid=10836] ppid=10813 vsize=2171992 CPUtime=0.92 cores=0-31
/proc/10815/task/10836/stat : 10836 (claspmt) S 10813 10815 10706 0 -1 4202560 7782 0 0 0 9 83 0 0 18 0 32 0 46292693 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604643160227296 0 0 -1 4 0 0 0
[pid=10815/tid=10837] ppid=10813 vsize=2171992 CPUtime=6.21 cores=0-31
/proc/10815/task/10837/stat : 10837 (claspmt) R 10813 10815 10706 0 -1 4202560 8403 0 0 0 541 80 0 0 25 0 32 0 46292693 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 4668712 0 0 4096 18946 0 0 0 -1 14 0 0 0
[pid=10815/tid=10838] ppid=10813 vsize=2171992 CPUtime=0.95 cores=0-31
/proc/10815/task/10838/stat : 10838 (claspmt) S 10813 10815 10706 0 -1 4202560 10284 0 0 0 16 79 0 0 18 0 32 0 46292693 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604638012104160 0 0 -1 8 0 0 0
[pid=10815/tid=10839] ppid=10813 vsize=2171992 CPUtime=0.95 cores=0-31
/proc/10815/task/10839/stat : 10839 (claspmt) S 10813 10815 10706 0 -1 4202560 9195 0 0 0 14 81 0 0 18 0 32 0 46292693 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604640068705760 0 0 -1 27 0 0 0
[pid=10815/tid=10840] ppid=10813 vsize=2171992 CPUtime=0.96 cores=0-31
/proc/10815/task/10840/stat : 10840 (claspmt) S 10813 10815 10706 0 -1 4202560 7897 0 0 0 10 86 0 0 18 0 32 0 46292693 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604629366963680 0 0 -1 2 0 0 0
[pid=10815/tid=10841] ppid=10813 vsize=2171992 CPUtime=0.8 cores=0-31
/proc/10815/task/10841/stat : 10841 (claspmt) S 10813 10815 10706 0 -1 4202560 7997 0 0 0 9 71 0 0 18 0 32 0 46292693 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604485751283168 0 0 -1 24 0 0 0
[pid=10815/tid=10842] ppid=10813 vsize=2171992 CPUtime=0.88 cores=0-31
/proc/10815/task/10842/stat : 10842 (claspmt) S 10813 10815 10706 0 -1 4202560 10774 0 0 0 16 72 0 0 18 0 32 0 46292693 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604485751291360 0 0 -1 29 0 0 0
[pid=10815/tid=10843] ppid=10813 vsize=2171992 CPUtime=0.87 cores=0-31
/proc/10815/task/10843/stat : 10843 (claspmt) S 10813 10815 10706 0 -1 4202560 10528 0 0 0 13 74 0 0 18 0 32 0 46292693 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604642679938528 0 0 -1 13 0 0 0
[pid=10815/tid=10844] ppid=10813 vsize=2171992 CPUtime=0.89 cores=0-31
/proc/10815/task/10844/stat : 10844 (claspmt) S 10813 10815 10706 0 -1 4202560 7900 0 0 0 9 80 0 0 18 0 32 0 46292693 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604499066772960 0 0 -1 30 0 0 0
[pid=10815/tid=10845] ppid=10813 vsize=2171992 CPUtime=6.04 cores=0-31
/proc/10815/task/10845/stat : 10845 (claspmt) R 10813 10815 10706 0 -1 4202560 8792 0 0 0 524 80 0 0 25 0 32 0 46292693 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 4669280 0 0 4096 18946 0 0 0 -1 28 0 0 0
[pid=10815/tid=10846] ppid=10813 vsize=2171992 CPUtime=0.82 cores=0-31
/proc/10815/task/10846/stat : 10846 (claspmt) S 10813 10815 10706 0 -1 4202560 10624 0 0 0 13 69 0 0 18 0 32 0 46292694 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604640152321504 0 0 -1 19 0 0 0
[pid=10815/tid=10847] ppid=10813 vsize=2171992 CPUtime=0.91 cores=0-31
/proc/10815/task/10847/stat : 10847 (claspmt) S 10813 10815 10706 0 -1 4202560 8370 0 0 0 14 77 0 0 18 0 32 0 46292694 2224119808 250843 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604639334292960 0 0 -1 16 0 0 0
Current children cumulated CPU time (s) 54.71
Current children cumulated vsize (KiB) 2171992

[startup+25.5021 s]
/proc/loadavg: 18.86 25.23 27.93 5/445 10847
/proc/meminfo: memFree=261029192/264257624 swapFree=67111528/67111528
[pid=10815] ppid=10813 vsize=2171992 CPUtime=105.9 cores=0-31
/proc/10815/stat : 10815 (claspmt) S 10813 10815 10706 0 -1 4202496 327861 0 0 0 8126 2464 0 0 18 0 32 0 46292130 2224119808 250865 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446744073709551615 0 0 17 10 0 0 0
/proc/10815/statm: 542998 250865 440 189 0 537347 0
[pid=10815/tid=10817] ppid=10813 vsize=2171992 CPUtime=0.82 cores=0-31
/proc/10815/task/10817/stat : 10817 (claspmt) S 10813 10815 10706 0 -1 4202560 7847 0 0 0 8 74 0 0 18 0 32 0 46292692 2224119808 250865 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604636497984992 0 0 -1 25 0 0 0
[pid=10815/tid=10818] ppid=10813 vsize=2171992 CPUtime=0.97 cores=0-31
/proc/10815/task/10818/stat : 10818 (claspmt) S 10813 10815 10706 0 -1 4202560 10314 0 0 0 13 84 0 0 18 0 32 0 46292692 2224119808 250865 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604502485605856 0 0 -1 10 0 0 0
[pid=10815/tid=10819] ppid=10813 vsize=2171992 CPUtime=1 cores=0-31
/proc/10815/task/10819/stat : 10819 (claspmt) S 10813 10815 10706 0 -1 4202560 9339 0 0 0 14 86 0 0 18 0 32 0 46292692 2224119808 250865 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604641547173344 0 0 -1 3 0 0 0
[pid=10815/tid=10820] ppid=10813 vsize=2171992 CPUtime=0.85 cores=0-31
/proc/10815/task/10820/stat : 10820 (claspmt) S 10813 10815 10706 0 -1 4202560 8034 0 0 0 9 76 0 0 18 0 32 0 46292692 2224119808 250865 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604502194986464 0 0 -1 7 0 0 0
[pid=10815/tid=10821] ppid=10813 vsize=2171992 CPUtime=18.91 cores=0-31
/proc/10815/task/10821/stat : 10821 (claspmt) R 10813 10815 10706 0 -1 4202560 8773 0 0 0 1814 77 0 0 25 0 32 0 46292692 2224119808 250865 270582939648 4194304 4964800 140736730161280 18446744073709551615 4399412 0 0 4096 18946 0 0 0 -1 17 0 0 0
[pid=10815/tid=10822] ppid=10813 vsize=2171992 CPUtime=0.98 cores=0-31
/proc/10815/task/10822/stat : 10822 (claspmt) S 10813 10815 10706 0 -1 4202560 10491 0 0 0 14 84 0 0 18 0 32 0 46292692 2224119808 250865 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604483509951968 0 0 -1 26 0 0 0
[pid=10815/tid=10823] ppid=10813 vsize=2171992 CPUtime=1.05 cores=0-31
/proc/10815/task/10823/stat : 10823 (claspmt) S 10813 10815 10706 0 -1 4202560 9403 0 0 0 14 91 0 0 18 0 32 0 46292692 2224119808 250865 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604637471604192 0 0 -1 23 0 0 0
[pid=10815/tid=10824] ppid=10813 vsize=2171992 CPUtime=0.86 cores=0-31
/proc/10815/task/10824/stat : 10824 (claspmt) S 10813 10815 10706 0 -1 4202560 7960 0 0 0 8 78 0 0 18 0 32 0 46292692 2224119808 250865 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604506127203808 0 0 -1 9 0 0 0
[pid=10815/tid=10825] ppid=10813 vsize=2171992 CPUtime=0.88 cores=0-31
/proc/10815/task/10825/stat : 10825 (claspmt) S 10813 10815 10706 0 -1 4202560 7988 0 0 0 8 80 0 0 18 0 32 0 46292692 2224119808 250865 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604639675497952 0 0 -1 5 0 0 0
[pid=10815/tid=10826] ppid=10813 vsize=2171992 CPUtime=0.99 cores=0-31
/proc/10815/task/10826/stat : 10826 (claspmt) S 10813 10815 10706 0 -1 4202560 10417 0 0 0 14 85 0 0 18 0 32 0 46292692 2224119808 250865 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604483388898784 0 0 -1 22 0 0 0
[pid=10815/tid=10827] ppid=10813 vsize=2171992 CPUtime=0.97 cores=0-31
/proc/10815/task/10827/stat : 10827 (claspmt) S 10813 10815 10706 0 -1 4202560 9988 0 0 0 14 83 0 0 18 0 32 0 46292692 2224119808 250865 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604505883393504 0 0 -1 6 0 0 0
[pid=10815/tid=10828] ppid=10813 vsize=2171992 CPUtime=0.85 cores=0-31
/proc/10815/task/10828/stat : 10828 (claspmt) S 10813 10815 10706 0 -1 4202560 7629 0 0 0 9 76 0 0 17 0 32 0 46292692 2224119808 250865 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604500622867936 0 0 -1 18 0 0 0
[pid=10815/tid=10829] ppid=10813 vsize=2171992 CPUtime=18.82 cores=0-31
/proc/10815/task/10829/stat : 10829 (claspmt) R 10813 10815 10706 0 -1 4202560 8695 0 0 0 1794 88 0 0 25 0 32 0 46292692 2224119808 250865 270582939648 4194304 4964800 140736730161280 18446744073709551615 4668773 0 0 4096 18946 0 0 0 -1 23 0 0 0
[pid=10815/tid=10830] ppid=10813 vsize=2171992 CPUtime=0.96 cores=0-31
/proc/10815/task/10830/stat : 10830 (claspmt) S 10813 10815 10706 0 -1 4202560 10288 0 0 0 15 81 0 0 18 0 32 0 46292692 2224119808 250865 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604640406150624 0 0 -1 12 0 0 0
[pid=10815/tid=10831] ppid=10813 vsize=2171992 CPUtime=0.98 cores=0-31
/proc/10815/task/10831/stat : 10831 (claspmt) S 10813 10815 10706 0 -1 4202560 9101 0 0 0 17 81 0 0 18 0 32 0 46292692 2224119808 250865 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604497121754592 0 0 -1 20 0 0 0
[pid=10815/tid=10832] ppid=10813 vsize=2171992 CPUtime=0.88 cores=0-31
/proc/10815/task/10832/stat : 10832 (claspmt) S 10813 10815 10706 0 -1 4202560 7984 0 0 0 10 78 0 0 18 0 32 0 46292692 2224119808 250865 270582939648 4194304 4964800 140736730161280 18446744073709551615 258662772457 0 0 4096 18946 18446604638949555680 0 0 -1 14 0 0 0

################
# More data... #
################

/proc/10815/task/10828/stat : 10828 (claspmt) R 10813 10815 10706 0 -1 4202560 64052 0 0 0 472232 274 0 0 25 0 32 0 46292692 13898678272 3183003 270582939648 4194304 4964800 140736730161280 18446744073709551615 4669509 0 0 4096 18946 0 0 0 -1 18 0 0 0
[pid=10815/tid=10829] ppid=10813 vsize=13572928 CPUtime=5070.93 cores=0-31
/proc/10815/task/10829/stat : 10829 (claspmt) R 10813 10815 10706 0 -1 4202560 52219 0 0 0 506879 214 0 0 25 0 32 0 46292692 13898678272 3183003 270582939648 4194304 4964800 140736730161280 18446744073709551615 4670371 0 0 4096 18946 0 0 0 -1 27 0 0 0
[pid=10815/tid=10830] ppid=10813 vsize=13572928 CPUtime=4716.24 cores=0-31
/proc/10815/task/10830/stat : 10830 (claspmt) R 10813 10815 10706 0 -1 4202560 103157 0 0 0 471216 408 0 0 25 0 32 0 46292692 13898678272 3183003 270582939648 4194304 4964800 140736730161280 18446744073709551615 4321121 0 0 4096 18946 0 0 0 -1 19 0 0 0
[pid=10815/tid=10831] ppid=10813 vsize=13572928 CPUtime=4723.96 cores=0-31
/proc/10815/task/10831/stat : 10831 (claspmt) R 10813 10815 10706 0 -1 4202560 256444 0 0 0 471577 819 0 0 25 0 32 0 46292692 13898678272 3183003 270582939648 4194304 4964800 140736730161280 18446744073709551615 4678036 0 0 4096 18946 0 0 0 -1 26 0 0 0
[pid=10815/tid=10832] ppid=10813 vsize=13572928 CPUtime=4725.23 cores=0-31
/proc/10815/task/10832/stat : 10832 (claspmt) R 10813 10815 10706 0 -1 4202560 22896 0 0 0 472382 141 0 0 25 0 32 0 46292692 13898678272 3183003 270582939648 4194304 4964800 140736730161280 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 14 0 0 0
[pid=10815/tid=10833] ppid=10813 vsize=13572928 CPUtime=4725.21 cores=0-31
/proc/10815/task/10833/stat : 10833 (claspmt) R 10813 10815 10706 0 -1 4202560 25877 0 0 0 472371 150 0 0 25 0 32 0 46292692 13898678272 3183003 270582939648 4194304 4964800 140736730161280 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=10815/tid=10834] ppid=10813 vsize=13572928 CPUtime=4723.6 cores=0-31
/proc/10815/task/10834/stat : 10834 (claspmt) R 10813 10815 10706 0 -1 4202560 192985 0 0 0 471716 644 0 0 25 0 32 0 46292692 13898678272 3183003 270582939648 4194304 4964800 140736730161280 18446744073709551615 4670332 0 0 4096 18946 0 0 0 -1 29 0 0 0
[pid=10815/tid=10835] ppid=10813 vsize=13572928 CPUtime=4717.85 cores=0-31
/proc/10815/task/10835/stat : 10835 (claspmt) R 10813 10815 10706 0 -1 4202560 123576 0 0 0 471300 485 0 0 21 0 32 0 46292693 13898678272 3183003 270582939648 4194304 4964800 140736730161280 18446744073709551615 4670534 0 0 4096 18946 0 0 0 -1 17 0 0 0
[pid=10815/tid=10836] ppid=10813 vsize=13572928 CPUtime=4724.46 cores=0-31
/proc/10815/task/10836/stat : 10836 (claspmt) R 10813 10815 10706 0 -1 4202560 84323 0 0 0 472110 336 0 0 25 0 32 0 46292693 13898678272 3183003 270582939648 4194304 4964800 140736730161280 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 24 0 0 0
[pid=10815/tid=10837] ppid=10813 vsize=13572928 CPUtime=5074.43 cores=0-31
/proc/10815/task/10837/stat : 10837 (claspmt) R 10813 10815 10706 0 -1 4202560 46015 0 0 0 507255 188 0 0 19 0 32 0 46292693 13898678272 3183003 270582939648 4194304 4964800 140736730161280 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=10815/tid=10838] ppid=10813 vsize=13572928 CPUtime=4724.92 cores=0-31
/proc/10815/task/10838/stat : 10838 (claspmt) R 10813 10815 10706 0 -1 4202560 95579 0 0 0 472119 373 0 0 25 0 32 0 46292693 13898678272 3183003 270582939648 4194304 4964800 140736730161280 18446744073709551615 4668712 0 0 4096 18946 0 0 0 -1 8 0 0 0
[pid=10815/tid=10839] ppid=10813 vsize=13572928 CPUtime=4693.93 cores=0-31
/proc/10815/task/10839/stat : 10839 (claspmt) R 10813 10815 10706 0 -1 4202560 244891 0 0 0 468588 805 0 0 25 0 32 0 46292693 13898678272 3183003 270582939648 4194304 4964800 140736730161280 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=10815/tid=10840] ppid=10813 vsize=13572928 CPUtime=4725.32 cores=0-31
/proc/10815/task/10840/stat : 10840 (claspmt) R 10813 10815 10706 0 -1 4202560 27335 0 0 0 472369 163 0 0 25 0 32 0 46292693 13898678272 3183003 270582939648 4194304 4964800 140736730161280 18446744073709551615 4668734 0 0 4096 18946 0 0 0 -1 25 0 0 0
[pid=10815/tid=10841] ppid=10813 vsize=13572928 CPUtime=4724.33 cores=0-31
/proc/10815/task/10841/stat : 10841 (claspmt) R 10813 10815 10706 0 -1 4202560 24954 0 0 0 472301 132 0 0 25 0 32 0 46292693 13898678272 3183003 270582939648 4194304 4964800 140736730161280 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=10815/tid=10842] ppid=10813 vsize=13572928 CPUtime=4717.33 cores=0-31
/proc/10815/task/10842/stat : 10842 (claspmt) R 10813 10815 10706 0 -1 4202560 177196 0 0 0 471091 642 0 0 25 0 32 0 46292693 13898678272 3183003 270582939648 4194304 4964800 140736730161280 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 31 0 0 0
[pid=10815/tid=10843] ppid=10813 vsize=13572928 CPUtime=4724.9 cores=0-31
/proc/10815/task/10843/stat : 10843 (claspmt) R 10813 10815 10706 0 -1 4202560 86114 0 0 0 472134 356 0 0 25 0 32 0 46292693 13898678272 3183003 270582939648 4194304 4964800 140736730161280 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 13 0 0 0
[pid=10815/tid=10844] ppid=10813 vsize=13572928 CPUtime=4711.86 cores=0-31
/proc/10815/task/10844/stat : 10844 (claspmt) R 10813 10815 10706 0 -1 4202560 76864 0 0 0 470872 314 0 0 25 0 32 0 46292693 13898678272 3183003 270582939648 4194304 4964800 140736730161280 18446744073709551615 4668712 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=10815/tid=10845] ppid=10813 vsize=13572928 CPUtime=5069.86 cores=0-31
/proc/10815/task/10845/stat : 10845 (claspmt) R 10813 10815 10706 0 -1 4202560 36011 0 0 0 506768 218 0 0 25 0 32 0 46292693 13898678272 3183003 270582939648 4194304 4964800 140736730161280 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 16 0 0 0
[pid=10815/tid=10846] ppid=10813 vsize=13572928 CPUtime=4706.02 cores=0-31
/proc/10815/task/10846/stat : 10846 (claspmt) R 10813 10815 10706 0 -1 4202560 101820 0 0 0 470222 380 0 0 25 0 32 0 46292694 13898678272 3183003 270582939648 4194304 4964800 140736730161280 18446744073709551615 4660157 0 0 4096 18946 0 0 0 -1 12 0 0 0
[pid=10815/tid=10847] ppid=10813 vsize=13572928 CPUtime=4724.4 cores=0-31
/proc/10815/task/10847/stat : 10847 (claspmt) R 10813 10815 10706 0 -1 4202560 259898 0 0 0 471656 784 0 0 25 0 32 0 46292694 13898678272 3183003 270582939648 4194304 4964800 140736730161280 18446744073709551615 4321054 0 0 4096 18946 0 0 0 -1 20 0 0 0
Current children cumulated CPU time (s) 152458
Current children cumulated vsize (KiB) 13572928



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5100 s]
/proc/loadavg: 32.23 32.17 32.03 33/445 10995
/proc/meminfo: memFree=249114132/264257624 swapFree=67111528/67111528
[pid=10815] ppid=10813 vsize=13704000 CPUtime=153025 cores=0-31
/proc/10815/stat : 10815 (claspmt) R 10813 10815 10706 0 -1 4202496 3311333 0 0 0 15290077 12385 0 0 25 0 32 0 46292130 14032896000 3223851 270582939648 4194304 4964800 140736730161280 18446744073709551615 4670180 0 0 4096 18946 18446744073709551615 0 0 17 10 0 0 0
/proc/10815/statm: 3426000 3223851 463 189 0 3420349 0
[pid=10815/tid=10817] ppid=10813 vsize=13704000 CPUtime=4742.71 cores=0-31
/proc/10815/task/10817/stat : 10817 (claspmt) R 10813 10815 10706 0 -1 4202560 27731 0 0 0 474124 147 0 0 25 0 32 0 46292692 14032896000 3223852 270582939648 4194304 4964800 140736730161280 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 21 0 0 0
[pid=10815/tid=10818] ppid=10813 vsize=13704000 CPUtime=4742.57 cores=0-31
/proc/10815/task/10818/stat : 10818 (claspmt) R 10813 10815 10706 0 -1 4202560 211259 0 0 0 473487 770 0 0 20 0 32 0 46292692 14032896000 3223852 270582939648 4194304 4964800 140736730161280 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 22 0 0 0
[pid=10815/tid=10819] ppid=10813 vsize=13704000 CPUtime=4740.27 cores=0-31
/proc/10815/task/10819/stat : 10819 (claspmt) R 10813 10815 10706 0 -1 4202560 122097 0 0 0 473549 478 0 0 25 0 32 0 46292692 14032896000 3223852 270582939648 4194304 4964800 140736730161280 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 23 0 0 0
[pid=10815/tid=10820] ppid=10813 vsize=13704000 CPUtime=4732.16 cores=0-31
/proc/10815/task/10820/stat : 10820 (claspmt) R 10813 10815 10706 0 -1 4202560 77524 0 0 0 472914 302 0 0 19 0 32 0 46292692 14032896000 3223852 270582939648 4194304 4964800 140736730161280 18446744073709551615 4370264 0 0 4096 18946 0 0 0 -1 11 0 0 0
[pid=10815/tid=10821] ppid=10813 vsize=13704000 CPUtime=5090.83 cores=0-31
/proc/10815/task/10821/stat : 10821 (claspmt) R 10813 10815 10706 0 -1 4202560 47174 0 0 0 508889 194 0 0 25 0 32 0 46292692 14032896000 3223852 270582939648 4194304 4964800 140736730161280 18446744073709551615 4668681 0 0 4096 18946 0 0 0 -1 9 0 0 0
[pid=10815/tid=10822] ppid=10813 vsize=13704000 CPUtime=4727.25 cores=0-31
/proc/10815/task/10822/stat : 10822 (claspmt) R 10813 10815 10706 0 -1 4202560 95055 0 0 0 472320 405 0 0 25 0 32 0 46292692 14032896000 3223852 270582939648 4194304 4964800 140736730161280 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=10815/tid=10823] ppid=10813 vsize=13704000 CPUtime=4738.21 cores=0-31
/proc/10815/task/10823/stat : 10823 (claspmt) R 10813 10815 10706 0 -1 4202560 190909 0 0 0 473148 673 0 0 25 0 32 0 46292692 14032896000 3223852 270582939648 4194304 4964800 140736730161280 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 28 0 0 0
[pid=10815/tid=10824] ppid=10813 vsize=13704000 CPUtime=4738.11 cores=0-31
/proc/10815/task/10824/stat : 10824 (claspmt) R 10813 10815 10706 0 -1 4202560 30350 0 0 0 473638 173 0 0 25 0 32 0 46292692 14032896000 3223852 270582939648 4194304 4964800 140736730161280 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 30 0 0 0
[pid=10815/tid=10825] ppid=10813 vsize=13704000 CPUtime=4742.78 cores=0-31
/proc/10815/task/10825/stat : 10825 (claspmt) R 10813 10815 10706 0 -1 4202560 22213 0 0 0 474150 128 0 0 25 0 32 0 46292692 14032896000 3223852 270582939648 4194304 4964800 140736730161280 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=10815/tid=10826] ppid=10813 vsize=13704000 CPUtime=4742.59 cores=0-31
/proc/10815/task/10826/stat : 10826 (claspmt) R 10813 10815 10706 0 -1 4202560 178807 0 0 0 473604 655 0 0 25 0 32 0 46292692 14032896000 3223852 270582939648 4194304 4964800 140736730161280 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=10815/tid=10827] ppid=10813 vsize=13704000 CPUtime=4735.62 cores=0-31
/proc/10815/task/10827/stat : 10827 (claspmt) R 10813 10815 10706 0 -1 4202560 112405 0 0 0 473121 441 0 0 19 0 32 0 46292692 14032896000 3223852 270582939648 4194304 4964800 140736730161280 18446744073709551615 4669542 0 0 4096 18946 0 0 0 -1 15 0 0 0
[pid=10815/tid=10828] ppid=10813 vsize=13704000 CPUtime=4742.77 cores=0-31
/proc/10815/task/10828/stat : 10828 (claspmt) R 10813 10815 10706 0 -1 4202560 65396 0 0 0 473998 279 0 0 25 0 32 0 46292692 14032896000 3223852 270582939648 4194304 4964800 140736730161280 18446744073709551615 4370264 0 0 4096 18946 0 0 0 -1 18 0 0 0
[pid=10815/tid=10829] ppid=10813 vsize=13704000 CPUtime=5088.63 cores=0-31
/proc/10815/task/10829/stat : 10829 (claspmt) R 10813 10815 10706 0 -1 4202560 57330 0 0 0 508633 230 0 0 19 0 32 0 46292692 14032896000 3223852 270582939648 4194304 4964800 140736730161280 18446744073709551615 4632289 0 0 4096 18946 0 0 0 -1 27 0 0 0
[pid=10815/tid=10830] ppid=10813 vsize=13704000 CPUtime=4733.95 cores=0-31
/proc/10815/task/10830/stat : 10830 (claspmt) R 10813 10815 10706 0 -1 4202560 103464 0 0 0 472985 410 0 0 25 0 32 0 46292692 14032896000 3223852 270582939648 4194304 4964800 140736730161280 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 19 0 0 0
[pid=10815/tid=10831] ppid=10813 vsize=13704000 CPUtime=4741.68 cores=0-31
/proc/10815/task/10831/stat : 10831 (claspmt) R 10813 10815 10706 0 -1 4202560 258206 0 0 0 473343 825 0 0 25 0 32 0 46292692 14032896000 3223852 270582939648 4194304 4964800 140736730161280 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 26 0 0 0
[pid=10815/tid=10832] ppid=10813 vsize=13704000 CPUtime=4742.95 cores=0-31
/proc/10815/task/10832/stat : 10832 (claspmt) R 10813 10815 10706 0 -1 4202560 22980 0 0 0 474154 141 0 0 25 0 32 0 46292692 14032896000 3223852 270582939648 4194304 4964800 140736730161280 18446744073709551615 4668734 0 0 4096 18946 0 0 0 -1 14 0 0 0
[pid=10815/tid=10833] ppid=10813 vsize=13704000 CPUtime=4742.93 cores=0-31
/proc/10815/task/10833/stat : 10833 (claspmt) R 10813 10815 10706 0 -1 4202560 25879 0 0 0 474143 150 0 0 25 0 32 0 46292692 14032896000 3223852 270582939648 4194304 4964800 140736730161280 18446744073709551615 4669509 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=10815/tid=10834] ppid=10813 vsize=13704000 CPUtime=4741.31 cores=0-31
/proc/10815/task/10834/stat : 10834 (claspmt) R 10813 10815 10706 0 -1 4202560 193418 0 0 0 473486 645 0 0 25 0 32 0 46292692 14032896000 3223852 270582939648 4194304 4964800 140736730161280 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 29 0 0 0
[pid=10815/tid=10835] ppid=10813 vsize=13704000 CPUtime=4735.55 cores=0-31
/proc/10815/task/10835/stat : 10835 (claspmt) R 10813 10815 10706 0 -1 4202560 126953 0 0 0 473062 493 0 0 21 0 32 0 46292693 14032896000 3223852 270582939648 4194304 4964800 140736730161280 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 17 0 0 0
[pid=10815/tid=10836] ppid=10813 vsize=13704000 CPUtime=4742.18 cores=0-31
/proc/10815/task/10836/stat : 10836 (claspmt) R 10813 10815 10706 0 -1 4202560 84689 0 0 0 473880 338 0 0 25 0 32 0 46292693 14032896000 3223853 270582939648 4194304 4964800 140736730161280 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 24 0 0 0
[pid=10815/tid=10837] ppid=10813 vsize=13704000 CPUtime=5092.14 cores=0-31
/proc/10815/task/10837/stat : 10837 (claspmt) R 10813 10815 10706 0 -1 4202560 49122 0 0 0 509017 197 0 0 25 0 32 0 46292693 14032896000 3223853 270582939648 4194304 4964800 140736730161280 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=10815/tid=10838] ppid=10813 vsize=13704000 CPUtime=4742.64 cores=0-31
/proc/10815/task/10838/stat : 10838 (claspmt) R 10813 10815 10706 0 -1 4202560 95760 0 0 0 473890 374 0 0 25 0 32 0 46292693 14032896000 3223854 270582939648 4194304 4964800 140736730161280 18446744073709551615 4653507 0 0 4096 18946 0 0 0 -1 8 0 0 0
[pid=10815/tid=10839] ppid=10813 vsize=13704000 CPUtime=4711.65 cores=0-31
/proc/10815/task/10839/stat : 10839 (claspmt) R 10813 10815 10706 0 -1 4202560 247015 0 0 0 470353 812 0 0 25 0 32 0 46292693 14032896000 3223855 270582939648 4194304 4964800 140736730161280 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=10815/tid=10840] ppid=10813 vsize=13704000 CPUtime=4743.03 cores=0-31
/proc/10815/task/10840/stat : 10840 (claspmt) R 10813 10815 10706 0 -1 4202560 27508 0 0 0 474140 163 0 0 25 0 32 0 46292693 14032896000 3223855 270582939648 4194304 4964800 140736730161280 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 25 0 0 0
[pid=10815/tid=10841] ppid=10813 vsize=13704000 CPUtime=4742.05 cores=0-31
/proc/10815/task/10841/stat : 10841 (claspmt) R 10813 10815 10706 0 -1 4202560 25068 0 0 0 474073 132 0 0 25 0 32 0 46292693 14032896000 3223855 270582939648 4194304 4964800 140736730161280 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=10815/tid=10842] ppid=10813 vsize=13704000 CPUtime=4734.54 cores=0-31
/proc/10815/task/10842/stat : 10842 (claspmt) R 10813 10815 10706 0 -1 4202560 178054 0 0 0 472808 646 0 0 20 0 32 0 46292693 14032896000 3223855 270582939648 4194304 4964800 140736730161280 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 31 0 0 0
[pid=10815/tid=10843] ppid=10813 vsize=13704000 CPUtime=4742.61 cores=0-31
/proc/10815/task/10843/stat : 10843 (claspmt) R 10813 10815 10706 0 -1 4202560 86498 0 0 0 473904 357 0 0 18 0 32 0 46292693 14032896000 3223856 270582939648 4194304 4964800 140736730161280 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 13 0 0 0
[pid=10815/tid=10844] ppid=10813 vsize=13704000 CPUtime=4729.57 cores=0-31
/proc/10815/task/10844/stat : 10844 (claspmt) R 10813 10815 10706 0 -1 4202560 79407 0 0 0 472634 323 0 0 22 0 32 0 46292693 14032896000 3223856 270582939648 4194304 4964800 140736730161280 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=10815/tid=10845] ppid=10813 vsize=13704000 CPUtime=5087.57 cores=0-31
/proc/10815/task/10845/stat : 10845 (claspmt) R 10813 10815 10706 0 -1 4202560 37480 0 0 0 508531 226 0 0 25 0 32 0 46292693 14032896000 3223856 270582939648 4194304 4964800 140736730161280 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 16 0 0 0
[pid=10815/tid=10846] ppid=10813 vsize=13704000 CPUtime=4723.71 cores=0-31
/proc/10815/task/10846/stat : 10846 (claspmt) R 10813 10815 10706 0 -1 4202560 102762 0 0 0 471988 383 0 0 25 0 32 0 46292694 14032896000 3223856 270582939648 4194304 4964800 140736730161280 18446744073709551615 4669560 0 0 4096 18946 0 0 0 -1 12 0 0 0
[pid=10815/tid=10847] ppid=10813 vsize=13704000 CPUtime=4742.12 cores=0-31
/proc/10815/task/10847/stat : 10847 (claspmt) R 10813 10815 10706 0 -1 4202560 261295 0 0 0 473423 789 0 0 19 0 32 0 46292694 14032896000 3223856 270582939648 4194304 4964800 140736730161280 18446744073709551615 4670437 0 0 4096 18946 0 0 0 -1 20 0 0 0
Current children cumulated CPU time (s) 153025
Current children cumulated vsize (KiB) 13704000

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)

Sending SIGKILL to -10815

Solver just ended. Dumping a history of the last processes samples

[startup+5100.6 s]
/proc/loadavg: 29.73 31.65 31.86 2/415 10996
/proc/meminfo: memFree=249146736/264257624 swapFree=67111528/67111528
[pid=10815] ppid=10813 vsize=13580872 CPUtime=153026 cores=0-31
/proc/10815/stat : 10815 (claspmt) R 10813 10815 10706 0 -1 4202496 3311368 0 0 0 15290166 12385 0 0 25 0 1 0 46292130 13906812928 3215610 270582939648 4194304 4964800 140736730161280 18446744073709551615 258650604658 0 0 4096 18946 0 0 0 17 10 0 0 0
/proc/10815/statm: 3395218 3215610 474 189 0 3389567 0
Current children cumulated CPU time (s) 153026
Current children cumulated vsize (KiB) 13580872

[startup+5101.4 s]
/proc/loadavg: 29.73 31.65 31.86 2/415 10996
/proc/meminfo: memFree=249146736/264257624 swapFree=67111528/67111528
[pid=10815] ppid=10813 vsize=13556368 CPUtime=153026 cores=0-31
/proc/10815/stat : 10815 (claspmt) R 10813 10815 10706 0 -1 4202496 3311368 0 0 0 15290243 12385 0 0 25 0 1 0 46292130 13881720832 3209470 270582939648 4194304 4964800 140736730161280 18446744073709551615 258650610907 0 0 4096 18946 0 0 0 17 10 0 0 0
/proc/10815/statm: 3389092 3209470 474 189 0 3383441 0
Current children cumulated CPU time (s) 153026
Current children cumulated vsize (KiB) 13556368

[startup+5102.2 s]
/proc/loadavg: 29.73 31.65 31.86 2/415 10996
/proc/meminfo: memFree=249198708/264257624 swapFree=67111528/67111528
[pid=10815] ppid=10813 vsize=0 CPUtime=153027 cores=0-31
/proc/10815/stat : 10815 (claspmt) R 10813 10815 10706 0 -1 4203524 3311368 0 0 0 15290305 12403 0 0 25 0 1 0 46292130 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 10 0 0 0
/proc/10815/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 153027
Current children cumulated vsize (KiB) 0

[startup+5102.6 s]
/proc/loadavg: 29.73 31.65 31.86 2/415 10996
/proc/meminfo: memFree=249198708/264257624 swapFree=67111528/67111528
[pid=10815] ppid=10813 vsize=0 CPUtime=153027 cores=0-31
/proc/10815/stat : 10815 (claspmt) R 10813 10815 10706 0 -1 4203524 3311368 0 0 0 15290305 12443 0 0 25 0 1 0 46292130 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 10 0 0 0
/proc/10815/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 153027
Current children cumulated vsize (KiB) 0

[startup+5102.7 s]
/proc/loadavg: 29.73 31.65 31.86 2/415 10996
/proc/meminfo: memFree=249198708/264257624 swapFree=67111528/67111528
[pid=10815] ppid=10813 vsize=0 CPUtime=153028 cores=0-31
/proc/10815/stat : 10815 (claspmt) R 10813 10815 10706 0 -1 4203524 3311368 0 0 0 15290305 12453 0 0 25 0 1 0 46292130 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 10 0 0 0
/proc/10815/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 153028
Current children cumulated vsize (KiB) 0

Child ended because it received signal 9 (SIGKILL)
Real time (s): 5102.73
CPU time (s): 153028
CPU user time (s): 152903
CPU system time (s): 124.564
CPU usage (%): 2998.93
Max. virtual memory (cumulated for all children) (KiB): 13704000

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 152903
system time used= 124.564
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3311368
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 606770
involuntary context switches= 379149

runsolver used 30.2254 second user time and 112.88 second system time

The end

Launcher Data 

Begin job on node200 at 2011-06-11 18:17:32
IDJOB=3472509
IDBENCH=84248
IDSOLVER=1961
FILE ID=node200/3472509-1307809051
RUNJOBID= node200-1307808729-10720
PBS_JOBID= 13550803
Free space on /tmp= 61728 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT03/industrial/schuppan/l2s/dme-03-1-k247-unsat.shuffled-as.sat03-407.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3472509-1307809051/watcher-3472509-1307809051 -o /tmp/evaluation-result-3472509-1307809051/solver-3472509-1307809051 -C 160000 -W 5100 -M 256000  HOME/claspmt -t32 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3472509-1307809051.cnf

TIME LIMIT= 160000 seconds
MEMORY LIMIT= 256000 MiB
NBCORE= 32

MD5SUM BENCH= 3fef216b3efe77c0f10ea0e0be664a94
RANDOM SEED=1655927536

node200.alineos.net Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 64
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 96
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 66
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 98
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.99
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 68
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 100
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.03
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 70
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 102
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 16
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 17
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 48
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 18
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 80
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 19
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 112
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.97
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 20
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 21
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 50
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.10
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 22
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 82
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.10
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 114
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 24
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 25
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 26
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 84
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 27
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 116
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 28
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 29
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 54
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 30
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 86
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 31
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 118
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     264257624 kB
MemFree:      262030608 kB
Buffers:        292592 kB
Cached:        1610704 kB
SwapCached:          0 kB
Active:         335440 kB
Inactive:      1631296 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     264257624 kB
LowFree:      262030608 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           15988 kB
Writeback:           0 kB
AnonPages:       63488 kB
Mapped:          14532 kB
Slab:           185580 kB
PageTables:       4412 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  199240340 kB
Committed_AS:   168164 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296372 kB
VmallocChunk: 34359440603 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 61724 MiB
End job on node200 at 2011-06-11 19:42:35