Trace number 3472379

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 155952 5103.74

General information on the benchmark

Nameapplication/rintanen/SATPlanning/
sokoban-sequential-p145-microban-sequential.080-SAT.cnf
MD5SUM1ca09a2b911e29894c6603aee8d3e672
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark26196.5
Satisfiable
(Un)Satisfiability was proved
Number of variables175084
Number of clauses2826936
Sum of the clauses size6454456
Maximum clause length78
Minimum clause length1
Number of clauses of size 1696
Number of clauses of size 22728320
Number of clauses of size 37680
Number of clauses of size 435520
Number of clauses of size 515360
Number of clauses of size over 539360

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...586718/instance-3472379-1307586718.cnf
3.69/3.77	c Solving...
155947.38/5100.03	
155947.38/5100.03	*** INTERRUPTED! ***
155947.38/5100.03	*** Shutting down threads...

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3472379-1307586718/watcher-3472379-1307586718 -o /tmp/evaluation-result-3472379-1307586718/solver-3472379-1307586718 -C 160000 -W 5100 -M 256000 HOME/claspmt -t32 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3472379-1307586718.cnf 

running on 32 cores: 0,4,8,12,16,20,24,28,1,5,9,13,17,21,25,29,2,6,10,14,18,22,26,30,3,7,11,15,19,23,27,31

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 160000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 160030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 262144000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 262195200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 27.28 31.07 31.68 1/413 25387
/proc/meminfo: memFree=262622664/264257624 swapFree=67111528/67111528
[pid=25387] ppid=25385 vsize=43108 CPUtime=0.02 cores=0-31
/proc/25387/stat : 25387 (claspmt) R 25385 25387 25361 0 -1 4202496 5042 0 0 0 2 0 0 0 22 0 1 0 24058889 44142592 4971 270582939648 4194304 4964800 140735124092688 18446744073709551615 4700412 0 0 4096 18946 0 0 0 17 12 0 0 0
/proc/25387/statm: 10777 4979 373 189 0 5651 0

[startup+0.0679931 s]
/proc/loadavg: 27.28 31.07 31.68 1/413 25387
/proc/meminfo: memFree=262622664/264257624 swapFree=67111528/67111528
[pid=25387] ppid=25385 vsize=55420 CPUtime=0.06 cores=0-31
/proc/25387/stat : 25387 (claspmt) R 25385 25387 25361 0 -1 4202496 10366 0 0 0 5 1 0 0 22 0 1 0 24058889 56750080 8928 270582939648 4194304 4964800 140735124092688 18446744073709551615 258650955760 0 0 4096 18946 0 0 0 17 12 0 0 0
/proc/25387/statm: 13855 8929 386 189 0 8729 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 55420

[startup+0.100986 s]
/proc/loadavg: 27.28 31.07 31.68 1/413 25387
/proc/meminfo: memFree=262622664/264257624 swapFree=67111528/67111528
[pid=25387] ppid=25385 vsize=55692 CPUtime=0.09 cores=0-31
/proc/25387/stat : 25387 (claspmt) R 25385 25387 25361 0 -1 4202496 10434 0 0 0 8 1 0 0 22 0 1 0 24058889 57028608 8996 270582939648 4194304 4964800 140735124092688 18446744073709551615 258650955760 0 0 4096 18946 0 0 0 17 12 0 0 0
/proc/25387/statm: 13923 8996 386 189 0 8797 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 55692

[startup+0.301037 s]
/proc/loadavg: 27.28 31.07 31.68 1/413 25387
/proc/meminfo: memFree=262622664/264257624 swapFree=67111528/67111528
[pid=25387] ppid=25385 vsize=78192 CPUtime=0.29 cores=0-31
/proc/25387/stat : 25387 (claspmt) R 25385 25387 25361 0 -1 4202496 18386 0 0 0 26 3 0 0 23 0 1 0 24058889 80068608 14278 270582939648 4194304 4964800 140735124092688 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 17 12 0 0 0
/proc/25387/statm: 19548 14279 386 189 0 14422 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 78192

[startup+0.701049 s]
/proc/loadavg: 27.28 31.07 31.68 1/413 25387
/proc/meminfo: memFree=262622664/264257624 swapFree=67111528/67111528
[pid=25387] ppid=25385 vsize=130644 CPUtime=0.69 cores=0-31
/proc/25387/stat : 25387 (claspmt) R 25385 25387 25361 0 -1 4202496 37137 0 0 0 62 7 0 0 25 0 1 0 24058889 133779456 26534 270582939648 4194304 4964800 140735124092688 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 17 12 0 0 0
/proc/25387/statm: 32661 26536 386 189 0 27535 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 130644

[startup+1.50106 s]
/proc/loadavg: 27.28 31.07 31.68 2/414 25388
/proc/meminfo: memFree=262476940/264257624 swapFree=67111528/67111528
[pid=25387] ppid=25385 vsize=236792 CPUtime=1.49 cores=0-31
/proc/25387/stat : 25387 (claspmt) R 25385 25387 25361 0 -1 4202496 67870 0 0 0 135 14 0 0 25 0 1 0 24058889 242475008 50901 270582939648 4194304 4964800 140735124092688 18446744073709551615 4615783 0 0 4096 18946 0 0 0 17 12 0 0 0
/proc/25387/statm: 59198 50901 396 189 0 54072 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 236792

[startup+3.1011 s]
/proc/loadavg: 27.28 31.07 31.68 2/414 25388
/proc/meminfo: memFree=262419244/264257624 swapFree=67111528/67111528
[pid=25387] ppid=25385 vsize=236792 CPUtime=3.09 cores=0-31
/proc/25387/stat : 25387 (claspmt) R 25385 25387 25361 0 -1 4202496 67943 0 0 0 295 14 0 0 25 0 1 0 24058889 242475008 50974 270582939648 4194304 4964800 140735124092688 18446744073709551615 4601745 0 0 4096 18946 0 0 0 17 12 0 0 0
/proc/25387/statm: 59198 50974 401 189 0 54072 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 236792

[startup+6.30216 s]
/proc/loadavg: 27.50 31.05 31.67 6/445 25419
/proc/meminfo: memFree=261861420/264257624 swapFree=67111528/67111528
[pid=25387] ppid=25385 vsize=2364376 CPUtime=27.74 cores=0-31
/proc/25387/stat : 25387 (claspmt) S 25385 25387 25361 0 -1 4202496 277310 0 0 0 1171 1603 0 0 18 0 32 0 24058889 2421121024 213130 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/25387/statm: 591094 213130 440 189 0 585443 0
[pid=25387/tid=25389] ppid=25385 vsize=2364376 CPUtime=0.62 cores=0-31
/proc/25387/task/25389/stat : 25389 (claspmt) S 25385 25387 25361 0 -1 4202560 6414 0 0 0 9 53 0 0 18 0 32 0 24059267 2421121024 213130 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604483707125216 0 0 -1 9 0 0 0
[pid=25387/tid=25390] ppid=25385 vsize=2364376 CPUtime=0.71 cores=0-31
/proc/25387/task/25390/stat : 25390 (claspmt) S 25385 25387 25361 0 -1 4202560 7690 0 0 0 23 48 0 0 18 0 32 0 24059267 2421121024 213131 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604637775273440 0 0 -1 23 0 0 0
[pid=25387/tid=25391] ppid=25385 vsize=2364376 CPUtime=0.79 cores=0-31
/proc/25387/task/25391/stat : 25391 (claspmt) S 25385 25387 25361 0 -1 4202560 8005 0 0 0 23 56 0 0 18 0 32 0 24059267 2421121024 213131 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604639914991072 0 0 -1 15 0 0 0
[pid=25387/tid=25392] ppid=25385 vsize=2364376 CPUtime=0.57 cores=0-31
/proc/25387/task/25392/stat : 25392 (claspmt) S 25385 25387 25361 0 -1 4202560 6438 0 0 0 5 52 0 0 18 0 32 0 24059267 2421121024 213131 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604641227390432 0 0 -1 8 0 0 0
[pid=25387/tid=25393] ppid=25385 vsize=2364376 CPUtime=1.26 cores=0-31
/proc/25387/task/25393/stat : 25393 (claspmt) R 25385 25387 25361 0 -1 4202560 7711 0 0 0 71 55 0 0 18 0 32 0 24059267 2421121024 213131 270582939648 4194304 4964800 140735124092688 18446744073709551615 4669136 0 0 4096 18946 0 0 0 -1 15 0 0 0
[pid=25387/tid=25394] ppid=25385 vsize=2364376 CPUtime=0.76 cores=0-31
/proc/25387/task/25394/stat : 25394 (claspmt) S 25385 25387 25361 0 -1 4202560 7747 0 0 0 20 56 0 0 18 0 32 0 24059267 2421121024 213131 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604504799485408 0 0 -1 7 0 0 0
[pid=25387/tid=25395] ppid=25385 vsize=2364376 CPUtime=0.85 cores=0-31
/proc/25387/task/25395/stat : 25395 (claspmt) S 25385 25387 25361 0 -1 4202560 8480 0 0 0 25 60 0 0 18 0 32 0 24059267 2421121024 213131 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604641227365856 0 0 -1 25 0 0 0
[pid=25387/tid=25396] ppid=25385 vsize=2364376 CPUtime=0.64 cores=0-31
/proc/25387/task/25396/stat : 25396 (claspmt) S 25385 25387 25361 0 -1 4202560 6608 0 0 0 9 55 0 0 18 0 32 0 24059267 2421121024 213131 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604638822915552 0 0 -1 5 0 0 0
[pid=25387/tid=25397] ppid=25385 vsize=2364376 CPUtime=0.6 cores=0-31
/proc/25387/task/25397/stat : 25397 (claspmt) S 25385 25387 25361 0 -1 4202560 6338 0 0 0 8 52 0 0 18 0 32 0 24059267 2421121024 213131 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604643719765472 0 0 -1 13 0 0 0
[pid=25387/tid=25398] ppid=25385 vsize=2364376 CPUtime=0.79 cores=0-31
/proc/25387/task/25398/stat : 25398 (claspmt) S 25385 25387 25361 0 -1 4202560 7647 0 0 0 22 57 0 0 18 0 32 0 24059267 2421121024 213131 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604640715308512 0 0 -1 1 0 0 0
[pid=25387/tid=25399] ppid=25385 vsize=2364376 CPUtime=0.69 cores=0-31
/proc/25387/task/25399/stat : 25399 (claspmt) S 25385 25387 25361 0 -1 4202560 8363 0 0 0 23 46 0 0 18 0 32 0 24059267 2421121024 213131 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604636386287072 0 0 -1 11 0 0 0
[pid=25387/tid=25400] ppid=25385 vsize=2364376 CPUtime=0.63 cores=0-31
/proc/25387/task/25400/stat : 25400 (claspmt) S 25385 25387 25361 0 -1 4202560 6371 0 0 0 8 55 0 0 18 0 32 0 24059267 2421121024 213131 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604485026078176 0 0 -1 29 0 0 0
[pid=25387/tid=25401] ppid=25385 vsize=2364376 CPUtime=1.29 cores=0-31
/proc/25387/task/25401/stat : 25401 (claspmt) R 25385 25387 25361 0 -1 4202560 7720 0 0 0 73 56 0 0 18 0 32 0 24059267 2421121024 213131 270582939648 4194304 4964800 140735124092688 18446744073709551615 4669003 0 0 4096 18946 0 0 0 -1 28 0 0 0
[pid=25387/tid=25402] ppid=25385 vsize=2364376 CPUtime=0.79 cores=0-31
/proc/25387/task/25402/stat : 25402 (claspmt) S 25385 25387 25361 0 -1 4202560 7912 0 0 0 22 57 0 0 18 0 32 0 24059267 2421121024 213131 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604639314001376 0 0 -1 30 0 0 0
[pid=25387/tid=25403] ppid=25385 vsize=2364376 CPUtime=0.77 cores=0-31
/proc/25387/task/25403/stat : 25403 (claspmt) S 25385 25387 25361 0 -1 4202560 8162 0 0 0 25 52 0 0 18 0 32 0 24059267 2421121024 213131 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604640477887968 0 0 -1 3 0 0 0
[pid=25387/tid=25404] ppid=25385 vsize=2364376 CPUtime=0.58 cores=0-31
/proc/25387/task/25404/stat : 25404 (claspmt) S 25385 25387 25361 0 -1 4202560 5741 0 0 0 6 52 0 0 18 0 32 0 24059267 2421121024 213131 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604639754051040 0 0 -1 4 0 0 0
[pid=25387/tid=25405] ppid=25385 vsize=2364376 CPUtime=0.09 cores=0-31
/proc/25387/task/25405/stat : 25405 (claspmt) S 25385 25387 25361 0 -1 4202560 517 0 0 0 8 1 0 0 20 0 32 0 24059267 2421121024 213131 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604570363772384 0 0 -1 18 0 0 0
[pid=25387/tid=25406] ppid=25385 vsize=2364376 CPUtime=0.76 cores=0-31
/proc/25387/task/25406/stat : 25406 (claspmt) S 25385 25387 25361 0 -1 4202560 5493 0 0 0 21 55 0 0 18 0 32 0 24059267 2421121024 213132 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604501358026208 0 0 -1 1 0 0 0
[pid=25387/tid=25407] ppid=25385 vsize=2364376 CPUtime=0.78 cores=0-31
/proc/25387/task/25407/stat : 25407 (claspmt) S 25385 25387 25361 0 -1 4202560 7617 0 0 0 22 56 0 0 18 0 32 0 24059268 2421121024 213132 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604636922838496 0 0 -1 6 0 0 0
[pid=25387/tid=25408] ppid=25385 vsize=2364376 CPUtime=0.58 cores=0-31
/proc/25387/task/25408/stat : 25408 (claspmt) S 25385 25387 25361 0 -1 4202560 5828 0 0 0 7 51 0 0 18 0 32 0 24059268 2421121024 213132 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604639423765984 0 0 -1 20 0 0 0
[pid=25387/tid=25409] ppid=25385 vsize=2364376 CPUtime=1.36 cores=0-31
/proc/25387/task/25409/stat : 25409 (claspmt) R 25385 25387 25361 0 -1 4202560 7574 0 0 0 90 46 0 0 18 0 32 0 24059268 2421121024 213132 270582939648 4194304 4964800 140735124092688 18446744073709551615 4342206 0 0 4096 18946 0 0 0 -1 26 0 0 0
[pid=25387/tid=25410] ppid=25385 vsize=2364376 CPUtime=0.59 cores=0-31
/proc/25387/task/25410/stat : 25410 (claspmt) S 25385 25387 25361 0 -1 4202560 5593 0 0 0 23 36 0 0 18 0 32 0 24059268 2421121024 213132 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604636610248160 0 0 -1 27 0 0 0
[pid=25387/tid=25411] ppid=25385 vsize=2364376 CPUtime=0.8 cores=0-31
/proc/25387/task/25411/stat : 25411 (claspmt) S 25385 25387 25361 0 -1 4202560 6995 0 0 0 26 54 0 0 18 0 32 0 24059268 2421121024 213132 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604636560186848 0 0 -1 14 0 0 0
[pid=25387/tid=25412] ppid=25385 vsize=2364376 CPUtime=0.62 cores=0-31
/proc/25387/task/25412/stat : 25412 (claspmt) S 25385 25387 25361 0 -1 4202560 6839 0 0 0 7 55 0 0 18 0 32 0 24059268 2421121024 213132 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604641227357664 0 0 -1 22 0 0 0
[pid=25387/tid=25413] ppid=25385 vsize=2364376 CPUtime=0.49 cores=0-31
/proc/25387/task/25413/stat : 25413 (claspmt) S 25385 25387 25361 0 -1 4202560 4764 0 0 0 7 42 0 0 18 0 32 0 24059268 2421121024 213132 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604642552987104 0 0 -1 21 0 0 0
[pid=25387/tid=25414] ppid=25385 vsize=2364376 CPUtime=0.75 cores=0-31
/proc/25387/task/25414/stat : 25414 (claspmt) S 25385 25387 25361 0 -1 4202560 7694 0 0 0 22 53 0 0 18 0 32 0 24059268 2421121024 213132 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604644029595104 0 0 -1 27 0 0 0
[pid=25387/tid=25415] ppid=25385 vsize=2364376 CPUtime=0.77 cores=0-31
/proc/25387/task/25415/stat : 25415 (claspmt) S 25385 25387 25361 0 -1 4202560 7310 0 0 0 23 54 0 0 18 0 32 0 24059268 2421121024 213132 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604636766150112 0 0 -1 25 0 0 0
[pid=25387/tid=25416] ppid=25385 vsize=2364376 CPUtime=0.55 cores=0-31
/proc/25387/task/25416/stat : 25416 (claspmt) S 25385 25387 25361 0 -1 4202560 5077 0 0 0 5 50 0 0 18 0 32 0 24059268 2421121024 213132 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604640087997920 0 0 -1 24 0 0 0
[pid=25387/tid=25417] ppid=25385 vsize=2364376 CPUtime=1.63 cores=0-31
/proc/25387/task/25417/stat : 25417 (claspmt) R 25385 25387 25361 0 -1 4202560 6024 0 0 0 110 53 0 0 19 0 32 0 24059268 2421121024 213132 270582939648 4194304 4964800 140735124092688 18446744073709551615 4668546 0 0 4096 18946 0 0 0 -1 25 0 0 0
[pid=25387/tid=25418] ppid=25385 vsize=2364376 CPUtime=0.77 cores=0-31
/proc/25387/task/25418/stat : 25418 (claspmt) S 25385 25387 25361 0 -1 4202560 8034 0 0 0 22 55 0 0 18 0 32 0 24059268 2421121024 213132 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604636720029152 0 0 -1 17 0 0 0
[pid=25387/tid=25419] ppid=25385 vsize=2364376 CPUtime=0.74 cores=0-31
/proc/25387/task/25419/stat : 25419 (claspmt) S 25385 25387 25361 0 -1 4202560 6549 0 0 0 27 47 0 0 18 0 32 0 24059268 2421121024 213132 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604638600666592 0 0 -1 10 0 0 0
Current children cumulated CPU time (s) 27.74
Current children cumulated vsize (KiB) 2364376

[startup+12.7014 s]
/proc/loadavg: 25.62 30.60 31.52 5/445 25419
/proc/meminfo: memFree=261766564/264257624 swapFree=67111528/67111528
[pid=25387] ppid=25385 vsize=2364376 CPUtime=53.32 cores=0-31
/proc/25387/stat : 25387 (claspmt) S 25385 25387 25361 0 -1 4202496 278680 0 0 0 3728 1604 0 0 18 0 32 0 24058889 2421121024 214500 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/25387/statm: 591094 214500 440 189 0 585443 0
[pid=25387/tid=25389] ppid=25385 vsize=2364376 CPUtime=0.62 cores=0-31
/proc/25387/task/25389/stat : 25389 (claspmt) S 25385 25387 25361 0 -1 4202560 6414 0 0 0 9 53 0 0 18 0 32 0 24059267 2421121024 214500 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604483707125216 0 0 -1 9 0 0 0
[pid=25387/tid=25390] ppid=25385 vsize=2364376 CPUtime=0.71 cores=0-31
/proc/25387/task/25390/stat : 25390 (claspmt) S 25385 25387 25361 0 -1 4202560 7690 0 0 0 23 48 0 0 18 0 32 0 24059267 2421121024 214500 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604637775273440 0 0 -1 23 0 0 0
[pid=25387/tid=25391] ppid=25385 vsize=2364376 CPUtime=0.79 cores=0-31
/proc/25387/task/25391/stat : 25391 (claspmt) S 25385 25387 25361 0 -1 4202560 8005 0 0 0 23 56 0 0 18 0 32 0 24059267 2421121024 214500 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604639914991072 0 0 -1 15 0 0 0
[pid=25387/tid=25392] ppid=25385 vsize=2364376 CPUtime=0.57 cores=0-31
/proc/25387/task/25392/stat : 25392 (claspmt) S 25385 25387 25361 0 -1 4202560 6438 0 0 0 5 52 0 0 18 0 32 0 24059267 2421121024 214500 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604641227390432 0 0 -1 8 0 0 0
[pid=25387/tid=25393] ppid=25385 vsize=2364376 CPUtime=7.66 cores=0-31
/proc/25387/task/25393/stat : 25393 (claspmt) R 25385 25387 25361 0 -1 4202560 8031 0 0 0 711 55 0 0 25 0 32 0 24059267 2421121024 214500 270582939648 4194304 4964800 140735124092688 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 15 0 0 0
[pid=25387/tid=25394] ppid=25385 vsize=2364376 CPUtime=0.76 cores=0-31
/proc/25387/task/25394/stat : 25394 (claspmt) S 25385 25387 25361 0 -1 4202560 7747 0 0 0 20 56 0 0 18 0 32 0 24059267 2421121024 214500 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604504799485408 0 0 -1 7 0 0 0
[pid=25387/tid=25395] ppid=25385 vsize=2364376 CPUtime=0.85 cores=0-31
/proc/25387/task/25395/stat : 25395 (claspmt) S 25385 25387 25361 0 -1 4202560 8480 0 0 0 25 60 0 0 18 0 32 0 24059267 2421121024 214500 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604641227365856 0 0 -1 25 0 0 0
[pid=25387/tid=25396] ppid=25385 vsize=2364376 CPUtime=0.64 cores=0-31
/proc/25387/task/25396/stat : 25396 (claspmt) S 25385 25387 25361 0 -1 4202560 6608 0 0 0 9 55 0 0 18 0 32 0 24059267 2421121024 214500 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604638822915552 0 0 -1 5 0 0 0
[pid=25387/tid=25397] ppid=25385 vsize=2364376 CPUtime=0.6 cores=0-31
/proc/25387/task/25397/stat : 25397 (claspmt) S 25385 25387 25361 0 -1 4202560 6338 0 0 0 8 52 0 0 18 0 32 0 24059267 2421121024 214500 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604643719765472 0 0 -1 13 0 0 0
[pid=25387/tid=25398] ppid=25385 vsize=2364376 CPUtime=0.79 cores=0-31
/proc/25387/task/25398/stat : 25398 (claspmt) S 25385 25387 25361 0 -1 4202560 7647 0 0 0 22 57 0 0 18 0 32 0 24059267 2421121024 214500 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604640715308512 0 0 -1 1 0 0 0
[pid=25387/tid=25399] ppid=25385 vsize=2364376 CPUtime=0.69 cores=0-31
/proc/25387/task/25399/stat : 25399 (claspmt) S 25385 25387 25361 0 -1 4202560 8363 0 0 0 23 46 0 0 18 0 32 0 24059267 2421121024 214500 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604636386287072 0 0 -1 11 0 0 0
[pid=25387/tid=25400] ppid=25385 vsize=2364376 CPUtime=0.63 cores=0-31
/proc/25387/task/25400/stat : 25400 (claspmt) S 25385 25387 25361 0 -1 4202560 6371 0 0 0 8 55 0 0 18 0 32 0 24059267 2421121024 214500 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604485026078176 0 0 -1 29 0 0 0
[pid=25387/tid=25401] ppid=25385 vsize=2364376 CPUtime=7.68 cores=0-31
/proc/25387/task/25401/stat : 25401 (claspmt) R 25385 25387 25361 0 -1 4202560 8117 0 0 0 712 56 0 0 25 0 32 0 24059267 2421121024 214500 270582939648 4194304 4964800 140735124092688 18446744073709551615 4399415 0 0 4096 18946 0 0 0 -1 28 0 0 0
[pid=25387/tid=25402] ppid=25385 vsize=2364376 CPUtime=0.79 cores=0-31
/proc/25387/task/25402/stat : 25402 (claspmt) S 25385 25387 25361 0 -1 4202560 7912 0 0 0 22 57 0 0 18 0 32 0 24059267 2421121024 214500 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604639314001376 0 0 -1 30 0 0 0
[pid=25387/tid=25403] ppid=25385 vsize=2364376 CPUtime=0.77 cores=0-31
/proc/25387/task/25403/stat : 25403 (claspmt) S 25385 25387 25361 0 -1 4202560 8162 0 0 0 25 52 0 0 18 0 32 0 24059267 2421121024 214500 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604640477887968 0 0 -1 3 0 0 0
[pid=25387/tid=25404] ppid=25385 vsize=2364376 CPUtime=0.58 cores=0-31
/proc/25387/task/25404/stat : 25404 (claspmt) S 25385 25387 25361 0 -1 4202560 5741 0 0 0 6 52 0 0 18 0 32 0 24059267 2421121024 214500 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604639754051040 0 0 -1 4 0 0 0
[pid=25387/tid=25405] ppid=25385 vsize=2364376 CPUtime=0.09 cores=0-31
/proc/25387/task/25405/stat : 25405 (claspmt) S 25385 25387 25361 0 -1 4202560 517 0 0 0 8 1 0 0 20 0 32 0 24059267 2421121024 214500 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604570363772384 0 0 -1 18 0 0 0
[pid=25387/tid=25406] ppid=25385 vsize=2364376 CPUtime=0.76 cores=0-31
/proc/25387/task/25406/stat : 25406 (claspmt) S 25385 25387 25361 0 -1 4202560 5493 0 0 0 21 55 0 0 18 0 32 0 24059267 2421121024 214500 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604501358026208 0 0 -1 1 0 0 0
[pid=25387/tid=25407] ppid=25385 vsize=2364376 CPUtime=0.78 cores=0-31
/proc/25387/task/25407/stat : 25407 (claspmt) S 25385 25387 25361 0 -1 4202560 7617 0 0 0 22 56 0 0 18 0 32 0 24059268 2421121024 214500 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604636922838496 0 0 -1 6 0 0 0
[pid=25387/tid=25408] ppid=25385 vsize=2364376 CPUtime=0.58 cores=0-31
/proc/25387/task/25408/stat : 25408 (claspmt) S 25385 25387 25361 0 -1 4202560 5828 0 0 0 7 51 0 0 18 0 32 0 24059268 2421121024 214500 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662772457 0 0 4096 18946 18446604639423765984 0 0 -1 20 0 0 0
[pid=25387/tid=25409] ppid=25385 vsize=2364376 CPUtime=7.77 cores=0-31

################
# More data... #
################

[pid=25387/tid=25406] ppid=25385 vsize=29468016 CPUtime=4827.92 cores=0-31
/proc/25387/task/25406/stat : 25406 (claspmt) R 25385 25387 25361 0 -1 4202560 774278 0 0 0 481780 1012 0 0 25 0 32 0 24059267 30175248384 7112847 270582939648 4194304 4964800 140735124092688 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 17 0 0 0
[pid=25387/tid=25407] ppid=25385 vsize=29468016 CPUtime=4822.14 cores=0-31
/proc/25387/task/25407/stat : 25407 (claspmt) R 25385 25387 25361 0 -1 4202560 170577 0 0 0 481844 370 0 0 25 0 32 0 24059268 30175248384 7112847 270582939648 4194304 4964800 140735124092688 18446744073709551615 4684107 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25387/tid=25408] ppid=25385 vsize=29468016 CPUtime=4828.51 cores=0-31
/proc/25387/task/25408/stat : 25408 (claspmt) R 25385 25387 25361 0 -1 4202560 195685 0 0 0 482377 474 0 0 25 0 32 0 24059268 30175248384 7112847 270582939648 4194304 4964800 140735124092688 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 13 0 0 0
[pid=25387/tid=25409] ppid=25385 vsize=29468016 CPUtime=5076.06 cores=0-31
/proc/25387/task/25409/stat : 25409 (claspmt) R 25385 25387 25361 0 -1 4202560 8554 0 0 0 507558 48 0 0 25 0 32 0 24059268 30175248384 7112847 270582939648 4194304 4964800 140735124092688 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 26 0 0 0
[pid=25387/tid=25410] ppid=25385 vsize=29468016 CPUtime=4804.16 cores=0-31
/proc/25387/task/25410/stat : 25410 (claspmt) R 25385 25387 25361 0 -1 4202560 203649 0 0 0 479970 446 0 0 25 0 32 0 24059268 30175248384 7112847 270582939648 4194304 4964800 140735124092688 18446744073709551615 4669131 0 0 4096 18946 0 0 0 -1 19 0 0 0
[pid=25387/tid=25411] ppid=25385 vsize=29468016 CPUtime=4826.46 cores=0-31
/proc/25387/task/25411/stat : 25411 (claspmt) R 25385 25387 25361 0 -1 4202560 269345 0 0 0 481931 715 0 0 25 0 32 0 24059268 30175248384 7112847 270582939648 4194304 4964800 140735124092688 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=25387/tid=25412] ppid=25385 vsize=29468016 CPUtime=4827.84 cores=0-31
/proc/25387/task/25412/stat : 25412 (claspmt) R 25385 25387 25361 0 -1 4202560 76812 0 0 0 482515 269 0 0 25 0 32 0 24059268 30175248384 7112847 270582939648 4194304 4964800 140735124092688 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=25387/tid=25413] ppid=25385 vsize=29468016 CPUtime=4828.25 cores=0-31
/proc/25387/task/25413/stat : 25413 (claspmt) R 25385 25387 25361 0 -1 4202560 67889 0 0 0 482585 240 0 0 25 0 32 0 24059268 30175248384 7112847 270582939648 4194304 4964800 140735124092688 18446744073709551615 4670670 0 0 4096 18946 0 0 0 -1 23 0 0 0
[pid=25387/tid=25414] ppid=25385 vsize=29468016 CPUtime=4824.7 cores=0-31
/proc/25387/task/25414/stat : 25414 (claspmt) R 25385 25387 25361 0 -1 4202560 635582 0 0 0 481598 872 0 0 25 0 32 0 24059268 30175248384 7112847 270582939648 4194304 4964800 140735124092688 18446744073709551615 4627365 0 0 4096 18946 0 0 0 -1 16 0 0 0
[pid=25387/tid=25415] ppid=25385 vsize=29468016 CPUtime=4829.19 cores=0-31
/proc/25387/task/25415/stat : 25415 (claspmt) R 25385 25387 25361 0 -1 4202560 193927 0 0 0 482476 443 0 0 25 0 32 0 24059268 30175248384 7112847 270582939648 4194304 4964800 140735124092688 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=25387/tid=25416] ppid=25385 vsize=29468016 CPUtime=4825.49 cores=0-31
/proc/25387/task/25416/stat : 25416 (claspmt) R 25385 25387 25361 0 -1 4202560 183208 0 0 0 482093 456 0 0 25 0 32 0 24059268 30175248384 7112847 270582939648 4194304 4964800 140735124092688 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 11 0 0 0
[pid=25387/tid=25417] ppid=25385 vsize=29468016 CPUtime=5074.1 cores=0-31
/proc/25387/task/25417/stat : 25417 (claspmt) R 25385 25387 25361 0 -1 4202560 6967 0 0 0 507355 55 0 0 25 0 32 0 24059268 30175248384 7112847 270582939648 4194304 4964800 140735124092688 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 8 0 0 0
[pid=25387/tid=25418] ppid=25385 vsize=29468016 CPUtime=4826.2 cores=0-31
/proc/25387/task/25418/stat : 25418 (claspmt) R 25385 25387 25361 0 -1 4202560 186203 0 0 0 482236 384 0 0 25 0 32 0 24059268 30175248384 7112847 270582939648 4194304 4964800 140735124092688 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 30 0 0 0
[pid=25387/tid=25419] ppid=25385 vsize=29468016 CPUtime=4829.35 cores=0-31
/proc/25387/task/25419/stat : 25419 (claspmt) R 25385 25387 25361 0 -1 4202560 297409 0 0 0 482167 768 0 0 25 0 32 0 24059268 30175248384 7112847 270582939648 4194304 4964800 140735124092688 18446744073709551615 4669978 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 155381
Current children cumulated vsize (KiB) 29468016



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5100 s]
/proc/loadavg: 32.12 32.13 32.02 33/445 25570
/proc/meminfo: memFree=234125580/264257624 swapFree=67111528/67111528
[pid=25387] ppid=25385 vsize=29468016 CPUtime=155947 cores=0-31
/proc/25387/stat : 25387 (claspmt) R 25385 25387 25361 0 -1 4202496 7179333 0 0 0 15579997 14740 0 0 25 0 32 0 24058889 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4669509 0 0 4096 18946 18446744073709551615 0 0 17 22 0 0 0
/proc/25387/statm: 7367004 7112851 463 189 0 7361353 0
[pid=25387/tid=25389] ppid=25385 vsize=29468016 CPUtime=4846.94 cores=0-31
/proc/25387/task/25389/stat : 25389 (claspmt) R 25385 25387 25361 0 -1 4202560 94432 0 0 0 484395 299 0 0 25 0 32 0 24059267 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25387/tid=25390] ppid=25385 vsize=29468016 CPUtime=4843.47 cores=0-31
/proc/25387/task/25390/stat : 25390 (claspmt) R 25385 25387 25361 0 -1 4202560 782400 0 0 0 483287 1060 0 0 25 0 32 0 24059267 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4363491 0 0 4096 18946 0 0 0 -1 28 0 0 0
[pid=25387/tid=25391] ppid=25385 vsize=29468016 CPUtime=4837.78 cores=0-31
/proc/25387/task/25391/stat : 25391 (claspmt) R 25385 25387 25361 0 -1 4202560 222814 0 0 0 483262 516 0 0 25 0 32 0 24059267 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4669386 0 0 4096 18946 0 0 0 -1 27 0 0 0
[pid=25387/tid=25392] ppid=25385 vsize=29468016 CPUtime=4843.65 cores=0-31
/proc/25387/task/25392/stat : 25392 (claspmt) R 25385 25387 25361 0 -1 4202560 185406 0 0 0 483895 470 0 0 25 0 32 0 24059267 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 24 0 0 0
[pid=25387/tid=25393] ppid=25385 vsize=29468016 CPUtime=5092.42 cores=0-31
/proc/25387/task/25393/stat : 25393 (claspmt) R 25385 25387 25361 0 -1 4202560 8689 0 0 0 509186 56 0 0 25 0 32 0 24059267 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 15 0 0 0
[pid=25387/tid=25394] ppid=25385 vsize=29468016 CPUtime=4821.1 cores=0-31
/proc/25387/task/25394/stat : 25394 (claspmt) R 25385 25387 25361 0 -1 4202560 228418 0 0 0 481580 530 0 0 25 0 32 0 24059267 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 12 0 0 0
[pid=25387/tid=25395] ppid=25385 vsize=29468016 CPUtime=4825.03 cores=0-31
/proc/25387/task/25395/stat : 25395 (claspmt) R 25385 25387 25361 0 -1 4202560 241490 0 0 0 481898 605 0 0 25 0 32 0 24059267 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 20 0 0 0
[pid=25387/tid=25396] ppid=25385 vsize=29468016 CPUtime=4847.33 cores=0-31
/proc/25387/task/25396/stat : 25396 (claspmt) R 25385 25387 25361 0 -1 4202560 88414 0 0 0 484417 316 0 0 25 0 32 0 24059267 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 10 0 0 0
[pid=25387/tid=25397] ppid=25385 vsize=29468016 CPUtime=4843.64 cores=0-31
/proc/25387/task/25397/stat : 25397 (claspmt) R 25385 25387 25361 0 -1 4202560 90948 0 0 0 484052 312 0 0 25 0 32 0 24059267 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 9 0 0 0
[pid=25387/tid=25398] ppid=25385 vsize=29468016 CPUtime=4838.82 cores=0-31
/proc/25387/task/25398/stat : 25398 (claspmt) R 25385 25387 25361 0 -1 4202560 747179 0 0 0 482812 1070 0 0 25 0 32 0 24059267 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4668552 0 0 4096 18946 0 0 0 -1 31 0 0 0
[pid=25387/tid=25399] ppid=25385 vsize=29468016 CPUtime=4847.12 cores=0-31
/proc/25387/task/25399/stat : 25399 (claspmt) R 25385 25387 25361 0 -1 4202560 257043 0 0 0 484181 531 0 0 25 0 32 0 24059267 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4669011 0 0 4096 18946 0 0 0 -1 25 0 0 0
[pid=25387/tid=25400] ppid=25385 vsize=29468016 CPUtime=4847.05 cores=0-31
/proc/25387/task/25400/stat : 25400 (claspmt) R 25385 25387 25361 0 -1 4202560 199433 0 0 0 484197 508 0 0 25 0 32 0 24059267 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 14 0 0 0
[pid=25387/tid=25401] ppid=25385 vsize=29468016 CPUtime=5090.81 cores=0-31
/proc/25387/task/25401/stat : 25401 (claspmt) R 25385 25387 25361 0 -1 4202560 8812 0 0 0 509011 70 0 0 25 0 32 0 24059267 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4399339 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25387/tid=25402] ppid=25385 vsize=29468016 CPUtime=4838.27 cores=0-31
/proc/25387/task/25402/stat : 25402 (claspmt) R 25385 25387 25361 0 -1 4202560 221302 0 0 0 483372 455 0 0 25 0 32 0 24059267 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=25387/tid=25403] ppid=25385 vsize=29468016 CPUtime=4843.85 cores=0-31
/proc/25387/task/25403/stat : 25403 (claspmt) R 25385 25387 25361 0 -1 4202560 231855 0 0 0 483784 601 0 0 25 0 32 0 24059267 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 29 0 0 0
[pid=25387/tid=25404] ppid=25385 vsize=29468016 CPUtime=4846.51 cores=0-31
/proc/25387/task/25404/stat : 25404 (claspmt) R 25385 25387 25361 0 -1 4202560 80836 0 0 0 484362 289 0 0 25 0 32 0 24059267 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4668987 0 0 4096 18946 0 0 0 -1 18 0 0 0
[pid=25387/tid=25405] ppid=25385 vsize=29468016 CPUtime=4844.48 cores=0-31
/proc/25387/task/25405/stat : 25405 (claspmt) R 25385 25387 25361 0 -1 4202560 75588 0 0 0 484203 245 0 0 25 0 32 0 24059267 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 21 0 0 0
[pid=25387/tid=25406] ppid=25385 vsize=29468016 CPUtime=4845.64 cores=0-31
/proc/25387/task/25406/stat : 25406 (claspmt) R 25385 25387 25361 0 -1 4202560 774278 0 0 0 483552 1012 0 0 25 0 32 0 24059267 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4669503 0 0 4096 18946 0 0 0 -1 17 0 0 0
[pid=25387/tid=25407] ppid=25385 vsize=29468016 CPUtime=4839.86 cores=0-31
/proc/25387/task/25407/stat : 25407 (claspmt) R 25385 25387 25361 0 -1 4202560 170577 0 0 0 483616 370 0 0 25 0 32 0 24059268 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4668987 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25387/tid=25408] ppid=25385 vsize=29468016 CPUtime=4846.23 cores=0-31
/proc/25387/task/25408/stat : 25408 (claspmt) R 25385 25387 25361 0 -1 4202560 195685 0 0 0 484149 474 0 0 25 0 32 0 24059268 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 13 0 0 0
[pid=25387/tid=25409] ppid=25385 vsize=29468016 CPUtime=5093.78 cores=0-31
/proc/25387/task/25409/stat : 25409 (claspmt) R 25385 25387 25361 0 -1 4202560 8554 0 0 0 509330 48 0 0 25 0 32 0 24059268 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 26 0 0 0
[pid=25387/tid=25410] ppid=25385 vsize=29468016 CPUtime=4821.87 cores=0-31
/proc/25387/task/25410/stat : 25410 (claspmt) R 25385 25387 25361 0 -1 4202560 203649 0 0 0 481741 446 0 0 25 0 32 0 24059268 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4321097 0 0 4096 18946 0 0 0 -1 19 0 0 0
[pid=25387/tid=25411] ppid=25385 vsize=29468016 CPUtime=4844.18 cores=0-31
/proc/25387/task/25411/stat : 25411 (claspmt) R 25385 25387 25361 0 -1 4202560 269345 0 0 0 483703 715 0 0 25 0 32 0 24059268 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4669022 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=25387/tid=25412] ppid=25385 vsize=29468016 CPUtime=4845.56 cores=0-31
/proc/25387/task/25412/stat : 25412 (claspmt) R 25385 25387 25361 0 -1 4202560 76812 0 0 0 484287 269 0 0 25 0 32 0 24059268 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=25387/tid=25413] ppid=25385 vsize=29468016 CPUtime=4845.97 cores=0-31
/proc/25387/task/25413/stat : 25413 (claspmt) R 25385 25387 25361 0 -1 4202560 67889 0 0 0 484357 240 0 0 25 0 32 0 24059268 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4669471 0 0 4096 18946 0 0 0 -1 23 0 0 0
[pid=25387/tid=25414] ppid=25385 vsize=29468016 CPUtime=4842.4 cores=0-31
/proc/25387/task/25414/stat : 25414 (claspmt) R 25385 25387 25361 0 -1 4202560 635586 0 0 0 483368 872 0 0 25 0 32 0 24059268 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4313761 0 0 4096 18946 0 0 0 -1 16 0 0 0
[pid=25387/tid=25415] ppid=25385 vsize=29468016 CPUtime=4846.91 cores=0-31
/proc/25387/task/25415/stat : 25415 (claspmt) R 25385 25387 25361 0 -1 4202560 193927 0 0 0 484248 443 0 0 25 0 32 0 24059268 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=25387/tid=25416] ppid=25385 vsize=29468016 CPUtime=4843.21 cores=0-31
/proc/25387/task/25416/stat : 25416 (claspmt) R 25385 25387 25361 0 -1 4202560 183208 0 0 0 483865 456 0 0 25 0 32 0 24059268 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 11 0 0 0
[pid=25387/tid=25417] ppid=25385 vsize=29468016 CPUtime=5091.82 cores=0-31
/proc/25387/task/25417/stat : 25417 (claspmt) R 25385 25387 25361 0 -1 4202560 6967 0 0 0 509127 55 0 0 25 0 32 0 24059268 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4669496 0 0 4096 18946 0 0 0 -1 8 0 0 0
[pid=25387/tid=25418] ppid=25385 vsize=29468016 CPUtime=4843.91 cores=0-31
/proc/25387/task/25418/stat : 25418 (claspmt) R 25385 25387 25361 0 -1 4202560 186203 0 0 0 484007 384 0 0 25 0 32 0 24059268 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 30 0 0 0
[pid=25387/tid=25419] ppid=25385 vsize=29468016 CPUtime=4847.07 cores=0-31
/proc/25387/task/25419/stat : 25419 (claspmt) R 25385 25387 25361 0 -1 4202560 297409 0 0 0 483939 768 0 0 25 0 32 0 24059268 30175248384 7112851 270582939648 4194304 4964800 140735124092688 18446744073709551615 4627652 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 155947
Current children cumulated vsize (KiB) 29468016

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)

Sending SIGKILL to -25387

Solver just ended. Dumping a history of the last processes samples

[startup+5100.6 s]
/proc/loadavg: 32.12 32.13 32.02 2/416 25571
/proc/meminfo: memFree=234125812/264257624 swapFree=67111528/67111528
[pid=25387] ppid=25385 vsize=29402416 CPUtime=155948 cores=0-31
/proc/25387/stat : 25387 (claspmt) S 25385 25387 25361 0 -1 4202496 7179335 0 0 0 15580110 14740 0 0 25 0 2 0 24058889 30108073984 7112821 270582939648 4194304 4964800 140735124092688 18446744073709551615 258662759221 0 0 4096 18946 18446744073709551615 0 0 17 22 0 0 0
/proc/25387/statm: 7350604 7112821 465 189 0 7344953 0
[pid=25387/tid=25414] ppid=25385 vsize=29402416 CPUtime=4843.01 cores=0-31
/proc/25387/task/25414/stat : 25414 (claspmt) R 25385 25387 25361 0 -1 4202560 635586 0 0 0 483429 872 0 0 25 0 2 0 24059268 30108073984 7112821 270582939648 4194304 4964800 140735124092688 18446744073709551615 4313751 0 0 4096 18946 0 0 0 -1 16 0 0 0
Current children cumulated CPU time (s) 155948
Current children cumulated vsize (KiB) 29402416

[startup+5102.2 s]
/proc/loadavg: 32.12 32.13 32.02 2/416 25571
/proc/meminfo: memFree=234125812/264257624 swapFree=67111528/67111528
[pid=25387] ppid=25385 vsize=0 CPUtime=155950 cores=0-31
/proc/25387/stat : 25387 (claspmt) R 25385 25387 25361 0 -1 4203524 7179335 0 0 0 15580250 14757 0 0 18 0 1 0 24058889 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 22 0 0 0
/proc/25387/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 155950
Current children cumulated vsize (KiB) 0

[startup+5103 s]
/proc/loadavg: 32.12 32.13 32.02 2/414 25571
/proc/meminfo: memFree=242377052/264257624 swapFree=67111528/67111528
[pid=25387] ppid=25385 vsize=0 CPUtime=155951 cores=0-31
/proc/25387/stat : 25387 (claspmt) R 25385 25387 25361 0 -1 4203524 7179335 0 0 0 15580250 14837 0 0 19 0 1 0 24058889 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 22 0 0 0
/proc/25387/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 155951
Current children cumulated vsize (KiB) 0

[startup+5103.4 s]
/proc/loadavg: 32.12 32.13 32.02 2/414 25571
/proc/meminfo: memFree=242377052/264257624 swapFree=67111528/67111528
[pid=25387] ppid=25385 vsize=0 CPUtime=155951 cores=0-31
/proc/25387/stat : 25387 (claspmt) R 25385 25387 25361 0 -1 4203524 7179335 0 0 0 15580250 14877 0 0 19 0 1 0 24058889 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 22 0 0 0
/proc/25387/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 155951
Current children cumulated vsize (KiB) 0

[startup+5103.6 s]
/proc/loadavg: 32.12 32.13 32.02 2/414 25571
/proc/meminfo: memFree=242377052/264257624 swapFree=67111528/67111528
[pid=25387] ppid=25385 vsize=0 CPUtime=155951 cores=0-31
/proc/25387/stat : 25387 (claspmt) R 25385 25387 25361 0 -1 4203524 7179335 0 0 0 15580250 14897 0 0 20 0 1 0 24058889 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 22 0 0 0
/proc/25387/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 155951
Current children cumulated vsize (KiB) 0

[startup+5103.7 s]
/proc/loadavg: 32.12 32.13 32.02 2/414 25571
/proc/meminfo: memFree=242377052/264257624 swapFree=67111528/67111528
[pid=25387] ppid=25385 vsize=0 CPUtime=155952 cores=0-31
/proc/25387/stat : 25387 (claspmt) R 25385 25387 25361 0 -1 4203524 7179335 0 0 0 15580250 14907 0 0 20 0 1 0 24058889 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 22 0 0 0
/proc/25387/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 155952
Current children cumulated vsize (KiB) 0

Child ended because it received signal 9 (SIGKILL)
Real time (s): 5103.74
CPU time (s): 155952
CPU user time (s): 155802
CPU system time (s): 149.113
CPU usage (%): 3055.63
Max. virtual memory (cumulated for all children) (KiB): 29468016

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 155803
system time used= 149.113
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 7179335
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 472416
involuntary context switches= 354716

runsolver used 28.0077 second user time and 115.544 second system time

The end

Launcher Data 

Begin job on node200 at 2011-06-09 04:32:00
IDJOB=3472379
IDBENCH=82394
IDSOLVER=1961
FILE ID=node200/3472379-1307586718
RUNJOBID= node200-1307586718-25375
PBS_JOBID= 13547689
Free space on /tmp= 61696 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT11/application/rintanen/SATPlanning/sokoban-sequential-p145-microban-sequential.080-SAT.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3472379-1307586718/watcher-3472379-1307586718 -o /tmp/evaluation-result-3472379-1307586718/solver-3472379-1307586718 -C 160000 -W 5100 -M 256000  HOME/claspmt -t32 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3472379-1307586718.cnf

TIME LIMIT= 160000 seconds
MEMORY LIMIT= 256000 MiB
NBCORE= 32

MD5SUM BENCH= 1ca09a2b911e29894c6603aee8d3e672
RANDOM SEED=1755658808

node200.alineos.net Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 64
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 96
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 66
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 98
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.99
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 68
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 100
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.03
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 70
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 102
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 16
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 17
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 48
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 18
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 80
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 19
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 112
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.97
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 20
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 21
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 50
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.10
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 22
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 82
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.10
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 114
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 24
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 25
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 26
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 84
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 27
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 116
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 28
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 29
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 54
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 30
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 86
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 31
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 118
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     264257624 kB
MemFree:      262623060 kB
Buffers:        207988 kB
Cached:        1136536 kB
SwapCached:          0 kB
Active:         276616 kB
Inactive:      1131124 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     264257624 kB
LowFree:      262623060 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           51480 kB
Writeback:           0 kB
AnonPages:       63400 kB
Mapped:          14476 kB
Slab:           145892 kB
PageTables:       4444 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  199240340 kB
Committed_AS:   159868 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296372 kB
VmallocChunk: 34359440603 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 61692 MiB
End job on node200 at 2011-06-09 05:57:04