Trace number 3472286

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 20429.6 5102.25

General information on the benchmark

Nameindustrial/vliw_sat_4.0/
9vliw_m_9stages_iq3_C1_bug6.cnf
MD5SUMabc624ca713aad9563ab3886d918a526
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark37.1444
Satisfiable
(Un)Satisfiability was proved
Number of variables521192
Number of clauses13378781
Sum of the clauses size39205391
Maximum clause length6689
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 28714698
Number of clauses of size 34209419
Number of clauses of size 425257
Number of clauses of size 515681
Number of clauses of size over 5413725

Solver Data 

0.00/0.01	c clasp version 2.0.0-RC2
0.00/0.01	c Reading from ...403950/instance-3472286-1307403950.cnf
5.99/6.05	c Solving...
20427.29/5100.02	
20427.29/5100.02	*** INTERRUPTED! ***
20427.29/5100.02	*** Shutting down threads...

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3472286-1307403950/watcher-3472286-1307403950 -o /tmp/evaluation-result-3472286-1307403950/solver-3472286-1307403950 -C 160000 -W 5100 -M 256000 HOME/claspmt -t32 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3472286-1307403950.cnf 

running on 32 cores: 0,4,8,12,16,20,24,28,1,5,9,13,17,21,25,29,2,6,10,14,18,22,26,30,3,7,11,15,19,23,27,31

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 160000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 160030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 262144000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 262195200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.34 3.87 4.06 1/413 11905
/proc/meminfo: memFree=263230764/264257624 swapFree=67111528/67111528
[pid=11905] ppid=11903 vsize=87052 CPUtime=0.03 cores=0-31
/proc/11905/stat : 11905 (claspmt) R 11903 11905 11878 0 -1 4202496 1935 0 0 0 3 0 0 0 25 0 1 0 5783146 89141248 1865 270582939648 4194304 4964800 140733234300256 18446744073709551615 4700412 0 0 4096 18946 0 0 0 17 25 0 0 0
/proc/11905/statm: 21763 1870 373 189 0 16637 0

[startup+0.129488 s]
/proc/loadavg: 3.34 3.87 4.06 1/413 11905
/proc/meminfo: memFree=263230764/264257624 swapFree=67111528/67111528
[pid=11905] ppid=11903 vsize=87052 CPUtime=0.12 cores=0-31
/proc/11905/stat : 11905 (claspmt) R 11903 11905 11878 0 -1 4202496 9387 0 0 0 10 2 0 0 25 0 1 0 5783146 89141248 9317 270582939648 4194304 4964800 140733234300256 18446744073709551615 4700412 0 0 4096 18946 0 0 0 17 25 0 0 0
/proc/11905/statm: 21763 9322 373 189 0 16637 0
Current children cumulated CPU time (s) 0.12
Current children cumulated vsize (KiB) 87052

[startup+0.200472 s]
/proc/loadavg: 3.34 3.87 4.06 1/413 11905
/proc/meminfo: memFree=263230764/264257624 swapFree=67111528/67111528
[pid=11905] ppid=11903 vsize=87052 CPUtime=0.19 cores=0-31
/proc/11905/stat : 11905 (claspmt) R 11903 11905 11878 0 -1 4202496 15038 0 0 0 16 3 0 0 25 0 1 0 5783146 89141248 14968 270582939648 4194304 4964800 140733234300256 18446744073709551615 4700412 0 0 4096 18946 0 0 0 17 25 0 0 0
/proc/11905/statm: 21763 14973 373 189 0 16637 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 87052

[startup+0.300492 s]
/proc/loadavg: 3.34 3.87 4.06 1/413 11905
/proc/meminfo: memFree=263230764/264257624 swapFree=67111528/67111528
[pid=11905] ppid=11903 vsize=117592 CPUtime=0.29 cores=0-31
/proc/11905/stat : 11905 (claspmt) R 11903 11905 11878 0 -1 4202496 28766 0 0 0 23 6 0 0 25 0 1 0 5783146 120414208 24625 270582939648 4194304 4964800 140733234300256 18446744073709551615 258650999114 0 0 4096 18946 0 0 0 17 25 0 0 0
/proc/11905/statm: 30925 24632 377 189 0 25799 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 117592

[startup+0.700505 s]
/proc/loadavg: 3.34 3.87 4.06 1/413 11905
/proc/meminfo: memFree=263230764/264257624 swapFree=67111528/67111528
[pid=11905] ppid=11903 vsize=135744 CPUtime=0.69 cores=0-31
/proc/11905/stat : 11905 (claspmt) R 11903 11905 11878 0 -1 4202496 32888 0 0 0 60 9 0 0 25 0 1 0 5783146 139001856 28664 270582939648 4194304 4964800 140733234300256 18446744073709551615 258650955760 0 0 4096 18946 0 0 0 17 25 0 0 0
/proc/11905/statm: 33936 28664 385 189 0 28810 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 135744

[startup+1.50053 s]
/proc/loadavg: 3.15 3.82 4.04 2/414 11906
/proc/meminfo: memFree=263096900/264257624 swapFree=67111528/67111528
[pid=11905] ppid=11903 vsize=177220 CPUtime=1.49 cores=0-31
/proc/11905/stat : 11905 (claspmt) R 11903 11905 11878 0 -1 4202496 43717 0 0 0 134 15 0 0 25 0 1 0 5783146 181473280 38888 270582939648 4194304 4964800 140733234300256 18446744073709551615 258650955760 0 0 4096 18946 0 0 0 17 25 0 0 0
/proc/11905/statm: 44305 38889 385 189 0 39179 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 177220

[startup+3.10059 s]
/proc/loadavg: 3.15 3.82 4.04 2/414 11906
/proc/meminfo: memFree=263038744/264257624 swapFree=67111528/67111528
[pid=11905] ppid=11903 vsize=263872 CPUtime=3.09 cores=0-31
/proc/11905/stat : 11905 (claspmt) R 11903 11905 11878 0 -1 4202496 65966 0 0 0 284 25 0 0 25 0 1 0 5783146 270204928 60197 270582939648 4194304 4964800 140733234300256 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 17 25 0 0 0
/proc/11905/statm: 65968 60198 385 189 0 60842 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 263872

[startup+6.30293 s]
/proc/loadavg: 3.15 3.82 4.04 2/414 11906
/proc/meminfo: memFree=262850604/264257624 swapFree=67111528/67111528
[pid=11905] ppid=11903 vsize=1548040 CPUtime=12.8 cores=0-31
/proc/11905/stat : 11905 (claspmt) S 11903 11905 11878 0 -1 4202496 192007 0 0 0 577 703 0 0 18 0 32 0 5783146 1585192960 164320 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446744073709551615 0 0 17 25 0 0 0
/proc/11905/statm: 387010 164333 424 189 0 381359 0
Current children cumulated CPU time (s) 12.8
Current children cumulated vsize (KiB) 1548040

[startup+12.7028 s]
/proc/loadavg: 4.58 4.09 4.13 7/445 11937
/proc/meminfo: memFree=260810804/264257624 swapFree=67111528/67111528
[pid=11905] ppid=11903 vsize=3844968 CPUtime=69.93 cores=0-31
/proc/11905/stat : 11905 (claspmt) S 11903 11905 11878 0 -1 4202496 785901 0 0 0 1297 5696 0 0 18 0 32 0 5783146 3937247232 642778 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446744073709551615 0 0 17 25 0 0 0
/proc/11905/statm: 961242 642780 428 189 0 955591 0
[pid=11905/tid=11907] ppid=11903 vsize=3844968 CPUtime=2.06 cores=0-31
/proc/11905/task/11907/stat : 11907 (claspmt) S 11903 11905 11878 0 -1 4202560 21371 0 0 0 22 184 0 0 15 0 32 0 5783751 3937247232 642789 270582939648 4194304 4964800 140733234300256 18446744073709551615 258651060958 0 0 4096 18946 18446744071566930144 0 0 -1 26 0 0 0
[pid=11905/tid=11908] ppid=11903 vsize=3844968 CPUtime=2.04 cores=0-31
/proc/11905/task/11908/stat : 11908 (claspmt) D 11903 11905 11878 0 -1 4202560 21040 0 0 0 20 184 0 0 16 0 32 0 5783751 3937247232 642796 270582939648 4194304 4964800 140733234300256 18446744073709551615 258650653289 0 0 4096 18946 0 0 0 -1 24 0 0 0
[pid=11905/tid=11909] ppid=11903 vsize=3844968 CPUtime=2.08 cores=0-31
/proc/11905/task/11909/stat : 11909 (claspmt) S 11903 11905 11878 0 -1 4202560 21667 0 0 0 18 190 0 0 16 0 32 0 5783751 3937247232 642802 270582939648 4194304 4964800 140733234300256 18446744073709551615 258651060958 0 0 4096 18946 18446604568489336288 0 0 -1 14 0 0 0
[pid=11905/tid=11910] ppid=11903 vsize=3844968 CPUtime=2.06 cores=0-31
/proc/11905/task/11910/stat : 11910 (claspmt) D 11903 11905 11878 0 -1 4202560 18583 0 0 0 24 182 0 0 16 0 32 0 5783751 3937247232 642806 270582939648 4194304 4964800 140733234300256 18446744073709551615 4685574 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11905/tid=11911] ppid=11903 vsize=3844968 CPUtime=1.99 cores=0-31
/proc/11905/task/11911/stat : 11911 (claspmt) D 11903 11905 11878 0 -1 4202560 22857 0 0 0 33 166 0 0 16 0 32 0 5783751 3937247232 642806 270582939648 4194304 4964800 140733234300256 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 -1 16 0 0 0
[pid=11905/tid=11912] ppid=11903 vsize=3844968 CPUtime=2.11 cores=0-31
/proc/11905/task/11912/stat : 11912 (claspmt) R 11903 11905 11878 0 -1 4202560 20511 0 0 0 22 189 0 0 16 0 32 0 5783751 3937247232 642806 270582939648 4194304 4964800 140733234300256 18446744073709551615 258650616986 0 0 4096 18946 38150144 0 0 -1 3 0 0 0
[pid=11905/tid=11913] ppid=11903 vsize=3844968 CPUtime=1.87 cores=0-31
/proc/11905/task/11913/stat : 11913 (claspmt) D 11903 11905 11878 0 -1 4202560 19748 0 0 0 22 165 0 0 16 0 32 0 5783751 3937247232 642808 270582939648 4194304 4964800 140733234300256 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 23 0 0 0
[pid=11905/tid=11914] ppid=11903 vsize=3844968 CPUtime=2.13 cores=0-31
/proc/11905/task/11914/stat : 11914 (claspmt) D 11903 11905 11878 0 -1 4202560 20933 0 0 0 20 193 0 0 16 0 32 0 5783751 3937247232 642815 270582939648 4194304 4964800 140733234300256 18446744073709551615 4685574 0 0 4096 18946 0 0 0 -1 29 0 0 0
[pid=11905/tid=11915] ppid=11903 vsize=3844968 CPUtime=2.03 cores=0-31
/proc/11905/task/11915/stat : 11915 (claspmt) R 11903 11905 11878 0 -1 4202560 22757 0 0 0 31 172 0 0 16 0 32 0 5783751 3937247232 642825 270582939648 4194304 4964800 140733234300256 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11905/tid=11916] ppid=11903 vsize=3844968 CPUtime=2.04 cores=0-31
/proc/11905/task/11916/stat : 11916 (claspmt) R 11903 11905 11878 0 -1 4202560 22011 0 0 0 23 181 0 0 16 0 32 0 5783751 3937247232 642837 270582939648 4194304 4964800 140733234300256 18446744073709551615 4685574 0 0 4096 18946 0 0 0 -1 28 0 0 0
[pid=11905/tid=11917] ppid=11903 vsize=3844968 CPUtime=2.08 cores=0-31
/proc/11905/task/11917/stat : 11917 (claspmt) D 11903 11905 11878 0 -1 4202560 20053 0 0 0 21 187 0 0 15 0 32 0 5783751 3937247232 642837 270582939648 4194304 4964800 140733234300256 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11905/tid=11918] ppid=11903 vsize=3844968 CPUtime=2.11 cores=0-31
/proc/11905/task/11918/stat : 11918 (claspmt) D 11903 11905 11878 0 -1 4202560 21013 0 0 0 23 188 0 0 15 0 32 0 5783751 3937247232 642841 270582939648 4194304 4964800 140733234300256 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11905/tid=11919] ppid=11903 vsize=3844968 CPUtime=1.91 cores=0-31
/proc/11905/task/11919/stat : 11919 (claspmt) R 11903 11905 11878 0 -1 4202560 21277 0 0 0 23 168 0 0 16 0 32 0 5783751 3937247232 642842 270582939648 4194304 4964800 140733234300256 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 15 0 0 0
[pid=11905/tid=11920] ppid=11903 vsize=3844968 CPUtime=2.11 cores=0-31
/proc/11905/task/11920/stat : 11920 (claspmt) D 11903 11905 11878 0 -1 4202560 21248 0 0 0 22 189 0 0 16 0 32 0 5783751 3937247232 642843 270582939648 4194304 4964800 140733234300256 18446744073709551615 258650653289 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11905/tid=11921] ppid=11903 vsize=3844968 CPUtime=2.09 cores=0-31
/proc/11905/task/11921/stat : 11921 (claspmt) D 11903 11905 11878 0 -1 4202560 22122 0 0 0 20 189 0 0 16 0 32 0 5783751 3937247232 642845 270582939648 4194304 4964800 140733234300256 18446744073709551615 258650653299 0 0 4096 18946 0 0 0 -1 25 0 0 0
[pid=11905/tid=11922] ppid=11903 vsize=3844968 CPUtime=1.99 cores=0-31
/proc/11905/task/11922/stat : 11922 (claspmt) D 11903 11905 11878 0 -1 4202560 22317 0 0 0 26 173 0 0 16 0 32 0 5783751 3937247232 642849 270582939648 4194304 4964800 140733234300256 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 21 0 0 0
[pid=11905/tid=11923] ppid=11903 vsize=3844968 CPUtime=2.02 cores=0-31
/proc/11905/task/11923/stat : 11923 (claspmt) D 11903 11905 11878 0 -1 4202560 22010 0 0 0 26 176 0 0 16 0 32 0 5783751 3937247232 642852 270582939648 4194304 4964800 140733234300256 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 27 0 0 0
[pid=11905/tid=11924] ppid=11903 vsize=3844968 CPUtime=2.12 cores=0-31
/proc/11905/task/11924/stat : 11924 (claspmt) S 11903 11905 11878 0 -1 4202560 20565 0 0 0 25 187 0 0 16 0 32 0 5783751 3937247232 642871 270582939648 4194304 4964800 140733234300256 18446744073709551615 258651060958 0 0 4096 18946 18446744071566930304 0 0 -1 4 0 0 0
[pid=11905/tid=11925] ppid=11903 vsize=3844968 CPUtime=2.1 cores=0-31
/proc/11905/task/11925/stat : 11925 (claspmt) S 11903 11905 11878 0 -1 4202560 21901 0 0 0 20 190 0 0 15 0 32 0 5783751 3937247232 642871 270582939648 4194304 4964800 140733234300256 18446744073709551615 258651060958 0 0 4096 18946 18446744071566930384 0 0 -1 1 0 0 0
[pid=11905/tid=11926] ppid=11903 vsize=3844968 CPUtime=2.08 cores=0-31
/proc/11905/task/11926/stat : 11926 (claspmt) D 11903 11905 11878 0 -1 4202560 21209 0 0 0 22 186 0 0 16 0 32 0 5783751 3937247232 642871 270582939648 4194304 4964800 140733234300256 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 17 0 0 0
[pid=11905/tid=11927] ppid=11903 vsize=3844968 CPUtime=2.11 cores=0-31
/proc/11905/task/11927/stat : 11927 (claspmt) D 11903 11905 11878 0 -1 4202560 21228 0 0 0 25 186 0 0 16 0 32 0 5783751 3937247232 642873 270582939648 4194304 4964800 140733234300256 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 13 0 0 0
[pid=11905/tid=11928] ppid=11903 vsize=3844968 CPUtime=1.95 cores=0-31
/proc/11905/task/11928/stat : 11928 (claspmt) S 11903 11905 11878 0 -1 4202560 24199 0 0 0 31 164 0 0 15 0 32 0 5783751 3937247232 642873 270582939648 4194304 4964800 140733234300256 18446744073709551615 258651060958 0 0 4096 18946 18446744071566929064 0 0 -1 8 0 0 0
[pid=11905/tid=11929] ppid=11903 vsize=3844968 CPUtime=2.08 cores=0-31
/proc/11905/task/11929/stat : 11929 (claspmt) D 11903 11905 11878 0 -1 4202560 22612 0 0 0 18 190 0 0 16 0 32 0 5783751 3937247232 642876 270582939648 4194304 4964800 140733234300256 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 10 0 0 0
[pid=11905/tid=11930] ppid=11903 vsize=3844968 CPUtime=2.07 cores=0-31
/proc/11905/task/11930/stat : 11930 (claspmt) D 11903 11905 11878 0 -1 4202560 22281 0 0 0 26 181 0 0 15 0 32 0 5783752 3937247232 642878 270582939648 4194304 4964800 140733234300256 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 11 0 0 0
[pid=11905/tid=11931] ppid=11903 vsize=3844968 CPUtime=2.1 cores=0-31
/proc/11905/task/11931/stat : 11931 (claspmt) D 11903 11905 11878 0 -1 4202560 21448 0 0 0 29 181 0 0 16 0 32 0 5783752 3937247232 642880 270582939648 4194304 4964800 140733234300256 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 12 0 0 0
[pid=11905/tid=11932] ppid=11903 vsize=3844968 CPUtime=2.07 cores=0-31
/proc/11905/task/11932/stat : 11932 (claspmt) D 11903 11905 11878 0 -1 4202560 21660 0 0 0 20 187 0 0 16 0 32 0 5783752 3937247232 642883 270582939648 4194304 4964800 140733234300256 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 22 0 0 0
[pid=11905/tid=11933] ppid=11903 vsize=3844968 CPUtime=1.74 cores=0-31
/proc/11905/task/11933/stat : 11933 (claspmt) S 11903 11905 11878 0 -1 4202560 21660 0 0 0 22 152 0 0 16 0 32 0 5783752 3937247232 642890 270582939648 4194304 4964800 140733234300256 18446744073709551615 258651060958 0 0 4096 18946 18446744071566933224 0 0 -1 19 0 0 0
[pid=11905/tid=11934] ppid=11903 vsize=3844968 CPUtime=2.11 cores=0-31
/proc/11905/task/11934/stat : 11934 (claspmt) D 11903 11905 11878 0 -1 4202560 21244 0 0 0 24 187 0 0 16 0 32 0 5783752 3937247232 642895 270582939648 4194304 4964800 140733234300256 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 9 0 0 0
[pid=11905/tid=11935] ppid=11903 vsize=3844968 CPUtime=2.1 cores=0-31
/proc/11905/task/11935/stat : 11935 (claspmt) D 11903 11905 11878 0 -1 4202560 21415 0 0 0 23 187 0 0 15 0 32 0 5783752 3937247232 642900 270582939648 4194304 4964800 140733234300256 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 18 0 0 0
[pid=11905/tid=11936] ppid=11903 vsize=3844968 CPUtime=2.08 cores=0-31
/proc/11905/task/11936/stat : 11936 (claspmt) S 11903 11905 11878 0 -1 4202560 22159 0 0 0 23 185 0 0 16 0 32 0 5783752 3937247232 642904 270582939648 4194304 4964800 140733234300256 18446744073709551615 258651060958 0 0 4096 18946 18446744071566930904 0 0 -1 20 0 0 0
[pid=11905/tid=11937] ppid=11903 vsize=3844968 CPUtime=2.11 cores=0-31
/proc/11905/task/11937/stat : 11937 (claspmt) D 11903 11905 11878 0 -1 4202560 21743 0 0 0 21 190 0 0 16 0 32 0 5783752 3937247232 642908 270582939648 4194304 4964800 140733234300256 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 30 0 0 0
Current children cumulated CPU time (s) 69.93
Current children cumulated vsize (KiB) 3844968

[startup+25.5022 s]
/proc/loadavg: 5.82 4.38 4.22 5/445 11937
/proc/meminfo: memFree=259451272/264257624 swapFree=67111528/67111528
[pid=11905] ppid=11903 vsize=5042004 CPUtime=133.04 cores=0-31
/proc/11905/stat : 11905 (claspmt) S 11903 11905 11878 0 -1 4202496 1089154 0 0 0 6697 6607 0 0 18 0 32 0 5783146 5163012096 945459 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446744073709551615 0 0 17 25 0 0 0
/proc/11905/statm: 1260501 945459 433 189 0 1254850 0
[pid=11905/tid=11907] ppid=11903 vsize=5042004 CPUtime=2.49 cores=0-31
/proc/11905/task/11907/stat : 11907 (claspmt) S 11903 11905 11878 0 -1 4202560 27651 0 0 0 37 212 0 0 16 0 32 0 5783751 5163012096 945459 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604568821571040 0 0 -1 26 0 0 0
[pid=11905/tid=11908] ppid=11903 vsize=5042004 CPUtime=3.23 cores=0-31
/proc/11905/task/11908/stat : 11908 (claspmt) S 11903 11905 11878 0 -1 4202560 34518 0 0 0 109 214 0 0 16 0 32 0 5783751 5163012096 945459 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604574196358624 0 0 -1 24 0 0 0
[pid=11905/tid=11909] ppid=11903 vsize=5042004 CPUtime=3.2 cores=0-31
/proc/11905/task/11909/stat : 11909 (claspmt) S 11903 11905 11878 0 -1 4202560 32729 0 0 0 100 220 0 0 16 0 32 0 5783751 5163012096 945459 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604568986099168 0 0 -1 14 0 0 0
[pid=11905/tid=11910] ppid=11903 vsize=5042004 CPUtime=2.61 cores=0-31
/proc/11905/task/11910/stat : 11910 (claspmt) S 11903 11905 11878 0 -1 4202560 27065 0 0 0 49 212 0 0 16 0 32 0 5783751 5163012096 945459 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604568780340704 0 0 -1 7 0 0 0
[pid=11905/tid=11911] ppid=11903 vsize=5042004 CPUtime=11.8 cores=0-31
/proc/11905/task/11911/stat : 11911 (claspmt) R 11903 11905 11878 0 -1 4202560 33031 0 0 0 986 194 0 0 25 0 32 0 5783751 5163012096 945459 270582939648 4194304 4964800 140733234300256 18446744073709551615 4669131 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11905/tid=11912] ppid=11903 vsize=5042004 CPUtime=3.34 cores=0-31
/proc/11905/task/11912/stat : 11912 (claspmt) S 11903 11905 11878 0 -1 4202560 34679 0 0 0 112 222 0 0 16 0 32 0 5783751 5163012096 945459 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604572395593184 0 0 -1 3 0 0 0
[pid=11905/tid=11913] ppid=11903 vsize=5042004 CPUtime=3.37 cores=0-31
/proc/11905/task/11913/stat : 11913 (claspmt) S 11903 11905 11878 0 -1 4202560 30881 0 0 0 134 203 0 0 16 0 32 0 5783751 5163012096 945459 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604569548996064 0 0 -1 23 0 0 0
[pid=11905/tid=11914] ppid=11903 vsize=5042004 CPUtime=2.54 cores=0-31
/proc/11905/task/11914/stat : 11914 (claspmt) S 11903 11905 11878 0 -1 4202560 27586 0 0 0 35 219 0 0 16 0 32 0 5783751 5163012096 945459 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604574709022176 0 0 -1 29 0 0 0
[pid=11905/tid=11915] ppid=11903 vsize=5042004 CPUtime=2.37 cores=0-31
/proc/11905/task/11915/stat : 11915 (claspmt) S 11903 11905 11878 0 -1 4202560 27544 0 0 0 46 191 0 0 16 0 32 0 5783751 5163012096 945459 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604639231000032 0 0 -1 7 0 0 0
[pid=11905/tid=11916] ppid=11903 vsize=5042004 CPUtime=3.24 cores=0-31
/proc/11905/task/11916/stat : 11916 (claspmt) S 11903 11905 11878 0 -1 4202560 35110 0 0 0 111 213 0 0 16 0 32 0 5783751 5163012096 945459 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604572395601376 0 0 -1 28 0 0 0
[pid=11905/tid=11917] ppid=11903 vsize=5042004 CPUtime=3.35 cores=0-31
/proc/11905/task/11917/stat : 11917 (claspmt) S 11903 11905 11878 0 -1 4202560 33435 0 0 0 114 221 0 0 16 0 32 0 5783751 5163012096 945459 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604574639513056 0 0 -1 4 0 0 0
[pid=11905/tid=11918] ppid=11903 vsize=5042004 CPUtime=2.51 cores=0-31
/proc/11905/task/11918/stat : 11918 (claspmt) S 11903 11905 11878 0 -1 4202560 27601 0 0 0 37 214 0 0 16 0 32 0 5783751 5163012096 945459 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604568780316128 0 0 -1 6 0 0 0
[pid=11905/tid=11919] ppid=11903 vsize=5042004 CPUtime=11.39 cores=0-31
/proc/11905/task/11919/stat : 11919 (claspmt) R 11903 11905 11878 0 -1 4202560 31127 0 0 0 937 202 0 0 25 0 32 0 5783751 5163012096 945459 270582939648 4194304 4964800 140733234300256 18446744073709551615 4669131 0 0 4096 18946 0 0 0 -1 23 0 0 0
[pid=11905/tid=11920] ppid=11903 vsize=5042004 CPUtime=3.23 cores=0-31
/proc/11905/task/11920/stat : 11920 (claspmt) S 11903 11905 11878 0 -1 4202560 31844 0 0 0 105 218 0 0 16 0 32 0 5783751 5163012096 945459 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604572395576800 0 0 -1 5 0 0 0
[pid=11905/tid=11921] ppid=11903 vsize=5042004 CPUtime=3.65 cores=0-31
/proc/11905/task/11921/stat : 11921 (claspmt) S 11903 11905 11878 0 -1 4202560 31993 0 0 0 137 228 0 0 17 0 32 0 5783751 5163012096 945459 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604571766996448 0 0 -1 25 0 0 0
[pid=11905/tid=11922] ppid=11903 vsize=5042004 CPUtime=2.33 cores=0-31
/proc/11905/task/11922/stat : 11922 (claspmt) S 11903 11905 11878 0 -1 4202560 27522 0 0 0 39 194 0 0 16 0 32 0 5783751 5163012096 945459 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604568780291552 0 0 -1 21 0 0 0

################
# More data... #
################

/proc/11905/task/11918/stat : 11918 (claspmt) S 11903 11905 11878 0 -1 4202560 27601 0 0 0 37 214 0 0 16 0 32 0 5783751 5297229824 962513 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604568780316128 0 0 -1 6 0 0 0
[pid=11905/tid=11919] ppid=11903 vsize=5173076 CPUtime=5067.24 cores=0-31
/proc/11905/task/11919/stat : 11919 (claspmt) R 11903 11905 11878 0 -1 4202560 33924 0 0 0 506516 208 0 0 25 0 32 0 5783751 5297229824 962513 270582939648 4194304 4964800 140733234300256 18446744073709551615 4668900 0 0 4096 18946 0 0 0 -1 23 0 0 0
[pid=11905/tid=11920] ppid=11903 vsize=5173076 CPUtime=3.23 cores=0-31
/proc/11905/task/11920/stat : 11920 (claspmt) S 11903 11905 11878 0 -1 4202560 31844 0 0 0 105 218 0 0 16 0 32 0 5783751 5297229824 962513 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604572395576800 0 0 -1 5 0 0 0
[pid=11905/tid=11921] ppid=11903 vsize=5173076 CPUtime=3.65 cores=0-31
/proc/11905/task/11921/stat : 11921 (claspmt) S 11903 11905 11878 0 -1 4202560 31993 0 0 0 137 228 0 0 17 0 32 0 5783751 5297229824 962513 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604571766996448 0 0 -1 25 0 0 0
[pid=11905/tid=11922] ppid=11903 vsize=5173076 CPUtime=2.33 cores=0-31
/proc/11905/task/11922/stat : 11922 (claspmt) S 11903 11905 11878 0 -1 4202560 27522 0 0 0 39 194 0 0 16 0 32 0 5783751 5297229824 962513 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604568780291552 0 0 -1 21 0 0 0
[pid=11905/tid=11923] ppid=11903 vsize=5173076 CPUtime=2.39 cores=0-31
/proc/11905/task/11923/stat : 11923 (claspmt) S 11903 11905 11878 0 -1 4202560 27511 0 0 0 40 199 0 0 16 0 32 0 5783751 5297229824 962513 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604574196260320 0 0 -1 27 0 0 0
[pid=11905/tid=11924] ppid=11903 vsize=5173076 CPUtime=3.36 cores=0-31
/proc/11905/task/11924/stat : 11924 (claspmt) S 11903 11905 11878 0 -1 4202560 33842 0 0 0 114 222 0 0 16 0 32 0 5783751 5297229824 962513 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604574196243936 0 0 -1 4 0 0 0
[pid=11905/tid=11925] ppid=11903 vsize=5173076 CPUtime=3.25 cores=0-31
/proc/11905/task/11925/stat : 11925 (claspmt) S 11903 11905 11878 0 -1 4202560 34067 0 0 0 103 222 0 0 16 0 32 0 5783751 5297229824 962513 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604568626085344 0 0 -1 15 0 0 0
[pid=11905/tid=11926] ppid=11903 vsize=5173076 CPUtime=2.48 cores=0-31
/proc/11905/task/11926/stat : 11926 (claspmt) S 11903 11905 11878 0 -1 4202560 27550 0 0 0 38 210 0 0 16 0 32 0 5783751 5297229824 962513 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604569549004256 0 0 -1 17 0 0 0
[pid=11905/tid=11927] ppid=11903 vsize=5173076 CPUtime=5067.45 cores=0-31
/proc/11905/task/11927/stat : 11927 (claspmt) R 11903 11905 11878 0 -1 4202560 40787 0 0 0 506505 240 0 0 25 0 32 0 5783751 5297229824 962513 270582939648 4194304 4964800 140733234300256 18446744073709551615 4668900 0 0 4096 18946 0 0 0 -1 13 0 0 0
[pid=11905/tid=11928] ppid=11903 vsize=5173076 CPUtime=3.13 cores=0-31
/proc/11905/task/11928/stat : 11928 (claspmt) S 11903 11905 11878 0 -1 4202560 36463 0 0 0 123 190 0 0 16 0 32 0 5783751 5297229824 962513 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604571766963680 0 0 -1 8 0 0 0
[pid=11905/tid=11929] ppid=11903 vsize=5173076 CPUtime=3.59 cores=0-31
/proc/11905/task/11929/stat : 11929 (claspmt) S 11903 11905 11878 0 -1 4202560 31682 0 0 0 138 221 0 0 17 0 32 0 5783751 5297229824 962513 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604568489336288 0 0 -1 16 0 0 0
[pid=11905/tid=11930] ppid=11903 vsize=5173076 CPUtime=2.42 cores=0-31
/proc/11905/task/11930/stat : 11930 (claspmt) S 11903 11905 11878 0 -1 4202560 27425 0 0 0 41 201 0 0 16 0 32 0 5783752 5297229824 962513 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604568626200032 0 0 -1 11 0 0 0
[pid=11905/tid=11931] ppid=11903 vsize=5173076 CPUtime=2.51 cores=0-31
/proc/11905/task/11931/stat : 11931 (claspmt) S 11903 11905 11878 0 -1 4202560 27421 0 0 0 46 205 0 0 16 0 32 0 5783752 5297229824 962513 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604574196268512 0 0 -1 12 0 0 0
[pid=11905/tid=11932] ppid=11903 vsize=5173076 CPUtime=3.29 cores=0-31
/proc/11905/task/11932/stat : 11932 (claspmt) S 11903 11905 11878 0 -1 4202560 34003 0 0 0 111 218 0 0 16 0 32 0 5783752 5297229824 962513 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604574196366816 0 0 -1 22 0 0 0
[pid=11905/tid=11933] ppid=11903 vsize=5173076 CPUtime=2.9 cores=0-31
/proc/11905/task/11933/stat : 11933 (claspmt) S 11903 11905 11878 0 -1 4202560 34162 0 0 0 109 181 0 0 16 0 32 0 5783752 5297229824 962513 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604569548987872 0 0 -1 19 0 0 0
[pid=11905/tid=11934] ppid=11903 vsize=5173076 CPUtime=2.51 cores=0-31
/proc/11905/task/11934/stat : 11934 (claspmt) S 11903 11905 11878 0 -1 4202560 27236 0 0 0 39 212 0 0 16 0 32 0 5783752 5297229824 962513 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604574709177824 0 0 -1 9 0 0 0
[pid=11905/tid=11935] ppid=11903 vsize=5173076 CPUtime=5067.43 cores=0-31
/proc/11905/task/11935/stat : 11935 (claspmt) R 11903 11905 11878 0 -1 4202560 36646 0 0 0 506509 234 0 0 25 0 32 0 5783752 5297229824 962513 270582939648 4194304 4964800 140733234300256 18446744073709551615 4669011 0 0 4096 18946 0 0 0 -1 18 0 0 0
[pid=11905/tid=11936] ppid=11903 vsize=5173076 CPUtime=3.34 cores=0-31
/proc/11905/task/11936/stat : 11936 (claspmt) S 11903 11905 11878 0 -1 4202560 34795 0 0 0 120 214 0 0 16 0 32 0 5783752 5297229824 962513 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604571766988256 0 0 -1 20 0 0 0
[pid=11905/tid=11937] ppid=11903 vsize=5173076 CPUtime=3.69 cores=0-31
/proc/11905/task/11937/stat : 11937 (claspmt) S 11903 11905 11878 0 -1 4202560 32766 0 0 0 144 225 0 0 17 0 32 0 5783752 5297229824 962513 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604569298591200 0 0 -1 30 0 0 0
Current children cumulated CPU time (s) 20356.4
Current children cumulated vsize (KiB) 5173076



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5100 s]
/proc/loadavg: 4.00 4.02 4.01 5/445 12094
/proc/meminfo: memFree=259375700/264257624 swapFree=67111528/67111528
[pid=11905] ppid=11903 vsize=5173076 CPUtime=20427.3 cores=0-31
/proc/11905/stat : 11905 (claspmt) S 11903 11905 11878 0 -1 4202496 1106218 0 0 0 2036083 6646 0 0 18 0 32 0 5783146 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446744073709551615 0 0 17 25 0 0 0
/proc/11905/statm: 1293269 962517 433 189 0 1287618 0
[pid=11905/tid=11907] ppid=11903 vsize=5173076 CPUtime=2.49 cores=0-31
/proc/11905/task/11907/stat : 11907 (claspmt) S 11903 11905 11878 0 -1 4202560 27651 0 0 0 37 212 0 0 16 0 32 0 5783751 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604568821571040 0 0 -1 26 0 0 0
[pid=11905/tid=11908] ppid=11903 vsize=5173076 CPUtime=3.23 cores=0-31
/proc/11905/task/11908/stat : 11908 (claspmt) S 11903 11905 11878 0 -1 4202560 34518 0 0 0 109 214 0 0 16 0 32 0 5783751 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604574196358624 0 0 -1 24 0 0 0
[pid=11905/tid=11909] ppid=11903 vsize=5173076 CPUtime=3.2 cores=0-31
/proc/11905/task/11909/stat : 11909 (claspmt) S 11903 11905 11878 0 -1 4202560 32729 0 0 0 100 220 0 0 16 0 32 0 5783751 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604568986099168 0 0 -1 14 0 0 0
[pid=11905/tid=11910] ppid=11903 vsize=5173076 CPUtime=2.61 cores=0-31
/proc/11905/task/11910/stat : 11910 (claspmt) S 11903 11905 11878 0 -1 4202560 27065 0 0 0 49 212 0 0 16 0 32 0 5783751 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604568780340704 0 0 -1 7 0 0 0
[pid=11905/tid=11911] ppid=11903 vsize=5173076 CPUtime=5085.37 cores=0-31
/proc/11905/task/11911/stat : 11911 (claspmt) R 11903 11905 11878 0 -1 4202560 36652 0 0 0 508335 202 0 0 25 0 32 0 5783751 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 4669008 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11905/tid=11912] ppid=11903 vsize=5173076 CPUtime=3.34 cores=0-31
/proc/11905/task/11912/stat : 11912 (claspmt) S 11903 11905 11878 0 -1 4202560 34679 0 0 0 112 222 0 0 16 0 32 0 5783751 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604572395593184 0 0 -1 3 0 0 0
[pid=11905/tid=11913] ppid=11903 vsize=5173076 CPUtime=3.37 cores=0-31
/proc/11905/task/11913/stat : 11913 (claspmt) S 11903 11905 11878 0 -1 4202560 30881 0 0 0 134 203 0 0 16 0 32 0 5783751 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604569548996064 0 0 -1 23 0 0 0
[pid=11905/tid=11914] ppid=11903 vsize=5173076 CPUtime=2.54 cores=0-31
/proc/11905/task/11914/stat : 11914 (claspmt) S 11903 11905 11878 0 -1 4202560 27586 0 0 0 35 219 0 0 16 0 32 0 5783751 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604574709022176 0 0 -1 29 0 0 0
[pid=11905/tid=11915] ppid=11903 vsize=5173076 CPUtime=2.37 cores=0-31
/proc/11905/task/11915/stat : 11915 (claspmt) S 11903 11905 11878 0 -1 4202560 27544 0 0 0 46 191 0 0 16 0 32 0 5783751 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604639231000032 0 0 -1 7 0 0 0
[pid=11905/tid=11916] ppid=11903 vsize=5173076 CPUtime=3.24 cores=0-31
/proc/11905/task/11916/stat : 11916 (claspmt) S 11903 11905 11878 0 -1 4202560 35110 0 0 0 111 213 0 0 16 0 32 0 5783751 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604572395601376 0 0 -1 28 0 0 0
[pid=11905/tid=11917] ppid=11903 vsize=5173076 CPUtime=3.35 cores=0-31
/proc/11905/task/11917/stat : 11917 (claspmt) S 11903 11905 11878 0 -1 4202560 33435 0 0 0 114 221 0 0 16 0 32 0 5783751 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604574639513056 0 0 -1 4 0 0 0
[pid=11905/tid=11918] ppid=11903 vsize=5173076 CPUtime=2.51 cores=0-31
/proc/11905/task/11918/stat : 11918 (claspmt) S 11903 11905 11878 0 -1 4202560 27601 0 0 0 37 214 0 0 16 0 32 0 5783751 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604568780316128 0 0 -1 6 0 0 0
[pid=11905/tid=11919] ppid=11903 vsize=5173076 CPUtime=5084.96 cores=0-31
/proc/11905/task/11919/stat : 11919 (claspmt) R 11903 11905 11878 0 -1 4202560 33926 0 0 0 508288 208 0 0 25 0 32 0 5783751 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 4669131 0 0 4096 18946 0 0 0 -1 23 0 0 0
[pid=11905/tid=11920] ppid=11903 vsize=5173076 CPUtime=3.23 cores=0-31
/proc/11905/task/11920/stat : 11920 (claspmt) S 11903 11905 11878 0 -1 4202560 31844 0 0 0 105 218 0 0 16 0 32 0 5783751 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604572395576800 0 0 -1 5 0 0 0
[pid=11905/tid=11921] ppid=11903 vsize=5173076 CPUtime=3.65 cores=0-31
/proc/11905/task/11921/stat : 11921 (claspmt) S 11903 11905 11878 0 -1 4202560 31993 0 0 0 137 228 0 0 17 0 32 0 5783751 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604571766996448 0 0 -1 25 0 0 0
[pid=11905/tid=11922] ppid=11903 vsize=5173076 CPUtime=2.33 cores=0-31
/proc/11905/task/11922/stat : 11922 (claspmt) S 11903 11905 11878 0 -1 4202560 27522 0 0 0 39 194 0 0 16 0 32 0 5783751 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604568780291552 0 0 -1 21 0 0 0
[pid=11905/tid=11923] ppid=11903 vsize=5173076 CPUtime=2.39 cores=0-31
/proc/11905/task/11923/stat : 11923 (claspmt) S 11903 11905 11878 0 -1 4202560 27511 0 0 0 40 199 0 0 16 0 32 0 5783751 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604574196260320 0 0 -1 27 0 0 0
[pid=11905/tid=11924] ppid=11903 vsize=5173076 CPUtime=3.36 cores=0-31
/proc/11905/task/11924/stat : 11924 (claspmt) S 11903 11905 11878 0 -1 4202560 33842 0 0 0 114 222 0 0 16 0 32 0 5783751 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604574196243936 0 0 -1 4 0 0 0
[pid=11905/tid=11925] ppid=11903 vsize=5173076 CPUtime=3.25 cores=0-31
/proc/11905/task/11925/stat : 11925 (claspmt) S 11903 11905 11878 0 -1 4202560 34067 0 0 0 103 222 0 0 16 0 32 0 5783751 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604568626085344 0 0 -1 15 0 0 0
[pid=11905/tid=11926] ppid=11903 vsize=5173076 CPUtime=2.48 cores=0-31
/proc/11905/task/11926/stat : 11926 (claspmt) S 11903 11905 11878 0 -1 4202560 27550 0 0 0 38 210 0 0 16 0 32 0 5783751 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604569549004256 0 0 -1 17 0 0 0
[pid=11905/tid=11927] ppid=11903 vsize=5173076 CPUtime=5085.17 cores=0-31
/proc/11905/task/11927/stat : 11927 (claspmt) R 11903 11905 11878 0 -1 4202560 40787 0 0 0 508277 240 0 0 25 0 32 0 5783751 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 4669011 0 0 4096 18946 0 0 0 -1 13 0 0 0
[pid=11905/tid=11928] ppid=11903 vsize=5173076 CPUtime=3.13 cores=0-31
/proc/11905/task/11928/stat : 11928 (claspmt) S 11903 11905 11878 0 -1 4202560 36463 0 0 0 123 190 0 0 16 0 32 0 5783751 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604571766963680 0 0 -1 8 0 0 0
[pid=11905/tid=11929] ppid=11903 vsize=5173076 CPUtime=3.59 cores=0-31
/proc/11905/task/11929/stat : 11929 (claspmt) S 11903 11905 11878 0 -1 4202560 31682 0 0 0 138 221 0 0 17 0 32 0 5783751 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604568489336288 0 0 -1 16 0 0 0
[pid=11905/tid=11930] ppid=11903 vsize=5173076 CPUtime=2.42 cores=0-31
/proc/11905/task/11930/stat : 11930 (claspmt) S 11903 11905 11878 0 -1 4202560 27425 0 0 0 41 201 0 0 16 0 32 0 5783752 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604568626200032 0 0 -1 11 0 0 0
[pid=11905/tid=11931] ppid=11903 vsize=5173076 CPUtime=2.51 cores=0-31
/proc/11905/task/11931/stat : 11931 (claspmt) S 11903 11905 11878 0 -1 4202560 27421 0 0 0 46 205 0 0 16 0 32 0 5783752 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604574196268512 0 0 -1 12 0 0 0
[pid=11905/tid=11932] ppid=11903 vsize=5173076 CPUtime=3.29 cores=0-31
/proc/11905/task/11932/stat : 11932 (claspmt) S 11903 11905 11878 0 -1 4202560 34003 0 0 0 111 218 0 0 16 0 32 0 5783752 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604574196366816 0 0 -1 22 0 0 0
[pid=11905/tid=11933] ppid=11903 vsize=5173076 CPUtime=2.9 cores=0-31
/proc/11905/task/11933/stat : 11933 (claspmt) S 11903 11905 11878 0 -1 4202560 34162 0 0 0 109 181 0 0 16 0 32 0 5783752 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604569548987872 0 0 -1 19 0 0 0
[pid=11905/tid=11934] ppid=11903 vsize=5173076 CPUtime=2.51 cores=0-31
/proc/11905/task/11934/stat : 11934 (claspmt) S 11903 11905 11878 0 -1 4202560 27236 0 0 0 39 212 0 0 16 0 32 0 5783752 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604574709177824 0 0 -1 9 0 0 0
[pid=11905/tid=11935] ppid=11903 vsize=5173076 CPUtime=5085.15 cores=0-31
/proc/11905/task/11935/stat : 11935 (claspmt) R 11903 11905 11878 0 -1 4202560 36646 0 0 0 508281 234 0 0 25 0 32 0 5783752 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 18 0 0 0
[pid=11905/tid=11936] ppid=11903 vsize=5173076 CPUtime=3.34 cores=0-31
/proc/11905/task/11936/stat : 11936 (claspmt) S 11903 11905 11878 0 -1 4202560 34795 0 0 0 120 214 0 0 16 0 32 0 5783752 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604571766988256 0 0 -1 20 0 0 0
[pid=11905/tid=11937] ppid=11903 vsize=5173076 CPUtime=3.69 cores=0-31
/proc/11905/task/11937/stat : 11937 (claspmt) S 11903 11905 11878 0 -1 4202560 32766 0 0 0 144 225 0 0 17 0 32 0 5783752 5297229824 962517 270582939648 4194304 4964800 140733234300256 18446744073709551615 258662772457 0 0 4096 18946 18446604569298591200 0 0 -1 30 0 0 0
Current children cumulated CPU time (s) 20427.3
Current children cumulated vsize (KiB) 5173076

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)

Sending SIGKILL to -11905

Solver just ended. Dumping a history of the last processes samples

[startup+5100.7 s]
/proc/loadavg: 4.00 4.02 4.01 2/415 12095
/proc/meminfo: memFree=259488904/264257624 swapFree=67111528/67111528
[pid=11905] ppid=11903 vsize=4968348 CPUtime=20428.1 cores=0-31
/proc/11905/stat : 11905 (claspmt) R 11903 11905 11878 0 -1 4202496 1106231 0 0 0 2036158 6647 0 0 18 0 1 0 5783146 5087588352 933687 270582939648 4194304 4964800 140733234300256 18446744073709551615 258650613570 0 0 4096 18946 0 0 0 17 25 0 0 0
/proc/11905/statm: 1242087 933687 444 189 0 1236436 0
Current children cumulated CPU time (s) 20428.1
Current children cumulated vsize (KiB) 4968348

[startup+5101.5 s]
/proc/loadavg: 4.00 4.02 4.01 2/415 12095
/proc/meminfo: memFree=259488904/264257624 swapFree=67111528/67111528
[pid=11905] ppid=11903 vsize=4846188 CPUtime=20428.8 cores=0-31
/proc/11905/stat : 11905 (claspmt) R 11903 11905 11878 0 -1 4202496 1106231 0 0 0 2036233 6648 0 0 19 0 1 0 5783146 4962496512 892088 270582939648 4194304 4964800 140733234300256 18446744073709551615 4628834 0 0 4096 18946 0 0 0 17 25 0 0 0
/proc/11905/statm: 1211547 892088 444 189 0 1205896 0
Current children cumulated CPU time (s) 20428.8
Current children cumulated vsize (KiB) 4846188

[startup+5101.93 s]
/proc/loadavg: 3.76 3.97 3.99 2/415 12095
/proc/meminfo: memFree=259704416/264257624 swapFree=67111528/67111528
[pid=11905] ppid=11903 vsize=4772892 CPUtime=20429.2 cores=0-31
/proc/11905/stat : 11905 (claspmt) R 11903 11905 11878 0 -1 4202496 1106231 0 0 0 2036276 6648 0 0 20 0 1 0 5783146 4887441408 873601 270582939648 4194304 4964800 140733234300256 18446744073709551615 258650604632 0 0 4096 18946 0 0 0 17 25 0 0 0
/proc/11905/statm: 1193223 873601 444 189 0 1187572 0
Current children cumulated CPU time (s) 20429.2
Current children cumulated vsize (KiB) 4772892

[startup+5102.1 s]
/proc/loadavg: 3.76 3.97 3.99 2/415 12095
/proc/meminfo: memFree=259704416/264257624 swapFree=67111528/67111528
[pid=11905] ppid=11903 vsize=0 CPUtime=20429.4 cores=0-31
/proc/11905/stat : 11905 (claspmt) R 11903 11905 11878 0 -1 4203524 1106231 0 0 0 2036286 6655 0 0 20 0 1 0 5783146 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 25 0 0 0
/proc/11905/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 20429.4
Current children cumulated vsize (KiB) 0

[startup+5102.2 s]
/proc/loadavg: 3.76 3.97 3.99 2/415 12095
/proc/meminfo: memFree=259704416/264257624 swapFree=67111528/67111528
[pid=11905] ppid=11903 vsize=0 CPUtime=20429.5 cores=0-31
/proc/11905/stat : 11905 (claspmt) R 11903 11905 11878 0 -1 4203524 1106231 0 0 0 2036286 6665 0 0 21 0 1 0 5783146 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 25 0 0 0
/proc/11905/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 20429.5
Current children cumulated vsize (KiB) 0

Child ended because it received signal 9 (SIGKILL)
Real time (s): 5102.25
CPU time (s): 20429.6
CPU user time (s): 20362.9
CPU system time (s): 66.7059
CPU usage (%): 400.403
Max. virtual memory (cumulated for all children) (KiB): 5173076

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 20362.9
system time used= 66.7059
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1106231
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1110432
involuntary context switches= 35045

runsolver used 27.7788 second user time and 111.208 second system time

The end

Launcher Data 

Begin job on node200 at 2011-06-07 01:46:03
IDJOB=3472286
IDBENCH=20583
IDSOLVER=1961
FILE ID=node200/3472286-1307403950
RUNJOBID= node200-1307403950-11892
PBS_JOBID= 13509173
Free space on /tmp= 61444 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT07/industrial/vliw_sat_4.0/9vliw_m_9stages_iq3_C1_bug6.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3472286-1307403950/watcher-3472286-1307403950 -o /tmp/evaluation-result-3472286-1307403950/solver-3472286-1307403950 -C 160000 -W 5100 -M 256000  HOME/claspmt -t32 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3472286-1307403950.cnf

TIME LIMIT= 160000 seconds
MEMORY LIMIT= 256000 MiB
NBCORE= 32

MD5SUM BENCH= abc624ca713aad9563ab3886d918a526
RANDOM SEED=1028482441

node200.alineos.net Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 64
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 96
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 66
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 98
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.99
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 68
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 100
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.03
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 70
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 102
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 16
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 17
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 48
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 18
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 80
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 19
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 112
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.97
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 20
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 21
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 50
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.10
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 22
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 82
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.10
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 114
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 24
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 25
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 26
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 84
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 27
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 116
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 28
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 29
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 54
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 30
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 86
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 31
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 118
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     264257624 kB
MemFree:      263231168 kB
Buffers:         60788 kB
Cached:         748964 kB
SwapCached:          0 kB
Active:         448412 kB
Inactive:       424404 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     264257624 kB
LowFree:      263231168 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:          308712 kB
Writeback:           0 kB
AnonPages:       63000 kB
Mapped:          14416 kB
Slab:            76184 kB
PageTables:       4328 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  199240340 kB
Committed_AS:   172868 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296372 kB
VmallocChunk: 34359440603 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 61440 MiB
End job on node200 at 2011-06-07 03:11:05