Trace number 3472447

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 160018 5021.95

General information on the benchmark

Nameapplication/fuhs/slp-synthesis-AES/
slp-synthesis-aes-bottom26.cnf
MD5SUM98fee1752ddf8cd7cd77925ecf4bfe6e
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables73074
Number of clauses245746
Sum of the clauses size628346
Maximum clause length2232
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 2124096
Number of clauses of size 3120415
Number of clauses of size 4350
Number of clauses of size 524
Number of clauses of size over 5860

Solver Data 

0.00/0.01	c clasp version 2.0.0-RC2
0.00/0.01	c Reading from ...653112/instance-3472447-1307653112.cnf
1.29/1.32	c Solving...
160003.92/5008.35	
160003.92/5008.35	*** INTERRUPTED! ***
160003.92/5008.35	*** Shutting down threads...

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3472447-1307653112/watcher-3472447-1307653112 -o /tmp/evaluation-result-3472447-1307653112/solver-3472447-1307653112 -C 160000 -W 5100 -M 256000 HOME/claspmt -t32 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3472447-1307653112.cnf 

running on 32 cores: 0,4,8,12,16,20,24,28,1,5,9,13,17,21,25,29,2,6,10,14,18,22,26,30,3,7,11,15,19,23,27,31

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 160000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 160030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 262144000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 262195200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 29.50 31.50 31.83 1/412 32128
/proc/meminfo: memFree=262214980/264257624 swapFree=67111528/67111528
[pid=32128] ppid=32126 vsize=30172 CPUtime=0.02 cores=0-31
/proc/32128/stat : 32128 (claspmt) R 32126 32128 32098 0 -1 4202496 2273 0 0 0 2 0 0 0 21 0 1 0 30698116 30896128 2202 270582939648 4194304 4964800 140737030216864 18446744073709551615 4700412 0 0 4096 18946 0 0 0 17 16 0 0 0
/proc/32128/statm: 7543 2206 373 189 0 2417 0

[startup+0.131255 s]
/proc/loadavg: 29.50 31.50 31.83 1/412 32128
/proc/meminfo: memFree=262214980/264257624 swapFree=67111528/67111528
[pid=32128] ppid=32126 vsize=40516 CPUtime=0.12 cores=0-31
/proc/32128/stat : 32128 (claspmt) R 32126 32128 32098 0 -1 4202496 6332 0 0 0 11 1 0 0 21 0 1 0 30698116 41488384 5217 270582939648 4194304 4964800 140737030216864 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 17 16 0 0 0
/proc/32128/statm: 10129 5218 386 189 0 5003 0
Current children cumulated CPU time (s) 0.12
Current children cumulated vsize (KiB) 40516

[startup+0.201274 s]
/proc/loadavg: 29.50 31.50 31.83 1/412 32128
/proc/meminfo: memFree=262214980/264257624 swapFree=67111528/67111528
[pid=32128] ppid=32126 vsize=44752 CPUtime=0.19 cores=0-31
/proc/32128/stat : 32128 (claspmt) R 32126 32128 32098 0 -1 4202496 7686 0 0 0 17 2 0 0 22 0 1 0 30698116 45826048 6300 270582939648 4194304 4964800 140737030216864 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 17 16 0 0 0
/proc/32128/statm: 11188 6300 386 189 0 6062 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 44752

[startup+0.301256 s]
/proc/loadavg: 29.50 31.50 31.83 1/412 32128
/proc/meminfo: memFree=262214980/264257624 swapFree=67111528/67111528
[pid=32128] ppid=32126 vsize=54044 CPUtime=0.29 cores=0-31
/proc/32128/stat : 32128 (claspmt) R 32126 32128 32098 0 -1 4202496 10939 0 0 0 26 3 0 0 22 0 1 0 30698116 55341056 8434 270582939648 4194304 4964800 140737030216864 18446744073709551615 4617330 0 0 4096 18946 0 0 0 17 16 0 0 0
/proc/32128/statm: 13511 8436 396 189 0 8385 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 54044

[startup+0.700271 s]
/proc/loadavg: 29.50 31.50 31.83 1/412 32128
/proc/meminfo: memFree=262214980/264257624 swapFree=67111528/67111528
[pid=32128] ppid=32126 vsize=61884 CPUtime=0.69 cores=0-31
/proc/32128/stat : 32128 (claspmt) R 32126 32128 32098 0 -1 4202496 12874 0 0 0 65 4 0 0 23 0 1 0 30698116 63369216 10369 270582939648 4194304 4964800 140737030216864 18446744073709551615 4600351 0 0 4096 18946 0 0 0 17 16 0 0 0
/proc/32128/statm: 15471 10369 401 189 0 10345 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 61884

[startup+1.50236 s]
/proc/loadavg: 29.50 31.50 31.83 2/413 32129
/proc/meminfo: memFree=262169216/264257624 swapFree=67111528/67111528
[pid=32128] ppid=32126 vsize=506492 CPUtime=5.92 cores=0-31
/proc/32128/stat : 32128 (claspmt) S 32126 32128 32098 0 -1 4202496 51365 0 0 0 144 448 0 0 18 0 32 0 30698116 518647808 35713 270582939648 4194304 4964800 140737030216864 18446744073709551615 258662772457 0 0 4096 18946 18446744073709551615 0 0 17 16 0 0 0
/proc/32128/statm: 126623 35726 429 189 0 120972 0
Current children cumulated CPU time (s) 5.92
Current children cumulated vsize (KiB) 506492

[startup+3.10325 s]
/proc/loadavg: 29.50 31.50 31.83 5/444 32160
/proc/meminfo: memFree=261871228/264257624 swapFree=67111528/67111528
[pid=32128] ppid=32126 vsize=2102516 CPUtime=27.73 cores=0-31
/proc/32128/stat : 32128 (claspmt) R 32126 32128 32098 0 -1 4202496 111335 0 0 0 2073 700 0 0 18 0 32 0 30698116 2152976384 89952 270582939648 4194304 4964800 140737030216864 18446744073709551615 4628356 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/32128/statm: 525629 89953 461 189 0 519978 0
[pid=32128/tid=32130] ppid=32126 vsize=2102516 CPUtime=0.78 cores=0-31
/proc/32128/task/32130/stat : 32130 (claspmt) R 32126 32128 32098 0 -1 4202560 2499 0 0 0 54 24 0 0 16 0 32 0 30698249 2152976384 89953 270582939648 4194304 4964800 140737030216864 18446744073709551615 4669192 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=32128/tid=32131] ppid=32126 vsize=2102516 CPUtime=0.74 cores=0-31
/proc/32128/task/32131/stat : 32131 (claspmt) R 32126 32128 32098 0 -1 4202560 3465 0 0 0 54 20 0 0 16 0 32 0 30698249 2152976384 89953 270582939648 4194304 4964800 140737030216864 18446744073709551615 4668977 0 0 4096 18946 0 0 0 -1 30 0 0 0
[pid=32128/tid=32132] ppid=32126 vsize=2102516 CPUtime=0.78 cores=0-31
/proc/32128/task/32132/stat : 32132 (claspmt) R 32126 32128 32098 0 -1 4202560 3368 0 0 0 55 23 0 0 16 0 32 0 30698249 2152976384 89953 270582939648 4194304 4964800 140737030216864 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=32128/tid=32133] ppid=32126 vsize=2102516 CPUtime=0.74 cores=0-31
/proc/32128/task/32133/stat : 32133 (claspmt) R 32126 32128 32098 0 -1 4202560 2649 0 0 0 53 21 0 0 16 0 32 0 30698249 2152976384 89953 270582939648 4194304 4964800 140737030216864 18446744073709551615 4379895 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=32128/tid=32134] ppid=32126 vsize=2102516 CPUtime=1.24 cores=0-31
/proc/32128/task/32134/stat : 32134 (claspmt) R 32126 32128 32098 0 -1 4202560 3233 0 0 0 101 23 0 0 19 0 32 0 30698249 2152976384 89954 270582939648 4194304 4964800 140737030216864 18446744073709551615 4399339 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=32128/tid=32135] ppid=32126 vsize=2102516 CPUtime=0.78 cores=0-31
/proc/32128/task/32135/stat : 32135 (claspmt) R 32126 32128 32098 0 -1 4202560 3431 0 0 0 55 23 0 0 16 0 32 0 30698249 2152976384 89954 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 12 0 0 0
[pid=32128/tid=32136] ppid=32126 vsize=2102516 CPUtime=0.76 cores=0-31
/proc/32128/task/32136/stat : 32136 (claspmt) R 32126 32128 32098 0 -1 4202560 2913 0 0 0 54 22 0 0 16 0 32 0 30698249 2152976384 89954 270582939648 4194304 4964800 140737030216864 18446744073709551615 4668759 0 0 4096 18946 0 0 0 -1 15 0 0 0
[pid=32128/tid=32137] ppid=32126 vsize=2102516 CPUtime=0.76 cores=0-31
/proc/32128/task/32137/stat : 32137 (claspmt) R 32126 32128 32098 0 -1 4202560 2642 0 0 0 54 22 0 0 16 0 32 0 30698249 2152976384 89955 270582939648 4194304 4964800 140737030216864 18446744073709551615 4669535 0 0 4096 18946 0 0 0 -1 17 0 0 0
[pid=32128/tid=32138] ppid=32126 vsize=2102516 CPUtime=0.76 cores=0-31
/proc/32128/task/32138/stat : 32138 (claspmt) R 32126 32128 32098 0 -1 4202560 2809 0 0 0 55 21 0 0 16 0 32 0 30698249 2152976384 89955 270582939648 4194304 4964800 140737030216864 18446744073709551615 4370264 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=32128/tid=32139] ppid=32126 vsize=2102516 CPUtime=0.8 cores=0-31
/proc/32128/task/32139/stat : 32139 (claspmt) R 32126 32128 32098 0 -1 4202560 3454 0 0 0 55 25 0 0 16 0 32 0 30698249 2152976384 89955 270582939648 4194304 4964800 140737030216864 18446744073709551615 4342685 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=32128/tid=32140] ppid=32126 vsize=2102516 CPUtime=0.78 cores=0-31
/proc/32128/task/32140/stat : 32140 (claspmt) R 32126 32128 32098 0 -1 4202560 3391 0 0 0 56 22 0 0 16 0 32 0 30698249 2152976384 89955 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 18 0 0 0
[pid=32128/tid=32141] ppid=32126 vsize=2102516 CPUtime=0.74 cores=0-31
/proc/32128/task/32141/stat : 32141 (claspmt) R 32126 32128 32098 0 -1 4202560 2709 0 0 0 56 18 0 0 16 0 32 0 30698249 2152976384 89955 270582939648 4194304 4964800 140737030216864 18446744073709551615 4668753 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=32128/tid=32142] ppid=32126 vsize=2102516 CPUtime=1.26 cores=0-31
/proc/32128/task/32142/stat : 32142 (claspmt) R 32126 32128 32098 0 -1 4202560 3174 0 0 0 104 22 0 0 19 0 32 0 30698249 2152976384 89955 270582939648 4194304 4964800 140737030216864 18446744073709551615 4668709 0 0 4096 18946 0 0 0 -1 10 0 0 0
[pid=32128/tid=32143] ppid=32126 vsize=2102516 CPUtime=0.8 cores=0-31
/proc/32128/task/32143/stat : 32143 (claspmt) R 32126 32128 32098 0 -1 4202560 3418 0 0 0 56 24 0 0 16 0 32 0 30698249 2152976384 89956 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 16 0 0 0
[pid=32128/tid=32144] ppid=32126 vsize=2102516 CPUtime=0.75 cores=0-31
/proc/32128/task/32144/stat : 32144 (claspmt) R 32126 32128 32098 0 -1 4202560 3122 0 0 0 55 20 0 0 16 0 32 0 30698249 2152976384 89956 270582939648 4194304 4964800 140737030216864 18446744073709551615 4668737 0 0 4096 18946 0 0 0 -1 27 0 0 0
[pid=32128/tid=32145] ppid=32126 vsize=2102516 CPUtime=0.75 cores=0-31
/proc/32128/task/32145/stat : 32145 (claspmt) R 32126 32128 32098 0 -1 4202560 2836 0 0 0 55 20 0 0 16 0 32 0 30698249 2152976384 89957 270582939648 4194304 4964800 140737030216864 18446744073709551615 4370236 0 0 4096 18946 0 0 0 -1 14 0 0 0
[pid=32128/tid=32146] ppid=32126 vsize=2102516 CPUtime=0.73 cores=0-31
/proc/32128/task/32146/stat : 32146 (claspmt) R 32126 32128 32098 0 -1 4202560 2693 0 0 0 54 19 0 0 16 0 32 0 30698249 2152976384 89957 270582939648 4194304 4964800 140737030216864 18446744073709551615 4370384 0 0 4096 18946 0 0 0 -1 23 0 0 0
[pid=32128/tid=32147] ppid=32126 vsize=2102516 CPUtime=0.79 cores=0-31
/proc/32128/task/32147/stat : 32147 (claspmt) R 32126 32128 32098 0 -1 4202560 3446 0 0 0 55 24 0 0 16 0 32 0 30698249 2152976384 89957 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 21 0 0 0
[pid=32128/tid=32148] ppid=32126 vsize=2102516 CPUtime=0.79 cores=0-31
/proc/32128/task/32148/stat : 32148 (claspmt) R 32126 32128 32098 0 -1 4202560 3408 0 0 0 55 24 0 0 16 0 32 0 30698249 2152976384 89957 270582939648 4194304 4964800 140737030216864 18446744073709551615 4668789 0 0 4096 18946 0 0 0 -1 9 0 0 0
[pid=32128/tid=32149] ppid=32126 vsize=2102516 CPUtime=0.76 cores=0-31
/proc/32128/task/32149/stat : 32149 (claspmt) R 32126 32128 32098 0 -1 4202560 2795 0 0 0 54 22 0 0 16 0 32 0 30698249 2152976384 89957 270582939648 4194304 4964800 140737030216864 18446744073709551615 4668753 0 0 4096 18946 0 0 0 -1 13 0 0 0
[pid=32128/tid=32150] ppid=32126 vsize=2102516 CPUtime=1.22 cores=0-31
/proc/32128/task/32150/stat : 32150 (claspmt) R 32126 32128 32098 0 -1 4202560 3197 0 0 0 100 22 0 0 18 0 32 0 30698249 2152976384 89957 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 24 0 0 0
[pid=32128/tid=32151] ppid=32126 vsize=2102516 CPUtime=0.77 cores=0-31
/proc/32128/task/32151/stat : 32151 (claspmt) R 32126 32128 32098 0 -1 4202560 3457 0 0 0 55 22 0 0 16 0 32 0 30698250 2152976384 89957 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670155 0 0 4096 18946 0 0 0 -1 28 0 0 0
[pid=32128/tid=32152] ppid=32126 vsize=2102516 CPUtime=0.79 cores=0-31
/proc/32128/task/32152/stat : 32152 (claspmt) R 32126 32128 32098 0 -1 4202560 3260 0 0 0 56 23 0 0 16 0 32 0 30698250 2152976384 89957 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 25 0 0 0
[pid=32128/tid=32153] ppid=32126 vsize=2102516 CPUtime=0.72 cores=0-31
/proc/32128/task/32153/stat : 32153 (claspmt) R 32126 32128 32098 0 -1 4202560 2728 0 0 0 52 20 0 0 16 0 32 0 30698250 2152976384 89957 270582939648 4194304 4964800 140737030216864 18446744073709551615 4370264 0 0 4096 18946 0 0 0 -1 22 0 0 0
[pid=32128/tid=32154] ppid=32126 vsize=2102516 CPUtime=0.77 cores=0-31
/proc/32128/task/32154/stat : 32154 (claspmt) R 32126 32128 32098 0 -1 4202560 2760 0 0 0 55 22 0 0 16 0 32 0 30698250 2152976384 89958 270582939648 4194304 4964800 140737030216864 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 26 0 0 0
[pid=32128/tid=32155] ppid=32126 vsize=2102516 CPUtime=0.77 cores=0-31
/proc/32128/task/32155/stat : 32155 (claspmt) R 32126 32128 32098 0 -1 4202560 3605 0 0 0 54 23 0 0 16 0 32 0 30698250 2152976384 89958 270582939648 4194304 4964800 140737030216864 18446744073709551615 4675555 0 0 4096 18946 0 0 0 -1 8 0 0 0
[pid=32128/tid=32156] ppid=32126 vsize=2102516 CPUtime=0.79 cores=0-31
/proc/32128/task/32156/stat : 32156 (claspmt) R 32126 32128 32098 0 -1 4202560 3587 0 0 0 54 25 0 0 16 0 32 0 30698250 2152976384 89958 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 20 0 0 0
[pid=32128/tid=32157] ppid=32126 vsize=2102516 CPUtime=0.77 cores=0-31
/proc/32128/task/32157/stat : 32157 (claspmt) R 32126 32128 32098 0 -1 4202560 2879 0 0 0 55 22 0 0 16 0 32 0 30698250 2152976384 89962 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670190 0 0 4096 18946 0 0 0 -1 29 0 0 0
[pid=32128/tid=32158] ppid=32126 vsize=2102516 CPUtime=1.21 cores=0-31
/proc/32128/task/32158/stat : 32158 (claspmt) R 32126 32128 32098 0 -1 4202560 3126 0 0 0 96 25 0 0 19 0 32 0 30698250 2152976384 89963 270582939648 4194304 4964800 140737030216864 18446744073709551615 4668763 0 0 4096 18946 0 0 0 -1 31 0 0 0
[pid=32128/tid=32159] ppid=32126 vsize=2102516 CPUtime=0.75 cores=0-31
/proc/32128/task/32159/stat : 32159 (claspmt) R 32126 32128 32098 0 -1 4202560 3382 0 0 0 56 19 0 0 16 0 32 0 30698250 2152976384 89963 270582939648 4194304 4964800 140737030216864 18446744073709551615 4678229 0 0 4096 18946 0 0 0 -1 11 0 0 0
[pid=32128/tid=32160] ppid=32126 vsize=2102516 CPUtime=0.7 cores=0-31
/proc/32128/task/32160/stat : 32160 (claspmt) R 32126 32128 32098 0 -1 4202560 2938 0 0 0 55 15 0 0 16 0 32 0 30698250 2152976384 89963 270582939648 4194304 4964800 140737030216864 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 19 0 0 0
Current children cumulated CPU time (s) 27.73
Current children cumulated vsize (KiB) 2102516

[startup+6.30236 s]
/proc/loadavg: 29.78 31.53 31.83 33/444 32160
/proc/meminfo: memFree=261788516/264257624 swapFree=67111528/67111528
[pid=32128] ppid=32126 vsize=2105164 CPUtime=129.69 cores=0-31
/proc/32128/stat : 32128 (claspmt) R 32126 32128 32098 0 -1 4202496 133704 0 0 0 12198 771 0 0 20 0 32 0 30698116 2155687936 112321 270582939648 4194304 4964800 140737030216864 18446744073709551615 4668773 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/32128/statm: 526291 112321 461 189 0 520640 0
[pid=32128/tid=32130] ppid=32126 vsize=2105164 CPUtime=3.97 cores=0-31
/proc/32128/task/32130/stat : 32130 (claspmt) R 32126 32128 32098 0 -1 4202560 3044 0 0 0 372 25 0 0 20 0 32 0 30698249 2155687936 112321 270582939648 4194304 4964800 140737030216864 18446744073709551615 4342167 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=32128/tid=32131] ppid=32126 vsize=2105164 CPUtime=3.93 cores=0-31
/proc/32128/task/32131/stat : 32131 (claspmt) R 32126 32128 32098 0 -1 4202560 4590 0 0 0 370 23 0 0 19 0 32 0 30698249 2155687936 112321 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 30 0 0 0
[pid=32128/tid=32132] ppid=32126 vsize=2105164 CPUtime=3.97 cores=0-31
/proc/32128/task/32132/stat : 32132 (claspmt) R 32126 32128 32098 0 -1 4202560 4481 0 0 0 371 26 0 0 18 0 32 0 30698249 2155687936 112321 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=32128/tid=32133] ppid=32126 vsize=2105164 CPUtime=3.94 cores=0-31
/proc/32128/task/32133/stat : 32133 (claspmt) R 32126 32128 32098 0 -1 4202560 3404 0 0 0 370 24 0 0 21 0 32 0 30698249 2155687936 112321 270582939648 4194304 4964800 140737030216864 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=32128/tid=32134] ppid=32126 vsize=2105164 CPUtime=4.44 cores=0-31
/proc/32128/task/32134/stat : 32134 (claspmt) R 32126 32128 32098 0 -1 4202560 3291 0 0 0 421 23 0 0 25 0 32 0 30698249 2155687936 112321 270582939648 4194304 4964800 140737030216864 18446744073709551615 4668672 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=32128/tid=32135] ppid=32126 vsize=2105164 CPUtime=3.97 cores=0-31
/proc/32128/task/32135/stat : 32135 (claspmt) R 32126 32128 32098 0 -1 4202560 4124 0 0 0 372 25 0 0 21 0 32 0 30698249 2155687936 112321 270582939648 4194304 4964800 140737030216864 18446744073709551615 4342232 0 0 4096 18946 0 0 0 -1 12 0 0 0
[pid=32128/tid=32136] ppid=32126 vsize=2105164 CPUtime=3.95 cores=0-31
/proc/32128/task/32136/stat : 32136 (claspmt) R 32126 32128 32098 0 -1 4202560 3902 0 0 0 368 27 0 0 21 0 32 0 30698249 2155687936 112321 270582939648 4194304 4964800 140737030216864 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 15 0 0 0
[pid=32128/tid=32137] ppid=32126 vsize=2105164 CPUtime=3.95 cores=0-31
/proc/32128/task/32137/stat : 32137 (claspmt) R 32126 32128 32098 0 -1 4202560 3226 0 0 0 371 24 0 0 21 0 32 0 30698249 2155687936 112322 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 17 0 0 0
[pid=32128/tid=32138] ppid=32126 vsize=2105164 CPUtime=3.95 cores=0-31
/proc/32128/task/32138/stat : 32138 (claspmt) R 32126 32128 32098 0 -1 4202560 3386 0 0 0 373 22 0 0 18 0 32 0 30698249 2155687936 112322 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=32128/tid=32139] ppid=32126 vsize=2105164 CPUtime=3.99 cores=0-31
/proc/32128/task/32139/stat : 32139 (claspmt) R 32126 32128 32098 0 -1 4202560 4337 0 0 0 372 27 0 0 21 0 32 0 30698249 2155687936 112322 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670313 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=32128/tid=32140] ppid=32126 vsize=2105164 CPUtime=3.96 cores=0-31
/proc/32128/task/32140/stat : 32140 (claspmt) R 32126 32128 32098 0 -1 4202560 4457 0 0 0 370 26 0 0 18 0 32 0 30698249 2155687936 112322 270582939648 4194304 4964800 140737030216864 18446744073709551615 4668773 0 0 4096 18946 0 0 0 -1 18 0 0 0
[pid=32128/tid=32141] ppid=32126 vsize=2105164 CPUtime=3.93 cores=0-31
/proc/32128/task/32141/stat : 32141 (claspmt) R 32126 32128 32098 0 -1 4202560 3432 0 0 0 372 21 0 0 19 0 32 0 30698249 2155687936 112322 270582939648 4194304 4964800 140737030216864 18446744073709551615 4320632 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=32128/tid=32142] ppid=32126 vsize=2105164 CPUtime=4.45 cores=0-31
/proc/32128/task/32142/stat : 32142 (claspmt) R 32126 32128 32098 0 -1 4202560 3225 0 0 0 423 22 0 0 25 0 32 0 30698249 2155687936 112322 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 10 0 0 0
[pid=32128/tid=32143] ppid=32126 vsize=2105164 CPUtime=3.99 cores=0-31
/proc/32128/task/32143/stat : 32143 (claspmt) R 32126 32128 32098 0 -1 4202560 4137 0 0 0 373 26 0 0 21 0 32 0 30698249 2155687936 112323 270582939648 4194304 4964800 140737030216864 18446744073709551615 4362754 0 0 4096 18946 0 0 0 -1 16 0 0 0
[pid=32128/tid=32144] ppid=32126 vsize=2105164 CPUtime=3.93 cores=0-31
/proc/32128/task/32144/stat : 32144 (claspmt) R 32126 32128 32098 0 -1 4202560 4101 0 0 0 371 22 0 0 21 0 32 0 30698249 2155687936 112323 270582939648 4194304 4964800 140737030216864 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 27 0 0 0
[pid=32128/tid=32145] ppid=32126 vsize=2105164 CPUtime=3.94 cores=0-31
/proc/32128/task/32145/stat : 32145 (claspmt) R 32126 32128 32098 0 -1 4202560 3565 0 0 0 371 23 0 0 21 0 32 0 30698249 2155687936 112323 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 14 0 0 0
[pid=32128/tid=32146] ppid=32126 vsize=2105164 CPUtime=3.93 cores=0-31
/proc/32128/task/32146/stat : 32146 (claspmt) R 32126 32128 32098 0 -1 4202560 3218 0 0 0 372 21 0 0 20 0 32 0 30698249 2155687936 112324 270582939648 4194304 4964800 140737030216864 18446744073709551615 4370288 0 0 4096 18946 0 0 0 -1 23 0 0 0
[pid=32128/tid=32147] ppid=32126 vsize=2105164 CPUtime=3.97 cores=0-31
/proc/32128/task/32147/stat : 32147 (claspmt) R 32126 32128 32098 0 -1 4202560 4384 0 0 0 370 27 0 0 18 0 32 0 30698249 2155687936 112324 270582939648 4194304 4964800 140737030216864 18446744073709551615 4668759 0 0 4096 18946 0 0 0 -1 21 0 0 0
[pid=32128/tid=32148] ppid=32126 vsize=2105164 CPUtime=3.98 cores=0-31
/proc/32128/task/32148/stat : 32148 (claspmt) R 32126 32128 32098 0 -1 4202560 4264 0 0 0 372 26 0 0 19 0 32 0 30698249 2155687936 112324 270582939648 4194304 4964800 140737030216864 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 9 0 0 0
[pid=32128/tid=32149] ppid=32126 vsize=2105164 CPUtime=3.94 cores=0-31
/proc/32128/task/32149/stat : 32149 (claspmt) R 32126 32128 32098 0 -1 4202560 3486 0 0 0 370 24 0 0 21 0 32 0 30698249 2155687936 112324 270582939648 4194304 4964800 140737030216864 18446744073709551615 4668709 0 0 4096 18946 0 0 0 -1 13 0 0 0
[pid=32128/tid=32150] ppid=32126 vsize=2105164 CPUtime=4.42 cores=0-31
/proc/32128/task/32150/stat : 32150 (claspmt) R 32126 32128 32098 0 -1 4202560 3262 0 0 0 420 22 0 0 25 0 32 0 30698249 2155687936 112324 270582939648 4194304 4964800 140737030216864 18446744073709551615 4465551 0 0 4096 18946 0 0 0 -1 24 0 0 0
[pid=32128/tid=32151] ppid=32126 vsize=2105164 CPUtime=3.96 cores=0-31
/proc/32128/task/32151/stat : 32151 (claspmt) R 32126 32128 32098 0 -1 4202560 4283 0 0 0 372 24 0 0 19 0 32 0 30698250 2155687936 112325 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 28 0 0 0
[pid=32128/tid=32152] ppid=32126 vsize=2105164 CPUtime=3.98 cores=0-31
/proc/32128/task/32152/stat : 32152 (claspmt) R 32126 32128 32098 0 -1 4202560 4149 0 0 0 372 26 0 0 21 0 32 0 30698250 2155687936 112325 270582939648 4194304 4964800 140737030216864 18446744073709551615 4668544 0 0 4096 18946 0 0 0 -1 25 0 0 0
[pid=32128/tid=32153] ppid=32126 vsize=2105164 CPUtime=3.91 cores=0-31
/proc/32128/task/32153/stat : 32153 (claspmt) R 32126 32128 32098 0 -1 4202560 3218 0 0 0 369 22 0 0 21 0 32 0 30698250 2155687936 112325 270582939648 4194304 4964800 140737030216864 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 22 0 0 0
[pid=32128/tid=32154] ppid=32126 vsize=2105164 CPUtime=3.95 cores=0-31
/proc/32128/task/32154/stat : 32154 (claspmt) R 32126 32128 32098 0 -1 4202560 3386 0 0 0 372 23 0 0 21 0 32 0 30698250 2155687936 112325 270582939648 4194304 4964800 140737030216864 18446744073709551615 4668773 0 0 4096 18946 0 0 0 -1 26 0 0 0

################
# More data... #
################

/proc/32128/task/32146/stat : 32146 (claspmt) R 32126 32128 32098 0 -1 4202560 25170 0 0 0 493365 94 0 0 25 0 32 0 30698249 7857618944 1663377 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670264 0 0 4096 18946 0 0 0 -1 15 0 0 0
[pid=32128/tid=32147] ppid=32126 vsize=7673456 CPUtime=4958.63 cores=0-31
/proc/32128/task/32147/stat : 32147 (claspmt) R 32126 32128 32098 0 -1 4202560 52424 0 0 0 495651 212 0 0 25 0 32 0 30698249 7857618944 1663377 270582939648 4194304 4964800 140737030216864 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 21 0 0 0
[pid=32128/tid=32148] ppid=32126 vsize=7673456 CPUtime=4958.49 cores=0-31
/proc/32128/task/32148/stat : 32148 (claspmt) R 32126 32128 32098 0 -1 4202560 43001 0 0 0 495666 183 0 0 25 0 32 0 30698249 7857618944 1663377 270582939648 4194304 4964800 140737030216864 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=32128/tid=32149] ppid=32126 vsize=7673456 CPUtime=4946.01 cores=0-31
/proc/32128/task/32149/stat : 32149 (claspmt) R 32126 32128 32098 0 -1 4202560 48268 0 0 0 494402 199 0 0 25 0 32 0 30698249 7857618944 1663377 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=32128/tid=32150] ppid=32126 vsize=7673456 CPUtime=4955.86 cores=0-31
/proc/32128/task/32150/stat : 32150 (claspmt) R 32126 32128 32098 0 -1 4202560 43284 0 0 0 495377 209 0 0 25 0 32 0 30698249 7857618944 1663377 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 20 0 0 0
[pid=32128/tid=32151] ppid=32126 vsize=7673456 CPUtime=4958.01 cores=0-31
/proc/32128/task/32151/stat : 32151 (claspmt) R 32126 32128 32098 0 -1 4202560 37521 0 0 0 495650 151 0 0 25 0 32 0 30698250 7857618944 1663377 270582939648 4194304 4964800 140737030216864 18446744073709551615 4684107 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=32128/tid=32152] ppid=32126 vsize=7673456 CPUtime=4940.02 cores=0-31
/proc/32128/task/32152/stat : 32152 (claspmt) R 32126 32128 32098 0 -1 4202560 146999 0 0 0 493406 596 0 0 25 0 32 0 30698250 7857618944 1663377 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 19 0 0 0
[pid=32128/tid=32153] ppid=32126 vsize=7673456 CPUtime=4946.25 cores=0-31
/proc/32128/task/32153/stat : 32153 (claspmt) R 32126 32128 32098 0 -1 4202560 31565 0 0 0 494484 141 0 0 25 0 32 0 30698250 7857618944 1663377 270582939648 4194304 4964800 140737030216864 18446744073709551615 4669509 0 0 4096 18946 0 0 0 -1 31 0 0 0
[pid=32128/tid=32154] ppid=32126 vsize=7673456 CPUtime=4958.17 cores=0-31
/proc/32128/task/32154/stat : 32154 (claspmt) R 32126 32128 32098 0 -1 4202560 22215 0 0 0 495724 93 0 0 25 0 32 0 30698250 7857618944 1663377 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=32128/tid=32155] ppid=32126 vsize=7673456 CPUtime=4958.67 cores=0-31
/proc/32128/task/32155/stat : 32155 (claspmt) R 32126 32128 32098 0 -1 4202560 56005 0 0 0 495655 212 0 0 25 0 32 0 30698250 7857618944 1663377 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670530 0 0 4096 18946 0 0 0 -1 8 0 0 0
[pid=32128/tid=32156] ppid=32126 vsize=7673456 CPUtime=4958.45 cores=0-31
/proc/32128/task/32156/stat : 32156 (claspmt) R 32126 32128 32098 0 -1 4202560 40365 0 0 0 495674 171 0 0 25 0 32 0 30698250 7857618944 1663377 270582939648 4194304 4964800 140737030216864 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 28 0 0 0
[pid=32128/tid=32157] ppid=32126 vsize=7673456 CPUtime=4958.59 cores=0-31
/proc/32128/task/32157/stat : 32157 (claspmt) R 32126 32128 32098 0 -1 4202560 58112 0 0 0 495618 241 0 0 25 0 32 0 30698250 7857618944 1663377 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 29 0 0 0
[pid=32128/tid=32158] ppid=32126 vsize=7673456 CPUtime=4948.73 cores=0-31
/proc/32128/task/32158/stat : 32158 (claspmt) R 32126 32128 32098 0 -1 4202560 37161 0 0 0 494725 148 0 0 25 0 32 0 30698250 7857618944 1663377 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 22 0 0 0
[pid=32128/tid=32159] ppid=32126 vsize=7673456 CPUtime=4957.49 cores=0-31
/proc/32128/task/32159/stat : 32159 (claspmt) R 32126 32128 32098 0 -1 4202560 31663 0 0 0 495621 128 0 0 25 0 32 0 30698250 7857618944 1663377 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 13 0 0 0
[pid=32128/tid=32160] ppid=32126 vsize=7673456 CPUtime=4957.73 cores=0-31
/proc/32128/task/32160/stat : 32160 (claspmt) R 32126 32128 32098 0 -1 4202560 106667 0 0 0 495343 430 0 0 25 0 32 0 30698250 7857618944 1663377 270582939648 4194304 4964800 140737030216864 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 158532
Current children cumulated vsize (KiB) 7673456



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5008.3 s]
/proc/loadavg: 32.06 32.05 32.04 34/444 32315
/proc/meminfo: memFree=255556496/264257624 swapFree=67111528/67111528
[pid=32128] ppid=32126 vsize=7673456 CPUtime=160004 cores=0-31
/proc/32128/stat : 32128 (claspmt) R 32126 32128 32098 0 -1 4202496 1697004 0 0 0 15993544 6849 0 0 25 0 32 0 30698116 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4653507 0 0 4096 18946 18446744073709551615 0 0 17 24 0 0 0
/proc/32128/statm: 1918364 1663474 463 189 0 1912713 0
[pid=32128/tid=32130] ppid=32126 vsize=7673456 CPUtime=4994.28 cores=0-31
/proc/32128/task/32130/stat : 32130 (claspmt) R 32126 32128 32098 0 -1 4202560 24903 0 0 0 499321 107 0 0 25 0 32 0 30698249 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=32128/tid=32131] ppid=32126 vsize=7673456 CPUtime=5004.71 cores=0-31
/proc/32128/task/32131/stat : 32131 (claspmt) R 32126 32128 32098 0 -1 4202560 58623 0 0 0 500250 221 0 0 25 0 32 0 30698249 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4669026 0 0 4096 18946 0 0 0 -1 30 0 0 0
[pid=32128/tid=32132] ppid=32126 vsize=7673456 CPUtime=4992.11 cores=0-31
/proc/32128/task/32132/stat : 32132 (claspmt) R 32126 32128 32098 0 -1 4202560 39387 0 0 0 499063 148 0 0 25 0 32 0 30698249 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=32128/tid=32133] ppid=32126 vsize=7673456 CPUtime=4996.91 cores=0-31
/proc/32128/task/32133/stat : 32133 (claspmt) R 32126 32128 32098 0 -1 4202560 48266 0 0 0 499493 198 0 0 25 0 32 0 30698249 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4668773 0 0 4096 18946 0 0 0 -1 27 0 0 0
[pid=32128/tid=32134] ppid=32126 vsize=7673456 CPUtime=4997.51 cores=0-31
/proc/32128/task/32134/stat : 32134 (claspmt) R 32126 32128 32098 0 -1 4202560 45488 0 0 0 499578 173 0 0 25 0 32 0 30698249 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 23 0 0 0
[pid=32128/tid=32135] ppid=32126 vsize=7673456 CPUtime=5004.65 cores=0-31
/proc/32128/task/32135/stat : 32135 (claspmt) R 32126 32128 32098 0 -1 4202560 36141 0 0 0 500310 155 0 0 25 0 32 0 30698249 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 12 0 0 0
[pid=32128/tid=32136] ppid=32126 vsize=7673456 CPUtime=4989.01 cores=0-31
/proc/32128/task/32136/stat : 32136 (claspmt) R 32126 32128 32098 0 -1 4202560 146842 0 0 0 498330 571 0 0 25 0 32 0 30698249 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4321038 0 0 4096 18946 0 0 0 -1 11 0 0 0
[pid=32128/tid=32137] ppid=32126 vsize=7673456 CPUtime=5004.64 cores=0-31
/proc/32128/task/32137/stat : 32137 (claspmt) R 32126 32128 32098 0 -1 4202560 36213 0 0 0 500297 167 0 0 25 0 32 0 30698249 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4687112 0 0 4096 18946 0 0 0 -1 17 0 0 0
[pid=32128/tid=32138] ppid=32126 vsize=7673456 CPUtime=5002.26 cores=0-31
/proc/32128/task/32138/stat : 32138 (claspmt) R 32126 32128 32098 0 -1 4202560 24410 0 0 0 500133 93 0 0 25 0 32 0 30698249 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 16 0 0 0
[pid=32128/tid=32139] ppid=32126 vsize=7673456 CPUtime=5004.68 cores=0-31
/proc/32128/task/32139/stat : 32139 (claspmt) R 32126 32128 32098 0 -1 4202560 62803 0 0 0 500222 246 0 0 25 0 32 0 30698249 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=32128/tid=32140] ppid=32126 vsize=7673456 CPUtime=5004.71 cores=0-31
/proc/32128/task/32140/stat : 32140 (claspmt) R 32126 32128 32098 0 -1 4202560 42632 0 0 0 500287 184 0 0 25 0 32 0 30698249 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 18 0 0 0
[pid=32128/tid=32141] ppid=32126 vsize=7673456 CPUtime=5004.32 cores=0-31
/proc/32128/task/32141/stat : 32141 (claspmt) R 32126 32128 32098 0 -1 4202560 47512 0 0 0 500228 204 0 0 25 0 32 0 30698249 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4320944 0 0 4096 18946 0 0 0 -1 9 0 0 0
[pid=32128/tid=32142] ppid=32126 vsize=7673456 CPUtime=5005.12 cores=0-31
/proc/32128/task/32142/stat : 32142 (claspmt) R 32126 32128 32098 0 -1 4202560 44527 0 0 0 500336 176 0 0 25 0 32 0 30698249 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4370264 0 0 4096 18946 0 0 0 -1 26 0 0 0
[pid=32128/tid=32143] ppid=32126 vsize=7673456 CPUtime=5004.73 cores=0-31
/proc/32128/task/32143/stat : 32143 (claspmt) R 32126 32128 32098 0 -1 4202560 39885 0 0 0 500304 169 0 0 25 0 32 0 30698249 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 10 0 0 0
[pid=32128/tid=32144] ppid=32126 vsize=7673456 CPUtime=4999.12 cores=0-31
/proc/32128/task/32144/stat : 32144 (claspmt) R 32126 32128 32098 0 -1 4202560 140844 0 0 0 499372 540 0 0 25 0 32 0 30698249 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 25 0 0 0
[pid=32128/tid=32145] ppid=32126 vsize=7673456 CPUtime=5004.65 cores=0-31
/proc/32128/task/32145/stat : 32145 (claspmt) R 32126 32128 32098 0 -1 4202560 33639 0 0 0 500322 143 0 0 25 0 32 0 30698249 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 14 0 0 0
[pid=32128/tid=32146] ppid=32126 vsize=7673456 CPUtime=4980.02 cores=0-31
/proc/32128/task/32146/stat : 32146 (claspmt) R 32126 32128 32098 0 -1 4202560 25429 0 0 0 497907 95 0 0 25 0 32 0 30698249 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 31 0 0 0
[pid=32128/tid=32147] ppid=32126 vsize=7673456 CPUtime=5004.66 cores=0-31
/proc/32128/task/32147/stat : 32147 (claspmt) R 32126 32128 32098 0 -1 4202560 52425 0 0 0 500254 212 0 0 25 0 32 0 30698249 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4320976 0 0 4096 18946 0 0 0 -1 21 0 0 0
[pid=32128/tid=32148] ppid=32126 vsize=7673456 CPUtime=5004.5 cores=0-31
/proc/32128/task/32148/stat : 32148 (claspmt) R 32126 32128 32098 0 -1 4202560 43001 0 0 0 500267 183 0 0 25 0 32 0 30698249 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670330 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=32128/tid=32149] ppid=32126 vsize=7673456 CPUtime=4992.04 cores=0-31
/proc/32128/task/32149/stat : 32149 (claspmt) R 32126 32128 32098 0 -1 4202560 48268 0 0 0 499005 199 0 0 25 0 32 0 30698249 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=32128/tid=32150] ppid=32126 vsize=7673456 CPUtime=5001.87 cores=0-31
/proc/32128/task/32150/stat : 32150 (claspmt) R 32126 32128 32098 0 -1 4202560 43285 0 0 0 499978 209 0 0 25 0 32 0 30698249 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 20 0 0 0
[pid=32128/tid=32151] ppid=32126 vsize=7673456 CPUtime=5004.03 cores=0-31
/proc/32128/task/32151/stat : 32151 (claspmt) R 32126 32128 32098 0 -1 4202560 37523 0 0 0 500252 151 0 0 25 0 32 0 30698250 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4342149 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=32128/tid=32152] ppid=32126 vsize=7673456 CPUtime=4986.05 cores=0-31
/proc/32128/task/32152/stat : 32152 (claspmt) R 32126 32128 32098 0 -1 4202560 147006 0 0 0 498009 596 0 0 25 0 32 0 30698250 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 258650653584 0 0 4096 18946 0 0 0 -1 19 0 0 0
[pid=32128/tid=32153] ppid=32126 vsize=7673456 CPUtime=4991.5 cores=0-31
/proc/32128/task/32153/stat : 32153 (claspmt) R 32126 32128 32098 0 -1 4202560 31565 0 0 0 499009 141 0 0 25 0 32 0 30698250 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 15 0 0 0
[pid=32128/tid=32154] ppid=32126 vsize=7673456 CPUtime=5004.2 cores=0-31
/proc/32128/task/32154/stat : 32154 (claspmt) R 32126 32128 32098 0 -1 4202560 22215 0 0 0 500327 93 0 0 25 0 32 0 30698250 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=32128/tid=32155] ppid=32126 vsize=7673456 CPUtime=5004.7 cores=0-31
/proc/32128/task/32155/stat : 32155 (claspmt) R 32126 32128 32098 0 -1 4202560 56006 0 0 0 500258 212 0 0 25 0 32 0 30698250 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 8 0 0 0
[pid=32128/tid=32156] ppid=32126 vsize=7673456 CPUtime=5004.48 cores=0-31
/proc/32128/task/32156/stat : 32156 (claspmt) R 32126 32128 32098 0 -1 4202560 40366 0 0 0 500277 171 0 0 25 0 32 0 30698250 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4669406 0 0 4096 18946 0 0 0 -1 28 0 0 0
[pid=32128/tid=32157] ppid=32126 vsize=7673456 CPUtime=5004.62 cores=0-31
/proc/32128/task/32157/stat : 32157 (claspmt) R 32126 32128 32098 0 -1 4202560 58112 0 0 0 500221 241 0 0 25 0 32 0 30698250 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 29 0 0 0
[pid=32128/tid=32158] ppid=32126 vsize=7673456 CPUtime=4994.76 cores=0-31
/proc/32128/task/32158/stat : 32158 (claspmt) R 32126 32128 32098 0 -1 4202560 37161 0 0 0 499328 148 0 0 25 0 32 0 30698250 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 22 0 0 0
[pid=32128/tid=32159] ppid=32126 vsize=7673456 CPUtime=5003.52 cores=0-31
/proc/32128/task/32159/stat : 32159 (claspmt) R 32126 32128 32098 0 -1 4202560 31663 0 0 0 500224 128 0 0 25 0 32 0 30698250 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 13 0 0 0
[pid=32128/tid=32160] ppid=32126 vsize=7673456 CPUtime=5003.76 cores=0-31
/proc/32128/task/32160/stat : 32160 (claspmt) R 32126 32128 32098 0 -1 4202560 106671 0 0 0 499946 430 0 0 25 0 32 0 30698250 7857618944 1663474 270582939648 4194304 4964800 140737030216864 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 160004
Current children cumulated vsize (KiB) 7673456

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)

Sending SIGKILL to -32128

Solver just ended. Dumping a history of the last processes samples

[startup+5011.1 s]
/proc/loadavg: 29.57 31.53 31.87 2/413 32316
/proc/meminfo: memFree=255179648/264257624 swapFree=67111528/67111528
[pid=32128] ppid=32126 vsize=7463360 CPUtime=160007 cores=0-31
/proc/32128/stat : 32128 (claspmt) R 32126 32128 32098 0 -1 4204032 1697015 0 0 0 15993716 6982 0 0 25 0 1 0 30698116 7642480640 1649556 270582939648 4194304 4964800 140737030216864 18446744073709551615 4628834 0 0 4096 18946 0 0 0 17 24 0 0 0
/proc/32128/statm: 1865840 1649556 474 189 0 1860189 0
Current children cumulated CPU time (s) 160007
Current children cumulated vsize (KiB) 7463360

[startup+5017.5 s]
/proc/loadavg: 27.36 31.04 31.71 2/413 32316
/proc/meminfo: memFree=251442672/264257624 swapFree=67111528/67111528
[pid=32128] ppid=32126 vsize=7463360 CPUtime=160013 cores=0-31
/proc/32128/stat : 32128 (claspmt) R 32126 32128 32098 0 -1 4204032 1697015 0 0 0 15993716 7622 0 0 25 0 1 0 30698116 7642480640 1649556 270582939648 4194304 4964800 140737030216864 18446744073709551615 4628834 0 0 4096 18946 0 0 0 17 28 0 0 0
/proc/32128/statm: 1865840 1649556 474 189 0 1860189 0
Current children cumulated CPU time (s) 160013
Current children cumulated vsize (KiB) 7463360

[startup+5019.1 s]
/proc/loadavg: 27.36 31.04 31.71 2/413 32316
/proc/meminfo: memFree=250762264/264257624 swapFree=67111528/67111528
[pid=32128] ppid=32126 vsize=7463360 CPUtime=160015 cores=0-31
/proc/32128/stat : 32128 (claspmt) R 32126 32128 32098 0 -1 4204032 1697015 0 0 0 15993716 7782 0 0 25 0 1 0 30698116 7642480640 1649556 270582939648 4194304 4964800 140737030216864 18446744073709551615 4628834 0 0 4096 18946 0 0 0 17 28 0 0 0
/proc/32128/statm: 1865840 1649556 474 189 0 1860189 0
Current children cumulated CPU time (s) 160015
Current children cumulated vsize (KiB) 7463360

[startup+5020.7 s]
/proc/loadavg: 25.41 30.58 31.56 2/413 32316
/proc/meminfo: memFree=249399952/264257624 swapFree=67111528/67111528
[pid=32128] ppid=32126 vsize=7463360 CPUtime=160017 cores=0-31
/proc/32128/stat : 32128 (claspmt) R 32126 32128 32098 0 -1 4204032 1697015 0 0 0 15993716 7942 0 0 25 0 1 0 30698116 7642480640 1649556 270582939648 4194304 4964800 140737030216864 18446744073709551615 4628834 0 0 4096 18946 0 0 0 17 28 0 0 0
/proc/32128/statm: 1865840 1649556 474 189 0 1860189 0
Current children cumulated CPU time (s) 160017
Current children cumulated vsize (KiB) 7463360

[startup+5021.5 s]
/proc/loadavg: 25.41 30.58 31.56 2/413 32316
/proc/meminfo: memFree=248719316/264257624 swapFree=67111528/67111528
[pid=32128] ppid=32126 vsize=7463360 CPUtime=160017 cores=0-31
/proc/32128/stat : 32128 (claspmt) R 32126 32128 32098 0 -1 4204032 1697015 0 0 0 15993716 8025 0 0 25 0 1 0 30698116 7642480640 1649556 270582939648 4194304 4964800 140737030216864 18446744073709551615 4628834 0 0 4096 18946 0 0 0 17 28 0 0 0
/proc/32128/statm: 1865840 1649556 474 189 0 1860189 0
Current children cumulated CPU time (s) 160017
Current children cumulated vsize (KiB) 7463360

[startup+5021.9 s]
/proc/loadavg: 25.41 30.58 31.56 2/413 32316
/proc/meminfo: memFree=248719316/264257624 swapFree=67111528/67111528
[pid=32128] ppid=32126 vsize=0 CPUtime=160018 cores=0-31
/proc/32128/stat : 32128 (claspmt) R 32126 32128 32098 0 -1 4204036 1697015 0 0 0 15993716 8062 0 0 25 0 1 0 30698116 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 28 0 0 0
/proc/32128/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 160018
Current children cumulated vsize (KiB) 0

Child ended because it received signal 24 (SIGXCPU)
Child dumped core
Real time (s): 5021.95
CPU time (s): 160018
CPU user time (s): 159937
CPU system time (s): 80.6737
CPU usage (%): 3186.37
Max. virtual memory (cumulated for all children) (KiB): 7673456

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 159937
system time used= 80.6737
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1697015
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 405419
involuntary context switches= 394599

runsolver used 29.9185 second user time and 111.711 second system time

The end

Launcher Data 

Begin job on node200 at 2011-06-09 22:58:32
IDJOB=3472447
IDBENCH=83107
IDSOLVER=1961
FILE ID=node200/3472447-1307653112
RUNJOBID= node200-1307653112-32112
PBS_JOBID= 13547803
Free space on /tmp= 61740 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT11/application/fuhs/slp-synthesis-AES/slp-synthesis-aes-bottom26.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3472447-1307653112/watcher-3472447-1307653112 -o /tmp/evaluation-result-3472447-1307653112/solver-3472447-1307653112 -C 160000 -W 5100 -M 256000  HOME/claspmt -t32 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3472447-1307653112.cnf

TIME LIMIT= 160000 seconds
MEMORY LIMIT= 256000 MiB
NBCORE= 32

MD5SUM BENCH= 98fee1752ddf8cd7cd77925ecf4bfe6e
RANDOM SEED=1254523522

node200.alineos.net Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 64
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 96
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 66
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 98
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.99
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 68
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 100
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.03
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 70
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 102
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 16
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 17
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 48
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 18
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 80
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 19
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 112
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.97
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 20
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 21
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 50
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.10
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 22
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 82
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.10
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 114
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 24
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 25
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 26
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 84
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 27
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 116
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 28
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 29
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 54
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 30
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 86
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 31
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 118
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     264257624 kB
MemFree:      262216012 kB
Buffers:        246644 kB
Cached:        1487540 kB
SwapCached:          0 kB
Active:         267164 kB
Inactive:      1530020 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     264257624 kB
LowFree:      262216012 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:            5424 kB
Writeback:           0 kB
AnonPages:       64100 kB
Mapped:          14108 kB
Slab:           163520 kB
PageTables:       4296 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  199240340 kB
Committed_AS:   169480 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296372 kB
VmallocChunk: 34359440603 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 55288 MiB
End job on node200 at 2011-06-10 00:22:14