Trace number 3372681

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 160005 5012.54

General information on the benchmark

Nameapplication/fuhs/slp-synthesis-AES/
slp-synthesis-aes-top23.cnf
MD5SUM6ee28191d649bfeef8c7d8c569a3cd59
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark38269.3
Satisfiable
(Un)Satisfiability was proved
Number of variables60823
Number of clauses193450
Sum of the clauses size472824
Maximum clause length2414
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 2117689
Number of clauses of size 374473
Number of clauses of size 4275
Number of clauses of size 521
Number of clauses of size over 5991

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3372681-1305947450/watcher-3372681-1305947450 -o /tmp/evaluation-result-3372681-1305947450/solver-3372681-1305947450 -C 160000 -W 5100 -M 266338304 plingeling -t 32 HOME/instance-3372681-1305947450.cnf 

running on 32 cores: 0,4,8,12,16,20,24,28,1,5,9,13,17,21,25,29,2,6,10,14,18,22,26,30,3,7,11,15,19,23,27,31

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 160000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 160030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 272730423296 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 272730474496 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 27.15 30.99 31.72 1/413 11331
/proc/meminfo: memFree=255504640/264257624 swapFree=67111528/67111528
[pid=11331] ppid=11329 vsize=32332 CPUtime=0.03 cores=0-31
/proc/11331/stat : 11331 (plingeling) R 11329 11331 11305 0 -1 4202496 6107 1240 0 0 1 1 0 1 20 0 1 0 706231470 33107968 6037 268435456000 4194304 4417452 140734999885504 18446744073709551615 258650649933 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/11331/statm: 8083 6048 138 55 0 5969 0

[startup+0.0451901 s]
/proc/loadavg: 27.15 30.99 31.72 1/413 11331
/proc/meminfo: memFree=255504640/264257624 swapFree=67111528/67111528
[pid=11331] ppid=11329 vsize=41584 CPUtime=0.04 cores=0-31
/proc/11331/stat : 11331 (plingeling) R 11329 11331 11305 0 -1 4202496 8356 1240 0 0 2 1 0 1 20 0 1 0 706231470 42582016 8286 268435456000 4194304 4417452 140734999885504 18446744073709551615 258650649933 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/11331/statm: 10396 8301 138 55 0 8282 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 41584

[startup+0.101199 s]
/proc/loadavg: 27.15 30.99 31.72 1/413 11331
/proc/meminfo: memFree=255504640/264257624 swapFree=67111528/67111528
[pid=11331] ppid=11329 vsize=51336 CPUtime=0.09 cores=0-31
/proc/11331/stat : 11331 (plingeling) R 11329 11331 11305 0 -1 4202496 10891 1240 0 0 6 2 0 1 20 0 1 0 706231470 52568064 10821 268435456000 4194304 4417452 140734999885504 18446744073709551615 258650652667 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/11331/statm: 12834 10836 141 55 0 10720 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 51336

[startup+0.301201 s]
/proc/loadavg: 27.15 30.99 31.72 1/413 11331
/proc/meminfo: memFree=255504640/264257624 swapFree=67111528/67111528
[pid=11331] ppid=11329 vsize=65212 CPUtime=0.3 cores=0-31
/proc/11331/stat : 11331 (plingeling) R 11329 11331 11305 0 -1 4202496 14378 1240 0 0 26 3 0 1 21 0 1 0 706231470 66777088 14308 268435456000 4194304 4417452 140734999885504 18446744073709551615 4264909 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/11331/statm: 16303 14308 146 55 0 14189 0
Current children cumulated CPU time (s) 0.3
Current children cumulated vsize (KiB) 65212

[startup+0.701231 s]
/proc/loadavg: 27.15 30.99 31.72 1/413 11331
/proc/meminfo: memFree=255504640/264257624 swapFree=67111528/67111528
[pid=11331] ppid=11329 vsize=93088 CPUtime=0.7 cores=0-31
/proc/11331/stat : 11331 (plingeling) R 11329 11331 11305 0 -1 4202496 21347 1240 0 0 64 5 0 1 22 0 1 0 706231470 95322112 21277 268435456000 4194304 4417452 140734999885504 18446744073709551615 258650574384 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/11331/statm: 23272 21277 146 55 0 21158 0
Current children cumulated CPU time (s) 0.7
Current children cumulated vsize (KiB) 93088

[startup+1.50127 s]
/proc/loadavg: 27.15 30.99 31.72 2/414 11337
/proc/meminfo: memFree=255373904/264257624 swapFree=67111528/67111528
[pid=11331] ppid=11329 vsize=171796 CPUtime=1.5 cores=0-31
/proc/11331/stat : 11331 (plingeling) R 11329 11331 11305 0 -1 4202496 41040 1240 0 0 140 9 0 1 25 0 1 0 706231470 175919104 40970 268435456000 4194304 4417452 140734999885504 18446744073709551615 4229142 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/11331/statm: 42949 40970 146 55 0 40835 0
Current children cumulated CPU time (s) 1.5
Current children cumulated vsize (KiB) 171796

[startup+3.10031 s]
/proc/loadavg: 25.05 30.49 31.55 2/414 11337
/proc/meminfo: memFree=255281188/264257624 swapFree=67111528/67111528
[pid=11331] ppid=11329 vsize=297264 CPUtime=3.09 cores=0-31
/proc/11331/stat : 11331 (plingeling) R 11329 11331 11305 0 -1 4202496 72375 1240 0 0 293 15 0 1 25 0 1 0 706231470 304398336 72305 268435456000 4194304 4417452 140734999885504 18446744073709551615 4259618 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/11331/statm: 74316 72305 146 55 0 72202 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 297264

[startup+6.3023 s]
/proc/loadavg: 25.05 30.49 31.55 33/446 11369
/proc/meminfo: memFree=255086924/264257624 swapFree=67111528/67111528
[pid=11331] ppid=11329 vsize=979628 CPUtime=89.57 cores=0-31
/proc/11331/stat : 11331 (plingeling) S 11329 11331 11305 0 -1 4202496 201958 1240 0 0 8829 127 0 1 19 0 33 0 706231470 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 30 0 0 0
/proc/11331/statm: 244907 103439 172 55 0 242793 0
[pid=11331/tid=11338] ppid=11329 vsize=979628 CPUtime=2.7 cores=0-31
/proc/11331/task/11338/stat : 11338 (plingeling) R 11329 11331 11305 0 -1 4202560 3131 1240 0 0 267 2 0 1 22 0 33 0 706231827 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4302145 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=11331/tid=11339] ppid=11329 vsize=979628 CPUtime=2.71 cores=0-31
/proc/11331/task/11339/stat : 11339 (plingeling) R 11329 11331 11305 0 -1 4202560 2679 1240 0 0 267 3 0 1 25 0 33 0 706231827 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4284341 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=11331/tid=11340] ppid=11329 vsize=979628 CPUtime=2.71 cores=0-31
/proc/11331/task/11340/stat : 11340 (plingeling) R 11329 11331 11305 0 -1 4202560 2700 1240 0 0 267 3 0 1 25 0 33 0 706231827 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4284595 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=11331/tid=11341] ppid=11329 vsize=979628 CPUtime=2.7 cores=0-31
/proc/11331/task/11341/stat : 11341 (plingeling) R 11329 11331 11305 0 -1 4202560 1048 1240 0 0 267 2 0 1 22 0 33 0 706231827 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364724 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11331/tid=11342] ppid=11329 vsize=979628 CPUtime=2.7 cores=0-31
/proc/11331/task/11342/stat : 11342 (plingeling) R 11329 11331 11305 0 -1 4202560 3231 1240 0 0 266 3 0 1 22 0 33 0 706231827 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4236964 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11331/tid=11343] ppid=11329 vsize=979628 CPUtime=2.7 cores=0-31
/proc/11331/task/11343/stat : 11343 (plingeling) R 11329 11331 11305 0 -1 4202560 3198 1240 0 0 265 4 0 1 25 0 33 0 706231827 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4275978 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=11331/tid=11344] ppid=11329 vsize=979628 CPUtime=2.71 cores=0-31
/proc/11331/task/11344/stat : 11344 (plingeling) R 11329 11331 11305 0 -1 4202560 3286 1240 0 0 268 2 0 1 22 0 33 0 706231827 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4268511 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11331/tid=11345] ppid=11329 vsize=979628 CPUtime=2.64 cores=0-31
/proc/11331/task/11345/stat : 11345 (plingeling) R 11329 11331 11305 0 -1 4202560 6056 1240 0 0 259 4 0 1 23 0 33 0 706231827 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4286960 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=11331/tid=11346] ppid=11329 vsize=979628 CPUtime=2.7 cores=0-31
/proc/11331/task/11346/stat : 11346 (plingeling) R 11329 11331 11305 0 -1 4202560 5535 1240 0 0 266 3 0 1 22 0 33 0 706231827 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4275735 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=11331/tid=11347] ppid=11329 vsize=979628 CPUtime=2.69 cores=0-31
/proc/11331/task/11347/stat : 11347 (plingeling) R 11329 11331 11305 0 -1 4202560 4897 1240 0 0 265 3 0 1 22 0 33 0 706231827 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=11331/tid=11348] ppid=11329 vsize=979628 CPUtime=2.69 cores=0-31
/proc/11331/task/11348/stat : 11348 (plingeling) R 11329 11331 11305 0 -1 4202560 3106 1240 0 0 265 3 0 1 25 0 33 0 706231828 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=11331/tid=11349] ppid=11329 vsize=979628 CPUtime=2.7 cores=0-31
/proc/11331/task/11349/stat : 11349 (plingeling) R 11329 11331 11305 0 -1 4202560 8715 1240 0 0 264 5 0 1 22 0 33 0 706231828 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4278271 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=11331/tid=11350] ppid=11329 vsize=979628 CPUtime=2.69 cores=0-31
/proc/11331/task/11350/stat : 11350 (plingeling) R 11329 11331 11305 0 -1 4202560 3780 1240 0 0 267 1 0 1 23 0 33 0 706231828 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4284626 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=11331/tid=11351] ppid=11329 vsize=979628 CPUtime=2.7 cores=0-31
/proc/11331/task/11351/stat : 11351 (plingeling) R 11329 11331 11305 0 -1 4202560 3074 1240 0 0 266 3 0 1 22 0 33 0 706231828 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4286414 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=11331/tid=11352] ppid=11329 vsize=979628 CPUtime=2.69 cores=0-31
/proc/11331/task/11352/stat : 11352 (plingeling) R 11329 11331 11305 0 -1 4202560 6176 1240 0 0 264 4 0 1 22 0 33 0 706231828 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4277594 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11331/tid=11353] ppid=11329 vsize=979628 CPUtime=2.69 cores=0-31
/proc/11331/task/11353/stat : 11353 (plingeling) R 11329 11331 11305 0 -1 4202560 3801 1240 0 0 265 3 0 1 25 0 33 0 706231828 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4286467 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=11331/tid=11354] ppid=11329 vsize=979628 CPUtime=2.69 cores=0-31
/proc/11331/task/11354/stat : 11354 (plingeling) R 11329 11331 11305 0 -1 4202560 2068 1240 0 0 265 3 0 1 25 0 33 0 706231828 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4287111 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=11331/tid=11355] ppid=11329 vsize=979628 CPUtime=2.69 cores=0-31
/proc/11331/task/11355/stat : 11355 (plingeling) R 11329 11331 11305 0 -1 4202560 3204 1240 0 0 265 3 0 1 25 0 33 0 706231828 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4284514 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=11331/tid=11356] ppid=11329 vsize=979628 CPUtime=2.67 cores=0-31
/proc/11331/task/11356/stat : 11356 (plingeling) R 11329 11331 11305 0 -1 4202560 5361 1240 0 0 264 2 0 1 22 0 33 0 706231829 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4281907 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=11331/tid=11357] ppid=11329 vsize=979628 CPUtime=2.69 cores=0-31
/proc/11331/task/11357/stat : 11357 (plingeling) R 11329 11331 11305 0 -1 4202560 3555 1240 0 0 265 3 0 1 22 0 33 0 706231829 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4286414 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=11331/tid=11358] ppid=11329 vsize=979628 CPUtime=2.69 cores=0-31
/proc/11331/task/11358/stat : 11358 (plingeling) R 11329 11331 11305 0 -1 4202560 7483 1240 0 0 264 4 0 1 22 0 33 0 706231829 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=11331/tid=11359] ppid=11329 vsize=979628 CPUtime=2.69 cores=0-31
/proc/11331/task/11359/stat : 11359 (plingeling) R 11329 11331 11305 0 -1 4202560 2606 1240 0 0 265 3 0 1 22 0 33 0 706231829 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4284484 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=11331/tid=11360] ppid=11329 vsize=979628 CPUtime=2.68 cores=0-31
/proc/11331/task/11360/stat : 11360 (plingeling) R 11329 11331 11305 0 -1 4202560 2864 1240 0 0 264 3 0 1 25 0 33 0 706231829 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=11331/tid=11361] ppid=11329 vsize=979628 CPUtime=2.69 cores=0-31
/proc/11331/task/11361/stat : 11361 (plingeling) R 11329 11331 11305 0 -1 4202560 3041 1240 0 0 266 2 0 1 25 0 33 0 706231829 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4284626 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=11331/tid=11362] ppid=11329 vsize=979628 CPUtime=2.69 cores=0-31
/proc/11331/task/11362/stat : 11362 (plingeling) R 11329 11331 11305 0 -1 4202560 8331 1240 0 0 264 4 0 1 22 0 33 0 706231829 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4284626 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=11331/tid=11363] ppid=11329 vsize=979628 CPUtime=2.69 cores=0-31
/proc/11331/task/11363/stat : 11363 (plingeling) R 11329 11331 11305 0 -1 4202560 4267 1240 0 0 265 3 0 1 22 0 33 0 706231829 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4275950 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=11331/tid=11364] ppid=11329 vsize=979628 CPUtime=2.69 cores=0-31
/proc/11331/task/11364/stat : 11364 (plingeling) R 11329 11331 11305 0 -1 4202560 3779 1240 0 0 265 3 0 1 22 0 33 0 706231829 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4264403 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=11331/tid=11365] ppid=11329 vsize=979628 CPUtime=2.69 cores=0-31
/proc/11331/task/11365/stat : 11365 (plingeling) R 11329 11331 11305 0 -1 4202560 4300 1240 0 0 265 3 0 1 22 0 33 0 706231829 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4284514 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=11331/tid=11366] ppid=11329 vsize=979628 CPUtime=2.56 cores=0-31
/proc/11331/task/11366/stat : 11366 (plingeling) R 11329 11331 11305 0 -1 4202560 3454 1240 0 0 255 0 0 1 25 0 33 0 706231829 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4284677 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=11331/tid=11367] ppid=11329 vsize=979628 CPUtime=2.69 cores=0-31
/proc/11331/task/11367/stat : 11367 (plingeling) R 11329 11331 11305 0 -1 4202560 4171 1240 0 0 265 3 0 1 22 0 33 0 706231829 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4284514 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=11331/tid=11368] ppid=11329 vsize=979628 CPUtime=2.68 cores=0-31
/proc/11331/task/11368/stat : 11368 (plingeling) R 11329 11331 11305 0 -1 4202560 2340 1240 0 0 264 3 0 1 25 0 33 0 706231829 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4264232 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=11331/tid=11369] ppid=11329 vsize=979628 CPUtime=2.67 cores=0-31
/proc/11331/task/11369/stat : 11369 (plingeling) R 11329 11331 11305 0 -1 4202560 4307 1240 0 0 263 3 0 1 25 0 33 0 706231829 1003139072 103439 268435456000 4194304 4417452 140734999885504 18446744073709551615 4278271 0 0 4096 17442 0 0 0 -1 28 0 0 0
Current children cumulated CPU time (s) 89.57
Current children cumulated vsize (KiB) 979628

[startup+12.7015 s]
/proc/loadavg: 26.12 30.54 31.56 33/446 11369
/proc/meminfo: memFree=254894264/264257624 swapFree=67111528/67111528
[pid=11331] ppid=11329 vsize=1123340 CPUtime=293.17 cores=0-31
/proc/11331/stat : 11331 (plingeling) S 11329 11331 11305 0 -1 4202496 266932 1240 0 0 29135 181 0 1 19 0 33 0 706231470 1150300160 146115 268435456000 4194304 4417452 140734999885504 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 30 0 0 0
/proc/11331/statm: 280835 146115 175 55 0 278721 0
[pid=11331/tid=11338] ppid=11329 vsize=1123340 CPUtime=9.07 cores=0-31
/proc/11331/task/11338/stat : 11338 (plingeling) R 11329 11331 11305 0 -1 4202560 5159 1240 0 0 903 3 0 1 25 0 33 0 706231827 1150300160 146115 268435456000 4194304 4417452 140734999885504 18446744073709551615 4258394 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=11331/tid=11339] ppid=11329 vsize=1123340 CPUtime=9.07 cores=0-31
/proc/11331/task/11339/stat : 11339 (plingeling) R 11329 11331 11305 0 -1 4202560 6092 1240 0 0 900 6 0 1 25 0 33 0 706231827 1150300160 146115 268435456000 4194304 4417452 140734999885504 18446744073709551615 4273277 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=11331/tid=11340] ppid=11329 vsize=1123340 CPUtime=9.1 cores=0-31
/proc/11331/task/11340/stat : 11340 (plingeling) R 11329 11331 11305 0 -1 4202560 2700 1240 0 0 906 3 0 1 25 0 33 0 706231827 1150300160 146115 268435456000 4194304 4417452 140734999885504 18446744073709551615 4232705 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=11331/tid=11341] ppid=11329 vsize=1123340 CPUtime=9.06 cores=0-31
/proc/11331/task/11341/stat : 11341 (plingeling) R 11329 11331 11305 0 -1 4202560 1048 1240 0 0 903 2 0 1 25 0 33 0 706231827 1150300160 146115 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=11331/tid=11342] ppid=11329 vsize=1123340 CPUtime=9.09 cores=0-31
/proc/11331/task/11342/stat : 11342 (plingeling) R 11329 11331 11305 0 -1 4202560 5432 1240 0 0 902 6 0 1 25 0 33 0 706231827 1150300160 146115 268435456000 4194304 4417452 140734999885504 18446744073709551615 4258201 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=11331/tid=11343] ppid=11329 vsize=1123340 CPUtime=9.09 cores=0-31
/proc/11331/task/11343/stat : 11343 (plingeling) R 11329 11331 11305 0 -1 4202560 5430 1240 0 0 902 6 0 1 25 0 33 0 706231827 1150300160 146115 268435456000 4194304 4417452 140734999885504 18446744073709551615 4250017 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=11331/tid=11344] ppid=11329 vsize=1123340 CPUtime=9.08 cores=0-31
/proc/11331/task/11344/stat : 11344 (plingeling) R 11329 11331 11305 0 -1 4202560 3798 1240 0 0 904 3 0 1 25 0 33 0 706231827 1150300160 146115 268435456000 4194304 4417452 140734999885504 18446744073709551615 4250936 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=11331/tid=11345] ppid=11329 vsize=1123340 CPUtime=8.82 cores=0-31
/proc/11331/task/11345/stat : 11345 (plingeling) R 11329 11331 11305 0 -1 4202560 9000 1240 0 0 874 7 0 1 25 0 33 0 706231827 1150300160 146115 268435456000 4194304 4417452 140734999885504 18446744073709551615 4252389 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=11331/tid=11346] ppid=11329 vsize=1123340 CPUtime=9.08 cores=0-31
/proc/11331/task/11346/stat : 11346 (plingeling) R 11329 11331 11305 0 -1 4202560 7592 1240 0 0 901 6 0 1 25 0 33 0 706231827 1150300160 146115 268435456000 4194304 4417452 140734999885504 18446744073709551615 4236319 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11331/tid=11347] ppid=11329 vsize=1123340 CPUtime=9.09 cores=0-31
/proc/11331/task/11347/stat : 11347 (plingeling) R 11329 11331 11305 0 -1 4202560 8691 1240 0 0 901 7 0 1 25 0 33 0 706231827 1150300160 146115 268435456000 4194304 4417452 140734999885504 18446744073709551615 4258540 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11331/tid=11348] ppid=11329 vsize=1123340 CPUtime=9.07 cores=0-31
/proc/11331/task/11348/stat : 11348 (plingeling) R 11329 11331 11305 0 -1 4202560 3106 1240 0 0 903 3 0 1 25 0 33 0 706231828 1150300160 146115 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364604 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=11331/tid=11349] ppid=11329 vsize=1123340 CPUtime=9.09 cores=0-31
/proc/11331/task/11349/stat : 11349 (plingeling) R 11329 11331 11305 0 -1 4202560 10874 1240 0 0 900 8 0 1 25 0 33 0 706231828 1150300160 146115 268435456000 4194304 4417452 140734999885504 18446744073709551615 4335984 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=11331/tid=11350] ppid=11329 vsize=1123340 CPUtime=9.03 cores=0-31
/proc/11331/task/11350/stat : 11350 (plingeling) R 11329 11331 11305 0 -1 4202560 6748 1240 0 0 897 5 0 1 25 0 33 0 706231828 1150300160 146115 268435456000 4194304 4417452 140734999885504 18446744073709551615 4235411 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=11331/tid=11351] ppid=11329 vsize=1123340 CPUtime=9.04 cores=0-31
/proc/11331/task/11351/stat : 11351 (plingeling) R 11329 11331 11305 0 -1 4202560 5666 1240 0 0 899 4 0 1 25 0 33 0 706231828 1150300160 146115 268435456000 4194304 4417452 140734999885504 18446744073709551615 4249976 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=11331/tid=11352] ppid=11329 vsize=1123340 CPUtime=9.09 cores=0-31
/proc/11331/task/11352/stat : 11352 (plingeling) R 11329 11331 11305 0 -1 4202560 7082 1240 0 0 904 4 0 1 25 0 33 0 706231828 1150300160 146115 268435456000 4194304 4417452 140734999885504 18446744073709551615 4264911 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11331/tid=11353] ppid=11329 vsize=1123340 CPUtime=9.04 cores=0-31
/proc/11331/task/11353/stat : 11353 (plingeling) R 11329 11331 11305 0 -1 4202560 7684 1240 0 0 896 7 0 1 25 0 33 0 706231828 1150300160 146115 268435456000 4194304 4417452 140734999885504 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11331/tid=11354] ppid=11329 vsize=1123340 CPUtime=8.99 cores=0-31
/proc/11331/task/11354/stat : 11354 (plingeling) R 11329 11331 11305 0 -1 4202560 4636 1240 0 0 895 3 0 1 25 0 33 0 706231828 1150300160 146115 268435456000 4194304 4417452 140734999885504 18446744073709551615 4250022 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=11331/tid=11355] ppid=11329 vsize=1123340 CPUtime=9.08 cores=0-31
/proc/11331/task/11355/stat : 11355 (plingeling) R 11329 11331 11305 0 -1 4202560 6696 1240 0 0 903 4 0 1 25 0 33 0 706231828 1150300160 146115 268435456000 4194304 4417452 140734999885504 18446744073709551615 4258267 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=11331/tid=11356] ppid=11329 vsize=1123340 CPUtime=9.05 cores=0-31
/proc/11331/task/11356/stat : 11356 (plingeling) R 11329 11331 11305 0 -1 4202560 6074 1240 0 0 902 2 0 1 25 0 33 0 706231829 1150300160 146115 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=11331/tid=11357] ppid=11329 vsize=1123340 CPUtime=9.07 cores=0-31

################
# More data... #
################

[pid=11331/tid=11349] ppid=11329 vsize=12710964 CPUtime=4942.36 cores=0-31
/proc/11331/task/11349/stat : 11349 (plingeling) R 11329 11331 11305 0 -1 4202560 555797 1240 0 0 493553 682 0 1 25 0 33 0 706231828 13016027136 3053933 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=11331/tid=11350] ppid=11329 vsize=12710964 CPUtime=4952.94 cores=0-31
/proc/11331/task/11350/stat : 11350 (plingeling) R 11329 11331 11305 0 -1 4202560 559875 1240 0 0 494556 737 0 1 25 0 33 0 706231828 13016027136 3053933 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=11331/tid=11351] ppid=11329 vsize=12710964 CPUtime=4955.71 cores=0-31
/proc/11331/task/11351/stat : 11351 (plingeling) R 11329 11331 11305 0 -1 4202560 764701 1240 0 0 494578 992 0 1 25 0 33 0 706231828 13016027136 3053933 268435456000 4194304 4417452 140734999885504 18446744073709551615 4236649 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=11331/tid=11352] ppid=11329 vsize=12710964 CPUtime=4955.11 cores=0-31
/proc/11331/task/11352/stat : 11352 (plingeling) R 11329 11331 11305 0 -1 4202560 582302 1240 0 0 494700 810 0 1 25 0 33 0 706231828 13016027136 3053933 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364829 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=11331/tid=11353] ppid=11329 vsize=12710964 CPUtime=4954.24 cores=0-31
/proc/11331/task/11353/stat : 11353 (plingeling) R 11329 11331 11305 0 -1 4202560 566531 1240 0 0 494740 683 0 1 25 0 33 0 706231828 13016027136 3053933 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11331/tid=11354] ppid=11329 vsize=12710964 CPUtime=4954.27 cores=0-31
/proc/11331/task/11354/stat : 11354 (plingeling) R 11329 11331 11305 0 -1 4202560 571879 1240 0 0 494662 764 0 1 25 0 33 0 706231828 13016027136 3053933 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=11331/tid=11355] ppid=11329 vsize=12710964 CPUtime=4953.62 cores=0-31
/proc/11331/task/11355/stat : 11355 (plingeling) R 11329 11331 11305 0 -1 4202560 597916 1240 0 0 494555 806 0 1 25 0 33 0 706231828 13016027136 3053933 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11331/tid=11356] ppid=11329 vsize=12710964 CPUtime=4952.08 cores=0-31
/proc/11331/task/11356/stat : 11356 (plingeling) R 11329 11331 11305 0 -1 4202560 624959 1240 0 0 494361 846 0 1 25 0 33 0 706231829 13016027136 3053933 268435456000 4194304 4417452 140734999885504 18446744073709551615 4365027 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=11331/tid=11357] ppid=11329 vsize=12710964 CPUtime=4954.76 cores=0-31
/proc/11331/task/11357/stat : 11357 (plingeling) R 11329 11331 11305 0 -1 4202560 535036 1240 0 0 494724 751 0 1 25 0 33 0 706231829 13016027136 3053933 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11331/tid=11358] ppid=11329 vsize=12710964 CPUtime=4952.25 cores=0-31
/proc/11331/task/11358/stat : 11358 (plingeling) R 11329 11331 11305 0 -1 4202560 530235 1240 0 0 494409 815 0 1 25 0 33 0 706231829 13016027136 3053933 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364809 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=11331/tid=11359] ppid=11329 vsize=12710964 CPUtime=4954.58 cores=0-31
/proc/11331/task/11359/stat : 11359 (plingeling) R 11329 11331 11305 0 -1 4202560 734232 1240 0 0 494602 855 0 1 25 0 33 0 706231829 13016027136 3053933 268435456000 4194304 4417452 140734999885504 18446744073709551615 4366113 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=11331/tid=11360] ppid=11329 vsize=12710964 CPUtime=4954.44 cores=0-31
/proc/11331/task/11360/stat : 11360 (plingeling) R 11329 11331 11305 0 -1 4202560 627267 1240 0 0 494568 875 0 1 25 0 33 0 706231829 13016027136 3053933 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=11331/tid=11361] ppid=11329 vsize=12710964 CPUtime=4949.48 cores=0-31
/proc/11331/task/11361/stat : 11361 (plingeling) R 11329 11331 11305 0 -1 4202560 584933 1240 0 0 494352 595 0 1 25 0 33 0 706231829 13016027136 3053933 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=11331/tid=11362] ppid=11329 vsize=12710964 CPUtime=4954.78 cores=0-31
/proc/11331/task/11362/stat : 11362 (plingeling) R 11329 11331 11305 0 -1 4202560 546395 1240 0 0 494858 619 0 1 25 0 33 0 706231829 13016027136 3053933 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11331/tid=11363] ppid=11329 vsize=12710964 CPUtime=4951.13 cores=0-31
/proc/11331/task/11363/stat : 11363 (plingeling) R 11329 11331 11305 0 -1 4202560 542634 1240 0 0 494463 649 0 1 25 0 33 0 706231829 13016027136 3053933 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=11331/tid=11364] ppid=11329 vsize=12710964 CPUtime=4955.12 cores=0-31
/proc/11331/task/11364/stat : 11364 (plingeling) R 11329 11331 11305 0 -1 4202560 532720 1240 0 0 494876 635 0 1 25 0 33 0 706231829 13016027136 3053933 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=11331/tid=11365] ppid=11329 vsize=12710964 CPUtime=4915.86 cores=0-31
/proc/11331/task/11365/stat : 11365 (plingeling) R 11329 11331 11305 0 -1 4202560 549486 1240 0 0 490859 726 0 1 25 0 33 0 706231829 13016027136 3053933 268435456000 4194304 4417452 140734999885504 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=11331/tid=11366] ppid=11329 vsize=12710964 CPUtime=4954.36 cores=0-31
/proc/11331/task/11366/stat : 11366 (plingeling) R 11329 11331 11305 0 -1 4202560 621194 1240 0 0 494664 771 0 1 25 0 33 0 706231829 13016027136 3053933 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364821 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=11331/tid=11367] ppid=11329 vsize=12710964 CPUtime=4953.02 cores=0-31
/proc/11331/task/11367/stat : 11367 (plingeling) R 11329 11331 11305 0 -1 4202560 527049 1240 0 0 494828 473 0 1 25 0 33 0 706231829 13016027136 3053933 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=11331/tid=11368] ppid=11329 vsize=12710964 CPUtime=4957.18 cores=0-31
/proc/11331/task/11368/stat : 11368 (plingeling) R 11329 11331 11305 0 -1 4202560 487706 1240 0 0 495231 486 0 1 25 0 33 0 706231829 13016027136 3053933 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=11331/tid=11369] ppid=11329 vsize=12710964 CPUtime=4954.2 cores=0-31
/proc/11331/task/11369/stat : 11369 (plingeling) R 11329 11331 11305 0 -1 4202560 637330 1240 0 0 494609 810 0 1 25 0 33 0 706231829 13016027136 3053933 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 20 0 0 0
Current children cumulated CPU time (s) 158428
Current children cumulated vsize (KiB) 12710964



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5011.62 s]
/proc/loadavg: 32.02 32.06 32.06 33/446 11516
/proc/meminfo: memFree=243496808/264257624 swapFree=67111528/67111528
[pid=11331] ppid=11329 vsize=12475952 CPUtime=160004 cores=0-31
/proc/11331/stat : 11331 (plingeling) S 11329 11331 11305 0 -1 4202496 18981511 1240 0 0 15976764 23618 0 1 19 0 33 0 706231470 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 30 0 0 0
/proc/11331/statm: 3118988 2997139 175 55 0 3116874 0
[pid=11331/tid=11338] ppid=11329 vsize=12475952 CPUtime=5003.83 cores=0-31
/proc/11331/task/11338/stat : 11338 (plingeling) R 11329 11331 11305 0 -1 4202560 654478 1240 0 0 499222 1160 0 1 25 0 33 0 706231827 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=11331/tid=11339] ppid=11329 vsize=12475952 CPUtime=5005.16 cores=0-31
/proc/11331/task/11339/stat : 11339 (plingeling) R 11329 11331 11305 0 -1 4202560 687556 1240 0 0 499768 747 0 1 25 0 33 0 706231827 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=11331/tid=11340] ppid=11329 vsize=12475952 CPUtime=5004.39 cores=0-31
/proc/11331/task/11340/stat : 11340 (plingeling) R 11329 11331 11305 0 -1 4202560 516667 1240 0 0 499863 575 0 1 25 0 33 0 706231827 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4366806 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=11331/tid=11341] ppid=11329 vsize=12475952 CPUtime=5001.78 cores=0-31
/proc/11331/task/11341/stat : 11341 (plingeling) R 11329 11331 11305 0 -1 4202560 404873 1240 0 0 499838 339 0 1 25 0 33 0 706231827 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4237189 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=11331/tid=11342] ppid=11329 vsize=12475952 CPUtime=5000.27 cores=0-31
/proc/11331/task/11342/stat : 11342 (plingeling) R 11329 11331 11305 0 -1 4202560 560098 1240 0 0 499433 593 0 1 25 0 33 0 706231827 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=11331/tid=11343] ppid=11329 vsize=12475952 CPUtime=5003.17 cores=0-31
/proc/11331/task/11343/stat : 11343 (plingeling) R 11329 11331 11305 0 -1 4202560 617069 1240 0 0 499439 877 0 1 25 0 33 0 706231827 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=11331/tid=11344] ppid=11329 vsize=12475952 CPUtime=5004.83 cores=0-31
/proc/11331/task/11344/stat : 11344 (plingeling) R 11329 11331 11305 0 -1 4202560 564209 1240 0 0 499781 701 0 1 25 0 33 0 706231827 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=11331/tid=11345] ppid=11329 vsize=12475952 CPUtime=4996.15 cores=0-31
/proc/11331/task/11345/stat : 11345 (plingeling) R 11329 11331 11305 0 -1 4202560 639770 1240 0 0 498906 708 0 1 25 0 33 0 706231827 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=11331/tid=11346] ppid=11329 vsize=12475952 CPUtime=5002 cores=0-31
/proc/11331/task/11346/stat : 11346 (plingeling) R 11329 11331 11305 0 -1 4202560 572257 1240 0 0 499464 735 0 1 25 0 33 0 706231827 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=11331/tid=11347] ppid=11329 vsize=12475952 CPUtime=5002.65 cores=0-31
/proc/11331/task/11347/stat : 11347 (plingeling) R 11329 11331 11305 0 -1 4202560 742063 1240 0 0 499349 915 0 1 25 0 33 0 706231827 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=11331/tid=11348] ppid=11329 vsize=12475952 CPUtime=4960.59 cores=0-31
/proc/11331/task/11348/stat : 11348 (plingeling) R 11329 11331 11305 0 -1 4202560 525814 1240 0 0 495369 689 0 1 25 0 33 0 706231828 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364984 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=11331/tid=11349] ppid=11329 vsize=12475952 CPUtime=4991.69 cores=0-31
/proc/11331/task/11349/stat : 11349 (plingeling) R 11329 11331 11305 0 -1 4202560 555797 1240 0 0 498486 682 0 1 25 0 33 0 706231828 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4365004 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=11331/tid=11350] ppid=11329 vsize=12475952 CPUtime=5002.27 cores=0-31
/proc/11331/task/11350/stat : 11350 (plingeling) R 11329 11331 11305 0 -1 4202560 559875 1240 0 0 499489 737 0 1 25 0 33 0 706231828 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4366141 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=11331/tid=11351] ppid=11329 vsize=12475952 CPUtime=5005.04 cores=0-31
/proc/11331/task/11351/stat : 11351 (plingeling) R 11329 11331 11305 0 -1 4202560 764701 1240 0 0 499511 992 0 1 25 0 33 0 706231828 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4365058 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=11331/tid=11352] ppid=11329 vsize=12475952 CPUtime=5004.44 cores=0-31
/proc/11331/task/11352/stat : 11352 (plingeling) R 11329 11331 11305 0 -1 4202560 582302 1240 0 0 499633 810 0 1 25 0 33 0 706231828 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=11331/tid=11353] ppid=11329 vsize=12475952 CPUtime=5003.57 cores=0-31
/proc/11331/task/11353/stat : 11353 (plingeling) R 11329 11331 11305 0 -1 4202560 566531 1240 0 0 499673 683 0 1 25 0 33 0 706231828 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=11331/tid=11354] ppid=11329 vsize=12475952 CPUtime=5003.58 cores=0-31
/proc/11331/task/11354/stat : 11354 (plingeling) R 11329 11331 11305 0 -1 4202560 571879 1240 0 0 499593 764 0 1 25 0 33 0 706231828 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=11331/tid=11355] ppid=11329 vsize=12475952 CPUtime=5002.96 cores=0-31
/proc/11331/task/11355/stat : 11355 (plingeling) R 11329 11331 11305 0 -1 4202560 597916 1240 0 0 499489 806 0 1 25 0 33 0 706231828 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=11331/tid=11356] ppid=11329 vsize=12475952 CPUtime=5001.41 cores=0-31
/proc/11331/task/11356/stat : 11356 (plingeling) R 11329 11331 11305 0 -1 4202560 624959 1240 0 0 499294 846 0 1 25 0 33 0 706231829 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=11331/tid=11357] ppid=11329 vsize=12475952 CPUtime=5004.09 cores=0-31
/proc/11331/task/11357/stat : 11357 (plingeling) R 11329 11331 11305 0 -1 4202560 564234 1240 0 0 499596 812 0 1 25 0 33 0 706231829 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=11331/tid=11358] ppid=11329 vsize=12475952 CPUtime=5001.58 cores=0-31
/proc/11331/task/11358/stat : 11358 (plingeling) R 11329 11331 11305 0 -1 4202560 530235 1240 0 0 499342 815 0 1 25 0 33 0 706231829 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=11331/tid=11359] ppid=11329 vsize=12475952 CPUtime=5003.33 cores=0-31
/proc/11331/task/11359/stat : 11359 (plingeling) R 11329 11331 11305 0 -1 4202560 794067 1240 0 0 499435 897 0 1 25 0 33 0 706231829 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=11331/tid=11360] ppid=11329 vsize=12475952 CPUtime=5003.78 cores=0-31
/proc/11331/task/11360/stat : 11360 (plingeling) R 11329 11331 11305 0 -1 4202560 627267 1240 0 0 499502 875 0 1 25 0 33 0 706231829 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364716 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=11331/tid=11361] ppid=11329 vsize=12475952 CPUtime=4998.81 cores=0-31
/proc/11331/task/11361/stat : 11361 (plingeling) R 11329 11331 11305 0 -1 4202560 639966 1240 0 0 499223 657 0 1 25 0 33 0 706231829 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=11331/tid=11362] ppid=11329 vsize=12475952 CPUtime=5004.11 cores=0-31
/proc/11331/task/11362/stat : 11362 (plingeling) R 11329 11331 11305 0 -1 4202560 546395 1240 0 0 499791 619 0 1 25 0 33 0 706231829 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4366400 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=11331/tid=11363] ppid=11329 vsize=12475952 CPUtime=5000.46 cores=0-31
/proc/11331/task/11363/stat : 11363 (plingeling) R 11329 11331 11305 0 -1 4202560 542634 1240 0 0 499396 649 0 1 25 0 33 0 706231829 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4366400 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=11331/tid=11364] ppid=11329 vsize=12475952 CPUtime=5004.45 cores=0-31
/proc/11331/task/11364/stat : 11364 (plingeling) R 11329 11331 11305 0 -1 4202560 532720 1240 0 0 499809 635 0 1 25 0 33 0 706231829 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=11331/tid=11365] ppid=11329 vsize=12475952 CPUtime=4963.79 cores=0-31
/proc/11331/task/11365/stat : 11365 (plingeling) R 11329 11331 11305 0 -1 4202560 549486 1240 0 0 495651 727 0 1 25 0 33 0 706231829 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=11331/tid=11366] ppid=11329 vsize=12475952 CPUtime=5003.69 cores=0-31
/proc/11331/task/11366/stat : 11366 (plingeling) R 11329 11331 11305 0 -1 4202560 621194 1240 0 0 499597 771 0 1 25 0 33 0 706231829 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=11331/tid=11367] ppid=11329 vsize=12475952 CPUtime=5002.35 cores=0-31
/proc/11331/task/11367/stat : 11367 (plingeling) R 11329 11331 11305 0 -1 4202560 527049 1240 0 0 499761 473 0 1 25 0 33 0 706231829 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=11331/tid=11368] ppid=11329 vsize=12475952 CPUtime=5006.52 cores=0-31
/proc/11331/task/11368/stat : 11368 (plingeling) R 11329 11331 11305 0 -1 4202560 487706 1240 0 0 500165 486 0 1 25 0 33 0 706231829 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=11331/tid=11369] ppid=11329 vsize=12475952 CPUtime=5003.54 cores=0-31
/proc/11331/task/11369/stat : 11369 (plingeling) R 11329 11331 11305 0 -1 4202560 637330 1240 0 0 499543 810 0 1 25 0 33 0 706231829 12775374848 2997139 268435456000 4194304 4417452 140734999885504 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 20 0 0 0
Current children cumulated CPU time (s) 160004
Current children cumulated vsize (KiB) 12475952

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5011.8 s]
/proc/loadavg: 32.02 32.06 32.06 33/446 11516
/proc/meminfo: memFree=243496808/264257624 swapFree=67111528/67111528
[pid=11331] ppid=11329 vsize=0 CPUtime=160004 cores=0-31
/proc/11331/stat : 11331 (plingeling) Z 11329 11331 11305 0 -1 4203548 18981511 1240 0 0 15976787 23633 0 1 18 0 2 0 706231470 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 30 0 0 0
/proc/11331/statm: 0 0 0 0 0 0 0
[pid=11331/tid=11366] ppid=11329 vsize=0 CPUtime=5003.84 cores=0-31
/proc/11331/task/11366/stat : 11366 (plingeling) R 11329 11331 11305 0 -1 4203588 621194 1240 0 0 499597 786 0 1 25 0 2 0 706231829 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 30 0 0 0
Current children cumulated CPU time (s) 160004
Current children cumulated vsize (KiB) 0

[startup+5012.2 s]
/proc/loadavg: 32.02 32.06 32.06 33/446 11516
/proc/meminfo: memFree=243496808/264257624 swapFree=67111528/67111528
[pid=11331] ppid=11329 vsize=0 CPUtime=160005 cores=0-31
/proc/11331/stat : 11331 (plingeling) Z 11329 11331 11305 0 -1 4203548 18981511 1240 0 0 15976787 23673 0 1 18 0 2 0 706231470 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 30 0 0 0
/proc/11331/statm: 0 0 0 0 0 0 0
[pid=11331/tid=11366] ppid=11329 vsize=0 CPUtime=5004.24 cores=0-31
/proc/11331/task/11366/stat : 11366 (plingeling) R 11329 11331 11305 0 -1 4203588 621194 1240 0 0 499597 826 0 1 25 0 2 0 706231829 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 30 0 0 0
Current children cumulated CPU time (s) 160005
Current children cumulated vsize (KiB) 0

[startup+5012.4 s]
/proc/loadavg: 32.02 32.06 32.06 33/446 11516
/proc/meminfo: memFree=243496808/264257624 swapFree=67111528/67111528
[pid=11331] ppid=11329 vsize=0 CPUtime=160005 cores=0-31
/proc/11331/stat : 11331 (plingeling) Z 11329 11331 11305 0 -1 4203548 18981511 1240 0 0 15976787 23693 0 1 18 0 2 0 706231470 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 30 0 0 0
/proc/11331/statm: 0 0 0 0 0 0 0
[pid=11331/tid=11366] ppid=11329 vsize=0 CPUtime=5004.44 cores=0-31
/proc/11331/task/11366/stat : 11366 (plingeling) R 11329 11331 11305 0 -1 4203588 621194 1240 0 0 499597 846 0 1 25 0 2 0 706231829 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 30 0 0 0
Current children cumulated CPU time (s) 160005
Current children cumulated vsize (KiB) 0

[startup+5012.5 s]
/proc/loadavg: 32.02 32.06 32.06 33/446 11516
/proc/meminfo: memFree=243496808/264257624 swapFree=67111528/67111528
[pid=11331] ppid=11329 vsize=0 CPUtime=160005 cores=0-31
/proc/11331/stat : 11331 (plingeling) Z 11329 11331 11305 0 -1 4203548 18981511 1240 0 0 15976787 23703 0 1 18 0 2 0 706231470 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 30 0 0 0
/proc/11331/statm: 0 0 0 0 0 0 0
[pid=11331/tid=11366] ppid=11329 vsize=0 CPUtime=5004.54 cores=0-31
/proc/11331/task/11366/stat : 11366 (plingeling) R 11329 11331 11305 0 -1 4203588 621194 1240 0 0 499597 856 0 1 25 0 2 0 706231829 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 30 0 0 0
Current children cumulated CPU time (s) 160005
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5012.54
CPU time (s): 160005
CPU user time (s): 159768
CPU system time (s): 237.079
CPU usage (%): 3192.09
Max. virtual memory (cumulated for all children) (KiB): 13131768

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 159768
system time used= 237.079
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 18982751
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 8629
involuntary context switches= 407881

runsolver used 25.1202 second user time and 116.29 second system time

The end

Launcher Data 

Begin job on node200 at 2011-05-21 05:10:51
IDJOB=3372681
IDBENCH=83114
IDSOLVER=1852
FILE ID=node200/3372681-1305947450
RUNJOBID= node200-1305947449-11319
PBS_JOBID= 13387784
Free space on /tmp= 62044 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/application/fuhs/slp-synthesis-AES/slp-synthesis-aes-top23.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3372681-1305947450/watcher-3372681-1305947450 -o /tmp/evaluation-result-3372681-1305947450/solver-3372681-1305947450 -C 160000 -W 5100 -M 266338304  plingeling -t 32 HOME/instance-3372681-1305947450.cnf

TIME LIMIT= 160000 seconds
MEMORY LIMIT= 266338304 MiB
NBCORE= 32

MD5SUM BENCH= 6ee28191d649bfeef8c7d8c569a3cd59
RANDOM SEED=1211654412

node200.alineos.net Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 64
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 96
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 66
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 98
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 68
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 100
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 70
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.98
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 102
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 16
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 17
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 48
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 18
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 80
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 19
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 112
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 20
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 21
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 50
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 22
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 82
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 114
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.23
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 24
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 25
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 26
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 84
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 27
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 116
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 28
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 29
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 54
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 30
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 86
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 31
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 118
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     264257624 kB
MemFree:      255504920 kB
Buffers:        453424 kB
Cached:        7899384 kB
SwapCached:          0 kB
Active:        7411748 kB
Inactive:      1009676 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     264257624 kB
LowFree:      255504920 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:            3764 kB
Writeback:           0 kB
AnonPages:       69476 kB
Mapped:          14452 kB
Slab:           252572 kB
PageTables:       4552 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  199240340 kB
Committed_AS:   186980 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296372 kB
VmallocChunk: 34359440603 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 62044 MiB
End job on node200 at 2011-05-21 06:34:24