Trace number 3372402

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 160003 5035.02

General information on the benchmark

Nameapplication/rintanen/SATPlanning/
sokoban-sequential-p145-microban-sequential.070-NOTKNOWN.cnf
MD5SUMf52ec4dacdcf65c3aedb61fb46ade01b
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark24553.6
Satisfiable
(Un)Satisfiability was proved
Number of variables153284
Number of clauses2473656
Sum of the clauses size5647736
Maximum clause length78
Minimum clause length1
Number of clauses of size 1696
Number of clauses of size 22387280
Number of clauses of size 36720
Number of clauses of size 431080
Number of clauses of size 513440
Number of clauses of size over 534440

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3372402-1305572749/watcher-3372402-1305572749 -o /tmp/evaluation-result-3372402-1305572749/solver-3372402-1305572749 -C 160000 -W 5100 -M 266338304 plingeling -t 32 HOME/instance-3372402-1305572749.cnf 

running on 32 cores: 0,4,8,12,16,20,24,28,1,5,9,13,17,21,25,29,2,6,10,14,18,22,26,30,3,7,11,15,19,23,27,31

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 160000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 160030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 272730423296 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 272730474496 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 14.83 24.99 29.30 1/413 21132
/proc/meminfo: memFree=255474692/264257624 swapFree=67111528/67111528
[pid=21132] ppid=21130 vsize=17232 CPUtime=0.01 cores=0-31
/proc/21132/stat : 21132 (plingeling) R 21130 21132 20910 0 -1 4202496 2376 1240 0 0 1 0 0 0 22 0 1 0 668761548 17645568 2308 268435456000 4194304 4417452 140736666960064 18446744073709551615 4231256 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/21132/statm: 4308 2308 141 55 0 2194 0

[startup+0.127761 s]
/proc/loadavg: 14.83 24.99 29.30 1/413 21132
/proc/meminfo: memFree=255474692/264257624 swapFree=67111528/67111528
[pid=21132] ppid=21130 vsize=25316 CPUtime=0.11 cores=0-31
/proc/21132/stat : 21132 (plingeling) R 21130 21132 20910 0 -1 4202496 4416 1240 0 0 10 1 0 0 22 0 1 0 668761548 25923584 4348 268435456000 4194304 4417452 140736666960064 18446744073709551615 4231321 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/21132/statm: 6329 4348 144 55 0 4215 0
Current children cumulated CPU time (s) 0.11
Current children cumulated vsize (KiB) 25316

[startup+0.200776 s]
/proc/loadavg: 14.83 24.99 29.30 1/413 21132
/proc/meminfo: memFree=255474692/264257624 swapFree=67111528/67111528
[pid=21132] ppid=21130 vsize=35304 CPUtime=0.18 cores=0-31
/proc/21132/stat : 21132 (plingeling) R 21130 21132 20910 0 -1 4202496 6915 1240 0 0 17 1 0 0 23 0 1 0 668761548 36151296 6847 268435456000 4194304 4417452 140736666960064 18446744073709551615 4237514 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/21132/statm: 8826 6847 146 55 0 6712 0
Current children cumulated CPU time (s) 0.18
Current children cumulated vsize (KiB) 35304

[startup+0.300766 s]
/proc/loadavg: 14.83 24.99 29.30 1/413 21132
/proc/meminfo: memFree=255474692/264257624 swapFree=67111528/67111528
[pid=21132] ppid=21130 vsize=35304 CPUtime=0.28 cores=0-31
/proc/21132/stat : 21132 (plingeling) R 21130 21132 20910 0 -1 4202496 6915 1240 0 0 27 1 0 0 23 0 1 0 668761548 36151296 6847 268435456000 4194304 4417452 140736666960064 18446744073709551615 4227801 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/21132/statm: 8826 6847 146 55 0 6712 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 35304

[startup+0.700785 s]
/proc/loadavg: 14.83 24.99 29.30 1/413 21132
/proc/meminfo: memFree=255474692/264257624 swapFree=67111528/67111528
[pid=21132] ppid=21130 vsize=60300 CPUtime=0.68 cores=0-31
/proc/21132/stat : 21132 (plingeling) R 21130 21132 20910 0 -1 4202496 13156 1240 0 0 66 2 0 0 25 0 1 0 668761548 61747200 13088 268435456000 4194304 4417452 140736666960064 18446744073709551615 258650955760 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/21132/statm: 15075 13088 146 55 0 12961 0
Current children cumulated CPU time (s) 0.68
Current children cumulated vsize (KiB) 60300

[startup+1.50081 s]
/proc/loadavg: 14.83 24.99 29.30 2/414 21138
/proc/meminfo: memFree=255399036/264257624 swapFree=67111528/67111528
[pid=21132] ppid=21130 vsize=117636 CPUtime=1.49 cores=0-31
/proc/21132/stat : 21132 (plingeling) R 21130 21132 20910 0 -1 4202496 27466 1240 0 0 143 6 0 0 25 0 1 0 668761548 120459264 27398 268435456000 4194304 4417452 140736666960064 18446744073709551615 258650955760 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/21132/statm: 29409 27398 146 55 0 27295 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 117636

[startup+3.10087 s]
/proc/loadavg: 14.83 24.99 29.30 2/414 21138
/proc/meminfo: memFree=255336748/264257624 swapFree=67111528/67111528
[pid=21132] ppid=21130 vsize=210460 CPUtime=3.08 cores=0-31
/proc/21132/stat : 21132 (plingeling) R 21130 21132 20910 0 -1 4202496 50500 1240 0 0 297 11 0 0 25 0 1 0 668761548 215511040 50408 268435456000 4194304 4417452 140736666960064 18446744073709551615 258650649933 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/21132/statm: 52615 50426 146 55 0 50501 0
Current children cumulated CPU time (s) 3.08
Current children cumulated vsize (KiB) 210460

[startup+6.30102 s]
/proc/loadavg: 13.72 24.60 29.14 2/414 21138
/proc/meminfo: memFree=255066504/264257624 swapFree=67111528/67111528
[pid=21132] ppid=21130 vsize=479104 CPUtime=6.28 cores=0-31
/proc/21132/stat : 21132 (plingeling) R 21130 21132 20910 0 -1 4202496 117921 1240 0 0 603 25 0 0 25 0 1 0 668761548 490602496 117797 268435456000 4194304 4417452 140736666960064 18446744073709551615 258650955760 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/21132/statm: 119776 117797 146 55 0 117662 0
Current children cumulated CPU time (s) 6.28
Current children cumulated vsize (KiB) 479104

[startup+12.7012 s]
/proc/loadavg: 12.70 24.20 28.99 2/414 21138
/proc/meminfo: memFree=254562476/264257624 swapFree=67111528/67111528
[pid=21132] ppid=21130 vsize=924864 CPUtime=12.68 cores=0-31
/proc/21132/stat : 21132 (plingeling) R 21130 21132 20910 0 -1 4202496 229361 1240 0 0 1220 48 0 0 20 0 1 0 668761548 947060736 229237 268435456000 4194304 4417452 140736666960064 18446744073709551615 4264959 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/21132/statm: 231216 229237 146 55 0 229102 0
Current children cumulated CPU time (s) 12.68
Current children cumulated vsize (KiB) 924864

[startup+25.5007 s]
/proc/loadavg: 10.11 23.07 28.54 2/414 21138
/proc/meminfo: memFree=253679980/264257624 swapFree=67111528/67111528
[pid=21132] ppid=21130 vsize=1812640 CPUtime=25.48 cores=0-31
/proc/21132/stat : 21132 (plingeling) R 21130 21132 20910 0 -1 4202496 451322 1240 0 0 2458 90 0 0 25 0 1 0 668761548 1856143360 451181 268435456000 4194304 4417452 140736666960064 18446744073709551615 4264940 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/21132/statm: 453160 451181 146 55 0 451046 0
Current children cumulated CPU time (s) 25.48
Current children cumulated vsize (KiB) 1812640

[startup+51.1024 s]
/proc/loadavg: 17.58 23.78 28.63 33/446 21170
/proc/meminfo: memFree=253618024/264257624 swapFree=67111528/67111528
[pid=21132] ppid=21130 vsize=2352020 CPUtime=791.92 cores=0-31
/proc/21132/stat : 21132 (plingeling) S 21130 21132 20910 0 -1 4202496 786393 1240 0 0 78892 300 0 0 18 0 33 0 668761548 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 15 0 0 0
/proc/21132/statm: 588005 462871 175 55 0 585891 0
[pid=21132/tid=21139] ppid=21130 vsize=2352020 CPUtime=23.94 cores=0-31
/proc/21132/task/21139/stat : 21139 (plingeling) R 21130 21132 20910 0 -1 4202560 15372 1240 0 0 2385 9 0 0 25 0 33 0 668764253 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4265473 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21132/tid=21140] ppid=21130 vsize=2352020 CPUtime=23.93 cores=0-31
/proc/21132/task/21140/stat : 21140 (plingeling) R 21130 21132 20910 0 -1 4202560 9823 1240 0 0 2388 5 0 0 25 0 33 0 668764253 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21132/tid=21141] ppid=21130 vsize=2352020 CPUtime=23.94 cores=0-31
/proc/21132/task/21141/stat : 21141 (plingeling) R 21130 21132 20910 0 -1 4202560 12877 1240 0 0 2389 5 0 0 25 0 33 0 668764253 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=21132/tid=21142] ppid=21130 vsize=2352020 CPUtime=23.95 cores=0-31
/proc/21132/task/21142/stat : 21142 (plingeling) R 21130 21132 20910 0 -1 4202560 2422 1240 0 0 2394 1 0 0 25 0 33 0 668764253 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4237169 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=21132/tid=21143] ppid=21130 vsize=2352020 CPUtime=23.94 cores=0-31
/proc/21132/task/21143/stat : 21143 (plingeling) R 21130 21132 20910 0 -1 4202560 8241 1240 0 0 2392 2 0 0 25 0 33 0 668764253 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=21132/tid=21144] ppid=21130 vsize=2352020 CPUtime=23.94 cores=0-31
/proc/21132/task/21144/stat : 21144 (plingeling) R 21130 21132 20910 0 -1 4202560 7480 1240 0 0 2388 6 0 0 25 0 33 0 668764254 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=21132/tid=21145] ppid=21130 vsize=2352020 CPUtime=23.93 cores=0-31
/proc/21132/task/21145/stat : 21145 (plingeling) R 21130 21132 20910 0 -1 4202560 3452 1240 0 0 2389 4 0 0 25 0 33 0 668764254 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=21132/tid=21146] ppid=21130 vsize=2352020 CPUtime=23.92 cores=0-31
/proc/21132/task/21146/stat : 21146 (plingeling) R 21130 21132 20910 0 -1 4202560 14279 1240 0 0 2385 7 0 0 25 0 33 0 668764254 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4353598 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=21132/tid=21147] ppid=21130 vsize=2352020 CPUtime=23.94 cores=0-31
/proc/21132/task/21147/stat : 21147 (plingeling) R 21130 21132 20910 0 -1 4202560 13748 1240 0 0 2386 8 0 0 25 0 33 0 668764254 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=21132/tid=21148] ppid=21130 vsize=2352020 CPUtime=23.93 cores=0-31
/proc/21132/task/21148/stat : 21148 (plingeling) R 21130 21132 20910 0 -1 4202560 4790 1240 0 0 2387 6 0 0 25 0 33 0 668764254 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21132/tid=21149] ppid=21130 vsize=2352020 CPUtime=23.92 cores=0-31
/proc/21132/task/21149/stat : 21149 (plingeling) R 21130 21132 20910 0 -1 4202560 11693 1240 0 0 2384 8 0 0 25 0 33 0 668764255 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4236888 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=21132/tid=21150] ppid=21130 vsize=2352020 CPUtime=23.9 cores=0-31
/proc/21132/task/21150/stat : 21150 (plingeling) R 21130 21132 20910 0 -1 4202560 14918 1240 0 0 2384 6 0 0 25 0 33 0 668764255 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4232617 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=21132/tid=21151] ppid=21130 vsize=2352020 CPUtime=23.9 cores=0-31
/proc/21132/task/21151/stat : 21151 (plingeling) R 21130 21132 20910 0 -1 4202560 16230 1240 0 0 2381 9 0 0 25 0 33 0 668764255 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4354497 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=21132/tid=21152] ppid=21130 vsize=2352020 CPUtime=23.92 cores=0-31
/proc/21132/task/21152/stat : 21152 (plingeling) R 21130 21132 20910 0 -1 4202560 11502 1240 0 0 2386 6 0 0 25 0 33 0 668764255 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4232791 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=21132/tid=21153] ppid=21130 vsize=2352020 CPUtime=23.91 cores=0-31
/proc/21132/task/21153/stat : 21153 (plingeling) R 21130 21132 20910 0 -1 4202560 13674 1240 0 0 2383 8 0 0 25 0 33 0 668764256 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=21132/tid=21154] ppid=21130 vsize=2352020 CPUtime=23.92 cores=0-31
/proc/21132/task/21154/stat : 21154 (plingeling) R 21130 21132 20910 0 -1 4202560 7484 1240 0 0 2385 7 0 0 19 0 33 0 668764256 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=21132/tid=21155] ppid=21130 vsize=2352020 CPUtime=23.39 cores=0-31
/proc/21132/task/21155/stat : 21155 (plingeling) R 21130 21132 20910 0 -1 4202560 13116 1240 0 0 2331 8 0 0 25 0 33 0 668764256 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=21132/tid=21156] ppid=21130 vsize=2352020 CPUtime=23.9 cores=0-31
/proc/21132/task/21156/stat : 21156 (plingeling) R 21130 21132 20910 0 -1 4202560 12595 1240 0 0 2384 6 0 0 25 0 33 0 668764256 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4364587 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=21132/tid=21157] ppid=21130 vsize=2352020 CPUtime=23.93 cores=0-31
/proc/21132/task/21157/stat : 21157 (plingeling) R 21130 21132 20910 0 -1 4202560 4916 1240 0 0 2387 6 0 0 25 0 33 0 668764256 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=21132/tid=21158] ppid=21130 vsize=2352020 CPUtime=23.91 cores=0-31
/proc/21132/task/21158/stat : 21158 (plingeling) R 21130 21132 20910 0 -1 4202560 8138 1240 0 0 2385 6 0 0 25 0 33 0 668764257 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=21132/tid=21159] ppid=21130 vsize=2352020 CPUtime=23.88 cores=0-31
/proc/21132/task/21159/stat : 21159 (plingeling) R 21130 21132 20910 0 -1 4202560 16946 1240 0 0 2381 7 0 0 25 0 33 0 668764257 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4357856 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=21132/tid=21160] ppid=21130 vsize=2352020 CPUtime=23.84 cores=0-31
/proc/21132/task/21160/stat : 21160 (plingeling) R 21130 21132 20910 0 -1 4202560 12162 1240 0 0 2377 7 0 0 25 0 33 0 668764257 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21132/tid=21161] ppid=21130 vsize=2352020 CPUtime=23.94 cores=0-31
/proc/21132/task/21161/stat : 21161 (plingeling) R 21130 21132 20910 0 -1 4202560 15266 1240 0 0 2385 9 0 0 25 0 33 0 668764257 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=21132/tid=21162] ppid=21130 vsize=2352020 CPUtime=23.91 cores=0-31
/proc/21132/task/21162/stat : 21162 (plingeling) R 21130 21132 20910 0 -1 4202560 7578 1240 0 0 2386 5 0 0 25 0 33 0 668764258 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4232654 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=21132/tid=21163] ppid=21130 vsize=2352020 CPUtime=23.72 cores=0-31
/proc/21132/task/21163/stat : 21163 (plingeling) R 21130 21132 20910 0 -1 4202560 3223 1240 0 0 2371 1 0 0 25 0 33 0 668764258 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=21132/tid=21164] ppid=21130 vsize=2352020 CPUtime=23.91 cores=0-31
/proc/21132/task/21164/stat : 21164 (plingeling) R 21130 21132 20910 0 -1 4202560 3278 1240 0 0 2386 5 0 0 25 0 33 0 668764259 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=21132/tid=21165] ppid=21130 vsize=2352020 CPUtime=23.93 cores=0-31
/proc/21132/task/21165/stat : 21165 (plingeling) R 21130 21132 20910 0 -1 4202560 16450 1240 0 0 2384 9 0 0 25 0 33 0 668764259 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=21132/tid=21166] ppid=21130 vsize=2352020 CPUtime=23.94 cores=0-31
/proc/21132/task/21166/stat : 21166 (plingeling) R 21130 21132 20910 0 -1 4202560 16077 1240 0 0 2386 8 0 0 25 0 33 0 668764259 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21132/tid=21167] ppid=21130 vsize=2352020 CPUtime=23.87 cores=0-31
/proc/21132/task/21167/stat : 21167 (plingeling) R 21130 21132 20910 0 -1 4202560 8709 1240 0 0 2382 5 0 0 25 0 33 0 668764259 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21132/tid=21168] ppid=21130 vsize=2352020 CPUtime=23.78 cores=0-31
/proc/21132/task/21168/stat : 21168 (plingeling) R 21130 21132 20910 0 -1 4202560 13766 1240 0 0 2374 4 0 0 25 0 33 0 668764260 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21132/tid=21169] ppid=21130 vsize=2352020 CPUtime=23.93 cores=0-31
/proc/21132/task/21169/stat : 21169 (plingeling) R 21130 21132 20910 0 -1 4202560 11420 1240 0 0 2388 5 0 0 25 0 33 0 668764260 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=21132/tid=21170] ppid=21130 vsize=2352020 CPUtime=23.94 cores=0-31
/proc/21132/task/21170/stat : 21170 (plingeling) R 21130 21132 20910 0 -1 4202560 3408 1240 0 0 2391 3 0 0 25 0 33 0 668764260 2408468480 462871 268435456000 4194304 4417452 140736666960064 18446744073709551615 4232693 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 791.92
Current children cumulated vsize (KiB) 2352020

[startup+102.302 s]
/proc/loadavg: 26.28 25.18 28.83 33/446 21172
/proc/meminfo: memFree=253548128/264257624 swapFree=67111528/67111528
[pid=21132] ppid=21130 vsize=2443208 CPUtime=2429.12 cores=0-31
/proc/21132/stat : 21132 (plingeling) S 21130 21132 20910 0 -1 4202496 813455 1240 0 0 242604 308 0 0 18 0 33 0 668761548 2501844992 486130 268435456000 4194304 4417452 140736666960064 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 15 0 0 0
/proc/21132/statm: 610802 486130 175 55 0 608688 0
[pid=21132/tid=21139] ppid=21130 vsize=2443208 CPUtime=75.15 cores=0-31
/proc/21132/task/21139/stat : 21139 (plingeling) R 21130 21132 20910 0 -1 4202560 16081 1240 0 0 7505 10 0 0 25 0 33 0 668764253 2501844992 486130 268435456000 4194304 4417452 140736666960064 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21132/tid=21140] ppid=21130 vsize=2443208 CPUtime=75.14 cores=0-31
/proc/21132/task/21140/stat : 21140 (plingeling) R 21130 21132 20910 0 -1 4202560 9951 1240 0 0 7509 5 0 0 25 0 33 0 668764253 2501844992 486130 268435456000 4194304 4417452 140736666960064 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21132/tid=21141] ppid=21130 vsize=2443208 CPUtime=75.15 cores=0-31
/proc/21132/task/21141/stat : 21141 (plingeling) R 21130 21132 20910 0 -1 4202560 12877 1240 0 0 7510 5 0 0 25 0 33 0 668764253 2501844992 486130 268435456000 4194304 4417452 140736666960064 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=21132/tid=21142] ppid=21130 vsize=2443208 CPUtime=75.16 cores=0-31
/proc/21132/task/21142/stat : 21142 (plingeling) R 21130 21132 20910 0 -1 4202560 7438 1240 0 0 7514 2 0 0 25 0 33 0 668764253 2501844992 486130 268435456000 4194304 4417452 140736666960064 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=21132/tid=21143] ppid=21130 vsize=2443208 CPUtime=75.15 cores=0-31
/proc/21132/task/21143/stat : 21143 (plingeling) R 21130 21132 20910 0 -1 4202560 9220 1240 0 0 7513 2 0 0 25 0 33 0 668764253 2501844992 486130 268435456000 4194304 4417452 140736666960064 18446744073709551615 4236971 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=21132/tid=21144] ppid=21130 vsize=2443208 CPUtime=75.15 cores=0-31
/proc/21132/task/21144/stat : 21144 (plingeling) R 21130 21132 20910 0 -1 4202560 7589 1240 0 0 7509 6 0 0 25 0 33 0 668764254 2501844992 486130 268435456000 4194304 4417452 140736666960064 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 8 0 0 0

################
# More data... #
################

[pid=21132/tid=21150] ppid=21130 vsize=12186292 CPUtime=4986.98 cores=0-31
/proc/21132/task/21150/stat : 21150 (plingeling) R 21130 21132 20910 0 -1 4202560 671764 1240 0 0 498127 571 0 0 25 0 33 0 668764255 12478763008 2882620 268435456000 4194304 4417452 140736666960064 18446744073709551615 4358116 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=21132/tid=21151] ppid=21130 vsize=12186292 CPUtime=4991.65 cores=0-31
/proc/21132/task/21151/stat : 21151 (plingeling) R 21130 21132 20910 0 -1 4202560 574646 1240 0 0 498691 474 0 0 25 0 33 0 668764255 12478763008 2882620 268435456000 4194304 4417452 140736666960064 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=21132/tid=21152] ppid=21130 vsize=12186292 CPUtime=4990.93 cores=0-31
/proc/21132/task/21152/stat : 21152 (plingeling) R 21130 21132 20910 0 -1 4202560 662919 1240 0 0 498527 566 0 0 25 0 33 0 668764255 12478763008 2882620 268435456000 4194304 4417452 140736666960064 18446744073709551615 4354481 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21132/tid=21153] ppid=21130 vsize=12186292 CPUtime=4992.07 cores=0-31
/proc/21132/task/21153/stat : 21153 (plingeling) R 21130 21132 20910 0 -1 4202560 559181 1240 0 0 498824 383 0 0 25 0 33 0 668764256 12478763008 2882620 268435456000 4194304 4417452 140736666960064 18446744073709551615 4357410 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=21132/tid=21154] ppid=21130 vsize=12186292 CPUtime=4990.72 cores=0-31
/proc/21132/task/21154/stat : 21154 (plingeling) R 21130 21132 20910 0 -1 4202560 779669 1240 0 0 498275 797 0 0 25 0 33 0 668764256 12478763008 2882620 268435456000 4194304 4417452 140736666960064 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=21132/tid=21155] ppid=21130 vsize=12186292 CPUtime=4973.18 cores=0-31
/proc/21132/task/21155/stat : 21155 (plingeling) R 21130 21132 20910 0 -1 4202560 513990 1240 0 0 496884 434 0 0 25 0 33 0 668764256 12478763008 2882620 268435456000 4194304 4417452 140736666960064 18446744073709551615 4366688 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=21132/tid=21156] ppid=21130 vsize=12186292 CPUtime=4992.81 cores=0-31
/proc/21132/task/21156/stat : 21156 (plingeling) R 21130 21132 20910 0 -1 4202560 522058 1240 0 0 498955 326 0 0 25 0 33 0 668764256 12478763008 2882620 268435456000 4194304 4417452 140736666960064 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=21132/tid=21157] ppid=21130 vsize=12186292 CPUtime=4992.33 cores=0-31
/proc/21132/task/21157/stat : 21157 (plingeling) R 21130 21132 20910 0 -1 4202560 509161 1240 0 0 498820 413 0 0 25 0 33 0 668764256 12478763008 2882620 268435456000 4194304 4417452 140736666960064 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=21132/tid=21158] ppid=21130 vsize=12186292 CPUtime=4991.32 cores=0-31
/proc/21132/task/21158/stat : 21158 (plingeling) R 21130 21132 20910 0 -1 4202560 384664 1240 0 0 498899 233 0 0 25 0 33 0 668764257 12478763008 2882620 268435456000 4194304 4417452 140736666960064 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=21132/tid=21159] ppid=21130 vsize=12186292 CPUtime=4983.22 cores=0-31
/proc/21132/task/21159/stat : 21159 (plingeling) R 21130 21132 20910 0 -1 4202560 526582 1240 0 0 497951 371 0 0 25 0 33 0 668764257 12478763008 2882620 268435456000 4194304 4417452 140736666960064 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=21132/tid=21160] ppid=21130 vsize=12186292 CPUtime=4985.45 cores=0-31
/proc/21132/task/21160/stat : 21160 (plingeling) R 21130 21132 20910 0 -1 4202560 1528062 1240 0 0 497756 789 0 0 25 0 33 0 668764257 12478763008 2882620 268435456000 4194304 4417452 140736666960064 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=21132/tid=21161] ppid=21130 vsize=12186292 CPUtime=4989.91 cores=0-31
/proc/21132/task/21161/stat : 21161 (plingeling) R 21130 21132 20910 0 -1 4202560 653781 1240 0 0 498396 595 0 0 25 0 33 0 668764257 12478763008 2882620 268435456000 4194304 4417452 140736666960064 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=21132/tid=21162] ppid=21130 vsize=12186292 CPUtime=4988.92 cores=0-31
/proc/21132/task/21162/stat : 21162 (plingeling) R 21130 21132 20910 0 -1 4202560 482971 1240 0 0 498450 442 0 0 25 0 33 0 668764258 12478763008 2882620 268435456000 4194304 4417452 140736666960064 18446744073709551615 4357970 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=21132/tid=21163] ppid=21130 vsize=12186292 CPUtime=4991.72 cores=0-31
/proc/21132/task/21163/stat : 21163 (plingeling) R 21130 21132 20910 0 -1 4202560 529261 1240 0 0 498812 360 0 0 25 0 33 0 668764258 12478763008 2882620 268435456000 4194304 4417452 140736666960064 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21132/tid=21164] ppid=21130 vsize=12186292 CPUtime=4978.3 cores=0-31
/proc/21132/task/21164/stat : 21164 (plingeling) R 21130 21132 20910 0 -1 4202560 242679 1240 0 0 497701 129 0 0 25 0 33 0 668764259 12478763008 2882620 268435456000 4194304 4417452 140736666960064 18446744073709551615 4358116 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=21132/tid=21165] ppid=21130 vsize=12186292 CPUtime=4989.05 cores=0-31
/proc/21132/task/21165/stat : 21165 (plingeling) R 21130 21132 20910 0 -1 4202560 682959 1240 0 0 498288 617 0 0 25 0 33 0 668764259 12478763008 2882620 268435456000 4194304 4417452 140736666960064 18446744073709551615 4357283 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=21132/tid=21166] ppid=21130 vsize=12186292 CPUtime=4993.31 cores=0-31
/proc/21132/task/21166/stat : 21166 (plingeling) R 21130 21132 20910 0 -1 4202560 497032 1240 0 0 498975 356 0 0 25 0 33 0 668764259 12478763008 2882620 268435456000 4194304 4417452 140736666960064 18446744073709551615 4232787 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21132/tid=21167] ppid=21130 vsize=12186292 CPUtime=4982.14 cores=0-31
/proc/21132/task/21167/stat : 21167 (plingeling) R 21130 21132 20910 0 -1 4202560 571883 1240 0 0 497758 456 0 0 25 0 33 0 668764259 12478763008 2882620 268435456000 4194304 4417452 140736666960064 18446744073709551615 4357207 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21132/tid=21168] ppid=21130 vsize=12186292 CPUtime=4990.71 cores=0-31
/proc/21132/task/21168/stat : 21168 (plingeling) R 21130 21132 20910 0 -1 4202560 490585 1240 0 0 498612 459 0 0 25 0 33 0 668764260 12478763008 2882620 268435456000 4194304 4417452 140736666960064 18446744073709551615 4265373 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21132/tid=21169] ppid=21130 vsize=12186292 CPUtime=4990.41 cores=0-31
/proc/21132/task/21169/stat : 21169 (plingeling) R 21130 21132 20910 0 -1 4202560 604947 1240 0 0 498712 329 0 0 25 0 33 0 668764260 12478763008 2882620 268435456000 4194304 4417452 140736666960064 18446744073709551615 4357207 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=21132/tid=21170] ppid=21130 vsize=12186292 CPUtime=4982.99 cores=0-31
/proc/21132/task/21170/stat : 21170 (plingeling) R 21130 21132 20910 0 -1 4202560 528634 1240 0 0 497817 482 0 0 25 0 33 0 668764260 12478763008 2882620 268435456000 4194304 4417452 140736666960064 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 26 0 0 0
Current children cumulated CPU time (s) 159625
Current children cumulated vsize (KiB) 12186292



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5034.1 s]
/proc/loadavg: 32.12 32.19 32.11 33/446 21319
/proc/meminfo: memFree=243871604/264257624 swapFree=67111528/67111528
[pid=21132] ppid=21130 vsize=12258900 CPUtime=160002 cores=0-31
/proc/21132/stat : 21132 (plingeling) S 21130 21132 20910 0 -1 4202496 21567746 1240 0 0 15982576 17646 0 0 18 0 33 0 668761548 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 15 0 0 0
/proc/21132/statm: 3064725 2900773 175 55 0 3062611 0
[pid=21132/tid=21139] ppid=21130 vsize=12258900 CPUtime=5002.43 cores=0-31
/proc/21132/task/21139/stat : 21139 (plingeling) R 21130 21132 20910 0 -1 4202560 831114 1240 0 0 499439 804 0 0 25 0 33 0 668764253 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4264940 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21132/tid=21140] ppid=21130 vsize=12258900 CPUtime=4988.01 cores=0-31
/proc/21132/task/21140/stat : 21140 (plingeling) R 21130 21132 20910 0 -1 4202560 575247 1240 0 0 498335 466 0 0 25 0 33 0 668764253 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4320859 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=21132/tid=21141] ppid=21130 vsize=12258900 CPUtime=5002.41 cores=0-31
/proc/21132/task/21141/stat : 21141 (plingeling) R 21130 21132 20910 0 -1 4202560 684133 1240 0 0 499726 515 0 0 25 0 33 0 668764253 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4302950 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=21132/tid=21142] ppid=21130 vsize=12258900 CPUtime=4993.29 cores=0-31
/proc/21132/task/21142/stat : 21142 (plingeling) R 21130 21132 20910 0 -1 4202560 1956935 1240 0 0 497207 2122 0 0 25 0 33 0 668764253 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4353636 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=21132/tid=21143] ppid=21130 vsize=12258900 CPUtime=5002.68 cores=0-31
/proc/21132/task/21143/stat : 21143 (plingeling) R 21130 21132 20910 0 -1 4202560 583563 1240 0 0 499855 413 0 0 25 0 33 0 668764253 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4232801 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=21132/tid=21144] ppid=21130 vsize=12258900 CPUtime=4999 cores=0-31
/proc/21132/task/21144/stat : 21144 (plingeling) R 21130 21132 20910 0 -1 4202560 626452 1240 0 0 499265 635 0 0 25 0 33 0 668764254 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=21132/tid=21145] ppid=21130 vsize=12258900 CPUtime=4994.94 cores=0-31
/proc/21132/task/21145/stat : 21145 (plingeling) R 21130 21132 20910 0 -1 4202560 1033224 1240 0 0 498336 1158 0 0 25 0 33 0 668764254 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4232577 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=21132/tid=21146] ppid=21130 vsize=12258900 CPUtime=4994.06 cores=0-31
/proc/21132/task/21146/stat : 21146 (plingeling) R 21130 21132 20910 0 -1 4202560 421344 1240 0 0 499045 361 0 0 25 0 33 0 668764254 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4353531 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=21132/tid=21147] ppid=21130 vsize=12258900 CPUtime=4998.31 cores=0-31
/proc/21132/task/21147/stat : 21147 (plingeling) R 21130 21132 20910 0 -1 4202560 515046 1240 0 0 499376 455 0 0 25 0 33 0 668764254 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4365996 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=21132/tid=21148] ppid=21130 vsize=12258900 CPUtime=5001.87 cores=0-31
/proc/21132/task/21148/stat : 21148 (plingeling) R 21130 21132 20910 0 -1 4202560 620435 1240 0 0 499657 530 0 0 25 0 33 0 668764254 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=21132/tid=21149] ppid=21130 vsize=12258900 CPUtime=5002.03 cores=0-31
/proc/21132/task/21149/stat : 21149 (plingeling) R 21130 21132 20910 0 -1 4202560 673518 1240 0 0 499797 406 0 0 25 0 33 0 668764255 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21132/tid=21150] ppid=21130 vsize=12258900 CPUtime=4998.8 cores=0-31
/proc/21132/task/21150/stat : 21150 (plingeling) R 21130 21132 20910 0 -1 4202560 671764 1240 0 0 499309 571 0 0 25 0 33 0 668764255 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4354515 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=21132/tid=21151] ppid=21130 vsize=12258900 CPUtime=5003.41 cores=0-31
/proc/21132/task/21151/stat : 21151 (plingeling) R 21130 21132 20910 0 -1 4202560 580175 1240 0 0 499853 488 0 0 25 0 33 0 668764255 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=21132/tid=21152] ppid=21130 vsize=12258900 CPUtime=5002.74 cores=0-31
/proc/21132/task/21152/stat : 21152 (plingeling) R 21130 21132 20910 0 -1 4202560 663591 1240 0 0 499708 566 0 0 25 0 33 0 668764255 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4354332 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=21132/tid=21153] ppid=21130 vsize=12258900 CPUtime=5003.89 cores=0-31
/proc/21132/task/21153/stat : 21153 (plingeling) R 21130 21132 20910 0 -1 4202560 559181 1240 0 0 500006 383 0 0 25 0 33 0 668764256 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4321266 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=21132/tid=21154] ppid=21130 vsize=12258900 CPUtime=5002.54 cores=0-31
/proc/21132/task/21154/stat : 21154 (plingeling) R 21130 21132 20910 0 -1 4202560 779669 1240 0 0 499457 797 0 0 25 0 33 0 668764256 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=21132/tid=21155] ppid=21130 vsize=12258900 CPUtime=4985 cores=0-31
/proc/21132/task/21155/stat : 21155 (plingeling) R 21130 21132 20910 0 -1 4202560 515076 1240 0 0 498066 434 0 0 25 0 33 0 668764256 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4357554 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=21132/tid=21156] ppid=21130 vsize=12258900 CPUtime=5004.63 cores=0-31
/proc/21132/task/21156/stat : 21156 (plingeling) R 21130 21132 20910 0 -1 4202560 522058 1240 0 0 500137 326 0 0 25 0 33 0 668764256 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4366491 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=21132/tid=21157] ppid=21130 vsize=12258900 CPUtime=5004.14 cores=0-31
/proc/21132/task/21157/stat : 21157 (plingeling) R 21130 21132 20910 0 -1 4202560 513577 1240 0 0 499989 425 0 0 25 0 33 0 668764256 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=21132/tid=21158] ppid=21130 vsize=12258900 CPUtime=5003.14 cores=0-31
/proc/21132/task/21158/stat : 21158 (plingeling) R 21130 21132 20910 0 -1 4202560 384664 1240 0 0 500081 233 0 0 25 0 33 0 668764257 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=21132/tid=21159] ppid=21130 vsize=12258900 CPUtime=4995.04 cores=0-31
/proc/21132/task/21159/stat : 21159 (plingeling) R 21130 21132 20910 0 -1 4202560 527521 1240 0 0 499133 371 0 0 25 0 33 0 668764257 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4302145 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=21132/tid=21160] ppid=21130 vsize=12258900 CPUtime=4997.27 cores=0-31
/proc/21132/task/21160/stat : 21160 (plingeling) R 21130 21132 20910 0 -1 4202560 1528062 1240 0 0 498938 789 0 0 25 0 33 0 668764257 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4232584 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=21132/tid=21161] ppid=21130 vsize=12258900 CPUtime=5001.74 cores=0-31
/proc/21132/task/21161/stat : 21161 (plingeling) R 21130 21132 20910 0 -1 4202560 653781 1240 0 0 499578 596 0 0 25 0 33 0 668764257 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=21132/tid=21162] ppid=21130 vsize=12258900 CPUtime=5000.73 cores=0-31
/proc/21132/task/21162/stat : 21162 (plingeling) R 21130 21132 20910 0 -1 4202560 483243 1240 0 0 499631 442 0 0 25 0 33 0 668764258 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4236891 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=21132/tid=21163] ppid=21130 vsize=12258900 CPUtime=5003.53 cores=0-31
/proc/21132/task/21163/stat : 21163 (plingeling) R 21130 21132 20910 0 -1 4202560 532318 1240 0 0 499993 360 0 0 25 0 33 0 668764258 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4356995 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21132/tid=21164] ppid=21130 vsize=12258900 CPUtime=4990.12 cores=0-31
/proc/21132/task/21164/stat : 21164 (plingeling) R 21130 21132 20910 0 -1 4202560 242679 1240 0 0 498883 129 0 0 25 0 33 0 668764259 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4358748 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=21132/tid=21165] ppid=21130 vsize=12258900 CPUtime=5000.87 cores=0-31
/proc/21132/task/21165/stat : 21165 (plingeling) R 21130 21132 20910 0 -1 4202560 684846 1240 0 0 499470 617 0 0 25 0 33 0 668764259 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4236735 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=21132/tid=21166] ppid=21130 vsize=12258900 CPUtime=5005.13 cores=0-31
/proc/21132/task/21166/stat : 21166 (plingeling) R 21130 21132 20910 0 -1 4202560 497032 1240 0 0 500157 356 0 0 25 0 33 0 668764259 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4263216 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=21132/tid=21167] ppid=21130 vsize=12258900 CPUtime=4993.96 cores=0-31
/proc/21132/task/21167/stat : 21167 (plingeling) R 21130 21132 20910 0 -1 4202560 572775 1240 0 0 498940 456 0 0 25 0 33 0 668764259 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4357283 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=21132/tid=21168] ppid=21130 vsize=12258900 CPUtime=5002.53 cores=0-31
/proc/21132/task/21168/stat : 21168 (plingeling) R 21130 21132 20910 0 -1 4202560 540070 1240 0 0 499731 522 0 0 25 0 33 0 668764260 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4353557 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=21132/tid=21169] ppid=21130 vsize=12258900 CPUtime=5002.16 cores=0-31
/proc/21132/task/21169/stat : 21169 (plingeling) R 21130 21132 20910 0 -1 4202560 605026 1240 0 0 499887 329 0 0 25 0 33 0 668764260 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4356995 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=21132/tid=21170] ppid=21130 vsize=12258900 CPUtime=4994.5 cores=0-31
/proc/21132/task/21170/stat : 21170 (plingeling) R 21130 21132 20910 0 -1 4202560 538267 1240 0 0 498964 486 0 0 25 0 33 0 668764260 12553113600 2900773 268435456000 4194304 4417452 140736666960064 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 26 0 0 0
Current children cumulated CPU time (s) 160002
Current children cumulated vsize (KiB) 12258900

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5034.3 s]
/proc/loadavg: 32.12 32.19 32.11 33/446 21319
/proc/meminfo: memFree=243871604/264257624 swapFree=67111528/67111528
[pid=21132] ppid=21130 vsize=0 CPUtime=160003 cores=0-31
/proc/21132/stat : 21132 (plingeling) Z 21130 21132 20910 0 -1 4203548 21567746 1240 0 0 15982598 17663 0 0 18 0 2 0 668761548 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 15 0 0 0
/proc/21132/statm: 0 0 0 0 0 0 0
[pid=21132/tid=21139] ppid=21130 vsize=0 CPUtime=5002.61 cores=0-31
/proc/21132/task/21139/stat : 21139 (plingeling) R 21130 21132 20910 0 -1 4203588 831114 1240 0 0 499440 821 0 0 25 0 2 0 668764253 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 160003
Current children cumulated vsize (KiB) 0

[startup+5034.7 s]
/proc/loadavg: 32.12 32.19 32.11 2/416 21320
/proc/meminfo: memFree=250946808/264257624 swapFree=67111528/67111528
[pid=21132] ppid=21130 vsize=0 CPUtime=160003 cores=0-31
/proc/21132/stat : 21132 (plingeling) Z 21130 21132 20910 0 -1 4203548 21567746 1240 0 0 15982598 17706 0 0 18 0 2 0 668761548 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 15 0 0 0
/proc/21132/statm: 0 0 0 0 0 0 0
[pid=21132/tid=21139] ppid=21130 vsize=0 CPUtime=5003.05 cores=0-31
/proc/21132/task/21139/stat : 21139 (plingeling) R 21130 21132 20910 0 -1 4203588 831114 1240 0 0 499440 865 0 0 25 0 2 0 668764253 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 160003
Current children cumulated vsize (KiB) 0

[startup+5034.9 s]
/proc/loadavg: 32.12 32.19 32.11 2/416 21320
/proc/meminfo: memFree=250946808/264257624 swapFree=67111528/67111528
[pid=21132] ppid=21130 vsize=0 CPUtime=160003 cores=0-31
/proc/21132/stat : 21132 (plingeling) Z 21130 21132 20910 0 -1 4203548 21567746 1240 0 0 15982598 17723 0 0 18 0 2 0 668761548 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 15 0 0 0
/proc/21132/statm: 0 0 0 0 0 0 0
[pid=21132/tid=21139] ppid=21130 vsize=0 CPUtime=5003.21 cores=0-31
/proc/21132/task/21139/stat : 21139 (plingeling) R 21130 21132 20910 0 -1 4203588 831114 1240 0 0 499440 881 0 0 25 0 2 0 668764253 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 160003
Current children cumulated vsize (KiB) 0

[startup+5035 s]
/proc/loadavg: 32.12 32.19 32.11 2/416 21320
/proc/meminfo: memFree=250946808/264257624 swapFree=67111528/67111528
[pid=21132] ppid=21130 vsize=0 CPUtime=160003 cores=0-31
/proc/21132/stat : 21132 (plingeling) Z 21130 21132 20910 0 -1 4203548 21567746 1240 0 0 15982598 17733 0 0 18 0 2 0 668761548 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 15 0 0 0
/proc/21132/statm: 0 0 0 0 0 0 0
[pid=21132/tid=21139] ppid=21130 vsize=0 CPUtime=5003.31 cores=0-31
/proc/21132/task/21139/stat : 21139 (plingeling) R 21130 21132 20910 0 -1 4203588 831114 1240 0 0 499440 891 0 0 25 0 2 0 668764253 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 160003
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5035.02
CPU time (s): 160003
CPU user time (s): 159826
CPU system time (s): 177.361
CPU usage (%): 3177.81
Max. virtual memory (cumulated for all children) (KiB): 13295380

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 159826
system time used= 177.361
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 21568986
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 14650
involuntary context switches= 417933

runsolver used 26.6759 second user time and 115.784 second system time

The end

Launcher Data 

Begin job on node200 at 2011-05-16 21:05:51
IDJOB=3372402
IDBENCH=82363
IDSOLVER=1852
FILE ID=node200/3372402-1305572749
RUNJOBID= node200-1305572633-20924
PBS_JOBID= 13285553
Free space on /tmp= 62008 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/application/rintanen/SATPlanning/sokoban-sequential-p145-microban-sequential.070-NOTKNOWN.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3372402-1305572749/watcher-3372402-1305572749 -o /tmp/evaluation-result-3372402-1305572749/solver-3372402-1305572749 -C 160000 -W 5100 -M 266338304  plingeling -t 32 HOME/instance-3372402-1305572749.cnf

TIME LIMIT= 160000 seconds
MEMORY LIMIT= 266338304 MiB
NBCORE= 32

MD5SUM BENCH= f52ec4dacdcf65c3aedb61fb46ade01b
RANDOM SEED=1184326852

node200.alineos.net Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 64
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 96
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 66
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 98
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 68
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 100
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 70
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.98
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 102
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 16
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 17
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 48
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 18
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 80
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 19
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 112
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 20
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 21
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 50
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 22
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 82
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 114
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.23
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 24
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 25
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 26
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 84
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 27
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 116
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 28
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 29
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 54
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 30
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 86
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 31
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 118
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     264257624 kB
MemFree:      255474856 kB
Buffers:        446172 kB
Cached:        7914044 kB
SwapCached:          0 kB
Active:        7438996 kB
Inactive:       994832 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     264257624 kB
LowFree:      255474856 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           43576 kB
Writeback:           0 kB
AnonPages:       73800 kB
Mapped:          14536 kB
Slab:           252624 kB
PageTables:       4504 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  199240340 kB
Committed_AS:   198772 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296372 kB
VmallocChunk: 34359440603 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 62008 MiB
End job on node200 at 2011-05-16 22:29:48