Trace number 3372441

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 160006 5031.9

General information on the benchmark

Nameapplication/rintanen/SATPlanning/
openstacks-sequencedstrips-nonadl-nonnegated-os-sequencedstrips-p30_3.025-NOTKNOWN.cnf
MD5SUM3c5c73741ba21d5f39c7f436ba834743
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark56366.2
Satisfiable
(Un)Satisfiability was proved
Number of variables95456
Number of clauses483561
Sum of the clauses size1149911
Maximum clause length63
Minimum clause length1
Number of clauses of size 1211
Number of clauses of size 2477300
Number of clauses of size 31500
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 54550

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3372441-1305612533/watcher-3372441-1305612533 -o /tmp/evaluation-result-3372441-1305612533/solver-3372441-1305612533 -C 160000 -W 5100 -M 266338304 plingeling -t 32 HOME/instance-3372441-1305612533.cnf 

running on 32 cores: 0,4,8,12,16,20,24,28,1,5,9,13,17,21,25,29,2,6,10,14,18,22,26,30,3,7,11,15,19,23,27,31

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 160000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 160030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 272730423296 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 272730474496 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 27.18 31.02 31.68 1/413 23832
/proc/meminfo: memFree=255526052/264257624 swapFree=67111528/67111528
[pid=23832] ppid=23830 vsize=14336 CPUtime=0.03 cores=0-31
/proc/23832/stat : 23832 (plingeling) R 23830 23832 23806 0 -1 4202496 1661 1242 0 0 2 0 0 1 18 0 1 0 672739766 14680064 1593 268435456000 4194304 4417452 140734366480928 18446744073709551615 4231477 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/23832/statm: 3584 1593 141 55 0 1470 0

[startup+0.111293 s]
/proc/loadavg: 27.18 31.02 31.68 1/413 23832
/proc/meminfo: memFree=255526052/264257624 swapFree=67111528/67111528
[pid=23832] ppid=23830 vsize=17280 CPUtime=0.1 cores=0-31
/proc/23832/stat : 23832 (plingeling) R 23830 23832 23806 0 -1 4202496 2372 1242 0 0 9 0 0 1 18 0 1 0 672739766 17694720 2304 268435456000 4194304 4417452 140734366480928 18446744073709551615 4235997 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/23832/statm: 4320 2304 141 55 0 2206 0
Current children cumulated CPU time (s) 0.1
Current children cumulated vsize (KiB) 17280

[startup+0.201305 s]
/proc/loadavg: 27.18 31.02 31.68 1/413 23832
/proc/meminfo: memFree=255526052/264257624 swapFree=67111528/67111528
[pid=23832] ppid=23830 vsize=22468 CPUtime=0.2 cores=0-31
/proc/23832/stat : 23832 (plingeling) R 23830 23832 23806 0 -1 4202496 3690 1242 0 0 18 1 0 1 18 0 1 0 672739766 23007232 3622 268435456000 4194304 4417452 140734366480928 18446744073709551615 4231321 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/23832/statm: 5617 3622 144 55 0 3503 0
Current children cumulated CPU time (s) 0.2
Current children cumulated vsize (KiB) 22468

[startup+0.301297 s]
/proc/loadavg: 27.18 31.02 31.68 1/413 23832
/proc/meminfo: memFree=255526052/264257624 swapFree=67111528/67111528
[pid=23832] ppid=23830 vsize=32296 CPUtime=0.3 cores=0-31
/proc/23832/stat : 23832 (plingeling) R 23830 23832 23806 0 -1 4202496 6154 1242 0 0 27 2 0 1 19 0 1 0 672739766 33071104 6086 268435456000 4194304 4417452 140734366480928 18446744073709551615 4236706 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/23832/statm: 8074 6086 145 55 0 5960 0
Current children cumulated CPU time (s) 0.3
Current children cumulated vsize (KiB) 32296

[startup+0.700311 s]
/proc/loadavg: 27.18 31.02 31.68 1/413 23832
/proc/meminfo: memFree=255526052/264257624 swapFree=67111528/67111528
[pid=23832] ppid=23830 vsize=55408 CPUtime=0.69 cores=0-31
/proc/23832/stat : 23832 (plingeling) R 23830 23832 23806 0 -1 4202496 11925 1242 0 0 65 3 0 1 19 0 1 0 672739766 56737792 11857 268435456000 4194304 4417452 140734366480928 18446744073709551615 4262065 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/23832/statm: 13852 11857 146 55 0 11738 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 55408

[startup+1.50035 s]
/proc/loadavg: 27.18 31.02 31.68 2/414 23838
/proc/meminfo: memFree=255429276/264257624 swapFree=67111528/67111528
[pid=23832] ppid=23830 vsize=114204 CPUtime=1.49 cores=0-31
/proc/23832/stat : 23832 (plingeling) R 23830 23832 23806 0 -1 4202496 26640 1242 0 0 142 6 0 1 21 0 1 0 672739766 116944896 26572 268435456000 4194304 4417452 140734366480928 18446744073709551615 4231250 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/23832/statm: 28551 26572 146 55 0 26437 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 114204

[startup+3.1004 s]
/proc/loadavg: 25.08 30.53 31.52 2/414 23838
/proc/meminfo: memFree=255322512/264257624 swapFree=67111528/67111528
[pid=23832] ppid=23830 vsize=251864 CPUtime=3.09 cores=0-31
/proc/23832/stat : 23832 (plingeling) R 23830 23832 23806 0 -1 4202496 61055 1242 0 0 295 13 0 1 25 0 1 0 672739766 257908736 60987 268435456000 4194304 4417452 140734366480928 18446744073709551615 258650955760 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/23832/statm: 62966 60987 146 55 0 60852 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 251864

[startup+6.30039 s]
/proc/loadavg: 25.08 30.53 31.52 2/414 23838
/proc/meminfo: memFree=255052560/264257624 swapFree=67111528/67111528
[pid=23832] ppid=23830 vsize=928276 CPUtime=18.7 cores=0-31
/proc/23832/stat : 23832 (plingeling) S 23830 23832 23806 0 -1 4202496 176391 1242 0 0 1743 126 0 1 18 0 33 0 672739766 950554624 114856 268435456000 4194304 4417452 140734366480928 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/23832/statm: 232069 114337 170 55 0 229955 0
Current children cumulated CPU time (s) 18.7
Current children cumulated vsize (KiB) 928276

[startup+12.7016 s]
/proc/loadavg: 26.15 30.57 31.52 33/446 23870
/proc/meminfo: memFree=255004964/264257624 swapFree=67111528/67111528
[pid=23832] ppid=23830 vsize=949800 CPUtime=223.18 cores=0-31
/proc/23832/stat : 23832 (plingeling) S 23830 23832 23806 0 -1 4202496 286932 1242 0 0 22158 159 0 1 18 0 33 0 672739766 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/23832/statm: 237450 132961 175 55 0 235336 0
[pid=23832/tid=23839] ppid=23830 vsize=949800 CPUtime=6.83 cores=0-31
/proc/23832/task/23839/stat : 23839 (plingeling) R 23830 23832 23806 0 -1 4202560 5271 1242 0 0 679 3 0 1 25 0 33 0 672740352 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4250943 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23832/tid=23840] ppid=23830 vsize=949800 CPUtime=6.82 cores=0-31
/proc/23832/task/23840/stat : 23840 (plingeling) R 23830 23832 23806 0 -1 4202560 4654 1242 0 0 678 3 0 1 25 0 33 0 672740352 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23832/tid=23841] ppid=23830 vsize=949800 CPUtime=6.82 cores=0-31
/proc/23832/task/23841/stat : 23841 (plingeling) R 23830 23832 23806 0 -1 4202560 2541 1242 0 0 679 2 0 1 25 0 33 0 672740352 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4357130 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23832/tid=23842] ppid=23830 vsize=949800 CPUtime=6.78 cores=0-31
/proc/23832/task/23842/stat : 23842 (plingeling) R 23830 23832 23806 0 -1 4202560 3826 1242 0 0 674 3 0 1 25 0 33 0 672740352 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=23832/tid=23843] ppid=23830 vsize=949800 CPUtime=6.81 cores=0-31
/proc/23832/task/23843/stat : 23843 (plingeling) R 23830 23832 23806 0 -1 4202560 5369 1242 0 0 675 5 0 1 25 0 33 0 672740352 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=23832/tid=23844] ppid=23830 vsize=949800 CPUtime=6.82 cores=0-31
/proc/23832/task/23844/stat : 23844 (plingeling) R 23830 23832 23806 0 -1 4202560 3258 1242 0 0 679 2 0 1 25 0 33 0 672740352 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4365155 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23832/tid=23845] ppid=23830 vsize=949800 CPUtime=6.81 cores=0-31
/proc/23832/task/23845/stat : 23845 (plingeling) R 23830 23832 23806 0 -1 4202560 4546 1242 0 0 677 3 0 1 25 0 33 0 672740352 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4237371 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=23832/tid=23846] ppid=23830 vsize=949800 CPUtime=6.81 cores=0-31
/proc/23832/task/23846/stat : 23846 (plingeling) R 23830 23832 23806 0 -1 4202560 7164 1242 0 0 676 4 0 1 25 0 33 0 672740352 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4259922 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=23832/tid=23847] ppid=23830 vsize=949800 CPUtime=6.81 cores=0-31
/proc/23832/task/23847/stat : 23847 (plingeling) R 23830 23832 23806 0 -1 4202560 7400 1242 0 0 676 4 0 1 25 0 33 0 672740352 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23832/tid=23848] ppid=23830 vsize=949800 CPUtime=6.81 cores=0-31
/proc/23832/task/23848/stat : 23848 (plingeling) R 23830 23832 23806 0 -1 4202560 3601 1242 0 0 677 3 0 1 25 0 33 0 672740352 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23832/tid=23849] ppid=23830 vsize=949800 CPUtime=6.81 cores=0-31
/proc/23832/task/23849/stat : 23849 (plingeling) R 23830 23832 23806 0 -1 4202560 3765 1242 0 0 676 4 0 1 25 0 33 0 672740353 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=23832/tid=23850] ppid=23830 vsize=949800 CPUtime=6.72 cores=0-31
/proc/23832/task/23850/stat : 23850 (plingeling) R 23830 23832 23806 0 -1 4202560 2475 1242 0 0 668 3 0 1 25 0 33 0 672740353 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=23832/tid=23851] ppid=23830 vsize=949800 CPUtime=6.81 cores=0-31
/proc/23832/task/23851/stat : 23851 (plingeling) R 23830 23832 23806 0 -1 4202560 4335 1242 0 0 676 4 0 1 25 0 33 0 672740353 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4320938 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=23832/tid=23852] ppid=23830 vsize=949800 CPUtime=6.82 cores=0-31
/proc/23832/task/23852/stat : 23852 (plingeling) R 23830 23832 23806 0 -1 4202560 6055 1242 0 0 677 4 0 1 25 0 33 0 672740353 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=23832/tid=23853] ppid=23830 vsize=949800 CPUtime=6.8 cores=0-31
/proc/23832/task/23853/stat : 23853 (plingeling) R 23830 23832 23806 0 -1 4202560 5205 1242 0 0 675 4 0 1 25 0 33 0 672740353 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4357291 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=23832/tid=23854] ppid=23830 vsize=949800 CPUtime=6.8 cores=0-31
/proc/23832/task/23854/stat : 23854 (plingeling) R 23830 23832 23806 0 -1 4202560 7324 1242 0 0 674 5 0 1 25 0 33 0 672740353 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4358093 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=23832/tid=23855] ppid=23830 vsize=949800 CPUtime=6.8 cores=0-31
/proc/23832/task/23855/stat : 23855 (plingeling) R 23830 23832 23806 0 -1 4202560 8628 1242 0 0 673 6 0 1 25 0 33 0 672740353 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4357207 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=23832/tid=23856] ppid=23830 vsize=949800 CPUtime=6.8 cores=0-31
/proc/23832/task/23856/stat : 23856 (plingeling) R 23830 23832 23806 0 -1 4202560 9350 1242 0 0 674 5 0 1 25 0 33 0 672740354 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4232670 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=23832/tid=23857] ppid=23830 vsize=949800 CPUtime=6.79 cores=0-31
/proc/23832/task/23857/stat : 23857 (plingeling) R 23830 23832 23806 0 -1 4202560 3063 1242 0 0 675 3 0 1 25 0 33 0 672740354 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=23832/tid=23858] ppid=23830 vsize=949800 CPUtime=6.72 cores=0-31
/proc/23832/task/23858/stat : 23858 (plingeling) R 23830 23832 23806 0 -1 4202560 8120 1242 0 0 668 3 0 1 25 0 33 0 672740354 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4250957 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=23832/tid=23859] ppid=23830 vsize=949800 CPUtime=6.79 cores=0-31
/proc/23832/task/23859/stat : 23859 (plingeling) R 23830 23832 23806 0 -1 4202560 3466 1242 0 0 675 3 0 1 25 0 33 0 672740354 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4250713 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=23832/tid=23860] ppid=23830 vsize=949800 CPUtime=6.8 cores=0-31
/proc/23832/task/23860/stat : 23860 (plingeling) R 23830 23832 23806 0 -1 4202560 4332 1242 0 0 675 4 0 1 25 0 33 0 672740354 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=23832/tid=23861] ppid=23830 vsize=949800 CPUtime=6.78 cores=0-31
/proc/23832/task/23861/stat : 23861 (plingeling) R 23830 23832 23806 0 -1 4202560 5094 1242 0 0 673 4 0 1 25 0 33 0 672740354 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4250548 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=23832/tid=23862] ppid=23830 vsize=949800 CPUtime=6.79 cores=0-31
/proc/23832/task/23862/stat : 23862 (plingeling) R 23830 23832 23806 0 -1 4202560 10778 1242 0 0 672 6 0 1 25 0 33 0 672740354 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4357232 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=23832/tid=23863] ppid=23830 vsize=949800 CPUtime=6.79 cores=0-31
/proc/23832/task/23863/stat : 23863 (plingeling) R 23830 23832 23806 0 -1 4202560 8750 1242 0 0 674 4 0 1 25 0 33 0 672740354 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4366646 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=23832/tid=23864] ppid=23830 vsize=949800 CPUtime=6.79 cores=0-31
/proc/23832/task/23864/stat : 23864 (plingeling) R 23830 23832 23806 0 -1 4202560 3219 1242 0 0 675 3 0 1 25 0 33 0 672740355 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=23832/tid=23865] ppid=23830 vsize=949800 CPUtime=6.79 cores=0-31
/proc/23832/task/23865/stat : 23865 (plingeling) R 23830 23832 23806 0 -1 4202560 2951 1242 0 0 675 3 0 1 25 0 33 0 672740355 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4366065 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=23832/tid=23866] ppid=23830 vsize=949800 CPUtime=6.68 cores=0-31
/proc/23832/task/23866/stat : 23866 (plingeling) R 23830 23832 23806 0 -1 4202560 6154 1242 0 0 663 4 0 1 25 0 33 0 672740355 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=23832/tid=23867] ppid=23830 vsize=949800 CPUtime=6.79 cores=0-31
/proc/23832/task/23867/stat : 23867 (plingeling) R 23830 23832 23806 0 -1 4202560 2231 1242 0 0 675 3 0 1 25 0 33 0 672740355 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4232537 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=23832/tid=23868] ppid=23830 vsize=949800 CPUtime=6.79 cores=0-31
/proc/23832/task/23868/stat : 23868 (plingeling) R 23830 23832 23806 0 -1 4202560 3611 1242 0 0 674 4 0 1 25 0 33 0 672740355 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4357260 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23832/tid=23869] ppid=23830 vsize=949800 CPUtime=6.77 cores=0-31
/proc/23832/task/23869/stat : 23869 (plingeling) R 23830 23832 23806 0 -1 4202560 4008 1242 0 0 673 3 0 1 25 0 33 0 672740355 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364587 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=23832/tid=23870] ppid=23830 vsize=949800 CPUtime=6.78 cores=0-31
/proc/23832/task/23870/stat : 23870 (plingeling) R 23830 23832 23806 0 -1 4202560 8051 1242 0 0 673 4 0 1 25 0 33 0 672740355 972595200 132961 268435456000 4194304 4417452 140734366480928 18446744073709551615 4232680 0 0 4096 17442 0 0 0 -1 25 0 0 0
Current children cumulated CPU time (s) 223.18
Current children cumulated vsize (KiB) 949800

[startup+25.5021 s]
/proc/loadavg: 27.05 30.62 31.53 33/446 23870
/proc/meminfo: memFree=254602108/264257624 swapFree=67111528/67111528
[pid=23832] ppid=23830 vsize=1345676 CPUtime=632.07 cores=0-31
/proc/23832/stat : 23832 (plingeling) S 23830 23832 23806 0 -1 4202496 408270 1242 0 0 63015 191 0 1 18 0 33 0 672739766 1377972224 225300 268435456000 4194304 4417452 140734366480928 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/23832/statm: 336419 225300 175 55 0 334305 0
[pid=23832/tid=23839] ppid=23830 vsize=1345676 CPUtime=19.63 cores=0-31
/proc/23832/task/23839/stat : 23839 (plingeling) R 23830 23832 23806 0 -1 4202560 7273 1242 0 0 1958 4 0 1 25 0 33 0 672740352 1377972224 225300 268435456000 4194304 4417452 140734366480928 18446744073709551615 4354506 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23832/tid=23840] ppid=23830 vsize=1345676 CPUtime=19.62 cores=0-31
/proc/23832/task/23840/stat : 23840 (plingeling) R 23830 23832 23806 0 -1 4202560 9931 1242 0 0 1957 4 0 1 25 0 33 0 672740352 1377972224 225300 268435456000 4194304 4417452 140734366480928 18446744073709551615 4321595 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23832/tid=23841] ppid=23830 vsize=1345676 CPUtime=19.6 cores=0-31
/proc/23832/task/23841/stat : 23841 (plingeling) R 23830 23832 23806 0 -1 4202560 5115 1242 0 0 1956 3 0 1 25 0 33 0 672740352 1377972224 225300 268435456000 4194304 4417452 140734366480928 18446744073709551615 4232623 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=23832/tid=23842] ppid=23830 vsize=1345676 CPUtime=19.58 cores=0-31
/proc/23832/task/23842/stat : 23842 (plingeling) R 23830 23832 23806 0 -1 4202560 14577 1242 0 0 1949 8 0 1 25 0 33 0 672740352 1377972224 225300 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=23832/tid=23843] ppid=23830 vsize=1345676 CPUtime=19.61 cores=0-31
/proc/23832/task/23843/stat : 23843 (plingeling) R 23830 23832 23806 0 -1 4202560 9762 1242 0 0 1954 6 0 1 25 0 33 0 672740352 1377972224 225300 268435456000 4194304 4417452 140734366480928 18446744073709551615 4365150 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=23832/tid=23844] ppid=23830 vsize=1345676 CPUtime=19.61 cores=0-31
/proc/23832/task/23844/stat : 23844 (plingeling) R 23830 23832 23806 0 -1 4202560 6699 1242 0 0 1957 3 0 1 25 0 33 0 672740352 1377972224 225300 268435456000 4194304 4417452 140734366480928 18446744073709551615 4232654 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23832/tid=23845] ppid=23830 vsize=1345676 CPUtime=19.59 cores=0-31
/proc/23832/task/23845/stat : 23845 (plingeling) R 23830 23832 23806 0 -1 4202560 7967 1242 0 0 1955 3 0 1 25 0 33 0 672740352 1377972224 225300 268435456000 4194304 4417452 140734366480928 18446744073709551615 4281860 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=23832/tid=23846] ppid=23830 vsize=1345676 CPUtime=19.61 cores=0-31
/proc/23832/task/23846/stat : 23846 (plingeling) R 23830 23832 23806 0 -1 4202560 13511 1242 0 0 1954 6 0 1 25 0 33 0 672740352 1377972224 225300 268435456000 4194304 4417452 140734366480928 18446744073709551615 4358106 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=23832/tid=23847] ppid=23830 vsize=1345676 CPUtime=19.59 cores=0-31
/proc/23832/task/23847/stat : 23847 (plingeling) R 23830 23832 23806 0 -1 4202560 11972 1242 0 0 1953 5 0 1 25 0 33 0 672740352 1377972224 225300 268435456000 4194304 4417452 140734366480928 18446744073709551615 4232654 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23832/tid=23848] ppid=23830 vsize=1345676 CPUtime=19.52 cores=0-31
/proc/23832/task/23848/stat : 23848 (plingeling) R 23830 23832 23806 0 -1 4202560 6656 1242 0 0 1947 4 0 1 25 0 33 0 672740352 1377972224 225300 268435456000 4194304 4417452 140734366480928 18446744073709551615 4320800 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=23832/tid=23849] ppid=23830 vsize=1345676 CPUtime=19.59 cores=0-31
/proc/23832/task/23849/stat : 23849 (plingeling) R 23830 23832 23806 0 -1 4202560 8207 1242 0 0 1954 4 0 1 25 0 33 0 672740353 1377972224 225300 268435456000 4194304 4417452 140734366480928 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=23832/tid=23850] ppid=23830 vsize=1345676 CPUtime=19.43 cores=0-31
/proc/23832/task/23850/stat : 23850 (plingeling) R 23830 23832 23806 0 -1 4202560 3031 1242 0 0 1939 3 0 1 25 0 33 0 672740353 1377972224 225300 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364461 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=23832/tid=23851] ppid=23830 vsize=1345676 CPUtime=19.61 cores=0-31
/proc/23832/task/23851/stat : 23851 (plingeling) R 23830 23832 23806 0 -1 4202560 8570 1242 0 0 1955 5 0 1 25 0 33 0 672740353 1377972224 225300 268435456000 4194304 4417452 140734366480928 18446744073709551615 4263136 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=23832/tid=23852] ppid=23830 vsize=1345676 CPUtime=19.6 cores=0-31
/proc/23832/task/23852/stat : 23852 (plingeling) R 23830 23832 23806 0 -1 4202560 11204 1242 0 0 1954 5 0 1 25 0 33 0 672740353 1377972224 225300 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364542 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=23832/tid=23853] ppid=23830 vsize=1345676 CPUtime=19.59 cores=0-31
/proc/23832/task/23853/stat : 23853 (plingeling) R 23830 23832 23806 0 -1 4202560 6989 1242 0 0 1953 5 0 1 25 0 33 0 672740353 1377972224 225300 268435456000 4194304 4417452 140734366480928 18446744073709551615 4357204 0 0 4096 17442 0 0 0 -1 9 0 0 0

################
# More data... #
################

/proc/23832/task/23857/stat : 23857 (plingeling) R 23830 23832 23806 0 -1 4202560 2286894 1242 0 0 495610 3503 0 1 25 0 33 0 672740354 50843410432 12253017 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23832/tid=23858] ppid=23830 vsize=49651768 CPUtime=4998.03 cores=0-31
/proc/23832/task/23858/stat : 23858 (plingeling) R 23830 23832 23806 0 -1 4202560 1832666 1242 0 0 496998 2804 0 1 25 0 33 0 672740354 50843410432 12253017 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=23832/tid=23859] ppid=23830 vsize=49651768 CPUtime=4994.71 cores=0-31
/proc/23832/task/23859/stat : 23859 (plingeling) R 23830 23832 23806 0 -1 4202560 1648216 1242 0 0 497313 2157 0 1 25 0 33 0 672740354 50843410432 12253017 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=23832/tid=23860] ppid=23830 vsize=49651768 CPUtime=4994.68 cores=0-31
/proc/23832/task/23860/stat : 23860 (plingeling) R 23830 23832 23806 0 -1 4202560 2068460 1242 0 0 496409 3058 0 1 25 0 33 0 672740354 50843410432 12253017 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=23832/tid=23861] ppid=23830 vsize=49651768 CPUtime=4989.26 cores=0-31
/proc/23832/task/23861/stat : 23861 (plingeling) R 23830 23832 23806 0 -1 4202560 1842932 1242 0 0 495904 3021 0 1 25 0 33 0 672740354 50843410432 12253017 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23832/tid=23862] ppid=23830 vsize=49651768 CPUtime=4992.72 cores=0-31
/proc/23832/task/23862/stat : 23862 (plingeling) R 23830 23832 23806 0 -1 4202560 2606730 1242 0 0 494855 4416 0 1 25 0 33 0 672740354 50843410432 12253017 268435456000 4194304 4417452 140734366480928 18446744073709551615 4260714 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=23832/tid=23863] ppid=23830 vsize=49651768 CPUtime=4994.99 cores=0-31
/proc/23832/task/23863/stat : 23863 (plingeling) R 23830 23832 23806 0 -1 4202560 1471727 1242 0 0 497708 1790 0 1 25 0 33 0 672740354 50843410432 12253017 268435456000 4194304 4417452 140734366480928 18446744073709551615 4366385 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23832/tid=23864] ppid=23830 vsize=49651768 CPUtime=4983.47 cores=0-31
/proc/23832/task/23864/stat : 23864 (plingeling) R 23830 23832 23806 0 -1 4202560 1557159 1242 0 0 496391 1955 0 1 25 0 33 0 672740355 50843410432 12253017 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=23832/tid=23865] ppid=23830 vsize=49651768 CPUtime=4997.52 cores=0-31
/proc/23832/task/23865/stat : 23865 (plingeling) R 23830 23832 23806 0 -1 4202560 2194968 1242 0 0 496564 3187 0 1 25 0 33 0 672740355 50843410432 12253017 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=23832/tid=23866] ppid=23830 vsize=49651768 CPUtime=4995.04 cores=0-31
/proc/23832/task/23866/stat : 23866 (plingeling) R 23830 23832 23806 0 -1 4202560 1812796 1242 0 0 497371 2132 0 1 25 0 33 0 672740355 50843410432 12253017 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23832/tid=23867] ppid=23830 vsize=49651768 CPUtime=5003.39 cores=0-31
/proc/23832/task/23867/stat : 23867 (plingeling) R 23830 23832 23806 0 -1 4202560 1464070 1242 0 0 497730 2608 0 1 25 0 33 0 672740355 50843410432 12253017 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=23832/tid=23868] ppid=23830 vsize=49651768 CPUtime=4984.4 cores=0-31
/proc/23832/task/23868/stat : 23868 (plingeling) R 23830 23832 23806 0 -1 4202560 2290809 1242 0 0 495355 3084 0 1 25 0 33 0 672740355 50843410432 12253017 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=23832/tid=23869] ppid=23830 vsize=49651768 CPUtime=4992.28 cores=0-31
/proc/23832/task/23869/stat : 23869 (plingeling) R 23830 23832 23806 0 -1 4202560 1579597 1242 0 0 496984 2243 0 1 25 0 33 0 672740355 50843410432 12253017 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=23832/tid=23870] ppid=23830 vsize=49651768 CPUtime=4997.12 cores=0-31
/proc/23832/task/23870/stat : 23870 (plingeling) R 23830 23832 23806 0 -1 4202560 1924995 1242 0 0 496444 3267 0 1 25 0 33 0 672740355 50843410432 12253017 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364873 0 0 4096 17442 0 0 0 -1 30 0 0 0
Current children cumulated CPU time (s) 159781
Current children cumulated vsize (KiB) 49651768



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5029.2 s]
/proc/loadavg: 32.10 32.10 32.08 34/446 24019
/proc/meminfo: memFree=206274892/264257624 swapFree=67111528/67111528
[pid=23832] ppid=23830 vsize=49795680 CPUtime=160003 cores=0-31
/proc/23832/stat : 23832 (plingeling) S 23830 23832 23806 0 -1 4202496 61328403 1242 0 0 15909812 90488 0 1 18 0 33 0 672739766 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/23832/statm: 12448920 12288995 175 55 0 12446806 0
[pid=23832/tid=23839] ppid=23830 vsize=49795680 CPUtime=5005.48 cores=0-31
/proc/23832/task/23839/stat : 23839 (plingeling) R 23830 23832 23806 0 -1 4202560 1838537 1242 0 0 497603 2944 0 1 25 0 33 0 672740352 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=23832/tid=23840] ppid=23830 vsize=49795680 CPUtime=5003.44 cores=0-31
/proc/23832/task/23840/stat : 23840 (plingeling) R 23830 23832 23806 0 -1 4202560 1514781 1242 0 0 498272 2071 0 1 25 0 33 0 672740352 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=23832/tid=23841] ppid=23830 vsize=49795680 CPUtime=4991.69 cores=0-31
/proc/23832/task/23841/stat : 23841 (plingeling) R 23830 23832 23806 0 -1 4202560 1686254 1242 0 0 496263 2905 0 1 25 0 33 0 672740352 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23832/tid=23842] ppid=23830 vsize=49795680 CPUtime=4983.25 cores=0-31
/proc/23832/task/23842/stat : 23842 (plingeling) R 23830 23832 23806 0 -1 4202560 3783309 1242 0 0 492784 5540 0 1 25 0 33 0 672740352 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4236956 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=23832/tid=23843] ppid=23830 vsize=49795680 CPUtime=4997.3 cores=0-31
/proc/23832/task/23843/stat : 23843 (plingeling) R 23830 23832 23806 0 -1 4202560 2091665 1242 0 0 496573 3156 0 1 25 0 33 0 672740352 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=23832/tid=23844] ppid=23830 vsize=49795680 CPUtime=5002.75 cores=0-31
/proc/23832/task/23844/stat : 23844 (plingeling) R 23830 23832 23806 0 -1 4202560 1718038 1242 0 0 498172 2102 0 1 25 0 33 0 672740352 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=23832/tid=23845] ppid=23830 vsize=49795680 CPUtime=5005.98 cores=0-31
/proc/23832/task/23845/stat : 23845 (plingeling) R 23830 23832 23806 0 -1 4202560 1048372 1242 0 0 499224 1373 0 1 25 0 33 0 672740352 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=23832/tid=23846] ppid=23830 vsize=49795680 CPUtime=5001.28 cores=0-31
/proc/23832/task/23846/stat : 23846 (plingeling) R 23830 23832 23806 0 -1 4202560 1702794 1242 0 0 497565 2562 0 1 25 0 33 0 672740352 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23832/tid=23847] ppid=23830 vsize=49795680 CPUtime=4985.85 cores=0-31
/proc/23832/task/23847/stat : 23847 (plingeling) R 23830 23832 23806 0 -1 4202560 2124059 1242 0 0 495758 2826 0 1 25 0 33 0 672740352 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=23832/tid=23848] ppid=23830 vsize=49795680 CPUtime=5007.54 cores=0-31
/proc/23832/task/23848/stat : 23848 (plingeling) R 23830 23832 23806 0 -1 4202560 1568916 1242 0 0 498650 2103 0 1 25 0 33 0 672740352 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4358748 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=23832/tid=23849] ppid=23830 vsize=49795680 CPUtime=4997.15 cores=0-31
/proc/23832/task/23849/stat : 23849 (plingeling) R 23830 23832 23806 0 -1 4202560 2456886 1242 0 0 495426 4288 0 1 25 0 33 0 672740353 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=23832/tid=23850] ppid=23830 vsize=49795680 CPUtime=5001.94 cores=0-31
/proc/23832/task/23850/stat : 23850 (plingeling) R 23830 23832 23806 0 -1 4202560 1770240 1242 0 0 497640 2553 0 1 25 0 33 0 672740353 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=23832/tid=23851] ppid=23830 vsize=49795680 CPUtime=5005.36 cores=0-31
/proc/23832/task/23851/stat : 23851 (plingeling) R 23830 23832 23806 0 -1 4202560 2176644 1242 0 0 497137 3398 0 1 25 0 33 0 672740353 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4309982 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=23832/tid=23852] ppid=23830 vsize=49795680 CPUtime=5005.99 cores=0-31
/proc/23832/task/23852/stat : 23852 (plingeling) R 23830 23832 23806 0 -1 4202560 1287469 1242 0 0 498893 1705 0 1 25 0 33 0 672740353 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=23832/tid=23853] ppid=23830 vsize=49795680 CPUtime=5000.19 cores=0-31
/proc/23832/task/23853/stat : 23853 (plingeling) R 23830 23832 23806 0 -1 4202560 2019148 1242 0 0 497191 2827 0 1 25 0 33 0 672740353 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4258513 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=23832/tid=23854] ppid=23830 vsize=49795680 CPUtime=5002.32 cores=0-31
/proc/23832/task/23854/stat : 23854 (plingeling) R 23830 23832 23806 0 -1 4202560 1813862 1242 0 0 497259 2972 0 1 25 0 33 0 672740353 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=23832/tid=23855] ppid=23830 vsize=49795680 CPUtime=4997.4 cores=0-31
/proc/23832/task/23855/stat : 23855 (plingeling) R 23830 23832 23806 0 -1 4202560 2267687 1242 0 0 496006 3733 0 1 25 0 33 0 672740353 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23832/tid=23856] ppid=23830 vsize=49795680 CPUtime=4996.62 cores=0-31
/proc/23832/task/23856/stat : 23856 (plingeling) R 23830 23832 23806 0 -1 4202560 1759386 1242 0 0 497494 2167 0 1 25 0 33 0 672740354 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4270279 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=23832/tid=23857] ppid=23830 vsize=49795680 CPUtime=4998.08 cores=0-31
/proc/23832/task/23857/stat : 23857 (plingeling) R 23830 23832 23806 0 -1 4202560 2286894 1242 0 0 496304 3503 0 1 25 0 33 0 672740354 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23832/tid=23858] ppid=23830 vsize=49795680 CPUtime=5004.74 cores=0-31
/proc/23832/task/23858/stat : 23858 (plingeling) R 23830 23832 23806 0 -1 4202560 1832666 1242 0 0 497669 2804 0 1 25 0 33 0 672740354 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=23832/tid=23859] ppid=23830 vsize=49795680 CPUtime=5001.65 cores=0-31
/proc/23832/task/23859/stat : 23859 (plingeling) R 23830 23832 23806 0 -1 4202560 1648216 1242 0 0 498007 2157 0 1 25 0 33 0 672740354 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=23832/tid=23860] ppid=23830 vsize=49795680 CPUtime=5001.62 cores=0-31
/proc/23832/task/23860/stat : 23860 (plingeling) R 23830 23832 23806 0 -1 4202560 2068460 1242 0 0 497103 3058 0 1 25 0 33 0 672740354 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=23832/tid=23861] ppid=23830 vsize=49795680 CPUtime=4996.19 cores=0-31
/proc/23832/task/23861/stat : 23861 (plingeling) R 23830 23832 23806 0 -1 4202560 1842932 1242 0 0 496597 3021 0 1 25 0 33 0 672740354 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23832/tid=23862] ppid=23830 vsize=49795680 CPUtime=4999.66 cores=0-31
/proc/23832/task/23862/stat : 23862 (plingeling) R 23830 23832 23806 0 -1 4202560 2606730 1242 0 0 495549 4416 0 1 25 0 33 0 672740354 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4234636 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=23832/tid=23863] ppid=23830 vsize=49795680 CPUtime=5001.93 cores=0-31
/proc/23832/task/23863/stat : 23863 (plingeling) R 23830 23832 23806 0 -1 4202560 1471727 1242 0 0 498402 1790 0 1 25 0 33 0 672740354 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=23832/tid=23864] ppid=23830 vsize=49795680 CPUtime=4990.41 cores=0-31
/proc/23832/task/23864/stat : 23864 (plingeling) R 23830 23832 23806 0 -1 4202560 1557159 1242 0 0 497085 1955 0 1 25 0 33 0 672740355 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=23832/tid=23865] ppid=23830 vsize=49795680 CPUtime=5004.46 cores=0-31
/proc/23832/task/23865/stat : 23865 (plingeling) R 23830 23832 23806 0 -1 4202560 2194968 1242 0 0 497258 3187 0 1 25 0 33 0 672740355 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=23832/tid=23866] ppid=23830 vsize=49795680 CPUtime=5001.98 cores=0-31
/proc/23832/task/23866/stat : 23866 (plingeling) R 23830 23832 23806 0 -1 4202560 1812796 1242 0 0 498065 2132 0 1 25 0 33 0 672740355 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23832/tid=23867] ppid=23830 vsize=49795680 CPUtime=5010.33 cores=0-31
/proc/23832/task/23867/stat : 23867 (plingeling) R 23830 23832 23806 0 -1 4202560 1464070 1242 0 0 498424 2608 0 1 25 0 33 0 672740355 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4275960 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=23832/tid=23868] ppid=23830 vsize=49795680 CPUtime=4991.34 cores=0-31
/proc/23832/task/23868/stat : 23868 (plingeling) R 23830 23832 23806 0 -1 4202560 2290809 1242 0 0 496049 3084 0 1 25 0 33 0 672740355 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=23832/tid=23869] ppid=23830 vsize=49795680 CPUtime=4999.22 cores=0-31
/proc/23832/task/23869/stat : 23869 (plingeling) R 23830 23832 23806 0 -1 4202560 1579597 1242 0 0 497678 2243 0 1 25 0 33 0 672740355 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=23832/tid=23870] ppid=23830 vsize=49795680 CPUtime=5004.06 cores=0-31
/proc/23832/task/23870/stat : 23870 (plingeling) R 23830 23832 23806 0 -1 4202560 1924995 1242 0 0 497138 3267 0 1 25 0 33 0 672740355 50990776320 12288995 268435456000 4194304 4417452 140734366480928 18446744073709551615 4250715 0 0 4096 17442 0 0 0 -1 30 0 0 0
Current children cumulated CPU time (s) 160003
Current children cumulated vsize (KiB) 49795680

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)

Sending SIGKILL to -23832

Solver just ended. Dumping a history of the last processes samples

[startup+5030.3 s]
/proc/loadavg: 32.10 32.10 32.08 2/416 24020
/proc/meminfo: memFree=224325896/264257624 swapFree=67111528/67111528
[pid=23832] ppid=23830 vsize=0 CPUtime=160004 cores=0-31
/proc/23832/stat : 23832 (plingeling) Z 23830 23832 23806 0 -1 4203548 61328403 1242 0 0 15909834 90597 0 1 18 0 2 0 672739766 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/23832/statm: 0 0 0 0 0 0 0
[pid=23832/tid=23857] ppid=23830 vsize=0 CPUtime=4999.17 cores=0-31
/proc/23832/task/23857/stat : 23857 (plingeling) R 23830 23832 23806 0 -1 4203588 2286894 1242 0 0 496305 3611 0 1 25 0 2 0 672740354 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 160004
Current children cumulated vsize (KiB) 0

[startup+5031.1 s]
/proc/loadavg: 32.10 32.10 32.08 2/416 24020
/proc/meminfo: memFree=224325896/264257624 swapFree=67111528/67111528
[pid=23832] ppid=23830 vsize=0 CPUtime=160005 cores=0-31
/proc/23832/stat : 23832 (plingeling) Z 23830 23832 23806 0 -1 4203548 61328403 1242 0 0 15909834 90674 0 1 18 0 2 0 672739766 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/23832/statm: 0 0 0 0 0 0 0
[pid=23832/tid=23857] ppid=23830 vsize=0 CPUtime=4999.94 cores=0-31
/proc/23832/task/23857/stat : 23857 (plingeling) R 23830 23832 23806 0 -1 4203588 2286894 1242 0 0 496305 3688 0 1 25 0 2 0 672740354 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 160005
Current children cumulated vsize (KiB) 0

[startup+5031.53 s]
/proc/loadavg: 32.10 32.10 32.08 2/415 24020
/proc/meminfo: memFree=246240144/264257624 swapFree=67111528/67111528
[pid=23832] ppid=23830 vsize=0 CPUtime=160006 cores=0-31
/proc/23832/stat : 23832 (plingeling) Z 23830 23832 23806 0 -1 4203548 61328403 1242 0 0 15909834 90717 0 1 18 0 2 0 672739766 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/23832/statm: 0 0 0 0 0 0 0
[pid=23832/tid=23857] ppid=23830 vsize=0 CPUtime=5000.37 cores=0-31
/proc/23832/task/23857/stat : 23857 (plingeling) R 23830 23832 23806 0 -1 4203588 2286894 1242 0 0 496305 3731 0 1 25 0 2 0 672740354 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 160006
Current children cumulated vsize (KiB) 0

[startup+5031.7 s]
/proc/loadavg: 32.10 32.10 32.08 2/415 24020
/proc/meminfo: memFree=246240144/264257624 swapFree=67111528/67111528
[pid=23832] ppid=23830 vsize=0 CPUtime=160006 cores=0-31
/proc/23832/stat : 23832 (plingeling) Z 23830 23832 23806 0 -1 4203548 61328403 1242 0 0 15909834 90734 0 1 18 0 2 0 672739766 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/23832/statm: 0 0 0 0 0 0 0
[pid=23832/tid=23857] ppid=23830 vsize=0 CPUtime=5000.54 cores=0-31
/proc/23832/task/23857/stat : 23857 (plingeling) R 23830 23832 23806 0 -1 4203588 2286894 1242 0 0 496305 3748 0 1 25 0 2 0 672740354 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 160006
Current children cumulated vsize (KiB) 0

[startup+5031.8 s]
/proc/loadavg: 32.10 32.10 32.08 2/415 24020
/proc/meminfo: memFree=246240144/264257624 swapFree=67111528/67111528
[pid=23832] ppid=23830 vsize=0 CPUtime=160006 cores=0-31
/proc/23832/stat : 23832 (plingeling) Z 23830 23832 23806 0 -1 4203548 61328403 1242 0 0 15909834 90744 0 1 18 0 2 0 672739766 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/23832/statm: 0 0 0 0 0 0 0
[pid=23832/tid=23857] ppid=23830 vsize=0 CPUtime=5000.64 cores=0-31
/proc/23832/task/23857/stat : 23857 (plingeling) R 23830 23832 23806 0 -1 4203588 2286894 1242 0 0 496305 3758 0 1 25 0 2 0 672740354 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 160006
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5031.9
CPU time (s): 160006
CPU user time (s): 159098
CPU system time (s): 907.548
CPU usage (%): 3179.83
Max. virtual memory (cumulated for all children) (KiB): 53430800

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 159098
system time used= 907.548
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 61329645
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 24276
involuntary context switches= 399895

runsolver used 27.4168 second user time and 115.057 second system time

The end

Launcher Data 

Begin job on node200 at 2011-05-17 08:08:54
IDJOB=3372441
IDBENCH=82381
IDSOLVER=1852
FILE ID=node200/3372441-1305612533
RUNJOBID= node200-1305612533-23820
PBS_JOBID= 13285563
Free space on /tmp= 62044 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/application/rintanen/SATPlanning/openstacks-sequencedstrips-nonadl-nonnegated-os-sequencedstrips-p30_3.025-NOTKNOWN.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3372441-1305612533/watcher-3372441-1305612533 -o /tmp/evaluation-result-3372441-1305612533/solver-3372441-1305612533 -C 160000 -W 5100 -M 266338304  plingeling -t 32 HOME/instance-3372441-1305612533.cnf

TIME LIMIT= 160000 seconds
MEMORY LIMIT= 266338304 MiB
NBCORE= 32

MD5SUM BENCH= 3c5c73741ba21d5f39c7f436ba834743
RANDOM SEED=1005095793

node200.alineos.net Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 64
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 96
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 66
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 98
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 68
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 100
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 70
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.98
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 102
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 16
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 17
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 48
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 18
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 80
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 19
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 112
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 20
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 21
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 50
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 22
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 82
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 114
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.23
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 24
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 25
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 26
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 84
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 27
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 116
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 28
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 29
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 54
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 30
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 86
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 31
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 118
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     264257624 kB
MemFree:      255526324 kB
Buffers:        448200 kB
Cached:        7883460 kB
SwapCached:          0 kB
Active:        7402408 kB
Inactive:       998396 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     264257624 kB
LowFree:      255526324 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:            8068 kB
Writeback:           0 kB
AnonPages:       70072 kB
Mapped:          13484 kB
Slab:           251648 kB
PageTables:       4720 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  199240340 kB
Committed_AS:   186864 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296372 kB
VmallocChunk: 34359440603 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 62044 MiB
End job on node200 at 2011-05-17 09:32:45