Trace number 3372114

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 160003 5019.86

General information on the benchmark

Nameindustrial/anbulagan/
hard-unsat/partial-5-11-u.cnf
MD5SUM8e22d84b1ae625d44fbf300a32374a2d
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables164158
Number of clauses730719
Sum of the clauses size1660140
Maximum clause length10
Minimum clause length1
Number of clauses of size 112474
Number of clauses of size 2600009
Number of clauses of size 387256
Number of clauses of size 43440
Number of clauses of size 513760
Number of clauses of size over 513780

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3372114-1305428031/watcher-3372114-1305428031 -o /tmp/evaluation-result-3372114-1305428031/solver-3372114-1305428031 -C 160000 -W 5100 -M 266338304 plingeling -t 32 HOME/instance-3372114-1305428031.cnf 

running on 32 cores: 0,4,8,12,16,20,24,28,1,5,9,13,17,21,25,29,2,6,10,14,18,22,26,30,3,7,11,15,19,23,27,31

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 160000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 160030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 272730423296 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 272730474496 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 29.55 32.12 32.55 1/413 9154
/proc/meminfo: memFree=255579884/264257624 swapFree=67111528/67111528
[pid=9154] ppid=9152 vsize=21264 CPUtime=0.01 cores=0-31
/proc/9154/stat : 9154 (plingeling) R 9152 9154 9128 0 -1 4202496 3387 1246 0 0 1 0 0 0 22 0 1 0 654289597 21774336 3319 268435456000 4194304 4417452 140735282813712 18446744073709551615 4320831 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/9154/statm: 5316 3319 144 55 0 3202 0

[startup+0.115802 s]
/proc/loadavg: 29.55 32.12 32.55 1/413 9154
/proc/meminfo: memFree=255579884/264257624 swapFree=67111528/67111528
[pid=9154] ppid=9152 vsize=52084 CPUtime=0.1 cores=0-31
/proc/9154/stat : 9154 (plingeling) R 9152 9154 9128 0 -1 4202496 11077 1246 0 0 8 2 0 0 22 0 1 0 654289597 53334016 11009 268435456000 4194304 4417452 140735282813712 18446744073709551615 4209599 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/9154/statm: 13021 11009 145 55 0 10907 0
Current children cumulated CPU time (s) 0.1
Current children cumulated vsize (KiB) 52084

[startup+0.200831 s]
/proc/loadavg: 29.55 32.12 32.55 1/413 9154
/proc/meminfo: memFree=255579884/264257624 swapFree=67111528/67111528
[pid=9154] ppid=9152 vsize=55288 CPUtime=0.18 cores=0-31
/proc/9154/stat : 9154 (plingeling) R 9152 9154 9128 0 -1 4202496 11910 1246 0 0 16 2 0 0 22 0 1 0 654289597 56614912 11842 268435456000 4194304 4417452 140735282813712 18446744073709551615 258650574380 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/9154/statm: 13822 11842 145 55 0 11708 0
Current children cumulated CPU time (s) 0.18
Current children cumulated vsize (KiB) 55288

[startup+0.300788 s]
/proc/loadavg: 29.55 32.12 32.55 1/413 9154
/proc/meminfo: memFree=255579884/264257624 swapFree=67111528/67111528
[pid=9154] ppid=9152 vsize=92932 CPUtime=0.29 cores=0-31
/proc/9154/stat : 9154 (plingeling) R 9152 9154 9128 0 -1 4202496 21321 1246 0 0 25 4 0 0 23 0 1 0 654289597 95162368 21253 268435456000 4194304 4417452 140735282813712 18446744073709551615 4227863 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/9154/statm: 23233 21253 145 55 0 21119 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 92932

[startup+0.700808 s]
/proc/loadavg: 29.55 32.12 32.55 1/413 9154
/proc/meminfo: memFree=255579884/264257624 swapFree=67111528/67111528
[pid=9154] ppid=9152 vsize=124860 CPUtime=0.69 cores=0-31
/proc/9154/stat : 9154 (plingeling) R 9152 9154 9128 0 -1 4202496 29545 1246 0 0 63 6 0 0 25 0 1 0 654289597 127856640 29235 268435456000 4194304 4417452 140735282813712 18446744073709551615 4262065 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/9154/statm: 31215 29235 145 55 0 29101 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 124860

[startup+1.50087 s]
/proc/loadavg: 29.55 32.12 32.55 2/414 9160
/proc/meminfo: memFree=255388980/264257624 swapFree=67111528/67111528
[pid=9154] ppid=9152 vsize=231808 CPUtime=1.49 cores=0-31
/proc/9154/stat : 9154 (plingeling) R 9152 9154 9128 0 -1 4202496 56282 1246 0 0 138 11 0 0 25 0 1 0 654289597 237371392 55972 268435456000 4194304 4417452 140735282813712 18446744073709551615 4262019 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/9154/statm: 57952 55972 145 55 0 55838 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 231808

[startup+3.10088 s]
/proc/loadavg: 27.27 31.60 32.38 2/414 9160
/proc/meminfo: memFree=255205576/264257624 swapFree=67111528/67111528
[pid=9154] ppid=9152 vsize=447788 CPUtime=3.09 cores=0-31
/proc/9154/stat : 9154 (plingeling) R 9152 9154 9128 0 -1 4202496 110317 1246 0 0 287 22 0 0 25 0 1 0 654289597 458534912 109967 268435456000 4194304 4417452 140735282813712 18446744073709551615 4320705 0 0 4096 17442 0 0 0 17 8 0 0 0
/proc/9154/statm: 111947 109967 145 55 0 109833 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 447788

[startup+6.301 s]
/proc/loadavg: 27.27 31.60 32.38 2/414 9162
/proc/meminfo: memFree=254839324/264257624 swapFree=67111528/67111528
[pid=9154] ppid=9152 vsize=747612 CPUtime=6.28 cores=0-31
/proc/9154/stat : 9154 (plingeling) R 9152 9154 9128 0 -1 4202496 185273 1246 0 0 592 36 0 0 25 0 1 0 654289597 765554688 184923 268435456000 4194304 4417452 140735282813712 18446744073709551615 4259962 0 0 4096 17442 0 0 0 17 8 0 0 0
/proc/9154/statm: 186903 184923 145 55 0 184789 0
Current children cumulated CPU time (s) 6.28
Current children cumulated vsize (KiB) 747612

[startup+12.7012 s]
/proc/loadavg: 23.23 30.59 32.05 2/414 9162
/proc/meminfo: memFree=254415028/264257624 swapFree=67111528/67111528
[pid=9154] ppid=9152 vsize=1172536 CPUtime=12.68 cores=0-31
/proc/9154/stat : 9154 (plingeling) R 9152 9154 9128 0 -1 4202496 291505 1246 0 0 1211 57 0 0 25 0 1 0 654289597 1200676864 291155 268435456000 4194304 4417452 140735282813712 18446744073709551615 4231321 0 0 4096 17442 0 0 0 17 8 0 0 0
/proc/9154/statm: 293134 291155 146 55 0 291020 0
Current children cumulated CPU time (s) 12.68
Current children cumulated vsize (KiB) 1172536

[startup+25.5025 s]
/proc/loadavg: 24.58 30.64 32.04 33/446 9194
/proc/meminfo: memFree=254648904/264257624 swapFree=67111528/67111528
[pid=9154] ppid=9152 vsize=1368540 CPUtime=398.67 cores=0-31
/proc/9154/stat : 9154 (plingeling) S 9152 9154 9128 0 -1 4202496 468720 1246 0 0 39625 242 0 0 19 0 33 0 654289597 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 8 0 0 0
/proc/9154/statm: 342135 225837 175 55 0 340021 0
[pid=9154/tid=9163] ppid=9152 vsize=1368540 CPUtime=12.08 cores=0-31
/proc/9154/task/9163/stat : 9163 (plingeling) R 9152 9154 9128 0 -1 4202560 5423 1246 0 0 1204 4 0 0 25 0 33 0 654290928 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364653 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=9154/tid=9164] ppid=9152 vsize=1368540 CPUtime=12.03 cores=0-31
/proc/9154/task/9164/stat : 9164 (plingeling) R 9152 9154 9128 0 -1 4202560 4512 1246 0 0 1199 4 0 0 25 0 33 0 654290928 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4236800 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9154/tid=9165] ppid=9152 vsize=1368540 CPUtime=12.05 cores=0-31
/proc/9154/task/9165/stat : 9165 (plingeling) R 9152 9154 9128 0 -1 4202560 3653 1246 0 0 1201 4 0 0 25 0 33 0 654290928 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=9154/tid=9166] ppid=9152 vsize=1368540 CPUtime=12.04 cores=0-31
/proc/9154/task/9166/stat : 9166 (plingeling) R 9152 9154 9128 0 -1 4202560 2803 1246 0 0 1199 5 0 0 25 0 33 0 654290928 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4281949 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=9154/tid=9167] ppid=9152 vsize=1368540 CPUtime=12.03 cores=0-31
/proc/9154/task/9167/stat : 9167 (plingeling) R 9152 9154 9128 0 -1 4202560 5476 1246 0 0 1197 6 0 0 25 0 33 0 654290928 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=9154/tid=9168] ppid=9152 vsize=1368540 CPUtime=12.04 cores=0-31
/proc/9154/task/9168/stat : 9168 (plingeling) R 9152 9154 9128 0 -1 4202560 6101 1246 0 0 1199 5 0 0 25 0 33 0 654290928 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=9154/tid=9169] ppid=9152 vsize=1368540 CPUtime=12.03 cores=0-31
/proc/9154/task/9169/stat : 9169 (plingeling) R 9152 9154 9128 0 -1 4202560 4005 1246 0 0 1197 6 0 0 25 0 33 0 654290928 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4236649 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9154/tid=9170] ppid=9152 vsize=1368540 CPUtime=12.01 cores=0-31
/proc/9154/task/9170/stat : 9170 (plingeling) R 9152 9154 9128 0 -1 4202560 4099 1246 0 0 1198 3 0 0 25 0 33 0 654290928 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=9154/tid=9171] ppid=9152 vsize=1368540 CPUtime=12.08 cores=0-31
/proc/9154/task/9171/stat : 9171 (plingeling) R 9152 9154 9128 0 -1 4202560 5406 1246 0 0 1204 4 0 0 25 0 33 0 654290928 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4251008 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=9154/tid=9172] ppid=9152 vsize=1368540 CPUtime=12.04 cores=0-31
/proc/9154/task/9172/stat : 9172 (plingeling) R 9152 9154 9128 0 -1 4202560 7063 1246 0 0 1196 8 0 0 25 0 33 0 654290929 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364650 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=9154/tid=9173] ppid=9152 vsize=1368540 CPUtime=12.04 cores=0-31
/proc/9154/task/9173/stat : 9173 (plingeling) R 9152 9154 9128 0 -1 4202560 9250 1246 0 0 1198 6 0 0 25 0 33 0 654290929 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4357024 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=9154/tid=9174] ppid=9152 vsize=1368540 CPUtime=11.96 cores=0-31
/proc/9154/task/9174/stat : 9174 (plingeling) R 9152 9154 9128 0 -1 4202560 4412 1246 0 0 1192 4 0 0 25 0 33 0 654290929 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9154/tid=9175] ppid=9152 vsize=1368540 CPUtime=11.89 cores=0-31
/proc/9154/task/9175/stat : 9175 (plingeling) R 9152 9154 9128 0 -1 4202560 6370 1246 0 0 1185 4 0 0 25 0 33 0 654290929 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4356995 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=9154/tid=9176] ppid=9152 vsize=1368540 CPUtime=12.01 cores=0-31
/proc/9154/task/9176/stat : 9176 (plingeling) R 9152 9154 9128 0 -1 4202560 4423 1246 0 0 1195 6 0 0 25 0 33 0 654290929 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364466 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=9154/tid=9177] ppid=9152 vsize=1368540 CPUtime=12.03 cores=0-31
/proc/9154/task/9177/stat : 9177 (plingeling) R 9152 9154 9128 0 -1 4202560 5376 1246 0 0 1197 6 0 0 25 0 33 0 654290929 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4265373 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=9154/tid=9178] ppid=9152 vsize=1368540 CPUtime=12.03 cores=0-31
/proc/9154/task/9178/stat : 9178 (plingeling) R 9152 9154 9128 0 -1 4202560 5783 1246 0 0 1197 6 0 0 25 0 33 0 654290929 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=9154/tid=9179] ppid=9152 vsize=1368540 CPUtime=12.05 cores=0-31
/proc/9154/task/9179/stat : 9179 (plingeling) R 9152 9154 9128 0 -1 4202560 4550 1246 0 0 1200 5 0 0 25 0 33 0 654290929 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364466 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9154/tid=9180] ppid=9152 vsize=1368540 CPUtime=12.05 cores=0-31
/proc/9154/task/9180/stat : 9180 (plingeling) R 9152 9154 9128 0 -1 4202560 5054 1246 0 0 1199 6 0 0 25 0 33 0 654290929 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4265465 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=9154/tid=9181] ppid=9152 vsize=1368540 CPUtime=12.03 cores=0-31
/proc/9154/task/9181/stat : 9181 (plingeling) R 9152 9154 9128 0 -1 4202560 9132 1246 0 0 1196 7 0 0 25 0 33 0 654290929 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=9154/tid=9182] ppid=9152 vsize=1368540 CPUtime=12.03 cores=0-31
/proc/9154/task/9182/stat : 9182 (plingeling) R 9152 9154 9128 0 -1 4202560 5168 1246 0 0 1196 7 0 0 25 0 33 0 654290929 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=9154/tid=9183] ppid=9152 vsize=1368540 CPUtime=12.04 cores=0-31
/proc/9154/task/9183/stat : 9183 (plingeling) R 9152 9154 9128 0 -1 4202560 4257 1246 0 0 1199 5 0 0 25 0 33 0 654290929 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4236616 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=9154/tid=9184] ppid=9152 vsize=1368540 CPUtime=12.08 cores=0-31
/proc/9154/task/9184/stat : 9184 (plingeling) R 9152 9154 9128 0 -1 4202560 5534 1246 0 0 1203 5 0 0 25 0 33 0 654290929 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364650 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9154/tid=9185] ppid=9152 vsize=1368540 CPUtime=12.05 cores=0-31
/proc/9154/task/9185/stat : 9185 (plingeling) R 9152 9154 9128 0 -1 4202560 4099 1246 0 0 1201 4 0 0 25 0 33 0 654290929 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4236649 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=9154/tid=9186] ppid=9152 vsize=1368540 CPUtime=12.05 cores=0-31
/proc/9154/task/9186/stat : 9186 (plingeling) R 9152 9154 9128 0 -1 4202560 4268 1246 0 0 1199 6 0 0 25 0 33 0 654290929 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=9154/tid=9187] ppid=9152 vsize=1368540 CPUtime=12.09 cores=0-31
/proc/9154/task/9187/stat : 9187 (plingeling) R 9152 9154 9128 0 -1 4202560 4544 1246 0 0 1206 3 0 0 25 0 33 0 654290930 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=9154/tid=9188] ppid=9152 vsize=1368540 CPUtime=12.06 cores=0-31
/proc/9154/task/9188/stat : 9188 (plingeling) R 9152 9154 9128 0 -1 4202560 5383 1246 0 0 1201 5 0 0 25 0 33 0 654290930 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=9154/tid=9189] ppid=9152 vsize=1368540 CPUtime=12.04 cores=0-31
/proc/9154/task/9189/stat : 9189 (plingeling) R 9152 9154 9128 0 -1 4202560 3775 1246 0 0 1200 4 0 0 25 0 33 0 654290930 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4232584 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=9154/tid=9190] ppid=9152 vsize=1368540 CPUtime=12.03 cores=0-31
/proc/9154/task/9190/stat : 9190 (plingeling) R 9152 9154 9128 0 -1 4202560 10334 1246 0 0 1195 8 0 0 25 0 33 0 654290930 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364604 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=9154/tid=9191] ppid=9152 vsize=1368540 CPUtime=12.09 cores=0-31
/proc/9154/task/9191/stat : 9191 (plingeling) R 9152 9154 9128 0 -1 4202560 4542 1246 0 0 1205 4 0 0 25 0 33 0 654290930 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=9154/tid=9192] ppid=9152 vsize=1368540 CPUtime=12.03 cores=0-31
/proc/9154/task/9192/stat : 9192 (plingeling) R 9152 9154 9128 0 -1 4202560 8898 1246 0 0 1195 8 0 0 25 0 33 0 654290930 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9154/tid=9193] ppid=9152 vsize=1368540 CPUtime=11.95 cores=0-31
/proc/9154/task/9193/stat : 9193 (plingeling) R 9152 9154 9128 0 -1 4202560 5344 1246 0 0 1191 4 0 0 25 0 33 0 654290930 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364466 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=9154/tid=9194] ppid=9152 vsize=1368540 CPUtime=12.02 cores=0-31
/proc/9154/task/9194/stat : 9194 (plingeling) R 9152 9154 9128 0 -1 4202560 8137 1246 0 0 1198 4 0 0 25 0 33 0 654290930 1401384960 225837 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 398.67
Current children cumulated vsize (KiB) 1368540

[startup+51.1024 s]
/proc/loadavg: 27.11 30.75 32.04 33/446 9194
/proc/meminfo: memFree=254593032/264257624 swapFree=67111528/67111528
[pid=9154] ppid=9152 vsize=1434264 CPUtime=1216.89 cores=0-31
/proc/9154/stat : 9154 (plingeling) S 9152 9154 9128 0 -1 4202496 507670 1246 0 0 121436 253 0 0 19 0 33 0 654289597 1468686336 238265 268435456000 4194304 4417452 140735282813712 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 8 0 0 0
/proc/9154/statm: 358566 238265 175 55 0 356452 0
[pid=9154/tid=9163] ppid=9152 vsize=1434264 CPUtime=37.67 cores=0-31
/proc/9154/task/9163/stat : 9163 (plingeling) R 9152 9154 9128 0 -1 4202560 5444 1246 0 0 3763 4 0 0 25 0 33 0 654290928 1468686336 238265 268435456000 4194304 4417452 140735282813712 18446744073709551615 4250957 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=9154/tid=9164] ppid=9152 vsize=1434264 CPUtime=37.62 cores=0-31
/proc/9154/task/9164/stat : 9164 (plingeling) R 9152 9154 9128 0 -1 4202560 4533 1246 0 0 3758 4 0 0 25 0 33 0 654290928 1468686336 238265 268435456000 4194304 4417452 140735282813712 18446744073709551615 4232666 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9154/tid=9165] ppid=9152 vsize=1434264 CPUtime=37.65 cores=0-31
/proc/9154/task/9165/stat : 9165 (plingeling) R 9152 9154 9128 0 -1 4202560 3688 1246 0 0 3761 4 0 0 25 0 33 0 654290928 1468686336 238265 268435456000 4194304 4417452 140735282813712 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=9154/tid=9166] ppid=9152 vsize=1434264 CPUtime=37.63 cores=0-31
/proc/9154/task/9166/stat : 9166 (plingeling) R 9152 9154 9128 0 -1 4202560 3105 1246 0 0 3758 5 0 0 25 0 33 0 654290928 1468686336 238265 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=9154/tid=9167] ppid=9152 vsize=1434264 CPUtime=37.62 cores=0-31
/proc/9154/task/9167/stat : 9167 (plingeling) R 9152 9154 9128 0 -1 4202560 6196 1246 0 0 3756 6 0 0 25 0 33 0 654290928 1468686336 238265 268435456000 4194304 4417452 140735282813712 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=9154/tid=9168] ppid=9152 vsize=1434264 CPUtime=36.9 cores=0-31
/proc/9154/task/9168/stat : 9168 (plingeling) R 9152 9154 9128 0 -1 4202560 6122 1246 0 0 3685 5 0 0 25 0 33 0 654290928 1468686336 238265 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=9154/tid=9169] ppid=9152 vsize=1434264 CPUtime=37.62 cores=0-31
/proc/9154/task/9169/stat : 9169 (plingeling) R 9152 9154 9128 0 -1 4202560 4246 1246 0 0 3756 6 0 0 25 0 33 0 654290928 1468686336 238265 268435456000 4194304 4417452 140735282813712 18446744073709551615 4236772 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9154/tid=9170] ppid=9152 vsize=1434264 CPUtime=37.59 cores=0-31
/proc/9154/task/9170/stat : 9170 (plingeling) R 9152 9154 9128 0 -1 4202560 5344 1246 0 0 3755 4 0 0 25 0 33 0 654290928 1468686336 238265 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364650 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=9154/tid=9171] ppid=9152 vsize=1434264 CPUtime=37.68 cores=0-31
/proc/9154/task/9171/stat : 9171 (plingeling) R 9152 9154 9128 0 -1 4202560 6653 1246 0 0 3763 5 0 0 25 0 33 0 654290928 1468686336 238265 268435456000 4194304 4417452 140735282813712 18446744073709551615 4357142 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=9154/tid=9172] ppid=9152 vsize=1434264 CPUtime=37.63 cores=0-31
/proc/9154/task/9172/stat : 9172 (plingeling) R 9152 9154 9128 0 -1 4202560 7083 1246 0 0 3755 8 0 0 25 0 33 0 654290929 1468686336 238265 268435456000 4194304 4417452 140735282813712 18446744073709551615 4357970 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=9154/tid=9173] ppid=9152 vsize=1434264 CPUtime=37.63 cores=0-31

################
# More data... #
################

/proc/9154/task/9168/stat : 9168 (plingeling) R 9152 9154 9128 0 -1 4202560 506117 1246 0 0 493769 181 0 0 25 0 33 0 654290928 8553324544 1975133 268435456000 4194304 4417452 140735282813712 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=9154/tid=9169] ppid=9152 vsize=8352856 CPUtime=4939.74 cores=0-31
/proc/9154/task/9169/stat : 9169 (plingeling) R 9152 9154 9128 0 -1 4202560 262938 1246 0 0 493785 189 0 0 25 0 33 0 654290928 8553324544 1975133 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=9154/tid=9170] ppid=9152 vsize=8352856 CPUtime=4943.26 cores=0-31
/proc/9154/task/9170/stat : 9170 (plingeling) R 9152 9154 9128 0 -1 4202560 303283 1246 0 0 494163 163 0 0 25 0 33 0 654290928 8553324544 1975133 268435456000 4194304 4417452 140735282813712 18446744073709551615 4346484 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9154/tid=9171] ppid=9152 vsize=8352856 CPUtime=4940.23 cores=0-31
/proc/9154/task/9171/stat : 9171 (plingeling) R 9152 9154 9128 0 -1 4202560 269803 1246 0 0 493921 102 0 0 25 0 33 0 654290928 8553324544 1975133 268435456000 4194304 4417452 140735282813712 18446744073709551615 4310641 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=9154/tid=9172] ppid=9152 vsize=8352856 CPUtime=4947.57 cores=0-31
/proc/9154/task/9172/stat : 9172 (plingeling) R 9152 9154 9128 0 -1 4202560 665639 1246 0 0 494469 288 0 0 25 0 33 0 654290929 8553324544 1975133 268435456000 4194304 4417452 140735282813712 18446744073709551615 4366065 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=9154/tid=9173] ppid=9152 vsize=8352856 CPUtime=4930.15 cores=0-31
/proc/9154/task/9173/stat : 9173 (plingeling) R 9152 9154 9128 0 -1 4202560 357576 1246 0 0 492665 350 0 0 25 0 33 0 654290929 8553324544 1975133 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=9154/tid=9174] ppid=9152 vsize=8352856 CPUtime=4947.18 cores=0-31
/proc/9154/task/9174/stat : 9174 (plingeling) R 9152 9154 9128 0 -1 4202560 251005 1246 0 0 494628 90 0 0 25 0 33 0 654290929 8553324544 1975133 268435456000 4194304 4417452 140735282813712 18446744073709551615 4357263 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=9154/tid=9175] ppid=9152 vsize=8352856 CPUtime=4937.34 cores=0-31
/proc/9154/task/9175/stat : 9175 (plingeling) R 9152 9154 9128 0 -1 4202560 346359 1246 0 0 493597 137 0 0 25 0 33 0 654290929 8553324544 1975133 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9154/tid=9176] ppid=9152 vsize=8352856 CPUtime=4945.89 cores=0-31
/proc/9154/task/9176/stat : 9176 (plingeling) R 9152 9154 9128 0 -1 4202560 238442 1246 0 0 494370 219 0 0 25 0 33 0 654290929 8553324544 1975133 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=9154/tid=9177] ppid=9152 vsize=8352856 CPUtime=4934.47 cores=0-31
/proc/9154/task/9177/stat : 9177 (plingeling) R 9152 9154 9128 0 -1 4202560 592511 1246 0 0 493282 165 0 0 25 0 33 0 654290929 8553324544 1975133 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364807 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9154/tid=9178] ppid=9152 vsize=8352856 CPUtime=4938.1 cores=0-31
/proc/9154/task/9178/stat : 9178 (plingeling) R 9152 9154 9128 0 -1 4202560 251752 1246 0 0 493598 212 0 0 25 0 33 0 654290929 8553324544 1975133 268435456000 4194304 4417452 140735282813712 18446744073709551615 4263080 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=9154/tid=9179] ppid=9152 vsize=8352856 CPUtime=4947.38 cores=0-31
/proc/9154/task/9179/stat : 9179 (plingeling) R 9152 9154 9128 0 -1 4202560 194819 1246 0 0 494562 176 0 0 25 0 33 0 654290929 8553324544 1975133 268435456000 4194304 4417452 140735282813712 18446744073709551615 4365058 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=9154/tid=9180] ppid=9152 vsize=8352856 CPUtime=4946.3 cores=0-31
/proc/9154/task/9180/stat : 9180 (plingeling) R 9152 9154 9128 0 -1 4202560 207551 1246 0 0 494500 130 0 0 25 0 33 0 654290929 8553324544 1975133 268435456000 4194304 4417452 140735282813712 18446744073709551615 4356995 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=9154/tid=9181] ppid=9152 vsize=8352856 CPUtime=4946.83 cores=0-31
/proc/9154/task/9181/stat : 9181 (plingeling) R 9152 9154 9128 0 -1 4202560 509553 1246 0 0 494392 291 0 0 25 0 33 0 654290929 8553324544 1975133 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364488 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=9154/tid=9182] ppid=9152 vsize=8352856 CPUtime=4945.8 cores=0-31
/proc/9154/task/9182/stat : 9182 (plingeling) R 9152 9154 9128 0 -1 4202560 418126 1246 0 0 494276 304 0 0 25 0 33 0 654290929 8553324544 1975133 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=9154/tid=9183] ppid=9152 vsize=8352856 CPUtime=4930.66 cores=0-31
/proc/9154/task/9183/stat : 9183 (plingeling) R 9152 9154 9128 0 -1 4202560 310173 1246 0 0 492913 153 0 0 25 0 33 0 654290929 8553324544 1975133 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=9154/tid=9184] ppid=9152 vsize=8352856 CPUtime=4947.41 cores=0-31
/proc/9154/task/9184/stat : 9184 (plingeling) R 9152 9154 9128 0 -1 4202560 571886 1246 0 0 494517 224 0 0 25 0 33 0 654290929 8553324544 1975133 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9154/tid=9185] ppid=9152 vsize=8352856 CPUtime=4946.63 cores=0-31
/proc/9154/task/9185/stat : 9185 (plingeling) R 9152 9154 9128 0 -1 4202560 601400 1246 0 0 494115 548 0 0 25 0 33 0 654290929 8553324544 1975133 268435456000 4194304 4417452 140735282813712 18446744073709551615 4251091 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=9154/tid=9186] ppid=9152 vsize=8352856 CPUtime=4947.2 cores=0-31
/proc/9154/task/9186/stat : 9186 (plingeling) R 9152 9154 9128 0 -1 4202560 232010 1246 0 0 494625 95 0 0 25 0 33 0 654290929 8553324544 1975133 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364572 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9154/tid=9187] ppid=9152 vsize=8352856 CPUtime=4947.53 cores=0-31
/proc/9154/task/9187/stat : 9187 (plingeling) R 9152 9154 9128 0 -1 4202560 282068 1246 0 0 494570 183 0 0 25 0 33 0 654290930 8553324544 1975133 268435456000 4194304 4417452 140735282813712 18446744073709551615 4263213 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=9154/tid=9188] ppid=9152 vsize=8352856 CPUtime=4938.88 cores=0-31
/proc/9154/task/9188/stat : 9188 (plingeling) R 9152 9154 9128 0 -1 4202560 369589 1246 0 0 493588 300 0 0 25 0 33 0 654290930 8553324544 1975133 268435456000 4194304 4417452 140735282813712 18446744073709551615 4311411 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=9154/tid=9189] ppid=9152 vsize=8352856 CPUtime=4947.3 cores=0-31
/proc/9154/task/9189/stat : 9189 (plingeling) R 9152 9154 9128 0 -1 4202560 354195 1246 0 0 494565 165 0 0 25 0 33 0 654290930 8553324544 1975133 268435456000 4194304 4417452 140735282813712 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=9154/tid=9190] ppid=9152 vsize=8352856 CPUtime=4945.62 cores=0-31
/proc/9154/task/9190/stat : 9190 (plingeling) R 9152 9154 9128 0 -1 4202560 347957 1246 0 0 494369 193 0 0 25 0 33 0 654290930 8553324544 1975133 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=9154/tid=9191] ppid=9152 vsize=8352856 CPUtime=4933.07 cores=0-31
/proc/9154/task/9191/stat : 9191 (plingeling) R 9152 9154 9128 0 -1 4202560 243181 1246 0 0 493167 140 0 0 25 0 33 0 654290930 8553324544 1975133 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=9154/tid=9192] ppid=9152 vsize=8352856 CPUtime=4943.79 cores=0-31
/proc/9154/task/9192/stat : 9192 (plingeling) R 9152 9154 9128 0 -1 4202560 335841 1246 0 0 494246 133 0 0 25 0 33 0 654290930 8553324544 1975133 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9154/tid=9193] ppid=9152 vsize=8352856 CPUtime=4947.04 cores=0-31
/proc/9154/task/9193/stat : 9193 (plingeling) R 9152 9154 9128 0 -1 4202560 268029 1246 0 0 494548 156 0 0 25 0 33 0 654290930 8553324544 1975133 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=9154/tid=9194] ppid=9152 vsize=8352856 CPUtime=4940.21 cores=0-31
/proc/9154/task/9194/stat : 9194 (plingeling) R 9152 9154 9128 0 -1 4202560 395331 1246 0 0 493827 194 0 0 25 0 33 0 654290930 8553324544 1975133 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 28 0 0 0
Current children cumulated CPU time (s) 158183
Current children cumulated vsize (KiB) 8352856



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5019.2 s]
/proc/loadavg: 32.06 32.06 32.08 33/446 9347
/proc/meminfo: memFree=247615392/264257624 swapFree=67111528/67111528
[pid=9154] ppid=9152 vsize=8482052 CPUtime=160002 cores=0-31
/proc/9154/stat : 9154 (plingeling) S 9152 9154 9128 0 -1 4202496 13091306 1246 0 0 15992003 8170 0 0 19 0 33 0 654289597 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 8 0 0 0
/proc/9154/statm: 2120513 1987321 175 55 0 2118399 0
[pid=9154/tid=9163] ppid=9152 vsize=8482052 CPUtime=5004.48 cores=0-31
/proc/9154/task/9163/stat : 9163 (plingeling) R 9152 9154 9128 0 -1 4202560 305767 1246 0 0 500280 168 0 0 25 0 33 0 654290928 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=9154/tid=9164] ppid=9152 vsize=8482052 CPUtime=5004.26 cores=0-31
/proc/9154/task/9164/stat : 9164 (plingeling) R 9152 9154 9128 0 -1 4202560 1036841 1246 0 0 499689 737 0 0 25 0 33 0 654290928 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4346677 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9154/tid=9165] ppid=9152 vsize=8482052 CPUtime=4992.75 cores=0-31
/proc/9154/task/9165/stat : 9165 (plingeling) R 9152 9154 9128 0 -1 4202560 584641 1246 0 0 499042 233 0 0 25 0 33 0 654290928 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364650 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=9154/tid=9166] ppid=9152 vsize=8482052 CPUtime=5003.76 cores=0-31
/proc/9154/task/9166/stat : 9166 (plingeling) R 9152 9154 9128 0 -1 4202560 621800 1246 0 0 499430 946 0 0 25 0 33 0 654290928 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=9154/tid=9167] ppid=9152 vsize=8482052 CPUtime=5003.38 cores=0-31
/proc/9154/task/9167/stat : 9167 (plingeling) R 9152 9154 9128 0 -1 4202560 397929 1246 0 0 499975 363 0 0 25 0 33 0 654290928 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=9154/tid=9168] ppid=9152 vsize=8482052 CPUtime=4996.34 cores=0-31
/proc/9154/task/9168/stat : 9168 (plingeling) R 9152 9154 9128 0 -1 4202560 508132 1246 0 0 499452 182 0 0 25 0 33 0 654290928 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4281633 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=9154/tid=9169] ppid=9152 vsize=8482052 CPUtime=4996.62 cores=0-31
/proc/9154/task/9169/stat : 9169 (plingeling) R 9152 9154 9128 0 -1 4202560 262938 1246 0 0 499472 190 0 0 25 0 33 0 654290928 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4354937 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=9154/tid=9170] ppid=9152 vsize=8482052 CPUtime=5000.17 cores=0-31
/proc/9154/task/9170/stat : 9170 (plingeling) R 9152 9154 9128 0 -1 4202560 317855 1246 0 0 499824 193 0 0 25 0 33 0 654290928 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9154/tid=9171] ppid=9152 vsize=8482052 CPUtime=4995.52 cores=0-31
/proc/9154/task/9171/stat : 9171 (plingeling) R 9152 9154 9128 0 -1 4202560 278048 1246 0 0 499427 125 0 0 25 0 33 0 654290928 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4354506 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=9154/tid=9172] ppid=9152 vsize=8482052 CPUtime=5004.47 cores=0-31
/proc/9154/task/9172/stat : 9172 (plingeling) R 9152 9154 9128 0 -1 4202560 678310 1246 0 0 500157 290 0 0 25 0 33 0 654290929 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=9154/tid=9173] ppid=9152 vsize=8482052 CPUtime=4987.06 cores=0-31
/proc/9154/task/9173/stat : 9173 (plingeling) R 9152 9154 9128 0 -1 4202560 360219 1246 0 0 498355 351 0 0 25 0 33 0 654290929 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=9154/tid=9174] ppid=9152 vsize=8482052 CPUtime=5003.84 cores=0-31
/proc/9154/task/9174/stat : 9174 (plingeling) R 9152 9154 9128 0 -1 4202560 258176 1246 0 0 500292 92 0 0 25 0 33 0 654290929 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4357283 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=9154/tid=9175] ppid=9152 vsize=8482052 CPUtime=4994.25 cores=0-31
/proc/9154/task/9175/stat : 9175 (plingeling) R 9152 9154 9128 0 -1 4202560 346359 1246 0 0 499288 137 0 0 25 0 33 0 654290929 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4237176 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9154/tid=9176] ppid=9152 vsize=8482052 CPUtime=5002.65 cores=0-31
/proc/9154/task/9176/stat : 9176 (plingeling) R 9152 9154 9128 0 -1 4202560 238442 1246 0 0 500046 219 0 0 25 0 33 0 654290929 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=9154/tid=9177] ppid=9152 vsize=8482052 CPUtime=4991.38 cores=0-31
/proc/9154/task/9177/stat : 9177 (plingeling) R 9152 9154 9128 0 -1 4202560 599764 1246 0 0 498968 170 0 0 25 0 33 0 654290929 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9154/tid=9178] ppid=9152 vsize=8482052 CPUtime=4995.01 cores=0-31
/proc/9154/task/9178/stat : 9178 (plingeling) R 9152 9154 9128 0 -1 4202560 256644 1246 0 0 499284 217 0 0 25 0 33 0 654290929 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=9154/tid=9179] ppid=9152 vsize=8482052 CPUtime=5004.27 cores=0-31
/proc/9154/task/9179/stat : 9179 (plingeling) R 9152 9154 9128 0 -1 4202560 194819 1246 0 0 500251 176 0 0 25 0 33 0 654290929 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=9154/tid=9180] ppid=9152 vsize=8482052 CPUtime=5003.21 cores=0-31
/proc/9154/task/9180/stat : 9180 (plingeling) R 9152 9154 9128 0 -1 4202560 208434 1246 0 0 500191 130 0 0 25 0 33 0 654290929 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4237290 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=9154/tid=9181] ppid=9152 vsize=8482052 CPUtime=5003.75 cores=0-31
/proc/9154/task/9181/stat : 9181 (plingeling) R 9152 9154 9128 0 -1 4202560 509565 1246 0 0 500083 292 0 0 25 0 33 0 654290929 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=9154/tid=9182] ppid=9152 vsize=8482052 CPUtime=5002.71 cores=0-31
/proc/9154/task/9182/stat : 9182 (plingeling) R 9152 9154 9128 0 -1 4202560 421746 1246 0 0 499966 305 0 0 25 0 33 0 654290929 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4357207 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=9154/tid=9183] ppid=9152 vsize=8482052 CPUtime=4987.57 cores=0-31
/proc/9154/task/9183/stat : 9183 (plingeling) R 9152 9154 9128 0 -1 4202560 310173 1246 0 0 498604 153 0 0 25 0 33 0 654290929 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=9154/tid=9184] ppid=9152 vsize=8482052 CPUtime=5004.32 cores=0-31
/proc/9154/task/9184/stat : 9184 (plingeling) R 9152 9154 9128 0 -1 4202560 571898 1246 0 0 500208 224 0 0 25 0 33 0 654290929 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364807 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9154/tid=9185] ppid=9152 vsize=8482052 CPUtime=5003.54 cores=0-31
/proc/9154/task/9185/stat : 9185 (plingeling) R 9152 9154 9128 0 -1 4202560 614785 1246 0 0 499780 574 0 0 25 0 33 0 654290929 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=9154/tid=9186] ppid=9152 vsize=8482052 CPUtime=5004.11 cores=0-31
/proc/9154/task/9186/stat : 9186 (plingeling) R 9152 9154 9128 0 -1 4202560 232010 1246 0 0 500316 95 0 0 25 0 33 0 654290929 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364587 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9154/tid=9187] ppid=9152 vsize=8482052 CPUtime=5004.44 cores=0-31
/proc/9154/task/9187/stat : 9187 (plingeling) R 9152 9154 9128 0 -1 4202560 290036 1246 0 0 500239 205 0 0 25 0 33 0 654290930 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=9154/tid=9188] ppid=9152 vsize=8482052 CPUtime=4995.79 cores=0-31
/proc/9154/task/9188/stat : 9188 (plingeling) R 9152 9154 9128 0 -1 4202560 378931 1246 0 0 499266 313 0 0 25 0 33 0 654290930 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=9154/tid=9189] ppid=9152 vsize=8482052 CPUtime=5004.21 cores=0-31
/proc/9154/task/9189/stat : 9189 (plingeling) R 9152 9154 9128 0 -1 4202560 354195 1246 0 0 500256 165 0 0 25 0 33 0 654290930 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=9154/tid=9190] ppid=9152 vsize=8482052 CPUtime=5002.53 cores=0-31
/proc/9154/task/9190/stat : 9190 (plingeling) R 9152 9154 9128 0 -1 4202560 347957 1246 0 0 500060 193 0 0 25 0 33 0 654290930 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=9154/tid=9191] ppid=9152 vsize=8482052 CPUtime=4989.98 cores=0-31
/proc/9154/task/9191/stat : 9191 (plingeling) R 9152 9154 9128 0 -1 4202560 305325 1246 0 0 498845 153 0 0 25 0 33 0 654290930 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=9154/tid=9192] ppid=9152 vsize=8482052 CPUtime=5000.7 cores=0-31
/proc/9154/task/9192/stat : 9192 (plingeling) R 9152 9154 9128 0 -1 4202560 344033 1246 0 0 499915 155 0 0 25 0 33 0 654290930 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9154/tid=9193] ppid=9152 vsize=8482052 CPUtime=5003.95 cores=0-31
/proc/9154/task/9193/stat : 9193 (plingeling) R 9152 9154 9128 0 -1 4202560 268657 1246 0 0 500238 157 0 0 25 0 33 0 654290930 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=9154/tid=9194] ppid=9152 vsize=8482052 CPUtime=4997.11 cores=0-31
/proc/9154/task/9194/stat : 9194 (plingeling) R 9152 9154 9128 0 -1 4202560 395331 1246 0 0 499517 194 0 0 25 0 33 0 654290930 8685621248 1987321 268435456000 4194304 4417452 140735282813712 18446744073709551615 4355830 0 0 4096 17442 0 0 0 -1 28 0 0 0
Current children cumulated CPU time (s) 160002
Current children cumulated vsize (KiB) 8482052

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5019.5 s]
/proc/loadavg: 32.06 32.06 32.08 2/416 9348
/proc/meminfo: memFree=248681668/264257624 swapFree=67111528/67111528
[pid=9154] ppid=9152 vsize=0 CPUtime=160002 cores=0-31
/proc/9154/stat : 9154 (plingeling) Z 9152 9154 9128 0 -1 4203548 13091306 1246 0 0 15992025 8197 0 0 18 0 2 0 654289597 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 8 0 0 0
/proc/9154/statm: 0 0 0 0 0 0 0
[pid=9154/tid=9163] ppid=9152 vsize=0 CPUtime=5004.75 cores=0-31
/proc/9154/task/9163/stat : 9163 (plingeling) R 9152 9154 9128 0 -1 4203588 305767 1246 0 0 500280 195 0 0 25 0 2 0 654290928 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 8 0 0 0
Current children cumulated CPU time (s) 160002
Current children cumulated vsize (KiB) 0

[startup+5019.7 s]
/proc/loadavg: 32.06 32.06 32.08 2/416 9348
/proc/meminfo: memFree=248681668/264257624 swapFree=67111528/67111528
[pid=9154] ppid=9152 vsize=0 CPUtime=160002 cores=0-31
/proc/9154/stat : 9154 (plingeling) Z 9152 9154 9128 0 -1 4203548 13091306 1246 0 0 15992025 8217 0 0 18 0 2 0 654289597 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 8 0 0 0
/proc/9154/statm: 0 0 0 0 0 0 0
[pid=9154/tid=9163] ppid=9152 vsize=0 CPUtime=5004.95 cores=0-31
/proc/9154/task/9163/stat : 9163 (plingeling) R 9152 9154 9128 0 -1 4203588 305767 1246 0 0 500280 215 0 0 25 0 2 0 654290928 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 8 0 0 0
Current children cumulated CPU time (s) 160002
Current children cumulated vsize (KiB) 0

[startup+5019.8 s]
/proc/loadavg: 32.06 32.06 32.08 2/416 9348
/proc/meminfo: memFree=248681668/264257624 swapFree=67111528/67111528
[pid=9154] ppid=9152 vsize=0 CPUtime=160003 cores=0-31
/proc/9154/stat : 9154 (plingeling) Z 9152 9154 9128 0 -1 4203548 13091306 1246 0 0 15992025 8227 0 0 18 0 2 0 654289597 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 8 0 0 0
/proc/9154/statm: 0 0 0 0 0 0 0
[pid=9154/tid=9163] ppid=9152 vsize=0 CPUtime=5005.05 cores=0-31
/proc/9154/task/9163/stat : 9163 (plingeling) R 9152 9154 9128 0 -1 4203588 305767 1246 0 0 500280 225 0 0 25 0 2 0 654290928 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 8 0 0 0
Current children cumulated CPU time (s) 160003
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5019.86
CPU time (s): 160003
CPU user time (s): 159920
CPU system time (s): 82.3345
CPU usage (%): 3187.39
Max. virtual memory (cumulated for all children) (KiB): 8733836

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 159920
system time used= 82.3345
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 13092552
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 11697
involuntary context switches= 403183

runsolver used 27.6798 second user time and 114.789 second system time

The end

Launcher Data 

Begin job on node200 at 2011-05-15 04:53:52
IDJOB=3372114
IDBENCH=20386
IDSOLVER=1852
FILE ID=node200/3372114-1305428031
RUNJOBID= node200-1305428031-9142
PBS_JOBID= 13285501
Free space on /tmp= 62040 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT07/industrial/anbulagan/hard-unsat/partial-5-11-u.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3372114-1305428031/watcher-3372114-1305428031 -o /tmp/evaluation-result-3372114-1305428031/solver-3372114-1305428031 -C 160000 -W 5100 -M 266338304  plingeling -t 32 HOME/instance-3372114-1305428031.cnf

TIME LIMIT= 160000 seconds
MEMORY LIMIT= 266338304 MiB
NBCORE= 32

MD5SUM BENCH= 8e22d84b1ae625d44fbf300a32374a2d
RANDOM SEED=406508206

node200.alineos.net Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 64
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 96
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 66
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 98
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 68
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 100
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 70
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.98
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 102
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 16
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 17
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 48
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 18
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 80
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 19
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 112
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 20
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 21
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 50
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 22
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 82
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 114
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.23
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 24
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 25
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 26
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 84
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 27
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 116
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 28
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 29
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 54
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 30
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 86
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 31
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 118
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     264257624 kB
MemFree:      255580040 kB
Buffers:        439072 kB
Cached:        7841460 kB
SwapCached:          0 kB
Active:        7390820 kB
Inactive:       958244 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     264257624 kB
LowFree:      255580040 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           13092 kB
Writeback:           4 kB
AnonPages:       69428 kB
Mapped:          13756 kB
Slab:           250432 kB
PageTables:       4860 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  199240340 kB
Committed_AS:   181568 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296372 kB
VmallocChunk: 34359440603 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 62036 MiB
End job on node200 at 2011-05-15 06:17:33