Trace number 3372129

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 160002 5014.04

General information on the benchmark

Nameindustrial/palacios/hard/
sortnet-7-ipc5-h15-unsat.cnf
MD5SUM0683489092a5eb0c881dd17c2c45d08e
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables105623
Number of clauses374254
Sum of the clauses size911890
Maximum clause length43
Minimum clause length1
Number of clauses of size 1128
Number of clauses of size 2269133
Number of clauses of size 399944
Number of clauses of size 4407
Number of clauses of size 5309
Number of clauses of size over 54333

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3372129-1305442023/watcher-3372129-1305442023 -o /tmp/evaluation-result-3372129-1305442023/solver-3372129-1305442023 -C 160000 -W 5100 -M 266338304 plingeling -t 32 HOME/instance-3372129-1305442023.cnf 

running on 32 cores: 0,4,8,12,16,20,24,28,1,5,9,13,17,21,25,29,2,6,10,14,18,22,26,30,3,7,11,15,19,23,27,31

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 160000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 160030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 272730423296 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 272730474496 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 29.44 31.41 31.77 1/413 9986
/proc/meminfo: memFree=255576932/264257624 swapFree=67111528/67111528
[pid=9986] ppid=9984 vsize=13860 CPUtime=0.02 cores=0-31
/proc/9986/stat : 9986 (plingeling) R 9984 9986 9960 0 -1 4202496 1523 1238 0 0 2 0 0 0 20 0 1 0 655688762 14192640 1454 268435456000 4194304 4417452 140733254012832 18446744073709551615 258650649806 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/9986/statm: 3465 1456 145 55 0 1351 0

[startup+0.066917 s]
/proc/loadavg: 29.44 31.41 31.77 1/413 9986
/proc/meminfo: memFree=255576932/264257624 swapFree=67111528/67111528
[pid=9986] ppid=9984 vsize=16408 CPUtime=0.05 cores=0-31
/proc/9986/stat : 9986 (plingeling) R 9984 9986 9960 0 -1 4202496 2151 1238 0 0 5 0 0 0 20 0 1 0 655688762 16801792 2082 268435456000 4194304 4417452 140733254012832 18446744073709551615 258650653097 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/9986/statm: 4102 2092 145 55 0 1988 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 16408

[startup+0.100917 s]
/proc/loadavg: 29.44 31.41 31.77 1/413 9986
/proc/meminfo: memFree=255576932/264257624 swapFree=67111528/67111528
[pid=9986] ppid=9984 vsize=18752 CPUtime=0.09 cores=0-31
/proc/9986/stat : 9986 (plingeling) R 9984 9986 9960 0 -1 4202496 2746 1238 0 0 8 1 0 0 20 0 1 0 655688762 19202048 2677 268435456000 4194304 4417452 140733254012832 18446744073709551615 4227779 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/9986/statm: 4688 2677 145 55 0 2574 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 18752

[startup+0.300941 s]
/proc/loadavg: 29.44 31.41 31.77 1/413 9986
/proc/meminfo: memFree=255576932/264257624 swapFree=67111528/67111528
[pid=9986] ppid=9984 vsize=31448 CPUtime=0.29 cores=0-31
/proc/9986/stat : 9986 (plingeling) R 9984 9986 9960 0 -1 4202496 5953 1238 0 0 27 2 0 0 21 0 1 0 655688762 32202752 5884 268435456000 4194304 4417452 140733254012832 18446744073709551615 4231321 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/9986/statm: 7862 5884 146 55 0 5748 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 31448

[startup+0.700928 s]
/proc/loadavg: 29.44 31.41 31.77 1/413 9986
/proc/meminfo: memFree=255576932/264257624 swapFree=67111528/67111528
[pid=9986] ppid=9984 vsize=56824 CPUtime=0.68 cores=0-31
/proc/9986/stat : 9986 (plingeling) R 9984 9986 9960 0 -1 4202496 12235 1238 0 0 65 3 0 0 22 0 1 0 655688762 58187776 12166 268435456000 4194304 4417452 140733254012832 18446744073709551615 258650649933 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/9986/statm: 14206 12181 146 55 0 12092 0
Current children cumulated CPU time (s) 0.68
Current children cumulated vsize (KiB) 56824

[startup+1.50095 s]
/proc/loadavg: 27.16 30.91 31.61 2/414 9992
/proc/meminfo: memFree=255499136/264257624 swapFree=67111528/67111528
[pid=9986] ppid=9984 vsize=155916 CPUtime=1.48 cores=0-31
/proc/9986/stat : 9986 (plingeling) R 9984 9986 9960 0 -1 4202496 37070 1238 0 0 140 8 0 0 25 0 1 0 655688762 159657984 37001 268435456000 4194304 4417452 140733254012832 18446744073709551615 4259618 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/9986/statm: 38979 37001 146 55 0 36865 0
Current children cumulated CPU time (s) 1.48
Current children cumulated vsize (KiB) 155916

[startup+3.10102 s]
/proc/loadavg: 27.16 30.91 31.61 2/414 9992
/proc/meminfo: memFree=255423372/264257624 swapFree=67111528/67111528
[pid=9986] ppid=9984 vsize=300996 CPUtime=3.09 cores=0-31
/proc/9986/stat : 9986 (plingeling) R 9984 9986 9960 0 -1 4202496 73395 1238 0 0 293 16 0 0 25 0 1 0 655688762 308219904 73271 268435456000 4194304 4417452 140733254012832 18446744073709551615 258650955760 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/9986/statm: 75249 73271 146 55 0 73135 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 300996

[startup+6.30113 s]
/proc/loadavg: 27.16 30.91 31.61 2/414 9992
/proc/meminfo: memFree=254992840/264257624 swapFree=67111528/67111528
[pid=9986] ppid=9984 vsize=592292 CPUtime=6.29 cores=0-31
/proc/9986/stat : 9986 (plingeling) R 9984 9986 9960 0 -1 4202496 146219 1238 0 0 598 31 0 0 25 0 1 0 655688762 606507008 146095 268435456000 4194304 4417452 140733254012832 18446744073709551615 4231321 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/9986/statm: 148073 146095 146 55 0 145959 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 592292

[startup+12.7022 s]
/proc/loadavg: 24.58 30.22 31.37 31/446 10024
/proc/meminfo: memFree=254432056/264257624 swapFree=67111528/67111528
[pid=9986] ppid=9984 vsize=1575040 CPUtime=172.23 cores=0-31
/proc/9986/stat : 9986 (plingeling) S 9984 9986 9960 0 -1 4202496 494955 1238 0 0 16789 434 0 0 18 0 33 0 655688762 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 25 0 0 0
/proc/9986/statm: 393760 287753 173 55 0 391646 0
[pid=9986/tid=9993] ppid=9984 vsize=1575040 CPUtime=5.3 cores=0-31
/proc/9986/task/9993/stat : 9993 (plingeling) R 9984 9986 9960 0 -1 4202560 8044 1238 0 0 522 8 0 0 21 0 33 0 655689488 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4261096 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=9986/tid=9994] ppid=9984 vsize=1575040 CPUtime=5.28 cores=0-31
/proc/9986/task/9994/stat : 9994 (plingeling) R 9984 9986 9960 0 -1 4202560 11225 1238 0 0 517 11 0 0 25 0 33 0 655689488 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4255698 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9986/tid=9995] ppid=9984 vsize=1575040 CPUtime=5.25 cores=0-31
/proc/9986/task/9995/stat : 9995 (plingeling) R 9984 9986 9960 0 -1 4202560 11096 1238 0 0 516 9 0 0 25 0 33 0 655689489 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4346484 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=9986/tid=9996] ppid=9984 vsize=1575040 CPUtime=5.35 cores=0-31
/proc/9986/task/9996/stat : 9996 (plingeling) R 9984 9986 9960 0 -1 4202560 2267 1238 0 0 533 2 0 0 25 0 33 0 655689489 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=9986/tid=9997] ppid=9984 vsize=1575040 CPUtime=4.84 cores=0-31
/proc/9986/task/9997/stat : 9997 (plingeling) R 9984 9986 9960 0 -1 4202560 11596 1238 0 0 468 16 0 0 21 0 33 0 655689489 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4234242 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=9986/tid=9998] ppid=9984 vsize=1575040 CPUtime=5.19 cores=0-31
/proc/9986/task/9998/stat : 9998 (plingeling) R 9984 9986 9960 0 -1 4202560 5681 1238 0 0 512 7 0 0 25 0 33 0 655689489 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4234314 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=9986/tid=9999] ppid=9984 vsize=1575040 CPUtime=4.91 cores=0-31
/proc/9986/task/9999/stat : 9999 (plingeling) R 9984 9986 9960 0 -1 4202560 10372 1238 0 0 484 7 0 0 21 0 33 0 655689489 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4346488 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=9986/tid=10000] ppid=9984 vsize=1575040 CPUtime=5.2 cores=0-31
/proc/9986/task/10000/stat : 10000 (plingeling) R 9984 9986 9960 0 -1 4202560 10853 1238 0 0 513 7 0 0 25 0 33 0 655689489 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4259008 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=9986/tid=10001] ppid=9984 vsize=1575040 CPUtime=4.91 cores=0-31
/proc/9986/task/10001/stat : 10001 (plingeling) R 9984 9986 9960 0 -1 4202560 16231 1238 0 0 473 18 0 0 21 0 33 0 655689489 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4270927 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=9986/tid=10002] ppid=9984 vsize=1575040 CPUtime=5.26 cores=0-31
/proc/9986/task/10002/stat : 10002 (plingeling) R 9984 9986 9960 0 -1 4202560 13295 1238 0 0 512 14 0 0 25 0 33 0 655689489 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4256047 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=9986/tid=10003] ppid=9984 vsize=1575040 CPUtime=4.94 cores=0-31
/proc/9986/task/10003/stat : 10003 (plingeling) R 9984 9986 9960 0 -1 4202560 8569 1238 0 0 478 16 0 0 22 0 33 0 655689489 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4233483 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=9986/tid=10004] ppid=9984 vsize=1575040 CPUtime=5.35 cores=0-31
/proc/9986/task/10004/stat : 10004 (plingeling) R 9984 9986 9960 0 -1 4202560 13591 1238 0 0 523 12 0 0 25 0 33 0 655689489 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4274913 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9986/tid=10005] ppid=9984 vsize=1575040 CPUtime=5.16 cores=0-31
/proc/9986/task/10005/stat : 10005 (plingeling) R 9984 9986 9960 0 -1 4202560 11700 1238 0 0 502 14 0 0 18 0 33 0 655689489 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4270966 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=9986/tid=10006] ppid=9984 vsize=1575040 CPUtime=4.91 cores=0-31
/proc/9986/task/10006/stat : 10006 (plingeling) R 9984 9986 9960 0 -1 4202560 12447 1238 0 0 482 9 0 0 22 0 33 0 655689490 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4261142 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=9986/tid=10007] ppid=9984 vsize=1575040 CPUtime=5.15 cores=0-31
/proc/9986/task/10007/stat : 10007 (plingeling) R 9984 9986 9960 0 -1 4202560 10643 1238 0 0 502 13 0 0 25 0 33 0 655689490 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4302029 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=9986/tid=10008] ppid=9984 vsize=1575040 CPUtime=5.29 cores=0-31
/proc/9986/task/10008/stat : 10008 (plingeling) R 9984 9986 9960 0 -1 4202560 6087 1238 0 0 523 6 0 0 25 0 33 0 655689490 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4302058 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=9986/tid=10009] ppid=9984 vsize=1575040 CPUtime=5.23 cores=0-31
/proc/9986/task/10009/stat : 10009 (plingeling) R 9984 9986 9960 0 -1 4202560 9625 1238 0 0 509 14 0 0 25 0 33 0 655689490 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4317317 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9986/tid=10010] ppid=9984 vsize=1575040 CPUtime=5.02 cores=0-31
/proc/9986/task/10010/stat : 10010 (plingeling) R 9984 9986 9960 0 -1 4202560 10596 1238 0 0 486 16 0 0 25 0 33 0 655689490 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4255615 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=9986/tid=10011] ppid=9984 vsize=1575040 CPUtime=5.25 cores=0-31
/proc/9986/task/10011/stat : 10011 (plingeling) R 9984 9986 9960 0 -1 4202560 10216 1238 0 0 510 15 0 0 21 0 33 0 655689490 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4346285 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=9986/tid=10012] ppid=9984 vsize=1575040 CPUtime=5.29 cores=0-31
/proc/9986/task/10012/stat : 10012 (plingeling) R 9984 9986 9960 0 -1 4202560 10944 1238 0 0 517 12 0 0 25 0 33 0 655689490 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4346361 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=9986/tid=10013] ppid=9984 vsize=1575040 CPUtime=5.19 cores=0-31
/proc/9986/task/10013/stat : 10013 (plingeling) R 9984 9986 9960 0 -1 4202560 11812 1238 0 0 506 13 0 0 25 0 33 0 655689490 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4274888 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=9986/tid=10014] ppid=9984 vsize=1575040 CPUtime=4.94 cores=0-31
/proc/9986/task/10014/stat : 10014 (plingeling) R 9984 9986 9960 0 -1 4202560 14886 1238 0 0 476 18 0 0 22 0 33 0 655689490 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4346361 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=9986/tid=10015] ppid=9984 vsize=1575040 CPUtime=5.13 cores=0-31
/proc/9986/task/10015/stat : 10015 (plingeling) R 9984 9986 9960 0 -1 4202560 8160 1238 0 0 501 12 0 0 25 0 33 0 655689490 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4346639 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9986/tid=10016] ppid=9984 vsize=1575040 CPUtime=5.02 cores=0-31
/proc/9986/task/10016/stat : 10016 (plingeling) R 9984 9986 9960 0 -1 4202560 9299 1238 0 0 486 16 0 0 21 0 33 0 655689490 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4270966 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9986/tid=10017] ppid=9984 vsize=1575040 CPUtime=5.2 cores=0-31
/proc/9986/task/10017/stat : 10017 (plingeling) R 9984 9986 9960 0 -1 4202560 7705 1238 0 0 508 12 0 0 21 0 33 0 655689490 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4255662 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=9986/tid=10018] ppid=9984 vsize=1575040 CPUtime=4.91 cores=0-31
/proc/9986/task/10018/stat : 10018 (plingeling) R 9984 9986 9960 0 -1 4202560 12957 1238 0 0 473 18 0 0 21 0 33 0 655689490 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4346546 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=9986/tid=10019] ppid=9984 vsize=1575040 CPUtime=5.27 cores=0-31
/proc/9986/task/10019/stat : 10019 (plingeling) R 9984 9986 9960 0 -1 4202560 9674 1238 0 0 514 13 0 0 25 0 33 0 655689490 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4301955 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=9986/tid=10020] ppid=9984 vsize=1575040 CPUtime=5.3 cores=0-31
/proc/9986/task/10020/stat : 10020 (plingeling) R 9984 9986 9960 0 -1 4202560 10194 1238 0 0 522 8 0 0 25 0 33 0 655689490 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4255662 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=9986/tid=10021] ppid=9984 vsize=1575040 CPUtime=5.09 cores=0-31
/proc/9986/task/10021/stat : 10021 (plingeling) R 9984 9986 9960 0 -1 4202560 20995 1238 0 0 494 15 0 0 25 0 33 0 655689491 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4255628 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=9986/tid=10022] ppid=9984 vsize=1575040 CPUtime=5.17 cores=0-31
/proc/9986/task/10022/stat : 10022 (plingeling) R 9984 9986 9960 0 -1 4202560 12948 1238 0 0 500 17 0 0 18 0 33 0 655689491 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4270916 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=9986/tid=10023] ppid=9984 vsize=1575040 CPUtime=5.2 cores=0-31
/proc/9986/task/10023/stat : 10023 (plingeling) R 9984 9986 9960 0 -1 4202560 14270 1238 0 0 511 9 0 0 25 0 33 0 655689491 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4346484 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9986/tid=10024] ppid=9984 vsize=1575040 CPUtime=5.17 cores=0-31
/proc/9986/task/10024/stat : 10024 (plingeling) R 9984 9986 9960 0 -1 4202560 10719 1238 0 0 508 9 0 0 25 0 33 0 655689491 1612840960 287753 268435456000 4194304 4417452 140733254012832 18446744073709551615 4270970 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 172.23
Current children cumulated vsize (KiB) 1575040

[startup+25.5016 s]
/proc/loadavg: 25.72 30.28 31.38 33/446 10024
/proc/meminfo: memFree=254066988/264257624 swapFree=67111528/67111528
[pid=9986] ppid=9984 vsize=1960688 CPUtime=578.06 cores=0-31
/proc/9986/stat : 9986 (plingeling) S 9984 9986 9960 0 -1 4202496 602804 1238 0 0 57172 634 0 0 18 0 33 0 655688762 2007744512 373506 268435456000 4194304 4417452 140733254012832 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 25 0 0 0
/proc/9986/statm: 490172 373506 175 55 0 488058 0
[pid=9986/tid=9993] ppid=9984 vsize=1960688 CPUtime=17.91 cores=0-31
/proc/9986/task/9993/stat : 9993 (plingeling) R 9984 9986 9960 0 -1 4202560 13299 1238 0 0 1769 22 0 0 25 0 33 0 655689488 2007744512 373506 268435456000 4194304 4417452 140733254012832 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=9986/tid=9994] ppid=9984 vsize=1960688 CPUtime=17.81 cores=0-31
/proc/9986/task/9994/stat : 9994 (plingeling) R 9984 9986 9960 0 -1 4202560 16480 1238 0 0 1768 13 0 0 25 0 33 0 655689488 2007744512 373506 268435456000 4194304 4417452 140733254012832 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=9986/tid=9995] ppid=9984 vsize=1960688 CPUtime=18.04 cores=0-31
/proc/9986/task/9995/stat : 9995 (plingeling) R 9984 9986 9960 0 -1 4202560 11096 1238 0 0 1795 9 0 0 25 0 33 0 655689489 2007744512 373506 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364587 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=9986/tid=9996] ppid=9984 vsize=1960688 CPUtime=18.15 cores=0-31
/proc/9986/task/9996/stat : 9996 (plingeling) R 9984 9986 9960 0 -1 4202560 2267 1238 0 0 1813 2 0 0 25 0 33 0 655689489 2007744512 373506 268435456000 4194304 4417452 140733254012832 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=9986/tid=9997] ppid=9984 vsize=1960688 CPUtime=17.33 cores=0-31
/proc/9986/task/9997/stat : 9997 (plingeling) R 9984 9986 9960 0 -1 4202560 17843 1238 0 0 1702 31 0 0 25 0 33 0 655689489 2007744512 373506 268435456000 4194304 4417452 140733254012832 18446744073709551615 4249305 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=9986/tid=9998] ppid=9984 vsize=1960688 CPUtime=17.68 cores=0-31
/proc/9986/task/9998/stat : 9998 (plingeling) R 9984 9986 9960 0 -1 4202560 10936 1238 0 0 1747 21 0 0 25 0 33 0 655689489 2007744512 373506 268435456000 4194304 4417452 140733254012832 18446744073709551615 4249193 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9986/tid=9999] ppid=9984 vsize=1960688 CPUtime=17.71 cores=0-31
/proc/9986/task/9999/stat : 9999 (plingeling) R 9984 9986 9960 0 -1 4202560 10372 1238 0 0 1764 7 0 0 25 0 33 0 655689489 2007744512 373506 268435456000 4194304 4417452 140733254012832 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=9986/tid=10000] ppid=9984 vsize=1960688 CPUtime=17.74 cores=0-31
/proc/9986/task/10000/stat : 10000 (plingeling) R 9984 9986 9960 0 -1 4202560 12012 1238 0 0 1764 10 0 0 25 0 33 0 655689489 2007744512 373506 268435456000 4194304 4417452 140733254012832 18446744073709551615 4235999 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9986/tid=10001] ppid=9984 vsize=1960688 CPUtime=17.53 cores=0-31
/proc/9986/task/10001/stat : 10001 (plingeling) R 9984 9986 9960 0 -1 4202560 21486 1238 0 0 1721 32 0 0 25 0 33 0 655689489 2007744512 373506 268435456000 4194304 4417452 140733254012832 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9986/tid=10002] ppid=9984 vsize=1960688 CPUtime=18.03 cores=0-31
/proc/9986/task/10002/stat : 10002 (plingeling) R 9984 9986 9960 0 -1 4202560 18550 1238 0 0 1787 16 0 0 25 0 33 0 655689489 2007744512 373506 268435456000 4194304 4417452 140733254012832 18446744073709551615 4337285 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=9986/tid=10003] ppid=9984 vsize=1960688 CPUtime=17.74 cores=0-31
/proc/9986/task/10003/stat : 10003 (plingeling) R 9984 9986 9960 0 -1 4202560 8648 1238 0 0 1758 16 0 0 25 0 33 0 655689489 2007744512 373506 268435456000 4194304 4417452 140733254012832 18446744073709551615 4236784 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=9986/tid=10004] ppid=9984 vsize=1960688 CPUtime=18.05 cores=0-31
/proc/9986/task/10004/stat : 10004 (plingeling) R 9984 9986 9960 0 -1 4202560 18846 1238 0 0 1779 26 0 0 25 0 33 0 655689489 2007744512 373506 268435456000 4194304 4417452 140733254012832 18446744073709551615 4249478 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=9986/tid=10005] ppid=9984 vsize=1960688 CPUtime=17.62 cores=0-31
/proc/9986/task/10005/stat : 10005 (plingeling) R 9984 9986 9960 0 -1 4202560 16955 1238 0 0 1743 19 0 0 25 0 33 0 655689489 2007744512 373506 268435456000 4194304 4417452 140733254012832 18446744073709551615 4249143 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=9986/tid=10006] ppid=9984 vsize=1960688 CPUtime=17.69 cores=0-31
/proc/9986/task/10006/stat : 10006 (plingeling) R 9984 9986 9960 0 -1 4202560 19750 1238 0 0 1756 13 0 0 25 0 33 0 655689490 2007744512 373506 268435456000 4194304 4417452 140733254012832 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=9986/tid=10007] ppid=9984 vsize=1960688 CPUtime=17.95 cores=0-31
/proc/9986/task/10007/stat : 10007 (plingeling) R 9984 9986 9960 0 -1 4202560 10643 1238 0 0 1782 13 0 0 25 0 33 0 655689490 2007744512 373506 268435456000 4194304 4417452 140733254012832 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 13 0 0 0

################
# More data... #
################

/proc/9986/task/9998/stat : 9998 (plingeling) R 9984 9986 9960 0 -1 4202560 375492 1238 0 0 494783 542 0 0 25 0 33 0 655689489 11861680128 2779689 268435456000 4194304 4417452 140733254012832 18446744073709551615 4366065 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=9986/tid=9999] ppid=9984 vsize=11583672 CPUtime=4952.98 cores=0-31
/proc/9986/task/9999/stat : 9999 (plingeling) R 9984 9986 9960 0 -1 4202560 211023 1238 0 0 495041 257 0 0 25 0 33 0 655689489 11861680128 2779689 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=9986/tid=10000] ppid=9984 vsize=11583672 CPUtime=4953.29 cores=0-31
/proc/9986/task/10000/stat : 10000 (plingeling) R 9984 9986 9960 0 -1 4202560 195148 1238 0 0 495054 275 0 0 25 0 33 0 655689489 11861680128 2779689 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=9986/tid=10001] ppid=9984 vsize=11583672 CPUtime=4922.92 cores=0-31
/proc/9986/task/10001/stat : 10001 (plingeling) R 9984 9986 9960 0 -1 4202560 247543 1238 0 0 491989 303 0 0 25 0 33 0 655689489 11861680128 2779689 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9986/tid=10002] ppid=9984 vsize=11583672 CPUtime=4953.78 cores=0-31
/proc/9986/task/10002/stat : 10002 (plingeling) R 9984 9986 9960 0 -1 4202560 201563 1238 0 0 495173 205 0 0 25 0 33 0 655689489 11861680128 2779689 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=9986/tid=10003] ppid=9984 vsize=11583672 CPUtime=4953.57 cores=0-31
/proc/9986/task/10003/stat : 10003 (plingeling) R 9984 9986 9960 0 -1 4202560 219856 1238 0 0 495061 296 0 0 25 0 33 0 655689489 11861680128 2779689 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9986/tid=10004] ppid=9984 vsize=11583672 CPUtime=4952.63 cores=0-31
/proc/9986/task/10004/stat : 10004 (plingeling) R 9984 9986 9960 0 -1 4202560 434670 1238 0 0 494580 683 0 0 25 0 33 0 655689489 11861680128 2779689 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=9986/tid=10005] ppid=9984 vsize=11583672 CPUtime=4953.45 cores=0-31
/proc/9986/task/10005/stat : 10005 (plingeling) R 9984 9986 9960 0 -1 4202560 262228 1238 0 0 495032 313 0 0 25 0 33 0 655689489 11861680128 2779689 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=9986/tid=10006] ppid=9984 vsize=11583672 CPUtime=4948.31 cores=0-31
/proc/9986/task/10006/stat : 10006 (plingeling) R 9984 9986 9960 0 -1 4202560 311012 1238 0 0 494460 371 0 0 25 0 33 0 655689490 11861680128 2779689 268435456000 4194304 4417452 140733254012832 18446744073709551615 4366400 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9986/tid=10007] ppid=9984 vsize=11583672 CPUtime=4953.53 cores=0-31
/proc/9986/task/10007/stat : 10007 (plingeling) R 9984 9986 9960 0 -1 4202560 226601 1238 0 0 495110 243 0 0 25 0 33 0 655689490 11861680128 2779689 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=9986/tid=10008] ppid=9984 vsize=11583672 CPUtime=4953.44 cores=0-31
/proc/9986/task/10008/stat : 10008 (plingeling) R 9984 9986 9960 0 -1 4202560 244482 1238 0 0 495066 278 0 0 25 0 33 0 655689490 11861680128 2779689 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=9986/tid=10009] ppid=9984 vsize=11583672 CPUtime=4953.82 cores=0-31
/proc/9986/task/10009/stat : 10009 (plingeling) R 9984 9986 9960 0 -1 4202560 279417 1238 0 0 495029 353 0 0 25 0 33 0 655689490 11861680128 2779689 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=9986/tid=10010] ppid=9984 vsize=11583672 CPUtime=4952.99 cores=0-31
/proc/9986/task/10010/stat : 10010 (plingeling) R 9984 9986 9960 0 -1 4202560 227805 1238 0 0 495048 251 0 0 25 0 33 0 655689490 11861680128 2779689 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=9986/tid=10011] ppid=9984 vsize=11583672 CPUtime=4953.48 cores=0-31
/proc/9986/task/10011/stat : 10011 (plingeling) R 9984 9986 9960 0 -1 4202560 177931 1238 0 0 495056 292 0 0 25 0 33 0 655689490 11861680128 2779689 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=9986/tid=10012] ppid=9984 vsize=11583672 CPUtime=4953.12 cores=0-31
/proc/9986/task/10012/stat : 10012 (plingeling) R 9984 9986 9960 0 -1 4202560 201141 1238 0 0 495045 267 0 0 25 0 33 0 655689490 11861680128 2779689 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=9986/tid=10013] ppid=9984 vsize=11583672 CPUtime=4953.21 cores=0-31
/proc/9986/task/10013/stat : 10013 (plingeling) R 9984 9986 9960 0 -1 4202560 255735 1238 0 0 494991 330 0 0 25 0 33 0 655689490 11861680128 2779689 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=9986/tid=10014] ppid=9984 vsize=11583672 CPUtime=4953.1 cores=0-31
/proc/9986/task/10014/stat : 10014 (plingeling) R 9984 9986 9960 0 -1 4202560 275998 1238 0 0 494867 443 0 0 25 0 33 0 655689490 11861680128 2779689 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=9986/tid=10015] ppid=9984 vsize=11583672 CPUtime=4953.6 cores=0-31
/proc/9986/task/10015/stat : 10015 (plingeling) R 9984 9986 9960 0 -1 4202560 216059 1238 0 0 495089 271 0 0 25 0 33 0 655689490 11861680128 2779689 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9986/tid=10016] ppid=9984 vsize=11583672 CPUtime=4952.79 cores=0-31
/proc/9986/task/10016/stat : 10016 (plingeling) R 9984 9986 9960 0 -1 4202560 255176 1238 0 0 494945 334 0 0 25 0 33 0 655689490 11861680128 2779689 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9986/tid=10017] ppid=9984 vsize=11583672 CPUtime=4951.47 cores=0-31
/proc/9986/task/10017/stat : 10017 (plingeling) R 9984 9986 9960 0 -1 4202560 236139 1238 0 0 494859 288 0 0 25 0 33 0 655689490 11861680128 2779689 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=9986/tid=10018] ppid=9984 vsize=11583672 CPUtime=4953.37 cores=0-31
/proc/9986/task/10018/stat : 10018 (plingeling) R 9984 9986 9960 0 -1 4202560 359839 1238 0 0 494798 539 0 0 25 0 33 0 655689490 11861680128 2779689 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=9986/tid=10019] ppid=9984 vsize=11583672 CPUtime=4953.73 cores=0-31
/proc/9986/task/10019/stat : 10019 (plingeling) R 9984 9986 9960 0 -1 4202560 186461 1238 0 0 495124 249 0 0 25 0 33 0 655689490 11861680128 2779689 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=9986/tid=10020] ppid=9984 vsize=11583672 CPUtime=4888.04 cores=0-31
/proc/9986/task/10020/stat : 10020 (plingeling) R 9984 9986 9960 0 -1 4202560 306409 1238 0 0 488361 443 0 0 25 0 33 0 655689490 11861680128 2779689 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=9986/tid=10021] ppid=9984 vsize=11583672 CPUtime=4953.71 cores=0-31
/proc/9986/task/10021/stat : 10021 (plingeling) R 9984 9986 9960 0 -1 4202560 300224 1238 0 0 495111 260 0 0 25 0 33 0 655689491 11861680128 2779689 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=9986/tid=10022] ppid=9984 vsize=11583672 CPUtime=4953.72 cores=0-31
/proc/9986/task/10022/stat : 10022 (plingeling) R 9984 9986 9960 0 -1 4202560 289342 1238 0 0 494936 436 0 0 25 0 33 0 655689491 11861680128 2779689 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9986/tid=10023] ppid=9984 vsize=11583672 CPUtime=4921.98 cores=0-31
/proc/9986/task/10023/stat : 10023 (plingeling) R 9984 9986 9960 0 -1 4202560 199380 1238 0 0 491953 245 0 0 25 0 33 0 655689491 11861680128 2779689 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=9986/tid=10024] ppid=9984 vsize=11583672 CPUtime=4953.73 cores=0-31
/proc/9986/task/10024/stat : 10024 (plingeling) R 9984 9986 9960 0 -1 4202560 208588 1238 0 0 495104 269 0 0 25 0 33 0 655689491 11861680128 2779689 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 15 0 0 0
Current children cumulated CPU time (s) 158374
Current children cumulated vsize (KiB) 11583672



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5013.2 s]
/proc/loadavg: 32.25 32.09 32.02 33/446 10179
/proc/meminfo: memFree=244330500/264257624 swapFree=67111528/67111528
[pid=9986] ppid=9984 vsize=11703420 CPUtime=160001 cores=0-31
/proc/9986/stat : 9986 (plingeling) S 9984 9986 9960 0 -1 4202496 8166771 1238 0 0 15989514 10625 0 0 18 0 33 0 655688762 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 25 0 0 0
/proc/9986/statm: 2925855 2809626 175 55 0 2923741 0
[pid=9986/tid=9993] ppid=9984 vsize=11703420 CPUtime=5003.35 cores=0-31
/proc/9986/task/9993/stat : 9993 (plingeling) R 9984 9986 9960 0 -1 4202560 245817 1238 0 0 500040 295 0 0 25 0 33 0 655689488 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=9986/tid=9994] ppid=9984 vsize=11703420 CPUtime=4997.02 cores=0-31
/proc/9986/task/9994/stat : 9994 (plingeling) R 9984 9986 9960 0 -1 4202560 270144 1238 0 0 499384 318 0 0 25 0 33 0 655689488 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=9986/tid=9995] ppid=9984 vsize=11703420 CPUtime=5004.7 cores=0-31
/proc/9986/task/9995/stat : 9995 (plingeling) R 9984 9986 9960 0 -1 4202560 193896 1238 0 0 500184 286 0 0 25 0 33 0 655689489 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=9986/tid=9996] ppid=9984 vsize=11703420 CPUtime=5004.7 cores=0-31
/proc/9986/task/9996/stat : 9996 (plingeling) R 9984 9986 9960 0 -1 4202560 65148 1238 0 0 500406 64 0 0 25 0 33 0 655689489 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4353557 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=9986/tid=9997] ppid=9984 vsize=11703420 CPUtime=5003.92 cores=0-31
/proc/9986/task/9997/stat : 9997 (plingeling) R 9984 9986 9960 0 -1 4202560 288380 1238 0 0 499914 478 0 0 25 0 33 0 655689489 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=9986/tid=9998] ppid=9984 vsize=11703420 CPUtime=5004.16 cores=0-31
/proc/9986/task/9998/stat : 9998 (plingeling) R 9984 9986 9960 0 -1 4202560 375492 1238 0 0 499874 542 0 0 25 0 33 0 655689489 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4353568 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=9986/tid=9999] ppid=9984 vsize=11703420 CPUtime=5003.89 cores=0-31
/proc/9986/task/9999/stat : 9999 (plingeling) R 9984 9986 9960 0 -1 4202560 211023 1238 0 0 500132 257 0 0 25 0 33 0 655689489 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=9986/tid=10000] ppid=9984 vsize=11703420 CPUtime=5004.2 cores=0-31
/proc/9986/task/10000/stat : 10000 (plingeling) R 9984 9986 9960 0 -1 4202560 195148 1238 0 0 500145 275 0 0 25 0 33 0 655689489 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=9986/tid=10001] ppid=9984 vsize=11703420 CPUtime=4973.83 cores=0-31
/proc/9986/task/10001/stat : 10001 (plingeling) R 9984 9986 9960 0 -1 4202560 247543 1238 0 0 497080 303 0 0 25 0 33 0 655689489 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9986/tid=10002] ppid=9984 vsize=11703420 CPUtime=5004.69 cores=0-31
/proc/9986/task/10002/stat : 10002 (plingeling) R 9984 9986 9960 0 -1 4202560 201563 1238 0 0 500264 205 0 0 25 0 33 0 655689489 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=9986/tid=10003] ppid=9984 vsize=11703420 CPUtime=5004.48 cores=0-31
/proc/9986/task/10003/stat : 10003 (plingeling) R 9984 9986 9960 0 -1 4202560 219856 1238 0 0 500152 296 0 0 25 0 33 0 655689489 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=9986/tid=10004] ppid=9984 vsize=11703420 CPUtime=5003.54 cores=0-31
/proc/9986/task/10004/stat : 10004 (plingeling) R 9984 9986 9960 0 -1 4202560 434670 1238 0 0 499671 683 0 0 25 0 33 0 655689489 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364399 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=9986/tid=10005] ppid=9984 vsize=11703420 CPUtime=5004.36 cores=0-31
/proc/9986/task/10005/stat : 10005 (plingeling) R 9984 9986 9960 0 -1 4202560 294245 1238 0 0 500046 390 0 0 25 0 33 0 655689489 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=9986/tid=10006] ppid=9984 vsize=11703420 CPUtime=4999.22 cores=0-31
/proc/9986/task/10006/stat : 10006 (plingeling) R 9984 9986 9960 0 -1 4202560 311012 1238 0 0 499551 371 0 0 25 0 33 0 655689490 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364812 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=9986/tid=10007] ppid=9984 vsize=11703420 CPUtime=5004.44 cores=0-31
/proc/9986/task/10007/stat : 10007 (plingeling) R 9984 9986 9960 0 -1 4202560 226601 1238 0 0 500201 243 0 0 25 0 33 0 655689490 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=9986/tid=10008] ppid=9984 vsize=11703420 CPUtime=5004.36 cores=0-31
/proc/9986/task/10008/stat : 10008 (plingeling) R 9984 9986 9960 0 -1 4202560 244482 1238 0 0 500158 278 0 0 25 0 33 0 655689490 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=9986/tid=10009] ppid=9984 vsize=11703420 CPUtime=5004.73 cores=0-31
/proc/9986/task/10009/stat : 10009 (plingeling) R 9984 9986 9960 0 -1 4202560 279417 1238 0 0 500120 353 0 0 25 0 33 0 655689490 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=9986/tid=10010] ppid=9984 vsize=11703420 CPUtime=5003.89 cores=0-31
/proc/9986/task/10010/stat : 10010 (plingeling) R 9984 9986 9960 0 -1 4202560 227805 1238 0 0 500138 251 0 0 25 0 33 0 655689490 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=9986/tid=10011] ppid=9984 vsize=11703420 CPUtime=5004.4 cores=0-31
/proc/9986/task/10011/stat : 10011 (plingeling) R 9984 9986 9960 0 -1 4202560 185342 1238 0 0 500127 313 0 0 25 0 33 0 655689490 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=9986/tid=10012] ppid=9984 vsize=11703420 CPUtime=5004.03 cores=0-31
/proc/9986/task/10012/stat : 10012 (plingeling) R 9984 9986 9960 0 -1 4202560 201141 1238 0 0 500136 267 0 0 25 0 33 0 655689490 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=9986/tid=10013] ppid=9984 vsize=11703420 CPUtime=5004.12 cores=0-31
/proc/9986/task/10013/stat : 10013 (plingeling) R 9984 9986 9960 0 -1 4202560 255735 1238 0 0 500082 330 0 0 25 0 33 0 655689490 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=9986/tid=10014] ppid=9984 vsize=11703420 CPUtime=5004.01 cores=0-31
/proc/9986/task/10014/stat : 10014 (plingeling) R 9984 9986 9960 0 -1 4202560 275998 1238 0 0 499958 443 0 0 25 0 33 0 655689490 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4365004 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=9986/tid=10015] ppid=9984 vsize=11703420 CPUtime=5004.52 cores=0-31
/proc/9986/task/10015/stat : 10015 (plingeling) R 9984 9986 9960 0 -1 4202560 216059 1238 0 0 500181 271 0 0 25 0 33 0 655689490 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9986/tid=10016] ppid=9984 vsize=11703420 CPUtime=5003.7 cores=0-31
/proc/9986/task/10016/stat : 10016 (plingeling) R 9984 9986 9960 0 -1 4202560 267605 1238 0 0 500033 337 0 0 25 0 33 0 655689490 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364737 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9986/tid=10017] ppid=9984 vsize=11703420 CPUtime=5002.36 cores=0-31
/proc/9986/task/10017/stat : 10017 (plingeling) R 9984 9986 9960 0 -1 4202560 236139 1238 0 0 499948 288 0 0 25 0 33 0 655689490 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=9986/tid=10018] ppid=9984 vsize=11703420 CPUtime=5004.29 cores=0-31
/proc/9986/task/10018/stat : 10018 (plingeling) R 9984 9986 9960 0 -1 4202560 359848 1238 0 0 499890 539 0 0 25 0 33 0 655689490 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=9986/tid=10019] ppid=9984 vsize=11703420 CPUtime=5004.64 cores=0-31
/proc/9986/task/10019/stat : 10019 (plingeling) R 9984 9986 9960 0 -1 4202560 186461 1238 0 0 500215 249 0 0 25 0 33 0 655689490 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=9986/tid=10020] ppid=9984 vsize=11703420 CPUtime=4937.5 cores=0-31
/proc/9986/task/10020/stat : 10020 (plingeling) R 9984 9986 9960 0 -1 4202560 306409 1238 0 0 493307 443 0 0 25 0 33 0 655689490 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=9986/tid=10021] ppid=9984 vsize=11703420 CPUtime=5004.62 cores=0-31
/proc/9986/task/10021/stat : 10021 (plingeling) R 9984 9986 9960 0 -1 4202560 300224 1238 0 0 500202 260 0 0 25 0 33 0 655689491 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=9986/tid=10022] ppid=9984 vsize=11703420 CPUtime=5004.63 cores=0-31
/proc/9986/task/10022/stat : 10022 (plingeling) R 9984 9986 9960 0 -1 4202560 289342 1238 0 0 500027 436 0 0 25 0 33 0 655689491 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=9986/tid=10023] ppid=9984 vsize=11703420 CPUtime=4972.9 cores=0-31
/proc/9986/task/10023/stat : 10023 (plingeling) R 9984 9986 9960 0 -1 4202560 199380 1238 0 0 497045 245 0 0 25 0 33 0 655689491 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4277864 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=9986/tid=10024] ppid=9984 vsize=11703420 CPUtime=5004.64 cores=0-31
/proc/9986/task/10024/stat : 10024 (plingeling) R 9984 9986 9960 0 -1 4202560 208588 1238 0 0 500195 269 0 0 25 0 33 0 655689491 11984302080 2809626 268435456000 4194304 4417452 140733254012832 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 15 0 0 0
Current children cumulated CPU time (s) 160001
Current children cumulated vsize (KiB) 11703420

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5013.4 s]
/proc/loadavg: 32.25 32.09 32.02 33/446 10179
/proc/meminfo: memFree=244330500/264257624 swapFree=67111528/67111528
[pid=9986] ppid=9984 vsize=0 CPUtime=160002 cores=0-31
/proc/9986/stat : 9986 (plingeling) Z 9984 9986 9960 0 -1 4203548 8166771 1238 0 0 15989537 10642 0 0 18 0 2 0 655688762 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 25 0 0 0
/proc/9986/statm: 0 0 0 0 0 0 0
[pid=9986/tid=10013] ppid=9984 vsize=0 CPUtime=5004.3 cores=0-31
/proc/9986/task/10013/stat : 10013 (plingeling) R 9984 9986 9960 0 -1 4203588 255735 1238 0 0 500083 347 0 0 25 0 2 0 655689490 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 25 0 0 0
Current children cumulated CPU time (s) 160002
Current children cumulated vsize (KiB) 0

[startup+5013.84 s]
/proc/loadavg: 32.25 32.09 32.02 2/416 10180
/proc/meminfo: memFree=250849916/264257624 swapFree=67111528/67111528
[pid=9986] ppid=9984 vsize=0 CPUtime=160002 cores=0-31
/proc/9986/stat : 9986 (plingeling) Z 9984 9986 9960 0 -1 4203548 8166771 1238 0 0 15989537 10686 0 0 18 0 2 0 655688762 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 25 0 0 0
/proc/9986/statm: 0 0 0 0 0 0 0
[pid=9986/tid=10013] ppid=9984 vsize=0 CPUtime=5004.74 cores=0-31
/proc/9986/task/10013/stat : 10013 (plingeling) R 9984 9986 9960 0 -1 4203588 255735 1238 0 0 500083 391 0 0 25 0 2 0 655689490 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 25 0 0 0
Current children cumulated CPU time (s) 160002
Current children cumulated vsize (KiB) 0

[startup+5014 s]
/proc/loadavg: 32.25 32.09 32.02 2/416 10180
/proc/meminfo: memFree=250849916/264257624 swapFree=67111528/67111528
[pid=9986] ppid=9984 vsize=0 CPUtime=160002 cores=0-31
/proc/9986/stat : 9986 (plingeling) Z 9984 9986 9960 0 -1 4203548 8166771 1238 0 0 15989537 10702 0 0 18 0 2 0 655688762 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 25 0 0 0
/proc/9986/statm: 0 0 0 0 0 0 0
[pid=9986/tid=10013] ppid=9984 vsize=0 CPUtime=5004.9 cores=0-31
/proc/9986/task/10013/stat : 10013 (plingeling) R 9984 9986 9960 0 -1 4203588 255735 1238 0 0 500083 407 0 0 25 0 2 0 655689490 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 25 0 0 0
Current children cumulated CPU time (s) 160002
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5014.04
CPU time (s): 160002
CPU user time (s): 159895
CPU system time (s): 107.07
CPU usage (%): 3191.09
Max. virtual memory (cumulated for all children) (KiB): 11703420

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 159895
system time used= 107.07
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 8168009
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 4441
involuntary context switches= 418778

runsolver used 26.7849 second user time and 114.906 second system time

The end

Launcher Data 

Begin job on node200 at 2011-05-15 08:47:04
IDJOB=3372129
IDBENCH=20406
IDSOLVER=1852
FILE ID=node200/3372129-1305442023
RUNJOBID= node200-1305442023-9974
PBS_JOBID= 13285509
Free space on /tmp= 62044 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT07/industrial/palacios/hard/sortnet-7-ipc5-h15-unsat.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3372129-1305442023/watcher-3372129-1305442023 -o /tmp/evaluation-result-3372129-1305442023/solver-3372129-1305442023 -C 160000 -W 5100 -M 266338304  plingeling -t 32 HOME/instance-3372129-1305442023.cnf

TIME LIMIT= 160000 seconds
MEMORY LIMIT= 266338304 MiB
NBCORE= 32

MD5SUM BENCH= 0683489092a5eb0c881dd17c2c45d08e
RANDOM SEED=307370830

node200.alineos.net Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 64
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 96
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 66
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 98
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 68
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 100
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 70
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.98
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 102
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 16
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 17
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 48
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 18
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 80
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 19
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 112
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 20
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 21
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 50
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 22
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 82
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 114
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.23
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 24
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 25
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 26
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 84
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 27
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 116
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 28
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 29
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 54
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 30
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 86
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 31
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 118
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     264257624 kB
MemFree:      255577212 kB
Buffers:        439300 kB
Cached:        7835208 kB
SwapCached:          0 kB
Active:        7385140 kB
Inactive:       958088 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     264257624 kB
LowFree:      255577212 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:            6596 kB
Writeback:           0 kB
AnonPages:       70908 kB
Mapped:          13604 kB
Slab:           250248 kB
PageTables:       4912 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  199240340 kB
Committed_AS:   180432 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296372 kB
VmallocChunk: 34359440603 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 62044 MiB
End job on node200 at 2011-05-15 10:10:39