Trace number 3372852

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 160005 5056.09

General information on the benchmark

NameSAT03/industrial/schuppan/l2s/
dme-03-1-k247-unsat.shuffled-as.sat03-407.cnf
MD5SUM3fef216b3efe77c0f10ea0e0be664a94
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables261352
Number of clauses773077
Sum of the clauses size1985588
Maximum clause length248
Minimum clause length1
Number of clauses of size 11592
Number of clauses of size 2357488
Number of clauses of size 3413748
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 5249

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3372852-1306103524/watcher-3372852-1306103524 -o /tmp/evaluation-result-3372852-1306103524/solver-3372852-1306103524 -C 160000 -W 5100 -M 266338304 plingeling -t 32 HOME/instance-3372852-1306103524.cnf 

running on 32 cores: 0,4,8,12,16,20,24,28,1,5,9,13,17,21,25,29,2,6,10,14,18,22,26,30,3,7,11,15,19,23,27,31

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 160000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 160030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 272730423296 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 272730474496 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 29.52 31.58 31.91 1/413 27965
/proc/meminfo: memFree=255442008/264257624 swapFree=67111528/67111528
[pid=27965] ppid=27963 vsize=48968 CPUtime=0.02 cores=0-31
/proc/27965/stat : 27965 (plingeling) R 27963 27965 27724 0 -1 4202496 10152 1250 0 0 0 2 0 0 22 0 1 0 721838921 50143232 10083 268435456000 4194304 4417452 140737088365296 18446744073709551615 258650649933 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/27965/statm: 12242 10097 138 55 0 10128 0

[startup+0.064668 s]
/proc/loadavg: 29.52 31.58 31.91 1/413 27965
/proc/meminfo: memFree=255442008/264257624 swapFree=67111528/67111528
[pid=27965] ppid=27963 vsize=81768 CPUtime=0.05 cores=0-31
/proc/27965/stat : 27965 (plingeling) R 27963 27965 27724 0 -1 4202496 17706 1250 0 0 1 4 0 0 22 0 1 0 721838921 83730432 17637 268435456000 4194304 4417452 140737088365296 18446744073709551615 258650649933 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/27965/statm: 20442 17657 138 55 0 18328 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 81768

[startup+0.10066 s]
/proc/loadavg: 29.52 31.58 31.91 1/413 27965
/proc/meminfo: memFree=255442008/264257624 swapFree=67111528/67111528
[pid=27965] ppid=27963 vsize=122768 CPUtime=0.09 cores=0-31
/proc/27965/stat : 27965 (plingeling) R 27963 27965 27724 0 -1 4202496 28764 1250 0 0 3 6 0 0 22 0 1 0 721838921 125714432 28694 268435456000 4194304 4417452 140737088365296 18446744073709551615 258650649933 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/27965/statm: 31717 28712 138 55 0 29603 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 122768

[startup+0.300685 s]
/proc/loadavg: 29.52 31.58 31.91 1/413 27965
/proc/meminfo: memFree=255442008/264257624 swapFree=67111528/67111528
[pid=27965] ppid=27963 vsize=174908 CPUtime=0.29 cores=0-31
/proc/27965/stat : 27965 (plingeling) R 27963 27965 27724 0 -1 4202496 41818 1250 0 0 21 8 0 0 23 0 1 0 721838921 179105792 41748 268435456000 4194304 4417452 140737088365296 18446744073709551615 4228036 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/27965/statm: 43727 41748 146 55 0 41613 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 174908

[startup+0.700696 s]
/proc/loadavg: 29.52 31.58 31.91 1/413 27965
/proc/meminfo: memFree=255442008/264257624 swapFree=67111528/67111528
[pid=27965] ppid=27963 vsize=210860 CPUtime=0.69 cores=0-31
/proc/27965/stat : 27965 (plingeling) R 27963 27965 27724 0 -1 4202496 50806 1250 0 0 58 11 0 0 25 0 1 0 721838921 215920640 50736 268435456000 4194304 4417452 140737088365296 18446744073709551615 4231321 0 0 4096 17442 0 0 0 17 20 0 0 0
/proc/27965/statm: 52715 50736 146 55 0 50601 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 210860

[startup+1.50072 s]
/proc/loadavg: 29.52 31.58 31.91 2/414 27971
/proc/meminfo: memFree=255238528/264257624 swapFree=67111528/67111528
[pid=27965] ppid=27963 vsize=278448 CPUtime=1.48 cores=0-31
/proc/27965/stat : 27965 (plingeling) R 27963 27965 27724 0 -1 4202496 67760 1250 0 0 134 14 0 0 25 0 1 0 721838921 285130752 67633 268435456000 4194304 4417452 140737088365296 18446744073709551615 4264909 0 0 4096 17442 0 0 0 17 20 0 0 0
/proc/27965/statm: 69612 67633 146 55 0 67498 0
Current children cumulated CPU time (s) 1.48
Current children cumulated vsize (KiB) 278448

[startup+3.10077 s]
/proc/loadavg: 27.23 31.07 31.74 2/414 27971
/proc/meminfo: memFree=255152472/264257624 swapFree=67111528/67111528
[pid=27965] ppid=27963 vsize=411736 CPUtime=3.08 cores=0-31
/proc/27965/stat : 27965 (plingeling) R 27963 27965 27724 0 -1 4202496 101082 1250 0 0 286 22 0 0 25 0 1 0 721838921 421617664 100955 268435456000 4194304 4417452 140737088365296 18446744073709551615 4231477 0 0 4096 17442 0 0 0 17 20 0 0 0
/proc/27965/statm: 102934 100955 146 55 0 100820 0
Current children cumulated CPU time (s) 3.08
Current children cumulated vsize (KiB) 411736

[startup+6.30089 s]
/proc/loadavg: 27.23 31.07 31.74 2/414 27971
/proc/meminfo: memFree=254769580/264257624 swapFree=67111528/67111528
[pid=27965] ppid=27963 vsize=679196 CPUtime=6.29 cores=0-31
/proc/27965/stat : 27965 (plingeling) R 27963 27965 27724 0 -1 4202496 167947 1250 0 0 592 37 0 0 25 0 1 0 721838921 695496704 167820 268435456000 4194304 4417452 140737088365296 18446744073709551615 4231477 0 0 4096 17442 0 0 0 17 20 0 0 0
/proc/27965/statm: 169799 167820 146 55 0 167685 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 679196

[startup+12.7011 s]
/proc/loadavg: 25.13 30.57 31.58 2/414 27971
/proc/meminfo: memFree=254704900/264257624 swapFree=67111528/67111528
[pid=27965] ppid=27963 vsize=744732 CPUtime=12.68 cores=0-31
/proc/27965/stat : 27965 (plingeling) R 27963 27965 27724 0 -1 4202496 184331 1250 0 0 1228 40 0 0 25 0 1 0 721838921 762605568 184204 268435456000 4194304 4417452 140737088365296 18446744073709551615 4262041 0 0 4096 17442 0 0 0 17 20 0 0 0
/proc/27965/statm: 186183 184204 146 55 0 184069 0
Current children cumulated CPU time (s) 12.68
Current children cumulated vsize (KiB) 744732

[startup+25.5006 s]
/proc/loadavg: 19.79 29.12 31.08 2/414 27973
/proc/meminfo: memFree=254312532/264257624 swapFree=67111528/67111528
[pid=27965] ppid=27963 vsize=1137948 CPUtime=25.48 cores=0-31
/proc/27965/stat : 27965 (plingeling) R 27963 27965 27724 0 -1 4202496 282635 1250 0 0 2489 59 0 0 25 0 1 0 721838921 1165258752 282508 268435456000 4194304 4417452 140737088365296 18446744073709551615 4259186 0 0 4096 17442 0 0 0 17 20 0 0 0
/proc/27965/statm: 284487 282508 146 55 0 282373 0
Current children cumulated CPU time (s) 25.48
Current children cumulated vsize (KiB) 1137948

[startup+51.1023 s]
/proc/loadavg: 15.86 27.38 30.45 33/446 28005
/proc/meminfo: memFree=253073908/264257624 swapFree=67111528/67111528
[pid=27965] ppid=27963 vsize=2725224 CPUtime=313.97 cores=0-31
/proc/27965/stat : 27965 (plingeling) S 27963 27965 27724 0 -1 4202496 754141 1250 0 0 31138 259 0 0 18 0 33 0 721838921 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 20 0 0 0
/proc/27965/statm: 681306 590299 171 55 0 679192 0
[pid=27965/tid=27974] ppid=27963 vsize=2725224 CPUtime=8.57 cores=0-31
/proc/27965/task/27974/stat : 27974 (plingeling) R 27963 27965 27724 0 -1 4202560 11665 1250 0 0 848 9 0 0 25 0 33 0 721843172 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4265480 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=27965/tid=27975] ppid=27963 vsize=2725224 CPUtime=8.57 cores=0-31
/proc/27965/task/27975/stat : 27975 (plingeling) R 27963 27965 27724 0 -1 4202560 15258 1250 0 0 850 7 0 0 25 0 33 0 721843172 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4275892 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=27965/tid=27976] ppid=27963 vsize=2725224 CPUtime=8.57 cores=0-31
/proc/27965/task/27976/stat : 27976 (plingeling) R 27963 27965 27724 0 -1 4202560 9422 1250 0 0 851 6 0 0 25 0 33 0 721843172 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4264199 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27965/tid=27977] ppid=27963 vsize=2725224 CPUtime=8.57 cores=0-31
/proc/27965/task/27977/stat : 27977 (plingeling) R 27963 27965 27724 0 -1 4202560 4371 1250 0 0 850 7 0 0 25 0 33 0 721843172 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=27965/tid=27978] ppid=27963 vsize=2725224 CPUtime=8.57 cores=0-31
/proc/27965/task/27978/stat : 27978 (plingeling) R 27963 27965 27724 0 -1 4202560 14850 1250 0 0 849 8 0 0 25 0 33 0 721843172 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4265360 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=27965/tid=27979] ppid=27963 vsize=2725224 CPUtime=8.52 cores=0-31
/proc/27965/task/27979/stat : 27979 (plingeling) R 27963 27965 27724 0 -1 4202560 11318 1250 0 0 850 2 0 0 25 0 33 0 721843172 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4276482 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=27965/tid=27980] ppid=27963 vsize=2725224 CPUtime=8.46 cores=0-31
/proc/27965/task/27980/stat : 27980 (plingeling) R 27963 27965 27724 0 -1 4202560 7158 1250 0 0 841 5 0 0 25 0 33 0 721843172 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4261326 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=27965/tid=27981] ppid=27963 vsize=2725224 CPUtime=8.53 cores=0-31
/proc/27965/task/27981/stat : 27981 (plingeling) R 27963 27965 27724 0 -1 4202560 10695 1250 0 0 850 3 0 0 25 0 33 0 721843172 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4302131 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=27965/tid=27982] ppid=27963 vsize=2725224 CPUtime=8.57 cores=0-31
/proc/27965/task/27982/stat : 27982 (plingeling) R 27963 27965 27724 0 -1 4202560 11312 1250 0 0 849 8 0 0 25 0 33 0 721843172 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4275944 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=27965/tid=27983] ppid=27963 vsize=2725224 CPUtime=8.56 cores=0-31
/proc/27965/task/27983/stat : 27983 (plingeling) R 27963 27965 27724 0 -1 4202560 7193 1250 0 0 855 1 0 0 25 0 33 0 721843172 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4301776 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=27965/tid=27984] ppid=27963 vsize=2725224 CPUtime=8.57 cores=0-31
/proc/27965/task/27984/stat : 27984 (plingeling) R 27963 27965 27724 0 -1 4202560 9103 1250 0 0 849 8 0 0 25 0 33 0 721843172 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4265383 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27965/tid=27985] ppid=27963 vsize=2725224 CPUtime=8.57 cores=0-31
/proc/27965/task/27985/stat : 27985 (plingeling) R 27963 27965 27724 0 -1 4202560 7095 1250 0 0 855 2 0 0 25 0 33 0 721843172 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4265832 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=27965/tid=27986] ppid=27963 vsize=2725224 CPUtime=8.55 cores=0-31
/proc/27965/task/27986/stat : 27986 (plingeling) R 27963 27965 27724 0 -1 4202560 8579 1250 0 0 853 2 0 0 25 0 33 0 721843172 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4301955 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27965/tid=27987] ppid=27963 vsize=2725224 CPUtime=8.51 cores=0-31
/proc/27965/task/27987/stat : 27987 (plingeling) R 27963 27965 27724 0 -1 4202560 11315 1250 0 0 843 8 0 0 25 0 33 0 721843172 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4276477 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27965/tid=27988] ppid=27963 vsize=2725224 CPUtime=8.55 cores=0-31
/proc/27965/task/27988/stat : 27988 (plingeling) R 27963 27965 27724 0 -1 4202560 9115 1250 0 0 848 7 0 0 25 0 33 0 721843172 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=27965/tid=27989] ppid=27963 vsize=2725224 CPUtime=8.55 cores=0-31
/proc/27965/task/27989/stat : 27989 (plingeling) R 27963 27965 27724 0 -1 4202560 11315 1250 0 0 847 8 0 0 25 0 33 0 721843172 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4275892 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=27965/tid=27990] ppid=27963 vsize=2725224 CPUtime=8.34 cores=0-31
/proc/27965/task/27990/stat : 27990 (plingeling) R 27963 27965 27724 0 -1 4202560 18284 1250 0 0 825 9 0 0 25 0 33 0 721843172 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4237169 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=27965/tid=27991] ppid=27963 vsize=2725224 CPUtime=8.55 cores=0-31
/proc/27965/task/27991/stat : 27991 (plingeling) R 27963 27965 27724 0 -1 4202560 18382 1250 0 0 846 9 0 0 25 0 33 0 721843172 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=27965/tid=27992] ppid=27963 vsize=2725224 CPUtime=8.56 cores=0-31
/proc/27965/task/27992/stat : 27992 (plingeling) R 27963 27965 27724 0 -1 4202560 12280 1250 0 0 853 3 0 0 25 0 33 0 721843173 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4270927 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=27965/tid=27993] ppid=27963 vsize=2725224 CPUtime=8.55 cores=0-31
/proc/27965/task/27993/stat : 27993 (plingeling) R 27963 27965 27724 0 -1 4202560 10997 1250 0 0 848 7 0 0 25 0 33 0 721843173 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4277662 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=27965/tid=27994] ppid=27963 vsize=2725224 CPUtime=8.3 cores=0-31
/proc/27965/task/27994/stat : 27994 (plingeling) R 27963 27965 27724 0 -1 4202560 7562 1250 0 0 828 2 0 0 25 0 33 0 721843173 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4275646 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=27965/tid=27995] ppid=27963 vsize=2725224 CPUtime=8.43 cores=0-31
/proc/27965/task/27995/stat : 27995 (plingeling) R 27963 27965 27724 0 -1 4202560 11251 1250 0 0 840 3 0 0 25 0 33 0 721843173 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=27965/tid=27996] ppid=27963 vsize=2725224 CPUtime=8.43 cores=0-31
/proc/27965/task/27996/stat : 27996 (plingeling) R 27963 27965 27724 0 -1 4202560 10609 1250 0 0 841 2 0 0 25 0 33 0 721843173 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4236784 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=27965/tid=27997] ppid=27963 vsize=2725224 CPUtime=8.29 cores=0-31
/proc/27965/task/27997/stat : 27997 (plingeling) R 27963 27965 27724 0 -1 4202560 10998 1250 0 0 827 2 0 0 25 0 33 0 721843173 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4277545 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=27965/tid=27998] ppid=27963 vsize=2725224 CPUtime=8.26 cores=0-31
/proc/27965/task/27998/stat : 27998 (plingeling) R 27963 27965 27724 0 -1 4202560 16546 1250 0 0 822 4 0 0 25 0 33 0 721843173 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4276673 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=27965/tid=27999] ppid=27963 vsize=2725224 CPUtime=8.26 cores=0-31
/proc/27965/task/27999/stat : 27999 (plingeling) R 27963 27965 27724 0 -1 4202560 7094 1250 0 0 825 1 0 0 25 0 33 0 721843173 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4275892 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=27965/tid=28000] ppid=27963 vsize=2725224 CPUtime=8.54 cores=0-31
/proc/27965/task/28000/stat : 28000 (plingeling) R 27963 27965 27724 0 -1 4202560 10104 1250 0 0 847 7 0 0 25 0 33 0 721843173 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4264403 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=27965/tid=28001] ppid=27963 vsize=2725224 CPUtime=8.57 cores=0-31
/proc/27965/task/28001/stat : 28001 (plingeling) R 27963 27965 27724 0 -1 4202560 7093 1250 0 0 855 2 0 0 25 0 33 0 721843173 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4276673 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=27965/tid=28002] ppid=27963 vsize=2725224 CPUtime=8.55 cores=0-31
/proc/27965/task/28002/stat : 28002 (plingeling) R 27963 27965 27724 0 -1 4202560 7156 1250 0 0 847 8 0 0 25 0 33 0 721843173 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=27965/tid=28003] ppid=27963 vsize=2725224 CPUtime=8.26 cores=0-31
/proc/27965/task/28003/stat : 28003 (plingeling) R 27963 27965 27724 0 -1 4202560 10463 1250 0 0 824 2 0 0 25 0 33 0 721843173 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4275944 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=27965/tid=28004] ppid=27963 vsize=2725224 CPUtime=8.26 cores=0-31
/proc/27965/task/28004/stat : 28004 (plingeling) R 27963 27965 27724 0 -1 4202560 10363 1250 0 0 824 2 0 0 25 0 33 0 721843179 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4275748 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=27965/tid=28005] ppid=27963 vsize=2725224 CPUtime=8.24 cores=0-31
/proc/27965/task/28005/stat : 28005 (plingeling) R 27963 27965 27724 0 -1 4202560 10743 1250 0 0 822 2 0 0 25 0 33 0 721843204 2790629376 590299 268435456000 4194304 4417452 140737088365296 18446744073709551615 4276491 0 0 4096 17442 0 0 0 -1 15 0 0 0
Current children cumulated CPU time (s) 313.97
Current children cumulated vsize (KiB) 2725224

[startup+102.322 s]
/proc/loadavg: 25.60 28.17 30.54 33/446 28007
/proc/meminfo: memFree=253688500/264257624 swapFree=67111528/67111528
[pid=27965] ppid=27963 vsize=2182076 CPUtime=1950.43 cores=0-31
/proc/27965/stat : 27965 (plingeling) S 27963 27965 27724 0 -1 4202496 872780 1250 0 0 194696 347 0 0 18 0 33 0 721838921 2234445824 436420 268435456000 4194304 4417452 140737088365296 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 20 0 0 0
/proc/27965/statm: 545519 436420 175 55 0 543405 0
[pid=27965/tid=27974] ppid=27963 vsize=2182076 CPUtime=59.66 cores=0-31
/proc/27965/task/27974/stat : 27974 (plingeling) R 27963 27965 27724 0 -1 4202560 17539 1250 0 0 5956 10 0 0 25 0 33 0 721843172 2234445824 436420 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27965/tid=27975] ppid=27963 vsize=2182076 CPUtime=59.78 cores=0-31
/proc/27965/task/27975/stat : 27975 (plingeling) R 27963 27965 27724 0 -1 4202560 23182 1250 0 0 5964 14 0 0 25 0 33 0 721843172 2234445824 436420 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=27965/tid=27976] ppid=27963 vsize=2182076 CPUtime=59.78 cores=0-31
/proc/27965/task/27976/stat : 27976 (plingeling) R 27963 27965 27724 0 -1 4202560 13880 1250 0 0 5970 8 0 0 25 0 33 0 721843172 2234445824 436420 268435456000 4194304 4417452 140737088365296 18446744073709551615 4302058 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=27965/tid=27977] ppid=27963 vsize=2182076 CPUtime=59.78 cores=0-31
/proc/27965/task/27977/stat : 27977 (plingeling) R 27963 27965 27724 0 -1 4202560 4372 1250 0 0 5971 7 0 0 25 0 33 0 721843172 2234445824 436420 268435456000 4194304 4417452 140737088365296 18446744073709551615 4232715 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=27965/tid=27978] ppid=27963 vsize=2182076 CPUtime=59.76 cores=0-31
/proc/27965/task/27978/stat : 27978 (plingeling) R 27963 27965 27724 0 -1 4202560 19108 1250 0 0 5963 13 0 0 25 0 33 0 721843172 2234445824 436420 268435456000 4194304 4417452 140737088365296 18446744073709551615 4302105 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=27965/tid=27979] ppid=27963 vsize=2182076 CPUtime=59.72 cores=0-31
/proc/27965/task/27979/stat : 27979 (plingeling) R 27963 27965 27724 0 -1 4202560 11386 1250 0 0 5969 3 0 0 25 0 33 0 721843172 2234445824 436420 268435456000 4194304 4417452 140737088365296 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 18 0 0 0

################
# More data... #
################

[pid=27965/tid=27987] ppid=27963 vsize=35323684 CPUtime=4971.21 cores=0-31
/proc/27965/task/27987/stat : 27987 (plingeling) R 27963 27965 27724 0 -1 4202560 1031874 1250 0 0 495738 1383 0 0 25 0 33 0 721843172 36171452416 8690331 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=27965/tid=27988] ppid=27963 vsize=35323684 CPUtime=4965.16 cores=0-31
/proc/27965/task/27988/stat : 27988 (plingeling) R 27963 27965 27724 0 -1 4202560 1594505 1250 0 0 493785 2731 0 0 25 0 33 0 721843172 36171452416 8690331 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=27965/tid=27989] ppid=27963 vsize=35323684 CPUtime=4971.08 cores=0-31
/proc/27965/task/27989/stat : 27989 (plingeling) R 27963 27965 27724 0 -1 4202560 858677 1250 0 0 496265 843 0 0 25 0 33 0 721843172 36171452416 8690331 268435456000 4194304 4417452 140737088365296 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=27965/tid=27990] ppid=27963 vsize=35323684 CPUtime=4969.53 cores=0-31
/proc/27965/task/27990/stat : 27990 (plingeling) R 27963 27965 27724 0 -1 4202560 312243 1250 0 0 496791 162 0 0 25 0 33 0 721843172 36171452416 8690331 268435456000 4194304 4417452 140737088365296 18446744073709551615 4365023 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=27965/tid=27991] ppid=27963 vsize=35323684 CPUtime=4969.28 cores=0-31
/proc/27965/task/27991/stat : 27991 (plingeling) R 27963 27965 27724 0 -1 4202560 1033346 1250 0 0 495665 1263 0 0 25 0 33 0 721843172 36171452416 8690331 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=27965/tid=27992] ppid=27963 vsize=35323684 CPUtime=4965.67 cores=0-31
/proc/27965/task/27992/stat : 27992 (plingeling) R 27963 27965 27724 0 -1 4202560 1286096 1250 0 0 494764 1803 0 0 25 0 33 0 721843173 36171452416 8690331 268435456000 4194304 4417452 140737088365296 18446744073709551615 4365036 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=27965/tid=27993] ppid=27963 vsize=35323684 CPUtime=4964.14 cores=0-31
/proc/27965/task/27993/stat : 27993 (plingeling) R 27963 27965 27724 0 -1 4202560 1199198 1250 0 0 494670 1744 0 0 25 0 33 0 721843173 36171452416 8690331 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27965/tid=27994] ppid=27963 vsize=35323684 CPUtime=4956.16 cores=0-31
/proc/27965/task/27994/stat : 27994 (plingeling) R 27963 27965 27724 0 -1 4202560 1110902 1250 0 0 494210 1406 0 0 25 0 33 0 721843173 36171452416 8690331 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=27965/tid=27995] ppid=27963 vsize=35323684 CPUtime=4968.13 cores=0-31
/proc/27965/task/27995/stat : 27995 (plingeling) R 27963 27965 27724 0 -1 4202560 1404578 1250 0 0 494519 2294 0 0 25 0 33 0 721843173 36171452416 8690331 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=27965/tid=27996] ppid=27963 vsize=35323684 CPUtime=4967.86 cores=0-31
/proc/27965/task/27996/stat : 27996 (plingeling) R 27963 27965 27724 0 -1 4202560 1547197 1250 0 0 494583 2203 0 0 25 0 33 0 721843173 36171452416 8690331 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=27965/tid=27997] ppid=27963 vsize=35323684 CPUtime=4967.86 cores=0-31
/proc/27965/task/27997/stat : 27997 (plingeling) R 27963 27965 27724 0 -1 4202560 915558 1250 0 0 495654 1132 0 0 25 0 33 0 721843173 36171452416 8690331 268435456000 4194304 4417452 140737088365296 18446744073709551615 4354506 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=27965/tid=27998] ppid=27963 vsize=35323684 CPUtime=4971.34 cores=0-31
/proc/27965/task/27998/stat : 27998 (plingeling) R 27963 27965 27724 0 -1 4202560 923745 1250 0 0 496014 1120 0 0 25 0 33 0 721843173 36171452416 8690331 268435456000 4194304 4417452 140737088365296 18446744073709551615 4263109 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=27965/tid=27999] ppid=27963 vsize=35323684 CPUtime=4969.78 cores=0-31
/proc/27965/task/27999/stat : 27999 (plingeling) R 27963 27965 27724 0 -1 4202560 1328326 1250 0 0 494712 2266 0 0 25 0 33 0 721843173 36171452416 8690331 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=27965/tid=28000] ppid=27963 vsize=35323684 CPUtime=4960.84 cores=0-31
/proc/27965/task/28000/stat : 28000 (plingeling) R 27963 27965 27724 0 -1 4202560 1173722 1250 0 0 494534 1550 0 0 25 0 33 0 721843173 36171452416 8690331 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364741 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=27965/tid=28001] ppid=27963 vsize=35323684 CPUtime=4971.34 cores=0-31
/proc/27965/task/28001/stat : 28001 (plingeling) R 27963 27965 27724 0 -1 4202560 951146 1250 0 0 496126 1008 0 0 25 0 33 0 721843173 36171452416 8690331 268435456000 4194304 4417452 140737088365296 18446744073709551615 4357207 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=27965/tid=28002] ppid=27963 vsize=35323684 CPUtime=4963.32 cores=0-31
/proc/27965/task/28002/stat : 28002 (plingeling) R 27963 27965 27724 0 -1 4202560 1140183 1250 0 0 494709 1623 0 0 25 0 33 0 721843173 36171452416 8690331 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=27965/tid=28003] ppid=27963 vsize=35323684 CPUtime=4973.52 cores=0-31
/proc/27965/task/28003/stat : 28003 (plingeling) R 27963 27965 27724 0 -1 4202560 1266769 1250 0 0 495363 1989 0 0 25 0 33 0 721843173 36171452416 8690331 268435456000 4194304 4417452 140737088365296 18446744073709551615 4353830 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=27965/tid=28004] ppid=27963 vsize=35323684 CPUtime=4965.91 cores=0-31
/proc/27965/task/28004/stat : 28004 (plingeling) R 27963 27965 27724 0 -1 4202560 1518589 1250 0 0 494380 2211 0 0 25 0 33 0 721843179 36171452416 8690331 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27965/tid=28005] ppid=27963 vsize=35323684 CPUtime=4966.08 cores=0-31
/proc/27965/task/28005/stat : 28005 (plingeling) R 27963 27965 27724 0 -1 4202560 833963 1250 0 0 495600 1008 0 0 25 0 33 0 721843204 36171452416 8690331 268435456000 4194304 4417452 140737088365296 18446744073709551615 4357436 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 158991
Current children cumulated vsize (KiB) 35323684



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5054 s]
/proc/loadavg: 32.12 32.11 32.06 33/446 28151
/proc/meminfo: memFree=220098768/264257624 swapFree=67111528/67111528
[pid=27965] ppid=27963 vsize=35878292 CPUtime=160003 cores=0-31
/proc/27965/stat : 27965 (plingeling) S 27963 27965 27724 0 -1 4202496 37989449 1250 0 0 15945068 55212 0 0 18 0 33 0 721838921 36739371008 8826047 268435456000 4194304 4417452 140737088365296 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 20 0 0 0
/proc/27965/statm: 8969573 8826055 175 55 0 8967459 0
[pid=27965/tid=27974] ppid=27963 vsize=35878292 CPUtime=4999.63 cores=0-31
/proc/27965/task/27974/stat : 27974 (plingeling) R 27963 27965 27724 0 -1 4202560 1211584 1250 0 0 498252 1711 0 0 25 0 33 0 721843172 36739371008 8826098 268435456000 4194304 4417452 140737088365296 18446744073709551615 4263136 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=27965/tid=27975] ppid=27963 vsize=35878292 CPUtime=4995.49 cores=0-31
/proc/27965/task/27975/stat : 27975 (plingeling) R 27963 27965 27724 0 -1 4202560 1119446 1250 0 0 497722 1827 0 0 25 0 33 0 721843172 36739371008 8826110 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=27965/tid=27976] ppid=27963 vsize=35878292 CPUtime=4991.81 cores=0-31
/proc/27965/task/27976/stat : 27976 (plingeling) R 27963 27965 27724 0 -1 4202560 2039690 1250 0 0 495626 3555 0 0 25 0 33 0 721843172 36739371008 8826121 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=27965/tid=27977] ppid=27963 vsize=35878292 CPUtime=5004.71 cores=0-31
/proc/27965/task/27977/stat : 27977 (plingeling) R 27963 27965 27724 0 -1 4202560 1950622 1250 0 0 496694 3777 0 0 25 0 33 0 721843172 36739371008 8826131 268435456000 4194304 4417452 140737088365296 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=27965/tid=27978] ppid=27963 vsize=35878292 CPUtime=5004.55 cores=0-31
/proc/27965/task/27978/stat : 27978 (plingeling) R 27963 27965 27724 0 -1 4202560 1318262 1250 0 0 498475 1980 0 0 25 0 33 0 721843172 36739371008 8826141 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=27965/tid=27979] ppid=27963 vsize=35878292 CPUtime=5002.23 cores=0-31
/proc/27965/task/27979/stat : 27979 (plingeling) R 27963 27965 27724 0 -1 4202560 1168362 1250 0 0 498520 1703 0 0 25 0 33 0 721843172 36739371008 8826152 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364604 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=27965/tid=27980] ppid=27963 vsize=35878292 CPUtime=4986.41 cores=0-31
/proc/27965/task/27980/stat : 27980 (plingeling) R 27963 27965 27724 0 -1 4202560 902522 1250 0 0 497265 1376 0 0 25 0 33 0 721843172 36739371008 8826161 268435456000 4194304 4417452 140737088365296 18446744073709551615 4366141 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=27965/tid=27981] ppid=27963 vsize=35878292 CPUtime=5002.82 cores=0-31
/proc/27965/task/27981/stat : 27981 (plingeling) R 27963 27965 27724 0 -1 4202560 1044097 1250 0 0 498795 1487 0 0 25 0 33 0 721843172 36739371008 8826172 268435456000 4194304 4417452 140737088365296 18446744073709551615 258650649918 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=27965/tid=27982] ppid=27963 vsize=35878292 CPUtime=5002.9 cores=0-31
/proc/27965/task/27982/stat : 27982 (plingeling) R 27963 27965 27724 0 -1 4202560 1045013 1250 0 0 498856 1434 0 0 25 0 33 0 721843172 36739371008 8826182 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=27965/tid=27983] ppid=27963 vsize=35878292 CPUtime=4998.81 cores=0-31
/proc/27965/task/27983/stat : 27983 (plingeling) R 27963 27965 27724 0 -1 4202560 742297 1250 0 0 499058 823 0 0 25 0 33 0 721843172 36739371008 8826192 268435456000 4194304 4417452 140737088365296 18446744073709551615 4354532 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=27965/tid=27984] ppid=27963 vsize=35878292 CPUtime=5002.07 cores=0-31
/proc/27965/task/27984/stat : 27984 (plingeling) R 27963 27965 27724 0 -1 4202560 1503561 1250 0 0 497577 2630 0 0 25 0 33 0 721843172 36739371008 8826203 268435456000 4194304 4417452 140737088365296 18446744073709551615 4346372 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=27965/tid=27985] ppid=27963 vsize=35878292 CPUtime=4988.81 cores=0-31
/proc/27965/task/27985/stat : 27985 (plingeling) R 27963 27965 27724 0 -1 4202560 960287 1250 0 0 497612 1269 0 0 25 0 33 0 721843172 36739371008 8826213 268435456000 4194304 4417452 140737088365296 18446744073709551615 4365027 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=27965/tid=27986] ppid=27963 vsize=35878292 CPUtime=5000.37 cores=0-31
/proc/27965/task/27986/stat : 27986 (plingeling) R 27963 27965 27724 0 -1 4202560 956334 1250 0 0 498595 1442 0 0 25 0 33 0 721843172 36739371008 8826223 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=27965/tid=27987] ppid=27963 vsize=35878292 CPUtime=5002.93 cores=0-31
/proc/27965/task/27987/stat : 27987 (plingeling) R 27963 27965 27724 0 -1 4202560 1031874 1250 0 0 498910 1383 0 0 25 0 33 0 721843172 36739371008 8826233 268435456000 4194304 4417452 140737088365296 18446744073709551615 4357168 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=27965/tid=27988] ppid=27963 vsize=35878292 CPUtime=4996.87 cores=0-31
/proc/27965/task/27988/stat : 27988 (plingeling) R 27963 27965 27724 0 -1 4202560 1594505 1250 0 0 496956 2731 0 0 25 0 33 0 721843172 36739371008 8826244 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=27965/tid=27989] ppid=27963 vsize=35878292 CPUtime=5002.8 cores=0-31
/proc/27965/task/27989/stat : 27989 (plingeling) R 27963 27965 27724 0 -1 4202560 858677 1250 0 0 499437 843 0 0 25 0 33 0 721843172 36739371008 8826254 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=27965/tid=27990] ppid=27963 vsize=35878292 CPUtime=5001.24 cores=0-31
/proc/27965/task/27990/stat : 27990 (plingeling) R 27963 27965 27724 0 -1 4202560 336812 1250 0 0 499936 188 0 0 25 0 33 0 721843172 36739371008 8826265 268435456000 4194304 4417452 140737088365296 18446744073709551615 4357013 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=27965/tid=27991] ppid=27963 vsize=35878292 CPUtime=5000.99 cores=0-31
/proc/27965/task/27991/stat : 27991 (plingeling) R 27963 27965 27724 0 -1 4202560 1033346 1250 0 0 498836 1263 0 0 25 0 33 0 721843172 36739371008 8826276 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=27965/tid=27992] ppid=27963 vsize=35878292 CPUtime=4997.39 cores=0-31
/proc/27965/task/27992/stat : 27992 (plingeling) R 27963 27965 27724 0 -1 4202560 1286096 1250 0 0 497936 1803 0 0 25 0 33 0 721843173 36739371008 8826286 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=27965/tid=27993] ppid=27963 vsize=35878292 CPUtime=4995.85 cores=0-31
/proc/27965/task/27993/stat : 27993 (plingeling) R 27963 27965 27724 0 -1 4202560 1199198 1250 0 0 497841 1744 0 0 25 0 33 0 721843173 36739371008 8826296 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=27965/tid=27994] ppid=27963 vsize=35878292 CPUtime=4987.28 cores=0-31
/proc/27965/task/27994/stat : 27994 (plingeling) R 27963 27965 27724 0 -1 4202560 1113650 1250 0 0 497318 1410 0 0 25 0 33 0 721843173 36739371008 8826307 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=27965/tid=27995] ppid=27963 vsize=35878292 CPUtime=4999.85 cores=0-31
/proc/27965/task/27995/stat : 27995 (plingeling) R 27963 27965 27724 0 -1 4202560 1404578 1250 0 0 497691 2294 0 0 25 0 33 0 721843173 36739371008 8826316 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364741 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=27965/tid=27996] ppid=27963 vsize=35878292 CPUtime=4999.58 cores=0-31
/proc/27965/task/27996/stat : 27996 (plingeling) R 27963 27965 27724 0 -1 4202560 1547197 1250 0 0 497755 2203 0 0 25 0 33 0 721843173 36739371008 8826331 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=27965/tid=27997] ppid=27963 vsize=35878292 CPUtime=4998.78 cores=0-31
/proc/27965/task/27997/stat : 27997 (plingeling) R 27963 27965 27724 0 -1 4202560 1068430 1250 0 0 498425 1453 0 0 25 0 33 0 721843173 36739371008 8826341 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=27965/tid=27998] ppid=27963 vsize=35878292 CPUtime=5003.06 cores=0-31
/proc/27965/task/27998/stat : 27998 (plingeling) R 27963 27965 27724 0 -1 4202560 923745 1250 0 0 499186 1120 0 0 25 0 33 0 721843173 36739371008 8826351 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=27965/tid=27999] ppid=27963 vsize=35878292 CPUtime=5001.5 cores=0-31
/proc/27965/task/27999/stat : 27999 (plingeling) R 27963 27965 27724 0 -1 4202560 1328326 1250 0 0 497884 2266 0 0 25 0 33 0 721843173 36739371008 8826361 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=27965/tid=28000] ppid=27963 vsize=35878292 CPUtime=4992.56 cores=0-31
/proc/27965/task/28000/stat : 28000 (plingeling) R 27963 27965 27724 0 -1 4202560 1173722 1250 0 0 497706 1550 0 0 25 0 33 0 721843173 36739371008 8826371 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=27965/tid=28001] ppid=27963 vsize=35878292 CPUtime=5003.05 cores=0-31
/proc/27965/task/28001/stat : 28001 (plingeling) R 27963 27965 27724 0 -1 4202560 951146 1250 0 0 499297 1008 0 0 25 0 33 0 721843173 36739371008 8826380 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=27965/tid=28002] ppid=27963 vsize=35878292 CPUtime=4995.03 cores=0-31
/proc/27965/task/28002/stat : 28002 (plingeling) R 27963 27965 27724 0 -1 4202560 1142423 1250 0 0 497874 1629 0 0 25 0 33 0 721843173 36739371008 8826390 268435456000 4194304 4417452 140737088365296 18446744073709551615 4277545 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=27965/tid=28003] ppid=27963 vsize=35878292 CPUtime=5005.23 cores=0-31
/proc/27965/task/28003/stat : 28003 (plingeling) R 27963 27965 27724 0 -1 4202560 1266769 1250 0 0 498534 1989 0 0 25 0 33 0 721843173 36739371008 8826401 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=27965/tid=28004] ppid=27963 vsize=35878292 CPUtime=4997.61 cores=0-31
/proc/27965/task/28004/stat : 28004 (plingeling) R 27963 27965 27724 0 -1 4202560 1518589 1250 0 0 497550 2211 0 0 25 0 33 0 721843179 36739371008 8826410 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=27965/tid=28005] ppid=27963 vsize=35878292 CPUtime=4997.79 cores=0-31
/proc/27965/task/28005/stat : 28005 (plingeling) R 27963 27965 27724 0 -1 4202560 833963 1250 0 0 498771 1008 0 0 25 0 33 0 721843204 36739371008 8826421 268435456000 4194304 4417452 140737088365296 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 160003
Current children cumulated vsize (KiB) 35878292

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)

Sending SIGKILL to -27965

Solver just ended. Dumping a history of the last processes samples

[startup+5054.2 s]
/proc/loadavg: 32.12 32.11 32.06 33/446 28151
/proc/meminfo: memFree=220098768/264257624 swapFree=67111528/67111528
[pid=27965] ppid=27963 vsize=0 CPUtime=160003 cores=0-31
/proc/27965/stat : 27965 (plingeling) Z 27963 27965 27724 0 -1 4203548 37991535 1250 0 0 15945090 55229 0 0 18 0 2 0 721838921 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 20 0 0 0
/proc/27965/statm: 0 0 0 0 0 0 0
[pid=27965/tid=27974] ppid=27963 vsize=0 CPUtime=4999.81 cores=0-31
/proc/27965/task/27974/stat : 27974 (plingeling) R 27963 27965 27724 0 -1 4203588 1211584 1250 0 0 498253 1728 0 0 25 0 2 0 721843172 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 20 0 0 0
Current children cumulated CPU time (s) 160003
Current children cumulated vsize (KiB) 0

[startup+5055 s]
/proc/loadavg: 32.12 32.11 32.06 2/416 28152
/proc/meminfo: memFree=224438140/264257624 swapFree=67111528/67111528
[pid=27965] ppid=27963 vsize=0 CPUtime=160004 cores=0-31
/proc/27965/stat : 27965 (plingeling) Z 27963 27965 27724 0 -1 4203548 37991535 1250 0 0 15945090 55309 0 0 18 0 2 0 721838921 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 20 0 0 0
/proc/27965/statm: 0 0 0 0 0 0 0
[pid=27965/tid=27974] ppid=27963 vsize=0 CPUtime=5000.61 cores=0-31
/proc/27965/task/27974/stat : 27974 (plingeling) R 27963 27965 27724 0 -1 4203588 1211584 1250 0 0 498253 1808 0 0 25 0 2 0 721843172 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 20 0 0 0
Current children cumulated CPU time (s) 160004
Current children cumulated vsize (KiB) 0

[startup+5055.8 s]
/proc/loadavg: 32.12 32.11 32.06 2/416 28152
/proc/meminfo: memFree=244557872/264257624 swapFree=67111528/67111528
[pid=27965] ppid=27963 vsize=0 CPUtime=160005 cores=0-31
/proc/27965/stat : 27965 (plingeling) Z 27963 27965 27724 0 -1 4203548 37991535 1250 0 0 15945090 55389 0 0 18 0 2 0 721838921 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 20 0 0 0
/proc/27965/statm: 0 0 0 0 0 0 0
[pid=27965/tid=27974] ppid=27963 vsize=0 CPUtime=5001.41 cores=0-31
/proc/27965/task/27974/stat : 27974 (plingeling) R 27963 27965 27724 0 -1 4203588 1211584 1250 0 0 498253 1888 0 0 25 0 2 0 721843172 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 20 0 0 0
Current children cumulated CPU time (s) 160005
Current children cumulated vsize (KiB) 0

[startup+5056 s]
/proc/loadavg: 32.12 32.11 32.06 2/416 28152
/proc/meminfo: memFree=244557872/264257624 swapFree=67111528/67111528
[pid=27965] ppid=27963 vsize=0 CPUtime=160005 cores=0-31
/proc/27965/stat : 27965 (plingeling) Z 27963 27965 27724 0 -1 4203548 37991535 1250 0 0 15945090 55409 0 0 18 0 2 0 721838921 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 20 0 0 0
/proc/27965/statm: 0 0 0 0 0 0 0
[pid=27965/tid=27974] ppid=27963 vsize=0 CPUtime=5001.61 cores=0-31
/proc/27965/task/27974/stat : 27974 (plingeling) R 27963 27965 27724 0 -1 4203588 1211584 1250 0 0 498253 1908 0 0 25 0 2 0 721843172 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 20 0 0 0
Current children cumulated CPU time (s) 160005
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5056.09
CPU time (s): 160005
CPU user time (s): 159451
CPU system time (s): 554.193
CPU usage (%): 3164.6
Max. virtual memory (cumulated for all children) (KiB): 37137736

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 159451
system time used= 554.193
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 37992785
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 16336
involuntary context switches= 406555

runsolver used 26.213 second user time and 116.677 second system time

The end

Launcher Data 

Begin job on node200 at 2011-05-23 00:32:05
IDJOB=3372852
IDBENCH=84248
IDSOLVER=1852
FILE ID=node200/3372852-1306103524
RUNJOBID= node200-1306098483-27738
PBS_JOBID= 13387822
Free space on /tmp= 62032 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT03/industrial/schuppan/l2s/dme-03-1-k247-unsat.shuffled-as.sat03-407.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3372852-1306103524/watcher-3372852-1306103524 -o /tmp/evaluation-result-3372852-1306103524/solver-3372852-1306103524 -C 160000 -W 5100 -M 266338304  plingeling -t 32 HOME/instance-3372852-1306103524.cnf

TIME LIMIT= 160000 seconds
MEMORY LIMIT= 266338304 MiB
NBCORE= 32

MD5SUM BENCH= 3fef216b3efe77c0f10ea0e0be664a94
RANDOM SEED=1436812812

node200.alineos.net Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 64
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 96
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 66
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 98
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 68
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 100
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 70
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.98
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 102
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 16
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 17
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 48
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 18
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 80
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 19
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 112
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 20
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 21
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 50
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 22
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 82
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 114
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.23
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 24
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 25
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 26
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 84
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 27
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 116
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 28
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 29
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 54
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 30
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 86
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 31
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 118
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     264257624 kB
MemFree:      255442164 kB
Buffers:        455984 kB
Cached:        7955548 kB
SwapCached:          0 kB
Active:        7508936 kB
Inactive:       971696 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     264257624 kB
LowFree:      255442164 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           15380 kB
Writeback:           0 kB
AnonPages:       69120 kB
Mapped:          14532 kB
Slab:           253148 kB
PageTables:       4452 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  199240340 kB
Committed_AS:   199608 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296372 kB
VmallocChunk: 34359440603 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 62032 MiB
End job on node200 at 2011-05-23 01:56:21