Trace number 3372426

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 158322 5102.49

General information on the benchmark

Nameapplication/rintanen/SATPlanning/
openstacks-sequencedstrips-nonadl-nonnegated-os-sequencedstrips-p30_3.035-NOTKNOWN.cnf
MD5SUMc4319228342e10028de028a9b67c3404
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables133566
Number of clauses676901
Sum of the clauses size1609791
Maximum clause length63
Minimum clause length1
Number of clauses of size 1211
Number of clauses of size 2668220
Number of clauses of size 32100
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 56370

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3372426-1305594093/watcher-3372426-1305594093 -o /tmp/evaluation-result-3372426-1305594093/solver-3372426-1305594093 -C 160000 -W 5100 -M 266338304 plingeling -t 32 HOME/instance-3372426-1305594093.cnf 

running on 32 cores: 0,4,8,12,16,20,24,28,1,5,9,13,17,21,25,29,2,6,10,14,18,22,26,30,3,7,11,15,19,23,27,31

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 160000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 160030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 272730423296 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 272730474496 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 27.36 31.07 30.80 1/413 22397
/proc/meminfo: memFree=255516356/264257624 swapFree=67111528/67111528
[pid=22397] ppid=22395 vsize=16932 CPUtime=0.02 cores=0-31
/proc/22397/stat : 22397 (plingeling) R 22395 22397 22371 0 -1 4202496 2317 1240 0 0 2 0 0 0 21 0 1 0 670895786 17338368 2247 268435456000 4194304 4417452 140736020152880 18446744073709551615 4235999 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/22397/statm: 4233 2247 141 55 0 2119 0

[startup+0.109184 s]
/proc/loadavg: 27.36 31.07 30.80 1/413 22397
/proc/meminfo: memFree=255516356/264257624 swapFree=67111528/67111528
[pid=22397] ppid=22395 vsize=23192 CPUtime=0.09 cores=0-31
/proc/22397/stat : 22397 (plingeling) R 22395 22397 22371 0 -1 4202496 3883 1240 0 0 9 0 0 0 21 0 1 0 670895786 23748608 3813 268435456000 4194304 4417452 140736020152880 18446744073709551615 4231277 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/22397/statm: 5798 3813 144 55 0 3684 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 23192

[startup+0.20116 s]
/proc/loadavg: 27.36 31.07 30.80 1/413 22397
/proc/meminfo: memFree=255516356/264257624 swapFree=67111528/67111528
[pid=22397] ppid=22395 vsize=36152 CPUtime=0.19 cores=0-31
/proc/22397/stat : 22397 (plingeling) R 22395 22397 22371 0 -1 4202496 7105 1240 0 0 18 1 0 0 21 0 1 0 670895786 37019648 7035 268435456000 4194304 4417452 140736020152880 18446744073709551615 4235968 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/22397/statm: 9038 7035 146 55 0 6924 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 36152

[startup+0.301144 s]
/proc/loadavg: 27.36 31.07 30.80 1/413 22397
/proc/meminfo: memFree=255516356/264257624 swapFree=67111528/67111528
[pid=22397] ppid=22395 vsize=55852 CPUtime=0.29 cores=0-31
/proc/22397/stat : 22397 (plingeling) R 22395 22397 22371 0 -1 4202496 12038 1240 0 0 27 2 0 0 22 0 1 0 670895786 57192448 11968 268435456000 4194304 4417452 140736020152880 18446744073709551615 4209658 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/22397/statm: 13963 11968 146 55 0 11849 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 55852

[startup+0.701169 s]
/proc/loadavg: 27.36 31.07 30.80 1/413 22397
/proc/meminfo: memFree=255516356/264257624 swapFree=67111528/67111528
[pid=22397] ppid=22395 vsize=58228 CPUtime=0.69 cores=0-31
/proc/22397/stat : 22397 (plingeling) R 22395 22397 22371 0 -1 4202496 12632 1240 0 0 67 2 0 0 23 0 1 0 670895786 59625472 12562 268435456000 4194304 4417452 140736020152880 18446744073709551615 4231454 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/22397/statm: 14557 12562 146 55 0 12443 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 58228

[startup+1.50119 s]
/proc/loadavg: 25.25 30.57 30.64 2/414 22403
/proc/meminfo: memFree=255413980/264257624 swapFree=67111528/67111528
[pid=22397] ppid=22395 vsize=114648 CPUtime=1.49 cores=0-31
/proc/22397/stat : 22397 (plingeling) R 22395 22397 22371 0 -1 4202496 26753 1240 0 0 144 5 0 0 25 0 1 0 670895786 117399552 26683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4235985 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/22397/statm: 28662 26683 146 55 0 26548 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 114648

[startup+3.10125 s]
/proc/loadavg: 25.25 30.57 30.64 2/414 22403
/proc/meminfo: memFree=255306084/264257624 swapFree=67111528/67111528
[pid=22397] ppid=22395 vsize=252308 CPUtime=3.09 cores=0-31
/proc/22397/stat : 22397 (plingeling) R 22395 22397 22371 0 -1 4202496 61168 1240 0 0 296 13 0 0 25 0 1 0 670895786 258363392 61098 268435456000 4194304 4417452 140736020152880 18446744073709551615 4259140 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/22397/statm: 63077 61098 146 55 0 60963 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 252308

[startup+6.30036 s]
/proc/loadavg: 25.25 30.57 30.64 2/414 22403
/proc/meminfo: memFree=255042260/264257624 swapFree=67111528/67111528
[pid=22397] ppid=22395 vsize=481276 CPUtime=6.28 cores=0-31
/proc/22397/stat : 22397 (plingeling) R 22395 22397 22371 0 -1 4202496 118410 1240 0 0 604 24 0 0 25 0 1 0 670895786 492826624 118340 268435456000 4194304 4417452 140736020152880 18446744073709551615 4264940 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/22397/statm: 120319 118340 146 55 0 118205 0
Current children cumulated CPU time (s) 6.28
Current children cumulated vsize (KiB) 481276

[startup+12.7025 s]
/proc/loadavg: 24.00 30.11 30.49 33/446 22435
/proc/meminfo: memFree=254623992/264257624 swapFree=67111528/67111528
[pid=22397] ppid=22395 vsize=1277168 CPUtime=132.25 cores=0-31
/proc/22397/stat : 22397 (plingeling) S 22395 22397 22371 0 -1 4202496 518382 1240 0 0 12999 226 0 0 18 0 33 0 670895786 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/22397/statm: 319292 213378 174 55 0 317178 0
[pid=22397/tid=22404] ppid=22395 vsize=1277168 CPUtime=3.87 cores=0-31
/proc/22397/task/22404/stat : 22404 (plingeling) R 22395 22397 22371 0 -1 4202560 6723 1240 0 0 382 5 0 0 25 0 33 0 670896660 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4331796 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=22397/tid=22405] ppid=22395 vsize=1277168 CPUtime=3.88 cores=0-31
/proc/22397/task/22405/stat : 22405 (plingeling) R 22395 22397 22371 0 -1 4202560 10692 1240 0 0 381 7 0 0 25 0 33 0 670896660 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4268586 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22397/tid=22406] ppid=22395 vsize=1277168 CPUtime=3.87 cores=0-31
/proc/22397/task/22406/stat : 22406 (plingeling) R 22395 22397 22371 0 -1 4202560 12732 1240 0 0 381 6 0 0 25 0 33 0 670896660 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4233384 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=22397/tid=22407] ppid=22395 vsize=1277168 CPUtime=3.9 cores=0-31
/proc/22397/task/22407/stat : 22407 (plingeling) R 22395 22397 22371 0 -1 4202560 2905 1240 0 0 385 5 0 0 25 0 33 0 670896660 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4357283 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22397/tid=22408] ppid=22395 vsize=1277168 CPUtime=3.84 cores=0-31
/proc/22397/task/22408/stat : 22408 (plingeling) R 22395 22397 22371 0 -1 4202560 6920 1240 0 0 379 5 0 0 25 0 33 0 670896660 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4259147 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=22397/tid=22409] ppid=22395 vsize=1277168 CPUtime=3.89 cores=0-31
/proc/22397/task/22409/stat : 22409 (plingeling) R 22395 22397 22371 0 -1 4202560 15635 1240 0 0 382 7 0 0 25 0 33 0 670896660 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4331796 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22397/tid=22410] ppid=22395 vsize=1277168 CPUtime=3.86 cores=0-31
/proc/22397/task/22410/stat : 22410 (plingeling) R 22395 22397 22371 0 -1 4202560 12327 1240 0 0 382 4 0 0 25 0 33 0 670896661 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4330515 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=22397/tid=22411] ppid=22395 vsize=1277168 CPUtime=3.88 cores=0-31
/proc/22397/task/22411/stat : 22411 (plingeling) R 22395 22397 22371 0 -1 4202560 8687 1240 0 0 381 7 0 0 25 0 33 0 670896661 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4346458 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=22397/tid=22412] ppid=22395 vsize=1277168 CPUtime=3.86 cores=0-31
/proc/22397/task/22412/stat : 22412 (plingeling) R 22395 22397 22371 0 -1 4202560 11485 1240 0 0 380 6 0 0 25 0 33 0 670896661 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4317337 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=22397/tid=22413] ppid=22395 vsize=1277168 CPUtime=3.84 cores=0-31
/proc/22397/task/22413/stat : 22413 (plingeling) R 22395 22397 22371 0 -1 4202560 10330 1240 0 0 378 6 0 0 25 0 33 0 670896661 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4330445 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=22397/tid=22414] ppid=22395 vsize=1277168 CPUtime=3.89 cores=0-31
/proc/22397/task/22414/stat : 22414 (plingeling) R 22395 22397 22371 0 -1 4202560 8862 1240 0 0 382 7 0 0 25 0 33 0 670896661 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4330515 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=22397/tid=22415] ppid=22395 vsize=1277168 CPUtime=3.9 cores=0-31
/proc/22397/task/22415/stat : 22415 (plingeling) R 22395 22397 22371 0 -1 4202560 6054 1240 0 0 384 6 0 0 25 0 33 0 670896662 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4255694 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=22397/tid=22416] ppid=22395 vsize=1277168 CPUtime=3.9 cores=0-31
/proc/22397/task/22416/stat : 22416 (plingeling) R 22395 22397 22371 0 -1 4202560 9050 1240 0 0 383 7 0 0 25 0 33 0 670896662 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4265832 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=22397/tid=22417] ppid=22395 vsize=1277168 CPUtime=3.91 cores=0-31
/proc/22397/task/22417/stat : 22417 (plingeling) R 22395 22397 22371 0 -1 4202560 7282 1240 0 0 384 7 0 0 25 0 33 0 670896662 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4270883 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=22397/tid=22418] ppid=22395 vsize=1277168 CPUtime=3.9 cores=0-31
/proc/22397/task/22418/stat : 22418 (plingeling) R 22395 22397 22371 0 -1 4202560 8299 1240 0 0 386 4 0 0 25 0 33 0 670896662 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4330515 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22397/tid=22419] ppid=22395 vsize=1277168 CPUtime=3.87 cores=0-31
/proc/22397/task/22419/stat : 22419 (plingeling) R 22395 22397 22371 0 -1 4202560 6672 1240 0 0 384 3 0 0 25 0 33 0 670896662 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4234135 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=22397/tid=22420] ppid=22395 vsize=1277168 CPUtime=3.85 cores=0-31
/proc/22397/task/22420/stat : 22420 (plingeling) R 22395 22397 22371 0 -1 4202560 9570 1240 0 0 378 7 0 0 25 0 33 0 670896662 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4330445 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=22397/tid=22421] ppid=22395 vsize=1277168 CPUtime=3.79 cores=0-31
/proc/22397/task/22421/stat : 22421 (plingeling) R 22395 22397 22371 0 -1 4202560 12470 1240 0 0 374 5 0 0 25 0 33 0 670896666 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4270966 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=22397/tid=22422] ppid=22395 vsize=1277168 CPUtime=3.85 cores=0-31
/proc/22397/task/22422/stat : 22422 (plingeling) R 22395 22397 22371 0 -1 4202560 6808 1240 0 0 382 3 0 0 25 0 33 0 670896667 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4320366 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=22397/tid=22423] ppid=22395 vsize=1277168 CPUtime=3.85 cores=0-31
/proc/22397/task/22423/stat : 22423 (plingeling) R 22395 22397 22371 0 -1 4202560 11610 1240 0 0 380 5 0 0 25 0 33 0 670896667 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4346647 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=22397/tid=22424] ppid=22395 vsize=1277168 CPUtime=3.85 cores=0-31
/proc/22397/task/22424/stat : 22424 (plingeling) R 22395 22397 22371 0 -1 4202560 9828 1240 0 0 381 4 0 0 25 0 33 0 670896667 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4346309 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=22397/tid=22425] ppid=22395 vsize=1277168 CPUtime=3.81 cores=0-31
/proc/22397/task/22425/stat : 22425 (plingeling) R 22395 22397 22371 0 -1 4202560 9857 1240 0 0 375 6 0 0 25 0 33 0 670896668 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4256889 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=22397/tid=22426] ppid=22395 vsize=1277168 CPUtime=3.85 cores=0-31
/proc/22397/task/22426/stat : 22426 (plingeling) R 22395 22397 22371 0 -1 4202560 9336 1240 0 0 380 5 0 0 25 0 33 0 670896668 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4330789 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=22397/tid=22427] ppid=22395 vsize=1277168 CPUtime=3.84 cores=0-31
/proc/22397/task/22427/stat : 22427 (plingeling) R 22395 22397 22371 0 -1 4202560 5016 1240 0 0 377 7 0 0 25 0 33 0 670896668 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4346458 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22397/tid=22428] ppid=22395 vsize=1277168 CPUtime=3.83 cores=0-31
/proc/22397/task/22428/stat : 22428 (plingeling) R 22395 22397 22371 0 -1 4202560 10422 1240 0 0 376 7 0 0 25 0 33 0 670896668 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4330612 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=22397/tid=22429] ppid=22395 vsize=1277168 CPUtime=3.85 cores=0-31
/proc/22397/task/22429/stat : 22429 (plingeling) R 22395 22397 22371 0 -1 4202560 2768 1240 0 0 383 2 0 0 20 0 33 0 670896668 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4274888 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22397/tid=22430] ppid=22395 vsize=1277168 CPUtime=3.73 cores=0-31
/proc/22397/task/22430/stat : 22430 (plingeling) R 22395 22397 22371 0 -1 4202560 7098 1240 0 0 370 3 0 0 25 0 33 0 670896668 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4346458 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=22397/tid=22431] ppid=22395 vsize=1277168 CPUtime=3.8 cores=0-31
/proc/22397/task/22431/stat : 22431 (plingeling) R 22395 22397 22371 0 -1 4202560 10105 1240 0 0 377 3 0 0 25 0 33 0 670896668 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4317286 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=22397/tid=22432] ppid=22395 vsize=1277168 CPUtime=3.83 cores=0-31
/proc/22397/task/22432/stat : 22432 (plingeling) R 22395 22397 22371 0 -1 4202560 6210 1240 0 0 380 3 0 0 25 0 33 0 670896669 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4268638 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=22397/tid=22433] ppid=22395 vsize=1277168 CPUtime=3.82 cores=0-31
/proc/22397/task/22433/stat : 22433 (plingeling) R 22395 22397 22371 0 -1 4202560 10056 1240 0 0 379 3 0 0 25 0 33 0 670896670 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4275475 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=22397/tid=22434] ppid=22395 vsize=1277168 CPUtime=3.82 cores=0-31
/proc/22397/task/22434/stat : 22434 (plingeling) R 22395 22397 22371 0 -1 4202560 9239 1240 0 0 378 4 0 0 25 0 33 0 670896670 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4320498 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=22397/tid=22435] ppid=22395 vsize=1277168 CPUtime=3.79 cores=0-31
/proc/22397/task/22435/stat : 22435 (plingeling) R 22395 22397 22371 0 -1 4202560 12997 1240 0 0 374 5 0 0 25 0 33 0 670896670 1307820032 213378 268435456000 4194304 4417452 140736020152880 18446744073709551615 4234264 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 132.25
Current children cumulated vsize (KiB) 1277168

[startup+25.5019 s]
/proc/loadavg: 25.23 30.18 30.50 33/446 22437
/proc/meminfo: memFree=254377360/264257624 swapFree=67111528/67111528
[pid=22397] ppid=22395 vsize=1612752 CPUtime=541.3 cores=0-31
/proc/22397/stat : 22397 (plingeling) S 22395 22397 22371 0 -1 4202496 607965 1240 0 0 53876 254 0 0 18 0 33 0 670895786 1651458048 282739 268435456000 4194304 4417452 140736020152880 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/22397/statm: 403188 282739 175 55 0 401074 0
[pid=22397/tid=22404] ppid=22395 vsize=1612752 CPUtime=16.67 cores=0-31
/proc/22397/task/22404/stat : 22404 (plingeling) R 22395 22397 22371 0 -1 4202560 10185 1240 0 0 1661 6 0 0 25 0 33 0 670896660 1651458048 282739 268435456000 4194304 4417452 140736020152880 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=22397/tid=22405] ppid=22395 vsize=1612752 CPUtime=16.67 cores=0-31
/proc/22397/task/22405/stat : 22405 (plingeling) R 22395 22397 22371 0 -1 4202560 12710 1240 0 0 1659 8 0 0 25 0 33 0 670896660 1651458048 282739 268435456000 4194304 4417452 140736020152880 18446744073709551615 4232598 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22397/tid=22406] ppid=22395 vsize=1612752 CPUtime=16.66 cores=0-31
/proc/22397/task/22406/stat : 22406 (plingeling) R 22395 22397 22371 0 -1 4202560 14958 1240 0 0 1660 6 0 0 25 0 33 0 670896660 1651458048 282739 268435456000 4194304 4417452 140736020152880 18446744073709551615 4356852 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=22397/tid=22407] ppid=22395 vsize=1612752 CPUtime=16.7 cores=0-31
/proc/22397/task/22407/stat : 22407 (plingeling) R 22395 22397 22371 0 -1 4202560 17000 1240 0 0 1661 9 0 0 25 0 33 0 670896660 1651458048 282739 268435456000 4194304 4417452 140736020152880 18446744073709551615 4357907 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22397/tid=22408] ppid=22395 vsize=1612752 CPUtime=16.65 cores=0-31
/proc/22397/task/22408/stat : 22408 (plingeling) R 22395 22397 22371 0 -1 4202560 7438 1240 0 0 1659 6 0 0 25 0 33 0 670896660 1651458048 282739 268435456000 4194304 4417452 140736020152880 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=22397/tid=22409] ppid=22395 vsize=1612752 CPUtime=16.68 cores=0-31
/proc/22397/task/22409/stat : 22409 (plingeling) R 22395 22397 22371 0 -1 4202560 19492 1240 0 0 1660 8 0 0 25 0 33 0 670896660 1651458048 282739 268435456000 4194304 4417452 140736020152880 18446744073709551615 4353557 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22397/tid=22410] ppid=22395 vsize=1612752 CPUtime=16.65 cores=0-31
/proc/22397/task/22410/stat : 22410 (plingeling) R 22395 22397 22371 0 -1 4202560 13970 1240 0 0 1661 4 0 0 25 0 33 0 670896661 1651458048 282739 268435456000 4194304 4417452 140736020152880 18446744073709551615 4365647 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=22397/tid=22411] ppid=22395 vsize=1612752 CPUtime=16.3 cores=0-31
/proc/22397/task/22411/stat : 22411 (plingeling) R 22395 22397 22371 0 -1 4202560 9813 1240 0 0 1623 7 0 0 25 0 33 0 670896661 1651458048 282739 268435456000 4194304 4417452 140736020152880 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=22397/tid=22412] ppid=22395 vsize=1612752 CPUtime=16.65 cores=0-31
/proc/22397/task/22412/stat : 22412 (plingeling) R 22395 22397 22371 0 -1 4202560 14643 1240 0 0 1658 7 0 0 18 0 33 0 670896661 1651458048 282739 268435456000 4194304 4417452 140736020152880 18446744073709551615 4354497 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=22397/tid=22413] ppid=22395 vsize=1612752 CPUtime=16.63 cores=0-31
/proc/22397/task/22413/stat : 22413 (plingeling) R 22395 22397 22371 0 -1 4202560 12705 1240 0 0 1656 7 0 0 25 0 33 0 670896661 1651458048 282739 268435456000 4194304 4417452 140736020152880 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=22397/tid=22414] ppid=22395 vsize=1612752 CPUtime=16.69 cores=0-31
/proc/22397/task/22414/stat : 22414 (plingeling) R 22395 22397 22371 0 -1 4202560 10500 1240 0 0 1662 7 0 0 25 0 33 0 670896661 1651458048 282739 268435456000 4194304 4417452 140736020152880 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=22397/tid=22415] ppid=22395 vsize=1612752 CPUtime=16.7 cores=0-31
/proc/22397/task/22415/stat : 22415 (plingeling) R 22395 22397 22371 0 -1 4202560 7822 1240 0 0 1663 7 0 0 25 0 33 0 670896662 1651458048 282739 268435456000 4194304 4417452 140736020152880 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=22397/tid=22416] ppid=22395 vsize=1612752 CPUtime=16.7 cores=0-31
/proc/22397/task/22416/stat : 22416 (plingeling) R 22395 22397 22371 0 -1 4202560 11278 1240 0 0 1662 8 0 0 25 0 33 0 670896662 1651458048 282739 268435456000 4194304 4417452 140736020152880 18446744073709551615 4232464 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=22397/tid=22417] ppid=22395 vsize=1612752 CPUtime=16.69 cores=0-31
/proc/22397/task/22417/stat : 22417 (plingeling) R 22395 22397 22371 0 -1 4202560 9583 1240 0 0 1662 7 0 0 25 0 33 0 670896662 1651458048 282739 268435456000 4194304 4417452 140736020152880 18446744073709551615 4364466 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=22397/tid=22418] ppid=22395 vsize=1612752 CPUtime=16.69 cores=0-31
/proc/22397/task/22418/stat : 22418 (plingeling) R 22395 22397 22371 0 -1 4202560 11788 1240 0 0 1664 5 0 0 25 0 33 0 670896662 1651458048 282739 268435456000 4194304 4417452 140736020152880 18446744073709551615 4265499 0 0 4096 17442 0 0 0 -1 7 0 0 0

################
# More data... #
################

[pid=22397/tid=22417] ppid=22395 vsize=30227240 CPUtime=4923.85 cores=0-31
/proc/22397/task/22417/stat : 22417 (plingeling) D 22395 22397 22371 0 -1 4202560 3993764 1240 0 0 486527 5858 0 0 19 0 33 0 670896662 30952693760 7422953 268435456000 4194304 4417452 140736020152880 18446744073709551615 258650999114 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=22397/tid=22418] ppid=22395 vsize=30227240 CPUtime=4918.24 cores=0-31
/proc/22397/task/22418/stat : 22418 (plingeling) D 22395 22397 22371 0 -1 4202560 3219344 1240 0 0 487640 4184 0 0 19 0 33 0 670896662 30952693760 7422953 268435456000 4194304 4417452 140736020152880 18446744073709551615 258651013786 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=22397/tid=22419] ppid=22395 vsize=30227240 CPUtime=4939 cores=0-31
/proc/22397/task/22419/stat : 22419 (plingeling) R 22395 22397 22371 0 -1 4202560 3640335 1240 0 0 488944 4956 0 0 25 0 33 0 670896662 30952693760 7422953 268435456000 4194304 4417452 140736020152880 18446744073709551615 4309183 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=22397/tid=22420] ppid=22395 vsize=30227240 CPUtime=4974.61 cores=0-31
/proc/22397/task/22420/stat : 22420 (plingeling) R 22395 22397 22371 0 -1 4202560 5650454 1240 0 0 488150 9311 0 0 25 0 33 0 670896662 30952693760 7422953 268435456000 4194304 4417452 140736020152880 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=22397/tid=22421] ppid=22395 vsize=30227240 CPUtime=4905.49 cores=0-31
/proc/22397/task/22421/stat : 22421 (plingeling) R 22395 22397 22371 0 -1 4202560 3823953 1240 0 0 485349 5200 0 0 25 0 33 0 670896666 30952693760 7422953 268435456000 4194304 4417452 140736020152880 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=22397/tid=22422] ppid=22395 vsize=30227240 CPUtime=4928.02 cores=0-31
/proc/22397/task/22422/stat : 22422 (plingeling) R 22395 22397 22371 0 -1 4202560 3834023 1240 0 0 487542 5260 0 0 25 0 33 0 670896667 30952693760 7422953 268435456000 4194304 4417452 140736020152880 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=22397/tid=22423] ppid=22395 vsize=30227240 CPUtime=4923.84 cores=0-31
/proc/22397/task/22423/stat : 22423 (plingeling) R 22395 22397 22371 0 -1 4202560 4308904 1240 0 0 486430 5954 0 0 20 0 33 0 670896667 30952693760 7422953 268435456000 4194304 4417452 140736020152880 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22397/tid=22424] ppid=22395 vsize=30227240 CPUtime=4949.32 cores=0-31
/proc/22397/task/22424/stat : 22424 (plingeling) R 22395 22397 22371 0 -1 4202560 4104343 1240 0 0 489043 5889 0 0 25 0 33 0 670896667 30952693760 7422953 268435456000 4194304 4417452 140736020152880 18446744073709551615 4364604 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=22397/tid=22425] ppid=22395 vsize=30227240 CPUtime=4943.89 cores=0-31
/proc/22397/task/22425/stat : 22425 (plingeling) D 22395 22397 22371 0 -1 4202560 4121279 1240 0 0 488423 5966 0 0 18 0 33 0 670896668 30952693760 7422953 268435456000 4194304 4417452 140736020152880 18446744073709551615 258650649923 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=22397/tid=22426] ppid=22395 vsize=30227240 CPUtime=4920.48 cores=0-31
/proc/22397/task/22426/stat : 22426 (plingeling) R 22395 22397 22371 0 -1 4202560 4249675 1240 0 0 485848 6200 0 0 25 0 33 0 670896668 30952693760 7422953 268435456000 4194304 4417452 140736020152880 18446744073709551615 4232654 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=22397/tid=22427] ppid=22395 vsize=30227240 CPUtime=4944.63 cores=0-31
/proc/22397/task/22427/stat : 22427 (plingeling) R 22395 22397 22371 0 -1 4202560 4183455 1240 0 0 488267 6196 0 0 25 0 33 0 670896668 30952693760 7422953 268435456000 4194304 4417452 140736020152880 18446744073709551615 4358008 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22397/tid=22428] ppid=22395 vsize=30227240 CPUtime=4890.27 cores=0-31
/proc/22397/task/22428/stat : 22428 (plingeling) R 22395 22397 22371 0 -1 4202560 4227012 1240 0 0 483312 5715 0 0 20 0 33 0 670896668 30952693760 7422953 268435456000 4194304 4417452 140736020152880 18446744073709551615 4346322 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=22397/tid=22429] ppid=22395 vsize=30227240 CPUtime=4895.18 cores=0-31
/proc/22397/task/22429/stat : 22429 (plingeling) R 22395 22397 22371 0 -1 4202560 4522964 1240 0 0 483705 5813 0 0 25 0 33 0 670896668 30952693760 7422953 268435456000 4194304 4417452 140736020152880 18446744073709551615 4357862 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=22397/tid=22430] ppid=22395 vsize=30227240 CPUtime=4941.54 cores=0-31
/proc/22397/task/22430/stat : 22430 (plingeling) R 22395 22397 22371 0 -1 4202560 4958199 1240 0 0 486049 8105 0 0 25 0 33 0 670896668 30952693760 7422953 268435456000 4194304 4417452 140736020152880 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=22397/tid=22431] ppid=22395 vsize=30227240 CPUtime=4897.78 cores=0-31
/proc/22397/task/22431/stat : 22431 (plingeling) R 22395 22397 22371 0 -1 4202560 4158405 1240 0 0 484113 5665 0 0 25 0 33 0 670896668 30952693760 7422953 268435456000 4194304 4417452 140736020152880 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22397/tid=22432] ppid=22395 vsize=30227240 CPUtime=4905.75 cores=0-31
/proc/22397/task/22432/stat : 22432 (plingeling) R 22395 22397 22371 0 -1 4202560 3658813 1240 0 0 486046 4529 0 0 21 0 33 0 670896669 30952693760 7422953 268435456000 4194304 4417452 140736020152880 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=22397/tid=22433] ppid=22395 vsize=30227240 CPUtime=4899.86 cores=0-31
/proc/22397/task/22433/stat : 22433 (plingeling) R 22395 22397 22371 0 -1 4202560 4190200 1240 0 0 484516 5470 0 0 25 0 33 0 670896670 30952693760 7422953 268435456000 4194304 4417452 140736020152880 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22397/tid=22434] ppid=22395 vsize=30227240 CPUtime=4935.45 cores=0-31
/proc/22397/task/22434/stat : 22434 (plingeling) R 22395 22397 22371 0 -1 4202560 3537999 1240 0 0 488519 5026 0 0 25 0 33 0 670896670 30952693760 7422953 268435456000 4194304 4417452 140736020152880 18446744073709551615 4354497 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=22397/tid=22435] ppid=22395 vsize=30227240 CPUtime=4913.77 cores=0-31
/proc/22397/task/22435/stat : 22435 (plingeling) R 22395 22397 22371 0 -1 4202560 3968061 1240 0 0 485979 5398 0 0 25 0 33 0 670896670 30952693760 7422953 268435456000 4194304 4417452 140736020152880 18446744073709551615 4357265 0 0 4096 17442 0 0 0 -1 21 0 0 0
Current children cumulated CPU time (s) 157831
Current children cumulated vsize (KiB) 30227240



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5100 s]
/proc/loadavg: 32.02 31.99 31.98 33/446 23057
/proc/meminfo: memFree=223169084/264257624 swapFree=67111528/67111528
[pid=22397] ppid=22395 vsize=32751072 CPUtime=158319 cores=0-31
/proc/22397/stat : 22397 (plingeling) S 22395 22397 22371 0 -1 4202496 136192635 1240 0 0 15636744 195141 0 0 18 0 33 0 670895786 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/22397/statm: 8187768 8074683 175 55 0 8185654 0
[pid=22397/tid=22404] ppid=22395 vsize=32751072 CPUtime=4967.65 cores=0-31
/proc/22397/task/22404/stat : 22404 (plingeling) R 22395 22397 22371 0 -1 4202560 3762014 1240 0 0 491213 5552 0 0 25 0 33 0 670896660 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4356992 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=22397/tid=22405] ppid=22395 vsize=32751072 CPUtime=4970.88 cores=0-31
/proc/22397/task/22405/stat : 22405 (plingeling) R 22395 22397 22371 0 -1 4202560 5006301 1240 0 0 489761 7327 0 0 25 0 33 0 670896660 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4308533 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=22397/tid=22406] ppid=22395 vsize=32751072 CPUtime=4957.13 cores=0-31
/proc/22397/task/22406/stat : 22406 (plingeling) R 22395 22397 22371 0 -1 4202560 3292094 1240 0 0 491231 4482 0 0 25 0 33 0 670896660 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=22397/tid=22407] ppid=22395 vsize=32751072 CPUtime=5029.62 cores=0-31
/proc/22397/task/22407/stat : 22407 (plingeling) R 22395 22397 22371 0 -1 4202560 6589403 1240 0 0 490923 12039 0 0 25 0 33 0 670896660 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4355827 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=22397/tid=22408] ppid=22395 vsize=32751072 CPUtime=4947.67 cores=0-31
/proc/22397/task/22408/stat : 22408 (plingeling) R 22395 22397 22371 0 -1 4202560 4780452 1240 0 0 487633 7134 0 0 25 0 33 0 670896660 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=22397/tid=22409] ppid=22395 vsize=32751072 CPUtime=4941.15 cores=0-31
/proc/22397/task/22409/stat : 22409 (plingeling) R 22395 22397 22371 0 -1 4202560 4599957 1240 0 0 487874 6241 0 0 25 0 33 0 670896660 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4364461 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=22397/tid=22410] ppid=22395 vsize=32751072 CPUtime=4934.58 cores=0-31
/proc/22397/task/22410/stat : 22410 (plingeling) R 22395 22397 22371 0 -1 4202560 3495893 1240 0 0 488895 4563 0 0 25 0 33 0 670896661 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4250786 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=22397/tid=22411] ppid=22395 vsize=32751072 CPUtime=4960.94 cores=0-31
/proc/22397/task/22411/stat : 22411 (plingeling) R 22395 22397 22371 0 -1 4202560 5389450 1240 0 0 486875 9219 0 0 22 0 33 0 670896661 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=22397/tid=22412] ppid=22395 vsize=32751072 CPUtime=4972.57 cores=0-31
/proc/22397/task/22412/stat : 22412 (plingeling) R 22395 22397 22371 0 -1 4202560 3665131 1240 0 0 493156 4101 0 0 25 0 33 0 670896661 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=22397/tid=22413] ppid=22395 vsize=32751072 CPUtime=4900.38 cores=0-31
/proc/22397/task/22413/stat : 22413 (plingeling) R 22395 22397 22371 0 -1 4202560 3957621 1240 0 0 484665 5373 0 0 22 0 33 0 670896661 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4364829 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=22397/tid=22414] ppid=22395 vsize=32751072 CPUtime=4998.16 cores=0-31
/proc/22397/task/22414/stat : 22414 (plingeling) R 22395 22397 22371 0 -1 4202560 4212029 1240 0 0 492980 6836 0 0 25 0 33 0 670896661 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4236655 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=22397/tid=22415] ppid=22395 vsize=32751072 CPUtime=4945.53 cores=0-31
/proc/22397/task/22415/stat : 22415 (plingeling) R 22395 22397 22371 0 -1 4202560 4055797 1240 0 0 489678 4875 0 0 22 0 33 0 670896662 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4364587 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=22397/tid=22416] ppid=22395 vsize=32751072 CPUtime=4945.93 cores=0-31
/proc/22397/task/22416/stat : 22416 (plingeling) R 22395 22397 22371 0 -1 4202560 4422350 1240 0 0 488647 5946 0 0 18 0 33 0 670896662 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=22397/tid=22417] ppid=22395 vsize=32751072 CPUtime=4940.6 cores=0-31
/proc/22397/task/22417/stat : 22417 (plingeling) R 22395 22397 22371 0 -1 4202560 4030374 1240 0 0 488161 5899 0 0 25 0 33 0 670896662 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4259618 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=22397/tid=22418] ppid=22395 vsize=32751072 CPUtime=4935.77 cores=0-31
/proc/22397/task/22418/stat : 22418 (plingeling) R 22395 22397 22371 0 -1 4202560 3238424 1240 0 0 489377 4200 0 0 25 0 33 0 670896662 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4364453 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=22397/tid=22419] ppid=22395 vsize=32751072 CPUtime=4954.54 cores=0-31
/proc/22397/task/22419/stat : 22419 (plingeling) R 22395 22397 22371 0 -1 4202560 3670830 1240 0 0 490490 4964 0 0 18 0 33 0 670896662 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 258651013786 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=22397/tid=22420] ppid=22395 vsize=32751072 CPUtime=4991.51 cores=0-31
/proc/22397/task/22420/stat : 22420 (plingeling) R 22395 22397 22371 0 -1 4202560 5670166 1240 0 0 489834 9317 0 0 25 0 33 0 670896662 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=22397/tid=22421] ppid=22395 vsize=32751072 CPUtime=4922.46 cores=0-31
/proc/22397/task/22421/stat : 22421 (plingeling) R 22395 22397 22371 0 -1 4202560 3834323 1240 0 0 487017 5229 0 0 25 0 33 0 670896666 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4236732 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=22397/tid=22422] ppid=22395 vsize=32751072 CPUtime=4945.74 cores=0-31
/proc/22397/task/22422/stat : 22422 (plingeling) R 22395 22397 22371 0 -1 4202560 3834023 1240 0 0 489314 5260 0 0 25 0 33 0 670896667 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=22397/tid=22423] ppid=22395 vsize=32751072 CPUtime=4937.68 cores=0-31
/proc/22397/task/22423/stat : 22423 (plingeling) R 22395 22397 22371 0 -1 4202560 4317697 1240 0 0 487807 5961 0 0 22 0 33 0 670896667 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=22397/tid=22424] ppid=22395 vsize=32751072 CPUtime=4967.04 cores=0-31
/proc/22397/task/22424/stat : 22424 (plingeling) R 22395 22397 22371 0 -1 4202560 4104343 1240 0 0 490815 5889 0 0 25 0 33 0 670896667 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4364604 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=22397/tid=22425] ppid=22395 vsize=32751072 CPUtime=4953.05 cores=0-31
/proc/22397/task/22425/stat : 22425 (plingeling) R 22395 22397 22371 0 -1 4202560 4220281 1240 0 0 489083 6222 0 0 22 0 33 0 670896668 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4232584 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=22397/tid=22426] ppid=22395 vsize=32751072 CPUtime=4934.9 cores=0-31
/proc/22397/task/22426/stat : 22426 (plingeling) R 22395 22397 22371 0 -1 4202560 4290739 1240 0 0 487177 6313 0 0 25 0 33 0 670896668 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4365027 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=22397/tid=22427] ppid=22395 vsize=32751072 CPUtime=4961.3 cores=0-31
/proc/22397/task/22427/stat : 22427 (plingeling) R 22395 22397 22371 0 -1 4202560 4191647 1240 0 0 489906 6224 0 0 25 0 33 0 670896668 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=22397/tid=22428] ppid=22395 vsize=32751072 CPUtime=4903.68 cores=0-31
/proc/22397/task/22428/stat : 22428 (plingeling) R 22395 22397 22371 0 -1 4202560 4243577 1240 0 0 484635 5733 0 0 25 0 33 0 670896668 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4364558 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=22397/tid=22429] ppid=22395 vsize=32751072 CPUtime=4910 cores=0-31
/proc/22397/task/22429/stat : 22429 (plingeling) R 22395 22397 22371 0 -1 4202560 4538431 1240 0 0 485169 5831 0 0 25 0 33 0 670896668 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=22397/tid=22430] ppid=22395 vsize=32751072 CPUtime=4958.86 cores=0-31
/proc/22397/task/22430/stat : 22430 (plingeling) R 22395 22397 22371 0 -1 4202560 4972369 1240 0 0 487741 8145 0 0 22 0 33 0 670896668 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4237202 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=22397/tid=22431] ppid=22395 vsize=32751072 CPUtime=4915.38 cores=0-31
/proc/22397/task/22431/stat : 22431 (plingeling) R 22395 22397 22371 0 -1 4202560 4158405 1240 0 0 485873 5665 0 0 25 0 33 0 670896668 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=22397/tid=22432] ppid=22395 vsize=32751072 CPUtime=4911.11 cores=0-31
/proc/22397/task/22432/stat : 22432 (plingeling) R 22395 22397 22371 0 -1 4202560 3667820 1240 0 0 486557 4554 0 0 22 0 33 0 670896669 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=22397/tid=22433] ppid=22395 vsize=32751072 CPUtime=4911.91 cores=0-31
/proc/22397/task/22433/stat : 22433 (plingeling) R 22395 22397 22371 0 -1 4202560 4208839 1240 0 0 485686 5505 0 0 18 0 33 0 670896670 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=22397/tid=22434] ppid=22395 vsize=32751072 CPUtime=4953.04 cores=0-31
/proc/22397/task/22434/stat : 22434 (plingeling) R 22395 22397 22371 0 -1 4202560 3542431 1240 0 0 490266 5038 0 0 25 0 33 0 670896670 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=22397/tid=22435] ppid=22395 vsize=32751072 CPUtime=4929.06 cores=0-31
/proc/22397/task/22435/stat : 22435 (plingeling) R 22395 22397 22371 0 -1 4202560 3999087 1240 0 0 487464 5442 0 0 25 0 33 0 670896670 33537097728 8074683 268435456000 4194304 4417452 140736020152880 18446744073709551615 4237202 0 0 4096 17442 0 0 0 -1 21 0 0 0
Current children cumulated CPU time (s) 158319
Current children cumulated vsize (KiB) 32751072

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)

Sending SIGKILL to -22397

Solver just ended. Dumping a history of the last processes samples

[startup+5100.8 s]
/proc/loadavg: 29.53 31.47 31.82 2/416 23058
/proc/meminfo: memFree=231500340/264257624 swapFree=67111528/67111528
[pid=22397] ppid=22395 vsize=0 CPUtime=158320 cores=0-31
/proc/22397/stat : 22397 (plingeling) Z 22395 22397 22371 0 -1 4203548 136192635 1240 0 0 15636765 195223 0 0 18 0 2 0 670895786 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/22397/statm: 0 0 0 0 0 0 0
[pid=22397/tid=22419] ppid=22395 vsize=0 CPUtime=4955.35 cores=0-31
/proc/22397/task/22419/stat : 22419 (plingeling) R 22395 22397 22371 0 -1 4203588 3670830 1240 0 0 490490 5045 0 0 19 0 2 0 670896662 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 18 0 0 0
Current children cumulated CPU time (s) 158320
Current children cumulated vsize (KiB) 0

[startup+5101.6 s]
/proc/loadavg: 29.53 31.47 31.82 2/416 23058
/proc/meminfo: memFree=231500340/264257624 swapFree=67111528/67111528
[pid=22397] ppid=22395 vsize=0 CPUtime=158321 cores=0-31
/proc/22397/stat : 22397 (plingeling) Z 22395 22397 22371 0 -1 4203548 136192635 1240 0 0 15636765 195299 0 0 18 0 2 0 670895786 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/22397/statm: 0 0 0 0 0 0 0
[pid=22397/tid=22419] ppid=22395 vsize=0 CPUtime=4956.11 cores=0-31
/proc/22397/task/22419/stat : 22419 (plingeling) R 22395 22397 22371 0 -1 4203588 3670830 1240 0 0 490490 5121 0 0 20 0 2 0 670896662 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 18 0 0 0
Current children cumulated CPU time (s) 158321
Current children cumulated vsize (KiB) 0

[startup+5102.03 s]
/proc/loadavg: 29.53 31.47 31.82 2/416 23058
/proc/meminfo: memFree=247432212/264257624 swapFree=67111528/67111528
[pid=22397] ppid=22395 vsize=0 CPUtime=158321 cores=0-31
/proc/22397/stat : 22397 (plingeling) Z 22395 22397 22371 0 -1 4203548 136192635 1240 0 0 15636765 195343 0 0 18 0 2 0 670895786 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/22397/statm: 0 0 0 0 0 0 0
[pid=22397/tid=22419] ppid=22395 vsize=0 CPUtime=4956.54 cores=0-31
/proc/22397/task/22419/stat : 22419 (plingeling) R 22395 22397 22371 0 -1 4203588 3670830 1240 0 0 490490 5164 0 0 21 0 2 0 670896662 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 18 0 0 0
Current children cumulated CPU time (s) 158321
Current children cumulated vsize (KiB) 0

[startup+5102.4 s]
/proc/loadavg: 29.53 31.47 31.82 2/416 23058
/proc/meminfo: memFree=247432212/264257624 swapFree=67111528/67111528
[pid=22397] ppid=22395 vsize=0 CPUtime=158321 cores=0-31
/proc/22397/stat : 22397 (plingeling) Z 22395 22397 22371 0 -1 4203548 136192635 1240 0 0 15636765 195379 0 0 18 0 2 0 670895786 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/22397/statm: 0 0 0 0 0 0 0
[pid=22397/tid=22419] ppid=22395 vsize=0 CPUtime=4956.91 cores=0-31
/proc/22397/task/22419/stat : 22419 (plingeling) R 22395 22397 22371 0 -1 4203588 3670830 1240 0 0 490490 5201 0 0 22 0 2 0 670896662 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 18 0 0 0
Current children cumulated CPU time (s) 158321
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5102.49
CPU time (s): 158322
CPU user time (s): 156368
CPU system time (s): 1953.89
CPU usage (%): 3102.83
Max. virtual memory (cumulated for all children) (KiB): 37712836

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 156368
system time used= 1953.89
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 136193875
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 93661
involuntary context switches= 387391

runsolver used 29.4565 second user time and 117.885 second system time

The end

Launcher Data 

Begin job on node200 at 2011-05-17 03:01:34
IDJOB=3372426
IDBENCH=82372
IDSOLVER=1852
FILE ID=node200/3372426-1305594093
RUNJOBID= node200-1305594093-22385
PBS_JOBID= 13285559
Free space on /tmp= 62040 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/application/rintanen/SATPlanning/openstacks-sequencedstrips-nonadl-nonnegated-os-sequencedstrips-p30_3.035-NOTKNOWN.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3372426-1305594093/watcher-3372426-1305594093 -o /tmp/evaluation-result-3372426-1305594093/solver-3372426-1305594093 -C 160000 -W 5100 -M 266338304  plingeling -t 32 HOME/instance-3372426-1305594093.cnf

TIME LIMIT= 160000 seconds
MEMORY LIMIT= 266338304 MiB
NBCORE= 32

MD5SUM BENCH= c4319228342e10028de028a9b67c3404
RANDOM SEED=2038640190

node200.alineos.net Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 64
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 96
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 66
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 98
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 68
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 100
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 70
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.98
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 102
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 16
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 17
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 48
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 18
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 80
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 19
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 112
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 20
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 21
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 50
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 22
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 82
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 114
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.23
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 24
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 25
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 26
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 84
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 27
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 116
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 28
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 29
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 54
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 30
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 86
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 31
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 118
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     264257624 kB
MemFree:      255516636 kB
Buffers:        447452 kB
Cached:        7886512 kB
SwapCached:          0 kB
Active:        7404740 kB
Inactive:       997768 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     264257624 kB
LowFree:      255516636 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           12488 kB
Writeback:           4 kB
AnonPages:       68328 kB
Mapped:          13836 kB
Slab:           251688 kB
PageTables:       4388 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  199240340 kB
Committed_AS:   191624 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296372 kB
VmallocChunk: 34359440603 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 62040 MiB
End job on node200 at 2011-05-17 04:26:36