Trace number 3372840

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 160004 5024.28

General information on the benchmark

NameSAT03/industrial/li/li-test4/
li-test4-100.shuffled-as.sat03-370.cnf
MD5SUM5e5606ae7367b9272c60adc115594912
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables36809
Number of clauses142491
Sum of the clauses size559708
Maximum clause length172
Minimum clause length1
Number of clauses of size 114
Number of clauses of size 224558
Number of clauses of size 367458
Number of clauses of size 449000
Number of clauses of size 51
Number of clauses of size over 51460

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3372840-1306075477/watcher-3372840-1306075477 -o /tmp/evaluation-result-3372840-1306075477/solver-3372840-1306075477 -C 160000 -W 5100 -M 266338304 plingeling -t 32 HOME/instance-3372840-1306075477.cnf 

running on 32 cores: 0,4,8,12,16,20,24,28,1,5,9,13,17,21,25,29,2,6,10,14,18,22,26,30,3,7,11,15,19,23,27,31

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 160000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 160030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 272730423296 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 272730474496 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 29.52 31.56 31.92 1/413 26447
/proc/meminfo: memFree=255496912/264257624 swapFree=67111528/67111528
[pid=26447] ppid=26445 vsize=30680 CPUtime=0.03 cores=0-31
/proc/26447/stat : 26447 (plingeling) R 26445 26447 26421 0 -1 4202496 5631 1250 0 0 1 1 0 1 18 0 1 0 719034170 31416320 5562 268435456000 4194304 4417452 140736041086384 18446744073709551615 258650649933 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/26447/statm: 7670 5573 145 55 0 5556 0

[startup+0.0503921 s]
/proc/loadavg: 29.52 31.56 31.92 1/413 26447
/proc/meminfo: memFree=255496912/264257624 swapFree=67111528/67111528
[pid=26447] ppid=26445 vsize=42456 CPUtime=0.04 cores=0-31
/proc/26447/stat : 26447 (plingeling) R 26445 26447 26421 0 -1 4202496 8678 1250 0 0 1 2 0 1 18 0 1 0 719034170 43474944 8609 268435456000 4194304 4417452 140736041086384 18446744073709551615 258650649933 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/26447/statm: 10614 8613 146 55 0 8500 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 42456

[startup+0.100384 s]
/proc/loadavg: 29.52 31.56 31.92 1/413 26447
/proc/meminfo: memFree=255496912/264257624 swapFree=67111528/67111528
[pid=26447] ppid=26445 vsize=49592 CPUtime=0.09 cores=0-31
/proc/26447/stat : 26447 (plingeling) R 26445 26447 26421 0 -1 4202496 10457 1250 0 0 6 2 0 1 18 0 1 0 719034170 50782208 10388 268435456000 4194304 4417452 140736041086384 18446744073709551615 258650618162 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/26447/statm: 12398 10403 146 55 0 10284 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 49592

[startup+0.300409 s]
/proc/loadavg: 29.52 31.56 31.92 1/413 26447
/proc/meminfo: memFree=255496912/264257624 swapFree=67111528/67111528
[pid=26447] ppid=26445 vsize=75180 CPUtime=0.3 cores=0-31
/proc/26447/stat : 26447 (plingeling) R 26445 26447 26421 0 -1 4202496 16870 1250 0 0 25 4 0 1 19 0 1 0 719034170 76984320 16801 268435456000 4194304 4417452 140736041086384 18446744073709551615 4265009 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/26447/statm: 18795 16801 146 55 0 16681 0
Current children cumulated CPU time (s) 0.3
Current children cumulated vsize (KiB) 75180

[startup+0.700397 s]
/proc/loadavg: 29.52 31.56 31.92 1/413 26447
/proc/meminfo: memFree=255496912/264257624 swapFree=67111528/67111528
[pid=26447] ppid=26445 vsize=107760 CPUtime=0.7 cores=0-31
/proc/26447/stat : 26447 (plingeling) R 26445 26447 26421 0 -1 4202496 25031 1250 0 0 63 6 0 1 19 0 1 0 719034170 110346240 24962 268435456000 4194304 4417452 140736041086384 18446744073709551615 4231321 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/26447/statm: 26940 24962 146 55 0 24826 0
Current children cumulated CPU time (s) 0.7
Current children cumulated vsize (KiB) 107760

[startup+1.50045 s]
/proc/loadavg: 29.52 31.56 31.92 3/414 26453
/proc/meminfo: memFree=255390000/264257624 swapFree=67111528/67111528
[pid=26447] ppid=26445 vsize=126852 CPUtime=1.49 cores=0-31
/proc/26447/stat : 26447 (plingeling) R 26445 26447 26421 0 -1 4202496 29804 1250 0 0 141 7 0 1 21 0 1 0 719034170 129896448 29735 268435456000 4194304 4417452 140736041086384 18446744073709551615 4323168 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/26447/statm: 31713 29735 146 55 0 29599 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 126852

[startup+3.10051 s]
/proc/loadavg: 29.52 31.56 31.92 2/414 26453
/proc/meminfo: memFree=255292752/264257624 swapFree=67111528/67111528
[pid=26447] ppid=26445 vsize=244988 CPUtime=3.09 cores=0-31
/proc/26447/stat : 26447 (plingeling) R 26445 26447 26421 0 -1 4202496 59338 1250 0 0 295 13 0 1 25 0 1 0 719034170 250867712 59269 268435456000 4194304 4417452 140736041086384 18446744073709551615 4322574 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/26447/statm: 61247 59269 146 55 0 59133 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 244988

[startup+6.30062 s]
/proc/loadavg: 27.24 31.05 31.75 2/414 26453
/proc/meminfo: memFree=255177220/264257624 swapFree=67111528/67111528
[pid=26447] ppid=26445 vsize=361224 CPUtime=6.29 cores=0-31
/proc/26447/stat : 26447 (plingeling) R 26445 26447 26421 0 -1 4202496 88397 1250 0 0 609 19 0 1 25 0 1 0 719034170 369893376 88328 268435456000 4194304 4417452 140736041086384 18446744073709551615 4259222 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/26447/statm: 90306 88328 146 55 0 88192 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 361224

[startup+12.7017 s]
/proc/loadavg: 27.62 31.07 31.75 33/446 26487
/proc/meminfo: memFree=254963896/264257624 swapFree=67111528/67111528
[pid=26447] ppid=26445 vsize=944620 CPUtime=155.55 cores=0-31
/proc/26447/stat : 26447 (plingeling) S 26445 26447 26421 0 -1 4202496 178154 1250 0 0 15508 46 0 1 18 0 33 0 719034170 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 30 0 0 0
/proc/26447/statm: 236155 129615 173 55 0 234041 0
[pid=26447/tid=26454] ppid=26445 vsize=944620 CPUtime=4.63 cores=0-31
/proc/26447/task/26454/stat : 26454 (plingeling) R 26445 26447 26421 0 -1 4202560 1963 1250 0 0 462 0 0 1 25 0 33 0 719034977 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4277864 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=26447/tid=26455] ppid=26445 vsize=944620 CPUtime=4.62 cores=0-31
/proc/26447/task/26455/stat : 26455 (plingeling) R 26445 26447 26421 0 -1 4202560 2208 1250 0 0 461 0 0 1 25 0 33 0 719034977 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4277864 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=26447/tid=26456] ppid=26445 vsize=944620 CPUtime=4.63 cores=0-31
/proc/26447/task/26456/stat : 26456 (plingeling) R 26445 26447 26421 0 -1 4202560 936 1250 0 0 462 0 0 1 25 0 33 0 719034977 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4234242 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26447/tid=26457] ppid=26445 vsize=944620 CPUtime=4.63 cores=0-31
/proc/26447/task/26457/stat : 26457 (plingeling) R 26445 26447 26421 0 -1 4202560 436 1250 0 0 462 0 0 1 25 0 33 0 719034977 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26447/tid=26458] ppid=26445 vsize=944620 CPUtime=4.63 cores=0-31
/proc/26447/task/26458/stat : 26458 (plingeling) R 26445 26447 26421 0 -1 4202560 774 1250 0 0 462 0 0 1 25 0 33 0 719034977 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4260714 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=26447/tid=26459] ppid=26445 vsize=944620 CPUtime=4.63 cores=0-31
/proc/26447/task/26459/stat : 26459 (plingeling) R 26445 26447 26421 0 -1 4202560 802 1250 0 0 462 0 0 1 25 0 33 0 719034978 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4234794 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=26447/tid=26460] ppid=26445 vsize=944620 CPUtime=4.62 cores=0-31
/proc/26447/task/26460/stat : 26460 (plingeling) R 26445 26447 26421 0 -1 4202560 788 1250 0 0 461 0 0 1 25 0 33 0 719034978 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4277545 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26447/tid=26461] ppid=26445 vsize=944620 CPUtime=4.62 cores=0-31
/proc/26447/task/26461/stat : 26461 (plingeling) R 26445 26447 26421 0 -1 4202560 2779 1250 0 0 461 0 0 1 25 0 33 0 719034978 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4277545 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26447/tid=26462] ppid=26445 vsize=944620 CPUtime=4.62 cores=0-31
/proc/26447/task/26462/stat : 26462 (plingeling) R 26445 26447 26421 0 -1 4202560 1776 1250 0 0 461 0 0 1 25 0 33 0 719034978 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4277545 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26447/tid=26463] ppid=26445 vsize=944620 CPUtime=4.62 cores=0-31
/proc/26447/task/26463/stat : 26463 (plingeling) R 26445 26447 26421 0 -1 4202560 2706 1250 0 0 460 1 0 1 25 0 33 0 719034978 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4275856 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=26447/tid=26464] ppid=26445 vsize=944620 CPUtime=4.62 cores=0-31
/proc/26447/task/26464/stat : 26464 (plingeling) R 26445 26447 26421 0 -1 4202560 1716 1250 0 0 461 0 0 1 25 0 33 0 719034978 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4326634 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=26447/tid=26465] ppid=26445 vsize=944620 CPUtime=4.62 cores=0-31
/proc/26447/task/26465/stat : 26465 (plingeling) R 26445 26447 26421 0 -1 4202560 1008 1250 0 0 461 0 0 1 25 0 33 0 719034978 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4270966 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=26447/tid=26466] ppid=26445 vsize=944620 CPUtime=4.62 cores=0-31
/proc/26447/task/26466/stat : 26466 (plingeling) R 26445 26447 26421 0 -1 4202560 1777 1250 0 0 461 0 0 1 25 0 33 0 719034978 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4301955 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=26447/tid=26467] ppid=26445 vsize=944620 CPUtime=4.62 cores=0-31
/proc/26447/task/26467/stat : 26467 (plingeling) R 26445 26447 26421 0 -1 4202560 2202 1250 0 0 460 1 0 1 25 0 33 0 719034978 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4277545 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=26447/tid=26468] ppid=26445 vsize=944620 CPUtime=4.48 cores=0-31
/proc/26447/task/26468/stat : 26468 (plingeling) R 26445 26447 26421 0 -1 4202560 1966 1250 0 0 447 0 0 1 25 0 33 0 719034978 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4265411 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=26447/tid=26469] ppid=26445 vsize=944620 CPUtime=4.61 cores=0-31
/proc/26447/task/26469/stat : 26469 (plingeling) R 26445 26447 26421 0 -1 4202560 2444 1250 0 0 460 0 0 1 25 0 33 0 719034978 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4260714 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26447/tid=26470] ppid=26445 vsize=944620 CPUtime=4.61 cores=0-31
/proc/26447/task/26470/stat : 26470 (plingeling) R 26445 26447 26421 0 -1 4202560 1941 1250 0 0 460 0 0 1 25 0 33 0 719034978 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4270927 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=26447/tid=26471] ppid=26445 vsize=944620 CPUtime=4.61 cores=0-31
/proc/26447/task/26471/stat : 26471 (plingeling) R 26445 26447 26421 0 -1 4202560 1703 1250 0 0 460 0 0 1 25 0 33 0 719034978 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4278111 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=26447/tid=26472] ppid=26445 vsize=944620 CPUtime=4.61 cores=0-31
/proc/26447/task/26472/stat : 26472 (plingeling) R 26445 26447 26421 0 -1 4202560 937 1250 0 0 460 0 0 1 25 0 33 0 719034978 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4260519 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=26447/tid=26473] ppid=26445 vsize=944620 CPUtime=4.61 cores=0-31
/proc/26447/task/26473/stat : 26473 (plingeling) R 26445 26447 26421 0 -1 4202560 1777 1250 0 0 460 0 0 1 25 0 33 0 719034978 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4277594 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=26447/tid=26474] ppid=26445 vsize=944620 CPUtime=4.62 cores=0-31
/proc/26447/task/26474/stat : 26474 (plingeling) R 26445 26447 26421 0 -1 4202560 2887 1250 0 0 461 0 0 1 25 0 33 0 719034978 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4236649 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=26447/tid=26475] ppid=26445 vsize=944620 CPUtime=4.61 cores=0-31
/proc/26447/task/26475/stat : 26475 (plingeling) R 26445 26447 26421 0 -1 4202560 1991 1250 0 0 460 0 0 1 25 0 33 0 719034978 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4277864 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=26447/tid=26476] ppid=26445 vsize=944620 CPUtime=4.62 cores=0-31
/proc/26447/task/26476/stat : 26476 (plingeling) R 26445 26447 26421 0 -1 4202560 2157 1250 0 0 460 1 0 1 25 0 33 0 719034979 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4264403 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=26447/tid=26477] ppid=26445 vsize=944620 CPUtime=4.61 cores=0-31
/proc/26447/task/26477/stat : 26477 (plingeling) R 26445 26447 26421 0 -1 4202560 1740 1250 0 0 460 0 0 1 25 0 33 0 719034979 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4277545 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=26447/tid=26478] ppid=26445 vsize=944620 CPUtime=4.61 cores=0-31
/proc/26447/task/26478/stat : 26478 (plingeling) R 26445 26447 26421 0 -1 4202560 2004 1250 0 0 460 0 0 1 25 0 33 0 719034979 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4234734 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=26447/tid=26479] ppid=26445 vsize=944620 CPUtime=4.61 cores=0-31
/proc/26447/task/26479/stat : 26479 (plingeling) R 26445 26447 26421 0 -1 4202560 2457 1250 0 0 459 1 0 1 25 0 33 0 719034979 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=26447/tid=26480] ppid=26445 vsize=944620 CPUtime=4.6 cores=0-31
/proc/26447/task/26480/stat : 26480 (plingeling) R 26445 26447 26421 0 -1 4202560 648 1250 0 0 459 0 0 1 25 0 33 0 719034979 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4265408 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=26447/tid=26481] ppid=26445 vsize=944620 CPUtime=4.61 cores=0-31
/proc/26447/task/26481/stat : 26481 (plingeling) R 26445 26447 26421 0 -1 4202560 1509 1250 0 0 460 0 0 1 25 0 33 0 719034979 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4277563 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=26447/tid=26482] ppid=26445 vsize=944620 CPUtime=4.6 cores=0-31
/proc/26447/task/26482/stat : 26482 (plingeling) R 26445 26447 26421 0 -1 4202560 2563 1250 0 0 459 0 0 1 25 0 33 0 719034980 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4277864 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=26447/tid=26483] ppid=26445 vsize=944620 CPUtime=4.6 cores=0-31
/proc/26447/task/26483/stat : 26483 (plingeling) R 26445 26447 26421 0 -1 4202560 2499 1250 0 0 459 0 0 1 25 0 33 0 719034980 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=26447/tid=26484] ppid=26445 vsize=944620 CPUtime=4.6 cores=0-31
/proc/26447/task/26484/stat : 26484 (plingeling) R 26445 26447 26421 0 -1 4202560 1509 1250 0 0 459 0 0 1 25 0 33 0 719034980 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=26447/tid=26485] ppid=26445 vsize=944620 CPUtime=4.55 cores=0-31
/proc/26447/task/26485/stat : 26485 (plingeling) R 26445 26447 26421 0 -1 4202560 2346 1250 0 0 454 0 0 1 25 0 33 0 719034980 967290880 129615 268435456000 4194304 4417452 140736041086384 18446744073709551615 4277545 0 0 4096 17442 0 0 0 -1 29 0 0 0
Current children cumulated CPU time (s) 155.55
Current children cumulated vsize (KiB) 944620

[startup+25.5022 s]
/proc/loadavg: 28.59 31.11 31.76 33/446 26487
/proc/meminfo: memFree=254991996/264257624 swapFree=67111528/67111528
[pid=26447] ppid=26445 vsize=914452 CPUtime=564.26 cores=0-31
/proc/26447/stat : 26447 (plingeling) S 26445 26447 26421 0 -1 4202496 190847 1250 0 0 56358 67 0 1 18 0 33 0 719034170 936398848 123092 268435456000 4194304 4417452 140736041086384 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 30 0 0 0
/proc/26447/statm: 228613 123092 175 55 0 226499 0
[pid=26447/tid=26454] ppid=26445 vsize=914452 CPUtime=17.37 cores=0-31
/proc/26447/task/26454/stat : 26454 (plingeling) R 26445 26447 26421 0 -1 4202560 2312 1250 0 0 1735 1 0 1 25 0 33 0 719034977 936398848 123092 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=26447/tid=26455] ppid=26445 vsize=914452 CPUtime=17.41 cores=0-31
/proc/26447/task/26455/stat : 26455 (plingeling) R 26445 26447 26421 0 -1 4202560 2749 1250 0 0 1740 0 0 1 25 0 33 0 719034977 936398848 123092 268435456000 4194304 4417452 140736041086384 18446744073709551615 4365198 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=26447/tid=26456] ppid=26445 vsize=914452 CPUtime=17.43 cores=0-31
/proc/26447/task/26456/stat : 26456 (plingeling) R 26445 26447 26421 0 -1 4202560 936 1250 0 0 1742 0 0 1 25 0 33 0 719034977 936398848 123092 268435456000 4194304 4417452 140736041086384 18446744073709551615 4365579 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26447/tid=26457] ppid=26445 vsize=914452 CPUtime=17.43 cores=0-31
/proc/26447/task/26457/stat : 26457 (plingeling) R 26445 26447 26421 0 -1 4202560 436 1250 0 0 1742 0 0 1 25 0 33 0 719034977 936398848 123092 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26447/tid=26458] ppid=26445 vsize=914452 CPUtime=17.42 cores=0-31
/proc/26447/task/26458/stat : 26458 (plingeling) R 26445 26447 26421 0 -1 4202560 1123 1250 0 0 1740 1 0 1 25 0 33 0 719034977 936398848 123092 268435456000 4194304 4417452 140736041086384 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=26447/tid=26459] ppid=26445 vsize=914452 CPUtime=17.42 cores=0-31
/proc/26447/task/26459/stat : 26459 (plingeling) R 26445 26447 26421 0 -1 4202560 1151 1250 0 0 1741 0 0 1 25 0 33 0 719034978 936398848 123092 268435456000 4194304 4417452 140736041086384 18446744073709551615 4232640 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=26447/tid=26460] ppid=26445 vsize=914452 CPUtime=17.42 cores=0-31
/proc/26447/task/26460/stat : 26460 (plingeling) R 26445 26447 26421 0 -1 4202560 788 1250 0 0 1741 0 0 1 25 0 33 0 719034978 936398848 123092 268435456000 4194304 4417452 140736041086384 18446744073709551615 4351526 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26447/tid=26461] ppid=26445 vsize=914452 CPUtime=17.36 cores=0-31
/proc/26447/task/26461/stat : 26461 (plingeling) R 26445 26447 26421 0 -1 4202560 3256 1250 0 0 1734 1 0 1 25 0 33 0 719034978 936398848 123092 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26447/tid=26462] ppid=26445 vsize=914452 CPUtime=17.41 cores=0-31
/proc/26447/task/26462/stat : 26462 (plingeling) R 26445 26447 26421 0 -1 4202560 2125 1250 0 0 1740 0 0 1 25 0 33 0 719034978 936398848 123092 268435456000 4194304 4417452 140736041086384 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26447/tid=26463] ppid=26445 vsize=914452 CPUtime=17.4 cores=0-31
/proc/26447/task/26463/stat : 26463 (plingeling) R 26445 26447 26421 0 -1 4202560 3517 1250 0 0 1737 2 0 1 25 0 33 0 719034978 936398848 123092 268435456000 4194304 4417452 140736041086384 18446744073709551615 4232693 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=26447/tid=26464] ppid=26445 vsize=914452 CPUtime=17.42 cores=0-31
/proc/26447/task/26464/stat : 26464 (plingeling) R 26445 26447 26421 0 -1 4202560 1716 1250 0 0 1741 0 0 1 25 0 33 0 719034978 936398848 123092 268435456000 4194304 4417452 140736041086384 18446744073709551615 4232747 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=26447/tid=26465] ppid=26445 vsize=914452 CPUtime=17.41 cores=0-31
/proc/26447/task/26465/stat : 26465 (plingeling) R 26445 26447 26421 0 -1 4202560 2194 1250 0 0 1739 1 0 1 25 0 33 0 719034978 936398848 123092 268435456000 4194304 4417452 140736041086384 18446744073709551615 4232795 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=26447/tid=26466] ppid=26445 vsize=914452 CPUtime=17.42 cores=0-31
/proc/26447/task/26466/stat : 26466 (plingeling) R 26445 26447 26421 0 -1 4202560 2716 1250 0 0 1739 2 0 1 25 0 33 0 719034978 936398848 123092 268435456000 4194304 4417452 140736041086384 18446744073709551615 4232801 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=26447/tid=26467] ppid=26445 vsize=914452 CPUtime=17.42 cores=0-31
/proc/26447/task/26467/stat : 26467 (plingeling) R 26445 26447 26421 0 -1 4202560 2807 1250 0 0 1740 1 0 1 25 0 33 0 719034978 936398848 123092 268435456000 4194304 4417452 140736041086384 18446744073709551615 4232791 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=26447/tid=26468] ppid=26445 vsize=914452 CPUtime=17.21 cores=0-31
/proc/26447/task/26468/stat : 26468 (plingeling) R 26445 26447 26421 0 -1 4202560 1966 1250 0 0 1720 0 0 1 25 0 33 0 719034978 936398848 123092 268435456000 4194304 4417452 140736041086384 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 19 0 0 0

################
# More data... #
################

[pid=26447/tid=26467] ppid=26445 vsize=33494328 CPUtime=4944.43 cores=0-31
/proc/26447/task/26467/stat : 26467 (plingeling) R 26445 26447 26421 0 -1 4202560 1174524 1250 0 0 492767 1675 0 1 25 0 33 0 719034978 34298191872 8258545 268435456000 4194304 4417452 140736041086384 18446744073709551615 4353512 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=26447/tid=26468] ppid=26445 vsize=33494328 CPUtime=4944.42 cores=0-31
/proc/26447/task/26468/stat : 26468 (plingeling) R 26445 26447 26421 0 -1 4202560 1439534 1250 0 0 492469 1972 0 1 25 0 33 0 719034978 34298191872 8258545 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=26447/tid=26469] ppid=26445 vsize=33494328 CPUtime=4923.12 cores=0-31
/proc/26447/task/26469/stat : 26469 (plingeling) R 26445 26447 26421 0 -1 4202560 1146969 1250 0 0 490898 1413 0 1 25 0 33 0 719034978 34298191872 8258545 268435456000 4194304 4417452 140736041086384 18446744073709551615 4251076 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26447/tid=26470] ppid=26445 vsize=33494328 CPUtime=4943.74 cores=0-31
/proc/26447/task/26470/stat : 26470 (plingeling) R 26445 26447 26421 0 -1 4202560 1235615 1250 0 0 492722 1651 0 1 25 0 33 0 719034978 34298191872 8258545 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=26447/tid=26471] ppid=26445 vsize=33494328 CPUtime=4944.64 cores=0-31
/proc/26447/task/26471/stat : 26471 (plingeling) R 26445 26447 26421 0 -1 4202560 1803086 1250 0 0 491560 2903 0 1 25 0 33 0 719034978 34298191872 8258545 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=26447/tid=26472] ppid=26445 vsize=33494328 CPUtime=4942.29 cores=0-31
/proc/26447/task/26472/stat : 26472 (plingeling) R 26445 26447 26421 0 -1 4202560 980562 1250 0 0 493160 1068 0 1 25 0 33 0 719034978 34298191872 8258545 268435456000 4194304 4417452 140736041086384 18446744073709551615 4237349 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=26447/tid=26473] ppid=26445 vsize=33494328 CPUtime=4943.61 cores=0-31
/proc/26447/task/26473/stat : 26473 (plingeling) R 26445 26447 26421 0 -1 4202560 1336000 1250 0 0 492607 1753 0 1 25 0 33 0 719034978 34298191872 8258545 268435456000 4194304 4417452 140736041086384 18446744073709551615 4265418 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=26447/tid=26474] ppid=26445 vsize=33494328 CPUtime=4926.31 cores=0-31
/proc/26447/task/26474/stat : 26474 (plingeling) R 26445 26447 26421 0 -1 4202560 1249492 1250 0 0 491030 1600 0 1 25 0 33 0 719034978 34298191872 8258545 268435456000 4194304 4417452 140736041086384 18446744073709551615 4353653 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=26447/tid=26475] ppid=26445 vsize=33494328 CPUtime=4946.15 cores=0-31
/proc/26447/task/26475/stat : 26475 (plingeling) R 26445 26447 26421 0 -1 4202560 1315504 1250 0 0 492757 1857 0 1 25 0 33 0 719034978 34298191872 8258545 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=26447/tid=26476] ppid=26445 vsize=33494328 CPUtime=4935.69 cores=0-31
/proc/26447/task/26476/stat : 26476 (plingeling) R 26445 26447 26421 0 -1 4202560 1340862 1250 0 0 491550 2018 0 1 25 0 33 0 719034979 34298191872 8258545 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26447/tid=26477] ppid=26445 vsize=33494328 CPUtime=4940.42 cores=0-31
/proc/26447/task/26477/stat : 26477 (plingeling) R 26445 26447 26421 0 -1 4202560 1419450 1250 0 0 491762 2279 0 1 25 0 33 0 719034979 34298191872 8258545 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=26447/tid=26478] ppid=26445 vsize=33494328 CPUtime=4944.72 cores=0-31
/proc/26447/task/26478/stat : 26478 (plingeling) R 26445 26447 26421 0 -1 4202560 892039 1250 0 0 493484 987 0 1 25 0 33 0 719034979 34298191872 8258545 268435456000 4194304 4417452 140736041086384 18446744073709551615 4237290 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=26447/tid=26479] ppid=26445 vsize=33494328 CPUtime=4933.7 cores=0-31
/proc/26447/task/26479/stat : 26479 (plingeling) R 26445 26447 26421 0 -1 4202560 1207337 1250 0 0 491821 1548 0 1 25 0 33 0 719034979 34298191872 8258545 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=26447/tid=26480] ppid=26445 vsize=33494328 CPUtime=4931.54 cores=0-31
/proc/26447/task/26480/stat : 26480 (plingeling) R 26445 26447 26421 0 -1 4202560 1112600 1250 0 0 491410 1743 0 1 25 0 33 0 719034979 34298191872 8258545 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364650 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=26447/tid=26481] ppid=26445 vsize=33494328 CPUtime=4937.63 cores=0-31
/proc/26447/task/26481/stat : 26481 (plingeling) R 26445 26447 26421 0 -1 4202560 1071329 1250 0 0 492115 1647 0 1 25 0 33 0 719034979 34298191872 8258545 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26447/tid=26482] ppid=26445 vsize=33494328 CPUtime=4929.24 cores=0-31
/proc/26447/task/26482/stat : 26482 (plingeling) R 26445 26447 26421 0 -1 4202560 1329756 1250 0 0 491205 1718 0 1 25 0 33 0 719034980 34298191872 8258545 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=26447/tid=26483] ppid=26445 vsize=33494328 CPUtime=4943.95 cores=0-31
/proc/26447/task/26483/stat : 26483 (plingeling) R 26445 26447 26421 0 -1 4202560 1730236 1250 0 0 491310 3084 0 1 25 0 33 0 719034980 34298191872 8258545 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26447/tid=26484] ppid=26445 vsize=33494328 CPUtime=4932.77 cores=0-31
/proc/26447/task/26484/stat : 26484 (plingeling) R 26445 26447 26421 0 -1 4202560 1156351 1250 0 0 491848 1428 0 1 25 0 33 0 719034980 34298191872 8258545 268435456000 4194304 4417452 140736041086384 18446744073709551615 4236784 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=26447/tid=26485] ppid=26445 vsize=33494328 CPUtime=4945.64 cores=0-31
/proc/26447/task/26485/stat : 26485 (plingeling) R 26445 26447 26421 0 -1 4202560 918340 1250 0 0 493591 972 0 1 25 0 33 0 719034980 34298191872 8258545 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 22 0 0 0
Current children cumulated CPU time (s) 158092
Current children cumulated vsize (KiB) 33494328



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5022.2 s]
/proc/loadavg: 32.04 32.04 32.06 34/446 26640
/proc/meminfo: memFree=220651172/264257624 swapFree=67111528/67111528
[pid=26447] ppid=26445 vsize=35242408 CPUtime=160002 cores=0-31
/proc/26447/stat : 26447 (plingeling) S 26445 26447 26421 0 -1 4202496 40588347 1250 0 0 15944036 56166 0 1 18 0 33 0 719034170 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 30 0 0 0
/proc/26447/statm: 8810602 8695565 175 55 0 8808488 0
[pid=26447/tid=26454] ppid=26445 vsize=35242408 CPUtime=5003.15 cores=0-31
/proc/26447/task/26454/stat : 26454 (plingeling) R 26445 26447 26421 0 -1 4202560 1360568 1250 0 0 498195 2119 0 1 18 0 33 0 719034977 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=26447/tid=26455] ppid=26445 vsize=35242408 CPUtime=5002.8 cores=0-31
/proc/26447/task/26455/stat : 26455 (plingeling) R 26445 26447 26421 0 -1 4202560 1302733 1250 0 0 498791 1488 0 1 25 0 33 0 719034977 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4237290 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=26447/tid=26456] ppid=26445 vsize=35242408 CPUtime=4998.41 cores=0-31
/proc/26447/task/26456/stat : 26456 (plingeling) R 26445 26447 26421 0 -1 4202560 1239785 1250 0 0 498235 1605 0 1 25 0 33 0 719034977 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4309899 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=26447/tid=26457] ppid=26445 vsize=35242408 CPUtime=5006.1 cores=0-31
/proc/26447/task/26457/stat : 26457 (plingeling) R 26445 26447 26421 0 -1 4202560 1902405 1250 0 0 496643 3966 0 1 25 0 33 0 719034977 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=26447/tid=26458] ppid=26445 vsize=35242408 CPUtime=5000.47 cores=0-31
/proc/26447/task/26458/stat : 26458 (plingeling) R 26445 26447 26421 0 -1 4202560 1383072 1250 0 0 498345 1701 0 1 25 0 33 0 719034977 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=26447/tid=26459] ppid=26445 vsize=35242408 CPUtime=5002.18 cores=0-31
/proc/26447/task/26459/stat : 26459 (plingeling) R 26445 26447 26421 0 -1 4202560 1101507 1250 0 0 498683 1534 0 1 25 0 33 0 719034978 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4366065 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26447/tid=26460] ppid=26445 vsize=35242408 CPUtime=5004.81 cores=0-31
/proc/26447/task/26460/stat : 26460 (plingeling) R 26445 26447 26421 0 -1 4202560 1010449 1250 0 0 499296 1184 0 1 25 0 33 0 719034978 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4236845 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=26447/tid=26461] ppid=26445 vsize=35242408 CPUtime=5001.11 cores=0-31
/proc/26447/task/26461/stat : 26461 (plingeling) R 26445 26447 26421 0 -1 4202560 1175013 1250 0 0 498956 1154 0 1 25 0 33 0 719034978 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=26447/tid=26462] ppid=26445 vsize=35242408 CPUtime=4997.81 cores=0-31
/proc/26447/task/26462/stat : 26462 (plingeling) R 26445 26447 26421 0 -1 4202560 1229504 1250 0 0 498202 1578 0 1 25 0 33 0 719034978 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=26447/tid=26463] ppid=26445 vsize=35242408 CPUtime=4995.66 cores=0-31
/proc/26447/task/26463/stat : 26463 (plingeling) R 26445 26447 26421 0 -1 4202560 1325460 1250 0 0 497787 1778 0 1 25 0 33 0 719034978 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=26447/tid=26464] ppid=26445 vsize=35242408 CPUtime=5009.72 cores=0-31
/proc/26447/task/26464/stat : 26464 (plingeling) R 26445 26447 26421 0 -1 4202560 1026837 1250 0 0 499898 1073 0 1 25 0 33 0 719034978 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=26447/tid=26465] ppid=26445 vsize=35242408 CPUtime=5002.31 cores=0-31
/proc/26447/task/26465/stat : 26465 (plingeling) R 26445 26447 26421 0 -1 4202560 1059541 1250 0 0 498886 1344 0 1 25 0 33 0 719034978 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26447/tid=26466] ppid=26445 vsize=35242408 CPUtime=5001.24 cores=0-31
/proc/26447/task/26466/stat : 26466 (plingeling) R 26445 26447 26421 0 -1 4202560 1008906 1250 0 0 498828 1295 0 1 25 0 33 0 719034978 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=26447/tid=26467] ppid=26445 vsize=35242408 CPUtime=5003.65 cores=0-31
/proc/26447/task/26467/stat : 26467 (plingeling) R 26445 26447 26421 0 -1 4202560 1181057 1250 0 0 498678 1686 0 1 25 0 33 0 719034978 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=26447/tid=26468] ppid=26445 vsize=35242408 CPUtime=5004.06 cores=0-31
/proc/26447/task/26468/stat : 26468 (plingeling) R 26445 26447 26421 0 -1 4202560 1439534 1250 0 0 498433 1972 0 1 25 0 33 0 719034978 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4305912 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=26447/tid=26469] ppid=26445 vsize=35242408 CPUtime=4983.03 cores=0-31
/proc/26447/task/26469/stat : 26469 (plingeling) R 26445 26447 26421 0 -1 4202560 1146969 1250 0 0 496889 1413 0 1 25 0 33 0 719034978 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26447/tid=26470] ppid=26445 vsize=35242408 CPUtime=5003.65 cores=0-31
/proc/26447/task/26470/stat : 26470 (plingeling) R 26445 26447 26421 0 -1 4202560 1235615 1250 0 0 498713 1651 0 1 25 0 33 0 719034978 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4357263 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=26447/tid=26471] ppid=26445 vsize=35242408 CPUtime=5004.55 cores=0-31
/proc/26447/task/26471/stat : 26471 (plingeling) R 26445 26447 26421 0 -1 4202560 1822330 1250 0 0 497499 2955 0 1 25 0 33 0 719034978 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=26447/tid=26472] ppid=26445 vsize=35242408 CPUtime=5002.11 cores=0-31
/proc/26447/task/26472/stat : 26472 (plingeling) R 26445 26447 26421 0 -1 4202560 1094136 1250 0 0 498964 1246 0 1 25 0 33 0 719034978 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=26447/tid=26473] ppid=26445 vsize=35242408 CPUtime=5003.52 cores=0-31
/proc/26447/task/26473/stat : 26473 (plingeling) R 26445 26447 26421 0 -1 4202560 1336000 1250 0 0 498598 1753 0 1 25 0 33 0 719034978 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=26447/tid=26474] ppid=26445 vsize=35242408 CPUtime=4986.22 cores=0-31
/proc/26447/task/26474/stat : 26474 (plingeling) R 26445 26447 26421 0 -1 4202560 1249492 1250 0 0 497021 1600 0 1 25 0 33 0 719034978 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4308211 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=26447/tid=26475] ppid=26445 vsize=35242408 CPUtime=5005.81 cores=0-31
/proc/26447/task/26475/stat : 26475 (plingeling) R 26445 26447 26421 0 -1 4202560 1336475 1250 0 0 498694 1886 0 1 25 0 33 0 719034978 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4263109 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=26447/tid=26476] ppid=26445 vsize=35242408 CPUtime=4995.6 cores=0-31
/proc/26447/task/26476/stat : 26476 (plingeling) R 26445 26447 26421 0 -1 4202560 1360263 1250 0 0 497489 2070 0 1 25 0 33 0 719034979 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26447/tid=26477] ppid=26445 vsize=35242408 CPUtime=5000.33 cores=0-31
/proc/26447/task/26477/stat : 26477 (plingeling) R 26445 26447 26421 0 -1 4202560 1419450 1250 0 0 497753 2279 0 1 25 0 33 0 719034979 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=26447/tid=26478] ppid=26445 vsize=35242408 CPUtime=5004.16 cores=0-31
/proc/26447/task/26478/stat : 26478 (plingeling) R 26445 26447 26421 0 -1 4202560 907215 1250 0 0 499385 1030 0 1 25 0 33 0 719034979 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=26447/tid=26479] ppid=26445 vsize=35242408 CPUtime=4992.52 cores=0-31
/proc/26447/task/26479/stat : 26479 (plingeling) R 26445 26447 26421 0 -1 4202560 1207337 1250 0 0 497703 1548 0 1 25 0 33 0 719034979 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=26447/tid=26480] ppid=26445 vsize=35242408 CPUtime=4991.36 cores=0-31
/proc/26447/task/26480/stat : 26480 (plingeling) R 26445 26447 26421 0 -1 4202560 1112600 1250 0 0 497392 1743 0 1 25 0 33 0 719034979 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=26447/tid=26481] ppid=26445 vsize=35242408 CPUtime=4997.54 cores=0-31
/proc/26447/task/26481/stat : 26481 (plingeling) R 26445 26447 26421 0 -1 4202560 1071329 1250 0 0 498106 1647 0 1 25 0 33 0 719034979 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364508 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26447/tid=26482] ppid=26445 vsize=35242408 CPUtime=4989.1 cores=0-31
/proc/26447/task/26482/stat : 26482 (plingeling) R 26445 26447 26421 0 -1 4202560 1459273 1250 0 0 496890 2019 0 1 25 0 33 0 719034980 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=26447/tid=26483] ppid=26445 vsize=35242408 CPUtime=5003.85 cores=0-31
/proc/26447/task/26483/stat : 26483 (plingeling) R 26445 26447 26421 0 -1 4202560 1730236 1250 0 0 497300 3084 0 1 25 0 33 0 719034980 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4353557 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26447/tid=26484] ppid=26445 vsize=35242408 CPUtime=4991.93 cores=0-31
/proc/26447/task/26484/stat : 26484 (plingeling) R 26445 26447 26421 0 -1 4202560 1311868 1250 0 0 497443 1749 0 1 21 0 33 0 719034980 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=26447/tid=26485] ppid=26445 vsize=35242408 CPUtime=5005.2 cores=0-31
/proc/26447/task/26485/stat : 26485 (plingeling) R 26445 26447 26421 0 -1 4202560 920183 1250 0 0 499546 973 0 1 25 0 33 0 719034980 36088225792 8695565 268435456000 4194304 4417452 140736041086384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 22 0 0 0
Current children cumulated CPU time (s) 160002
Current children cumulated vsize (KiB) 35242408

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

[startup+5022.33 s]
/proc/loadavg: 32.04 32.04 32.06 34/446 26640
/proc/meminfo: memFree=220651172/264257624 swapFree=67111528/67111528
[pid=26447] ppid=26445 vsize=0 CPUtime=160002 cores=0-31
/proc/26447/stat : 26447 (plingeling) Z 26445 26447 26421 0 -1 4203548 40588347 1250 0 0 15944058 56176 0 1 18 0 2 0 719034170 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 30 0 0 0
/proc/26447/statm: 0 0 0 0 0 0 0
[pid=26447/tid=26478] ppid=26445 vsize=0 CPUtime=5004.25 cores=0-31
/proc/26447/task/26478/stat : 26478 (plingeling) R 26445 26447 26421 0 -1 4203588 907215 1250 0 0 499385 1039 0 1 25 0 2 0 719034979 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 30 0 0 0
Current children cumulated CPU time (s) 160002
Current children cumulated vsize (KiB) 0

Sending SIGKILL to process tree (bottom up)

Sending SIGKILL to -26447

Solver just ended. Dumping a history of the last processes samples

[startup+5023 s]
/proc/loadavg: 32.04 32.04 32.06 2/416 26641
/proc/meminfo: memFree=223867600/264257624 swapFree=67111528/67111528
[pid=26447] ppid=26445 vsize=0 CPUtime=160003 cores=0-31
/proc/26447/stat : 26447 (plingeling) Z 26445 26447 26421 0 -1 4203548 40588347 1250 0 0 15944058 56243 0 1 18 0 2 0 719034170 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 30 0 0 0
/proc/26447/statm: 0 0 0 0 0 0 0
[pid=26447/tid=26478] ppid=26445 vsize=0 CPUtime=5004.93 cores=0-31
/proc/26447/task/26478/stat : 26478 (plingeling) R 26445 26447 26421 0 -1 4203588 907215 1250 0 0 499385 1107 0 1 25 0 2 0 719034979 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 30 0 0 0
Current children cumulated CPU time (s) 160003
Current children cumulated vsize (KiB) 0

[startup+5023.8 s]
/proc/loadavg: 32.04 32.04 32.06 2/416 26641
/proc/meminfo: memFree=242034592/264257624 swapFree=67111528/67111528
[pid=26447] ppid=26445 vsize=0 CPUtime=160004 cores=0-31
/proc/26447/stat : 26447 (plingeling) Z 26445 26447 26421 0 -1 4203548 40588347 1250 0 0 15944058 56323 0 1 18 0 2 0 719034170 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 30 0 0 0
/proc/26447/statm: 0 0 0 0 0 0 0
[pid=26447/tid=26478] ppid=26445 vsize=0 CPUtime=5005.73 cores=0-31
/proc/26447/task/26478/stat : 26478 (plingeling) R 26445 26447 26421 0 -1 4203588 907215 1250 0 0 499385 1187 0 1 25 0 2 0 719034979 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 30 0 0 0
Current children cumulated CPU time (s) 160004
Current children cumulated vsize (KiB) 0

[startup+5024.2 s]
/proc/loadavg: 32.04 32.04 32.06 2/416 26641
/proc/meminfo: memFree=242034592/264257624 swapFree=67111528/67111528
[pid=26447] ppid=26445 vsize=0 CPUtime=160004 cores=0-31
/proc/26447/stat : 26447 (plingeling) Z 26445 26447 26421 0 -1 4203548 40588347 1250 0 0 15944058 56363 0 1 18 0 2 0 719034170 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 30 0 0 0
/proc/26447/statm: 0 0 0 0 0 0 0
[pid=26447/tid=26478] ppid=26445 vsize=0 CPUtime=5006.13 cores=0-31
/proc/26447/task/26478/stat : 26478 (plingeling) R 26445 26447 26421 0 -1 4203588 907215 1250 0 0 499385 1227 0 1 25 0 2 0 719034979 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 30 0 0 0
Current children cumulated CPU time (s) 160004
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5024.28
CPU time (s): 160004
CPU user time (s): 159441
CPU system time (s): 563.727
CPU usage (%): 3184.62
Max. virtual memory (cumulated for all children) (KiB): 35242408

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 159441
system time used= 563.727
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 40589597
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 8724
involuntary context switches= 402993

runsolver used 25.9681 second user time and 115.977 second system time

The end

Launcher Data 

Begin job on node200 at 2011-05-22 16:44:38
IDJOB=3372840
IDBENCH=84227
IDSOLVER=1852
FILE ID=node200/3372840-1306075477
RUNJOBID= node200-1306075477-26435
PBS_JOBID= 13387815
Free space on /tmp= 62044 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT03/industrial/li/li-test4/li-test4-100.shuffled-as.sat03-370.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3372840-1306075477/watcher-3372840-1306075477 -o /tmp/evaluation-result-3372840-1306075477/solver-3372840-1306075477 -C 160000 -W 5100 -M 266338304  plingeling -t 32 HOME/instance-3372840-1306075477.cnf

TIME LIMIT= 160000 seconds
MEMORY LIMIT= 266338304 MiB
NBCORE= 32

MD5SUM BENCH= 5e5606ae7367b9272c60adc115594912
RANDOM SEED=426813060

node200.alineos.net Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 64
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 96
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 66
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 98
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 68
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 100
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 70
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.98
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 102
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 16
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 17
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 48
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 18
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 80
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 19
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 112
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 20
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 21
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 50
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 22
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 82
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 114
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.23
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 24
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 25
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 26
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 84
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 27
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 116
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 28
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 29
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 54
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 30
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 86
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 31
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 118
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     264257624 kB
MemFree:      255497192 kB
Buffers:        455344 kB
Cached:        7902688 kB
SwapCached:          0 kB
Active:        7455356 kB
Inactive:       971744 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     264257624 kB
LowFree:      255497192 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:            3696 kB
Writeback:           0 kB
AnonPages:       70168 kB
Mapped:          13484 kB
Slab:           253044 kB
PageTables:       3996 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  199240340 kB
Committed_AS:   187064 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296372 kB
VmallocChunk: 34359440603 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 62040 MiB
End job on node200 at 2011-05-22 18:08:22