Trace number 3372555

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 160004 5020.38

General information on the benchmark

Nameapplication/jarvisalo/smtqfbv-aigs/
smtlib-qfbv-aigs-VS3-benchmark-S2-tseitin.cnf
MD5SUMb967fca11e85ef5b614d3452b1c4b2ad
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables257030
Number of clauses769313
Sum of the clauses size1795061
Maximum clause length3
Minimum clause length1
Number of clauses of size 12
Number of clauses of size 2512874
Number of clauses of size 3256437
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3372555-1305675990/watcher-3372555-1305675990 -o /tmp/evaluation-result-3372555-1305675990/solver-3372555-1305675990 -C 160000 -W 5100 -M 266338304 plingeling -t 32 HOME/instance-3372555-1305675990.cnf 

running on 32 cores: 0,4,8,12,16,20,24,28,1,5,9,13,17,21,25,29,2,6,10,14,18,22,26,30,3,7,11,15,19,23,27,31

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 160000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 160030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 272730423296 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 272730474496 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 11.96 16.69 21.57 1/413 28523
/proc/meminfo: memFree=255480676/264257624 swapFree=67111528/67111528
[pid=28523] ppid=28521 vsize=18924 CPUtime=0.02 cores=0-31
/proc/28523/stat : 28523 (plingeling) R 28521 28523 27975 0 -1 4202496 2785 1242 0 0 2 0 0 0 19 0 1 0 679085515 19378176 2716 268435456000 4194304 4417452 140736431069568 18446744073709551615 258650649801 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/28523/statm: 4731 2735 145 55 0 2617 0

[startup+0.09446 s]
/proc/loadavg: 11.96 16.69 21.57 1/413 28523
/proc/meminfo: memFree=255480676/264257624 swapFree=67111528/67111528
[pid=28523] ppid=28521 vsize=25752 CPUtime=0.08 cores=0-31
/proc/28523/stat : 28523 (plingeling) R 28521 28523 27975 0 -1 4202496 4527 1242 0 0 8 0 0 0 19 0 1 0 679085515 26370048 4458 268435456000 4194304 4417452 140736431069568 18446744073709551615 4227906 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/28523/statm: 6438 4458 145 55 0 4324 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 25752

[startup+0.100508 s]
/proc/loadavg: 11.96 16.69 21.57 1/413 28523
/proc/meminfo: memFree=255480676/264257624 swapFree=67111528/67111528
[pid=28523] ppid=28521 vsize=30268 CPUtime=0.08 cores=0-31
/proc/28523/stat : 28523 (plingeling) R 28521 28523 27975 0 -1 4202496 5657 1242 0 0 8 0 0 0 19 0 1 0 679085515 30994432 5588 268435456000 4194304 4417452 140736431069568 18446744073709551615 258650649821 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/28523/statm: 7567 5588 146 55 0 5453 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 30268

[startup+0.300468 s]
/proc/loadavg: 11.96 16.69 21.57 1/413 28523
/proc/meminfo: memFree=255480676/264257624 swapFree=67111528/67111528
[pid=28523] ppid=28521 vsize=48928 CPUtime=0.29 cores=0-31
/proc/28523/stat : 28523 (plingeling) R 28521 28523 27975 0 -1 4202496 10322 1242 0 0 27 2 0 0 20 0 1 0 679085515 50102272 10237 268435456000 4194304 4417452 140736431069568 18446744073709551615 4264909 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/28523/statm: 12232 10237 146 55 0 10118 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 48928

[startup+0.700484 s]
/proc/loadavg: 11.96 16.69 21.57 1/413 28523
/proc/meminfo: memFree=255480676/264257624 swapFree=67111528/67111528
[pid=28523] ppid=28521 vsize=83500 CPUtime=0.69 cores=0-31
/proc/28523/stat : 28523 (plingeling) R 28521 28523 27975 0 -1 4202496 18949 1242 0 0 65 4 0 0 20 0 1 0 679085515 85504000 18864 268435456000 4194304 4417452 140736431069568 18446744073709551615 4259017 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/28523/statm: 20875 18864 146 55 0 18761 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 83500

[startup+1.50052 s]
/proc/loadavg: 11.96 16.69 21.57 2/414 28529
/proc/meminfo: memFree=255333960/264257624 swapFree=67111528/67111528
[pid=28523] ppid=28521 vsize=153916 CPUtime=1.48 cores=0-31
/proc/28523/stat : 28523 (plingeling) R 28521 28523 27975 0 -1 4202496 36585 1242 0 0 141 7 0 0 23 0 1 0 679085515 157609984 36500 268435456000 4194304 4417452 140736431069568 18446744073709551615 4274358 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/28523/statm: 38479 36500 146 55 0 36365 0
Current children cumulated CPU time (s) 1.48
Current children cumulated vsize (KiB) 153916

[startup+3.10056 s]
/proc/loadavg: 11.08 16.43 21.46 2/414 28529
/proc/meminfo: memFree=255194136/264257624 swapFree=67111528/67111528
[pid=28523] ppid=28521 vsize=294428 CPUtime=3.08 cores=0-31
/proc/28523/stat : 28523 (plingeling) R 28521 28523 27975 0 -1 4202496 71714 1242 0 0 294 14 0 0 25 0 1 0 679085515 301494272 71628 268435456000 4194304 4417452 140736431069568 18446744073709551615 4264940 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/28523/statm: 73607 71628 146 55 0 71493 0
Current children cumulated CPU time (s) 3.08
Current children cumulated vsize (KiB) 294428

[startup+6.30069 s]
/proc/loadavg: 11.08 16.43 21.46 2/414 28529
/proc/meminfo: memFree=254912936/264257624 swapFree=67111528/67111528
[pid=28523] ppid=28521 vsize=575932 CPUtime=6.29 cores=0-31
/proc/28523/stat : 28523 (plingeling) R 28521 28523 27975 0 -1 4202496 142095 1242 0 0 600 29 0 0 25 0 1 0 679085515 589754368 142004 268435456000 4194304 4417452 140736431069568 18446744073709551615 4274347 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/28523/statm: 143983 142004 146 55 0 141869 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 575932

[startup+12.7009 s]
/proc/loadavg: 9.53 15.92 21.24 2/414 28529
/proc/meminfo: memFree=254356780/264257624 swapFree=67111528/67111528
[pid=28523] ppid=28521 vsize=1137568 CPUtime=12.68 cores=0-31
/proc/28523/stat : 28523 (plingeling) R 28521 28523 27975 0 -1 4202496 282557 1242 0 0 1211 57 0 0 25 0 1 0 679085515 1164869632 282413 268435456000 4194304 4417452 140736431069568 18446744073709551615 4264926 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/28523/statm: 284392 282413 146 55 0 282278 0
Current children cumulated CPU time (s) 12.68
Current children cumulated vsize (KiB) 1137568

[startup+25.5022 s]
/proc/loadavg: 12.98 16.45 21.36 33/446 28561
/proc/meminfo: memFree=253315892/264257624 swapFree=67111528/67111528
[pid=28523] ppid=28521 vsize=2771640 CPUtime=357.85 cores=0-31
/proc/28523/stat : 28523 (plingeling) S 28521 28523 27975 0 -1 4202496 1036388 1242 0 0 34846 939 0 0 19 0 33 0 679085515 2838159360 537612 268435456000 4194304 4417452 140736431069568 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 29 0 0 0
/proc/28523/statm: 692910 537630 174 55 0 690796 0
[pid=28523/tid=28530] ppid=28521 vsize=2771640 CPUtime=10.68 cores=0-31
/proc/28523/task/28530/stat : 28530 (plingeling) R 28521 28523 27975 0 -1 4202560 30176 1242 0 0 1038 30 0 0 25 0 33 0 679086942 2838159360 537636 268435456000 4194304 4417452 140736431069568 18446744073709551615 4273677 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=28523/tid=28531] ppid=28521 vsize=2771640 CPUtime=10.71 cores=0-31
/proc/28523/task/28531/stat : 28531 (plingeling) R 28521 28523 27975 0 -1 4202560 33205 1242 0 0 1031 40 0 0 25 0 33 0 679086942 2838159360 537644 268435456000 4194304 4417452 140736431069568 18446744073709551615 4272261 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28523/tid=28532] ppid=28521 vsize=2771640 CPUtime=10.68 cores=0-31
/proc/28523/task/28532/stat : 28532 (plingeling) R 28521 28523 27975 0 -1 4202560 22602 1242 0 0 1042 26 0 0 18 0 33 0 679086942 2838159360 537651 268435456000 4194304 4417452 140736431069568 18446744073709551615 4235999 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=28523/tid=28533] ppid=28521 vsize=2771640 CPUtime=11.19 cores=0-31
/proc/28523/task/28533/stat : 28533 (plingeling) R 28521 28523 27975 0 -1 4202560 4485 1242 0 0 1110 9 0 0 25 0 33 0 679086942 2838159360 537659 268435456000 4194304 4417452 140736431069568 18446744073709551615 4353557 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=28523/tid=28534] ppid=28521 vsize=2771640 CPUtime=10.52 cores=0-31
/proc/28523/task/28534/stat : 28534 (plingeling) R 28521 28523 27975 0 -1 4202560 19764 1242 0 0 1022 30 0 0 25 0 33 0 679086942 2838159360 537667 268435456000 4194304 4417452 140736431069568 18446744073709551615 4236148 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=28523/tid=28535] ppid=28521 vsize=2771640 CPUtime=10.49 cores=0-31
/proc/28523/task/28535/stat : 28535 (plingeling) R 28521 28523 27975 0 -1 4202560 27903 1242 0 0 1017 32 0 0 25 0 33 0 679086942 2838159360 537674 268435456000 4194304 4417452 140736431069568 18446744073709551615 4262065 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=28523/tid=28536] ppid=28521 vsize=2771640 CPUtime=11.08 cores=0-31
/proc/28523/task/28536/stat : 28536 (plingeling) R 28521 28523 27975 0 -1 4202560 22714 1242 0 0 1083 25 0 0 25 0 33 0 679086942 2838159360 537681 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364587 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28523/tid=28537] ppid=28521 vsize=2771640 CPUtime=10.67 cores=0-31
/proc/28523/task/28537/stat : 28537 (plingeling) R 28521 28523 27975 0 -1 4202560 23692 1242 0 0 1038 29 0 0 25 0 33 0 679086942 2838159360 537689 268435456000 4194304 4417452 140736431069568 18446744073709551615 4240123 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=28523/tid=28538] ppid=28521 vsize=2771640 CPUtime=10.78 cores=0-31
/proc/28523/task/28538/stat : 28538 (plingeling) R 28521 28523 27975 0 -1 4202560 16649 1242 0 0 1065 13 0 0 25 0 33 0 679086942 2838159360 537696 268435456000 4194304 4417452 140736431069568 18446744073709551615 4252459 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=28523/tid=28539] ppid=28521 vsize=2771640 CPUtime=10.73 cores=0-31
/proc/28523/task/28539/stat : 28539 (plingeling) R 28521 28523 27975 0 -1 4202560 24028 1242 0 0 1055 18 0 0 25 0 33 0 679086942 2838159360 537703 268435456000 4194304 4417452 140736431069568 18446744073709551615 4233495 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=28523/tid=28540] ppid=28521 vsize=2771640 CPUtime=10.57 cores=0-31
/proc/28523/task/28540/stat : 28540 (plingeling) R 28521 28523 27975 0 -1 4202560 20109 1242 0 0 1031 26 0 0 25 0 33 0 679086942 2838159360 537710 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=28523/tid=28541] ppid=28521 vsize=2771640 CPUtime=10.6 cores=0-31
/proc/28523/task/28541/stat : 28541 (plingeling) R 28521 28523 27975 0 -1 4202560 16641 1242 0 0 1040 20 0 0 25 0 33 0 679086942 2838159360 537718 268435456000 4194304 4417452 140736431069568 18446744073709551615 258650653381 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=28523/tid=28542] ppid=28521 vsize=2771640 CPUtime=10.89 cores=0-31
/proc/28523/task/28542/stat : 28542 (plingeling) R 28521 28523 27975 0 -1 4202560 11368 1242 0 0 1057 32 0 0 25 0 33 0 679086942 2838159360 537725 268435456000 4194304 4417452 140736431069568 18446744073709551615 4272961 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=28523/tid=28543] ppid=28521 vsize=2771640 CPUtime=10.68 cores=0-31
/proc/28523/task/28543/stat : 28543 (plingeling) R 28521 28523 27975 0 -1 4202560 30105 1242 0 0 1045 23 0 0 25 0 33 0 679086942 2838159360 537732 268435456000 4194304 4417452 140736431069568 18446744073709551615 4252379 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=28523/tid=28544] ppid=28521 vsize=2771640 CPUtime=11.04 cores=0-31
/proc/28523/task/28544/stat : 28544 (plingeling) R 28521 28523 27975 0 -1 4202560 20884 1242 0 0 1087 17 0 0 25 0 33 0 679086943 2838159360 537739 268435456000 4194304 4417452 140736431069568 18446744073709551615 4343423 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=28523/tid=28545] ppid=28521 vsize=2771640 CPUtime=10.75 cores=0-31
/proc/28523/task/28545/stat : 28545 (plingeling) R 28521 28523 27975 0 -1 4202560 27069 1242 0 0 1040 35 0 0 25 0 33 0 679086943 2838159360 537747 268435456000 4194304 4417452 140736431069568 18446744073709551615 4262037 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=28523/tid=28546] ppid=28521 vsize=2771640 CPUtime=10.61 cores=0-31
/proc/28523/task/28546/stat : 28546 (plingeling) R 28521 28523 27975 0 -1 4202560 18432 1242 0 0 1030 31 0 0 25 0 33 0 679086943 2838159360 537754 268435456000 4194304 4417452 140736431069568 18446744073709551615 4253420 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=28523/tid=28547] ppid=28521 vsize=2771640 CPUtime=10.4 cores=0-31
/proc/28523/task/28547/stat : 28547 (plingeling) R 28521 28523 27975 0 -1 4202560 24838 1242 0 0 1017 23 0 0 25 0 33 0 679086943 2838159360 537762 268435456000 4194304 4417452 140736431069568 18446744073709551615 4253336 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=28523/tid=28548] ppid=28521 vsize=2771640 CPUtime=11.04 cores=0-31
/proc/28523/task/28548/stat : 28548 (plingeling) R 28521 28523 27975 0 -1 4202560 21651 1242 0 0 1080 24 0 0 25 0 33 0 679086943 2838159360 537768 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=28523/tid=28549] ppid=28521 vsize=2771640 CPUtime=11 cores=0-31
/proc/28523/task/28549/stat : 28549 (plingeling) R 28521 28523 27975 0 -1 4202560 20538 1242 0 0 1069 31 0 0 25 0 33 0 679086943 2838159360 537775 268435456000 4194304 4417452 140736431069568 18446744073709551615 4252336 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=28523/tid=28550] ppid=28521 vsize=2771640 CPUtime=10.7 cores=0-31
/proc/28523/task/28550/stat : 28550 (plingeling) R 28521 28523 27975 0 -1 4202560 23070 1242 0 0 1042 28 0 0 25 0 33 0 679086943 2838159360 537782 268435456000 4194304 4417452 140736431069568 18446744073709551615 4252064 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=28523/tid=28551] ppid=28521 vsize=2771640 CPUtime=10.45 cores=0-31
/proc/28523/task/28551/stat : 28551 (plingeling) R 28521 28523 27975 0 -1 4202560 32180 1242 0 0 1013 32 0 0 25 0 33 0 679086943 2838159360 537789 268435456000 4194304 4417452 140736431069568 18446744073709551615 4252240 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=28523/tid=28552] ppid=28521 vsize=2771640 CPUtime=10.96 cores=0-31
/proc/28523/task/28552/stat : 28552 (plingeling) R 28521 28523 27975 0 -1 4202560 27199 1242 0 0 1080 16 0 0 25 0 33 0 679086943 2838159360 537797 268435456000 4194304 4417452 140736431069568 18446744073709551615 4232648 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=28523/tid=28553] ppid=28521 vsize=2771640 CPUtime=10.88 cores=0-31
/proc/28523/task/28553/stat : 28553 (plingeling) R 28521 28523 27975 0 -1 4202560 21891 1242 0 0 1054 34 0 0 25 0 33 0 679086943 2838159360 537804 268435456000 4194304 4417452 140736431069568 18446744073709551615 4226190 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=28523/tid=28554] ppid=28521 vsize=2771640 CPUtime=10.55 cores=0-31
/proc/28523/task/28554/stat : 28554 (plingeling) R 28521 28523 27975 0 -1 4202560 27359 1242 0 0 1028 27 0 0 25 0 33 0 679086943 2838159360 537811 268435456000 4194304 4417452 140736431069568 18446744073709551615 4252045 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=28523/tid=28555] ppid=28521 vsize=2771640 CPUtime=10.94 cores=0-31
/proc/28523/task/28555/stat : 28555 (plingeling) R 28521 28523 27975 0 -1 4202560 15765 1242 0 0 1075 19 0 0 25 0 33 0 679086943 2838159360 537818 268435456000 4194304 4417452 140736431069568 18446744073709551615 4252475 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=28523/tid=28556] ppid=28521 vsize=2771640 CPUtime=10.48 cores=0-31
/proc/28523/task/28556/stat : 28556 (plingeling) R 28521 28523 27975 0 -1 4202560 19092 1242 0 0 1026 22 0 0 18 0 33 0 679086943 2838159360 537826 268435456000 4194304 4417452 140736431069568 18446744073709551615 4270702 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=28523/tid=28557] ppid=28521 vsize=2771640 CPUtime=10.78 cores=0-31
/proc/28523/task/28557/stat : 28557 (plingeling) R 28521 28523 27975 0 -1 4202560 30692 1242 0 0 1041 37 0 0 25 0 33 0 679086943 2838159360 537832 268435456000 4194304 4417452 140736431069568 18446744073709551615 4252515 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28523/tid=28558] ppid=28521 vsize=2771640 CPUtime=10.51 cores=0-31
/proc/28523/task/28558/stat : 28558 (plingeling) R 28521 28523 27975 0 -1 4202560 33132 1242 0 0 1011 40 0 0 25 0 33 0 679086943 2838159360 537839 268435456000 4194304 4417452 140736431069568 18446744073709551615 4273083 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=28523/tid=28559] ppid=28521 vsize=2771640 CPUtime=10.61 cores=0-31
/proc/28523/task/28559/stat : 28559 (plingeling) R 28521 28523 27975 0 -1 4202560 32820 1242 0 0 1026 35 0 0 25 0 33 0 679086943 2838159360 537846 268435456000 4194304 4417452 140736431069568 18446744073709551615 4253179 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=28523/tid=28560] ppid=28521 vsize=2771640 CPUtime=10.69 cores=0-31
/proc/28523/task/28560/stat : 28560 (plingeling) R 28521 28523 27975 0 -1 4202560 31408 1242 0 0 1033 36 0 0 25 0 33 0 679086944 2838159360 537854 268435456000 4194304 4417452 140736431069568 18446744073709551615 4249130 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=28523/tid=28561] ppid=28521 vsize=2771640 CPUtime=10.64 cores=0-31
/proc/28523/task/28561/stat : 28561 (plingeling) R 28521 28523 27975 0 -1 4202560 22435 1242 0 0 1038 26 0 0 25 0 33 0 679086944 2838159360 537861 268435456000 4194304 4417452 140736431069568 18446744073709551615 4320455 0 0 4096 17442 0 0 0 -1 21 0 0 0
Current children cumulated CPU time (s) 357.85
Current children cumulated vsize (KiB) 2771640

[startup+51.1021 s]
/proc/loadavg: 19.47 17.70 21.64 33/446 28563
/proc/meminfo: memFree=253205944/264257624 swapFree=67111528/67111528
[pid=28523] ppid=28521 vsize=2847040 CPUtime=1176.11 cores=0-31
/proc/28523/stat : 28523 (plingeling) S 28521 28523 27975 0 -1 4202496 1092096 1242 0 0 116658 953 0 0 19 0 33 0 679085515 2915368960 568197 268435456000 4194304 4417452 140736431069568 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 29 0 0 0
/proc/28523/statm: 711760 568197 174 55 0 709646 0
[pid=28523/tid=28530] ppid=28521 vsize=2847040 CPUtime=36.27 cores=0-31
/proc/28523/task/28530/stat : 28530 (plingeling) R 28521 28523 27975 0 -1 4202560 32738 1242 0 0 3596 31 0 0 25 0 33 0 679086942 2915368960 568197 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=28523/tid=28531] ppid=28521 vsize=2847040 CPUtime=36.3 cores=0-31
/proc/28523/task/28531/stat : 28531 (plingeling) R 28521 28523 27975 0 -1 4202560 34231 1242 0 0 3590 40 0 0 25 0 33 0 679086942 2915368960 568197 268435456000 4194304 4417452 140736431069568 18446744073709551615 4365962 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28523/tid=28532] ppid=28521 vsize=2847040 CPUtime=36.27 cores=0-31
/proc/28523/task/28532/stat : 28532 (plingeling) R 28521 28523 27975 0 -1 4202560 24139 1242 0 0 3601 26 0 0 25 0 33 0 679086942 2915368960 568197 268435456000 4194304 4417452 140736431069568 18446744073709551615 4348404 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=28523/tid=28533] ppid=28521 vsize=2847040 CPUtime=36.77 cores=0-31
/proc/28523/task/28533/stat : 28533 (plingeling) R 28521 28523 27975 0 -1 4202560 4485 1242 0 0 3668 9 0 0 25 0 33 0 679086942 2915368960 568197 268435456000 4194304 4417452 140736431069568 18446744073709551615 4365990 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=28523/tid=28534] ppid=28521 vsize=2847040 CPUtime=36.1 cores=0-31
/proc/28523/task/28534/stat : 28534 (plingeling) R 28521 28523 27975 0 -1 4202560 20790 1242 0 0 3579 31 0 0 25 0 33 0 679086942 2915368960 568197 268435456000 4194304 4417452 140736431069568 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=28523/tid=28535] ppid=28521 vsize=2847040 CPUtime=36.08 cores=0-31
/proc/28523/task/28535/stat : 28535 (plingeling) R 28521 28523 27975 0 -1 4202560 30977 1242 0 0 3576 32 0 0 25 0 33 0 679086942 2915368960 568197 268435456000 4194304 4417452 140736431069568 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=28523/tid=28536] ppid=28521 vsize=2847040 CPUtime=36.68 cores=0-31
/proc/28523/task/28536/stat : 28536 (plingeling) R 28521 28523 27975 0 -1 4202560 22900 1242 0 0 3642 26 0 0 25 0 33 0 679086942 2915368960 568197 268435456000 4194304 4417452 140736431069568 18446744073709551615 4236649 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28523/tid=28537] ppid=28521 vsize=2847040 CPUtime=36.27 cores=0-31
/proc/28523/task/28537/stat : 28537 (plingeling) R 28521 28523 27975 0 -1 4202560 25742 1242 0 0 3597 30 0 0 25 0 33 0 679086942 2915368960 568197 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=28523/tid=28538] ppid=28521 vsize=2847040 CPUtime=36.37 cores=0-31
/proc/28523/task/28538/stat : 28538 (plingeling) R 28521 28523 27975 0 -1 4202560 16651 1242 0 0 3624 13 0 0 25 0 33 0 679086942 2915368960 568197 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364576 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=28523/tid=28539] ppid=28521 vsize=2847040 CPUtime=36.32 cores=0-31
/proc/28523/task/28539/stat : 28539 (plingeling) R 28521 28523 27975 0 -1 4202560 25566 1242 0 0 3614 18 0 0 25 0 33 0 679086942 2915368960 568197 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=28523/tid=28540] ppid=28521 vsize=2847040 CPUtime=36.16 cores=0-31

################
# More data... #
################

[pid=28523/tid=28530] ppid=28521 vsize=4583776 CPUtime=4946.35 cores=0-31
/proc/28523/task/28530/stat : 28530 (plingeling) R 28521 28523 27975 0 -1 4202560 118358 1242 0 0 494585 50 0 0 25 0 33 0 679086942 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4250936 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=28523/tid=28531] ppid=28521 vsize=4583776 CPUtime=4943.12 cores=0-31
/proc/28523/task/28531/stat : 28531 (plingeling) R 28521 28523 27975 0 -1 4202560 126842 1242 0 0 494200 112 0 0 25 0 33 0 679086942 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28523/tid=28532] ppid=28521 vsize=4583776 CPUtime=4946.31 cores=0-31
/proc/28523/task/28532/stat : 28532 (plingeling) R 28521 28523 27975 0 -1 4202560 171333 1242 0 0 494540 91 0 0 25 0 33 0 679086942 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=28523/tid=28533] ppid=28521 vsize=4583776 CPUtime=4906.11 cores=0-31
/proc/28523/task/28533/stat : 28533 (plingeling) R 28521 28523 27975 0 -1 4202560 16806 1242 0 0 490577 34 0 0 25 0 33 0 679086942 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=28523/tid=28534] ppid=28521 vsize=4583776 CPUtime=4945.16 cores=0-31
/proc/28523/task/28534/stat : 28534 (plingeling) R 28521 28523 27975 0 -1 4202560 131899 1242 0 0 494343 173 0 0 25 0 33 0 679086942 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=28523/tid=28535] ppid=28521 vsize=4583776 CPUtime=4946.27 cores=0-31
/proc/28523/task/28535/stat : 28535 (plingeling) R 28521 28523 27975 0 -1 4202560 179815 1242 0 0 494462 165 0 0 25 0 33 0 679086942 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4366094 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=28523/tid=28536] ppid=28521 vsize=4583776 CPUtime=4946.91 cores=0-31
/proc/28523/task/28536/stat : 28536 (plingeling) R 28521 28523 27975 0 -1 4202560 113890 1242 0 0 494635 56 0 0 25 0 33 0 679086942 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28523/tid=28537] ppid=28521 vsize=4583776 CPUtime=4946.51 cores=0-31
/proc/28523/task/28537/stat : 28537 (plingeling) R 28521 28523 27975 0 -1 4202560 129679 1242 0 0 494547 104 0 0 25 0 33 0 679086942 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=28523/tid=28538] ppid=28521 vsize=4583776 CPUtime=4946.53 cores=0-31
/proc/28523/task/28538/stat : 28538 (plingeling) R 28521 28523 27975 0 -1 4202560 138779 1242 0 0 494588 65 0 0 25 0 33 0 679086942 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=28523/tid=28539] ppid=28521 vsize=4583776 CPUtime=4946.53 cores=0-31
/proc/28523/task/28539/stat : 28539 (plingeling) R 28521 28523 27975 0 -1 4202560 150257 1242 0 0 494540 113 0 0 25 0 33 0 679086942 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=28523/tid=28540] ppid=28521 vsize=4583776 CPUtime=4945.55 cores=0-31
/proc/28523/task/28540/stat : 28540 (plingeling) R 28521 28523 27975 0 -1 4202560 128378 1242 0 0 494425 130 0 0 25 0 33 0 679086942 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364650 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=28523/tid=28541] ppid=28521 vsize=4583776 CPUtime=4923.25 cores=0-31
/proc/28523/task/28541/stat : 28541 (plingeling) R 28521 28523 27975 0 -1 4202560 160509 1242 0 0 492216 109 0 0 25 0 33 0 679086942 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4365959 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=28523/tid=28542] ppid=28521 vsize=4583776 CPUtime=4946.69 cores=0-31
/proc/28523/task/28542/stat : 28542 (plingeling) R 28521 28523 27975 0 -1 4202560 146479 1242 0 0 494529 140 0 0 25 0 33 0 679086942 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=28523/tid=28543] ppid=28521 vsize=4583776 CPUtime=4946.32 cores=0-31
/proc/28523/task/28543/stat : 28543 (plingeling) R 28521 28523 27975 0 -1 4202560 192482 1242 0 0 494469 163 0 0 25 0 33 0 679086942 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=28523/tid=28544] ppid=28521 vsize=4583776 CPUtime=4945.3 cores=0-31
/proc/28523/task/28544/stat : 28544 (plingeling) R 28521 28523 27975 0 -1 4202560 66457 1242 0 0 494479 51 0 0 25 0 33 0 679086943 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=28523/tid=28545] ppid=28521 vsize=4583776 CPUtime=4946.5 cores=0-31
/proc/28523/task/28545/stat : 28545 (plingeling) R 28521 28523 27975 0 -1 4202560 159791 1242 0 0 494527 123 0 0 25 0 33 0 679086943 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=28523/tid=28546] ppid=28521 vsize=4583776 CPUtime=4946.39 cores=0-31
/proc/28523/task/28546/stat : 28546 (plingeling) R 28521 28523 27975 0 -1 4202560 159103 1242 0 0 494476 163 0 0 25 0 33 0 679086943 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=28523/tid=28547] ppid=28521 vsize=4583776 CPUtime=4946.01 cores=0-31
/proc/28523/task/28547/stat : 28547 (plingeling) R 28521 28523 27975 0 -1 4202560 211062 1242 0 0 494495 106 0 0 25 0 33 0 679086943 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=28523/tid=28548] ppid=28521 vsize=4583776 CPUtime=4915.91 cores=0-31
/proc/28523/task/28548/stat : 28548 (plingeling) R 28521 28523 27975 0 -1 4202560 175784 1242 0 0 491433 158 0 0 25 0 33 0 679086943 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4357540 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=28523/tid=28549] ppid=28521 vsize=4583776 CPUtime=4938.23 cores=0-31
/proc/28523/task/28549/stat : 28549 (plingeling) R 28521 28523 27975 0 -1 4202560 163816 1242 0 0 493646 177 0 0 25 0 33 0 679086943 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=28523/tid=28550] ppid=28521 vsize=4583776 CPUtime=4946.53 cores=0-31
/proc/28523/task/28550/stat : 28550 (plingeling) R 28521 28523 27975 0 -1 4202560 76193 1242 0 0 494612 41 0 0 25 0 33 0 679086943 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=28523/tid=28551] ppid=28521 vsize=4583776 CPUtime=4937.8 cores=0-31
/proc/28523/task/28551/stat : 28551 (plingeling) R 28521 28523 27975 0 -1 4202560 155055 1242 0 0 493646 134 0 0 25 0 33 0 679086943 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=28523/tid=28552] ppid=28521 vsize=4583776 CPUtime=4946.76 cores=0-31
/proc/28523/task/28552/stat : 28552 (plingeling) R 28521 28523 27975 0 -1 4202560 158960 1242 0 0 494572 104 0 0 25 0 33 0 679086943 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364650 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=28523/tid=28553] ppid=28521 vsize=4583776 CPUtime=4944.19 cores=0-31
/proc/28523/task/28553/stat : 28553 (plingeling) R 28521 28523 27975 0 -1 4202560 152524 1242 0 0 494273 146 0 0 25 0 33 0 679086943 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=28523/tid=28554] ppid=28521 vsize=4583776 CPUtime=4946.38 cores=0-31
/proc/28523/task/28554/stat : 28554 (plingeling) R 28521 28523 27975 0 -1 4202560 109624 1242 0 0 494545 93 0 0 25 0 33 0 679086943 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=28523/tid=28555] ppid=28521 vsize=4583776 CPUtime=4932.75 cores=0-31
/proc/28523/task/28555/stat : 28555 (plingeling) R 28521 28523 27975 0 -1 4202560 185984 1242 0 0 493127 148 0 0 25 0 33 0 679086943 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=28523/tid=28556] ppid=28521 vsize=4583776 CPUtime=4946.34 cores=0-31
/proc/28523/task/28556/stat : 28556 (plingeling) R 28521 28523 27975 0 -1 4202560 142269 1242 0 0 494541 93 0 0 25 0 33 0 679086943 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=28523/tid=28557] ppid=28521 vsize=4583776 CPUtime=4946.55 cores=0-31
/proc/28523/task/28557/stat : 28557 (plingeling) R 28521 28523 27975 0 -1 4202560 68848 1242 0 0 494568 87 0 0 25 0 33 0 679086943 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28523/tid=28558] ppid=28521 vsize=4583776 CPUtime=4946.33 cores=0-31
/proc/28523/task/28558/stat : 28558 (plingeling) R 28521 28523 27975 0 -1 4202560 137536 1242 0 0 494534 99 0 0 25 0 33 0 679086943 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=28523/tid=28559] ppid=28521 vsize=4583776 CPUtime=4937.58 cores=0-31
/proc/28523/task/28559/stat : 28559 (plingeling) R 28521 28523 27975 0 -1 4202560 207959 1242 0 0 493578 180 0 0 25 0 33 0 679086943 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4277545 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=28523/tid=28560] ppid=28521 vsize=4583776 CPUtime=4946.52 cores=0-31
/proc/28523/task/28560/stat : 28560 (plingeling) R 28521 28523 27975 0 -1 4202560 55110 1242 0 0 494588 64 0 0 25 0 33 0 679086944 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364829 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=28523/tid=28561] ppid=28521 vsize=4583776 CPUtime=4946.46 cores=0-31
/proc/28523/task/28561/stat : 28561 (plingeling) R 28521 28523 27975 0 -1 4202560 57099 1242 0 0 494612 34 0 0 25 0 33 0 679086944 4693786624 985031 268435456000 4194304 4417452 140736431069568 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 21 0 0 0
Current children cumulated CPU time (s) 158159
Current children cumulated vsize (KiB) 4583776



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5020 s]
/proc/loadavg: 32.06 32.11 32.07 33/446 28716
/proc/meminfo: memFree=251595172/264257624 swapFree=67111528/67111528
[pid=28523] ppid=28521 vsize=4547176 CPUtime=160004 cores=0-31
/proc/28523/stat : 28523 (plingeling) S 28521 28523 27975 0 -1 4202496 4705109 1242 0 0 15996742 3629 0 0 19 0 33 0 679085515 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 29 0 0 0
/proc/28523/statm: 1136794 976035 175 55 0 1134680 0
[pid=28523/tid=28530] ppid=28521 vsize=4547176 CPUtime=5004.06 cores=0-31
/proc/28523/task/28530/stat : 28530 (plingeling) R 28521 28523 27975 0 -1 4202560 118358 1242 0 0 500356 50 0 0 25 0 33 0 679086942 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4277864 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=28523/tid=28531] ppid=28521 vsize=4547176 CPUtime=5000.84 cores=0-31
/proc/28523/task/28531/stat : 28531 (plingeling) R 28521 28523 27975 0 -1 4202560 128465 1242 0 0 499971 113 0 0 25 0 33 0 679086942 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364741 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28523/tid=28532] ppid=28521 vsize=4547176 CPUtime=5004.02 cores=0-31
/proc/28523/task/28532/stat : 28532 (plingeling) R 28521 28523 27975 0 -1 4202560 174051 1242 0 0 500310 92 0 0 25 0 33 0 679086942 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=28523/tid=28533] ppid=28521 vsize=4547176 CPUtime=4963.82 cores=0-31
/proc/28523/task/28533/stat : 28533 (plingeling) R 28521 28523 27975 0 -1 4202560 16806 1242 0 0 496348 34 0 0 25 0 33 0 679086942 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=28523/tid=28534] ppid=28521 vsize=4547176 CPUtime=5002.86 cores=0-31
/proc/28523/task/28534/stat : 28534 (plingeling) R 28521 28523 27975 0 -1 4202560 133511 1242 0 0 500113 173 0 0 25 0 33 0 679086942 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=28523/tid=28535] ppid=28521 vsize=4547176 CPUtime=5003.98 cores=0-31
/proc/28523/task/28535/stat : 28535 (plingeling) R 28521 28523 27975 0 -1 4202560 190693 1242 0 0 500203 195 0 0 25 0 33 0 679086942 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=28523/tid=28536] ppid=28521 vsize=4547176 CPUtime=5004.61 cores=0-31
/proc/28523/task/28536/stat : 28536 (plingeling) R 28521 28523 27975 0 -1 4202560 114713 1242 0 0 500405 56 0 0 25 0 33 0 679086942 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28523/tid=28537] ppid=28521 vsize=4547176 CPUtime=5004.22 cores=0-31
/proc/28523/task/28537/stat : 28537 (plingeling) R 28521 28523 27975 0 -1 4202560 129679 1242 0 0 500318 104 0 0 25 0 33 0 679086942 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=28523/tid=28538] ppid=28521 vsize=4547176 CPUtime=5004.25 cores=0-31
/proc/28523/task/28538/stat : 28538 (plingeling) R 28521 28523 27975 0 -1 4202560 140504 1242 0 0 500359 66 0 0 25 0 33 0 679086942 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=28523/tid=28539] ppid=28521 vsize=4547176 CPUtime=5004.24 cores=0-31
/proc/28523/task/28539/stat : 28539 (plingeling) R 28521 28523 27975 0 -1 4202560 153787 1242 0 0 500310 114 0 0 25 0 33 0 679086942 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4236649 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=28523/tid=28540] ppid=28521 vsize=4547176 CPUtime=5003.27 cores=0-31
/proc/28523/task/28540/stat : 28540 (plingeling) R 28521 28523 27975 0 -1 4202560 128378 1242 0 0 500196 131 0 0 25 0 33 0 679086942 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=28523/tid=28541] ppid=28521 vsize=4547176 CPUtime=4979.29 cores=0-31
/proc/28523/task/28541/stat : 28541 (plingeling) R 28521 28523 27975 0 -1 4202560 161357 1242 0 0 497820 109 0 0 25 0 33 0 679086942 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=28523/tid=28542] ppid=28521 vsize=4547176 CPUtime=5004.4 cores=0-31
/proc/28523/task/28542/stat : 28542 (plingeling) R 28521 28523 27975 0 -1 4202560 146479 1242 0 0 500300 140 0 0 25 0 33 0 679086942 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=28523/tid=28543] ppid=28521 vsize=4547176 CPUtime=5004.03 cores=0-31
/proc/28523/task/28543/stat : 28543 (plingeling) R 28521 28523 27975 0 -1 4202560 193272 1242 0 0 500240 163 0 0 25 0 33 0 679086942 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=28523/tid=28544] ppid=28521 vsize=4547176 CPUtime=5002.98 cores=0-31
/proc/28523/task/28544/stat : 28544 (plingeling) R 28521 28523 27975 0 -1 4202560 67324 1242 0 0 500247 51 0 0 25 0 33 0 679086943 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4353598 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=28523/tid=28545] ppid=28521 vsize=4547176 CPUtime=5004.21 cores=0-31
/proc/28523/task/28545/stat : 28545 (plingeling) R 28521 28523 27975 0 -1 4202560 161580 1242 0 0 500297 124 0 0 25 0 33 0 679086943 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=28523/tid=28546] ppid=28521 vsize=4547176 CPUtime=5004.1 cores=0-31
/proc/28523/task/28546/stat : 28546 (plingeling) R 28521 28523 27975 0 -1 4202560 167295 1242 0 0 500245 165 0 0 25 0 33 0 679086943 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=28523/tid=28547] ppid=28521 vsize=4547176 CPUtime=5003.73 cores=0-31
/proc/28523/task/28547/stat : 28547 (plingeling) R 28521 28523 27975 0 -1 4202560 211062 1242 0 0 500267 106 0 0 25 0 33 0 679086943 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364604 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=28523/tid=28548] ppid=28521 vsize=4547176 CPUtime=4973.62 cores=0-31
/proc/28523/task/28548/stat : 28548 (plingeling) R 28521 28523 27975 0 -1 4202560 187426 1242 0 0 497202 160 0 0 25 0 33 0 679086943 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=28523/tid=28549] ppid=28521 vsize=4547176 CPUtime=4995.94 cores=0-31
/proc/28523/task/28549/stat : 28549 (plingeling) R 28521 28523 27975 0 -1 4202560 166031 1242 0 0 499417 177 0 0 25 0 33 0 679086943 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4252708 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=28523/tid=28550] ppid=28521 vsize=4547176 CPUtime=5004.24 cores=0-31
/proc/28523/task/28550/stat : 28550 (plingeling) R 28521 28523 27975 0 -1 4202560 76193 1242 0 0 500383 41 0 0 25 0 33 0 679086943 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=28523/tid=28551] ppid=28521 vsize=4547176 CPUtime=4995.5 cores=0-31
/proc/28523/task/28551/stat : 28551 (plingeling) R 28521 28523 27975 0 -1 4202560 156593 1242 0 0 499416 134 0 0 25 0 33 0 679086943 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=28523/tid=28552] ppid=28521 vsize=4547176 CPUtime=5004.47 cores=0-31
/proc/28523/task/28552/stat : 28552 (plingeling) R 28521 28523 27975 0 -1 4202560 159755 1242 0 0 500343 104 0 0 25 0 33 0 679086943 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=28523/tid=28553] ppid=28521 vsize=4547176 CPUtime=5001.9 cores=0-31
/proc/28523/task/28553/stat : 28553 (plingeling) R 28521 28523 27975 0 -1 4202560 157011 1242 0 0 500043 147 0 0 25 0 33 0 679086943 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364572 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=28523/tid=28554] ppid=28521 vsize=4547176 CPUtime=5004.09 cores=0-31
/proc/28523/task/28554/stat : 28554 (plingeling) R 28521 28523 27975 0 -1 4202560 111428 1242 0 0 500315 94 0 0 25 0 33 0 679086943 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=28523/tid=28555] ppid=28521 vsize=4547176 CPUtime=4990.46 cores=0-31
/proc/28523/task/28555/stat : 28555 (plingeling) R 28521 28523 27975 0 -1 4202560 189633 1242 0 0 498897 149 0 0 25 0 33 0 679086943 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=28523/tid=28556] ppid=28521 vsize=4547176 CPUtime=5004.05 cores=0-31
/proc/28523/task/28556/stat : 28556 (plingeling) R 28521 28523 27975 0 -1 4202560 144715 1242 0 0 500311 94 0 0 25 0 33 0 679086943 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=28523/tid=28557] ppid=28521 vsize=4547176 CPUtime=5004.26 cores=0-31
/proc/28523/task/28557/stat : 28557 (plingeling) R 28521 28523 27975 0 -1 4202560 68848 1242 0 0 500339 87 0 0 25 0 33 0 679086943 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28523/tid=28558] ppid=28521 vsize=4547176 CPUtime=5004.04 cores=0-31
/proc/28523/task/28558/stat : 28558 (plingeling) R 28521 28523 27975 0 -1 4202560 137536 1242 0 0 500305 99 0 0 25 0 33 0 679086943 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=28523/tid=28559] ppid=28521 vsize=4547176 CPUtime=4995.28 cores=0-31
/proc/28523/task/28559/stat : 28559 (plingeling) R 28521 28523 27975 0 -1 4202560 216151 1242 0 0 499347 181 0 0 25 0 33 0 679086943 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4250571 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=28523/tid=28560] ppid=28521 vsize=4547176 CPUtime=5004.23 cores=0-31
/proc/28523/task/28560/stat : 28560 (plingeling) R 28521 28523 27975 0 -1 4202560 55110 1242 0 0 500359 64 0 0 25 0 33 0 679086944 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=28523/tid=28561] ppid=28521 vsize=4547176 CPUtime=5004.16 cores=0-31
/proc/28523/task/28561/stat : 28561 (plingeling) R 28521 28523 27975 0 -1 4202560 58766 1242 0 0 500378 38 0 0 25 0 33 0 679086944 4656308224 976035 268435456000 4194304 4417452 140736431069568 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 21 0 0 0
Current children cumulated CPU time (s) 160004
Current children cumulated vsize (KiB) 4547176

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5020.2 s]
/proc/loadavg: 32.06 32.11 32.07 33/446 28716
/proc/meminfo: memFree=251595172/264257624 swapFree=67111528/67111528
[pid=28523] ppid=28521 vsize=0 CPUtime=160004 cores=0-31
/proc/28523/stat : 28523 (plingeling) Z 28521 28523 27975 0 -1 4203548 4705109 1242 0 0 15996764 3646 0 0 18 0 2 0 679085515 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 29 0 0 0
/proc/28523/statm: 0 0 0 0 0 0 0
[pid=28523/tid=28538] ppid=28521 vsize=0 CPUtime=5004.43 cores=0-31
/proc/28523/task/28538/stat : 28538 (plingeling) R 28521 28523 27975 0 -1 4203588 140504 1242 0 0 500360 83 0 0 25 0 2 0 679086942 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 29 0 0 0
Current children cumulated CPU time (s) 160004
Current children cumulated vsize (KiB) 0

[startup+5020.3 s]
/proc/loadavg: 32.06 32.11 32.07 2/416 28717
/proc/meminfo: memFree=254675820/264257624 swapFree=67111528/67111528
[pid=28523] ppid=28521 vsize=0 CPUtime=160004 cores=0-31
/proc/28523/stat : 28523 (plingeling) Z 28521 28523 27975 0 -1 4203548 4705109 1242 0 0 15996764 3659 0 0 18 0 2 0 679085515 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 29 0 0 0
/proc/28523/statm: 0 0 0 0 0 0 0
[pid=28523/tid=28538] ppid=28521 vsize=0 CPUtime=5004.56 cores=0-31
/proc/28523/task/28538/stat : 28538 (plingeling) R 28521 28523 27975 0 -1 4203588 140504 1242 0 0 500360 96 0 0 25 0 2 0 679086942 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 29 0 0 0
Current children cumulated CPU time (s) 160004
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5020.38
CPU time (s): 160004
CPU user time (s): 159968
CPU system time (s): 36.6444
CPU usage (%): 3187.1
Max. virtual memory (cumulated for all children) (KiB): 4680900

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 159968
system time used= 36.6444
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 4706351
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 6340
involuntary context switches= 422371

runsolver used 25.5871 second user time and 116.219 second system time

The end

Launcher Data 

Begin job on node200 at 2011-05-18 01:46:31
IDJOB=3372555
IDBENCH=82459
IDSOLVER=1852
FILE ID=node200/3372555-1305675990
RUNJOBID= node200-1305675585-27989
PBS_JOBID= 13357376
Free space on /tmp= 62036 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/application/jarvisalo/smtqfbv-aigs/smtlib-qfbv-aigs-VS3-benchmark-S2-tseitin.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3372555-1305675990/watcher-3372555-1305675990 -o /tmp/evaluation-result-3372555-1305675990/solver-3372555-1305675990 -C 160000 -W 5100 -M 266338304  plingeling -t 32 HOME/instance-3372555-1305675990.cnf

TIME LIMIT= 160000 seconds
MEMORY LIMIT= 266338304 MiB
NBCORE= 32

MD5SUM BENCH= b967fca11e85ef5b614d3452b1c4b2ad
RANDOM SEED=889417396

node200.alineos.net Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 64
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 96
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 66
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 98
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 68
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 100
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 70
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.98
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 102
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 16
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 17
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 48
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 18
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 80
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 19
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 112
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 20
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 21
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 50
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 22
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 82
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 114
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.23
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 24
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 25
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 26
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 84
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 27
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 116
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 28
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 29
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 54
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 30
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 86
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 31
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 118
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     264257624 kB
MemFree:      255480964 kB
Buffers:        451464 kB
Cached:        7899952 kB
SwapCached:          0 kB
Active:        7417956 kB
Inactive:      1007372 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     264257624 kB
LowFree:      255480964 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           13956 kB
Writeback:           0 kB
AnonPages:       74076 kB
Mapped:          14536 kB
Slab:           252184 kB
PageTables:       4568 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  199240340 kB
Committed_AS:   206636 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296372 kB
VmallocChunk: 34359440603 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 62036 MiB
End job on node200 at 2011-05-18 03:10:13