Trace number 3728796

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
pwbo 2.02MSAT (TO) 1800.36 900.216

General information on the benchmark

Name/SOFT-SMALLINT-LIN/PB06/submitted-PB05/aloul/FPGA_SAT05/
normalized-chnl20_25_pb.cnf.cr--soft-0-100-0.wbo
MD5SUMcad14126bbe64a82703c3144c4e347cd
Bench CategorySOFT-SMALLINT-LIN (only soft constraints, small integers, linear constraints)
Best result obtained on this benchmarkMOPT
Best cost obtained on this benchmark10
Best CPU time to get the best result obtained on this benchmark0.25696
Max-Satisfiable
Max-(Un)Satisfiability was proved
Best value of the cost
Optimality of the best cost was proved
Number of variables1000
Total number of constraints90
Number of soft constraints90
Number of constraints which are clauses50
Number of constraints which are cardinality constraints (but not clauses)40
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint20
Maximum length of a constraint25
Top cost 4573
Min constraint cost 1
Max constraint cost 98
Sum of constraints costs 4572
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 26
Number of bits of the biggest sum of numbers5
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Quality of the solution as a function of time


Download the above graph as a PDF file / as an EPS file

Solver Data 

0.00/0.00	c ==============================[ Solver Version ]===============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  pwbo:                     2.02 beta - May, 2012                            |
0.00/0.00	c |                                                                             |
0.00/0.00	c Parsing wbo file format. File HOME/instance-3728796-1338673584.wbo.
0.00/0.00	c Instance file HOME/instance-3728796-1338673584.wbo
0.00/0.00	c File size is 17159 bytes.
0.00/0.00	c Highest Coefficient sum: 26
0.00/0.00	c ============================[ Problem Statistics ]=============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  Parsing time:          0.00         s                                      |
0.00/0.00	c |  Number Variables:      1090                                                |
0.00/0.00	c |  Number Clauses:        50                                                  |
0.00/0.00	c |  Number Cardinality:    0                                                   |
0.00/0.00	c |  Number PB Constraints: 40                                                  |
0.00/0.00	c |                                                                             |
0.00/0.00	c |  Problem Type:          Optimization                                        |
0.00/0.00	c |  Objective Function:    PB Constraint                                       |
0.00/0.00	c |                                                                             |
0.00/0.00	c ===============================================================================
1800.04/900.21	o 669
1800.04/900.21	o 659
1800.04/900.21	o 628
1800.04/900.21	o 352
1800.04/900.21	o 332
1800.04/900.21	o 328
1800.04/900.21	o 326
1800.04/900.21	o 319
1800.04/900.21	o 315
1800.04/900.21	o 308
1800.04/900.21	o 289
1800.04/900.21	o 262
1800.04/900.21	o 250
1800.04/900.21	o 247
1800.04/900.21	o 243
1800.04/900.21	o 232
1800.04/900.21	o 229
1800.04/900.21	o 213
1800.04/900.21	o 212
1800.04/900.21	o 206
1800.04/900.21	o 179
1800.04/900.21	o 178
1800.04/900.21	o 120
1800.04/900.21	o 92
1800.04/900.21	o 80
1800.04/900.21	o 71
1800.04/900.21	o 67
1800.04/900.21	o 55
1800.04/900.21	o 11
1800.04/900.21	o 10
1800.04/900.21	c ===============================================================================
1800.04/900.21	c Wall Clock Time: 900.21 s
1800.04/900.21	c Best Solution: 10
1800.04/900.21	s SATISFIABLE
1800.04/900.21	v -x1 -x2 -x3 -x4 -x5 -x6 x7 -x8 -x9 -x10 -x11 -x12 -x13 -x14 -x15 -x16 -x17 -x18 -x19 -x20 -x21 x22 -x23 -x24 -x25 -x26 -x27 -x28 -x29 -x30 -x31 -x32 -x33 -x34 -x35 -x36 -x37 -x38 -x39 -x40 -x41 -x42 -x43 -x44 -x45 -x46 -x47 -x48 -x49 -x50 -x51 -x52 -x53 x54 -x55 -x56 -x57 -x58 -x59 -x60 -x61 -x62 -x63 -x64 -x65 -x66 -x67 -x68 -x69 -x70 -x71 -x72 -x73 -x74 -x75 -x76 -x77 -x78 x79 -x80 -x81 -x82 -x83 -x84 -x85 -x86 -x87 -x88 -x89 x90 -x91 -x92 -x93 -x94 -x95 -x96 -x97 -x98 -x99 -x100 -x101 -x102 -x103 -x104 -x105 -x106 -x107 -x108 -x109 -x110 -x111 -x112 -x113 -x114 -x115 -x116 -x117 -x118 -x119 x120 -x121 -x122 -x123 -x124 -x125 -x126 -x127 -x128 -x129 x130 -x131 -x132 -x133 -x134 -x135 -x136 -x137 -x138 -x139 -x140 -x141 -x142 -x143 -x144 -x145 -x146 -x147 -x148 -x149 -x150 x151 -x152 -x153 -x154 -x155 -x156 -x157 -x158 -x159 -x160 x161 -x162 -x163 -x164 -x165 -x166 -x167 -x168 -x169 -x170 -x171 -x172 -x173 -x174 -x175 -x176 -x177 -x178 -x179 -x180 -x181 -x182 -x183 -x184 -x185 -x186 -x187 -x188 -x189 -x190 -x191 -x192 -x193 -x194 -x195 -x196 -x197 -x198 x199 -x200 -x201 x202 -x203 -x204 -x205 -x206 -x207 -x208 -x209 -x210 -x211 -x212 -x213 -x214 -x215 -x216 -x217 -x218 -x219 -x220 -x221 -x222 -x223 -x224 -x225 -x226 -x227 -x228 -x229 -x230 -x231 -x232 x233 -x234 -x235 -x236 -x237 -x238 -x239 -x240 -x241 -x242 -x243 -x244 -x245 -x246 -x247 -x248 -x249 -x250 -x251 -x252 -x253 -x254 -x255 -x256 x257 -x258 -x259 -x260 -x261 x262 -x263 -x264 -x265 -x266 -x267 -x268 -x269 -x270 -x271 -x272 -x273 -x274 -x275 -x276 -x277 -x278 -x279 -x280 -x281 x282 -x283 -x284 -x285 -x286 -x287 -x288 -x289 -x290 -x291 -x292 -x293 -x294 -x295 -x296 -x297 -x298 -x299 -x300 -x301 x302 -x303 -x304 -x305 -x306 -x307 -x308 -x309 -x310 -x311 -x312 -x313 -x314 -x315 -x316 -x317 -x318 -x319 -x320 -x321 -x322 -x323 -x324 -x325 x326 -x327 -x328 -x329 -x330 -x331 -x332 -x333 -x334 -x335 -x336 -x337 -x338 -x339 -x340 -x341 -x342 -x343 -x344 -x345 -x346 -x347 -x348 -x349 -x350 -x351 -x352 -x353 -x354 -x355 -x356 -x357 x358 -x359 -x360 -x361 x362 -x363 -x364 -x365 -x366 -x367 -x368 -x369 -x370 -x371 -x372 -x373 -x374 -x375 -x376 -x377 -x378 -x379 -x380 -x381 -x382 -x383 -x384 -x385 -x386 x387 -x388 -x389 -x390 -x391 -x392 -x393 -x394 -x395 -x396 -x397 -x398 -x399 -x400 -x401 -x402 -x403 -x404 -x405 -x406 -x407 -x408 -x409 -x410 -x411 -x412 -x413 -x414 -x415 -x416 -x417 -x418 -x419 x420 -x421 -x422 -x423 -x424 -x425 -x426 -x427 -x428 -x429 -x430 -x431 x432 -x433 -x434 -x435 -x436 -x437 -x438 -x439 -x440 -x441 -x442 -x443 -x444 -x445 -x446 -x447 -x448 -x449 -x450 -x451 -x452 -x453 -x454 x455 -x456 -x457 -x458 -x459 -x460 -x461 -x462 -x463 -x464 -x465 -x466 -x467 -x468 -x469 -x470 -x471 -x472 -x473 -x474 -x475 -x476 x477 -x478 -x479 -x480 x481 -x482 -x483 -x484 -x485 -x486 -x487 -x488 -x489 -x490 -x491 -x492 -x493 -x494 -x495 -x496 -x497 -x498 -x499 -x500 -x501 -x502 -x503 -x504 -x505 -x506 -x507 x508 -x509 -x510 -x511 -x512 -x513 -x514 -x515 -x516 -x517 -x518 -x519 -x520 -x521 -x522 -x523 -x524 -x525 -x526 -x527 -x528 -x529 -x530 -x531 -x532 -x533 -x534 x535 -x536 -x537 -x538 -x539 -x540 -x541 -x542 -x543 -x544 -x545 -x546 -x547 -x548 -x549 -x550 -x551 -x552 -x553 -x554 -x555 x556 -x557 -x558 -x559 -x560 -x561 -x562 -x563 -x564 -x565 -x566 -x567 -x568 -x569 -x570 -x571 -x572 -x573 -x574 x575 -x576 -x577 -x578 -x579 -x580 -x581 x582 -x583 -x584 -x585 -x586 -x587 -x588 -x589 -x590 -x591 -x592 -x593 -x594 -x595 -x596 -x597 -x598 -x599 -x600 -x601 -x602 -x603 -x604 -x605 -x606 -x607 -x608 -x609 -x610 -x611 -x612 x613 -x614 -x615 -x616 -x617 -x618 -x619 -x620 -x621 -x622 -x623 -x624 -x625 -x626 -x627 -x628 -x629 -x630 x631 -x632 -x633 -x634 -x635 -x636 -x637 -x638 -x639 -x640 -x641 -x642 -x643 -x644 -x645 -x646 -x647 -x648 -x649 -x650 -x651 x652 -x653 -x654 -x655 -x656 -x657 -x658 -x659 -x660 -x661 -x662 -x663 -x664 -x665 -x666 -x667 -x668 -x669 -x670 -x671 -x672 -x673 -x674 x675 -x676 -x677 -x678 -x679 -x680 -x681 -x682 x683 -x684 -x685 -x686 -x687 -x688 -x689 -x690 -x691 -x692 -x693 -x694 -x695 -x696 -x697 -x698 -x699 -x700 -x701 -x702 -x703 -x704 -x705 -x706 -x707 -x708 -x709 -x710 -x711 -x712 -x713 -x714 -x715 x716 -x717 -x718 -x719 -x720 -x721 -x722 -x723 -x724 -x725 -x726 -x727 -x728 -x729 -x730 -x731 -x732 -x733 -x734 x735 -x736 -x737 -x738 -x739 -x740 -x741 -x742 -x743 -x744 -x745 -x746 -x747 -x748 x749 -x750 -x751 -x752 -x753 -x754 -x755 -x756 -x757 -x758 -x759 -x760 -x761 -x762 -x763 -x764 x765 -x766 -x767 -x768 -x769 -x770 -x771 -x772 -x773 -x774 -x775 -x776 -x777 -x778 -x779 -x780 -x781 -x782 -x783 -x784 -x785 -x786 -x787 -x788 x789 -x790 -x791 -x792 -x793 -x794 -x795 -x796 -x797 -x798 -x799 -x800 -x801 -x802 -x803 -x804 -x805 -x806 -x807 x808 -x809 -x810 -x811 -x812 -x813 -x814 -x815 -x816 -x817 -x818 -x819 -x820 -x821 -x822 x823 -x824 -x825 -x826 -x827 -x828 -x829 -x830 -x831 -x832 -x833 -x834 -x835 -x836 -x837 -x838 -x839 -x840 -x841 -x842 -x843 -x844 x845 -x846 -x847 -x848 -x849 -x850 -x851 -x852 -x853 -x854 -x855 -x856 -x857 -x858 -x859 -x860 -x861 -x862 -x863 -x864 -x865 -x866 -x867 -x868 -x869 -x870 -x871 -x872 -x873 -x874 -x875 -x876 -x877 x878 -x879 -x880 -x881 -x882 -x883 -x884 -x885 x886 -x887 -x888 -x889 -x890 -x891 -x892 -x893 -x894 -x895 -x896 -x897 -x898 -x899 -x900 -x901 -x902 -x903 -x904 -x905 -x906 -x907 -x908 -x909 -x910 -x911 -x912 -x913 x914 -x915 -x916 -x917 -x918 -x919 -x920 -x921 -x922 -x923 x924 -x925 -x926 -x927 -x928 -x929 -x930 -x931 -x932 -x933 -x934 -x935 -x936 -x937 -x938 -x939 -x940 -x941 -x942 -x943 -x944 -x945 -x946 -x947 -x948 -x949 -x950 -x951 -x952 -x953 -x954 x955 -x956 -x957 -x958 -x959 -x960 -x961 -x962 -x963 -x964 -x965 -x966 -x967 -x968 -x969 -x970 -x971 -x972 -x973 -x974 x975 -x976 -x977 -x978 -x979 -x980 -x981 -x982 -x983 x984 -x985 -x986 -x987 -x988 -x989 -x990 -x991 -x992 -x993 -x994 -x995 -x996 -x997 -x998 -x999 -x1000

Verifier Data 

OK	10

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3728796-1338673584/watcher-3728796-1338673584 -o /tmp/evaluation-result-3728796-1338673584/solver-3728796-1338673584 -C 1800 -W 1900 -M 15500 pwbo2.02 -file-format=wbo -verbosity=0 HOME/instance-3728796-1338673584.wbo 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.68 3.93 3.89 4/180 6560
/proc/meminfo: memFree=28916104/32873844 swapFree=7380/7380
[pid=6560] ppid=6558 vsize=162812 CPUtime=0 cores=0,2,4,6
/proc/6560/stat : 6560 (pwbo2.02) S 6558 6560 5745 0 -1 4202496 1096 0 0 0 0 0 0 0 20 0 3 0 333483233 166719488 1044 33554432000 4194304 5964478 140734126200384 140734126199904 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6560/statm: 40703 1047 188 433 0 40267 0
[pid=6560/tid=6561] ppid=6558 vsize=162812 CPUtime=0 cores=0,2,4,6
/proc/6560/task/6561/stat : 6561 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 140 0 0 0 0 0 0 0 20 0 3 0 333483233 166719488 1052 33554432000 4194304 5964478 140734126200384 139643682949736 5136420 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=6560/tid=6562] ppid=6558 vsize=162812 CPUtime=0 cores=0,2,4,6
/proc/6560/task/6562/stat : 6562 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 76 0 0 0 0 0 0 0 20 0 3 0 333483233 166719488 1053 33554432000 4194304 5964478 140734126200384 139643672459512 18446744073699066235 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0

[startup+0.0458069 s]
/proc/loadavg: 3.68 3.93 3.89 4/180 6560
/proc/meminfo: memFree=28916104/32873844 swapFree=7380/7380
[pid=6560] ppid=6558 vsize=162812 CPUtime=0.08 cores=0,2,4,6
/proc/6560/stat : 6560 (pwbo2.02) S 6558 6560 5745 0 -1 4202496 1346 0 0 0 8 0 0 0 20 0 3 0 333483233 166719488 1294 33554432000 4194304 5964478 140734126200384 140734126199904 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6560/statm: 40703 1294 198 433 0 40267 0
[pid=6560/tid=6561] ppid=6558 vsize=162812 CPUtime=0.03 cores=0,2,4,6
/proc/6560/task/6561/stat : 6561 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 290 0 0 0 3 0 0 0 20 0 3 0 333483233 166719488 1294 33554432000 4194304 5964478 140734126200384 139643682949112 5136420 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=6560/tid=6562] ppid=6558 vsize=162812 CPUtime=0.03 cores=0,2,4,6
/proc/6560/task/6562/stat : 6562 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 168 0 0 0 3 0 0 0 20 0 3 0 333483233 166719488 1294 33554432000 4194304 5964478 140734126200384 139643672459560 4381960 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 162812

[startup+0.100254 s]
/proc/loadavg: 3.68 3.93 3.89 4/180 6560
/proc/meminfo: memFree=28916104/32873844 swapFree=7380/7380
[pid=6560] ppid=6558 vsize=162812 CPUtime=0.19 cores=0,2,4,6
/proc/6560/stat : 6560 (pwbo2.02) S 6558 6560 5745 0 -1 4202496 1629 0 0 0 19 0 0 0 20 0 3 0 333483233 166719488 1577 33554432000 4194304 5964478 140734126200384 140734126199904 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6560/statm: 40703 1577 198 433 0 40267 0
[pid=6560/tid=6561] ppid=6558 vsize=162812 CPUtime=0.08 cores=0,2,4,6
/proc/6560/task/6561/stat : 6561 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 436 0 0 0 8 0 0 0 20 0 3 0 333483233 166719488 1577 33554432000 4194304 5964478 140734126200384 139643682949112 5136420 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=6560/tid=6562] ppid=6558 vsize=162812 CPUtime=0.08 cores=0,2,4,6
/proc/6560/task/6562/stat : 6562 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 305 0 0 0 8 0 0 0 20 0 3 0 333483233 166719488 1577 33554432000 4194304 5964478 140734126200384 139643672459576 4400476 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 162812

[startup+0.300217 s]
/proc/loadavg: 3.68 3.93 3.89 4/180 6560
/proc/meminfo: memFree=28916104/32873844 swapFree=7380/7380
[pid=6560] ppid=6558 vsize=162812 CPUtime=0.58 cores=0,2,4,6
/proc/6560/stat : 6560 (pwbo2.02) S 6558 6560 5745 0 -1 4202496 2553 0 0 0 58 0 0 0 20 0 3 0 333483233 166719488 2501 33554432000 4194304 5964478 140734126200384 140734126199904 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6560/statm: 40703 2501 198 433 0 40267 0
[pid=6560/tid=6561] ppid=6558 vsize=162812 CPUtime=0.28 cores=0,2,4,6
/proc/6560/task/6561/stat : 6561 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 913 0 0 0 28 0 0 0 20 0 3 0 333483233 166719488 2501 33554432000 4194304 5964478 140734126200384 139643682949112 5136420 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=6560/tid=6562] ppid=6558 vsize=162812 CPUtime=0.28 cores=0,2,4,6
/proc/6560/task/6562/stat : 6562 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 752 0 0 0 28 0 0 0 20 0 3 0 333483233 166719488 2501 33554432000 4194304 5964478 140734126200384 139643672459384 4400087 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 0.58
Current children cumulated vsize (KiB) 162812

[startup+0.700238 s]
/proc/loadavg: 3.68 3.93 3.89 4/180 6560
/proc/meminfo: memFree=28916104/32873844 swapFree=7380/7380
[pid=6560] ppid=6558 vsize=162812 CPUtime=1.38 cores=0,2,4,6
/proc/6560/stat : 6560 (pwbo2.02) S 6558 6560 5745 0 -1 4202496 4030 0 0 0 137 1 0 0 20 0 3 0 333483233 166719488 3978 33554432000 4194304 5964478 140734126200384 140734126199904 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6560/statm: 40703 3978 198 433 0 40267 0
[pid=6560/tid=6561] ppid=6558 vsize=162812 CPUtime=0.68 cores=0,2,4,6
/proc/6560/task/6561/stat : 6561 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 1680 0 0 0 68 0 0 0 20 0 3 0 333483233 166719488 3978 33554432000 4194304 5964478 140734126200384 139643682948920 5136420 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=6560/tid=6562] ppid=6558 vsize=162812 CPUtime=0.68 cores=0,2,4,6
/proc/6560/task/6562/stat : 6562 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 1462 0 0 0 68 0 0 0 20 0 3 0 333483233 166719488 3978 33554432000 4194304 5964478 140734126200384 139643672459384 4400067 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 1.38
Current children cumulated vsize (KiB) 162812

[startup+1.50024 s]
/proc/loadavg: 3.68 3.93 3.89 5/183 6563
/proc/meminfo: memFree=28879848/32873844 swapFree=7380/7380
[pid=6560] ppid=6558 vsize=163100 CPUtime=2.98 cores=0,2,4,6
/proc/6560/stat : 6560 (pwbo2.02) S 6558 6560 5745 0 -1 4202496 6060 0 0 0 296 2 0 0 20 0 3 0 333483233 167014400 6008 33554432000 4194304 5964478 140734126200384 140734126199904 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6560/statm: 40775 6008 198 433 0 40339 0
[pid=6560/tid=6561] ppid=6558 vsize=163100 CPUtime=1.48 cores=0,2,4,6
/proc/6560/task/6561/stat : 6561 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 2749 0 0 0 147 1 0 0 20 0 3 0 333483233 167014400 6008 33554432000 4194304 5964478 140734126200384 139643682948920 4400091 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=6560/tid=6562] ppid=6558 vsize=163100 CPUtime=1.48 cores=0,2,4,6
/proc/6560/task/6562/stat : 6562 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 2423 0 0 0 147 1 0 0 20 0 3 0 333483233 167014400 6008 33554432000 4194304 5964478 140734126200384 139643672459384 18446744073699066235 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 2.98
Current children cumulated vsize (KiB) 163100

[startup+3.10026 s]
/proc/loadavg: 3.68 3.93 3.89 5/183 6563
/proc/meminfo: memFree=28866456/32873844 swapFree=7380/7380
[pid=6560] ppid=6558 vsize=163232 CPUtime=6.19 cores=0,2,4,6
/proc/6560/stat : 6560 (pwbo2.02) S 6558 6560 5745 0 -1 4202496 7157 0 0 0 616 3 0 0 20 0 3 0 333483233 167149568 7105 33554432000 4194304 5964478 140734126200384 140734126199904 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6560/statm: 40808 7105 198 433 0 40372 0
[pid=6560/tid=6561] ppid=6558 vsize=163232 CPUtime=3.09 cores=0,2,4,6
/proc/6560/task/6561/stat : 6561 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 3400 0 0 0 307 2 0 0 20 0 3 0 333483233 167149568 7105 33554432000 4194304 5964478 140734126200384 139643682949112 4400091 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=6560/tid=6562] ppid=6558 vsize=163232 CPUtime=3.08 cores=0,2,4,6
/proc/6560/task/6562/stat : 6562 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 2869 0 0 0 306 2 0 0 20 0 3 0 333483233 167149568 7105 33554432000 4194304 5964478 140734126200384 139643672459576 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 6.19
Current children cumulated vsize (KiB) 163232

[startup+6.3003 s]
/proc/loadavg: 3.70 3.93 3.89 5/183 6563
/proc/meminfo: memFree=28831976/32873844 swapFree=7380/7380
[pid=6560] ppid=6558 vsize=163508 CPUtime=12.58 cores=0,2,4,6
/proc/6560/stat : 6560 (pwbo2.02) S 6558 6560 5745 0 -1 4202496 10319 0 0 0 1250 8 0 0 20 0 3 0 333483233 167432192 10267 33554432000 4194304 5964478 140734126200384 140734126199904 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6560/statm: 40877 10267 198 433 0 40441 0
[pid=6560/tid=6561] ppid=6558 vsize=163508 CPUtime=6.28 cores=0,2,4,6
/proc/6560/task/6561/stat : 6561 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 4743 0 0 0 625 3 0 0 20 0 3 0 333483233 167432192 10267 33554432000 4194304 5964478 140734126200384 139643682948920 4220672 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=6560/tid=6562] ppid=6558 vsize=163508 CPUtime=6.28 cores=0,2,4,6
/proc/6560/task/6562/stat : 6562 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 4688 0 0 0 622 6 0 0 20 0 3 0 333483233 167432192 10267 33554432000 4194304 5964478 140734126200384 139643672459576 5136420 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 12.58
Current children cumulated vsize (KiB) 163508

[startup+12.7003 s]
/proc/loadavg: 3.73 3.94 3.89 5/183 6563
/proc/meminfo: memFree=28784132/32873844 swapFree=7380/7380
[pid=6560] ppid=6558 vsize=163644 CPUtime=25.37 cores=0,2,4,6
/proc/6560/stat : 6560 (pwbo2.02) S 6558 6560 5745 0 -1 4202496 15079 0 0 0 2523 14 0 0 20 0 3 0 333483233 167571456 15027 33554432000 4194304 5964478 140734126200384 140734126199904 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6560/statm: 40911 15027 198 433 0 40475 0
[pid=6560/tid=6561] ppid=6558 vsize=163644 CPUtime=12.68 cores=0,2,4,6
/proc/6560/task/6561/stat : 6561 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 7947 0 0 0 1261 7 0 0 20 0 3 0 333483233 167571456 15027 33554432000 4194304 5964478 140734126200384 139643682948920 4221120 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=6560/tid=6562] ppid=6558 vsize=163644 CPUtime=12.68 cores=0,2,4,6
/proc/6560/task/6562/stat : 6562 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 6244 0 0 0 1259 9 0 0 20 0 3 0 333483233 167571456 15027 33554432000 4194304 5964478 140734126200384 139643672459576 4220672 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 25.37
Current children cumulated vsize (KiB) 163644

[startup+25.5004 s]
/proc/loadavg: 3.79 3.94 3.89 5/183 6563
/proc/meminfo: memFree=28728844/32873844 swapFree=7380/7380
[pid=6560] ppid=6558 vsize=163752 CPUtime=50.97 cores=0,2,4,6
/proc/6560/stat : 6560 (pwbo2.02) S 6558 6560 5745 0 -1 4202496 19143 0 0 0 5077 20 0 0 20 0 3 0 333483233 167682048 19091 33554432000 4194304 5964478 140734126200384 140734126199904 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6560/statm: 40938 19091 198 433 0 40502 0
[pid=6560/tid=6561] ppid=6558 vsize=163752 CPUtime=25.48 cores=0,2,4,6
/proc/6560/task/6561/stat : 6561 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 11808 0 0 0 2535 13 0 0 20 0 3 0 333483233 167682048 19091 33554432000 4194304 5964478 140734126200384 139643682949112 4400087 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=6560/tid=6562] ppid=6558 vsize=163752 CPUtime=25.48 cores=0,2,4,6
/proc/6560/task/6562/stat : 6562 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 6447 0 0 0 2539 9 0 0 20 0 3 0 333483233 167682048 19091 33554432000 4194304 5964478 140734126200384 139643672459384 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 50.97
Current children cumulated vsize (KiB) 163752

[startup+51.1004 s]
/proc/loadavg: 3.86 3.94 3.89 5/183 6563
/proc/meminfo: memFree=28630356/32873844 swapFree=7380/7380
[pid=6560] ppid=6558 vsize=164724 CPUtime=102.17 cores=0,2,4,6
/proc/6560/stat : 6560 (pwbo2.02) S 6558 6560 5745 0 -1 4202496 25583 0 0 0 10187 30 0 0 20 0 3 0 333483233 168677376 25531 33554432000 4194304 5964478 140734126200384 140734126199904 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6560/statm: 41181 25531 198 433 0 40745 0
[pid=6560/tid=6561] ppid=6558 vsize=164724 CPUtime=51.08 cores=0,2,4,6
/proc/6560/task/6561/stat : 6561 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 15231 0 0 0 5091 17 0 0 20 0 3 0 333483233 168677376 25531 33554432000 4194304 5964478 140734126200384 139643682949112 4400076 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=6560/tid=6562] ppid=6558 vsize=164724 CPUtime=51.08 cores=0,2,4,6
/proc/6560/task/6562/stat : 6562 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 9464 0 0 0 5094 14 0 0 20 0 3 0 333483233 168677376 25531 33554432000 4194304 5964478 140734126200384 139643672459576 4221120 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 102.17
Current children cumulated vsize (KiB) 164724

[startup+102.3 s]
/proc/loadavg: 3.94 3.95 3.90 5/183 6563
/proc/meminfo: memFree=28563120/32873844 swapFree=7380/7380
[pid=6560] ppid=6558 vsize=231428 CPUtime=204.57 cores=0,2,4,6
/proc/6560/stat : 6560 (pwbo2.02) S 6558 6560 5745 0 -1 4202496 29964 0 0 0 20417 40 0 0 20 0 3 0 333483233 236982272 29912 33554432000 4194304 5964478 140734126200384 140734126199904 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6560/statm: 57857 29912 198 433 0 57421 0
[pid=6560/tid=6561] ppid=6558 vsize=231428 CPUtime=102.29 cores=0,2,4,6
/proc/6560/task/6561/stat : 6561 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 17891 0 0 0 10207 22 0 0 20 0 3 0 333483233 236982272 29912 33554432000 4194304 5964478 140734126200384 139643682949112 4400542 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=6560/tid=6562] ppid=6558 vsize=231428 CPUtime=102.28 cores=0,2,4,6
/proc/6560/task/6562/stat : 6562 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 11185 0 0 0 10207 21 0 0 20 0 3 0 333483233 236982272 29912 33554432000 4194304 5964478 140734126200384 139643672459576 4400087 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 204.57
Current children cumulated vsize (KiB) 231428

[startup+162.3 s]
/proc/loadavg: 3.98 3.95 3.90 5/183 6563
/proc/meminfo: memFree=28480348/32873844 swapFree=7380/7380
[pid=6560] ppid=6558 vsize=233472 CPUtime=324.51 cores=0,2,4,6
/proc/6560/stat : 6560 (pwbo2.02) S 6558 6560 5745 0 -1 4202496 34105 0 0 0 32402 49 0 0 20 0 3 0 333483233 239075328 34117 33554432000 4194304 5964478 140734126200384 140734126199904 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6560/statm: 58368 34117 198 433 0 57932 0
[pid=6560/tid=6561] ppid=6558 vsize=233472 CPUtime=162.26 cores=0,2,4,6
/proc/6560/task/6561/stat : 6561 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 17926 0 0 0 16203 23 0 0 20 0 3 0 333483233 239075328 34117 33554432000 4194304 5964478 140734126200384 139643682949112 4400067 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=6560/tid=6562] ppid=6558 vsize=233472 CPUtime=162.27 cores=0,2,4,6
/proc/6560/task/6562/stat : 6562 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 15291 0 0 0 16197 30 0 0 20 0 3 0 333483233 239075328 34117 33554432000 4194304 5964478 140734126200384 139643672459400 4400089 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 324.51
Current children cumulated vsize (KiB) 233472

[startup+222.307 s]
/proc/loadavg: 3.99 3.96 3.91 5/184 6567
/proc/meminfo: memFree=28384744/32873844 swapFree=7380/7380
[pid=6560] ppid=6558 vsize=234480 CPUtime=444.48 cores=0,2,4,6
/proc/6560/stat : 6560 (pwbo2.02) S 6558 6560 5745 0 -1 4202496 34362 0 0 0 44399 49 0 0 20 0 3 0 333483233 240107520 36140 33554432000 4194304 5964478 140734126200384 140734126199904 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6560/statm: 58620 36140 198 433 0 58184 0
[pid=6560/tid=6561] ppid=6558 vsize=234480 CPUtime=222.27 cores=0,2,4,6
/proc/6560/task/6561/stat : 6561 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 18102 0 0 0 22202 25 0 0 20 0 3 0 333483233 240107520 36140 33554432000 4194304 5964478 140734126200384 139643682948920 4400091 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=6560/tid=6562] ppid=6558 vsize=234480 CPUtime=222.26 cores=0,2,4,6
/proc/6560/task/6562/stat : 6562 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 15372 0 0 0 22194 32 0 0 20 0 3 0 333483233 240107520 36140 33554432000 4194304 5964478 140734126200384 139643672459384 4400115 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 444.48
Current children cumulated vsize (KiB) 234480

################
# More data... #
################

/proc/6560/task/6561/stat : 6561 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 20841 0 0 0 28197 30 0 0 20 0 3 0 333483233 310829056 42161 33554432000 4194304 5964478 140734126200384 139643682949112 4220672 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=6560/tid=6562] ppid=6558 vsize=303544 CPUtime=282.25 cores=0,2,4,6
/proc/6560/task/6562/stat : 6562 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 18658 0 0 0 28187 38 0 0 20 0 3 0 333483233 310829056 42161 33554432000 4194304 5964478 140734126200384 139643672459576 4221984 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 564.45
Current children cumulated vsize (KiB) 303544

[startup+342.3 s]
/proc/loadavg: 3.99 3.97 3.91 5/183 6567
/proc/meminfo: memFree=28294544/32873844 swapFree=7380/7380
[pid=6560] ppid=6558 vsize=303976 CPUtime=684.43 cores=0,2,4,6
/proc/6560/stat : 6560 (pwbo2.02) S 6558 6560 5745 0 -1 4202496 41522 0 0 0 68381 62 0 0 20 0 3 0 333483233 311271424 43296 33554432000 4194304 5964478 140734126200384 140734126199904 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6560/statm: 75994 43296 198 433 0 75558 0
[pid=6560/tid=6561] ppid=6558 vsize=303976 CPUtime=342.27 cores=0,2,4,6
/proc/6560/task/6561/stat : 6561 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 21886 0 0 0 34193 34 0 0 20 0 3 0 333483233 311271424 43296 33554432000 4194304 5964478 140734126200384 139643682949112 4400097 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=6560/tid=6562] ppid=6558 vsize=303976 CPUtime=342.25 cores=0,2,4,6
/proc/6560/task/6562/stat : 6562 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 18748 0 0 0 34186 39 0 0 20 0 3 0 333483233 311271424 43296 33554432000 4194304 5964478 140734126200384 139643672459336 4400078 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 684.43
Current children cumulated vsize (KiB) 303976

[startup+402.3 s]
/proc/loadavg: 3.99 3.97 3.91 5/184 6578
/proc/meminfo: memFree=28286128/32873844 swapFree=7380/7380
[pid=6560] ppid=6558 vsize=304620 CPUtime=804.41 cores=0,2,4,6
/proc/6560/stat : 6560 (pwbo2.02) S 6558 6560 5745 0 -1 4202496 42951 0 0 0 80375 66 0 0 20 0 3 0 333483233 311930880 44725 33554432000 4194304 5964478 140734126200384 140734126199904 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6560/statm: 76155 44725 198 433 0 75719 0
[pid=6560/tid=6561] ppid=6558 vsize=304620 CPUtime=402.27 cores=0,2,4,6
/proc/6560/task/6561/stat : 6561 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 21887 0 0 0 40192 35 0 0 20 0 3 0 333483233 311930880 44725 33554432000 4194304 5964478 140734126200384 139643682949112 4400115 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=6560/tid=6562] ppid=6558 vsize=304620 CPUtime=402.25 cores=0,2,4,6
/proc/6560/task/6562/stat : 6562 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 20176 0 0 0 40181 44 0 0 20 0 3 0 333483233 311930880 44725 33554432000 4194304 5964478 140734126200384 139643672459576 4400016 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 804.41
Current children cumulated vsize (KiB) 304620

[startup+462.3 s]
/proc/loadavg: 3.99 3.97 3.91 5/183 6578
/proc/meminfo: memFree=28210008/32873844 swapFree=7380/7380
[pid=6560] ppid=6558 vsize=304908 CPUtime=924.39 cores=0,2,4,6
/proc/6560/stat : 6560 (pwbo2.02) S 6558 6560 5745 0 -1 4202496 44006 0 0 0 92372 67 0 0 20 0 3 0 333483233 312225792 45780 33554432000 4194304 5964478 140734126200384 140734126199904 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6560/statm: 76227 45780 198 433 0 75791 0
[pid=6560/tid=6561] ppid=6558 vsize=304908 CPUtime=462.28 cores=0,2,4,6
/proc/6560/task/6561/stat : 6561 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 22276 0 0 0 46192 36 0 0 20 0 3 0 333483233 312225792 45780 33554432000 4194304 5964478 140734126200384 139643682949112 4400091 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=6560/tid=6562] ppid=6558 vsize=304908 CPUtime=462.25 cores=0,2,4,6
/proc/6560/task/6562/stat : 6562 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 20842 0 0 0 46179 46 0 0 20 0 3 0 333483233 312225792 45780 33554432000 4194304 5964478 140734126200384 139643672459576 4400087 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 924.39
Current children cumulated vsize (KiB) 304908

[startup+522.3 s]
/proc/loadavg: 4.04 3.99 3.91 5/183 6579
/proc/meminfo: memFree=28115388/32873844 swapFree=7380/7380
[pid=6560] ppid=6558 vsize=305580 CPUtime=1044.37 cores=0,2,4,6
/proc/6560/stat : 6560 (pwbo2.02) S 6558 6560 5745 0 -1 4202496 47619 0 0 0 104364 73 0 0 20 0 3 0 333483233 312913920 49590 33554432000 4194304 5964478 140734126200384 140734126199904 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6560/statm: 76395 49590 198 433 0 75959 0
[pid=6560/tid=6561] ppid=6558 vsize=305580 CPUtime=522.28 cores=0,2,4,6
/proc/6560/task/6561/stat : 6561 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 25877 0 0 0 52186 42 0 0 20 0 3 0 333483233 312913920 49590 33554432000 4194304 5964478 140734126200384 139643682949112 4220672 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=6560/tid=6562] ppid=6558 vsize=305580 CPUtime=522.24 cores=0,2,4,6
/proc/6560/task/6562/stat : 6562 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 20854 0 0 0 52178 46 0 0 20 0 3 0 333483233 312913920 49590 33554432000 4194304 5964478 140734126200384 139643672459576 4221120 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1044.37
Current children cumulated vsize (KiB) 305580

[startup+582.3 s]
/proc/loadavg: 4.01 3.99 3.91 5/183 6579
/proc/meminfo: memFree=27972384/32873844 swapFree=7380/7380
[pid=6560] ppid=6558 vsize=306372 CPUtime=1164.33 cores=0,2,4,6
/proc/6560/stat : 6560 (pwbo2.02) S 6558 6560 5745 0 -1 4202496 47911 0 0 0 116360 73 0 0 20 0 3 0 333483233 313724928 49888 33554432000 4194304 5964478 140734126200384 140734126199904 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6560/statm: 76593 49888 198 433 0 76157 0
[pid=6560/tid=6561] ppid=6558 vsize=306372 CPUtime=582.26 cores=0,2,4,6
/proc/6560/task/6561/stat : 6561 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 26115 0 0 0 58183 43 0 0 20 0 3 0 333483233 313724928 49888 33554432000 4194304 5964478 140734126200384 139643682948920 4400056 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=6560/tid=6562] ppid=6558 vsize=306372 CPUtime=582.24 cores=0,2,4,6
/proc/6560/task/6562/stat : 6562 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 20908 0 0 0 58177 47 0 0 20 0 3 0 333483233 313724928 49888 33554432000 4194304 5964478 140734126200384 139643672459336 4400091 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1164.33
Current children cumulated vsize (KiB) 306372

[startup+642.3 s]
/proc/loadavg: 4.00 3.99 3.91 5/183 6579
/proc/meminfo: memFree=27895860/32873844 swapFree=7380/7380
[pid=6560] ppid=6558 vsize=307560 CPUtime=1284.31 cores=0,2,4,6
/proc/6560/stat : 6560 (pwbo2.02) S 6558 6560 5745 0 -1 4202496 48152 0 0 0 128358 73 0 0 20 0 3 0 333483233 314941440 50127 33554432000 4194304 5964478 140734126200384 140734126199904 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6560/statm: 76890 50127 198 433 0 76454 0
[pid=6560/tid=6561] ppid=6558 vsize=307560 CPUtime=642.27 cores=0,2,4,6
/proc/6560/task/6561/stat : 6561 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 26353 0 0 0 64183 44 0 0 20 0 3 0 333483233 314941440 50127 33554432000 4194304 5964478 140734126200384 139643682948920 4400069 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=6560/tid=6562] ppid=6558 vsize=307560 CPUtime=642.23 cores=0,2,4,6
/proc/6560/task/6562/stat : 6562 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 20911 0 0 0 64175 48 0 0 20 0 3 0 333483233 314941440 50127 33554432000 4194304 5964478 140734126200384 139643672459336 4400109 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1284.31
Current children cumulated vsize (KiB) 307560

[startup+702.3 s]
/proc/loadavg: 4.00 3.99 3.91 5/183 6579
/proc/meminfo: memFree=27889476/32873844 swapFree=7380/7380
[pid=6560] ppid=6558 vsize=314508 CPUtime=1404.29 cores=0,2,4,6
/proc/6560/stat : 6560 (pwbo2.02) S 6558 6560 5745 0 -1 4202496 49752 0 0 0 140355 74 0 0 20 0 3 0 333483233 322056192 51601 33554432000 4194304 5964478 140734126200384 140734126199904 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6560/statm: 78627 51601 198 433 0 78191 0
[pid=6560/tid=6561] ppid=6558 vsize=314508 CPUtime=702.27 cores=0,2,4,6
/proc/6560/task/6561/stat : 6561 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 27951 0 0 0 70181 46 0 0 20 0 3 0 333483233 322056192 51601 33554432000 4194304 5964478 140734126200384 139643682949160 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=6560/tid=6562] ppid=6558 vsize=314508 CPUtime=702.23 cores=0,2,4,6
/proc/6560/task/6562/stat : 6562 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 20913 0 0 0 70174 49 0 0 20 0 3 0 333483233 322056192 51601 33554432000 4194304 5964478 140734126200384 139643672459336 4400056 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1404.29
Current children cumulated vsize (KiB) 314508

[startup+762.3 s]
/proc/loadavg: 4.06 4.00 3.92 5/183 6579
/proc/meminfo: memFree=27884856/32873844 swapFree=7380/7380
[pid=6560] ppid=6558 vsize=316992 CPUtime=1524.29 cores=0,2,4,6
/proc/6560/stat : 6560 (pwbo2.02) S 6558 6560 5745 0 -1 4202496 50831 0 0 0 152353 76 0 0 20 0 3 0 333483233 324599808 52676 33554432000 4194304 5964478 140734126200384 140734126199904 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6560/statm: 79248 52676 198 433 0 78812 0
[pid=6560/tid=6561] ppid=6558 vsize=316992 CPUtime=762.28 cores=0,2,4,6
/proc/6560/task/6561/stat : 6561 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 27957 0 0 0 76181 47 0 0 20 0 3 0 333483233 324599808 52676 33554432000 4194304 5964478 140734126200384 139643682949160 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=6560/tid=6562] ppid=6558 vsize=316992 CPUtime=762.24 cores=0,2,4,6
/proc/6560/task/6562/stat : 6562 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 21986 0 0 0 76173 51 0 0 20 0 3 0 333483233 324599808 52676 33554432000 4194304 5964478 140734126200384 139643672459576 4400069 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1524.29
Current children cumulated vsize (KiB) 316992

[startup+822.3 s]
/proc/loadavg: 4.02 4.00 3.92 5/184 6583
/proc/meminfo: memFree=27878380/32873844 swapFree=7380/7380
[pid=6560] ppid=6558 vsize=318052 CPUtime=1644.25 cores=0,2,4,6
/proc/6560/stat : 6560 (pwbo2.02) S 6558 6560 5745 0 -1 4202496 52340 0 0 0 164346 79 0 0 20 0 3 0 333483233 325685248 54128 33554432000 4194304 5964478 140734126200384 140734126199904 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6560/statm: 79513 54128 198 433 0 79077 0
[pid=6560/tid=6561] ppid=6558 vsize=318052 CPUtime=822.28 cores=0,2,4,6
/proc/6560/task/6561/stat : 6561 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 27963 0 0 0 82180 48 0 0 20 0 3 0 333483233 325685248 54128 33554432000 4194304 5964478 140734126200384 139643682949160 4400078 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=6560/tid=6562] ppid=6558 vsize=318052 CPUtime=822.23 cores=0,2,4,6
/proc/6560/task/6562/stat : 6562 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 23489 0 0 0 82167 56 0 0 20 0 3 0 333483233 325685248 54128 33554432000 4194304 5964478 140734126200384 139643672459576 4400087 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1644.25
Current children cumulated vsize (KiB) 318052

[startup+882.307 s]
/proc/loadavg: 4.01 4.00 3.92 5/184 6606
/proc/meminfo: memFree=27857248/32873844 swapFree=7380/7380
[pid=6560] ppid=6558 vsize=319256 CPUtime=1764.25 cores=0,2,4,6
/proc/6560/stat : 6560 (pwbo2.02) S 6558 6560 5745 0 -1 4202496 55234 0 0 0 176342 83 0 0 20 0 3 0 333483233 326918144 57022 33554432000 4194304 5964478 140734126200384 140734126199904 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6560/statm: 79814 57022 198 433 0 79378 0
[pid=6560/tid=6561] ppid=6558 vsize=319256 CPUtime=882.29 cores=0,2,4,6
/proc/6560/task/6561/stat : 6561 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 30243 0 0 0 88177 52 0 0 20 0 3 0 333483233 326918144 57022 33554432000 4194304 5964478 140734126200384 139643682949160 4400083 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=6560/tid=6562] ppid=6558 vsize=319256 CPUtime=882.23 cores=0,2,4,6
/proc/6560/task/6562/stat : 6562 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 24103 0 0 0 88165 58 0 0 20 0 3 0 333483233 326918144 57022 33554432000 4194304 5964478 140734126200384 139643672459384 4400056 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1764.25
Current children cumulated vsize (KiB) 319256



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+900.2 s]
/proc/loadavg: 4.00 4.00 3.92 5/184 6606
/proc/meminfo: memFree=27851912/32873844 swapFree=7380/7380
[pid=6560] ppid=6558 vsize=319976 CPUtime=1800.04 cores=0,2,4,6
/proc/6560/stat : 6560 (pwbo2.02) S 6558 6560 5745 0 -1 4202496 56349 0 0 0 179921 83 0 0 20 0 3 0 333483233 327655424 58137 33554432000 4194304 5964478 140734126200384 140734126199904 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6560/statm: 79994 58137 198 433 0 79558 0
[pid=6560/tid=6561] ppid=6558 vsize=319976 CPUtime=900.19 cores=0,2,4,6
/proc/6560/task/6561/stat : 6561 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 31239 0 0 0 89967 52 0 0 20 0 3 0 333483233 327655424 58137 33554432000 4194304 5964478 140734126200384 139643682948920 4400087 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=6560/tid=6562] ppid=6558 vsize=319976 CPUtime=900.13 cores=0,2,4,6
/proc/6560/task/6562/stat : 6562 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 24222 0 0 0 89955 58 0 0 20 0 3 0 333483233 327655424 58137 33554432000 4194304 5964478 140734126200384 139643672459384 4400091 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1800.04
Current children cumulated vsize (KiB) 319976

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+900.2 s]
/proc/loadavg: 4.00 4.00 3.92 5/184 6606
/proc/meminfo: memFree=27851912/32873844 swapFree=7380/7380
[pid=6560] ppid=6558 vsize=319976 CPUtime=1800.04 cores=0,2,4,6
/proc/6560/stat : 6560 (pwbo2.02) S 6558 6560 5745 0 -1 4202496 56349 0 0 0 179921 83 0 0 20 0 3 0 333483233 327655424 58137 33554432000 4194304 5964478 140734126200384 140734126199904 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/6560/statm: 79994 58137 198 433 0 79558 0
[pid=6560/tid=6561] ppid=6558 vsize=319976 CPUtime=900.19 cores=0,2,4,6
/proc/6560/task/6561/stat : 6561 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 31239 0 0 0 89967 52 0 0 20 0 3 0 333483233 327655424 58137 33554432000 4194304 5964478 140734126200384 139643682948920 4400087 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=6560/tid=6562] ppid=6558 vsize=319976 CPUtime=900.13 cores=0,2,4,6
/proc/6560/task/6562/stat : 6562 (pwbo2.02) R 6558 6560 5745 0 -1 4202560 24222 0 0 0 89955 58 0 0 20 0 3 0 333483233 327655424 58137 33554432000 4194304 5964478 140734126200384 139643672459384 4400091 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1800.04
Current children cumulated vsize (KiB) 319976

Child status: 0
Real time (s): 900.216
CPU time (s): 1800.36
CPU user time (s): 1799.23
CPU system time (s): 1.13283
CPU usage (%): 199.992
Max. virtual memory (cumulated for all children) (KiB): 319976

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1799.23
system time used= 1.13283
maximum resident set size= 232556
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 56351
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 200
involuntary context switches= 1891

runsolver used 1.9807 second user time and 4.96524 second system time

The end

Launcher Data 

Begin job on node133 at 2012-06-02 23:46:24
IDJOB=3728796
IDBENCH=79162
IDSOLVER=2316
FILE ID=node133/3728796-1338673584
RUNJOBID= node133-1338669974-6147
PBS_JOBID= 14636490
Free space on /tmp= 70480 MiB

SOLVER NAME= pwbo 2.02
BENCH NAME= PB10/normalized-PB10/SOFT-SMALLINT-LIN/PB06/submitted-PB05/aloul/FPGA_SAT05/normalized-chnl20_25_pb.cnf.cr--soft-0-100-0.wbo
COMMAND LINE= pwbo2.02 -file-format=wbo -verbosity=0 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3728796-1338673584/watcher-3728796-1338673584 -o /tmp/evaluation-result-3728796-1338673584/solver-3728796-1338673584 -C 1800 -W 1900 -M 15500  pwbo2.02 -file-format=wbo -verbosity=0 HOME/instance-3728796-1338673584.wbo

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= cad14126bbe64a82703c3144c4e347cd
RANDOM SEED=1376652223

node133.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.540
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.540
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.540
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.540
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.540
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.540
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.540
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.540
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        28916848 kB
Buffers:          366448 kB
Cached:          2699684 kB
SwapCached:          676 kB
Active:          1345344 kB
Inactive:        1737620 kB
Active(anon):       7868 kB
Inactive(anon):    10240 kB
Active(file):    1337476 kB
Inactive(file):  1727380 kB
Unevictable:        7380 kB
Mlocked:            7380 kB
SwapTotal:      67108856 kB
SwapFree:       67107040 kB
Dirty:             10900 kB
Writeback:             0 kB
AnonPages:         22792 kB
Mapped:            11112 kB
Shmem:                 0 kB
Slab:             732668 kB
SReclaimable:     174608 kB
SUnreclaim:       558060 kB
KernelStack:        1456 kB
PageTables:         4360 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     156516 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346376 kB
VmallocChunk:   34341987860 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70472 MiB
End job on node133 at 2012-06-03 00:01:27