Trace number 3712793

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
pwbo 2.01MSAT (TO) 1800.11 900.323

General information on the benchmark

Name/SOFT-SMALLINT-LIN/PB06/submitted-PB06/roussel/
normalized-pigeon-cardinality-35-30--soft-0-100-0.wbo
MD5SUM169186b65db4bc0391cc8aa4ed776345
Bench CategorySOFT-SMALLINT-LIN (only soft constraints, small integers, linear constraints)
Best result obtained on this benchmarkMOPT
Best cost obtained on this benchmark3
Best CPU time to get the best result obtained on this benchmark0.047992
Max-Satisfiable
Max-(Un)Satisfiability was proved
Best value of the cost
Optimality of the best cost was proved
Number of variables1050
Total number of constraints65
Number of soft constraints65
Number of constraints which are clauses35
Number of constraints which are cardinality constraints (but not clauses)30
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint30
Maximum length of a constraint35
Top cost 3654
Min constraint cost 1
Max constraint cost 99
Sum of constraints costs 3653
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 36
Number of bits of the biggest sum of numbers6
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Quality of the solution as a function of time


Download the above graph as a PDF file / as an EPS file

Solver Data 

0.00/0.00	c ==============================[ Solver Version ]===============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  pwbo:                    2.01 beta - May, 2012                            |
0.00/0.00	c |                                                                             |
0.00/0.00	c Parsing wbo file format. File HOME/instance-3712793-1338202144.wbo.
0.00/0.00	c Instance file HOME/instance-3712793-1338202144.wbo
0.00/0.00	c File size is 17750 bytes.
0.00/0.00	c Highest Coefficient sum: 36
0.00/0.00	c ============================[ Problem Statistics ]=============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  Parsing time:          0.00         s                                      |
0.00/0.00	c |  Number Variables:      1115                                                |
0.00/0.00	c |  Number Clauses:        35                                                  |
0.00/0.00	c |  Number Cardinality:    0                                                   |
0.00/0.00	c |  Number PB Constraints: 30                                                  |
0.00/0.00	c |                                                                             |
0.00/0.00	c |  Problem Type:          Optimization                                        |
0.00/0.00	c |  Objective Function:    PB Constraint                                       |
0.00/0.00	c |                                                                             |
0.00/0.00	c ===============================================================================
1800.11/900.31	o 303
1800.11/900.31	o 280
1800.11/900.31	o 108
1800.11/900.31	o 98
1800.11/900.31	o 69
1800.11/900.31	o 34
1800.11/900.31	o 25
1800.11/900.31	o 23
1800.11/900.31	o 21
1800.11/900.31	o 19
1800.11/900.31	o 17
1800.11/900.31	o 10
1800.11/900.31	o 8
1800.11/900.31	o 7
1800.11/900.31	o 6
1800.11/900.31	o 5
1800.11/900.31	o 3
1800.11/900.31	c ===============================================================================
1800.11/900.31	c Wall Clock Time: 900.31 s
1800.11/900.31	c Best Solution: 3
1800.11/900.31	s SATISFIABLE
1800.11/900.31	v -x1 -x2 -x3 -x4 -x5 -x6 x7 -x8 -x9 -x10 -x11 -x12 -x13 -x14 -x15 -x16 -x17 -x18 -x19 -x20 -x21 -x22 -x23 -x24 -x25 -x26 -x27 -x28 -x29 -x30 -x31 -x32 x33 -x34 -x35 -x36 -x37 -x38 -x39 -x40 -x41 -x42 -x43 -x44 -x45 -x46 -x47 -x48 -x49 -x50 -x51 -x52 -x53 -x54 -x55 -x56 -x57 -x58 -x59 -x60 -x61 -x62 -x63 -x64 -x65 -x66 -x67 -x68 -x69 -x70 -x71 -x72 -x73 -x74 -x75 -x76 -x77 -x78 -x79 -x80 -x81 -x82 -x83 -x84 -x85 -x86 -x87 x88 -x89 -x90 -x91 -x92 -x93 -x94 -x95 -x96 -x97 -x98 -x99 -x100 -x101 -x102 -x103 -x104 -x105 -x106 -x107 -x108 x109 -x110 -x111 -x112 -x113 -x114 -x115 -x116 -x117 -x118 -x119 -x120 -x121 -x122 -x123 -x124 -x125 -x126 -x127 -x128 -x129 -x130 -x131 -x132 -x133 -x134 -x135 -x136 -x137 -x138 -x139 -x140 -x141 -x142 -x143 -x144 -x145 x146 -x147 -x148 -x149 -x150 -x151 -x152 -x153 -x154 -x155 -x156 -x157 -x158 x159 -x160 -x161 -x162 -x163 -x164 -x165 -x166 -x167 -x168 -x169 -x170 -x171 -x172 -x173 -x174 -x175 -x176 -x177 -x178 -x179 -x180 -x181 -x182 -x183 -x184 -x185 -x186 -x187 -x188 -x189 -x190 -x191 -x192 -x193 -x194 -x195 -x196 x197 -x198 -x199 -x200 -x201 -x202 -x203 -x204 -x205 -x206 -x207 -x208 -x209 -x210 -x211 -x212 -x213 -x214 -x215 -x216 -x217 -x218 -x219 -x220 -x221 -x222 -x223 -x224 -x225 -x226 -x227 -x228 -x229 -x230 -x231 -x232 -x233 -x234 -x235 -x236 -x237 -x238 -x239 x240 -x241 -x242 -x243 -x244 -x245 -x246 -x247 -x248 -x249 -x250 -x251 -x252 -x253 -x254 -x255 -x256 x257 -x258 -x259 -x260 -x261 -x262 -x263 -x264 -x265 -x266 -x267 -x268 -x269 -x270 -x271 -x272 -x273 -x274 -x275 -x276 -x277 -x278 -x279 -x280 -x281 -x282 -x283 -x284 -x285 -x286 x287 -x288 -x289 -x290 -x291 -x292 -x293 -x294 -x295 -x296 -x297 -x298 -x299 -x300 -x301 -x302 -x303 -x304 -x305 -x306 -x307 -x308 -x309 -x310 -x311 -x312 -x313 -x314 -x315 -x316 -x317 -x318 -x319 x320 -x321 -x322 -x323 -x324 -x325 -x326 -x327 -x328 -x329 -x330 -x331 -x332 -x333 -x334 -x335 -x336 -x337 -x338 -x339 -x340 -x341 x342 -x343 -x344 -x345 -x346 -x347 -x348 -x349 -x350 -x351 -x352 -x353 -x354 -x355 -x356 -x357 -x358 -x359 -x360 -x361 -x362 -x363 -x364 -x365 -x366 -x367 -x368 -x369 -x370 -x371 -x372 -x373 -x374 -x375 x376 -x377 -x378 -x379 -x380 -x381 -x382 -x383 -x384 -x385 -x386 -x387 -x388 -x389 -x390 -x391 -x392 -x393 -x394 -x395 -x396 -x397 -x398 -x399 -x400 x401 -x402 -x403 -x404 -x405 -x406 -x407 -x408 -x409 -x410 -x411 -x412 -x413 -x414 -x415 -x416 -x417 -x418 -x419 -x420 -x421 -x422 -x423 -x424 -x425 -x426 -x427 -x428 -x429 x430 -x431 -x432 -x433 -x434 -x435 -x436 -x437 -x438 -x439 -x440 -x441 -x442 -x443 -x444 -x445 -x446 -x447 -x448 -x449 -x450 x451 -x452 -x453 -x454 -x455 -x456 -x457 -x458 -x459 -x460 -x461 -x462 -x463 -x464 -x465 -x466 -x467 -x468 -x469 -x470 -x471 -x472 -x473 -x474 -x475 -x476 -x477 -x478 -x479 -x480 -x481 -x482 -x483 -x484 -x485 -x486 -x487 -x488 -x489 -x490 -x491 -x492 -x493 -x494 -x495 -x496 -x497 -x498 -x499 -x500 -x501 -x502 x503 -x504 -x505 -x506 -x507 -x508 -x509 -x510 -x511 -x512 -x513 -x514 -x515 -x516 -x517 -x518 -x519 -x520 -x521 -x522 -x523 -x524 -x525 -x526 x527 -x528 -x529 -x530 -x531 -x532 -x533 -x534 -x535 -x536 -x537 -x538 -x539 -x540 -x541 -x542 -x543 -x544 -x545 -x546 -x547 -x548 -x549 -x550 -x551 -x552 -x553 -x554 -x555 -x556 -x557 -x558 -x559 -x560 x561 -x562 -x563 -x564 -x565 -x566 -x567 -x568 -x569 -x570 -x571 -x572 -x573 -x574 -x575 -x576 -x577 -x578 -x579 -x580 -x581 -x582 -x583 -x584 -x585 -x586 -x587 -x588 -x589 -x590 -x591 -x592 -x593 -x594 x595 -x596 -x597 -x598 -x599 -x600 -x601 -x602 -x603 -x604 -x605 -x606 -x607 -x608 -x609 -x610 -x611 -x612 -x613 -x614 x615 -x616 -x617 -x618 -x619 -x620 -x621 -x622 -x623 -x624 -x625 -x626 -x627 -x628 -x629 -x630 -x631 -x632 -x633 -x634 -x635 -x636 -x637 -x638 -x639 -x640 -x641 -x642 -x643 -x644 -x645 -x646 -x647 x648 -x649 -x650 -x651 -x652 -x653 -x654 -x655 -x656 -x657 -x658 -x659 -x660 -x661 -x662 -x663 -x664 -x665 -x666 -x667 -x668 -x669 -x670 -x671 -x672 -x673 -x674 -x675 -x676 x677 -x678 -x679 -x680 -x681 -x682 -x683 -x684 -x685 -x686 -x687 -x688 -x689 -x690 -x691 x692 -x693 -x694 -x695 -x696 -x697 -x698 -x699 -x700 -x701 -x702 -x703 -x704 -x705 -x706 -x707 -x708 -x709 -x710 -x711 -x712 -x713 -x714 -x715 -x716 -x717 -x718 -x719 -x720 -x721 -x722 -x723 -x724 -x725 -x726 -x727 -x728 -x729 -x730 -x731 -x732 -x733 -x734 -x735 -x736 -x737 -x738 -x739 -x740 -x741 -x742 -x743 x744 -x745 -x746 -x747 -x748 -x749 -x750 -x751 -x752 -x753 -x754 -x755 -x756 -x757 -x758 -x759 -x760 -x761 -x762 -x763 x764 -x765 -x766 -x767 -x768 -x769 -x770 -x771 -x772 -x773 -x774 -x775 -x776 -x777 -x778 -x779 -x780 -x781 -x782 -x783 -x784 -x785 -x786 -x787 -x788 -x789 -x790 -x791 -x792 -x793 -x794 -x795 -x796 x797 -x798 -x799 -x800 -x801 -x802 -x803 -x804 -x805 -x806 -x807 -x808 -x809 -x810 -x811 -x812 -x813 -x814 -x815 -x816 -x817 x818 -x819 -x820 -x821 -x822 -x823 -x824 -x825 -x826 -x827 -x828 -x829 -x830 -x831 -x832 -x833 -x834 -x835 -x836 -x837 -x838 -x839 -x840 -x841 -x842 -x843 -x844 -x845 -x846 -x847 -x848 -x849 -x850 -x851 -x852 -x853 -x854 -x855 -x856 -x857 -x858 -x859 -x860 -x861 -x862 -x863 -x864 -x865 -x866 -x867 -x868 x869 -x870 -x871 -x872 -x873 -x874 -x875 -x876 -x877 -x878 -x879 -x880 -x881 -x882 -x883 -x884 -x885 -x886 -x887 -x888 -x889 -x890 -x891 x892 -x893 -x894 -x895 -x896 -x897 -x898 -x899 -x900 -x901 -x902 -x903 -x904 -x905 -x906 -x907 -x908 -x909 -x910 -x911 -x912 -x913 -x914 -x915 -x916 -x917 -x918 -x919 -x920 -x921 -x922 -x923 -x924 -x925 -x926 x927 -x928 -x929 -x930 -x931 -x932 -x933 -x934 -x935 x936 -x937 -x938 -x939 -x940 -x941 -x942 -x943 -x944 -x945 -x946 -x947 -x948 -x949 -x950 -x951 -x952 -x953 -x954 -x955 -x956 -x957 -x958 -x959 -x960 -x961 -x962 -x963 -x964 x965 -x966 -x967 -x968 -x969 -x970 -x971 -x972 -x973 -x974 -x975 -x976 -x977 -x978 -x979 -x980 -x981 -x982 -x983 -x984 -x985 -x986 -x987 -x988 -x989 -x990 -x991 -x992 -x993 -x994 -x995 -x996 -x997 -x998 -x999 -x1000 -x1001 -x1002 x1003 -x1004 -x1005 -x1006 -x1007 -x1008 -x1009 -x1010 -x1011 -x1012 -x1013 -x1014 -x1015 -x1016 -x1017 -x1018 -x1019 -x1020 -x1021 -x1022 -x1023 x1024 -x1025 -x1026 -x1027 -x1028 -x1029 -x1030 -x1031 -x1032 -x1033 -x1034 -x1035 -x1036 -x1037 -x1038 -x1039 -x1040 -x1041 -x1042 -x1043 -x1044 -x1045 -x1046 -x1047 -x1048 -x1049 -x1050

Verifier Data 

OK	3

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3712793-1338202144/watcher-3712793-1338202144 -o /tmp/evaluation-result-3712793-1338202144/solver-3712793-1338202144 -C 1800 -W 1900 -M 15500 pwbo2.01 -file-format=wbo -verbosity=0 HOME/instance-3712793-1338202144.wbo 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.84 3.97 3.95 5/178 15956
/proc/meminfo: memFree=30328700/32873648 swapFree=7116/7116
[pid=15956] ppid=15954 vsize=5084 CPUtime=0 cores=0,2,4,6
/proc/15956/stat : 15956 (pwbo2.01) R 15954 15956 14514 0 -1 4202496 587 0 0 0 0 0 0 0 20 0 1 0 286339830 5206016 535 33554432000 4194304 5964478 140737164069120 140735832244952 4297148 0 0 4096 16387 0 0 0 17 4 0 0 0 0 0
/proc/15956/statm: 1271 547 170 433 0 835 0

[startup+0.0680179 s]
/proc/loadavg: 3.84 3.97 3.95 5/178 15956
/proc/meminfo: memFree=30328700/32873648 swapFree=7116/7116
[pid=15956] ppid=15954 vsize=162836 CPUtime=0.11 cores=0,2,4,6
/proc/15956/stat : 15956 (pwbo2.01) S 15954 15956 14514 0 -1 4202496 1531 0 0 0 11 0 0 0 20 0 3 0 286339830 166744064 1479 33554432000 4194304 5964478 140737164069120 140737164068640 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/15956/statm: 40709 1479 198 433 0 40273 0
Current children cumulated CPU time (s) 0.11
Current children cumulated vsize (KiB) 162836

[startup+0.100163 s]
/proc/loadavg: 3.84 3.97 3.95 5/178 15956
/proc/meminfo: memFree=30328700/32873648 swapFree=7116/7116
[pid=15956] ppid=15954 vsize=162836 CPUtime=0.18 cores=0,2,4,6
/proc/15956/stat : 15956 (pwbo2.01) S 15954 15956 14514 0 -1 4202496 1703 0 0 0 18 0 0 0 20 0 3 0 286339830 166744064 1651 33554432000 4194304 5964478 140737164069120 140737164068640 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/15956/statm: 40709 1651 198 433 0 40273 0
Current children cumulated CPU time (s) 0.18
Current children cumulated vsize (KiB) 162836

[startup+0.300119 s]
/proc/loadavg: 3.84 3.97 3.95 5/178 15956
/proc/meminfo: memFree=30328700/32873648 swapFree=7116/7116
[pid=15956] ppid=15954 vsize=162836 CPUtime=0.57 cores=0,2,4,6
/proc/15956/stat : 15956 (pwbo2.01) S 15954 15956 14514 0 -1 4202496 2673 0 0 0 57 0 0 0 20 0 3 0 286339830 166744064 2621 33554432000 4194304 5964478 140737164069120 140737164068640 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/15956/statm: 40709 2621 198 433 0 40273 0
Current children cumulated CPU time (s) 0.57
Current children cumulated vsize (KiB) 162836

[startup+0.700121 s]
/proc/loadavg: 3.84 3.97 3.95 5/178 15956
/proc/meminfo: memFree=30328700/32873648 swapFree=7116/7116
[pid=15956] ppid=15954 vsize=162836 CPUtime=1.38 cores=0,2,4,6
/proc/15956/stat : 15956 (pwbo2.01) S 15954 15956 14514 0 -1 4202496 4375 0 0 0 137 1 0 0 20 0 3 0 286339830 166744064 4323 33554432000 4194304 5964478 140737164069120 140737164068640 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/15956/statm: 40709 4323 198 433 0 40273 0
Current children cumulated CPU time (s) 1.38
Current children cumulated vsize (KiB) 162836

[startup+1.50024 s]
/proc/loadavg: 3.84 3.97 3.95 6/181 15959
/proc/meminfo: memFree=30307068/32873648 swapFree=7116/7116
[pid=15956] ppid=15954 vsize=162836 CPUtime=2.98 cores=0,2,4,6
/proc/15956/stat : 15956 (pwbo2.01) S 15954 15956 14514 0 -1 4202496 7162 0 0 0 295 3 0 0 20 0 3 0 286339830 166744064 7110 33554432000 4194304 5964478 140737164069120 140737164068640 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/15956/statm: 40709 7110 198 433 0 40273 0
[pid=15956/tid=15958] ppid=15954 vsize=162836 CPUtime=1.47 cores=0,2,4,6
/proc/15956/task/15958/stat : 15958 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 3305 0 0 0 146 1 0 0 20 0 3 0 286339831 166744064 7110 33554432000 4194304 5964478 140737164069120 139668123539448 4400071 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=15956/tid=15959] ppid=15954 vsize=162836 CPUtime=1.47 cores=0,2,4,6
/proc/15956/task/15959/stat : 15959 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 2963 0 0 0 146 1 0 0 20 0 3 0 286339831 166744064 7110 33554432000 4194304 5964478 140737164069120 139668113049912 5136404 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 2.98
Current children cumulated vsize (KiB) 162836

[startup+3.10023 s]
/proc/loadavg: 3.84 3.97 3.95 5/181 15959
/proc/meminfo: memFree=30295288/32873648 swapFree=7116/7116
[pid=15956] ppid=15954 vsize=163256 CPUtime=6.17 cores=0,2,4,6
/proc/15956/stat : 15956 (pwbo2.01) S 15954 15956 14514 0 -1 4202496 9279 0 0 0 614 3 0 0 20 0 3 0 286339830 167174144 9227 33554432000 4194304 5964478 140737164069120 140737164068640 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/15956/statm: 40814 9228 198 433 0 40378 0
[pid=15956/tid=15958] ppid=15954 vsize=163256 CPUtime=3.07 cores=0,2,4,6
/proc/15956/task/15958/stat : 15958 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 4400 0 0 0 306 1 0 0 20 0 3 0 286339831 167174144 9228 33554432000 4194304 5964478 140737164069120 139668123539448 4409686 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=15956/tid=15959] ppid=15954 vsize=163256 CPUtime=3.08 cores=0,2,4,6
/proc/15956/task/15959/stat : 15959 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 3986 0 0 0 306 2 0 0 20 0 3 0 286339831 167174144 9228 33554432000 4194304 5964478 140737164069120 139668113049960 4400099 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 6.17
Current children cumulated vsize (KiB) 163256

[startup+6.30026 s]
/proc/loadavg: 3.85 3.97 3.95 5/181 15959
/proc/meminfo: memFree=30280284/32873648 swapFree=7116/7116
[pid=15956] ppid=15954 vsize=163400 CPUtime=12.57 cores=0,2,4,6
/proc/15956/stat : 15956 (pwbo2.01) S 15954 15956 14514 0 -1 4202496 12853 0 0 0 1253 4 0 0 20 0 3 0 286339830 167321600 12801 33554432000 4194304 5964478 140737164069120 140737164068640 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/15956/statm: 40850 12801 198 433 0 40414 0
[pid=15956/tid=15958] ppid=15954 vsize=163400 CPUtime=6.26 cores=0,2,4,6
/proc/15956/task/15958/stat : 15958 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 6586 0 0 0 624 2 0 0 20 0 3 0 286339831 167321600 12801 33554432000 4194304 5964478 140737164069120 139668123539448 4382015 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=15956/tid=15959] ppid=15954 vsize=163400 CPUtime=6.27 cores=0,2,4,6
/proc/15956/task/15959/stat : 15959 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 5373 0 0 0 625 2 0 0 20 0 3 0 286339831 167321600 12801 33554432000 4194304 5964478 140737164069120 139668113049720 4221984 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 12.57
Current children cumulated vsize (KiB) 163400

[startup+12.7002 s]
/proc/loadavg: 3.86 3.97 3.95 5/181 15959
/proc/meminfo: memFree=30252484/32873648 swapFree=7116/7116
[pid=15956] ppid=15954 vsize=163668 CPUtime=25.37 cores=0,2,4,6
/proc/15956/stat : 15956 (pwbo2.01) S 15954 15956 14514 0 -1 4202496 18930 0 0 0 2530 7 0 0 20 0 3 0 286339830 167596032 18878 33554432000 4194304 5964478 140737164069120 140737164068640 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/15956/statm: 40917 18878 198 433 0 40481 0
[pid=15956/tid=15958] ppid=15954 vsize=163668 CPUtime=12.64 cores=0,2,4,6
/proc/15956/task/15958/stat : 15958 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 9958 0 0 0 1261 3 0 0 20 0 3 0 286339831 167596032 18878 33554432000 4194304 5964478 140737164069120 139668123539448 4221120 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=15956/tid=15959] ppid=15954 vsize=163668 CPUtime=12.64 cores=0,2,4,6
/proc/15956/task/15959/stat : 15959 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 8078 0 0 0 1261 3 0 0 20 0 3 0 286339831 167596032 18878 33554432000 4194304 5964478 140737164069120 139668113049736 4399904 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 25.37
Current children cumulated vsize (KiB) 163668

[startup+25.5003 s]
/proc/loadavg: 3.89 3.97 3.95 5/181 15959
/proc/meminfo: memFree=30211440/32873648 swapFree=7116/7116
[pid=15956] ppid=15954 vsize=164024 CPUtime=50.96 cores=0,2,4,6
/proc/15956/stat : 15956 (pwbo2.01) S 15954 15956 14514 0 -1 4202496 29455 0 0 0 5085 11 0 0 20 0 3 0 286339830 167960576 29403 33554432000 4194304 5964478 140737164069120 140737164068640 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/15956/statm: 41006 29403 198 433 0 40570 0
[pid=15956/tid=15958] ppid=15954 vsize=164024 CPUtime=25.41 cores=0,2,4,6
/proc/15956/task/15958/stat : 15958 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 14950 0 0 0 2536 5 0 0 20 0 3 0 286339831 167960576 29403 33554432000 4194304 5964478 140737164069120 139668123539256 4220672 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=15956/tid=15959] ppid=15954 vsize=164024 CPUtime=25.41 cores=0,2,4,6
/proc/15956/task/15959/stat : 15959 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 13611 0 0 0 2535 6 0 0 20 0 3 0 286339831 167960576 29403 33554432000 4194304 5964478 140737164069120 139668113049960 4399904 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 50.96
Current children cumulated vsize (KiB) 164024

[startup+51.1003 s]
/proc/loadavg: 3.93 3.97 3.95 5/181 15959
/proc/meminfo: memFree=30160708/32873648 swapFree=7116/7116
[pid=15956] ppid=15954 vsize=295688 CPUtime=102.15 cores=0,2,4,6
/proc/15956/stat : 15956 (pwbo2.01) S 15954 15956 14514 0 -1 4202496 42734 0 0 0 10197 18 0 0 20 0 3 0 286339830 302784512 42682 33554432000 4194304 5964478 140737164069120 140737164068640 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/15956/statm: 73922 42682 198 433 0 73486 0
[pid=15956/tid=15958] ppid=15954 vsize=295688 CPUtime=50.94 cores=0,2,4,6
/proc/15956/task/15958/stat : 15958 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 21880 0 0 0 5086 8 0 0 20 0 3 0 286339831 302784512 42682 33554432000 4194304 5964478 140737164069120 139668123539448 5136404 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=15956/tid=15959] ppid=15954 vsize=295688 CPUtime=50.94 cores=0,2,4,6
/proc/15956/task/15959/stat : 15959 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 19960 0 0 0 5084 10 0 0 20 0 3 0 286339831 302784512 42682 33554432000 4194304 5964478 140737164069120 139668113049720 4400071 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 102.15
Current children cumulated vsize (KiB) 295688

[startup+102.307 s]
/proc/loadavg: 3.97 3.97 3.95 5/182 15962
/proc/meminfo: memFree=30097236/32873648 swapFree=7116/7116
[pid=15956] ppid=15954 vsize=296744 CPUtime=204.55 cores=0,2,4,6
/proc/15956/stat : 15956 (pwbo2.01) S 15954 15956 14514 0 -1 4202496 57632 0 0 0 20425 30 0 0 20 0 3 0 286339830 303865856 57580 33554432000 4194304 5964478 140737164069120 140737164068640 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/15956/statm: 74186 57580 198 433 0 73750 0
[pid=15956/tid=15958] ppid=15954 vsize=296744 CPUtime=102.02 cores=0,2,4,6
/proc/15956/task/15958/stat : 15958 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 30401 0 0 0 10187 15 0 0 20 0 3 0 286339831 303865856 57580 33554432000 4194304 5964478 140737164069120 139668123539448 4400040 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=15956/tid=15959] ppid=15954 vsize=296744 CPUtime=102.01 cores=0,2,4,6
/proc/15956/task/15959/stat : 15959 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 26337 0 0 0 10184 17 0 0 20 0 3 0 286339831 303865856 57580 33554432000 4194304 5964478 140737164069120 139668113049912 4400071 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 204.55
Current children cumulated vsize (KiB) 296744

[startup+162.3 s]
/proc/loadavg: 3.99 3.97 3.95 5/181 15962
/proc/meminfo: memFree=30067360/32873648 swapFree=7116/7116
[pid=15956] ppid=15954 vsize=362744 CPUtime=324.52 cores=0,2,4,6
/proc/15956/stat : 15956 (pwbo2.01) S 15954 15956 14514 0 -1 4202496 65338 0 0 0 32417 35 0 0 20 0 3 0 286339830 371449856 65279 33554432000 4194304 5964478 140737164069120 140737164068640 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/15956/statm: 90686 65279 198 433 0 90250 0
[pid=15956/tid=15958] ppid=15954 vsize=362744 CPUtime=161.86 cores=0,2,4,6
/proc/15956/task/15958/stat : 15958 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 37998 0 0 0 16166 20 0 0 20 0 3 0 286339831 371449856 65279 33554432000 4194304 5964478 140737164069120 139668123539496 4400099 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=15956/tid=15959] ppid=15954 vsize=362744 CPUtime=161.84 cores=0,2,4,6
/proc/15956/task/15959/stat : 15959 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 26446 0 0 0 16166 18 0 0 20 0 3 0 286339831 371449856 65279 33554432000 4194304 5964478 140737164069120 139668113049672 4221120 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 324.52
Current children cumulated vsize (KiB) 362744

[startup+222.3 s]
/proc/loadavg: 4.03 3.99 3.95 5/181 15963
/proc/meminfo: memFree=29953676/32873648 swapFree=7116/7116
[pid=15956] ppid=15954 vsize=366164 CPUtime=444.5 cores=0,2,4,6
/proc/15956/stat : 15956 (pwbo2.01) S 15954 15956 14514 0 -1 4202496 79909 0 0 0 44402 48 0 0 20 0 3 0 286339830 374951936 79847 33554432000 4194304 5964478 140737164069120 140737164068640 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/15956/statm: 91541 79847 198 433 0 91105 0
[pid=15956/tid=15958] ppid=15954 vsize=366164 CPUtime=221.7 cores=0,2,4,6
/proc/15956/task/15958/stat : 15958 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 45552 0 0 0 22144 26 0 0 20 0 3 0 286339831 374951936 79847 33554432000 4194304 5964478 140737164069120 139668123539448 4400062 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=15956/tid=15959] ppid=15954 vsize=366164 CPUtime=221.68 cores=0,2,4,6
/proc/15956/task/15959/stat : 15959 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 33463 0 0 0 22141 27 0 0 20 0 3 0 286339831 374951936 79847 33554432000 4194304 5964478 140737164069120 139668113049720 4400002 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 444.5
Current children cumulated vsize (KiB) 366164

[startup+282.3 s]
/proc/loadavg: 3.91 3.96 3.94 5/181 16004
/proc/meminfo: memFree=30994492/32873648 swapFree=7116/7116
[pid=15956] ppid=15954 vsize=432008 CPUtime=564.46 cores=0,2,4,6
/proc/15956/stat : 15956 (pwbo2.01) S 15954 15956 14514 0 -1 4202496 81033 0 0 0 56397 49 0 0 20 0 3 0 286339830 442376192 81410 33554432000 4194304 5964478 140737164069120 140737164068640 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/15956/statm: 108002 81410 198 433 0 107566 0
[pid=15956/tid=15958] ppid=15954 vsize=432008 CPUtime=281.53 cores=0,2,4,6
/proc/15956/task/15958/stat : 15958 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 45826 0 0 0 28125 28 0 0 20 0 3 0 286339831 442376192 81410 33554432000 4194304 5964478 140737164069120 139668123539448 4382005 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=15956/tid=15959] ppid=15954 vsize=432008 CPUtime=281.52 cores=0,2,4,6
/proc/15956/task/15959/stat : 15959 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 34313 0 0 0 28124 28 0 0 20 0 3 0 286339831 442376192 81410 33554432000 4194304 5964478 140737164069120 139668113049672 4400071 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 564.46
Current children cumulated vsize (KiB) 432008

[startup+342.3 s]
/proc/loadavg: 3.97 3.96 3.94 5/181 16004
/proc/meminfo: memFree=30829848/32873648 swapFree=7116/7116
[pid=15956] ppid=15954 vsize=499920 CPUtime=684.41 cores=0,2,4,6
/proc/15956/stat : 15956 (pwbo2.01) S 15954 15956 14514 0 -1 4202496 100236 0 0 0 68376 65 0 0 20 0 3 0 286339830 511918080 100691 33554432000 4194304 5964478 140737164069120 140737164068640 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/15956/statm: 124980 100691 198 433 0 124544 0

################
# More data... #
################

[pid=15956/tid=15959] ppid=15954 vsize=499920 CPUtime=341.33 cores=0,2,4,6
/proc/15956/task/15959/stat : 15959 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 43309 0 0 0 34095 38 0 0 20 0 3 0 286339831 511918080 100691 33554432000 4194304 5964478 140737164069120 139668113049912 4400081 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 684.41
Current children cumulated vsize (KiB) 499920

[startup+402.3 s]
/proc/loadavg: 3.99 3.97 3.94 5/181 16004
/proc/meminfo: memFree=30791312/32873648 swapFree=7116/7116
[pid=15956] ppid=15954 vsize=501020 CPUtime=804.35 cores=0,2,4,6
/proc/15956/stat : 15956 (pwbo2.01) S 15954 15956 14514 0 -1 4202496 102003 0 0 0 80369 66 0 0 20 0 3 0 286339830 513044480 102580 33554432000 4194304 5964478 140737164069120 140737164068640 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/15956/statm: 125255 102580 198 433 0 124819 0
[pid=15956/tid=15958] ppid=15954 vsize=501020 CPUtime=401.2 cores=0,2,4,6
/proc/15956/task/15958/stat : 15958 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 57715 0 0 0 40082 38 0 0 20 0 3 0 286339831 513044480 102580 33554432000 4194304 5964478 140737164069120 139668123539256 4400087 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=15956/tid=15959] ppid=15954 vsize=501020 CPUtime=401.16 cores=0,2,4,6
/proc/15956/task/15959/stat : 15959 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 43394 0 0 0 40077 39 0 0 20 0 3 0 286339831 513044480 102580 33554432000 4194304 5964478 140737164069120 139668113049736 4400087 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 804.35
Current children cumulated vsize (KiB) 501020

[startup+462.3 s]
/proc/loadavg: 3.99 3.97 3.94 5/181 16004
/proc/meminfo: memFree=30726012/32873648 swapFree=7116/7116
[pid=15956] ppid=15954 vsize=501916 CPUtime=924.3 cores=0,2,4,6
/proc/15956/stat : 15956 (pwbo2.01) S 15954 15956 14514 0 -1 4202496 103627 0 0 0 92363 67 0 0 20 0 3 0 286339830 513961984 105382 33554432000 4194304 5964478 140737164069120 140737164068640 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/15956/statm: 125479 105382 198 433 0 125043 0
[pid=15956/tid=15958] ppid=15954 vsize=501916 CPUtime=461.02 cores=0,2,4,6
/proc/15956/task/15958/stat : 15958 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 59219 0 0 0 46062 40 0 0 20 0 3 0 286339831 513961984 105382 33554432000 4194304 5964478 140737164069120 139668123539256 4400062 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=15956/tid=15959] ppid=15954 vsize=501916 CPUtime=461 cores=0,2,4,6
/proc/15956/task/15959/stat : 15959 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 43514 0 0 0 46059 41 0 0 20 0 3 0 286339831 513961984 105382 33554432000 4194304 5964478 140737164069120 139668113049672 4400087 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 924.3
Current children cumulated vsize (KiB) 501916

[startup+522.3 s]
/proc/loadavg: 3.99 3.97 3.94 5/181 16005
/proc/meminfo: memFree=30614692/32873648 swapFree=7116/7116
[pid=15956] ppid=15954 vsize=571088 CPUtime=1044.28 cores=0,2,4,6
/proc/15956/stat : 15956 (pwbo2.01) S 15954 15956 14514 0 -1 4202496 117421 0 0 0 104349 79 0 0 20 0 3 0 286339830 584794112 119176 33554432000 4194304 5964478 140737164069120 140737164068640 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/15956/statm: 142772 119176 198 433 0 142336 0
[pid=15956/tid=15958] ppid=15954 vsize=571088 CPUtime=520.87 cores=0,2,4,6
/proc/15956/task/15958/stat : 15958 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 67955 0 0 0 52039 48 0 0 20 0 3 0 286339831 584794112 119176 33554432000 4194304 5964478 140737164069120 139668123539448 4400612 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=15956/tid=15959] ppid=15954 vsize=571088 CPUtime=520.85 cores=0,2,4,6
/proc/15956/task/15959/stat : 15959 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 48572 0 0 0 52041 44 0 0 20 0 3 0 286339831 584794112 119176 33554432000 4194304 5964478 140737164069120 139668113049912 4221120 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 1044.28
Current children cumulated vsize (KiB) 571088

[startup+582.3 s]
/proc/loadavg: 3.99 3.97 3.94 5/181 16005
/proc/meminfo: memFree=30580868/32873648 swapFree=7116/7116
[pid=15956] ppid=15954 vsize=573264 CPUtime=1164.26 cores=0,2,4,6
/proc/15956/stat : 15956 (pwbo2.01) S 15954 15956 14514 0 -1 4202496 117840 0 0 0 116347 79 0 0 20 0 3 0 286339830 587022336 119595 33554432000 4194304 5964478 140737164069120 140737164068640 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/15956/statm: 143316 119595 198 433 0 142880 0
[pid=15956/tid=15958] ppid=15954 vsize=573264 CPUtime=580.71 cores=0,2,4,6
/proc/15956/task/15958/stat : 15958 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 68284 0 0 0 58022 49 0 0 20 0 3 0 286339831 587022336 119595 33554432000 4194304 5964478 140737164069120 139668123539496 4400075 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=15956/tid=15959] ppid=15954 vsize=573264 CPUtime=580.69 cores=0,2,4,6
/proc/15956/task/15959/stat : 15959 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 48662 0 0 0 58024 45 0 0 20 0 3 0 286339831 587022336 119595 33554432000 4194304 5964478 140737164069120 139668113049720 4220672 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 1164.26
Current children cumulated vsize (KiB) 573264

[startup+642.3 s]
/proc/loadavg: 3.99 3.97 3.94 5/181 16005
/proc/meminfo: memFree=30524740/32873648 swapFree=7116/7116
[pid=15956] ppid=15954 vsize=643452 CPUtime=1284.25 cores=0,2,4,6
/proc/15956/stat : 15956 (pwbo2.01) S 15954 15956 14514 0 -1 4202496 126444 0 0 0 128338 87 0 0 20 0 3 0 286339830 658894848 128199 33554432000 4194304 5964478 140737164069120 140737164068640 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/15956/statm: 160863 128199 198 433 0 160427 0
[pid=15956/tid=15958] ppid=15954 vsize=643452 CPUtime=640.56 cores=0,2,4,6
/proc/15956/task/15958/stat : 15958 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 73702 0 0 0 64001 55 0 0 20 0 3 0 286339831 658894848 128199 33554432000 4194304 5964478 140737164069120 139668123539448 4221120 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=15956/tid=15959] ppid=15954 vsize=643452 CPUtime=640.55 cores=0,2,4,6
/proc/15956/task/15959/stat : 15959 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 51848 0 0 0 64005 50 0 0 20 0 3 0 286339831 658894848 128199 33554432000 4194304 5964478 140737164069120 139668113049912 4220672 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 1284.25
Current children cumulated vsize (KiB) 643452

[startup+702.3 s]
/proc/loadavg: 4.08 4.00 3.95 5/182 16008
/proc/meminfo: memFree=30492276/32873648 swapFree=7116/7116
[pid=15956] ppid=15954 vsize=579068 CPUtime=1404.22 cores=0,2,4,6
/proc/15956/stat : 15956 (pwbo2.01) S 15954 15956 14514 0 -1 4202496 132059 0 0 0 140330 92 0 0 20 0 3 0 286339830 592965632 133081 33554432000 4194304 5964478 140737164069120 140737164068640 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/15956/statm: 144767 133081 198 433 0 144331 0
[pid=15956/tid=15958] ppid=15954 vsize=579068 CPUtime=700.4 cores=0,2,4,6
/proc/15956/task/15958/stat : 15958 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 79316 0 0 0 69980 60 0 0 20 0 3 0 286339831 592965632 133081 33554432000 4194304 5964478 140737164069120 139668123539448 5136404 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=15956/tid=15959] ppid=15954 vsize=579068 CPUtime=700.38 cores=0,2,4,6
/proc/15956/task/15959/stat : 15959 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 51849 0 0 0 69987 51 0 0 20 0 3 0 286339831 592965632 133081 33554432000 4194304 5964478 140737164069120 139668113049944 4221120 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 1404.22
Current children cumulated vsize (KiB) 579068

[startup+762.307 s]
/proc/loadavg: 4.03 4.00 3.95 5/182 16031
/proc/meminfo: memFree=30455996/32873648 swapFree=7116/7116
[pid=15956] ppid=15954 vsize=710572 CPUtime=1524.18 cores=0,2,4,6
/proc/15956/stat : 15956 (pwbo2.01) S 15954 15956 14514 0 -1 4202496 137880 0 0 0 152323 95 0 0 20 0 3 0 286339830 727625728 138982 33554432000 4194304 5964478 140737164069120 140737164068640 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/15956/statm: 177643 138982 198 433 0 177207 0
[pid=15956/tid=15958] ppid=15954 vsize=710572 CPUtime=760.24 cores=0,2,4,6
/proc/15956/task/15958/stat : 15958 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 84544 0 0 0 75960 64 0 0 20 0 3 0 286339831 727625728 138982 33554432000 4194304 5964478 140737164069120 139668123539448 4400075 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=15956/tid=15959] ppid=15954 vsize=710572 CPUtime=760.21 cores=0,2,4,6
/proc/15956/task/15959/stat : 15959 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 52442 0 0 0 75969 52 0 0 20 0 3 0 286339831 727625728 138982 33554432000 4194304 5964478 140737164069120 139668113049720 4221120 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 1524.18
Current children cumulated vsize (KiB) 710572

[startup+822.3 s]
/proc/loadavg: 4.05 4.01 3.96 5/181 16032
/proc/meminfo: memFree=30361516/32873648 swapFree=7116/7116
[pid=15956] ppid=15954 vsize=715684 CPUtime=1644.12 cores=0,2,4,6
/proc/15956/stat : 15956 (pwbo2.01) S 15954 15956 14514 0 -1 4202496 157912 0 0 0 164303 109 0 0 20 0 3 0 286339830 732860416 159153 33554432000 4194304 5964478 140737164069120 140737164068640 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/15956/statm: 178921 159153 198 433 0 178485 0
[pid=15956/tid=15958] ppid=15954 vsize=715684 CPUtime=820.05 cores=0,2,4,6
/proc/15956/task/15958/stat : 15958 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 90139 0 0 0 81936 69 0 0 20 0 3 0 286339831 732860416 159153 33554432000 4194304 5964478 140737164069120 139668123539448 4221984 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=15956/tid=15959] ppid=15954 vsize=715684 CPUtime=820.05 cores=0,2,4,6
/proc/15956/task/15959/stat : 15959 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 66879 0 0 0 81941 64 0 0 20 0 3 0 286339831 732860416 159153 33554432000 4194304 5964478 140737164069120 139668113049912 4400075 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 1644.12
Current children cumulated vsize (KiB) 715684

[startup+882.3 s]
/proc/loadavg: 4.02 4.01 3.96 5/181 16032
/proc/meminfo: memFree=30328580/32873648 swapFree=7116/7116
[pid=15956] ppid=15954 vsize=719212 CPUtime=1764.1 cores=0,2,4,6
/proc/15956/stat : 15956 (pwbo2.01) S 15954 15956 14514 0 -1 4202496 161458 0 0 0 176297 113 0 0 20 0 3 0 286339830 736473088 162698 33554432000 4194304 5964478 140737164069120 140737164068640 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/15956/statm: 179803 162698 198 433 0 179367 0
[pid=15956/tid=15958] ppid=15954 vsize=719212 CPUtime=879.9 cores=0,2,4,6
/proc/15956/task/15958/stat : 15958 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 92341 0 0 0 87918 72 0 0 20 0 3 0 286339831 736473088 162698 33554432000 4194304 5964478 140737164069120 139668123539448 4400075 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=15956/tid=15959] ppid=15954 vsize=719212 CPUtime=879.9 cores=0,2,4,6
/proc/15956/task/15959/stat : 15959 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 68223 0 0 0 87924 66 0 0 20 0 3 0 286339831 736473088 162698 33554432000 4194304 5964478 140737164069120 139668113049912 4400051 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 1764.1
Current children cumulated vsize (KiB) 719212



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+900.3 s]
/proc/loadavg: 4.01 4.01 3.96 5/181 16032
/proc/meminfo: memFree=30323496/32873648 swapFree=7116/7116
[pid=15956] ppid=15954 vsize=719212 CPUtime=1800.11 cores=0,2,4,6
/proc/15956/stat : 15956 (pwbo2.01) S 15954 15956 14514 0 -1 4202496 161473 0 0 0 179898 113 0 0 20 0 3 0 286339830 736473088 162713 33554432000 4194304 5964478 140737164069120 140737164068640 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/15956/statm: 179803 162713 198 433 0 179367 0
[pid=15956/tid=15958] ppid=15954 vsize=719212 CPUtime=897.86 cores=0,2,4,6
/proc/15956/task/15958/stat : 15958 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 92355 0 0 0 89714 72 0 0 20 0 3 0 286339831 736473088 162713 33554432000 4194304 5964478 140737164069120 139668123539448 4381686 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=15956/tid=15959] ppid=15954 vsize=719212 CPUtime=897.86 cores=0,2,4,6
/proc/15956/task/15959/stat : 15959 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 68224 0 0 0 89720 66 0 0 20 0 3 0 286339831 736473088 162713 33554432000 4194304 5964478 140737164069120 139668113049912 4400075 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 1800.11
Current children cumulated vsize (KiB) 719212

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+900.3 s]
/proc/loadavg: 4.01 4.01 3.96 5/181 16032
/proc/meminfo: memFree=30323496/32873648 swapFree=7116/7116
[pid=15956] ppid=15954 vsize=719212 CPUtime=1800.11 cores=0,2,4,6
/proc/15956/stat : 15956 (pwbo2.01) S 15954 15956 14514 0 -1 4202496 161473 0 0 0 179898 113 0 0 20 0 3 0 286339830 736473088 162713 33554432000 4194304 5964478 140737164069120 140737164068640 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/15956/statm: 179803 162713 198 433 0 179367 0
[pid=15956/tid=15958] ppid=15954 vsize=719212 CPUtime=897.86 cores=0,2,4,6
/proc/15956/task/15958/stat : 15958 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 92355 0 0 0 89714 72 0 0 20 0 3 0 286339831 736473088 162713 33554432000 4194304 5964478 140737164069120 139668123539448 4381686 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=15956/tid=15959] ppid=15954 vsize=719212 CPUtime=897.86 cores=0,2,4,6
/proc/15956/task/15959/stat : 15959 (pwbo2.01) R 15954 15956 14514 0 -1 4202560 68224 0 0 0 89720 66 0 0 20 0 3 0 286339831 736473088 162713 33554432000 4194304 5964478 140737164069120 139668113049912 4400075 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 1800.11
Current children cumulated vsize (KiB) 719212

Child status: 0

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 15956 and gives
#  childrusage.ru_utime.tv_sec=1794
#  childrusage.ru_utime.tv_usec=350217
#  childrusage.ru_stime.tv_sec=1
#  childrusage.ru_stime.tv_usec=409785
# CPU time returned by wait4() is 1795.76
# while last known CPU time is 1800.11
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 900.323
CPU time (s): 1800.11
CPU user time (s): 1798.98
CPU system time (s): 1.13
CPU usage (%): 199.941
Max. virtual memory (cumulated for all children) (KiB): 719212

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1794.35
system time used= 1.40978
maximum resident set size= 650860
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 161475
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 532
involuntary context switches= 2062

runsolver used 1.85272 second user time and 4.98424 second system time

The end

Launcher Data 

Begin job on node123 at 2012-05-28 12:49:04
IDJOB=3712793
IDBENCH=79184
IDSOLVER=2292
FILE ID=node123/3712793-1338202144
RUNJOBID= node123-1338195143-14530
PBS_JOBID= 14624606
Free space on /tmp= 71676 MiB

SOLVER NAME= pwbo 2.01
BENCH NAME= PB10/normalized-PB10/SOFT-SMALLINT-LIN/PB06/submitted-PB06/roussel/normalized-pigeon-cardinality-35-30--soft-0-100-0.wbo
COMMAND LINE= pwbo2.01 -file-format=wbo -verbosity=0 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3712793-1338202144/watcher-3712793-1338202144 -o /tmp/evaluation-result-3712793-1338202144/solver-3712793-1338202144 -C 1800 -W 1900 -M 15500  pwbo2.01 -file-format=wbo -verbosity=0 HOME/instance-3712793-1338202144.wbo

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 169186b65db4bc0391cc8aa4ed776345
RANDOM SEED=111834634

node123.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.43
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873648 kB
MemFree:        30328964 kB
Buffers:          310232 kB
Cached:           277044 kB
SwapCached:          956 kB
Active:          1402532 kB
Inactive:         303340 kB
Active(anon):    1112092 kB
Inactive(anon):     8336 kB
Active(file):     290440 kB
Inactive(file):   295004 kB
Unevictable:        7116 kB
Mlocked:            7116 kB
SwapTotal:      67108856 kB
SwapFree:       67102956 kB
Dirty:              5504 kB
Writeback:             0 kB
AnonPages:       1124988 kB
Mapped:            11056 kB
Shmem:                 0 kB
Slab:             698952 kB
SReclaimable:     143356 kB
SUnreclaim:       555596 kB
KernelStack:        1472 kB
PageTables:         5796 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545680 kB
Committed_AS:     163664 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346400 kB
VmallocChunk:   34341988052 kB
HardwareCorrupted:     0 kB
AnonHugePages:    348160 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7680 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 71672 MiB
End job on node123 at 2012-05-28 13:04:06