Trace number 3726309

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
pwbo 2.02SAT (TO)127 1800.12 900.316

General information on the benchmark

Name/OPT-SMALLINT-LIN/heinz/
normalized-neos-538867.opb
MD5SUM3b75c8039d6468258a1aa0343ce1fddb
Bench CategoryOPT-SMALLINT-LIN (optimisation, small integers, linear constraints)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark122
Best CPU time to get the best result obtained on this benchmark58.879
Has Objective FunctionYES
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables792
Total number of constraints1170
Number of constraints which are clauses1143
Number of constraints which are cardinality constraints (but not clauses)27
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint3
Maximum length of a constraint18
Number of terms in the objective function 666
Biggest coefficient in the objective function 80
Number of bits for the biggest coefficient in the objective function 7
Sum of the numbers in the objective function 18684
Number of bits of the sum of numbers in the objective function 15
Biggest number in a constraint 80
Number of bits of the biggest number in a constraint 7
Biggest sum of numbers in a constraint 18684
Number of bits of the biggest sum of numbers15
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Quality of the solution as a function of time


Download the above graph as a PDF file / as an EPS file

Solver Data 

0.00/0.00	c ==============================[ Solver Version ]===============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  pwbo:                     2.02 beta - May, 2012                            |
0.00/0.00	c |                                                                             |
0.00/0.00	c Parsing opb file format. File HOME/instance-3726309-1338658652.opb.
0.00/0.00	c Instance file HOME/instance-3726309-1338658652.opb
0.00/0.00	c File size is 46134 bytes.
0.00/0.00	c Cost Offset: 0
0.00/0.00	c Highest Coefficient sum: 21
0.00/0.00	c ============================[ Problem Statistics ]=============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  Parsing time:          0.01         s                                      |
0.00/0.00	c |  Number Variables:      792                                                 |
0.00/0.00	c |  Number Clauses:        1161                                                |
0.00/0.00	c |  Number Cardinality:    27                                                  |
0.00/0.00	c |  Number PB Constraints: 0                                                   |
0.00/0.00	c |                                                                             |
0.00/0.00	c |  Problem Type:          Optimization                                        |
0.00/0.00	c |  Objective Function:    PB Constraint                                       |
0.00/0.00	c |                                                                             |
0.00/0.00	c ===============================================================================
1800.12/900.31	o 477
1800.12/900.31	o 472
1800.12/900.31	o 467
1800.12/900.31	o 452
1800.12/900.31	o 417
1800.12/900.31	o 407
1800.12/900.31	o 387
1800.12/900.31	o 372
1800.12/900.31	o 362
1800.12/900.31	o 352
1800.12/900.31	o 342
1800.12/900.31	o 337
1800.12/900.31	o 317
1800.12/900.31	o 307
1800.12/900.31	o 292
1800.12/900.31	o 267
1800.12/900.31	o 247
1800.12/900.31	o 242
1800.12/900.31	o 232
1800.12/900.31	o 222
1800.12/900.31	o 217
1800.12/900.31	o 212
1800.12/900.31	o 207
1800.12/900.31	o 197
1800.12/900.31	o 187
1800.12/900.31	o 182
1800.12/900.31	o 177
1800.12/900.31	o 167
1800.12/900.31	o 142
1800.12/900.31	o 132
1800.12/900.31	o 127
1800.12/900.31	c ===============================================================================
1800.12/900.31	c Wall Clock Time: 900.31 s
1800.12/900.31	c Best Solution: 127
1800.12/900.31	s SATISFIABLE
1800.12/900.31	v -x1 -x2 x3 -x4 -x5 -x6 -x7 -x8 -x9 -x10 -x11 -x12 -x13 -x14 -x15 -x16 -x17 -x18 -x19 -x20 -x21 -x22 -x23 -x24 -x25 -x26 -x27 -x28 -x29 -x30 -x31 -x32 -x33 -x34 -x35 -x36 -x37 -x38 -x39 -x40 -x41 -x42 -x43 -x44 -x45 -x46 -x47 -x48 x49 -x50 -x51 -x52 -x53 -x54 -x55 -x56 -x57 -x58 -x59 -x60 -x61 -x62 -x63 -x64 -x65 -x66 -x67 -x68 -x69 -x70 -x71 -x72 -x73 -x74 -x75 -x76 -x77 -x78 -x79 -x80 -x81 -x82 -x83 -x84 -x85 -x86 x87 -x88 -x89 -x90 -x91 -x92 -x93 -x94 -x95 -x96 -x97 -x98 -x99 -x100 -x101 -x102 -x103 -x104 -x105 -x106 -x107 -x108 -x109 -x110 -x111 -x112 -x113 -x114 -x115 -x116 -x117 -x118 -x119 -x120 -x121 -x122 -x123 -x124 -x125 -x126 -x127 -x128 -x129 -x130 -x131 -x132 -x133 -x134 -x135 -x136 -x137 -x138 -x139 -x140 -x141 -x142 -x143 x144 -x145 -x146 -x147 x148 -x149 -x150 -x151 -x152 -x153 -x154 -x155 -x156 -x157 -x158 -x159 -x160 -x161 -x162 -x163 -x164 -x165 -x166 -x167 -x168 -x169 -x170 -x171 -x172 -x173 -x174 -x175 -x176 -x177 -x178 -x179 -x180 -x181 -x182 -x183 -x184 -x185 -x186 -x187 -x188 -x189 -x190 -x191 -x192 -x193 -x194 -x195 -x196 -x197 -x198 x199 -x200 -x201 -x202 -x203 -x204 -x205 -x206 -x207 -x208 -x209 -x210 -x211 -x212 -x213 -x214 -x215 -x216 -x217 -x218 -x219 -x220 -x221 -x222 -x223 -x224 -x225 -x226 -x227 -x228 -x229 -x230 -x231 -x232 -x233 -x234 -x235 -x236 -x237 -x238 -x239 -x240 -x241 -x242 -x243 -x244 -x245 -x246 -x247 -x248 -x249 -x250 -x251 -x252 -x253 -x254 -x255 -x256 -x257 -x258 -x259 -x260 -x261 -x262 -x263 -x264 -x265 -x266 -x267 -x268 -x269 -x270 -x271 -x272 -x273 -x274 -x275 -x276 -x277 -x278 -x279 -x280 -x281 -x282 -x283 -x284 -x285 -x286 -x287 -x288 -x289 -x290 -x291 -x292 -x293 -x294 -x295 -x296 -x297 -x298 -x299 -x300 -x301 -x302 -x303 -x304 -x305 -x306 -x307 -x308 -x309 -x310 -x311 -x312 -x313 -x314 -x315 -x316 -x317 -x318 -x319 -x320 -x321 -x322 -x323 -x324 -x325 -x326 -x327 -x328 -x329 -x330 -x331 -x332 -x333 -x334 -x335 -x336 -x337 -x338 -x339 -x340 -x341 -x342 -x343 -x344 -x345 -x346 -x347 -x348 -x349 -x350 -x351 -x352 -x353 -x354 -x355 -x356 -x357 -x358 -x359 -x360 -x361 -x362 -x363 -x364 -x365 -x366 -x367 -x368 -x369 -x370 -x371 -x372 -x373 -x374 -x375 -x376 -x377 -x378 -x379 -x380 -x381 -x382 -x383 -x384 -x385 -x386 -x387 -x388 -x389 -x390 -x391 -x392 -x393 -x394 -x395 -x396 -x397 -x398 -x399 -x400 -x401 -x402 -x403 -x404 -x405 -x406 -x407 -x408 -x409 -x410 -x411 -x412 -x413 -x414 -x415 -x416 -x417 -x418 -x419 -x420 -x421 -x422 -x423 -x424 -x425 -x426 -x427 -x428 -x429 -x430 -x431 -x432 -x433 -x434 -x435 -x436 -x437 -x438 -x439 -x440 -x441 -x442 -x443 -x444 -x445 -x446 -x447 -x448 -x449 -x450 -x451 -x452 -x453 -x454 -x455 -x456 -x457 -x458 -x459 -x460 -x461 -x462 -x463 -x464 -x465 -x466 -x467 -x468 -x469 -x470 -x471 -x472 -x473 -x474 -x475 -x476 -x477 -x478 -x479 -x480 -x481 -x482 -x483 -x484 -x485 -x486 -x487 -x488 -x489 -x490 -x491 -x492 -x493 -x494 -x495 -x496 -x497 -x498 -x499 -x500 -x501 -x502 -x503 -x504 x505 -x506 -x507 -x508 -x509 -x510 -x511 -x512 -x513 -x514 -x515 -x516 x517 -x518 -x519 -x520 -x521 -x522 -x523 -x524 -x525 x526 -x527 -x528 -x529 -x530 -x531 -x532 -x533 -x534 -x535 -x536 -x537 x538 -x539 -x540 -x541 -x542 x543 -x544 -x545 -x546 -x547 -x548 -x549 -x550 -x551 x552 -x553 -x554 -x555 -x556 -x557 -x558 -x559 x560 -x561 -x562 -x563 -x564 -x565 -x566 -x567 -x568 x569 -x570 -x571 -x572 -x573 -x574 -x575 -x576 -x577 -x578 -x579 -x580 x581 -x582 -x583 -x584 -x585 -x586 -x587 -x588 -x589 x590 -x591 -x592 -x593 -x594 -x595 -x596 -x597 -x598 -x599 -x600 -x601 x602 -x603 -x604 -x605 -x606 -x607 -x608 -x609 -x610 x611 -x612 -x613 -x614 -x615 -x616 -x617 x618 -x619 -x620 -x621 -x622 -x623 -x624 -x625 -x626 x627 -x628 -x629 -x630 -x631 -x632 -x633 -x634 -x635 x636 -x637 -x638 -x639 -x640 -x641 -x642 -x643 -x644 -x645 -x646 -x647 x648 -x649 -x650 -x651 -x652 -x653 -x654 -x655 -x656 x657 -x658 -x659 -x660 -x661 -x662 -x663 -x664 -x665 x666 x667 -x668 -x669 -x670 -x671 -x672 -x673 -x674 -x675 x676 -x677 -x678 -x679 -x680 -x681 -x682 -x683 -x684 -x685 -x686 -x687 x688 -x689 -x690 -x691 -x692 -x693 -x694 -x695 -x696 x697 -x698 -x699 -x700 -x701 -x702 -x703 x704 -x705 -x706 -x707 -x708 -x709 -x710 -x711 -x712 -x713 -x714 -x715 x716 -x717 -x718 -x719 -x720 -x721 -x722 -x723 -x724 -x725 -x726 -x727 x728 -x729 -x730 -x731 -x732 -x733 -x734 -x735 x736 -x737 -x738 -x739 -x740 -x741 -x742 -x743 -x744 -x745 -x746 -x747 -x748 -x749 -x750 -x751 -x752 -x753 -x754 -x755 -x756 -x757 -x758 -x759 -x760 -x761 -x762 -x763 -x764 -x765 -x766 -x767 -x768 -x769 -x770 -x771 -x772 -x773 -x774 -x775 -x776 -x777 -x778 -x779 -x780 -x781 -x782 -x783 -x784 -x785 -x786 -x787 -x788 -x789 -x790 -x791 -x792

Verifier Data 

OK	127

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3726309-1338658652/watcher-3726309-1338658652 -o /tmp/evaluation-result-3726309-1338658652/solver-3726309-1338658652 -C 1800 -W 1900 -M 15500 pwbo2.02 -file-format=opb -verbosity=0 HOME/instance-3726309-1338658652.opb 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.92 3.99 3.96 2/174 12959
/proc/meminfo: memFree=29130296/32873844 swapFree=7120/7120
[pid=12959] ppid=12957 vsize=4700 CPUtime=0 cores=0,2,4,6
/proc/12959/stat : 12959 (pwbo2.02) R 12957 12959 10378 0 -1 4202496 426 0 0 0 0 0 0 0 20 0 1 0 331990090 4812800 366 33554432000 4194304 5964478 140737053592416 140736670365480 4297174 0 0 4096 16387 0 0 0 17 0 0 0 0 0 0
/proc/12959/statm: 1175 378 168 433 0 739 0

[startup+0.028742 s]
/proc/loadavg: 3.92 3.99 3.96 2/174 12959
/proc/meminfo: memFree=29130296/32873844 swapFree=7120/7120
[pid=12959] ppid=12957 vsize=162768 CPUtime=0.04 cores=0,2,4,6
/proc/12959/stat : 12959 (pwbo2.02) S 12957 12959 10378 0 -1 4202496 1329 0 0 0 4 0 0 0 20 0 3 0 331990090 166674432 1268 33554432000 4194304 5964478 140737053592416 140737053591936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/12959/statm: 40692 1279 198 433 0 40256 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 162768

[startup+0.100161 s]
/proc/loadavg: 3.92 3.99 3.96 2/174 12959
/proc/meminfo: memFree=29130296/32873844 swapFree=7120/7120
[pid=12959] ppid=12957 vsize=162768 CPUtime=0.18 cores=0,2,4,6
/proc/12959/stat : 12959 (pwbo2.02) S 12957 12959 10378 0 -1 4202496 1870 0 0 0 18 0 0 0 20 0 3 0 331990090 166674432 1782 33554432000 4194304 5964478 140737053592416 140737053591936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/12959/statm: 40692 1782 198 433 0 40256 0
Current children cumulated CPU time (s) 0.18
Current children cumulated vsize (KiB) 162768

[startup+0.300129 s]
/proc/loadavg: 3.92 3.99 3.96 2/174 12959
/proc/meminfo: memFree=29130296/32873844 swapFree=7120/7120
[pid=12959] ppid=12957 vsize=162768 CPUtime=0.58 cores=0,2,4,6
/proc/12959/stat : 12959 (pwbo2.02) S 12957 12959 10378 0 -1 4202496 2924 0 0 0 58 0 0 0 20 0 3 0 331990090 166674432 2582 33554432000 4194304 5964478 140737053592416 140737053591936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/12959/statm: 40692 2582 198 433 0 40256 0
Current children cumulated CPU time (s) 0.58
Current children cumulated vsize (KiB) 162768

[startup+0.700153 s]
/proc/loadavg: 3.92 3.99 3.96 2/174 12959
/proc/meminfo: memFree=29130296/32873844 swapFree=7120/7120
[pid=12959] ppid=12957 vsize=162900 CPUtime=1.37 cores=0,2,4,6
/proc/12959/stat : 12959 (pwbo2.02) S 12957 12959 10378 0 -1 4202496 3845 0 0 0 137 0 0 0 20 0 3 0 331990090 166809600 3142 33554432000 4194304 5964478 140737053592416 140737053591936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/12959/statm: 40725 3142 198 433 0 40289 0
Current children cumulated CPU time (s) 1.37
Current children cumulated vsize (KiB) 162900

[startup+1.50032 s]
/proc/loadavg: 3.92 3.99 3.96 4/176 12990
/proc/meminfo: memFree=29111800/32873844 swapFree=7120/7120
[pid=12959] ppid=12957 vsize=162900 CPUtime=2.98 cores=0,2,4,6
/proc/12959/stat : 12959 (pwbo2.02) S 12957 12959 10378 0 -1 4202496 4406 0 0 0 297 1 0 0 20 0 3 0 331990090 166809600 3663 33554432000 4194304 5964478 140737053592416 140737053591936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/12959/statm: 40725 3663 198 433 0 40289 0
[pid=12959/tid=12961] ppid=12957 vsize=162900 CPUtime=1.47 cores=0,2,4,6
/proc/12959/task/12961/stat : 12961 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 2819 0 0 0 147 0 0 0 20 0 3 0 331990091 166809600 3663 33554432000 4194304 5964478 140737053592416 140264761047384 18446744073699066235 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12959/tid=12962] ppid=12957 vsize=162900 CPUtime=1.47 cores=0,2,4,6
/proc/12959/task/12962/stat : 12962 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 703 0 0 0 147 0 0 0 20 0 3 0 331990091 166809600 3663 33554432000 4194304 5964478 140737053592416 140264750557408 4288610 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 2.98
Current children cumulated vsize (KiB) 162900

[startup+3.10032 s]
/proc/loadavg: 3.92 3.99 3.96 4/179 13000
/proc/meminfo: memFree=28940440/32873844 swapFree=7120/7120
[pid=12959] ppid=12957 vsize=164188 CPUtime=6.17 cores=0,2,4,6
/proc/12959/stat : 12959 (pwbo2.02) S 12957 12959 10378 0 -1 4202496 5189 0 0 0 616 1 0 0 20 0 3 0 331990090 168128512 4443 33554432000 4194304 5964478 140737053592416 140737053591936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/12959/statm: 41047 4443 198 433 0 40611 0
[pid=12959/tid=12961] ppid=12957 vsize=164188 CPUtime=3.07 cores=0,2,4,6
/proc/12959/task/12961/stat : 12961 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 3424 0 0 0 307 0 0 0 20 0 3 0 331990091 168128512 4443 33554432000 4194304 5964478 140737053592416 140264761047368 4222416 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12959/tid=12962] ppid=12957 vsize=164188 CPUtime=3.07 cores=0,2,4,6
/proc/12959/task/12962/stat : 12962 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 881 0 0 0 307 0 0 0 20 0 3 0 331990091 168128512 4443 33554432000 4194304 5964478 140737053592416 140264750557848 4392962 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 6.17
Current children cumulated vsize (KiB) 164188

[startup+6.30035 s]
/proc/loadavg: 3.85 3.97 3.96 5/181 13002
/proc/meminfo: memFree=28478116/32873844 swapFree=7120/7120
[pid=12959] ppid=12957 vsize=164452 CPUtime=12.58 cores=0,2,4,6
/proc/12959/stat : 12959 (pwbo2.02) S 12957 12959 10378 0 -1 4202496 6664 0 0 0 1256 2 0 0 20 0 3 0 331990090 168398848 5621 33554432000 4194304 5964478 140737053592416 140737053591936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/12959/statm: 41113 5621 198 433 0 40677 0
[pid=12959/tid=12961] ppid=12957 vsize=164452 CPUtime=6.26 cores=0,2,4,6
/proc/12959/task/12961/stat : 12961 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 4522 0 0 0 626 0 0 0 20 0 3 0 331990091 168398848 5621 33554432000 4194304 5964478 140737053592416 140264761047992 4221120 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12959/tid=12962] ppid=12957 vsize=164452 CPUtime=6.26 cores=0,2,4,6
/proc/12959/task/12962/stat : 12962 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 1258 0 0 0 625 1 0 0 20 0 3 0 331990091 168398848 5621 33554432000 4194304 5964478 140737053592416 140264750557256 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 12.58
Current children cumulated vsize (KiB) 164452

[startup+12.7004 s]
/proc/loadavg: 3.86 3.97 3.96 5/181 13002
/proc/meminfo: memFree=28398492/32873844 swapFree=7120/7120
[pid=12959] ppid=12957 vsize=164856 CPUtime=25.37 cores=0,2,4,6
/proc/12959/stat : 12959 (pwbo2.02) S 12957 12959 10378 0 -1 4202496 10070 0 0 0 2535 2 0 0 20 0 3 0 331990090 168812544 9027 33554432000 4194304 5964478 140737053592416 140737053591936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/12959/statm: 41214 9027 198 433 0 40778 0
[pid=12959/tid=12961] ppid=12957 vsize=164856 CPUtime=12.65 cores=0,2,4,6
/proc/12959/task/12961/stat : 12961 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 6408 0 0 0 1264 1 0 0 20 0 3 0 331990091 168812544 9027 33554432000 4194304 5964478 140737053592416 140264761046840 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12959/tid=12962] ppid=12957 vsize=164856 CPUtime=12.64 cores=0,2,4,6
/proc/12959/task/12962/stat : 12962 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 2778 0 0 0 1263 1 0 0 20 0 3 0 331990091 168812544 9027 33554432000 4194304 5964478 140737053592416 140264750557496 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 25.37
Current children cumulated vsize (KiB) 164856

[startup+25.5004 s]
/proc/loadavg: 3.89 3.97 3.96 5/181 13002
/proc/meminfo: memFree=28317484/32873844 swapFree=7120/7120
[pid=12959] ppid=12957 vsize=163708 CPUtime=50.96 cores=0,2,4,6
/proc/12959/stat : 12959 (pwbo2.02) S 12957 12959 10378 0 -1 4202496 12891 0 0 0 5092 4 0 0 20 0 3 0 331990090 167636992 11596 33554432000 4194304 5964478 140737053592416 140737053591936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/12959/statm: 40927 11596 198 433 0 40491 0
[pid=12959/tid=12961] ppid=12957 vsize=163708 CPUtime=25.41 cores=0,2,4,6
/proc/12959/task/12961/stat : 12961 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 9144 0 0 0 2539 2 0 0 20 0 3 0 331990091 167636992 11596 33554432000 4194304 5964478 140737053592416 140264761047136 4400056 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12959/tid=12962] ppid=12957 vsize=163708 CPUtime=25.4 cores=0,2,4,6
/proc/12959/task/12962/stat : 12962 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 2863 0 0 0 2539 1 0 0 20 0 3 0 331990091 167636992 11596 33554432000 4194304 5964478 140737053592416 140264750557408 4403560 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 50.96
Current children cumulated vsize (KiB) 163708

[startup+51.1003 s]
/proc/loadavg: 3.93 3.97 3.96 5/181 13002
/proc/meminfo: memFree=28283288/32873844 swapFree=7120/7120
[pid=12959] ppid=12957 vsize=164244 CPUtime=102.14 cores=0,2,4,6
/proc/12959/stat : 12959 (pwbo2.02) S 12957 12959 10378 0 -1 4202496 13944 0 0 0 10210 4 0 0 20 0 3 0 331990090 168185856 13369 33554432000 4194304 5964478 140737053592416 140737053591936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/12959/statm: 41061 13369 198 433 0 40625 0
[pid=12959/tid=12961] ppid=12957 vsize=164244 CPUtime=50.93 cores=0,2,4,6
/proc/12959/task/12961/stat : 12961 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 9788 0 0 0 5091 2 0 0 20 0 3 0 331990091 168185856 13369 33554432000 4194304 5964478 140737053592416 140264761047032 4400087 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12959/tid=12962] ppid=12957 vsize=164244 CPUtime=50.93 cores=0,2,4,6
/proc/12959/task/12962/stat : 12962 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 3272 0 0 0 5092 1 0 0 20 0 3 0 331990091 168185856 13370 33554432000 4194304 5964478 140737053592416 140264750557496 4403721 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 102.14
Current children cumulated vsize (KiB) 164244

[startup+102.307 s]
/proc/loadavg: 3.97 3.97 3.96 5/181 13002
/proc/meminfo: memFree=28227484/32873844 swapFree=7120/7120
[pid=12959] ppid=12957 vsize=232496 CPUtime=204.54 cores=0,2,4,6
/proc/12959/stat : 12959 (pwbo2.02) S 12957 12959 10378 0 -1 4202496 26087 0 0 0 20444 10 0 0 20 0 3 0 331990090 238075904 25512 33554432000 4194304 5964478 140737053592416 140737053591936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/12959/statm: 58124 25512 198 433 0 57688 0
[pid=12959/tid=12961] ppid=12957 vsize=232496 CPUtime=102.01 cores=0,2,4,6
/proc/12959/task/12961/stat : 12961 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 19672 0 0 0 10193 8 0 0 20 0 3 0 331990091 238075904 25512 33554432000 4194304 5964478 140737053592416 140264761046840 4400519 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12959/tid=12962] ppid=12957 vsize=232496 CPUtime=102.01 cores=0,2,4,6
/proc/12959/task/12962/stat : 12962 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 5531 0 0 0 10198 3 0 0 20 0 3 0 331990091 238075904 25512 33554432000 4194304 5964478 140737053592416 140264750557256 4222416 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 204.54
Current children cumulated vsize (KiB) 232496

[startup+162.3 s]
/proc/loadavg: 3.99 3.97 3.96 5/182 13006
/proc/meminfo: memFree=28140208/32873844 swapFree=7120/7120
[pid=12959] ppid=12957 vsize=235036 CPUtime=324.51 cores=0,2,4,6
/proc/12959/stat : 12959 (pwbo2.02) S 12957 12959 10378 0 -1 4202496 38114 0 0 0 32435 16 0 0 20 0 3 0 331990090 240676864 37538 33554432000 4194304 5964478 140737053592416 140737053591936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/12959/statm: 58759 37538 198 433 0 58323 0
[pid=12959/tid=12961] ppid=12957 vsize=235036 CPUtime=161.85 cores=0,2,4,6
/proc/12959/task/12961/stat : 12961 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 27508 0 0 0 16173 12 0 0 20 0 3 0 331990091 240676864 37538 33554432000 4194304 5964478 140737053592416 140264761047080 4389160 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12959/tid=12962] ppid=12957 vsize=235036 CPUtime=161.84 cores=0,2,4,6
/proc/12959/task/12962/stat : 12962 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 9722 0 0 0 16177 7 0 0 20 0 3 0 331990091 240676864 37538 33554432000 4194304 5964478 140737053592416 140264750557256 4399912 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 324.51
Current children cumulated vsize (KiB) 235036

[startup+222.3 s]
/proc/loadavg: 3.99 3.97 3.96 5/181 13006
/proc/meminfo: memFree=27968016/32873844 swapFree=7120/7120
[pid=12959] ppid=12957 vsize=302560 CPUtime=444.49 cores=0,2,4,6
/proc/12959/stat : 12959 (pwbo2.02) S 12957 12959 10378 0 -1 4202496 46304 0 0 0 44428 21 0 0 20 0 3 0 331990090 309821440 45722 33554432000 4194304 5964478 140737053592416 140737053591936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/12959/statm: 75640 45722 198 433 0 75204 0
[pid=12959/tid=12961] ppid=12957 vsize=302560 CPUtime=221.69 cores=0,2,4,6
/proc/12959/task/12961/stat : 12961 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 35145 0 0 0 22151 18 0 0 20 0 3 0 331990091 309821440 45722 33554432000 4194304 5964478 140737053592416 140264761046840 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12959/tid=12962] ppid=12957 vsize=302560 CPUtime=221.69 cores=0,2,4,6
/proc/12959/task/12962/stat : 12962 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 10275 0 0 0 22161 8 0 0 20 0 3 0 331990091 309821440 45722 33554432000 4194304 5964478 140737053592416 140264750557256 4403370 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 444.49
Current children cumulated vsize (KiB) 302560

[startup+282.3 s]
/proc/loadavg: 4.04 3.99 3.96 5/181 13006
/proc/meminfo: memFree=27936000/32873844 swapFree=7120/7120
[pid=12959] ppid=12957 vsize=304240 CPUtime=564.47 cores=0,2,4,6
/proc/12959/stat : 12959 (pwbo2.02) S 12957 12959 10378 0 -1 4202496 50602 0 0 0 56423 24 0 0 20 0 3 0 331990090 311541760 50019 33554432000 4194304 5964478 140737053592416 140737053591936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/12959/statm: 76060 50019 198 433 0 75624 0
[pid=12959/tid=12961] ppid=12957 vsize=304240 CPUtime=281.54 cores=0,2,4,6
/proc/12959/task/12961/stat : 12961 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 37535 0 0 0 28134 20 0 0 20 0 3 0 331990091 311541760 50019 33554432000 4194304 5964478 140737053592416 140264761046840 4222416 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12959/tid=12962] ppid=12957 vsize=304240 CPUtime=281.53 cores=0,2,4,6
/proc/12959/task/12962/stat : 12962 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 12183 0 0 0 28141 12 0 0 20 0 3 0 331990091 311541760 50019 33554432000 4194304 5964478 140737053592416 140264750557256 4400476 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 564.47
Current children cumulated vsize (KiB) 304240

[startup+342.3 s]
/proc/loadavg: 4.01 3.99 3.96 5/181 13006
/proc/meminfo: memFree=27881340/32873844 swapFree=7120/7120
[pid=12959] ppid=12957 vsize=305304 CPUtime=684.45 cores=0,2,4,6
/proc/12959/stat : 12959 (pwbo2.02) S 12957 12959 10378 0 -1 4202496 61178 0 0 0 68413 32 0 0 20 0 3 0 331990090 312631296 60594 33554432000 4194304 5964478 140737053592416 140737053591936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/12959/statm: 76326 60594 198 433 0 75890 0

################
# More data... #
################

[pid=12959/tid=12962] ppid=12957 vsize=305304 CPUtime=341.37 cores=0,2,4,6
/proc/12959/task/12962/stat : 12962 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 12991 0 0 0 34124 13 0 0 20 0 3 0 331990091 312631296 60594 33554432000 4194304 5964478 140737053592416 140264750557304 4403727 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 684.45
Current children cumulated vsize (KiB) 305304

[startup+402.3 s]
/proc/loadavg: 4.00 3.99 3.96 5/181 13006
/proc/meminfo: memFree=27808600/32873844 swapFree=7120/7120
[pid=12959] ppid=12957 vsize=371800 CPUtime=804.43 cores=0,2,4,6
/proc/12959/stat : 12959 (pwbo2.02) S 12957 12959 10378 0 -1 4202496 76119 0 0 0 80397 46 0 0 20 0 3 0 331990090 380723200 75535 33554432000 4194304 5964478 140737053592416 140737053591936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/12959/statm: 92950 75535 198 433 0 92514 0
[pid=12959/tid=12961] ppid=12957 vsize=371800 CPUtime=401.24 cores=0,2,4,6
/proc/12959/task/12961/stat : 12961 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 60210 0 0 0 40082 42 0 0 20 0 3 0 331990091 380723200 75535 33554432000 4194304 5964478 140737053592416 140264761047032 4400528 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12959/tid=12962] ppid=12957 vsize=371800 CPUtime=401.21 cores=0,2,4,6
/proc/12959/task/12962/stat : 12962 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 15025 0 0 0 40105 16 0 0 20 0 3 0 331990091 380723200 75535 33554432000 4194304 5964478 140737053592416 140264750557304 4400516 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 804.43
Current children cumulated vsize (KiB) 371800

[startup+462.3 s]
/proc/loadavg: 4.00 3.99 3.96 5/181 13007
/proc/meminfo: memFree=27790036/32873844 swapFree=7120/7120
[pid=12959] ppid=12957 vsize=373740 CPUtime=924.41 cores=0,2,4,6
/proc/12959/stat : 12959 (pwbo2.02) S 12957 12959 10378 0 -1 4202496 77841 0 0 0 92394 47 0 0 20 0 3 0 331990090 382709760 77256 33554432000 4194304 5964478 140737053592416 140737053591936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/12959/statm: 93435 77256 198 433 0 92999 0
[pid=12959/tid=12961] ppid=12957 vsize=373740 CPUtime=461.08 cores=0,2,4,6
/proc/12959/task/12961/stat : 12961 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 61193 0 0 0 46065 43 0 0 20 0 3 0 331990091 382709760 77256 33554432000 4194304 5964478 140737053592416 140264761047080 4400010 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12959/tid=12962] ppid=12957 vsize=373740 CPUtime=461.06 cores=0,2,4,6
/proc/12959/task/12962/stat : 12962 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 15764 0 0 0 46089 17 0 0 20 0 3 0 331990091 382709760 77256 33554432000 4194304 5964478 140737053592416 140264750557496 4222416 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 924.41
Current children cumulated vsize (KiB) 373740

[startup+522.3 s]
/proc/loadavg: 4.00 3.99 3.96 5/181 13007
/proc/meminfo: memFree=27734140/32873844 swapFree=7120/7120
[pid=12959] ppid=12957 vsize=439876 CPUtime=1044.4 cores=0,2,4,6
/proc/12959/stat : 12959 (pwbo2.02) S 12957 12959 10378 0 -1 4202496 90748 0 0 0 104375 65 0 0 20 0 3 0 331990090 450433024 90163 33554432000 4194304 5964478 140737053592416 140737053591936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/12959/statm: 109969 90163 198 433 0 109533 0
[pid=12959/tid=12961] ppid=12957 vsize=439876 CPUtime=520.93 cores=0,2,4,6
/proc/12959/task/12961/stat : 12961 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 72531 0 0 0 52035 58 0 0 20 0 3 0 331990091 450433024 90163 33554432000 4194304 5964478 140737053592416 140264761047032 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12959/tid=12962] ppid=12957 vsize=439876 CPUtime=520.9 cores=0,2,4,6
/proc/12959/task/12962/stat : 12962 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 17333 0 0 0 52070 20 0 0 20 0 3 0 331990091 450433024 90163 33554432000 4194304 5964478 140737053592416 140264750557304 4399915 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1044.4
Current children cumulated vsize (KiB) 439876

[startup+582.3 s]
/proc/loadavg: 4.00 3.99 3.96 5/181 13007
/proc/meminfo: memFree=27673784/32873844 swapFree=7120/7120
[pid=12959] ppid=12957 vsize=441264 CPUtime=1164.37 cores=0,2,4,6
/proc/12959/stat : 12959 (pwbo2.02) S 12957 12959 10378 0 -1 4202496 96395 0 0 0 116366 71 0 0 20 0 3 0 331990090 451854336 95810 33554432000 4194304 5964478 140737053592416 140737053591936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/12959/statm: 110316 95810 198 433 0 109880 0
[pid=12959/tid=12961] ppid=12957 vsize=441264 CPUtime=580.78 cores=0,2,4,6
/proc/12959/task/12961/stat : 12961 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 78013 0 0 0 58012 66 0 0 20 0 3 0 331990091 451854336 95810 33554432000 4194304 5964478 140737053592416 140264761047080 4388838 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12959/tid=12962] ppid=12957 vsize=441264 CPUtime=580.74 cores=0,2,4,6
/proc/12959/task/12962/stat : 12962 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 17498 0 0 0 58053 21 0 0 20 0 3 0 331990091 451854336 95810 33554432000 4194304 5964478 140737053592416 140264750557256 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1164.37
Current children cumulated vsize (KiB) 441264

[startup+642.3 s]
/proc/loadavg: 4.00 3.99 3.96 5/181 13007
/proc/meminfo: memFree=27567704/32873844 swapFree=7120/7120
[pid=12959] ppid=12957 vsize=508572 CPUtime=1284.34 cores=0,2,4,6
/proc/12959/stat : 12959 (pwbo2.02) S 12957 12959 10378 0 -1 4202496 99168 0 0 0 128359 75 0 0 20 0 3 0 331990090 520777728 99116 33554432000 4194304 5964478 140737053592416 140737053591936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/12959/statm: 127143 99116 198 433 0 126707 0
[pid=12959/tid=12961] ppid=12957 vsize=508572 CPUtime=640.61 cores=0,2,4,6
/proc/12959/task/12961/stat : 12961 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 78103 0 0 0 63994 67 0 0 20 0 3 0 331990091 520777728 99116 33554432000 4194304 5964478 140737053592416 140264761047080 4400087 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12959/tid=12962] ppid=12957 vsize=508572 CPUtime=640.59 cores=0,2,4,6
/proc/12959/task/12962/stat : 12962 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 20181 0 0 0 64033 26 0 0 20 0 3 0 331990091 520777728 99116 33554432000 4194304 5964478 140737053592416 140264750557256 4403727 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1284.34
Current children cumulated vsize (KiB) 508572

[startup+702.3 s]
/proc/loadavg: 4.03 4.00 3.97 5/181 13007
/proc/meminfo: memFree=27430900/32873844 swapFree=7120/7120
[pid=12959] ppid=12957 vsize=509296 CPUtime=1404.25 cores=0,2,4,6
/proc/12959/stat : 12959 (pwbo2.02) S 12957 12959 10378 0 -1 4202496 99353 0 0 0 140350 75 0 0 20 0 3 0 331990090 521519104 99685 33554432000 4194304 5964478 140737053592416 140737053591936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/12959/statm: 127324 99685 198 433 0 126888 0
[pid=12959/tid=12961] ppid=12957 vsize=509296 CPUtime=700.39 cores=0,2,4,6
/proc/12959/task/12961/stat : 12961 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 78288 0 0 0 69971 68 0 0 20 0 3 0 331990091 521519104 99685 33554432000 4194304 5964478 140737053592416 140264761046840 4409232 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12959/tid=12962] ppid=12957 vsize=509296 CPUtime=700.42 cores=0,2,4,6
/proc/12959/task/12962/stat : 12962 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 20181 0 0 0 70015 27 0 0 20 0 3 0 331990091 521519104 99685 33554432000 4194304 5964478 140737053592416 140264750557256 4222464 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1404.25
Current children cumulated vsize (KiB) 509296

[startup+762.307 s]
/proc/loadavg: 4.01 4.00 3.97 5/182 13011
/proc/meminfo: memFree=27234016/32873844 swapFree=7120/7120
[pid=12959] ppid=12957 vsize=509580 CPUtime=1524.2 cores=0,2,4,6
/proc/12959/stat : 12959 (pwbo2.02) S 12957 12959 10378 0 -1 4202496 99486 0 0 0 152345 75 0 0 20 0 3 0 331990090 521809920 99935 33554432000 4194304 5964478 140737053592416 140737053591936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/12959/statm: 127395 99935 198 433 0 126959 0
[pid=12959/tid=12961] ppid=12957 vsize=509580 CPUtime=760.2 cores=0,2,4,6
/proc/12959/task/12961/stat : 12961 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 78359 0 0 0 75951 69 0 0 20 0 3 0 331990091 521809920 99935 33554432000 4194304 5964478 140737053592416 140264761046840 4221120 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12959/tid=12962] ppid=12957 vsize=509580 CPUtime=760.27 cores=0,2,4,6
/proc/12959/task/12962/stat : 12962 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 20243 0 0 0 75999 28 0 0 20 0 3 0 331990091 521809920 99935 33554432000 4194304 5964478 140737053592416 140264750557256 4389170 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1524.2
Current children cumulated vsize (KiB) 509580

[startup+822.3 s]
/proc/loadavg: 4.00 4.00 3.97 5/181 13011
/proc/meminfo: memFree=27113620/32873844 swapFree=7120/7120
[pid=12959] ppid=12957 vsize=510664 CPUtime=1644.14 cores=0,2,4,6
/proc/12959/stat : 12959 (pwbo2.02) S 12957 12959 10378 0 -1 4202496 99665 0 0 0 164338 76 0 0 20 0 3 0 331990090 522919936 100254 33554432000 4194304 5964478 140737053592416 140737053591936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/12959/statm: 127666 100254 198 433 0 127230 0
[pid=12959/tid=12961] ppid=12957 vsize=510664 CPUtime=820.03 cores=0,2,4,6
/proc/12959/task/12961/stat : 12961 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 78391 0 0 0 81933 70 0 0 20 0 3 0 331990091 522919936 100254 33554432000 4194304 5964478 140737053592416 140264761047208 4400006 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12959/tid=12962] ppid=12957 vsize=510664 CPUtime=820.08 cores=0,2,4,6
/proc/12959/task/12962/stat : 12962 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 20390 0 0 0 81979 29 0 0 20 0 3 0 331990091 522919936 100254 33554432000 4194304 5964478 140737053592416 140264750557256 4403664 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1644.14
Current children cumulated vsize (KiB) 510664

[startup+882.3 s]
/proc/loadavg: 4.00 4.00 3.97 5/181 13011
/proc/meminfo: memFree=26871908/32873844 swapFree=7120/7120
[pid=12959] ppid=12957 vsize=511768 CPUtime=1764.11 cores=0,2,4,6
/proc/12959/stat : 12959 (pwbo2.02) S 12957 12959 10378 0 -1 4202496 102957 0 0 0 176331 80 0 0 20 0 3 0 331990090 524050432 103555 33554432000 4194304 5964478 140737053592416 140737053591936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/12959/statm: 127942 103555 198 433 0 127506 0
[pid=12959/tid=12961] ppid=12957 vsize=511768 CPUtime=879.87 cores=0,2,4,6
/proc/12959/task/12961/stat : 12961 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 81603 0 0 0 87912 75 0 0 20 0 3 0 331990091 524050432 103555 33554432000 4194304 5964478 140737053592416 140264761047032 4400038 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12959/tid=12962] ppid=12957 vsize=511768 CPUtime=879.92 cores=0,2,4,6
/proc/12959/task/12962/stat : 12962 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 20470 0 0 0 87962 30 0 0 20 0 3 0 331990091 524050432 103555 33554432000 4194304 5964478 140737053592416 140264750557256 4399975 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1764.11
Current children cumulated vsize (KiB) 511768



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+900.3 s]
/proc/loadavg: 4.00 4.00 3.97 5/181 13011
/proc/meminfo: memFree=26795028/32873844 swapFree=7120/7120
[pid=12959] ppid=12957 vsize=578264 CPUtime=1800.12 cores=0,2,4,6
/proc/12959/stat : 12959 (pwbo2.02) S 12957 12959 10378 0 -1 4202496 106845 0 0 0 179929 83 0 0 20 0 3 0 331990090 592142336 107354 33554432000 4194304 5964478 140737053592416 140737053591936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/12959/statm: 144566 107354 198 433 0 144130 0
[pid=12959/tid=12961] ppid=12957 vsize=578264 CPUtime=897.83 cores=0,2,4,6
/proc/12959/task/12961/stat : 12961 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 85464 0 0 0 89705 78 0 0 20 0 3 0 331990091 592142336 107354 33554432000 4194304 5964478 140737053592416 140264761047032 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12959/tid=12962] ppid=12957 vsize=578264 CPUtime=897.88 cores=0,2,4,6
/proc/12959/task/12962/stat : 12962 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 20497 0 0 0 89758 30 0 0 20 0 3 0 331990091 592142336 107354 33554432000 4194304 5964478 140737053592416 140264750557256 4222416 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1800.12
Current children cumulated vsize (KiB) 578264

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+900.3 s]
/proc/loadavg: 4.00 4.00 3.97 5/181 13011
/proc/meminfo: memFree=26795028/32873844 swapFree=7120/7120
[pid=12959] ppid=12957 vsize=578264 CPUtime=1800.12 cores=0,2,4,6
/proc/12959/stat : 12959 (pwbo2.02) S 12957 12959 10378 0 -1 4202496 106845 0 0 0 179929 83 0 0 20 0 3 0 331990090 592142336 107354 33554432000 4194304 5964478 140737053592416 140737053591936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/12959/statm: 144566 107354 198 433 0 144130 0
[pid=12959/tid=12961] ppid=12957 vsize=578264 CPUtime=897.83 cores=0,2,4,6
/proc/12959/task/12961/stat : 12961 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 85464 0 0 0 89705 78 0 0 20 0 3 0 331990091 592142336 107354 33554432000 4194304 5964478 140737053592416 140264761047032 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12959/tid=12962] ppid=12957 vsize=578264 CPUtime=897.88 cores=0,2,4,6
/proc/12959/task/12962/stat : 12962 (pwbo2.02) R 12957 12959 10378 0 -1 4202560 20497 0 0 0 89758 30 0 0 20 0 3 0 331990091 592142336 107354 33554432000 4194304 5964478 140737053592416 140264750557256 4222416 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1800.12
Current children cumulated vsize (KiB) 578264

Child status: 0

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 12959 and gives
#  childrusage.ru_utime.tv_sec=1794
#  childrusage.ru_utime.tv_usec=652171
#  childrusage.ru_stime.tv_sec=1
#  childrusage.ru_stime.tv_usec=100832
# CPU time returned by wait4() is 1795.75
# while last known CPU time is 1800.12
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 900.316
CPU time (s): 1800.12
CPU user time (s): 1799.29
CPU system time (s): 0.83
CPU usage (%): 199.943
Max. virtual memory (cumulated for all children) (KiB): 578264

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1794.65
system time used= 1.10083
maximum resident set size= 429424
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 106847
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 245
involuntary context switches= 1873

runsolver used 1.71974 second user time and 5.17521 second system time

The end

Launcher Data 

Begin job on node142 at 2012-06-02 19:37:32
IDJOB=3726309
IDBENCH=89854
IDSOLVER=2309
FILE ID=node142/3726309-1338658652
RUNJOBID= node142-1338651410-11883
PBS_JOBID= 14636438
Free space on /tmp= 71196 MiB

SOLVER NAME= pwbo 2.02
BENCH NAME= PB11/normalized-PB11/OPT-SMALLINT-LIN/heinz/normalized-neos-538867.opb
COMMAND LINE= pwbo2.02 -file-format=opb -verbosity=0 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3726309-1338658652/watcher-3726309-1338658652 -o /tmp/evaluation-result-3726309-1338658652/solver-3726309-1338658652 -C 1800 -W 1900 -M 15500  pwbo2.02 -file-format=opb -verbosity=0 HOME/instance-3726309-1338658652.opb

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 3b75c8039d6468258a1aa0343ce1fddb
RANDOM SEED=793556893

node142.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.776
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.776
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.776
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5391.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.776
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.776
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.776
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.776
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.776
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        29130452 kB
Buffers:          226172 kB
Cached:          2689868 kB
SwapCached:            0 kB
Active:          1081132 kB
Inactive:        1850152 kB
Active(anon):       6548 kB
Inactive(anon):    11236 kB
Active(file):    1074584 kB
Inactive(file):  1838916 kB
Unevictable:        7120 kB
Mlocked:            7120 kB
SwapTotal:      67108856 kB
SwapFree:       67108856 kB
Dirty:              5668 kB
Writeback:             4 kB
AnonPages:         21948 kB
Mapped:            10312 kB
Shmem:               200 kB
Slab:             673432 kB
SReclaimable:     115348 kB
SUnreclaim:       558084 kB
KernelStack:        1472 kB
PageTables:         3776 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     140844 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 71176 MiB
End job on node142 at 2012-06-02 19:52:34