Trace number 3703625

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
pwbo 2.0SAT (TO)-37 1800.13 900.437

General information on the benchmark

Namenormalized-PB06/OPT-SMALLINT/web/www.nlsde.buaa.edu.cn/
~kexu/benchmarks/frb40-19-opb/normalized-frb40-19-4.opb
MD5SUM7262f9f4ab1a3ada75b3b140acf97f7d
Bench CategoryOPT-SMALLINT (optimisation, small integers)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark-40
Best CPU time to get the best result obtained on this benchmark269.661
Has Objective FunctionYES
SatisfiableYES
(Un)Satisfiability was provedYES
Best value of the objective function -40
Optimality of the best value was proved NO
Number of variables760
Total number of constraints41605
Number of constraints which are clauses41605
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint2
Maximum length of a constraint2
Number of terms in the objective function 760
Biggest coefficient in the objective function 1
Number of bits for the biggest coefficient in the objective function 1
Sum of the numbers in the objective function 760
Number of bits of the sum of numbers in the objective function 10
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 760
Number of bits of the biggest sum of numbers10
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Quality of the solution as a function of time


Download the above graph as a PDF file / as an EPS file

Solver Data 

0.00/0.00	c ==============================[ Solver Version ]===============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  pwbo:                    2.0 beta - March, 2012                            |
0.00/0.00	c |                                                                             |
0.00/0.00	c Parsing opb file format. File HOME/instance-3703625-1338091451.opb.
0.00/0.00	c Instance file HOME/instance-3703625-1338091451.opb
0.00/0.00	c File size is 993315 bytes.
0.00/0.07	c Cost Offset: -760
0.00/0.07	c Highest Coefficient sum: 3
0.00/0.07	c ============================[ Problem Statistics ]=============================
0.00/0.07	c |                                                                             |
0.00/0.07	c |  Parsing time:          0.07         s                                      |
0.00/0.07	c |  Number Variables:      760                                                 |
0.00/0.07	c |  Number Clauses:        41605                                               |
0.00/0.07	c |  Number Cardinality:    0                                                   |
0.00/0.07	c |  Number PB Constraints: 0                                                   |
0.00/0.07	c |                                                                             |
0.00/0.09	c |  Problem Type:          Optimization                                        |
0.00/0.09	c |  Objective Function:    Cardinality Constraint                              |
0.00/0.09	c |                                                                             |
0.00/0.09	c ===============================================================================
1800.13/900.41	o -29
1800.13/900.41	o -30
1800.13/900.41	o -31
1800.13/900.41	o -32
1800.13/900.41	o -33
1800.13/900.41	o -34
1800.13/900.41	o -35
1800.13/900.41	o -36
1800.13/900.41	o -37
1800.13/900.41	c ===============================================================================
1800.13/900.41	c Wall Clock Time: 900.41 s
1800.13/900.41	c Best Solution: -37
1800.13/900.41	s SATISFIABLE
1800.13/900.41	v -x760 -x759 -x758 -x757 -x756 x755 -x754 -x753 -x752 -x751 -x750 -x749 -x748 -x747 -x746 -x745 -x744 -x743 -x742 -x741 -x740 -x739 -x738 -x737 -x736 -x735 -x734 -x733 -x732 -x731 -x730 -x729 -x728 x727 -x726 -x725 -x724 -x723 -x722 -x721 -x720 -x719 -x718 -x717 -x716 -x715 -x714 -x713 x712 -x711 -x710 -x709 -x708 -x707 -x706 -x705 -x704 -x703 -x702 -x701 -x700 -x699 -x698 -x697 -x696 -x695 -x694 -x693 -x692 -x691 -x690 -x689 -x688 -x687 -x686 -x685 -x684 -x683 -x682 -x681 -x680 -x679 -x678 -x677 x676 -x675 -x674 -x673 -x672 -x671 -x670 -x669 -x668 -x667 -x666 -x665 -x664 -x663 -x662 -x661 x660 -x659 -x658 -x657 -x656 -x655 -x654 -x653 -x652 -x651 -x650 -x649 -x648 -x647 -x646 -x645 -x644 -x643 -x642 -x641 -x640 -x639 -x638 -x637 -x636 -x635 x634 -x633 -x632 -x631 -x630 -x629 -x628 -x627 -x626 -x625 -x624 -x623 -x622 -x621 -x620 -x619 -x618 -x617 x616 -x615 -x614 -x613 -x612 -x611 -x610 -x609 -x608 x607 -x606 -x605 -x604 -x603 -x602 -x601 -x600 -x599 -x598 -x597 -x596 -x595 -x594 -x593 -x592 -x591 -x590 -x589 -x588 x587 -x586 -x585 -x584 -x583 -x582 -x581 -x580 -x579 -x578 -x577 -x576 -x575 -x574 -x573 -x572 -x571 -x570 -x569 -x568 x567 -x566 -x565 -x564 -x563 -x562 -x561 -x560 -x559 -x558 -x557 -x556 -x555 -x554 -x553 -x552 -x551 -x550 -x549 x548 -x547 -x546 -x545 -x544 -x543 -x542 -x541 -x540 -x539 -x538 -x537 -x536 -x535 -x534 -x533 -x532 -x531 -x530 -x529 -x528 x527 -x526 -x525 -x524 -x523 -x522 -x521 -x520 -x519 -x518 -x517 -x516 -x515 -x514 -x513 -x512 -x511 -x510 -x509 -x508 -x507 -x506 -x505 -x504 -x503 -x502 -x501 -x500 -x499 x498 -x497 -x496 -x495 -x494 -x493 -x492 -x491 -x490 -x489 x488 -x487 -x486 -x485 -x484 -x483 -x482 -x481 -x480 -x479 -x478 -x477 -x476 -x475 -x474 -x473 -x472 x471 -x470 -x469 -x468 -x467 -x466 -x465 -x464 -x463 -x462 -x461 -x460 -x459 -x458 -x457 -x456 -x455 -x454 -x453 -x452 -x451 -x450 -x449 -x448 -x447 -x446 -x445 -x444 -x443 -x442 -x441 x440 -x439 -x438 -x437 -x436 -x435 -x434 -x433 -x432 -x431 -x430 -x429 -x428 -x427 -x426 x425 -x424 -x423 -x422 -x421 -x420 -x419 -x418 -x417 x416 -x415 -x414 -x413 -x412 -x411 -x410 -x409 -x408 -x407 -x406 -x405 -x404 -x403 -x402 -x401 -x400 -x399 -x398 -x397 -x396 -x395 -x394 -x393 -x392 -x391 -x390 -x389 -x388 x387 -x386 -x385 -x384 -x383 -x382 -x381 -x380 -x379 -x378 -x377 -x376 -x375 x374 -x373 -x372 -x371 -x370 -x369 -x368 -x367 -x366 -x365 -x364 -x363 -x362 -x361 -x360 -x359 -x358 -x357 -x356 -x355 -x354 -x353 -x352 -x351 -x350 -x349 -x348 -x347 -x346 -x345 x344 -x343 -x342 x341 -x340 -x339 -x338 -x337 -x336 -x335 -x334 -x333 -x332 -x331 -x330 -x329 -x328 -x327 -x326 -x325 -x324 -x323 x322 -x321 -x320 -x319 -x318 -x317 -x316 -x315 -x314 -x313 -x312 -x311 -x310 -x309 -x308 -x307 -x306 -x305 -x304 -x303 -x302 -x301 -x300 -x299 -x298 -x297 -x296 -x295 -x294 -x293 -x292 -x291 -x290 -x289 -x288 -x287 -x286 -x285 -x284 -x283 -x282 -x281 -x280 -x279 -x278 -x277 -x276 -x275 -x274 -x273 -x272 -x271 -x270 x269 -x268 -x267 -x266 -x265 x264 -x263 -x262 -x261 -x260 -x259 -x258 -x257 -x256 -x255 -x254 -x253 -x252 -x251 -x250 -x249 -x248 -x247 -x246 -x245 -x244 -x243 x242 -x241 -x240 -x239 -x238 -x237 -x236 -x235 -x234 -x233 -x232 -x231 -x230 -x229 -x228 -x227 -x226 -x225 -x224 -x223 -x222 -x221 -x220 -x219 -x218 x217 -x216 -x215 -x214 -x213 -x212 -x211 -x210 -x209 -x208 -x207 -x206 -x205 -x204 -x203 -x202 -x201 -x200 -x199 -x198 x197 -x196 -x195 -x194 -x193 -x192 -x191 -x190 -x189 -x188 -x187 -x186 -x185 -x184 -x183 -x182 -x181 -x180 -x179 -x178 -x177 -x176 x175 -x174 -x173 -x172 -x171 -x170 x169 -x168 -x167 -x166 -x165 -x164 -x163 -x162 -x161 -x160 -x159 -x158 -x157 -x156 -x155 -x154 -x153 x152 -x151 -x150 -x149 -x148 -x147 -x146 -x145 -x144 -x143 -x142 -x141 -x140 -x139 -x138 -x137 -x136 -x135 -x134 -x133 -x132 -x131 -x130 -x129 -x128 -x127 -x126 -x125 -x124 x123 -x122 -x121 -x120 -x119 -x118 -x117 -x116 -x115 -x114 -x113 x112 -x111 -x110 -x109 -x108 -x107 -x106 -x105 -x104 -x103 -x102 -x101 -x100 -x99 -x98 -x97 -x96 -x95 x94 -x93 -x92 -x91 -x90 -x89 -x88 -x87 -x86 -x85 -x84 -x83 -x82 -x81 -x80 -x79 -x78 -x77 -x76 -x75 -x74 -x73 -x72 -x71 -x70 -x69 -x68 -x67 -x66 x65 -x64 -x63 -x62 -x61 -x60 -x59 -x58 x57 -x56 -x55 -x54 -x53 -x52 -x51 -x50 -x49 -x48 -x47 -x46 -x45 -x44 -x43 -x42 -x41 -x40 -x39 -x38 -x37 -x36 -x35 x34 -x33 -x32 -x31 -x30 -x29 -x28 -x27 -x26 -x25 -x24 -x23 -x22 -x21 -x20 -x19 -x18 -x17 -x16 -x15 -x14 -x13 -x12 -x11 -x10 -x9 -x8 -x7 -x6 -x5 -x4 -x3 -x2 -x1

Verifier Data 

OK	-37

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3703625-1338091451/watcher-3703625-1338091451 -o /tmp/evaluation-result-3703625-1338091451/solver-3703625-1338091451 -C 1800 -W 1900 -M 15500 pwbo2.0 -file-format=opb -verbosity=0 HOME/instance-3703625-1338091451.opb 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.99 3.95 3.83 4/178 12220
/proc/meminfo: memFree=30933912/32873844 swapFree=7364/7364
[pid=12220] ppid=12218 vsize=5300 CPUtime=0 cores=0,2,4,6
/proc/12220/stat : 12220 (pwbo2.0) R 12218 12220 11438 0 -1 4202496 424 0 0 0 0 0 0 0 20 0 1 0 275273714 5427200 376 33554432000 4194304 5965102 140736280057696 140736280055752 5137146 0 0 4096 16387 0 0 0 17 4 0 0 0 0 0
/proc/12220/statm: 1325 377 169 433 0 646 0

[startup+0.10523 s]
/proc/loadavg: 3.99 3.95 3.83 4/178 12220
/proc/meminfo: memFree=30933912/32873844 swapFree=7364/7364
[pid=12220] ppid=12218 vsize=171868 CPUtime=0.11 cores=0,2,4,6
/proc/12220/stat : 12220 (pwbo2.0) S 12218 12220 11438 0 -1 4202496 6707 0 0 0 10 1 0 0 20 0 3 0 275273714 175992832 6349 33554432000 4194304 5965102 140736280057696 140736280057216 4547341 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/12220/statm: 42967 6355 196 433 0 42531 0
Current children cumulated CPU time (s) 0.11
Current children cumulated vsize (KiB) 171868

[startup+0.200234 s]
/proc/loadavg: 3.99 3.95 3.83 4/178 12220
/proc/meminfo: memFree=30933912/32873844 swapFree=7364/7364
[pid=12220] ppid=12218 vsize=174916 CPUtime=0.29 cores=0,2,4,6
/proc/12220/stat : 12220 (pwbo2.0) S 12218 12220 11438 0 -1 4202496 9566 0 0 0 27 2 0 0 20 0 3 0 275273714 179113984 8856 33554432000 4194304 5965102 140736280057696 140736280057216 4547341 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/12220/statm: 43729 8856 203 433 0 43293 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 174916

[startup+0.300242 s]
/proc/loadavg: 3.99 3.95 3.83 4/178 12220
/proc/meminfo: memFree=30933912/32873844 swapFree=7364/7364
[pid=12220] ppid=12218 vsize=174916 CPUtime=0.48 cores=0,2,4,6
/proc/12220/stat : 12220 (pwbo2.0) S 12218 12220 11438 0 -1 4202496 9906 0 0 0 46 2 0 0 20 0 3 0 275273714 179113984 8870 33554432000 4194304 5965102 140736280057696 140736280057216 4547341 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/12220/statm: 43729 8870 204 433 0 43293 0
Current children cumulated CPU time (s) 0.48
Current children cumulated vsize (KiB) 174916

[startup+0.700175 s]
/proc/loadavg: 3.99 3.95 3.83 4/178 12220
/proc/meminfo: memFree=30933912/32873844 swapFree=7364/7364
[pid=12220] ppid=12218 vsize=174916 CPUtime=1.28 cores=0,2,4,6
/proc/12220/stat : 12220 (pwbo2.0) S 12218 12220 11438 0 -1 4202496 10904 0 0 0 126 2 0 0 20 0 3 0 275273714 179113984 8861 33554432000 4194304 5965102 140736280057696 140736280057216 4547341 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/12220/statm: 43729 8861 205 433 0 43293 0
Current children cumulated CPU time (s) 1.28
Current children cumulated vsize (KiB) 174916

[startup+1.5004 s]
/proc/loadavg: 3.99 3.95 3.83 5/181 12223
/proc/meminfo: memFree=30899300/32873844 swapFree=7364/7364
[pid=12220] ppid=12218 vsize=174916 CPUtime=2.88 cores=0,2,4,6
/proc/12220/stat : 12220 (pwbo2.0) S 12218 12220 11438 0 -1 4202496 12726 0 0 0 285 3 0 0 20 0 3 0 275273714 179113984 8932 33554432000 4194304 5965102 140736280057696 140736280057216 4547341 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/12220/statm: 43729 8932 205 433 0 43293 0
[pid=12220/tid=12222] ppid=12218 vsize=174916 CPUtime=1.4 cores=0,2,4,6
/proc/12220/task/12222/stat : 12222 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 6123 0 0 0 137 3 0 0 20 0 3 0 275273721 179113984 8932 33554432000 4194304 5965102 140736280057696 139910396385688 5131557 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12220/tid=12223] ppid=12218 vsize=174916 CPUtime=1.4 cores=0,2,4,6
/proc/12220/task/12223/stat : 12223 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 3839 0 0 0 139 1 0 0 20 0 3 0 275273721 179113984 8932 33554432000 4194304 5965102 140736280057696 139910385895880 4400955 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 2.88
Current children cumulated vsize (KiB) 174916

[startup+3.1004 s]
/proc/loadavg: 3.99 3.95 3.83 5/181 12223
/proc/meminfo: memFree=30897052/32873844 swapFree=7364/7364
[pid=12220] ppid=12218 vsize=174916 CPUtime=6.07 cores=0,2,4,6
/proc/12220/stat : 12220 (pwbo2.0) S 12218 12220 11438 0 -1 4202496 13450 0 0 0 604 3 0 0 20 0 3 0 275273714 179113984 9656 33554432000 4194304 5965102 140736280057696 140736280057216 4547341 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/12220/statm: 43729 9656 205 433 0 43293 0
[pid=12220/tid=12222] ppid=12218 vsize=174916 CPUtime=2.99 cores=0,2,4,6
/proc/12220/task/12222/stat : 12222 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 6501 0 0 0 296 3 0 0 20 0 3 0 275273721 179113984 9656 33554432000 4194304 5965102 140736280057696 139910396385816 5136313 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12220/tid=12223] ppid=12218 vsize=174916 CPUtime=3 cores=0,2,4,6
/proc/12220/task/12223/stat : 12223 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 4185 0 0 0 299 1 0 0 20 0 3 0 275273721 179113984 9656 33554432000 4194304 5965102 140736280057696 139910385895544 4393958 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 6.07
Current children cumulated vsize (KiB) 174916

[startup+6.3004 s]
/proc/loadavg: 3.99 3.95 3.83 5/181 12223
/proc/meminfo: memFree=30877708/32873844 swapFree=7364/7364
[pid=12220] ppid=12218 vsize=174916 CPUtime=12.47 cores=0,2,4,6
/proc/12220/stat : 12220 (pwbo2.0) S 12218 12220 11438 0 -1 4202496 20096 0 0 0 1241 6 0 0 20 0 3 0 275273714 179113984 13107 33554432000 4194304 5965102 140736280057696 140736280057216 4547341 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/12220/statm: 43729 13108 206 433 0 43293 0
[pid=12220/tid=12222] ppid=12218 vsize=174916 CPUtime=6.19 cores=0,2,4,6
/proc/12220/task/12222/stat : 12222 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 10643 0 0 0 615 4 0 0 20 0 3 0 275273721 179113984 13108 33554432000 4194304 5965102 140736280057696 139910396385816 4420712 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12220/tid=12223] ppid=12218 vsize=174916 CPUtime=6.19 cores=0,2,4,6
/proc/12220/task/12223/stat : 12223 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 6690 0 0 0 617 2 0 0 20 0 3 0 275273721 179113984 13108 33554432000 4194304 5965102 140736280057696 139910385895736 5137076 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 12.47
Current children cumulated vsize (KiB) 174916

[startup+12.7004 s]
/proc/loadavg: 3.99 3.95 3.83 5/181 12223
/proc/meminfo: memFree=30834628/32873844 swapFree=7364/7364
[pid=12220] ppid=12218 vsize=174916 CPUtime=25.26 cores=0,2,4,6
/proc/12220/stat : 12220 (pwbo2.0) S 12218 12220 11438 0 -1 4202496 30689 0 0 0 2515 11 0 0 20 0 3 0 275273714 179113984 20941 33554432000 4194304 5965102 140736280057696 140736280057216 4547341 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/12220/statm: 43729 20941 206 433 0 43293 0
[pid=12220/tid=12222] ppid=12218 vsize=174916 CPUtime=12.57 cores=0,2,4,6
/proc/12220/task/12222/stat : 12222 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 15832 0 0 0 1252 5 0 0 20 0 3 0 275273721 179113984 20941 33554432000 4194304 5965102 140736280057696 139910396385272 4401041 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12220/tid=12223] ppid=12218 vsize=174916 CPUtime=12.56 cores=0,2,4,6
/proc/12220/task/12223/stat : 12223 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 12093 0 0 0 1250 6 0 0 20 0 3 0 275273721 179113984 20941 33554432000 4194304 5965102 140736280057696 139910385895736 4401015 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 25.26
Current children cumulated vsize (KiB) 174916

[startup+25.5004 s]
/proc/loadavg: 3.99 3.95 3.83 5/181 12223
/proc/meminfo: memFree=30777968/32873844 swapFree=7364/7364
[pid=12220] ppid=12218 vsize=241176 CPUtime=50.84 cores=0,2,4,6
/proc/12220/stat : 12220 (pwbo2.0) S 12218 12220 11438 0 -1 4202496 43960 0 0 0 5064 20 0 0 20 0 3 0 275273714 246964224 34206 33554432000 4194304 5965102 140736280057696 140736280057216 4547341 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/12220/statm: 60294 34206 206 433 0 59858 0
[pid=12220/tid=12222] ppid=12218 vsize=241176 CPUtime=25.33 cores=0,2,4,6
/proc/12220/task/12222/stat : 12222 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 20202 0 0 0 2527 6 0 0 20 0 3 0 275273721 246964224 34206 33554432000 4194304 5965102 140736280057696 139910396385816 4401472 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12220/tid=12223] ppid=12218 vsize=241176 CPUtime=25.32 cores=0,2,4,6
/proc/12220/task/12223/stat : 12223 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 20994 0 0 0 2519 13 0 0 20 0 3 0 275273721 246964224 34206 33554432000 4194304 5965102 140736280057696 139910385895736 4400960 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 50.84
Current children cumulated vsize (KiB) 241176

[startup+51.1004 s]
/proc/loadavg: 3.99 3.95 3.84 5/181 12223
/proc/meminfo: memFree=30700792/32873844 swapFree=7364/7364
[pid=12220] ppid=12218 vsize=307472 CPUtime=102.01 cores=0,2,4,6
/proc/12220/stat : 12220 (pwbo2.0) S 12218 12220 11438 0 -1 4202496 62365 0 0 0 10170 31 0 0 20 0 3 0 275273714 314851328 51708 33554432000 4194304 5965102 140736280057696 140736280057216 4547341 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/12220/statm: 76868 51708 206 433 0 76432 0
[pid=12220/tid=12222] ppid=12218 vsize=307472 CPUtime=50.86 cores=0,2,4,6
/proc/12220/task/12222/stat : 12222 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 23325 0 0 0 5079 7 0 0 20 0 3 0 275273721 314851328 51708 33554432000 4194304 5965102 140736280057696 139910396385080 4220672 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12220/tid=12223] ppid=12218 vsize=307472 CPUtime=50.83 cores=0,2,4,6
/proc/12220/task/12223/stat : 12223 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 36276 0 0 0 5060 23 0 0 20 0 3 0 275273721 314851328 51708 33554432000 4194304 5965102 140736280057696 139910385895880 4401475 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 102.01
Current children cumulated vsize (KiB) 307472

[startup+102.3 s]
/proc/loadavg: 3.99 3.96 3.84 5/181 12223
/proc/meminfo: memFree=30591120/32873844 swapFree=7364/7364
[pid=12220] ppid=12218 vsize=439680 CPUtime=204.36 cores=0,2,4,6
/proc/12220/stat : 12220 (pwbo2.0) S 12218 12220 11438 0 -1 4202496 86765 0 0 0 20388 48 0 0 20 0 3 0 275273714 450232320 77134 33554432000 4194304 5965102 140736280057696 140736280057216 4547341 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/12220/statm: 109920 77134 206 433 0 109484 0
[pid=12220/tid=12222] ppid=12218 vsize=439680 CPUtime=101.91 cores=0,2,4,6
/proc/12220/task/12222/stat : 12222 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 32755 0 0 0 10177 14 0 0 20 0 3 0 275273721 450232320 77134 33554432000 4194304 5965102 140736280057696 139910396385080 4220672 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12220/tid=12223] ppid=12218 vsize=439680 CPUtime=101.9 cores=0,2,4,6
/proc/12220/task/12223/stat : 12223 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 51246 0 0 0 10154 36 0 0 20 0 3 0 275273721 450232320 77134 33554432000 4194304 5965102 140736280057696 139910385895880 4401127 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 204.36
Current children cumulated vsize (KiB) 439680

[startup+162.3 s]
/proc/loadavg: 4.04 3.98 3.85 5/181 12223
/proc/meminfo: memFree=30410428/32873844 swapFree=7364/7364
[pid=12220] ppid=12218 vsize=506680 CPUtime=324.33 cores=0,2,4,6
/proc/12220/stat : 12220 (pwbo2.0) S 12218 12220 11438 0 -1 4202496 124923 0 0 0 32353 80 0 0 20 0 3 0 275273714 518840320 115292 33554432000 4194304 5965102 140736280057696 140736280057216 4547341 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/12220/statm: 126670 115292 206 433 0 126234 0
[pid=12220/tid=12222] ppid=12218 vsize=506680 CPUtime=161.75 cores=0,2,4,6
/proc/12220/task/12222/stat : 12222 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 42107 0 0 0 16152 23 0 0 20 0 3 0 275273721 518840320 115292 33554432000 4194304 5965102 140736280057696 139910396385080 4380048 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12220/tid=12223] ppid=12218 vsize=506680 CPUtime=161.73 cores=0,2,4,6
/proc/12220/task/12223/stat : 12223 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 80052 0 0 0 16113 60 0 0 20 0 3 0 275273721 518840320 115292 33554432000 4194304 5965102 140736280057696 139910385895736 4400960 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 324.33
Current children cumulated vsize (KiB) 506680

[startup+222.307 s]
/proc/loadavg: 4.01 3.98 3.86 5/181 12223
/proc/meminfo: memFree=30239212/32873844 swapFree=7364/7364
[pid=12220] ppid=12218 vsize=703692 CPUtime=444.32 cores=0,2,4,6
/proc/12220/stat : 12220 (pwbo2.0) S 12218 12220 11438 0 -1 4202496 152588 0 0 0 44327 105 0 0 20 0 3 0 275273714 720580608 142957 33554432000 4194304 5965102 140736280057696 140736280057216 4547341 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/12220/statm: 175923 142957 206 433 0 175487 0
[pid=12220/tid=12222] ppid=12218 vsize=703692 CPUtime=221.6 cores=0,2,4,6
/proc/12220/task/12222/stat : 12222 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 46648 0 0 0 22133 27 0 0 20 0 3 0 275273721 720580608 142957 33554432000 4194304 5965102 140736280057696 139910396385320 4400960 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12220/tid=12223] ppid=12218 vsize=703692 CPUtime=221.58 cores=0,2,4,6
/proc/12220/task/12223/stat : 12223 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 103176 0 0 0 22075 83 0 0 20 0 3 0 275273721 720580608 142957 33554432000 4194304 5965102 140736280057696 139910385895736 4221120 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 444.32
Current children cumulated vsize (KiB) 703692

[startup+282.3 s]
/proc/loadavg: 4.00 3.98 3.87 5/181 12224
/proc/meminfo: memFree=30160704/32873844 swapFree=7364/7364
[pid=12220] ppid=12218 vsize=704500 CPUtime=564.29 cores=0,2,4,6
/proc/12220/stat : 12220 (pwbo2.0) S 12218 12220 11438 0 -1 4202496 163926 0 0 0 56307 122 0 0 20 0 3 0 275273714 721408000 154295 33554432000 4194304 5965102 140736280057696 140736280057216 4547341 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/12220/statm: 176125 154295 206 433 0 175689 0
[pid=12220/tid=12222] ppid=12218 vsize=704500 CPUtime=281.44 cores=0,2,4,6
/proc/12220/task/12222/stat : 12222 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 55074 0 0 0 28103 41 0 0 20 0 3 0 275273721 721408000 154295 33554432000 4194304 5965102 140736280057696 139910396385080 4401475 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12220/tid=12223] ppid=12218 vsize=704500 CPUtime=281.42 cores=0,2,4,6
/proc/12220/task/12223/stat : 12223 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 106088 0 0 0 28054 88 0 0 20 0 3 0 275273721 721408000 154295 33554432000 4194304 5965102 140736280057696 139910385895496 4401056 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 564.29
Current children cumulated vsize (KiB) 704500

[startup+342.3 s]
/proc/loadavg: 4.00 3.98 3.87 5/181 12224
/proc/meminfo: memFree=30146568/32873844 swapFree=7364/7364
[pid=12220] ppid=12218 vsize=704632 CPUtime=684.28 cores=0,2,4,6
/proc/12220/stat : 12220 (pwbo2.0) S 12218 12220 11438 0 -1 4202496 164877 0 0 0 68305 123 0 0 20 0 3 0 275273714 721543168 155239 33554432000 4194304 5965102 140736280057696 140736280057216 4547341 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/12220/statm: 176158 155239 206 433 0 175722 0

################
# More data... #
################

[pid=12220/tid=12223] ppid=12218 vsize=704632 CPUtime=341.26 cores=0,2,4,6
/proc/12220/task/12223/stat : 12223 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 106153 0 0 0 34037 89 0 0 20 0 3 0 275273721 721543168 155239 33554432000 4194304 5965102 140736280057696 139910385895496 4400960 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 684.28
Current children cumulated vsize (KiB) 704632

[startup+402.3 s]
/proc/loadavg: 4.00 3.98 3.88 5/182 12227
/proc/meminfo: memFree=30034844/32873844 swapFree=7364/7364
[pid=12220] ppid=12218 vsize=836012 CPUtime=804.2 cores=0,2,4,6
/proc/12220/stat : 12220 (pwbo2.0) S 12218 12220 11438 0 -1 4202496 181670 0 0 0 80282 138 0 0 20 0 3 0 275273714 856076288 172387 33554432000 4194304 5965102 140736280057696 140736280057216 4547341 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/12220/statm: 209003 172387 206 433 0 208567 0
[pid=12220/tid=12222] ppid=12218 vsize=836012 CPUtime=401.1 cores=0,2,4,6
/proc/12220/task/12222/stat : 12222 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 61353 0 0 0 40062 48 0 0 20 0 3 0 275273721 856076288 172387 33554432000 4194304 5965102 140736280057696 139910396385272 4401107 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12220/tid=12223] ppid=12218 vsize=836012 CPUtime=401.09 cores=0,2,4,6
/proc/12220/task/12223/stat : 12223 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 117553 0 0 0 40007 102 0 0 20 0 3 0 275273721 856076288 172387 33554432000 4194304 5965102 140736280057696 139910385895736 4401131 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 804.2
Current children cumulated vsize (KiB) 836012

[startup+462.3 s]
/proc/loadavg: 4.00 3.98 3.88 5/181 12227
/proc/meminfo: memFree=29877400/32873844 swapFree=7364/7364
[pid=12220] ppid=12218 vsize=902644 CPUtime=924.15 cores=0,2,4,6
/proc/12220/stat : 12220 (pwbo2.0) S 12218 12220 11438 0 -1 4202496 209080 0 0 0 92233 182 0 0 20 0 3 0 275273714 924307456 199965 33554432000 4194304 5965102 140736280057696 140736280057216 4547341 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/12220/statm: 225661 199965 206 433 0 225225 0
[pid=12220/tid=12222] ppid=12218 vsize=902644 CPUtime=460.94 cores=0,2,4,6
/proc/12220/task/12222/stat : 12222 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 67519 0 0 0 46037 57 0 0 20 0 3 0 275273721 924307456 199965 33554432000 4194304 5965102 140736280057696 139910396385080 4400960 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12220/tid=12223] ppid=12218 vsize=902644 CPUtime=460.89 cores=0,2,4,6
/proc/12220/task/12223/stat : 12223 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 138797 0 0 0 45953 136 0 0 20 0 3 0 275273721 924307456 199965 33554432000 4194304 5965102 140736280057696 139910385895736 4400960 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 924.15
Current children cumulated vsize (KiB) 902644

[startup+522.3 s]
/proc/loadavg: 4.05 3.99 3.89 5/181 12227
/proc/meminfo: memFree=29769176/32873844 swapFree=7364/7364
[pid=12220] ppid=12218 vsize=969200 CPUtime=1044.1 cores=0,2,4,6
/proc/12220/stat : 12220 (pwbo2.0) S 12218 12220 11438 0 -1 4202496 232219 0 0 0 104169 241 0 0 20 0 3 0 275273714 992460800 223072 33554432000 4194304 5965102 140736280057696 140736280057216 4547341 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/12220/statm: 242300 223072 206 433 0 241864 0
[pid=12220/tid=12222] ppid=12218 vsize=969200 CPUtime=520.78 cores=0,2,4,6
/proc/12220/task/12222/stat : 12222 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 73599 0 0 0 52001 77 0 0 20 0 3 0 275273721 992460800 223072 33554432000 4194304 5965102 140736280057696 139910396385320 4401118 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12220/tid=12223] ppid=12218 vsize=969200 CPUtime=520.69 cores=0,2,4,6
/proc/12220/task/12223/stat : 12223 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 155856 0 0 0 51891 178 0 0 20 0 3 0 275273721 992460800 223072 33554432000 4194304 5965102 140736280057696 139910385895736 4400960 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1044.1
Current children cumulated vsize (KiB) 969200

[startup+582.3 s]
/proc/loadavg: 4.02 3.99 3.90 5/181 12228
/proc/meminfo: memFree=29743412/32873844 swapFree=7364/7364
[pid=12220] ppid=12218 vsize=1036992 CPUtime=1164.05 cores=0,2,4,6
/proc/12220/stat : 12220 (pwbo2.0) S 12218 12220 11438 0 -1 4202496 237087 0 0 0 116150 255 0 0 20 0 3 0 275273714 1061879808 228115 33554432000 4194304 5965102 140736280057696 140736280057216 4547341 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/12220/statm: 259248 228115 206 433 0 258812 0
[pid=12220/tid=12222] ppid=12218 vsize=1036992 CPUtime=580.6 cores=0,2,4,6
/proc/12220/task/12222/stat : 12222 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 78442 0 0 0 57968 92 0 0 20 0 3 0 275273721 1061879808 228115 33554432000 4194304 5965102 140736280057696 139910396385080 4401127 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12220/tid=12223] ppid=12218 vsize=1036992 CPUtime=580.52 cores=0,2,4,6
/proc/12220/task/12223/stat : 12223 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 155881 0 0 0 57873 179 0 0 20 0 3 0 275273721 1061879808 228115 33554432000 4194304 5965102 140736280057696 139910385895736 4401137 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1164.05
Current children cumulated vsize (KiB) 1036992

[startup+642.3 s]
/proc/loadavg: 4.00 3.99 3.91 5/181 12228
/proc/meminfo: memFree=29669660/32873844 swapFree=7364/7364
[pid=12220] ppid=12218 vsize=1037128 CPUtime=1284.02 cores=0,2,4,6
/proc/12220/stat : 12220 (pwbo2.0) S 12218 12220 11438 0 -1 4202496 237677 0 0 0 128147 255 0 0 20 0 3 0 275273714 1062019072 228656 33554432000 4194304 5965102 140736280057696 140736280057216 4547341 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/12220/statm: 259282 228656 206 433 0 258846 0
[pid=12220/tid=12222] ppid=12218 vsize=1037128 CPUtime=640.45 cores=0,2,4,6
/proc/12220/task/12222/stat : 12222 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 78946 0 0 0 63952 93 0 0 20 0 3 0 275273721 1062019072 228656 33554432000 4194304 5965102 140736280057696 139910396385080 4400960 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12220/tid=12223] ppid=12218 vsize=1037128 CPUtime=640.32 cores=0,2,4,6
/proc/12220/task/12223/stat : 12223 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 155967 0 0 0 63852 180 0 0 20 0 3 0 275273721 1062019072 228656 33554432000 4194304 5965102 140736280057696 139910385895496 4401155 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1284.02
Current children cumulated vsize (KiB) 1037128

[startup+702.3 s]
/proc/loadavg: 4.00 3.99 3.91 5/181 12269
/proc/meminfo: memFree=30160108/32873844 swapFree=7364/7364
[pid=12220] ppid=12218 vsize=1038356 CPUtime=1403.98 cores=0,2,4,6
/proc/12220/stat : 12220 (pwbo2.0) S 12218 12220 11438 0 -1 4202496 239513 0 0 0 140142 256 0 0 20 0 3 0 275273714 1063276544 230362 33554432000 4194304 5965102 140736280057696 140736280057216 4547341 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/12220/statm: 259589 230362 206 433 0 259153 0
[pid=12220/tid=12222] ppid=12218 vsize=1038356 CPUtime=700.29 cores=0,2,4,6
/proc/12220/task/12222/stat : 12222 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 80706 0 0 0 69935 94 0 0 20 0 3 0 275273721 1063276544 230362 33554432000 4194304 5965102 140736280057696 139910396385080 4400960 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12220/tid=12223] ppid=12218 vsize=1038356 CPUtime=700.13 cores=0,2,4,6
/proc/12220/task/12223/stat : 12223 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 156043 0 0 0 69832 181 0 0 20 0 3 0 275273721 1063276544 230362 33554432000 4194304 5965102 140736280057696 139910385895496 4401137 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1403.98
Current children cumulated vsize (KiB) 1038356

[startup+762.3 s]
/proc/loadavg: 4.00 3.99 3.91 5/181 12269
/proc/meminfo: memFree=30031580/32873844 swapFree=7364/7364
[pid=12220] ppid=12218 vsize=1039944 CPUtime=1523.98 cores=0,2,4,6
/proc/12220/stat : 12220 (pwbo2.0) S 12218 12220 11438 0 -1 4202496 243857 0 0 0 152138 260 0 0 20 0 3 0 275273714 1064902656 234757 33554432000 4194304 5965102 140736280057696 140736280057216 4547341 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/12220/statm: 259986 234757 206 433 0 259550 0
[pid=12220/tid=12222] ppid=12218 vsize=1039944 CPUtime=760.15 cores=0,2,4,6
/proc/12220/task/12222/stat : 12222 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 85004 0 0 0 75916 99 0 0 20 0 3 0 275273721 1064902656 234757 33554432000 4194304 5965102 140736280057696 139910396385320 5137146 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12220/tid=12223] ppid=12218 vsize=1039944 CPUtime=759.98 cores=0,2,4,6
/proc/12220/task/12223/stat : 12223 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 156089 0 0 0 75816 182 0 0 20 0 3 0 275273721 1064902656 234757 33554432000 4194304 5965102 140736280057696 139910385895496 4389776 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1523.98
Current children cumulated vsize (KiB) 1039944

[startup+822.3 s]
/proc/loadavg: 4.00 3.99 3.91 5/181 12269
/proc/meminfo: memFree=29902448/32873844 swapFree=7364/7364
[pid=12220] ppid=12218 vsize=1039944 CPUtime=1643.94 cores=0,2,4,6
/proc/12220/stat : 12220 (pwbo2.0) S 12218 12220 11438 0 -1 4202496 250229 0 0 0 164129 265 0 0 20 0 3 0 275273714 1064902656 241129 33554432000 4194304 5965102 140736280057696 140736280057216 4547341 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/12220/statm: 259986 241129 206 433 0 259550 0
[pid=12220/tid=12222] ppid=12218 vsize=1039944 CPUtime=819.99 cores=0,2,4,6
/proc/12220/task/12222/stat : 12222 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 91375 0 0 0 81893 106 0 0 20 0 3 0 275273721 1064902656 241129 33554432000 4194304 5965102 140736280057696 139910396385272 4400960 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12220/tid=12223] ppid=12218 vsize=1039944 CPUtime=819.82 cores=0,2,4,6
/proc/12220/task/12223/stat : 12223 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 156090 0 0 0 81799 183 0 0 20 0 3 0 275273721 1064902656 241129 33554432000 4194304 5965102 140736280057696 139910385895496 4400955 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1643.94
Current children cumulated vsize (KiB) 1039944

[startup+882.307 s]
/proc/loadavg: 4.00 3.99 3.91 5/181 12270
/proc/meminfo: memFree=29798068/32873844 swapFree=7364/7364
[pid=12220] ppid=12218 vsize=1106340 CPUtime=1763.94 cores=0,2,4,6
/proc/12220/stat : 12220 (pwbo2.0) S 12218 12220 11438 0 -1 4202496 255873 0 0 0 176122 272 0 0 20 0 3 0 275273714 1132892160 246773 33554432000 4194304 5965102 140736280057696 140736280057216 4547341 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/12220/statm: 276585 246773 206 433 0 276149 0
[pid=12220/tid=12222] ppid=12218 vsize=1106340 CPUtime=879.84 cores=0,2,4,6
/proc/12220/task/12222/stat : 12222 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 96996 0 0 0 87870 114 0 0 20 0 3 0 275273721 1132892160 246773 33554432000 4194304 5965102 140736280057696 139910396385272 4221120 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12220/tid=12223] ppid=12218 vsize=1106340 CPUtime=879.67 cores=0,2,4,6
/proc/12220/task/12223/stat : 12223 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 156113 0 0 0 87783 184 0 0 20 0 3 0 275273721 1132892160 246773 33554432000 4194304 5965102 140736280057696 139910385895496 4400960 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1763.94
Current children cumulated vsize (KiB) 1106340



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+900.4 s]
/proc/loadavg: 4.00 3.99 3.91 5/181 12270
/proc/meminfo: memFree=29791836/32873844 swapFree=7364/7364
[pid=12220] ppid=12218 vsize=1107064 CPUtime=1800.13 cores=0,2,4,6
/proc/12220/stat : 12220 (pwbo2.0) S 12218 12220 11438 0 -1 4202496 257358 0 0 0 179739 274 0 0 20 0 3 0 275273714 1133633536 248258 33554432000 4194304 5965102 140736280057696 140736280057216 4547341 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/12220/statm: 276766 248258 206 433 0 276330 0
[pid=12220/tid=12222] ppid=12218 vsize=1107064 CPUtime=897.89 cores=0,2,4,6
/proc/12220/task/12222/stat : 12222 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 98465 0 0 0 89673 116 0 0 20 0 3 0 275273721 1133633536 248258 33554432000 4194304 5965102 140736280057696 139910396385272 4400960 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12220/tid=12223] ppid=12218 vsize=1107064 CPUtime=897.72 cores=0,2,4,6
/proc/12220/task/12223/stat : 12223 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 156129 0 0 0 89588 184 0 0 20 0 3 0 275273721 1133633536 248258 33554432000 4194304 5965102 140736280057696 139910385895496 4401107 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1800.13
Current children cumulated vsize (KiB) 1107064

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+900.4 s]
/proc/loadavg: 4.00 3.99 3.91 5/181 12270
/proc/meminfo: memFree=29791836/32873844 swapFree=7364/7364
[pid=12220] ppid=12218 vsize=1107064 CPUtime=1800.13 cores=0,2,4,6
/proc/12220/stat : 12220 (pwbo2.0) S 12218 12220 11438 0 -1 4202496 257358 0 0 0 179739 274 0 0 20 0 3 0 275273714 1133633536 248258 33554432000 4194304 5965102 140736280057696 140736280057216 4547341 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/12220/statm: 276766 248258 206 433 0 276330 0
[pid=12220/tid=12222] ppid=12218 vsize=1107064 CPUtime=897.89 cores=0,2,4,6
/proc/12220/task/12222/stat : 12222 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 98465 0 0 0 89673 116 0 0 20 0 3 0 275273721 1133633536 248258 33554432000 4194304 5965102 140736280057696 139910396385272 4400960 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=12220/tid=12223] ppid=12218 vsize=1107064 CPUtime=897.72 cores=0,2,4,6
/proc/12220/task/12223/stat : 12223 (pwbo2.0) R 12218 12220 11438 0 -1 4202560 156129 0 0 0 89588 184 0 0 20 0 3 0 275273721 1133633536 248258 33554432000 4194304 5965102 140736280057696 139910385895496 4401107 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1800.13
Current children cumulated vsize (KiB) 1107064

Child status: 0

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 12220 and gives
#  childrusage.ru_utime.tv_sec=1792
#  childrusage.ru_utime.tv_usec=711466
#  childrusage.ru_stime.tv_sec=3
#  childrusage.ru_stime.tv_usec=40537
# CPU time returned by wait4() is 1795.75
# while last known CPU time is 1800.13
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 900.437
CPU time (s): 1800.13
CPU user time (s): 1797.39
CPU system time (s): 2.74
CPU usage (%): 199.917
Max. virtual memory (cumulated for all children) (KiB): 1107064

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1792.71
system time used= 3.04054
maximum resident set size= 993048
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 257362
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1370
involuntary context switches= 1945

runsolver used 1.84072 second user time and 5.14522 second system time

The end

Launcher Data 

Begin job on node109 at 2012-05-27 06:04:11
IDJOB=3703625
IDBENCH=2244
IDSOLVER=2279
FILE ID=node109/3703625-1338091451
RUNJOBID= node109-1338088487-11454
PBS_JOBID= 14624462
Free space on /tmp= 71344 MiB

SOLVER NAME= pwbo 2.0
BENCH NAME= PB06//final/normalized-PB06/OPT-SMALLINT/web/www.nlsde.buaa.edu.cn/~kexu/benchmarks/frb40-19-opb/normalized-frb40-19-4.opb
COMMAND LINE= pwbo2.0 -file-format=opb -verbosity=0 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3703625-1338091451/watcher-3703625-1338091451 -o /tmp/evaluation-result-3703625-1338091451/solver-3703625-1338091451 -C 1800 -W 1900 -M 15500  pwbo2.0 -file-format=opb -verbosity=0 HOME/instance-3703625-1338091451.opb

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 7262f9f4ab1a3ada75b3b140acf97f7d
RANDOM SEED=890074155

node109.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.264
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.52
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.264
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.264
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.264
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.264
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.264
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.264
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.264
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        30934812 kB
Buffers:          249396 kB
Cached:           544084 kB
SwapCached:            0 kB
Active:          1025420 kB
Inactive:         121464 kB
Active(anon):     345272 kB
Inactive(anon):    10916 kB
Active(file):     680148 kB
Inactive(file):   110548 kB
Unevictable:        7364 kB
Mlocked:            7364 kB
SwapTotal:      67108856 kB
SwapFree:       67108856 kB
Dirty:              6224 kB
Writeback:             0 kB
AnonPages:        331964 kB
Mapped:            15092 kB
Shmem:               200 kB
Slab:             650528 kB
SReclaimable:      92940 kB
SUnreclaim:       557588 kB
KernelStack:        1504 kB
PageTables:         4836 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     163524 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:    174080 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 71336 MiB
End job on node109 at 2012-05-27 06:19:14