Trace number 3725859

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
pwbo 2.02SAT (TO)68 1800.16 900.32

General information on the benchmark

Namenormalized-PB06/OPT-SMALLINT/submitted-PB05/
manquinho/ttp/normalized-circ6_3.opb
MD5SUMff9be653d1d3ea3b1a82846016e94df6
Bench CategoryOPT-SMALLINT (optimisation, small integers)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark64
Best CPU time to get the best result obtained on this benchmark1800.03
Has Objective FunctionYES
SatisfiableYES
(Un)Satisfiability was provedYES
Best value of the objective function 64
Optimality of the best value was proved NO
Number of variables540
Total number of constraints4476
Number of constraints which are clauses2532
Number of constraints which are cardinality constraints (but not clauses)264
Number of constraints which are nor clauses,nor cardinality constraints1680
Minimum length of a constraint2
Maximum length of a constraint20
Number of terms in the objective function 180
Biggest coefficient in the objective function 3
Number of bits for the biggest coefficient in the objective function 2
Sum of the numbers in the objective function 324
Number of bits of the sum of numbers in the objective function 9
Biggest number in a constraint 16
Number of bits of the biggest number in a constraint 5
Biggest sum of numbers in a constraint 324
Number of bits of the biggest sum of numbers9
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Quality of the solution as a function of time


Download the above graph as a PDF file / as an EPS file

Solver Data 

0.00/0.00	c ==============================[ Solver Version ]===============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  pwbo:                     2.02 beta - May, 2012                            |
0.00/0.00	c |                                                                             |
0.00/0.00	c Parsing opb file format. File HOME/instance-3725859-1338653236.opb.
0.00/0.00	c Instance file HOME/instance-3725859-1338653236.opb
0.00/0.00	c File size is 202558 bytes.
0.00/0.02	c Cost Offset: 0
0.00/0.02	c Highest Coefficient sum: 40
0.00/0.02	c ============================[ Problem Statistics ]=============================
0.00/0.02	c |                                                                             |
0.00/0.02	c |  Parsing time:          0.02         s                                      |
0.00/0.02	c |  Number Variables:      540                                                 |
0.00/0.02	c |  Number Clauses:        2682                                                |
0.00/0.02	c |  Number Cardinality:    264                                                 |
0.00/0.02	c |  Number PB Constraints: 1680                                                |
0.00/0.02	c |                                                                             |
0.00/0.02	c |  Problem Type:          Optimization                                        |
0.00/0.02	c |  Objective Function:    Cardinality Constraint                              |
0.00/0.02	c |                                                                             |
0.00/0.02	c ===============================================================================
1800.16/900.31	o 94
1800.16/900.31	o 92
1800.16/900.31	o 80
1800.16/900.31	o 78
1800.16/900.31	o 76
1800.16/900.31	o 74
1800.16/900.31	o 72
1800.16/900.31	o 70
1800.16/900.31	o 68
1800.16/900.31	c ===============================================================================
1800.16/900.31	c Wall Clock Time: 900.31 s
1800.16/900.31	c Best Solution: 68
1800.16/900.31	s SATISFIABLE
1800.16/900.31	v x361 -x362 -x363 -x364 x365 -x366 x367 -x368 -x369 -x370 x371 -x372 -x373 -x374 -x375 x376 -x377 -x378 -x379 -x380 -x381 -x382 -x383 x384 -x385 -x386 -x387 -x388 -x389 x390 x391 -x392 -x393 -x394 -x395 -x396 x397 x398 -x399 x400 x401 -x402 -x403 -x404 -x405 -x406 x407 -x408 -x409 -x410 -x411 x412 -x413 -x414 -x415 -x416 -x417 -x418 -x419 x420 -x421 -x422 -x423 -x424 x425 x426 -x427 -x428 -x429 -x430 -x431 x432 x433 x434 -x435 -x436 -x437 x438 -x439 -x440 -x441 -x442 x443 -x444 -x445 -x446 -x447 x448 -x449 -x450 -x451 -x452 x453 -x454 -x455 -x456 x457 -x458 -x459 -x460 -x461 -x462 x463 -x464 -x465 -x466 x467 -x468 x469 -x470 -x471 -x472 -x473 -x474 x475 x476 -x477 -x478 -x479 -x480 -x481 -x482 -x483 -x484 x485 -x486 x487 -x488 -x489 -x490 -x491 -x492 x493 -x494 -x495 -x496 -x497 -x498 x499 -x500 x501 x502 -x503 -x504 -x505 -x506 -x507 -x508 -x509 x510 -x511 -x512 -x513 x514 -x515 x516 -x517 -x518 -x519 -x520 -x521 -x522 x523 -x524 -x525 -x526 -x527 -x528 -x529 x530 -x531 -x532 -x533 -x534 x535 -x536 x537 x538 -x539 -x540 -x2 -x38 -x74 -x110 -x146 x182 -x218 -x254 -x290 -x326 -x3 -x39 -x75 -x111 -x147 -x183 x219 -x255 -x291 -x327 -x4 -x40 x76 -x112 -x148 -x184 -x220 -x256 -x292 -x328 -x5 x41 -x77 -x113 -x149 -x185 -x221 -x257 -x293 -x329 x6 -x42 -x78 -x114 -x150 -x186 -x222 -x258 -x294 -x330 -x7 -x43 -x79 -x115 -x151 -x187 -x223 -x259 -x295 x331 -x9 -x45 -x81 -x117 -x153 -x189 -x225 -x261 x297 -x333 -x10 -x46 -x82 -x118 x154 -x190 -x226 -x262 -x298 -x334 -x11 -x47 x83 -x119 -x155 -x191 -x227 -x263 -x299 -x335 -x12 x48 -x84 -x120 -x156 -x192 -x228 -x264 -x300 -x336 -x13 -x49 -x85 -x121 x157 -x193 -x229 -x265 -x301 -x337 -x14 -x50 -x86 x122 -x158 -x194 -x230 -x266 -x302 -x338 -x16 -x52 -x88 -x124 -x160 -x196 -x232 -x268 -x304 x340 x17 -x53 -x89 -x125 -x161 -x197 -x233 -x269 -x305 -x341 -x18 -x54 -x90 -x126 -x162 x198 -x234 -x270 -x306 -x342 -x19 -x55 -x91 -x127 -x163 -x199 -x235 x271 -x307 -x343 x20 -x56 -x92 -x128 -x164 -x200 -x236 -x272 -x308 -x344 -x21 x57 -x93 -x129 -x165 -x201 -x237 -x273 -x309 -x345 -x23 -x59 -x95 -x131 -x167 x203 -x239 -x275 -x311 -x347 -x24 -x60 -x96 -x132 -x168 -x204 x240 -x276 -x312 -x348 -x25 -x61 -x97 x133 -x169 -x205 -x241 -x277 -x313 -x349 -x26 -x62 -x98 -x134 -x170 -x206 x242 -x278 -x314 -x350 -x27 -x63 -x99 -x135 -x171 -x207 -x243 x279 -x315 -x351 -x28 -x64 -x100 -x136 -x172 -x208 -x244 -x280 x316 -x352 -x30 -x66 -x102 -x138 x174 -x210 -x246 -x282 -x318 -x354 -x31 -x67 -x103 -x139 -x175 -x211 -x247 -x283 x319 -x355 -x32 -x68 -x104 -x140 -x176 -x212 -x248 x284 -x320 -x356 -x33 -x69 x105 -x141 -x177 -x213 -x249 -x285 -x321 -x357 -x34 -x70 -x106 x142 -x178 -x214 -x250 -x286 -x322 -x358 -x35 -x71 -x107 -x143 -x179 -x215 -x251 -x287 -x323 x359 x8 -x44 -x80 x116 -x152 x188 x224 x260 -x296 -x332 -x15 x51 x87 -x123 -x159 -x195 x231 x267 x303 -x339 -x22 -x58 x94 x130 x166 -x202 -x238 -x274 x310 x346 x29 x65 x101 -x137 -x173 x209 -x245 -x281 -x317 x353 x36 x72 -x108 -x144 x180 x216 x252 -x288 -x324 -x360 -x1 -x37 -x73 x109 x145 -x181 -x217 x253 x289 x325

Verifier Data 

OK	68

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3725859-1338653236/watcher-3725859-1338653236 -o /tmp/evaluation-result-3725859-1338653236/solver-3725859-1338653236 -C 1800 -W 1900 -M 15500 pwbo2.02 -file-format=opb -verbosity=0 HOME/instance-3725859-1338653236.opb 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.00 3.98 3.58 2/171 29863
/proc/meminfo: memFree=27424156/32873844 swapFree=6608/6608
[pid=29863] ppid=29861 vsize=4556 CPUtime=0 cores=0,2,4,6
/proc/29863/stat : 29863 (pwbo2.02) R 29861 29863 28266 0 -1 4202496 423 0 0 0 0 0 0 0 20 0 1 0 331448414 4665344 376 33554432000 4194304 5964478 140735523478688 140734063763800 5136490 0 0 4096 16387 0 0 0 17 0 0 0 0 0 0
/proc/29863/statm: 1139 376 169 433 0 653 0

[startup+0.09395 s]
/proc/loadavg: 4.00 3.98 3.58 2/171 29863
/proc/meminfo: memFree=27424156/32873844 swapFree=6608/6608
[pid=29863] ppid=29861 vsize=166524 CPUtime=0.15 cores=0,2,4,6
/proc/29863/stat : 29863 (pwbo2.02) S 29861 29863 28266 0 -1 4202496 3315 0 0 0 14 1 0 0 20 0 3 0 331448414 170520576 3174 33554432000 4194304 5964478 140735523478688 140735523478208 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/29863/statm: 41631 3174 207 433 0 41195 0
Current children cumulated CPU time (s) 0.15
Current children cumulated vsize (KiB) 166524

[startup+0.100203 s]
/proc/loadavg: 4.00 3.98 3.58 2/171 29863
/proc/meminfo: memFree=27424156/32873844 swapFree=6608/6608
[pid=29863] ppid=29861 vsize=166524 CPUtime=0.17 cores=0,2,4,6
/proc/29863/stat : 29863 (pwbo2.02) S 29861 29863 28266 0 -1 4202496 3370 0 0 0 16 1 0 0 20 0 3 0 331448414 170520576 3229 33554432000 4194304 5964478 140735523478688 140735523478208 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/29863/statm: 41631 3229 207 433 0 41195 0
Current children cumulated CPU time (s) 0.17
Current children cumulated vsize (KiB) 166524

[startup+0.300153 s]
/proc/loadavg: 4.00 3.98 3.58 2/171 29863
/proc/meminfo: memFree=27424156/32873844 swapFree=6608/6608
[pid=29863] ppid=29861 vsize=166524 CPUtime=0.57 cores=0,2,4,6
/proc/29863/stat : 29863 (pwbo2.02) S 29861 29863 28266 0 -1 4202496 4393 0 0 0 55 2 0 0 20 0 3 0 331448414 170520576 4252 33554432000 4194304 5964478 140735523478688 140735523478208 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/29863/statm: 41631 4252 207 433 0 41195 0
Current children cumulated CPU time (s) 0.57
Current children cumulated vsize (KiB) 166524

[startup+0.700147 s]
/proc/loadavg: 4.00 3.98 3.58 2/171 29863
/proc/meminfo: memFree=27424156/32873844 swapFree=6608/6608
[pid=29863] ppid=29861 vsize=166524 CPUtime=1.36 cores=0,2,4,6
/proc/29863/stat : 29863 (pwbo2.02) S 29861 29863 28266 0 -1 4202496 5132 0 0 0 134 2 0 0 20 0 3 0 331448414 170520576 4991 33554432000 4194304 5964478 140735523478688 140735523478208 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/29863/statm: 41631 4991 207 433 0 41195 0
Current children cumulated CPU time (s) 1.36
Current children cumulated vsize (KiB) 166524

[startup+1.50032 s]
/proc/loadavg: 4.00 3.98 3.58 5/181 29884
/proc/meminfo: memFree=27373052/32873844 swapFree=6608/6608
[pid=29863] ppid=29861 vsize=166800 CPUtime=2.96 cores=0,2,4,6
/proc/29863/stat : 29863 (pwbo2.02) S 29861 29863 28266 0 -1 4202496 6296 0 0 0 293 3 0 0 20 0 3 0 331448414 170803200 6155 33554432000 4194304 5964478 140735523478688 140735523478208 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/29863/statm: 41700 6155 207 433 0 41264 0
[pid=29863/tid=29866] ppid=29861 vsize=166800 CPUtime=1.46 cores=0,2,4,6
/proc/29863/task/29866/stat : 29866 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 2906 0 0 0 143 3 0 0 20 0 3 0 331448416 170803200 6155 33554432000 4194304 5964478 140735523478688 140384755922928 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=29863/tid=29867] ppid=29861 vsize=166800 CPUtime=1.45 cores=0,2,4,6
/proc/29863/task/29867/stat : 29867 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 2014 0 0 0 145 0 0 0 20 0 3 0 331448416 170803200 6155 33554432000 4194304 5964478 140735523478688 140384745433208 5136420 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 2.96
Current children cumulated vsize (KiB) 166800

[startup+3.10033 s]
/proc/loadavg: 4.00 3.98 3.58 5/181 29884
/proc/meminfo: memFree=27333728/32873844 swapFree=6608/6608
[pid=29863] ppid=29861 vsize=167412 CPUtime=6.15 cores=0,2,4,6
/proc/29863/stat : 29863 (pwbo2.02) S 29861 29863 28266 0 -1 4202496 8243 0 0 0 611 4 0 0 20 0 3 0 331448414 171429888 8102 33554432000 4194304 5964478 140735523478688 140735523478208 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/29863/statm: 41853 8102 207 433 0 41417 0
[pid=29863/tid=29866] ppid=29861 vsize=167412 CPUtime=3.05 cores=0,2,4,6
/proc/29863/task/29866/stat : 29866 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 3887 0 0 0 302 3 0 0 20 0 3 0 331448416 171429888 8102 33554432000 4194304 5964478 140735523478688 140384755922984 4221120 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=29863/tid=29867] ppid=29861 vsize=167412 CPUtime=3.05 cores=0,2,4,6
/proc/29863/task/29867/stat : 29867 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 2980 0 0 0 304 1 0 0 20 0 3 0 331448416 171429888 8102 33554432000 4194304 5964478 140735523478688 140384745433400 5136420 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 6.15
Current children cumulated vsize (KiB) 167412

[startup+6.30033 s]
/proc/loadavg: 4.00 3.98 3.58 5/181 29884
/proc/meminfo: memFree=27266396/32873844 swapFree=6608/6608
[pid=29863] ppid=29861 vsize=167748 CPUtime=12.55 cores=0,2,4,6
/proc/29863/stat : 29863 (pwbo2.02) S 29861 29863 28266 0 -1 4202496 10478 0 0 0 1250 5 0 0 20 0 3 0 331448414 171773952 10337 33554432000 4194304 5964478 140735523478688 140735523478208 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/29863/statm: 41937 10337 207 433 0 41501 0
[pid=29863/tid=29866] ppid=29861 vsize=167748 CPUtime=6.25 cores=0,2,4,6
/proc/29863/task/29866/stat : 29866 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 5541 0 0 0 622 3 0 0 20 0 3 0 331448416 171773952 10337 33554432000 4194304 5964478 140735523478688 140384755922744 4400557 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=29863/tid=29867] ppid=29861 vsize=167748 CPUtime=6.24 cores=0,2,4,6
/proc/29863/task/29867/stat : 29867 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 3561 0 0 0 623 1 0 0 20 0 3 0 331448416 171773952 10337 33554432000 4194304 5964478 140735523478688 140384745433312 4392829 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 12.55
Current children cumulated vsize (KiB) 167748

[startup+12.7003 s]
/proc/loadavg: 4.00 3.98 3.59 5/181 29884
/proc/meminfo: memFree=27169964/32873844 swapFree=6608/6608
[pid=29863] ppid=29861 vsize=168252 CPUtime=25.35 cores=0,2,4,6
/proc/29863/stat : 29863 (pwbo2.02) S 29861 29863 28266 0 -1 4202496 13479 0 0 0 2528 7 0 0 20 0 3 0 331448414 172290048 13688 33554432000 4194304 5964478 140735523478688 140735523478208 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/29863/statm: 42063 13688 207 433 0 41627 0
[pid=29863/tid=29866] ppid=29861 vsize=168252 CPUtime=12.63 cores=0,2,4,6
/proc/29863/task/29866/stat : 29866 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 7629 0 0 0 1259 4 0 0 20 0 3 0 331448416 172290048 13688 33554432000 4194304 5964478 140735523478688 140384755922744 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=29863/tid=29867] ppid=29861 vsize=168252 CPUtime=12.62 cores=0,2,4,6
/proc/29863/task/29867/stat : 29867 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 4474 0 0 0 1260 2 0 0 20 0 3 0 331448416 172290048 13688 33554432000 4194304 5964478 140735523478688 140384745433400 5136420 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 25.35
Current children cumulated vsize (KiB) 168252

[startup+25.5003 s]
/proc/loadavg: 4.00 3.98 3.59 5/181 29884
/proc/meminfo: memFree=27045840/32873844 swapFree=6608/6608
[pid=29863] ppid=29861 vsize=169056 CPUtime=50.93 cores=0,2,4,6
/proc/29863/stat : 29863 (pwbo2.02) S 29861 29863 28266 0 -1 4202496 17224 0 0 0 5084 9 0 0 20 0 3 0 331448414 173113344 17427 33554432000 4194304 5964478 140735523478688 140735523478208 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/29863/statm: 42264 17427 207 433 0 41828 0
[pid=29863/tid=29866] ppid=29861 vsize=169056 CPUtime=25.39 cores=0,2,4,6
/proc/29863/task/29866/stat : 29866 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 10240 0 0 0 2532 7 0 0 20 0 3 0 331448416 173113344 17427 33554432000 4194304 5964478 140735523478688 140384755922744 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=29863/tid=29867] ppid=29861 vsize=169056 CPUtime=25.38 cores=0,2,4,6
/proc/29863/task/29867/stat : 29867 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 5608 0 0 0 2536 2 0 0 20 0 3 0 331448416 173113344 17427 33554432000 4194304 5964478 140735523478688 140384745433160 4399915 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 50.93
Current children cumulated vsize (KiB) 169056

[startup+51.1003 s]
/proc/loadavg: 4.00 3.98 3.60 5/181 29884
/proc/meminfo: memFree=26975612/32873844 swapFree=6608/6608
[pid=29863] ppid=29861 vsize=170288 CPUtime=102.1 cores=0,2,4,6
/proc/29863/stat : 29863 (pwbo2.02) S 29861 29863 28266 0 -1 4202496 24840 0 0 0 10195 15 0 0 20 0 3 0 331448414 174374912 25052 33554432000 4194304 5964478 140735523478688 140735523478208 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/29863/statm: 42572 25052 207 433 0 42136 0
[pid=29863/tid=29866] ppid=29861 vsize=170288 CPUtime=50.9 cores=0,2,4,6
/proc/29863/task/29866/stat : 29866 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 14976 0 0 0 5080 10 0 0 20 0 3 0 331448416 174374912 25052 33554432000 4194304 5964478 140735523478688 140384755922984 4400042 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=29863/tid=29867] ppid=29861 vsize=170288 CPUtime=50.92 cores=0,2,4,6
/proc/29863/task/29867/stat : 29867 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 8488 0 0 0 5088 4 0 0 20 0 3 0 331448416 174374912 25052 33554432000 4194304 5964478 140735523478688 140384745433160 4221120 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 102.1
Current children cumulated vsize (KiB) 170288

[startup+102.307 s]
/proc/loadavg: 4.00 3.98 3.62 5/181 29885
/proc/meminfo: memFree=26869092/32873844 swapFree=6608/6608
[pid=29863] ppid=29861 vsize=237580 CPUtime=204.5 cores=0,2,4,6
/proc/29863/stat : 29863 (pwbo2.02) S 29861 29863 28266 0 -1 4202496 35851 0 0 0 20424 26 0 0 20 0 3 0 331448414 243281920 37222 33554432000 4194304 5964478 140735523478688 140735523478208 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/29863/statm: 59395 37222 207 433 0 58959 0
[pid=29863/tid=29866] ppid=29861 vsize=237580 CPUtime=101.98 cores=0,2,4,6
/proc/29863/task/29866/stat : 29866 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 21322 0 0 0 10180 18 0 0 20 0 3 0 331448416 243281920 37222 33554432000 4194304 5964478 140735523478688 140384755922744 4399912 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=29863/tid=29867] ppid=29861 vsize=237580 CPUtime=101.99 cores=0,2,4,6
/proc/29863/task/29867/stat : 29867 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 13153 0 0 0 10190 9 0 0 20 0 3 0 331448416 243281920 37222 33554432000 4194304 5964478 140735523478688 140384745433400 4399915 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 204.5
Current children cumulated vsize (KiB) 237580

[startup+162.3 s]
/proc/loadavg: 4.00 3.98 3.64 5/181 29885
/proc/meminfo: memFree=26832912/32873844 swapFree=6608/6608
[pid=29863] ppid=29861 vsize=239504 CPUtime=324.47 cores=0,2,4,6
/proc/29863/stat : 29863 (pwbo2.02) S 29861 29863 28266 0 -1 4202496 43339 0 0 0 32414 33 0 0 20 0 3 0 331448414 245252096 44710 33554432000 4194304 5964478 140735523478688 140735523478208 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/29863/statm: 59876 44710 207 433 0 59440 0
[pid=29863/tid=29866] ppid=29861 vsize=239504 CPUtime=161.82 cores=0,2,4,6
/proc/29863/task/29866/stat : 29866 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 27276 0 0 0 16158 24 0 0 20 0 3 0 331448416 245252096 44710 33554432000 4194304 5964478 140735523478688 140384755922984 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=29863/tid=29867] ppid=29861 vsize=239504 CPUtime=161.83 cores=0,2,4,6
/proc/29863/task/29867/stat : 29867 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 14687 0 0 0 16171 12 0 0 20 0 3 0 331448416 245252096 44710 33554432000 4194304 5964478 140735523478688 140384745433400 4400078 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 324.47
Current children cumulated vsize (KiB) 239504

[startup+222.3 s]
/proc/loadavg: 4.00 3.98 3.66 5/182 29888
/proc/meminfo: memFree=26761880/32873844 swapFree=6608/6608
[pid=29863] ppid=29861 vsize=240916 CPUtime=444.45 cores=0,2,4,6
/proc/29863/stat : 29863 (pwbo2.02) S 29861 29863 28266 0 -1 4202496 51207 0 0 0 44403 42 0 0 20 0 3 0 331448414 246697984 52578 33554432000 4194304 5964478 140735523478688 140735523478208 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/29863/statm: 60229 52578 207 433 0 59793 0
[pid=29863/tid=29866] ppid=29861 vsize=240916 CPUtime=221.66 cores=0,2,4,6
/proc/29863/task/29866/stat : 29866 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 31469 0 0 0 22136 30 0 0 20 0 3 0 331448416 246697984 52578 33554432000 4194304 5964478 140735523478688 140384755922744 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=29863/tid=29867] ppid=29861 vsize=240916 CPUtime=221.68 cores=0,2,4,6
/proc/29863/task/29867/stat : 29867 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 18362 0 0 0 22151 17 0 0 20 0 3 0 331448416 246697984 52578 33554432000 4194304 5964478 140735523478688 140384745433160 4392918 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 444.45
Current children cumulated vsize (KiB) 240916

[startup+282.3 s]
/proc/loadavg: 4.00 3.98 3.68 5/181 29888
/proc/meminfo: memFree=26701156/32873844 swapFree=6608/6608
[pid=29863] ppid=29861 vsize=373300 CPUtime=564.44 cores=0,2,4,6
/proc/29863/stat : 29863 (pwbo2.02) S 29861 29863 28266 0 -1 4202496 57395 0 0 0 56392 52 0 0 20 0 3 0 331448414 382259200 58766 33554432000 4194304 5964478 140735523478688 140735523478208 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/29863/statm: 93325 58766 207 433 0 92889 0
[pid=29863/tid=29866] ppid=29861 vsize=373300 CPUtime=281.51 cores=0,2,4,6
/proc/29863/task/29866/stat : 29866 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 33069 0 0 0 28119 32 0 0 20 0 3 0 331448416 382259200 58766 33554432000 4194304 5964478 140735523478688 140384755922744 4221120 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=29863/tid=29867] ppid=29861 vsize=373300 CPUtime=281.52 cores=0,2,4,6
/proc/29863/task/29867/stat : 29867 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 22950 0 0 0 28125 27 0 0 20 0 3 0 331448416 382259200 58766 33554432000 4194304 5964478 140735523478688 140384745433448 4400078 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 564.44
Current children cumulated vsize (KiB) 373300

[startup+342.3 s]
/proc/loadavg: 4.12 4.01 3.71 5/181 29888
/proc/meminfo: memFree=26641872/32873844 swapFree=6608/6608
[pid=29863] ppid=29861 vsize=375564 CPUtime=684.4 cores=0,2,4,6
/proc/29863/stat : 29863 (pwbo2.02) S 29861 29863 28266 0 -1 4202496 67067 0 0 0 68367 73 0 0 20 0 3 0 331448414 384577536 68510 33554432000 4194304 5964478 140735523478688 140735523478208 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/29863/statm: 93891 68510 207 433 0 93455 0

################
# More data... #
################

[pid=29863/tid=29867] ppid=29861 vsize=375564 CPUtime=341.36 cores=0,2,4,6
/proc/29863/task/29867/stat : 29867 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 26778 0 0 0 34101 35 0 0 20 0 3 0 331448416 384577536 68510 33554432000 4194304 5964478 140735523478688 140384745433400 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 684.4
Current children cumulated vsize (KiB) 375564

[startup+402.3 s]
/proc/loadavg: 4.04 4.00 3.72 5/181 29888
/proc/meminfo: memFree=26597276/32873844 swapFree=6608/6608
[pid=29863] ppid=29861 vsize=376312 CPUtime=804.35 cores=0,2,4,6
/proc/29863/stat : 29863 (pwbo2.02) S 29861 29863 28266 0 -1 4202496 73895 0 0 0 80343 92 0 0 20 0 3 0 331448414 385343488 75427 33554432000 4194304 5964478 140735523478688 140735523478208 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/29863/statm: 94078 75427 207 433 0 93642 0
[pid=29863/tid=29866] ppid=29861 vsize=376312 CPUtime=401.19 cores=0,2,4,6
/proc/29863/task/29866/stat : 29866 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 45569 0 0 0 40052 67 0 0 20 0 3 0 331448416 385343488 75427 33554432000 4194304 5964478 140735523478688 140384755922936 4400432 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=29863/tid=29867] ppid=29861 vsize=376312 CPUtime=401.19 cores=0,2,4,6
/proc/29863/task/29867/stat : 29867 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 26950 0 0 0 40083 36 0 0 20 0 3 0 331448416 385343488 75427 33554432000 4194304 5964478 140735523478688 140384745433160 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 804.35
Current children cumulated vsize (KiB) 376312

[startup+462.3 s]
/proc/loadavg: 4.01 4.00 3.74 5/181 29889
/proc/meminfo: memFree=26578360/32873844 swapFree=6608/6608
[pid=29863] ppid=29861 vsize=377696 CPUtime=924.31 cores=0,2,4,6
/proc/29863/stat : 29863 (pwbo2.02) S 29861 29863 28266 0 -1 4202496 76346 0 0 0 92334 97 0 0 20 0 3 0 331448414 386760704 78018 33554432000 4194304 5964478 140735523478688 140735523478208 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/29863/statm: 94424 78018 207 433 0 93988 0
[pid=29863/tid=29866] ppid=29861 vsize=377696 CPUtime=461.01 cores=0,2,4,6
/proc/29863/task/29866/stat : 29866 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 46025 0 0 0 46032 69 0 0 20 0 3 0 331448416 386760704 78018 33554432000 4194304 5964478 140735523478688 140384755922936 4399932 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=29863/tid=29867] ppid=29861 vsize=377696 CPUtime=461.03 cores=0,2,4,6
/proc/29863/task/29867/stat : 29867 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 28945 0 0 0 46063 40 0 0 20 0 3 0 331448416 386760704 78018 33554432000 4194304 5964478 140735523478688 140384745433400 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 924.31
Current children cumulated vsize (KiB) 377696

[startup+522.3 s]
/proc/loadavg: 4.00 4.00 3.75 5/181 29889
/proc/meminfo: memFree=26540568/32873844 swapFree=6608/6608
[pid=29863] ppid=29861 vsize=379556 CPUtime=1044.28 cores=0,2,4,6
/proc/29863/stat : 29863 (pwbo2.02) S 29861 29863 28266 0 -1 4202496 82390 0 0 0 104322 106 0 0 20 0 3 0 331448414 388665344 84059 33554432000 4194304 5964478 140735523478688 140735523478208 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/29863/statm: 94889 84059 207 433 0 94453 0
[pid=29863/tid=29866] ppid=29861 vsize=379556 CPUtime=520.85 cores=0,2,4,6
/proc/29863/task/29866/stat : 29866 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 46510 0 0 0 52015 70 0 0 20 0 3 0 331448416 388665344 84059 33554432000 4194304 5964478 140735523478688 140384755922984 4403348 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=29863/tid=29867] ppid=29861 vsize=379556 CPUtime=520.87 cores=0,2,4,6
/proc/29863/task/29867/stat : 29867 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 34504 0 0 0 52038 49 0 0 20 0 3 0 331448416 388665344 84059 33554432000 4194304 5964478 140735523478688 140384745433400 4403343 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1044.28
Current children cumulated vsize (KiB) 379556

[startup+582.3 s]
/proc/loadavg: 4.00 4.00 3.76 5/181 29889
/proc/meminfo: memFree=26531240/32873844 swapFree=6608/6608
[pid=29863] ppid=29861 vsize=380080 CPUtime=1164.26 cores=0,2,4,6
/proc/29863/stat : 29863 (pwbo2.02) S 29861 29863 28266 0 -1 4202496 82659 0 0 0 116320 106 0 0 20 0 3 0 331448414 389201920 84328 33554432000 4194304 5964478 140735523478688 140735523478208 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/29863/statm: 95020 84328 207 433 0 94584 0
[pid=29863/tid=29866] ppid=29861 vsize=380080 CPUtime=580.69 cores=0,2,4,6
/proc/29863/task/29866/stat : 29866 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 46707 0 0 0 57998 71 0 0 20 0 3 0 331448416 389201920 84328 33554432000 4194304 5964478 140735523478688 140384755922744 4400103 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=29863/tid=29867] ppid=29861 vsize=380080 CPUtime=580.72 cores=0,2,4,6
/proc/29863/task/29867/stat : 29867 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 34576 0 0 0 58021 51 0 0 20 0 3 0 331448416 389201920 84328 33554432000 4194304 5964478 140735523478688 140384745433160 4220672 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1164.26
Current children cumulated vsize (KiB) 380080

[startup+642.3 s]
/proc/loadavg: 4.00 4.00 3.77 5/181 29889
/proc/meminfo: memFree=26510696/32873844 swapFree=6608/6608
[pid=29863] ppid=29861 vsize=446764 CPUtime=1284.26 cores=0,2,4,6
/proc/29863/stat : 29863 (pwbo2.02) S 29861 29863 28266 0 -1 4202496 87742 0 0 0 128310 116 0 0 20 0 3 0 331448414 457486336 89411 33554432000 4194304 5964478 140735523478688 140735523478208 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/29863/statm: 111691 89411 207 433 0 111255 0
[pid=29863/tid=29866] ppid=29861 vsize=446764 CPUtime=640.55 cores=0,2,4,6
/proc/29863/task/29866/stat : 29866 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 51667 0 0 0 63973 82 0 0 20 0 3 0 331448416 457486336 89411 33554432000 4194304 5964478 140735523478688 140384755922936 4221120 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=29863/tid=29867] ppid=29861 vsize=446764 CPUtime=640.57 cores=0,2,4,6
/proc/29863/task/29867/stat : 29867 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 34699 0 0 0 64005 52 0 0 20 0 3 0 331448416 457486336 89411 33554432000 4194304 5964478 140735523478688 140384745433208 4400091 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1284.26
Current children cumulated vsize (KiB) 446764

[startup+702.3 s]
/proc/loadavg: 4.00 4.00 3.78 5/181 29889
/proc/meminfo: memFree=26490112/32873844 swapFree=6608/6608
[pid=29863] ppid=29861 vsize=449240 CPUtime=1404.22 cores=0,2,4,6
/proc/29863/stat : 29863 (pwbo2.02) S 29861 29863 28266 0 -1 4202496 92831 0 0 0 140300 122 0 0 20 0 3 0 331448414 460021760 94491 33554432000 4194304 5964478 140735523478688 140735523478208 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/29863/statm: 112310 94491 207 433 0 111874 0
[pid=29863/tid=29866] ppid=29861 vsize=449240 CPUtime=700.39 cores=0,2,4,6
/proc/29863/task/29866/stat : 29866 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 56621 0 0 0 69950 89 0 0 20 0 3 0 331448416 460021760 94491 33554432000 4194304 5964478 140735523478688 140384755922936 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=29863/tid=29867] ppid=29861 vsize=449240 CPUtime=700.41 cores=0,2,4,6
/proc/29863/task/29867/stat : 29867 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 34834 0 0 0 69988 53 0 0 20 0 3 0 331448416 460021760 94491 33554432000 4194304 5964478 140735523478688 140384745433208 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1404.22
Current children cumulated vsize (KiB) 449240

[startup+762.307 s]
/proc/loadavg: 4.00 4.00 3.80 5/181 29890
/proc/meminfo: memFree=26451300/32873844 swapFree=6608/6608
[pid=29863] ppid=29861 vsize=452264 CPUtime=1524.22 cores=0,2,4,6
/proc/29863/stat : 29863 (pwbo2.02) S 29861 29863 28266 0 -1 4202496 97347 0 0 0 152288 134 0 0 20 0 3 0 331448414 463118336 99007 33554432000 4194304 5964478 140735523478688 140735523478208 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/29863/statm: 113066 99007 207 433 0 112630 0
[pid=29863/tid=29866] ppid=29861 vsize=452264 CPUtime=760.24 cores=0,2,4,6
/proc/29863/task/29866/stat : 29866 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 60940 0 0 0 75922 102 0 0 20 0 3 0 331448416 463118336 99007 33554432000 4194304 5964478 140735523478688 140384755922936 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=29863/tid=29867] ppid=29861 vsize=452264 CPUtime=760.26 cores=0,2,4,6
/proc/29863/task/29867/stat : 29867 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 35031 0 0 0 75972 54 0 0 20 0 3 0 331448416 463118336 99007 33554432000 4194304 5964478 140735523478688 140384745433160 4221120 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1524.22
Current children cumulated vsize (KiB) 452264

[startup+822.3 s]
/proc/loadavg: 4.00 4.00 3.81 5/182 29893
/proc/meminfo: memFree=26431948/32873844 swapFree=6608/6608
[pid=29863] ppid=29861 vsize=454804 CPUtime=1644.19 cores=0,2,4,6
/proc/29863/stat : 29863 (pwbo2.02) S 29861 29863 28266 0 -1 4202496 101892 0 0 0 164272 147 0 0 20 0 3 0 331448414 465719296 103552 33554432000 4194304 5964478 140735523478688 140735523478208 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/29863/statm: 113701 103552 207 433 0 113265 0
[pid=29863/tid=29866] ppid=29861 vsize=454804 CPUtime=820.08 cores=0,2,4,6
/proc/29863/task/29866/stat : 29866 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 65386 0 0 0 81893 115 0 0 20 0 3 0 331448416 465719296 103552 33554432000 4194304 5964478 140735523478688 140384755922744 4399941 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=29863/tid=29867] ppid=29861 vsize=454804 CPUtime=820.1 cores=0,2,4,6
/proc/29863/task/29867/stat : 29867 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 35130 0 0 0 81955 55 0 0 20 0 3 0 331448416 465719296 103552 33554432000 4194304 5964478 140735523478688 140384745433160 4400519 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1644.19
Current children cumulated vsize (KiB) 454804

[startup+882.3 s]
/proc/loadavg: 4.00 4.00 3.82 5/181 29893
/proc/meminfo: memFree=26410952/32873844 swapFree=6608/6608
[pid=29863] ppid=29861 vsize=522564 CPUtime=1764.17 cores=0,2,4,6
/proc/29863/stat : 29863 (pwbo2.02) S 29861 29863 28266 0 -1 4202496 106745 0 0 0 176258 159 0 0 20 0 3 0 331448414 535105536 108579 33554432000 4194304 5964478 140735523478688 140735523478208 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/29863/statm: 130641 108579 207 433 0 130205 0
[pid=29863/tid=29866] ppid=29861 vsize=522564 CPUtime=879.91 cores=0,2,4,6
/proc/29863/task/29866/stat : 29866 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 66404 0 0 0 87872 119 0 0 20 0 3 0 331448416 535105536 108579 33554432000 4194304 5964478 140735523478688 140384755922744 4403343 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=29863/tid=29867] ppid=29861 vsize=522564 CPUtime=879.95 cores=0,2,4,6
/proc/29863/task/29867/stat : 29867 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 38965 0 0 0 87930 65 0 0 20 0 3 0 331448416 535105536 108579 33554432000 4194304 5964478 140735523478688 140384745433208 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1764.17
Current children cumulated vsize (KiB) 522564



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+900.3 s]
/proc/loadavg: 4.00 4.00 3.82 5/181 29893
/proc/meminfo: memFree=26400024/32873844 swapFree=6608/6608
[pid=29863] ppid=29861 vsize=522852 CPUtime=1800.16 cores=0,2,4,6
/proc/29863/stat : 29863 (pwbo2.02) S 29861 29863 28266 0 -1 4202496 108188 0 0 0 179855 161 0 0 20 0 3 0 331448414 535400448 110077 33554432000 4194304 5964478 140735523478688 140735523478208 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/29863/statm: 130713 110077 207 433 0 130277 0
[pid=29863/tid=29866] ppid=29861 vsize=522852 CPUtime=897.86 cores=0,2,4,6
/proc/29863/task/29866/stat : 29866 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 66464 0 0 0 89667 119 0 0 20 0 3 0 331448416 535400448 110077 33554432000 4194304 5964478 140735523478688 140384755922744 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=29863/tid=29867] ppid=29861 vsize=522852 CPUtime=897.91 cores=0,2,4,6
/proc/29863/task/29867/stat : 29867 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 40348 0 0 0 89723 68 0 0 20 0 3 0 331448416 535400448 110077 33554432000 4194304 5964478 140735523478688 140384745433400 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1800.16
Current children cumulated vsize (KiB) 522852

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+900.3 s]
/proc/loadavg: 4.00 4.00 3.82 5/181 29893
/proc/meminfo: memFree=26400024/32873844 swapFree=6608/6608
[pid=29863] ppid=29861 vsize=522852 CPUtime=1800.16 cores=0,2,4,6
/proc/29863/stat : 29863 (pwbo2.02) S 29861 29863 28266 0 -1 4202496 108188 0 0 0 179855 161 0 0 20 0 3 0 331448414 535400448 110077 33554432000 4194304 5964478 140735523478688 140735523478208 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/29863/statm: 130713 110077 207 433 0 130277 0
[pid=29863/tid=29866] ppid=29861 vsize=522852 CPUtime=897.86 cores=0,2,4,6
/proc/29863/task/29866/stat : 29866 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 66464 0 0 0 89667 119 0 0 20 0 3 0 331448416 535400448 110077 33554432000 4194304 5964478 140735523478688 140384755922744 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=29863/tid=29867] ppid=29861 vsize=522852 CPUtime=897.91 cores=0,2,4,6
/proc/29863/task/29867/stat : 29867 (pwbo2.02) R 29861 29863 28266 0 -1 4202560 40348 0 0 0 89723 68 0 0 20 0 3 0 331448416 535400448 110077 33554432000 4194304 5964478 140735523478688 140384745433400 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1800.16
Current children cumulated vsize (KiB) 522852

Child status: 0

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 29863 and gives
#  childrusage.ru_utime.tv_sec=1793
#  childrusage.ru_utime.tv_usec=947278
#  childrusage.ru_stime.tv_sec=1
#  childrusage.ru_stime.tv_usec=890712
# CPU time returned by wait4() is 1795.84
# while last known CPU time is 1800.16
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 900.32
CPU time (s): 1800.16
CPU user time (s): 1798.55
CPU system time (s): 1.61
CPU usage (%): 199.947
Max. virtual memory (cumulated for all children) (KiB): 522852

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1793.95
system time used= 1.89071
maximum resident set size= 440312
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 108189
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 322
involuntary context switches= 1870

runsolver used 1.62075 second user time and 5.2542 second system time

The end

Launcher Data 

Begin job on node140 at 2012-06-02 18:07:16
IDJOB=3725859
IDBENCH=2007
IDSOLVER=2309
FILE ID=node140/3725859-1338653236
RUNJOBID= node140-1338651425-29649
PBS_JOBID= 14636440
Free space on /tmp= 70604 MiB

SOLVER NAME= pwbo 2.02
BENCH NAME= PB06//final/normalized-PB06/OPT-SMALLINT/submitted-PB05/manquinho/ttp/normalized-circ6_3.opb
COMMAND LINE= pwbo2.02 -file-format=opb -verbosity=0 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3725859-1338653236/watcher-3725859-1338653236 -o /tmp/evaluation-result-3725859-1338653236/solver-3725859-1338653236 -C 1800 -W 1900 -M 15500  pwbo2.02 -file-format=opb -verbosity=0 HOME/instance-3725859-1338653236.opb

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= ff9be653d1d3ea3b1a82846016e94df6
RANDOM SEED=1373807438

node140.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.88
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        27424040 kB
Buffers:          389904 kB
Cached:          4163124 kB
SwapCached:         1664 kB
Active:          1238892 kB
Inactive:        3324612 kB
Active(anon):       2536 kB
Inactive(anon):     9416 kB
Active(file):    1236356 kB
Inactive(file):  3315196 kB
Unevictable:        6608 kB
Mlocked:            6608 kB
SwapTotal:      67108856 kB
SwapFree:       67105156 kB
Dirty:              5712 kB
Writeback:             0 kB
AnonPages:         14580 kB
Mapped:             9756 kB
Shmem:                 8 kB
Slab:             747304 kB
SReclaimable:     189268 kB
SUnreclaim:       558036 kB
KernelStack:        1424 kB
PageTables:         3576 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     103912 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70612 MiB
End job on node140 at 2012-06-02 18:22:18