Trace number 3728608

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
pwbo 2.02MSAT (TO) 1800.06 900.226

General information on the benchmark

Name/PARTIAL-SMALLINT-LIN/PB06/submitted-PB06/roussel/
normalized-pigeon-cardinality-21-20--soft-66-100-0.wbo
MD5SUM92adeb187a48a927ab5078673cfa945b
Bench CategoryPARTIAL-SMALLINT-LIN (both soft and hard constraints, small integers, linear constraints)
Best result obtained on this benchmarkMOPT
Best cost obtained on this benchmark1
Best CPU time to get the best result obtained on this benchmark0.028994
Max-Satisfiable
Max-(Un)Satisfiability was proved
Best value of the cost
Optimality of the best cost was proved
Number of variables420
Total number of constraints41
Number of soft constraints11
Number of constraints which are clauses21
Number of constraints which are cardinality constraints (but not clauses)20
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint20
Maximum length of a constraint21
Top cost 551
Min constraint cost 1
Max constraint cost 93
Sum of constraints costs 550
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 22
Number of bits of the biggest sum of numbers5
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Quality of the solution as a function of time


Download the above graph as a PDF file / as an EPS file

Solver Data 

0.00/0.00	c ==============================[ Solver Version ]===============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  pwbo:                     2.02 beta - May, 2012                            |
0.00/0.00	c |                                                                             |
0.00/0.00	c Parsing wbo file format. File HOME/instance-3728608-1338671291.wbo.
0.00/0.00	c Instance file HOME/instance-3728608-1338671291.wbo
0.00/0.00	c File size is 7125 bytes.
0.00/0.00	c Highest Coefficient sum: 22
0.00/0.00	c ============================[ Problem Statistics ]=============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  Parsing time:          0.00         s                                      |
0.00/0.00	c |  Number Variables:      431                                                 |
0.00/0.00	c |  Number Clauses:        21                                                  |
0.00/0.00	c |  Number Cardinality:    13                                                  |
0.00/0.00	c |  Number PB Constraints: 7                                                   |
0.00/0.00	c |                                                                             |
0.00/0.00	c |  Problem Type:          Optimization                                        |
0.00/0.00	c |  Objective Function:    Cardinality Constraint                              |
0.00/0.00	c |                                                                             |
0.00/0.00	c ===============================================================================
1800.06/900.21	o 63
1800.06/900.21	o 46
1800.06/900.21	o 23
1800.06/900.21	o 1
1800.06/900.21	c ===============================================================================
1800.06/900.21	c Wall Clock Time: 900.21 s
1800.06/900.21	c Best Solution: 1
1800.06/900.21	s SATISFIABLE
1800.06/900.21	v x1 -x2 -x3 -x4 -x5 -x6 -x7 -x8 -x9 -x10 -x11 -x12 -x13 -x14 -x15 -x16 -x17 -x18 -x19 -x20 -x21 -x22 -x23 -x24 -x25 x26 -x27 -x28 -x29 -x30 -x31 -x32 -x33 -x34 -x35 -x36 -x37 -x38 -x39 -x40 -x41 -x42 -x43 -x44 -x45 -x46 x47 -x48 -x49 -x50 -x51 -x52 -x53 -x54 -x55 -x56 -x57 -x58 -x59 -x60 -x61 -x62 -x63 -x64 -x65 -x66 -x67 -x68 -x69 -x70 -x71 -x72 -x73 -x74 -x75 -x76 -x77 -x78 x79 -x80 -x81 -x82 -x83 -x84 -x85 -x86 -x87 -x88 -x89 -x90 -x91 -x92 -x93 -x94 -x95 -x96 -x97 -x98 -x99 -x100 -x101 -x102 -x103 -x104 -x105 -x106 -x107 -x108 -x109 -x110 -x111 -x112 -x113 -x114 -x115 -x116 -x117 x118 -x119 -x120 -x121 -x122 -x123 -x124 -x125 -x126 -x127 -x128 -x129 -x130 -x131 -x132 -x133 -x134 -x135 -x136 -x137 -x138 -x139 x140 -x141 -x142 -x143 -x144 -x145 -x146 -x147 -x148 -x149 -x150 -x151 -x152 x153 -x154 -x155 -x156 -x157 -x158 -x159 -x160 -x161 -x162 -x163 -x164 -x165 -x166 -x167 -x168 x169 -x170 -x171 -x172 -x173 -x174 -x175 -x176 -x177 -x178 -x179 -x180 -x181 -x182 -x183 -x184 -x185 -x186 -x187 -x188 -x189 -x190 -x191 -x192 -x193 x194 -x195 -x196 -x197 -x198 -x199 -x200 -x201 -x202 -x203 -x204 -x205 -x206 -x207 -x208 -x209 -x210 x211 -x212 -x213 -x214 -x215 -x216 -x217 -x218 -x219 -x220 -x221 -x222 -x223 -x224 -x225 -x226 -x227 -x228 -x229 -x230 -x231 x232 -x233 -x234 -x235 -x236 -x237 -x238 -x239 -x240 -x241 x242 -x243 -x244 -x245 -x246 -x247 -x248 -x249 -x250 -x251 -x252 -x253 -x254 -x255 -x256 -x257 -x258 -x259 -x260 -x261 -x262 -x263 x264 -x265 -x266 -x267 -x268 -x269 -x270 -x271 -x272 -x273 -x274 -x275 -x276 -x277 -x278 -x279 -x280 -x281 -x282 -x283 -x284 -x285 -x286 -x287 -x288 -x289 -x290 -x291 -x292 -x293 -x294 x295 -x296 -x297 -x298 -x299 -x300 -x301 -x302 -x303 -x304 -x305 -x306 -x307 x308 -x309 -x310 -x311 -x312 -x313 -x314 -x315 -x316 -x317 -x318 -x319 -x320 -x321 -x322 -x323 -x324 -x325 -x326 -x327 -x328 -x329 x330 -x331 -x332 -x333 -x334 -x335 -x336 -x337 -x338 -x339 -x340 -x341 -x342 -x343 -x344 -x345 -x346 -x347 -x348 -x349 -x350 -x351 -x352 -x353 -x354 -x355 -x356 x357 -x358 -x359 -x360 -x361 -x362 -x363 -x364 -x365 -x366 -x367 -x368 -x369 -x370 -x371 -x372 -x373 -x374 -x375 x376 -x377 -x378 -x379 -x380 -x381 -x382 -x383 -x384 x385 -x386 -x387 -x388 -x389 -x390 -x391 -x392 -x393 -x394 -x395 -x396 -x397 -x398 -x399 -x400 -x401 -x402 x403 -x404 -x405 -x406 -x407 -x408 -x409 -x410 -x411 -x412 -x413 -x414 -x415 -x416 -x417 -x418 -x419 -x420

Verifier Data 

OK	1

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3728608-1338671291/watcher-3728608-1338671291 -o /tmp/evaluation-result-3728608-1338671291/solver-3728608-1338671291 -C 1800 -W 1900 -M 15500 pwbo2.02 -file-format=wbo -verbosity=0 HOME/instance-3728608-1338671291.wbo 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.00 3.99 3.67 4/178 4143
/proc/meminfo: memFree=28463556/32873844 swapFree=7632/7632
[pid=4143] ppid=4141 vsize=163424 CPUtime=0 cores=0,2,4,6
/proc/4143/stat : 4143 (pwbo2.02) S 4141 4143 3617 0 -1 4202496 1222 0 0 0 0 0 0 0 20 0 3 0 333253434 167346176 1174 33554432000 4194304 5964478 140734423370416 140734423369936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4143/statm: 40856 1185 205 433 0 40420 0
[pid=4143/tid=4144] ppid=4141 vsize=163236 CPUtime=0 cores=0,2,4,6
/proc/4143/task/4144/stat : 4144 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 72 0 0 0 0 0 0 0 20 0 3 0 333253434 167153664 1151 33554432000 4194304 5964478 140734423370416 140124545759224 5136420 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4143/tid=4145] ppid=4141 vsize=163236 CPUtime=0 cores=0,2,4,6
/proc/4143/task/4145/stat : 4145 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 387 0 0 0 0 0 0 0 20 0 3 0 333253434 167153664 1155 33554432000 4194304 5964478 140734423370416 140124535268872 4395406 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0

[startup+0.080388 s]
/proc/loadavg: 4.00 3.99 3.67 4/178 4143
/proc/meminfo: memFree=28463556/32873844 swapFree=7632/7632
[pid=4143] ppid=4141 vsize=166144 CPUtime=0.15 cores=0,2,4,6
/proc/4143/stat : 4143 (pwbo2.02) S 4141 4143 3617 0 -1 4202496 2998 0 0 0 13 2 0 0 20 0 3 0 333253434 170131456 2902 33554432000 4194304 5964478 140734423370416 140734423369936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4143/statm: 41536 2902 207 433 0 41100 0
[pid=4143/tid=4144] ppid=4141 vsize=166144 CPUtime=0.06 cores=0,2,4,6
/proc/4143/task/4144/stat : 4144 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 333 0 0 0 6 0 0 0 20 0 3 0 333253434 170131456 2902 33554432000 4194304 5964478 140734423370416 140124545759032 5136420 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4143/tid=4145] ppid=4141 vsize=166144 CPUtime=0.06 cores=0,2,4,6
/proc/4143/task/4145/stat : 4145 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 1873 0 0 0 6 0 0 0 20 0 3 0 333253434 170131456 2902 33554432000 4194304 5964478 140734423370416 140124535269688 4220672 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 0.15
Current children cumulated vsize (KiB) 166144

[startup+0.10038 s]
/proc/loadavg: 4.00 3.99 3.67 4/178 4143
/proc/meminfo: memFree=28463556/32873844 swapFree=7632/7632
[pid=4143] ppid=4141 vsize=166144 CPUtime=0.18 cores=0,2,4,6
/proc/4143/stat : 4143 (pwbo2.02) S 4141 4143 3617 0 -1 4202496 3103 0 0 0 16 2 0 0 20 0 3 0 333253434 170131456 3007 33554432000 4194304 5964478 140734423370416 140734423369936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4143/statm: 41536 3007 207 433 0 41100 0
[pid=4143/tid=4144] ppid=4141 vsize=166144 CPUtime=0.08 cores=0,2,4,6
/proc/4143/task/4144/stat : 4144 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 393 0 0 0 7 1 0 0 20 0 3 0 333253434 170131456 3007 33554432000 4194304 5964478 140734423370416 140124545759224 5136420 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4143/tid=4145] ppid=4141 vsize=166144 CPUtime=0.08 cores=0,2,4,6
/proc/4143/task/4145/stat : 4145 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 1918 0 0 0 7 1 0 0 20 0 3 0 333253434 170131456 3007 33554432000 4194304 5964478 140734423370416 140124535269688 5136420 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 0.18
Current children cumulated vsize (KiB) 166144

[startup+0.300289 s]
/proc/loadavg: 4.00 3.99 3.67 4/178 4143
/proc/meminfo: memFree=28463556/32873844 swapFree=7632/7632
[pid=4143] ppid=4141 vsize=166144 CPUtime=0.58 cores=0,2,4,6
/proc/4143/stat : 4143 (pwbo2.02) S 4141 4143 3617 0 -1 4202496 3901 0 0 0 54 4 0 0 20 0 3 0 333253434 170131456 3805 33554432000 4194304 5964478 140734423370416 140734423369936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4143/statm: 41536 3806 207 433 0 41100 0
[pid=4143/tid=4144] ppid=4141 vsize=166144 CPUtime=0.28 cores=0,2,4,6
/proc/4143/task/4144/stat : 4144 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 741 0 0 0 26 2 0 0 20 0 3 0 333253434 170131456 3806 33554432000 4194304 5964478 140734423370416 140124545759224 4382021 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4143/tid=4145] ppid=4141 vsize=166144 CPUtime=0.28 cores=0,2,4,6
/proc/4143/task/4145/stat : 4145 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 2369 0 0 0 25 3 0 0 20 0 3 0 333253434 170131456 3806 33554432000 4194304 5964478 140734423370416 140124535269496 5136420 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 0.58
Current children cumulated vsize (KiB) 166144

[startup+0.700282 s]
/proc/loadavg: 4.00 3.99 3.67 4/178 4143
/proc/meminfo: memFree=28463556/32873844 swapFree=7632/7632
[pid=4143] ppid=4141 vsize=166144 CPUtime=1.38 cores=0,2,4,6
/proc/4143/stat : 4143 (pwbo2.02) S 4141 4143 3617 0 -1 4202496 5065 0 0 0 129 9 0 0 20 0 3 0 333253434 170131456 4969 33554432000 4194304 5964478 140734423370416 140734423369936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4143/statm: 41536 4969 207 433 0 41100 0
[pid=4143/tid=4144] ppid=4141 vsize=166144 CPUtime=0.68 cores=0,2,4,6
/proc/4143/task/4144/stat : 4144 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 1227 0 0 0 64 4 0 0 20 0 3 0 333253434 170131456 4969 33554432000 4194304 5964478 140734423370416 140124545759224 4400516 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4143/tid=4145] ppid=4141 vsize=166144 CPUtime=0.68 cores=0,2,4,6
/proc/4143/task/4145/stat : 4145 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 3046 0 0 0 62 6 0 0 20 0 3 0 333253434 170131456 4969 33554432000 4194304 5964478 140734423370416 140124535269736 5136420 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1.38
Current children cumulated vsize (KiB) 166144

[startup+1.50028 s]
/proc/loadavg: 4.00 3.99 3.67 5/181 4146
/proc/meminfo: memFree=28387084/32873844 swapFree=7632/7632
[pid=4143] ppid=4141 vsize=166288 CPUtime=2.98 cores=0,2,4,6
/proc/4143/stat : 4143 (pwbo2.02) S 4141 4143 3617 0 -1 4202496 7018 0 0 0 287 11 0 0 20 0 3 0 333253434 170278912 6922 33554432000 4194304 5964478 140734423370416 140734423369936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4143/statm: 41572 6922 207 433 0 41136 0
[pid=4143/tid=4144] ppid=4141 vsize=166288 CPUtime=1.48 cores=0,2,4,6
/proc/4143/task/4144/stat : 4144 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 2262 0 0 0 143 5 0 0 20 0 3 0 333253434 170278912 6922 33554432000 4194304 5964478 140734423370416 140124545759032 5136420 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4143/tid=4145] ppid=4141 vsize=166288 CPUtime=1.48 cores=0,2,4,6
/proc/4143/task/4145/stat : 4145 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 3965 0 0 0 141 7 0 0 20 0 3 0 333253434 170278912 6923 33554432000 4194304 5964478 140734423370416 140124535269688 5136420 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 2.98
Current children cumulated vsize (KiB) 166288

[startup+3.1003 s]
/proc/loadavg: 4.00 3.99 3.67 5/181 4146
/proc/meminfo: memFree=28364260/32873844 swapFree=7632/7632
[pid=4143] ppid=4141 vsize=166636 CPUtime=6.18 cores=0,2,4,6
/proc/4143/stat : 4143 (pwbo2.02) S 4141 4143 3617 0 -1 4202496 9598 0 0 0 606 12 0 0 20 0 3 0 333253434 170635264 9502 33554432000 4194304 5964478 140734423370416 140734423369936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4143/statm: 41659 9502 207 433 0 41223 0
[pid=4143/tid=4144] ppid=4141 vsize=166636 CPUtime=3.07 cores=0,2,4,6
/proc/4143/task/4144/stat : 4144 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 3120 0 0 0 302 5 0 0 20 0 3 0 333253434 170635264 9502 33554432000 4194304 5964478 140734423370416 140124545759224 5136420 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4143/tid=4145] ppid=4141 vsize=166636 CPUtime=3.08 cores=0,2,4,6
/proc/4143/task/4145/stat : 4145 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 5686 0 0 0 301 7 0 0 20 0 3 0 333253434 170635264 9502 33554432000 4194304 5964478 140734423370416 140124535269688 4400087 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 6.18
Current children cumulated vsize (KiB) 166636

[startup+6.30032 s]
/proc/loadavg: 4.00 3.99 3.67 5/181 4146
/proc/meminfo: memFree=28318256/32873844 swapFree=7632/7632
[pid=4143] ppid=4141 vsize=166636 CPUtime=12.57 cores=0,2,4,6
/proc/4143/stat : 4143 (pwbo2.02) S 4141 4143 3617 0 -1 4202496 12036 0 0 0 1244 13 0 0 20 0 3 0 333253434 170635264 11940 33554432000 4194304 5964478 140734423370416 140734423369936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4143/statm: 41659 11940 207 433 0 41223 0
[pid=4143/tid=4144] ppid=4141 vsize=166636 CPUtime=6.26 cores=0,2,4,6
/proc/4143/task/4144/stat : 4144 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 4521 0 0 0 620 6 0 0 20 0 3 0 333253434 170635264 11940 33554432000 4194304 5964478 140734423370416 140124545759224 4220672 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4143/tid=4145] ppid=4141 vsize=166636 CPUtime=6.26 cores=0,2,4,6
/proc/4143/task/4145/stat : 4145 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 6723 0 0 0 619 7 0 0 20 0 3 0 333253434 170635264 11940 33554432000 4194304 5964478 140734423370416 140124535269688 5136420 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 12.57
Current children cumulated vsize (KiB) 166636

[startup+12.7003 s]
/proc/loadavg: 4.00 3.99 3.67 5/181 4147
/proc/meminfo: memFree=28261320/32873844 swapFree=7632/7632
[pid=4143] ppid=4141 vsize=166956 CPUtime=25.38 cores=0,2,4,6
/proc/4143/stat : 4143 (pwbo2.02) S 4141 4143 3617 0 -1 4202496 14851 0 0 0 2523 15 0 0 20 0 3 0 333253434 170962944 14755 33554432000 4194304 5964478 140734423370416 140734423369936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4143/statm: 41739 14755 207 433 0 41303 0
[pid=4143/tid=4144] ppid=4141 vsize=166956 CPUtime=12.65 cores=0,2,4,6
/proc/4143/task/4144/stat : 4144 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 5578 0 0 0 1259 6 0 0 20 0 3 0 333253434 170962944 14755 33554432000 4194304 5964478 140734423370416 140124545759224 5136420 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4143/tid=4145] ppid=4141 vsize=166956 CPUtime=12.65 cores=0,2,4,6
/proc/4143/task/4145/stat : 4145 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 8481 0 0 0 1257 8 0 0 20 0 3 0 333253434 170962944 14755 33554432000 4194304 5964478 140734423370416 140124535269736 4221120 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 25.38
Current children cumulated vsize (KiB) 166956

[startup+25.5003 s]
/proc/loadavg: 4.00 3.99 3.68 5/181 4147
/proc/meminfo: memFree=28160756/32873844 swapFree=7632/7632
[pid=4143] ppid=4141 vsize=167376 CPUtime=50.97 cores=0,2,4,6
/proc/4143/stat : 4143 (pwbo2.02) S 4141 4143 3617 0 -1 4202496 18398 0 0 0 5079 18 0 0 20 0 3 0 333253434 171393024 18302 33554432000 4194304 5964478 140734423370416 140734423369936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4143/statm: 41844 18302 207 433 0 41408 0
[pid=4143/tid=4144] ppid=4141 vsize=167376 CPUtime=25.41 cores=0,2,4,6
/proc/4143/task/4144/stat : 4144 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 8037 0 0 0 2533 8 0 0 20 0 3 0 333253434 171393024 18302 33554432000 4194304 5964478 140734423370416 140124545759224 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4143/tid=4145] ppid=4141 vsize=167376 CPUtime=25.42 cores=0,2,4,6
/proc/4143/task/4145/stat : 4145 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 9569 0 0 0 2532 10 0 0 20 0 3 0 333253434 171393024 18302 33554432000 4194304 5964478 140734423370416 140124535269688 4221120 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 50.97
Current children cumulated vsize (KiB) 167376

[startup+51.1003 s]
/proc/loadavg: 4.00 3.99 3.68 5/181 4147
/proc/meminfo: memFree=28018604/32873844 swapFree=7632/7632
[pid=4143] ppid=4141 vsize=167804 CPUtime=102.17 cores=0,2,4,6
/proc/4143/stat : 4143 (pwbo2.02) S 4141 4143 3617 0 -1 4202496 22390 0 0 0 10195 22 0 0 20 0 3 0 333253434 171831296 22294 33554432000 4194304 5964478 140734423370416 140734423369936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4143/statm: 41951 22294 207 433 0 41515 0
[pid=4143/tid=4144] ppid=4141 vsize=167804 CPUtime=50.95 cores=0,2,4,6
/proc/4143/task/4144/stat : 4144 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 10002 0 0 0 5085 10 0 0 20 0 3 0 333253434 171831296 22294 33554432000 4194304 5964478 140734423370416 140124545759224 4221120 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4143/tid=4145] ppid=4141 vsize=167804 CPUtime=50.95 cores=0,2,4,6
/proc/4143/task/4145/stat : 4145 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 11596 0 0 0 5084 11 0 0 20 0 3 0 333253434 171831296 22294 33554432000 4194304 5964478 140734423370416 140124535269496 4400432 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 102.17
Current children cumulated vsize (KiB) 167804

[startup+102.306 s]
/proc/loadavg: 4.04 4.00 3.71 5/181 4147
/proc/meminfo: memFree=27756512/32873844 swapFree=7632/7632
[pid=4143] ppid=4141 vsize=168384 CPUtime=204.56 cores=0,2,4,6
/proc/4143/stat : 4143 (pwbo2.02) S 4141 4143 3617 0 -1 4202496 25413 0 0 0 20431 25 0 0 20 0 3 0 333253434 172425216 25317 33554432000 4194304 5964478 140734423370416 140734423369936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4143/statm: 42096 25317 207 433 0 41660 0
[pid=4143/tid=4144] ppid=4141 vsize=168384 CPUtime=102.02 cores=0,2,4,6
/proc/4143/task/4144/stat : 4144 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 11816 0 0 0 10189 13 0 0 20 0 3 0 333253434 172425216 25317 33554432000 4194304 5964478 140734423370416 140124545759224 4400087 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4143/tid=4145] ppid=4141 vsize=168384 CPUtime=102.03 cores=0,2,4,6
/proc/4143/task/4145/stat : 4145 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 12805 0 0 0 10190 13 0 0 20 0 3 0 333253434 172425216 25317 33554432000 4194304 5964478 140734423370416 140124535269512 4221120 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 204.56
Current children cumulated vsize (KiB) 168384

[startup+162.3 s]
/proc/loadavg: 4.02 4.00 3.72 5/182 4150
/proc/meminfo: memFree=27446364/32873844 swapFree=7632/7632
[pid=4143] ppid=4141 vsize=239360 CPUtime=324.52 cores=0,2,4,6
/proc/4143/stat : 4143 (pwbo2.02) S 4141 4143 3617 0 -1 4202496 38367 0 0 0 32416 36 0 0 20 0 3 0 333253434 245104640 38445 33554432000 4194304 5964478 140734423370416 140734423369936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4143/statm: 59840 38445 207 433 0 59404 0
[pid=4143/tid=4144] ppid=4141 vsize=239360 CPUtime=161.86 cores=0,2,4,6
/proc/4143/task/4144/stat : 4144 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 16120 0 0 0 16168 18 0 0 20 0 3 0 333253434 245104640 38445 33554432000 4194304 5964478 140734423370416 140124545759224 4389567 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4143/tid=4145] ppid=4141 vsize=239360 CPUtime=161.86 cores=0,2,4,6
/proc/4143/task/4145/stat : 4145 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 21455 0 0 0 16165 21 0 0 20 0 3 0 333253434 245104640 38445 33554432000 4194304 5964478 140734423370416 140124535269688 4400115 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 324.52
Current children cumulated vsize (KiB) 239360

[startup+222.3 s]
/proc/loadavg: 4.00 4.00 3.74 5/181 4150
/proc/meminfo: memFree=27244716/32873844 swapFree=7632/7632
[pid=4143] ppid=4141 vsize=305616 CPUtime=444.45 cores=0,2,4,6
/proc/4143/stat : 4143 (pwbo2.02) S 4141 4143 3617 0 -1 4202496 42017 0 0 0 44406 39 0 0 20 0 3 0 333253434 312950784 42245 33554432000 4194304 5964478 140734423370416 140734423369936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4143/statm: 76404 42245 207 433 0 75968 0
[pid=4143/tid=4144] ppid=4141 vsize=305616 CPUtime=221.68 cores=0,2,4,6
/proc/4143/task/4144/stat : 4144 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 19726 0 0 0 22145 23 0 0 20 0 3 0 333253434 312950784 42245 33554432000 4194304 5964478 140734423370416 140124545759224 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4143/tid=4145] ppid=4141 vsize=305616 CPUtime=221.69 cores=0,2,4,6
/proc/4143/task/4145/stat : 4145 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 21499 0 0 0 22147 22 0 0 20 0 3 0 333253434 312950784 42245 33554432000 4194304 5964478 140734423370416 140124535269736 4221120 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 444.45
Current children cumulated vsize (KiB) 305616

################
# More data... #
################

[pid=4143/tid=4145] ppid=4141 vsize=309264 CPUtime=281.53 cores=0,2,4,6
/proc/4143/task/4145/stat : 4145 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 22769 0 0 0 28129 24 0 0 20 0 3 0 333253434 316686336 49713 33554432000 4194304 5964478 140734423370416 140124535269688 4400069 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 564.43
Current children cumulated vsize (KiB) 309264

[startup+342.3 s]
/proc/loadavg: 4.07 4.02 3.76 5/181 4151
/proc/meminfo: memFree=26942372/32873844 swapFree=7632/7632
[pid=4143] ppid=4141 vsize=310832 CPUtime=684.41 cores=0,2,4,6
/proc/4143/stat : 4143 (pwbo2.02) S 4141 4143 3617 0 -1 4202496 54695 0 0 0 68392 49 0 0 20 0 3 0 333253434 318291968 56107 33554432000 4194304 5964478 140734423370416 140734423369936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4143/statm: 77708 56107 207 433 0 77272 0
[pid=4143/tid=4144] ppid=4141 vsize=310832 CPUtime=341.37 cores=0,2,4,6
/proc/4143/task/4144/stat : 4144 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 24704 0 0 0 34108 29 0 0 20 0 3 0 333253434 318291968 56107 33554432000 4194304 5964478 140734423370416 140124545759256 4400091 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4143/tid=4145] ppid=4141 vsize=310832 CPUtime=341.38 cores=0,2,4,6
/proc/4143/task/4145/stat : 4145 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 29199 0 0 0 34109 29 0 0 20 0 3 0 333253434 318291968 56107 33554432000 4194304 5964478 140734423370416 140124535269688 4400078 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 684.41
Current children cumulated vsize (KiB) 310832

[startup+402.3 s]
/proc/loadavg: 4.02 4.01 3.78 5/181 4151
/proc/meminfo: memFree=26759004/32873844 swapFree=7632/7632
[pid=4143] ppid=4141 vsize=312272 CPUtime=804.39 cores=0,2,4,6
/proc/4143/stat : 4143 (pwbo2.02) S 4141 4143 3617 0 -1 4202496 57785 0 0 0 80387 52 0 0 20 0 3 0 333253434 319766528 59197 33554432000 4194304 5964478 140734423370416 140734423369936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4143/statm: 78068 59197 207 433 0 77632 0
[pid=4143/tid=4144] ppid=4141 vsize=312272 CPUtime=401.22 cores=0,2,4,6
/proc/4143/task/4144/stat : 4144 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 27788 0 0 0 40089 33 0 0 20 0 3 0 333253434 319766528 59197 33554432000 4194304 5964478 140734423370416 140124545759272 4400006 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4143/tid=4145] ppid=4141 vsize=312272 CPUtime=401.23 cores=0,2,4,6
/proc/4143/task/4145/stat : 4145 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 29205 0 0 0 40093 30 0 0 20 0 3 0 333253434 319766528 59197 33554432000 4194304 5964478 140734423370416 140124535269688 4400103 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 804.39
Current children cumulated vsize (KiB) 312272

[startup+462.3 s]
/proc/loadavg: 4.01 4.00 3.79 5/181 4151
/proc/meminfo: memFree=26449404/32873844 swapFree=7632/7632
[pid=4143] ppid=4141 vsize=313568 CPUtime=924.37 cores=0,2,4,6
/proc/4143/stat : 4143 (pwbo2.02) S 4141 4143 3617 0 -1 4202496 60501 0 0 0 92383 54 0 0 20 0 3 0 333253434 321093632 61913 33554432000 4194304 5964478 140734423370416 140734423369936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4143/statm: 78392 61913 207 433 0 77956 0
[pid=4143/tid=4144] ppid=4141 vsize=313568 CPUtime=461.07 cores=0,2,4,6
/proc/4143/task/4144/stat : 4144 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 30454 0 0 0 46071 36 0 0 20 0 3 0 333253434 321093632 61913 33554432000 4194304 5964478 140734423370416 140124545759224 4400087 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4143/tid=4145] ppid=4141 vsize=313568 CPUtime=461.07 cores=0,2,4,6
/proc/4143/task/4145/stat : 4145 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 29255 0 0 0 46076 31 0 0 20 0 3 0 333253434 321093632 61913 33554432000 4194304 5964478 140734423370416 140124535269496 4400087 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 924.37
Current children cumulated vsize (KiB) 313568

[startup+522.3 s]
/proc/loadavg: 4.00 4.00 3.80 5/181 4151
/proc/meminfo: memFree=26330640/32873844 swapFree=7632/7632
[pid=4143] ppid=4141 vsize=317604 CPUtime=1044.36 cores=0,2,4,6
/proc/4143/stat : 4143 (pwbo2.02) S 4141 4143 3617 0 -1 4202496 62015 0 0 0 104380 56 0 0 20 0 3 0 333253434 325226496 63411 33554432000 4194304 5964478 140734423370416 140734423369936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4143/statm: 79401 63411 207 433 0 78965 0
[pid=4143/tid=4144] ppid=4141 vsize=317604 CPUtime=520.91 cores=0,2,4,6
/proc/4143/task/4144/stat : 4144 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 30624 0 0 0 52054 37 0 0 20 0 3 0 333253434 325226496 63411 33554432000 4194304 5964478 140734423370416 140124545759032 4220672 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4143/tid=4145] ppid=4141 vsize=317604 CPUtime=520.92 cores=0,2,4,6
/proc/4143/task/4145/stat : 4145 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 30599 0 0 0 52060 32 0 0 20 0 3 0 333253434 325226496 63411 33554432000 4194304 5964478 140734423370416 140124535269736 4400087 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1044.36
Current children cumulated vsize (KiB) 317604

[startup+582.3 s]
/proc/loadavg: 4.05 4.01 3.82 5/181 4151
/proc/meminfo: memFree=26250936/32873844 swapFree=7632/7632
[pid=4143] ppid=4141 vsize=321544 CPUtime=1164.34 cores=0,2,4,6
/proc/4143/stat : 4143 (pwbo2.02) S 4141 4143 3617 0 -1 4202496 67772 0 0 0 116373 61 0 0 20 0 3 0 333253434 329261056 69168 33554432000 4194304 5964478 140734423370416 140734423369936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4143/statm: 80386 69168 207 433 0 79950 0
[pid=4143/tid=4144] ppid=4141 vsize=321544 CPUtime=580.76 cores=0,2,4,6
/proc/4143/task/4144/stat : 4144 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 34431 0 0 0 58036 40 0 0 20 0 3 0 333253434 329261056 69168 33554432000 4194304 5964478 140734423370416 140124545759224 4221120 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4143/tid=4145] ppid=4141 vsize=321544 CPUtime=580.77 cores=0,2,4,6
/proc/4143/task/4145/stat : 4145 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 32549 0 0 0 58041 36 0 0 20 0 3 0 333253434 329261056 69168 33554432000 4194304 5964478 140734423370416 140124535269688 4400089 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1164.34
Current children cumulated vsize (KiB) 321544

[startup+642.3 s]
/proc/loadavg: 4.02 4.01 3.82 5/181 4152
/proc/meminfo: memFree=26106752/32873844 swapFree=7632/7632
[pid=4143] ppid=4141 vsize=390188 CPUtime=1284.34 cores=0,2,4,6
/proc/4143/stat : 4143 (pwbo2.02) S 4141 4143 3617 0 -1 4202496 76279 0 0 0 128366 68 0 0 20 0 3 0 333253434 399552512 77675 33554432000 4194304 5964478 140734423370416 140734423369936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4143/statm: 97547 77675 207 433 0 97111 0
[pid=4143/tid=4144] ppid=4141 vsize=390188 CPUtime=640.61 cores=0,2,4,6
/proc/4143/task/4144/stat : 4144 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 41017 0 0 0 64014 47 0 0 20 0 3 0 333253434 399552512 77675 33554432000 4194304 5964478 140734423370416 140124545759224 4400115 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4143/tid=4145] ppid=4141 vsize=390188 CPUtime=640.62 cores=0,2,4,6
/proc/4143/task/4145/stat : 4145 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 34470 0 0 0 64024 38 0 0 20 0 3 0 333253434 399552512 77675 33554432000 4194304 5964478 140734423370416 140124535269688 4400795 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1284.34
Current children cumulated vsize (KiB) 390188

[startup+702.3 s]
/proc/loadavg: 4.00 4.00 3.83 5/181 4152
/proc/meminfo: memFree=25982904/32873844 swapFree=7632/7632
[pid=4143] ppid=4141 vsize=458836 CPUtime=1404.3 cores=0,2,4,6
/proc/4143/stat : 4143 (pwbo2.02) S 4141 4143 3617 0 -1 4202496 81848 0 0 0 140356 74 0 0 20 0 3 0 333253434 469848064 83244 33554432000 4194304 5964478 140734423370416 140734423369936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4143/statm: 114709 83244 207 433 0 114273 0
[pid=4143/tid=4144] ppid=4141 vsize=458836 CPUtime=700.45 cores=0,2,4,6
/proc/4143/task/4144/stat : 4144 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 43800 0 0 0 69995 50 0 0 20 0 3 0 333253434 469848064 83244 33554432000 4194304 5964478 140734423370416 140124545759272 4400091 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4143/tid=4145] ppid=4141 vsize=458836 CPUtime=700.46 cores=0,2,4,6
/proc/4143/task/4145/stat : 4145 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 37256 0 0 0 70003 43 0 0 20 0 3 0 333253434 469848064 83244 33554432000 4194304 5964478 140734423370416 140124535269688 5136420 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1404.3
Current children cumulated vsize (KiB) 458836

[startup+762.306 s]
/proc/loadavg: 4.00 4.00 3.84 5/182 4155
/proc/meminfo: memFree=25759476/32873844 swapFree=7632/7632
[pid=4143] ppid=4141 vsize=462340 CPUtime=1524.29 cores=0,2,4,6
/proc/4143/stat : 4143 (pwbo2.02) S 4141 4143 3617 0 -1 4202496 87241 0 0 0 152350 79 0 0 20 0 3 0 333253434 473436160 88637 33554432000 4194304 5964478 140734423370416 140734423369936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4143/statm: 115585 88637 207 433 0 115149 0
[pid=4143/tid=4144] ppid=4141 vsize=462340 CPUtime=760.31 cores=0,2,4,6
/proc/4143/task/4144/stat : 4144 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 46467 0 0 0 75977 54 0 0 20 0 3 0 333253434 473436160 88637 33554432000 4194304 5964478 140734423370416 140124545759224 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4143/tid=4145] ppid=4141 vsize=462340 CPUtime=760.32 cores=0,2,4,6
/proc/4143/task/4145/stat : 4145 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 39982 0 0 0 75986 46 0 0 20 0 3 0 333253434 473436160 88637 33554432000 4194304 5964478 140734423370416 140124535269688 4400087 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1524.29
Current children cumulated vsize (KiB) 462340

[startup+822.3 s]
/proc/loadavg: 4.00 4.00 3.84 5/181 4155
/proc/meminfo: memFree=25547536/32873844 swapFree=7632/7632
[pid=4143] ppid=4141 vsize=463528 CPUtime=1644.26 cores=0,2,4,6
/proc/4143/stat : 4143 (pwbo2.02) S 4141 4143 3617 0 -1 4202496 89688 0 0 0 164344 82 0 0 20 0 3 0 333253434 474652672 91084 33554432000 4194304 5964478 140734423370416 140734423369936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4143/statm: 115882 91084 207 433 0 115446 0
[pid=4143/tid=4144] ppid=4141 vsize=463528 CPUtime=820.15 cores=0,2,4,6
/proc/4143/task/4144/stat : 4144 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 48506 0 0 0 81958 57 0 0 20 0 3 0 333253434 474652672 91084 33554432000 4194304 5964478 140734423370416 140124545759224 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4143/tid=4145] ppid=4141 vsize=463528 CPUtime=820.16 cores=0,2,4,6
/proc/4143/task/4145/stat : 4145 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 40390 0 0 0 81968 48 0 0 20 0 3 0 333253434 474652672 91084 33554432000 4194304 5964478 140734423370416 140124535269688 4400078 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1644.26
Current children cumulated vsize (KiB) 463528

[startup+882.3 s]
/proc/loadavg: 4.00 4.00 3.85 5/181 4155
/proc/meminfo: memFree=25302788/32873844 swapFree=7632/7632
[pid=4143] ppid=4141 vsize=464248 CPUtime=1764.25 cores=0,2,4,6
/proc/4143/stat : 4143 (pwbo2.02) S 4141 4143 3617 0 -1 4202496 90074 0 0 0 176342 83 0 0 20 0 3 0 333253434 475389952 91470 33554432000 4194304 5964478 140734423370416 140734423369936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4143/statm: 116062 91470 207 433 0 115626 0
[pid=4143/tid=4144] ppid=4141 vsize=464248 CPUtime=880 cores=0,2,4,6
/proc/4143/task/4144/stat : 4144 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 48733 0 0 0 87942 58 0 0 20 0 3 0 333253434 475389952 91470 33554432000 4194304 5964478 140734423370416 140124545759224 4400078 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4143/tid=4145] ppid=4141 vsize=464248 CPUtime=880 cores=0,2,4,6
/proc/4143/task/4145/stat : 4145 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 40549 0 0 0 87950 50 0 0 20 0 3 0 333253434 475389952 91470 33554432000 4194304 5964478 140734423370416 140124535269496 4400078 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1764.25
Current children cumulated vsize (KiB) 464248



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+900.2 s]
/proc/loadavg: 4.00 4.00 3.85 5/181 4155
/proc/meminfo: memFree=25286840/32873844 swapFree=7632/7632
[pid=4143] ppid=4141 vsize=464248 CPUtime=1800.06 cores=0,2,4,6
/proc/4143/stat : 4143 (pwbo2.02) S 4141 4143 3617 0 -1 4202496 90075 0 0 0 179923 83 0 0 20 0 3 0 333253434 475389952 91381 33554432000 4194304 5964478 140734423370416 140734423369936 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4143/statm: 116062 91381 207 433 0 115626 0
[pid=4143/tid=4144] ppid=4141 vsize=464248 CPUtime=897.86 cores=0,2,4,6
/proc/4143/task/4144/stat : 4144 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 48734 0 0 0 89728 58 0 0 20 0 3 0 333253434 475389952 91381 33554432000 4194304 5964478 140734423370416 140124545759256 4400067 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4143/tid=4145] ppid=4141 vsize=464248 CPUtime=897.86 cores=0,2,4,6
/proc/4143/task/4145/stat : 4145 (pwbo2.02) R 4141 4143 3617 0 -1 4202560 40549 0 0 0 89736 50 0 0 20 0 3 0 333253434 475389952 91381 33554432000 4194304 5964478 140734423370416 140124535269496 4400087 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1800.06
Current children cumulated vsize (KiB) 464248

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child status: 0

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 4143 and gives
#  childrusage.ru_utime.tv_sec=1794
#  childrusage.ru_utime.tv_usec=659170
#  childrusage.ru_stime.tv_sec=1
#  childrusage.ru_stime.tv_usec=110831
# CPU time returned by wait4() is 1795.77
# while last known CPU time is 1800.06
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 900.226
CPU time (s): 1800.06
CPU user time (s): 1799.23
CPU system time (s): 0.83
CPU usage (%): 199.956
Max. virtual memory (cumulated for all children) (KiB): 464248

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1794.66
system time used= 1.11083
maximum resident set size= 365880
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 90076
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 250
involuntary context switches= 1883

runsolver used 1.71074 second user time and 4.87626 second system time

The end

Launcher Data 

Begin job on node141 at 2012-06-02 23:08:11
IDJOB=3728608
IDBENCH=78718
IDSOLVER=2316
FILE ID=node141/3728608-1338671291
RUNJOBID= node141-1338669486-3911
PBS_JOBID= 14636518
Free space on /tmp= 70716 MiB

SOLVER NAME= pwbo 2.02
BENCH NAME= PB10/normalized-PB10/PARTIAL-SMALLINT-LIN/PB06/submitted-PB06/roussel/normalized-pigeon-cardinality-21-20--soft-66-100-0.wbo
COMMAND LINE= pwbo2.02 -file-format=wbo -verbosity=0 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3728608-1338671291/watcher-3728608-1338671291 -o /tmp/evaluation-result-3728608-1338671291/solver-3728608-1338671291 -C 1800 -W 1900 -M 15500  pwbo2.02 -file-format=wbo -verbosity=0 HOME/instance-3728608-1338671291.wbo

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 92adeb187a48a927ab5078673cfa945b
RANDOM SEED=1642590649

node141.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.744
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.48
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.744
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.744
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.93
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.744
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.744
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.744
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.744
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.744
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        28467416 kB
Buffers:          383392 kB
Cached:          3120640 kB
SwapCached:          216 kB
Active:          1225848 kB
Inactive:        2300348 kB
Active(anon):      13268 kB
Inactive(anon):    11300 kB
Active(file):    1212580 kB
Inactive(file):  2289048 kB
Unevictable:        7632 kB
Mlocked:            7632 kB
SwapTotal:      67108856 kB
SwapFree:       67108640 kB
Dirty:             10916 kB
Writeback:             0 kB
AnonPages:         29148 kB
Mapped:            11404 kB
Shmem:               172 kB
Slab:             738840 kB
SReclaimable:     181036 kB
SUnreclaim:       557804 kB
KernelStack:        1448 kB
PageTables:         3992 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     159700 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70712 MiB
End job on node141 at 2012-06-02 23:23:13