Trace number 3725761

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
pwbo 2.02SAT (TO)309 1800.04 900.215

General information on the benchmark

Namenormalized-PB06/OPT-SMALLINT/mps-v2-20-10/MIPLIB/
miplib/normalized-mps-v2-20-10-mod008.opb
MD5SUM4dd9ac2644c29d272d2675c7fc366ed6
Bench CategoryOPT-SMALLINT (optimisation, small integers)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark307
Best CPU time to get the best result obtained on this benchmark0.305953
Has Objective FunctionYES
SatisfiableYES
(Un)Satisfiability was provedYES
Best value of the objective function 307
Optimality of the best value was proved YES
Number of variables319
Total number of constraints6
Number of constraints which are clauses0
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints6
Minimum length of a constraint186
Maximum length of a constraint231
Number of terms in the objective function 319
Biggest coefficient in the objective function 87
Number of bits for the biggest coefficient in the objective function 7
Sum of the numbers in the objective function 23554
Number of bits of the sum of numbers in the objective function 15
Biggest number in a constraint 22000
Number of bits of the biggest number in a constraint 15
Biggest sum of numbers in a constraint 1027256
Number of bits of the biggest sum of numbers20
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Quality of the solution as a function of time


Download the above graph as a PDF file / as an EPS file

Solver Data 

0.00/0.00	c ==============================[ Solver Version ]===============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  pwbo:                     2.02 beta - May, 2012                            |
0.00/0.00	c |                                                                             |
0.00/0.00	c Parsing opb file format. File HOME/instance-3725761-1338651430.opb.
0.00/0.00	c Instance file HOME/instance-3725761-1338651430.opb
0.00/0.00	c File size is 16026 bytes.
0.00/0.00	c Cost Offset: 0
0.00/0.00	c Highest Coefficient sum: 1.02726e+06
0.00/0.00	c ============================[ Problem Statistics ]=============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  Parsing time:          0.00         s                                      |
0.00/0.00	c |  Number Variables:      319                                                 |
0.00/0.00	c |  Number Clauses:        0                                                   |
0.00/0.00	c |  Number Cardinality:    0                                                   |
0.00/0.00	c |  Number PB Constraints: 6                                                   |
0.00/0.00	c |                                                                             |
0.00/0.00	c |  Problem Type:          Optimization                                        |
0.00/0.00	c |  Objective Function:    PB Constraint                                       |
0.00/0.00	c |                                                                             |
0.00/0.00	c ===============================================================================
1800.04/900.21	o 604
1800.04/900.21	o 543
1800.04/900.21	o 423
1800.04/900.21	o 419
1800.04/900.21	o 418
1800.04/900.21	o 410
1800.04/900.21	o 384
1800.04/900.21	o 381
1800.04/900.21	o 377
1800.04/900.21	o 376
1800.04/900.21	o 375
1800.04/900.21	o 374
1800.04/900.21	o 373
1800.04/900.21	o 369
1800.04/900.21	o 326
1800.04/900.21	o 323
1800.04/900.21	o 319
1800.04/900.21	o 317
1800.04/900.21	o 312
1800.04/900.21	o 311
1800.04/900.21	o 309
1800.04/900.21	c ===============================================================================
1800.04/900.21	c Wall Clock Time: 900.21 s
1800.04/900.21	c Best Solution: 309
1800.04/900.21	s SATISFIABLE
1800.04/900.21	v -x111 x222 -x254 -x265 -x276 -x287 -x298 -x309 -x1 -x12 -x23 -x34 -x45 -x56 -x67 -x78 -x89 -x100 -x112 -x123 -x134 -x145 -x156 -x167 -x178 -x189 -x200 -x211 -x223 -x234 -x245 -x247 -x248 -x249 -x250 -x251 -x252 -x253 -x255 -x256 -x257 -x258 -x259 -x260 -x261 -x262 -x263 -x264 -x266 -x267 -x268 -x269 -x270 -x271 -x272 -x273 -x274 -x275 -x277 -x278 -x279 -x280 -x281 -x282 -x283 -x284 -x285 -x286 x288 -x289 -x290 -x291 -x292 -x293 -x294 -x295 -x296 -x297 -x299 -x300 -x301 -x302 -x303 -x304 x305 -x306 -x307 -x308 -x310 -x311 -x312 -x313 -x314 -x315 -x316 -x317 -x318 -x319 -x2 -x3 -x4 -x5 -x6 -x7 -x8 -x9 -x10 -x11 -x13 -x14 -x15 -x16 -x17 -x18 -x19 -x20 x21 -x22 -x24 -x25 -x26 -x27 -x28 x29 -x30 -x31 -x32 -x33 -x35 -x36 -x37 -x38 -x39 -x40 -x41 -x42 -x43 -x44 -x46 -x47 -x48 -x49 -x50 -x51 -x52 -x53 -x54 -x55 -x57 -x58 -x59 -x60 -x61 -x62 -x63 -x64 -x65 -x66 -x68 -x69 -x70 -x71 -x72 -x73 -x74 -x75 -x76 -x77 -x79 -x80 -x81 -x82 -x83 -x84 -x85 -x86 -x87 -x88 -x90 -x91 -x92 -x93 -x94 -x95 -x96 -x97 -x98 -x99 -x101 -x102 -x103 -x104 -x105 -x106 -x107 -x108 -x109 -x110 -x113 -x114 -x115 -x116 -x117 -x118 -x119 -x120 -x121 -x122 -x124 -x125 -x126 -x127 -x128 -x129 -x130 -x131 -x132 -x133 -x135 -x136 -x137 -x138 -x139 -x140 -x141 -x142 -x143 -x144 -x146 -x147 -x148 -x149 -x150 -x151 -x152 -x153 -x154 -x155 -x157 -x158 -x159 -x160 -x161 -x162 -x163 -x164 -x165 -x166 -x168 -x169 -x170 -x171 -x172 -x173 -x174 -x175 -x176 -x177 -x179 -x180 -x181 -x182 -x183 -x184 -x185 -x186 -x187 -x188 -x190 -x191 -x192 -x193 -x194 -x195 -x196 -x197 -x198 -x199 -x201 -x202 -x203 -x204 -x205 -x206 -x207 -x208 -x209 -x210 -x212 -x213 -x214 -x215 -x216 -x217 -x218 -x219 -x220 -x221 -x224 -x225 -x226 -x227 -x228 -x229 -x230 -x231 -x232 -x233 -x235 -x236 -x237 -x238 -x239 -x240 -x241 -x242 -x243 -x244 -x246

Verifier Data 

OK	309

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3725761-1338651430/watcher-3725761-1338651430 -o /tmp/evaluation-result-3725761-1338651430/solver-3725761-1338651430 -C 1800 -W 1900 -M 15500 pwbo2.02 -file-format=opb -verbosity=0 HOME/instance-3725761-1338651430.opb 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.31 1.79 1.92 4/177 23943
/proc/meminfo: memFree=28246848/32873844 swapFree=7124/7124
[pid=23943] ppid=23941 vsize=162456 CPUtime=0 cores=1,3,5,7
/proc/23943/stat : 23943 (pwbo2.02) S 23941 23943 22415 0 -1 4202496 1061 0 0 0 0 0 0 0 20 0 3 0 331268381 166354944 1008 33554432000 4194304 5964478 140735366874736 140735366874256 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23943/statm: 40614 1009 195 433 0 40178 0
[pid=23943/tid=23945] ppid=23941 vsize=162456 CPUtime=0 cores=1,3,5,7
/proc/23943/task/23945/stat : 23945 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 147 0 0 0 0 0 0 0 20 0 3 0 331268381 166354944 1009 33554432000 4194304 5964478 140735366874736 139904902203816 5136420 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=23943/tid=23946] ppid=23941 vsize=162456 CPUtime=0 cores=1,3,5,7
/proc/23943/task/23946/stat : 23946 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 123 0 0 0 0 0 0 0 20 0 3 0 331268381 166354944 1009 33554432000 4194304 5964478 140735366874736 139904891713992 18446744073699066235 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0

[startup+0.0786469 s]
/proc/loadavg: 1.31 1.79 1.92 4/177 23943
/proc/meminfo: memFree=28246848/32873844 swapFree=7124/7124
[pid=23943] ppid=23941 vsize=162456 CPUtime=0.14 cores=1,3,5,7
/proc/23943/stat : 23943 (pwbo2.02) S 23941 23943 22415 0 -1 4202496 1555 0 0 0 14 0 0 0 20 0 3 0 331268381 166354944 1502 33554432000 4194304 5964478 140735366874736 140735366874256 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23943/statm: 40614 1502 197 433 0 40178 0
[pid=23943/tid=23945] ppid=23941 vsize=162456 CPUtime=0.06 cores=1,3,5,7
/proc/23943/task/23945/stat : 23945 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 416 0 0 0 6 0 0 0 20 0 3 0 331268381 166354944 1502 33554432000 4194304 5964478 140735366874736 139904902203752 4403290 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=23943/tid=23946] ppid=23941 vsize=162456 CPUtime=0.06 cores=1,3,5,7
/proc/23943/task/23946/stat : 23946 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 347 0 0 0 6 0 0 0 20 0 3 0 331268381 166354944 1502 33554432000 4194304 5964478 140735366874736 139904891713368 4403717 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 0.14
Current children cumulated vsize (KiB) 162456

[startup+0.100288 s]
/proc/loadavg: 1.31 1.79 1.92 4/177 23943
/proc/meminfo: memFree=28246848/32873844 swapFree=7124/7124
[pid=23943] ppid=23941 vsize=162456 CPUtime=0.18 cores=1,3,5,7
/proc/23943/stat : 23943 (pwbo2.02) S 23941 23943 22415 0 -1 4202496 1623 0 0 0 18 0 0 0 20 0 3 0 331268381 166354944 1567 33554432000 4194304 5964478 140735366874736 140735366874256 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23943/statm: 40614 1567 197 433 0 40178 0
[pid=23943/tid=23945] ppid=23941 vsize=162456 CPUtime=0.08 cores=1,3,5,7
/proc/23943/task/23945/stat : 23945 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 416 0 0 0 8 0 0 0 20 0 3 0 331268381 166354944 1567 33554432000 4194304 5964478 140735366874736 139904902203752 4222416 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=23943/tid=23946] ppid=23941 vsize=162456 CPUtime=0.08 cores=1,3,5,7
/proc/23943/task/23946/stat : 23946 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 415 0 0 0 8 0 0 0 20 0 3 0 331268381 166354944 1567 33554432000 4194304 5964478 140735366874736 139904891713368 4222304 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 0.18
Current children cumulated vsize (KiB) 162456

[startup+0.300243 s]
/proc/loadavg: 1.31 1.79 1.92 4/177 23943
/proc/meminfo: memFree=28246848/32873844 swapFree=7124/7124
[pid=23943] ppid=23941 vsize=162456 CPUtime=0.58 cores=1,3,5,7
/proc/23943/stat : 23943 (pwbo2.02) S 23941 23943 22415 0 -1 4202496 2054 0 0 0 58 0 0 0 20 0 3 0 331268381 166354944 1988 33554432000 4194304 5964478 140735366874736 140735366874256 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23943/statm: 40614 1988 197 433 0 40178 0
[pid=23943/tid=23945] ppid=23941 vsize=162456 CPUtime=0.28 cores=1,3,5,7
/proc/23943/task/23945/stat : 23945 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 605 0 0 0 28 0 0 0 20 0 3 0 331268381 166354944 1988 33554432000 4194304 5964478 140735366874736 139904902203752 4392805 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=23943/tid=23946] ppid=23941 vsize=162456 CPUtime=0.28 cores=1,3,5,7
/proc/23943/task/23946/stat : 23946 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 657 0 0 0 28 0 0 0 20 0 3 0 331268381 166354944 1988 33554432000 4194304 5964478 140735366874736 139904891713368 4389199 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 0.58
Current children cumulated vsize (KiB) 162456

[startup+0.700244 s]
/proc/loadavg: 1.31 1.79 1.92 4/177 23943
/proc/meminfo: memFree=28246848/32873844 swapFree=7124/7124
[pid=23943] ppid=23941 vsize=162648 CPUtime=1.39 cores=1,3,5,7
/proc/23943/stat : 23943 (pwbo2.02) S 23941 23943 22415 0 -1 4202496 2714 0 0 0 138 1 0 0 20 0 3 0 331268381 166551552 2582 33554432000 4194304 5964478 140735366874736 140735366874256 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23943/statm: 40662 2582 197 433 0 40226 0
[pid=23943/tid=23945] ppid=23941 vsize=162648 CPUtime=0.68 cores=1,3,5,7
/proc/23943/task/23945/stat : 23945 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 829 0 0 0 68 0 0 0 20 0 3 0 331268381 166551552 2582 33554432000 4194304 5964478 140735366874736 139904902203752 4400010 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=23943/tid=23946] ppid=23941 vsize=162648 CPUtime=0.68 cores=1,3,5,7
/proc/23943/task/23946/stat : 23946 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 1093 0 0 0 68 0 0 0 20 0 3 0 331268381 166551552 2582 33554432000 4194304 5964478 140735366874736 139904891713368 5136420 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1.39
Current children cumulated vsize (KiB) 162648

[startup+1.50024 s]
/proc/loadavg: 1.31 1.79 1.92 6/183 23949
/proc/meminfo: memFree=28205824/32873844 swapFree=7124/7124
[pid=23943] ppid=23941 vsize=162456 CPUtime=2.98 cores=1,3,5,7
/proc/23943/stat : 23943 (pwbo2.02) S 23941 23943 22415 0 -1 4202496 4323 0 0 0 297 1 0 0 20 0 3 0 331268381 166354944 4140 33554432000 4194304 5964478 140735366874736 140735366874256 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23943/statm: 40614 4141 197 433 0 40178 0
[pid=23943/tid=23945] ppid=23941 vsize=162456 CPUtime=1.48 cores=1,3,5,7
/proc/23943/task/23945/stat : 23945 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 1164 0 0 0 148 0 0 0 20 0 3 0 331268381 166354944 4141 33554432000 4194304 5964478 140735366874736 139904902204344 4389129 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=23943/tid=23946] ppid=23941 vsize=162456 CPUtime=1.48 cores=1,3,5,7
/proc/23943/task/23946/stat : 23946 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 2368 0 0 0 148 0 0 0 20 0 3 0 331268381 166354944 4141 33554432000 4194304 5964478 140735366874736 139904891713848 5136420 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 2.98
Current children cumulated vsize (KiB) 162456

[startup+3.10025 s]
/proc/loadavg: 1.31 1.79 1.92 6/183 23949
/proc/meminfo: memFree=28184992/32873844 swapFree=7124/7124
[pid=23943] ppid=23941 vsize=162456 CPUtime=6.18 cores=1,3,5,7
/proc/23943/stat : 23943 (pwbo2.02) S 23941 23943 22415 0 -1 4202496 6370 0 0 0 614 4 0 0 20 0 3 0 331268381 166354944 5891 33554432000 4194304 5964478 140735366874736 140735366874256 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23943/statm: 40614 5891 197 433 0 40178 0
[pid=23943/tid=23945] ppid=23941 vsize=162456 CPUtime=3.07 cores=1,3,5,7
/proc/23943/task/23945/stat : 23945 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 1959 0 0 0 306 1 0 0 20 0 3 0 331268381 166354944 5891 33554432000 4194304 5964478 140735366874736 139904902204232 4403713 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=23943/tid=23946] ppid=23941 vsize=162456 CPUtime=3.07 cores=1,3,5,7
/proc/23943/task/23946/stat : 23946 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 3619 0 0 0 304 3 0 0 20 0 3 0 331268381 166354944 5891 33554432000 4194304 5964478 140735366874736 139904891713656 4400508 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 6.18
Current children cumulated vsize (KiB) 162456

[startup+6.30031 s]
/proc/loadavg: 1.53 1.83 1.94 6/183 23949
/proc/meminfo: memFree=28145172/32873844 swapFree=7124/7124
[pid=23943] ppid=23941 vsize=162456 CPUtime=12.58 cores=1,3,5,7
/proc/23943/stat : 23943 (pwbo2.02) S 23941 23943 22415 0 -1 4202496 8821 0 0 0 1253 5 0 0 20 0 3 0 331268381 166354944 7825 33554432000 4194304 5964478 140735366874736 140735366874256 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23943/statm: 40614 7825 197 433 0 40178 0
[pid=23943/tid=23945] ppid=23941 vsize=162456 CPUtime=6.26 cores=1,3,5,7
/proc/23943/task/23945/stat : 23945 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 3316 0 0 0 625 1 0 0 20 0 3 0 331268381 166354944 7825 33554432000 4194304 5964478 140735366874736 139904902203800 4403713 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=23943/tid=23946] ppid=23941 vsize=162456 CPUtime=6.26 cores=1,3,5,7
/proc/23943/task/23946/stat : 23946 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 4713 0 0 0 622 4 0 0 20 0 3 0 331268381 166354944 7825 33554432000 4194304 5964478 140735366874736 139904891713368 4403677 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 12.58
Current children cumulated vsize (KiB) 162456

[startup+12.7003 s]
/proc/loadavg: 1.73 1.86 1.95 6/183 23949
/proc/meminfo: memFree=28071680/32873844 swapFree=7124/7124
[pid=23943] ppid=23941 vsize=162456 CPUtime=25.37 cores=1,3,5,7
/proc/23943/stat : 23943 (pwbo2.02) S 23941 23943 22415 0 -1 4202496 12194 0 0 0 2531 6 0 0 20 0 3 0 331268381 166354944 10461 33554432000 4194304 5964478 140735366874736 140735366874256 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23943/statm: 40614 10461 197 433 0 40178 0
[pid=23943/tid=23945] ppid=23941 vsize=162456 CPUtime=12.65 cores=1,3,5,7
/proc/23943/task/23945/stat : 23945 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 6033 0 0 0 1264 1 0 0 20 0 3 0 331268381 166354944 10461 33554432000 4194304 5964478 140735366874736 139904902203720 4400010 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=23943/tid=23946] ppid=23941 vsize=162456 CPUtime=12.64 cores=1,3,5,7
/proc/23943/task/23946/stat : 23946 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 5369 0 0 0 1260 4 0 0 20 0 3 0 331268381 166354944 10461 33554432000 4194304 5964478 140735366874736 139904891713880 4400087 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 25.37
Current children cumulated vsize (KiB) 162456

[startup+25.5004 s]
/proc/loadavg: 2.38 2.00 1.99 6/183 23949
/proc/meminfo: memFree=27992180/32873844 swapFree=7124/7124
[pid=23943] ppid=23941 vsize=163420 CPUtime=50.96 cores=1,3,5,7
/proc/23943/stat : 23943 (pwbo2.02) S 23941 23943 22415 0 -1 4202496 17090 0 0 0 5085 11 0 0 20 0 3 0 331268381 167342080 16651 33554432000 4194304 5964478 140735366874736 140735366874256 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23943/statm: 40855 16651 197 433 0 40419 0
[pid=23943/tid=23945] ppid=23941 vsize=163420 CPUtime=25.41 cores=1,3,5,7
/proc/23943/task/23945/stat : 23945 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 10754 0 0 0 2534 7 0 0 20 0 3 0 331268381 167342080 16651 33554432000 4194304 5964478 140735366874736 139904902203720 4403590 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=23943/tid=23946] ppid=23941 vsize=163420 CPUtime=25.41 cores=1,3,5,7
/proc/23943/task/23946/stat : 23946 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 5544 0 0 0 2537 4 0 0 20 0 3 0 331268381 167342080 16651 33554432000 4194304 5964478 140735366874736 139904891714200 4400024 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 50.96
Current children cumulated vsize (KiB) 163420

[startup+51.1004 s]
/proc/loadavg: 2.93 2.16 2.04 6/183 23949
/proc/meminfo: memFree=27856228/32873844 swapFree=7124/7124
[pid=23943] ppid=23941 vsize=164384 CPUtime=102.16 cores=1,3,5,7
/proc/23943/stat : 23943 (pwbo2.02) S 23941 23943 22415 0 -1 4202496 25464 0 0 0 10199 17 0 0 20 0 3 0 331268381 168329216 24823 33554432000 4194304 5964478 140735366874736 140735366874256 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23943/statm: 41096 24823 197 433 0 40660 0
[pid=23943/tid=23945] ppid=23941 vsize=164384 CPUtime=50.94 cores=1,3,5,7
/proc/23943/task/23945/stat : 23945 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 12861 0 0 0 5086 8 0 0 20 0 3 0 331268381 168329216 24823 33554432000 4194304 5964478 140735366874736 139904902204344 4399920 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=23943/tid=23946] ppid=23941 vsize=164384 CPUtime=50.94 cores=1,3,5,7
/proc/23943/task/23946/stat : 23946 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 11811 0 0 0 5086 8 0 0 20 0 3 0 331268381 168329216 24823 33554432000 4194304 5964478 140735366874736 139904891713848 4403717 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 102.16
Current children cumulated vsize (KiB) 164384

[startup+102.307 s]
/proc/loadavg: 3.53 2.44 2.14 6/183 23949
/proc/meminfo: memFree=27632124/32873844 swapFree=7124/7124
[pid=23943] ppid=23941 vsize=229920 CPUtime=204.56 cores=1,3,5,7
/proc/23943/stat : 23943 (pwbo2.02) S 23941 23943 22415 0 -1 4202496 36474 0 0 0 20425 31 0 0 20 0 3 0 331268381 235438080 35833 33554432000 4194304 5964478 140735366874736 140735366874256 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23943/statm: 57480 35833 197 433 0 57044 0
[pid=23943/tid=23945] ppid=23941 vsize=229920 CPUtime=102.02 cores=1,3,5,7
/proc/23943/task/23945/stat : 23945 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 20165 0 0 0 10181 21 0 0 20 0 3 0 331268381 235438080 35833 33554432000 4194304 5964478 140735366874736 139904902203192 4399920 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=23943/tid=23946] ppid=23941 vsize=229920 CPUtime=102.02 cores=1,3,5,7
/proc/23943/task/23946/stat : 23946 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 15517 0 0 0 10191 11 0 0 20 0 3 0 331268381 235438080 35833 33554432000 4194304 5964478 140735366874736 139904891713848 4399986 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 204.56
Current children cumulated vsize (KiB) 229920

[startup+162.3 s]
/proc/loadavg: 3.83 2.72 2.26 6/183 23949
/proc/meminfo: memFree=27493980/32873844 swapFree=7124/7124
[pid=23943] ppid=23941 vsize=296416 CPUtime=324.52 cores=1,3,5,7
/proc/23943/stat : 23943 (pwbo2.02) S 23941 23943 22415 0 -1 4202496 47932 0 0 0 32410 42 0 0 20 0 3 0 331268381 303529984 46748 33554432000 4194304 5964478 140735366874736 140735366874256 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23943/statm: 74104 46748 197 433 0 73668 0
[pid=23943/tid=23945] ppid=23941 vsize=296416 CPUtime=161.85 cores=1,3,5,7
/proc/23943/task/23945/stat : 23945 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 25082 0 0 0 16157 28 0 0 20 0 3 0 331268381 303529984 46748 33554432000 4194304 5964478 140735366874736 139904902204344 4387948 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=23943/tid=23946] ppid=23941 vsize=296416 CPUtime=161.86 cores=1,3,5,7
/proc/23943/task/23946/stat : 23946 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 22058 0 0 0 16168 18 0 0 20 0 3 0 331268381 303529984 46748 33554432000 4194304 5964478 140735366874736 139904891713896 4389072 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 324.52
Current children cumulated vsize (KiB) 296416

[startup+222.3 s]
/proc/loadavg: 3.93 2.95 2.36 6/184 23952
/proc/meminfo: memFree=27410304/32873844 swapFree=7124/7124
[pid=23943] ppid=23941 vsize=293528 CPUtime=444.5 cores=1,3,5,7
/proc/23943/stat : 23943 (pwbo2.02) S 23941 23943 22415 0 -1 4202496 52727 0 0 0 44404 46 0 0 20 0 3 0 331268381 300572672 51003 33554432000 4194304 5964478 140735366874736 140735366874256 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23943/statm: 73382 51003 197 433 0 72946 0
[pid=23943/tid=23945] ppid=23941 vsize=293528 CPUtime=221.7 cores=1,3,5,7
/proc/23943/task/23945/stat : 23945 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 26976 0 0 0 22140 30 0 0 20 0 3 0 331268381 300572672 51003 33554432000 4194304 5964478 140735366874736 139904902203752 4400087 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=23943/tid=23946] ppid=23941 vsize=293528 CPUtime=221.7 cores=1,3,5,7
/proc/23943/task/23946/stat : 23946 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 24959 0 0 0 22149 21 0 0 20 0 3 0 331268381 300572672 51003 33554432000 4194304 5964478 140735366874736 139904891713896 4379296 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 444.5
Current children cumulated vsize (KiB) 293528

################
# More data... #
################

[pid=23943/tid=23946] ppid=23941 vsize=293528 CPUtime=281.54 cores=1,3,5,7
/proc/23943/task/23946/stat : 23946 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 27380 0 0 0 28129 25 0 0 20 0 3 0 331268381 300572672 55863 33554432000 4194304 5964478 140735366874736 139904891714200 4399920 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 564.46
Current children cumulated vsize (KiB) 293528

[startup+342.3 s]
/proc/loadavg: 3.99 3.29 2.56 6/183 23953
/proc/meminfo: memFree=27298100/32873844 swapFree=7124/7124
[pid=23943] ppid=23941 vsize=293528 CPUtime=684.42 cores=1,3,5,7
/proc/23943/stat : 23943 (pwbo2.02) S 23941 23943 22415 0 -1 4202496 57737 0 0 0 68391 51 0 0 20 0 3 0 331268381 300572672 56200 33554432000 4194304 5964478 140735366874736 140735366874256 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23943/statm: 73382 56200 197 433 0 72946 0
[pid=23943/tid=23945] ppid=23941 vsize=293528 CPUtime=341.34 cores=1,3,5,7
/proc/23943/task/23945/stat : 23945 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 29543 0 0 0 34100 34 0 0 20 0 3 0 331268381 300572672 56200 33554432000 4194304 5964478 140735366874736 139904902203192 4222464 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=23943/tid=23946] ppid=23941 vsize=293528 CPUtime=341.36 cores=1,3,5,7
/proc/23943/task/23946/stat : 23946 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 27402 0 0 0 34110 26 0 0 20 0 3 0 331268381 300572672 56200 33554432000 4194304 5964478 140735366874736 139904891714200 4222464 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 684.42
Current children cumulated vsize (KiB) 293528

[startup+402.3 s]
/proc/loadavg: 3.99 3.42 2.64 6/183 23953
/proc/meminfo: memFree=27285912/32873844 swapFree=7124/7124
[pid=23943] ppid=23941 vsize=293528 CPUtime=804.39 cores=1,3,5,7
/proc/23943/stat : 23943 (pwbo2.02) S 23941 23943 22415 0 -1 4202496 60102 0 0 0 80387 52 0 0 20 0 3 0 331268381 300572672 58563 33554432000 4194304 5964478 140735366874736 140735366874256 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23943/statm: 73382 58563 197 433 0 72946 0
[pid=23943/tid=23945] ppid=23941 vsize=293528 CPUtime=401.19 cores=1,3,5,7
/proc/23943/task/23945/stat : 23945 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 31908 0 0 0 40083 36 0 0 20 0 3 0 331268381 300572672 58563 33554432000 4194304 5964478 140735366874736 139904902203752 4403717 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=23943/tid=23946] ppid=23941 vsize=293528 CPUtime=401.21 cores=1,3,5,7
/proc/23943/task/23946/stat : 23946 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 27402 0 0 0 40094 27 0 0 20 0 3 0 331268381 300572672 58563 33554432000 4194304 5964478 140735366874736 139904891714200 4220672 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 804.39
Current children cumulated vsize (KiB) 293528

[startup+462.3 s]
/proc/loadavg: 3.99 3.52 2.72 6/183 23953
/proc/meminfo: memFree=27266364/32873844 swapFree=7124/7124
[pid=23943] ppid=23941 vsize=293528 CPUtime=924.37 cores=1,3,5,7
/proc/23943/stat : 23943 (pwbo2.02) S 23941 23943 22415 0 -1 4202496 60680 0 0 0 92385 52 0 0 20 0 3 0 331268381 300572672 59141 33554432000 4194304 5964478 140735366874736 140735366874256 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23943/statm: 73382 59141 197 433 0 72946 0
[pid=23943/tid=23945] ppid=23941 vsize=293528 CPUtime=461.03 cores=1,3,5,7
/proc/23943/task/23945/stat : 23945 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 32486 0 0 0 46065 38 0 0 20 0 3 0 331268381 300572672 59141 33554432000 4194304 5964478 140735366874736 139904902203752 4399920 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=23943/tid=23946] ppid=23941 vsize=293528 CPUtime=461.06 cores=1,3,5,7
/proc/23943/task/23946/stat : 23946 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 27402 0 0 0 46078 28 0 0 20 0 3 0 331268381 300572672 59141 33554432000 4194304 5964478 140735366874736 139904891714200 4403692 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 924.37
Current children cumulated vsize (KiB) 293528

[startup+522.3 s]
/proc/loadavg: 4.06 3.62 2.81 6/183 23953
/proc/meminfo: memFree=27264788/32873844 swapFree=7124/7124
[pid=23943] ppid=23941 vsize=295692 CPUtime=1044.35 cores=1,3,5,7
/proc/23943/stat : 23943 (pwbo2.02) S 23941 23943 22415 0 -1 4202496 61100 0 0 0 104383 52 0 0 20 0 3 0 331268381 302788608 59561 33554432000 4194304 5964478 140735366874736 140735366874256 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23943/statm: 73923 59561 197 433 0 73487 0
[pid=23943/tid=23945] ppid=23941 vsize=295692 CPUtime=520.87 cores=1,3,5,7
/proc/23943/task/23945/stat : 23945 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 32904 0 0 0 52049 38 0 0 20 0 3 0 331268381 302788608 59561 33554432000 4194304 5964478 140735366874736 139904902203736 4221120 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=23943/tid=23946] ppid=23941 vsize=295692 CPUtime=520.9 cores=1,3,5,7
/proc/23943/task/23946/stat : 23946 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 27404 0 0 0 52062 28 0 0 20 0 3 0 331268381 302788608 59561 33554432000 4194304 5964478 140735366874736 139904891714200 4399920 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1044.35
Current children cumulated vsize (KiB) 295692

[startup+582.3 s]
/proc/loadavg: 4.02 3.69 2.88 6/183 23954
/proc/meminfo: memFree=27257404/32873844 swapFree=7124/7124
[pid=23943] ppid=23941 vsize=297856 CPUtime=1164.34 cores=1,3,5,7
/proc/23943/stat : 23943 (pwbo2.02) S 23941 23943 22415 0 -1 4202496 62205 0 0 0 116381 53 0 0 20 0 3 0 331268381 305004544 59974 33554432000 4194304 5964478 140735366874736 140735366874256 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23943/statm: 74464 59974 197 433 0 74028 0
[pid=23943/tid=23945] ppid=23941 vsize=297856 CPUtime=580.72 cores=1,3,5,7
/proc/23943/task/23945/stat : 23945 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 34000 0 0 0 58033 39 0 0 20 0 3 0 331268381 305004544 59974 33554432000 4194304 5964478 140735366874736 139904902204344 4400401 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=23943/tid=23946] ppid=23941 vsize=297856 CPUtime=580.75 cores=1,3,5,7
/proc/23943/task/23946/stat : 23946 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 27413 0 0 0 58046 29 0 0 20 0 3 0 331268381 305004544 59974 33554432000 4194304 5964478 140735366874736 139904891714200 4221984 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1164.34
Current children cumulated vsize (KiB) 297856

[startup+642.3 s]
/proc/loadavg: 4.00 3.74 2.95 6/183 23954
/proc/meminfo: memFree=27240828/32873844 swapFree=7124/7124
[pid=23943] ppid=23941 vsize=293528 CPUtime=1284.33 cores=1,3,5,7
/proc/23943/stat : 23943 (pwbo2.02) S 23941 23943 22415 0 -1 4202496 62558 0 0 0 128380 53 0 0 20 0 3 0 331268381 300572672 59575 33554432000 4194304 5964478 140735366874736 140735366874256 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23943/statm: 73382 59575 197 433 0 72946 0
[pid=23943/tid=23945] ppid=23941 vsize=293528 CPUtime=640.56 cores=1,3,5,7
/proc/23943/task/23945/stat : 23945 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 34353 0 0 0 64016 40 0 0 20 0 3 0 331268381 300572672 59575 33554432000 4194304 5964478 140735366874736 139904902204344 4403396 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=23943/tid=23946] ppid=23941 vsize=293528 CPUtime=640.6 cores=1,3,5,7
/proc/23943/task/23946/stat : 23946 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 27413 0 0 0 64030 30 0 0 20 0 3 0 331268381 300572672 59575 33554432000 4194304 5964478 140735366874736 139904891714200 4399920 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1284.33
Current children cumulated vsize (KiB) 293528

[startup+702.3 s]
/proc/loadavg: 4.00 3.79 3.01 6/183 23954
/proc/meminfo: memFree=27220864/32873844 swapFree=7124/7124
[pid=23943] ppid=23941 vsize=293528 CPUtime=1404.3 cores=1,3,5,7
/proc/23943/stat : 23943 (pwbo2.02) S 23941 23943 22415 0 -1 4202496 67630 0 0 0 140373 57 0 0 20 0 3 0 331268381 300572672 64647 33554432000 4194304 5964478 140735366874736 140735366874256 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23943/statm: 73382 64647 197 433 0 72946 0
[pid=23943/tid=23945] ppid=23941 vsize=293528 CPUtime=700.4 cores=1,3,5,7
/proc/23943/task/23945/stat : 23945 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 36093 0 0 0 69999 41 0 0 20 0 3 0 331268381 300572672 64647 33554432000 4194304 5964478 140735366874736 139904902203736 4403713 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=23943/tid=23946] ppid=23941 vsize=293528 CPUtime=700.44 cores=1,3,5,7
/proc/23943/task/23946/stat : 23946 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 30745 0 0 0 70009 35 0 0 20 0 3 0 331268381 300572672 64647 33554432000 4194304 5964478 140735366874736 139904891713848 4400087 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1404.3
Current children cumulated vsize (KiB) 293528

[startup+762.307 s]
/proc/loadavg: 4.00 3.82 3.07 6/184 23955
/proc/meminfo: memFree=27220916/32873844 swapFree=7124/7124
[pid=23943] ppid=23941 vsize=293528 CPUtime=1524.3 cores=1,3,5,7
/proc/23943/stat : 23943 (pwbo2.02) S 23941 23943 22415 0 -1 4202496 67630 0 0 0 152373 57 0 0 20 0 3 0 331268381 300572672 64647 33554432000 4194304 5964478 140735366874736 140735366874256 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23943/statm: 73382 64647 197 433 0 72946 0
[pid=23943/tid=23945] ppid=23941 vsize=293528 CPUtime=760.25 cores=1,3,5,7
/proc/23943/task/23945/stat : 23945 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 36093 0 0 0 75983 42 0 0 20 0 3 0 331268381 300572672 64647 33554432000 4194304 5964478 140735366874736 139904902203736 4221984 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=23943/tid=23946] ppid=23941 vsize=293528 CPUtime=760.3 cores=1,3,5,7
/proc/23943/task/23946/stat : 23946 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 30745 0 0 0 75994 36 0 0 20 0 3 0 331268381 300572672 64647 33554432000 4194304 5964478 140735366874736 139904891713848 4221984 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1524.3
Current children cumulated vsize (KiB) 293528

[startup+822.3 s]
/proc/loadavg: 4.00 3.85 3.12 6/184 23957
/proc/meminfo: memFree=27210996/32873844 swapFree=7124/7124
[pid=23943] ppid=23941 vsize=359064 CPUtime=1644.26 cores=1,3,5,7
/proc/23943/stat : 23943 (pwbo2.02) S 23941 23943 22415 0 -1 4202496 71045 0 0 0 164366 60 0 0 20 0 3 0 331268381 367681536 67074 33554432000 4194304 5964478 140735366874736 140735366874256 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23943/statm: 89766 67074 197 433 0 89330 0
[pid=23943/tid=23945] ppid=23941 vsize=359064 CPUtime=820.09 cores=1,3,5,7
/proc/23943/task/23945/stat : 23945 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 36244 0 0 0 81966 43 0 0 20 0 3 0 331268381 367681536 67074 33554432000 4194304 5964478 140735366874736 139904902204232 4221984 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=23943/tid=23946] ppid=23941 vsize=359064 CPUtime=820.14 cores=1,3,5,7
/proc/23943/task/23946/stat : 23946 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 34009 0 0 0 81974 40 0 0 20 0 3 0 331268381 367681536 67074 33554432000 4194304 5964478 140735366874736 139904891713848 4399920 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1644.26
Current children cumulated vsize (KiB) 359064

[startup+882.3 s]
/proc/loadavg: 4.00 3.88 3.18 6/183 23958
/proc/meminfo: memFree=27201352/32873844 swapFree=7124/7124
[pid=23943] ppid=23941 vsize=359064 CPUtime=1764.23 cores=1,3,5,7
/proc/23943/stat : 23943 (pwbo2.02) S 23941 23943 22415 0 -1 4202496 73364 0 0 0 176361 62 0 0 20 0 3 0 331268381 367681536 69613 33554432000 4194304 5964478 140735366874736 140735366874256 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23943/statm: 89766 69613 197 433 0 89330 0
[pid=23943/tid=23945] ppid=23941 vsize=359064 CPUtime=879.93 cores=1,3,5,7
/proc/23943/task/23945/stat : 23945 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 37012 0 0 0 87948 45 0 0 20 0 3 0 331268381 367681536 69613 33554432000 4194304 5964478 140735366874736 139904902203928 4221120 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=23943/tid=23946] ppid=23941 vsize=359064 CPUtime=879.97 cores=1,3,5,7
/proc/23943/task/23946/stat : 23946 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 35560 0 0 0 87954 43 0 0 20 0 3 0 331268381 367681536 69613 33554432000 4194304 5964478 140735366874736 139904891713848 4379316 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1764.23
Current children cumulated vsize (KiB) 359064



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+900.2 s]
/proc/loadavg: 4.00 3.88 3.19 5/183 23958
/proc/meminfo: memFree=27201352/32873844 swapFree=7124/7124
[pid=23943] ppid=23941 vsize=359064 CPUtime=1800.04 cores=1,3,5,7
/proc/23943/stat : 23943 (pwbo2.02) S 23941 23943 22415 0 -1 4202496 73364 0 0 0 179942 62 0 0 20 0 3 0 331268381 367681536 69613 33554432000 4194304 5964478 140735366874736 140735366874256 4546685 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23943/statm: 89766 69613 197 433 0 89330 0
[pid=23943/tid=23945] ppid=23941 vsize=359064 CPUtime=897.79 cores=1,3,5,7
/proc/23943/task/23945/stat : 23945 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 37012 0 0 0 89734 45 0 0 20 0 3 0 331268381 367681536 69613 33554432000 4194304 5964478 140735366874736 139904902203928 4221120 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=23943/tid=23946] ppid=23941 vsize=359064 CPUtime=897.83 cores=1,3,5,7
/proc/23943/task/23946/stat : 23946 (pwbo2.02) R 23941 23943 22415 0 -1 4202560 35560 0 0 0 89740 43 0 0 20 0 3 0 331268381 367681536 69613 33554432000 4194304 5964478 140735366874736 139904891713848 4403698 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1800.04
Current children cumulated vsize (KiB) 359064

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child status: 0

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 23943 and gives
#  childrusage.ru_utime.tv_sec=1794
#  childrusage.ru_utime.tv_usec=769153
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=900863
# CPU time returned by wait4() is 1795.67
# while last known CPU time is 1800.04
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 900.215
CPU time (s): 1800.04
CPU user time (s): 1799.42
CPU system time (s): 0.62
CPU usage (%): 199.957
Max. virtual memory (cumulated for all children) (KiB): 359064

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1794.77
system time used= 0.900863
maximum resident set size= 278460
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 73366
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 147
involuntary context switches= 1876

runsolver used 3.81242 second user time and 3.78442 second system time

The end

Launcher Data 

Begin job on node134 at 2012-06-02 17:37:10
IDJOB=3725761
IDBENCH=1854
IDSOLVER=2309
FILE ID=node134/3725761-1338651430
RUNJOBID= node134-1338651430-23917
PBS_JOBID= 14636446
Free space on /tmp= 70356 MiB

SOLVER NAME= pwbo 2.02
BENCH NAME= PB06//final/normalized-PB06/OPT-SMALLINT/mps-v2-20-10/MIPLIB/miplib/normalized-mps-v2-20-10-mod008.opb
COMMAND LINE= pwbo2.02 -file-format=opb -verbosity=0 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3725761-1338651430/watcher-3725761-1338651430 -o /tmp/evaluation-result-3725761-1338651430/solver-3725761-1338651430 -C 1800 -W 1900 -M 15500  pwbo2.02 -file-format=opb -verbosity=0 HOME/instance-3725761-1338651430.opb

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 4dd9ac2644c29d272d2675c7fc366ed6
RANDOM SEED=739256189

node134.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.278
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.278
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.278
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.278
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.59
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.278
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.278
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.278
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.57
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.278
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        28247476 kB
Buffers:          455800 kB
Cached:          3267664 kB
SwapCached:           76 kB
Active:          1716392 kB
Inactive:        2017884 kB
Active(anon):       2048 kB
Inactive(anon):    10508 kB
Active(file):    1714344 kB
Inactive(file):  2007376 kB
Unevictable:        7124 kB
Mlocked:            7124 kB
SwapTotal:      67108856 kB
SwapFree:       67107756 kB
Dirty:             10872 kB
Writeback:            12 kB
AnonPages:         17324 kB
Mapped:            10356 kB
Shmem:                96 kB
Slab:             753700 kB
SReclaimable:     195372 kB
SUnreclaim:       558328 kB
KernelStack:        1440 kB
PageTables:         3928 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     106012 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346376 kB
VmallocChunk:   34341987860 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70348 MiB
End job on node134 at 2012-06-02 17:52:12