Trace number 3726063

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
pwbo 2.02SAT (TO)44 1800.57 900.322

General information on the benchmark

Namenormalized-PB06/OPT-SMALLINT/submitted-PB06/
manquiho/market-split/normalized-opt-market-split_8_70_1.opb
MD5SUM0263d0a7c5e3320bd7cce9534c14bf27
Bench CategoryOPT-SMALLINT (optimisation, small integers)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark15
Best CPU time to get the best result obtained on this benchmark1792.11
Has Objective FunctionYES
SatisfiableYES
(Un)Satisfiability was provedYES
Best value of the objective function 32
Optimality of the best value was proved NO
Number of variables198
Total number of constraints16
Number of constraints which are clauses0
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints16
Minimum length of a constraint85
Maximum length of a constraint86
Number of terms in the objective function 128
Biggest coefficient in the objective function 128
Number of bits for the biggest coefficient in the objective function 8
Sum of the numbers in the objective function 4080
Number of bits of the sum of numbers in the objective function 12
Biggest number in a constraint 1884
Number of bits of the biggest number in a constraint 11
Biggest sum of numbers in a constraint 6162
Number of bits of the biggest sum of numbers13
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Quality of the solution as a function of time


Download the above graph as a PDF file / as an EPS file

Solver Data 

0.00/0.00	c ==============================[ Solver Version ]===============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  pwbo:                     2.02 beta - May, 2012                            |
0.00/0.00	c |                                                                             |
0.00/0.00	c Parsing opb file format. File HOME/instance-3726063-1338656773.opb.
0.00/0.00	c Instance file HOME/instance-3726063-1338656773.opb
0.00/0.00	c File size is 13239 bytes.
0.00/0.00	c Cost Offset: 0
0.00/0.00	c Highest Coefficient sum: 6162
0.00/0.00	c ============================[ Problem Statistics ]=============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  Parsing time:          0.00         s                                      |
0.00/0.00	c |  Number Variables:      198                                                 |
0.00/0.00	c |  Number Clauses:        0                                                   |
0.00/0.00	c |  Number Cardinality:    0                                                   |
0.00/0.00	c |  Number PB Constraints: 16                                                  |
0.00/0.00	c |                                                                             |
0.00/0.00	c |  Problem Type:          Optimization                                        |
0.00/0.00	c |  Objective Function:    PB Constraint                                       |
0.00/0.00	c |                                                                             |
0.00/0.00	c ===============================================================================
1800.19/900.31	o 1039
1800.19/900.31	o 904
1800.19/900.31	o 901
1800.19/900.31	o 874
1800.19/900.31	o 590
1800.19/900.31	o 502
1800.19/900.31	o 369
1800.19/900.31	o 360
1800.19/900.31	o 340
1800.19/900.31	o 320
1800.19/900.31	o 295
1800.19/900.31	o 289
1800.19/900.31	o 281
1800.19/900.31	o 271
1800.19/900.31	o 205
1800.19/900.31	o 179
1800.19/900.31	o 166
1800.19/900.31	o 164
1800.19/900.31	o 162
1800.19/900.31	o 159
1800.19/900.31	o 150
1800.19/900.31	o 98
1800.19/900.31	o 92
1800.19/900.31	o 69
1800.19/900.31	o 67
1800.19/900.31	o 63
1800.19/900.31	o 61
1800.19/900.31	o 58
1800.19/900.31	o 53
1800.19/900.31	o 51
1800.19/900.31	o 44
1800.19/900.31	c ===============================================================================
1800.19/900.31	c Wall Clock Time: 900.31 s
1800.19/900.31	c Best Solution: 44
1800.19/900.31	s SATISFIABLE
1800.19/900.31	v -x1 -x2 -x3 -x4 -x5 -x6 -x7 -x8 x9 x10 x11 -x12 -x13 -x14 -x15 -x16 -x17 -x18 -x19 -x20 -x21 -x22 -x23 -x24 -x25 -x26 -x27 -x28 -x29 -x30 -x31 -x32 -x33 -x34 -x35 -x36 -x37 -x38 -x39 -x40 -x41 -x42 -x43 -x44 -x45 -x46 -x47 -x48 -x49 -x50 -x51 -x52 -x53 -x54 -x55 -x56 -x57 -x58 x59 x60 -x61 -x62 -x63 -x64 -x65 x66 -x67 -x68 -x69 -x70 -x71 -x72 -x73 -x74 -x75 -x76 -x77 -x78 -x79 -x80 x81 -x82 -x83 -x84 x85 -x86 -x87 -x88 -x89 -x90 -x91 -x92 -x93 -x94 -x95 -x96 -x97 -x98 -x99 -x100 -x101 -x102 -x103 -x104 -x105 -x106 -x107 -x108 -x109 -x110 -x111 -x112 -x113 x114 x115 -x116 -x117 -x118 -x119 -x120 -x121 -x122 -x123 -x124 -x125 -x126 -x127 -x128 x129 x130 x131 -x132 -x133 -x134 -x135 -x136 x137 -x138 -x139 x140 x141 x142 -x144 x145 x146 x147 -x148 -x149 -x150 x151 -x152 -x153 x154 x155 x156 -x157 -x158 x159 x160 -x161 x162 -x163 x164 -x165 x166 -x167 x168 -x169 x170 -x171 -x172 -x173 -x174 -x175 x176 x177 -x178 x179 x180 x181 x182 x183 -x184 -x185 -x186 -x187 -x188 x189 -x190 x191 -x192 -x193 x194 -x195 -x196 x197 x198 x143

Verifier Data 

OK	44

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3726063-1338656773/watcher-3726063-1338656773 -o /tmp/evaluation-result-3726063-1338656773/solver-3726063-1338656773 -C 1800 -W 1900 -M 15500 pwbo2.02 -file-format=opb -verbosity=0 HOME/instance-3726063-1338656773.opb 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.00 4.00 3.93 4/180 4338
/proc/meminfo: memFree=28739952/32873844 swapFree=7380/7380
[pid=4338] ppid=4336 vsize=162500 CPUtime=0 cores=1,3,5,7
/proc/4338/stat : 4338 (pwbo2.02) S 4336 4338 2131 0 -1 4202496 883 0 0 0 0 0 0 0 20 0 3 0 331802122 166400000 832 33554432000 4194304 5964478 140735935254624 140735935254144 4546685 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/4338/statm: 40625 832 194 433 0 40189 0
[pid=4338/tid=4339] ppid=4336 vsize=162500 CPUtime=0 cores=1,3,5,7
/proc/4338/task/4339/stat : 4339 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 50 0 0 0 0 0 0 0 20 0 3 0 331802122 166400000 832 33554432000 4194304 5964478 140735935254624 140124101413936 4381960 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=4338/tid=4340] ppid=4336 vsize=162500 CPUtime=0 cores=1,3,5,7
/proc/4338/task/4340/stat : 4340 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 48 0 0 0 0 0 0 0 20 0 3 0 331802122 166400000 832 33554432000 4194304 5964478 140735935254624 140124090923992 4222416 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0

[startup+0.0333821 s]
/proc/loadavg: 4.00 4.00 3.93 4/180 4338
/proc/meminfo: memFree=28739952/32873844 swapFree=7380/7380
[pid=4338] ppid=4336 vsize=162500 CPUtime=0.05 cores=1,3,5,7
/proc/4338/stat : 4338 (pwbo2.02) S 4336 4338 2131 0 -1 4202496 1002 0 0 0 5 0 0 0 20 0 3 0 331802122 166400000 951 33554432000 4194304 5964478 140735935254624 140735935254144 4546685 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/4338/statm: 40625 951 196 433 0 40189 0
[pid=4338/tid=4339] ppid=4336 vsize=162500 CPUtime=0.01 cores=1,3,5,7
/proc/4338/task/4339/stat : 4339 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 102 0 0 0 1 0 0 0 20 0 3 0 331802122 166400000 951 33554432000 4194304 5964478 140735935254624 140124101412904 5136420 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=4338/tid=4340] ppid=4336 vsize=162500 CPUtime=0.01 cores=1,3,5,7
/proc/4338/task/4340/stat : 4340 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 115 0 0 0 1 0 0 0 20 0 3 0 331802122 166400000 951 33554432000 4194304 5964478 140735935254624 140124090923992 4389393 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 162500

[startup+0.100291 s]
/proc/loadavg: 4.00 4.00 3.93 4/180 4338
/proc/meminfo: memFree=28739952/32873844 swapFree=7380/7380
[pid=4338] ppid=4336 vsize=162500 CPUtime=0.19 cores=1,3,5,7
/proc/4338/stat : 4338 (pwbo2.02) S 4336 4338 2131 0 -1 4202496 1270 0 0 0 19 0 0 0 20 0 3 0 331802122 166400000 1219 33554432000 4194304 5964478 140735935254624 140735935254144 4546685 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/4338/statm: 40625 1219 196 433 0 40189 0
[pid=4338/tid=4339] ppid=4336 vsize=162500 CPUtime=0.08 cores=1,3,5,7
/proc/4338/task/4339/stat : 4339 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 239 0 0 0 8 0 0 0 20 0 3 0 331802122 166400000 1219 33554432000 4194304 5964478 140735935254624 140124101412856 4400001 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=4338/tid=4340] ppid=4336 vsize=162500 CPUtime=0.08 cores=1,3,5,7
/proc/4338/task/4340/stat : 4340 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 246 0 0 0 8 0 0 0 20 0 3 0 331802122 166400000 1220 33554432000 4194304 5964478 140735935254624 140124090923368 4389940 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 162500

[startup+0.300245 s]
/proc/loadavg: 4.00 4.00 3.93 4/180 4338
/proc/meminfo: memFree=28739952/32873844 swapFree=7380/7380
[pid=4338] ppid=4336 vsize=162500 CPUtime=0.58 cores=1,3,5,7
/proc/4338/stat : 4338 (pwbo2.02) S 4336 4338 2131 0 -1 4202496 1909 0 0 0 58 0 0 0 20 0 3 0 331802122 166400000 1858 33554432000 4194304 5964478 140735935254624 140735935254144 4546685 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/4338/statm: 40625 1858 196 433 0 40189 0
[pid=4338/tid=4339] ppid=4336 vsize=162500 CPUtime=0.28 cores=1,3,5,7
/proc/4338/task/4339/stat : 4339 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 598 0 0 0 28 0 0 0 20 0 3 0 331802122 166400000 1858 33554432000 4194304 5964478 140735935254624 140124101412856 4220672 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=4338/tid=4340] ppid=4336 vsize=162500 CPUtime=0.28 cores=1,3,5,7
/proc/4338/task/4340/stat : 4340 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 526 0 0 0 28 0 0 0 20 0 3 0 331802122 166400000 1858 33554432000 4194304 5964478 140735935254624 140124090923368 4389072 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 0.58
Current children cumulated vsize (KiB) 162500

[startup+0.700276 s]
/proc/loadavg: 4.00 4.00 3.93 4/180 4338
/proc/meminfo: memFree=28739952/32873844 swapFree=7380/7380
[pid=4338] ppid=4336 vsize=162500 CPUtime=1.39 cores=1,3,5,7
/proc/4338/stat : 4338 (pwbo2.02) S 4336 4338 2131 0 -1 4202496 2911 0 0 0 138 1 0 0 20 0 3 0 331802122 166400000 2860 33554432000 4194304 5964478 140735935254624 140735935254144 4546685 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/4338/statm: 40625 2860 196 433 0 40189 0
[pid=4338/tid=4339] ppid=4336 vsize=162500 CPUtime=0.68 cores=1,3,5,7
/proc/4338/task/4339/stat : 4339 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 1216 0 0 0 68 0 0 0 20 0 3 0 331802122 166400000 2860 33554432000 4194304 5964478 140735935254624 140124101412904 4222464 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=4338/tid=4340] ppid=4336 vsize=162500 CPUtime=0.68 cores=1,3,5,7
/proc/4338/task/4340/stat : 4340 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 910 0 0 0 68 0 0 0 20 0 3 0 331802122 166400000 2860 33554432000 4194304 5964478 140735935254624 140124090923320 4403717 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1.39
Current children cumulated vsize (KiB) 162500

[startup+1.50029 s]
/proc/loadavg: 4.00 4.00 3.93 5/183 4341
/proc/meminfo: memFree=28727536/32873844 swapFree=7380/7380
[pid=4338] ppid=4336 vsize=162644 CPUtime=2.98 cores=1,3,5,7
/proc/4338/stat : 4338 (pwbo2.02) S 4336 4338 2131 0 -1 4202496 3594 0 0 0 297 1 0 0 20 0 3 0 331802122 166547456 3543 33554432000 4194304 5964478 140735935254624 140735935254144 4546685 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/4338/statm: 40661 3543 196 433 0 40225 0
[pid=4338/tid=4339] ppid=4336 vsize=162644 CPUtime=1.48 cores=1,3,5,7
/proc/4338/task/4339/stat : 4339 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 1754 0 0 0 148 0 0 0 20 0 3 0 331802122 166547456 3543 33554432000 4194304 5964478 140735935254624 140124101412664 4221984 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=4338/tid=4340] ppid=4336 vsize=162644 CPUtime=1.48 cores=1,3,5,7
/proc/4338/task/4340/stat : 4340 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 1055 0 0 0 148 0 0 0 20 0 3 0 331802122 166547456 3543 33554432000 4194304 5964478 140735935254624 140124090923320 4389147 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 2.98
Current children cumulated vsize (KiB) 162644

[startup+3.10037 s]
/proc/loadavg: 4.00 4.00 3.93 5/183 4341
/proc/meminfo: memFree=28724296/32873844 swapFree=7380/7380
[pid=4338] ppid=4336 vsize=162860 CPUtime=6.19 cores=1,3,5,7
/proc/4338/stat : 4338 (pwbo2.02) S 4336 4338 2131 0 -1 4202496 4863 0 0 0 617 2 0 0 20 0 3 0 331802122 166768640 4812 33554432000 4194304 5964478 140735935254624 140735935254144 4546685 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/4338/statm: 40715 4812 196 433 0 40279 0
[pid=4338/tid=4339] ppid=4336 vsize=162860 CPUtime=3.08 cores=1,3,5,7
/proc/4338/task/4339/stat : 4339 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 2717 0 0 0 308 0 0 0 20 0 3 0 331802122 166768640 4812 33554432000 4194304 5964478 140735935254624 140124101412904 4399920 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=4338/tid=4340] ppid=4336 vsize=162860 CPUtime=3.08 cores=1,3,5,7
/proc/4338/task/4340/stat : 4340 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 1361 0 0 0 308 0 0 0 20 0 3 0 331802122 166768640 4812 33554432000 4194304 5964478 140735935254624 140124090923320 4392962 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 6.19
Current children cumulated vsize (KiB) 162860

[startup+6.30038 s]
/proc/loadavg: 4.00 4.00 3.93 5/183 4341
/proc/meminfo: memFree=28710152/32873844 swapFree=7380/7380
[pid=4338] ppid=4336 vsize=162932 CPUtime=12.59 cores=1,3,5,7
/proc/4338/stat : 4338 (pwbo2.02) S 4336 4338 2131 0 -1 4202496 7108 0 0 0 1256 3 0 0 20 0 3 0 331802122 166842368 7057 33554432000 4194304 5964478 140735935254624 140735935254144 4546685 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/4338/statm: 40733 7057 196 433 0 40297 0
[pid=4338/tid=4339] ppid=4336 vsize=162932 CPUtime=6.28 cores=1,3,5,7
/proc/4338/task/4339/stat : 4339 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 4172 0 0 0 626 2 0 0 20 0 3 0 331802122 166842368 7057 33554432000 4194304 5964478 140735935254624 140124101412856 5136420 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=4338/tid=4340] ppid=4336 vsize=162932 CPUtime=6.29 cores=1,3,5,7
/proc/4338/task/4340/stat : 4340 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 2151 0 0 0 627 2 0 0 20 0 3 0 331802122 166842368 7057 33554432000 4194304 5964478 140735935254624 140124090923320 4388815 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 12.59
Current children cumulated vsize (KiB) 162932

[startup+12.7004 s]
/proc/loadavg: 4.00 4.00 3.93 5/183 4341
/proc/meminfo: memFree=28691956/32873844 swapFree=7380/7380
[pid=4338] ppid=4336 vsize=163308 CPUtime=25.38 cores=1,3,5,7
/proc/4338/stat : 4338 (pwbo2.02) S 4336 4338 2131 0 -1 4202496 11366 0 0 0 2533 5 0 0 20 0 3 0 331802122 167227392 11315 33554432000 4194304 5964478 140735935254624 140735935254144 4546685 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/4338/statm: 40827 11315 196 433 0 40391 0
[pid=4338/tid=4339] ppid=4336 vsize=163308 CPUtime=12.68 cores=1,3,5,7
/proc/4338/task/4339/stat : 4339 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 6717 0 0 0 1265 3 0 0 20 0 3 0 331802122 167227392 11315 33554432000 4194304 5964478 140735935254624 140124101412856 5136420 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=4338/tid=4340] ppid=4336 vsize=163308 CPUtime=12.68 cores=1,3,5,7
/proc/4338/task/4340/stat : 4340 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 3864 0 0 0 1265 3 0 0 20 0 3 0 331802122 167227392 11315 33554432000 4194304 5964478 140735935254624 140124090923320 5136420 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 25.38
Current children cumulated vsize (KiB) 163308

[startup+25.5004 s]
/proc/loadavg: 4.00 4.00 3.93 5/183 4341
/proc/meminfo: memFree=28667136/32873844 swapFree=7380/7380
[pid=4338] ppid=4336 vsize=163468 CPUtime=50.98 cores=1,3,5,7
/proc/4338/stat : 4338 (pwbo2.02) S 4336 4338 2131 0 -1 4202496 15534 0 0 0 5090 8 0 0 20 0 3 0 331802122 167391232 15483 33554432000 4194304 5964478 140735935254624 140735935254144 4546685 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/4338/statm: 40867 15483 196 433 0 40431 0
[pid=4338/tid=4339] ppid=4336 vsize=163468 CPUtime=25.48 cores=1,3,5,7
/proc/4338/task/4339/stat : 4339 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 10608 0 0 0 2542 6 0 0 20 0 3 0 331802122 167391232 15483 33554432000 4194304 5964478 140735935254624 140124101412856 4400632 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=4338/tid=4340] ppid=4336 vsize=163468 CPUtime=25.48 cores=1,3,5,7
/proc/4338/task/4340/stat : 4340 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 4141 0 0 0 2545 3 0 0 20 0 3 0 331802122 167391232 15483 33554432000 4194304 5964478 140735935254624 140124090923320 4400542 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 50.98
Current children cumulated vsize (KiB) 163468

[startup+51.1004 s]
/proc/loadavg: 4.00 4.00 3.93 5/183 4341
/proc/meminfo: memFree=28644692/32873844 swapFree=7380/7380
[pid=4338] ppid=4336 vsize=163708 CPUtime=102.17 cores=1,3,5,7
/proc/4338/stat : 4338 (pwbo2.02) S 4336 4338 2131 0 -1 4202496 19603 0 0 0 10205 12 0 0 20 0 3 0 331802122 167636992 19552 33554432000 4194304 5964478 140735935254624 140735935254144 4546685 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/4338/statm: 40927 19552 196 433 0 40491 0
[pid=4338/tid=4339] ppid=4336 vsize=163708 CPUtime=51.08 cores=1,3,5,7
/proc/4338/task/4339/stat : 4339 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 13138 0 0 0 5100 8 0 0 20 0 3 0 331802122 167636992 19552 33554432000 4194304 5964478 140735935254624 140124101412856 4399920 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=4338/tid=4340] ppid=4336 vsize=163708 CPUtime=51.08 cores=1,3,5,7
/proc/4338/task/4340/stat : 4340 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 5680 0 0 0 5103 5 0 0 20 0 3 0 331802122 167636992 19552 33554432000 4194304 5964478 140735935254624 140124090923320 5130901 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 102.17
Current children cumulated vsize (KiB) 163708

[startup+102.307 s]
/proc/loadavg: 4.00 4.00 3.93 5/183 4341
/proc/meminfo: memFree=28592612/32873844 swapFree=7380/7380
[pid=4338] ppid=4336 vsize=230384 CPUtime=204.57 cores=1,3,5,7
/proc/4338/stat : 4338 (pwbo2.02) S 4336 4338 2131 0 -1 4202496 28670 0 0 0 20436 21 0 0 20 0 3 0 331802122 235913216 28619 33554432000 4194304 5964478 140735935254624 140735935254144 4546685 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/4338/statm: 57596 28619 196 433 0 57160 0
[pid=4338/tid=4339] ppid=4336 vsize=230384 CPUtime=102.29 cores=1,3,5,7
/proc/4338/task/4339/stat : 4339 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 20358 0 0 0 10213 16 0 0 20 0 3 0 331802122 235913216 28619 33554432000 4194304 5964478 140735935254624 140124101412856 4221120 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=4338/tid=4340] ppid=4336 vsize=230384 CPUtime=102.29 cores=1,3,5,7
/proc/4338/task/4340/stat : 4340 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 7527 0 0 0 10220 9 0 0 20 0 3 0 331802122 235913216 28619 33554432000 4194304 5964478 140735935254624 140124090923320 4399920 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 204.57
Current children cumulated vsize (KiB) 230384

[startup+162.3 s]
/proc/loadavg: 4.00 4.00 3.93 5/183 4341
/proc/meminfo: memFree=28542428/32873844 swapFree=7380/7380
[pid=4338] ppid=4336 vsize=232184 CPUtime=324.52 cores=1,3,5,7
/proc/4338/stat : 4338 (pwbo2.02) S 4336 4338 2131 0 -1 4202496 36064 0 0 0 32420 32 0 0 20 0 3 0 331802122 237756416 35819 33554432000 4194304 5964478 140735935254624 140735935254144 4546685 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/4338/statm: 58046 35819 196 433 0 57610 0
[pid=4338/tid=4339] ppid=4336 vsize=232184 CPUtime=162.27 cores=1,3,5,7
/proc/4338/task/4339/stat : 4339 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 25559 0 0 0 16202 25 0 0 20 0 3 0 331802122 237756416 35819 33554432000 4194304 5964478 140735935254624 140124101412856 4399938 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=4338/tid=4340] ppid=4336 vsize=232184 CPUtime=162.29 cores=1,3,5,7
/proc/4338/task/4340/stat : 4340 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 9720 0 0 0 16216 13 0 0 20 0 3 0 331802122 237756416 35819 33554432000 4194304 5964478 140735935254624 140124090923352 4399920 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 324.52
Current children cumulated vsize (KiB) 232184

[startup+222.3 s]
/proc/loadavg: 4.00 4.00 3.93 5/183 4341
/proc/meminfo: memFree=28498560/32873844 swapFree=7380/7380
[pid=4338] ppid=4336 vsize=233336 CPUtime=444.45 cores=1,3,5,7
/proc/4338/stat : 4338 (pwbo2.02) S 4336 4338 2131 0 -1 4202496 44233 0 0 0 44399 46 0 0 20 0 3 0 331802122 238936064 44036 33554432000 4194304 5964478 140735935254624 140735935254144 4546685 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/4338/statm: 58334 44036 196 433 0 57898 0
[pid=4338/tid=4339] ppid=4336 vsize=233336 CPUtime=222.22 cores=1,3,5,7
/proc/4338/task/4339/stat : 4339 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 32269 0 0 0 22185 37 0 0 20 0 3 0 331802122 238936064 44036 33554432000 4194304 5964478 140735935254624 140124101412856 4221120 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=4338/tid=4340] ppid=4336 vsize=233336 CPUtime=222.29 cores=1,3,5,7
/proc/4338/task/4340/stat : 4340 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 11179 0 0 0 22213 16 0 0 20 0 3 0 331802122 238936064 44036 33554432000 4194304 5964478 140735935254624 140124090923320 4399920 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 444.45
Current children cumulated vsize (KiB) 233336

################
# More data... #
################

/proc/4338/task/4339/stat : 4339 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 38130 0 0 0 28174 48 0 0 20 0 3 0 331802122 307777536 51143 33554432000 4194304 5964478 140735935254624 140124101412856 4399920 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=4338/tid=4340] ppid=4336 vsize=300564 CPUtime=282.27 cores=1,3,5,7
/proc/4338/task/4340/stat : 4340 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 11197 0 0 0 28210 17 0 0 20 0 3 0 331802122 307777536 51143 33554432000 4194304 5964478 140735935254624 140124090923320 4399975 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 564.41
Current children cumulated vsize (KiB) 300564

[startup+342.3 s]
/proc/loadavg: 4.00 4.00 3.93 5/183 4345
/proc/meminfo: memFree=28413752/32873844 swapFree=7380/7380
[pid=4338] ppid=4336 vsize=302580 CPUtime=684.39 cores=1,3,5,7
/proc/4338/stat : 4338 (pwbo2.02) S 4336 4338 2131 0 -1 4202496 56046 0 0 0 68371 68 0 0 20 0 3 0 331802122 309841920 57073 33554432000 4194304 5964478 140735935254624 140735935254144 4546685 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/4338/statm: 75645 57073 196 433 0 75209 0
[pid=4338/tid=4339] ppid=4336 vsize=302580 CPUtime=342.22 cores=1,3,5,7
/proc/4338/task/4339/stat : 4339 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 44063 0 0 0 34161 61 0 0 20 0 3 0 331802122 309841920 57073 33554432000 4194304 5964478 140735935254624 140124101412856 4399920 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=4338/tid=4340] ppid=4336 vsize=302580 CPUtime=342.27 cores=1,3,5,7
/proc/4338/task/4340/stat : 4340 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 11198 0 0 0 34209 18 0 0 20 0 3 0 331802122 309841920 57073 33554432000 4194304 5964478 140735935254624 140124090923320 4399938 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 684.39
Current children cumulated vsize (KiB) 302580

[startup+402.3 s]
/proc/loadavg: 4.00 4.00 3.93 5/183 4345
/proc/meminfo: memFree=28376860/32873844 swapFree=7380/7380
[pid=4338] ppid=4336 vsize=303084 CPUtime=804.37 cores=1,3,5,7
/proc/4338/stat : 4338 (pwbo2.02) S 4336 4338 2131 0 -1 4202496 59841 0 0 0 80363 74 0 0 20 0 3 0 331802122 310358016 60775 33554432000 4194304 5964478 140735935254624 140735935254144 4546685 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/4338/statm: 75771 60775 196 433 0 75335 0
[pid=4338/tid=4339] ppid=4336 vsize=303084 CPUtime=402.23 cores=1,3,5,7
/proc/4338/task/4339/stat : 4339 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 47589 0 0 0 40155 68 0 0 20 0 3 0 331802122 310358016 60775 33554432000 4194304 5964478 140735935254624 140124101412376 5136420 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=4338/tid=4340] ppid=4336 vsize=303084 CPUtime=402.28 cores=1,3,5,7
/proc/4338/task/4340/stat : 4340 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 11467 0 0 0 40209 19 0 0 20 0 3 0 331802122 310358016 60775 33554432000 4194304 5964478 140735935254624 140124090923352 4399920 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 804.37
Current children cumulated vsize (KiB) 303084

[startup+462.3 s]
/proc/loadavg: 4.00 4.00 3.93 5/183 4345
/proc/meminfo: memFree=28359120/32873844 swapFree=7380/7380
[pid=4338] ppid=4336 vsize=303588 CPUtime=924.35 cores=1,3,5,7
/proc/4338/stat : 4338 (pwbo2.02) S 4336 4338 2131 0 -1 4202496 62337 0 0 0 92356 79 0 0 20 0 3 0 331802122 310874112 63175 33554432000 4194304 5964478 140735935254624 140735935254144 4546685 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/4338/statm: 75897 63175 196 433 0 75461 0
[pid=4338/tid=4339] ppid=4336 vsize=303588 CPUtime=462.23 cores=1,3,5,7
/proc/4338/task/4339/stat : 4339 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 48927 0 0 0 46152 71 0 0 20 0 3 0 331802122 310874112 63175 33554432000 4194304 5964478 140735935254624 140124101412904 4399938 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=4338/tid=4340] ppid=4336 vsize=303588 CPUtime=462.28 cores=1,3,5,7
/proc/4338/task/4340/stat : 4340 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 12625 0 0 0 46205 23 0 0 20 0 3 0 331802122 310874112 63175 33554432000 4194304 5964478 140735935254624 140124090923320 4221120 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 924.35
Current children cumulated vsize (KiB) 303588

[startup+522.3 s]
/proc/loadavg: 4.00 4.00 3.93 5/183 4386
/proc/meminfo: memFree=28493868/32873844 swapFree=7380/7380
[pid=4338] ppid=4336 vsize=369340 CPUtime=1044.3 cores=1,3,5,7
/proc/4338/stat : 4338 (pwbo2.02) S 4336 4338 2131 0 -1 4202496 65965 0 0 0 104346 84 0 0 20 0 3 0 331802122 378204160 66822 33554432000 4194304 5964478 140735935254624 140735935254144 4546685 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/4338/statm: 92335 66822 196 433 0 91899 0
[pid=4338/tid=4339] ppid=4336 vsize=369340 CPUtime=522.2 cores=1,3,5,7
/proc/4338/task/4339/stat : 4339 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 52167 0 0 0 52144 76 0 0 20 0 3 0 331802122 378204160 66822 33554432000 4194304 5964478 140735935254624 140124101412856 4399912 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=4338/tid=4340] ppid=4336 vsize=369340 CPUtime=522.28 cores=1,3,5,7
/proc/4338/task/4340/stat : 4340 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 13013 0 0 0 52205 23 0 0 20 0 3 0 331802122 378204160 66822 33554432000 4194304 5964478 140735935254624 140124090923320 4400024 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1044.3
Current children cumulated vsize (KiB) 369340

[startup+582.3 s]
/proc/loadavg: 4.06 4.01 3.93 5/183 4467
/proc/meminfo: memFree=27993968/32873844 swapFree=7380/7380
[pid=4338] ppid=4336 vsize=371428 CPUtime=1164.29 cores=1,3,5,7
/proc/4338/stat : 4338 (pwbo2.02) S 4336 4338 2131 0 -1 4202496 73682 0 0 0 116327 102 0 0 20 0 3 0 331802122 380342272 74539 33554432000 4194304 5964478 140735935254624 140735935254144 4546685 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/4338/statm: 92857 74539 196 433 0 92421 0
[pid=4338/tid=4339] ppid=4336 vsize=371428 CPUtime=582.2 cores=1,3,5,7
/proc/4338/task/4339/stat : 4339 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 57703 0 0 0 58129 91 0 0 20 0 3 0 331802122 380342272 74539 33554432000 4194304 5964478 140735935254624 140124101412856 4399920 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=4338/tid=4340] ppid=4336 vsize=371428 CPUtime=582.29 cores=1,3,5,7
/proc/4338/task/4340/stat : 4340 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 15194 0 0 0 58201 28 0 0 20 0 3 0 331802122 380342272 74539 33554432000 4194304 5964478 140735935254624 140124090923368 4221120 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1164.29
Current children cumulated vsize (KiB) 371428

[startup+642.3 s]
/proc/loadavg: 4.02 4.01 3.93 5/183 4467
/proc/meminfo: memFree=27950672/32873844 swapFree=7380/7380
[pid=4338] ppid=4336 vsize=372184 CPUtime=1284.27 cores=1,3,5,7
/proc/4338/stat : 4338 (pwbo2.02) S 4336 4338 2131 0 -1 4202496 76661 0 0 0 128318 109 0 0 20 0 3 0 331802122 381116416 77510 33554432000 4194304 5964478 140735935254624 140735935254144 4546685 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/4338/statm: 93046 77510 196 433 0 92610 0
[pid=4338/tid=4339] ppid=4336 vsize=372184 CPUtime=642.21 cores=1,3,5,7
/proc/4338/task/4339/stat : 4339 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 59434 0 0 0 64125 96 0 0 20 0 3 0 331802122 381116416 77510 33554432000 4194304 5964478 140735935254624 140124101412856 4221120 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=4338/tid=4340] ppid=4336 vsize=372184 CPUtime=642.29 cores=1,3,5,7
/proc/4338/task/4340/stat : 4340 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 16442 0 0 0 64196 33 0 0 20 0 3 0 331802122 381116416 77510 33554432000 4194304 5964478 140735935254624 140124090923320 4399920 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1284.27
Current children cumulated vsize (KiB) 372184

[startup+702.3 s]
/proc/loadavg: 4.00 4.00 3.93 5/183 4467
/proc/meminfo: memFree=27853428/32873844 swapFree=7380/7380
[pid=4338] ppid=4336 vsize=373820 CPUtime=1404.23 cores=1,3,5,7
/proc/4338/stat : 4338 (pwbo2.02) S 4336 4338 2131 0 -1 4202496 80066 0 0 0 140308 115 0 0 20 0 3 0 331802122 382791680 79673 33554432000 4194304 5964478 140735935254624 140735935254144 4546685 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/4338/statm: 93455 79673 196 433 0 93019 0
[pid=4338/tid=4339] ppid=4336 vsize=373820 CPUtime=702.2 cores=1,3,5,7
/proc/4338/task/4339/stat : 4339 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 62839 0 0 0 70117 103 0 0 20 0 3 0 331802122 382791680 79673 33554432000 4194304 5964478 140735935254624 140124101412904 4400018 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=4338/tid=4340] ppid=4336 vsize=373820 CPUtime=702.28 cores=1,3,5,7
/proc/4338/task/4340/stat : 4340 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 16442 0 0 0 70194 34 0 0 20 0 3 0 331802122 382791680 79673 33554432000 4194304 5964478 140735935254624 140124090923352 4403677 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1404.23
Current children cumulated vsize (KiB) 373820

[startup+762.307 s]
/proc/loadavg: 4.00 4.00 3.93 5/183 4467
/proc/meminfo: memFree=27766396/32873844 swapFree=7380/7380
[pid=4338] ppid=4336 vsize=375364 CPUtime=1524.23 cores=1,3,5,7
/proc/4338/stat : 4338 (pwbo2.02) S 4336 4338 2131 0 -1 4202496 85148 0 0 0 152300 123 0 0 20 0 3 0 331802122 384372736 84755 33554432000 4194304 5964478 140735935254624 140735935254144 4546685 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/4338/statm: 93841 84755 196 433 0 93405 0
[pid=4338/tid=4339] ppid=4336 vsize=375364 CPUtime=762.21 cores=1,3,5,7
/proc/4338/task/4339/stat : 4339 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 66706 0 0 0 76110 111 0 0 20 0 3 0 331802122 384372736 84755 33554432000 4194304 5964478 140735935254624 140124101412856 4399920 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=4338/tid=4340] ppid=4336 vsize=375364 CPUtime=762.29 cores=1,3,5,7
/proc/4338/task/4340/stat : 4340 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 17657 0 0 0 76193 36 0 0 20 0 3 0 331802122 384372736 84755 33554432000 4194304 5964478 140735935254624 140124090923128 4399920 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1524.23
Current children cumulated vsize (KiB) 375364

[startup+822.3 s]
/proc/loadavg: 4.00 4.00 3.93 5/183 4467
/proc/meminfo: memFree=27742992/32873844 swapFree=7380/7380
[pid=4338] ppid=4336 vsize=441672 CPUtime=1644.2 cores=1,3,5,7
/proc/4338/stat : 4338 (pwbo2.02) S 4336 4338 2131 0 -1 4202496 88098 0 0 0 164293 127 0 0 20 0 3 0 331802122 452272128 87705 33554432000 4194304 5964478 140735935254624 140735935254144 4546685 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/4338/statm: 110418 87705 196 433 0 109982 0
[pid=4338/tid=4339] ppid=4336 vsize=441672 CPUtime=822.21 cores=1,3,5,7
/proc/4338/task/4339/stat : 4339 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 69146 0 0 0 82106 115 0 0 20 0 3 0 331802122 452272128 87705 33554432000 4194304 5964478 140735935254624 140124101412856 4399920 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=4338/tid=4340] ppid=4336 vsize=441672 CPUtime=822.29 cores=1,3,5,7
/proc/4338/task/4340/stat : 4340 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 18167 0 0 0 82192 37 0 0 20 0 3 0 331802122 452272128 87705 33554432000 4194304 5964478 140735935254624 140124090923368 4399920 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1644.2
Current children cumulated vsize (KiB) 441672

[startup+882.3 s]
/proc/loadavg: 4.00 4.00 3.93 5/184 4471
/proc/meminfo: memFree=27719832/32873844 swapFree=7380/7380
[pid=4338] ppid=4336 vsize=507424 CPUtime=1764.18 cores=1,3,5,7
/proc/4338/stat : 4338 (pwbo2.02) S 4336 4338 2131 0 -1 4202496 90978 0 0 0 176283 135 0 0 20 0 3 0 331802122 519602176 90585 33554432000 4194304 5964478 140735935254624 140735935254144 4546685 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/4338/statm: 126856 90585 196 433 0 126420 0
[pid=4338/tid=4339] ppid=4336 vsize=507424 CPUtime=882.21 cores=1,3,5,7
/proc/4338/task/4339/stat : 4339 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 70877 0 0 0 88101 120 0 0 20 0 3 0 331802122 519602176 90585 33554432000 4194304 5964478 140735935254624 140124101412664 4399920 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=4338/tid=4340] ppid=4336 vsize=507424 CPUtime=882.29 cores=1,3,5,7
/proc/4338/task/4340/stat : 4340 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 19316 0 0 0 88186 43 0 0 20 0 3 0 331802122 519602176 90585 33554432000 4194304 5964478 140735935254624 140124090922840 4400528 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1764.18
Current children cumulated vsize (KiB) 507424



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+900.3 s]
/proc/loadavg: 4.00 4.00 3.93 5/183 4471
/proc/meminfo: memFree=27714676/32873844 swapFree=7380/7380
[pid=4338] ppid=4336 vsize=507424 CPUtime=1800.19 cores=1,3,5,7
/proc/4338/stat : 4338 (pwbo2.02) S 4336 4338 2131 0 -1 4202496 90978 0 0 0 179884 135 0 0 20 0 3 0 331802122 519602176 90585 33554432000 4194304 5964478 140735935254624 140735935254144 4546685 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/4338/statm: 126856 90585 196 433 0 126420 0
[pid=4338/tid=4339] ppid=4336 vsize=507424 CPUtime=900.22 cores=1,3,5,7
/proc/4338/task/4339/stat : 4339 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 70877 0 0 0 89902 120 0 0 20 0 3 0 331802122 519602176 90585 33554432000 4194304 5964478 140735935254624 140124101412664 4399920 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=4338/tid=4340] ppid=4336 vsize=507424 CPUtime=900.3 cores=1,3,5,7
/proc/4338/task/4340/stat : 4340 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 19316 0 0 0 89987 43 0 0 20 0 3 0 331802122 519602176 90585 33554432000 4194304 5964478 140735935254624 140124090922840 4399920 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1800.19
Current children cumulated vsize (KiB) 507424

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+900.3 s]
/proc/loadavg: 4.00 4.00 3.93 5/183 4471
/proc/meminfo: memFree=27714676/32873844 swapFree=7380/7380
[pid=4338] ppid=4336 vsize=507424 CPUtime=1800.19 cores=1,3,5,7
/proc/4338/stat : 4338 (pwbo2.02) S 4336 4338 2131 0 -1 4202496 90978 0 0 0 179884 135 0 0 20 0 3 0 331802122 519602176 90585 33554432000 4194304 5964478 140735935254624 140735935254144 4546685 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/4338/statm: 126856 90585 196 433 0 126420 0
[pid=4338/tid=4339] ppid=4336 vsize=507424 CPUtime=900.22 cores=1,3,5,7
/proc/4338/task/4339/stat : 4339 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 70877 0 0 0 89902 120 0 0 20 0 3 0 331802122 519602176 90585 33554432000 4194304 5964478 140735935254624 140124101412664 4399920 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=4338/tid=4340] ppid=4336 vsize=507424 CPUtime=900.3 cores=1,3,5,7
/proc/4338/task/4340/stat : 4340 (pwbo2.02) R 4336 4338 2131 0 -1 4202560 19316 0 0 0 89987 43 0 0 20 0 3 0 331802122 519602176 90585 33554432000 4194304 5964478 140735935254624 140124090922840 4399920 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1800.19
Current children cumulated vsize (KiB) 507424

Child status: 0
Real time (s): 900.322
CPU time (s): 1800.57
CPU user time (s): 1798.92
CPU system time (s): 1.65175
CPU usage (%): 199.992
Max. virtual memory (cumulated for all children) (KiB): 507424

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1798.92
system time used= 1.65175
maximum resident set size= 362344
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 90979
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 237
involuntary context switches= 1845

runsolver used 2.19267 second user time and 5.61315 second system time

The end

Launcher Data 

Begin job on node133 at 2012-06-02 19:06:13
IDJOB=3726063
IDBENCH=2920
IDSOLVER=2309
FILE ID=node133/3726063-1338656773
RUNJOBID= node133-1338651663-3450
PBS_JOBID= 14636447
Free space on /tmp= 70480 MiB

SOLVER NAME= pwbo 2.02
BENCH NAME= PB06/final/normalized-PB06/OPT-SMALLINT/submitted-PB06/manquiho/market-split/normalized-opt-market-split_8_70_1.opb
COMMAND LINE= pwbo2.02 -file-format=opb -verbosity=0 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3726063-1338656773/watcher-3726063-1338656773 -o /tmp/evaluation-result-3726063-1338656773/solver-3726063-1338656773 -C 1800 -W 1900 -M 15500  pwbo2.02 -file-format=opb -verbosity=0 HOME/instance-3726063-1338656773.opb

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 0263d0a7c5e3320bd7cce9534c14bf27
RANDOM SEED=1777567251

node133.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.540
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.540
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.540
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.540
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.540
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.540
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.540
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.540
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        28741216 kB
Buffers:          361352 kB
Cached:          2665616 kB
SwapCached:          676 kB
Active:          1550280 kB
Inactive:        1711940 kB
Active(anon):     226416 kB
Inactive(anon):    10240 kB
Active(file):    1323864 kB
Inactive(file):  1701700 kB
Unevictable:        7380 kB
Mlocked:            7380 kB
SwapTotal:      67108856 kB
SwapFree:       67107040 kB
Dirty:              5468 kB
Writeback:             0 kB
AnonPages:        240620 kB
Mapped:            11112 kB
Shmem:                 0 kB
Slab:             732064 kB
SReclaimable:     174152 kB
SUnreclaim:       557912 kB
KernelStack:        1512 kB
PageTables:         4268 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     190740 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346376 kB
VmallocChunk:   34341987860 kB
HardwareCorrupted:     0 kB
AnonHugePages:    157696 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70456 MiB
End job on node133 at 2012-06-02 19:21:16