Trace number 3712736

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
pwbo 2.01MSAT (TO) 1800.14 900.492

General information on the benchmark

Name/SOFT-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/
tsp/normalized-t3002.11tsp11.1900557574--soft-0-100-0.wbo
MD5SUMc0d431beb578ab67ce1970cb297cf650
Bench CategorySOFT-SMALLINT-LIN (only soft constraints, small integers, linear constraints)
Best result obtained on this benchmarkMOPT
Best cost obtained on this benchmark6
Best CPU time to get the best result obtained on this benchmark59.355
Max-Satisfiable
Max-(Un)Satisfiability was proved
Best value of the cost
Optimality of the best cost was proved
Number of variables231
Total number of constraints2707
Number of soft constraints2707
Number of constraints which are clauses2684
Number of constraints which are cardinality constraints (but not clauses)22
Number of constraints which are nor clauses,nor cardinality constraints1
Minimum length of a constraint2
Maximum length of a constraint110
Top cost 137973
Min constraint cost 1
Max constraint cost 100
Sum of constraints costs 137972
Biggest number in a constraint 25
Number of bits of the biggest number in a constraint 5
Biggest sum of numbers in a constraint 689
Number of bits of the biggest sum of numbers10
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Quality of the solution as a function of time


Download the above graph as a PDF file / as an EPS file

Solver Data 

0.00/0.00	c ==============================[ Solver Version ]===============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  pwbo:                    2.01 beta - May, 2012                            |
0.00/0.00	c |                                                                             |
0.00/0.00	c Parsing wbo file format. File HOME/instance-3712736-1338203095.wbo.
0.00/0.00	c Instance file HOME/instance-3712736-1338203095.wbo
0.00/0.00	c File size is 96792 bytes.
0.00/0.01	c Highest Coefficient sum: 689
0.00/0.01	c ============================[ Problem Statistics ]=============================
0.00/0.01	c |                                                                             |
0.00/0.01	c |  Parsing time:          0.01         s                                      |
0.00/0.01	c |  Number Variables:      2938                                                |
0.00/0.01	c |  Number Clauses:        2684                                                |
0.00/0.01	c |  Number Cardinality:    0                                                   |
0.00/0.01	c |  Number PB Constraints: 23                                                  |
0.00/0.01	c |                                                                             |
0.00/0.01	c |  Problem Type:          Optimization                                        |
0.00/0.01	c |  Objective Function:    PB Constraint                                       |
0.00/0.01	c |                                                                             |
0.00/0.01	c ===============================================================================
1800.14/900.41	o 857
1800.14/900.41	o 806
1800.14/900.41	o 727
1800.14/900.41	o 695
1800.14/900.41	o 683
1800.14/900.41	o 659
1800.14/900.41	o 649
1800.14/900.41	o 632
1800.14/900.41	o 631
1800.14/900.41	o 601
1800.14/900.41	o 513
1800.14/900.41	o 508
1800.14/900.41	o 481
1800.14/900.41	o 438
1800.14/900.41	o 436
1800.14/900.41	o 398
1800.14/900.41	o 393
1800.14/900.41	o 391
1800.14/900.41	o 371
1800.14/900.41	o 366
1800.14/900.41	o 332
1800.14/900.41	o 319
1800.14/900.41	o 316
1800.14/900.41	o 315
1800.14/900.41	o 312
1800.14/900.41	o 310
1800.14/900.41	o 292
1800.14/900.41	o 291
1800.14/900.41	o 285
1800.14/900.41	o 240
1800.14/900.41	o 239
1800.14/900.41	o 226
1800.14/900.41	o 218
1800.14/900.41	o 216
1800.14/900.41	o 208
1800.14/900.41	o 202
1800.14/900.41	o 196
1800.14/900.41	o 193
1800.14/900.41	o 188
1800.14/900.41	o 187
1800.14/900.41	o 153
1800.14/900.41	o 148
1800.14/900.41	o 145
1800.14/900.41	o 142
1800.14/900.41	o 126
1800.14/900.41	o 115
1800.14/900.41	o 108
1800.14/900.41	o 99
1800.14/900.41	o 89
1800.14/900.41	o 85
1800.14/900.41	o 76
1800.14/900.41	o 71
1800.14/900.41	o 67
1800.14/900.41	o 63
1800.14/900.41	o 56
1800.14/900.41	o 53
1800.14/900.41	o 52
1800.14/900.41	o 50
1800.14/900.41	o 42
1800.14/900.41	o 41
1800.14/900.41	o 39
1800.14/900.41	o 38
1800.14/900.41	o 35
1800.14/900.41	o 34
1800.14/900.41	o 32
1800.14/900.41	o 31
1800.14/900.41	o 29
1800.14/900.41	o 22
1800.14/900.41	o 21
1800.14/900.41	o 15
1800.14/900.41	o 14
1800.14/900.41	o 12
1800.14/900.41	o 11
1800.14/900.41	o 9
1800.14/900.41	o 7
1800.14/900.41	o 6
1800.14/900.41	c ===============================================================================
1800.14/900.41	c Wall Clock Time: 900.41 s
1800.14/900.41	c Best Solution: 6
1800.14/900.41	s SATISFIABLE
1800.14/900.41	v -x1 -x2 x3 -x4 -x5 -x6 -x7 -x8 -x9 -x10 -x11 -x12 -x13 -x14 -x15 -x16 -x17 x18 -x19 -x20 -x21 -x22 -x23 -x24 -x25 -x26 -x27 -x28 -x29 x30 -x31 -x32 -x33 -x34 -x35 -x36 -x37 -x38 x39 -x40 -x41 -x42 -x43 -x44 -x45 -x46 -x47 x48 -x49 -x50 -x51 -x52 -x53 -x54 -x55 -x56 -x57 -x58 -x59 -x60 -x61 x62 -x63 -x64 -x65 -x66 -x67 -x68 x69 -x70 -x71 -x72 -x73 -x74 -x75 -x76 -x77 x78 -x79 -x80 -x81 -x82 -x83 -x84 -x85 -x86 -x87 -x88 -x89 -x90 -x91 -x92 -x93 -x94 -x95 -x96 -x97 -x98 x99 -x100 -x101 -x102 -x103 x104 -x105 -x106 -x107 -x108 -x109 -x110 -x111 -x112 -x113 -x114 -x115 -x116 -x117 -x118 -x119 x120 -x121 -x122 x123 -x124 -x125 -x126 -x127 -x128 -x129 -x130 -x131 -x132 -x133 -x134 -x135 -x136 -x137 -x138 -x139 -x140 -x141 -x142 -x143 -x144 -x145 -x146 x147 -x148 -x149 -x150 -x151 -x152 -x153 -x154 -x155 x156 -x157 -x158 -x159 -x160 -x161 -x162 -x163 -x164 -x165 -x166 -x167 -x168 -x169 -x170 x171 -x172 -x173 -x174 -x175 -x176 -x177 x178 -x179 -x180 -x181 -x182 -x183 -x184 -x185 -x186 -x187 -x188 -x189 -x190 -x191 -x192 -x193 -x194 -x195 -x196 -x197 -x198 x199 -x200 -x201 -x202 -x203 -x204 -x205 -x206 -x207 -x208 -x209 -x210 -x211 -x212 -x213 -x214 -x215 x216 -x217 -x218 -x219 -x220 -x221 x222 -x223 -x224 -x225 -x226 -x227 -x228 -x229 -x230 -x231

Verifier Data 

OK	6

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3712736-1338203095/watcher-3712736-1338203095 -o /tmp/evaluation-result-3712736-1338203095/solver-3712736-1338203095 -C 1800 -W 1900 -M 15500 pwbo2.01 -file-format=wbo -verbosity=0 HOME/instance-3712736-1338203095.wbo 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.98 4.05 4.01 4/180 24604
/proc/meminfo: memFree=30343968/32873844 swapFree=7112/7112
[pid=24604] ppid=24602 vsize=4572 CPUtime=0 cores=0,2,4,6
/proc/24604/stat : 24604 (pwbo2.01) R 24602 24604 22874 0 -1 4202496 435 0 0 0 0 0 0 0 20 0 1 0 286434875 4681728 388 33554432000 4194304 5964478 140736509446256 140736509444504 4395513 0 0 4096 16387 0 0 0 17 4 0 0 0 0 0
/proc/24604/statm: 1143 388 168 433 0 683 0

[startup+0.10229 s]
/proc/loadavg: 3.98 4.05 4.01 4/180 24604
/proc/meminfo: memFree=30343968/32873844 swapFree=7112/7112
[pid=24604] ppid=24602 vsize=164272 CPUtime=0.18 cores=0,2,4,6
/proc/24604/stat : 24604 (pwbo2.01) S 24602 24604 22874 0 -1 4202496 3176 0 0 0 17 1 0 0 20 0 3 0 286434875 168214528 2560 33554432000 4194304 5964478 140736509446256 140736509445776 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/24604/statm: 41068 2560 198 433 0 40632 0
Current children cumulated CPU time (s) 0.18
Current children cumulated vsize (KiB) 164272

[startup+0.200203 s]
/proc/loadavg: 3.98 4.05 4.01 4/180 24604
/proc/meminfo: memFree=30343968/32873844 swapFree=7112/7112
[pid=24604] ppid=24602 vsize=164272 CPUtime=0.37 cores=0,2,4,6
/proc/24604/stat : 24604 (pwbo2.01) S 24602 24604 22874 0 -1 4202496 4566 0 0 0 36 1 0 0 20 0 3 0 286434875 168214528 3230 33554432000 4194304 5964478 140736509446256 140736509445776 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/24604/statm: 41068 3230 198 433 0 40632 0
Current children cumulated CPU time (s) 0.37
Current children cumulated vsize (KiB) 164272

[startup+0.300167 s]
/proc/loadavg: 3.98 4.05 4.01 4/180 24604
/proc/meminfo: memFree=30343968/32873844 swapFree=7112/7112
[pid=24604] ppid=24602 vsize=164272 CPUtime=0.57 cores=0,2,4,6
/proc/24604/stat : 24604 (pwbo2.01) S 24602 24604 22874 0 -1 4202496 5269 0 0 0 56 1 0 0 20 0 3 0 286434875 168214528 3529 33554432000 4194304 5964478 140736509446256 140736509445776 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/24604/statm: 41068 3529 198 433 0 40632 0
Current children cumulated CPU time (s) 0.57
Current children cumulated vsize (KiB) 164272

[startup+0.70014 s]
/proc/loadavg: 3.98 4.05 4.01 4/180 24604
/proc/meminfo: memFree=30343968/32873844 swapFree=7112/7112
[pid=24604] ppid=24602 vsize=164272 CPUtime=1.36 cores=0,2,4,6
/proc/24604/stat : 24604 (pwbo2.01) S 24602 24604 22874 0 -1 4202496 7272 0 0 0 135 1 0 0 20 0 3 0 286434875 168214528 4811 33554432000 4194304 5964478 140736509446256 140736509445776 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/24604/statm: 41068 4811 198 433 0 40632 0
Current children cumulated CPU time (s) 1.36
Current children cumulated vsize (KiB) 164272

[startup+1.5003 s]
/proc/loadavg: 3.98 4.05 4.01 5/183 24607
/proc/meminfo: memFree=30317904/32873844 swapFree=7112/7112
[pid=24604] ppid=24602 vsize=164272 CPUtime=2.96 cores=0,2,4,6
/proc/24604/stat : 24604 (pwbo2.01) S 24602 24604 22874 0 -1 4202496 10223 0 0 0 293 3 0 0 20 0 3 0 286434875 168214528 6536 33554432000 4194304 5964478 140736509446256 140736509445776 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/24604/statm: 41068 6536 198 433 0 40632 0
[pid=24604/tid=24606] ppid=24602 vsize=164272 CPUtime=1.46 cores=0,2,4,6
/proc/24604/task/24606/stat : 24606 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 3422 0 0 0 145 1 0 0 20 0 3 0 286434876 168214528 6536 33554432000 4194304 5964478 140736509446256 140455064877048 4400051 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=24604/tid=24607] ppid=24602 vsize=164272 CPUtime=1.46 cores=0,2,4,6
/proc/24604/task/24607/stat : 24607 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 5711 0 0 0 145 1 0 0 20 0 3 0 286434876 168214528 6536 33554432000 4194304 5964478 140736509446256 140455054387864 4403701 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 2.96
Current children cumulated vsize (KiB) 164272

[startup+3.10031 s]
/proc/loadavg: 3.98 4.05 4.01 5/183 24607
/proc/meminfo: memFree=30308580/32873844 swapFree=7112/7112
[pid=24604] ppid=24602 vsize=164272 CPUtime=6.16 cores=0,2,4,6
/proc/24604/stat : 24604 (pwbo2.01) S 24602 24604 22874 0 -1 4202496 13402 0 0 0 610 6 0 0 20 0 3 0 286434875 168214528 9303 33554432000 4194304 5964478 140736509446256 140736509445776 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/24604/statm: 41068 9303 198 433 0 40632 0
[pid=24604/tid=24606] ppid=24602 vsize=164272 CPUtime=3.06 cores=0,2,4,6
/proc/24604/task/24606/stat : 24606 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 5840 0 0 0 302 4 0 0 20 0 3 0 286434876 168214528 9303 33554432000 4194304 5964478 140736509446256 140455064877048 4221120 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=24604/tid=24607] ppid=24602 vsize=164272 CPUtime=3.06 cores=0,2,4,6
/proc/24604/task/24607/stat : 24607 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 6472 0 0 0 305 1 0 0 20 0 3 0 286434876 168214528 9303 33554432000 4194304 5964478 140736509446256 140455054387864 4221984 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 6.16
Current children cumulated vsize (KiB) 164272

[startup+6.30031 s]
/proc/loadavg: 4.06 4.07 4.01 5/183 24607
/proc/meminfo: memFree=30277828/32873844 swapFree=7112/7112
[pid=24604] ppid=24602 vsize=164272 CPUtime=12.56 cores=0,2,4,6
/proc/24604/stat : 24604 (pwbo2.01) S 24602 24604 22874 0 -1 4202496 20223 0 0 0 1241 15 0 0 20 0 3 0 286434875 168214528 15554 33554432000 4194304 5964478 140736509446256 140736509445776 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/24604/statm: 41068 15554 198 433 0 40632 0
[pid=24604/tid=24606] ppid=24602 vsize=164272 CPUtime=6.25 cores=0,2,4,6
/proc/24604/task/24606/stat : 24606 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 11769 0 0 0 612 13 0 0 20 0 3 0 286434876 168214528 15554 33554432000 4194304 5964478 140736509446256 140455064877048 5136404 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=24604/tid=24607] ppid=24602 vsize=164272 CPUtime=6.25 cores=0,2,4,6
/proc/24604/task/24607/stat : 24607 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 7364 0 0 0 624 1 0 0 20 0 3 0 286434876 168214528 15554 33554432000 4194304 5964478 140736509446256 140455054387720 4400071 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 12.56
Current children cumulated vsize (KiB) 164272

[startup+12.7003 s]
/proc/loadavg: 4.06 4.07 4.01 5/183 24607
/proc/meminfo: memFree=30227228/32873844 swapFree=7112/7112
[pid=24604] ppid=24602 vsize=229808 CPUtime=25.36 cores=0,2,4,6
/proc/24604/stat : 24604 (pwbo2.01) S 24602 24604 22874 0 -1 4202496 28566 0 0 0 2512 24 0 0 20 0 3 0 286434875 235323392 23895 33554432000 4194304 5964478 140736509446256 140736509445776 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/24604/statm: 57452 23895 198 433 0 57016 0
[pid=24604/tid=24606] ppid=24602 vsize=229808 CPUtime=12.64 cores=0,2,4,6
/proc/24604/task/24606/stat : 24606 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 19534 0 0 0 1242 22 0 0 20 0 3 0 286434876 235323392 23895 33554432000 4194304 5964478 140736509446256 140455064877048 4391484 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=24604/tid=24607] ppid=24602 vsize=229808 CPUtime=12.63 cores=0,2,4,6
/proc/24604/task/24607/stat : 24607 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 7942 0 0 0 1262 1 0 0 20 0 3 0 286434876 235323392 23895 33554432000 4194304 5964478 140736509446256 140455054387720 4221984 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 25.36
Current children cumulated vsize (KiB) 229808

[startup+25.5004 s]
/proc/loadavg: 4.04 4.06 4.01 5/183 24607
/proc/meminfo: memFree=30128012/32873844 swapFree=7112/7112
[pid=24604] ppid=24602 vsize=295476 CPUtime=50.95 cores=0,2,4,6
/proc/24604/stat : 24604 (pwbo2.01) S 24602 24604 22874 0 -1 4202496 48483 0 0 0 5065 30 0 0 20 0 3 0 286434875 302567424 43608 33554432000 4194304 5964478 140736509446256 140736509445776 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/24604/statm: 73869 43608 198 433 0 73433 0
[pid=24604/tid=24606] ppid=24602 vsize=295476 CPUtime=25.4 cores=0,2,4,6
/proc/24604/task/24606/stat : 24606 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 38481 0 0 0 2513 27 0 0 20 0 3 0 286434876 302567424 43608 33554432000 4194304 5964478 140736509446256 140455064877048 4400040 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=24604/tid=24607] ppid=24602 vsize=295476 CPUtime=25.4 cores=0,2,4,6
/proc/24604/task/24607/stat : 24607 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 8912 0 0 0 2538 2 0 0 20 0 3 0 286434876 302567424 43608 33554432000 4194304 5964478 140736509446256 140455054387512 4403648 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 50.95
Current children cumulated vsize (KiB) 295476

[startup+51.1004 s]
/proc/loadavg: 4.03 4.06 4.01 5/183 24607
/proc/meminfo: memFree=29978188/32873844 swapFree=7112/7112
[pid=24604] ppid=24602 vsize=362580 CPUtime=102.14 cores=0,2,4,6
/proc/24604/stat : 24604 (pwbo2.01) S 24602 24604 22874 0 -1 4202496 78254 0 0 0 10160 54 0 0 20 0 3 0 286434875 371281920 73070 33554432000 4194304 5964478 140736509446256 140736509445776 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/24604/statm: 90645 73070 198 433 0 90209 0
[pid=24604/tid=24606] ppid=24602 vsize=362580 CPUtime=50.94 cores=0,2,4,6
/proc/24604/task/24606/stat : 24606 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 56526 0 0 0 5051 43 0 0 20 0 3 0 286434876 371281920 73070 33554432000 4194304 5964478 140736509446256 140455064877048 4387891 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=24604/tid=24607] ppid=24602 vsize=362580 CPUtime=50.93 cores=0,2,4,6
/proc/24604/task/24607/stat : 24607 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 20638 0 0 0 5083 10 0 0 20 0 3 0 286434876 371281920 73070 33554432000 4194304 5964478 140736509446256 140455054387560 4403701 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 102.14
Current children cumulated vsize (KiB) 362580

[startup+102.3 s]
/proc/loadavg: 4.06 4.06 4.01 5/183 24607
/proc/meminfo: memFree=29730720/32873844 swapFree=7112/7112
[pid=24604] ppid=24602 vsize=560732 CPUtime=204.54 cores=0,2,4,6
/proc/24604/stat : 24604 (pwbo2.01) S 24602 24604 22874 0 -1 4202496 119423 0 0 0 20380 74 0 0 20 0 3 0 286434875 574189568 114059 33554432000 4194304 5964478 140736509446256 140736509445776 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/24604/statm: 140183 114059 198 433 0 139747 0
[pid=24604/tid=24606] ppid=24602 vsize=560732 CPUtime=102.01 cores=0,2,4,6
/proc/24604/task/24606/stat : 24606 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 73287 0 0 0 10146 55 0 0 20 0 3 0 286434876 574189568 114059 33554432000 4194304 5964478 140736509446256 140455064877048 5136404 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=24604/tid=24607] ppid=24602 vsize=560732 CPUtime=102 cores=0,2,4,6
/proc/24604/task/24607/stat : 24607 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 45046 0 0 0 10180 20 0 0 20 0 3 0 286434876 574189568 114059 33554432000 4194304 5964478 140736509446256 140455054387864 4403676 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 204.54
Current children cumulated vsize (KiB) 560732

[startup+162.3 s]
/proc/loadavg: 3.95 4.03 4.00 5/183 24648
/proc/meminfo: memFree=30423696/32873844 swapFree=7112/7112
[pid=24604] ppid=24602 vsize=823660 CPUtime=324.48 cores=0,2,4,6
/proc/24604/stat : 24604 (pwbo2.01) S 24602 24604 22874 0 -1 4202496 173534 0 0 0 32354 94 0 0 20 0 3 0 286434875 843427840 168534 33554432000 4194304 5964478 140736509446256 140736509445776 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/24604/statm: 205915 168534 198 433 0 205479 0
[pid=24604/tid=24606] ppid=24602 vsize=823660 CPUtime=161.83 cores=0,2,4,6
/proc/24604/task/24606/stat : 24606 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 115567 0 0 0 16110 73 0 0 20 0 3 0 286434876 843427840 168534 33554432000 4194304 5964478 140736509446256 140455064877048 4399904 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=24604/tid=24607] ppid=24602 vsize=823660 CPUtime=161.84 cores=0,2,4,6
/proc/24604/task/24607/stat : 24607 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 56877 0 0 0 16160 24 0 0 20 0 3 0 286434876 843427840 168534 33554432000 4194304 5964478 140736509446256 140455054387272 4400053 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 324.48
Current children cumulated vsize (KiB) 823660

[startup+222.307 s]
/proc/loadavg: 3.98 4.02 4.00 5/183 24649
/proc/meminfo: memFree=30070944/32873844 swapFree=7112/7112
[pid=24604] ppid=24602 vsize=890796 CPUtime=444.43 cores=0,2,4,6
/proc/24604/stat : 24604 (pwbo2.01) S 24602 24604 22874 0 -1 4202496 189839 0 0 0 44342 101 0 0 20 0 3 0 286434875 912175104 183716 33554432000 4194304 5964478 140736509446256 140736509445776 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/24604/statm: 222699 183716 198 433 0 222263 0
[pid=24604/tid=24606] ppid=24602 vsize=890796 CPUtime=221.67 cores=0,2,4,6
/proc/24604/task/24606/stat : 24606 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 115599 0 0 0 22093 74 0 0 20 0 3 0 286434876 912175104 183716 33554432000 4194304 5964478 140736509446256 140455064876960 4400051 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=24604/tid=24607] ppid=24602 vsize=890796 CPUtime=221.66 cores=0,2,4,6
/proc/24604/task/24607/stat : 24607 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 73150 0 0 0 22134 32 0 0 20 0 3 0 286434876 912175104 183716 33554432000 4194304 5964478 140736509446256 140455054387544 4399904 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 444.43
Current children cumulated vsize (KiB) 890796

[startup+282.3 s]
/proc/loadavg: 3.99 4.02 4.00 5/183 24649
/proc/meminfo: memFree=29728608/32873844 swapFree=7112/7112
[pid=24604] ppid=24602 vsize=1022984 CPUtime=564.34 cores=0,2,4,6
/proc/24604/stat : 24604 (pwbo2.01) S 24602 24604 22874 0 -1 4202496 224320 0 0 0 56321 113 0 0 20 0 3 0 286434875 1047535616 218194 33554432000 4194304 5964478 140736509446256 140736509445776 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/24604/statm: 255746 218194 198 433 0 255310 0
[pid=24604/tid=24606] ppid=24602 vsize=1022984 CPUtime=281.47 cores=0,2,4,6
/proc/24604/task/24606/stat : 24606 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 133538 0 0 0 28066 81 0 0 20 0 3 0 286434876 1047535616 218194 33554432000 4194304 5964478 140736509446256 140455064877048 4400099 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=24604/tid=24607] ppid=24602 vsize=1022984 CPUtime=281.48 cores=0,2,4,6
/proc/24604/task/24607/stat : 24607 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 89692 0 0 0 28108 40 0 0 20 0 3 0 286434876 1047535616 218194 33554432000 4194304 5964478 140736509446256 140455054387512 4400460 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 564.34
Current children cumulated vsize (KiB) 1022984

[startup+342.3 s]
/proc/loadavg: 4.04 4.03 4.00 5/184 24652
/proc/meminfo: memFree=29491788/32873844 swapFree=7112/7112
[pid=24604] ppid=24602 vsize=1154956 CPUtime=684.28 cores=0,2,4,6
/proc/24604/stat : 24604 (pwbo2.01) S 24602 24604 22874 0 -1 4202496 262518 0 0 0 68298 130 0 0 20 0 3 0 286434875 1182674944 256291 33554432000 4194304 5964478 140736509446256 140736509445776 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/24604/statm: 288739 256291 198 433 0 288303 0

################
# More data... #
################

[pid=24604/tid=24607] ppid=24602 vsize=1154956 CPUtime=341.31 cores=0,2,4,6
/proc/24604/task/24607/stat : 24607 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 90245 0 0 0 34090 41 0 0 20 0 3 0 286434876 1182674944 256291 33554432000 4194304 5964478 140736509446256 140455054387544 4390961 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 684.28
Current children cumulated vsize (KiB) 1154956

[startup+402.3 s]
/proc/loadavg: 4.01 4.02 4.00 5/183 24652
/proc/meminfo: memFree=29074688/32873844 swapFree=7112/7112
[pid=24604] ppid=24602 vsize=1286568 CPUtime=804.21 cores=0,2,4,6
/proc/24604/stat : 24604 (pwbo2.01) S 24602 24604 22874 0 -1 4202496 304313 0 0 0 80272 149 0 0 20 0 3 0 286434875 1317445632 298058 33554432000 4194304 5964478 140736509446256 140736509445776 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/24604/statm: 321642 298058 198 433 0 321206 0
[pid=24604/tid=24606] ppid=24602 vsize=1286568 CPUtime=401.1 cores=0,2,4,6
/proc/24604/task/24606/stat : 24606 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 212963 0 0 0 39992 118 0 0 20 0 3 0 286434876 1317445632 298058 33554432000 4194304 5964478 140736509446256 140455064877048 4220672 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=24604/tid=24607] ppid=24602 vsize=1286568 CPUtime=401.15 cores=0,2,4,6
/proc/24604/task/24607/stat : 24607 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 90260 0 0 0 40073 42 0 0 20 0 3 0 286434876 1317445632 298058 33554432000 4194304 5964478 140736509446256 140455054387616 4403661 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 804.21
Current children cumulated vsize (KiB) 1286568

[startup+462.3 s]
/proc/loadavg: 4.00 4.02 4.00 5/183 24652
/proc/meminfo: memFree=28792864/32873844 swapFree=7112/7112
[pid=24604] ppid=24602 vsize=1417972 CPUtime=924.14 cores=0,2,4,6
/proc/24604/stat : 24604 (pwbo2.01) S 24602 24604 22874 0 -1 4202496 332078 0 0 0 92251 163 0 0 20 0 3 0 286434875 1452003328 325793 33554432000 4194304 5964478 140736509446256 140736509445776 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/24604/statm: 354493 325793 198 433 0 354057 0
[pid=24604/tid=24606] ppid=24602 vsize=1417972 CPUtime=460.9 cores=0,2,4,6
/proc/24604/task/24606/stat : 24606 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 240633 0 0 0 45958 132 0 0 20 0 3 0 286434876 1452003328 325793 33554432000 4194304 5964478 140736509446256 140455064876856 4400099 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=24604/tid=24607] ppid=24602 vsize=1417972 CPUtime=460.99 cores=0,2,4,6
/proc/24604/task/24607/stat : 24607 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 90355 0 0 0 46056 43 0 0 20 0 3 0 286434876 1452003328 325793 33554432000 4194304 5964478 140736509446256 140455054387272 4222416 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 924.14
Current children cumulated vsize (KiB) 1417972

[startup+522.3 s]
/proc/loadavg: 4.04 4.03 4.00 5/183 24653
/proc/meminfo: memFree=28659220/32873844 swapFree=7112/7112
[pid=24604] ppid=24602 vsize=1418304 CPUtime=1044.07 cores=0,2,4,6
/proc/24604/stat : 24604 (pwbo2.01) S 24602 24604 22874 0 -1 4202496 335816 0 0 0 104243 164 0 0 20 0 3 0 286434875 1452343296 329485 33554432000 4194304 5964478 140736509446256 140736509445776 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/24604/statm: 354576 329485 198 433 0 354140 0
[pid=24604/tid=24606] ppid=24602 vsize=1418304 CPUtime=520.7 cores=0,2,4,6
/proc/24604/task/24606/stat : 24606 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 240681 0 0 0 51937 133 0 0 20 0 3 0 286434876 1452343296 329485 33554432000 4194304 5964478 140736509446256 140455064876856 4400075 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=24604/tid=24607] ppid=24602 vsize=1418304 CPUtime=520.83 cores=0,2,4,6
/proc/24604/task/24607/stat : 24607 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 94045 0 0 0 52038 45 0 0 20 0 3 0 286434876 1452343296 329485 33554432000 4194304 5964478 140736509446256 140455054387272 4400071 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1044.07
Current children cumulated vsize (KiB) 1418304

[startup+582.3 s]
/proc/loadavg: 4.01 4.02 4.00 5/183 24653
/proc/meminfo: memFree=28506992/32873844 swapFree=7112/7112
[pid=24604] ppid=24602 vsize=1418492 CPUtime=1164.03 cores=0,2,4,6
/proc/24604/stat : 24604 (pwbo2.01) S 24602 24604 22874 0 -1 4202496 340121 0 0 0 116236 167 0 0 20 0 3 0 286434875 1452535808 335217 33554432000 4194304 5964478 140736509446256 140736509445776 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/24604/statm: 354623 335217 198 433 0 354187 0
[pid=24604/tid=24606] ppid=24602 vsize=1418492 CPUtime=580.54 cores=0,2,4,6
/proc/24604/task/24606/stat : 24606 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 240692 0 0 0 57920 134 0 0 20 0 3 0 286434876 1452535808 335217 33554432000 4194304 5964478 140736509446256 140455064876856 4400051 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=24604/tid=24607] ppid=24602 vsize=1418492 CPUtime=580.65 cores=0,2,4,6
/proc/24604/task/24607/stat : 24607 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 98339 0 0 0 58017 48 0 0 20 0 3 0 286434876 1452535808 335217 33554432000 4194304 5964478 140736509446256 140455054387864 4222416 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1164.03
Current children cumulated vsize (KiB) 1418492

[startup+642.3 s]
/proc/loadavg: 4.00 4.01 4.00 5/183 24653
/proc/meminfo: memFree=28463248/32873844 swapFree=7112/7112
[pid=24604] ppid=24602 vsize=1418808 CPUtime=1284.01 cores=0,2,4,6
/proc/24604/stat : 24604 (pwbo2.01) S 24602 24604 22874 0 -1 4202496 340185 0 0 0 128234 167 0 0 20 0 3 0 286434875 1452859392 335281 33554432000 4194304 5964478 140736509446256 140736509445776 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/24604/statm: 354702 335281 198 433 0 354266 0
[pid=24604/tid=24606] ppid=24602 vsize=1418808 CPUtime=640.39 cores=0,2,4,6
/proc/24604/task/24606/stat : 24606 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 240705 0 0 0 63904 135 0 0 20 0 3 0 286434876 1452859392 335281 33554432000 4194304 5964478 140736509446256 140455064876960 4400081 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=24604/tid=24607] ppid=24602 vsize=1418808 CPUtime=640.49 cores=0,2,4,6
/proc/24604/task/24607/stat : 24607 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 98390 0 0 0 64000 49 0 0 20 0 3 0 286434876 1452859392 335281 33554432000 4194304 5964478 140736509446256 140455054387272 4220672 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1284.01
Current children cumulated vsize (KiB) 1418808

[startup+702.3 s]
/proc/loadavg: 4.00 4.01 4.00 5/183 24653
/proc/meminfo: memFree=28454660/32873844 swapFree=7112/7112
[pid=24604] ppid=24602 vsize=1419348 CPUtime=1403.99 cores=0,2,4,6
/proc/24604/stat : 24604 (pwbo2.01) S 24602 24604 22874 0 -1 4202496 340323 0 0 0 140232 167 0 0 20 0 3 0 286434875 1453412352 335419 33554432000 4194304 5964478 140736509446256 140736509445776 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/24604/statm: 354837 335419 198 433 0 354401 0
[pid=24604/tid=24606] ppid=24602 vsize=1419348 CPUtime=700.24 cores=0,2,4,6
/proc/24604/task/24606/stat : 24606 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 240728 0 0 0 69888 136 0 0 20 0 3 0 286434876 1453412352 335419 33554432000 4194304 5964478 140736509446256 140455064876960 4220672 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=24604/tid=24607] ppid=24602 vsize=1419348 CPUtime=700.34 cores=0,2,4,6
/proc/24604/task/24607/stat : 24607 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 98505 0 0 0 69984 50 0 0 20 0 3 0 286434876 1453412352 335419 33554432000 4194304 5964478 140736509446256 140455054387272 4399994 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1403.99
Current children cumulated vsize (KiB) 1419348

[startup+762.3 s]
/proc/loadavg: 4.00 4.00 4.00 5/183 24653
/proc/meminfo: memFree=28242772/32873844 swapFree=7112/7112
[pid=24604] ppid=24602 vsize=1550752 CPUtime=1523.98 cores=0,2,4,6
/proc/24604/stat : 24604 (pwbo2.01) S 24602 24604 22874 0 -1 4202496 380103 0 0 0 152215 183 0 0 20 0 3 0 286434875 1587970048 375199 33554432000 4194304 5964478 140736509446256 140736509445776 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/24604/statm: 387688 375199 198 433 0 387252 0
[pid=24604/tid=24606] ppid=24602 vsize=1550752 CPUtime=760.09 cores=0,2,4,6
/proc/24604/task/24606/stat : 24606 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 260143 0 0 0 75863 146 0 0 20 0 3 0 286434876 1587970048 375199 33554432000 4194304 5964478 140736509446256 140455064877048 4221120 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=24604/tid=24607] ppid=24602 vsize=1550752 CPUtime=760.19 cores=0,2,4,6
/proc/24604/task/24607/stat : 24607 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 118870 0 0 0 75961 58 0 0 20 0 3 0 286434876 1587970048 375199 33554432000 4194304 5964478 140736509446256 140455054387512 4400081 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1523.98
Current children cumulated vsize (KiB) 1550752

[startup+822.3 s]
/proc/loadavg: 4.05 4.01 4.00 5/183 24654
/proc/meminfo: memFree=27974092/32873844 swapFree=7112/7112
[pid=24604] ppid=24602 vsize=1682548 CPUtime=1643.96 cores=0,2,4,6
/proc/24604/stat : 24604 (pwbo2.01) S 24602 24604 22874 0 -1 4202496 409664 0 0 0 164198 198 0 0 20 0 3 0 286434875 1722929152 404672 33554432000 4194304 5964478 140736509446256 140736509445776 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/24604/statm: 420637 404672 198 433 0 420201 0
[pid=24604/tid=24606] ppid=24602 vsize=1682548 CPUtime=819.93 cores=0,2,4,6
/proc/24604/task/24606/stat : 24606 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 289677 0 0 0 81831 162 0 0 20 0 3 0 286434876 1722929152 404672 33554432000 4194304 5964478 140736509446256 140455064877048 4400051 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=24604/tid=24607] ppid=24602 vsize=1682548 CPUtime=820.03 cores=0,2,4,6
/proc/24604/task/24607/stat : 24607 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 118897 0 0 0 81944 59 0 0 20 0 3 0 286434876 1722929152 404672 33554432000 4194304 5964478 140736509446256 140455054387544 4399904 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1643.96
Current children cumulated vsize (KiB) 1682548

[startup+882.307 s]
/proc/loadavg: 4.02 4.01 4.00 5/183 24654
/proc/meminfo: memFree=27697228/32873844 swapFree=7112/7112
[pid=24604] ppid=24602 vsize=1879772 CPUtime=1763.95 cores=0,2,4,6
/proc/24604/stat : 24604 (pwbo2.01) S 24602 24604 22874 0 -1 4202496 454729 0 0 0 176178 217 0 0 20 0 3 0 286434875 1924886528 449737 33554432000 4194304 5964478 140736509446256 140736509445776 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/24604/statm: 469943 449737 198 433 0 469507 0
[pid=24604/tid=24606] ppid=24602 vsize=1879772 CPUtime=879.78 cores=0,2,4,6
/proc/24604/task/24606/stat : 24606 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 322239 0 0 0 87801 177 0 0 20 0 3 0 286434876 1924886528 449737 33554432000 4194304 5964478 140736509446256 140455064877048 4400040 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=24604/tid=24607] ppid=24602 vsize=1879772 CPUtime=879.88 cores=0,2,4,6
/proc/24604/task/24607/stat : 24607 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 131400 0 0 0 87924 64 0 0 20 0 3 0 286434876 1924886528 449737 33554432000 4194304 5964478 140736509446256 140455054387616 4403701 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1763.95
Current children cumulated vsize (KiB) 1879772



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+900.4 s]
/proc/loadavg: 4.01 4.01 4.00 5/183 24654
/proc/meminfo: memFree=27632220/32873844 swapFree=7112/7112
[pid=24604] ppid=24602 vsize=1945544 CPUtime=1800.14 cores=0,2,4,6
/proc/24604/stat : 24604 (pwbo2.01) S 24602 24604 22874 0 -1 4202496 464066 0 0 0 179793 221 0 0 20 0 3 0 286434875 1992237056 459074 33554432000 4194304 5964478 140736509446256 140736509445776 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/24604/statm: 486386 459074 198 433 0 485950 0
[pid=24604/tid=24606] ppid=24602 vsize=1945544 CPUtime=897.83 cores=0,2,4,6
/proc/24604/task/24606/stat : 24606 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 331505 0 0 0 89601 182 0 0 20 0 3 0 286434876 1992237056 459074 33554432000 4194304 5964478 140736509446256 140455064877048 4400051 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=24604/tid=24607] ppid=24602 vsize=1945544 CPUtime=897.94 cores=0,2,4,6
/proc/24604/task/24607/stat : 24607 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 131471 0 0 0 89730 64 0 0 20 0 3 0 286434876 1992237056 459074 33554432000 4194304 5964478 140736509446256 140455054387272 4400500 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1800.14
Current children cumulated vsize (KiB) 1945544

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+900.4 s]
/proc/loadavg: 4.01 4.01 4.00 5/183 24654
/proc/meminfo: memFree=27632220/32873844 swapFree=7112/7112
[pid=24604] ppid=24602 vsize=1945544 CPUtime=1800.14 cores=0,2,4,6
/proc/24604/stat : 24604 (pwbo2.01) S 24602 24604 22874 0 -1 4202496 464066 0 0 0 179793 221 0 0 20 0 3 0 286434875 1992237056 459074 33554432000 4194304 5964478 140736509446256 140736509445776 4546669 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/24604/statm: 486386 459074 198 433 0 485950 0
[pid=24604/tid=24606] ppid=24602 vsize=1945544 CPUtime=897.83 cores=0,2,4,6
/proc/24604/task/24606/stat : 24606 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 331505 0 0 0 89601 182 0 0 20 0 3 0 286434876 1992237056 459074 33554432000 4194304 5964478 140736509446256 140455064877048 4400051 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=24604/tid=24607] ppid=24602 vsize=1945544 CPUtime=897.94 cores=0,2,4,6
/proc/24604/task/24607/stat : 24607 (pwbo2.01) R 24602 24604 22874 0 -1 4202560 131471 0 0 0 89730 64 0 0 20 0 3 0 286434876 1992237056 459074 33554432000 4194304 5964478 140736509446256 140455054387272 4400500 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1800.14
Current children cumulated vsize (KiB) 1945544

Child status: 0

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 24604 and gives
#  childrusage.ru_utime.tv_sec=1793
#  childrusage.ru_utime.tv_usec=334371
#  childrusage.ru_stime.tv_sec=2
#  childrusage.ru_stime.tv_usec=563610
# CPU time returned by wait4() is 1795.9
# while last known CPU time is 1800.14
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 900.492
CPU time (s): 1800.14
CPU user time (s): 1797.93
CPU system time (s): 2.21
CPU usage (%): 199.906
Max. virtual memory (cumulated for all children) (KiB): 1945544

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1793.33
system time used= 2.56361
maximum resident set size= 1836304
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 464068
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 738
involuntary context switches= 1967

runsolver used 1.58176 second user time and 5.35319 second system time

The end

Launcher Data 

Begin job on node132 at 2012-05-28 13:04:55
IDJOB=3712736
IDBENCH=79258
IDSOLVER=2292
FILE ID=node132/3712736-1338203095
RUNJOBID= node132-1338194011-22890
PBS_JOBID= 14624592
Free space on /tmp= 71632 MiB

SOLVER NAME= pwbo 2.01
BENCH NAME= PB10/normalized-PB10/SOFT-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/tsp/normalized-t3002.11tsp11.1900557574--soft-0-100-0.wbo
COMMAND LINE= pwbo2.01 -file-format=wbo -verbosity=0 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3712736-1338203095/watcher-3712736-1338203095 -o /tmp/evaluation-result-3712736-1338203095/solver-3712736-1338203095 -C 1800 -W 1900 -M 15500  pwbo2.01 -file-format=wbo -verbosity=0 HOME/instance-3712736-1338203095.wbo

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= c0d431beb578ab67ce1970cb297cf650
RANDOM SEED=26566709

node132.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.832
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.832
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.832
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.832
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.832
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.832
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.832
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.832
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        30344588 kB
Buffers:          463632 kB
Cached:           204684 kB
SwapCached:         2284 kB
Active:          1469032 kB
Inactive:         168248 kB
Active(anon):     961092 kB
Inactive(anon):     8148 kB
Active(file):     507940 kB
Inactive(file):   160100 kB
Unevictable:        7112 kB
Mlocked:            7112 kB
SwapTotal:      67108856 kB
SwapFree:       67103740 kB
Dirty:              5444 kB
Writeback:             8 kB
AnonPages:        972212 kB
Mapped:            11784 kB
Shmem:                 8 kB
Slab:             750696 kB
SReclaimable:     192192 kB
SUnreclaim:       558504 kB
KernelStack:        1472 kB
PageTables:         5808 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     162260 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346376 kB
VmallocChunk:   34341987860 kB
HardwareCorrupted:     0 kB
AnonHugePages:    751616 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 71628 MiB
End job on node132 at 2012-05-28 13:19:58