Trace number 3728835

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
pwbo 2.02MSAT (TO) 1800.03 900.394

General information on the benchmark

Name/SOFT-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/
tsp/normalized-t3002.11tsp11.1900584106--soft-0-100-0.wbo
MD5SUMd3ccf217ec482c2d8ac120ae50401a4a
Bench CategorySOFT-SMALLINT-LIN (only soft constraints, small integers, linear constraints)
Best result obtained on this benchmarkMOPT
Best cost obtained on this benchmark7
Best CPU time to get the best result obtained on this benchmark72.143
Max-Satisfiable
Max-(Un)Satisfiability was proved
Best value of the cost
Optimality of the best cost was proved
Number of variables231
Total number of constraints2707
Number of soft constraints2707
Number of constraints which are clauses2684
Number of constraints which are cardinality constraints (but not clauses)22
Number of constraints which are nor clauses,nor cardinality constraints1
Minimum length of a constraint2
Maximum length of a constraint110
Top cost 137946
Min constraint cost 1
Max constraint cost 100
Sum of constraints costs 137945
Biggest number in a constraint 25
Number of bits of the biggest number in a constraint 5
Biggest sum of numbers in a constraint 737
Number of bits of the biggest sum of numbers10
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Quality of the solution as a function of time


Download the above graph as a PDF file / as an EPS file

Solver Data 

0.00/0.00	c ==============================[ Solver Version ]===============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  pwbo:                     2.02 beta - May, 2012                            |
0.00/0.00	c |                                                                             |
0.00/0.00	c Parsing wbo file format. File HOME/instance-3728835-1338674752.wbo.
0.00/0.00	c Instance file HOME/instance-3728835-1338674752.wbo
0.00/0.00	c File size is 96771 bytes.
0.00/0.01	c Highest Coefficient sum: 737
0.00/0.01	c ============================[ Problem Statistics ]=============================
0.00/0.01	c |                                                                             |
0.00/0.01	c |  Parsing time:          0.01         s                                      |
0.00/0.01	c |  Number Variables:      2938                                                |
0.00/0.01	c |  Number Clauses:        2684                                                |
0.00/0.01	c |  Number Cardinality:    0                                                   |
0.00/0.01	c |  Number PB Constraints: 23                                                  |
0.00/0.01	c |                                                                             |
0.00/0.01	c |  Problem Type:          Optimization                                        |
0.00/0.01	c |  Objective Function:    PB Constraint                                       |
0.00/0.01	c |                                                                             |
0.00/0.01	c ===============================================================================
1800.03/900.31	o 883
1800.03/900.31	o 709
1800.03/900.31	o 589
1800.03/900.31	o 578
1800.03/900.31	o 554
1800.03/900.31	o 550
1800.03/900.31	o 543
1800.03/900.31	o 540
1800.03/900.31	o 302
1800.03/900.31	o 293
1800.03/900.31	o 265
1800.03/900.31	o 256
1800.03/900.31	o 230
1800.03/900.31	o 221
1800.03/900.31	o 193
1800.03/900.31	o 184
1800.03/900.31	o 166
1800.03/900.31	o 160
1800.03/900.31	o 142
1800.03/900.31	o 136
1800.03/900.31	o 135
1800.03/900.31	o 130
1800.03/900.31	o 129
1800.03/900.31	o 124
1800.03/900.31	o 112
1800.03/900.31	o 103
1800.03/900.31	o 101
1800.03/900.31	o 100
1800.03/900.31	o 91
1800.03/900.31	o 90
1800.03/900.31	o 75
1800.03/900.31	o 73
1800.03/900.31	o 66
1800.03/900.31	o 63
1800.03/900.31	o 61
1800.03/900.31	o 57
1800.03/900.31	o 52
1800.03/900.31	o 48
1800.03/900.31	o 33
1800.03/900.31	o 31
1800.03/900.31	o 29
1800.03/900.31	o 23
1800.03/900.31	o 22
1800.03/900.31	o 21
1800.03/900.31	o 20
1800.03/900.31	o 18
1800.03/900.31	o 17
1800.03/900.31	o 16
1800.03/900.31	o 13
1800.03/900.31	o 11
1800.03/900.31	o 10
1800.03/900.31	o 9
1800.03/900.31	o 7
1800.03/900.31	c ===============================================================================
1800.03/900.31	c Wall Clock Time: 900.31 s
1800.03/900.31	c Best Solution: 7
1800.03/900.31	s SATISFIABLE
1800.03/900.31	v x1 -x2 -x3 -x4 -x5 -x6 -x7 -x8 -x9 -x10 -x11 -x12 -x13 -x14 -x15 -x16 -x17 x18 -x19 -x20 -x21 -x22 -x23 -x24 -x25 x26 -x27 -x28 -x29 -x30 -x31 -x32 -x33 -x34 -x35 x36 -x37 -x38 -x39 -x40 -x41 -x42 -x43 -x44 -x45 -x46 -x47 -x48 -x49 -x50 x51 -x52 -x53 -x54 -x55 -x56 -x57 -x58 -x59 -x60 -x61 -x62 -x63 -x64 -x65 x66 -x67 -x68 -x69 -x70 -x71 -x72 -x73 x74 -x75 -x76 -x77 -x78 -x79 -x80 -x81 x82 -x83 -x84 -x85 -x86 -x87 -x88 -x89 x90 -x91 -x92 -x93 -x94 -x95 -x96 -x97 -x98 -x99 -x100 -x101 -x102 -x103 -x104 x105 -x106 -x107 -x108 -x109 -x110 -x111 -x112 -x113 -x114 -x115 -x116 -x117 -x118 -x119 x120 -x121 -x122 -x123 -x124 -x125 -x126 -x127 -x128 -x129 x130 -x131 -x132 -x133 -x134 x135 -x136 -x137 -x138 -x139 -x140 -x141 -x142 -x143 x144 -x145 -x146 -x147 -x148 -x149 -x150 -x151 -x152 -x153 -x154 -x155 -x156 -x157 -x158 -x159 -x160 -x161 -x162 -x163 -x164 -x165 -x166 -x167 x168 -x169 -x170 -x171 -x172 x173 -x174 -x175 -x176 -x177 -x178 -x179 -x180 -x181 -x182 -x183 x184 -x185 -x186 -x187 -x188 -x189 -x190 -x191 -x192 -x193 -x194 -x195 -x196 -x197 -x198 -x199 -x200 x201 x202 -x203 -x204 -x205 -x206 -x207 -x208 -x209 -x210 -x211 -x212 -x213 -x214 -x215 -x216 -x217 -x218 -x219 -x220 -x221 -x222 -x223 -x224 -x225 -x226 -x227 -x228 -x229 -x230 -x231

Verifier Data 

OK	7

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3728835-1338674752/watcher-3728835-1338674752 -o /tmp/evaluation-result-3728835-1338674752/solver-3728835-1338674752 -C 1800 -W 1900 -M 15500 pwbo2.02 -file-format=wbo -verbosity=0 HOME/instance-3728835-1338674752.wbo 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.06 4.02 3.93 4/178 27735
/proc/meminfo: memFree=29931912/32873844 swapFree=7372/7372
[pid=27735] ppid=27733 vsize=4572 CPUtime=0 cores=0,2,4,6
/proc/27735/stat : 27735 (pwbo2.02) R 27733 27735 26396 0 -1 4202496 434 0 0 0 0 0 0 0 20 0 1 0 333603828 4681728 387 33554432000 4194304 5964478 140733969285904 140733969284152 4395529 0 0 4096 16387 0 0 0 17 4 0 0 0 0 0
/proc/27735/statm: 1143 387 168 433 0 683 0

[startup+0.052 s]
/proc/loadavg: 4.06 4.02 3.93 4/178 27735
/proc/meminfo: memFree=29931912/32873844 swapFree=7372/7372
[pid=27735] ppid=27733 vsize=164272 CPUtime=0.07 cores=0,2,4,6
/proc/27735/stat : 27735 (pwbo2.02) S 27733 27735 26396 0 -1 4202496 2584 0 0 0 7 0 0 0 20 0 3 0 333603828 168214528 2192 33554432000 4194304 5964478 140733969285904 140733969285424 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/27735/statm: 41068 2192 198 433 0 40632 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 164272

[startup+0.10018 s]
/proc/loadavg: 4.06 4.02 3.93 4/178 27735
/proc/meminfo: memFree=29931912/32873844 swapFree=7372/7372
[pid=27735] ppid=27733 vsize=164272 CPUtime=0.17 cores=0,2,4,6
/proc/27735/stat : 27735 (pwbo2.02) S 27733 27735 26396 0 -1 4202496 3234 0 0 0 17 0 0 0 20 0 3 0 333603828 168214528 2336 33554432000 4194304 5964478 140733969285904 140733969285424 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/27735/statm: 41068 2336 198 433 0 40632 0
Current children cumulated CPU time (s) 0.17
Current children cumulated vsize (KiB) 164272

[startup+0.300132 s]
/proc/loadavg: 4.06 4.02 3.93 4/178 27735
/proc/meminfo: memFree=29931912/32873844 swapFree=7372/7372
[pid=27735] ppid=27733 vsize=164272 CPUtime=0.57 cores=0,2,4,6
/proc/27735/stat : 27735 (pwbo2.02) S 27733 27735 26396 0 -1 4202496 5473 0 0 0 56 1 0 0 20 0 3 0 333603828 168214528 3522 33554432000 4194304 5964478 140733969285904 140733969285424 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/27735/statm: 41068 3522 198 433 0 40632 0
Current children cumulated CPU time (s) 0.57
Current children cumulated vsize (KiB) 164272

[startup+0.700141 s]
/proc/loadavg: 4.06 4.02 3.93 4/178 27735
/proc/meminfo: memFree=29931912/32873844 swapFree=7372/7372
[pid=27735] ppid=27733 vsize=164272 CPUtime=1.37 cores=0,2,4,6
/proc/27735/stat : 27735 (pwbo2.02) S 27733 27735 26396 0 -1 4202496 7495 0 0 0 136 1 0 0 20 0 3 0 333603828 168214528 4696 33554432000 4194304 5964478 140733969285904 140733969285424 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/27735/statm: 41068 4696 198 433 0 40632 0
Current children cumulated CPU time (s) 1.37
Current children cumulated vsize (KiB) 164272

[startup+1.50032 s]
/proc/loadavg: 4.06 4.02 3.93 5/181 27738
/proc/meminfo: memFree=29895680/32873844 swapFree=7372/7372
[pid=27735] ppid=27733 vsize=164272 CPUtime=2.97 cores=0,2,4,6
/proc/27735/stat : 27735 (pwbo2.02) S 27733 27735 26396 0 -1 4202496 9535 0 0 0 294 3 0 0 20 0 3 0 333603828 168214528 6426 33554432000 4194304 5964478 140733969285904 140733969285424 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/27735/statm: 41068 6426 198 433 0 40632 0
[pid=27735/tid=27737] ppid=27733 vsize=164272 CPUtime=1.47 cores=0,2,4,6
/proc/27735/task/27737/stat : 27737 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 3588 0 0 0 144 3 0 0 20 0 3 0 333603829 168214528 6426 33554432000 4194304 5964478 140733969285904 139904763295736 5136420 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27735/tid=27738] ppid=27733 vsize=164272 CPUtime=1.46 cores=0,2,4,6
/proc/27735/task/27738/stat : 27738 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 4857 0 0 0 146 0 0 0 20 0 3 0 333603829 168214528 6426 33554432000 4194304 5964478 140733969285904 139904752806408 4403717 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 2.97
Current children cumulated vsize (KiB) 164272

[startup+3.10032 s]
/proc/loadavg: 4.06 4.02 3.93 5/181 27738
/proc/meminfo: memFree=29879436/32873844 swapFree=7372/7372
[pid=27735] ppid=27733 vsize=164272 CPUtime=6.17 cores=0,2,4,6
/proc/27735/stat : 27735 (pwbo2.02) S 27733 27735 26396 0 -1 4202496 11778 0 0 0 610 7 0 0 20 0 3 0 333603828 168214528 8295 33554432000 4194304 5964478 140733969285904 140733969285424 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/27735/statm: 41068 8295 198 433 0 40632 0
[pid=27735/tid=27737] ppid=27733 vsize=164272 CPUtime=3.06 cores=0,2,4,6
/proc/27735/task/27737/stat : 27737 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 5093 0 0 0 301 5 0 0 20 0 3 0 333603829 168214528 8295 33554432000 4194304 5964478 140733969285904 139904763295736 4399912 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27735/tid=27738] ppid=27733 vsize=164272 CPUtime=3.06 cores=0,2,4,6
/proc/27735/task/27738/stat : 27738 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 5595 0 0 0 305 1 0 0 20 0 3 0 333603829 168214528 8295 33554432000 4194304 5964478 140733969285904 139904752806344 4403717 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 6.17
Current children cumulated vsize (KiB) 164272

[startup+6.30032 s]
/proc/loadavg: 4.06 4.01 3.93 5/181 27738
/proc/meminfo: memFree=29821404/32873844 swapFree=7372/7372
[pid=27735] ppid=27733 vsize=164272 CPUtime=12.57 cores=0,2,4,6
/proc/27735/stat : 27735 (pwbo2.02) S 27733 27735 26396 0 -1 4202496 19063 0 0 0 1239 18 0 0 20 0 3 0 333603828 168214528 15238 33554432000 4194304 5964478 140733969285904 140733969285424 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/27735/statm: 41068 15238 198 433 0 40632 0
[pid=27735/tid=27737] ppid=27733 vsize=164272 CPUtime=6.25 cores=0,2,4,6
/proc/27735/task/27737/stat : 27737 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 11704 0 0 0 609 16 0 0 20 0 3 0 333603829 168214528 15238 33554432000 4194304 5964478 140733969285904 139904763295736 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27735/tid=27738] ppid=27733 vsize=164272 CPUtime=6.25 cores=0,2,4,6
/proc/27735/task/27738/stat : 27738 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 6269 0 0 0 624 1 0 0 20 0 3 0 333603829 168214528 15238 33554432000 4194304 5964478 140733969285904 139904752806552 4403717 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 12.57
Current children cumulated vsize (KiB) 164272

[startup+12.7003 s]
/proc/loadavg: 4.05 4.01 3.93 5/181 27738
/proc/meminfo: memFree=29718988/32873844 swapFree=7372/7372
[pid=27735] ppid=27733 vsize=229940 CPUtime=25.37 cores=0,2,4,6
/proc/27735/stat : 27735 (pwbo2.02) S 27733 27735 26396 0 -1 4202496 29503 0 0 0 2512 25 0 0 20 0 3 0 333603828 235458560 25199 33554432000 4194304 5964478 140733969285904 140733969285424 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/27735/statm: 57485 25199 198 433 0 57049 0
[pid=27735/tid=27737] ppid=27733 vsize=229940 CPUtime=12.64 cores=0,2,4,6
/proc/27735/task/27737/stat : 27737 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 21159 0 0 0 1241 23 0 0 20 0 3 0 333603829 235458560 25199 33554432000 4194304 5964478 140733969285904 139904763295784 4381601 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27735/tid=27738] ppid=27733 vsize=229940 CPUtime=12.63 cores=0,2,4,6
/proc/27735/task/27738/stat : 27738 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 7254 0 0 0 1262 1 0 0 20 0 3 0 333603829 235458560 25199 33554432000 4194304 5964478 140733969285904 139904752806552 4392835 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 25.37
Current children cumulated vsize (KiB) 229940

[startup+25.5003 s]
/proc/loadavg: 4.04 4.01 3.93 5/181 27738
/proc/meminfo: memFree=29530740/32873844 swapFree=7372/7372
[pid=27735] ppid=27733 vsize=295876 CPUtime=50.96 cores=0,2,4,6
/proc/27735/stat : 27735 (pwbo2.02) S 27733 27735 26396 0 -1 4202496 50227 0 0 0 5056 40 0 0 20 0 3 0 333603828 302977024 45820 33554432000 4194304 5964478 140733969285904 140733969285424 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/27735/statm: 73969 45820 198 433 0 73533 0
[pid=27735/tid=27737] ppid=27733 vsize=295876 CPUtime=25.4 cores=0,2,4,6
/proc/27735/task/27737/stat : 27737 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 40417 0 0 0 2502 38 0 0 20 0 3 0 333603829 302977024 45820 33554432000 4194304 5964478 140733969285904 139904763295784 4400087 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27735/tid=27738] ppid=27733 vsize=295876 CPUtime=25.4 cores=0,2,4,6
/proc/27735/task/27738/stat : 27738 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 8720 0 0 0 2538 2 0 0 20 0 3 0 333603829 302977024 45820 33554432000 4194304 5964478 140733969285904 139904752806200 4221984 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 50.96
Current children cumulated vsize (KiB) 295876

[startup+51.1004 s]
/proc/loadavg: 4.02 4.01 3.93 5/181 27739
/proc/meminfo: memFree=29319768/32873844 swapFree=7372/7372
[pid=27735] ppid=27733 vsize=363416 CPUtime=102.14 cores=0,2,4,6
/proc/27735/stat : 27735 (pwbo2.02) S 27733 27735 26396 0 -1 4202496 79527 0 0 0 10146 68 0 0 20 0 3 0 333603828 372137984 75120 33554432000 4194304 5964478 140733969285904 140733969285424 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/27735/statm: 90854 75120 198 433 0 90418 0
[pid=27735/tid=27737] ppid=27733 vsize=363416 CPUtime=50.93 cores=0,2,4,6
/proc/27735/task/27737/stat : 27737 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 63667 0 0 0 5035 58 0 0 20 0 3 0 333603829 372137984 75120 33554432000 4194304 5964478 140733969285904 139904763295736 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27735/tid=27738] ppid=27733 vsize=363416 CPUtime=50.93 cores=0,2,4,6
/proc/27735/task/27738/stat : 27738 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 14770 0 0 0 5083 10 0 0 20 0 3 0 333603829 372137984 75120 33554432000 4194304 5964478 140733969285904 139904752806200 4403698 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 102.14
Current children cumulated vsize (KiB) 363416

[startup+102.307 s]
/proc/loadavg: 4.01 4.01 3.93 5/181 27739
/proc/meminfo: memFree=29076772/32873844 swapFree=7372/7372
[pid=27735] ppid=27733 vsize=496768 CPUtime=204.54 cores=0,2,4,6
/proc/27735/stat : 27735 (pwbo2.02) S 27733 27735 26396 0 -1 4202496 101682 0 0 0 20361 93 0 0 20 0 3 0 333603828 508690432 97153 33554432000 4194304 5964478 140733969285904 140733969285424 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/27735/statm: 124192 97153 198 433 0 123756 0
[pid=27735/tid=27737] ppid=27733 vsize=496768 CPUtime=102 cores=0,2,4,6
/proc/27735/task/27737/stat : 27737 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 75044 0 0 0 10132 68 0 0 20 0 3 0 333603829 508690432 97153 33554432000 4194304 5964478 140733969285904 139904763295768 4400078 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27735/tid=27738] ppid=27733 vsize=496768 CPUtime=102 cores=0,2,4,6
/proc/27735/task/27738/stat : 27738 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 25548 0 0 0 10173 27 0 0 20 0 3 0 333603829 508690432 97153 33554432000 4194304 5964478 140733969285904 139904752806200 4221984 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 204.54
Current children cumulated vsize (KiB) 496768

[startup+162.3 s]
/proc/loadavg: 4.00 4.00 3.93 5/181 27739
/proc/meminfo: memFree=28762212/32873844 swapFree=7372/7372
[pid=27735] ppid=27733 vsize=627988 CPUtime=324.51 cores=0,2,4,6
/proc/27735/stat : 27735 (pwbo2.02) S 27733 27735 26396 0 -1 4202496 138730 0 0 0 32315 136 0 0 20 0 3 0 333603828 643059712 134058 33554432000 4194304 5964478 140733969285904 140733969285424 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/27735/statm: 156997 134058 200 433 0 156561 0
[pid=27735/tid=27737] ppid=27733 vsize=627988 CPUtime=161.84 cores=0,2,4,6
/proc/27735/task/27737/stat : 27737 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 102263 0 0 0 16089 95 0 0 20 0 3 0 333603829 643059712 134058 33554432000 4194304 5964478 140733969285904 139904763295648 18446744073699066235 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27735/tid=27738] ppid=27733 vsize=627988 CPUtime=161.85 cores=0,2,4,6
/proc/27735/task/27738/stat : 27738 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 35377 0 0 0 16141 44 0 0 20 0 3 0 333603829 643059712 134058 33554432000 4194304 5964478 140733969285904 139904752805960 4222416 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 324.51
Current children cumulated vsize (KiB) 627988

[startup+222.3 s]
/proc/loadavg: 4.00 4.00 3.93 5/181 27739
/proc/meminfo: memFree=28559872/32873844 swapFree=7372/7372
[pid=27735] ppid=27733 vsize=693672 CPUtime=444.49 cores=0,2,4,6
/proc/27735/stat : 27735 (pwbo2.02) S 27733 27735 26396 0 -1 4202496 152657 0 0 0 44306 143 0 0 20 0 3 0 333603828 710320128 147799 33554432000 4194304 5964478 140733969285904 140733969285424 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/27735/statm: 173418 147799 200 433 0 172982 0
[pid=27735/tid=27737] ppid=27733 vsize=693672 CPUtime=221.68 cores=0,2,4,6
/proc/27735/task/27737/stat : 27737 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 108745 0 0 0 22070 98 0 0 20 0 3 0 333603829 710320128 147799 33554432000 4194304 5964478 140733969285904 139904763295736 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27735/tid=27738] ppid=27733 vsize=693672 CPUtime=221.69 cores=0,2,4,6
/proc/27735/task/27738/stat : 27738 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 42822 0 0 0 22119 50 0 0 20 0 3 0 333603829 710320128 147799 33554432000 4194304 5964478 140733969285904 139904752806304 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 444.49
Current children cumulated vsize (KiB) 693672

[startup+282.3 s]
/proc/loadavg: 4.00 4.00 3.93 5/182 27742
/proc/meminfo: memFree=28245648/32873844 swapFree=7372/7372
[pid=27735] ppid=27733 vsize=890996 CPUtime=564.46 cores=0,2,4,6
/proc/27735/stat : 27735 (pwbo2.02) S 27733 27735 26396 0 -1 4202496 192927 0 0 0 56252 194 0 0 20 0 3 0 333603828 912379904 188428 33554432000 4194304 5964478 140733969285904 140733969285424 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/27735/statm: 222749 188428 200 433 0 222313 0
[pid=27735/tid=27737] ppid=27733 vsize=890996 CPUtime=281.52 cores=0,2,4,6
/proc/27735/task/27737/stat : 27737 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 131811 0 0 0 28032 120 0 0 20 0 3 0 333603829 912379904 188428 33554432000 4194304 5964478 140733969285904 139904763295736 5136420 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27735/tid=27738] ppid=27733 vsize=890996 CPUtime=281.53 cores=0,2,4,6
/proc/27735/task/27738/stat : 27738 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 60026 0 0 0 28072 81 0 0 20 0 3 0 333603829 912379904 188428 33554432000 4194304 5964478 140733969285904 139904752805960 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 564.46
Current children cumulated vsize (KiB) 890996

[startup+342.3 s]
/proc/loadavg: 4.00 4.00 3.93 5/181 27743
/proc/meminfo: memFree=27757124/32873844 swapFree=7372/7372
[pid=27735] ppid=27733 vsize=1155080 CPUtime=684.4 cores=0,2,4,6
/proc/27735/stat : 27735 (pwbo2.02) S 27733 27735 26396 0 -1 4202496 263066 0 0 0 68184 256 0 0 20 0 3 0 333603828 1182801920 258529 33554432000 4194304 5964478 140733969285904 140733969285424 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/27735/statm: 288770 258529 200 433 0 288334 0

################
# More data... #
################

[pid=27735/tid=27738] ppid=27733 vsize=1155080 CPUtime=341.35 cores=0,2,4,6
/proc/27735/task/27738/stat : 27738 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 63757 0 0 0 34046 89 0 0 20 0 3 0 333603829 1182801920 258529 33554432000 4194304 5964478 140733969285904 139904752806200 4400091 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 684.4
Current children cumulated vsize (KiB) 1155080

[startup+402.3 s]
/proc/loadavg: 4.00 4.00 3.93 5/181 27743
/proc/meminfo: memFree=27504936/32873844 swapFree=7372/7372
[pid=27735] ppid=27733 vsize=1221088 CPUtime=804.36 cores=0,2,4,6
/proc/27735/stat : 27735 (pwbo2.02) S 27733 27735 26396 0 -1 4202496 286310 0 0 0 80151 285 0 0 20 0 3 0 333603828 1250394112 281760 33554432000 4194304 5964478 140733969285904 140733969285424 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/27735/statm: 305272 281760 200 433 0 304836 0
[pid=27735/tid=27737] ppid=27733 vsize=1221088 CPUtime=401.16 cores=0,2,4,6
/proc/27735/task/27737/stat : 27737 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 211163 0 0 0 39923 193 0 0 20 0 3 0 333603829 1250394112 281760 33554432000 4194304 5964478 140733969285904 139904763295648 4400067 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27735/tid=27738] ppid=27733 vsize=1221088 CPUtime=401.2 cores=0,2,4,6
/proc/27735/task/27738/stat : 27738 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 74057 0 0 0 40017 103 0 0 20 0 3 0 333603829 1250394112 281760 33554432000 4194304 5964478 140733969285904 139904752806304 4400024 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 804.36
Current children cumulated vsize (KiB) 1221088

[startup+462.3 s]
/proc/loadavg: 4.00 4.00 3.93 5/181 27743
/proc/meminfo: memFree=27300860/32873844 swapFree=7372/7372
[pid=27735] ppid=27733 vsize=1287572 CPUtime=924.31 cores=0,2,4,6
/proc/27735/stat : 27735 (pwbo2.02) S 27733 27735 26396 0 -1 4202496 302461 0 0 0 92117 314 0 0 20 0 3 0 333603828 1318473728 297845 33554432000 4194304 5964478 140733969285904 140733969285424 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/27735/statm: 321893 297845 200 433 0 321457 0
[pid=27735/tid=27737] ppid=27733 vsize=1287572 CPUtime=460.99 cores=0,2,4,6
/proc/27735/task/27737/stat : 27737 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 211219 0 0 0 45906 193 0 0 20 0 3 0 333603829 1318473728 297845 33554432000 4194304 5964478 140733969285904 139904763295648 4220672 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27735/tid=27738] ppid=27733 vsize=1287572 CPUtime=461.02 cores=0,2,4,6
/proc/27735/task/27738/stat : 27738 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 90152 0 0 0 45969 133 0 0 20 0 3 0 333603829 1318473728 297845 33554432000 4194304 5964478 140733969285904 139904752806112 4400115 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 924.31
Current children cumulated vsize (KiB) 1287572

[startup+522.3 s]
/proc/loadavg: 4.00 4.00 3.93 5/181 27743
/proc/meminfo: memFree=27170936/32873844 swapFree=7372/7372
[pid=27735] ppid=27733 vsize=1419200 CPUtime=1044.25 cores=0,2,4,6
/proc/27735/stat : 27735 (pwbo2.02) S 27733 27735 26396 0 -1 4202496 329352 0 0 0 104081 344 0 0 20 0 3 0 333603828 1453260800 324572 33554432000 4194304 5964478 140733969285904 140733969285424 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/27735/statm: 354800 324572 200 433 0 354364 0
[pid=27735/tid=27737] ppid=27733 vsize=1419200 CPUtime=520.8 cores=0,2,4,6
/proc/27735/task/27737/stat : 27737 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 233951 0 0 0 51860 220 0 0 20 0 3 0 333603829 1453260800 324572 33554432000 4194304 5964478 140733969285904 139904763295736 4400089 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27735/tid=27738] ppid=27733 vsize=1419200 CPUtime=520.87 cores=0,2,4,6
/proc/27735/task/27738/stat : 27738 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 94311 0 0 0 51950 137 0 0 20 0 3 0 333603829 1453260800 324573 33554432000 4194304 5964478 140733969285904 139904752806200 4403677 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1044.25
Current children cumulated vsize (KiB) 1419200

[startup+582.3 s]
/proc/loadavg: 4.04 4.01 3.93 5/181 27743
/proc/meminfo: memFree=26846456/32873844 swapFree=7372/7372
[pid=27735] ppid=27733 vsize=1682620 CPUtime=1164.17 cores=0,2,4,6
/proc/27735/stat : 27735 (pwbo2.02) S 27733 27735 26396 0 -1 4202496 405543 0 0 0 115991 426 0 0 20 0 3 0 333603828 1723002880 400716 33554432000 4194304 5964478 140733969285904 140733969285424 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/27735/statm: 420655 400716 200 433 0 420219 0
[pid=27735/tid=27737] ppid=27733 vsize=1682620 CPUtime=580.6 cores=0,2,4,6
/proc/27735/task/27737/stat : 27737 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 278900 0 0 0 57790 270 0 0 20 0 3 0 333603829 1723002880 400716 33554432000 4194304 5964478 140733969285904 139904763295736 4220672 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27735/tid=27738] ppid=27733 vsize=1682620 CPUtime=580.71 cores=0,2,4,6
/proc/27735/task/27738/stat : 27738 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 125553 0 0 0 57899 172 0 0 20 0 3 0 333603829 1723002880 400716 33554432000 4194304 5964478 140733969285904 139904752806192 4403692 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1164.17
Current children cumulated vsize (KiB) 1682620

[startup+642.3 s]
/proc/loadavg: 4.01 4.01 3.93 5/180 27743
/proc/meminfo: memFree=26596756/32873844 swapFree=7372/7372
[pid=27735] ppid=27733 vsize=1945240 CPUtime=1284.14 cores=0,2,4,6
/proc/27735/stat : 27735 (pwbo2.02) S 27733 27735 26396 0 -1 4202496 455471 0 0 0 127931 483 0 0 20 0 3 0 333603828 1991925760 450518 33554432000 4194304 5964478 140733969285904 140733969285424 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/27735/statm: 486310 450518 200 433 0 485874 0
[pid=27735/tid=27737] ppid=27733 vsize=1945240 CPUtime=640.42 cores=0,2,4,6
/proc/27735/task/27737/stat : 27737 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 328721 0 0 0 63714 328 0 0 20 0 3 0 333603829 1991925760 450518 33554432000 4194304 5964478 140733969285904 139904763295736 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27735/tid=27738] ppid=27733 vsize=1945240 CPUtime=640.55 cores=0,2,4,6
/proc/27735/task/27738/stat : 27738 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 125660 0 0 0 63882 173 0 0 20 0 3 0 333603829 1991925760 450518 33554432000 4194304 5964478 140733969285904 139904752805960 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1284.14
Current children cumulated vsize (KiB) 1945240

[startup+702.3 s]
/proc/loadavg: 4.00 4.00 3.93 5/181 27744
/proc/meminfo: memFree=26476268/32873844 swapFree=7372/7372
[pid=27735] ppid=27733 vsize=2012904 CPUtime=1404.08 cores=0,2,4,6
/proc/27735/stat : 27735 (pwbo2.02) S 27733 27735 26396 0 -1 4202496 484897 0 0 0 139896 512 0 0 20 0 3 0 333603828 2061213696 481162 33554432000 4194304 5964478 140733969285904 140733969285424 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/27735/statm: 503226 481162 200 433 0 502790 0
[pid=27735/tid=27737] ppid=27733 vsize=2012904 CPUtime=700.25 cores=0,2,4,6
/proc/27735/task/27737/stat : 27737 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 355819 0 0 0 69668 357 0 0 20 0 3 0 333603829 2061213696 481162 33554432000 4194304 5964478 140733969285904 139904763295736 4221120 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27735/tid=27738] ppid=27733 vsize=2012904 CPUtime=700.37 cores=0,2,4,6
/proc/27735/task/27738/stat : 27738 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 127988 0 0 0 69863 174 0 0 20 0 3 0 333603829 2061213696 481162 33554432000 4194304 5964478 140733969285904 139904752806200 4400001 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1404.08
Current children cumulated vsize (KiB) 2012904

[startup+762.307 s]
/proc/loadavg: 4.00 4.00 3.93 5/181 27744
/proc/meminfo: memFree=26391604/32873844 swapFree=7372/7372
[pid=27735] ppid=27733 vsize=2014396 CPUtime=1524.08 cores=0,2,4,6
/proc/27735/stat : 27735 (pwbo2.02) S 27733 27735 26396 0 -1 4202496 486880 0 0 0 151894 514 0 0 20 0 3 0 333603828 2062741504 483145 33554432000 4194304 5964478 140733969285904 140733969285424 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/27735/statm: 503599 483145 200 433 0 503163 0
[pid=27735/tid=27737] ppid=27733 vsize=2014396 CPUtime=760.1 cores=0,2,4,6
/proc/27735/task/27737/stat : 27737 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 355833 0 0 0 75652 358 0 0 20 0 3 0 333603829 2062741504 483145 33554432000 4194304 5964478 140733969285904 139904763295736 4400087 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27735/tid=27738] ppid=27733 vsize=2014396 CPUtime=760.23 cores=0,2,4,6
/proc/27735/task/27738/stat : 27738 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 129957 0 0 0 75846 177 0 0 20 0 3 0 333603829 2062741504 483145 33554432000 4194304 5964478 140733969285904 139904752805960 4403721 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1524.08
Current children cumulated vsize (KiB) 2014396

[startup+822.3 s]
/proc/loadavg: 4.00 4.00 3.93 5/181 27744
/proc/meminfo: memFree=26324796/32873844 swapFree=7372/7372
[pid=27735] ppid=27733 vsize=1946712 CPUtime=1644.04 cores=0,2,4,6
/proc/27735/stat : 27735 (pwbo2.02) S 27733 27735 26396 0 -1 4202496 486942 0 0 0 163890 514 0 0 20 0 3 0 333603828 1993433088 469045 33554432000 4194304 5964478 140733969285904 140733969285424 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/27735/statm: 486678 469045 200 433 0 486242 0
[pid=27735/tid=27737] ppid=27733 vsize=1946712 CPUtime=819.93 cores=0,2,4,6
/proc/27735/task/27737/stat : 27737 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 355848 0 0 0 81633 360 0 0 20 0 3 0 333603829 1993433088 469045 33554432000 4194304 5964478 140733969285904 139904763295880 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27735/tid=27738] ppid=27733 vsize=1946712 CPUtime=820.07 cores=0,2,4,6
/proc/27735/task/27738/stat : 27738 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 130004 0 0 0 81829 178 0 0 20 0 3 0 333603829 1993433088 469045 33554432000 4194304 5964478 140733969285904 139904752806112 4400787 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1644.04
Current children cumulated vsize (KiB) 1946712

[startup+882.3 s]
/proc/loadavg: 4.00 4.00 3.93 5/182 27747
/proc/meminfo: memFree=26172916/32873844 swapFree=7372/7372
[pid=27735] ppid=27733 vsize=2012676 CPUtime=1764.03 cores=0,2,4,6
/proc/27735/stat : 27735 (pwbo2.02) S 27733 27735 26396 0 -1 4202496 498656 0 0 0 175874 529 0 0 20 0 3 0 333603828 2060980224 480759 33554432000 4194304 5964478 140733969285904 140733969285424 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/27735/statm: 503169 480759 200 433 0 502733 0
[pid=27735/tid=27737] ppid=27733 vsize=2012676 CPUtime=879.78 cores=0,2,4,6
/proc/27735/task/27737/stat : 27737 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 355923 0 0 0 87617 361 0 0 20 0 3 0 333603829 2060980224 480759 33554432000 4194304 5964478 140733969285904 139904763295648 18446744073699066235 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27735/tid=27738] ppid=27733 vsize=2012676 CPUtime=879.91 cores=0,2,4,6
/proc/27735/task/27738/stat : 27738 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 141643 0 0 0 87797 194 0 0 20 0 3 0 333603829 2060980224 480759 33554432000 4194304 5964478 140733969285904 139904752806304 4400076 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1764.03
Current children cumulated vsize (KiB) 2012676



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+900.3 s]
/proc/loadavg: 3.43 3.87 3.89 3/174 27772
/proc/meminfo: memFree=28192444/32873844 swapFree=7372/7372
[pid=27735] ppid=27733 vsize=2012676 CPUtime=1800.03 cores=0,2,4,6
/proc/27735/stat : 27735 (pwbo2.02) S 27733 27735 26396 0 -1 4202496 498694 0 0 0 179474 529 0 0 20 0 3 0 333603828 2060980224 480942 33554432000 4194304 5964478 140733969285904 140733969285424 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/27735/statm: 503169 480942 200 433 0 502733 0
[pid=27735/tid=27737] ppid=27733 vsize=2012676 CPUtime=897.74 cores=0,2,4,6
/proc/27735/task/27737/stat : 27737 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 355956 0 0 0 89413 361 0 0 20 0 3 0 333603829 2060980224 480942 33554432000 4194304 5964478 140733969285904 139904763295648 4400091 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27735/tid=27738] ppid=27733 vsize=2012676 CPUtime=897.87 cores=0,2,4,6
/proc/27735/task/27738/stat : 27738 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 141648 0 0 0 89593 194 0 0 20 0 3 0 333603829 2060980224 480942 33554432000 4194304 5964478 140733969285904 139904752806304 4403664 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1800.03
Current children cumulated vsize (KiB) 2012676

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+900.3 s]
/proc/loadavg: 3.43 3.87 3.89 3/174 27772
/proc/meminfo: memFree=28192444/32873844 swapFree=7372/7372
[pid=27735] ppid=27733 vsize=2012676 CPUtime=1800.03 cores=0,2,4,6
/proc/27735/stat : 27735 (pwbo2.02) S 27733 27735 26396 0 -1 4202496 498694 0 0 0 179474 529 0 0 20 0 3 0 333603828 2060980224 480942 33554432000 4194304 5964478 140733969285904 140733969285424 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/27735/statm: 503169 480942 200 433 0 502733 0
[pid=27735/tid=27737] ppid=27733 vsize=2012676 CPUtime=897.74 cores=0,2,4,6
/proc/27735/task/27737/stat : 27737 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 355956 0 0 0 89413 361 0 0 20 0 3 0 333603829 2060980224 480942 33554432000 4194304 5964478 140733969285904 139904763295648 4400091 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27735/tid=27738] ppid=27733 vsize=2012676 CPUtime=897.87 cores=0,2,4,6
/proc/27735/task/27738/stat : 27738 (pwbo2.02) R 27733 27735 26396 0 -1 4202560 141648 0 0 0 89593 194 0 0 20 0 3 0 333603829 2060980224 480942 33554432000 4194304 5964478 140733969285904 139904752806304 4403664 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1800.03
Current children cumulated vsize (KiB) 2012676

Child status: 0

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 27735 and gives
#  childrusage.ru_utime.tv_sec=1790
#  childrusage.ru_utime.tv_usec=86865
#  childrusage.ru_stime.tv_sec=5
#  childrusage.ru_stime.tv_usec=651140
# CPU time returned by wait4() is 1795.74
# while last known CPU time is 1800.03
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 900.394
CPU time (s): 1800.03
CPU user time (s): 1794.74
CPU system time (s): 5.29
CPU usage (%): 199.916
Max. virtual memory (cumulated for all children) (KiB): 2014680

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1790.09
system time used= 5.65114
maximum resident set size= 1932788
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 498695
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 730
involuntary context switches= 1994

runsolver used 1.3008 second user time and 5.58415 second system time

The end

Launcher Data 

Begin job on node109 at 2012-06-03 00:05:53
IDJOB=3728835
IDBENCH=79289
IDSOLVER=2316
FILE ID=node109/3728835-1338674752
RUNJOBID= node109-1338670195-26999
PBS_JOBID= 14636486
Free space on /tmp= 70708 MiB

SOLVER NAME= pwbo 2.02
BENCH NAME= PB10/normalized-PB10/SOFT-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/tsp/normalized-t3002.11tsp11.1900584106--soft-0-100-0.wbo
COMMAND LINE= pwbo2.02 -file-format=wbo -verbosity=0 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3728835-1338674752/watcher-3728835-1338674752 -o /tmp/evaluation-result-3728835-1338674752/solver-3728835-1338674752 -C 1800 -W 1900 -M 15500  pwbo2.02 -file-format=wbo -verbosity=0 HOME/instance-3728835-1338674752.wbo

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= d3ccf217ec482c2d8ac120ae50401a4a
RANDOM SEED=1641590329

node109.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.264
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.52
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.264
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.264
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.264
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.264
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.264
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.264
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.264
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        29932068 kB
Buffers:          327764 kB
Cached:          1547780 kB
SwapCached:          944 kB
Active:          1080020 kB
Inactive:         987256 kB
Active(anon):     186016 kB
Inactive(anon):     7796 kB
Active(file):     894004 kB
Inactive(file):   979460 kB
Unevictable:        7372 kB
Mlocked:            7372 kB
SwapTotal:      67108856 kB
SwapFree:       67105492 kB
Dirty:             11320 kB
Writeback:             0 kB
AnonPages:        232828 kB
Mapped:            10840 kB
Shmem:                 4 kB
Slab:             736000 kB
SReclaimable:     178268 kB
SUnreclaim:       557732 kB
KernelStack:        1504 kB
PageTables:         4528 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     158212 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:     34816 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70708 MiB
End job on node109 at 2012-06-03 00:20:55