Trace number 3726058

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
pwbo 2.02SAT (TO)14 1800.48 900.321

General information on the benchmark

Namenormalized-PB06/OPT-SMALLINT/submitted-PB06/
manquiho/market-split/normalized-opt-market-split_6_50_1.opb
MD5SUMcdfa46b59876181d5d89fd18757b8cfb
Bench CategoryOPT-SMALLINT (optimisation, small integers)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark5
Best CPU time to get the best result obtained on this benchmark1790.16
Has Objective FunctionYES
SatisfiableYES
(Un)Satisfiability was provedYES
Best value of the objective function 5
Optimality of the best value was proved NO
Number of variables146
Total number of constraints12
Number of constraints which are clauses0
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints12
Minimum length of a constraint66
Maximum length of a constraint66
Number of terms in the objective function 96
Biggest coefficient in the objective function 128
Number of bits for the biggest coefficient in the objective function 8
Sum of the numbers in the objective function 3060
Number of bits of the sum of numbers in the objective function 12
Biggest number in a constraint 1273
Number of bits of the biggest number in a constraint 11
Biggest sum of numbers in a constraint 4329
Number of bits of the biggest sum of numbers13
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Quality of the solution as a function of time


Download the above graph as a PDF file / as an EPS file

Solver Data 

0.00/0.00	c ==============================[ Solver Version ]===============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  pwbo:                     2.02 beta - May, 2012                            |
0.00/0.00	c |                                                                             |
0.00/0.00	c Parsing opb file format. File HOME/instance-3726058-1338656202.opb.
0.00/0.00	c Instance file HOME/instance-3726058-1338656202.opb
0.00/0.00	c File size is 7798 bytes.
0.00/0.00	c Cost Offset: 0
0.00/0.00	c Highest Coefficient sum: 4329
0.00/0.00	c ============================[ Problem Statistics ]=============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  Parsing time:          0.00         s                                      |
0.00/0.00	c |  Number Variables:      146                                                 |
0.00/0.00	c |  Number Clauses:        0                                                   |
0.00/0.00	c |  Number Cardinality:    0                                                   |
0.00/0.00	c |  Number PB Constraints: 12                                                  |
0.00/0.00	c |                                                                             |
0.00/0.00	c |  Problem Type:          Optimization                                        |
0.00/0.00	c |  Objective Function:    Cardinality Constraint                              |
0.00/0.00	c |                                                                             |
0.00/0.00	c ===============================================================================
1800.10/900.31	o 1026
1800.10/900.31	o 963
1800.10/900.31	o 941
1800.10/900.31	o 870
1800.10/900.31	o 854
1800.10/900.31	o 828
1800.10/900.31	o 618
1800.10/900.31	o 586
1800.10/900.31	o 576
1800.10/900.31	o 480
1800.10/900.31	o 472
1800.10/900.31	o 284
1800.10/900.31	o 282
1800.10/900.31	o 248
1800.10/900.31	o 220
1800.10/900.31	o 216
1800.10/900.31	o 160
1800.10/900.31	o 114
1800.10/900.31	o 110
1800.10/900.31	o 76
1800.10/900.31	o 62
1800.10/900.31	o 60
1800.10/900.31	o 49
1800.10/900.31	o 48
1800.10/900.31	o 37
1800.10/900.31	o 32
1800.10/900.31	o 26
1800.10/900.31	o 22
1800.10/900.31	o 21
1800.10/900.31	o 19
1800.10/900.31	o 18
1800.10/900.31	o 14
1800.10/900.31	c ===============================================================================
1800.10/900.31	c Wall Clock Time: 900.31 s
1800.10/900.31	c Best Solution: 14
1800.10/900.31	s SATISFIABLE
1800.10/900.31	v -x1 -x2 -x3 -x4 -x5 -x6 -x7 -x8 -x9 -x10 -x11 -x12 -x13 -x14 -x15 -x16 -x17 x18 -x19 -x20 -x21 -x22 -x23 -x24 -x25 -x26 -x27 -x28 -x29 -x30 -x31 -x32 -x33 -x34 -x35 -x36 -x37 -x38 -x39 -x40 -x41 -x42 -x43 -x44 -x45 -x46 -x47 -x48 -x49 -x50 -x51 -x52 -x53 -x54 -x55 -x56 -x57 x58 x59 -x60 -x61 -x62 -x63 -x64 x65 x66 -x67 -x68 -x69 -x70 -x71 -x72 -x73 -x74 -x75 -x76 -x77 -x78 -x79 -x80 -x81 -x82 -x83 -x84 -x85 -x86 -x87 -x88 x89 x90 -x91 -x92 -x93 -x94 -x95 -x96 -x97 x98 x99 -x100 x101 x102 -x103 -x104 -x105 x106 x107 x108 x109 -x110 -x111 x112 x113 x114 -x115 -x116 x117 x118 -x119 x120 -x121 x122 x123 -x124 -x125 -x126 -x127 -x128 x129 -x130 -x131 x132 -x133 -x134 x135 -x136 x137 x138 x139 -x140 x141 x142 x143 -x144 -x145 -x146

Verifier Data 

OK	14

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3726058-1338656202/watcher-3726058-1338656202 -o /tmp/evaluation-result-3726058-1338656202/solver-3726058-1338656202 -C 1800 -W 1900 -M 15500 pwbo2.02 -file-format=opb -verbosity=0 HOME/instance-3726058-1338656202.opb 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.99 3.98 3.94 4/178 27115
/proc/meminfo: memFree=31141040/32873844 swapFree=7116/7116
[pid=27115] ppid=27113 vsize=163000 CPUtime=0 cores=0,2,4,6
/proc/27115/stat : 27115 (pwbo2.02) S 27113 27115 24826 0 -1 4202496 1140 0 0 0 0 0 0 0 20 0 3 0 331747213 166912000 1091 33554432000 4194304 5964478 140736385489344 140736385488864 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/27115/statm: 40750 1093 202 433 0 40314 0
[pid=27115/tid=27116] ppid=27113 vsize=163000 CPUtime=0 cores=0,2,4,6
/proc/27115/task/27116/stat : 27116 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 43 0 0 0 0 0 0 0 20 0 3 0 331747214 166912000 1096 33554432000 4194304 5964478 140736385489344 140625618230168 4389172 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27115/tid=27117] ppid=27113 vsize=163000 CPUtime=0 cores=0,2,4,6
/proc/27115/task/27117/stat : 27117 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 343 0 0 0 0 0 0 0 20 0 3 0 331747214 166912000 1099 33554432000 4194304 5964478 140736385489344 140734541505768 4395339 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0

[startup+0.0630149 s]
/proc/loadavg: 3.99 3.98 3.94 4/178 27115
/proc/meminfo: memFree=31141040/32873844 swapFree=7116/7116
[pid=27115] ppid=27113 vsize=176940 CPUtime=0.1 cores=0,2,4,6
/proc/27115/stat : 27115 (pwbo2.02) S 27113 27115 24826 0 -1 4202496 7141 0 0 0 9 1 0 0 20 0 3 0 331747213 181186560 7055 33554432000 4194304 5964478 140736385489344 140736385488864 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/27115/statm: 44235 7068 204 433 0 43799 0
[pid=27115/tid=27116] ppid=27113 vsize=176940 CPUtime=0.04 cores=0,2,4,6
/proc/27115/task/27116/stat : 27116 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 190 0 0 0 4 0 0 0 20 0 3 0 331747214 181186560 7069 33554432000 4194304 5964478 140736385489344 140625618229240 5142704 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27115/tid=27117] ppid=27113 vsize=176940 CPUtime=0.04 cores=0,2,4,6
/proc/27115/task/27117/stat : 27117 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 6207 0 0 0 3 1 0 0 20 0 3 0 331747214 181186560 7073 33554432000 4194304 5964478 140736385489344 140625607738856 5136420 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 0.1
Current children cumulated vsize (KiB) 176940

[startup+0.100349 s]
/proc/loadavg: 3.99 3.98 3.94 4/178 27115
/proc/meminfo: memFree=31141040/32873844 swapFree=7116/7116
[pid=27115] ppid=27113 vsize=189524 CPUtime=0.18 cores=0,2,4,6
/proc/27115/stat : 27115 (pwbo2.02) S 27113 27115 24826 0 -1 4202496 11123 0 0 0 16 2 0 0 20 0 3 0 331747213 194072576 10997 33554432000 4194304 5964478 140736385489344 140736385488864 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/27115/statm: 47381 10997 204 433 0 46945 0
[pid=27115/tid=27116] ppid=27113 vsize=189524 CPUtime=0.08 cores=0,2,4,6
/proc/27115/task/27116/stat : 27116 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 274 0 0 0 8 0 0 0 20 0 3 0 331747214 194072576 11004 33554432000 4194304 5964478 140736385489344 140625618229240 4222464 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27115/tid=27117] ppid=27113 vsize=189524 CPUtime=0.08 cores=0,2,4,6
/proc/27115/task/27117/stat : 27117 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 10096 0 0 0 6 2 0 0 20 0 3 0 331747214 194072576 11006 33554432000 4194304 5964478 140736385489344 140625607739048 5244295 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 0.18
Current children cumulated vsize (KiB) 189524

[startup+0.300326 s]
/proc/loadavg: 3.99 3.98 3.94 4/178 27115
/proc/meminfo: memFree=31141040/32873844 swapFree=7116/7116
[pid=27115] ppid=27113 vsize=196112 CPUtime=0.58 cores=0,2,4,6
/proc/27115/stat : 27115 (pwbo2.02) S 27113 27115 24826 0 -1 4202496 14827 0 0 0 54 4 0 0 20 0 3 0 331747213 200818688 14652 33554432000 4194304 5964478 140736385489344 140736385488864 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/27115/statm: 49028 14652 205 433 0 48592 0
[pid=27115/tid=27116] ppid=27113 vsize=196112 CPUtime=0.28 cores=0,2,4,6
/proc/27115/task/27116/stat : 27116 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 667 0 0 0 26 2 0 0 20 0 3 0 331747214 200818688 14652 33554432000 4194304 5964478 140736385489344 140625618229288 4399854 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27115/tid=27117] ppid=27113 vsize=196112 CPUtime=0.28 cores=0,2,4,6
/proc/27115/task/27117/stat : 27117 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 13398 0 0 0 25 3 0 0 20 0 3 0 331747214 200818688 14652 33554432000 4194304 5964478 140736385489344 140625607739624 4392918 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 0.58
Current children cumulated vsize (KiB) 196112

[startup+0.700322 s]
/proc/loadavg: 3.99 3.98 3.94 4/178 27115
/proc/meminfo: memFree=31141040/32873844 swapFree=7116/7116
[pid=27115] ppid=27113 vsize=196112 CPUtime=1.38 cores=0,2,4,6
/proc/27115/stat : 27115 (pwbo2.02) S 27113 27115 24826 0 -1 4202496 15510 0 0 0 132 6 0 0 20 0 3 0 331747213 200818688 15335 33554432000 4194304 5964478 140736385489344 140736385488864 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/27115/statm: 49028 15335 205 433 0 48592 0
[pid=27115/tid=27116] ppid=27113 vsize=196112 CPUtime=0.68 cores=0,2,4,6
/proc/27115/task/27116/stat : 27116 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 1295 0 0 0 65 3 0 0 20 0 3 0 331747214 200818688 15335 33554432000 4194304 5964478 140736385489344 140625618229288 4222416 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27115/tid=27117] ppid=27113 vsize=196112 CPUtime=0.68 cores=0,2,4,6
/proc/27115/task/27117/stat : 27117 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 13453 0 0 0 65 3 0 0 20 0 3 0 331747214 200818688 15335 33554432000 4194304 5964478 140736385489344 140625607739704 4392696 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 1.38
Current children cumulated vsize (KiB) 196112

[startup+1.50032 s]
/proc/loadavg: 3.99 3.98 3.94 5/181 27118
/proc/meminfo: memFree=31079264/32873844 swapFree=7116/7116
[pid=27115] ppid=27113 vsize=196256 CPUtime=2.99 cores=0,2,4,6
/proc/27115/stat : 27115 (pwbo2.02) S 27113 27115 24826 0 -1 4202496 16086 0 0 0 291 8 0 0 20 0 3 0 331747213 200966144 15911 33554432000 4194304 5964478 140736385489344 140736385488864 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/27115/statm: 49064 15911 205 433 0 48628 0
[pid=27115/tid=27116] ppid=27113 vsize=196256 CPUtime=1.48 cores=0,2,4,6
/proc/27115/task/27116/stat : 27116 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 1799 0 0 0 143 5 0 0 20 0 3 0 331747214 200966144 15911 33554432000 4194304 5964478 140736385489344 140625618229288 4389164 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27115/tid=27117] ppid=27113 vsize=196256 CPUtime=1.48 cores=0,2,4,6
/proc/27115/task/27117/stat : 27117 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 13525 0 0 0 145 3 0 0 20 0 3 0 331747214 200966144 15911 33554432000 4194304 5964478 140736385489344 140625607739576 4399920 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 2.99
Current children cumulated vsize (KiB) 196256

[startup+3.10032 s]
/proc/loadavg: 3.99 3.98 3.94 5/181 27118
/proc/meminfo: memFree=31076644/32873844 swapFree=7116/7116
[pid=27115] ppid=27113 vsize=196328 CPUtime=6.18 cores=0,2,4,6
/proc/27115/stat : 27115 (pwbo2.02) S 27113 27115 24826 0 -1 4202496 16924 0 0 0 608 10 0 0 20 0 3 0 331747213 201039872 16749 33554432000 4194304 5964478 140736385489344 140736385488864 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/27115/statm: 49082 16749 205 433 0 48646 0
[pid=27115/tid=27116] ppid=27113 vsize=196328 CPUtime=3.08 cores=0,2,4,6
/proc/27115/task/27116/stat : 27116 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 2558 0 0 0 300 8 0 0 20 0 3 0 331747214 201039872 16749 33554432000 4194304 5964478 140736385489344 140625618229240 4400038 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27115/tid=27117] ppid=27113 vsize=196328 CPUtime=3.08 cores=0,2,4,6
/proc/27115/task/27117/stat : 27117 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 13604 0 0 0 305 3 0 0 20 0 3 0 331747214 201039872 16749 33554432000 4194304 5964478 140736385489344 140625607739640 4392918 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 6.18
Current children cumulated vsize (KiB) 196328

[startup+6.30029 s]
/proc/loadavg: 3.99 3.98 3.94 5/181 27118
/proc/meminfo: memFree=31069256/32873844 swapFree=7116/7116
[pid=27115] ppid=27113 vsize=196436 CPUtime=12.58 cores=0,2,4,6
/proc/27115/stat : 27115 (pwbo2.02) S 27113 27115 24826 0 -1 4202496 18752 0 0 0 1245 13 0 0 20 0 3 0 331747213 201150464 18577 33554432000 4194304 5964478 140736385489344 140736385488864 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/27115/statm: 49109 18577 205 433 0 48673 0
[pid=27115/tid=27116] ppid=27113 vsize=196436 CPUtime=6.28 cores=0,2,4,6
/proc/27115/task/27116/stat : 27116 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 4278 0 0 0 617 11 0 0 20 0 3 0 331747214 201150464 18577 33554432000 4194304 5964478 140736385489344 140625618229048 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27115/tid=27117] ppid=27113 vsize=196436 CPUtime=6.28 cores=0,2,4,6
/proc/27115/task/27117/stat : 27117 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 13712 0 0 0 625 3 0 0 20 0 3 0 331747214 201150464 18577 33554432000 4194304 5964478 140736385489344 140625607739704 4392738 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 12.58
Current children cumulated vsize (KiB) 196436

[startup+12.7004 s]
/proc/loadavg: 3.99 3.98 3.94 5/181 27118
/proc/meminfo: memFree=31056260/32873844 swapFree=7116/7116
[pid=27115] ppid=27113 vsize=196596 CPUtime=25.38 cores=0,2,4,6
/proc/27115/stat : 27115 (pwbo2.02) S 27113 27115 24826 0 -1 4202496 21661 0 0 0 2521 17 0 0 20 0 3 0 331747213 201314304 21486 33554432000 4194304 5964478 140736385489344 140736385488864 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/27115/statm: 49149 21486 205 433 0 48713 0
[pid=27115/tid=27116] ppid=27113 vsize=196596 CPUtime=12.68 cores=0,2,4,6
/proc/27115/task/27116/stat : 27116 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 6957 0 0 0 1254 14 0 0 20 0 3 0 331747214 201314304 21486 33554432000 4194304 5964478 140736385489344 140625618229240 4382257 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27115/tid=27117] ppid=27113 vsize=196596 CPUtime=12.68 cores=0,2,4,6
/proc/27115/task/27117/stat : 27117 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 13942 0 0 0 1265 3 0 0 20 0 3 0 331747214 201314304 21486 33554432000 4194304 5964478 140736385489344 140625607739752 4403343 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 25.38
Current children cumulated vsize (KiB) 196596

[startup+25.5003 s]
/proc/loadavg: 3.99 3.98 3.94 5/181 27118
/proc/meminfo: memFree=31030284/32873844 swapFree=7116/7116
[pid=27115] ppid=27113 vsize=197100 CPUtime=50.97 cores=0,2,4,6
/proc/27115/stat : 27115 (pwbo2.02) S 27113 27115 24826 0 -1 4202496 26107 0 0 0 5069 28 0 0 20 0 3 0 331747213 201830400 25936 33554432000 4194304 5964478 140736385489344 140736385488864 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/27115/statm: 49275 25936 205 433 0 48839 0
[pid=27115/tid=27116] ppid=27113 vsize=197100 CPUtime=25.47 cores=0,2,4,6
/proc/27115/task/27116/stat : 27116 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 9769 0 0 0 2525 22 0 0 20 0 3 0 331747214 201830400 25936 33554432000 4194304 5964478 140736385489344 140625618229240 4400018 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27115/tid=27117] ppid=27113 vsize=197100 CPUtime=25.48 cores=0,2,4,6
/proc/27115/task/27117/stat : 27117 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 15576 0 0 0 2541 7 0 0 20 0 3 0 331747214 201830400 25936 33554432000 4194304 5964478 140736385489344 140625607739704 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 50.97
Current children cumulated vsize (KiB) 197100

[startup+51.1003 s]
/proc/loadavg: 3.99 3.98 3.94 5/181 27118
/proc/meminfo: memFree=30961292/32873844 swapFree=7116/7116
[pid=27115] ppid=27113 vsize=198352 CPUtime=102.13 cores=0,2,4,6
/proc/27115/stat : 27115 (pwbo2.02) S 27113 27115 24826 0 -1 4202496 35262 0 0 0 10177 36 0 0 20 0 3 0 331747213 203112448 35155 33554432000 4194304 5964478 140736385489344 140736385488864 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/27115/statm: 49588 35155 205 433 0 49152 0
[pid=27115/tid=27116] ppid=27113 vsize=198352 CPUtime=51.06 cores=0,2,4,6
/proc/27115/task/27116/stat : 27116 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 13551 0 0 0 5080 26 0 0 20 0 3 0 331747214 203112448 35155 33554432000 4194304 5964478 140736385489344 140625618229240 4400069 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27115/tid=27117] ppid=27113 vsize=198352 CPUtime=51.07 cores=0,2,4,6
/proc/27115/task/27117/stat : 27117 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 20949 0 0 0 5096 11 0 0 20 0 3 0 331747214 203112448 35155 33554432000 4194304 5964478 140736385489344 140625607739752 4400542 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 102.13
Current children cumulated vsize (KiB) 198352

[startup+102.307 s]
/proc/loadavg: 3.99 3.98 3.94 5/181 27118
/proc/meminfo: memFree=30898284/32873844 swapFree=7116/7116
[pid=27115] ppid=27113 vsize=331836 CPUtime=204.49 cores=0,2,4,6
/proc/27115/stat : 27115 (pwbo2.02) S 27113 27115 24826 0 -1 4202496 46912 0 0 0 20396 53 0 0 20 0 3 0 331747213 339800064 46885 33554432000 4194304 5964478 140736385489344 140736385488864 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/27115/statm: 82959 46885 205 433 0 82523 0
[pid=27115/tid=27116] ppid=27113 vsize=331836 CPUtime=102.26 cores=0,2,4,6
/proc/27115/task/27116/stat : 27116 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 20121 0 0 0 10188 38 0 0 20 0 3 0 331747214 339800064 46885 33554432000 4194304 5964478 140736385489344 140625618229288 4399938 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27115/tid=27117] ppid=27113 vsize=331836 CPUtime=102.26 cores=0,2,4,6
/proc/27115/task/27117/stat : 27117 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 26029 0 0 0 10208 18 0 0 20 0 3 0 331747214 339800064 46885 33554432000 4194304 5964478 140736385489344 140625607739704 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 204.49
Current children cumulated vsize (KiB) 331836

[startup+162.3 s]
/proc/loadavg: 3.99 3.98 3.94 5/181 27118
/proc/meminfo: memFree=30849428/32873844 swapFree=7116/7116
[pid=27115] ppid=27113 vsize=335704 CPUtime=324.46 cores=0,2,4,6
/proc/27115/stat : 27115 (pwbo2.02) S 27113 27115 24826 0 -1 4202496 56973 0 0 0 32375 71 0 0 20 0 3 0 331747213 343760896 58478 33554432000 4194304 5964478 140736385489344 140736385488864 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/27115/statm: 83926 58478 205 433 0 83490 0
[pid=27115/tid=27116] ppid=27113 vsize=335704 CPUtime=162.25 cores=0,2,4,6
/proc/27115/task/27116/stat : 27116 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 26583 0 0 0 16173 52 0 0 20 0 3 0 331747214 343760896 58478 33554432000 4194304 5964478 140736385489344 140625618229240 5139166 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27115/tid=27117] ppid=27113 vsize=335704 CPUtime=162.25 cores=0,2,4,6
/proc/27115/task/27117/stat : 27117 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 29628 0 0 0 16200 25 0 0 20 0 3 0 331747214 343760896 58478 33554432000 4194304 5964478 140736385489344 140625607739704 4221120 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 324.46
Current children cumulated vsize (KiB) 335704

[startup+222.3 s]
/proc/loadavg: 3.99 3.98 3.94 5/182 27121
/proc/meminfo: memFree=30759380/32873844 swapFree=7116/7116
[pid=27115] ppid=27113 vsize=338440 CPUtime=444.44 cores=0,2,4,6
/proc/27115/stat : 27115 (pwbo2.02) S 27113 27115 24826 0 -1 4202496 64355 0 0 0 44360 84 0 0 20 0 3 0 331747213 346562560 65860 33554432000 4194304 5964478 140736385489344 140736385488864 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/27115/statm: 84610 65860 205 433 0 84174 0
[pid=27115/tid=27116] ppid=27113 vsize=338440 CPUtime=222.25 cores=0,2,4,6
/proc/27115/task/27116/stat : 27116 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 30992 0 0 0 22163 62 0 0 20 0 3 0 331747214 346562560 65860 33554432000 4194304 5964478 140736385489344 140625618229240 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27115/tid=27117] ppid=27113 vsize=338440 CPUtime=222.26 cores=0,2,4,6
/proc/27115/task/27117/stat : 27117 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 32601 0 0 0 22196 30 0 0 20 0 3 0 331747214 346562560 65860 33554432000 4194304 5964478 140736385489344 140625607739704 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 444.44
Current children cumulated vsize (KiB) 338440

################
# More data... #
################

/proc/27115/task/27116/stat : 27116 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 35880 0 0 0 28157 69 0 0 20 0 3 0 331747214 416804864 73205 33554432000 4194304 5964478 140736385489344 140625618229240 4221120 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27115/tid=27117] ppid=27113 vsize=407036 CPUtime=282.26 cores=0,2,4,6
/proc/27115/task/27117/stat : 27117 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 35058 0 0 0 28192 34 0 0 20 0 3 0 331747214 416804864 73205 33554432000 4194304 5964478 140736385489344 140625607739704 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 564.43
Current children cumulated vsize (KiB) 407036

[startup+342.3 s]
/proc/loadavg: 3.99 3.98 3.94 5/181 27145
/proc/meminfo: memFree=30620880/32873844 swapFree=7116/7116
[pid=27115] ppid=27113 vsize=411328 CPUtime=684.41 cores=0,2,4,6
/proc/27115/stat : 27115 (pwbo2.02) S 27113 27115 24826 0 -1 4202496 78285 0 0 0 68332 109 0 0 20 0 3 0 331747213 421199872 79785 33554432000 4194304 5964478 140736385489344 140736385488864 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/27115/statm: 102832 79785 205 433 0 102396 0
[pid=27115/tid=27116] ppid=27113 vsize=411328 CPUtime=342.26 cores=0,2,4,6
/proc/27115/task/27116/stat : 27116 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 40190 0 0 0 34146 80 0 0 20 0 3 0 331747214 421199872 79785 33554432000 4194304 5964478 140736385489344 140625618229240 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27115/tid=27117] ppid=27113 vsize=411328 CPUtime=342.26 cores=0,2,4,6
/proc/27115/task/27117/stat : 27117 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 37333 0 0 0 34186 40 0 0 20 0 3 0 331747214 421199872 79785 33554432000 4194304 5964478 140736385489344 140625607739704 4400519 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 684.41
Current children cumulated vsize (KiB) 411328

[startup+402.3 s]
/proc/loadavg: 4.03 4.00 3.95 5/181 27145
/proc/meminfo: memFree=30595868/32873844 swapFree=7116/7116
[pid=27115] ppid=27113 vsize=413484 CPUtime=804.38 cores=0,2,4,6
/proc/27115/stat : 27115 (pwbo2.02) S 27113 27115 24826 0 -1 4202496 82683 0 0 0 80322 116 0 0 20 0 3 0 331747213 423407616 84174 33554432000 4194304 5964478 140736385489344 140736385488864 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/27115/statm: 103371 84174 205 433 0 102935 0
[pid=27115/tid=27116] ppid=27113 vsize=413484 CPUtime=402.25 cores=0,2,4,6
/proc/27115/task/27116/stat : 27116 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 42571 0 0 0 40141 84 0 0 20 0 3 0 331747214 423407616 84174 33554432000 4194304 5964478 140736385489344 140625618229240 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27115/tid=27117] ppid=27113 vsize=413484 CPUtime=402.27 cores=0,2,4,6
/proc/27115/task/27117/stat : 27117 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 39350 0 0 0 40182 45 0 0 20 0 3 0 331747214 423407616 84174 33554432000 4194304 5964478 140736385489344 140625607739704 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 804.38
Current children cumulated vsize (KiB) 413484

[startup+462.3 s]
/proc/loadavg: 4.01 4.00 3.95 5/181 27145
/proc/meminfo: memFree=30543288/32873844 swapFree=7116/7116
[pid=27115] ppid=27113 vsize=415892 CPUtime=924.32 cores=0,2,4,6
/proc/27115/stat : 27115 (pwbo2.02) S 27113 27115 24826 0 -1 4202496 88223 0 0 0 92308 124 0 0 20 0 3 0 331747213 425873408 89797 33554432000 4194304 5964478 140736385489344 140736385488864 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/27115/statm: 103973 89797 205 433 0 103537 0
[pid=27115/tid=27116] ppid=27113 vsize=415892 CPUtime=462.24 cores=0,2,4,6
/proc/27115/task/27116/stat : 27116 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 46138 0 0 0 46133 91 0 0 20 0 3 0 331747214 425873408 89797 33554432000 4194304 5964478 140736385489344 140625618229288 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27115/tid=27117] ppid=27113 vsize=415892 CPUtime=462.24 cores=0,2,4,6
/proc/27115/task/27117/stat : 27117 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 41323 0 0 0 46176 48 0 0 20 0 3 0 331747214 425873408 89797 33554432000 4194304 5964478 140736385489344 140625607739704 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 924.32
Current children cumulated vsize (KiB) 415892

[startup+522.3 s]
/proc/loadavg: 4.00 4.00 3.95 5/181 27145
/proc/meminfo: memFree=30523752/32873844 swapFree=7116/7116
[pid=27115] ppid=27113 vsize=483960 CPUtime=1044.26 cores=0,2,4,6
/proc/27115/stat : 27115 (pwbo2.02) S 27113 27115 24826 0 -1 4202496 93484 0 0 0 104295 131 0 0 20 0 3 0 331747213 495575040 95037 33554432000 4194304 5964478 140736385489344 140736385488864 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/27115/statm: 120990 95037 205 433 0 120554 0
[pid=27115/tid=27116] ppid=27113 vsize=483960 CPUtime=522.2 cores=0,2,4,6
/proc/27115/task/27116/stat : 27116 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 49460 0 0 0 52124 96 0 0 20 0 3 0 331747214 495575040 95037 33554432000 4194304 5964478 140736385489344 140625618229240 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27115/tid=27117] ppid=27113 vsize=483960 CPUtime=522.24 cores=0,2,4,6
/proc/27115/task/27117/stat : 27117 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 43262 0 0 0 52173 51 0 0 20 0 3 0 331747214 495575040 95037 33554432000 4194304 5964478 140736385489344 140625607739704 4400542 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1044.26
Current children cumulated vsize (KiB) 483960

[startup+582.3 s]
/proc/loadavg: 4.00 4.00 3.95 5/180 27145
/proc/meminfo: memFree=30493748/32873844 swapFree=7116/7116
[pid=27115] ppid=27113 vsize=551840 CPUtime=1164.24 cores=0,2,4,6
/proc/27115/stat : 27115 (pwbo2.02) S 27113 27115 24826 0 -1 4202496 100994 0 0 0 116279 145 0 0 20 0 3 0 331747213 565084160 102551 33554432000 4194304 5964478 140736385489344 140736385488864 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/27115/statm: 137960 102551 205 433 0 137524 0
[pid=27115/tid=27116] ppid=27113 vsize=551840 CPUtime=582.21 cores=0,2,4,6
/proc/27115/task/27116/stat : 27116 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 55276 0 0 0 58113 108 0 0 20 0 3 0 331747214 565084160 102551 33554432000 4194304 5964478 140736385489344 140625618229288 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27115/tid=27117] ppid=27113 vsize=551840 CPUtime=582.24 cores=0,2,4,6
/proc/27115/task/27117/stat : 27117 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 44956 0 0 0 58170 54 0 0 20 0 3 0 331747214 565084160 102551 33554432000 4194304 5964478 140736385489344 140625607739704 4221120 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1164.24
Current children cumulated vsize (KiB) 551840

[startup+642.3 s]
/proc/loadavg: 4.00 4.00 3.95 5/181 27187
/proc/meminfo: memFree=31007216/32873844 swapFree=7116/7116
[pid=27115] ppid=27113 vsize=553332 CPUtime=1284.22 cores=0,2,4,6
/proc/27115/stat : 27115 (pwbo2.02) S 27113 27115 24826 0 -1 4202496 105408 0 0 0 128268 154 0 0 20 0 3 0 331747213 566611968 106974 33554432000 4194304 5964478 140736385489344 140736385488864 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/27115/statm: 138333 106974 205 433 0 137897 0
[pid=27115/tid=27116] ppid=27113 vsize=553332 CPUtime=642.21 cores=0,2,4,6
/proc/27115/task/27116/stat : 27116 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 58030 0 0 0 64106 115 0 0 20 0 3 0 331747214 566611968 106974 33554432000 4194304 5964478 140736385489344 140625618229240 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27115/tid=27117] ppid=27113 vsize=553332 CPUtime=642.24 cores=0,2,4,6
/proc/27115/task/27117/stat : 27117 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 46616 0 0 0 64166 58 0 0 20 0 3 0 331747214 566611968 106974 33554432000 4194304 5964478 140736385489344 140625607739704 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1284.22
Current children cumulated vsize (KiB) 553332

[startup+702.3 s]
/proc/loadavg: 4.07 4.01 3.95 5/181 27187
/proc/meminfo: memFree=30931652/32873844 swapFree=7116/7116
[pid=27115] ppid=27113 vsize=557544 CPUtime=1404.15 cores=0,2,4,6
/proc/27115/stat : 27115 (pwbo2.02) S 27113 27115 24826 0 -1 4202496 109038 0 0 0 140254 161 0 0 20 0 3 0 331747213 570925056 110582 33554432000 4194304 5964478 140736385489344 140736385488864 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/27115/statm: 139386 110582 205 433 0 138950 0
[pid=27115/tid=27116] ppid=27113 vsize=557544 CPUtime=702.17 cores=0,2,4,6
/proc/27115/task/27116/stat : 27116 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 59931 0 0 0 70098 119 0 0 20 0 3 0 331747214 570925056 110582 33554432000 4194304 5964478 140736385489344 140625618229240 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27115/tid=27117] ppid=27113 vsize=557544 CPUtime=702.24 cores=0,2,4,6
/proc/27115/task/27117/stat : 27117 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 48345 0 0 0 70161 63 0 0 20 0 3 0 331747214 570925056 110582 33554432000 4194304 5964478 140736385489344 140625607739704 4399938 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1404.15
Current children cumulated vsize (KiB) 557544

[startup+762.307 s]
/proc/loadavg: 4.02 4.01 3.95 5/181 27187
/proc/meminfo: memFree=30864556/32873844 swapFree=7116/7116
[pid=27115] ppid=27113 vsize=559484 CPUtime=1524.14 cores=0,2,4,6
/proc/27115/stat : 27115 (pwbo2.02) S 27113 27115 24826 0 -1 4202496 116139 0 0 0 152236 178 0 0 20 0 3 0 331747213 572911616 117858 33554432000 4194304 5964478 140736385489344 140736385488864 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/27115/statm: 139871 117858 205 433 0 139435 0
[pid=27115/tid=27116] ppid=27113 vsize=559484 CPUtime=762.18 cores=0,2,4,6
/proc/27115/task/27116/stat : 27116 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 65781 0 0 0 76083 135 0 0 20 0 3 0 331747214 572911616 117858 33554432000 4194304 5964478 140736385489344 140625618229240 4379248 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27115/tid=27117] ppid=27113 vsize=559484 CPUtime=762.24 cores=0,2,4,6
/proc/27115/task/27117/stat : 27117 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 49596 0 0 0 76158 66 0 0 20 0 3 0 331747214 572911616 117858 33554432000 4194304 5964478 140736385489344 140625607739704 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1524.14
Current children cumulated vsize (KiB) 559484

[startup+822.3 s]
/proc/loadavg: 4.01 4.00 3.95 5/182 27190
/proc/meminfo: memFree=30830944/32873844 swapFree=7116/7116
[pid=27115] ppid=27113 vsize=560240 CPUtime=1644.1 cores=0,2,4,6
/proc/27115/stat : 27115 (pwbo2.02) S 27113 27115 24826 0 -1 4202496 117684 0 0 0 164229 181 0 0 20 0 3 0 331747213 573685760 119403 33554432000 4194304 5964478 140736385489344 140736385488864 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/27115/statm: 140060 119403 205 433 0 139624 0
[pid=27115/tid=27116] ppid=27113 vsize=560240 CPUtime=822.18 cores=0,2,4,6
/proc/27115/task/27116/stat : 27116 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 65915 0 0 0 82082 136 0 0 20 0 3 0 331747214 573685760 119403 33554432000 4194304 5964478 140736385489344 140625618229048 4222314 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27115/tid=27117] ppid=27113 vsize=560240 CPUtime=822.24 cores=0,2,4,6
/proc/27115/task/27117/stat : 27117 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 51007 0 0 0 82154 70 0 0 20 0 3 0 331747214 573685760 119403 33554432000 4194304 5964478 140736385489344 140625607739704 4399952 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1644.1
Current children cumulated vsize (KiB) 560240

[startup+882.3 s]
/proc/loadavg: 4.04 4.01 3.96 5/181 27190
/proc/meminfo: memFree=30785072/32873844 swapFree=7116/7116
[pid=27115] ppid=27113 vsize=626692 CPUtime=1764.09 cores=0,2,4,6
/proc/27115/stat : 27115 (pwbo2.02) S 27113 27115 24826 0 -1 4202496 123064 0 0 0 176216 193 0 0 20 0 3 0 331747213 641732608 124783 33554432000 4194304 5964478 140736385489344 140736385488864 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/27115/statm: 156673 124783 205 433 0 156237 0
[pid=27115/tid=27116] ppid=27113 vsize=626692 CPUtime=882.18 cores=0,2,4,6
/proc/27115/task/27116/stat : 27116 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 70016 0 0 0 88072 146 0 0 20 0 3 0 331747214 641732608 124783 33554432000 4194304 5964478 140736385489344 140625618229240 4221120 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27115/tid=27117] ppid=27113 vsize=626692 CPUtime=882.24 cores=0,2,4,6
/proc/27115/task/27117/stat : 27117 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 52286 0 0 0 88150 74 0 0 20 0 3 0 331747214 641732608 124783 33554432000 4194304 5964478 140736385489344 140625607739704 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1764.09
Current children cumulated vsize (KiB) 626692



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+900.3 s]
/proc/loadavg: 4.11 4.03 3.96 5/181 27191
/proc/meminfo: memFree=30761856/32873844 swapFree=7116/7116
[pid=27115] ppid=27113 vsize=626924 CPUtime=1800.1 cores=0,2,4,6
/proc/27115/stat : 27115 (pwbo2.02) S 27113 27115 24826 0 -1 4202496 125598 0 0 0 179812 198 0 0 20 0 3 0 331747213 641970176 127317 33554432000 4194304 5964478 140736385489344 140736385488864 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/27115/statm: 156731 127317 205 433 0 156295 0
[pid=27115/tid=27116] ppid=27113 vsize=626924 CPUtime=900.19 cores=0,2,4,6
/proc/27115/task/27116/stat : 27116 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 72201 0 0 0 89868 151 0 0 20 0 3 0 331747214 641970176 127317 33554432000 4194304 5964478 140736385489344 140625618229240 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27115/tid=27117] ppid=27113 vsize=626924 CPUtime=900.25 cores=0,2,4,6
/proc/27115/task/27117/stat : 27117 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 52635 0 0 0 89951 74 0 0 20 0 3 0 331747214 641970176 127317 33554432000 4194304 5964478 140736385489344 140625607739704 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1800.1
Current children cumulated vsize (KiB) 626924

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+900.3 s]
/proc/loadavg: 4.11 4.03 3.96 5/181 27191
/proc/meminfo: memFree=30761856/32873844 swapFree=7116/7116
[pid=27115] ppid=27113 vsize=626924 CPUtime=1800.1 cores=0,2,4,6
/proc/27115/stat : 27115 (pwbo2.02) S 27113 27115 24826 0 -1 4202496 125598 0 0 0 179812 198 0 0 20 0 3 0 331747213 641970176 127317 33554432000 4194304 5964478 140736385489344 140736385488864 4546685 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/27115/statm: 156731 127317 205 433 0 156295 0
[pid=27115/tid=27116] ppid=27113 vsize=626924 CPUtime=900.19 cores=0,2,4,6
/proc/27115/task/27116/stat : 27116 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 72201 0 0 0 89868 151 0 0 20 0 3 0 331747214 641970176 127317 33554432000 4194304 5964478 140736385489344 140625618229240 4399920 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=27115/tid=27117] ppid=27113 vsize=626924 CPUtime=900.25 cores=0,2,4,6
/proc/27115/task/27117/stat : 27117 (pwbo2.02) R 27113 27115 24826 0 -1 4202560 52635 0 0 0 89951 74 0 0 20 0 3 0 331747214 641970176 127317 33554432000 4194304 5964478 140736385489344 140625607739704 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1800.1
Current children cumulated vsize (KiB) 626924

Child status: 0
Real time (s): 900.321
CPU time (s): 1800.48
CPU user time (s): 1798.2
CPU system time (s): 2.28165
CPU usage (%): 199.982
Max. virtual memory (cumulated for all children) (KiB): 626924

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1798.2
system time used= 2.28165
maximum resident set size= 509272
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 125599
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 470
involuntary context switches= 1973

runsolver used 1.09383 second user time and 5.8921 second system time

The end

Launcher Data 

Begin job on node111 at 2012-06-02 18:56:42
IDJOB=3726058
IDBENCH=2902
IDSOLVER=2309
FILE ID=node111/3726058-1338656202
RUNJOBID= node111-1338651416-26330
PBS_JOBID= 14636469
Free space on /tmp= 67392 MiB

SOLVER NAME= pwbo 2.02
BENCH NAME= PB06/final/normalized-PB06/OPT-SMALLINT/submitted-PB06/manquiho/market-split/normalized-opt-market-split_6_50_1.opb
COMMAND LINE= pwbo2.02 -file-format=opb -verbosity=0 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3726058-1338656202/watcher-3726058-1338656202 -o /tmp/evaluation-result-3726058-1338656202/solver-3726058-1338656202 -C 1800 -W 1900 -M 15500  pwbo2.02 -file-format=opb -verbosity=0 HOME/instance-3726058-1338656202.opb

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= cdfa46b59876181d5d89fd18757b8cfb
RANDOM SEED=449452573

node111.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.152
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.30
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.152
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.152
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.152
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.152
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.152
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.152
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.152
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        31140668 kB
Buffers:          150072 kB
Cached:           429068 kB
SwapCached:         1352 kB
Active:           489240 kB
Inactive:         459376 kB
Active(anon):     370504 kB
Inactive(anon):     1120 kB
Active(file):     118736 kB
Inactive(file):   458256 kB
Unevictable:        7116 kB
Mlocked:            7116 kB
SwapTotal:      67108856 kB
SwapFree:       67100048 kB
Dirty:              5492 kB
Writeback:             0 kB
AnonPages:        375352 kB
Mapped:             9556 kB
Shmem:                 0 kB
Slab:             644324 kB
SReclaimable:      86616 kB
SUnreclaim:       557708 kB
KernelStack:        1472 kB
PageTables:         4656 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     173000 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:    253952 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 67384 MiB
End job on node111 at 2012-06-02 19:11:45