Trace number 3703463

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
pwbo 2.0SAT (TO)1523 1800.53 900.324

General information on the benchmark

Namenormalized-PB06/OPT-SMALLINT/mps-v2-20-10/
MIPLIB/miplib/normalized-mps-v2-20-10-p0201.opb
MD5SUM303eaeab4b7ccd67c7fab31d13d20bc2
Bench CategoryOPT-SMALLINT (optimisation, small integers)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark1523
Best CPU time to get the best result obtained on this benchmark0.200968
Has Objective FunctionYES
SatisfiableYES
(Un)Satisfiability was provedYES
Best value of the objective function 1523
Optimality of the best value was proved YES
Number of variables201
Total number of constraints133
Number of constraints which are clauses20
Number of constraints which are cardinality constraints (but not clauses)26
Number of constraints which are nor clauses,nor cardinality constraints87
Minimum length of a constraint3
Maximum length of a constraint67
Number of terms in the objective function 201
Biggest coefficient in the objective function 1920
Number of bits for the biggest coefficient in the objective function 11
Sum of the numbers in the objective function 19980
Number of bits of the sum of numbers in the objective function 15
Biggest number in a constraint 1920
Number of bits of the biggest number in a constraint 11
Biggest sum of numbers in a constraint 19980
Number of bits of the biggest sum of numbers15
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Quality of the solution as a function of time


Download the above graph as a PDF file / as an EPS file

Solver Data 

0.00/0.00	c ==============================[ Solver Version ]===============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  pwbo:                    2.0 beta - March, 2012                            |
0.00/0.00	c |                                                                             |
0.00/0.00	c Parsing opb file format. File HOME/instance-3703463-1338085948.opb.
0.00/0.00	c Instance file HOME/instance-3703463-1338085948.opb
0.00/0.00	c File size is 17810 bytes.
0.00/0.00	c Cost Offset: 0
0.00/0.00	c Highest Coefficient sum: 777
0.00/0.00	c ============================[ Problem Statistics ]=============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  Parsing time:          0.00         s                                      |
0.00/0.00	c |  Number Variables:      201                                                 |
0.00/0.00	c |  Number Clauses:        20                                                  |
0.00/0.00	c |  Number Cardinality:    80                                                  |
0.00/0.00	c |  Number PB Constraints: 33                                                  |
0.00/0.00	c |                                                                             |
0.00/0.00	c |  Problem Type:          Optimization                                        |
0.00/0.00	c |  Objective Function:    PB Constraint                                       |
0.00/0.00	c |                                                                             |
0.00/0.00	c ===============================================================================
1800.18/900.31	o 2590
1800.18/900.31	o 2578
1800.18/900.31	o 2568
1800.18/900.31	o 2554
1800.18/900.31	o 2371
1800.18/900.31	o 2321
1800.18/900.31	o 2265
1800.18/900.31	o 2233
1800.18/900.31	o 2221
1800.18/900.31	o 2188
1800.18/900.31	o 2176
1800.18/900.31	o 2164
1800.18/900.31	o 2155
1800.18/900.31	o 1992
1800.18/900.31	o 1990
1800.18/900.31	o 1967
1800.18/900.31	o 1956
1800.18/900.31	o 1955
1800.18/900.31	o 1950
1800.18/900.31	o 1910
1800.18/900.31	o 1890
1800.18/900.31	o 1880
1800.18/900.31	o 1878
1800.18/900.31	o 1850
1800.18/900.31	o 1822
1800.18/900.31	o 1814
1800.18/900.31	o 1802
1800.18/900.31	o 1794
1800.18/900.31	o 1789
1800.18/900.31	o 1784
1800.18/900.31	o 1760
1800.18/900.31	o 1743
1800.18/900.31	o 1742
1800.18/900.31	o 1735
1800.18/900.31	o 1725
1800.18/900.31	o 1716
1800.18/900.31	o 1711
1800.18/900.31	o 1709
1800.18/900.31	o 1704
1800.18/900.31	o 1693
1800.18/900.31	o 1685
1800.18/900.31	o 1682
1800.18/900.31	o 1675
1800.18/900.31	o 1671
1800.18/900.31	o 1659
1800.18/900.31	o 1651
1800.18/900.31	o 1648
1800.18/900.31	o 1636
1800.18/900.31	o 1633
1800.18/900.31	o 1632
1800.18/900.31	o 1623
1800.18/900.31	o 1613
1800.18/900.31	o 1603
1800.18/900.31	o 1599
1800.18/900.31	o 1597
1800.18/900.31	o 1579
1800.18/900.31	o 1573
1800.18/900.31	o 1561
1800.18/900.31	o 1559
1800.18/900.31	o 1555
1800.18/900.31	o 1543
1800.18/900.31	o 1535
1800.18/900.31	o 1523
1800.18/900.31	c ===============================================================================
1800.18/900.31	c Wall Clock Time: 900.31 s
1800.18/900.31	c Best Solution: 1523
1800.18/900.31	s SATISFIABLE
1800.18/900.31	v -x1 x2 -x3 x4 -x5 -x6 -x7 -x8 -x9 -x10 -x11 -x12 -x13 -x14 -x15 -x16 -x17 -x18 -x19 -x20 -x21 -x22 -x23 -x24 -x25 x26 -x27 -x28 -x29 -x30 -x31 -x32 -x33 -x34 -x35 -x36 -x37 -x38 x39 -x40 -x41 -x42 -x43 x44 -x45 -x46 -x47 -x48 -x49 -x50 -x51 -x52 -x53 -x54 -x55 -x56 x57 -x58 x59 -x60 -x61 -x62 -x63 -x64 -x65 -x66 -x67 -x68 -x69 -x70 -x71 -x72 -x73 -x74 x75 x76 -x77 -x78 -x79 -x80 -x81 -x82 -x83 -x84 -x85 -x86 -x87 -x88 -x89 -x90 -x91 -x92 x93 -x94 -x95 -x96 x97 -x98 -x99 -x100 -x101 -x102 -x103 -x104 -x105 -x106 -x107 -x108 -x109 -x110 x111 -x112 -x113 -x114 -x115 x116 -x117 -x118 -x119 -x120 -x121 -x122 -x123 -x124 -x125 -x126 -x127 -x128 x129 -x130 x131 -x132 -x133 -x134 -x135 -x136 -x137 -x138 -x139 -x140 -x141 -x142 -x143 -x144 -x145 -x146 x147 x148 -x149 -x150 -x151 -x152 -x153 -x154 -x155 -x156 -x157 -x158 -x159 -x160 -x161 -x162 -x163 -x164 x165 -x166 -x167 -x168 x169 -x170 -x171 -x172 -x173 -x174 -x175 -x176 -x177 -x178 -x179 -x180 -x181 -x182 x183 -x184 -x185 -x186 -x187 x188 -x189 -x190 -x191 -x192 -x193 -x194 -x195 -x196 -x197 -x198 -x199 -x200 x201

Verifier Data 

OK	1523

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3703463-1338085948/watcher-3703463-1338085948 -o /tmp/evaluation-result-3703463-1338085948/solver-3703463-1338085948 -C 1800 -W 1900 -M 15500 pwbo2.0 -file-format=opb -verbosity=0 HOME/instance-3703463-1338085948.opb 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.00 3.97 3.67 4/180 16233
/proc/meminfo: memFree=28777708/32873844 swapFree=7372/7372
[pid=16233] ppid=16231 vsize=162480 CPUtime=0 cores=1,3,5,7
/proc/16233/stat : 16233 (pwbo2.0) S 16231 16233 15889 0 -1 4202496 912 0 0 0 0 0 0 0 20 0 3 0 274719612 166379520 1881 33554432000 4194304 5965102 140736949773648 140736949773168 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/16233/statm: 40620 1881 197 433 0 40184 0
[pid=16233/tid=16234] ppid=16231 vsize=162480 CPUtime=0 cores=1,3,5,7
/proc/16233/task/16234/stat : 16234 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 89 0 0 0 0 0 0 0 20 0 3 0 274719612 166379520 1884 33554432000 4194304 5965102 140736949773648 139858605505496 5137076 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=16233/tid=16235] ppid=16231 vsize=162480 CPUtime=0 cores=1,3,5,7
/proc/16233/task/16235/stat : 16235 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 22 0 0 0 0 0 0 0 20 0 3 0 274719612 166379520 1884 33554432000 4194304 5965102 140736949773648 139858595015728 4221120 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0

[startup+0.0328501 s]
/proc/loadavg: 4.00 3.97 3.67 4/180 16233
/proc/meminfo: memFree=28777708/32873844 swapFree=7372/7372
[pid=16233] ppid=16231 vsize=162480 CPUtime=0.05 cores=1,3,5,7
/proc/16233/stat : 16233 (pwbo2.0) S 16231 16233 15889 0 -1 4202496 1354 0 0 0 5 0 0 0 20 0 3 0 274719612 166379520 2284 33554432000 4194304 5965102 140736949773648 140736949773168 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/16233/statm: 40620 2284 200 433 0 40184 0
[pid=16233/tid=16234] ppid=16231 vsize=162480 CPUtime=0.01 cores=1,3,5,7
/proc/16233/task/16234/stat : 16234 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 362 0 0 0 1 0 0 0 20 0 3 0 274719612 166379520 2284 33554432000 4194304 5965102 140736949773648 139858605504856 4404717 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=16233/tid=16235] ppid=16231 vsize=162480 CPUtime=0.01 cores=1,3,5,7
/proc/16233/task/16235/stat : 16235 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 188 0 0 0 1 0 0 0 20 0 3 0 274719612 166379520 2284 33554432000 4194304 5965102 140736949773648 139858595015112 4221984 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 162480

[startup+0.10032 s]
/proc/loadavg: 4.00 3.97 3.67 4/180 16233
/proc/meminfo: memFree=28777708/32873844 swapFree=7372/7372
[pid=16233] ppid=16231 vsize=162480 CPUtime=0.18 cores=1,3,5,7
/proc/16233/stat : 16233 (pwbo2.0) S 16231 16233 15889 0 -1 4202496 1706 0 0 0 18 0 0 0 20 0 3 0 274719612 166379520 2622 33554432000 4194304 5965102 140736949773648 140736949773168 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/16233/statm: 40620 2622 200 433 0 40184 0
[pid=16233/tid=16234] ppid=16231 vsize=162480 CPUtime=0.08 cores=1,3,5,7
/proc/16233/task/16234/stat : 16234 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 594 0 0 0 8 0 0 0 20 0 3 0 274719612 166379520 2622 33554432000 4194304 5965102 140736949773648 139858605504496 5131417 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=16233/tid=16235] ppid=16231 vsize=162480 CPUtime=0.08 cores=1,3,5,7
/proc/16233/task/16235/stat : 16235 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 308 0 0 0 8 0 0 0 20 0 3 0 274719612 166379520 2622 33554432000 4194304 5965102 140736949773648 139858595014944 4404732 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 0.18
Current children cumulated vsize (KiB) 162480

[startup+0.300273 s]
/proc/loadavg: 4.00 3.97 3.67 4/180 16233
/proc/meminfo: memFree=28777708/32873844 swapFree=7372/7372
[pid=16233] ppid=16231 vsize=162480 CPUtime=0.58 cores=1,3,5,7
/proc/16233/stat : 16233 (pwbo2.0) S 16231 16233 15889 0 -1 4202496 2476 0 0 0 58 0 0 0 20 0 3 0 274719612 166379520 3253 33554432000 4194304 5965102 140736949773648 140736949773168 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/16233/statm: 40620 3253 200 433 0 40184 0
[pid=16233/tid=16234] ppid=16231 vsize=162480 CPUtime=0.28 cores=1,3,5,7
/proc/16233/task/16234/stat : 16234 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 1133 0 0 0 28 0 0 0 20 0 3 0 274719612 166379520 3253 33554432000 4194304 5965102 140736949773648 139858605504856 4404446 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=16233/tid=16235] ppid=16231 vsize=162480 CPUtime=0.28 cores=1,3,5,7
/proc/16233/task/16235/stat : 16235 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 539 0 0 0 28 0 0 0 20 0 3 0 274719612 166379520 3253 33554432000 4194304 5965102 140736949773648 139858595015016 5137076 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 0.58
Current children cumulated vsize (KiB) 162480

[startup+0.700303 s]
/proc/loadavg: 4.00 3.97 3.67 4/180 16233
/proc/meminfo: memFree=28777708/32873844 swapFree=7372/7372
[pid=16233] ppid=16231 vsize=162612 CPUtime=1.39 cores=1,3,5,7
/proc/16233/stat : 16233 (pwbo2.0) S 16231 16233 15889 0 -1 4202496 3308 0 0 0 138 1 0 0 20 0 3 0 274719612 166514688 3736 33554432000 4194304 5965102 140736949773648 140736949773168 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/16233/statm: 40653 3736 200 433 0 40217 0
[pid=16233/tid=16234] ppid=16231 vsize=162612 CPUtime=0.68 cores=1,3,5,7
/proc/16233/task/16234/stat : 16234 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 1870 0 0 0 68 0 0 0 20 0 3 0 274719612 166514688 3736 33554432000 4194304 5965102 140736949773648 139858605505464 4404738 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=16233/tid=16235] ppid=16231 vsize=162612 CPUtime=0.68 cores=1,3,5,7
/proc/16233/task/16235/stat : 16235 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 634 0 0 0 68 0 0 0 20 0 3 0 274719612 166514688 3736 33554432000 4194304 5965102 140736949773648 139858595015320 4394008 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1.39
Current children cumulated vsize (KiB) 162612

[startup+1.50028 s]
/proc/loadavg: 4.00 3.97 3.67 5/183 16236
/proc/meminfo: memFree=28763064/32873844 swapFree=7372/7372
[pid=16233] ppid=16231 vsize=162612 CPUtime=2.98 cores=1,3,5,7
/proc/16233/stat : 16233 (pwbo2.0) S 16231 16233 15889 0 -1 4202496 3933 0 0 0 297 1 0 0 20 0 3 0 274719612 166514688 4323 33554432000 4194304 5965102 140736949773648 140736949773168 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/16233/statm: 40653 4323 200 433 0 40217 0
[pid=16233/tid=16234] ppid=16231 vsize=162612 CPUtime=1.48 cores=1,3,5,7
/proc/16233/task/16234/stat : 16234 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 2257 0 0 0 148 0 0 0 20 0 3 0 274719612 166514688 4323 33554432000 4194304 5965102 140736949773648 139858605504856 4400960 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=16233/tid=16235] ppid=16231 vsize=162612 CPUtime=1.48 cores=1,3,5,7
/proc/16233/task/16235/stat : 16235 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 872 0 0 0 148 0 0 0 20 0 3 0 274719612 166514688 4323 33554432000 4194304 5965102 140736949773648 139858595015320 4222416 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 2.98
Current children cumulated vsize (KiB) 162612

[startup+3.1003 s]
/proc/loadavg: 4.00 3.97 3.67 5/183 16236
/proc/meminfo: memFree=28757848/32873844 swapFree=7372/7372
[pid=16233] ppid=16231 vsize=163468 CPUtime=6.18 cores=1,3,5,7
/proc/16233/stat : 16233 (pwbo2.0) S 16231 16233 15889 0 -1 4202496 4850 0 0 0 617 1 0 0 20 0 3 0 274719612 167391232 5237 33554432000 4194304 5965102 140736949773648 140736949773168 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/16233/statm: 40867 5237 200 433 0 40431 0
[pid=16233/tid=16234] ppid=16231 vsize=163468 CPUtime=3.08 cores=1,3,5,7
/proc/16233/task/16234/stat : 16234 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 3083 0 0 0 308 0 0 0 20 0 3 0 274719612 167391232 5237 33554432000 4194304 5965102 140736949773648 139858605504840 4393747 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=16233/tid=16235] ppid=16231 vsize=163468 CPUtime=3.08 cores=1,3,5,7
/proc/16233/task/16235/stat : 16235 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 963 0 0 0 308 0 0 0 20 0 3 0 274719612 167391232 5237 33554432000 4194304 5965102 140736949773648 139858595015320 4390219 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 6.18
Current children cumulated vsize (KiB) 163468

[startup+6.30033 s]
/proc/loadavg: 4.00 3.97 3.67 5/183 16236
/proc/meminfo: memFree=28754500/32873844 swapFree=7372/7372
[pid=16233] ppid=16231 vsize=163040 CPUtime=12.58 cores=1,3,5,7
/proc/16233/stat : 16233 (pwbo2.0) S 16231 16233 15889 0 -1 4202496 6104 0 0 0 1256 2 0 0 20 0 3 0 274719612 166952960 6288 33554432000 4194304 5965102 140736949773648 140736949773168 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/16233/statm: 40760 6288 200 433 0 40324 0
[pid=16233/tid=16234] ppid=16231 vsize=163040 CPUtime=6.28 cores=1,3,5,7
/proc/16233/task/16234/stat : 16234 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 3775 0 0 0 627 1 0 0 20 0 3 0 274719612 166952960 6288 33554432000 4194304 5965102 140736949773648 139858605504504 4401116 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=16233/tid=16235] ppid=16231 vsize=163040 CPUtime=6.29 cores=1,3,5,7
/proc/16233/task/16235/stat : 16235 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 1525 0 0 0 627 2 0 0 20 0 3 0 274719612 166952960 6288 33554432000 4194304 5965102 140736949773648 139858595015016 4404767 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 12.58
Current children cumulated vsize (KiB) 163040

[startup+12.7003 s]
/proc/loadavg: 4.00 3.97 3.67 5/183 16236
/proc/meminfo: memFree=28739148/32873844 swapFree=7372/7372
[pid=16233] ppid=16231 vsize=163900 CPUtime=25.38 cores=1,3,5,7
/proc/16233/stat : 16233 (pwbo2.0) S 16231 16233 15889 0 -1 4202496 9642 0 0 0 2534 4 0 0 20 0 3 0 274719612 167833600 9730 33554432000 4194304 5965102 140736949773648 140736949773168 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/16233/statm: 40975 9730 200 433 0 40539 0
[pid=16233/tid=16234] ppid=16231 vsize=163900 CPUtime=12.68 cores=1,3,5,7
/proc/16233/task/16234/stat : 16234 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 6549 0 0 0 1265 3 0 0 20 0 3 0 274719612 167833600 9730 33554432000 4194304 5965102 140736949773648 139858605505464 4393958 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=16233/tid=16235] ppid=16231 vsize=163900 CPUtime=12.68 cores=1,3,5,7
/proc/16233/task/16235/stat : 16235 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 2289 0 0 0 1266 2 0 0 20 0 3 0 274719612 167833600 9730 33554432000 4194304 5965102 140736949773648 139858595014904 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 25.38
Current children cumulated vsize (KiB) 163900

[startup+25.5003 s]
/proc/loadavg: 4.00 3.97 3.67 5/183 16236
/proc/meminfo: memFree=28736916/32873844 swapFree=7372/7372
[pid=16233] ppid=16231 vsize=163900 CPUtime=50.97 cores=1,3,5,7
/proc/16233/stat : 16233 (pwbo2.0) S 16231 16233 15889 0 -1 4202496 10548 0 0 0 5093 4 0 0 20 0 3 0 274719612 167833600 10636 33554432000 4194304 5965102 140736949773648 140736949773168 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/16233/statm: 40975 10636 200 433 0 40539 0
[pid=16233/tid=16234] ppid=16231 vsize=163900 CPUtime=25.48 cores=1,3,5,7
/proc/16233/task/16234/stat : 16234 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 6919 0 0 0 2545 3 0 0 20 0 3 0 274719612 167833600 10636 33554432000 4194304 5965102 140736949773648 139858605505464 5137076 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=16233/tid=16235] ppid=16231 vsize=163900 CPUtime=25.48 cores=1,3,5,7
/proc/16233/task/16235/stat : 16235 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 2825 0 0 0 2546 2 0 0 20 0 3 0 274719612 167833600 10636 33554432000 4194304 5965102 140736949773648 139858595014968 18446744073699066235 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 50.97
Current children cumulated vsize (KiB) 163900

[startup+51.1004 s]
/proc/loadavg: 4.00 3.97 3.68 5/183 16477
/proc/meminfo: memFree=29957120/32873844 swapFree=7372/7372
[pid=16233] ppid=16231 vsize=164516 CPUtime=102.15 cores=1,3,5,7
/proc/16233/stat : 16233 (pwbo2.0) S 16231 16233 15889 0 -1 4202496 17088 0 0 0 10207 8 0 0 20 0 3 0 274719612 168464384 16739 33554432000 4194304 5965102 140736949773648 140736949773168 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/16233/statm: 41129 16739 200 433 0 40693 0
[pid=16233/tid=16234] ppid=16231 vsize=164516 CPUtime=51.07 cores=1,3,5,7
/proc/16233/task/16234/stat : 16234 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 11692 0 0 0 5101 6 0 0 20 0 3 0 274719612 168464384 16739 33554432000 4194304 5965102 140736949773648 139858605505464 4404383 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=16233/tid=16235] ppid=16231 vsize=164516 CPUtime=51.07 cores=1,3,5,7
/proc/16233/task/16235/stat : 16235 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 4592 0 0 0 5104 3 0 0 20 0 3 0 274719612 168464384 16739 33554432000 4194304 5965102 140736949773648 139858595014968 4400963 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 102.15
Current children cumulated vsize (KiB) 164516

[startup+102.307 s]
/proc/loadavg: 4.00 3.97 3.70 5/183 16477
/proc/meminfo: memFree=29868336/32873844 swapFree=7372/7372
[pid=16233] ppid=16231 vsize=164756 CPUtime=204.53 cores=1,3,5,7
/proc/16233/stat : 16233 (pwbo2.0) S 16231 16233 15889 0 -1 4202496 23475 0 0 0 20439 14 0 0 20 0 3 0 274719612 168710144 24329 33554432000 4194304 5965102 140736949773648 140736949773168 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/16233/statm: 41189 24329 200 433 0 40753 0
[pid=16233/tid=16234] ppid=16231 vsize=164756 CPUtime=102.26 cores=1,3,5,7
/proc/16233/task/16234/stat : 16234 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 15938 0 0 0 10214 12 0 0 20 0 3 0 274719612 168710144 24329 33554432000 4194304 5965102 140736949773648 139858605504312 4400978 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=16233/tid=16235] ppid=16231 vsize=164756 CPUtime=102.28 cores=1,3,5,7
/proc/16233/task/16235/stat : 16235 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 6733 0 0 0 10222 6 0 0 20 0 3 0 274719612 168710144 24329 33554432000 4194304 5965102 140736949773648 139858595015000 4404717 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 204.53
Current children cumulated vsize (KiB) 164756

[startup+162.3 s]
/proc/loadavg: 4.00 3.97 3.71 5/184 16519
/proc/meminfo: memFree=29876848/32873844 swapFree=7372/7372
[pid=16233] ppid=16231 vsize=231152 CPUtime=324.5 cores=1,3,5,7
/proc/16233/stat : 16233 (pwbo2.0) S 16231 16233 15889 0 -1 4202496 31057 0 0 0 32425 25 0 0 20 0 3 0 274719612 236699648 31911 33554432000 4194304 5965102 140736949773648 140736949773168 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/16233/statm: 57788 31911 200 433 0 57352 0
[pid=16233/tid=16234] ppid=16231 vsize=231152 CPUtime=162.26 cores=1,3,5,7
/proc/16233/task/16234/stat : 16234 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 23496 0 0 0 16203 23 0 0 20 0 3 0 274719612 236699648 31911 33554432000 4194304 5965102 140736949773648 139858605504552 4404664 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=16233/tid=16235] ppid=16231 vsize=231152 CPUtime=162.28 cores=1,3,5,7
/proc/16233/task/16235/stat : 16235 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 6757 0 0 0 16221 7 0 0 20 0 3 0 274719612 236699648 31911 33554432000 4194304 5965102 140736949773648 139858595014728 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 324.5
Current children cumulated vsize (KiB) 231152

[startup+222.3 s]
/proc/loadavg: 4.00 3.97 3.73 5/183 16519
/proc/meminfo: memFree=29752920/32873844 swapFree=7372/7372
[pid=16233] ppid=16231 vsize=231688 CPUtime=444.47 cores=1,3,5,7
/proc/16233/stat : 16233 (pwbo2.0) S 16231 16233 15889 0 -1 4202496 36925 0 0 0 44415 32 0 0 20 0 3 0 274719612 237248512 37797 33554432000 4194304 5965102 140736949773648 140736949773168 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/16233/statm: 57922 37797 200 433 0 57486 0
[pid=16233/tid=16234] ppid=16231 vsize=231688 CPUtime=222.26 cores=1,3,5,7
/proc/16233/task/16234/stat : 16234 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 27243 0 0 0 22197 29 0 0 20 0 3 0 274719612 237248512 37797 33554432000 4194304 5965102 140736949773648 139858605504312 4401137 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=16233/tid=16235] ppid=16231 vsize=231688 CPUtime=222.28 cores=1,3,5,7
/proc/16233/task/16235/stat : 16235 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 8878 0 0 0 22217 11 0 0 20 0 3 0 274719612 237248512 37797 33554432000 4194304 5965102 140736949773648 139858595014776 4404410 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 444.47
Current children cumulated vsize (KiB) 231688

################
# More data... #
################

/proc/16233/task/16234/stat : 16234 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 33850 0 0 0 28180 42 0 0 20 0 3 0 274719612 237248512 46317 33554432000 4194304 5965102 140736949773648 139858605504504 4221984 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=16233/tid=16235] ppid=16231 vsize=231688 CPUtime=282.27 cores=1,3,5,7
/proc/16233/task/16235/stat : 16235 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 10579 0 0 0 28214 13 0 0 20 0 3 0 274719612 237248512 46317 33554432000 4194304 5965102 140736949773648 139858595014776 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 564.42
Current children cumulated vsize (KiB) 231688

[startup+342.3 s]
/proc/loadavg: 4.00 3.97 3.75 5/183 16519
/proc/meminfo: memFree=29587020/32873844 swapFree=7372/7372
[pid=16233] ppid=16231 vsize=298124 CPUtime=684.39 cores=1,3,5,7
/proc/16233/stat : 16233 (pwbo2.0) S 16231 16233 15889 0 -1 4202496 51418 0 0 0 68385 54 0 0 20 0 3 0 274719612 305278976 52594 33554432000 4194304 5965102 140736949773648 140736949773168 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/16233/statm: 74531 52594 200 433 0 74095 0
[pid=16233/tid=16234] ppid=16231 vsize=298124 CPUtime=342.22 cores=1,3,5,7
/proc/16233/task/16234/stat : 16234 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 37615 0 0 0 34173 49 0 0 20 0 3 0 274719612 305278976 52594 33554432000 4194304 5965102 140736949773648 139858605504312 4389809 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=16233/tid=16235] ppid=16231 vsize=298124 CPUtime=342.28 cores=1,3,5,7
/proc/16233/task/16235/stat : 16235 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 12999 0 0 0 34211 17 0 0 20 0 3 0 274719612 305278976 52594 33554432000 4194304 5965102 140736949773648 139858595014776 4380356 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 684.39
Current children cumulated vsize (KiB) 298124

[startup+402.3 s]
/proc/loadavg: 4.05 3.99 3.77 5/183 16519
/proc/meminfo: memFree=29484500/32873844 swapFree=7372/7372
[pid=16233] ppid=16231 vsize=298124 CPUtime=804.37 cores=1,3,5,7
/proc/16233/stat : 16233 (pwbo2.0) S 16231 16233 15889 0 -1 4202496 56626 0 0 0 80378 59 0 0 20 0 3 0 274719612 305278976 57799 33554432000 4194304 5965102 140736949773648 140736949773168 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/16233/statm: 74531 57799 200 433 0 74095 0
[pid=16233/tid=16234] ppid=16231 vsize=298124 CPUtime=402.22 cores=1,3,5,7
/proc/16233/task/16234/stat : 16234 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 42799 0 0 0 40167 55 0 0 20 0 3 0 274719612 305278976 57799 33554432000 4194304 5965102 140736949773648 139858605504312 4401462 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=16233/tid=16235] ppid=16231 vsize=298124 CPUtime=402.28 cores=1,3,5,7
/proc/16233/task/16235/stat : 16235 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 13023 0 0 0 40210 18 0 0 20 0 3 0 274719612 305278976 57799 33554432000 4194304 5965102 140736949773648 139858595015112 4390823 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 804.37
Current children cumulated vsize (KiB) 298124

[startup+462.3 s]
/proc/loadavg: 4.07 4.00 3.79 5/184 16523
/proc/meminfo: memFree=29395928/32873844 swapFree=7372/7372
[pid=16233] ppid=16231 vsize=298848 CPUtime=924.35 cores=1,3,5,7
/proc/16233/stat : 16233 (pwbo2.0) S 16231 16233 15889 0 -1 4202496 63291 0 0 0 92367 68 0 0 20 0 3 0 274719612 306020352 64464 33554432000 4194304 5965102 140736949773648 140736949773168 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/16233/statm: 74712 64464 200 433 0 74276 0
[pid=16233/tid=16234] ppid=16231 vsize=298848 CPUtime=462.22 cores=1,3,5,7
/proc/16233/task/16234/stat : 16234 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 49454 0 0 0 46157 65 0 0 20 0 3 0 274719612 306020352 64464 33554432000 4194304 5965102 140736949773648 139858605504504 4400960 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=16233/tid=16235] ppid=16231 vsize=298848 CPUtime=462.28 cores=1,3,5,7
/proc/16233/task/16235/stat : 16235 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 13033 0 0 0 46209 19 0 0 20 0 3 0 274719612 306020352 64464 33554432000 4194304 5965102 140736949773648 139858595015112 4380336 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 924.35
Current children cumulated vsize (KiB) 298848

[startup+522.3 s]
/proc/loadavg: 4.02 4.00 3.80 5/183 16523
/proc/meminfo: memFree=29359904/32873844 swapFree=7372/7372
[pid=16233] ppid=16231 vsize=364384 CPUtime=1044.33 cores=1,3,5,7
/proc/16233/stat : 16233 (pwbo2.0) S 16231 16233 15889 0 -1 4202496 67820 0 0 0 104358 75 0 0 20 0 3 0 274719612 373129216 69134 33554432000 4194304 5965102 140736949773648 140736949773168 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/16233/statm: 91096 69134 200 433 0 90660 0
[pid=16233/tid=16234] ppid=16231 vsize=364384 CPUtime=522.21 cores=1,3,5,7
/proc/16233/task/16234/stat : 16234 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 53983 0 0 0 52150 71 0 0 20 0 3 0 274719612 373129216 69134 33554432000 4194304 5965102 140736949773648 139858605504416 4400960 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=16233/tid=16235] ppid=16231 vsize=364384 CPUtime=522.29 cores=1,3,5,7
/proc/16233/task/16235/stat : 16235 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 13033 0 0 0 52210 19 0 0 20 0 3 0 274719612 373129216 69134 33554432000 4194304 5965102 140736949773648 139858595015072 4404704 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1044.33
Current children cumulated vsize (KiB) 364384

[startup+582.3 s]
/proc/loadavg: 4.01 4.00 3.81 5/183 16523
/proc/meminfo: memFree=29351284/32873844 swapFree=7372/7372
[pid=16233] ppid=16231 vsize=364384 CPUtime=1164.27 cores=1,3,5,7
/proc/16233/stat : 16233 (pwbo2.0) S 16231 16233 15889 0 -1 4202496 69709 0 0 0 116350 77 0 0 20 0 3 0 274719612 373129216 71299 33554432000 4194304 5965102 140736949773648 140736949773168 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/16233/statm: 91096 71299 200 433 0 90660 0
[pid=16233/tid=16234] ppid=16231 vsize=364384 CPUtime=582.18 cores=1,3,5,7
/proc/16233/task/16234/stat : 16234 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 55689 0 0 0 58144 74 0 0 20 0 3 0 274719612 373129216 71299 33554432000 4194304 5965102 140736949773648 139858605504504 4401096 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=16233/tid=16235] ppid=16231 vsize=364384 CPUtime=582.28 cores=1,3,5,7
/proc/16233/task/16235/stat : 16235 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 13216 0 0 0 58208 20 0 0 20 0 3 0 274719612 373129216 71299 33554432000 4194304 5965102 140736949773648 139858595014968 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1164.27
Current children cumulated vsize (KiB) 364384

[startup+642.3 s]
/proc/loadavg: 4.00 4.00 3.82 5/183 16523
/proc/meminfo: memFree=29261300/32873844 swapFree=7372/7372
[pid=16233] ppid=16231 vsize=364924 CPUtime=1284.25 cores=1,3,5,7
/proc/16233/stat : 16233 (pwbo2.0) S 16231 16233 15889 0 -1 4202496 76473 0 0 0 128335 90 0 0 20 0 3 0 274719612 373682176 78243 33554432000 4194304 5965102 140736949773648 140736949773168 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/16233/statm: 91231 78243 200 433 0 90795 0
[pid=16233/tid=16234] ppid=16231 vsize=364924 CPUtime=642.18 cores=1,3,5,7
/proc/16233/task/16234/stat : 16234 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 62063 0 0 0 64130 88 0 0 20 0 3 0 274719612 373682176 78243 33554432000 4194304 5965102 140736949773648 139858605504312 4389864 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=16233/tid=16235] ppid=16231 vsize=364924 CPUtime=642.29 cores=1,3,5,7
/proc/16233/task/16235/stat : 16235 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 13606 0 0 0 64207 22 0 0 20 0 3 0 274719612 373682176 78243 33554432000 4194304 5965102 140736949773648 139858595014488 4401009 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1284.25
Current children cumulated vsize (KiB) 364924

[startup+702.3 s]
/proc/loadavg: 4.00 4.00 3.83 5/183 16523
/proc/meminfo: memFree=29246048/32873844 swapFree=7372/7372
[pid=16233] ppid=16231 vsize=365060 CPUtime=1404.21 cores=1,3,5,7
/proc/16233/stat : 16233 (pwbo2.0) S 16231 16233 15889 0 -1 4202496 80024 0 0 0 140325 96 0 0 20 0 3 0 274719612 373821440 81794 33554432000 4194304 5965102 140736949773648 140736949773168 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/16233/statm: 91265 81794 200 433 0 90829 0
[pid=16233/tid=16234] ppid=16231 vsize=365060 CPUtime=702.17 cores=1,3,5,7
/proc/16233/task/16234/stat : 16234 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 65055 0 0 0 70123 94 0 0 20 0 3 0 274719612 373821440 81794 33554432000 4194304 5965102 140736949773648 139858605504504 4221120 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=16233/tid=16235] ppid=16231 vsize=365060 CPUtime=702.28 cores=1,3,5,7
/proc/16233/task/16235/stat : 16235 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 14165 0 0 0 70204 24 0 0 20 0 3 0 274719612 373821440 81794 33554432000 4194304 5965102 140736949773648 139858595014776 4221120 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1404.21
Current children cumulated vsize (KiB) 365060

[startup+762.307 s]
/proc/loadavg: 4.00 4.00 3.83 5/182 16523
/proc/meminfo: memFree=29232320/32873844 swapFree=7372/7372
[pid=16233] ppid=16231 vsize=365060 CPUtime=1524.22 cores=1,3,5,7
/proc/16233/stat : 16233 (pwbo2.0) S 16231 16233 15889 0 -1 4202496 82052 0 0 0 152321 101 0 0 20 0 3 0 274719612 373821440 83822 33554432000 4194304 5965102 140736949773648 140736949773168 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/16233/statm: 91265 83822 200 433 0 90829 0
[pid=16233/tid=16234] ppid=16231 vsize=365060 CPUtime=762.18 cores=1,3,5,7
/proc/16233/task/16234/stat : 16234 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 66381 0 0 0 76120 98 0 0 20 0 3 0 274719612 373821440 83822 33554432000 4194304 5965102 140736949773648 139858605504312 4391908 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=16233/tid=16235] ppid=16231 vsize=365060 CPUtime=762.29 cores=1,3,5,7
/proc/16233/task/16235/stat : 16235 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 14867 0 0 0 76203 26 0 0 20 0 3 0 274719612 373821440 83822 33554432000 4194304 5965102 140736949773648 139858595014776 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1524.22
Current children cumulated vsize (KiB) 365060

[startup+822.3 s]
/proc/loadavg: 4.00 4.00 3.84 5/183 16524
/proc/meminfo: memFree=29174928/32873844 swapFree=7372/7372
[pid=16233] ppid=16231 vsize=430596 CPUtime=1644.18 cores=1,3,5,7
/proc/16233/stat : 16233 (pwbo2.0) S 16231 16233 15889 0 -1 4202496 88198 0 0 0 164306 112 0 0 20 0 3 0 274719612 440930304 89968 33554432000 4194304 5965102 140736949773648 140736949773168 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/16233/statm: 107649 89968 200 433 0 107213 0
[pid=16233/tid=16234] ppid=16231 vsize=430596 CPUtime=822.18 cores=1,3,5,7
/proc/16233/task/16234/stat : 16234 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 72490 0 0 0 82108 110 0 0 20 0 3 0 274719612 440930304 89968 33554432000 4194304 5965102 140736949773648 139858605504552 4400960 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=16233/tid=16235] ppid=16231 vsize=430596 CPUtime=822.29 cores=1,3,5,7
/proc/16233/task/16235/stat : 16235 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 14904 0 0 0 82202 27 0 0 20 0 3 0 274719612 440930304 89968 33554432000 4194304 5965102 140736949773648 139858595014776 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1644.18
Current children cumulated vsize (KiB) 430596

[startup+882.3 s]
/proc/loadavg: 4.06 4.01 3.85 5/183 16524
/proc/meminfo: memFree=29068700/32873844 swapFree=7372/7372
[pid=16233] ppid=16231 vsize=496132 CPUtime=1764.17 cores=1,3,5,7
/proc/16233/stat : 16233 (pwbo2.0) S 16231 16233 15889 0 -1 4202496 92375 0 0 0 176299 118 0 0 20 0 3 0 274719612 508039168 94145 33554432000 4194304 5965102 140736949773648 140736949773168 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/16233/statm: 124033 94145 200 433 0 123597 0
[pid=16233/tid=16234] ppid=16231 vsize=496132 CPUtime=882.18 cores=1,3,5,7
/proc/16233/task/16234/stat : 16234 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 74896 0 0 0 88103 115 0 0 20 0 3 0 274719612 508039168 94145 33554432000 4194304 5965102 140736949773648 139858605504504 4400960 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=16233/tid=16235] ppid=16231 vsize=496132 CPUtime=882.29 cores=1,3,5,7
/proc/16233/task/16235/stat : 16235 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 16675 0 0 0 88199 30 0 0 20 0 3 0 274719612 508039168 94145 33554432000 4194304 5965102 140736949773648 139858595014776 4404410 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1764.17
Current children cumulated vsize (KiB) 496132



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+900.3 s]
/proc/loadavg: 4.04 4.01 3.85 5/183 16524
/proc/meminfo: memFree=29057044/32873844 swapFree=7372/7372
[pid=16233] ppid=16231 vsize=496132 CPUtime=1800.18 cores=1,3,5,7
/proc/16233/stat : 16233 (pwbo2.0) S 16231 16233 15889 0 -1 4202496 94596 0 0 0 179897 121 0 0 20 0 3 0 274719612 508039168 96365 33554432000 4194304 5965102 140736949773648 140736949773168 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/16233/statm: 124033 96365 200 433 0 123597 0
[pid=16233/tid=16234] ppid=16231 vsize=496132 CPUtime=900.19 cores=1,3,5,7
/proc/16233/task/16234/stat : 16234 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 76902 0 0 0 89901 118 0 0 20 0 3 0 274719612 508039168 96365 33554432000 4194304 5965102 140736949773648 139858605504312 4400960 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=16233/tid=16235] ppid=16231 vsize=496132 CPUtime=900.3 cores=1,3,5,7
/proc/16233/task/16235/stat : 16235 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 16890 0 0 0 90000 30 0 0 20 0 3 0 274719612 508039168 96365 33554432000 4194304 5965102 140736949773648 139858595014776 4404410 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1800.18
Current children cumulated vsize (KiB) 496132

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+900.3 s]
/proc/loadavg: 4.04 4.01 3.85 5/183 16524
/proc/meminfo: memFree=29057044/32873844 swapFree=7372/7372
[pid=16233] ppid=16231 vsize=496132 CPUtime=1800.18 cores=1,3,5,7
/proc/16233/stat : 16233 (pwbo2.0) S 16231 16233 15889 0 -1 4202496 94596 0 0 0 179897 121 0 0 20 0 3 0 274719612 508039168 96365 33554432000 4194304 5965102 140736949773648 140736949773168 4547341 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/16233/statm: 124033 96365 200 433 0 123597 0
[pid=16233/tid=16234] ppid=16231 vsize=496132 CPUtime=900.19 cores=1,3,5,7
/proc/16233/task/16234/stat : 16234 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 76902 0 0 0 89901 118 0 0 20 0 3 0 274719612 508039168 96365 33554432000 4194304 5965102 140736949773648 139858605504312 4400960 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=16233/tid=16235] ppid=16231 vsize=496132 CPUtime=900.3 cores=1,3,5,7
/proc/16233/task/16235/stat : 16235 (pwbo2.0) R 16231 16233 15889 0 -1 4202560 16890 0 0 0 90000 30 0 0 20 0 3 0 274719612 508039168 96365 33554432000 4194304 5965102 140736949773648 139858595014776 4404410 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1800.18
Current children cumulated vsize (KiB) 496132

Child status: 0
Real time (s): 900.324
CPU time (s): 1800.53
CPU user time (s): 1799.02
CPU system time (s): 1.51277
CPU usage (%): 199.987
Max. virtual memory (cumulated for all children) (KiB): 496132

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1799.02
system time used= 1.51277
maximum resident set size= 385472
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 94599
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 160
involuntary context switches= 2363

runsolver used 2.00969 second user time and 5.55616 second system time

The end

Launcher Data 

Begin job on node133 at 2012-05-27 04:32:28
IDJOB=3703463
IDBENCH=1858
IDSOLVER=2279
FILE ID=node133/3703463-1338085948
RUNJOBID= node133-1338084137-15906
PBS_JOBID= 14624424
Free space on /tmp= 71724 MiB

SOLVER NAME= pwbo 2.0
BENCH NAME= PB06//final/normalized-PB06/OPT-SMALLINT/mps-v2-20-10/MIPLIB/miplib/normalized-mps-v2-20-10-p0201.opb
COMMAND LINE= pwbo2.0 -file-format=opb -verbosity=0 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3703463-1338085948/watcher-3703463-1338085948 -o /tmp/evaluation-result-3703463-1338085948/solver-3703463-1338085948 -C 1800 -W 1900 -M 15500  pwbo2.0 -file-format=opb -verbosity=0 HOME/instance-3703463-1338085948.opb

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 303eaeab4b7ccd67c7fab31d13d20bc2
RANDOM SEED=1054609266

node133.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.540
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.540
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.540
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.540
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.540
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.540
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.540
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.540
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        28777864 kB
Buffers:          315176 kB
Cached:          1664000 kB
SwapCached:          104 kB
Active:          1835208 kB
Inactive:        1433288 kB
Active(anon):    1280396 kB
Inactive(anon):    11368 kB
Active(file):     554812 kB
Inactive(file):  1421920 kB
Unevictable:        7372 kB
Mlocked:            7372 kB
SwapTotal:      67108856 kB
SwapFree:       67108752 kB
Dirty:              5804 kB
Writeback:             0 kB
AnonPages:       1296404 kB
Mapped:            15184 kB
Shmem:               212 kB
Slab:             684892 kB
SReclaimable:     126576 kB
SUnreclaim:       558316 kB
KernelStack:        1520 kB
PageTables:         6616 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     181492 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346376 kB
VmallocChunk:   34341987860 kB
HardwareCorrupted:     0 kB
AnonHugePages:    915456 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 71716 MiB
End job on node133 at 2012-05-27 04:47:31