Trace number 3703761

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
pwbo 2.0SAT (TO)17 1800.17 900.315

General information on the benchmark

Namenormalized-PB06/OPT-SMALLINT/submitted-PB06/
manquiho/market-split/normalized-opt-market-split_7_60_3.opb
MD5SUM474916db0f2f0be5831d97827cee8e55
Bench CategoryOPT-SMALLINT (optimisation, small integers)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark8
Best CPU time to get the best result obtained on this benchmark1790.84
Has Objective FunctionYES
SatisfiableYES
(Un)Satisfiability was provedYES
Best value of the objective function 16
Optimality of the best value was proved NO
Number of variables172
Total number of constraints14
Number of constraints which are clauses0
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints14
Minimum length of a constraint75
Maximum length of a constraint76
Number of terms in the objective function 112
Biggest coefficient in the objective function 128
Number of bits for the biggest coefficient in the objective function 8
Sum of the numbers in the objective function 3570
Number of bits of the sum of numbers in the objective function 12
Biggest number in a constraint 1901
Number of bits of the biggest number in a constraint 11
Biggest sum of numbers in a constraint 6214
Number of bits of the biggest sum of numbers13
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Quality of the solution as a function of time


Download the above graph as a PDF file / as an EPS file

Solver Data 

0.00/0.00	c ==============================[ Solver Version ]===============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  pwbo:                    2.0 beta - March, 2012                            |
0.00/0.00	c |                                                                             |
0.00/0.00	c Parsing opb file format. File HOME/instance-3703761-1338094701.opb.
0.00/0.00	c Instance file HOME/instance-3703761-1338094701.opb
0.00/0.00	c File size is 10386 bytes.
0.00/0.00	c Cost Offset: 0
0.00/0.00	c Highest Coefficient sum: 6214
0.00/0.00	c ============================[ Problem Statistics ]=============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  Parsing time:          0.00         s                                      |
0.00/0.00	c |  Number Variables:      172                                                 |
0.00/0.00	c |  Number Clauses:        0                                                   |
0.00/0.00	c |  Number Cardinality:    0                                                   |
0.00/0.00	c |  Number PB Constraints: 14                                                  |
0.00/0.00	c |                                                                             |
0.00/0.00	c |  Problem Type:          Optimization                                        |
0.00/0.00	c |  Objective Function:    PB Constraint                                       |
0.00/0.00	c |                                                                             |
0.00/0.00	c ===============================================================================
1800.17/900.31	o 907
1800.17/900.31	o 785
1800.17/900.31	o 645
1800.17/900.31	o 643
1800.17/900.31	o 523
1800.17/900.31	o 519
1800.17/900.31	o 511
1800.17/900.31	o 428
1800.17/900.31	o 389
1800.17/900.31	o 281
1800.17/900.31	o 278
1800.17/900.31	o 228
1800.17/900.31	o 227
1800.17/900.31	o 176
1800.17/900.31	o 120
1800.17/900.31	o 118
1800.17/900.31	o 66
1800.17/900.31	o 64
1800.17/900.31	o 62
1800.17/900.31	o 60
1800.17/900.31	o 56
1800.17/900.31	o 50
1800.17/900.31	o 46
1800.17/900.31	o 38
1800.17/900.31	o 37
1800.17/900.31	o 30
1800.17/900.31	o 28
1800.17/900.31	o 25
1800.17/900.31	o 20
1800.17/900.31	o 17
1800.17/900.31	c ===============================================================================
1800.17/900.31	c Wall Clock Time: 900.31 s
1800.17/900.31	c Best Solution: 17
1800.17/900.31	s SATISFIABLE
1800.17/900.31	v -x1 -x2 -x3 -x4 -x5 -x6 -x7 -x8 x9 -x10 -x11 -x12 -x13 -x14 -x15 -x16 x17 -x18 -x19 -x20 -x21 -x22 -x23 -x24 -x25 -x26 -x27 -x28 -x29 -x30 -x31 -x32 -x33 -x34 -x35 -x36 -x37 -x38 -x39 -x40 -x41 x42 x43 -x44 -x45 -x46 -x47 -x48 -x49 -x50 -x51 -x52 -x53 -x54 -x55 -x56 -x57 -x58 -x59 -x60 -x61 -x62 -x63 -x64 -x65 -x66 -x67 -x68 -x69 -x70 -x71 -x72 x73 x74 -x75 -x76 -x77 -x78 -x79 -x80 -x81 x82 -x83 -x84 -x85 -x86 -x87 -x88 -x89 -x90 -x91 -x92 -x93 -x94 -x95 -x96 -x97 -x98 -x99 -x100 -x101 -x102 -x103 -x104 -x105 -x106 x107 -x108 -x109 -x110 -x111 -x112 x113 -x114 x115 -x116 -x117 -x118 x119 x120 -x121 -x122 -x123 x124 x125 -x126 x127 -x128 x129 x130 x131 x132 -x133 x134 x135 -x136 -x137 x138 x139 -x140 x141 -x142 x143 x144 x145 x146 x147 x148 -x149 -x150 x151 x152 x153 x154 -x155 -x156 x157 -x158 -x159 x160 -x161 -x162 x163 -x164 -x165 -x166 -x167 -x168 -x169 x170 -x171 -x172

Verifier Data 

OK	17

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3703761-1338094701/watcher-3703761-1338094701 -o /tmp/evaluation-result-3703761-1338094701/solver-3703761-1338094701 -C 1800 -W 1900 -M 15500 pwbo2.0 -file-format=opb -verbosity=0 HOME/instance-3703761-1338094701.opb 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.00 4.00 3.99 4/178 25904
/proc/meminfo: memFree=30217696/32873648 swapFree=7108/7108
[pid=25904] ppid=25902 vsize=162464 CPUtime=0 cores=1,3,5,7
/proc/25904/stat : 25904 (pwbo2.0) S 25902 25904 25163 0 -1 4202496 866 0 0 0 0 0 0 0 20 0 3 0 275595540 166363136 816 33554432000 4194304 5965102 140734066120816 140734066120336 4547341 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/25904/statm: 40616 816 196 433 0 40180 0
[pid=25904/tid=25905] ppid=25902 vsize=162464 CPUtime=0 cores=1,3,5,7
/proc/25904/task/25905/stat : 25905 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 44 0 0 0 0 0 0 0 20 0 3 0 275595541 166363136 817 33554432000 4194304 5965102 140734066120816 140653956054064 5137076 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=25904/tid=25906] ppid=25902 vsize=162464 CPUtime=0 cores=1,3,5,7
/proc/25904/task/25906/stat : 25906 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 48 0 0 0 0 0 0 0 20 0 3 0 275595541 166363136 817 33554432000 4194304 5965102 140734066120816 140653945564120 4392734 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0

[startup+0.071052 s]
/proc/loadavg: 4.00 4.00 3.99 4/178 25904
/proc/meminfo: memFree=30217696/32873648 swapFree=7108/7108
[pid=25904] ppid=25902 vsize=162464 CPUtime=0.13 cores=1,3,5,7
/proc/25904/stat : 25904 (pwbo2.0) S 25902 25904 25163 0 -1 4202496 1166 0 0 0 13 0 0 0 20 0 3 0 275595540 166363136 1116 33554432000 4194304 5965102 140734066120816 140734066120336 4547341 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/25904/statm: 40616 1116 199 433 0 40180 0
[pid=25904/tid=25905] ppid=25902 vsize=162464 CPUtime=0.05 cores=1,3,5,7
/proc/25904/task/25905/stat : 25905 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 204 0 0 0 5 0 0 0 20 0 3 0 275595541 166363136 1116 33554432000 4194304 5965102 140734066120816 140653956054064 5137076 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=25904/tid=25906] ppid=25902 vsize=162464 CPUtime=0.05 cores=1,3,5,7
/proc/25904/task/25906/stat : 25906 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 187 0 0 0 5 0 0 0 20 0 3 0 275595541 166363136 1116 33554432000 4194304 5965102 140734066120816 140653945564120 4404753 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 0.13
Current children cumulated vsize (KiB) 162464

[startup+0.100221 s]
/proc/loadavg: 4.00 4.00 3.99 4/178 25904
/proc/meminfo: memFree=30217696/32873648 swapFree=7108/7108
[pid=25904] ppid=25902 vsize=162464 CPUtime=0.19 cores=1,3,5,7
/proc/25904/stat : 25904 (pwbo2.0) S 25902 25904 25163 0 -1 4202496 1283 0 0 0 19 0 0 0 20 0 3 0 275595540 166363136 1233 33554432000 4194304 5965102 140734066120816 140734066120336 4547341 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/25904/statm: 40616 1233 199 433 0 40180 0
[pid=25904/tid=25905] ppid=25902 vsize=162464 CPUtime=0.08 cores=1,3,5,7
/proc/25904/task/25905/stat : 25905 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 272 0 0 0 8 0 0 0 20 0 3 0 275595541 166363136 1233 33554432000 4194304 5965102 140734066120816 140653956054064 5137076 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=25904/tid=25906] ppid=25902 vsize=162464 CPUtime=0.08 cores=1,3,5,7
/proc/25904/task/25906/stat : 25906 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 236 0 0 0 8 0 0 0 20 0 3 0 275595541 166363136 1233 33554432000 4194304 5965102 140734066120816 140653945563256 4404717 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 162464

[startup+0.300159 s]
/proc/loadavg: 4.00 4.00 3.99 4/178 25904
/proc/meminfo: memFree=30217696/32873648 swapFree=7108/7108
[pid=25904] ppid=25902 vsize=162464 CPUtime=0.58 cores=1,3,5,7
/proc/25904/stat : 25904 (pwbo2.0) S 25902 25904 25163 0 -1 4202496 1933 0 0 0 58 0 0 0 20 0 3 0 275595540 166363136 1883 33554432000 4194304 5965102 140734066120816 140734066120336 4547341 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/25904/statm: 40616 1883 199 433 0 40180 0
[pid=25904/tid=25905] ppid=25902 vsize=162464 CPUtime=0.28 cores=1,3,5,7
/proc/25904/task/25905/stat : 25905 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 659 0 0 0 28 0 0 0 20 0 3 0 275595541 166363136 1883 33554432000 4194304 5965102 140734066120816 140653956053032 5137076 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=25904/tid=25906] ppid=25902 vsize=162464 CPUtime=0.28 cores=1,3,5,7
/proc/25904/task/25906/stat : 25906 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 499 0 0 0 28 0 0 0 20 0 3 0 275595541 166363136 1883 33554432000 4194304 5965102 140734066120816 140653945563384 4391169 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 0.58
Current children cumulated vsize (KiB) 162464

[startup+0.700153 s]
/proc/loadavg: 4.00 4.00 3.99 4/178 25904
/proc/meminfo: memFree=30217696/32873648 swapFree=7108/7108
[pid=25904] ppid=25902 vsize=162464 CPUtime=1.38 cores=1,3,5,7
/proc/25904/stat : 25904 (pwbo2.0) S 25902 25904 25163 0 -1 4202496 2801 0 0 0 138 0 0 0 20 0 3 0 275595540 166363136 2751 33554432000 4194304 5965102 140734066120816 140734066120336 4547341 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/25904/statm: 40616 2751 199 433 0 40180 0
[pid=25904/tid=25905] ppid=25902 vsize=162464 CPUtime=0.68 cores=1,3,5,7
/proc/25904/task/25905/stat : 25905 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 1299 0 0 0 68 0 0 0 20 0 3 0 275595541 166363136 2751 33554432000 4194304 5965102 140734066120816 140653956053032 4390219 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=25904/tid=25906] ppid=25902 vsize=162464 CPUtime=0.68 cores=1,3,5,7
/proc/25904/task/25906/stat : 25906 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 727 0 0 0 68 0 0 0 20 0 3 0 275595541 166363136 2751 33554432000 4194304 5965102 140734066120816 140653945562968 4222416 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1.38
Current children cumulated vsize (KiB) 162464

[startup+1.50016 s]
/proc/loadavg: 4.00 4.00 3.99 5/181 25907
/proc/meminfo: memFree=30205024/32873648 swapFree=7108/7108
[pid=25904] ppid=25902 vsize=162752 CPUtime=2.98 cores=1,3,5,7
/proc/25904/stat : 25904 (pwbo2.0) S 25902 25904 25163 0 -1 4202496 3608 0 0 0 297 1 0 0 20 0 3 0 275595540 166658048 3558 33554432000 4194304 5965102 140734066120816 140734066120336 4547341 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/25904/statm: 40688 3558 199 433 0 40252 0
[pid=25904/tid=25905] ppid=25902 vsize=162752 CPUtime=1.48 cores=1,3,5,7
/proc/25904/task/25905/stat : 25905 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 1850 0 0 0 147 1 0 0 20 0 3 0 275595541 166658048 3558 33554432000 4194304 5965102 140734066120816 140653956052792 4382742 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=25904/tid=25906] ppid=25902 vsize=162752 CPUtime=1.48 cores=1,3,5,7
/proc/25904/task/25906/stat : 25906 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 983 0 0 0 148 0 0 0 20 0 3 0 275595541 166658048 3558 33554432000 4194304 5965102 140734066120816 140653945563256 4390112 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 2.98
Current children cumulated vsize (KiB) 162752

[startup+3.10026 s]
/proc/loadavg: 4.00 4.00 3.99 5/181 25907
/proc/meminfo: memFree=30201800/32873648 swapFree=7108/7108
[pid=25904] ppid=25902 vsize=162824 CPUtime=6.19 cores=1,3,5,7
/proc/25904/stat : 25904 (pwbo2.0) S 25902 25904 25163 0 -1 4202496 4607 0 0 0 617 2 0 0 20 0 3 0 275595540 166731776 4557 33554432000 4194304 5965102 140734066120816 140734066120336 4547341 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/25904/statm: 40706 4557 199 433 0 40270 0
[pid=25904/tid=25905] ppid=25902 vsize=162824 CPUtime=3.07 cores=1,3,5,7
/proc/25904/task/25905/stat : 25905 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 2670 0 0 0 306 1 0 0 20 0 3 0 275595541 166731776 4557 33554432000 4194304 5965102 140734066120816 140653956052984 4401118 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=25904/tid=25906] ppid=25902 vsize=162824 CPUtime=3.07 cores=1,3,5,7
/proc/25904/task/25906/stat : 25906 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 1162 0 0 0 307 0 0 0 20 0 3 0 275595541 166731776 4557 33554432000 4194304 5965102 140734066120816 140653945563448 4401582 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 6.19
Current children cumulated vsize (KiB) 162824

[startup+6.30028 s]
/proc/loadavg: 4.00 4.00 3.99 5/181 25907
/proc/meminfo: memFree=30188208/32873648 swapFree=7108/7108
[pid=25904] ppid=25902 vsize=162896 CPUtime=12.58 cores=1,3,5,7
/proc/25904/stat : 25904 (pwbo2.0) S 25902 25904 25163 0 -1 4202496 6886 0 0 0 1255 3 0 0 20 0 3 0 275595540 166805504 6836 33554432000 4194304 5965102 140734066120816 140734066120336 4547341 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/25904/statm: 40724 6836 199 433 0 40288 0
[pid=25904/tid=25905] ppid=25902 vsize=162896 CPUtime=6.26 cores=1,3,5,7
/proc/25904/task/25905/stat : 25905 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 4163 0 0 0 623 3 0 0 20 0 3 0 275595541 166805504 6836 33554432000 4194304 5965102 140734066120816 140653956052984 5137146 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=25904/tid=25906] ppid=25902 vsize=162896 CPUtime=6.26 cores=1,3,5,7
/proc/25904/task/25906/stat : 25906 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 1948 0 0 0 626 0 0 0 20 0 3 0 275595541 166805504 6836 33554432000 4194304 5965102 140734066120816 140653945563496 4390144 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 12.58
Current children cumulated vsize (KiB) 162896

[startup+12.7003 s]
/proc/loadavg: 4.07 4.02 4.00 5/181 25907
/proc/meminfo: memFree=30168756/32873648 swapFree=7108/7108
[pid=25904] ppid=25902 vsize=163164 CPUtime=25.38 cores=1,3,5,7
/proc/25904/stat : 25904 (pwbo2.0) S 25902 25904 25163 0 -1 4202496 9972 0 0 0 2533 5 0 0 20 0 3 0 275595540 167079936 9922 33554432000 4194304 5965102 140734066120816 140734066120336 4547341 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/25904/statm: 40791 9922 199 433 0 40355 0
[pid=25904/tid=25905] ppid=25902 vsize=163164 CPUtime=12.65 cores=1,3,5,7
/proc/25904/task/25905/stat : 25905 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 6946 0 0 0 1261 4 0 0 20 0 3 0 275595541 167079936 9922 33554432000 4194304 5965102 140734066120816 140653956053032 4394002 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=25904/tid=25906] ppid=25902 vsize=163164 CPUtime=12.65 cores=1,3,5,7
/proc/25904/task/25906/stat : 25906 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 2251 0 0 0 1264 1 0 0 20 0 3 0 275595541 167079936 9922 33554432000 4194304 5965102 140734066120816 140653945563448 4221120 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 25.38
Current children cumulated vsize (KiB) 163164

[startup+25.5003 s]
/proc/loadavg: 4.06 4.02 4.00 5/181 25907
/proc/meminfo: memFree=30146216/32873648 swapFree=7108/7108
[pid=25904] ppid=25902 vsize=163432 CPUtime=50.98 cores=1,3,5,7
/proc/25904/stat : 25904 (pwbo2.0) S 25902 25904 25163 0 -1 4202496 14614 0 0 0 5089 9 0 0 20 0 3 0 275595540 167354368 14564 33554432000 4194304 5965102 140734066120816 140734066120336 4547341 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/25904/statm: 40858 14564 199 433 0 40422 0
[pid=25904/tid=25905] ppid=25902 vsize=163432 CPUtime=25.41 cores=1,3,5,7
/proc/25904/task/25905/stat : 25905 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 9776 0 0 0 2535 6 0 0 20 0 3 0 275595541 167354368 14564 33554432000 4194304 5965102 140734066120816 140653956052984 5132431 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=25904/tid=25906] ppid=25902 vsize=163432 CPUtime=25.42 cores=1,3,5,7
/proc/25904/task/25906/stat : 25906 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 4063 0 0 0 2540 2 0 0 20 0 3 0 275595541 167354368 14564 33554432000 4194304 5965102 140734066120816 140653945562968 4401058 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 50.98
Current children cumulated vsize (KiB) 163432

[startup+51.1004 s]
/proc/loadavg: 4.04 4.01 4.00 5/181 25907
/proc/meminfo: memFree=30109604/32873648 swapFree=7108/7108
[pid=25904] ppid=25902 vsize=163672 CPUtime=102.17 cores=1,3,5,7
/proc/25904/stat : 25904 (pwbo2.0) S 25902 25904 25163 0 -1 4202496 20666 0 0 0 10201 16 0 0 20 0 3 0 275595540 167600128 20616 33554432000 4194304 5965102 140734066120816 140734066120336 4547341 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/25904/statm: 40918 20616 199 433 0 40482 0
[pid=25904/tid=25905] ppid=25902 vsize=163672 CPUtime=50.95 cores=1,3,5,7
/proc/25904/task/25905/stat : 25905 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 14607 0 0 0 5082 13 0 0 20 0 3 0 275595541 167600128 20616 33554432000 4194304 5965102 140734066120816 140653956052984 4400960 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=25904/tid=25906] ppid=25902 vsize=163672 CPUtime=50.95 cores=1,3,5,7
/proc/25904/task/25906/stat : 25906 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 5284 0 0 0 5092 3 0 0 20 0 3 0 275595541 167600128 20616 33554432000 4194304 5965102 140734066120816 140653945563448 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 102.17
Current children cumulated vsize (KiB) 163672

[startup+102.307 s]
/proc/loadavg: 4.08 4.02 4.00 5/182 25910
/proc/meminfo: memFree=30049648/32873648 swapFree=7108/7108
[pid=25904] ppid=25902 vsize=230168 CPUtime=204.55 cores=1,3,5,7
/proc/25904/stat : 25904 (pwbo2.0) S 25902 25904 25163 0 -1 4202496 29442 0 0 0 20426 29 0 0 20 0 3 0 275595540 235692032 29352 33554432000 4194304 5965102 140734066120816 140734066120336 4547341 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/25904/statm: 57542 29352 199 433 0 57106 0
[pid=25904/tid=25905] ppid=25902 vsize=230168 CPUtime=102.01 cores=1,3,5,7
/proc/25904/task/25905/stat : 25905 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 20757 0 0 0 10178 23 0 0 20 0 3 0 275595541 235692032 29352 33554432000 4194304 5965102 140734066120816 140653956052984 4400960 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=25904/tid=25906] ppid=25902 vsize=230168 CPUtime=102.03 cores=1,3,5,7
/proc/25904/task/25906/stat : 25906 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 7910 0 0 0 10196 7 0 0 20 0 3 0 275595541 235692032 29352 33554432000 4194304 5965102 140734066120816 140653945563448 4401009 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 204.55
Current children cumulated vsize (KiB) 230168

[startup+162.3 s]
/proc/loadavg: 4.03 4.02 4.00 5/182 25933
/proc/meminfo: memFree=29998556/32873648 swapFree=7108/7108
[pid=25904] ppid=25902 vsize=231680 CPUtime=324.48 cores=1,3,5,7
/proc/25904/stat : 25904 (pwbo2.0) S 25902 25904 25163 0 -1 4202496 37373 0 0 0 32405 43 0 0 20 0 3 0 275595540 237240320 37321 33554432000 4194304 5965102 140734066120816 140734066120336 4547341 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/25904/statm: 57920 37321 199 433 0 57484 0
[pid=25904/tid=25905] ppid=25902 vsize=231680 CPUtime=161.82 cores=1,3,5,7
/proc/25904/task/25905/stat : 25905 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 27049 0 0 0 16147 35 0 0 20 0 3 0 275595541 237240320 37321 33554432000 4194304 5965102 140734066120816 140653956052984 4400960 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=25904/tid=25906] ppid=25902 vsize=231680 CPUtime=161.85 cores=1,3,5,7
/proc/25904/task/25906/stat : 25906 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 9549 0 0 0 16174 11 0 0 20 0 3 0 275595541 237240320 37321 33554432000 4194304 5965102 140734066120816 140653945563480 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 324.48
Current children cumulated vsize (KiB) 231680

[startup+222.3 s]
/proc/loadavg: 4.01 4.01 4.00 5/181 25933
/proc/meminfo: memFree=29953828/32873648 swapFree=7108/7108
[pid=25904] ppid=25902 vsize=233444 CPUtime=444.46 cores=1,3,5,7
/proc/25904/stat : 25904 (pwbo2.0) S 25902 25904 25163 0 -1 4202496 45194 0 0 0 44389 57 0 0 20 0 3 0 275595540 239046656 46431 33554432000 4194304 5965102 140734066120816 140734066120336 4547341 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/25904/statm: 58361 46431 199 433 0 57925 0
[pid=25904/tid=25905] ppid=25902 vsize=233444 CPUtime=221.67 cores=1,3,5,7
/proc/25904/task/25905/stat : 25905 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 33158 0 0 0 22120 47 0 0 20 0 3 0 275595541 239046656 46431 33554432000 4194304 5965102 140734066120816 140653956053032 4400960 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=25904/tid=25906] ppid=25902 vsize=233444 CPUtime=221.69 cores=1,3,5,7
/proc/25904/task/25906/stat : 25906 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 11261 0 0 0 22154 15 0 0 20 0 3 0 275595541 239046656 46431 33554432000 4194304 5965102 140734066120816 140653945563448 4380032 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 444.46
Current children cumulated vsize (KiB) 233444

################
# More data... #
################

[pid=25904/tid=25906] ppid=25902 vsize=301860 CPUtime=341.38 cores=1,3,5,7
/proc/25904/task/25906/stat : 25906 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 11273 0 0 0 34121 17 0 0 20 0 3 0 275595541 309104640 55427 33554432000 4194304 5965102 140734066120816 140653945563480 4221120 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 684.42
Current children cumulated vsize (KiB) 301860

[startup+402.3 s]
/proc/loadavg: 4.00 4.00 4.00 5/181 25934
/proc/meminfo: memFree=29822484/32873648 swapFree=7108/7108
[pid=25904] ppid=25902 vsize=302760 CPUtime=804.38 cores=1,3,5,7
/proc/25904/stat : 25904 (pwbo2.0) S 25902 25904 25163 0 -1 4202496 59433 0 0 0 80355 83 0 0 20 0 3 0 275595540 310026240 60551 33554432000 4194304 5965102 140734066120816 140734066120336 4547341 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/25904/statm: 75690 60551 199 433 0 75254 0
[pid=25904/tid=25905] ppid=25902 vsize=302760 CPUtime=401.18 cores=1,3,5,7
/proc/25904/task/25905/stat : 25905 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 47383 0 0 0 40042 76 0 0 20 0 3 0 275595541 310026240 60551 33554432000 4194304 5965102 140734066120816 140653956052984 4390392 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=25904/tid=25906] ppid=25902 vsize=302760 CPUtime=401.23 cores=1,3,5,7
/proc/25904/task/25906/stat : 25906 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 11275 0 0 0 40105 18 0 0 20 0 3 0 275595541 310026240 60551 33554432000 4194304 5965102 140734066120816 140653945563480 4400952 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 804.38
Current children cumulated vsize (KiB) 302760

[startup+462.3 s]
/proc/loadavg: 4.00 4.00 4.00 5/181 25934
/proc/meminfo: memFree=29802060/32873648 swapFree=7108/7108
[pid=25904] ppid=25902 vsize=303876 CPUtime=924.35 cores=1,3,5,7
/proc/25904/stat : 25904 (pwbo2.0) S 25902 25904 25163 0 -1 4202496 62364 0 0 0 92347 88 0 0 20 0 3 0 275595540 311169024 63475 33554432000 4194304 5965102 140734066120816 140734066120336 4547341 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/25904/statm: 75969 63475 199 433 0 75533 0
[pid=25904/tid=25905] ppid=25902 vsize=303876 CPUtime=461.02 cores=1,3,5,7
/proc/25904/task/25905/stat : 25905 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 49918 0 0 0 46021 81 0 0 20 0 3 0 275595541 311169024 63475 33554432000 4194304 5965102 140734066120816 140653956053032 4400960 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=25904/tid=25906] ppid=25902 vsize=303876 CPUtime=461.08 cores=1,3,5,7
/proc/25904/task/25906/stat : 25906 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 11671 0 0 0 46088 20 0 0 20 0 3 0 275595541 311169024 63475 33554432000 4194304 5965102 140734066120816 140653945563448 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 924.35
Current children cumulated vsize (KiB) 303876

[startup+522.3 s]
/proc/loadavg: 4.00 4.00 4.00 5/181 25934
/proc/meminfo: memFree=29757352/32873648 swapFree=7108/7108
[pid=25904] ppid=25902 vsize=370924 CPUtime=1044.31 cores=1,3,5,7
/proc/25904/stat : 25904 (pwbo2.0) S 25902 25904 25163 0 -1 4202496 69101 0 0 0 104331 100 0 0 20 0 3 0 275595540 379826176 70099 33554432000 4194304 5965102 140734066120816 140734066120336 4547341 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/25904/statm: 92731 70099 199 433 0 92295 0
[pid=25904/tid=25905] ppid=25902 vsize=370924 CPUtime=520.85 cores=1,3,5,7
/proc/25904/task/25905/stat : 25905 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 55991 0 0 0 51993 92 0 0 20 0 3 0 275595541 379826176 70099 33554432000 4194304 5965102 140734066120816 140653956052984 4400960 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=25904/tid=25906] ppid=25902 vsize=370924 CPUtime=520.92 cores=1,3,5,7
/proc/25904/task/25906/stat : 25906 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 12335 0 0 0 52070 22 0 0 20 0 3 0 275595541 379826176 70099 33554432000 4194304 5965102 140734066120816 140653945563480 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1044.31
Current children cumulated vsize (KiB) 370924

[startup+582.3 s]
/proc/loadavg: 3.66 3.92 3.97 4/179 26014
/proc/meminfo: memFree=30090992/32873648 swapFree=7108/7108
[pid=25904] ppid=25902 vsize=372632 CPUtime=1164.28 cores=1,3,5,7
/proc/25904/stat : 25904 (pwbo2.0) S 25902 25904 25163 0 -1 4202496 73812 0 0 0 116319 109 0 0 20 0 3 0 275595540 381575168 74815 33554432000 4194304 5965102 140734066120816 140734066120336 4547341 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/25904/statm: 93158 74815 199 433 0 92722 0
[pid=25904/tid=25905] ppid=25902 vsize=372632 CPUtime=580.68 cores=1,3,5,7
/proc/25904/task/25905/stat : 25905 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 60702 0 0 0 57966 102 0 0 20 0 3 0 275595541 381575168 74815 33554432000 4194304 5965102 140734066120816 140653956052984 4401050 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=25904/tid=25906] ppid=25902 vsize=372632 CPUtime=580.76 cores=1,3,5,7
/proc/25904/task/25906/stat : 25906 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 12335 0 0 0 58053 23 0 0 20 0 3 0 275595541 381575168 74815 33554432000 4194304 5965102 140734066120816 140653945563480 4400978 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1164.28
Current children cumulated vsize (KiB) 372632

[startup+642.3 s]
/proc/loadavg: 3.68 3.87 3.95 5/181 26096
/proc/meminfo: memFree=29961860/32873648 swapFree=7108/7108
[pid=25904] ppid=25902 vsize=373388 CPUtime=1284.27 cores=1,3,5,7
/proc/25904/stat : 25904 (pwbo2.0) S 25902 25904 25163 0 -1 4202496 76001 0 0 0 128314 113 0 0 20 0 3 0 275595540 382349312 76998 33554432000 4194304 5965102 140734066120816 140734066120336 4547341 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/25904/statm: 93347 76998 199 433 0 92911 0
[pid=25904/tid=25905] ppid=25902 vsize=373388 CPUtime=640.53 cores=1,3,5,7
/proc/25904/task/25905/stat : 25905 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 62891 0 0 0 63947 106 0 0 20 0 3 0 275595541 382349312 76998 33554432000 4194304 5965102 140734066120816 140653956052792 4400960 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=25904/tid=25906] ppid=25902 vsize=373388 CPUtime=640.61 cores=1,3,5,7
/proc/25904/task/25906/stat : 25906 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 12335 0 0 0 64037 24 0 0 20 0 3 0 275595541 382349312 76998 33554432000 4194304 5965102 140734066120816 140653945563480 4221120 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1284.27
Current children cumulated vsize (KiB) 373388

[startup+702.3 s]
/proc/loadavg: 3.75 3.86 3.94 5/182 26299
/proc/meminfo: memFree=30065796/32873648 swapFree=7108/7108
[pid=25904] ppid=25902 vsize=373964 CPUtime=1404.24 cores=1,3,5,7
/proc/25904/stat : 25904 (pwbo2.0) S 25902 25904 25163 0 -1 4202496 80464 0 0 0 140305 119 0 0 20 0 3 0 275595540 382939136 81461 33554432000 4194304 5965102 140734066120816 140734066120336 4547341 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/25904/statm: 93491 81461 199 433 0 93055 0
[pid=25904/tid=25905] ppid=25902 vsize=373964 CPUtime=700.36 cores=1,3,5,7
/proc/25904/task/25905/stat : 25905 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 67354 0 0 0 69922 114 0 0 20 0 3 0 275595541 382939136 81461 33554432000 4194304 5965102 140734066120816 140653956053032 5139822 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=25904/tid=25906] ppid=25902 vsize=373964 CPUtime=700.46 cores=1,3,5,7
/proc/25904/task/25906/stat : 25906 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 12335 0 0 0 70021 25 0 0 20 0 3 0 275595541 382939136 81461 33554432000 4194304 5965102 140734066120816 140653945563480 4401009 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1404.24
Current children cumulated vsize (KiB) 373964

[startup+762.307 s]
/proc/loadavg: 3.91 3.88 3.94 5/181 26419
/proc/meminfo: memFree=29890828/32873648 swapFree=7108/7108
[pid=25904] ppid=25902 vsize=439896 CPUtime=1524.23 cores=1,3,5,7
/proc/25904/stat : 25904 (pwbo2.0) S 25902 25904 25163 0 -1 4202496 82445 0 0 0 152301 122 0 0 20 0 3 0 275595540 450453504 83496 33554432000 4194304 5965102 140734066120816 140734066120336 4547341 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/25904/statm: 109974 83496 199 433 0 109538 0
[pid=25904/tid=25905] ppid=25902 vsize=439896 CPUtime=760.22 cores=1,3,5,7
/proc/25904/task/25905/stat : 25905 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 69335 0 0 0 75904 118 0 0 20 0 3 0 275595541 450453504 83496 33554432000 4194304 5965102 140734066120816 140653956053032 4400960 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=25904/tid=25906] ppid=25902 vsize=439896 CPUtime=760.31 cores=1,3,5,7
/proc/25904/task/25906/stat : 25906 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 12335 0 0 0 76005 26 0 0 20 0 3 0 275595541 450453504 83496 33554432000 4194304 5965102 140734066120816 140653945563480 4222474 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1524.23
Current children cumulated vsize (KiB) 439896

[startup+822.3 s]
/proc/loadavg: 3.96 3.90 3.94 5/181 26419
/proc/meminfo: memFree=29815596/32873648 swapFree=7108/7108
[pid=25904] ppid=25902 vsize=440508 CPUtime=1644.2 cores=1,3,5,7
/proc/25904/stat : 25904 (pwbo2.0) S 25902 25904 25163 0 -1 4202496 86631 0 0 0 164291 129 0 0 20 0 3 0 275595540 451080192 87682 33554432000 4194304 5965102 140734066120816 140734066120336 4547341 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/25904/statm: 110127 87682 199 433 0 109691 0
[pid=25904/tid=25905] ppid=25902 vsize=440508 CPUtime=820.06 cores=1,3,5,7
/proc/25904/task/25905/stat : 25905 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 73521 0 0 0 81880 126 0 0 20 0 3 0 275595541 451080192 87682 33554432000 4194304 5965102 140734066120816 140653956052984 4400960 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=25904/tid=25906] ppid=25902 vsize=440508 CPUtime=820.15 cores=1,3,5,7
/proc/25904/task/25906/stat : 25906 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 12335 0 0 0 81988 27 0 0 20 0 3 0 275595541 451080192 87682 33554432000 4194304 5965102 140734066120816 140653945563480 4404748 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1644.2
Current children cumulated vsize (KiB) 440508

[startup+882.3 s]
/proc/loadavg: 3.98 3.92 3.94 5/181 26420
/proc/meminfo: memFree=29717048/32873648 swapFree=7108/7108
[pid=25904] ppid=25902 vsize=441280 CPUtime=1764.18 cores=1,3,5,7
/proc/25904/stat : 25904 (pwbo2.0) S 25902 25904 25163 0 -1 4202496 89457 0 0 0 176284 134 0 0 20 0 3 0 275595540 451870720 90502 33554432000 4194304 5965102 140734066120816 140734066120336 4547341 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/25904/statm: 110320 90502 199 433 0 109884 0
[pid=25904/tid=25905] ppid=25902 vsize=441280 CPUtime=879.89 cores=1,3,5,7
/proc/25904/task/25905/stat : 25905 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 76133 0 0 0 87858 131 0 0 20 0 3 0 275595541 451870720 90502 33554432000 4194304 5965102 140734066120816 140653956053032 4400960 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=25904/tid=25906] ppid=25902 vsize=441280 CPUtime=879.99 cores=1,3,5,7
/proc/25904/task/25906/stat : 25906 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 12549 0 0 0 87971 28 0 0 20 0 3 0 275595541 451870720 90502 33554432000 4194304 5965102 140734066120816 140653945563496 4401441 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1764.18
Current children cumulated vsize (KiB) 441280



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+900.3 s]
/proc/loadavg: 3.99 3.92 3.94 5/181 26420
/proc/meminfo: memFree=29677472/32873648 swapFree=7108/7108
[pid=25904] ppid=25902 vsize=441856 CPUtime=1800.17 cores=1,3,5,7
/proc/25904/stat : 25904 (pwbo2.0) S 25902 25904 25163 0 -1 4202496 89913 0 0 0 179883 134 0 0 20 0 3 0 275595540 452460544 90946 33554432000 4194304 5965102 140734066120816 140734066120336 4547341 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/25904/statm: 110464 90946 199 433 0 110028 0
[pid=25904/tid=25905] ppid=25902 vsize=441856 CPUtime=897.84 cores=1,3,5,7
/proc/25904/task/25905/stat : 25905 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 76167 0 0 0 89653 131 0 0 20 0 3 0 275595541 452460544 90946 33554432000 4194304 5965102 140734066120816 140653956053032 4400960 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=25904/tid=25906] ppid=25902 vsize=441856 CPUtime=897.95 cores=1,3,5,7
/proc/25904/task/25906/stat : 25906 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 12971 0 0 0 89766 29 0 0 20 0 3 0 275595541 452460544 90946 33554432000 4194304 5965102 140734066120816 140653945563256 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1800.17
Current children cumulated vsize (KiB) 441856

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+900.3 s]
/proc/loadavg: 3.99 3.92 3.94 5/181 26420
/proc/meminfo: memFree=29677472/32873648 swapFree=7108/7108
[pid=25904] ppid=25902 vsize=441856 CPUtime=1800.17 cores=1,3,5,7
/proc/25904/stat : 25904 (pwbo2.0) S 25902 25904 25163 0 -1 4202496 89913 0 0 0 179883 134 0 0 20 0 3 0 275595540 452460544 90946 33554432000 4194304 5965102 140734066120816 140734066120336 4547341 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/25904/statm: 110464 90946 199 433 0 110028 0
[pid=25904/tid=25905] ppid=25902 vsize=441856 CPUtime=897.84 cores=1,3,5,7
/proc/25904/task/25905/stat : 25905 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 76167 0 0 0 89653 131 0 0 20 0 3 0 275595541 452460544 90946 33554432000 4194304 5965102 140734066120816 140653956053032 4400960 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=25904/tid=25906] ppid=25902 vsize=441856 CPUtime=897.95 cores=1,3,5,7
/proc/25904/task/25906/stat : 25906 (pwbo2.0) R 25902 25904 25163 0 -1 4202560 12971 0 0 0 89766 29 0 0 20 0 3 0 275595541 452460544 90946 33554432000 4194304 5965102 140734066120816 140653945563256 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1800.17
Current children cumulated vsize (KiB) 441856

Child status: 0

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 25904 and gives
#  childrusage.ru_utime.tv_sec=1794
#  childrusage.ru_utime.tv_usec=209238
#  childrusage.ru_stime.tv_sec=1
#  childrusage.ru_stime.tv_usec=619753
# CPU time returned by wait4() is 1795.83
# while last known CPU time is 1800.17
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 900.315
CPU time (s): 1800.17
CPU user time (s): 1798.83
CPU system time (s): 1.34
CPU usage (%): 199.949
Max. virtual memory (cumulated for all children) (KiB): 441856

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1794.21
system time used= 1.61975
maximum resident set size= 363796
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 89916
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 216
involuntary context switches= 2174

runsolver used 2.00169 second user time and 5.04423 second system time

The end

Launcher Data 

Begin job on node123 at 2012-05-27 06:58:21
IDJOB=3703761
IDBENCH=2932
IDSOLVER=2279
FILE ID=node123/3703761-1338094701
RUNJOBID= node123-1338088025-25180
PBS_JOBID= 14624452
Free space on /tmp= 71688 MiB

SOLVER NAME= pwbo 2.0
BENCH NAME= PB06/final/normalized-PB06/OPT-SMALLINT/submitted-PB06/manquiho/market-split/normalized-opt-market-split_7_60_3.opb
COMMAND LINE= pwbo2.0 -file-format=opb -verbosity=0 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3703761-1338094701/watcher-3703761-1338094701 -o /tmp/evaluation-result-3703761-1338094701/solver-3703761-1338094701 -C 1800 -W 1900 -M 15500  pwbo2.0 -file-format=opb -verbosity=0 HOME/instance-3703761-1338094701.opb

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 474916db0f2f0be5831d97827cee8e55
RANDOM SEED=1008549911

node123.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.43
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.217
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873648 kB
MemFree:        30218216 kB
Buffers:          274844 kB
Cached:          1358776 kB
SwapCached:         1808 kB
Active:           892192 kB
Inactive:         966972 kB
Active(anon):     216868 kB
Inactive(anon):    10968 kB
Active(file):     675324 kB
Inactive(file):   956004 kB
Unevictable:        7108 kB
Mlocked:            7108 kB
SwapTotal:      67108856 kB
SwapFree:       67104820 kB
Dirty:              5452 kB
Writeback:             0 kB
AnonPages:        212740 kB
Mapped:            15084 kB
Shmem:               100 kB
Slab:             657800 kB
SReclaimable:     102260 kB
SUnreclaim:       555540 kB
KernelStack:        1504 kB
PageTables:         4608 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545680 kB
Committed_AS:     179144 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346400 kB
VmallocChunk:   34341988052 kB
HardwareCorrupted:     0 kB
AnonHugePages:    112640 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7680 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 71680 MiB
End job on node123 at 2012-05-27 07:13:23