Trace number 3728513

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
pwbo 2.02MSAT (TO) 1800.05 900.219

General information on the benchmark

Name/PARTIAL-SMALLINT-LIN/PB06/submitted-PB05/aloul/FPGA_SAT05/
normalized-chnl10_11_pb.cnf.cr--soft-33-100-0.wbo
MD5SUM0b915fd051dbce2f3b4ec1d342cdefca
Bench CategoryPARTIAL-SMALLINT-LIN (both soft and hard constraints, small integers, linear constraints)
Best result obtained on this benchmarkMOPT
Best cost obtained on this benchmark3
Best CPU time to get the best result obtained on this benchmark0.116981
Max-Satisfiable
Max-(Un)Satisfiability was proved
Best value of the cost
Optimality of the best cost was proved
Number of variables220
Total number of constraints42
Number of soft constraints30
Number of constraints which are clauses22
Number of constraints which are cardinality constraints (but not clauses)20
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint10
Maximum length of a constraint11
Top cost 1595
Min constraint cost 1
Max constraint cost 100
Sum of constraints costs 1594
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 12
Number of bits of the biggest sum of numbers4
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Quality of the solution as a function of time


Download the above graph as a PDF file / as an EPS file

Solver Data 

0.00/0.00	c ==============================[ Solver Version ]===============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  pwbo:                     2.02 beta - May, 2012                            |
0.00/0.00	c |                                                                             |
0.00/0.00	c Parsing wbo file format. File HOME/instance-3728513-1338670389.wbo.
0.00/0.00	c Instance file HOME/instance-3728513-1338670389.wbo
0.00/0.00	c File size is 4029 bytes.
0.00/0.00	c Highest Coefficient sum: 12
0.00/0.00	c ============================[ Problem Statistics ]=============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  Parsing time:          0.00         s                                      |
0.00/0.00	c |  Number Variables:      250                                                 |
0.00/0.00	c |  Number Clauses:        22                                                  |
0.00/0.00	c |  Number Cardinality:    4                                                   |
0.00/0.00	c |  Number PB Constraints: 16                                                  |
0.00/0.00	c |                                                                             |
0.00/0.00	c |  Problem Type:          Optimization                                        |
0.00/0.00	c |  Objective Function:    Cardinality Constraint                              |
0.00/0.00	c |                                                                             |
0.00/0.00	c ===============================================================================
1800.05/900.21	o 96
1800.05/900.21	o 84
1800.05/900.21	o 55
1800.05/900.21	o 24
1800.05/900.21	o 19
1800.05/900.21	o 3
1800.05/900.21	c ===============================================================================
1800.05/900.21	c Wall Clock Time: 900.21 s
1800.05/900.21	c Best Solution: 3
1800.05/900.21	s SATISFIABLE
1800.05/900.21	v -x1 -x2 -x3 -x4 -x5 x6 -x7 -x8 -x9 -x10 -x11 -x12 -x13 -x14 -x15 -x16 -x17 -x18 x19 -x20 -x21 -x22 -x23 -x24 -x25 -x26 x27 -x28 -x29 -x30 x31 -x32 -x33 -x34 -x35 -x36 -x37 -x38 -x39 -x40 -x41 -x42 -x43 -x44 -x45 -x46 -x47 -x48 -x49 x50 -x51 x52 -x53 -x54 -x55 -x56 -x57 -x58 -x59 -x60 -x61 -x62 -x63 -x64 -x65 -x66 x67 -x68 -x69 -x70 -x71 -x72 -x73 -x74 -x75 -x76 -x77 x78 -x79 -x80 -x81 -x82 -x83 -x84 -x85 -x86 x87 -x88 -x89 -x90 -x91 -x92 -x93 -x94 -x95 -x96 x97 -x98 -x99 -x100 -x101 -x102 -x103 -x104 -x105 -x106 -x107 -x108 x109 -x110 -x111 -x112 -x113 x114 -x115 -x116 -x117 -x118 -x119 -x120 -x121 -x122 -x123 -x124 x125 -x126 -x127 -x128 -x129 -x130 -x131 -x132 -x133 x134 -x135 -x136 -x137 -x138 -x139 -x140 -x141 -x142 -x143 -x144 x145 -x146 -x147 -x148 -x149 -x150 -x151 -x152 -x153 -x154 -x155 -x156 x157 -x158 -x159 -x160 -x161 -x162 x163 -x164 -x165 -x166 -x167 -x168 -x169 -x170 -x171 -x172 -x173 -x174 x175 -x176 -x177 -x178 -x179 -x180 x181 -x182 -x183 -x184 -x185 -x186 -x187 -x188 -x189 -x190 -x191 -x192 -x193 -x194 -x195 -x196 -x197 -x198 -x199 x200 -x201 -x202 -x203 -x204 -x205 x206 -x207 -x208 -x209 -x210 -x211 -x212 -x213 -x214 x215 -x216 -x217 -x218 -x219 -x220

Verifier Data 

OK	3

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3728513-1338670389/watcher-3728513-1338670389 -o /tmp/evaluation-result-3728513-1338670389/solver-3728513-1338670389 -C 1800 -W 1900 -M 15500 pwbo2.02 -file-format=wbo -verbosity=0 HOME/instance-3728513-1338670389.wbo 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.68 3.83 3.19 4/178 4033
/proc/meminfo: memFree=28424332/32873844 swapFree=7632/7632
[pid=4033] ppid=4031 vsize=163272 CPUtime=0 cores=0,2,4,6
/proc/4033/stat : 4033 (pwbo2.02) S 4031 4033 3617 0 -1 4202496 1277 0 0 0 0 0 0 0 20 0 3 0 333163199 167190528 1215 33554432000 4194304 5964478 140736525163488 140736525163008 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4033/statm: 40818 1216 205 433 0 40382 0
[pid=4033/tid=4034] ppid=4031 vsize=163272 CPUtime=0 cores=0,2,4,6
/proc/4033/task/4034/stat : 4034 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 60 0 0 0 0 0 0 0 20 0 3 0 333163199 167190528 1220 33554432000 4194304 5964478 140736525163488 140702402713592 4400076 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4033/tid=4035] ppid=4031 vsize=163368 CPUtime=0 cores=0,2,4,6
/proc/4033/task/4035/stat : 4035 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 455 0 0 0 0 0 0 0 20 0 3 0 333163199 167288832 1220 33554432000 4194304 5964478 140736525163488 140702392223288 5244234 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0

[startup+0.0297031 s]
/proc/loadavg: 3.68 3.83 3.19 4/178 4033
/proc/meminfo: memFree=28424332/32873844 swapFree=7632/7632
[pid=4033] ppid=4031 vsize=169208 CPUtime=0.05 cores=0,2,4,6
/proc/4033/stat : 4033 (pwbo2.02) S 4031 4033 3617 0 -1 4202496 3730 0 0 0 4 1 0 0 20 0 3 0 333163199 173268992 3668 33554432000 4194304 5964478 140736525163488 140736525163008 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4033/statm: 42302 3675 206 433 0 41866 0
[pid=4033/tid=4034] ppid=4031 vsize=169208 CPUtime=0.01 cores=0,2,4,6
/proc/4033/task/4034/stat : 4034 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 134 0 0 0 1 0 0 0 20 0 3 0 333163199 173268992 3677 33554432000 4194304 5964478 140736525163488 140702402713592 4221120 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4033/tid=4035] ppid=4031 vsize=169208 CPUtime=0.01 cores=0,2,4,6
/proc/4033/task/4035/stat : 4035 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 2842 0 0 0 1 0 0 0 20 0 3 0 333163199 173268992 3681 33554432000 4194304 5964478 140736525163488 140702392223144 5136420 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 169208

[startup+0.100388 s]
/proc/loadavg: 3.68 3.83 3.19 4/178 4033
/proc/meminfo: memFree=28424332/32873844 swapFree=7632/7632
[pid=4033] ppid=4031 vsize=175724 CPUtime=0.17 cores=0,2,4,6
/proc/4033/stat : 4033 (pwbo2.02) S 4031 4033 3617 0 -1 4202496 6831 0 0 0 14 3 0 0 20 0 3 0 333163199 179941376 6769 33554432000 4194304 5964478 140736525163488 140736525163008 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4033/statm: 43931 6769 207 433 0 43495 0
[pid=4033/tid=4034] ppid=4031 vsize=175724 CPUtime=0.08 cores=0,2,4,6
/proc/4033/task/4034/stat : 4034 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 287 0 0 0 8 0 0 0 20 0 3 0 333163199 179941376 6770 33554432000 4194304 5964478 140736525163488 140702402713400 5136420 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4033/tid=4035] ppid=4031 vsize=175724 CPUtime=0.08 cores=0,2,4,6
/proc/4033/task/4035/stat : 4035 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 5778 0 0 0 5 3 0 0 20 0 3 0 333163199 179941376 6770 33554432000 4194304 5964478 140736525163488 140702392224104 4399938 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 0.17
Current children cumulated vsize (KiB) 175724

[startup+0.300324 s]
/proc/loadavg: 3.68 3.83 3.19 4/178 4033
/proc/meminfo: memFree=28424332/32873844 swapFree=7632/7632
[pid=4033] ppid=4031 vsize=175724 CPUtime=0.57 cores=0,2,4,6
/proc/4033/stat : 4033 (pwbo2.02) S 4031 4033 3617 0 -1 4202496 7242 0 0 0 53 4 0 0 20 0 3 0 333163199 179941376 7180 33554432000 4194304 5964478 140736525163488 140736525163008 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4033/statm: 43931 7180 207 433 0 43495 0
[pid=4033/tid=4034] ppid=4031 vsize=175724 CPUtime=0.28 cores=0,2,4,6
/proc/4033/task/4034/stat : 4034 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 473 0 0 0 27 1 0 0 20 0 3 0 333163199 179941376 7180 33554432000 4194304 5964478 140736525163488 140702402713400 4221120 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4033/tid=4035] ppid=4031 vsize=175724 CPUtime=0.28 cores=0,2,4,6
/proc/4033/task/4035/stat : 4035 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 6002 0 0 0 24 4 0 0 20 0 3 0 333163199 179941376 7180 33554432000 4194304 5964478 140736525163488 140702392223976 5139089 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 0.57
Current children cumulated vsize (KiB) 175724

[startup+0.700284 s]
/proc/loadavg: 3.68 3.83 3.19 4/178 4033
/proc/meminfo: memFree=28424332/32873844 swapFree=7632/7632
[pid=4033] ppid=4031 vsize=175724 CPUtime=1.37 cores=0,2,4,6
/proc/4033/stat : 4033 (pwbo2.02) S 4031 4033 3617 0 -1 4202496 7990 0 0 0 131 6 0 0 20 0 3 0 333163199 179941376 7928 33554432000 4194304 5964478 140736525163488 140736525163008 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4033/statm: 43931 7928 207 433 0 43495 0
[pid=4033/tid=4034] ppid=4031 vsize=175724 CPUtime=0.68 cores=0,2,4,6
/proc/4033/task/4034/stat : 4034 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 842 0 0 0 66 2 0 0 20 0 3 0 333163199 179941376 7928 33554432000 4194304 5964478 140736525163488 140702402713592 5179243 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4033/tid=4035] ppid=4031 vsize=175724 CPUtime=0.68 cores=0,2,4,6
/proc/4033/task/4035/stat : 4035 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 6381 0 0 0 62 6 0 0 20 0 3 0 333163199 179941376 7928 33554432000 4194304 5964478 140736525163488 140702392223864 4400091 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1.37
Current children cumulated vsize (KiB) 175724

[startup+1.50028 s]
/proc/loadavg: 3.68 3.83 3.19 5/181 4036
/proc/meminfo: memFree=28383464/32873844 swapFree=7632/7632
[pid=4033] ppid=4031 vsize=175724 CPUtime=2.98 cores=0,2,4,6
/proc/4033/stat : 4033 (pwbo2.02) S 4031 4033 3617 0 -1 4202496 8880 0 0 0 290 8 0 0 20 0 3 0 333163199 179941376 8818 33554432000 4194304 5964478 140736525163488 140736525163008 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4033/statm: 43931 8818 207 433 0 43495 0
[pid=4033/tid=4034] ppid=4031 vsize=175724 CPUtime=1.48 cores=0,2,4,6
/proc/4033/task/4034/stat : 4034 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 1309 0 0 0 146 2 0 0 20 0 3 0 333163199 179941376 8818 33554432000 4194304 5964478 140736525163488 140702402713400 4400460 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4033/tid=4035] ppid=4031 vsize=175724 CPUtime=1.48 cores=0,2,4,6
/proc/4033/task/4035/stat : 4035 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 6804 0 0 0 141 7 0 0 20 0 3 0 333163199 179941376 8818 33554432000 4194304 5964478 140736525163488 140702392223864 4400091 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 2.98
Current children cumulated vsize (KiB) 175724

[startup+3.10029 s]
/proc/loadavg: 3.68 3.83 3.19 5/181 4036
/proc/meminfo: memFree=28375032/32873844 swapFree=7632/7632
[pid=4033] ppid=4031 vsize=176012 CPUtime=6.17 cores=0,2,4,6
/proc/4033/stat : 4033 (pwbo2.02) S 4031 4033 3617 0 -1 4202496 9710 0 0 0 608 9 0 0 20 0 3 0 333163199 180236288 9648 33554432000 4194304 5964478 140736525163488 140736525163008 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4033/statm: 44003 9648 207 433 0 43567 0
[pid=4033/tid=4034] ppid=4031 vsize=176012 CPUtime=3.08 cores=0,2,4,6
/proc/4033/task/4034/stat : 4034 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 1649 0 0 0 305 3 0 0 20 0 3 0 333163199 180236288 9648 33554432000 4194304 5964478 140736525163488 140702402713400 4220672 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4033/tid=4035] ppid=4031 vsize=176012 CPUtime=3.08 cores=0,2,4,6
/proc/4033/task/4035/stat : 4035 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 7294 0 0 0 301 7 0 0 20 0 3 0 333163199 180236288 9648 33554432000 4194304 5964478 140736525163488 140702392224056 4400087 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 6.17
Current children cumulated vsize (KiB) 176012

[startup+6.30032 s]
/proc/loadavg: 3.70 3.83 3.20 5/181 4036
/proc/meminfo: memFree=28357920/32873844 swapFree=7632/7632
[pid=4033] ppid=4031 vsize=176400 CPUtime=12.57 cores=0,2,4,6
/proc/4033/stat : 4033 (pwbo2.02) S 4031 4033 3617 0 -1 4202496 10775 0 0 0 1248 9 0 0 20 0 3 0 333163199 180633600 10713 33554432000 4194304 5964478 140736525163488 140736525163008 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4033/statm: 44100 10713 207 433 0 43664 0
[pid=4033/tid=4034] ppid=4031 vsize=176400 CPUtime=6.28 cores=0,2,4,6
/proc/4033/task/4034/stat : 4034 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 2129 0 0 0 625 3 0 0 20 0 3 0 333163199 180633600 10713 33554432000 4194304 5964478 140736525163488 140702402713400 4220672 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4033/tid=4035] ppid=4031 vsize=176400 CPUtime=6.28 cores=0,2,4,6
/proc/4033/task/4035/stat : 4035 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 7879 0 0 0 621 7 0 0 20 0 3 0 333163199 180633600 10713 33554432000 4194304 5964478 140736525163488 140702392224056 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 12.57
Current children cumulated vsize (KiB) 176400

[startup+12.7003 s]
/proc/loadavg: 3.73 3.83 3.20 5/181 4036
/proc/meminfo: memFree=28342236/32873844 swapFree=7632/7632
[pid=4033] ppid=4031 vsize=177936 CPUtime=25.37 cores=0,2,4,6
/proc/4033/stat : 4033 (pwbo2.02) S 4031 4033 3617 0 -1 4202496 12925 0 0 0 2526 11 0 0 20 0 3 0 333163199 182206464 12851 33554432000 4194304 5964478 140736525163488 140736525163008 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4033/statm: 44484 12851 207 433 0 44048 0
[pid=4033/tid=4034] ppid=4031 vsize=177936 CPUtime=12.65 cores=0,2,4,6
/proc/4033/task/4034/stat : 4034 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 2715 0 0 0 1262 3 0 0 20 0 3 0 333163199 182206464 12851 33554432000 4194304 5964478 140736525163488 140702402713400 4400097 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4033/tid=4035] ppid=4031 vsize=177936 CPUtime=12.65 cores=0,2,4,6
/proc/4033/task/4035/stat : 4035 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 9443 0 0 0 1258 7 0 0 20 0 3 0 333163199 182206464 12851 33554432000 4194304 5964478 140736525163488 140702392223864 4221120 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 25.37
Current children cumulated vsize (KiB) 177936

[startup+25.5003 s]
/proc/loadavg: 3.79 3.84 3.21 5/181 4036
/proc/meminfo: memFree=28311352/32873844 swapFree=7632/7632
[pid=4033] ppid=4031 vsize=181616 CPUtime=50.97 cores=0,2,4,6
/proc/4033/stat : 4033 (pwbo2.02) S 4031 4033 3617 0 -1 4202496 16261 0 0 0 5081 16 0 0 20 0 3 0 333163199 185974784 16182 33554432000 4194304 5964478 140736525163488 140736525163008 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4033/statm: 45404 16182 207 433 0 44968 0
[pid=4033/tid=4034] ppid=4031 vsize=181616 CPUtime=25.41 cores=0,2,4,6
/proc/4033/task/4034/stat : 4034 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 4043 0 0 0 2535 6 0 0 20 0 3 0 333163199 185974784 16182 33554432000 4194304 5964478 140736525163488 140702402713400 4221120 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=4033/tid=4035] ppid=4031 vsize=181616 CPUtime=25.42 cores=0,2,4,6
/proc/4033/task/4035/stat : 4035 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 11451 0 0 0 2533 9 0 0 20 0 3 0 333163199 185974784 16182 33554432000 4194304 5964478 140736525163488 140702392223864 4400056 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 50.97
Current children cumulated vsize (KiB) 181616

[startup+51.1003 s]
/proc/loadavg: 3.86 3.85 3.23 5/181 4036
/proc/meminfo: memFree=28273332/32873844 swapFree=7632/7632
[pid=4033] ppid=4031 vsize=185524 CPUtime=102.16 cores=0,2,4,6
/proc/4033/stat : 4033 (pwbo2.02) S 4031 4033 3617 0 -1 4202496 21026 0 0 0 10196 20 0 0 20 0 3 0 333163199 189976576 20931 33554432000 4194304 5964478 140736525163488 140736525163008 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4033/statm: 46381 20931 207 433 0 45945 0
[pid=4033/tid=4034] ppid=4031 vsize=185524 CPUtime=50.94 cores=0,2,4,6
/proc/4033/task/4034/stat : 4034 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 5841 0 0 0 5086 8 0 0 20 0 3 0 333163199 189976576 20931 33554432000 4194304 5964478 140736525163488 140702402713400 4399991 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=4033/tid=4035] ppid=4031 vsize=185524 CPUtime=50.95 cores=0,2,4,6
/proc/4033/task/4035/stat : 4035 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 14418 0 0 0 5084 11 0 0 20 0 3 0 333163199 189976576 20931 33554432000 4194304 5964478 140736525163488 140702392224056 4400103 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 102.16
Current children cumulated vsize (KiB) 185524

[startup+102.306 s]
/proc/loadavg: 3.94 3.87 3.27 5/181 4036
/proc/meminfo: memFree=28192468/32873844 swapFree=7636/7636
[pid=4033] ppid=4031 vsize=194192 CPUtime=204.54 cores=0,2,4,6
/proc/4033/stat : 4033 (pwbo2.02) S 4031 4033 3617 0 -1 4202496 27477 0 0 0 20428 26 0 0 20 0 3 0 333163199 198852608 27858 33554432000 4194304 5964478 140736525163488 140736525163008 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4033/statm: 48548 27858 207 433 0 48112 0
[pid=4033/tid=4034] ppid=4031 vsize=194192 CPUtime=102 cores=0,2,4,6
/proc/4033/task/4034/stat : 4034 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 8847 0 0 0 10187 13 0 0 20 0 3 0 333163199 198852608 27858 33554432000 4194304 5964478 140736525163488 140702402713400 4399920 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=4033/tid=4035] ppid=4031 vsize=194192 CPUtime=102.01 cores=0,2,4,6
/proc/4033/task/4035/stat : 4035 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 17863 0 0 0 10187 14 0 0 20 0 3 0 333163199 198852608 27858 33554432000 4194304 5964478 140736525163488 140702392223864 4400106 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 204.54
Current children cumulated vsize (KiB) 194192

[startup+162.3 s]
/proc/loadavg: 3.98 3.89 3.31 5/181 4036
/proc/meminfo: memFree=28133448/32873844 swapFree=7632/7632
[pid=4033] ppid=4031 vsize=201212 CPUtime=324.5 cores=0,2,4,6
/proc/4033/stat : 4033 (pwbo2.02) S 4031 4033 3617 0 -1 4202496 33728 0 0 0 32418 32 0 0 20 0 3 0 333163199 206041088 35897 33554432000 4194304 5964478 140736525163488 140736525163008 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4033/statm: 50303 35897 207 433 0 49867 0
[pid=4033/tid=4034] ppid=4031 vsize=201212 CPUtime=161.84 cores=0,2,4,6
/proc/4033/task/4034/stat : 4034 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 11501 0 0 0 16167 17 0 0 20 0 3 0 333163199 206041088 35897 33554432000 4194304 5964478 140736525163488 140702402713640 4400076 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=4033/tid=4035] ppid=4031 vsize=201212 CPUtime=161.85 cores=0,2,4,6
/proc/4033/task/4035/stat : 4035 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 21460 0 0 0 16167 18 0 0 20 0 3 0 333163199 206041088 35897 33554432000 4194304 5964478 140736525163488 140702392224056 4400097 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 324.5
Current children cumulated vsize (KiB) 201212

[startup+222.3 s]
/proc/loadavg: 3.99 3.91 3.35 5/181 4036
/proc/meminfo: memFree=28103640/32873844 swapFree=7632/7632
[pid=4033] ppid=4031 vsize=208652 CPUtime=444.48 cores=0,2,4,6
/proc/4033/stat : 4033 (pwbo2.02) S 4031 4033 3617 0 -1 4202496 37379 0 0 0 44413 35 0 0 20 0 3 0 333163199 213659648 39548 33554432000 4194304 5964478 140736525163488 140736525163008 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4033/statm: 52163 39548 207 433 0 51727 0
[pid=4033/tid=4034] ppid=4031 vsize=208652 CPUtime=221.68 cores=0,2,4,6
/proc/4033/task/4034/stat : 4034 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 12944 0 0 0 22149 19 0 0 20 0 3 0 333163199 213659648 39548 33554432000 4194304 5964478 140736525163488 140702402713592 4399920 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=4033/tid=4035] ppid=4031 vsize=208652 CPUtime=221.7 cores=0,2,4,6
/proc/4033/task/4035/stat : 4035 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 23668 0 0 0 22149 21 0 0 20 0 3 0 333163199 213659648 39548 33554432000 4194304 5964478 140736525163488 140702392224056 4400018 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 444.48
Current children cumulated vsize (KiB) 208652

################
# More data... #
################

[pid=4033/tid=4035] ppid=4031 vsize=277200 CPUtime=281.55 cores=0,2,4,6
/proc/4033/task/4035/stat : 4035 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 25183 0 0 0 28131 24 0 0 20 0 3 0 333163199 283852800 43358 33554432000 4194304 5964478 140736525163488 140702392224056 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 564.46
Current children cumulated vsize (KiB) 277200

[startup+342.3 s]
/proc/loadavg: 4.02 3.95 3.43 5/181 4037
/proc/meminfo: memFree=28011700/32873844 swapFree=7632/7632
[pid=4033] ppid=4031 vsize=286304 CPUtime=684.43 cores=0,2,4,6
/proc/4033/stat : 4033 (pwbo2.02) S 4031 4033 3617 0 -1 4202496 46843 0 0 0 68398 45 0 0 20 0 3 0 333163199 293175296 49101 33554432000 4194304 5964478 140736525163488 140736525163008 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4033/statm: 71576 49101 207 433 0 71140 0
[pid=4033/tid=4034] ppid=4031 vsize=286304 CPUtime=341.35 cores=0,2,4,6
/proc/4033/task/4034/stat : 4034 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 18117 0 0 0 34108 27 0 0 20 0 3 0 333163199 293175296 49101 33554432000 4194304 5964478 140736525163488 140702402713592 4379409 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=4033/tid=4035] ppid=4031 vsize=286304 CPUtime=341.39 cores=0,2,4,6
/proc/4033/task/4035/stat : 4035 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 27959 0 0 0 34112 27 0 0 20 0 3 0 333163199 293175296 49101 33554432000 4194304 5964478 140736525163488 140702392224056 4400115 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 684.43
Current children cumulated vsize (KiB) 286304

[startup+402.3 s]
/proc/loadavg: 4.00 3.95 3.46 5/181 4037
/proc/meminfo: memFree=27970076/32873844 swapFree=7632/7632
[pid=4033] ppid=4031 vsize=289268 CPUtime=804.41 cores=0,2,4,6
/proc/4033/stat : 4033 (pwbo2.02) S 4031 4033 3617 0 -1 4202496 48446 0 0 0 80394 47 0 0 20 0 3 0 333163199 296210432 50704 33554432000 4194304 5964478 140736525163488 140736525163008 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4033/statm: 72317 50704 207 433 0 71881 0
[pid=4033/tid=4034] ppid=4031 vsize=289268 CPUtime=401.19 cores=0,2,4,6
/proc/4033/task/4034/stat : 4034 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 18450 0 0 0 40090 29 0 0 20 0 3 0 333163199 296210432 50704 33554432000 4194304 5964478 140736525163488 140702402713640 4399920 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=4033/tid=4035] ppid=4031 vsize=289268 CPUtime=401.24 cores=0,2,4,6
/proc/4033/task/4035/stat : 4035 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 29229 0 0 0 40095 29 0 0 20 0 3 0 333163199 296210432 50704 33554432000 4194304 5964478 140736525163488 140702392224056 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 804.41
Current children cumulated vsize (KiB) 289268

[startup+462.3 s]
/proc/loadavg: 4.00 3.96 3.49 5/182 4040
/proc/meminfo: memFree=27905736/32873844 swapFree=7632/7632
[pid=4033] ppid=4031 vsize=363732 CPUtime=924.39 cores=0,2,4,6
/proc/4033/stat : 4033 (pwbo2.02) S 4031 4033 3617 0 -1 4202496 52781 0 0 0 92387 52 0 0 20 0 3 0 333163199 372461568 55039 33554432000 4194304 5964478 140736525163488 140736525163008 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4033/statm: 90933 55039 207 433 0 90497 0
[pid=4033/tid=4034] ppid=4031 vsize=363732 CPUtime=461.03 cores=0,2,4,6
/proc/4033/task/4034/stat : 4034 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 20542 0 0 0 46070 33 0 0 20 0 3 0 333163199 372461568 55039 33554432000 4194304 5964478 140736525163488 140702402713592 4400432 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=4033/tid=4035] ppid=4031 vsize=363732 CPUtime=461.08 cores=0,2,4,6
/proc/4033/task/4035/stat : 4035 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 31472 0 0 0 46076 32 0 0 20 0 3 0 333163199 372461568 55039 33554432000 4194304 5964478 140736525163488 140702392223864 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 924.39
Current children cumulated vsize (KiB) 363732

[startup+522.3 s]
/proc/loadavg: 4.00 3.96 3.52 5/182 4063
/proc/meminfo: memFree=27851052/32873844 swapFree=7632/7632
[pid=4033] ppid=4031 vsize=368844 CPUtime=1044.37 cores=0,2,4,6
/proc/4033/stat : 4033 (pwbo2.02) S 4031 4033 3617 0 -1 4202496 55162 0 0 0 104382 55 0 0 20 0 3 0 333163199 377696256 57420 33554432000 4194304 5964478 140736525163488 140736525163008 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4033/statm: 92211 57420 207 433 0 91775 0
[pid=4033/tid=4034] ppid=4031 vsize=368844 CPUtime=520.87 cores=0,2,4,6
/proc/4033/task/4034/stat : 4034 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 21240 0 0 0 52053 34 0 0 20 0 3 0 333163199 377696256 57420 33554432000 4194304 5964478 140736525163488 140702402713592 4400010 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=4033/tid=4035] ppid=4031 vsize=368844 CPUtime=520.93 cores=0,2,4,6
/proc/4033/task/4035/stat : 4035 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 33155 0 0 0 52059 34 0 0 20 0 3 0 333163199 377696256 57420 33554432000 4194304 5964478 140736525163488 140702392224056 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1044.37
Current children cumulated vsize (KiB) 368844

[startup+582.3 s]
/proc/loadavg: 4.00 3.97 3.55 5/181 4063
/proc/meminfo: memFree=27824804/32873844 swapFree=7632/7632
[pid=4033] ppid=4031 vsize=352404 CPUtime=1164.35 cores=0,2,4,6
/proc/4033/stat : 4033 (pwbo2.02) S 4031 4033 3617 0 -1 4202496 57708 0 0 0 116378 57 0 0 20 0 3 0 333163199 360861696 56088 33554432000 4194304 5964478 140736525163488 140736525163008 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4033/statm: 88101 56088 209 433 0 87665 0
[pid=4033/tid=4034] ppid=4031 vsize=352404 CPUtime=580.7 cores=0,2,4,6
/proc/4033/task/4034/stat : 4034 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 21656 0 0 0 58034 36 0 0 20 0 3 0 333163199 360861696 56088 33554432000 4194304 5964478 140736525163488 140702402714552 4400470 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=4033/tid=4035] ppid=4031 vsize=352404 CPUtime=580.77 cores=0,2,4,6
/proc/4033/task/4035/stat : 4035 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 35285 0 0 0 58041 36 0 0 20 0 3 0 333163199 360861696 56088 33554432000 4194304 5964478 140736525163488 140702392223928 4220672 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1164.35
Current children cumulated vsize (KiB) 352404

[startup+642.3 s]
/proc/loadavg: 4.00 3.97 3.57 5/181 4064
/proc/meminfo: memFree=27803280/32873844 swapFree=7632/7632
[pid=4033] ppid=4031 vsize=357452 CPUtime=1284.34 cores=0,2,4,6
/proc/4033/stat : 4033 (pwbo2.02) S 4031 4033 3617 0 -1 4202496 60354 0 0 0 128374 60 0 0 20 0 3 0 333163199 366030848 58794 33554432000 4194304 5964478 140736525163488 140736525163008 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4033/statm: 89363 58794 209 433 0 88927 0
[pid=4033/tid=4034] ppid=4031 vsize=357452 CPUtime=640.55 cores=0,2,4,6
/proc/4033/task/4034/stat : 4034 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 21656 0 0 0 64018 37 0 0 20 0 3 0 333163199 366030848 58794 33554432000 4194304 5964478 140736525163488 140702402714552 4399920 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=4033/tid=4035] ppid=4031 vsize=357452 CPUtime=640.62 cores=0,2,4,6
/proc/4033/task/4035/stat : 4035 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 37931 0 0 0 64021 41 0 0 20 0 3 0 333163199 366030848 58794 33554432000 4194304 5964478 140736525163488 140702392224056 4400087 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1284.34
Current children cumulated vsize (KiB) 357452

[startup+702.3 s]
/proc/loadavg: 4.00 3.97 3.60 5/181 4064
/proc/meminfo: memFree=27760800/32873844 swapFree=7632/7632
[pid=4033] ppid=4031 vsize=360020 CPUtime=1404.31 cores=0,2,4,6
/proc/4033/stat : 4033 (pwbo2.02) S 4031 4033 3617 0 -1 4202496 63280 0 0 0 140369 62 0 0 20 0 3 0 333163199 368660480 61715 33554432000 4194304 5964478 140736525163488 140736525163008 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4033/statm: 90005 61715 209 433 0 89569 0
[pid=4033/tid=4034] ppid=4031 vsize=360020 CPUtime=700.39 cores=0,2,4,6
/proc/4033/task/4034/stat : 4034 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 21656 0 0 0 70001 38 0 0 20 0 3 0 333163199 368660480 61715 33554432000 4194304 5964478 140736525163488 140702402714552 4399920 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=4033/tid=4035] ppid=4031 vsize=360020 CPUtime=700.46 cores=0,2,4,6
/proc/4033/task/4035/stat : 4035 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 40857 0 0 0 70002 44 0 0 20 0 3 0 333163199 368660480 61715 33554432000 4194304 5964478 140736525163488 140702392224056 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1404.31
Current children cumulated vsize (KiB) 360020

[startup+762.307 s]
/proc/loadavg: 4.06 3.99 3.63 5/181 4064
/proc/meminfo: memFree=27738224/32873844 swapFree=7632/7632
[pid=4033] ppid=4031 vsize=361776 CPUtime=1524.31 cores=0,2,4,6
/proc/4033/stat : 4033 (pwbo2.02) S 4031 4033 3617 0 -1 4202496 65877 0 0 0 152367 64 0 0 20 0 3 0 333163199 370458624 64312 33554432000 4194304 5964478 140736525163488 140736525163008 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4033/statm: 90444 64312 209 433 0 90008 0
[pid=4033/tid=4034] ppid=4031 vsize=361776 CPUtime=760.23 cores=0,2,4,6
/proc/4033/task/4034/stat : 4034 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 21656 0 0 0 75984 39 0 0 20 0 3 0 333163199 370458624 64312 33554432000 4194304 5964478 140736525163488 140702402714552 4399996 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=4033/tid=4035] ppid=4031 vsize=361776 CPUtime=760.32 cores=0,2,4,6
/proc/4033/task/4035/stat : 4035 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 43454 0 0 0 75986 46 0 0 20 0 3 0 333163199 370458624 64312 33554432000 4194304 5964478 140736525163488 140702392224056 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1524.31
Current children cumulated vsize (KiB) 361776

[startup+822.3 s]
/proc/loadavg: 4.02 3.99 3.65 5/181 4064
/proc/meminfo: memFree=27714664/32873844 swapFree=7632/7632
[pid=4033] ppid=4031 vsize=362016 CPUtime=1644.27 cores=0,2,4,6
/proc/4033/stat : 4033 (pwbo2.02) S 4031 4033 3617 0 -1 4202496 65977 0 0 0 164363 64 0 0 20 0 3 0 333163199 370704384 64412 33554432000 4194304 5964478 140736525163488 140736525163008 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4033/statm: 90504 64412 209 433 0 90068 0
[pid=4033/tid=4034] ppid=4031 vsize=362016 CPUtime=820.07 cores=0,2,4,6
/proc/4033/task/4034/stat : 4034 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 21656 0 0 0 81967 40 0 0 20 0 3 0 333163199 370704384 64412 33554432000 4194304 5964478 140736525163488 140702402714552 4399920 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=4033/tid=4035] ppid=4031 vsize=362016 CPUtime=820.16 cores=0,2,4,6
/proc/4033/task/4035/stat : 4035 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 43554 0 0 0 81969 47 0 0 20 0 3 0 333163199 370704384 64412 33554432000 4194304 5964478 140736525163488 140702392224056 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1644.27
Current children cumulated vsize (KiB) 362016

[startup+882.3 s]
/proc/loadavg: 4.01 3.99 3.66 5/181 4064
/proc/meminfo: memFree=27687880/32873844 swapFree=7632/7632
[pid=4033] ppid=4031 vsize=362016 CPUtime=1764.25 cores=0,2,4,6
/proc/4033/stat : 4033 (pwbo2.02) S 4031 4033 3617 0 -1 4202496 67430 0 0 0 176360 65 0 0 20 0 3 0 333163199 370704384 65863 33554432000 4194304 5964478 140736525163488 140736525163008 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4033/statm: 90504 65863 209 433 0 90068 0
[pid=4033/tid=4034] ppid=4031 vsize=362016 CPUtime=879.91 cores=0,2,4,6
/proc/4033/task/4034/stat : 4034 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 23109 0 0 0 87949 42 0 0 20 0 3 0 333163199 370704384 65863 33554432000 4194304 5964478 140736525163488 140702402713464 4399920 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=4033/tid=4035] ppid=4031 vsize=362016 CPUtime=880.01 cores=0,2,4,6
/proc/4033/task/4035/stat : 4035 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 43554 0 0 0 87953 48 0 0 20 0 3 0 333163199 370704384 65863 33554432000 4194304 5964478 140736525163488 140702392224056 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1764.25
Current children cumulated vsize (KiB) 362016



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+900.2 s]
/proc/loadavg: 4.00 3.99 3.67 3/177 4065
/proc/meminfo: memFree=28210728/32873844 swapFree=7632/7632
[pid=4033] ppid=4031 vsize=362016 CPUtime=1800.05 cores=0,2,4,6
/proc/4033/stat : 4033 (pwbo2.02) S 4031 4033 3617 0 -1 4202496 67864 0 0 0 179940 65 0 0 20 0 3 0 333163199 370704384 66297 33554432000 4194304 5964478 140736525163488 140736525163008 4546685 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/4033/statm: 90504 66297 209 433 0 90068 0
[pid=4033/tid=4034] ppid=4031 vsize=362016 CPUtime=897.77 cores=0,2,4,6
/proc/4033/task/4034/stat : 4034 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 23523 0 0 0 89734 43 0 0 20 0 3 0 333163199 370704384 66297 33554432000 4194304 5964478 140736525163488 140702402713400 4399920 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=4033/tid=4035] ppid=4031 vsize=362016 CPUtime=897.87 cores=0,2,4,6
/proc/4033/task/4035/stat : 4035 (pwbo2.02) R 4031 4033 3617 0 -1 4202560 43574 0 0 0 89739 48 0 0 20 0 3 0 333163199 370704384 66297 33554432000 4194304 5964478 140736525163488 140702392224056 4400038 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1800.05
Current children cumulated vsize (KiB) 362016

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child status: 0

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 4033 and gives
#  childrusage.ru_utime.tv_sec=1794
#  childrusage.ru_utime.tv_usec=738158
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=940856
# CPU time returned by wait4() is 1795.68
# while last known CPU time is 1800.05
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 900.219
CPU time (s): 1800.05
CPU user time (s): 1799.4
CPU system time (s): 0.65
CPU usage (%): 199.957
Max. virtual memory (cumulated for all children) (KiB): 369604

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1794.74
system time used= 0.940856
maximum resident set size= 265192
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 67865
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 226
involuntary context switches= 2281

runsolver used 1.87271 second user time and 4.87326 second system time

The end

Launcher Data 

Begin job on node141 at 2012-06-02 22:53:09
IDJOB=3728513
IDBENCH=78690
IDSOLVER=2316
FILE ID=node141/3728513-1338670389
RUNJOBID= node141-1338669486-3911
PBS_JOBID= 14636518
Free space on /tmp= 70716 MiB

SOLVER NAME= pwbo 2.02
BENCH NAME= PB10/normalized-PB10/PARTIAL-SMALLINT-LIN/PB06/submitted-PB05/aloul/FPGA_SAT05/normalized-chnl10_11_pb.cnf.cr--soft-33-100-0.wbo
COMMAND LINE= pwbo2.02 -file-format=wbo -verbosity=0 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3728513-1338670389/watcher-3728513-1338670389 -o /tmp/evaluation-result-3728513-1338670389/solver-3728513-1338670389 -C 1800 -W 1900 -M 15500  pwbo2.02 -file-format=wbo -verbosity=0 HOME/instance-3728513-1338670389.wbo

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 0b915fd051dbce2f3b4ec1d342cdefca
RANDOM SEED=547991272

node141.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.744
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.48
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.744
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.744
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.93
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.744
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.744
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.744
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.744
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.744
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        28424588 kB
Buffers:          383100 kB
Cached:          3120080 kB
SwapCached:          216 kB
Active:          1269172 kB
Inactive:        2300148 kB
Active(anon):      56832 kB
Inactive(anon):    11300 kB
Active(file):    1212340 kB
Inactive(file):  2288848 kB
Unevictable:        7632 kB
Mlocked:            7632 kB
SwapTotal:      67108856 kB
SwapFree:       67108640 kB
Dirty:             10932 kB
Writeback:             0 kB
AnonPages:         71164 kB
Mapped:            11388 kB
Shmem:               172 kB
Slab:             739000 kB
SReclaimable:     181040 kB
SUnreclaim:       557960 kB
KernelStack:        1432 kB
PageTables:         3904 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     184364 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70716 MiB
End job on node141 at 2012-06-02 23:08:11