Trace number 3703755

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
pwbo 2.0SAT (TO)6 1800.5 900.318

General information on the benchmark

Namenormalized-PB06/OPT-SMALLINT/submitted-PB06/
manquiho/market-split/normalized-opt-market-split_5_40_3.opb
MD5SUM455599f2e36aed56b051b7169c43fa6c
Bench CategoryOPT-SMALLINT (optimisation, small integers)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark0
Best CPU time to get the best result obtained on this benchmark305.689
Has Objective FunctionYES
SatisfiableYES
(Un)Satisfiability was provedYES
Best value of the objective function 0
Optimality of the best value was proved YES
Number of variables120
Total number of constraints10
Number of constraints which are clauses0
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints10
Minimum length of a constraint55
Maximum length of a constraint56
Number of terms in the objective function 80
Biggest coefficient in the objective function 128
Number of bits for the biggest coefficient in the objective function 8
Sum of the numbers in the objective function 2550
Number of bits of the sum of numbers in the objective function 12
Biggest number in a constraint 1162
Number of bits of the biggest number in a constraint 11
Biggest sum of numbers in a constraint 3997
Number of bits of the biggest sum of numbers12
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Quality of the solution as a function of time


Download the above graph as a PDF file / as an EPS file

Solver Data 

0.00/0.00	c ==============================[ Solver Version ]===============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  pwbo:                    2.0 beta - March, 2012                            |
0.00/0.00	c |                                                                             |
0.00/0.00	c Parsing opb file format. File HOME/instance-3703755-1338093768.opb.
0.00/0.00	c Instance file HOME/instance-3703755-1338093768.opb
0.00/0.00	c File size is 5435 bytes.
0.00/0.00	c Cost Offset: 0
0.00/0.00	c Highest Coefficient sum: 3997
0.00/0.00	c ============================[ Problem Statistics ]=============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  Parsing time:          0.00         s                                      |
0.00/0.00	c |  Number Variables:      120                                                 |
0.00/0.00	c |  Number Clauses:        0                                                   |
0.00/0.00	c |  Number Cardinality:    0                                                   |
0.00/0.00	c |  Number PB Constraints: 10                                                  |
0.00/0.00	c |                                                                             |
0.00/0.00	c |  Problem Type:          Optimization                                        |
0.00/0.00	c |  Objective Function:    Cardinality Constraint                              |
0.00/0.00	c |                                                                             |
0.00/0.00	c ===============================================================================
1800.16/900.31	o 924
1800.16/900.31	o 887
1800.16/900.31	o 885
1800.16/900.31	o 823
1800.16/900.31	o 819
1800.16/900.31	o 809
1800.16/900.31	o 758
1800.16/900.31	o 726
1800.16/900.31	o 634
1800.16/900.31	o 575
1800.16/900.31	o 547
1800.16/900.31	o 499
1800.16/900.31	o 447
1800.16/900.31	o 381
1800.16/900.31	o 326
1800.16/900.31	o 325
1800.16/900.31	o 296
1800.16/900.31	o 291
1800.16/900.31	o 191
1800.16/900.31	o 165
1800.16/900.31	o 163
1800.16/900.31	o 132
1800.16/900.31	o 126
1800.16/900.31	o 118
1800.16/900.31	o 116
1800.16/900.31	o 114
1800.16/900.31	o 99
1800.16/900.31	o 95
1800.16/900.31	o 93
1800.16/900.31	o 58
1800.16/900.31	o 42
1800.16/900.31	o 31
1800.16/900.31	o 25
1800.16/900.31	o 21
1800.16/900.31	o 20
1800.16/900.31	o 19
1800.16/900.31	o 18
1800.16/900.31	o 17
1800.16/900.31	o 13
1800.16/900.31	o 12
1800.16/900.31	o 11
1800.16/900.31	o 10
1800.16/900.31	o 9
1800.16/900.31	o 8
1800.16/900.31	o 6
1800.16/900.31	c ===============================================================================
1800.16/900.31	c Wall Clock Time: 900.31 s
1800.16/900.31	c Best Solution: 6
1800.16/900.31	s SATISFIABLE
1800.16/900.31	v -x1 -x2 -x3 -x4 -x5 -x6 -x7 -x8 -x9 -x10 x11 -x12 -x13 -x14 -x15 -x16 x17 -x18 -x19 -x20 -x21 -x22 -x23 -x24 -x25 -x26 -x27 -x28 -x29 -x30 -x31 -x32 -x33 -x34 -x35 -x36 -x37 -x38 -x39 -x40 -x41 -x42 -x43 -x44 -x45 -x46 -x47 -x48 -x49 -x50 -x51 -x52 -x53 -x54 -x55 -x56 x57 -x58 -x59 -x60 -x61 -x62 -x63 -x64 -x65 -x66 -x67 -x68 -x69 -x70 -x71 -x72 -x73 -x74 -x75 -x76 -x77 -x78 -x79 -x80 -x81 -x82 x83 x84 x85 -x86 x87 -x88 x89 x90 x91 x92 x93 -x94 -x95 x96 x97 x98 -x99 -x100 x101 -x102 -x103 -x104 x105 -x106 -x107 -x108 -x109 -x110 x111 -x112 x113 -x114 -x115 -x116 x117 x118 x119 x120

Verifier Data 

OK	6

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3703755-1338093768/watcher-3703755-1338093768 -o /tmp/evaluation-result-3703755-1338093768/solver-3703755-1338093768 -C 1800 -W 1900 -M 15500 pwbo2.0 -file-format=opb -verbosity=0 HOME/instance-3703755-1338093768.opb 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.00 4.00 3.96 4/178 5791
/proc/meminfo: memFree=30224580/32873648 swapFree=7368/7368
[pid=5791] ppid=5789 vsize=162956 CPUtime=0 cores=1,3,5,7
/proc/5791/stat : 5791 (pwbo2.0) S 5789 5791 3829 0 -1 4202496 1122 0 0 0 0 0 0 0 20 0 3 0 275500910 166866944 1073 33554432000 4194304 5965102 140736489368656 140736489368176 4547341 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/5791/statm: 40739 1075 205 433 0 40303 0
[pid=5791/tid=5792] ppid=5789 vsize=162956 CPUtime=0 cores=1,3,5,7
/proc/5791/task/5792/stat : 5792 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 39 0 0 0 0 0 0 0 20 0 3 0 275500910 166866944 1080 33554432000 4194304 5965102 140736489368656 140434702093160 4390842 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=5791/tid=5793] ppid=5789 vsize=162956 CPUtime=0 cores=1,3,5,7
/proc/5791/task/5793/stat : 5793 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 338 0 0 0 0 0 0 0 20 0 3 0 275500910 166866944 1082 33554432000 4194304 5965102 140736489368656 140734894134568 5137076 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0

[startup+0.0935941 s]
/proc/loadavg: 4.00 4.00 3.96 4/178 5791
/proc/meminfo: memFree=30224580/32873648 swapFree=7368/7368
[pid=5791] ppid=5789 vsize=184068 CPUtime=0.17 cores=1,3,5,7
/proc/5791/stat : 5791 (pwbo2.0) S 5789 5791 3829 0 -1 4202496 10475 0 0 0 10 7 0 0 20 0 3 0 275500910 188485632 10361 33554432000 4194304 5965102 140736489368656 140736489368176 4547341 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/5791/statm: 46017 10364 207 433 0 45581 0
[pid=5791/tid=5792] ppid=5789 vsize=184068 CPUtime=0.07 cores=1,3,5,7
/proc/5791/task/5792/stat : 5792 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 285 0 0 0 7 0 0 0 20 0 3 0 275500910 188485632 10371 33554432000 4194304 5965102 140736489368656 140434702092216 5137076 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=5791/tid=5793] ppid=5789 vsize=184068 CPUtime=0.07 cores=1,3,5,7
/proc/5791/task/5793/stat : 5793 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 9447 0 0 0 0 7 0 0 20 0 3 0 275500910 188485632 10372 33554432000 4194304 5965102 140736489368656 140434691601352 5137076 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 0.17
Current children cumulated vsize (KiB) 184068

[startup+0.100297 s]
/proc/loadavg: 4.00 4.00 3.96 4/178 5791
/proc/meminfo: memFree=30224580/32873648 swapFree=7368/7368
[pid=5791] ppid=5789 vsize=189532 CPUtime=0.19 cores=1,3,5,7
/proc/5791/stat : 5791 (pwbo2.0) S 5789 5791 3829 0 -1 4202496 11076 0 0 0 11 8 0 0 20 0 3 0 275500910 194080768 10962 33554432000 4194304 5965102 140736489368656 140736489368176 4547341 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/5791/statm: 47383 10963 207 433 0 46947 0
[pid=5791/tid=5792] ppid=5789 vsize=189532 CPUtime=0.08 cores=1,3,5,7
/proc/5791/task/5792/stat : 5792 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 298 0 0 0 8 0 0 0 20 0 3 0 275500910 194080768 10966 33554432000 4194304 5965102 140736489368656 140434702092280 4221984 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=5791/tid=5793] ppid=5789 vsize=189532 CPUtime=0.08 cores=1,3,5,7
/proc/5791/task/5793/stat : 5793 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 10029 0 0 0 0 8 0 0 20 0 3 0 275500910 194080768 10967 33554432000 4194304 5965102 140736489368656 140434691601800 5137076 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 189532

[startup+0.300329 s]
/proc/loadavg: 4.00 4.00 3.96 4/178 5791
/proc/meminfo: memFree=30224580/32873648 swapFree=7368/7368
[pid=5791] ppid=5789 vsize=190344 CPUtime=0.59 cores=1,3,5,7
/proc/5791/stat : 5791 (pwbo2.0) S 5789 5791 3829 0 -1 4202496 12528 0 0 0 50 9 0 0 20 0 3 0 275500910 194912256 12925 33554432000 4194304 5965102 140736489368656 140736489368176 4547341 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/5791/statm: 47586 12925 207 433 0 47150 0
[pid=5791/tid=5792] ppid=5789 vsize=190344 CPUtime=0.28 cores=1,3,5,7
/proc/5791/task/5792/stat : 5792 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 738 0 0 0 28 0 0 0 20 0 3 0 275500910 194912256 12925 33554432000 4194304 5965102 140736489368656 140434702092088 4404732 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=5791/tid=5793] ppid=5789 vsize=190344 CPUtime=0.28 cores=1,3,5,7
/proc/5791/task/5793/stat : 5793 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 11036 0 0 0 19 9 0 0 20 0 3 0 275500910 194912256 12925 33554432000 4194304 5965102 140736489368656 140434691602664 5137076 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 0.59
Current children cumulated vsize (KiB) 190344

[startup+0.700327 s]
/proc/loadavg: 4.00 4.00 3.96 4/178 5791
/proc/meminfo: memFree=30224580/32873648 swapFree=7368/7368
[pid=5791] ppid=5789 vsize=190664 CPUtime=1.39 cores=1,3,5,7
/proc/5791/stat : 5791 (pwbo2.0) S 5789 5791 3829 0 -1 4202496 13209 0 0 0 130 9 0 0 20 0 3 0 275500910 195239936 13606 33554432000 4194304 5965102 140736489368656 140736489368176 4547341 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/5791/statm: 47666 13607 207 433 0 47230 0
[pid=5791/tid=5792] ppid=5789 vsize=190664 CPUtime=0.68 cores=1,3,5,7
/proc/5791/task/5792/stat : 5792 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 1350 0 0 0 68 0 0 0 20 0 3 0 275500910 195239936 13607 33554432000 4194304 5965102 140736489368656 140434702092088 5137076 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=5791/tid=5793] ppid=5789 vsize=190664 CPUtime=0.68 cores=1,3,5,7
/proc/5791/task/5793/stat : 5793 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 11106 0 0 0 59 9 0 0 20 0 3 0 275500910 195239936 13607 33554432000 4194304 5965102 140736489368656 140434691602616 4389868 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 1.39
Current children cumulated vsize (KiB) 190664

[startup+1.50036 s]
/proc/loadavg: 4.00 4.00 3.96 5/181 5794
/proc/meminfo: memFree=30170032/32873648 swapFree=7368/7368
[pid=5791] ppid=5789 vsize=190664 CPUtime=2.99 cores=1,3,5,7
/proc/5791/stat : 5791 (pwbo2.0) S 5789 5791 3829 0 -1 4202496 13636 0 0 0 290 9 0 0 20 0 3 0 275500910 195239936 14033 33554432000 4194304 5965102 140736489368656 140736489368176 4547341 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/5791/statm: 47666 14033 208 433 0 47230 0
[pid=5791/tid=5792] ppid=5789 vsize=190664 CPUtime=1.48 cores=1,3,5,7
/proc/5791/task/5792/stat : 5792 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 1719 0 0 0 148 0 0 0 20 0 3 0 275500910 195239936 14033 33554432000 4194304 5965102 140736489368656 140434702092088 4383254 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=5791/tid=5793] ppid=5789 vsize=190664 CPUtime=1.48 cores=1,3,5,7
/proc/5791/task/5793/stat : 5793 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 11163 0 0 0 139 9 0 0 20 0 3 0 275500910 195239936 14033 33554432000 4194304 5965102 140736489368656 140434691602616 4400894 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 2.99
Current children cumulated vsize (KiB) 190664

[startup+3.10032 s]
/proc/loadavg: 4.00 4.00 3.96 5/181 5794
/proc/meminfo: memFree=30166188/32873648 swapFree=7368/7368
[pid=5791] ppid=5789 vsize=190736 CPUtime=6.18 cores=1,3,5,7
/proc/5791/stat : 5791 (pwbo2.0) S 5789 5791 3829 0 -1 4202496 14690 0 0 0 608 10 0 0 20 0 3 0 275500910 195313664 15087 33554432000 4194304 5965102 140736489368656 140736489368176 4547341 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/5791/statm: 47684 15087 208 433 0 47248 0
[pid=5791/tid=5792] ppid=5789 vsize=190736 CPUtime=3.08 cores=1,3,5,7
/proc/5791/task/5792/stat : 5792 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 2650 0 0 0 307 1 0 0 20 0 3 0 275500910 195313664 15087 33554432000 4194304 5965102 140736489368656 140434702092088 4401137 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=5791/tid=5793] ppid=5789 vsize=190736 CPUtime=3.08 cores=1,3,5,7
/proc/5791/task/5793/stat : 5793 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 11286 0 0 0 299 9 0 0 20 0 3 0 275500910 195313664 15087 33554432000 4194304 5965102 140736489368656 140434691602616 4410257 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 6.18
Current children cumulated vsize (KiB) 190736

[startup+6.30034 s]
/proc/loadavg: 4.00 4.00 3.96 5/181 5794
/proc/meminfo: memFree=30153736/32873648 swapFree=7368/7368
[pid=5791] ppid=5789 vsize=190844 CPUtime=12.58 cores=1,3,5,7
/proc/5791/stat : 5791 (pwbo2.0) S 5789 5791 3829 0 -1 4202496 16537 0 0 0 1247 11 0 0 20 0 3 0 275500910 195424256 16934 33554432000 4194304 5965102 140736489368656 140736489368176 4547341 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/5791/statm: 47711 16934 208 433 0 47275 0
[pid=5791/tid=5792] ppid=5789 vsize=190844 CPUtime=6.28 cores=1,3,5,7
/proc/5791/task/5792/stat : 5792 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 4322 0 0 0 626 2 0 0 20 0 3 0 275500910 195424256 16934 33554432000 4194304 5965102 140736489368656 140434702092328 5137076 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=5791/tid=5793] ppid=5789 vsize=190844 CPUtime=6.29 cores=1,3,5,7
/proc/5791/task/5793/stat : 5793 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 11461 0 0 0 619 10 0 0 20 0 3 0 275500910 195424256 16934 33554432000 4194304 5965102 140736489368656 140434691602744 4389645 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 12.58
Current children cumulated vsize (KiB) 190844

[startup+12.7003 s]
/proc/loadavg: 4.00 4.00 3.96 5/181 5794
/proc/meminfo: memFree=30133220/32873648 swapFree=7368/7368
[pid=5791] ppid=5789 vsize=191004 CPUtime=25.38 cores=1,3,5,7
/proc/5791/stat : 5791 (pwbo2.0) S 5789 5791 3829 0 -1 4202496 18947 0 0 0 2525 13 0 0 20 0 3 0 275500910 195588096 19344 33554432000 4194304 5965102 140736489368656 140736489368176 4547341 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/5791/statm: 47751 19344 208 433 0 47315 0
[pid=5791/tid=5792] ppid=5789 vsize=191004 CPUtime=12.68 cores=1,3,5,7
/proc/5791/task/5792/stat : 5792 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 6519 0 0 0 1264 4 0 0 20 0 3 0 275500910 195588096 19344 33554432000 4194304 5965102 140736489368656 140434702092280 5137076 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=5791/tid=5793] ppid=5789 vsize=191004 CPUtime=12.68 cores=1,3,5,7
/proc/5791/task/5793/stat : 5793 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 11674 0 0 0 1258 10 0 0 20 0 3 0 275500910 195588096 19344 33554432000 4194304 5965102 140736489368656 140434691602744 4400894 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 25.38
Current children cumulated vsize (KiB) 191004

[startup+25.5003 s]
/proc/loadavg: 4.00 4.00 3.96 5/181 5794
/proc/meminfo: memFree=30096756/32873648 swapFree=7368/7368
[pid=5791] ppid=5789 vsize=191488 CPUtime=50.97 cores=1,3,5,7
/proc/5791/stat : 5791 (pwbo2.0) S 5789 5791 3829 0 -1 4202496 24555 0 0 0 5076 21 0 0 20 0 3 0 275500910 196083712 24952 33554432000 4194304 5965102 140736489368656 140736489368176 4547341 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/5791/statm: 47872 24952 208 433 0 47436 0
[pid=5791/tid=5792] ppid=5789 vsize=191488 CPUtime=25.48 cores=1,3,5,7
/proc/5791/task/5792/stat : 5792 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 9140 0 0 0 2541 7 0 0 20 0 3 0 275500910 196083712 24952 33554432000 4194304 5965102 140736489368656 140434702092280 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=5791/tid=5793] ppid=5789 vsize=191488 CPUtime=25.48 cores=1,3,5,7
/proc/5791/task/5793/stat : 5793 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 14661 0 0 0 2533 15 0 0 20 0 3 0 275500910 196083712 24952 33554432000 4194304 5965102 140736489368656 140434691602552 5244967 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 50.97
Current children cumulated vsize (KiB) 191488

[startup+51.1004 s]
/proc/loadavg: 4.00 4.00 3.96 5/181 5794
/proc/meminfo: memFree=30055308/32873648 swapFree=7368/7368
[pid=5791] ppid=5789 vsize=193048 CPUtime=102.17 cores=1,3,5,7
/proc/5791/stat : 5791 (pwbo2.0) S 5789 5791 3829 0 -1 4202496 31377 0 0 0 10184 33 0 0 20 0 3 0 275500910 197681152 31774 33554432000 4194304 5965102 140736489368656 140736489368176 4547341 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/5791/statm: 48262 31774 208 433 0 47826 0
[pid=5791/tid=5792] ppid=5789 vsize=193048 CPUtime=51.08 cores=1,3,5,7
/proc/5791/task/5792/stat : 5792 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 12419 0 0 0 5096 12 0 0 20 0 3 0 275500910 197681152 31774 33554432000 4194304 5965102 140736489368656 140434702092280 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=5791/tid=5793] ppid=5789 vsize=193048 CPUtime=51.08 cores=1,3,5,7
/proc/5791/task/5793/stat : 5793 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 18204 0 0 0 5086 22 0 0 20 0 3 0 275500910 197681152 31774 33554432000 4194304 5965102 140736489368656 140434691602744 4400960 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 102.17
Current children cumulated vsize (KiB) 193048

[startup+102.307 s]
/proc/loadavg: 4.00 4.00 3.96 5/181 5794
/proc/meminfo: memFree=29974412/32873648 swapFree=7368/7368
[pid=5791] ppid=5789 vsize=263084 CPUtime=204.56 cores=1,3,5,7
/proc/5791/stat : 5791 (pwbo2.0) S 5789 5791 3829 0 -1 4202496 41642 0 0 0 20407 49 0 0 20 0 3 0 275500910 269398016 42032 33554432000 4194304 5965102 140736489368656 140736489368176 4547341 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/5791/statm: 65771 42032 208 433 0 65335 0
[pid=5791/tid=5792] ppid=5789 vsize=263084 CPUtime=102.28 cores=1,3,5,7
/proc/5791/task/5792/stat : 5792 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 19059 0 0 0 10203 25 0 0 20 0 3 0 275500910 269398016 42032 33554432000 4194304 5965102 140736489368656 140434702092280 4401143 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=5791/tid=5793] ppid=5789 vsize=263084 CPUtime=102.28 cores=1,3,5,7
/proc/5791/task/5793/stat : 5793 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 21829 0 0 0 10201 27 0 0 20 0 3 0 275500910 269398016 42032 33554432000 4194304 5965102 140736489368656 140434691602744 4390219 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 204.56
Current children cumulated vsize (KiB) 263084

[startup+162.3 s]
/proc/loadavg: 4.00 4.00 3.96 5/181 5795
/proc/meminfo: memFree=29908844/32873648 swapFree=7368/7368
[pid=5791] ppid=5789 vsize=333480 CPUtime=324.49 cores=1,3,5,7
/proc/5791/stat : 5791 (pwbo2.0) S 5789 5791 3829 0 -1 4202496 49400 0 0 0 32383 66 0 0 20 0 3 0 275500910 341483520 49829 33554432000 4194304 5965102 140736489368656 140736489368176 4547341 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/5791/statm: 83370 49829 208 433 0 82934 0
[pid=5791/tid=5792] ppid=5789 vsize=333480 CPUtime=162.23 cores=1,3,5,7
/proc/5791/task/5792/stat : 5792 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 23770 0 0 0 16186 37 0 0 20 0 3 0 275500910 341483520 49829 33554432000 4194304 5965102 140736489368656 140434702092280 4401462 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=5791/tid=5793] ppid=5789 vsize=333480 CPUtime=162.27 cores=1,3,5,7
/proc/5791/task/5793/stat : 5793 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 24876 0 0 0 16193 34 0 0 20 0 3 0 275500910 341483520 49829 33554432000 4194304 5965102 140736489368656 140434691602744 4400960 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 324.49
Current children cumulated vsize (KiB) 333480

[startup+222.3 s]
/proc/loadavg: 4.00 4.00 3.96 5/181 5795
/proc/meminfo: memFree=29845900/32873648 swapFree=7368/7368
[pid=5791] ppid=5789 vsize=336608 CPUtime=444.44 cores=1,3,5,7
/proc/5791/stat : 5791 (pwbo2.0) S 5789 5791 3829 0 -1 4202496 57171 0 0 0 44360 84 0 0 20 0 3 0 275500910 344686592 58973 33554432000 4194304 5965102 140736489368656 140736489368176 4547341 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/5791/statm: 84152 58973 208 433 0 83716 0
[pid=5791/tid=5792] ppid=5789 vsize=336608 CPUtime=222.23 cores=1,3,5,7
/proc/5791/task/5792/stat : 5792 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 29450 0 0 0 22170 53 0 0 20 0 3 0 275500910 344686592 58973 33554432000 4194304 5965102 140736489368656 140434702092280 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=5791/tid=5793] ppid=5789 vsize=336608 CPUtime=222.25 cores=1,3,5,7
/proc/5791/task/5793/stat : 5793 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 26967 0 0 0 22186 39 0 0 20 0 3 0 275500910 344686592 58973 33554432000 4194304 5965102 140736489368656 140434691602744 4400960 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 444.44
Current children cumulated vsize (KiB) 336608

################
# More data... #
################

/proc/5791/task/5792/stat : 5792 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 32555 0 0 0 28161 62 0 0 20 0 3 0 275500910 346750976 63826 33554432000 4194304 5965102 140736489368656 140434702092280 4401015 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=5791/tid=5793] ppid=5789 vsize=338624 CPUtime=282.26 cores=1,3,5,7
/proc/5791/task/5793/stat : 5793 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 28715 0 0 0 28183 43 0 0 20 0 3 0 275500910 346750976 63826 33554432000 4194304 5965102 140736489368656 140434691602744 4400960 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 564.42
Current children cumulated vsize (KiB) 338624

[startup+342.3 s]
/proc/loadavg: 4.00 4.00 3.96 5/181 5795
/proc/meminfo: memFree=29743824/32873648 swapFree=7368/7368
[pid=5791] ppid=5789 vsize=406908 CPUtime=684.4 cores=1,3,5,7
/proc/5791/stat : 5791 (pwbo2.0) S 5789 5791 3829 0 -1 4202496 68702 0 0 0 68328 112 0 0 20 0 3 0 275500910 416673792 70504 33554432000 4194304 5965102 140736489368656 140736489368176 4547341 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/5791/statm: 101727 70504 208 433 0 101291 0
[pid=5791/tid=5792] ppid=5789 vsize=406908 CPUtime=342.23 cores=1,3,5,7
/proc/5791/task/5792/stat : 5792 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 37599 0 0 0 34146 77 0 0 20 0 3 0 275500910 416673792 70504 33554432000 4194304 5965102 140736489368656 140434702092280 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=5791/tid=5793] ppid=5789 vsize=406908 CPUtime=342.26 cores=1,3,5,7
/proc/5791/task/5793/stat : 5793 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 30349 0 0 0 34180 46 0 0 20 0 3 0 275500910 416673792 70504 33554432000 4194304 5965102 140736489368656 140434691602744 4221120 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 684.4
Current children cumulated vsize (KiB) 406908

[startup+402.3 s]
/proc/loadavg: 4.00 4.00 3.96 5/181 5795
/proc/meminfo: memFree=29707952/32873648 swapFree=7368/7368
[pid=5791] ppid=5789 vsize=409388 CPUtime=804.36 cores=1,3,5,7
/proc/5791/stat : 5791 (pwbo2.0) S 5789 5791 3829 0 -1 4202496 72578 0 0 0 80316 120 0 0 20 0 3 0 275500910 419213312 74403 33554432000 4194304 5965102 140736489368656 140736489368176 4547341 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/5791/statm: 102347 74403 208 433 0 101911 0
[pid=5791/tid=5792] ppid=5789 vsize=409388 CPUtime=402.22 cores=1,3,5,7
/proc/5791/task/5792/stat : 5792 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 40138 0 0 0 40137 85 0 0 20 0 3 0 275500910 419213312 74403 33554432000 4194304 5965102 140736489368656 140434702092328 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=5791/tid=5793] ppid=5789 vsize=409388 CPUtime=402.25 cores=1,3,5,7
/proc/5791/task/5793/stat : 5793 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 31686 0 0 0 40176 49 0 0 20 0 3 0 275500910 419213312 74403 33554432000 4194304 5965102 140736489368656 140434691602744 4400960 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 804.36
Current children cumulated vsize (KiB) 409388

[startup+462.3 s]
/proc/loadavg: 4.00 4.00 3.96 5/182 5799
/proc/meminfo: memFree=29671352/32873648 swapFree=7368/7368
[pid=5791] ppid=5789 vsize=412272 CPUtime=924.33 cores=1,3,5,7
/proc/5791/stat : 5791 (pwbo2.0) S 5789 5791 3829 0 -1 4202496 78256 0 0 0 92301 132 0 0 20 0 3 0 275500910 422166528 80078 33554432000 4194304 5965102 140736489368656 140736489368176 4547341 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/5791/statm: 103068 80078 208 433 0 102632 0
[pid=5791/tid=5792] ppid=5789 vsize=412272 CPUtime=462.2 cores=1,3,5,7
/proc/5791/task/5792/stat : 5792 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 44584 0 0 0 46125 95 0 0 20 0 3 0 275500910 422166528 80078 33554432000 4194304 5965102 140736489368656 140434702092328 4400978 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=5791/tid=5793] ppid=5789 vsize=412272 CPUtime=462.25 cores=1,3,5,7
/proc/5791/task/5793/stat : 5793 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 32918 0 0 0 46173 52 0 0 20 0 3 0 275500910 422166528 80078 33554432000 4194304 5965102 140736489368656 140434691602744 4400960 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 924.33
Current children cumulated vsize (KiB) 412272

[startup+522.3 s]
/proc/loadavg: 4.15 4.03 3.97 5/181 5799
/proc/meminfo: memFree=29627388/32873648 swapFree=7368/7368
[pid=5791] ppid=5789 vsize=413368 CPUtime=1044.3 cores=1,3,5,7
/proc/5791/stat : 5791 (pwbo2.0) S 5789 5791 3829 0 -1 4202496 81861 0 0 0 104290 140 0 0 20 0 3 0 275500910 423288832 83683 33554432000 4194304 5965102 140736489368656 140736489368176 4547341 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/5791/statm: 103342 83683 208 433 0 102906 0
[pid=5791/tid=5792] ppid=5789 vsize=413368 CPUtime=522.2 cores=1,3,5,7
/proc/5791/task/5792/stat : 5792 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 47058 0 0 0 52118 102 0 0 20 0 3 0 275500910 423288832 83683 33554432000 4194304 5965102 140736489368656 140434702092328 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=5791/tid=5793] ppid=5789 vsize=413368 CPUtime=522.25 cores=1,3,5,7
/proc/5791/task/5793/stat : 5793 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 34049 0 0 0 52172 53 0 0 20 0 3 0 275500910 423288832 83683 33554432000 4194304 5965102 140736489368656 140434691602744 4400960 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1044.3
Current children cumulated vsize (KiB) 413368

[startup+582.3 s]
/proc/loadavg: 4.05 4.02 3.97 5/181 5799
/proc/meminfo: memFree=29577156/32873648 swapFree=7368/7368
[pid=5791] ppid=5789 vsize=481148 CPUtime=1164.28 cores=1,3,5,7
/proc/5791/stat : 5791 (pwbo2.0) S 5789 5791 3829 0 -1 4202496 86970 0 0 0 116278 150 0 0 20 0 3 0 275500910 492695552 88792 33554432000 4194304 5965102 140736489368656 140736489368176 4547341 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/5791/statm: 120287 88792 208 433 0 119851 0
[pid=5791/tid=5792] ppid=5789 vsize=481148 CPUtime=582.2 cores=1,3,5,7
/proc/5791/task/5792/stat : 5792 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 51003 0 0 0 58108 112 0 0 20 0 3 0 275500910 492695552 88792 33554432000 4194304 5965102 140736489368656 140434702092328 5137076 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=5791/tid=5793] ppid=5789 vsize=481148 CPUtime=582.26 cores=1,3,5,7
/proc/5791/task/5793/stat : 5793 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 35213 0 0 0 58170 56 0 0 20 0 3 0 275500910 492695552 88792 33554432000 4194304 5965102 140736489368656 140434691602744 4400960 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1164.28
Current children cumulated vsize (KiB) 481148

[startup+642.3 s]
/proc/loadavg: 4.02 4.02 3.97 5/181 5799
/proc/meminfo: memFree=29543400/32873648 swapFree=7368/7368
[pid=5791] ppid=5789 vsize=482168 CPUtime=1284.28 cores=1,3,5,7
/proc/5791/stat : 5791 (pwbo2.0) S 5789 5791 3829 0 -1 4202496 90297 0 0 0 128269 159 0 0 20 0 3 0 275500910 493740032 92119 33554432000 4194304 5965102 140736489368656 140736489368176 4547341 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/5791/statm: 120542 92119 208 433 0 120106 0
[pid=5791/tid=5792] ppid=5789 vsize=482168 CPUtime=642.21 cores=1,3,5,7
/proc/5791/task/5792/stat : 5792 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 53260 0 0 0 64102 119 0 0 20 0 3 0 275500910 493740032 92119 33554432000 4194304 5965102 140736489368656 140434702092328 4401004 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=5791/tid=5793] ppid=5789 vsize=482168 CPUtime=642.27 cores=1,3,5,7
/proc/5791/task/5793/stat : 5793 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 36283 0 0 0 64168 59 0 0 20 0 3 0 275500910 493740032 92119 33554432000 4194304 5965102 140736489368656 140434691602744 4400960 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1284.28
Current children cumulated vsize (KiB) 482168

[startup+702.3 s]
/proc/loadavg: 4.00 4.01 3.97 5/181 5799
/proc/meminfo: memFree=29519096/32873648 swapFree=7368/7368
[pid=5791] ppid=5789 vsize=483068 CPUtime=1404.25 cores=1,3,5,7
/proc/5791/stat : 5791 (pwbo2.0) S 5789 5791 3829 0 -1 4202496 95307 0 0 0 140254 171 0 0 20 0 3 0 275500910 494661632 97129 33554432000 4194304 5965102 140736489368656 140736489368176 4547341 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/5791/statm: 120767 97129 208 433 0 120331 0
[pid=5791/tid=5792] ppid=5789 vsize=483068 CPUtime=702.2 cores=1,3,5,7
/proc/5791/task/5792/stat : 5792 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 57545 0 0 0 70088 132 0 0 20 0 3 0 275500910 494661632 97129 33554432000 4194304 5965102 140736489368656 140434702092280 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=5791/tid=5793] ppid=5789 vsize=483068 CPUtime=702.26 cores=1,3,5,7
/proc/5791/task/5793/stat : 5793 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 37008 0 0 0 70166 60 0 0 20 0 3 0 275500910 494661632 97129 33554432000 4194304 5965102 140736489368656 140434691602616 4221120 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1404.25
Current children cumulated vsize (KiB) 483068

[startup+762.307 s]
/proc/loadavg: 4.00 4.01 3.97 5/181 5800
/proc/meminfo: memFree=29496488/32873648 swapFree=7368/7368
[pid=5791] ppid=5789 vsize=483248 CPUtime=1524.23 cores=1,3,5,7
/proc/5791/stat : 5791 (pwbo2.0) S 5789 5791 3829 0 -1 4202496 95853 0 0 0 152250 173 0 0 20 0 3 0 275500910 494845952 97841 33554432000 4194304 5965102 140736489368656 140736489368176 4547341 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/5791/statm: 120812 97841 208 433 0 120376 0
[pid=5791/tid=5792] ppid=5789 vsize=483248 CPUtime=762.2 cores=1,3,5,7
/proc/5791/task/5792/stat : 5792 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 58062 0 0 0 76085 135 0 0 20 0 3 0 275500910 494845952 97841 33554432000 4194304 5965102 140736489368656 140434702092280 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=5791/tid=5793] ppid=5789 vsize=483248 CPUtime=762.27 cores=1,3,5,7
/proc/5791/task/5793/stat : 5793 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 37037 0 0 0 76165 62 0 0 20 0 3 0 275500910 494845952 97841 33554432000 4194304 5965102 140736489368656 140434691602792 4400960 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1524.23
Current children cumulated vsize (KiB) 483248

[startup+822.3 s]
/proc/loadavg: 4.00 4.00 3.97 5/181 5800
/proc/meminfo: memFree=29473232/32873648 swapFree=7368/7368
[pid=5791] ppid=5789 vsize=484848 CPUtime=1644.18 cores=1,3,5,7
/proc/5791/stat : 5791 (pwbo2.0) S 5789 5791 3829 0 -1 4202496 99960 0 0 0 164234 184 0 0 20 0 3 0 275500910 496484352 101938 33554432000 4194304 5965102 140736489368656 140736489368176 4547341 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/5791/statm: 121212 101938 208 433 0 120776 0
[pid=5791/tid=5792] ppid=5789 vsize=484848 CPUtime=822.17 cores=1,3,5,7
/proc/5791/task/5792/stat : 5792 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 62154 0 0 0 82071 146 0 0 20 0 3 0 275500910 496484352 101938 33554432000 4194304 5965102 140736489368656 140434702092328 4221120 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=5791/tid=5793] ppid=5789 vsize=484848 CPUtime=822.26 cores=1,3,5,7
/proc/5791/task/5793/stat : 5793 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 37052 0 0 0 82163 63 0 0 20 0 3 0 275500910 496484352 101938 33554432000 4194304 5965102 140736489368656 140434691602792 4400960 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1644.18
Current children cumulated vsize (KiB) 484848

[startup+882.3 s]
/proc/loadavg: 3.91 3.97 3.96 5/181 5841
/proc/meminfo: memFree=29796932/32873648 swapFree=7368/7368
[pid=5791] ppid=5789 vsize=485700 CPUtime=1764.15 cores=1,3,5,7
/proc/5791/stat : 5791 (pwbo2.0) S 5789 5791 3829 0 -1 4202496 102012 0 0 0 176226 189 0 0 20 0 3 0 275500910 497356800 103990 33554432000 4194304 5965102 140736489368656 140736489368176 4547341 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/5791/statm: 121425 103990 208 433 0 120989 0
[pid=5791/tid=5792] ppid=5789 vsize=485700 CPUtime=882.17 cores=1,3,5,7
/proc/5791/task/5792/stat : 5792 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 64190 0 0 0 88064 153 0 0 20 0 3 0 275500910 497356800 103990 33554432000 4194304 5965102 140736489368656 140434702092328 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=5791/tid=5793] ppid=5789 vsize=485700 CPUtime=882.27 cores=1,3,5,7
/proc/5791/task/5793/stat : 5793 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 37068 0 0 0 88163 64 0 0 20 0 3 0 275500910 497356800 103990 33554432000 4194304 5965102 140736489368656 140434691602792 4401046 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1764.15
Current children cumulated vsize (KiB) 485700



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+900.3 s]
/proc/loadavg: 3.93 3.97 3.96 5/181 5841
/proc/meminfo: memFree=29781060/32873648 swapFree=7368/7368
[pid=5791] ppid=5789 vsize=485932 CPUtime=1800.16 cores=1,3,5,7
/proc/5791/stat : 5791 (pwbo2.0) S 5789 5791 3829 0 -1 4202496 102041 0 0 0 179827 189 0 0 20 0 3 0 275500910 497594368 104019 33554432000 4194304 5965102 140736489368656 140736489368176 4547341 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/5791/statm: 121483 104019 208 433 0 121047 0
[pid=5791/tid=5792] ppid=5789 vsize=485932 CPUtime=900.17 cores=1,3,5,7
/proc/5791/task/5792/stat : 5792 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 64203 0 0 0 89864 153 0 0 20 0 3 0 275500910 497594368 104019 33554432000 4194304 5965102 140736489368656 140434702092328 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=5791/tid=5793] ppid=5789 vsize=485932 CPUtime=900.28 cores=1,3,5,7
/proc/5791/task/5793/stat : 5793 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 37084 0 0 0 89964 64 0 0 20 0 3 0 275500910 497594368 104019 33554432000 4194304 5965102 140736489368656 140434691602792 4400960 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1800.16
Current children cumulated vsize (KiB) 485932

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+900.3 s]
/proc/loadavg: 3.93 3.97 3.96 5/181 5841
/proc/meminfo: memFree=29781060/32873648 swapFree=7368/7368
[pid=5791] ppid=5789 vsize=485932 CPUtime=1800.16 cores=1,3,5,7
/proc/5791/stat : 5791 (pwbo2.0) S 5789 5791 3829 0 -1 4202496 102041 0 0 0 179827 189 0 0 20 0 3 0 275500910 497594368 104019 33554432000 4194304 5965102 140736489368656 140736489368176 4547341 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/5791/statm: 121483 104019 208 433 0 121047 0
[pid=5791/tid=5792] ppid=5789 vsize=485932 CPUtime=900.17 cores=1,3,5,7
/proc/5791/task/5792/stat : 5792 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 64203 0 0 0 89864 153 0 0 20 0 3 0 275500910 497594368 104019 33554432000 4194304 5965102 140736489368656 140434702092328 4400960 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=5791/tid=5793] ppid=5789 vsize=485932 CPUtime=900.28 cores=1,3,5,7
/proc/5791/task/5793/stat : 5793 (pwbo2.0) R 5789 5791 3829 0 -1 4202560 37084 0 0 0 89964 64 0 0 20 0 3 0 275500910 497594368 104019 33554432000 4194304 5965102 140736489368656 140434691602792 4400960 0 0 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1800.16
Current children cumulated vsize (KiB) 485932

Child status: 0
Real time (s): 900.318
CPU time (s): 1800.5
CPU user time (s): 1798.31
CPU system time (s): 2.19067
CPU usage (%): 199.985
Max. virtual memory (cumulated for all children) (KiB): 485932

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1798.31
system time used= 2.19067
maximum resident set size= 416084
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 102043
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 414
involuntary context switches= 1954

runsolver used 1.9607 second user time and 5.07523 second system time

The end

Launcher Data 

Begin job on node149 at 2012-05-27 06:42:48
IDJOB=3703755
IDBENCH=2899
IDSOLVER=2279
FILE ID=node149/3703755-1338093768
RUNJOBID= node149-1338087336-3846
PBS_JOBID= 14624439
Free space on /tmp= 71708 MiB

SOLVER NAME= pwbo 2.0
BENCH NAME= PB06/final/normalized-PB06/OPT-SMALLINT/submitted-PB06/manquiho/market-split/normalized-opt-market-split_5_40_3.opb
COMMAND LINE= pwbo2.0 -file-format=opb -verbosity=0 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3703755-1338093768/watcher-3703755-1338093768 -o /tmp/evaluation-result-3703755-1338093768/solver-3703755-1338093768 -C 1800 -W 1900 -M 15500  pwbo2.0 -file-format=opb -verbosity=0 HOME/instance-3703755-1338093768.opb

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 455599f2e36aed56b051b7169c43fa6c
RANDOM SEED=636411012

node149.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.145
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.29
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.145
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.145
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5438.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.145
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.145
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.145
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5438.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.145
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.145
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873648 kB
MemFree:        30224844 kB
Buffers:          265556 kB
Cached:          1490272 kB
SwapCached:         2244 kB
Active:           774996 kB
Inactive:        1071552 kB
Active(anon):      83060 kB
Inactive(anon):     9052 kB
Active(file):     691936 kB
Inactive(file):  1062500 kB
Unevictable:        7368 kB
Mlocked:            7368 kB
SwapTotal:      67108856 kB
SwapFree:       67104796 kB
Dirty:              5492 kB
Writeback:             0 kB
AnonPages:         95056 kB
Mapped:            15016 kB
Shmem:                96 kB
Slab:             665632 kB
SReclaimable:     109744 kB
SUnreclaim:       555888 kB
KernelStack:        1472 kB
PageTables:         3776 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545680 kB
Committed_AS:     160308 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346400 kB
VmallocChunk:   34341988052 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7680 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 71700 MiB
End job on node149 at 2012-05-27 06:57:50