Trace number 3728725

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
pwbo 2.02MSAT (TO) 1800.43 900.315

General information on the benchmark

Name/PARTIAL-SMALLINT-LIN/wcsp/coloring/
normalized-queen5-5-3_wcsp.wbo
MD5SUM3af6143b2c580af9e4edcedcd162c161
Bench CategoryPARTIAL-SMALLINT-LIN (both soft and hard constraints, small integers, linear constraints)
Best result obtained on this benchmarkMSAT
Best cost obtained on this benchmark29
Best CPU time to get the best result obtained on this benchmark1797.3
Max-Satisfiable
Max-(Un)Satisfiability was proved
Best value of the cost
Optimality of the best cost was proved
Number of variables75
Total number of constraints505
Number of soft constraints480
Number of constraints which are clauses480
Number of constraints which are cardinality constraints (but not clauses)25
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint2
Maximum length of a constraint3
Top cost 161
Min constraint cost 1
Max constraint cost 1
Sum of constraints costs 480
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 4
Number of bits of the biggest sum of numbers3
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Quality of the solution as a function of time


Download the above graph as a PDF file / as an EPS file

Solver Data 

0.00/0.00	c ==============================[ Solver Version ]===============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  pwbo:                     2.02 beta - May, 2012                            |
0.00/0.00	c |                                                                             |
0.00/0.00	c Parsing wbo file format. File HOME/instance-3728725-1338675256.wbo.
0.00/0.00	c Instance file HOME/instance-3728725-1338675256.wbo
0.00/0.00	c File size is 13298 bytes.
0.00/0.00	c Highest Coefficient sum: 4
0.00/0.00	c ============================[ Problem Statistics ]=============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  Parsing time:          0.00         s                                      |
0.00/0.00	c |  Number Variables:      555                                                 |
0.00/0.00	c |  Number Clauses:        505                                                 |
0.00/0.00	c |  Number Cardinality:    25                                                  |
0.00/0.00	c |  Number PB Constraints: 0                                                   |
0.00/0.00	c |                                                                             |
0.00/0.00	c |  Problem Type:          Optimization                                        |
0.00/0.00	c |  Objective Function:    Cardinality Constraint                              |
0.00/0.00	c |                                                                             |
0.00/0.00	c ===============================================================================
1800.11/900.31	o 48
1800.11/900.31	o 46
1800.11/900.31	o 44
1800.11/900.31	o 43
1800.11/900.31	o 41
1800.11/900.31	o 40
1800.11/900.31	o 39
1800.11/900.31	o 38
1800.11/900.31	o 37
1800.11/900.31	o 36
1800.11/900.31	o 35
1800.11/900.31	o 34
1800.11/900.31	o 33
1800.11/900.31	o 32
1800.11/900.31	o 31
1800.11/900.31	o 30
1800.11/900.31	o 29
1800.11/900.31	c ===============================================================================
1800.11/900.31	c Wall Clock Time: 900.31 s
1800.11/900.31	c Best Solution: 29
1800.11/900.31	s SATISFIABLE
1800.11/900.31	v x1 -x2 -x3 -x4 -x5 x6 x7 -x8 -x9 -x10 x11 -x12 -x13 x14 -x15 -x16 x17 -x18 -x19 x20 -x21 x22 -x23 -x24 -x25 -x26 x27 x28 -x29 -x30 -x31 -x32 x33 x34 -x35 -x36 -x37 x38 -x39 -x40 x41 -x42 -x43 -x44 x45 -x46 -x47 x48 x49 -x50 -x51 -x52 -x53 x54 x55 -x56 -x57 -x58 x59 -x60 x61 -x62 -x63 -x64 x65 -x66 -x67 x68 -x69 x70 -x71 -x72 -x73 -x74 x75

Verifier Data 

OK	29

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3728725-1338675256/watcher-3728725-1338675256 -o /tmp/evaluation-result-3728725-1338675256/solver-3728725-1338675256 -C 1800 -W 1900 -M 15500 pwbo2.02 -file-format=wbo -verbosity=0 HOME/instance-3728725-1338675256.wbo 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.98 3.97 3.91 4/178 29414
/proc/meminfo: memFree=27705776/32873648 swapFree=7112/7112
[pid=29414] ppid=29412 vsize=162752 CPUtime=0 cores=0,2,4,6
/proc/29414/stat : 29414 (pwbo2.02) S 29412 29414 28397 0 -1 4202496 1080 0 0 0 0 0 0 0 20 0 3 0 333651510 166658048 1028 33554432000 4194304 5964478 140734357537040 140734357536560 4546685 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/29414/statm: 40688 1029 206 433 0 40252 0
[pid=29414/tid=29415] ppid=29412 vsize=162752 CPUtime=0 cores=0,2,4,6
/proc/29414/task/29415/stat : 29415 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 139 0 0 0 0 0 0 0 20 0 3 0 333651510 166658048 1039 33554432000 4194304 5964478 140734357537040 139798005652008 5136420 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=29414/tid=29416] ppid=29412 vsize=162752 CPUtime=0 cores=0,2,4,6
/proc/29414/task/29416/stat : 29416 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 148 0 0 0 0 0 0 0 20 0 3 0 333651510 166658048 1049 33554432000 4194304 5964478 140734357537040 139797995161160 5430298 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0

[startup+0.0719671 s]
/proc/loadavg: 3.98 3.97 3.91 4/178 29414
/proc/meminfo: memFree=27705776/32873648 swapFree=7112/7112
[pid=29414] ppid=29412 vsize=165512 CPUtime=0.13 cores=0,2,4,6
/proc/29414/stat : 29414 (pwbo2.02) S 29412 29414 28397 0 -1 4202496 2383 0 0 0 13 0 0 0 20 0 3 0 333651510 169484288 2331 33554432000 4194304 5964478 140734357537040 140734357536560 4546685 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/29414/statm: 41378 2331 207 433 0 40942 0
[pid=29414/tid=29415] ppid=29412 vsize=165512 CPUtime=0.05 cores=0,2,4,6
/proc/29414/task/29415/stat : 29415 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 264 0 0 0 5 0 0 0 20 0 3 0 333651510 169484288 2331 33554432000 4194304 5964478 140734357537040 139798005651992 4389085 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=29414/tid=29416] ppid=29412 vsize=165512 CPUtime=0.05 cores=0,2,4,6
/proc/29414/task/29416/stat : 29416 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 1306 0 0 0 5 0 0 0 20 0 3 0 333651510 169484288 2331 33554432000 4194304 5964478 140734357537040 139797995161304 4400661 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 0.13
Current children cumulated vsize (KiB) 165512

[startup+0.100254 s]
/proc/loadavg: 3.98 3.97 3.91 4/178 29414
/proc/meminfo: memFree=27705776/32873648 swapFree=7112/7112
[pid=29414] ppid=29412 vsize=165512 CPUtime=0.18 cores=0,2,4,6
/proc/29414/stat : 29414 (pwbo2.02) S 29412 29414 28397 0 -1 4202496 2420 0 0 0 18 0 0 0 20 0 3 0 333651510 169484288 2368 33554432000 4194304 5964478 140734357537040 140734357536560 4546685 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/29414/statm: 41378 2368 207 433 0 40942 0
[pid=29414/tid=29415] ppid=29412 vsize=165512 CPUtime=0.08 cores=0,2,4,6
/proc/29414/task/29415/stat : 29415 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 295 0 0 0 8 0 0 0 20 0 3 0 333651510 169484288 2368 33554432000 4194304 5964478 140734357537040 139798005651992 5136420 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=29414/tid=29416] ppid=29412 vsize=165512 CPUtime=0.08 cores=0,2,4,6
/proc/29414/task/29416/stat : 29416 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 1312 0 0 0 8 0 0 0 20 0 3 0 333651510 169484288 2368 33554432000 4194304 5964478 140734357537040 139797995161304 4399964 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 0.18
Current children cumulated vsize (KiB) 165512

[startup+0.300178 s]
/proc/loadavg: 3.98 3.97 3.91 4/178 29414
/proc/meminfo: memFree=27705776/32873648 swapFree=7112/7112
[pid=29414] ppid=29412 vsize=165512 CPUtime=0.58 cores=0,2,4,6
/proc/29414/stat : 29414 (pwbo2.02) S 29412 29414 28397 0 -1 4202496 2636 0 0 0 58 0 0 0 20 0 3 0 333651510 169484288 2584 33554432000 4194304 5964478 140734357537040 140734357536560 4546685 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/29414/statm: 41378 2584 207 433 0 40942 0
[pid=29414/tid=29415] ppid=29412 vsize=165512 CPUtime=0.28 cores=0,2,4,6
/proc/29414/task/29415/stat : 29415 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 454 0 0 0 28 0 0 0 20 0 3 0 333651510 169484288 2584 33554432000 4194304 5964478 140734357537040 139798005651992 4388622 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=29414/tid=29416] ppid=29412 vsize=165512 CPUtime=0.28 cores=0,2,4,6
/proc/29414/task/29416/stat : 29416 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 1369 0 0 0 28 0 0 0 20 0 3 0 333651510 169484288 2584 33554432000 4194304 5964478 140734357537040 139797995161304 4392738 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 0.58
Current children cumulated vsize (KiB) 165512

[startup+0.700184 s]
/proc/loadavg: 3.98 3.97 3.91 4/178 29414
/proc/meminfo: memFree=27705776/32873648 swapFree=7112/7112
[pid=29414] ppid=29412 vsize=165512 CPUtime=1.38 cores=0,2,4,6
/proc/29414/stat : 29414 (pwbo2.02) S 29412 29414 28397 0 -1 4202496 3053 0 0 0 138 0 0 0 20 0 3 0 333651510 169484288 3001 33554432000 4194304 5964478 140734357537040 140734357536560 4546685 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/29414/statm: 41378 3001 207 433 0 40942 0
[pid=29414/tid=29415] ppid=29412 vsize=165512 CPUtime=0.68 cores=0,2,4,6
/proc/29414/task/29415/stat : 29415 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 765 0 0 0 68 0 0 0 20 0 3 0 333651510 169484288 3001 33554432000 4194304 5964478 140734357537040 139798005651256 5136420 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=29414/tid=29416] ppid=29412 vsize=165512 CPUtime=0.68 cores=0,2,4,6
/proc/29414/task/29416/stat : 29416 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 1475 0 0 0 68 0 0 0 20 0 3 0 333651510 169484288 3001 33554432000 4194304 5964478 140734357537040 139797995161912 4403332 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1.38
Current children cumulated vsize (KiB) 165512

[startup+1.50019 s]
/proc/loadavg: 3.98 3.97 3.91 5/181 29417
/proc/meminfo: memFree=27693056/32873648 swapFree=7112/7112
[pid=29414] ppid=29412 vsize=165512 CPUtime=2.99 cores=0,2,4,6
/proc/29414/stat : 29414 (pwbo2.02) S 29412 29414 28397 0 -1 4202496 3731 0 0 0 298 1 0 0 20 0 3 0 333651510 169484288 3655 33554432000 4194304 5964478 140734357537040 140734357536560 4546685 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/29414/statm: 41378 3655 207 433 0 40942 0
[pid=29414/tid=29415] ppid=29412 vsize=165512 CPUtime=1.48 cores=0,2,4,6
/proc/29414/task/29415/stat : 29415 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 1241 0 0 0 148 0 0 0 20 0 3 0 333651510 169484288 3655 33554432000 4194304 5964478 140734357537040 139798005652408 4403671 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=29414/tid=29416] ppid=29412 vsize=165512 CPUtime=1.48 cores=0,2,4,6
/proc/29414/task/29416/stat : 29416 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 1677 0 0 0 148 0 0 0 20 0 3 0 333651510 169484288 3655 33554432000 4194304 5964478 140734357537040 139797995161960 4392863 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 2.99
Current children cumulated vsize (KiB) 165512

[startup+3.10027 s]
/proc/loadavg: 3.98 3.97 3.91 5/181 29417
/proc/meminfo: memFree=27690204/32873648 swapFree=7112/7112
[pid=29414] ppid=29412 vsize=165644 CPUtime=6.19 cores=0,2,4,6
/proc/29414/stat : 29414 (pwbo2.02) S 29412 29414 28397 0 -1 4202496 4540 0 0 0 618 1 0 0 20 0 3 0 333651510 169619456 4464 33554432000 4194304 5964478 140734357537040 140734357536560 4546685 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/29414/statm: 41411 4464 207 433 0 40975 0
[pid=29414/tid=29415] ppid=29412 vsize=165644 CPUtime=3.08 cores=0,2,4,6
/proc/29414/task/29415/stat : 29415 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 1687 0 0 0 308 0 0 0 20 0 3 0 333651510 169619456 4464 33554432000 4194304 5964478 140734357537040 139798005651448 4403575 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=29414/tid=29416] ppid=29412 vsize=165644 CPUtime=3.08 cores=0,2,4,6
/proc/29414/task/29416/stat : 29416 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 2040 0 0 0 308 0 0 0 20 0 3 0 333651510 169619456 4464 33554432000 4194304 5964478 140734357537040 139797995161960 4399854 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 6.19
Current children cumulated vsize (KiB) 165644

[startup+6.30029 s]
/proc/loadavg: 3.98 3.97 3.91 5/181 29417
/proc/meminfo: memFree=27678732/32873648 swapFree=7112/7112
[pid=29414] ppid=29412 vsize=166460 CPUtime=12.58 cores=0,2,4,6
/proc/29414/stat : 29414 (pwbo2.02) S 29412 29414 28397 0 -1 4202496 6712 0 0 0 1256 2 0 0 20 0 3 0 333651510 170455040 6636 33554432000 4194304 5964478 140734357537040 140734357536560 4546685 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/29414/statm: 41615 6636 207 433 0 41179 0
[pid=29414/tid=29415] ppid=29412 vsize=166460 CPUtime=6.28 cores=0,2,4,6
/proc/29414/task/29415/stat : 29415 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 3112 0 0 0 628 0 0 0 20 0 3 0 333651510 170455040 6636 33554432000 4194304 5964478 140734357537040 139798005651496 4389160 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=29414/tid=29416] ppid=29412 vsize=166460 CPUtime=6.28 cores=0,2,4,6
/proc/29414/task/29416/stat : 29416 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 2787 0 0 0 627 1 0 0 20 0 3 0 333651510 170455040 6636 33554432000 4194304 5964478 140734357537040 139797995161912 4431835 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 12.58
Current children cumulated vsize (KiB) 166460

[startup+12.7003 s]
/proc/loadavg: 3.98 3.97 3.91 5/181 29417
/proc/meminfo: memFree=27663016/32873648 swapFree=7112/7112
[pid=29414] ppid=29412 vsize=167012 CPUtime=25.38 cores=0,2,4,6
/proc/29414/stat : 29414 (pwbo2.02) S 29412 29414 28397 0 -1 4202496 10151 0 0 0 2534 4 0 0 20 0 3 0 333651510 171020288 10075 33554432000 4194304 5964478 140734357537040 140734357536560 4546685 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/29414/statm: 41753 10075 207 433 0 41317 0
[pid=29414/tid=29415] ppid=29412 vsize=167012 CPUtime=12.68 cores=0,2,4,6
/proc/29414/task/29415/stat : 29415 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 6062 0 0 0 1265 3 0 0 20 0 3 0 333651510 171020288 10075 33554432000 4194304 5964478 140734357537040 139798005651496 4400119 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=29414/tid=29416] ppid=29412 vsize=167012 CPUtime=12.68 cores=0,2,4,6
/proc/29414/task/29416/stat : 29416 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 3276 0 0 0 1266 2 0 0 20 0 3 0 333651510 171020288 10075 33554432000 4194304 5964478 140734357537040 139797995161912 4400087 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 25.38
Current children cumulated vsize (KiB) 167012

[startup+25.5002 s]
/proc/loadavg: 3.99 3.97 3.91 5/181 29417
/proc/meminfo: memFree=27642348/32873648 swapFree=7112/7112
[pid=29414] ppid=29412 vsize=167456 CPUtime=50.98 cores=0,2,4,6
/proc/29414/stat : 29414 (pwbo2.02) S 29412 29414 28397 0 -1 4202496 13961 0 0 0 5091 7 0 0 20 0 3 0 333651510 171474944 13885 33554432000 4194304 5964478 140734357537040 140734357536560 4546685 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/29414/statm: 41864 13885 207 433 0 41428 0
[pid=29414/tid=29415] ppid=29412 vsize=167456 CPUtime=25.49 cores=0,2,4,6
/proc/29414/task/29415/stat : 29415 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 9192 0 0 0 2543 6 0 0 20 0 3 0 333651510 171474944 13885 33554432000 4194304 5964478 140734357537040 139798005651496 5136420 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=29414/tid=29416] ppid=29412 vsize=167456 CPUtime=25.48 cores=0,2,4,6
/proc/29414/task/29416/stat : 29416 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 3956 0 0 0 2546 2 0 0 20 0 3 0 333651510 171474944 13885 33554432000 4194304 5964478 140734357537040 139797995161912 4399975 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 50.98
Current children cumulated vsize (KiB) 167456

[startup+51.1003 s]
/proc/loadavg: 3.99 3.97 3.91 5/181 29418
/proc/meminfo: memFree=27610112/32873648 swapFree=7112/7112
[pid=29414] ppid=29412 vsize=233848 CPUtime=102.17 cores=0,2,4,6
/proc/29414/stat : 29414 (pwbo2.02) S 29412 29414 28397 0 -1 4202496 22274 0 0 0 10205 12 0 0 20 0 3 0 333651510 239460352 22198 33554432000 4194304 5964478 140734357537040 140734357536560 4546685 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/29414/statm: 58462 22198 207 433 0 58026 0
[pid=29414/tid=29415] ppid=29412 vsize=233848 CPUtime=51.08 cores=0,2,4,6
/proc/29414/task/29415/stat : 29415 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 16582 0 0 0 5098 10 0 0 20 0 3 0 333651510 239460352 22198 33554432000 4194304 5964478 140734357537040 139798005651256 4222016 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=29414/tid=29416] ppid=29412 vsize=233848 CPUtime=51.08 cores=0,2,4,6
/proc/29414/task/29416/stat : 29416 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 4879 0 0 0 5105 3 0 0 20 0 3 0 333651510 239460352 22198 33554432000 4194304 5964478 140734357537040 139797995161912 4400621 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 102.17
Current children cumulated vsize (KiB) 233848

[startup+102.307 s]
/proc/loadavg: 3.99 3.97 3.91 5/181 29418
/proc/meminfo: memFree=27538920/32873648 swapFree=7112/7112
[pid=29414] ppid=29412 vsize=234852 CPUtime=204.56 cores=0,2,4,6
/proc/29414/stat : 29414 (pwbo2.02) S 29412 29414 28397 0 -1 4202496 26548 0 0 0 20440 16 0 0 20 0 3 0 333651510 240488448 26277 33554432000 4194304 5964478 140734357537040 140734357536560 4546685 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/29414/statm: 58713 26277 207 433 0 58277 0
[pid=29414/tid=29415] ppid=29412 vsize=234852 CPUtime=102.28 cores=0,2,4,6
/proc/29414/task/29415/stat : 29415 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 17024 0 0 0 10217 11 0 0 20 0 3 0 333651510 240488448 26277 33554432000 4194304 5964478 140734357537040 139798005652408 4389160 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=29414/tid=29416] ppid=29412 vsize=234852 CPUtime=102.29 cores=0,2,4,6
/proc/29414/task/29416/stat : 29416 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 8711 0 0 0 10221 8 0 0 20 0 3 0 333651510 240488448 26277 33554432000 4194304 5964478 140734357537040 139797995161912 4409220 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 204.56
Current children cumulated vsize (KiB) 234852

[startup+162.3 s]
/proc/loadavg: 3.99 3.97 3.91 5/181 29418
/proc/meminfo: memFree=27423512/32873648 swapFree=7112/7112
[pid=29414] ppid=29412 vsize=237868 CPUtime=324.5 cores=0,2,4,6
/proc/29414/stat : 29414 (pwbo2.02) S 29412 29414 28397 0 -1 4202496 40855 0 0 0 32421 29 0 0 20 0 3 0 333651510 243576832 40815 33554432000 4194304 5964478 140734357537040 140734357536560 4546685 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/29414/statm: 59467 40815 207 433 0 59031 0
[pid=29414/tid=29415] ppid=29412 vsize=237868 CPUtime=162.26 cores=0,2,4,6
/proc/29414/task/29415/stat : 29415 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 27919 0 0 0 16205 21 0 0 20 0 3 0 333651510 243576832 40815 33554432000 4194304 5964478 140734357537040 139798005651256 4222464 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=29414/tid=29416] ppid=29412 vsize=237868 CPUtime=162.27 cores=0,2,4,6
/proc/29414/task/29416/stat : 29416 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 12123 0 0 0 16213 14 0 0 20 0 3 0 333651510 243576832 40815 33554432000 4194304 5964478 140734357537040 139797995161912 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 324.5
Current children cumulated vsize (KiB) 237868

[startup+222.3 s]
/proc/loadavg: 3.99 3.97 3.91 5/181 29418
/proc/meminfo: memFree=27343580/32873648 swapFree=7112/7112
[pid=29414] ppid=29412 vsize=303924 CPUtime=444.45 cores=0,2,4,6
/proc/29414/stat : 29414 (pwbo2.02) S 29412 29414 28397 0 -1 4202496 53057 0 0 0 44405 40 0 0 20 0 3 0 333651510 311218176 54326 33554432000 4194304 5964478 140734357537040 140734357536560 4546685 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/29414/statm: 75981 54326 207 433 0 75545 0
[pid=29414/tid=29415] ppid=29412 vsize=303924 CPUtime=222.23 cores=0,2,4,6
/proc/29414/task/29415/stat : 29415 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 39876 0 0 0 22191 32 0 0 20 0 3 0 333651510 311218176 54326 33554432000 4194304 5964478 140734357537040 139798005651448 4399986 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=29414/tid=29416] ppid=29412 vsize=303924 CPUtime=222.27 cores=0,2,4,6
/proc/29414/task/29416/stat : 29416 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 12368 0 0 0 22212 15 0 0 20 0 3 0 333651510 311218176 54326 33554432000 4194304 5964478 140734357537040 139797995161672 4399854 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 444.45
Current children cumulated vsize (KiB) 303924

################
# More data... #
################

/proc/29414/task/29415/stat : 29415 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 45858 0 0 0 28183 40 0 0 20 0 3 0 333651510 312553472 64189 33554432000 4194304 5964478 140734357537040 139798005651496 4220672 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=29414/tid=29416] ppid=29412 vsize=305228 CPUtime=282.28 cores=0,2,4,6
/proc/29414/task/29416/stat : 29416 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 16249 0 0 0 28207 21 0 0 20 0 3 0 333651510 312553472 64189 33554432000 4194304 5964478 140734357537040 139797995161912 4400001 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 564.44
Current children cumulated vsize (KiB) 305228

[startup+342.3 s]
/proc/loadavg: 3.99 3.97 3.91 5/181 29418
/proc/meminfo: memFree=27148908/32873648 swapFree=7112/7112
[pid=29414] ppid=29412 vsize=372468 CPUtime=684.42 cores=0,2,4,6
/proc/29414/stat : 29414 (pwbo2.02) S 29412 29414 28397 0 -1 4202496 71803 0 0 0 68380 62 0 0 20 0 3 0 333651510 381407232 73072 33554432000 4194304 5964478 140734357537040 140734357536560 4546685 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/29414/statm: 93117 73072 207 433 0 92681 0
[pid=29414/tid=29415] ppid=29412 vsize=372468 CPUtime=342.23 cores=0,2,4,6
/proc/29414/task/29415/stat : 29415 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 53986 0 0 0 34173 50 0 0 20 0 3 0 333651510 381407232 73072 33554432000 4194304 5964478 140734357537040 139798005651496 4389170 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=29414/tid=29416] ppid=29412 vsize=372468 CPUtime=342.28 cores=0,2,4,6
/proc/29414/task/29416/stat : 29416 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 17004 0 0 0 34205 23 0 0 20 0 3 0 333651510 381407232 73072 33554432000 4194304 5964478 140734357537040 139797995161720 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 684.42
Current children cumulated vsize (KiB) 372468

[startup+402.3 s]
/proc/loadavg: 3.99 3.97 3.91 5/182 29422
/proc/meminfo: memFree=27089736/32873648 swapFree=7112/7112
[pid=29414] ppid=29412 vsize=373108 CPUtime=804.39 cores=0,2,4,6
/proc/29414/stat : 29414 (pwbo2.02) S 29412 29414 28397 0 -1 4202496 77386 0 0 0 80371 68 0 0 20 0 3 0 333651510 382062592 78652 33554432000 4194304 5964478 140734357537040 140734357536560 4546685 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/29414/statm: 93277 78652 207 433 0 92841 0
[pid=29414/tid=29415] ppid=29412 vsize=373108 CPUtime=402.23 cores=0,2,4,6
/proc/29414/task/29415/stat : 29415 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 59427 0 0 0 40166 57 0 0 20 0 3 0 333651510 382062592 78652 33554432000 4194304 5964478 140734357537040 139798005651256 4220672 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=29414/tid=29416] ppid=29412 vsize=373108 CPUtime=402.28 cores=0,2,4,6
/proc/29414/task/29416/stat : 29416 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 17146 0 0 0 40204 24 0 0 20 0 3 0 333651510 382062592 78652 33554432000 4194304 5964478 140734357537040 139797995161672 4399938 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 804.39
Current children cumulated vsize (KiB) 373108

[startup+462.3 s]
/proc/loadavg: 3.99 3.97 3.91 5/181 29422
/proc/meminfo: memFree=27021056/32873648 swapFree=7112/7112
[pid=29414] ppid=29412 vsize=505248 CPUtime=924.37 cores=0,2,4,6
/proc/29414/stat : 29414 (pwbo2.02) S 29412 29414 28397 0 -1 4202496 86513 0 0 0 92357 80 0 0 20 0 3 0 333651510 517373952 87928 33554432000 4194304 5964478 140734357537040 140734357536560 4546685 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/29414/statm: 126312 87928 207 433 0 125876 0
[pid=29414/tid=29415] ppid=29412 vsize=505248 CPUtime=462.22 cores=0,2,4,6
/proc/29414/task/29415/stat : 29415 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 66156 0 0 0 46155 67 0 0 20 0 3 0 333651510 517373952 87928 33554432000 4194304 5964478 140734357537040 139798005651448 4399964 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=29414/tid=29416] ppid=29412 vsize=505248 CPUtime=462.28 cores=0,2,4,6
/proc/29414/task/29416/stat : 29416 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 19544 0 0 0 46200 28 0 0 20 0 3 0 333651510 517373952 87928 33554432000 4194304 5964478 140734357537040 139797995161912 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 924.37
Current children cumulated vsize (KiB) 505248

[startup+522.3 s]
/proc/loadavg: 3.99 3.97 3.91 5/181 29422
/proc/meminfo: memFree=26983512/32873648 swapFree=7112/7112
[pid=29414] ppid=29412 vsize=505460 CPUtime=1044.3 cores=0,2,4,6
/proc/29414/stat : 29414 (pwbo2.02) S 29412 29414 28397 0 -1 4202496 95166 0 0 0 104342 88 0 0 20 0 3 0 333651510 517591040 96931 33554432000 4194304 5964478 140734357537040 140734357536560 4546685 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/29414/statm: 126365 96931 207 433 0 125929 0
[pid=29414/tid=29415] ppid=29412 vsize=505460 CPUtime=522.18 cores=0,2,4,6
/proc/29414/task/29415/stat : 29415 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 73791 0 0 0 52143 75 0 0 20 0 3 0 333651510 517591040 96931 33554432000 4194304 5964478 140734357537040 139798005651448 4399986 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=29414/tid=29416] ppid=29412 vsize=505460 CPUtime=522.28 cores=0,2,4,6
/proc/29414/task/29416/stat : 29416 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 20562 0 0 0 52199 29 0 0 20 0 3 0 333651510 517591040 96931 33554432000 4194304 5964478 140734357537040 139797995161672 4399828 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1044.3
Current children cumulated vsize (KiB) 505460

[startup+582.3 s]
/proc/loadavg: 3.99 3.97 3.91 5/181 29422
/proc/meminfo: memFree=26949000/32873648 swapFree=7112/7112
[pid=29414] ppid=29412 vsize=505460 CPUtime=1164.27 cores=0,2,4,6
/proc/29414/stat : 29414 (pwbo2.02) S 29412 29414 28397 0 -1 4202496 101947 0 0 0 116331 96 0 0 20 0 3 0 333651510 517591040 103708 33554432000 4194304 5964478 140734357537040 140734357536560 4546685 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/29414/statm: 126365 103708 207 433 0 125929 0
[pid=29414/tid=29415] ppid=29412 vsize=505460 CPUtime=582.18 cores=0,2,4,6
/proc/29414/task/29415/stat : 29415 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 80554 0 0 0 58134 84 0 0 20 0 3 0 333651510 517591040 103708 33554432000 4194304 5964478 140734357537040 139798005651448 4389129 0 0 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=29414/tid=29416] ppid=29412 vsize=505460 CPUtime=582.27 cores=0,2,4,6
/proc/29414/task/29416/stat : 29416 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 20580 0 0 0 58197 30 0 0 20 0 3 0 333651510 517591040 103708 33554432000 4194304 5964478 140734357537040 139797995161672 4399938 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1164.27
Current children cumulated vsize (KiB) 505460

[startup+642.3 s]
/proc/loadavg: 3.99 3.97 3.91 5/181 29422
/proc/meminfo: memFree=26903784/32873648 swapFree=7112/7112
[pid=29414] ppid=29412 vsize=571232 CPUtime=1284.26 cores=0,2,4,6
/proc/29414/stat : 29414 (pwbo2.02) S 29412 29414 28397 0 -1 4202496 111197 0 0 0 128320 106 0 0 20 0 3 0 333651510 584941568 112960 33554432000 4194304 5964478 140734357537040 140734357536560 4546685 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/29414/statm: 142808 112960 207 433 0 142372 0
[pid=29414/tid=29415] ppid=29412 vsize=571232 CPUtime=642.17 cores=0,2,4,6
/proc/29414/task/29415/stat : 29415 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 89749 0 0 0 64123 94 0 0 20 0 3 0 333651510 584941568 112960 33554432000 4194304 5964478 140734357537040 139798005651448 4388838 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=29414/tid=29416] ppid=29412 vsize=571232 CPUtime=642.28 cores=0,2,4,6
/proc/29414/task/29416/stat : 29416 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 20635 0 0 0 64197 31 0 0 20 0 3 0 333651510 584941568 112960 33554432000 4194304 5964478 140734357537040 139797995161672 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1284.26
Current children cumulated vsize (KiB) 571232

[startup+702.3 s]
/proc/loadavg: 4.03 3.99 3.91 5/181 29423
/proc/meminfo: memFree=26861508/32873648 swapFree=7112/7112
[pid=29414] ppid=29412 vsize=572104 CPUtime=1404.23 cores=0,2,4,6
/proc/29414/stat : 29414 (pwbo2.02) S 29412 29414 28397 0 -1 4202496 118866 0 0 0 140307 116 0 0 20 0 3 0 333651510 585834496 120723 33554432000 4194304 5964478 140734357537040 140734357536560 4546685 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/29414/statm: 143026 120723 207 433 0 142590 0
[pid=29414/tid=29415] ppid=29412 vsize=572104 CPUtime=702.16 cores=0,2,4,6
/proc/29414/task/29415/stat : 29415 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 95868 0 0 0 70113 103 0 0 20 0 3 0 333651510 585834496 120723 33554432000 4194304 5964478 140734357537040 139798005651496 4399920 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=29414/tid=29416] ppid=29412 vsize=572104 CPUtime=702.27 cores=0,2,4,6
/proc/29414/task/29416/stat : 29416 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 22185 0 0 0 70193 34 0 0 20 0 3 0 333651510 585834496 120723 33554432000 4194304 5964478 140734357537040 139797995161912 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1404.23
Current children cumulated vsize (KiB) 572104

[startup+762.307 s]
/proc/loadavg: 4.07 4.00 3.92 5/181 29423
/proc/meminfo: memFree=26815556/32873648 swapFree=7112/7112
[pid=29414] ppid=29412 vsize=637840 CPUtime=1524.22 cores=0,2,4,6
/proc/29414/stat : 29414 (pwbo2.02) S 29412 29414 28397 0 -1 4202496 125400 0 0 0 152298 124 0 0 20 0 3 0 333651510 653148160 127255 33554432000 4194304 5964478 140734357537040 140734357536560 4546685 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/29414/statm: 159460 127255 207 433 0 159024 0
[pid=29414/tid=29415] ppid=29412 vsize=637840 CPUtime=762.17 cores=0,2,4,6
/proc/29414/task/29415/stat : 29415 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 101276 0 0 0 76106 111 0 0 20 0 3 0 333651510 653148160 127255 33554432000 4194304 5964478 140734357537040 139798005651320 4400542 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=29414/tid=29416] ppid=29412 vsize=637840 CPUtime=762.28 cores=0,2,4,6
/proc/29414/task/29416/stat : 29416 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 23311 0 0 0 76192 36 0 0 20 0 3 0 333651510 653148160 127255 33554432000 4194304 5964478 140734357537040 139797995161672 4400435 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1524.22
Current children cumulated vsize (KiB) 637840

[startup+822.3 s]
/proc/loadavg: 2.82 3.66 3.81 3/173 29448
/proc/meminfo: memFree=27317628/32873648 swapFree=7112/7112
[pid=29414] ppid=29412 vsize=639192 CPUtime=1644.16 cores=0,2,4,6
/proc/29414/stat : 29414 (pwbo2.02) S 29412 29414 28397 0 -1 4202496 133300 0 0 0 164282 134 0 0 20 0 3 0 333651510 654532608 135465 33554432000 4194304 5964478 140734357537040 140734357536560 4546685 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/29414/statm: 159798 135465 207 433 0 159362 0
[pid=29414/tid=29415] ppid=29412 vsize=639192 CPUtime=822.14 cores=0,2,4,6
/proc/29414/task/29415/stat : 29415 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 108833 0 0 0 82093 121 0 0 20 0 3 0 333651510 654532608 135465 33554432000 4194304 5964478 140734357537040 139798005651448 4399920 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=29414/tid=29416] ppid=29412 vsize=639192 CPUtime=822.28 cores=0,2,4,6
/proc/29414/task/29416/stat : 29416 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 23654 0 0 0 82190 38 0 0 20 0 3 0 333651510 654532608 135465 33554432000 4194304 5964478 140734357537040 139797995161672 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1644.16
Current children cumulated vsize (KiB) 639192

[startup+882.3 s]
/proc/loadavg: 2.30 3.36 3.69 3/173 29448
/proc/meminfo: memFree=27308792/32873648 swapFree=7112/7112
[pid=29414] ppid=29412 vsize=639832 CPUtime=1764.11 cores=0,2,4,6
/proc/29414/stat : 29414 (pwbo2.02) S 29412 29414 28397 0 -1 4202496 135436 0 0 0 176276 135 0 0 20 0 3 0 333651510 655187968 137797 33554432000 4194304 5964478 140734357537040 140734357536560 4546685 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/29414/statm: 159958 137797 207 433 0 159522 0
[pid=29414/tid=29415] ppid=29412 vsize=639832 CPUtime=882.11 cores=0,2,4,6
/proc/29414/task/29415/stat : 29415 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 110840 0 0 0 88087 124 0 0 20 0 3 0 333651510 655187968 137797 33554432000 4194304 5964478 140734357537040 139798005651496 4399920 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=29414/tid=29416] ppid=29412 vsize=639832 CPUtime=882.27 cores=0,2,4,6
/proc/29414/task/29416/stat : 29416 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 23783 0 0 0 88188 39 0 0 20 0 3 0 333651510 655187968 137797 33554432000 4194304 5964478 140734357537040 139797995161672 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1764.11
Current children cumulated vsize (KiB) 639832



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+900.3 s]
/proc/loadavg: 2.21 3.27 3.65 3/173 29448
/proc/meminfo: memFree=27298420/32873648 swapFree=7112/7112
[pid=29414] ppid=29412 vsize=639832 CPUtime=1800.11 cores=0,2,4,6
/proc/29414/stat : 29414 (pwbo2.02) S 29412 29414 28397 0 -1 4202496 137926 0 0 0 179873 138 0 0 20 0 3 0 333651510 655187968 140307 33554432000 4194304 5964478 140734357537040 140734357536560 4546685 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/29414/statm: 159958 140307 207 433 0 159522 0
[pid=29414/tid=29415] ppid=29412 vsize=639832 CPUtime=900.11 cores=0,2,4,6
/proc/29414/task/29415/stat : 29415 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 113330 0 0 0 89885 126 0 0 20 0 3 0 333651510 655187968 140307 33554432000 4194304 5964478 140734357537040 139798005651448 4399920 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=29414/tid=29416] ppid=29412 vsize=639832 CPUtime=900.28 cores=0,2,4,6
/proc/29414/task/29416/stat : 29416 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 23783 0 0 0 89989 39 0 0 20 0 3 0 333651510 655187968 140307 33554432000 4194304 5964478 140734357537040 139797995161672 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1800.11
Current children cumulated vsize (KiB) 639832

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+900.3 s]
/proc/loadavg: 2.21 3.27 3.65 3/173 29448
/proc/meminfo: memFree=27298420/32873648 swapFree=7112/7112
[pid=29414] ppid=29412 vsize=639832 CPUtime=1800.11 cores=0,2,4,6
/proc/29414/stat : 29414 (pwbo2.02) S 29412 29414 28397 0 -1 4202496 137926 0 0 0 179873 138 0 0 20 0 3 0 333651510 655187968 140307 33554432000 4194304 5964478 140734357537040 140734357536560 4546685 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/29414/statm: 159958 140307 207 433 0 159522 0
[pid=29414/tid=29415] ppid=29412 vsize=639832 CPUtime=900.11 cores=0,2,4,6
/proc/29414/task/29415/stat : 29415 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 113330 0 0 0 89885 126 0 0 20 0 3 0 333651510 655187968 140307 33554432000 4194304 5964478 140734357537040 139798005651448 4399920 0 0 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=29414/tid=29416] ppid=29412 vsize=639832 CPUtime=900.28 cores=0,2,4,6
/proc/29414/task/29416/stat : 29416 (pwbo2.02) R 29412 29414 28397 0 -1 4202560 23783 0 0 0 89989 39 0 0 20 0 3 0 333651510 655187968 140307 33554432000 4194304 5964478 140734357537040 139797995161672 4399920 0 0 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1800.11
Current children cumulated vsize (KiB) 639832

Child status: 0
Real time (s): 900.315
CPU time (s): 1800.43
CPU user time (s): 1798.76
CPU system time (s): 1.67174
CPU usage (%): 199.978
Max. virtual memory (cumulated for all children) (KiB): 639832

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1798.76
system time used= 1.67175
maximum resident set size= 561232
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 137927
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 364
involuntary context switches= 2437

runsolver used 1.9407 second user time and 5.10522 second system time

The end

Launcher Data 

Begin job on node126 at 2012-06-03 00:14:16
IDJOB=3728725
IDBENCH=79662
IDSOLVER=2316
FILE ID=node126/3728725-1338675256
RUNJOBID= node126-1338671112-28628
PBS_JOBID= 14636504
Free space on /tmp= 70784 MiB

SOLVER NAME= pwbo 2.02
BENCH NAME= PB10/normalized-PB10/PARTIAL-SMALLINT-LIN/wcsp/coloring/normalized-queen5-5-3_wcsp.wbo
COMMAND LINE= pwbo2.02 -file-format=wbo -verbosity=0 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3728725-1338675256/watcher-3728725-1338675256 -o /tmp/evaluation-result-3728725-1338675256/solver-3728725-1338675256 -C 1800 -W 1900 -M 15500  pwbo2.02 -file-format=wbo -verbosity=0 HOME/instance-3728725-1338675256.wbo

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 3af6143b2c580af9e4edcedcd162c161
RANDOM SEED=1761739702

node126.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.456
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.456
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.456
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.456
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.456
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.456
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.456
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.456
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873648 kB
MemFree:        27705768 kB
Buffers:          398820 kB
Cached:          3748764 kB
SwapCached:         9476 kB
Active:          1606476 kB
Inactive:        2702492 kB
Active(anon):     151188 kB
Inactive(anon):    12180 kB
Active(file):    1455288 kB
Inactive(file):  2690312 kB
Unevictable:        7112 kB
Mlocked:            7112 kB
SwapTotal:      67108856 kB
SwapFree:       67094928 kB
Dirty:              5528 kB
Writeback:             0 kB
AnonPages:        159564 kB
Mapped:            10440 kB
Shmem:                 0 kB
Slab:             721780 kB
SReclaimable:     166104 kB
SUnreclaim:       555676 kB
KernelStack:        1496 kB
PageTables:         4356 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545680 kB
Committed_AS:     166904 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346400 kB
VmallocChunk:   34341988052 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7680 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70784 MiB
End job on node126 at 2012-06-03 00:29:18