Trace number 3712470

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
pwbo 2.01OPTIMUM 1319.77 660.151

General information on the benchmark

Name/PARTIAL-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/tsp/
normalized-t3002.11tsp11.1900567024--soft-33-100-0.wbo
MD5SUM05d9157cdbd186fca72e4e6dc0ec425d
Bench CategoryPARTIAL-SMALLINT-LIN (both soft and hard constraints, small integers, linear constraints)
Best result obtained on this benchmarkMOPT
Best cost obtained on this benchmark6
Best CPU time to get the best result obtained on this benchmark32.1441
Max-Satisfiable
Max-(Un)Satisfiability was proved
Best value of the cost
Optimality of the best cost was proved
Number of variables231
Total number of constraints2707
Number of soft constraints1839
Number of constraints which are clauses2684
Number of constraints which are cardinality constraints (but not clauses)22
Number of constraints which are nor clauses,nor cardinality constraints1
Minimum length of a constraint2
Maximum length of a constraint110
Top cost 91769
Min constraint cost 1
Max constraint cost 100
Sum of constraints costs 91768
Biggest number in a constraint 25
Number of bits of the biggest number in a constraint 5
Biggest sum of numbers in a constraint 755
Number of bits of the biggest sum of numbers10
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Quality of the solution as a function of time


Download the above graph as a PDF file / as an EPS file

Solver Data 

0.00/0.00	c ==============================[ Solver Version ]===============================
0.00/0.00	c |                                                                             |
0.00/0.00	c |  pwbo:                    2.01 beta - May, 2012                            |
0.00/0.00	c |                                                                             |
0.00/0.00	c Parsing wbo file format. File HOME/instance-3712470-1338198062.wbo.
0.00/0.00	c Instance file HOME/instance-3712470-1338198062.wbo
0.00/0.00	c File size is 92502 bytes.
0.00/0.01	c Highest Coefficient sum: 755
0.00/0.01	c ============================[ Problem Statistics ]=============================
0.00/0.01	c |                                                                             |
0.00/0.01	c |  Parsing time:          0.01         s                                      |
0.00/0.01	c |  Number Variables:      2070                                                |
0.00/0.01	c |  Number Clauses:        2684                                                |
0.00/0.01	c |  Number Cardinality:    9                                                   |
0.00/0.01	c |  Number PB Constraints: 14                                                  |
0.00/0.01	c |                                                                             |
0.00/0.01	c |  Problem Type:          Optimization                                        |
0.00/0.01	c |  Objective Function:    PB Constraint                                       |
0.00/0.01	c |                                                                             |
0.00/0.01	c ===============================================================================
1319.77/660.11	o 705
1319.77/660.11	o 613
1319.77/660.11	o 600
1319.77/660.11	o 414
1319.77/660.11	o 411
1319.77/660.11	o 335
1319.77/660.11	o 334
1319.77/660.11	o 308
1319.77/660.11	o 305
1319.77/660.11	o 261
1319.77/660.11	o 256
1319.77/660.11	o 241
1319.77/660.11	o 236
1319.77/660.11	o 229
1319.77/660.11	o 226
1319.77/660.11	o 203
1319.77/660.11	o 201
1319.77/660.11	o 195
1319.77/660.11	o 185
1319.77/660.11	o 181
1319.77/660.11	o 177
1319.77/660.11	o 171
1319.77/660.11	o 163
1319.77/660.11	o 158
1319.77/660.11	o 134
1319.77/660.11	o 133
1319.77/660.11	o 117
1319.77/660.11	o 110
1319.77/660.11	o 109
1319.77/660.11	o 107
1319.77/660.11	o 103
1319.77/660.11	o 88
1319.77/660.11	o 80
1319.77/660.11	o 75
1319.77/660.11	o 66
1319.77/660.11	o 63
1319.77/660.11	o 44
1319.77/660.11	o 37
1319.77/660.11	o 34
1319.77/660.11	o 32
1319.77/660.11	o 25
1319.77/660.11	o 19
1319.77/660.11	o 11
1319.77/660.11	o 10
1319.77/660.11	o 8
1319.77/660.11	o 7
1319.77/660.11	o 6
1319.77/660.11	c ===============================================================================
1319.77/660.11	c Wall Clock Time: 660.12 s
1319.77/660.11	c Best Solution: 6
1319.77/660.11	s OPTIMUM FOUND
1319.77/660.11	v -x1 -x2 -x3 -x4 -x5 x6 -x7 -x8 -x9 -x10 x11 -x12 -x13 -x14 -x15 -x16 -x17 -x18 -x19 -x20 -x21 -x22 -x23 -x24 -x25 -x26 -x27 -x28 -x29 -x30 x31 -x32 -x33 -x34 -x35 -x36 -x37 x38 -x39 -x40 -x41 -x42 -x43 -x44 -x45 -x46 -x47 x48 -x49 -x50 -x51 -x52 -x53 -x54 -x55 x56 -x57 -x58 -x59 -x60 -x61 -x62 -x63 -x64 -x65 -x66 -x67 -x68 -x69 -x70 -x71 -x72 x73 -x74 -x75 -x76 -x77 -x78 -x79 -x80 -x81 -x82 -x83 -x84 x85 -x86 -x87 -x88 -x89 -x90 -x91 -x92 -x93 -x94 -x95 -x96 -x97 x98 -x99 -x100 x101 -x102 -x103 -x104 -x105 -x106 -x107 -x108 -x109 -x110 -x111 -x112 -x113 -x114 -x115 -x116 -x117 -x118 -x119 -x120 x121 -x122 -x123 x124 -x125 -x126 -x127 -x128 -x129 -x130 -x131 -x132 -x133 -x134 -x135 -x136 -x137 -x138 -x139 -x140 -x141 -x142 -x143 -x144 -x145 x146 -x147 -x148 -x149 -x150 -x151 -x152 -x153 -x154 -x155 -x156 -x157 -x158 -x159 -x160 -x161 -x162 -x163 -x164 -x165 -x166 -x167 -x168 x169 -x170 -x171 -x172 -x173 -x174 -x175 -x176 -x177 -x178 -x179 -x180 x181 x182 -x183 -x184 -x185 -x186 -x187 -x188 -x189 -x190 -x191 -x192 -x193 -x194 -x195 -x196 -x197 x198 -x199 -x200 -x201 -x202 -x203 x204 -x205 -x206 -x207 -x208 -x209 -x210 -x211 -x212 -x213 -x214 -x215 -x216 -x217 -x218 x219 -x220 -x221 -x222 x223 -x224 -x225 -x226 -x227 -x228 -x229 -x230 -x231

Verifier Data 

OK	6

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3712470-1338198062/watcher-3712470-1338198062 -o /tmp/evaluation-result-3712470-1338198062/solver-3712470-1338198062 -C 1800 -W 1900 -M 15500 pwbo2.01 -file-format=wbo -verbosity=0 HOME/instance-3712470-1338198062.wbo 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.01 4.02 3.94 4/180 23384
/proc/meminfo: memFree=29890224/32873844 swapFree=7112/7112
[pid=23384] ppid=23382 vsize=4312 CPUtime=0 cores=1,3,5,7
/proc/23384/stat : 23384 (pwbo2.01) R 23382 23384 22874 0 -1 4202496 392 0 0 0 0 0 0 0 20 0 1 0 285931521 4415488 344 33554432000 4194304 5964478 140736455174544 140733712504552 4360528 0 0 4096 16387 0 0 0 17 5 0 0 0 0 0
/proc/23384/statm: 1078 344 166 433 0 619 0

[startup+0.063901 s]
/proc/loadavg: 4.01 4.02 3.94 4/180 23384
/proc/meminfo: memFree=29890224/32873844 swapFree=7112/7112
[pid=23384] ppid=23382 vsize=163768 CPUtime=0.1 cores=1,3,5,7
/proc/23384/stat : 23384 (pwbo2.01) S 23382 23384 22874 0 -1 4202496 2510 0 0 0 10 0 0 0 20 0 3 0 285931521 167698432 1996 33554432000 4194304 5964478 140736455174544 140736455174064 4546669 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23384/statm: 40942 1996 198 433 0 40506 0
Current children cumulated CPU time (s) 0.1
Current children cumulated vsize (KiB) 163768

[startup+0.100199 s]
/proc/loadavg: 4.01 4.02 3.94 4/180 23384
/proc/meminfo: memFree=29890224/32873844 swapFree=7112/7112
[pid=23384] ppid=23382 vsize=163768 CPUtime=0.18 cores=1,3,5,7
/proc/23384/stat : 23384 (pwbo2.01) S 23382 23384 22874 0 -1 4202496 2818 0 0 0 17 1 0 0 20 0 3 0 285931521 167698432 2131 33554432000 4194304 5964478 140736455174544 140736455174064 4546669 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23384/statm: 40942 2131 198 433 0 40506 0
Current children cumulated CPU time (s) 0.18
Current children cumulated vsize (KiB) 163768

[startup+0.30013 s]
/proc/loadavg: 4.01 4.02 3.94 4/180 23384
/proc/meminfo: memFree=29890224/32873844 swapFree=7112/7112
[pid=23384] ppid=23382 vsize=163768 CPUtime=0.57 cores=1,3,5,7
/proc/23384/stat : 23384 (pwbo2.01) S 23382 23384 22874 0 -1 4202496 4051 0 0 0 56 1 0 0 20 0 3 0 285931521 167698432 2901 33554432000 4194304 5964478 140736455174544 140736455174064 4546669 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23384/statm: 40942 2901 198 433 0 40506 0
Current children cumulated CPU time (s) 0.57
Current children cumulated vsize (KiB) 163768

[startup+0.700152 s]
/proc/loadavg: 4.01 4.02 3.94 4/180 23384
/proc/meminfo: memFree=29890224/32873844 swapFree=7112/7112
[pid=23384] ppid=23382 vsize=163768 CPUtime=1.37 cores=1,3,5,7
/proc/23384/stat : 23384 (pwbo2.01) S 23382 23384 22874 0 -1 4202496 5129 0 0 0 135 2 0 0 20 0 3 0 285931521 167698432 3757 33554432000 4194304 5964478 140736455174544 140736455174064 4546669 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23384/statm: 40942 3757 198 433 0 40506 0
Current children cumulated CPU time (s) 1.37
Current children cumulated vsize (KiB) 163768

[startup+1.5003 s]
/proc/loadavg: 4.01 4.02 3.94 5/183 23387
/proc/meminfo: memFree=29873816/32873844 swapFree=7112/7112
[pid=23384] ppid=23382 vsize=163768 CPUtime=2.96 cores=1,3,5,7
/proc/23384/stat : 23384 (pwbo2.01) S 23382 23384 22874 0 -1 4202496 6438 0 0 0 292 4 0 0 20 0 3 0 285931521 167698432 4943 33554432000 4194304 5964478 140736455174544 140736455174064 4546669 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23384/statm: 40942 4943 198 433 0 40506 0
[pid=23384/tid=23386] ppid=23382 vsize=163768 CPUtime=1.46 cores=1,3,5,7
/proc/23384/task/23386/stat : 23386 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 2833 0 0 0 143 3 0 0 20 0 3 0 285931522 167698432 4943 33554432000 4194304 5964478 140736455174544 140293920347960 5136404 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=23384/tid=23387] ppid=23382 vsize=163768 CPUtime=1.47 cores=1,3,5,7
/proc/23384/task/23387/stat : 23387 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 2586 0 0 0 147 0 0 0 20 0 3 0 285931522 167698432 4943 33554432000 4194304 5964478 140736455174544 140293909858824 4221984 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 2.96
Current children cumulated vsize (KiB) 163768

[startup+3.10034 s]
/proc/loadavg: 4.01 4.02 3.94 5/183 23387
/proc/meminfo: memFree=29865260/32873844 swapFree=7112/7112
[pid=23384] ppid=23382 vsize=163768 CPUtime=6.17 cores=1,3,5,7
/proc/23384/stat : 23384 (pwbo2.01) S 23382 23384 22874 0 -1 4202496 9172 0 0 0 609 8 0 0 20 0 3 0 285931521 167698432 7501 33554432000 4194304 5964478 140736455174544 140736455174064 4546669 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23384/statm: 40942 7501 198 433 0 40506 0
[pid=23384/tid=23386] ppid=23382 vsize=163768 CPUtime=3.06 cores=1,3,5,7
/proc/23384/task/23386/stat : 23386 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 5135 0 0 0 299 7 0 0 20 0 3 0 285931522 167698432 7501 33554432000 4194304 5964478 140736455174544 140293920348144 5136404 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=23384/tid=23387] ppid=23382 vsize=163768 CPUtime=3.06 cores=1,3,5,7
/proc/23384/task/23387/stat : 23387 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 3018 0 0 0 306 0 0 0 20 0 3 0 285931522 167698432 7501 33554432000 4194304 5964478 140736455174544 140293909858528 4221120 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 6.17
Current children cumulated vsize (KiB) 163768

[startup+6.30034 s]
/proc/loadavg: 4.01 4.02 3.94 5/183 23387
/proc/meminfo: memFree=29841940/32873844 swapFree=7112/7112
[pid=23384] ppid=23382 vsize=163900 CPUtime=12.57 cores=1,3,5,7
/proc/23384/stat : 23384 (pwbo2.01) S 23382 23384 22874 0 -1 4202496 14718 0 0 0 1242 15 0 0 20 0 3 0 285931521 167833600 12907 33554432000 4194304 5964478 140736455174544 140736455174064 4546669 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23384/statm: 40975 12907 198 433 0 40539 0
[pid=23384/tid=23386] ppid=23382 vsize=163900 CPUtime=6.25 cores=1,3,5,7
/proc/23384/task/23386/stat : 23386 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 9972 0 0 0 611 14 0 0 20 0 3 0 285931522 167833600 12907 33554432000 4194304 5964478 140736455174544 140293920348152 4400062 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=23384/tid=23387] ppid=23382 vsize=163900 CPUtime=6.25 cores=1,3,5,7
/proc/23384/task/23387/stat : 23387 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 3727 0 0 0 624 1 0 0 20 0 3 0 285931522 167833600 12907 33554432000 4194304 5964478 140736455174544 140293909858376 4403682 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 12.57
Current children cumulated vsize (KiB) 163900

[startup+12.7004 s]
/proc/loadavg: 4.01 4.02 3.94 5/183 23387
/proc/meminfo: memFree=29810452/32873844 swapFree=7112/7112
[pid=23384] ppid=23382 vsize=164444 CPUtime=25.36 cores=1,3,5,7
/proc/23384/stat : 23384 (pwbo2.01) S 23382 23384 22874 0 -1 4202496 22710 0 0 0 2510 26 0 0 20 0 3 0 285931521 168390656 20784 33554432000 4194304 5964478 140736455174544 140736455174064 4546669 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23384/statm: 41111 20784 198 433 0 40675 0
[pid=23384/tid=23386] ppid=23382 vsize=164444 CPUtime=12.63 cores=1,3,5,7
/proc/23384/task/23386/stat : 23386 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 16231 0 0 0 1239 24 0 0 20 0 3 0 285931522 168390656 20784 33554432000 4194304 5964478 140736455174544 140293920348152 4400503 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=23384/tid=23387] ppid=23382 vsize=164444 CPUtime=12.64 cores=1,3,5,7
/proc/23384/task/23387/stat : 23387 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 5460 0 0 0 1262 2 0 0 20 0 3 0 285931522 168390656 20784 33554432000 4194304 5964478 140736455174544 140293909858616 18446744073699066235 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 25.36
Current children cumulated vsize (KiB) 164444

[startup+25.5004 s]
/proc/loadavg: 4.01 4.02 3.94 5/183 23387
/proc/meminfo: memFree=29720892/32873844 swapFree=7112/7112
[pid=23384] ppid=23382 vsize=231624 CPUtime=50.96 cores=1,3,5,7
/proc/23384/stat : 23384 (pwbo2.01) S 23382 23384 22874 0 -1 4202496 44407 0 0 0 5055 41 0 0 20 0 3 0 285931521 237182976 42371 33554432000 4194304 5964478 140736455174544 140736455174064 4546669 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23384/statm: 57906 42373 198 433 0 57470 0
[pid=23384/tid=23386] ppid=23382 vsize=231624 CPUtime=25.4 cores=1,3,5,7
/proc/23384/task/23386/stat : 23386 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 27829 0 0 0 2506 34 0 0 20 0 3 0 285931522 237182976 42373 33554432000 4194304 5964478 140736455174544 140293920348152 5136404 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=23384/tid=23387] ppid=23382 vsize=231624 CPUtime=25.4 cores=1,3,5,7
/proc/23384/task/23387/stat : 23387 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 15561 0 0 0 2534 6 0 0 20 0 3 0 285931522 237182976 42373 33554432000 4194304 5964478 140736455174544 140293909858968 4432077 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 50.96
Current children cumulated vsize (KiB) 231624

[startup+51.1005 s]
/proc/loadavg: 4.00 4.02 3.94 5/183 23388
/proc/meminfo: memFree=29644600/32873844 swapFree=7112/7112
[pid=23384] ppid=23382 vsize=363400 CPUtime=102.15 cores=1,3,5,7
/proc/23384/stat : 23384 (pwbo2.01) S 23382 23384 22874 0 -1 4202496 62903 0 0 0 10156 59 0 0 20 0 3 0 285931521 372121600 60754 33554432000 4194304 5964478 140736455174544 140736455174064 4546669 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23384/statm: 90850 60754 198 433 0 90414 0
[pid=23384/tid=23386] ppid=23382 vsize=363400 CPUtime=50.93 cores=1,3,5,7
/proc/23384/task/23386/stat : 23386 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 40795 0 0 0 5043 50 0 0 20 0 3 0 285931522 372121600 60754 33554432000 4194304 5964478 140736455174544 140293920348064 4400081 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=23384/tid=23387] ppid=23382 vsize=363400 CPUtime=50.93 cores=1,3,5,7
/proc/23384/task/23387/stat : 23387 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 21089 0 0 0 5085 8 0 0 20 0 3 0 285931522 372121600 60754 33554432000 4194304 5964478 140736455174544 140293909858376 4388330 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 102.15
Current children cumulated vsize (KiB) 363400

[startup+102.3 s]
/proc/loadavg: 4.00 4.01 3.94 5/183 23388
/proc/meminfo: memFree=29360064/32873844 swapFree=7112/7112
[pid=23384] ppid=23382 vsize=627352 CPUtime=204.54 cores=1,3,5,7
/proc/23384/stat : 23384 (pwbo2.01) S 23382 23384 22874 0 -1 4202496 129493 0 0 0 20335 119 0 0 20 0 3 0 285931521 642408448 127050 33554432000 4194304 5964478 140736455174544 140736455174064 4546669 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23384/statm: 156838 127050 198 433 0 156402 0
[pid=23384/tid=23386] ppid=23382 vsize=627352 CPUtime=102 cores=1,3,5,7
/proc/23384/task/23386/stat : 23386 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 86246 0 0 0 10096 104 0 0 20 0 3 0 285931522 642408448 127050 33554432000 4194304 5964478 140736455174544 140293920348152 4400099 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=23384/tid=23387] ppid=23382 vsize=627352 CPUtime=102 cores=1,3,5,7
/proc/23384/task/23387/stat : 23387 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 42228 0 0 0 10183 17 0 0 20 0 3 0 285931522 642408448 127050 33554432000 4194304 5964478 140736455174544 140293909858616 4400051 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 204.54
Current children cumulated vsize (KiB) 627352

[startup+162.3 s]
/proc/loadavg: 4.00 4.01 3.94 5/183 23388
/proc/meminfo: memFree=29219600/32873844 swapFree=7112/7112
[pid=23384] ppid=23382 vsize=693940 CPUtime=324.51 cores=1,3,5,7
/proc/23384/stat : 23384 (pwbo2.01) S 23382 23384 22874 0 -1 4202496 146057 0 0 0 32323 128 0 0 20 0 3 0 285931521 710594560 143614 33554432000 4194304 5964478 140736455174544 140736455174064 4546669 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23384/statm: 173485 143614 198 433 0 173049 0
[pid=23384/tid=23386] ppid=23382 vsize=693940 CPUtime=161.84 cores=1,3,5,7
/proc/23384/task/23386/stat : 23386 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 86283 0 0 0 16079 105 0 0 20 0 3 0 285931522 710594560 143614 33554432000 4194304 5964478 140736455174544 140293920348064 4399904 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=23384/tid=23387] ppid=23382 vsize=693940 CPUtime=161.84 cores=1,3,5,7
/proc/23384/task/23387/stat : 23387 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 58755 0 0 0 16158 26 0 0 20 0 3 0 285931522 710594560 143614 33554432000 4194304 5964478 140736455174544 140293909858720 4400099 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 324.51
Current children cumulated vsize (KiB) 693940

[startup+222.307 s]
/proc/loadavg: 4.00 4.00 3.94 5/183 23388
/proc/meminfo: memFree=29097364/32873844 swapFree=7112/7112
[pid=23384] ppid=23382 vsize=759944 CPUtime=444.51 cores=1,3,5,7
/proc/23384/stat : 23384 (pwbo2.01) S 23382 23384 22874 0 -1 4202496 176581 0 0 0 44293 158 0 0 20 0 3 0 285931521 778182656 173910 33554432000 4194304 5964478 140736455174544 140736455174064 4546669 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23384/statm: 189986 173910 200 433 0 189550 0
[pid=23384/tid=23386] ppid=23382 vsize=759944 CPUtime=221.69 cores=1,3,5,7
/proc/23384/task/23386/stat : 23386 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 108217 0 0 0 22037 132 0 0 20 0 3 0 285931522 778182656 173910 33554432000 4194304 5964478 140736455174544 140293920348144 4400040 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=23384/tid=23387] ppid=23382 vsize=759944 CPUtime=221.69 cores=1,3,5,7
/proc/23384/task/23387/stat : 23387 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 67345 0 0 0 22138 31 0 0 20 0 3 0 285931522 778182656 173910 33554432000 4194304 5964478 140736455174544 140293909858616 4388475 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 444.51
Current children cumulated vsize (KiB) 759944

[startup+282.3 s]
/proc/loadavg: 4.00 4.00 3.94 5/183 23388
/proc/meminfo: memFree=29038120/32873844 swapFree=7112/7112
[pid=23384] ppid=23382 vsize=761012 CPUtime=564.47 cores=1,3,5,7
/proc/23384/stat : 23384 (pwbo2.01) S 23382 23384 22874 0 -1 4202496 177309 0 0 0 56289 158 0 0 20 0 3 0 285931521 779276288 174143 33554432000 4194304 5964478 140736455174544 140736455174064 4546669 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23384/statm: 190253 174143 200 433 0 189817 0
[pid=23384/tid=23386] ppid=23382 vsize=761012 CPUtime=281.53 cores=1,3,5,7
/proc/23384/task/23386/stat : 23386 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 108315 0 0 0 28020 133 0 0 20 0 3 0 285931522 779276288 174143 33554432000 4194304 5964478 140736455174544 140293920348064 4220672 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=23384/tid=23387] ppid=23382 vsize=761012 CPUtime=281.52 cores=1,3,5,7
/proc/23384/task/23387/stat : 23387 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 67975 0 0 0 28120 32 0 0 20 0 3 0 285931522 779276288 174143 33554432000 4194304 5964478 140736455174544 140293909858720 4400073 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 564.47
Current children cumulated vsize (KiB) 761012

[startup+342.3 s]
/proc/loadavg: 4.00 4.00 3.94 5/183 23550
/proc/meminfo: memFree=30558692/32873844 swapFree=7112/7112
[pid=23384] ppid=23382 vsize=761148 CPUtime=684.45 cores=1,3,5,7
/proc/23384/stat : 23384 (pwbo2.01) S 23382 23384 22874 0 -1 4202496 177353 0 0 0 68287 158 0 0 20 0 3 0 285931521 779415552 174187 33554432000 4194304 5964478 140736455174544 140736455174064 4546669 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23384/statm: 190287 174187 200 433 0 189851 0

################
# More data... #
################

/proc/loadavg: 3.88 3.97 3.93 4/184 23991
/proc/meminfo: memFree=30503016/32873844 swapFree=7112/7112
[pid=23384] ppid=23382 vsize=761296 CPUtime=804.41 cores=1,3,5,7
/proc/23384/stat : 23384 (pwbo2.01) S 23382 23384 22874 0 -1 4202496 180630 0 0 0 80277 164 0 0 20 0 3 0 285931521 779567104 177944 33554432000 4194304 5964478 140736455174544 140736455174064 4546669 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23384/statm: 190324 177944 200 433 0 189888 0
[pid=23384/tid=23386] ppid=23382 vsize=761296 CPUtime=401.21 cores=1,3,5,7
/proc/23384/task/23386/stat : 23386 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 111562 0 0 0 39980 141 0 0 20 0 3 0 285931522 779567104 177944 33554432000 4194304 5964478 140736455174544 140293920348152 4399904 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=23384/tid=23387] ppid=23382 vsize=761296 CPUtime=401.19 cores=1,3,5,7
/proc/23384/task/23387/stat : 23387 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 68049 0 0 0 40085 34 0 0 20 0 3 0 285931522 779567104 177944 33554432000 4194304 5964478 140736455174544 140293909858720 4399925 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 804.41
Current children cumulated vsize (KiB) 761296

[startup+462.3 s]
/proc/loadavg: 3.96 3.97 3.93 5/183 24031
/proc/meminfo: memFree=30451376/32873844 swapFree=7112/7112
[pid=23384] ppid=23382 vsize=826488 CPUtime=924.35 cores=1,3,5,7
/proc/23384/stat : 23384 (pwbo2.01) S 23382 23384 22874 0 -1 4202496 195206 0 0 0 92260 175 0 0 20 0 3 0 285931521 846323712 192322 33554432000 4194304 5964478 140736455174544 140736455174064 4546669 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23384/statm: 206622 192322 200 433 0 206186 0
[pid=23384/tid=23386] ppid=23382 vsize=826488 CPUtime=461.04 cores=1,3,5,7
/proc/23384/task/23386/stat : 23386 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 126018 0 0 0 45952 152 0 0 20 0 3 0 285931522 846323712 192322 33554432000 4194304 5964478 140736455174544 140293920348256 4399904 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=23384/tid=23387] ppid=23382 vsize=826488 CPUtime=461 cores=1,3,5,7
/proc/23384/task/23387/stat : 23387 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 68169 0 0 0 46065 35 0 0 20 0 3 0 285931522 846323712 192322 33554432000 4194304 5964478 140736455174544 140293909858720 4400051 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 924.35
Current children cumulated vsize (KiB) 826488

[startup+522.3 s]
/proc/loadavg: 3.98 3.97 3.93 5/183 24031
/proc/meminfo: memFree=30385932/32873844 swapFree=7112/7112
[pid=23384] ppid=23382 vsize=892592 CPUtime=1044.3 cores=1,3,5,7
/proc/23384/stat : 23384 (pwbo2.01) S 23382 23384 22874 0 -1 4202496 204127 0 0 0 104250 180 0 0 20 0 3 0 285931521 914014208 201222 33554432000 4194304 5964478 140736455174544 140736455174064 4546669 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23384/statm: 223148 201222 200 433 0 222712 0
[pid=23384/tid=23386] ppid=23382 vsize=892592 CPUtime=520.86 cores=1,3,5,7
/proc/23384/task/23386/stat : 23386 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 126041 0 0 0 51933 153 0 0 20 0 3 0 285931522 914014208 201222 33554432000 4194304 5964478 140736455174544 140293920348256 4222464 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=23384/tid=23387] ppid=23382 vsize=892592 CPUtime=520.84 cores=1,3,5,7
/proc/23384/task/23387/stat : 23387 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 77067 0 0 0 52044 40 0 0 20 0 3 0 285931522 914014208 201222 33554432000 4194304 5964478 140736455174544 140293909858616 4400051 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1044.3
Current children cumulated vsize (KiB) 892592

[startup+582.3 s]
/proc/loadavg: 3.99 3.97 3.93 5/184 24034
/proc/meminfo: memFree=30327052/32873844 swapFree=7112/7112
[pid=23384] ppid=23382 vsize=958288 CPUtime=1164.25 cores=1,3,5,7
/proc/23384/stat : 23384 (pwbo2.01) S 23382 23384 22874 0 -1 4202496 214092 0 0 0 116238 187 0 0 20 0 3 0 285931521 981286912 211149 33554432000 4194304 5964478 140736455174544 140736455174064 4546669 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23384/statm: 239572 211149 200 433 0 239136 0
[pid=23384/tid=23386] ppid=23382 vsize=958288 CPUtime=580.7 cores=1,3,5,7
/proc/23384/task/23386/stat : 23386 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 126061 0 0 0 57916 154 0 0 20 0 3 0 285931522 981286912 211149 33554432000 4194304 5964478 140736455174544 140293920348256 4399904 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=23384/tid=23387] ppid=23382 vsize=958288 CPUtime=580.64 cores=1,3,5,7
/proc/23384/task/23387/stat : 23387 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 87012 0 0 0 58016 48 0 0 20 0 3 0 285931522 981286912 211149 33554432000 4194304 5964478 140736455174544 140293909858616 4400040 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1164.25
Current children cumulated vsize (KiB) 958288

[startup+642.3 s]
/proc/loadavg: 3.99 3.97 3.93 5/183 24035
/proc/meminfo: memFree=30311348/32873844 swapFree=7112/7112
[pid=23384] ppid=23382 vsize=958856 CPUtime=1284.18 cores=1,3,5,7
/proc/23384/stat : 23384 (pwbo2.01) S 23382 23384 22874 0 -1 4202496 217161 0 0 0 128227 191 0 0 20 0 3 0 285931521 981868544 214151 33554432000 4194304 5964478 140736455174544 140736455174064 4546669 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23384/statm: 239714 214151 200 433 0 239278 0
[pid=23384/tid=23386] ppid=23382 vsize=958856 CPUtime=640.5 cores=1,3,5,7
/proc/23384/task/23386/stat : 23386 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 129036 0 0 0 63892 158 0 0 20 0 3 0 285931522 981868544 214151 33554432000 4194304 5964478 140736455174544 140293920348152 5136404 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=23384/tid=23387] ppid=23382 vsize=958856 CPUtime=640.49 cores=1,3,5,7
/proc/23384/task/23387/stat : 23387 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 87106 0 0 0 63999 50 0 0 20 0 3 0 285931522 981868544 214151 33554432000 4194304 5964478 140736455174544 140293909858376 4400040 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1284.18
Current children cumulated vsize (KiB) 958856

Solver just ended. Dumping a history of the last processes samples

[startup+652.8 s]
/proc/loadavg: 3.99 3.97 3.93 5/183 24035
/proc/meminfo: memFree=30296568/32873844 swapFree=7112/7112
[pid=23384] ppid=23382 vsize=1024532 CPUtime=1305.16 cores=1,3,5,7
/proc/23384/stat : 23384 (pwbo2.01) S 23382 23384 22874 0 -1 4202496 220135 0 0 0 130320 196 0 0 20 0 3 0 285931521 1049120768 217117 33554432000 4194304 5964478 140736455174544 140736455174064 4546669 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23384/statm: 256133 217117 200 433 0 255697 0
[pid=23384/tid=23386] ppid=23382 vsize=1024532 CPUtime=650.97 cores=1,3,5,7
/proc/23384/task/23386/stat : 23386 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 131982 0 0 0 64933 164 0 0 20 0 3 0 285931522 1049120768 217117 33554432000 4194304 5964478 140736455174544 140293920348152 4399904 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=23384/tid=23387] ppid=23382 vsize=1024532 CPUtime=650.96 cores=1,3,5,7
/proc/23384/task/23387/stat : 23387 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 87134 0 0 0 65046 50 0 0 20 0 3 0 285931522 1049120768 217117 33554432000 4194304 5964478 140736455174544 140293909858376 4400071 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1305.16
Current children cumulated vsize (KiB) 1024532

[startup+656 s]
/proc/loadavg: 3.99 3.97 3.93 5/183 24035
/proc/meminfo: memFree=30295932/32873844 swapFree=7112/7112
[pid=23384] ppid=23382 vsize=1024676 CPUtime=1311.55 cores=1,3,5,7
/proc/23384/stat : 23384 (pwbo2.01) S 23382 23384 22874 0 -1 4202496 220176 0 0 0 130959 196 0 0 20 0 3 0 285931521 1049268224 217150 33554432000 4194304 5964478 140736455174544 140736455174064 4546669 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23384/statm: 256169 217150 200 433 0 255733 0
[pid=23384/tid=23386] ppid=23382 vsize=1024676 CPUtime=654.16 cores=1,3,5,7
/proc/23384/task/23386/stat : 23386 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 131982 0 0 0 65252 164 0 0 20 0 3 0 285931522 1049268224 217150 33554432000 4194304 5964478 140736455174544 140293920348152 4400051 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=23384/tid=23387] ppid=23382 vsize=1024676 CPUtime=654.14 cores=1,3,5,7
/proc/23384/task/23387/stat : 23387 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 87175 0 0 0 65364 50 0 0 20 0 3 0 285931522 1049268224 217150 33554432000 4194304 5964478 140736455174544 140293909858376 4400062 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1311.55
Current children cumulated vsize (KiB) 1024676

[startup+657.6 s]
/proc/loadavg: 3.99 3.97 3.93 5/183 24035
/proc/meminfo: memFree=30295932/32873844 swapFree=7112/7112
[pid=23384] ppid=23382 vsize=1024676 CPUtime=1314.76 cores=1,3,5,7
/proc/23384/stat : 23384 (pwbo2.01) S 23382 23384 22874 0 -1 4202496 220505 0 0 0 131279 197 0 0 20 0 3 0 285931521 1049268224 217479 33554432000 4194304 5964478 140736455174544 140736455174064 4546669 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23384/statm: 256169 217479 200 433 0 255733 0
[pid=23384/tid=23386] ppid=23382 vsize=1024676 CPUtime=655.76 cores=1,3,5,7
/proc/23384/task/23386/stat : 23386 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 131982 0 0 0 65412 164 0 0 20 0 3 0 285931522 1049268224 217479 33554432000 4194304 5964478 140736455174544 140293920348152 4221120 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=23384/tid=23387] ppid=23382 vsize=1024676 CPUtime=655.74 cores=1,3,5,7
/proc/23384/task/23387/stat : 23387 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 87504 0 0 0 65524 50 0 0 20 0 3 0 285931522 1049268224 217479 33554432000 4194304 5964478 140736455174544 140293909858616 4399904 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1314.76
Current children cumulated vsize (KiB) 1024676

[startup+659.2 s]
/proc/loadavg: 3.99 3.97 3.93 5/183 24035
/proc/meminfo: memFree=30290352/32873844 swapFree=7112/7112
[pid=23384] ppid=23382 vsize=1024676 CPUtime=1317.95 cores=1,3,5,7
/proc/23384/stat : 23384 (pwbo2.01) S 23382 23384 22874 0 -1 4202496 221726 0 0 0 131598 197 0 0 20 0 3 0 285931521 1049268224 218700 33554432000 4194304 5964478 140736455174544 140736455174064 4546669 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23384/statm: 256169 218700 200 433 0 255733 0
[pid=23384/tid=23386] ppid=23382 vsize=1024676 CPUtime=657.35 cores=1,3,5,7
/proc/23384/task/23386/stat : 23386 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 131982 0 0 0 65571 164 0 0 20 0 3 0 285931522 1049268224 218700 33554432000 4194304 5964478 140736455174544 140293920348152 4399904 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=23384/tid=23387] ppid=23382 vsize=1024676 CPUtime=657.33 cores=1,3,5,7
/proc/23384/task/23387/stat : 23387 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 88725 0 0 0 65682 51 0 0 20 0 3 0 285931522 1049268224 218700 33554432000 4194304 5964478 140736455174544 140293909858616 4221120 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1317.95
Current children cumulated vsize (KiB) 1024676

[startup+659.6 s]
/proc/loadavg: 3.99 3.97 3.93 5/183 24035
/proc/meminfo: memFree=30290352/32873844 swapFree=7112/7112
[pid=23384] ppid=23382 vsize=1024676 CPUtime=1318.75 cores=1,3,5,7
/proc/23384/stat : 23384 (pwbo2.01) S 23382 23384 22874 0 -1 4202496 222021 0 0 0 131678 197 0 0 20 0 3 0 285931521 1049268224 218995 33554432000 4194304 5964478 140736455174544 140736455174064 4546669 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23384/statm: 256169 218995 200 433 0 255733 0
[pid=23384/tid=23386] ppid=23382 vsize=1024676 CPUtime=657.75 cores=1,3,5,7
/proc/23384/task/23386/stat : 23386 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 131982 0 0 0 65611 164 0 0 20 0 3 0 285931522 1049268224 218995 33554432000 4194304 5964478 140736455174544 140293920348152 4399904 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=23384/tid=23387] ppid=23382 vsize=1024676 CPUtime=657.73 cores=1,3,5,7
/proc/23384/task/23387/stat : 23387 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 89020 0 0 0 65722 51 0 0 20 0 3 0 285931522 1049268224 218995 33554432000 4194304 5964478 140736455174544 140293909858616 4220672 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1318.75
Current children cumulated vsize (KiB) 1024676

[startup+660 s]
/proc/loadavg: 3.99 3.97 3.93 5/183 24035
/proc/meminfo: memFree=30286632/32873844 swapFree=7112/7112
[pid=23384] ppid=23382 vsize=1024676 CPUtime=1319.57 cores=1,3,5,7
/proc/23384/stat : 23384 (pwbo2.01) S 23382 23384 22874 0 -1 4202496 222438 0 0 0 131759 198 0 0 20 0 3 0 285931521 1049268224 219412 33554432000 4194304 5964478 140736455174544 140736455174064 4546669 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23384/statm: 256169 219412 200 433 0 255733 0
[pid=23384/tid=23386] ppid=23382 vsize=1024676 CPUtime=658.16 cores=1,3,5,7
/proc/23384/task/23386/stat : 23386 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 131982 0 0 0 65652 164 0 0 20 0 3 0 285931522 1049268224 219412 33554432000 4194304 5964478 140736455174544 140293920348152 4221120 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=23384/tid=23387] ppid=23382 vsize=1024676 CPUtime=658.14 cores=1,3,5,7
/proc/23384/task/23387/stat : 23387 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 89437 0 0 0 65763 51 0 0 20 0 3 0 285931522 1049268224 219412 33554432000 4194304 5964478 140736455174544 140293909858616 4399970 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1319.57
Current children cumulated vsize (KiB) 1024676

[startup+660.107 s]
/proc/loadavg: 3.99 3.97 3.93 5/183 24035
/proc/meminfo: memFree=30286632/32873844 swapFree=7112/7112
[pid=23384] ppid=23382 vsize=1024676 CPUtime=1319.77 cores=1,3,5,7
/proc/23384/stat : 23384 (pwbo2.01) S 23382 23384 22874 0 -1 4202496 222504 0 0 0 131779 198 0 0 20 0 3 0 285931521 1049268224 219478 33554432000 4194304 5964478 140736455174544 140736455174064 4546669 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/23384/statm: 256169 219478 200 433 0 255733 0
[pid=23384/tid=23386] ppid=23382 vsize=1024676 CPUtime=658.26 cores=1,3,5,7
/proc/23384/task/23386/stat : 23386 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 131982 0 0 0 65662 164 0 0 20 0 3 0 285931522 1049268224 219480 33554432000 4194304 5964478 140736455174544 140293920348152 4221120 0 0 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=23384/tid=23387] ppid=23382 vsize=1024676 CPUtime=658.24 cores=1,3,5,7
/proc/23384/task/23387/stat : 23387 (pwbo2.01) R 23382 23384 22874 0 -1 4202560 89505 0 0 0 65773 51 0 0 20 0 3 0 285931522 1049268224 219480 33554432000 4194304 5964478 140736455174544 140293909858616 5136404 0 0 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1319.77
Current children cumulated vsize (KiB) 1024676

Child status: 255

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 23384 and gives
#  childrusage.ru_utime.tv_sec=1314
#  childrusage.ru_utime.tv_usec=378184
#  childrusage.ru_stime.tv_sec=2
#  childrusage.ru_stime.tv_usec=201665
# CPU time returned by wait4() is 1316.58
# while last known CPU time is 1319.77
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 660.151
CPU time (s): 1319.77
CPU user time (s): 1317.79
CPU system time (s): 1.98
CPU usage (%): 199.92
Max. virtual memory (cumulated for all children) (KiB): 1024676

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1314.38
system time used= 2.20167
maximum resident set size= 877956
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 222515
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 616
involuntary context switches= 1616

runsolver used 1.06484 second user time and 3.14352 second system time

The end

Launcher Data 

Begin job on node132 at 2012-05-28 11:41:02
IDJOB=3712470
IDBENCH=78995
IDSOLVER=2292
FILE ID=node132/3712470-1338198062
RUNJOBID= node132-1338194011-22891
PBS_JOBID= 14624592
Free space on /tmp= 71632 MiB

SOLVER NAME= pwbo 2.01
BENCH NAME= PB10/normalized-PB10/PARTIAL-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/tsp/normalized-t3002.11tsp11.1900567024--soft-33-100-0.wbo
COMMAND LINE= pwbo2.01 -file-format=wbo -verbosity=0 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3712470-1338198062/watcher-3712470-1338198062 -o /tmp/evaluation-result-3712470-1338198062/solver-3712470-1338198062 -C 1800 -W 1900 -M 15500  pwbo2.01 -file-format=wbo -verbosity=0 HOME/instance-3712470-1338198062.wbo

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 05d9157cdbd186fca72e4e6dc0ec425d
RANDOM SEED=816950627

node132.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.832
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.832
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.832
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.832
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.832
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.832
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.832
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.832
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        29890604 kB
Buffers:          461296 kB
Cached:           202892 kB
SwapCached:         2284 kB
Active:          1924536 kB
Inactive:         165636 kB
Active(anon):    1418012 kB
Inactive(anon):     8148 kB
Active(file):     506524 kB
Inactive(file):   157488 kB
Unevictable:        7112 kB
Mlocked:            7112 kB
SwapTotal:      67108856 kB
SwapFree:       67103740 kB
Dirty:              5448 kB
Writeback:             0 kB
AnonPages:       1428792 kB
Mapped:            11784 kB
Shmem:                 8 kB
Slab:             750300 kB
SReclaimable:     191912 kB
SUnreclaim:       558388 kB
KernelStack:        1488 kB
PageTables:         6416 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     173928 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346376 kB
VmallocChunk:   34341987860 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 71628 MiB
End job on node132 at 2012-05-28 11:52:02