Trace number 3729842

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PB10: pb_cplex 2010-06-29UNSAT 1146.14 342.648

General information on the benchmark

Namenormalized-PB06/SATUNSAT-SMALLINT/submitted-PB06/
namasivayam/tsp/normalized-t3002.11tsp11.1900670167.opb
MD5SUM3972d8c24730f4742ee881de3ad25c62
Bench CategoryDEC-SMALLINT (no optimisation, small integers)
Best result obtained on this benchmarkUNSAT
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark1.42778
Has Objective FunctionNO
SatisfiableNO
(Un)Satisfiability was provedYES
Best value of the objective function
Optimality of the best value was proved
Number of variables231
Total number of constraints2707
Number of constraints which are clauses2684
Number of constraints which are cardinality constraints (but not clauses)22
Number of constraints which are nor clauses,nor cardinality constraints1
Minimum length of a constraint2
Maximum length of a constraint110
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 25
Number of bits of the biggest number in a constraint 5
Biggest sum of numbers in a constraint 707
Number of bits of the biggest sum of numbers10
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c pb_cplex 1.0 - 20100701
0.00/0.00	c Parsing opb file format. File HOME/instance-3729842-1339246838.opb.
0.00/0.00	c Instance file HOME/instance-3729842-1339246838.opb
0.00/0.00	c File size is 83379 bytes.
0.00/0.01	c Highest Coefficient sum: 707
0.00/0.01	c  IBM ILOG License Manager: "IBM ILOG Optimization Suite for Academic Initiative" is accessing CPLEX 12 with option(s): "e m b q ".
0.00/0.06	c ============================[ Problem Statistics ]=============================
0.00/0.06	c |                                                                             |
0.00/0.06	c |  Parsing time:          0.01         s                                      |
0.00/0.06	c |  Number Variables:      231                                                 |
0.00/0.06	c |  Number Constraints:    2707                                                |
0.00/0.06	c ===============================================================================
1146.14/342.64	c Remaining time 1799.99 sec.
1146.14/342.64	s UNSATISFIABLE
1146.14/342.64	c Total CPU time              : 1138.49 s

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3729842-1339246838/watcher-3729842-1339246838 -o /tmp/evaluation-result-3729842-1339246838/solver-3729842-1339246838 -C 1800 -W 1900 -M 15500 run HOME/instance-3729842-1339246838.opb 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.00 0.00 0.00 3/178 1737
/proc/meminfo: memFree=30380696/32873844 swapFree=7124/7124
[pid=1737] ppid=1732 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1737/stat : 1737 (run) S 1732 1737 1671 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 34650921 9367552 270 33554432000 4194304 5064252 140734486019408 140734486018320 215143310926 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1737/statm: 2287 270 229 213 0 72 0

[startup+0.0547581 s]
/proc/loadavg: 0.00 0.00 0.00 3/178 1737
/proc/meminfo: memFree=30380696/32873844 swapFree=7124/7124
[pid=1737] ppid=1732 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1737/stat : 1737 (run) S 1732 1737 1671 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 34650921 9367552 270 33554432000 4194304 5064252 140734486019408 140734486018320 215143310926 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1737/statm: 2287 270 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.100248 s]
/proc/loadavg: 0.00 0.00 0.00 3/178 1737
/proc/meminfo: memFree=30380696/32873844 swapFree=7124/7124
[pid=1737] ppid=1732 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1737/stat : 1737 (run) S 1732 1737 1671 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 34650921 9367552 270 33554432000 4194304 5064252 140734486019408 140734486018320 215143310926 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1737/statm: 2287 270 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.300179 s]
/proc/loadavg: 0.00 0.00 0.00 3/178 1737
/proc/meminfo: memFree=30380696/32873844 swapFree=7124/7124
[pid=1737] ppid=1732 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1737/stat : 1737 (run) S 1732 1737 1671 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 34650921 9367552 270 33554432000 4194304 5064252 140734486019408 140734486018320 215143310926 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1737/statm: 2287 270 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.700216 s]
/proc/loadavg: 0.00 0.00 0.00 3/178 1737
/proc/meminfo: memFree=30380696/32873844 swapFree=7124/7124
[pid=1737] ppid=1732 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1737/stat : 1737 (run) S 1732 1737 1671 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 34650921 9367552 270 33554432000 4194304 5064252 140734486019408 140734486018320 215143310926 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1737/statm: 2287 270 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+1.50027 s]
/proc/loadavg: 0.16 0.03 0.01 9/196 1755
/proc/meminfo: memFree=30310576/32873844 swapFree=7124/7124
[pid=1737] ppid=1732 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1737/stat : 1737 (run) S 1732 1737 1671 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 34650921 9367552 270 33554432000 4194304 5064252 140734486019408 140734486018320 215143310926 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1737/statm: 2287 270 229 213 0 72 0
[pid=1739] ppid=1737 vsize=75912 CPUtime=2.32 cores=1,3,5,7
/proc/1739/stat : 1739 (pb_cplex) R 1737 1737 1671 0 -1 4202496 13170 0 0 0 223 9 0 0 20 0 1 0 34650921 77733888 5342 33554432000 134512640 146736320 4289570272 4289564056 136007289 0 0 4096 16387 0 0 0 17 3 0 0 3 0 0
/proc/1739/statm: 18978 5342 888 2985 0 15912 0
Current children cumulated CPU time (s) 2.32
Current children cumulated vsize (KiB) 85060

[startup+3.10045 s]
/proc/loadavg: 0.16 0.03 0.01 4/196 1783
/proc/meminfo: memFree=30308632/32873844 swapFree=7124/7124
[pid=1737] ppid=1732 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1737/stat : 1737 (run) S 1732 1737 1671 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 34650921 9367552 270 33554432000 4194304 5064252 140734486019408 140734486018320 215143310926 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1737/statm: 2287 270 229 213 0 72 0
[pid=1739] ppid=1737 vsize=128136 CPUtime=4.72 cores=1,3,5,7
/proc/1739/stat : 1739 (pb_cplex) R 1737 1737 1671 0 -1 4202496 25602 0 0 0 453 19 0 0 20 0 8 0 34650921 131211264 11563 33554432000 134512640 146736320 4289570272 4289562064 136008176 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 3 0 0
/proc/1739/statm: 32034 11563 896 2985 0 28968 0
[pid=1739/tid=1763] ppid=1737 vsize=128136 CPUtime=0.08 cores=1,3,5,7
/proc/1739/task/1763/stat : 1763 (pb_cplex) R 1737 1737 1671 0 -1 4202560 32 0 0 0 8 0 0 0 20 0 8 0 34651103 131211264 11563 33554432000 134512640 146736320 4289570272 4130422936 136007640 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=1739/tid=1764] ppid=1737 vsize=128136 CPUtime=0.05 cores=1,3,5,7
/proc/1739/task/1764/stat : 1764 (pb_cplex) R 1737 1737 1671 0 -1 4202560 490 0 0 0 5 0 0 0 20 0 8 0 34651103 131211264 11563 33554432000 134512640 146736320 4289570272 4140907568 136384857 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=1739/tid=1765] ppid=1737 vsize=128136 CPUtime=0.08 cores=1,3,5,7
/proc/1739/task/1765/stat : 1765 (pb_cplex) R 1737 1737 1671 0 -1 4202560 41 0 0 0 8 0 0 0 20 0 8 0 34651103 131211264 11563 33554432000 134512640 146736320 4289570272 4151393424 136008044 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=1739/tid=1766] ppid=1737 vsize=128136 CPUtime=0.08 cores=1,3,5,7
/proc/1739/task/1766/stat : 1766 (pb_cplex) R 1737 1737 1671 0 -1 4202560 103 0 0 0 8 0 0 0 20 0 8 0 34651103 131211264 11563 33554432000 134512640 146736320 4289570272 4117785576 139044505 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=1739/tid=1767] ppid=1737 vsize=128136 CPUtime=0.08 cores=1,3,5,7
/proc/1739/task/1767/stat : 1767 (pb_cplex) R 1737 1737 1671 0 -1 4202560 651 0 0 0 8 0 0 0 20 0 8 0 34651103 131211264 11563 33554432000 134512640 146736320 4289570272 4097825888 136006275 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=1739/tid=1768] ppid=1737 vsize=128136 CPUtime=0.05 cores=1,3,5,7
/proc/1739/task/1768/stat : 1768 (pb_cplex) R 1737 1737 1671 0 -1 4202560 565 0 0 0 5 0 0 0 20 0 8 0 34651103 131211264 11563 33554432000 134512640 146736320 4289570272 4087341128 139042883 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=1739/tid=1769] ppid=1737 vsize=128136 CPUtime=0.05 cores=1,3,5,7
/proc/1739/task/1769/stat : 1769 (pb_cplex) R 1737 1737 1671 0 -1 4202560 694 0 0 0 5 0 0 0 20 0 8 0 34651103 131211264 11563 33554432000 134512640 146736320 4289570272 4070562792 139044585 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 4.72
Current children cumulated vsize (KiB) 137284

[startup+6.30043 s]
/proc/loadavg: 1.43 0.30 0.10 13/196 1783
/proc/meminfo: memFree=30289584/32873844 swapFree=7124/7124
[pid=1737] ppid=1732 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1737/stat : 1737 (run) S 1732 1737 1671 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 34650921 9367552 270 33554432000 4194304 5064252 140734486019408 140734486018320 215143310926 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1737/statm: 2287 270 229 213 0 72 0
[pid=1739] ppid=1737 vsize=127112 CPUtime=15.8 cores=1,3,5,7
/proc/1739/stat : 1739 (pb_cplex) R 1737 1737 1671 0 -1 4202496 38040 0 0 0 1554 26 0 0 20 0 8 0 34650921 130162688 11614 33554432000 134512640 146736320 4289570272 4289563152 136008042 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 3 0 0
/proc/1739/statm: 31778 11614 896 2985 0 28712 0
[pid=1739/tid=1763] ppid=1737 vsize=127112 CPUtime=1.39 cores=1,3,5,7
/proc/1739/task/1763/stat : 1763 (pb_cplex) R 1737 1737 1671 0 -1 4202560 45 0 0 0 139 0 0 0 20 0 8 0 34651103 130162688 11614 33554432000 134512640 146736320 4289570272 4130421504 135468687 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=1739/tid=1764] ppid=1737 vsize=127112 CPUtime=1.34 cores=1,3,5,7
/proc/1739/task/1764/stat : 1764 (pb_cplex) S 1737 1737 1671 0 -1 4202560 686 0 0 0 132 2 0 0 20 0 8 0 34651103 130162688 11614 33554432000 134512640 146736320 4289570272 4140912272 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=1739/tid=1765] ppid=1737 vsize=127112 CPUtime=1.42 cores=1,3,5,7
/proc/1739/task/1765/stat : 1765 (pb_cplex) S 1737 1737 1671 0 -1 4202560 53 0 0 0 142 0 0 0 20 0 8 0 34651103 130162688 11614 33554432000 134512640 146736320 4289570272 4151398032 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=1739/tid=1766] ppid=1737 vsize=127112 CPUtime=1.51 cores=1,3,5,7
/proc/1739/task/1766/stat : 1766 (pb_cplex) R 1737 1737 1671 0 -1 4202560 115 0 0 0 151 0 0 0 20 0 8 0 34651103 130162688 11614 33554432000 134512640 146736320 4289570272 4117788220 137366911 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=1739/tid=1767] ppid=1737 vsize=127112 CPUtime=1.75 cores=1,3,5,7
/proc/1739/task/1767/stat : 1767 (pb_cplex) R 1737 1737 1671 0 -1 4202560 1087 0 0 0 175 0 0 0 20 0 8 0 34651103 130162688 11614 33554432000 134512640 146736320 4289570272 4097826968 136007152 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=1739/tid=1768] ppid=1737 vsize=127112 CPUtime=1.27 cores=1,3,5,7
/proc/1739/task/1768/stat : 1768 (pb_cplex) R 1737 1737 1671 0 -1 4202560 3493 0 0 0 125 2 0 0 20 0 8 0 34651103 130162688 11614 33554432000 134512640 146736320 4289570272 4087341776 135982686 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=1739/tid=1769] ppid=1737 vsize=127112 CPUtime=1.6 cores=1,3,5,7
/proc/1739/task/1769/stat : 1769 (pb_cplex) S 1737 1737 1671 0 -1 4202560 4381 0 0 0 158 2 0 0 20 0 8 0 34651103 130162688 11614 33554432000 134512640 146736320 4289570272 4070567568 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 15.8
Current children cumulated vsize (KiB) 136260

[startup+12.7004 s]
/proc/loadavg: 2.51 0.54 0.18 14/196 1783
/proc/meminfo: memFree=30285896/32873844 swapFree=7124/7124
[pid=1737] ppid=1732 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1737/stat : 1737 (run) S 1732 1737 1671 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 34650921 9367552 270 33554432000 4194304 5064252 140734486019408 140734486018320 215143310926 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1737/statm: 2287 270 229 213 0 72 0
[pid=1739] ppid=1737 vsize=130184 CPUtime=37.57 cores=1,3,5,7
/proc/1739/stat : 1739 (pb_cplex) S 1737 1737 1671 0 -1 4202496 43206 0 0 0 3716 41 0 0 20 0 8 0 34650921 133308416 12227 33554432000 134512640 146736320 4289570272 4289566672 144580073 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 3 0 0
/proc/1739/statm: 32546 12227 896 2985 0 29480 0
[pid=1739/tid=1763] ppid=1737 vsize=130184 CPUtime=4.01 cores=1,3,5,7
/proc/1739/task/1763/stat : 1763 (pb_cplex) R 1737 1737 1671 0 -1 4202560 46 0 0 0 399 2 0 0 20 0 8 0 34651103 133308416 12227 33554432000 134512640 146736320 4289570272 4130423040 136487075 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=1739/tid=1764] ppid=1737 vsize=130184 CPUtime=3.98 cores=1,3,5,7
/proc/1739/task/1764/stat : 1764 (pb_cplex) R 1737 1737 1671 0 -1 4202560 688 0 0 0 396 2 0 0 20 0 8 0 34651103 133308416 12227 33554432000 134512640 146736320 4289570272 4140909544 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=1739/tid=1765] ppid=1737 vsize=130184 CPUtime=4.14 cores=1,3,5,7
/proc/1739/task/1765/stat : 1765 (pb_cplex) R 1737 1737 1671 0 -1 4202560 54 0 0 0 411 3 0 0 20 0 8 0 34651103 133308416 12227 33554432000 134512640 146736320 4289570272 4151394944 139046527 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=1739/tid=1766] ppid=1737 vsize=130184 CPUtime=4.22 cores=1,3,5,7
/proc/1739/task/1766/stat : 1766 (pb_cplex) R 1737 1737 1671 0 -1 4202560 893 0 0 0 420 2 0 0 20 0 8 0 34651103 133308416 12227 33554432000 134512640 146736320 4289570272 4117786440 135526137 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=1739/tid=1767] ppid=1737 vsize=130184 CPUtime=4.52 cores=1,3,5,7
/proc/1739/task/1767/stat : 1767 (pb_cplex) R 1737 1737 1671 0 -1 4202560 1659 0 0 0 449 3 0 0 20 0 8 0 34651103 133308416 12227 33554432000 134512640 146736320 4289570272 4097827816 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=1739/tid=1768] ppid=1737 vsize=130184 CPUtime=3.94 cores=1,3,5,7
/proc/1739/task/1768/stat : 1768 (pb_cplex) R 1737 1737 1671 0 -1 4202560 4246 0 0 0 391 3 0 0 20 0 8 0 34651103 133308416 12227 33554432000 134512640 146736320 4289570272 4087341208 136007292 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=1739/tid=1769] ppid=1737 vsize=130184 CPUtime=4.22 cores=1,3,5,7
/proc/1739/task/1769/stat : 1769 (pb_cplex) R 1737 1737 1671 0 -1 4202560 6439 0 0 0 419 3 0 0 20 0 8 0 34651103 133308416 12227 33554432000 134512640 146736320 4289570272 4070564432 136403114 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
Current children cumulated CPU time (s) 37.57
Current children cumulated vsize (KiB) 139332

[startup+25.5004 s]
/proc/loadavg: 3.78 0.92 0.31 9/196 1784
/proc/meminfo: memFree=30275052/32873844 swapFree=7124/7124
[pid=1737] ppid=1732 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1737/stat : 1737 (run) S 1732 1737 1671 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 34650921 9367552 270 33554432000 4194304 5064252 140734486019408 140734486018320 215143310926 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1737/statm: 2287 270 229 213 0 72 0
[pid=1739] ppid=1737 vsize=143296 CPUtime=72.92 cores=1,3,5,7
/proc/1739/stat : 1739 (pb_cplex) R 1737 1737 1671 0 -1 4202496 56108 0 0 0 7230 62 0 0 20 0 8 0 34650921 146735104 14472 33554432000 134512640 146736320 4289570272 4289556368 136008335 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 3 0 0
/proc/1739/statm: 35824 14472 902 2985 0 32758 0
[pid=1739/tid=1763] ppid=1737 vsize=143296 CPUtime=7.96 cores=1,3,5,7
/proc/1739/task/1763/stat : 1763 (pb_cplex) S 1737 1737 1671 0 -1 4202560 704 0 0 0 791 5 0 0 20 0 8 0 34651103 146735104 14472 33554432000 134512640 146736320 4289570272 4130426512 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=1739/tid=1764] ppid=1737 vsize=143296 CPUtime=8.01 cores=1,3,5,7
/proc/1739/task/1764/stat : 1764 (pb_cplex) S 1737 1737 1671 0 -1 4202560 937 0 0 0 797 4 0 0 20 0 8 0 34651103 146735104 14472 33554432000 134512640 146736320 4289570272 4140912272 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=1739/tid=1765] ppid=1737 vsize=143296 CPUtime=8.17 cores=1,3,5,7
/proc/1739/task/1765/stat : 1765 (pb_cplex) S 1737 1737 1671 0 -1 4202560 248 0 0 0 812 5 0 0 20 0 8 0 34651103 146735104 14472 33554432000 134512640 146736320 4289570272 4151398032 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=1739/tid=1766] ppid=1737 vsize=143296 CPUtime=8.19 cores=1,3,5,7
/proc/1739/task/1766/stat : 1766 (pb_cplex) S 1737 1737 1671 0 -1 4202560 1226 0 0 0 814 5 0 0 20 0 8 0 34651103 146735104 14472 33554432000 134512640 146736320 4289570272 4117790352 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=1739/tid=1767] ppid=1737 vsize=143296 CPUtime=8.48 cores=1,3,5,7
/proc/1739/task/1767/stat : 1767 (pb_cplex) S 1737 1737 1671 0 -1 4202560 3129 0 0 0 843 5 0 0 20 0 8 0 34651103 146735104 14472 33554432000 134512640 146736320 4289570272 4097830544 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=1739/tid=1768] ppid=1737 vsize=143296 CPUtime=7.86 cores=1,3,5,7
/proc/1739/task/1768/stat : 1768 (pb_cplex) S 1737 1737 1671 0 -1 4202560 6088 0 0 0 780 6 0 0 20 0 8 0 34651103 146735104 14472 33554432000 134512640 146736320 4289570272 4087344784 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=1739/tid=1769] ppid=1737 vsize=143296 CPUtime=8.18 cores=1,3,5,7
/proc/1739/task/1769/stat : 1769 (pb_cplex) S 1737 1737 1671 0 -1 4202560 9286 0 0 0 812 6 0 0 20 0 8 0 34651103 146735104 14472 33554432000 134512640 146736320 4289570272 4070567568 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 72.92
Current children cumulated vsize (KiB) 152444

[startup+51.1004 s]
/proc/loadavg: 5.22 1.49 0.51 4/189 1861
/proc/meminfo: memFree=30305572/32873844 swapFree=7124/7124
[pid=1737] ppid=1732 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1737/stat : 1737 (run) S 1732 1737 1671 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 34650921 9367552 270 33554432000 4194304 5064252 140734486019408 140734486018320 215143310926 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1737/statm: 2287 270 229 213 0 72 0
[pid=1739] ppid=1737 vsize=134280 CPUtime=146.65 cores=1,3,5,7
/proc/1739/stat : 1739 (pb_cplex) R 1737 1737 1671 0 -1 4202496 78146 0 0 0 14558 107 0 0 20 0 8 0 34650921 137502720 13196 33554432000 134512640 146736320 4289570272 4289563944 144757825 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 3 0 0
/proc/1739/statm: 33570 13196 903 2985 0 30504 0
[pid=1739/tid=1763] ppid=1737 vsize=134280 CPUtime=16.25 cores=1,3,5,7
/proc/1739/task/1763/stat : 1763 (pb_cplex) R 1737 1737 1671 0 -1 4202560 712 0 0 0 1616 9 0 0 20 0 8 0 34651103 137502720 13196 33554432000 134512640 146736320 4289570272 4130423784 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=1739/tid=1764] ppid=1737 vsize=134280 CPUtime=16.53 cores=1,3,5,7
/proc/1739/task/1764/stat : 1764 (pb_cplex) S 1737 1737 1671 0 -1 4202560 1068 0 0 0 1643 10 0 0 20 0 8 0 34651103 137502720 13196 33554432000 134512640 146736320 4289570272 4140912272 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=1739/tid=1765] ppid=1737 vsize=134280 CPUtime=16.53 cores=1,3,5,7
/proc/1739/task/1765/stat : 1765 (pb_cplex) S 1737 1737 1671 0 -1 4202560 252 0 0 0 1642 11 0 0 20 0 8 0 34651103 137502720 13196 33554432000 134512640 146736320 4289570272 4151398032 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=1739/tid=1766] ppid=1737 vsize=134280 CPUtime=16.59 cores=1,3,5,7
/proc/1739/task/1766/stat : 1766 (pb_cplex) R 1737 1737 1671 0 -1 4202560 4030 0 0 0 1647 12 0 0 20 0 8 0 34651103 137502720 13196 33554432000 134512640 146736320 4289570272 4117786784 136006325 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=1739/tid=1767] ppid=1737 vsize=134280 CPUtime=16.78 cores=1,3,5,7
/proc/1739/task/1767/stat : 1767 (pb_cplex) R 1737 1737 1671 0 -1 4202560 4420 0 0 0 1668 10 0 0 20 0 8 0 34651103 137502720 13196 33554432000 134512640 146736320 4289570272 4097827208 136388081 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=1739/tid=1768] ppid=1737 vsize=134280 CPUtime=16.08 cores=1,3,5,7

################
# More data... #
################

/proc/loadavg: 9.91 4.24 1.60 16/195 1954
/proc/meminfo: memFree=29787932/32873844 swapFree=7124/7124
[pid=1737] ppid=1732 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1737/stat : 1737 (run) S 1732 1737 1671 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 34650921 9367552 270 33554432000 4194304 5064252 140734486019408 140734486018320 215143310926 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1737/statm: 2287 270 229 213 0 72 0
[pid=1739] ppid=1737 vsize=132232 CPUtime=530.64 cores=1,3,5,7
/proc/1739/stat : 1739 (pb_cplex) R 1737 1737 1671 0 -1 4202496 120248 0 0 0 52723 341 0 0 20 0 8 0 34650921 135405568 12927 33554432000 134512640 146736320 4289570272 4289563944 144757825 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 3 0 0
/proc/1739/statm: 33058 12927 924 2985 0 29992 0
[pid=1739/tid=1763] ppid=1737 vsize=132232 CPUtime=64.13 cores=1,3,5,7
/proc/1739/task/1763/stat : 1763 (pb_cplex) R 1737 1737 1671 0 -1 4202560 1265 0 0 0 6375 38 0 0 20 0 8 0 34651103 135405568 12927 33554432000 134512640 146736320 4289570272 4130422936 136007277 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=1739/tid=1764] ppid=1737 vsize=132232 CPUtime=64.26 cores=1,3,5,7
/proc/1739/task/1764/stat : 1764 (pb_cplex) R 1737 1737 1671 0 -1 4202560 4879 0 0 0 6388 38 0 0 20 0 8 0 34651103 135405568 12927 33554432000 134512640 146736320 4289570272 4140908752 136008245 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=1739/tid=1765] ppid=1737 vsize=132232 CPUtime=63.99 cores=1,3,5,7
/proc/1739/task/1765/stat : 1765 (pb_cplex) R 1737 1737 1671 0 -1 4202560 6394 0 0 0 6360 39 0 0 20 0 8 0 34651103 135405568 12927 33554432000 134512640 146736320 4289570272 4151394896 136403109 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=1739/tid=1766] ppid=1737 vsize=132232 CPUtime=64.41 cores=1,3,5,7
/proc/1739/task/1766/stat : 1766 (pb_cplex) R 1737 1737 1671 0 -1 4202560 8869 0 0 0 6401 40 0 0 20 0 8 0 34651103 135405568 12927 33554432000 134512640 146736320 4289570272 4117787524 135850298 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=1739/tid=1767] ppid=1737 vsize=132232 CPUtime=64.7 cores=1,3,5,7
/proc/1739/task/1767/stat : 1767 (pb_cplex) R 1737 1737 1671 0 -1 4202560 5831 0 0 0 6432 38 0 0 20 0 8 0 34651103 135405568 12927 33554432000 134512640 146736320 4289570272 4097827816 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=1739/tid=1768] ppid=1737 vsize=132232 CPUtime=63.43 cores=1,3,5,7
/proc/1739/task/1768/stat : 1768 (pb_cplex) R 1737 1737 1671 0 -1 4202560 16665 0 0 0 6302 41 0 0 20 0 8 0 34651103 135405568 12927 33554432000 134512640 146736320 4289570272 4087341208 136007626 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=1739/tid=1769] ppid=1737 vsize=132232 CPUtime=64.01 cores=1,3,5,7
/proc/1739/task/1769/stat : 1769 (pb_cplex) R 1737 1737 1671 0 -1 4202560 25011 0 0 0 6354 47 0 0 20 0 8 0 34651103 135405568 12927 33554432000 134512640 146736320 4289570272 4070564840 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 530.64
Current children cumulated vsize (KiB) 141380

[startup+222.3 s]
/proc/loadavg: 10.45 5.45 2.18 14/195 1954
/proc/meminfo: memFree=29697228/32873844 swapFree=7124/7124
[pid=1737] ppid=1732 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1737/stat : 1737 (run) S 1732 1737 1671 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 34650921 9367552 270 33554432000 4194304 5064252 140734486019408 140734486018320 215143310926 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1737/statm: 2287 270 229 213 0 72 0
[pid=1739] ppid=1737 vsize=132232 CPUtime=734.34 cores=1,3,5,7
/proc/1739/stat : 1739 (pb_cplex) R 1737 1737 1671 0 -1 4202496 140876 0 0 0 72960 474 0 0 20 0 8 0 34650921 135405568 12973 33554432000 134512640 146736320 4289570272 4289563168 139028318 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 3 0 0
/proc/1739/statm: 33058 12973 924 2985 0 29992 0
[pid=1739/tid=1763] ppid=1737 vsize=132232 CPUtime=89.33 cores=1,3,5,7
/proc/1739/task/1763/stat : 1763 (pb_cplex) S 1737 1737 1671 0 -1 4202560 2121 0 0 0 8878 55 0 0 20 0 8 0 34651103 135405568 12973 33554432000 134512640 146736320 4289570272 4130426512 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=1739/tid=1764] ppid=1737 vsize=132232 CPUtime=89.81 cores=1,3,5,7
/proc/1739/task/1764/stat : 1764 (pb_cplex) R 1737 1737 1671 0 -1 4202560 9977 0 0 0 8925 56 0 0 20 0 8 0 34651103 135405568 12973 33554432000 134512640 146736320 4289570272 4140909136 136403129 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=1739/tid=1765] ppid=1737 vsize=132232 CPUtime=89.42 cores=1,3,5,7
/proc/1739/task/1765/stat : 1765 (pb_cplex) R 1737 1737 1671 0 -1 4202560 7906 0 0 0 8886 56 0 0 20 0 8 0 34651103 135405568 12973 33554432000 134512640 146736320 4289570272 4151394428 136400477 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=1739/tid=1766] ppid=1737 vsize=132232 CPUtime=89.53 cores=1,3,5,7
/proc/1739/task/1766/stat : 1766 (pb_cplex) R 1737 1737 1671 0 -1 4202560 12009 0 0 0 8893 60 0 0 20 0 8 0 34651103 135405568 12973 33554432000 134512640 146736320 4289570272 4117787624 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=1739/tid=1767] ppid=1737 vsize=132232 CPUtime=89.93 cores=1,3,5,7
/proc/1739/task/1767/stat : 1767 (pb_cplex) R 1737 1737 1671 0 -1 4202560 6072 0 0 0 8938 55 0 0 20 0 8 0 34651103 135405568 12973 33554432000 134512640 146736320 4289570272 4097826856 139044505 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=1739/tid=1768] ppid=1737 vsize=132232 CPUtime=88.89 cores=1,3,5,7
/proc/1739/task/1768/stat : 1768 (pb_cplex) R 1737 1737 1671 0 -1 4202560 20650 0 0 0 8832 57 0 0 20 0 8 0 34651103 135405568 12973 33554432000 134512640 146736320 4289570272 4087340148 135450621 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=1739/tid=1769] ppid=1737 vsize=132232 CPUtime=89.19 cores=1,3,5,7
/proc/1739/task/1769/stat : 1769 (pb_cplex) R 1737 1737 1671 0 -1 4202560 28877 0 0 0 8854 65 0 0 20 0 8 0 34651103 135405568 12973 33554432000 134512640 146736320 4289570272 4070563992 136007073 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
Current children cumulated CPU time (s) 734.34
Current children cumulated vsize (KiB) 141380

[startup+282.3 s]
/proc/loadavg: 10.31 6.30 2.68 16/195 1954
/proc/meminfo: memFree=29598824/32873844 swapFree=7124/7124
[pid=1737] ppid=1732 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1737/stat : 1737 (run) S 1732 1737 1671 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 34650921 9367552 270 33554432000 4194304 5064252 140734486019408 140734486018320 215143310926 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1737/statm: 2287 270 229 213 0 72 0
[pid=1739] ppid=1737 vsize=132232 CPUtime=940.53 cores=1,3,5,7
/proc/1739/stat : 1739 (pb_cplex) R 1737 1737 1671 0 -1 4202496 163755 0 0 0 93455 598 0 0 20 0 8 0 34650921 135405568 12992 33554432000 134512640 146736320 4289570272 4289566592 136674344 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 3 0 0
/proc/1739/statm: 33058 12992 924 2985 0 29992 0
[pid=1739/tid=1763] ppid=1737 vsize=132232 CPUtime=114.89 cores=1,3,5,7
/proc/1739/task/1763/stat : 1763 (pb_cplex) S 1737 1737 1671 0 -1 4202560 6166 0 0 0 11417 72 0 0 20 0 8 0 34651103 135405568 12992 33554432000 134512640 146736320 4289570272 4130426340 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=1739/tid=1764] ppid=1737 vsize=132232 CPUtime=115.55 cores=1,3,5,7
/proc/1739/task/1764/stat : 1764 (pb_cplex) S 1737 1737 1671 0 -1 4202560 13460 0 0 0 11486 69 0 0 20 0 8 0 34651103 135405568 12992 33554432000 134512640 146736320 4289570272 4140912100 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=1739/tid=1765] ppid=1737 vsize=132232 CPUtime=114.82 cores=1,3,5,7
/proc/1739/task/1765/stat : 1765 (pb_cplex) R 1737 1737 1671 0 -1 4202560 10890 0 0 0 11410 72 0 0 20 0 8 0 34651103 135405568 12992 33554432000 134512640 146736320 4289570272 4151398000 144580073 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=1739/tid=1766] ppid=1737 vsize=132232 CPUtime=114.76 cores=1,3,5,7
/proc/1739/task/1766/stat : 1766 (pb_cplex) S 1737 1737 1671 0 -1 4202560 12510 0 0 0 11402 74 0 0 20 0 8 0 34651103 135405568 12992 33554432000 134512640 146736320 4289570272 4117790320 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=1739/tid=1767] ppid=1737 vsize=132232 CPUtime=115.72 cores=1,3,5,7
/proc/1739/task/1767/stat : 1767 (pb_cplex) S 1737 1737 1671 0 -1 4202560 11808 0 0 0 11500 72 0 0 20 0 8 0 34651103 135405568 12992 33554432000 134512640 146736320 4289570272 4097830512 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=1739/tid=1768] ppid=1737 vsize=132232 CPUtime=114.49 cores=1,3,5,7
/proc/1739/task/1768/stat : 1768 (pb_cplex) S 1737 1737 1671 0 -1 4202560 22522 0 0 0 11377 72 0 0 20 0 8 0 34651103 135405568 12992 33554432000 134512640 146736320 4289570272 4087344752 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=1739/tid=1769] ppid=1737 vsize=132232 CPUtime=114.68 cores=1,3,5,7
/proc/1739/task/1769/stat : 1769 (pb_cplex) S 1737 1737 1671 0 -1 4202560 33135 0 0 0 11385 83 0 0 20 0 8 0 34651103 135405568 12992 33554432000 134512640 146736320 4289570272 4070567536 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 940.53
Current children cumulated vsize (KiB) 141380

[startup+342.3 s]
/proc/loadavg: 11.29 7.33 3.26 16/195 1954
/proc/meminfo: memFree=29518152/32873844 swapFree=7124/7124
[pid=1737] ppid=1732 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1737/stat : 1737 (run) S 1732 1737 1671 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 34650921 9367552 270 33554432000 4194304 5064252 140734486019408 140734486018320 215143310926 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1737/statm: 2287 270 229 213 0 72 0
[pid=1739] ppid=1737 vsize=132232 CPUtime=1145.28 cores=1,3,5,7
/proc/1739/stat : 1739 (pb_cplex) R 1737 1737 1671 0 -1 4202496 179329 0 0 0 113815 713 0 0 20 0 8 0 34650921 135405568 12945 33554432000 134512640 146736320 4289570272 4289566700 144580332 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 3 0 0
/proc/1739/statm: 33058 12945 924 2985 0 29992 0
[pid=1739/tid=1763] ppid=1737 vsize=132232 CPUtime=140.23 cores=1,3,5,7
/proc/1739/task/1763/stat : 1763 (pb_cplex) R 1737 1737 1671 0 -1 4202560 7853 0 0 0 13936 87 0 0 20 0 8 0 34651103 135405568 12945 33554432000 134512640 146736320 4289570272 4130423784 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=1739/tid=1764] ppid=1737 vsize=132232 CPUtime=141.01 cores=1,3,5,7
/proc/1739/task/1764/stat : 1764 (pb_cplex) R 1737 1737 1671 0 -1 4202560 16763 0 0 0 14017 84 0 0 20 0 8 0 34651103 135405568 12945 33554432000 134512640 146736320 4289570272 4140910104 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=1739/tid=1765] ppid=1737 vsize=132232 CPUtime=139.93 cores=1,3,5,7
/proc/1739/task/1765/stat : 1765 (pb_cplex) R 1737 1737 1671 0 -1 4202560 12458 0 0 0 13906 87 0 0 20 0 8 0 34651103 135405568 12945 33554432000 134512640 146736320 4289570272 4151395304 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=1739/tid=1766] ppid=1737 vsize=132232 CPUtime=140.26 cores=1,3,5,7
/proc/1739/task/1766/stat : 1766 (pb_cplex) R 1737 1737 1671 0 -1 4202560 14087 0 0 0 13937 89 0 0 20 0 8 0 34651103 135405568 12945 33554432000 134512640 146736320 4289570272 4117790380 144580332 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=1739/tid=1767] ppid=1737 vsize=132232 CPUtime=141.31 cores=1,3,5,7
/proc/1739/task/1767/stat : 1767 (pb_cplex) R 1737 1737 1671 0 -1 4202560 12668 0 0 0 14043 88 0 0 20 0 8 0 34651103 135405568 12945 33554432000 134512640 146736320 4289570272 4097827408 136403109 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=1739/tid=1768] ppid=1737 vsize=132232 CPUtime=139.86 cores=1,3,5,7
/proc/1739/task/1768/stat : 1768 (pb_cplex) R 1737 1737 1671 0 -1 4202560 23543 0 0 0 13898 88 0 0 20 0 8 0 34651103 135405568 12945 33554432000 134512640 146736320 4289570272 4087344784 144580073 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=1739/tid=1769] ppid=1737 vsize=132232 CPUtime=140.01 cores=1,3,5,7
/proc/1739/task/1769/stat : 1769 (pb_cplex) S 1737 1737 1671 0 -1 4202560 36634 0 0 0 13901 100 0 0 20 0 8 0 34651103 135405568 12945 33554432000 134512640 146736320 4289570272 4070567568 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1145.28
Current children cumulated vsize (KiB) 141380

Solver just ended. Dumping a history of the last processes samples

[startup+342.5 s]
/proc/loadavg: 11.29 7.33 3.26 16/195 1954
/proc/meminfo: memFree=29518152/32873844 swapFree=7124/7124
[pid=1737] ppid=1732 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1737/stat : 1737 (run) S 1732 1737 1671 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 34650921 9367552 270 33554432000 4194304 5064252 140734486019408 140734486018320 215143310926 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1737/statm: 2287 270 229 213 0 72 0
[pid=1739] ppid=1737 vsize=132232 CPUtime=1145.97 cores=1,3,5,7
/proc/1739/stat : 1739 (pb_cplex) S 1737 1737 1671 0 -1 4202496 179329 0 0 0 113883 714 0 0 20 0 8 0 34650921 135405568 12945 33554432000 134512640 146736320 4289570272 4289566672 144580073 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 3 0 0
/proc/1739/statm: 33058 12945 924 2985 0 29992 0
[pid=1739/tid=1763] ppid=1737 vsize=132232 CPUtime=140.33 cores=1,3,5,7
/proc/1739/task/1763/stat : 1763 (pb_cplex) R 1737 1737 1671 0 -1 4202560 7853 0 0 0 13946 87 0 0 20 0 8 0 34651103 135405568 12945 33554432000 134512640 146736320 4289570272 4130423784 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=1739/tid=1764] ppid=1737 vsize=132232 CPUtime=141.07 cores=1,3,5,7
/proc/1739/task/1764/stat : 1764 (pb_cplex) R 1737 1737 1671 0 -1 4202560 16763 0 0 0 14023 84 0 0 20 0 8 0 34651103 135405568 12945 33554432000 134512640 146736320 4289570272 4140909544 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=1739/tid=1765] ppid=1737 vsize=132232 CPUtime=139.98 cores=1,3,5,7
/proc/1739/task/1765/stat : 1765 (pb_cplex) R 1737 1737 1671 0 -1 4202560 12458 0 0 0 13911 87 0 0 20 0 8 0 34651103 135405568 12945 33554432000 134512640 146736320 4289570272 4151395304 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=1739/tid=1766] ppid=1737 vsize=132232 CPUtime=140.37 cores=1,3,5,7
/proc/1739/task/1766/stat : 1766 (pb_cplex) S 1737 1737 1671 0 -1 4202560 14087 0 0 0 13948 89 0 0 20 0 8 0 34651103 135405568 12945 33554432000 134512640 146736320 4289570272 4117790352 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=1739/tid=1767] ppid=1737 vsize=132232 CPUtime=141.41 cores=1,3,5,7
/proc/1739/task/1767/stat : 1767 (pb_cplex) S 1737 1737 1671 0 -1 4202560 12668 0 0 0 14053 88 0 0 20 0 8 0 34651103 135405568 12945 33554432000 134512640 146736320 4289570272 4097830544 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=1739/tid=1768] ppid=1737 vsize=132232 CPUtime=139.95 cores=1,3,5,7
/proc/1739/task/1768/stat : 1768 (pb_cplex) S 1737 1737 1671 0 -1 4202560 23543 0 0 0 13907 88 0 0 20 0 8 0 34651103 135405568 12945 33554432000 134512640 146736320 4289570272 4087344784 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=1739/tid=1769] ppid=1737 vsize=132232 CPUtime=140.09 cores=1,3,5,7
/proc/1739/task/1769/stat : 1769 (pb_cplex) R 1737 1737 1671 0 -1 4202560 36634 0 0 0 13909 100 0 0 20 0 8 0 34651103 135405568 12945 33554432000 134512640 146736320 4289570272 4070567568 144580073 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1145.97
Current children cumulated vsize (KiB) 141380

[startup+342.6 s]
/proc/loadavg: 11.29 7.33 3.26 16/195 1954
/proc/meminfo: memFree=29518152/32873844 swapFree=7124/7124
[pid=1737] ppid=1732 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1737/stat : 1737 (run) S 1732 1737 1671 0 -1 4202496 355 0 0 0 0 0 0 0 20 0 1 0 34650921 9367552 270 33554432000 4194304 5064252 140734486019408 140734486018320 215143310926 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1737/statm: 2287 270 229 213 0 72 0
[pid=1739] ppid=1737 vsize=132232 CPUtime=1146.14 cores=1,3,5,7
/proc/1739/stat : 1739 (pb_cplex) S 1737 1737 1671 0 -1 4202496 179329 0 0 0 113900 714 0 0 20 0 8 0 34650921 135405568 12945 33554432000 134512640 146736320 4289570272 4289566672 144580073 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 3 0 0
/proc/1739/statm: 33058 12945 924 2985 0 29992 0
[pid=1739/tid=1763] ppid=1737 vsize=132232 CPUtime=140.41 cores=1,3,5,7
/proc/1739/task/1763/stat : 1763 (pb_cplex) S 1737 1737 1671 0 -1 4202560 7853 0 0 0 13954 87 0 0 20 0 8 0 34651103 135405568 12945 33554432000 134512640 146736320 4289570272 4130426512 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=1739/tid=1764] ppid=1737 vsize=132232 CPUtime=141.12 cores=1,3,5,7
/proc/1739/task/1764/stat : 1764 (pb_cplex) R 1737 1737 1671 0 -1 4202560 16763 0 0 0 14028 84 0 0 20 0 8 0 34651103 135405568 12945 33554432000 134512640 146736320 4289570272 4140908752 136008165 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=1739/tid=1765] ppid=1737 vsize=132232 CPUtime=140.01 cores=1,3,5,7
/proc/1739/task/1765/stat : 1765 (pb_cplex) S 1737 1737 1671 0 -1 4202560 12458 0 0 0 13913 88 0 0 20 0 8 0 34651103 135405568 12945 33554432000 134512640 146736320 4289570272 4151398032 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=1739/tid=1766] ppid=1737 vsize=132232 CPUtime=140.37 cores=1,3,5,7
/proc/1739/task/1766/stat : 1766 (pb_cplex) S 1737 1737 1671 0 -1 4202560 14087 0 0 0 13948 89 0 0 20 0 8 0 34651103 135405568 12945 33554432000 134512640 146736320 4289570272 4117790352 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=1739/tid=1767] ppid=1737 vsize=132232 CPUtime=141.41 cores=1,3,5,7
/proc/1739/task/1767/stat : 1767 (pb_cplex) S 1737 1737 1671 0 -1 4202560 12668 0 0 0 14053 88 0 0 20 0 8 0 34651103 135405568 12945 33554432000 134512640 146736320 4289570272 4097830544 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=1739/tid=1768] ppid=1737 vsize=132232 CPUtime=139.95 cores=1,3,5,7
/proc/1739/task/1768/stat : 1768 (pb_cplex) S 1737 1737 1671 0 -1 4202560 23543 0 0 0 13907 88 0 0 20 0 8 0 34651103 135405568 12945 33554432000 134512640 146736320 4289570272 4087344784 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=1739/tid=1769] ppid=1737 vsize=132232 CPUtime=140.09 cores=1,3,5,7
/proc/1739/task/1769/stat : 1769 (pb_cplex) S 1737 1737 1671 0 -1 4202560 36634 0 0 0 13909 100 0 0 20 0 8 0 34651103 135405568 12945 33554432000 134512640 146736320 4289570272 4070567568 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
Current children cumulated CPU time (s) 1146.14
Current children cumulated vsize (KiB) 141380

Child status: 0

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 1737 and gives
#  childrusage.ru_utime.tv_sec=1138
#  childrusage.ru_utime.tv_usec=494922
#  childrusage.ru_stime.tv_sec=7
#  childrusage.ru_stime.tv_usec=350882
# CPU time returned by wait4() is 1145.85
# while last known CPU time is 1146.14
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 342.648
CPU time (s): 1146.14
CPU user time (s): 1139
CPU system time (s): 7.14
CPU usage (%): 334.495
Max. virtual memory (cumulated for all children) (KiB): 152444

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1138.49
system time used= 7.35088
maximum resident set size= 58208
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 179707
page faults= 0
swaps= 0
block input operations= 80
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 224051
involuntary context switches= 75142

runsolver used 1.13283 second user time and 2.34464 second system time

The end

Launcher Data 

Begin job on node133 at 2012-06-09 15:00:39
IDJOB=3729842
IDBENCH=1351
IDSOLVER=2325
FILE ID=node133/3729842-1339246838
RUNJOBID= node133-1339246833-1689
PBS_JOBID= 14644348
Free space on /tmp= 68388 MiB

SOLVER NAME= PB10: pb_cplex 2010-06-29
BENCH NAME= PB06//final/normalized-PB06/SATUNSAT-SMALLINT/submitted-PB06/namasivayam/tsp/normalized-t3002.11tsp11.1900670167.opb
COMMAND LINE= run BENCHNAME
CONVERSION SCRIPT= PBconversionToLinear BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3729842-1339246838/watcher-3729842-1339246838 -o /tmp/evaluation-result-3729842-1339246838/solver-3729842-1339246838 -C 1800 -W 1900 -M 15500  run HOME/instance-3729842-1339246838.opb

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 3972d8c24730f4742ee881de3ad25c62
RANDOM SEED=1674398657

node133.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.139
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.27
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.139
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.139
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.139
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.139
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.139
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.139
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.52
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.139
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        30381280 kB
Buffers:          114640 kB
Cached:          1523100 kB
SwapCached:         2204 kB
Active:          1612760 kB
Inactive:          28252 kB
Active(anon):       2860 kB
Inactive(anon):     3468 kB
Active(file):    1609900 kB
Inactive(file):    24784 kB
Unevictable:        7124 kB
Mlocked:            7124 kB
SwapTotal:      67108856 kB
SwapFree:       67100196 kB
Dirty:             33204 kB
Writeback:             0 kB
AnonPages:          9720 kB
Mapped:             7164 kB
Shmem:                 0 kB
Slab:             712752 kB
SReclaimable:     154896 kB
SUnreclaim:       557856 kB
KernelStack:        1432 kB
PageTables:         4000 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     102712 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346376 kB
VmallocChunk:   34341987860 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

CONVERSION COMMAND LINE= PBconversionToLinear HOME/instance-3729842-1339246838.opb

Free space on /tmp at the end= 68348 MiB
End job on node133 at 2012-06-09 15:06:23