Trace number 2697441

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
pb_cplex 2010-07-01? (TO) 1800.27 422.316

General information on the benchmark

Name/SOFT-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/
tsp/normalized-t3002.11tsp11.1900582300--soft-0-100-0.wbo
MD5SUM0a3bbeff5fae384bd8f72d98174965d3
Bench CategorySOFT-SMALLINT-LIN (only soft constraints, small integers, linear constraints)
Best result obtained on this benchmarkMOPT
Best cost obtained on this benchmark1
Best CPU time to get the best result obtained on this benchmark385.443
Max-Satisfiable
Max-(Un)Satisfiability was proved
Best value of the cost
Optimality of the best cost was proved
Number of variables231
Total number of constraints2707
Number of soft constraints2707
Number of constraints which are clauses2684
Number of constraints which are cardinality constraints (but not clauses)22
Number of constraints which are nor clauses,nor cardinality constraints1
Minimum length of a constraint2
Maximum length of a constraint110
Top cost 134314
Min constraint cost 1
Max constraint cost 100
Sum of constraints costs 134313
Biggest number in a constraint 25
Number of bits of the biggest number in a constraint 5
Biggest sum of numbers in a constraint 683
Number of bits of the biggest sum of numbers10
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c pb_cplex 1.0 - 20100701
0.00/0.00	c Parsing wbo file format. File HOME/instance-2697441-1278055758.wbo.
0.00/0.00	c Instance file HOME/instance-2697441-1278055758.wbo
0.00/0.00	c File size is 96737 bytes.
1.39/1.43	c Highest Coefficient sum: 683
1.39/1.43	c  IBM ILOG License Manager: "IBM ILOG Optimization Suite for Academic Initiative" is accessing CPLEX 12 with option(s): "e m b q ".
1.39/1.44	c ============================[ Problem Statistics ]=============================
1.39/1.44	c |                                                                             |
1.39/1.44	c |  Parsing time:          1.43         s                                      |
1.39/1.44	c |  Number Variables:      2938                                                |
1.39/1.44	c |  Number Constraints:    2708                                                |
1.39/1.44	c ===============================================================================
1800.18/422.31	c Remaining time 1798.56 sec.
1800.18/422.31	c 
1800.18/422.31	c *** INTERRUPTED ***
1800.18/422.31	s UNKNOWN
1800.18/422.31	c Total CPU time              : 1785.43 s
1800.18/422.31	c Memory used           : 139.81 MB
1800.18/422.31	c CPU time              : 1785.43 s
1800.18/422.31	c 
1800.18/422.31	c *** INTERRUPTED ***

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver version 3.2.9 (svn:492) (c) roussel@cril.univ-artois.fr

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-2697441-1278055758/watcher-2697441-1278055758 -o /tmp/evaluation-result-2697441-1278055758/solver-2697441-1278055758 -C 1800 -W 2000 -M 3800 run HOME/instance-2697441-1278055758.wbo 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 3891200 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 3942400 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.20 6.29 4.38 4/179 563
/proc/meminfo: memFree=31058700/32950928 swapFree=67111528/67111528
[pid=563] ppid=561 vsize=8700 CPUtime=0
/proc/563/stat : 563 (run) S 561 563 32241 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 179167564 8908800 249 4089446400 4194304 4922060 140733965374512 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/563/statm: 2175 249 208 178 0 68 0

[startup+0.0453071 s]
/proc/loadavg: 7.20 6.29 4.38 4/179 563
/proc/meminfo: memFree=31058700/32950928 swapFree=67111528/67111528
[pid=563] ppid=561 vsize=8700 CPUtime=0
/proc/563/stat : 563 (run) S 561 563 32241 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 179167564 8908800 249 4089446400 4194304 4922060 140733965374512 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/563/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.100297 s]
/proc/loadavg: 7.20 6.29 4.38 4/179 563
/proc/meminfo: memFree=31058700/32950928 swapFree=67111528/67111528
[pid=563] ppid=561 vsize=8700 CPUtime=0
/proc/563/stat : 563 (run) S 561 563 32241 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 179167564 8908800 249 4089446400 4194304 4922060 140733965374512 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/563/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.30026 s]
/proc/loadavg: 7.20 6.29 4.38 4/179 563
/proc/meminfo: memFree=31058700/32950928 swapFree=67111528/67111528
[pid=563] ppid=561 vsize=8700 CPUtime=0
/proc/563/stat : 563 (run) S 561 563 32241 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 179167564 8908800 249 4089446400 4194304 4922060 140733965374512 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/563/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.70019 s]
/proc/loadavg: 7.20 6.29 4.38 4/179 563
/proc/meminfo: memFree=31058700/32950928 swapFree=67111528/67111528
[pid=563] ppid=561 vsize=8700 CPUtime=0
/proc/563/stat : 563 (run) S 561 563 32241 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 179167564 8908800 249 4089446400 4194304 4922060 140733965374512 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/563/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+1.50104 s]
/proc/loadavg: 7.20 6.29 4.38 3/181 565
/proc/meminfo: memFree=31056700/32950928 swapFree=67111528/67111528
[pid=563] ppid=561 vsize=8700 CPUtime=0
/proc/563/stat : 563 (run) S 561 563 32241 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 179167564 8908800 249 4089446400 4194304 4922060 140733965374512 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/563/statm: 2175 249 208 178 0 68 0
[pid=564] ppid=563 vsize=27332 CPUtime=1.49
/proc/564/stat : 564 (pb_cplex) R 563 563 32241 0 -1 4202496 3602 0 0 0 149 0 0 0 25 0 1 0 179167564 27987968 3132 4089446400 134512640 146736320 4289025232 18446744073709551615 139022580 0 0 4096 16387 0 0 0 17 3 0 0 0
/proc/564/statm: 6833 3132 979 2985 0 3375 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 36032

[startup+3.10081 s]
/proc/loadavg: 6.78 6.21 4.36 3/188 579
/proc/meminfo: memFree=31017832/32950928 swapFree=67111528/67111528
[pid=563] ppid=561 vsize=8700 CPUtime=0
/proc/563/stat : 563 (run) S 561 563 32241 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 179167564 8908800 249 4089446400 4194304 4922060 140733965374512 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/563/statm: 2175 249 208 178 0 68 0
[pid=564] ppid=563 vsize=97924 CPUtime=6.27
/proc/564/stat : 564 (pb_cplex) R 563 563 32241 0 -1 4202496 18439 0 0 0 618 9 0 0 16 0 1 0 179167564 100274176 9425 4089446400 134512640 146736320 4289025232 18446744073709551615 137084966 0 0 4096 16387 0 0 0 17 1 0 0 0
/proc/564/statm: 24481 9425 1071 2985 0 21023 0
Current children cumulated CPU time (s) 6.27
Current children cumulated vsize (KiB) 106624

[startup+6.33416 s]
/proc/loadavg: 6.78 6.21 4.36 10/195 626
/proc/meminfo: memFree=30887460/32950928 swapFree=67111528/67111528
[pid=563] ppid=561 vsize=8700 CPUtime=0
/proc/563/stat : 563 (run) S 561 563 32241 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 179167564 8908800 249 4089446400 4194304 4922060 140733965374512 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/563/statm: 2175 249 208 178 0 68 0
[pid=564] ppid=563 vsize=145772 CPUtime=18.37
/proc/564/stat : 564 (pb_cplex) R 563 563 32241 0 -1 4202496 30538 0 0 0 1817 20 0 0 19 0 8 0 179167564 149270528 14117 4089446400 134512640 146736320 4289025232 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0
/proc/564/statm: 36443 14117 1081 2985 0 32985 0
[pid=564/tid=613] ppid=563 vsize=145772 CPUtime=1.45
/proc/564/task/613/stat : 613 (pb_cplex) S 563 563 32241 0 -1 4202560 949 0 0 0 145 0 0 0 16 0 8 0 179167948 149270528 14117 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 7 0 0 0
[pid=564/tid=614] ppid=563 vsize=145772 CPUtime=1.24
/proc/564/task/614/stat : 614 (pb_cplex) R 563 563 32241 0 -1 4202560 860 0 0 0 124 0 0 0 16 0 8 0 179167948 149270528 14117 4089446400 134512640 146736320 4289025232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=564/tid=615] ppid=563 vsize=145772 CPUtime=1.61
/proc/564/task/615/stat : 615 (pb_cplex) R 563 563 32241 0 -1 4202560 2805 0 0 0 159 2 0 0 16 0 8 0 179167948 149270528 14117 4089446400 134512640 146736320 4289025232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=564/tid=616] ppid=563 vsize=145772 CPUtime=1.32
/proc/564/task/616/stat : 616 (pb_cplex) R 563 563 32241 0 -1 4202560 497 0 0 0 132 0 0 0 15 0 8 0 179167948 149270528 14117 4089446400 134512640 146736320 4289025232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=564/tid=617] ppid=563 vsize=145772 CPUtime=1.39
/proc/564/task/617/stat : 617 (pb_cplex) R 563 563 32241 0 -1 4202560 1231 0 0 0 138 1 0 0 16 0 8 0 179167948 149270528 14117 4089446400 134512640 146736320 4289025232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=564/tid=618] ppid=563 vsize=145772 CPUtime=1.44
/proc/564/task/618/stat : 618 (pb_cplex) R 563 563 32241 0 -1 4202560 764 0 0 0 143 1 0 0 16 0 8 0 179167948 149270528 14117 4089446400 134512640 146736320 4289025232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=564/tid=619] ppid=563 vsize=145772 CPUtime=1.11
/proc/564/task/619/stat : 619 (pb_cplex) R 563 563 32241 0 -1 4202560 3826 0 0 0 110 1 0 0 15 0 8 0 179167948 149270528 14117 4089446400 134512640 146736320 4289025232 18446744073709551615 135998972 0 2 4096 16387 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 18.37
Current children cumulated vsize (KiB) 154472

[startup+12.714 s]
/proc/loadavg: 6.68 6.21 4.38 10/195 626
/proc/meminfo: memFree=30893376/32950928 swapFree=67111528/67111528
[pid=563] ppid=561 vsize=8700 CPUtime=0
/proc/563/stat : 563 (run) S 561 563 32241 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 179167564 8908800 249 4089446400 4194304 4922060 140733965374512 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/563/statm: 2175 249 208 178 0 68 0
[pid=564] ppid=563 vsize=145772 CPUtime=49.25
/proc/564/stat : 564 (pb_cplex) R 563 563 32241 0 -1 4202496 43329 0 0 0 4874 51 0 0 15 0 8 0 179167564 149270528 14502 4089446400 134512640 146736320 4289025232 18446744073709551615 135854527 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0
/proc/564/statm: 36443 14502 1081 2985 0 32985 0
[pid=564/tid=613] ppid=563 vsize=145772 CPUtime=5.29
/proc/564/task/613/stat : 613 (pb_cplex) R 563 563 32241 0 -1 4202560 1165 0 0 0 525 4 0 0 15 0 8 0 179167948 149270528 14502 4089446400 134512640 146736320 4289025232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=564/tid=614] ppid=563 vsize=145772 CPUtime=5.28
/proc/564/task/614/stat : 614 (pb_cplex) R 563 563 32241 0 -1 4202560 2864 0 0 0 524 4 0 0 15 0 8 0 179167948 149270528 14502 4089446400 134512640 146736320 4289025232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=564/tid=615] ppid=563 vsize=145772 CPUtime=5.52
/proc/564/task/615/stat : 615 (pb_cplex) R 563 563 32241 0 -1 4202560 4215 0 0 0 546 6 0 0 15 0 8 0 179167948 149270528 14502 4089446400 134512640 146736320 4289025232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=564/tid=616] ppid=563 vsize=145772 CPUtime=5.24
/proc/564/task/616/stat : 616 (pb_cplex) R 563 563 32241 0 -1 4202560 745 0 0 0 520 4 0 0 16 0 8 0 179167948 149270528 14502 4089446400 134512640 146736320 4289025232 18446744073709551615 144580332 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=564/tid=617] ppid=563 vsize=145772 CPUtime=5.15
/proc/564/task/617/stat : 617 (pb_cplex) R 563 563 32241 0 -1 4202560 1489 0 0 0 509 6 0 0 15 0 8 0 179167948 149270528 14502 4089446400 134512640 146736320 4289025232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=564/tid=618] ppid=563 vsize=145772 CPUtime=5.3
/proc/564/task/618/stat : 618 (pb_cplex) R 563 563 32241 0 -1 4202560 2509 0 0 0 524 6 0 0 15 0 8 0 179167948 149270528 14502 4089446400 134512640 146736320 4289025232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=564/tid=619] ppid=563 vsize=145772 CPUtime=4.91
/proc/564/task/619/stat : 619 (pb_cplex) R 563 563 32241 0 -1 4202560 10731 0 0 0 486 5 0 0 15 0 8 0 179167948 149270528 14502 4089446400 134512640 146736320 4289025232 18446744073709551615 136403129 0 2 4096 16387 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 49.25
Current children cumulated vsize (KiB) 154472

[startup+25.5007 s]
/proc/loadavg: 6.12 6.10 4.37 3/195 641
/proc/meminfo: memFree=30893384/32950928 swapFree=67111528/67111528
[pid=563] ppid=561 vsize=8700 CPUtime=0
/proc/563/stat : 563 (run) S 561 563 32241 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 179167564 8908800 249 4089446400 4194304 4922060 140733965374512 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/563/statm: 2175 249 208 178 0 68 0
[pid=564] ppid=563 vsize=136048 CPUtime=97.41
/proc/564/stat : 564 (pb_cplex) R 563 563 32241 0 -1 4202496 68509 0 0 0 9638 103 0 0 21 0 8 0 179167564 139313152 12170 4089446400 134512640 146736320 4289025232 18446744073709551615 136007274 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0
/proc/564/statm: 34012 12170 1092 2985 0 30554 0
[pid=564/tid=635] ppid=563 vsize=136048 CPUtime=0
/proc/564/task/635/stat : 635 (pb_cplex) S 563 563 32241 0 -1 4202560 0 0 0 0 0 0 0 0 18 0 8 0 179170077 139313152 12170 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 0 0 0 0
[pid=564/tid=636] ppid=563 vsize=136048 CPUtime=0
/proc/564/task/636/stat : 636 (pb_cplex) S 563 563 32241 0 -1 4202560 0 0 0 0 0 0 0 0 19 0 8 0 179170077 139313152 12170 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 3 0 0 0
[pid=564/tid=637] ppid=563 vsize=136048 CPUtime=0
/proc/564/task/637/stat : 637 (pb_cplex) S 563 563 32241 0 -1 4202560 0 0 0 0 0 0 0 0 19 0 8 0 179170077 139313152 12170 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 3 0 0 0
[pid=564/tid=638] ppid=563 vsize=136048 CPUtime=0
/proc/564/task/638/stat : 638 (pb_cplex) S 563 563 32241 0 -1 4202560 0 0 0 0 0 0 0 0 19 0 8 0 179170077 139313152 12170 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 3 0 0 0
[pid=564/tid=639] ppid=563 vsize=136048 CPUtime=0
/proc/564/task/639/stat : 639 (pb_cplex) S 563 563 32241 0 -1 4202560 1 0 0 0 0 0 0 0 18 0 8 0 179170077 139313152 12170 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 0 0 0 0
[pid=564/tid=640] ppid=563 vsize=136048 CPUtime=0
/proc/564/task/640/stat : 640 (pb_cplex) S 563 563 32241 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 8 0 179170077 139313152 12170 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 0 0 0 0
[pid=564/tid=641] ppid=563 vsize=136048 CPUtime=0
/proc/564/task/641/stat : 641 (pb_cplex) S 563 563 32241 0 -1 4202560 1 0 0 0 0 0 0 0 21 0 8 0 179170077 139313152 12170 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 97.41
Current children cumulated vsize (KiB) 144748

[startup+51.101 s]
/proc/loadavg: 6.20 6.12 4.42 3/195 642
/proc/meminfo: memFree=30898064/32950928 swapFree=67111528/67111528
[pid=563] ppid=561 vsize=8700 CPUtime=0
/proc/563/stat : 563 (run) S 561 563 32241 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 179167564 8908800 249 4089446400 4194304 4922060 140733965374512 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/563/statm: 2175 249 208 178 0 68 0
[pid=564] ppid=563 vsize=139064 CPUtime=211.88
/proc/564/stat : 564 (pb_cplex) S 563 563 32241 0 -1 4202496 113616 0 0 0 20997 191 0 0 15 0 8 0 179167564 142401536 13086 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0
/proc/564/statm: 34766 13086 1092 2985 0 31308 0
[pid=564/tid=635] ppid=563 vsize=139064 CPUtime=13.32
/proc/564/task/635/stat : 635 (pb_cplex) S 563 563 32241 0 -1 4202560 8805 0 0 0 1320 12 0 0 16 0 8 0 179170077 142401536 13086 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 7 0 0 0
[pid=564/tid=636] ppid=563 vsize=139064 CPUtime=13.38
/proc/564/task/636/stat : 636 (pb_cplex) S 563 563 32241 0 -1 4202560 4069 0 0 0 1328 10 0 0 15 0 8 0 179170077 142401536 13086 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 0 0 0 0
[pid=564/tid=637] ppid=563 vsize=139064 CPUtime=13.32
/proc/564/task/637/stat : 637 (pb_cplex) S 563 563 32241 0 -1 4202560 3690 0 0 0 1321 11 0 0 15 0 8 0 179170077 142401536 13086 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 3 0 0 0
[pid=564/tid=638] ppid=563 vsize=139064 CPUtime=14.15
/proc/564/task/638/stat : 638 (pb_cplex) S 563 563 32241 0 -1 4202560 6885 0 0 0 1408 7 0 0 15 0 8 0 179170077 142401536 13086 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 4 0 0 0
[pid=564/tid=639] ppid=563 vsize=139064 CPUtime=14.37
/proc/564/task/639/stat : 639 (pb_cplex) S 563 563 32241 0 -1 4202560 3952 0 0 0 1426 11 0 0 15 0 8 0 179170077 142401536 13086 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 6 0 0 0
[pid=564/tid=640] ppid=563 vsize=139064 CPUtime=16.07
/proc/564/task/640/stat : 640 (pb_cplex) R 563 563 32241 0 -1 4202560 13770 0 0 0 1594 13 0 0 16 0 8 0 179170077 142401536 13086 4089446400 134512640 146736320 4289025232 18446744073709551615 136388603 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=564/tid=641] ppid=563 vsize=139064 CPUtime=13.54
/proc/564/task/641/stat : 641 (pb_cplex) S 563 563 32241 0 -1 4202560 1532 0 0 0 1344 10 0 0 15 0 8 0 179170077 142401536 13086 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 211.88
Current children cumulated vsize (KiB) 147764

[startup+102.308 s]
/proc/loadavg: 9.09 6.95 4.80 10/195 643
/proc/meminfo: memFree=30831344/32950928 swapFree=67111528/67111528
[pid=563] ppid=561 vsize=8700 CPUtime=0
/proc/563/stat : 563 (run) S 561 563 32241 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 179167564 8908800 249 4089446400 4194304 4922060 140733965374512 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/563/statm: 2175 249 208 178 0 68 0
[pid=564] ppid=563 vsize=144184 CPUtime=326.59
/proc/564/stat : 564 (pb_cplex) R 563 563 32241 0 -1 4202496 140654 0 0 0 32381 278 0 0 25 0 8 0 179167564 147644416 14136 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 0
/proc/564/statm: 36046 14136 1092 2985 0 32588 0
[pid=564/tid=635] ppid=563 vsize=144184 CPUtime=27.41
/proc/564/task/635/stat : 635 (pb_cplex) S 563 563 32241 0 -1 4202560 14907 0 0 0 2718 23 0 0 15 0 8 0 179170077 147644416 14136 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 6 0 0 0
[pid=564/tid=636] ppid=563 vsize=144184 CPUtime=27.68
/proc/564/task/636/stat : 636 (pb_cplex) S 563 563 32241 0 -1 4202560 10294 0 0 0 2750 18 0 0 15 0 8 0 179170077 147644416 14136 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 0 0 0 0
[pid=564/tid=637] ppid=563 vsize=144184 CPUtime=27.41
/proc/564/task/637/stat : 637 (pb_cplex) S 563 563 32241 0 -1 4202560 4532 0 0 0 2720 21 0 0 15 0 8 0 179170077 147644416 14136 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 4 0 0 0
[pid=564/tid=638] ppid=563 vsize=144184 CPUtime=28.29
/proc/564/task/638/stat : 638 (pb_cplex) S 563 563 32241 0 -1 4202560 8837 0 0 0 2812 17 0 0 15 0 8 0 179170077 147644416 14136 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 3 0 0 0

################
# More data... #
################

/proc/564/task/639/stat : 639 (pb_cplex) S 563 563 32241 0 -1 4202560 8029 0 0 0 2820 21 0 0 15 0 8 0 179170077 147644416 14136 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 1 0 0 0
[pid=564/tid=640] ppid=563 vsize=144184 CPUtime=29.94
/proc/564/task/640/stat : 640 (pb_cplex) S 563 563 32241 0 -1 4202560 14139 0 0 0 2970 24 0 0 15 0 8 0 179170077 147644416 14136 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 2 0 0 0
[pid=564/tid=641] ppid=563 vsize=144184 CPUtime=27.8
/proc/564/task/641/stat : 641 (pb_cplex) S 563 563 32241 0 -1 4202560 7492 0 0 0 2757 23 0 0 15 0 8 0 179170077 147644416 14136 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 326.59
Current children cumulated vsize (KiB) 152884

[startup+162.301 s]
/proc/loadavg: 10.15 7.67 5.18 14/195 644
/proc/meminfo: memFree=30850380/32950928 swapFree=67111528/67111528
[pid=563] ppid=561 vsize=8700 CPUtime=0
/proc/563/stat : 563 (run) S 561 563 32241 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 179167564 8908800 249 4089446400 4194304 4922060 140733965374512 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/563/statm: 2175 249 208 178 0 68 0
[pid=564] ppid=563 vsize=144184 CPUtime=514.46
/proc/564/stat : 564 (pb_cplex) R 563 563 32241 0 -1 4202496 149685 0 0 0 51013 433 0 0 19 0 8 0 179167564 147644416 14179 4089446400 134512640 146736320 4289025232 18446744073709551615 136403112 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0
/proc/564/statm: 36046 14179 1092 2985 0 32588 0
[pid=564/tid=635] ppid=563 vsize=144184 CPUtime=50.64
/proc/564/task/635/stat : 635 (pb_cplex) S 563 563 32241 0 -1 4202560 17173 0 0 0 5022 42 0 0 15 0 8 0 179170077 147644416 14179 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 2 0 0 0
[pid=564/tid=636] ppid=563 vsize=144184 CPUtime=51.02
/proc/564/task/636/stat : 636 (pb_cplex) S 563 563 32241 0 -1 4202560 12285 0 0 0 5072 30 0 0 15 0 8 0 179170077 147644416 14179 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 0 0 0 0
[pid=564/tid=637] ppid=563 vsize=144184 CPUtime=50.4
/proc/564/task/637/stat : 637 (pb_cplex) S 563 563 32241 0 -1 4202560 5375 0 0 0 5001 39 0 0 16 0 8 0 179170077 147644416 14179 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 4 0 0 0
[pid=564/tid=638] ppid=563 vsize=144184 CPUtime=51.56
/proc/564/task/638/stat : 638 (pb_cplex) S 563 563 32241 0 -1 4202560 9572 0 0 0 5118 38 0 0 15 0 8 0 179170077 147644416 14179 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 3 0 0 0
[pid=564/tid=639] ppid=563 vsize=144184 CPUtime=51.77
/proc/564/task/639/stat : 639 (pb_cplex) S 563 563 32241 0 -1 4202560 9042 0 0 0 5136 41 0 0 15 0 8 0 179170077 147644416 14179 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 1 0 0 0
[pid=564/tid=640] ppid=563 vsize=144184 CPUtime=53.08
/proc/564/task/640/stat : 640 (pb_cplex) S 563 563 32241 0 -1 4202560 14407 0 0 0 5265 43 0 0 15 0 8 0 179170077 147644416 14179 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 6 0 0 0
[pid=564/tid=641] ppid=563 vsize=144184 CPUtime=51.37
/proc/564/task/641/stat : 641 (pb_cplex) S 563 563 32241 0 -1 4202560 9394 0 0 0 5093 44 0 0 15 0 8 0 179170077 147644416 14179 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 514.46
Current children cumulated vsize (KiB) 152884

[startup+222.307 s]
/proc/loadavg: 11.25 8.35 5.56 17/195 646
/proc/meminfo: memFree=30765440/32950928 swapFree=67111528/67111528
[pid=563] ppid=561 vsize=8700 CPUtime=0
/proc/563/stat : 563 (run) S 561 563 32241 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 179167564 8908800 249 4089446400 4194304 4922060 140733965374512 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/563/statm: 2175 249 208 178 0 68 0
[pid=564] ppid=563 vsize=145208 CPUtime=826.24
/proc/564/stat : 564 (pb_cplex) R 563 563 32241 0 -1 4202496 170849 0 0 0 81925 699 0 0 16 0 8 0 179167564 148692992 14403 4089446400 134512640 146736320 4289025232 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/564/statm: 36302 14403 1092 2985 0 32844 0
[pid=564/tid=635] ppid=563 vsize=145208 CPUtime=89.78
/proc/564/task/635/stat : 635 (pb_cplex) R 563 563 32241 0 -1 4202560 18883 0 0 0 8901 77 0 0 16 0 8 0 179170077 148692992 14403 4089446400 134512640 146736320 4289025232 18446744073709551615 135999265 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=564/tid=636] ppid=563 vsize=145208 CPUtime=90.26
/proc/564/task/636/stat : 636 (pb_cplex) R 563 563 32241 0 -1 4202560 15256 0 0 0 8961 65 0 0 15 0 8 0 179170077 148692992 14403 4089446400 134512640 146736320 4289025232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=564/tid=637] ppid=563 vsize=145208 CPUtime=89.25
/proc/564/task/637/stat : 637 (pb_cplex) R 563 563 32241 0 -1 4202560 6541 0 0 0 8859 66 0 0 16 0 8 0 179170077 148692992 14403 4089446400 134512640 146736320 4289025232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=564/tid=638] ppid=563 vsize=145208 CPUtime=90.29
/proc/564/task/638/stat : 638 (pb_cplex) R 563 563 32241 0 -1 4202560 12034 0 0 0 8955 74 0 0 16 0 8 0 179170077 148692992 14403 4089446400 134512640 146736320 4289025232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=564/tid=639] ppid=563 vsize=145208 CPUtime=90.17
/proc/564/task/639/stat : 639 (pb_cplex) R 563 563 32241 0 -1 4202560 11257 0 0 0 8942 75 0 0 16 0 8 0 179170077 148692992 14403 4089446400 134512640 146736320 4289025232 18446744073709551615 136403117 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=564/tid=640] ppid=563 vsize=145208 CPUtime=91.7
/proc/564/task/640/stat : 640 (pb_cplex) R 563 563 32241 0 -1 4202560 20180 0 0 0 9089 81 0 0 16 0 8 0 179170077 148692992 14403 4089446400 134512640 146736320 4289025232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=564/tid=641] ppid=563 vsize=145208 CPUtime=90.21
/proc/564/task/641/stat : 641 (pb_cplex) R 563 563 32241 0 -1 4202560 14176 0 0 0 8942 79 0 0 16 0 8 0 179170077 148692992 14403 4089446400 134512640 146736320 4289025232 18446744073709551615 136007292 0 2 4096 16387 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 826.24
Current children cumulated vsize (KiB) 153908

[startup+282.301 s]
/proc/loadavg: 13.55 9.58 6.15 10/195 648
/proc/meminfo: memFree=30780440/32950928 swapFree=67111528/67111528
[pid=563] ppid=561 vsize=8700 CPUtime=0
/proc/563/stat : 563 (run) S 561 563 32241 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 179167564 8908800 249 4089446400 4194304 4922060 140733965374512 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/563/statm: 2175 249 208 178 0 68 0
[pid=564] ppid=563 vsize=143160 CPUtime=1173.65
/proc/564/stat : 564 (pb_cplex) S 563 563 32241 0 -1 4202496 195476 0 0 0 116379 986 0 0 15 0 8 0 179167564 146595840 14107 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/564/statm: 35790 14107 1092 2985 0 32332 0
[pid=564/tid=635] ppid=563 vsize=143160 CPUtime=132.93
/proc/564/task/635/stat : 635 (pb_cplex) S 563 563 32241 0 -1 4202560 20733 0 0 0 13178 115 0 0 15 0 8 0 179170077 146595840 14107 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 2 0 0 0
[pid=564/tid=636] ppid=563 vsize=143160 CPUtime=133.56
/proc/564/task/636/stat : 636 (pb_cplex) S 563 563 32241 0 -1 4202560 20371 0 0 0 13254 102 0 0 15 0 8 0 179170077 146595840 14107 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 6 0 0 0
[pid=564/tid=637] ppid=563 vsize=143160 CPUtime=132.75
/proc/564/task/637/stat : 637 (pb_cplex) S 563 563 32241 0 -1 4202560 10107 0 0 0 13171 104 0 0 15 0 8 0 179170077 146595840 14107 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 1 0 0 0
[pid=564/tid=638] ppid=563 vsize=143160 CPUtime=133.6
/proc/564/task/638/stat : 638 (pb_cplex) S 563 563 32241 0 -1 4202560 14877 0 0 0 13246 114 0 0 15 0 8 0 179170077 146595840 14107 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 7 0 0 0
[pid=564/tid=639] ppid=563 vsize=143160 CPUtime=133.43
/proc/564/task/639/stat : 639 (pb_cplex) S 563 563 32241 0 -1 4202560 13800 0 0 0 13231 112 0 0 15 0 8 0 179170077 146595840 14107 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 4 0 0 0
[pid=564/tid=640] ppid=563 vsize=143160 CPUtime=135.1
/proc/564/task/640/stat : 640 (pb_cplex) S 563 563 32241 0 -1 4202560 25642 0 0 0 13389 121 0 0 15 0 8 0 179170077 146595840 14107 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 3 0 0 0
[pid=564/tid=641] ppid=563 vsize=143160 CPUtime=133.99
/proc/564/task/641/stat : 641 (pb_cplex) R 563 563 32241 0 -1 4202560 16635 0 0 0 13282 117 0 0 25 0 8 0 179170077 146595840 14107 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1173.65
Current children cumulated vsize (KiB) 151860

[startup+342.301 s]
/proc/loadavg: 11.42 9.71 6.42 10/195 650
/proc/meminfo: memFree=30785020/32950928 swapFree=67111528/67111528
[pid=563] ppid=561 vsize=8700 CPUtime=0
/proc/563/stat : 563 (run) S 561 563 32241 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 179167564 8908800 249 4089446400 4194304 4922060 140733965374512 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/563/statm: 2175 249 208 178 0 68 0
[pid=564] ppid=563 vsize=143160 CPUtime=1260.38
/proc/564/stat : 564 (pb_cplex) S 563 563 32241 0 -1 4202496 199953 0 0 0 124978 1060 0 0 15 0 8 0 179167564 146595840 14107 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/564/statm: 35790 14107 1092 2985 0 32332 0
[pid=564/tid=635] ppid=563 vsize=143160 CPUtime=143.69
/proc/564/task/635/stat : 635 (pb_cplex) S 563 563 32241 0 -1 4202560 21349 0 0 0 14246 123 0 0 15 0 8 0 179170077 146595840 14107 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 2 0 0 0
[pid=564/tid=636] ppid=563 vsize=143160 CPUtime=144.43
/proc/564/task/636/stat : 636 (pb_cplex) S 563 563 32241 0 -1 4202560 21032 0 0 0 14331 112 0 0 15 0 8 0 179170077 146595840 14107 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 6 0 0 0
[pid=564/tid=637] ppid=563 vsize=143160 CPUtime=143.63
/proc/564/task/637/stat : 637 (pb_cplex) S 563 563 32241 0 -1 4202560 10604 0 0 0 14251 112 0 0 15 0 8 0 179170077 146595840 14107 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 1 0 0 0
[pid=564/tid=638] ppid=563 vsize=143160 CPUtime=144.4
/proc/564/task/638/stat : 638 (pb_cplex) S 563 563 32241 0 -1 4202560 15688 0 0 0 14317 123 0 0 15 0 8 0 179170077 146595840 14107 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 7 0 0 0
[pid=564/tid=639] ppid=563 vsize=143160 CPUtime=144.05
/proc/564/task/639/stat : 639 (pb_cplex) S 563 563 32241 0 -1 4202560 14664 0 0 0 14284 121 0 0 15 0 8 0 179170077 146595840 14107 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 4 0 0 0
[pid=564/tid=640] ppid=563 vsize=143160 CPUtime=145.84
/proc/564/task/640/stat : 640 (pb_cplex) S 563 563 32241 0 -1 4202560 25719 0 0 0 14455 129 0 0 15 0 8 0 179170077 146595840 14107 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 18446604453952691072 0 0 -1 3 0 0 0
[pid=564/tid=641] ppid=563 vsize=143160 CPUtime=145.39
/proc/564/task/641/stat : 641 (pb_cplex) R 563 563 32241 0 -1 4202560 17214 0 0 0 14406 133 0 0 25 0 8 0 179170077 146595840 14107 4089446400 134512640 146736320 4289025232 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1260.38
Current children cumulated vsize (KiB) 151860

[startup+402.301 s]
/proc/loadavg: 8.47 9.17 6.43 9/179 673
/proc/meminfo: memFree=31269088/32950928 swapFree=67111528/67111528
[pid=563] ppid=561 vsize=8700 CPUtime=0
/proc/563/stat : 563 (run) S 561 563 32241 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 179167564 8908800 249 4089446400 4194304 4922060 140733965374512 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/563/statm: 2175 249 208 178 0 68 0
[pid=564] ppid=563 vsize=143160 CPUtime=1662.47
/proc/564/stat : 564 (pb_cplex) R 563 563 32241 0 -1 4202496 220307 0 0 0 164867 1380 0 0 16 0 8 0 179167564 146595840 14108 4089446400 134512640 146736320 4289025232 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0
/proc/564/statm: 35790 14108 1092 2985 0 32332 0
[pid=564/tid=635] ppid=563 vsize=143160 CPUtime=194.11
/proc/564/task/635/stat : 635 (pb_cplex) R 563 563 32241 0 -1 4202560 23066 0 0 0 19246 165 0 0 16 0 8 0 179170077 146595840 14108 4089446400 134512640 146736320 4289025232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=564/tid=636] ppid=563 vsize=143160 CPUtime=194.36
/proc/564/task/636/stat : 636 (pb_cplex) R 563 563 32241 0 -1 4202560 23134 0 0 0 19281 155 0 0 16 0 8 0 179170077 146595840 14108 4089446400 134512640 146736320 4289025232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=564/tid=637] ppid=563 vsize=143160 CPUtime=193.75
/proc/564/task/637/stat : 637 (pb_cplex) R 563 563 32241 0 -1 4202560 13355 0 0 0 19225 150 0 0 15 0 8 0 179170077 146595840 14108 4089446400 134512640 146736320 4289025232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=564/tid=638] ppid=563 vsize=143160 CPUtime=194.35
/proc/564/task/638/stat : 638 (pb_cplex) R 563 563 32241 0 -1 4202560 17858 0 0 0 19276 159 0 0 16 0 8 0 179170077 146595840 14108 4089446400 134512640 146736320 4289025232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=564/tid=639] ppid=563 vsize=143160 CPUtime=194.34
/proc/564/task/639/stat : 639 (pb_cplex) R 563 563 32241 0 -1 4202560 18393 0 0 0 19272 162 0 0 16 0 8 0 179170077 146595840 14108 4089446400 134512640 146736320 4289025232 18446744073709551615 136384849 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=564/tid=640] ppid=563 vsize=143160 CPUtime=195.92
/proc/564/task/640/stat : 640 (pb_cplex) R 563 563 32241 0 -1 4202560 29595 0 0 0 19421 171 0 0 16 0 8 0 179170077 146595840 14108 4089446400 134512640 146736320 4289025232 18446744073709551615 136007602 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=564/tid=641] ppid=563 vsize=143160 CPUtime=195.94
/proc/564/task/641/stat : 641 (pb_cplex) R 563 563 32241 0 -1 4202560 19404 0 0 0 19422 172 0 0 15 0 8 0 179170077 146595840 14108 4089446400 134512640 146736320 4289025232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1662.47
Current children cumulated vsize (KiB) 151860



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+422.301 s]
/proc/loadavg: 8.05 9.03 6.45 10/179 674
/proc/meminfo: memFree=31268732/32950928 swapFree=67111528/67111528
[pid=563] ppid=561 vsize=8700 CPUtime=0
/proc/563/stat : 563 (run) S 561 563 32241 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 179167564 8908800 249 4089446400 4194304 4922060 140733965374512 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/563/statm: 2175 249 208 178 0 68 0
[pid=564] ppid=563 vsize=143160 CPUtime=1800.18
/proc/564/stat : 564 (pb_cplex) R 563 563 32241 0 -1 4202496 227702 0 0 0 178535 1483 0 0 16 0 8 0 179167564 146595840 14114 4089446400 134512640 146736320 4289025232 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0
/proc/564/statm: 35790 14114 1092 2985 0 32332 0
[pid=564/tid=635] ppid=563 vsize=143160 CPUtime=211.29
/proc/564/task/635/stat : 635 (pb_cplex) R 563 563 32241 0 -1 4202560 23676 0 0 0 20948 181 0 0 16 0 8 0 179170077 146595840 14114 4089446400 134512640 146736320 4289025232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=564/tid=636] ppid=563 vsize=143160 CPUtime=211.52
/proc/564/task/636/stat : 636 (pb_cplex) R 563 563 32241 0 -1 4202560 24069 0 0 0 20982 170 0 0 15 0 8 0 179170077 146595840 14114 4089446400 134512640 146736320 4289025232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=564/tid=637] ppid=563 vsize=143160 CPUtime=211
/proc/564/task/637/stat : 637 (pb_cplex) R 563 563 32241 0 -1 4202560 15676 0 0 0 20937 163 0 0 15 0 8 0 179170077 146595840 14114 4089446400 134512640 146736320 4289025232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=564/tid=638] ppid=563 vsize=143160 CPUtime=211.52
/proc/564/task/638/stat : 638 (pb_cplex) R 563 563 32241 0 -1 4202560 18893 0 0 0 20980 172 0 0 16 0 8 0 179170077 146595840 14114 4089446400 134512640 146736320 4289025232 18446744073709551615 135963982 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=564/tid=639] ppid=563 vsize=143160 CPUtime=211.64
/proc/564/task/639/stat : 639 (pb_cplex) R 563 563 32241 0 -1 4202560 18832 0 0 0 20991 173 0 0 16 0 8 0 179170077 146595840 14114 4089446400 134512640 146736320 4289025232 18446744073709551615 136475868 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=564/tid=640] ppid=563 vsize=143160 CPUtime=213.16
/proc/564/task/640/stat : 640 (pb_cplex) R 563 563 32241 0 -1 4202560 30515 0 0 0 21130 186 0 0 15 0 8 0 179170077 146595840 14114 4089446400 134512640 146736320 4289025232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=564/tid=641] ppid=563 vsize=143160 CPUtime=213.12
/proc/564/task/641/stat : 641 (pb_cplex) R 563 563 32241 0 -1 4202560 19938 0 0 0 21129 183 0 0 16 0 8 0 179170077 146595840 14114 4089446400 134512640 146736320 4289025232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 1800.18
Current children cumulated vsize (KiB) 151860

Sending SIGTERM to process tree (bottom up)
SSolver just ended. Dumping a history of the last processes samples

Child status: 0
Real time (s): 422.316
CPU time (s): 1800.27
CPU user time (s): 1785.43
CPU system time (s): 14.8437
CPU usage (%): 426.285
Max. virtual memory (cumulated for all children) (KiB): 164644

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1785.43
system time used= 14.8437
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 228076
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 260864
involuntary context switches= 11051

runsolver used 0.701893 second user time and 1.65675 second system time

The end

Launcher Data 

Begin job on node116 at 2010-07-02 09:29:18
IDJOB=2697441
IDBENCH=79294
IDSOLVER=1210
FILE ID=node116/2697441-1278055758
PBS_JOBID= 11198950
Free space on /tmp= 74172 MiB

SOLVER NAME= pb_cplex 2010-07-01
BENCH NAME= PB10/normalized-PB10/SOFT-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/tsp/normalized-t3002.11tsp11.1900582300--soft-0-100-0.wbo
COMMAND LINE= run BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-2697441-1278055758/watcher-2697441-1278055758 -o /tmp/evaluation-result-2697441-1278055758/solver-2697441-1278055758 -C 1800 -W 2000 -M 3800  run HOME/instance-2697441-1278055758.wbo

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 3800 MiB
MAX NB THREAD= 0

MD5SUM BENCH= 0a3bbeff5fae384bd8f72d98174965d3
RANDOM SEED=351057663

node116.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5381.00
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      31058608 kB
Buffers:        218676 kB
Cached:        1216540 kB
SwapCached:          0 kB
Active:         860560 kB
Inactive:       883168 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      31058608 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           16728 kB
Writeback:           0 kB
AnonPages:      308516 kB
Mapped:          19132 kB
Slab:            82436 kB
PageTables:       5740 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   518212 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 74176 MiB
End job on node116 at 2010-07-02 09:36:21