Trace number 2697808

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
pb_cplex 2010-07-01? (TO) 1800.18 416.973

General information on the benchmark

Name/PARTIAL-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/tsp/
normalized-t3002.11tsp11.1900545610--soft-66-100-0.wbo
MD5SUM75e2f830e9a185ffa43de126e90d54da
Bench CategoryPARTIAL-SMALLINT-LIN (both soft and hard constraints, small integers, linear constraints)
Best result obtained on this benchmarkMOPT
Best cost obtained on this benchmark1
Best CPU time to get the best result obtained on this benchmark306.438
Max-Satisfiable
Max-(Un)Satisfiability was proved
Best value of the cost
Optimality of the best cost was proved
Number of variables231
Total number of constraints2707
Number of soft constraints906
Number of constraints which are clauses2684
Number of constraints which are cardinality constraints (but not clauses)22
Number of constraints which are nor clauses,nor cardinality constraints1
Minimum length of a constraint2
Maximum length of a constraint110
Top cost 44555
Min constraint cost 1
Max constraint cost 100
Sum of constraints costs 44554
Biggest number in a constraint 25
Number of bits of the biggest number in a constraint 5
Biggest sum of numbers in a constraint 671
Number of bits of the biggest sum of numbers10
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c pb_cplex 1.0 - 20100701
0.00/0.00	c Parsing wbo file format. File HOME/instance-2697808-1278054863.wbo.
0.00/0.00	c Instance file HOME/instance-2697808-1278054863.wbo
0.00/0.00	c File size is 87887 bytes.
0.00/0.17	c Highest Coefficient sum: 671
0.00/0.17	c  IBM ILOG License Manager: "IBM ILOG Optimization Suite for Academic Initiative" is accessing CPLEX 12 with option(s): "e m b q ".
0.00/0.18	c ============================[ Problem Statistics ]=============================
0.00/0.18	c |                                                                             |
0.00/0.18	c |  Parsing time:          0.18         s                                      |
0.00/0.18	c |  Number Variables:      1137                                                |
0.00/0.18	c |  Number Constraints:    2708                                                |
0.00/0.18	c ===============================================================================

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver version 3.2.9 (svn:492) (c) roussel@cril.univ-artois.fr

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-2697808-1278054863/watcher-2697808-1278054863 -o /tmp/evaluation-result-2697808-1278054863/solver-2697808-1278054863 -C 1800 -W 2000 -M 3800 run HOME/instance-2697808-1278054863.wbo 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 3891200 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 3942400 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.00 0.00 0.00 2/173 25289
/proc/meminfo: memFree=31075192/32951124 swapFree=67111528/67111528
[pid=25289] ppid=25287 vsize=8700 CPUtime=0
/proc/25289/stat : 25289 (run) S 25287 25289 25060 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146780799 8908800 248 4089446400 4194304 4922060 140736667949584 18446744073709551615 254261435525 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25289/statm: 2175 248 208 178 0 68 0

[startup+0.0817019 s]
/proc/loadavg: 0.00 0.00 0.00 2/173 25289
/proc/meminfo: memFree=31075192/32951124 swapFree=67111528/67111528
[pid=25289] ppid=25287 vsize=8700 CPUtime=0
/proc/25289/stat : 25289 (run) S 25287 25289 25060 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146780799 8908800 248 4089446400 4194304 4922060 140736667949584 18446744073709551615 254261435525 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25289/statm: 2175 248 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.100694 s]
/proc/loadavg: 0.00 0.00 0.00 2/173 25289
/proc/meminfo: memFree=31075192/32951124 swapFree=67111528/67111528
[pid=25289] ppid=25287 vsize=8700 CPUtime=0
/proc/25289/stat : 25289 (run) S 25287 25289 25060 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146780799 8908800 248 4089446400 4194304 4922060 140736667949584 18446744073709551615 254261435525 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25289/statm: 2175 248 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.300605 s]
/proc/loadavg: 0.00 0.00 0.00 2/173 25289
/proc/meminfo: memFree=31075192/32951124 swapFree=67111528/67111528
[pid=25289] ppid=25287 vsize=8700 CPUtime=0
/proc/25289/stat : 25289 (run) S 25287 25289 25060 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146780799 8908800 248 4089446400 4194304 4922060 140736667949584 18446744073709551615 254261435525 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25289/statm: 2175 248 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.700478 s]
/proc/loadavg: 0.00 0.00 0.00 2/173 25289
/proc/meminfo: memFree=31075192/32951124 swapFree=67111528/67111528
[pid=25289] ppid=25287 vsize=8700 CPUtime=0
/proc/25289/stat : 25289 (run) S 25287 25289 25060 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146780799 8908800 248 4089446400 4194304 4922060 140736667949584 18446744073709551615 254261435525 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25289/statm: 2175 248 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+1.50045 s]
/proc/loadavg: 0.00 0.00 0.00 9/182 25305
/proc/meminfo: memFree=31033704/32951124 swapFree=67111528/67111528
[pid=25289] ppid=25287 vsize=8700 CPUtime=0
/proc/25289/stat : 25289 (run) S 25287 25289 25060 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146780799 8908800 248 4089446400 4194304 4922060 140736667949584 18446744073709551615 254261435525 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25289/statm: 2175 248 208 178 0 68 0
[pid=25290] ppid=25289 vsize=83328 CPUtime=3.39
/proc/25290/stat : 25290 (pb_cplex) R 25289 25289 25060 0 -1 4202496 17359 0 0 0 332 7 0 0 16 0 1 0 146780799 85327872 6486 4089446400 134512640 146736320 4294141232 18446744073709551615 139044583 0 0 4096 16387 0 0 0 17 2 0 0 0
/proc/25290/statm: 20832 6486 1074 2985 0 17374 0
Current children cumulated CPU time (s) 3.39
Current children cumulated vsize (KiB) 92028

[startup+3.14497 s]
/proc/loadavg: 0.00 0.00 0.00 9/182 25326
/proc/meminfo: memFree=31032216/32951124 swapFree=67111528/67111528
[pid=25289] ppid=25287 vsize=8700 CPUtime=0
/proc/25289/stat : 25289 (run) S 25287 25289 25060 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146780799 8908800 248 4089446400 4194304 4922060 140736667949584 18446744073709551615 254261435525 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25289/statm: 2175 248 208 178 0 68 0
[pid=25290] ppid=25289 vsize=135164 CPUtime=10.64
/proc/25290/stat : 25290 (pb_cplex) R 25289 25289 25060 0 -1 4202496 30275 0 0 0 1051 13 0 0 17 0 8 0 146780799 138407936 11257 4089446400 134512640 146736320 4294141232 18446744073709551615 135453265 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0
/proc/25290/statm: 33791 11257 1081 2985 0 30333 0
[pid=25290/tid=25320] ppid=25289 vsize=135164 CPUtime=0.69
/proc/25290/task/25320/stat : 25320 (pb_cplex) R 25289 25289 25060 0 -1 4202560 720 0 0 0 69 0 0 0 15 0 8 0 146780970 138407936 11257 4089446400 134512640 146736320 4294141232 18446744073709551615 136388065 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=25290/tid=25321] ppid=25289 vsize=135164 CPUtime=0.73
/proc/25290/task/25321/stat : 25321 (pb_cplex) R 25289 25289 25060 0 -1 4202560 1314 0 0 0 73 0 0 0 15 0 8 0 146780970 138407936 11257 4089446400 134512640 146736320 4294141232 18446744073709551615 135454134 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=25290/tid=25322] ppid=25289 vsize=135164 CPUtime=0.76
/proc/25290/task/25322/stat : 25322 (pb_cplex) R 25289 25289 25060 0 -1 4202560 88 0 0 0 76 0 0 0 16 0 8 0 146780970 138407936 11257 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=25290/tid=25323] ppid=25289 vsize=135164 CPUtime=0.68
/proc/25290/task/25323/stat : 25323 (pb_cplex) R 25289 25289 25060 0 -1 4202560 454 0 0 0 68 0 0 0 15 0 8 0 146780970 138407936 11257 4089446400 134512640 146736320 4294141232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=25290/tid=25324] ppid=25289 vsize=135164 CPUtime=1.19
/proc/25290/task/25324/stat : 25324 (pb_cplex) R 25289 25289 25060 0 -1 4202560 274 0 0 0 119 0 0 0 17 0 8 0 146780970 138407936 11257 4089446400 134512640 146736320 4294141232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=25290/tid=25325] ppid=25289 vsize=135164 CPUtime=0.67
/proc/25290/task/25325/stat : 25325 (pb_cplex) R 25289 25289 25060 0 -1 4202560 509 0 0 0 67 0 0 0 16 0 8 0 146780970 138407936 11257 4089446400 134512640 146736320 4294141232 18446744073709551615 135454198 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=25290/tid=25326] ppid=25289 vsize=135164 CPUtime=0.73
/proc/25290/task/25326/stat : 25326 (pb_cplex) R 25289 25289 25060 0 -1 4202560 60 0 0 0 73 0 0 0 15 0 8 0 146780970 138407936 11257 4089446400 134512640 146736320 4294141232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 10.64
Current children cumulated vsize (KiB) 143864

[startup+6.30349 s]
/proc/loadavg: 0.72 0.15 0.05 10/188 25337
/proc/meminfo: memFree=31008300/32951124 swapFree=67111528/67111528
[pid=25289] ppid=25287 vsize=8700 CPUtime=0
/proc/25289/stat : 25289 (run) S 25287 25289 25060 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146780799 8908800 248 4089446400 4194304 4922060 140736667949584 18446744073709551615 254261435525 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25289/statm: 2175 248 208 178 0 68 0
[pid=25290] ppid=25289 vsize=137212 CPUtime=30.45
/proc/25290/stat : 25290 (pb_cplex) R 25289 25289 25060 0 -1 4202496 39149 0 0 0 3010 35 0 0 15 0 8 0 146780799 140505088 11916 4089446400 134512640 146736320 4294141232 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0
/proc/25290/statm: 34303 11916 1080 2985 0 30845 0
[pid=25290/tid=25320] ppid=25289 vsize=137212 CPUtime=3.32
/proc/25290/task/25320/stat : 25320 (pb_cplex) R 25289 25289 25060 0 -1 4202560 1659 0 0 0 330 2 0 0 15 0 8 0 146780970 140505088 11916 4089446400 134512640 146736320 4294141232 18446744073709551615 136403112 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=25290/tid=25321] ppid=25289 vsize=137212 CPUtime=3.19
/proc/25290/task/25321/stat : 25321 (pb_cplex) R 25289 25289 25060 0 -1 4202560 4180 0 0 0 315 4 0 0 15 0 8 0 146780970 140505088 11916 4089446400 134512640 146736320 4294141232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=25290/tid=25322] ppid=25289 vsize=137212 CPUtime=3.19
/proc/25290/task/25322/stat : 25322 (pb_cplex) R 25289 25289 25060 0 -1 4202560 1370 0 0 0 316 3 0 0 16 0 8 0 146780970 140505088 11916 4089446400 134512640 146736320 4294141232 18446744073709551615 138412560 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=25290/tid=25323] ppid=25289 vsize=137212 CPUtime=3.2
/proc/25290/task/25323/stat : 25323 (pb_cplex) R 25289 25289 25060 0 -1 4202560 1091 0 0 0 317 3 0 0 15 0 8 0 146780970 140505088 11916 4089446400 134512640 146736320 4294141232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=25290/tid=25324] ppid=25289 vsize=137212 CPUtime=3.5
/proc/25290/task/25324/stat : 25324 (pb_cplex) R 25289 25289 25060 0 -1 4202560 973 0 0 0 348 2 0 0 16 0 8 0 146780970 140505088 11916 4089446400 134512640 146736320 4294141232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=25290/tid=25325] ppid=25289 vsize=137212 CPUtime=3.14
/proc/25290/task/25325/stat : 25325 (pb_cplex) R 25289 25289 25060 0 -1 4202560 733 0 0 0 312 2 0 0 15 0 8 0 146780970 140505088 11916 4089446400 134512640 146736320 4294141232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=25290/tid=25326] ppid=25289 vsize=137212 CPUtime=3.22
/proc/25290/task/25326/stat : 25326 (pb_cplex) R 25289 25289 25060 0 -1 4202560 1891 0 0 0 319 3 0 0 16 0 8 0 146780970 140505088 11916 4089446400 134512640 146736320 4294141232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 30.45
Current children cumulated vsize (KiB) 145912

[startup+12.7004 s]
/proc/loadavg: 1.46 0.31 0.10 12/195 25379
/proc/meminfo: memFree=30967092/32951124 swapFree=67111528/67111528
[pid=25289] ppid=25287 vsize=8700 CPUtime=0
/proc/25289/stat : 25289 (run) S 25287 25289 25060 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146780799 8908800 248 4089446400 4194304 4922060 140736667949584 18446744073709551615 254261435525 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25289/statm: 2175 248 208 178 0 68 0
[pid=25290] ppid=25289 vsize=151164 CPUtime=60.53
/proc/25290/stat : 25290 (pb_cplex) R 25289 25289 25060 0 -1 4202496 59289 0 0 0 5974 79 0 0 16 0 8 0 146780799 154791936 14805 4089446400 134512640 146736320 4294141232 18446744073709551615 136008039 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 0
/proc/25290/statm: 37791 14805 1094 2985 0 34333 0
[pid=25290/tid=25320] ppid=25289 vsize=151164 CPUtime=7.15
/proc/25290/task/25320/stat : 25320 (pb_cplex) R 25289 25289 25060 0 -1 4202560 5023 0 0 0 708 7 0 0 16 0 8 0 146780970 154791936 14805 4089446400 134512640 146736320 4294141232 18446744073709551615 144785853 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=25290/tid=25321] ppid=25289 vsize=151164 CPUtime=6.91
/proc/25290/task/25321/stat : 25321 (pb_cplex) R 25289 25289 25060 0 -1 4202560 6938 0 0 0 681 10 0 0 16 0 8 0 146780970 154791936 14805 4089446400 134512640 146736320 4294141232 18446744073709551615 144785853 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=25290/tid=25322] ppid=25289 vsize=151164 CPUtime=6.9
/proc/25290/task/25322/stat : 25322 (pb_cplex) R 25289 25289 25060 0 -1 4202560 4914 0 0 0 681 9 0 0 16 0 8 0 146780970 154791936 14805 4089446400 134512640 146736320 4294141232 18446744073709551615 136403112 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=25290/tid=25323] ppid=25289 vsize=151164 CPUtime=6.98
/proc/25290/task/25323/stat : 25323 (pb_cplex) R 25289 25289 25060 0 -1 4202560 6590 0 0 0 689 9 0 0 16 0 8 0 146780970 154791936 14805 4089446400 134512640 146736320 4294141232 18446744073709551615 135450824 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=25290/tid=25324] ppid=25289 vsize=151164 CPUtime=7.24
/proc/25290/task/25324/stat : 25324 (pb_cplex) R 25289 25289 25060 0 -1 4202560 2775 0 0 0 718 6 0 0 16 0 8 0 146780970 154791936 14805 4089446400 134512640 146736320 4294141232 18446744073709551615 136006945 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=25290/tid=25325] ppid=25289 vsize=151164 CPUtime=6.81
/proc/25290/task/25325/stat : 25325 (pb_cplex) R 25289 25289 25060 0 -1 4202560 1379 0 0 0 673 8 0 0 16 0 8 0 146780970 154791936 14805 4089446400 134512640 146736320 4294141232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=25290/tid=25326] ppid=25289 vsize=151164 CPUtime=6.83
/proc/25290/task/25326/stat : 25326 (pb_cplex) R 25289 25289 25060 0 -1 4202560 3754 0 0 0 674 9 0 0 16 0 8 0 146780970 154791936 14805 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 60.53
Current children cumulated vsize (KiB) 159864

[startup+25.5101 s]
/proc/loadavg: 3.63 0.85 0.28 3/188 25387
/proc/meminfo: memFree=30954500/32951124 swapFree=67111528/67111528
[pid=25289] ppid=25287 vsize=8700 CPUtime=0
/proc/25289/stat : 25289 (run) S 25287 25289 25060 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146780799 8908800 248 4089446400 4194304 4922060 140736667949584 18446744073709551615 254261435525 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25289/statm: 2175 248 208 178 0 68 0
[pid=25290] ppid=25289 vsize=135164 CPUtime=98.75
/proc/25290/stat : 25290 (pb_cplex) R 25289 25289 25060 0 -1 4202496 90348 0 0 0 9774 101 0 0 18 0 2 0 146780799 138407936 11608 4089446400 134512640 146736320 4294141232 18446744073709551615 136007584 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0
/proc/25290/statm: 33791 11608 1091 2985 0 30333 0
Current children cumulated CPU time (s) 98.75
Current children cumulated vsize (KiB) 143864

[startup+51.1016 s]
/proc/loadavg: 5.00 1.39 0.47 9/195 25408
/proc/meminfo: memFree=30929952/32951124 swapFree=67111528/67111528
[pid=25289] ppid=25287 vsize=8700 CPUtime=0
/proc/25289/stat : 25289 (run) S 25287 25289 25060 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146780799 8908800 248 4089446400 4194304 4922060 140736667949584 18446744073709551615 254261435525 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25289/statm: 2175 248 208 178 0 68 0
[pid=25290] ppid=25289 vsize=146996 CPUtime=211.85
/proc/25290/stat : 25290 (pb_cplex) S 25289 25289 25060 0 -1 4202496 136557 0 0 0 20997 188 0 0 16 0 8 0 146780799 150523904 14406 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/25290/statm: 36749 14406 1092 2985 0 33291 0
[pid=25290/tid=25395] ppid=25289 vsize=146996 CPUtime=14.01
/proc/25290/task/25395/stat : 25395 (pb_cplex) S 25289 25289 25060 0 -1 4202560 2400 0 0 0 1391 10 0 0 16 0 8 0 146783353 150523904 14406 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 5 0 0 0
[pid=25290/tid=25396] ppid=25289 vsize=146996 CPUtime=13.9
/proc/25290/task/25396/stat : 25396 (pb_cplex) S 25289 25289 25060 0 -1 4202560 2100 0 0 0 1383 7 0 0 16 0 8 0 146783353 150523904 14406 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 3 0 0 0
[pid=25290/tid=25397] ppid=25289 vsize=146996 CPUtime=13.88
/proc/25290/task/25397/stat : 25397 (pb_cplex) S 25289 25289 25060 0 -1 4202560 3794 0 0 0 1378 10 0 0 16 0 8 0 146783353 150523904 14406 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 7 0 0 0
[pid=25290/tid=25398] ppid=25289 vsize=146996 CPUtime=14.04
/proc/25290/task/25398/stat : 25398 (pb_cplex) S 25289 25289 25060 0 -1 4202560 4198 0 0 0 1393 11 0 0 15 0 8 0 146783353 150523904 14406 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 4 0 0 0
[pid=25290/tid=25399] ppid=25289 vsize=146996 CPUtime=13.65
/proc/25290/task/25399/stat : 25399 (pb_cplex) S 25289 25289 25060 0 -1 4202560 4464 0 0 0 1355 10 0 0 15 0 8 0 146783353 150523904 14406 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 1 0 0 0
[pid=25290/tid=25400] ppid=25289 vsize=146996 CPUtime=13.97
/proc/25290/task/25400/stat : 25400 (pb_cplex) S 25289 25289 25060 0 -1 4202560 7629 0 0 0 1386 11 0 0 15 0 8 0 146783353 150523904 14406 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 6 0 0 0
[pid=25290/tid=25401] ppid=25289 vsize=146996 CPUtime=14.31
/proc/25290/task/25401/stat : 25401 (pb_cplex) R 25289 25289 25060 0 -1 4202560 3898 0 0 0 1420 11 0 0 15 0 8 0 146783353 150523904 14406 4089446400 134512640 146736320 4294141232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 211.85
Current children cumulated vsize (KiB) 155696

[startup+102.31 s]
/proc/loadavg: 7.43 2.60 0.93 10/195 25417
/proc/meminfo: memFree=30917900/32951124 swapFree=67111528/67111528
[pid=25289] ppid=25287 vsize=8700 CPUtime=0
/proc/25289/stat : 25289 (run) S 25287 25289 25060 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146780799 8908800 248 4089446400 4194304 4922060 140736667949584 18446744073709551615 254261435525 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25289/statm: 2175 248 208 178 0 68 0
[pid=25290] ppid=25289 vsize=149044 CPUtime=438.53
/proc/25290/stat : 25290 (pb_cplex) S 25289 25289 25060 0 -1 4202496 156910 0 0 0 43480 373 0 0 16 0 8 0 146780799 152621056 14798 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0
/proc/25290/statm: 37261 14798 1092 2985 0 33803 0
[pid=25290/tid=25395] ppid=25289 vsize=149044 CPUtime=42.4
/proc/25290/task/25395/stat : 25395 (pb_cplex) S 25289 25289 25060 0 -1 4202560 4794 0 0 0 4208 32 0 0 15 0 8 0 146783353 152621056 14798 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 3 0 0 0
[pid=25290/tid=25396] ppid=25289 vsize=149044 CPUtime=41.93
/proc/25290/task/25396/stat : 25396 (pb_cplex) S 25289 25289 25060 0 -1 4202560 3644 0 0 0 4164 29 0 0 16 0 8 0 146783353 152621056 14798 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 3 0 0 0
[pid=25290/tid=25397] ppid=25289 vsize=149044 CPUtime=41.76
/proc/25290/task/25397/stat : 25397 (pb_cplex) S 25289 25289 25060 0 -1 4202560 4280 0 0 0 4142 34 0 0 15 0 8 0 146783353 152621056 14798 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 6 0 0 0
[pid=25290/tid=25398] ppid=25289 vsize=149044 CPUtime=42.41
/proc/25290/task/25398/stat : 25398 (pb_cplex) S 25289 25289 25060 0 -1 4202560 5140 0 0 0 4208 33 0 0 15 0 8 0 146783353 152621056 14798 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 5 0 0 0

################
# More data... #
################

/proc/25290/statm: 37261 14812 1092 2985 0 33803 0
[pid=25290/tid=25395] ppid=25289 vsize=149044 CPUtime=107.37
/proc/25290/task/25395/stat : 25395 (pb_cplex) S 25289 25289 25060 0 -1 4202560 6676 0 0 0 10648 89 0 0 15 0 8 0 146783353 152621056 14812 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 2 0 0 0
[pid=25290/tid=25396] ppid=25289 vsize=149044 CPUtime=106.97
/proc/25290/task/25396/stat : 25396 (pb_cplex) S 25289 25289 25060 0 -1 4202560 6469 0 0 0 10612 85 0 0 15 0 8 0 146783353 152621056 14812 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 2 0 0 0
[pid=25290/tid=25397] ppid=25289 vsize=149044 CPUtime=106.76
/proc/25290/task/25397/stat : 25397 (pb_cplex) S 25289 25289 25060 0 -1 4202560 7201 0 0 0 10587 89 0 0 15 0 8 0 146783353 152621056 14812 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 3 0 0 0
[pid=25290/tid=25398] ppid=25289 vsize=149044 CPUtime=107.48
/proc/25290/task/25398/stat : 25398 (pb_cplex) S 25289 25289 25060 0 -1 4202560 6786 0 0 0 10661 87 0 0 15 0 8 0 146783353 152621056 14812 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 3 0 0 0
[pid=25290/tid=25399] ppid=25289 vsize=149044 CPUtime=107.36
/proc/25290/task/25399/stat : 25399 (pb_cplex) S 25289 25289 25060 0 -1 4202560 9542 0 0 0 10646 90 0 0 15 0 8 0 146783353 152621056 14812 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 3 0 0 0
[pid=25290/tid=25400] ppid=25289 vsize=149044 CPUtime=106.46
/proc/25290/task/25400/stat : 25400 (pb_cplex) R 25289 25289 25060 0 -1 4202560 11964 0 0 0 10556 90 0 0 15 0 8 0 146783353 152621056 14812 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=25290/tid=25401] ppid=25289 vsize=149044 CPUtime=107.09
/proc/25290/task/25401/stat : 25401 (pb_cplex) S 25289 25289 25060 0 -1 4202560 11058 0 0 0 10616 93 0 0 15 0 8 0 146783353 152621056 14812 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 961.79
Current children cumulated vsize (KiB) 157744

[startup+282.301 s]
/proc/loadavg: 9.19 5.50 2.34 9/195 25428
/proc/meminfo: memFree=30915252/32951124 swapFree=67111528/67111528
[pid=25289] ppid=25287 vsize=8700 CPUtime=0
/proc/25289/stat : 25289 (run) S 25287 25289 25060 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146780799 8908800 248 4089446400 4194304 4922060 140736667949584 18446744073709551615 254261435525 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25289/statm: 2175 248 208 178 0 68 0
[pid=25290] ppid=25289 vsize=149044 CPUtime=1240.69
/proc/25290/stat : 25290 (pb_cplex) S 25289 25289 25060 0 -1 4202496 188231 0 0 0 123007 1062 0 0 15 0 8 0 146780799 152621056 14831 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0
/proc/25290/statm: 37261 14831 1092 2985 0 33803 0
[pid=25290/tid=25395] ppid=25289 vsize=149044 CPUtime=142.2
/proc/25290/task/25395/stat : 25395 (pb_cplex) S 25289 25289 25060 0 -1 4202560 8387 0 0 0 14101 119 0 0 15 0 8 0 146783353 152621056 14831 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 7 0 0 0
[pid=25290/tid=25396] ppid=25289 vsize=149044 CPUtime=141.32
/proc/25290/task/25396/stat : 25396 (pb_cplex) S 25289 25289 25060 0 -1 4202560 8743 0 0 0 14017 115 0 0 16 0 8 0 146783353 152621056 14831 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 5 0 0 0
[pid=25290/tid=25397] ppid=25289 vsize=149044 CPUtime=141.57
/proc/25290/task/25397/stat : 25397 (pb_cplex) R 25289 25289 25060 0 -1 4202560 8726 0 0 0 14036 121 0 0 15 0 8 0 146783353 152621056 14831 4089446400 134512640 146736320 4294141232 18446744073709551615 137412828 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=25290/tid=25398] ppid=25289 vsize=149044 CPUtime=142.11
/proc/25290/task/25398/stat : 25398 (pb_cplex) S 25289 25289 25060 0 -1 4202560 8019 0 0 0 14094 117 0 0 15 0 8 0 146783353 152621056 14831 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 6 0 0 0
[pid=25290/tid=25399] ppid=25289 vsize=149044 CPUtime=142.08
/proc/25290/task/25399/stat : 25399 (pb_cplex) S 25289 25289 25060 0 -1 4202560 11302 0 0 0 14090 118 0 0 15 0 8 0 146783353 152621056 14831 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 3 0 0 0
[pid=25290/tid=25400] ppid=25289 vsize=149044 CPUtime=141.08
/proc/25290/task/25400/stat : 25400 (pb_cplex) S 25289 25289 25060 0 -1 4202560 13367 0 0 0 13989 119 0 0 16 0 8 0 146783353 152621056 14831 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 7 0 0 0
[pid=25290/tid=25401] ppid=25289 vsize=149044 CPUtime=142.1
/proc/25290/task/25401/stat : 25401 (pb_cplex) S 25289 25289 25060 0 -1 4202560 12665 0 0 0 14085 125 0 0 15 0 8 0 146783353 152621056 14831 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1240.69
Current children cumulated vsize (KiB) 157744

[startup+342.34 s]
/proc/loadavg: 8.81 6.07 2.73 7/195 25521
/proc/meminfo: memFree=30946676/32951124 swapFree=67111528/67111528
[pid=25289] ppid=25287 vsize=8700 CPUtime=0
/proc/25289/stat : 25289 (run) S 25287 25289 25060 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146780799 8908800 248 4089446400 4194304 4922060 140736667949584 18446744073709551615 254261435525 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25289/statm: 2175 248 208 178 0 68 0
[pid=25290] ppid=25289 vsize=149044 CPUtime=1521.57
/proc/25290/stat : 25290 (pb_cplex) S 25289 25289 25060 0 -1 4202496 201662 0 0 0 150832 1325 0 0 15 0 8 0 146780799 152621056 14852 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0
/proc/25290/statm: 37261 14852 1110 2985 0 33803 0
[pid=25290/tid=25395] ppid=25289 vsize=149044 CPUtime=177.37
/proc/25290/task/25395/stat : 25395 (pb_cplex) R 25289 25289 25060 0 -1 4202560 10326 0 0 0 17587 150 0 0 15 0 8 0 146783353 152621056 14852 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=25290/tid=25396] ppid=25289 vsize=149044 CPUtime=176.34
/proc/25290/task/25396/stat : 25396 (pb_cplex) S 25289 25289 25060 0 -1 4202560 9674 0 0 0 17486 148 0 0 15 0 8 0 146783353 152621056 14852 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 4 0 0 0
[pid=25290/tid=25397] ppid=25289 vsize=149044 CPUtime=176.4
/proc/25290/task/25397/stat : 25397 (pb_cplex) S 25289 25289 25060 0 -1 4202560 10494 0 0 0 17486 154 0 0 15 0 8 0 146783353 152621056 14852 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 4 0 0 0
[pid=25290/tid=25398] ppid=25289 vsize=149044 CPUtime=177.2
/proc/25290/task/25398/stat : 25398 (pb_cplex) S 25289 25289 25060 0 -1 4202560 10538 0 0 0 17570 150 0 0 15 0 8 0 146783353 152621056 14852 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 4 0 0 0
[pid=25290/tid=25399] ppid=25289 vsize=149044 CPUtime=177
/proc/25290/task/25399/stat : 25399 (pb_cplex) S 25289 25289 25060 0 -1 4202560 12927 0 0 0 17547 153 0 0 15 0 8 0 146783353 152621056 14852 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 7 0 0 0
[pid=25290/tid=25400] ppid=25289 vsize=149044 CPUtime=175.97
/proc/25290/task/25400/stat : 25400 (pb_cplex) S 25289 25289 25060 0 -1 4202560 14365 0 0 0 17448 149 0 0 15 0 8 0 146783353 152621056 14852 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 7 0 0 0
[pid=25290/tid=25401] ppid=25289 vsize=149044 CPUtime=177.01
/proc/25290/task/25401/stat : 25401 (pb_cplex) S 25289 25289 25060 0 -1 4202560 14157 0 0 0 17543 158 0 0 16 0 8 0 146783353 152621056 14852 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 1521.57
Current children cumulated vsize (KiB) 157744

[startup+402.318 s]
/proc/loadavg: 9.27 6.71 3.16 9/195 25523
/proc/meminfo: memFree=30947452/32951124 swapFree=67111528/67111528
[pid=25289] ppid=25287 vsize=8700 CPUtime=0
/proc/25289/stat : 25289 (run) S 25287 25289 25060 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146780799 8908800 248 4089446400 4194304 4922060 140736667949584 18446744073709551615 254261435525 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25289/statm: 2175 248 208 178 0 68 0
[pid=25290] ppid=25289 vsize=149044 CPUtime=1743.99
/proc/25290/stat : 25290 (pb_cplex) S 25289 25289 25060 0 -1 4202496 210208 0 0 0 172893 1506 0 0 16 0 8 0 146780799 152621056 14841 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 0
/proc/25290/statm: 37261 14841 1110 2985 0 33803 0
[pid=25290/tid=25395] ppid=25289 vsize=149044 CPUtime=204.94
/proc/25290/task/25395/stat : 25395 (pb_cplex) S 25289 25289 25060 0 -1 4202560 11471 0 0 0 20322 172 0 0 15 0 8 0 146783353 152621056 14841 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 2 0 0 0
[pid=25290/tid=25396] ppid=25289 vsize=149044 CPUtime=203.67
/proc/25290/task/25396/stat : 25396 (pb_cplex) S 25289 25289 25060 0 -1 4202560 11318 0 0 0 20195 172 0 0 15 0 8 0 146783353 152621056 14841 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 7 0 0 0
[pid=25290/tid=25397] ppid=25289 vsize=149044 CPUtime=203.61
/proc/25290/task/25397/stat : 25397 (pb_cplex) R 25289 25289 25060 0 -1 4202560 11352 0 0 0 20186 175 0 0 15 0 8 0 146783353 152621056 14841 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=25290/tid=25398] ppid=25289 vsize=149044 CPUtime=204.93
/proc/25290/task/25398/stat : 25398 (pb_cplex) R 25289 25289 25060 0 -1 4202560 10933 0 0 0 20322 171 0 0 15 0 8 0 146783353 152621056 14841 4089446400 134512640 146736320 4294141232 18446744073709551615 135854577 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=25290/tid=25399] ppid=25289 vsize=149044 CPUtime=204.41
/proc/25290/task/25399/stat : 25399 (pb_cplex) S 25289 25289 25060 0 -1 4202560 13785 0 0 0 20267 174 0 0 15 0 8 0 146783353 152621056 14841 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 18446604471161255104 0 0 -1 5 0 0 0
[pid=25290/tid=25400] ppid=25289 vsize=149044 CPUtime=203.31
/proc/25290/task/25400/stat : 25400 (pb_cplex) R 25289 25289 25060 0 -1 4202560 15708 0 0 0 20160 171 0 0 15 0 8 0 146783353 152621056 14841 4089446400 134512640 146736320 4294141232 18446744073709551615 136017773 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=25290/tid=25401] ppid=25289 vsize=149044 CPUtime=204.52
/proc/25290/task/25401/stat : 25401 (pb_cplex) R 25289 25289 25060 0 -1 4202560 15213 0 0 0 20269 183 0 0 15 0 8 0 146783353 152621056 14841 4089446400 134512640 146736320 4294141232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1743.99
Current children cumulated vsize (KiB) 157744



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+416.907 s]
/proc/loadavg: 9.35 6.85 3.26 9/195 25523
/proc/meminfo: memFree=30945776/32951124 swapFree=67111528/67111528
[pid=25289] ppid=25287 vsize=8700 CPUtime=0
/proc/25289/stat : 25289 (run) S 25287 25289 25060 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146780799 8908800 248 4089446400 4194304 4922060 140736667949584 18446744073709551615 254261435525 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25289/statm: 2175 248 208 178 0 68 0
[pid=25290] ppid=25289 vsize=149044 CPUtime=1800.18
/proc/25290/stat : 25290 (pb_cplex) R 25289 25289 25060 0 -1 4202496 213727 0 0 0 178470 1548 0 0 16 0 8 0 146780799 152621056 14841 4089446400 134512640 146736320 4294141232 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0
/proc/25290/statm: 37261 14841 1110 2985 0 33803 0
[pid=25290/tid=25395] ppid=25289 vsize=149044 CPUtime=211.9
/proc/25290/task/25395/stat : 25395 (pb_cplex) R 25289 25289 25060 0 -1 4202560 11657 0 0 0 21013 177 0 0 15 0 8 0 146783353 152621056 14841 4089446400 134512640 146736320 4294141232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=25290/tid=25396] ppid=25289 vsize=149044 CPUtime=210.68
/proc/25290/task/25396/stat : 25396 (pb_cplex) R 25289 25289 25060 0 -1 4202560 11542 0 0 0 20889 179 0 0 16 0 8 0 146783353 152621056 14841 4089446400 134512640 146736320 4294141232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=25290/tid=25397] ppid=25289 vsize=149044 CPUtime=210.56
/proc/25290/task/25397/stat : 25397 (pb_cplex) R 25289 25289 25060 0 -1 4202560 11474 0 0 0 20876 180 0 0 15 0 8 0 146783353 152621056 14841 4089446400 134512640 146736320 4294141232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=25290/tid=25398] ppid=25289 vsize=149044 CPUtime=212.09
/proc/25290/task/25398/stat : 25398 (pb_cplex) R 25289 25289 25060 0 -1 4202560 11200 0 0 0 21032 177 0 0 15 0 8 0 146783353 152621056 14841 4089446400 134512640 146736320 4294141232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=25290/tid=25399] ppid=25289 vsize=149044 CPUtime=211.26
/proc/25290/task/25399/stat : 25399 (pb_cplex) R 25289 25289 25060 0 -1 4202560 14769 0 0 0 20946 180 0 0 15 0 8 0 146783353 152621056 14841 4089446400 134512640 146736320 4294141232 18446744073709551615 136403109 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=25290/tid=25400] ppid=25289 vsize=149044 CPUtime=210.1
/proc/25290/task/25400/stat : 25400 (pb_cplex) R 25289 25289 25060 0 -1 4202560 16499 0 0 0 20836 174 0 0 15 0 8 0 146783353 152621056 14841 4089446400 134512640 146736320 4294141232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=25290/tid=25401] ppid=25289 vsize=149044 CPUtime=211.41
/proc/25290/task/25401/stat : 25401 (pb_cplex) R 25289 25289 25060 0 -1 4202560 15595 0 0 0 20952 189 0 0 15 0 8 0 146783353 152621056 14841 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 1800.18
Current children cumulated vsize (KiB) 157744

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+416.907 s]
/proc/loadavg: 9.35 6.85 3.26 9/195 25523
/proc/meminfo: memFree=30945776/32951124 swapFree=67111528/67111528
[pid=25289] ppid=25287 vsize=8700 CPUtime=0
/proc/25289/stat : 25289 (run) S 25287 25289 25060 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146780799 8908800 248 4089446400 4194304 4922060 140736667949584 18446744073709551615 254261435525 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25289/statm: 2175 248 208 178 0 68 0
[pid=25290] ppid=25289 vsize=149044 CPUtime=1800.18
/proc/25290/stat : 25290 (pb_cplex) R 25289 25289 25060 0 -1 4202496 213727 0 0 0 178470 1548 0 0 16 0 8 0 146780799 152621056 14841 4089446400 134512640 146736320 4294141232 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0
/proc/25290/statm: 37261 14841 1110 2985 0 33803 0
[pid=25290/tid=25395] ppid=25289 vsize=149044 CPUtime=211.9
/proc/25290/task/25395/stat : 25395 (pb_cplex) R 25289 25289 25060 0 -1 4202560 11657 0 0 0 21013 177 0 0 15 0 8 0 146783353 152621056 14841 4089446400 134512640 146736320 4294141232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=25290/tid=25396] ppid=25289 vsize=149044 CPUtime=210.68
/proc/25290/task/25396/stat : 25396 (pb_cplex) R 25289 25289 25060 0 -1 4202560 11542 0 0 0 20889 179 0 0 16 0 8 0 146783353 152621056 14841 4089446400 134512640 146736320 4294141232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=25290/tid=25397] ppid=25289 vsize=149044 CPUtime=210.56
/proc/25290/task/25397/stat : 25397 (pb_cplex) R 25289 25289 25060 0 -1 4202560 11474 0 0 0 20876 180 0 0 15 0 8 0 146783353 152621056 14841 4089446400 134512640 146736320 4294141232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=25290/tid=25398] ppid=25289 vsize=149044 CPUtime=212.09
/proc/25290/task/25398/stat : 25398 (pb_cplex) R 25289 25289 25060 0 -1 4202560 11200 0 0 0 21032 177 0 0 15 0 8 0 146783353 152621056 14841 4089446400 134512640 146736320 4294141232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=25290/tid=25399] ppid=25289 vsize=149044 CPUtime=211.26
/proc/25290/task/25399/stat : 25399 (pb_cplex) R 25289 25289 25060 0 -1 4202560 14769 0 0 0 20946 180 0 0 15 0 8 0 146783353 152621056 14841 4089446400 134512640 146736320 4294141232 18446744073709551615 136403109 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=25290/tid=25400] ppid=25289 vsize=149044 CPUtime=210.1
/proc/25290/task/25400/stat : 25400 (pb_cplex) R 25289 25289 25060 0 -1 4202560 16499 0 0 0 20836 174 0 0 15 0 8 0 146783353 152621056 14841 4089446400 134512640 146736320 4294141232 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=25290/tid=25401] ppid=25289 vsize=149044 CPUtime=211.41
/proc/25290/task/25401/stat : 25401 (pb_cplex) R 25289 25289 25060 0 -1 4202560 15595 0 0 0 20952 189 0 0 15 0 8 0 146783353 152621056 14841 4089446400 134512640 146736320 4294141232 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 1800.18
Current children cumulated vsize (KiB) 157744

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 25289 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=0
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=999
# CPU time returned by wait4() is 0.000999
# while last known CPU time is 1800.18
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 416.973
CPU time (s): 1800.18
CPU user time (s): 1784.7
CPU system time (s): 15.48
CPU usage (%): 431.725
Max. virtual memory (cumulated for all children) (KiB): 170756

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0
system time used= 0.000999
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 364
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2
involuntary context switches= 2

runsolver used 0.814876 second user time and 1.65975 second system time

The end

Launcher Data 

Begin job on node140 at 2010-07-02 09:14:23
IDJOB=2697808
IDBENCH=78882
IDSOLVER=1210
FILE ID=node140/2697808-1278054863
PBS_JOBID= 11198902
Free space on /tmp= 74200 MiB

SOLVER NAME= pb_cplex 2010-07-01
BENCH NAME= PB10/normalized-PB10/PARTIAL-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/tsp/normalized-t3002.11tsp11.1900545610--soft-66-100-0.wbo
COMMAND LINE= run BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-2697808-1278054863/watcher-2697808-1278054863 -o /tmp/evaluation-result-2697808-1278054863/solver-2697808-1278054863 -C 1800 -W 2000 -M 3800  run HOME/instance-2697808-1278054863.wbo

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 3800 MiB
MAX NB THREAD= 0

MD5SUM BENCH= 75e2f830e9a185ffa43de126e90d54da
RANDOM SEED=6311563

node140.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.790
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.58
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.790
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.790
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.790
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.89
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.790
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5421.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.790
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.790
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.790
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      31075720 kB
Buffers:        219596 kB
Cached:        1444952 kB
SwapCached:          0 kB
Active:         867668 kB
Inactive:       860064 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      31075720 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           16988 kB
Writeback:           0 kB
AnonPages:       63136 kB
Mapped:          14416 kB
Slab:            84644 kB
PageTables:       4520 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   183020 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 74168 MiB
End job on node140 at 2010-07-02 09:21:22