Trace number 2697843

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
pb_cplex 2010-07-01? (TO) 1800.07 484.059

General information on the benchmark

Name/PARTIAL-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/tsp/
normalized-t3002.11tsp11.1900579996--soft-66-100-0.wbo
MD5SUM088560862f81f380de0695aae8f2f503
Bench CategoryPARTIAL-SMALLINT-LIN (both soft and hard constraints, small integers, linear constraints)
Best result obtained on this benchmarkMOPT
Best cost obtained on this benchmark1
Best CPU time to get the best result obtained on this benchmark373.059
Max-Satisfiable
Max-(Un)Satisfiability was proved
Best value of the cost
Optimality of the best cost was proved
Number of variables231
Total number of constraints2707
Number of soft constraints899
Number of constraints which are clauses2684
Number of constraints which are cardinality constraints (but not clauses)22
Number of constraints which are nor clauses,nor cardinality constraints1
Minimum length of a constraint2
Maximum length of a constraint110
Top cost 46330
Min constraint cost 1
Max constraint cost 100
Sum of constraints costs 46329
Biggest number in a constraint 25
Number of bits of the biggest number in a constraint 5
Biggest sum of numbers in a constraint 677
Number of bits of the biggest sum of numbers10
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c pb_cplex 1.0 - 20100701
0.00/0.00	c Parsing wbo file format. File HOME/instance-2697843-1278054865.wbo.
0.00/0.00	c Instance file HOME/instance-2697843-1278054865.wbo
0.00/0.00	c File size is 87870 bytes.
0.00/0.17	c Highest Coefficient sum: 677
0.00/0.17	c  IBM ILOG License Manager: "IBM ILOG Optimization Suite for Academic Initiative" is accessing CPLEX 12 with option(s): "e m b q ".
0.00/0.18	c ============================[ Problem Statistics ]=============================
0.00/0.18	c |                                                                             |
0.00/0.18	c |  Parsing time:          0.17         s                                      |
0.00/0.18	c |  Number Variables:      1130                                                |
0.00/0.18	c |  Number Constraints:    2708                                                |
0.00/0.18	c ===============================================================================
1800.04/484.05	c Remaining time 1799.83 sec.
1800.04/484.05	c 
1800.04/484.05	c *** INTERRUPTED ***
1800.04/484.05	s UNKNOWN
1800.04/484.05	c Total CPU time              : 1784.1 s
1800.04/484.05	c Memory used           : 139.53 MB
1800.04/484.05	c CPU time              : 1784.1 s
1800.04/484.05	c 
1800.04/484.05	c *** INTERRUPTED ***

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver version 3.2.9 (svn:492) (c) roussel@cril.univ-artois.fr

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-2697843-1278054865/watcher-2697843-1278054865 -o /tmp/evaluation-result-2697843-1278054865/solver-2697843-1278054865 -C 1800 -W 2000 -M 3800 run HOME/instance-2697843-1278054865.wbo 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 3891200 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 3942400 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.08 0.06 0.01 4/183 21853
/proc/meminfo: memFree=30745204/32951124 swapFree=67111528/67111528
[pid=21853] ppid=21851 vsize=8700 CPUtime=0
/proc/21853/stat : 21853 (run) S 21851 21853 21589 0 -1 4202496 365 0 0 0 0 0 0 0 19 0 1 0 146782134 8908800 249 4089446400 4194304 4922060 140733523597136 18446744073709551615 255377120389 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21853/statm: 2175 249 208 178 0 68 0

[startup+0.09924 s]
/proc/loadavg: 0.08 0.06 0.01 4/183 21853
/proc/meminfo: memFree=30745204/32951124 swapFree=67111528/67111528
[pid=21853] ppid=21851 vsize=8700 CPUtime=0
/proc/21853/stat : 21853 (run) S 21851 21853 21589 0 -1 4202496 365 0 0 0 0 0 0 0 19 0 1 0 146782134 8908800 249 4089446400 4194304 4922060 140733523597136 18446744073709551615 255377120389 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21853/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.10024 s]
/proc/loadavg: 0.08 0.06 0.01 4/183 21853
/proc/meminfo: memFree=30745204/32951124 swapFree=67111528/67111528
[pid=21853] ppid=21851 vsize=8700 CPUtime=0
/proc/21853/stat : 21853 (run) S 21851 21853 21589 0 -1 4202496 365 0 0 0 0 0 0 0 19 0 1 0 146782134 8908800 249 4089446400 4194304 4922060 140733523597136 18446744073709551615 255377120389 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21853/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.30224 s]
/proc/loadavg: 0.08 0.06 0.01 4/183 21853
/proc/meminfo: memFree=30745204/32951124 swapFree=67111528/67111528
[pid=21853] ppid=21851 vsize=8700 CPUtime=0
/proc/21853/stat : 21853 (run) S 21851 21853 21589 0 -1 4202496 365 0 0 0 0 0 0 0 19 0 1 0 146782134 8908800 249 4089446400 4194304 4922060 140733523597136 18446744073709551615 255377120389 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21853/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.700133 s]
/proc/loadavg: 0.08 0.06 0.01 4/183 21853
/proc/meminfo: memFree=30745204/32951124 swapFree=67111528/67111528
[pid=21853] ppid=21851 vsize=8700 CPUtime=0
/proc/21853/stat : 21853 (run) S 21851 21853 21589 0 -1 4202496 365 0 0 0 0 0 0 0 19 0 1 0 146782134 8908800 249 4089446400 4194304 4922060 140733523597136 18446744073709551615 255377120389 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21853/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+1.50107 s]
/proc/loadavg: 0.08 0.06 0.01 3/192 21890
/proc/meminfo: memFree=30679412/32951124 swapFree=67111528/67111528
[pid=21853] ppid=21851 vsize=8700 CPUtime=0
/proc/21853/stat : 21853 (run) S 21851 21853 21589 0 -1 4202496 365 0 0 0 0 0 0 0 19 0 1 0 146782134 8908800 249 4089446400 4194304 4922060 140733523597136 18446744073709551615 255377120389 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21853/statm: 2175 249 208 178 0 68 0
[pid=21854] ppid=21853 vsize=82072 CPUtime=3.01
/proc/21854/stat : 21854 (pb_cplex) R 21853 21853 21589 0 -1 4202496 14861 0 0 0 294 7 0 0 16 0 1 0 146782134 84041728 6363 4089446400 134512640 146736320 4290246560 18446744073709551615 137706534 0 0 4096 16387 0 0 0 17 5 0 0 0
/proc/21854/statm: 20518 6363 1076 2985 0 17060 0
Current children cumulated CPU time (s) 3.01
Current children cumulated vsize (KiB) 90772

[startup+3.10069 s]
/proc/loadavg: 0.08 0.06 0.01 3/199 21932
/proc/meminfo: memFree=30686920/32951124 swapFree=67111528/67111528
[pid=21853] ppid=21851 vsize=8700 CPUtime=0
/proc/21853/stat : 21853 (run) S 21851 21853 21589 0 -1 4202496 365 0 0 0 0 0 0 0 19 0 1 0 146782134 8908800 249 4089446400 4194304 4922060 140733523597136 18446744073709551615 255377120389 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21853/statm: 2175 249 208 178 0 68 0
[pid=21854] ppid=21853 vsize=127664 CPUtime=4.93
/proc/21854/stat : 21854 (pb_cplex) D 21853 21853 21589 0 -1 4202496 21479 0 0 0 484 9 0 0 18 0 8 0 146782134 130727936 9243 4089446400 134512640 146736320 4290246560 18446744073709551615 136084826 0 0 4096 16387 18446744073709551615 0 0 17 6 0 0 0
/proc/21854/statm: 31916 9243 1142 2985 0 28458 0
[pid=21854/tid=21926] ppid=21853 vsize=127664 CPUtime=0
/proc/21854/task/21926/stat : 21926 (pb_cplex) R 21853 21853 21589 0 -1 4202560 400 0 0 0 0 0 0 0 17 0 8 0 146782331 130727936 9243 4089446400 134512640 146736320 4290246560 18446744073709551615 139023063 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=21854/tid=21927] ppid=21853 vsize=127664 CPUtime=0.01
/proc/21854/task/21927/stat : 21927 (pb_cplex) R 21853 21853 21589 0 -1 4202560 524 0 0 0 1 0 0 0 16 0 8 0 146782331 130727936 9243 4089446400 134512640 146736320 4290246560 18446744073709551615 139042888 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=21854/tid=21928] ppid=21853 vsize=127664 CPUtime=0
/proc/21854/task/21928/stat : 21928 (pb_cplex) R 21853 21853 21589 0 -1 4202560 154 0 0 0 0 0 0 0 18 0 8 0 146782331 130727936 9243 4089446400 134512640 146736320 4290246560 18446744073709551615 139028313 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=21854/tid=21929] ppid=21853 vsize=127664 CPUtime=0.01
/proc/21854/task/21929/stat : 21929 (pb_cplex) R 21853 21853 21589 0 -1 4202560 638 0 0 0 1 0 0 0 16 0 8 0 146782331 130727936 9243 4089446400 134512640 146736320 4290246560 18446744073709551615 136403109 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=21854/tid=21930] ppid=21853 vsize=127664 CPUtime=0.01
/proc/21854/task/21930/stat : 21930 (pb_cplex) R 21853 21853 21589 0 -1 4202560 384 0 0 0 1 0 0 0 16 0 8 0 146782331 130727936 9243 4089446400 134512640 146736320 4290246560 18446744073709551615 136000532 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=21854/tid=21931] ppid=21853 vsize=127664 CPUtime=0
/proc/21854/task/21931/stat : 21931 (pb_cplex) D 21853 21853 21589 0 -1 4202560 246 0 0 0 0 0 0 0 16 0 8 0 146782331 130727936 9243 4089446400 134512640 146736320 4290246560 18446744073709551615 144797972 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=21854/tid=21932] ppid=21853 vsize=127664 CPUtime=0.01
/proc/21854/task/21932/stat : 21932 (pb_cplex) D 21853 21853 21589 0 -1 4202560 392 0 0 0 1 0 0 0 16 0 8 0 146782331 130727936 9243 4089446400 134512640 146736320 4290246560 18446744073709551615 144719394 0 2 4096 16387 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4.93
Current children cumulated vsize (KiB) 136364

[startup+6.30116 s]
/proc/loadavg: 0.24 0.10 0.02 4/199 21932
/proc/meminfo: memFree=30652696/32951124 swapFree=67111528/67111528
[pid=21853] ppid=21851 vsize=8700 CPUtime=0
/proc/21853/stat : 21853 (run) S 21851 21853 21589 0 -1 4202496 365 0 0 0 0 0 0 0 19 0 1 0 146782134 8908800 249 4089446400 4194304 4922060 140733523597136 18446744073709551615 255377120389 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21853/statm: 2175 249 208 178 0 68 0
[pid=21854] ppid=21853 vsize=136768 CPUtime=18.68
/proc/21854/stat : 21854 (pb_cplex) S 21853 21853 21589 0 -1 4202496 33460 0 0 0 1848 20 0 0 16 0 8 0 146782134 140050432 11878 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0
/proc/21854/statm: 34192 11878 1084 2985 0 30734 0
[pid=21854/tid=21926] ppid=21853 vsize=136768 CPUtime=1.67
/proc/21854/task/21926/stat : 21926 (pb_cplex) S 21853 21853 21589 0 -1 4202560 1440 0 0 0 166 1 0 0 16 0 8 0 146782331 140050432 11878 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 0 0 0 0
[pid=21854/tid=21927] ppid=21853 vsize=136768 CPUtime=1.71
/proc/21854/task/21927/stat : 21927 (pb_cplex) R 21853 21853 21589 0 -1 4202560 1862 0 0 0 170 1 0 0 15 0 8 0 146782331 140050432 11878 4089446400 134512640 146736320 4290246560 18446744073709551615 139044583 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=21854/tid=21928] ppid=21853 vsize=136768 CPUtime=1.91
/proc/21854/task/21928/stat : 21928 (pb_cplex) R 21853 21853 21589 0 -1 4202560 3802 0 0 0 190 1 0 0 15 0 8 0 146782331 140050432 11878 4089446400 134512640 146736320 4290246560 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=21854/tid=21929] ppid=21853 vsize=136768 CPUtime=1.95
/proc/21854/task/21929/stat : 21929 (pb_cplex) R 21853 21853 21589 0 -1 4202560 1573 0 0 0 194 1 0 0 15 0 8 0 146782331 140050432 11878 4089446400 134512640 146736320 4290246560 18446744073709551615 136000488 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=21854/tid=21930] ppid=21853 vsize=136768 CPUtime=1.63
/proc/21854/task/21930/stat : 21930 (pb_cplex) R 21853 21853 21589 0 -1 4202560 1998 0 0 0 162 1 0 0 15 0 8 0 146782331 140050432 11878 4089446400 134512640 146736320 4290246560 18446744073709551615 136364030 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=21854/tid=21931] ppid=21853 vsize=136768 CPUtime=1.78
/proc/21854/task/21931/stat : 21931 (pb_cplex) R 21853 21853 21589 0 -1 4202560 1990 0 0 0 177 1 0 0 15 0 8 0 146782331 140050432 11878 4089446400 134512640 146736320 4290246560 18446744073709551615 136403129 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=21854/tid=21932] ppid=21853 vsize=136768 CPUtime=1.62
/proc/21854/task/21932/stat : 21932 (pb_cplex) R 21853 21853 21589 0 -1 4202560 1164 0 0 0 162 0 0 0 15 0 8 0 146782331 140050432 11878 4089446400 134512640 146736320 4290246560 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 18.68
Current children cumulated vsize (KiB) 145468

[startup+12.7124 s]
/proc/loadavg: 0.94 0.24 0.07 9/199 21932
/proc/meminfo: memFree=30651092/32951124 swapFree=67111528/67111528
[pid=21853] ppid=21851 vsize=8700 CPUtime=0
/proc/21853/stat : 21853 (run) S 21851 21853 21589 0 -1 4202496 365 0 0 0 0 0 0 0 19 0 1 0 146782134 8908800 249 4089446400 4194304 4922060 140733523597136 18446744073709551615 255377120389 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21853/statm: 2175 249 208 178 0 68 0
[pid=21854] ppid=21853 vsize=138892 CPUtime=43.09
/proc/21854/stat : 21854 (pb_cplex) S 21853 21853 21589 0 -1 4202496 40615 0 0 0 4265 44 0 0 15 0 8 0 146782134 142225408 12284 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0
/proc/21854/statm: 34723 12284 1084 2985 0 31265 0
[pid=21854/tid=21926] ppid=21853 vsize=138892 CPUtime=4.65
/proc/21854/task/21926/stat : 21926 (pb_cplex) S 21853 21853 21589 0 -1 4202560 3413 0 0 0 461 4 0 0 16 0 8 0 146782331 142225408 12284 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 1 0 0 0
[pid=21854/tid=21927] ppid=21853 vsize=138892 CPUtime=4.7
/proc/21854/task/21927/stat : 21927 (pb_cplex) S 21853 21853 21589 0 -1 4202560 3081 0 0 0 466 4 0 0 15 0 8 0 146782331 142225408 12284 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 3 0 0 0
[pid=21854/tid=21928] ppid=21853 vsize=138892 CPUtime=4.9
/proc/21854/task/21928/stat : 21928 (pb_cplex) R 21853 21853 21589 0 -1 4202560 6762 0 0 0 486 4 0 0 15 0 8 0 146782331 142225408 12284 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=21854/tid=21929] ppid=21853 vsize=138892 CPUtime=5.04
/proc/21854/task/21929/stat : 21929 (pb_cplex) S 21853 21853 21589 0 -1 4202560 1916 0 0 0 499 5 0 0 15 0 8 0 146782331 142225408 12284 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 2 0 0 0
[pid=21854/tid=21930] ppid=21853 vsize=138892 CPUtime=4.66
/proc/21854/task/21930/stat : 21930 (pb_cplex) S 21853 21853 21589 0 -1 4202560 2231 0 0 0 462 4 0 0 15 0 8 0 146782331 142225408 12284 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 4 0 0 0
[pid=21854/tid=21931] ppid=21853 vsize=138892 CPUtime=4.85
/proc/21854/task/21931/stat : 21931 (pb_cplex) R 21853 21853 21589 0 -1 4202560 2144 0 0 0 481 4 0 0 15 0 8 0 146782331 142225408 12284 4089446400 134512640 146736320 4290246560 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=21854/tid=21932] ppid=21853 vsize=138892 CPUtime=4.64
/proc/21854/task/21932/stat : 21932 (pb_cplex) R 21853 21853 21589 0 -1 4202560 1407 0 0 0 460 4 0 0 15 0 8 0 146782331 142225408 12284 4089446400 134512640 146736320 4290246560 18446744073709551615 136403117 0 2 4096 16387 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 43.09
Current children cumulated vsize (KiB) 147592

[startup+25.5005 s]
/proc/loadavg: 3.03 0.74 0.23 10/199 21947
/proc/meminfo: memFree=30644640/32951124 swapFree=67111528/67111528
[pid=21853] ppid=21851 vsize=8700 CPUtime=0
/proc/21853/stat : 21853 (run) S 21851 21853 21589 0 -1 4202496 365 0 0 0 0 0 0 0 19 0 1 0 146782134 8908800 249 4089446400 4194304 4922060 140733523597136 18446744073709551615 255377120389 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21853/statm: 2175 249 208 178 0 68 0
[pid=21854] ppid=21853 vsize=135772 CPUtime=81.56
/proc/21854/stat : 21854 (pb_cplex) R 21853 21853 21589 0 -1 4202496 66308 0 0 0 8076 80 0 0 25 0 8 0 146782134 139030528 11811 4089446400 134512640 146736320 4290246560 18446744073709551615 136007292 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 0
/proc/21854/statm: 33943 11811 1087 2985 0 30485 0
[pid=21854/tid=21941] ppid=21853 vsize=135772 CPUtime=1.66
/proc/21854/task/21941/stat : 21941 (pb_cplex) R 21853 21853 21589 0 -1 4202560 1255 0 0 0 166 0 0 0 18 0 8 0 146784127 139030528 11811 4089446400 134512640 146736320 4290246560 18446744073709551615 136475778 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=21854/tid=21942] ppid=21853 vsize=135772 CPUtime=0.8
/proc/21854/task/21942/stat : 21942 (pb_cplex) R 21853 21853 21589 0 -1 4202560 1425 0 0 0 80 0 0 0 16 0 8 0 146784127 139030528 11811 4089446400 134512640 146736320 4290246560 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=21854/tid=21943] ppid=21853 vsize=135772 CPUtime=0.31
/proc/21854/task/21943/stat : 21943 (pb_cplex) R 21853 21853 21589 0 -1 4202560 997 0 0 0 31 0 0 0 16 0 8 0 146784127 139030528 11811 4089446400 134512640 146736320 4290246560 18446744073709551615 136475763 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=21854/tid=21944] ppid=21853 vsize=135772 CPUtime=0.99
/proc/21854/task/21944/stat : 21944 (pb_cplex) R 21853 21853 21589 0 -1 4202560 1226 0 0 0 99 0 0 0 16 0 8 0 146784127 139030528 11811 4089446400 134512640 146736320 4290246560 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=21854/tid=21945] ppid=21853 vsize=135772 CPUtime=3.66
/proc/21854/task/21945/stat : 21945 (pb_cplex) R 21853 21853 21589 0 -1 4202560 4927 0 0 0 365 1 0 0 18 0 8 0 146784127 139030528 11811 4089446400 134512640 146736320 4290246560 18446744073709551615 136403112 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=21854/tid=21946] ppid=21853 vsize=135772 CPUtime=3.35
/proc/21854/task/21946/stat : 21946 (pb_cplex) R 21853 21853 21589 0 -1 4202560 6897 0 0 0 334 1 0 0 17 0 8 0 146784127 139030528 11811 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=21854/tid=21947] ppid=21853 vsize=135772 CPUtime=1.52
/proc/21854/task/21947/stat : 21947 (pb_cplex) S 21853 21853 21589 0 -1 4202560 2011 0 0 0 152 0 0 0 18 0 8 0 146784127 139030528 11811 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 81.56
Current children cumulated vsize (KiB) 144472

[startup+51.1018 s]
/proc/loadavg: 4.66 1.28 0.42 10/199 21962
/proc/meminfo: memFree=30653276/32951124 swapFree=67111528/67111528
[pid=21853] ppid=21851 vsize=8700 CPUtime=0
/proc/21853/stat : 21853 (run) S 21851 21853 21589 0 -1 4202496 365 0 0 0 0 0 0 0 19 0 1 0 146782134 8908800 249 4089446400 4194304 4922060 140733523597136 18446744073709551615 255377120389 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21853/statm: 2175 249 208 178 0 68 0
[pid=21854] ppid=21853 vsize=139344 CPUtime=193.05
/proc/21854/stat : 21854 (pb_cplex) R 21853 21853 21589 0 -1 4202496 77721 0 0 0 19129 176 0 0 16 0 8 0 146782134 142688256 12862 4089446400 134512640 146736320 4290246560 18446744073709551615 139023058 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0
/proc/21854/statm: 34836 12862 1087 2985 0 31378 0
[pid=21854/tid=21941] ppid=21853 vsize=139344 CPUtime=15.48
/proc/21854/task/21941/stat : 21941 (pb_cplex) R 21853 21853 21589 0 -1 4202560 2434 0 0 0 1537 11 0 0 15 0 8 0 146784127 142688256 12862 4089446400 134512640 146736320 4290246560 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=21854/tid=21942] ppid=21853 vsize=139344 CPUtime=14.46
/proc/21854/task/21942/stat : 21942 (pb_cplex) R 21853 21853 21589 0 -1 4202560 2982 0 0 0 1435 11 0 0 15 0 8 0 146784127 142688256 12862 4089446400 134512640 146736320 4290246560 18446744073709551615 135454093 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=21854/tid=21943] ppid=21853 vsize=139344 CPUtime=14.02
/proc/21854/task/21943/stat : 21943 (pb_cplex) R 21853 21853 21589 0 -1 4202560 2355 0 0 0 1389 13 0 0 15 0 8 0 146784127 142688256 12862 4089446400 134512640 146736320 4290246560 18446744073709551615 136008168 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=21854/tid=21944] ppid=21853 vsize=139344 CPUtime=14.61
/proc/21854/task/21944/stat : 21944 (pb_cplex) R 21853 21853 21589 0 -1 4202560 2699 0 0 0 1450 11 0 0 15 0 8 0 146784127 142688256 12862 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=21854/tid=21945] ppid=21853 vsize=139344 CPUtime=17.45
/proc/21854/task/21945/stat : 21945 (pb_cplex) S 21853 21853 21589 0 -1 4202560 6445 0 0 0 1731 14 0 0 15 0 8 0 146784127 142688256 12862 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 6 0 0 0
[pid=21854/tid=21946] ppid=21853 vsize=139344 CPUtime=17.47
/proc/21854/task/21946/stat : 21946 (pb_cplex) S 21853 21853 21589 0 -1 4202560 9675 0 0 0 1732 15 0 0 15 0 8 0 146784127 142688256 12862 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 6 0 0 0
[pid=21854/tid=21947] ppid=21853 vsize=139344 CPUtime=15.62
/proc/21854/task/21947/stat : 21947 (pb_cplex) S 21853 21853 21589 0 -1 4202560 2397 0 0 0 1552 10 0 0 15 0 8 0 146784127 142688256 12862 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 193.05
Current children cumulated vsize (KiB) 148044

[startup+102.333 s]
/proc/loadavg: 7.20 2.56 0.91 9/199 21963
/proc/meminfo: memFree=30652140/32951124 swapFree=67111528/67111528

################
# More data... #
################

/proc/21854/task/21946/stat : 21946 (pb_cplex) S 21853 21853 21589 0 -1 4202560 12952 0 0 0 6888 60 0 0 15 0 8 0 146784127 145301504 13225 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 5 0 0 0
[pid=21854/tid=21947] ppid=21853 vsize=141896 CPUtime=67.54
/proc/21854/task/21947/stat : 21947 (pb_cplex) S 21853 21853 21589 0 -1 4202560 5916 0 0 0 6696 58 0 0 15 0 8 0 146784127 145301504 13225 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 612.39
Current children cumulated vsize (KiB) 150596

[startup+222.301 s]
/proc/loadavg: 9.12 4.75 1.90 8/199 21966
/proc/meminfo: memFree=30650364/32951124 swapFree=67111528/67111528
[pid=21853] ppid=21851 vsize=8700 CPUtime=0
/proc/21853/stat : 21853 (run) S 21851 21853 21589 0 -1 4202496 365 0 0 0 0 0 0 0 19 0 1 0 146782134 8908800 249 4089446400 4194304 4922060 140733523597136 18446744073709551615 255377120389 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21853/statm: 2175 249 208 178 0 68 0
[pid=21854] ppid=21853 vsize=141392 CPUtime=832.54
/proc/21854/stat : 21854 (pb_cplex) S 21853 21853 21589 0 -1 4202496 117466 0 0 0 82510 744 0 0 15 0 8 0 146782134 144785408 13173 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0
/proc/21854/statm: 35348 13173 1087 2985 0 31890 0
[pid=21854/tid=21941] ppid=21853 vsize=141392 CPUtime=94.76
/proc/21854/task/21941/stat : 21941 (pb_cplex) R 21853 21853 21589 0 -1 4202560 8221 0 0 0 9396 80 0 0 16 0 8 0 146784127 144785408 13173 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=21854/tid=21942] ppid=21853 vsize=141392 CPUtime=93.76
/proc/21854/task/21942/stat : 21942 (pb_cplex) S 21853 21853 21589 0 -1 4202560 8188 0 0 0 9294 82 0 0 15 0 8 0 146784127 144785408 13173 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 5 0 0 0
[pid=21854/tid=21943] ppid=21853 vsize=141392 CPUtime=92.64
/proc/21854/task/21943/stat : 21943 (pb_cplex) S 21853 21853 21589 0 -1 4202560 6209 0 0 0 9181 83 0 0 15 0 8 0 146784127 144785408 13173 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 0 0 0 0
[pid=21854/tid=21944] ppid=21853 vsize=141392 CPUtime=93.83
/proc/21854/task/21944/stat : 21944 (pb_cplex) S 21853 21853 21589 0 -1 4202560 4831 0 0 0 9300 83 0 0 15 0 8 0 146784127 144785408 13173 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 2 0 0 0
[pid=21854/tid=21945] ppid=21853 vsize=141392 CPUtime=96.35
/proc/21854/task/21945/stat : 21945 (pb_cplex) S 21853 21853 21589 0 -1 4202560 14198 0 0 0 9552 83 0 0 15 0 8 0 146784127 144785408 13173 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 3 0 0 0
[pid=21854/tid=21946] ppid=21853 vsize=141392 CPUtime=96.66
/proc/21854/task/21946/stat : 21946 (pb_cplex) S 21853 21853 21589 0 -1 4202560 14610 0 0 0 9579 87 0 0 15 0 8 0 146784127 144785408 13173 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 7 0 0 0
[pid=21854/tid=21947] ppid=21853 vsize=141392 CPUtime=94.84
/proc/21854/task/21947/stat : 21947 (pb_cplex) R 21853 21853 21589 0 -1 4202560 8472 0 0 0 9401 83 0 0 15 0 8 0 146784127 144785408 13173 4089446400 134512640 146736320 4290246560 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 832.54
Current children cumulated vsize (KiB) 150092

[startup+282.322 s]
/proc/loadavg: 8.86 5.47 2.32 10/199 21968
/proc/meminfo: memFree=30648240/32951124 swapFree=67111528/67111528
[pid=21853] ppid=21851 vsize=8700 CPUtime=0
/proc/21853/stat : 21853 (run) S 21851 21853 21589 0 -1 4202496 365 0 0 0 0 0 0 0 19 0 1 0 146782134 8908800 249 4089446400 4194304 4922060 140733523597136 18446744073709551615 255377120389 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21853/statm: 2175 249 208 178 0 68 0
[pid=21854] ppid=21853 vsize=143440 CPUtime=1050.89
/proc/21854/stat : 21854 (pb_cplex) S 21853 21853 21589 0 -1 4202496 131634 0 0 0 104150 939 0 0 15 0 8 0 146782134 146882560 13582 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 6 0 0 0
/proc/21854/statm: 35860 13582 1103 2985 0 32402 0
[pid=21854/tid=21941] ppid=21853 vsize=143440 CPUtime=121.69
/proc/21854/task/21941/stat : 21941 (pb_cplex) S 21853 21853 21589 0 -1 4202560 8732 0 0 0 12067 102 0 0 15 0 8 0 146784127 146882560 13582 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 6 0 0 0
[pid=21854/tid=21942] ppid=21853 vsize=143440 CPUtime=120.77
/proc/21854/task/21942/stat : 21942 (pb_cplex) S 21853 21853 21589 0 -1 4202560 9933 0 0 0 11970 107 0 0 16 0 8 0 146784127 146882560 13582 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 6 0 0 0
[pid=21854/tid=21943] ppid=21853 vsize=143440 CPUtime=119.57
/proc/21854/task/21943/stat : 21943 (pb_cplex) S 21853 21853 21589 0 -1 4202560 7834 0 0 0 11850 107 0 0 15 0 8 0 146784127 146882560 13582 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 6 0 0 0
[pid=21854/tid=21944] ppid=21853 vsize=143440 CPUtime=120.79
/proc/21854/task/21944/stat : 21944 (pb_cplex) R 21853 21853 21589 0 -1 4202560 6869 0 0 0 11972 107 0 0 15 0 8 0 146784127 146882560 13582 4089446400 134512640 146736320 4290246560 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=21854/tid=21945] ppid=21853 vsize=143440 CPUtime=123.4
/proc/21854/task/21945/stat : 21945 (pb_cplex) R 21853 21853 21589 0 -1 4202560 15048 0 0 0 12235 105 0 0 15 0 8 0 146784127 146882560 13582 4089446400 134512640 146736320 4290246560 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=21854/tid=21946] ppid=21853 vsize=143440 CPUtime=123.78
/proc/21854/task/21946/stat : 21946 (pb_cplex) R 21853 21853 21589 0 -1 4202560 19533 0 0 0 12267 111 0 0 15 0 8 0 146784127 146882560 13582 4089446400 134512640 146736320 4290246560 18446744073709551615 135468677 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=21854/tid=21947] ppid=21853 vsize=143440 CPUtime=121.57
/proc/21854/task/21947/stat : 21947 (pb_cplex) R 21853 21853 21589 0 -1 4202560 9457 0 0 0 12046 111 0 0 15 0 8 0 146784127 146882560 13582 4089446400 134512640 146736320 4290246560 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 1050.89
Current children cumulated vsize (KiB) 152140

[startup+342.305 s]
/proc/loadavg: 9.99 6.40 2.84 9/199 21969
/proc/meminfo: memFree=30651688/32951124 swapFree=67111528/67111528
[pid=21853] ppid=21851 vsize=8700 CPUtime=0
/proc/21853/stat : 21853 (run) S 21851 21853 21589 0 -1 4202496 365 0 0 0 0 0 0 0 19 0 1 0 146782134 8908800 249 4089446400 4194304 4922060 140733523597136 18446744073709551615 255377120389 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21853/statm: 2175 249 208 178 0 68 0
[pid=21854] ppid=21853 vsize=141584 CPUtime=1270.17
/proc/21854/stat : 21854 (pb_cplex) S 21853 21853 21589 0 -1 4202496 144617 0 0 0 125891 1126 0 0 16 0 8 0 146782134 144982016 13277 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 0
/proc/21854/statm: 35396 13277 1103 2985 0 31938 0
[pid=21854/tid=21941] ppid=21853 vsize=141584 CPUtime=148.96
/proc/21854/task/21941/stat : 21941 (pb_cplex) S 21853 21853 21589 0 -1 4202560 10977 0 0 0 14770 126 0 0 15 0 8 0 146784127 144982016 13277 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 5 0 0 0
[pid=21854/tid=21942] ppid=21853 vsize=141584 CPUtime=147.76
/proc/21854/task/21942/stat : 21942 (pb_cplex) R 21853 21853 21589 0 -1 4202560 11869 0 0 0 14647 129 0 0 15 0 8 0 146784127 144982016 13277 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=21854/tid=21943] ppid=21853 vsize=141584 CPUtime=146.92
/proc/21854/task/21943/stat : 21943 (pb_cplex) S 21853 21853 21589 0 -1 4202560 8620 0 0 0 14562 130 0 0 15 0 8 0 146784127 144982016 13277 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 1 0 0 0
[pid=21854/tid=21944] ppid=21853 vsize=141584 CPUtime=148.02
/proc/21854/task/21944/stat : 21944 (pb_cplex) S 21853 21853 21589 0 -1 4202560 8133 0 0 0 14671 131 0 0 15 0 8 0 146784127 144982016 13277 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 4 0 0 0
[pid=21854/tid=21945] ppid=21853 vsize=141584 CPUtime=150.22
/proc/21854/task/21945/stat : 21945 (pb_cplex) S 21853 21853 21589 0 -1 4202560 15981 0 0 0 14896 126 0 0 15 0 8 0 146784127 144982016 13277 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 2 0 0 0
[pid=21854/tid=21946] ppid=21853 vsize=141584 CPUtime=150.97
/proc/21854/task/21946/stat : 21946 (pb_cplex) S 21853 21853 21589 0 -1 4202560 20933 0 0 0 14962 135 0 0 15 0 8 0 146784127 144982016 13277 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 6 0 0 0
[pid=21854/tid=21947] ppid=21853 vsize=141584 CPUtime=148.62
/proc/21854/task/21947/stat : 21947 (pb_cplex) S 21853 21853 21589 0 -1 4202560 12576 0 0 0 14726 136 0 0 15 0 8 0 146784127 144982016 13277 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1270.17
Current children cumulated vsize (KiB) 150284

[startup+402.301 s]
/proc/loadavg: 9.71 6.97 3.25 9/199 21971
/proc/meminfo: memFree=30649904/32951124 swapFree=67111528/67111528
[pid=21853] ppid=21851 vsize=8700 CPUtime=0
/proc/21853/stat : 21853 (run) S 21851 21853 21589 0 -1 4202496 365 0 0 0 0 0 0 0 19 0 1 0 146782134 8908800 249 4089446400 4194304 4922060 140733523597136 18446744073709551615 255377120389 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21853/statm: 2175 249 208 178 0 68 0
[pid=21854] ppid=21853 vsize=141584 CPUtime=1490.47
/proc/21854/stat : 21854 (pb_cplex) S 21853 21853 21589 0 -1 4202496 161175 0 0 0 147721 1326 0 0 15 0 8 0 146782134 144982016 13372 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0
/proc/21854/statm: 35396 13372 1103 2985 0 31938 0
[pid=21854/tid=21941] ppid=21853 vsize=141584 CPUtime=176.15
/proc/21854/task/21941/stat : 21941 (pb_cplex) S 21853 21853 21589 0 -1 4202560 12474 0 0 0 17465 150 0 0 15 0 8 0 146784127 144982016 13372 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 1 0 0 0
[pid=21854/tid=21942] ppid=21853 vsize=141584 CPUtime=175.24
/proc/21854/task/21942/stat : 21942 (pb_cplex) R 21853 21853 21589 0 -1 4202560 15178 0 0 0 17370 154 0 0 15 0 8 0 146784127 144982016 13372 4089446400 134512640 146736320 4290246560 18446744073709551615 135450763 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=21854/tid=21943] ppid=21853 vsize=141584 CPUtime=174.11
/proc/21854/task/21943/stat : 21943 (pb_cplex) R 21853 21853 21589 0 -1 4202560 10531 0 0 0 17256 155 0 0 15 0 8 0 146784127 144982016 13372 4089446400 134512640 146736320 4290246560 18446744073709551615 136388115 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=21854/tid=21944] ppid=21853 vsize=141584 CPUtime=175.38
/proc/21854/task/21944/stat : 21944 (pb_cplex) S 21853 21853 21589 0 -1 4202560 9242 0 0 0 17380 158 0 0 15 0 8 0 146784127 144982016 13372 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 2 0 0 0
[pid=21854/tid=21945] ppid=21853 vsize=141584 CPUtime=177.51
/proc/21854/task/21945/stat : 21945 (pb_cplex) S 21853 21853 21589 0 -1 4202560 17503 0 0 0 17599 152 0 0 15 0 8 0 146784127 144982016 13372 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 3 0 0 0
[pid=21854/tid=21946] ppid=21853 vsize=141584 CPUtime=177.96
/proc/21854/task/21946/stat : 21946 (pb_cplex) R 21853 21853 21589 0 -1 4202560 23285 0 0 0 17639 157 0 0 15 0 8 0 146784127 144982016 13372 4089446400 134512640 146736320 4290246560 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=21854/tid=21947] ppid=21853 vsize=141584 CPUtime=175.85
/proc/21854/task/21947/stat : 21947 (pb_cplex) R 21853 21853 21589 0 -1 4202560 13482 0 0 0 17423 162 0 0 15 0 8 0 146784127 144982016 13372 4089446400 134512640 146736320 4290246560 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 1490.47
Current children cumulated vsize (KiB) 150284

[startup+462.301 s]
/proc/loadavg: 9.37 7.39 3.63 12/199 21972
/proc/meminfo: memFree=30650452/32951124 swapFree=67111528/67111528
[pid=21853] ppid=21851 vsize=8700 CPUtime=0
/proc/21853/stat : 21853 (run) S 21851 21853 21589 0 -1 4202496 365 0 0 0 0 0 0 0 19 0 1 0 146782134 8908800 249 4089446400 4194304 4922060 140733523597136 18446744073709551615 255377120389 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21853/statm: 2175 249 208 178 0 68 0
[pid=21854] ppid=21853 vsize=142876 CPUtime=1716.81
/proc/21854/stat : 21854 (pb_cplex) R 21853 21853 21589 0 -1 4202496 176159 0 0 0 170160 1521 0 0 16 0 8 0 146782134 146305024 13494 4089446400 134512640 146736320 4290246560 18446744073709551615 135526870 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/21854/statm: 35719 13494 1103 2985 0 32261 0
[pid=21854/tid=21941] ppid=21853 vsize=142876 CPUtime=204.32
/proc/21854/task/21941/stat : 21941 (pb_cplex) R 21853 21853 21589 0 -1 4202560 14452 0 0 0 20259 173 0 0 15 0 8 0 146784127 146305024 13494 4089446400 134512640 146736320 4290246560 18446744073709551615 135526884 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=21854/tid=21942] ppid=21853 vsize=142876 CPUtime=203.25
/proc/21854/task/21942/stat : 21942 (pb_cplex) R 21853 21853 21589 0 -1 4202560 17477 0 0 0 20149 176 0 0 15 0 8 0 146784127 146305024 13494 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=21854/tid=21943] ppid=21853 vsize=142876 CPUtime=201.96
/proc/21854/task/21943/stat : 21943 (pb_cplex) S 21853 21853 21589 0 -1 4202560 11762 0 0 0 20017 179 0 0 15 0 8 0 146784127 146305024 13494 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 2 0 0 0
[pid=21854/tid=21944] ppid=21853 vsize=142876 CPUtime=203.29
/proc/21854/task/21944/stat : 21944 (pb_cplex) S 21853 21853 21589 0 -1 4202560 10737 0 0 0 20146 183 0 0 15 0 8 0 146784127 146305024 13494 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 2 0 0 0
[pid=21854/tid=21945] ppid=21853 vsize=142876 CPUtime=205.6
/proc/21854/task/21945/stat : 21945 (pb_cplex) S 21853 21853 21589 0 -1 4202560 18964 0 0 0 20385 175 0 0 15 0 8 0 146784127 146305024 13494 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 2 0 0 0
[pid=21854/tid=21946] ppid=21853 vsize=142876 CPUtime=205.62
/proc/21854/task/21946/stat : 21946 (pb_cplex) S 21853 21853 21589 0 -1 4202560 25186 0 0 0 20380 182 0 0 15 0 8 0 146784127 146305024 13494 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 2 0 0 0
[pid=21854/tid=21947] ppid=21853 vsize=142876 CPUtime=204.05
/proc/21854/task/21947/stat : 21947 (pb_cplex) S 21853 21853 21589 0 -1 4202560 15463 0 0 0 20218 187 0 0 15 0 8 0 146784127 146305024 13494 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 1716.81
Current children cumulated vsize (KiB) 151576



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+484.044 s]
/proc/loadavg: 9.40 7.55 3.78 10/199 21980
/proc/meminfo: memFree=30633440/32951124 swapFree=67111528/67111528
[pid=21853] ppid=21851 vsize=8700 CPUtime=0
/proc/21853/stat : 21853 (run) S 21851 21853 21589 0 -1 4202496 365 0 0 0 0 0 0 0 19 0 1 0 146782134 8908800 249 4089446400 4194304 4922060 140733523597136 18446744073709551615 255377120389 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21853/statm: 2175 249 208 178 0 68 0
[pid=21854] ppid=21853 vsize=142876 CPUtime=1800.04
/proc/21854/stat : 21854 (pb_cplex) R 21853 21853 21589 0 -1 4202496 180440 0 0 0 178408 1596 0 0 15 0 8 0 146782134 146305024 13521 4089446400 134512640 146736320 4290246560 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 6 0 0 0
/proc/21854/statm: 35719 13521 1103 2985 0 32261 0
[pid=21854/tid=21941] ppid=21853 vsize=142876 CPUtime=214.75
/proc/21854/task/21941/stat : 21941 (pb_cplex) R 21853 21853 21589 0 -1 4202560 15240 0 0 0 21293 182 0 0 15 0 8 0 146784127 146305024 13521 4089446400 134512640 146736320 4290246560 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=21854/tid=21942] ppid=21853 vsize=142876 CPUtime=213.65
/proc/21854/task/21942/stat : 21942 (pb_cplex) R 21853 21853 21589 0 -1 4202560 17909 0 0 0 21179 186 0 0 15 0 8 0 146784127 146305024 13521 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=21854/tid=21943] ppid=21853 vsize=142876 CPUtime=212.12
/proc/21854/task/21943/stat : 21943 (pb_cplex) S 21853 21853 21589 0 -1 4202560 11978 0 0 0 21024 188 0 0 15 0 8 0 146784127 146305024 13521 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 2 0 0 0
[pid=21854/tid=21944] ppid=21853 vsize=142876 CPUtime=213.5
/proc/21854/task/21944/stat : 21944 (pb_cplex) S 21853 21853 21589 0 -1 4202560 11176 0 0 0 21158 192 0 0 15 0 8 0 146784127 146305024 13521 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 2 0 0 0
[pid=21854/tid=21945] ppid=21853 vsize=142876 CPUtime=216.04
/proc/21854/task/21945/stat : 21945 (pb_cplex) S 21853 21853 21589 0 -1 4202560 19792 0 0 0 21419 185 0 0 15 0 8 0 146784127 146305024 13521 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 2 0 0 0
[pid=21854/tid=21946] ppid=21853 vsize=142876 CPUtime=215.81
/proc/21854/task/21946/stat : 21946 (pb_cplex) S 21853 21853 21589 0 -1 4202560 25680 0 0 0 21389 192 0 0 16 0 8 0 146784127 146305024 13521 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 2 0 0 0
[pid=21854/tid=21947] ppid=21853 vsize=142876 CPUtime=214.35
/proc/21854/task/21947/stat : 21947 (pb_cplex) S 21853 21853 21589 0 -1 4202560 15757 0 0 0 21239 196 0 0 16 0 8 0 146784127 146305024 13521 4089446400 134512640 146736320 4290246560 18446744073709551615 144580073 0 2 4096 16387 18446604471087823552 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 1800.04
Current children cumulated vsize (KiB) 151576

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child status: 0
Real time (s): 484.059
CPU time (s): 1800.07
CPU user time (s): 1784.1
CPU system time (s): 15.9696
CPU usage (%): 371.87
Max. virtual memory (cumulated for all children) (KiB): 157452

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1784.1
system time used= 15.9696
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 180814
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 345312
involuntary context switches= 32716

runsolver used 0.934857 second user time and 1.9607 second system time

The end

Launcher Data 

Begin job on node135 at 2010-07-02 09:14:25
IDJOB=2697843
IDBENCH=78909
IDSOLVER=1210
FILE ID=node135/2697843-1278054865
PBS_JOBID= 11198911
Free space on /tmp= 74116 MiB

SOLVER NAME= pb_cplex 2010-07-01
BENCH NAME= PB10/normalized-PB10/PARTIAL-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/tsp/normalized-t3002.11tsp11.1900579996--soft-66-100-0.wbo
COMMAND LINE= run BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-2697843-1278054865/watcher-2697843-1278054865 -o /tmp/evaluation-result-2697843-1278054865/solver-2697843-1278054865 -C 1800 -W 2000 -M 3800  run HOME/instance-2697843-1278054865.wbo

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 3800 MiB
MAX NB THREAD= 0

MD5SUM BENCH= 088560862f81f380de0695aae8f2f503
RANDOM SEED=1729222425

node135.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.857
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.71
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.857
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.857
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.857
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 7186.14
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.857
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.857
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.857
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.857
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      30746352 kB
Buffers:        218544 kB
Cached:        1763336 kB
SwapCached:          0 kB
Active:         628804 kB
Inactive:      1423704 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      30746352 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           33676 kB
Writeback:           0 kB
AnonPages:       70156 kB
Mapped:          18840 kB
Slab:            87364 kB
PageTables:       4424 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   217580 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 74100 MiB
End job on node135 at 2010-07-02 09:22:29