Trace number 2697883

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
pb_cplex 2010-07-01? (TO) 1800.2 473.025

General information on the benchmark

Name/PARTIAL-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/tsp/
normalized-t3002.11tsp11.1900576151--soft-66-100-0.wbo
MD5SUMe60bec5d313c91289af640357d00991f
Bench CategoryPARTIAL-SMALLINT-LIN (both soft and hard constraints, small integers, linear constraints)
Best result obtained on this benchmarkMOPT
Best cost obtained on this benchmark1
Best CPU time to get the best result obtained on this benchmark302.619
Max-Satisfiable
Max-(Un)Satisfiability was proved
Best value of the cost
Optimality of the best cost was proved
Number of variables231
Total number of constraints2707
Number of soft constraints904
Number of constraints which are clauses2684
Number of constraints which are cardinality constraints (but not clauses)22
Number of constraints which are nor clauses,nor cardinality constraints1
Minimum length of a constraint2
Maximum length of a constraint110
Top cost 44120
Min constraint cost 1
Max constraint cost 100
Sum of constraints costs 44119
Biggest number in a constraint 25
Number of bits of the biggest number in a constraint 5
Biggest sum of numbers in a constraint 697
Number of bits of the biggest sum of numbers10
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c pb_cplex 1.0 - 20100701
0.00/0.00	c Parsing wbo file format. File HOME/instance-2697883-1278054866.wbo.
0.00/0.00	c Instance file HOME/instance-2697883-1278054866.wbo
0.00/0.00	c File size is 87879 bytes.
0.00/0.17	c Highest Coefficient sum: 697
0.00/0.17	c  IBM ILOG License Manager: "IBM ILOG Optimization Suite for Academic Initiative" is accessing CPLEX 12 with option(s): "e m b q ".
0.00/0.18	c ============================[ Problem Statistics ]=============================
0.00/0.18	c |                                                                             |
0.00/0.18	c |  Parsing time:          0.18         s                                      |
0.00/0.18	c |  Number Variables:      1135                                                |
0.00/0.18	c |  Number Constraints:    2708                                                |
0.00/0.18	c ===============================================================================
1800.14/473.02	c Remaining time 1799.82 sec.
1800.14/473.02	c 
1800.14/473.02	c *** INTERRUPTED ***
1800.14/473.02	s UNKNOWN
1800.14/473.02	c Total CPU time              : 1784.7 s
1800.14/473.02	c Memory used           : 147.37 MB
1800.14/473.02	c CPU time              : 1784.7 s
1800.14/473.02	c 
1800.14/473.02	c *** INTERRUPTED ***

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver version 3.2.9 (svn:492) (c) roussel@cril.univ-artois.fr

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-2697883-1278054866/watcher-2697883-1278054866 -o /tmp/evaluation-result-2697883-1278054866/solver-2697883-1278054866 -C 1800 -W 2000 -M 3800 run HOME/instance-2697883-1278054866.wbo 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 3891200 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 3942400 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.32 0.07 0.02 4/179 24264
/proc/meminfo: memFree=31281624/32951124 swapFree=67111528/67111528
[pid=24264] ppid=24262 vsize=8700 CPUtime=0
/proc/24264/stat : 24264 (run) S 24262 24264 23951 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146781304 8908800 248 4089446400 4194304 4922060 140733755896608 18446744073709551615 228095270021 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/24264/statm: 2175 248 208 178 0 68 0

[startup+0.03789 s]
/proc/loadavg: 0.32 0.07 0.02 4/179 24264
/proc/meminfo: memFree=31281624/32951124 swapFree=67111528/67111528
[pid=24264] ppid=24262 vsize=8700 CPUtime=0
/proc/24264/stat : 24264 (run) S 24262 24264 23951 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146781304 8908800 248 4089446400 4194304 4922060 140733755896608 18446744073709551615 228095270021 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/24264/statm: 2175 248 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.100878 s]
/proc/loadavg: 0.32 0.07 0.02 4/179 24264
/proc/meminfo: memFree=31281624/32951124 swapFree=67111528/67111528
[pid=24264] ppid=24262 vsize=8700 CPUtime=0
/proc/24264/stat : 24264 (run) S 24262 24264 23951 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146781304 8908800 248 4089446400 4194304 4922060 140733755896608 18446744073709551615 228095270021 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/24264/statm: 2175 248 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.300811 s]
/proc/loadavg: 0.32 0.07 0.02 4/179 24264
/proc/meminfo: memFree=31281624/32951124 swapFree=67111528/67111528
[pid=24264] ppid=24262 vsize=8700 CPUtime=0
/proc/24264/stat : 24264 (run) S 24262 24264 23951 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146781304 8908800 248 4089446400 4194304 4922060 140733755896608 18446744073709551615 228095270021 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/24264/statm: 2175 248 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.700708 s]
/proc/loadavg: 0.32 0.07 0.02 4/179 24264
/proc/meminfo: memFree=31281624/32951124 swapFree=67111528/67111528
[pid=24264] ppid=24262 vsize=8700 CPUtime=0
/proc/24264/stat : 24264 (run) S 24262 24264 23951 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146781304 8908800 248 4089446400 4194304 4922060 140733755896608 18446744073709551615 228095270021 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/24264/statm: 2175 248 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+1.50056 s]
/proc/loadavg: 0.32 0.07 0.02 3/195 24301
/proc/meminfo: memFree=31231896/32951124 swapFree=67111528/67111528
[pid=24264] ppid=24262 vsize=8700 CPUtime=0
/proc/24264/stat : 24264 (run) S 24262 24264 23951 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146781304 8908800 248 4089446400 4194304 4922060 140733755896608 18446744073709551615 228095270021 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/24264/statm: 2175 248 208 178 0 68 0
[pid=24265] ppid=24264 vsize=80624 CPUtime=3.18
/proc/24265/stat : 24265 (pb_cplex) R 24264 24264 23951 0 -1 4202496 16821 0 0 0 312 6 0 0 16 0 1 0 146781304 82558976 6346 4089446400 134512640 146736320 4294026784 18446744073709551615 135708715 0 0 4096 16387 0 0 0 17 5 0 0 0
/proc/24265/statm: 20156 6346 1072 2985 0 16698 0
Current children cumulated CPU time (s) 3.18
Current children cumulated vsize (KiB) 89324

[startup+3.10136 s]
/proc/loadavg: 0.32 0.07 0.02 3/195 24322
/proc/meminfo: memFree=31234960/32951124 swapFree=67111528/67111528
[pid=24264] ppid=24262 vsize=8700 CPUtime=0
/proc/24264/stat : 24264 (run) S 24262 24264 23951 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146781304 8908800 248 4089446400 4194304 4922060 140733755896608 18446744073709551615 228095270021 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/24264/statm: 2175 248 208 178 0 68 0
[pid=24265] ppid=24264 vsize=115440 CPUtime=5.2
/proc/24265/stat : 24265 (pb_cplex) R 24264 24264 23951 0 -1 4202496 24788 0 0 0 512 8 0 0 19 0 8 0 146781304 118210560 7133 4089446400 134512640 146736320 4294026784 18446744073709551615 136403116 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/24265/statm: 28860 7133 1082 2985 0 25402 0
[pid=24265/tid=24316] ppid=24264 vsize=115440 CPUtime=0
/proc/24265/task/24316/stat : 24316 (pb_cplex) S 24264 24264 23951 0 -1 4202560 0 0 0 0 0 0 0 0 17 0 8 0 146781493 118210560 7133 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 0 0 0 0
[pid=24265/tid=24317] ppid=24264 vsize=115440 CPUtime=0
/proc/24265/task/24317/stat : 24317 (pb_cplex) S 24264 24264 23951 0 -1 4202560 0 0 0 0 0 0 0 0 20 0 8 0 146781493 118210560 7133 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 2 0 0 0
[pid=24265/tid=24318] ppid=24264 vsize=115440 CPUtime=0
/proc/24265/task/24318/stat : 24318 (pb_cplex) S 24264 24264 23951 0 -1 4202560 2 0 0 0 0 0 0 0 17 0 8 0 146781493 118210560 7133 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 0 0 0 0
[pid=24265/tid=24319] ppid=24264 vsize=115440 CPUtime=0
/proc/24265/task/24319/stat : 24319 (pb_cplex) S 24264 24264 23951 0 -1 4202560 0 0 0 0 0 0 0 0 20 0 8 0 146781493 118210560 7133 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 3 0 0 0
[pid=24265/tid=24320] ppid=24264 vsize=115440 CPUtime=0
/proc/24265/task/24320/stat : 24320 (pb_cplex) S 24264 24264 23951 0 -1 4202560 1 0 0 0 0 0 0 0 17 0 8 0 146781493 118210560 7133 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 0 0 0 0
[pid=24265/tid=24321] ppid=24264 vsize=115440 CPUtime=0
/proc/24265/task/24321/stat : 24321 (pb_cplex) S 24264 24264 23951 0 -1 4202560 1 0 0 0 0 0 0 0 21 0 8 0 146781493 118210560 7133 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 2 0 0 0
[pid=24265/tid=24322] ppid=24264 vsize=115440 CPUtime=0
/proc/24265/task/24322/stat : 24322 (pb_cplex) S 24264 24264 23951 0 -1 4202560 1 0 0 0 0 0 0 0 21 0 8 0 146781493 118210560 7133 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 5.2
Current children cumulated vsize (KiB) 124140

[startup+6.30569 s]
/proc/loadavg: 0.45 0.10 0.03 11/195 24322
/proc/meminfo: memFree=31212888/32951124 swapFree=67111528/67111528
[pid=24264] ppid=24262 vsize=8700 CPUtime=0
/proc/24264/stat : 24264 (run) S 24262 24264 23951 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146781304 8908800 248 4089446400 4194304 4922060 140733755896608 18446744073709551615 228095270021 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/24264/statm: 2175 248 208 178 0 68 0
[pid=24265] ppid=24264 vsize=138816 CPUtime=17.87
/proc/24265/stat : 24265 (pb_cplex) R 24264 24264 23951 0 -1 4202496 33219 0 0 0 1769 18 0 0 16 0 8 0 146781304 142147584 11973 4089446400 134512640 146736320 4294026784 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/24265/statm: 34704 11973 1083 2985 0 31246 0
[pid=24265/tid=24316] ppid=24264 vsize=138816 CPUtime=1.59
/proc/24265/task/24316/stat : 24316 (pb_cplex) R 24264 24264 23951 0 -1 4202560 977 0 0 0 158 1 0 0 15 0 8 0 146781493 142147584 11973 4089446400 134512640 146736320 4294026784 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=24265/tid=24317] ppid=24264 vsize=138816 CPUtime=1.79
/proc/24265/task/24317/stat : 24317 (pb_cplex) R 24264 24264 23951 0 -1 4202560 949 0 0 0 179 0 0 0 15 0 8 0 146781493 142147584 11973 4089446400 134512640 146736320 4294026784 18446744073709551615 136017773 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=24265/tid=24318] ppid=24264 vsize=138816 CPUtime=1.7
/proc/24265/task/24318/stat : 24318 (pb_cplex) R 24264 24264 23951 0 -1 4202560 1667 0 0 0 169 1 0 0 15 0 8 0 146781493 142147584 11973 4089446400 134512640 146736320 4294026784 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=24265/tid=24319] ppid=24264 vsize=138816 CPUtime=1.63
/proc/24265/task/24319/stat : 24319 (pb_cplex) R 24264 24264 23951 0 -1 4202560 1195 0 0 0 162 1 0 0 15 0 8 0 146781493 142147584 11973 4089446400 134512640 146736320 4294026784 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=24265/tid=24320] ppid=24264 vsize=138816 CPUtime=1.67
/proc/24265/task/24320/stat : 24320 (pb_cplex) R 24264 24264 23951 0 -1 4202560 961 0 0 0 166 1 0 0 15 0 8 0 146781493 142147584 11973 4089446400 134512640 146736320 4294026784 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=24265/tid=24321] ppid=24264 vsize=138816 CPUtime=1.52
/proc/24265/task/24321/stat : 24321 (pb_cplex) R 24264 24264 23951 0 -1 4202560 919 0 0 0 151 1 0 0 15 0 8 0 146781493 142147584 11973 4089446400 134512640 146736320 4294026784 18446744073709551615 136015567 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=24265/tid=24322] ppid=24264 vsize=138816 CPUtime=1.28
/proc/24265/task/24322/stat : 24322 (pb_cplex) R 24264 24264 23951 0 -1 4202560 596 0 0 0 128 0 0 0 15 0 8 0 146781493 142147584 11973 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 17.87
Current children cumulated vsize (KiB) 147516

[startup+12.7015 s]
/proc/loadavg: 1.14 0.25 0.08 7/195 24322
/proc/meminfo: memFree=31207068/32951124 swapFree=67111528/67111528
[pid=24264] ppid=24262 vsize=8700 CPUtime=0
/proc/24264/stat : 24264 (run) S 24262 24264 23951 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146781304 8908800 248 4089446400 4194304 4922060 140733755896608 18446744073709551615 228095270021 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/24264/statm: 2175 248 208 178 0 68 0
[pid=24265] ppid=24264 vsize=139840 CPUtime=43.57
/proc/24265/stat : 24265 (pb_cplex) R 24264 24264 23951 0 -1 4202496 45592 0 0 0 4318 39 0 0 15 0 8 0 146781304 143196160 12422 4089446400 134512640 146736320 4294026784 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/24265/statm: 34960 12422 1083 2985 0 31502 0
[pid=24265/tid=24316] ppid=24264 vsize=139840 CPUtime=4.78
/proc/24265/task/24316/stat : 24316 (pb_cplex) R 24264 24264 23951 0 -1 4202560 4949 0 0 0 474 4 0 0 15 0 8 0 146781493 143196160 12422 4089446400 134512640 146736320 4294026784 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=24265/tid=24317] ppid=24264 vsize=139840 CPUtime=5.02
/proc/24265/task/24317/stat : 24317 (pb_cplex) R 24264 24264 23951 0 -1 4202560 1622 0 0 0 499 3 0 0 15 0 8 0 146781493 143196160 12422 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=24265/tid=24318] ppid=24264 vsize=139840 CPUtime=4.92
/proc/24265/task/24318/stat : 24318 (pb_cplex) S 24264 24264 23951 0 -1 4202560 2280 0 0 0 489 3 0 0 15 0 8 0 146781493 143196160 12422 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 6 0 0 0
[pid=24265/tid=24319] ppid=24264 vsize=139840 CPUtime=4.7
/proc/24265/task/24319/stat : 24319 (pb_cplex) S 24264 24264 23951 0 -1 4202560 2927 0 0 0 467 3 0 0 16 0 8 0 146781493 143196160 12422 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 6 0 0 0
[pid=24265/tid=24320] ppid=24264 vsize=139840 CPUtime=4.88
/proc/24265/task/24320/stat : 24320 (pb_cplex) S 24264 24264 23951 0 -1 4202560 1042 0 0 0 484 4 0 0 15 0 8 0 146781493 143196160 12422 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 6 0 0 0
[pid=24265/tid=24321] ppid=24264 vsize=139840 CPUtime=4.65
/proc/24265/task/24321/stat : 24321 (pb_cplex) S 24264 24264 23951 0 -1 4202560 1711 0 0 0 462 3 0 0 15 0 8 0 146781493 143196160 12422 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 3 0 0 0
[pid=24265/tid=24322] ppid=24264 vsize=139840 CPUtime=4.65
/proc/24265/task/24322/stat : 24322 (pb_cplex) S 24264 24264 23951 0 -1 4202560 5057 0 0 0 462 3 0 0 16 0 8 0 146781493 143196160 12422 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 43.57
Current children cumulated vsize (KiB) 148540

[startup+25.5012 s]
/proc/loadavg: 2.47 0.59 0.20 5/195 24337
/proc/meminfo: memFree=31192172/32951124 swapFree=67111528/67111528
[pid=24264] ppid=24262 vsize=8700 CPUtime=0
/proc/24264/stat : 24264 (run) S 24262 24264 23951 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146781304 8908800 248 4089446400 4194304 4922060 140733755896608 18446744073709551615 228095270021 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/24264/statm: 2175 248 208 178 0 68 0
[pid=24265] ppid=24264 vsize=140664 CPUtime=85.43
/proc/24265/stat : 24265 (pb_cplex) R 24264 24264 23951 0 -1 4202496 61352 0 0 0 8464 79 0 0 19 0 8 0 146781304 144039936 12532 4089446400 134512640 146736320 4294026784 18446744073709551615 139044505 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/24265/statm: 35166 12532 1086 2985 0 31708 0
[pid=24265/tid=24331] ppid=24264 vsize=140664 CPUtime=0
/proc/24265/task/24331/stat : 24331 (pb_cplex) S 24264 24264 23951 0 -1 4202560 0 0 0 0 0 0 0 0 17 0 8 0 146783737 144039936 12532 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 1 0 0 0
[pid=24265/tid=24332] ppid=24264 vsize=140664 CPUtime=0
/proc/24265/task/24332/stat : 24332 (pb_cplex) S 24264 24264 23951 0 -1 4202560 0 0 0 0 0 0 0 0 21 0 8 0 146783737 144039936 12532 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 6 0 0 0
[pid=24265/tid=24333] ppid=24264 vsize=140664 CPUtime=0
/proc/24265/task/24333/stat : 24333 (pb_cplex) S 24264 24264 23951 0 -1 4202560 0 0 0 0 0 0 0 0 21 0 8 0 146783737 144039936 12532 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 6 0 0 0
[pid=24265/tid=24334] ppid=24264 vsize=140664 CPUtime=0
/proc/24265/task/24334/stat : 24334 (pb_cplex) S 24264 24264 23951 0 -1 4202560 0 0 0 0 0 0 0 0 21 0 8 0 146783737 144039936 12532 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 6 0 0 0
[pid=24265/tid=24335] ppid=24264 vsize=140664 CPUtime=0
/proc/24265/task/24335/stat : 24335 (pb_cplex) S 24264 24264 23951 0 -1 4202560 1 0 0 0 0 0 0 0 17 0 8 0 146783737 144039936 12532 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 1 0 0 0
[pid=24265/tid=24336] ppid=24264 vsize=140664 CPUtime=0
/proc/24265/task/24336/stat : 24336 (pb_cplex) S 24264 24264 23951 0 -1 4202560 1 0 0 0 0 0 0 0 21 0 8 0 146783737 144039936 12532 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 6 0 0 0
[pid=24265/tid=24337] ppid=24264 vsize=140664 CPUtime=0
/proc/24265/task/24337/stat : 24337 (pb_cplex) S 24264 24264 23951 0 -1 4202560 1 0 0 0 0 0 0 0 17 0 8 0 146783737 144039936 12532 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 85.43
Current children cumulated vsize (KiB) 149364

[startup+51.1235 s]
/proc/loadavg: 4.74 1.27 0.43 12/195 24358
/proc/meminfo: memFree=31196604/32951124 swapFree=67111528/67111528
[pid=24264] ppid=24262 vsize=8700 CPUtime=0
/proc/24264/stat : 24264 (run) S 24262 24264 23951 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146781304 8908800 248 4089446400 4194304 4922060 140733755896608 18446744073709551615 228095270021 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/24264/statm: 2175 248 208 178 0 68 0
[pid=24265] ppid=24264 vsize=145784 CPUtime=175.39
/proc/24265/stat : 24265 (pb_cplex) R 24264 24264 23951 0 -1 4202496 96715 0 0 0 17393 146 0 0 16 0 8 0 146781304 149282816 14679 4089446400 134512640 146736320 4294026784 18446744073709551615 136007292 0 0 4096 16387 18446744073709551615 0 0 17 6 0 0 0
/proc/24265/statm: 36446 14679 1102 2985 0 32988 0
[pid=24265/tid=24331] ppid=24264 vsize=145784 CPUtime=10.01
/proc/24265/task/24331/stat : 24331 (pb_cplex) S 24264 24264 23951 0 -1 4202560 1358 0 0 0 995 6 0 0 16 0 8 0 146783737 149282816 14679 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 1 0 0 0
[pid=24265/tid=24332] ppid=24264 vsize=145784 CPUtime=11.88
/proc/24265/task/24332/stat : 24332 (pb_cplex) R 24264 24264 23951 0 -1 4202560 1470 0 0 0 1180 8 0 0 15 0 8 0 146783737 149282816 14679 4089446400 134512640 146736320 4294026784 18446744073709551615 136403117 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=24265/tid=24333] ppid=24264 vsize=145784 CPUtime=9.89
/proc/24265/task/24333/stat : 24333 (pb_cplex) R 24264 24264 23951 0 -1 4202560 3851 0 0 0 981 8 0 0 15 0 8 0 146783737 149282816 14679 4089446400 134512640 146736320 4294026784 18446744073709551615 136403112 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=24265/tid=24334] ppid=24264 vsize=145784 CPUtime=9.44
/proc/24265/task/24334/stat : 24334 (pb_cplex) R 24264 24264 23951 0 -1 4202560 3346 0 0 0 937 7 0 0 15 0 8 0 146783737 149282816 14679 4089446400 134512640 146736320 4294026784 18446744073709551615 136400653 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=24265/tid=24335] ppid=24264 vsize=145784 CPUtime=13.51
/proc/24265/task/24335/stat : 24335 (pb_cplex) R 24264 24264 23951 0 -1 4202560 2576 0 0 0 1343 8 0 0 15 0 8 0 146783737 149282816 14679 4089446400 134512640 146736320 4294026784 18446744073709551615 135854505 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=24265/tid=24336] ppid=24264 vsize=145784 CPUtime=9.84
/proc/24265/task/24336/stat : 24336 (pb_cplex) R 24264 24264 23951 0 -1 4202560 4997 0 0 0 975 9 0 0 15 0 8 0 146783737 149282816 14679 4089446400 134512640 146736320 4294026784 18446744073709551615 136007626 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=24265/tid=24337] ppid=24264 vsize=145784 CPUtime=13.32
/proc/24265/task/24337/stat : 24337 (pb_cplex) R 24264 24264 23951 0 -1 4202560 8081 0 0 0 1321 11 0 0 15 0 8 0 146783737 149282816 14679 4089446400 134512640 146736320 4294026784 18446744073709551615 136400469 0 2 4096 16387 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 175.39
Current children cumulated vsize (KiB) 154484

[startup+102.33 s]
/proc/loadavg: 7.54 2.56 0.91 10/195 24360
/proc/meminfo: memFree=31192788/32951124 swapFree=67111528/67111528

################
# More data... #
################

/proc/24265/task/24336/stat : 24336 (pb_cplex) R 24264 24264 23951 0 -1 4202560 15342 0 0 0 9048 83 0 0 15 0 8 0 146783737 153477120 15409 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=24265/tid=24337] ppid=24264 vsize=149880 CPUtime=95.44
/proc/24265/task/24337/stat : 24337 (pb_cplex) R 24264 24264 23951 0 -1 4202560 21599 0 0 0 9455 89 0 0 15 0 8 0 146783737 153477120 15409 4089446400 134512640 146736320 4294026784 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 837.54
Current children cumulated vsize (KiB) 158580

[startup+282.302 s]
/proc/loadavg: 9.23 5.69 2.42 10/195 24455
/proc/meminfo: memFree=31192488/32951124 swapFree=67111528/67111528
[pid=24264] ppid=24262 vsize=8700 CPUtime=0
/proc/24264/stat : 24264 (run) S 24262 24264 23951 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146781304 8908800 248 4089446400 4194304 4922060 140733755896608 18446744073709551615 228095270021 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/24264/statm: 2175 248 208 178 0 68 0
[pid=24265] ppid=24264 vsize=151928 CPUtime=1051.94
/proc/24265/stat : 24265 (pb_cplex) R 24264 24264 23951 0 -1 4202496 205949 0 0 0 104292 902 0 0 15 0 8 0 146781304 155574272 15701 4089446400 134512640 146736320 4294026784 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0
/proc/24265/statm: 37982 15701 1104 2985 0 34524 0
[pid=24265/tid=24331] ppid=24264 vsize=151928 CPUtime=117.76
/proc/24265/task/24331/stat : 24331 (pb_cplex) R 24264 24264 23951 0 -1 4202560 13500 0 0 0 11673 103 0 0 15 0 8 0 146783737 155574272 15701 4089446400 134512640 146736320 4294026784 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=24265/tid=24332] ppid=24264 vsize=151928 CPUtime=120.18
/proc/24265/task/24332/stat : 24332 (pb_cplex) R 24264 24264 23951 0 -1 4202560 9337 0 0 0 11920 98 0 0 15 0 8 0 146783737 155574272 15701 4089446400 134512640 146736320 4294026784 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=24265/tid=24333] ppid=24264 vsize=151928 CPUtime=118.14
/proc/24265/task/24333/stat : 24333 (pb_cplex) R 24264 24264 23951 0 -1 4202560 13651 0 0 0 11712 102 0 0 15 0 8 0 146783737 155574272 15701 4089446400 134512640 146736320 4294026784 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=24265/tid=24334] ppid=24264 vsize=151928 CPUtime=117.51
/proc/24265/task/24334/stat : 24334 (pb_cplex) R 24264 24264 23951 0 -1 4202560 20405 0 0 0 11653 98 0 0 15 0 8 0 146783737 155574272 15701 4089446400 134512640 146736320 4294026784 18446744073709551615 136007627 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=24265/tid=24335] ppid=24264 vsize=151928 CPUtime=122.11
/proc/24265/task/24335/stat : 24335 (pb_cplex) R 24264 24264 23951 0 -1 4202560 16150 0 0 0 12109 102 0 0 15 0 8 0 146783737 155574272 15701 4089446400 134512640 146736320 4294026784 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=24265/tid=24336] ppid=24264 vsize=151928 CPUtime=117.81
/proc/24265/task/24336/stat : 24336 (pb_cplex) R 24264 24264 23951 0 -1 4202560 16377 0 0 0 11677 104 0 0 15 0 8 0 146783737 155574272 15701 4089446400 134512640 146736320 4294026784 18446744073709551615 136387662 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=24265/tid=24337] ppid=24264 vsize=151928 CPUtime=121.72
/proc/24265/task/24337/stat : 24337 (pb_cplex) R 24264 24264 23951 0 -1 4202560 23265 0 0 0 12063 109 0 0 15 0 8 0 146783737 155574272 15701 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1051.94
Current children cumulated vsize (KiB) 160628

[startup+342.3 s]
/proc/loadavg: 9.15 6.28 2.82 9/195 24457
/proc/meminfo: memFree=31188712/32951124 swapFree=67111528/67111528
[pid=24264] ppid=24262 vsize=8700 CPUtime=0
/proc/24264/stat : 24264 (run) S 24262 24264 23951 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146781304 8908800 248 4089446400 4194304 4922060 140733755896608 18446744073709551615 228095270021 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/24264/statm: 2175 248 208 178 0 68 0
[pid=24265] ppid=24264 vsize=149880 CPUtime=1270.51
/proc/24265/stat : 24265 (pb_cplex) S 24264 24264 23951 0 -1 4202496 229484 0 0 0 125974 1077 0 0 15 0 8 0 146781304 153477120 15425 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0
/proc/24265/statm: 37470 15425 1104 2985 0 34012 0
[pid=24265/tid=24331] ppid=24264 vsize=149880 CPUtime=144.78
/proc/24265/task/24331/stat : 24331 (pb_cplex) S 24264 24264 23951 0 -1 4202560 21437 0 0 0 14354 124 0 0 16 0 8 0 146783737 153477120 15425 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 1 0 0 0
[pid=24265/tid=24332] ppid=24264 vsize=149880 CPUtime=147.18
/proc/24265/task/24332/stat : 24332 (pb_cplex) S 24264 24264 23951 0 -1 4202560 11551 0 0 0 14600 118 0 0 15 0 8 0 146783737 153477120 15425 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 1 0 0 0
[pid=24265/tid=24333] ppid=24264 vsize=149880 CPUtime=145.15
/proc/24265/task/24333/stat : 24333 (pb_cplex) S 24264 24264 23951 0 -1 4202560 15494 0 0 0 14390 125 0 0 15 0 8 0 146783737 153477120 15425 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 0 0 0 0
[pid=24265/tid=24334] ppid=24264 vsize=149880 CPUtime=144.34
/proc/24265/task/24334/stat : 24334 (pb_cplex) S 24264 24264 23951 0 -1 4202560 22589 0 0 0 14314 120 0 0 15 0 8 0 146783737 153477120 15425 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 1 0 0 0
[pid=24265/tid=24335] ppid=24264 vsize=149880 CPUtime=149.12
/proc/24265/task/24335/stat : 24335 (pb_cplex) R 24264 24264 23951 0 -1 4202560 19540 0 0 0 14785 127 0 0 16 0 8 0 146783737 153477120 15425 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=24265/tid=24336] ppid=24264 vsize=149880 CPUtime=144.78
/proc/24265/task/24336/stat : 24336 (pb_cplex) S 24264 24264 23951 0 -1 4202560 19010 0 0 0 14352 126 0 0 15 0 8 0 146783737 153477120 15425 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 4 0 0 0
[pid=24265/tid=24337] ppid=24264 vsize=149880 CPUtime=148.92
/proc/24265/task/24337/stat : 24337 (pb_cplex) S 24264 24264 23951 0 -1 4202560 24904 0 0 0 14763 129 0 0 15 0 8 0 146783737 153477120 15425 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1270.51
Current children cumulated vsize (KiB) 158580

[startup+402.3 s]
/proc/loadavg: 9.44 6.88 3.24 10/195 24529
/proc/meminfo: memFree=31199920/32951124 swapFree=67111528/67111528
[pid=24264] ppid=24262 vsize=8700 CPUtime=0
/proc/24264/stat : 24264 (run) S 24262 24264 23951 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146781304 8908800 248 4089446400 4194304 4922060 140733755896608 18446744073709551615 228095270021 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/24264/statm: 2175 248 208 178 0 68 0
[pid=24265] ppid=24264 vsize=149880 CPUtime=1521.85
/proc/24265/stat : 24265 (pb_cplex) R 24264 24264 23951 0 -1 4202496 252781 0 0 0 150886 1299 0 0 15 0 8 0 146781304 153477120 15408 4089446400 134512640 146736320 4294026784 18446744073709551615 136674344 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0
/proc/24265/statm: 37470 15408 1104 2985 0 34012 0
[pid=24265/tid=24331] ppid=24264 vsize=149880 CPUtime=176.02
/proc/24265/task/24331/stat : 24331 (pb_cplex) R 24264 24264 23951 0 -1 4202560 23540 0 0 0 17452 150 0 0 15 0 8 0 146783737 153477120 15408 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=24265/tid=24332] ppid=24264 vsize=149880 CPUtime=178.23
/proc/24265/task/24332/stat : 24332 (pb_cplex) S 24264 24264 23951 0 -1 4202560 14816 0 0 0 17676 147 0 0 15 0 8 0 146783737 153477120 15408 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 4 0 0 0
[pid=24265/tid=24333] ppid=24264 vsize=149880 CPUtime=176.03
/proc/24265/task/24333/stat : 24333 (pb_cplex) S 24264 24264 23951 0 -1 4202560 17433 0 0 0 17453 150 0 0 15 0 8 0 146783737 153477120 15408 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 4 0 0 0
[pid=24265/tid=24334] ppid=24264 vsize=149880 CPUtime=175.54
/proc/24265/task/24334/stat : 24334 (pb_cplex) S 24264 24264 23951 0 -1 4202560 27379 0 0 0 17407 147 0 0 15 0 8 0 146783737 153477120 15408 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 4 0 0 0
[pid=24265/tid=24335] ppid=24264 vsize=149880 CPUtime=180.48
/proc/24265/task/24335/stat : 24335 (pb_cplex) R 24264 24264 23951 0 -1 4202560 22986 0 0 0 17893 155 0 0 16 0 8 0 146783737 153477120 15408 4089446400 134512640 146736320 4294026784 18446744073709551615 144580332 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=24265/tid=24336] ppid=24264 vsize=149880 CPUtime=175.82
/proc/24265/task/24336/stat : 24336 (pb_cplex) S 24264 24264 23951 0 -1 4202560 20250 0 0 0 17430 152 0 0 15 0 8 0 146783737 153477120 15408 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 4 0 0 0
[pid=24265/tid=24337] ppid=24264 vsize=149880 CPUtime=180.07
/proc/24265/task/24337/stat : 24337 (pb_cplex) S 24264 24264 23951 0 -1 4202560 28029 0 0 0 17851 156 0 0 15 0 8 0 146783737 153477120 15408 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 1521.85
Current children cumulated vsize (KiB) 158580

[startup+462.302 s]
/proc/loadavg: 9.79 7.36 3.63 8/195 24544
/proc/meminfo: memFree=31201624/32951124 swapFree=67111528/67111528
[pid=24264] ppid=24262 vsize=8700 CPUtime=0
/proc/24264/stat : 24264 (run) S 24262 24264 23951 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146781304 8908800 248 4089446400 4194304 4922060 140733755896608 18446744073709551615 228095270021 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/24264/statm: 2175 248 208 178 0 68 0
[pid=24265] ppid=24264 vsize=150904 CPUtime=1757.9
/proc/24265/stat : 24265 (pb_cplex) S 24264 24264 23951 0 -1 4202496 273286 0 0 0 174281 1509 0 0 16 0 8 0 146781304 154525696 15652 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0
/proc/24265/statm: 37726 15652 1104 2985 0 34268 0
[pid=24265/tid=24331] ppid=24264 vsize=150904 CPUtime=205.29
/proc/24265/task/24331/stat : 24331 (pb_cplex) S 24264 24264 23951 0 -1 4202560 25629 0 0 0 20356 173 0 0 16 0 8 0 146783737 154525696 15652 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 7 0 0 0
[pid=24265/tid=24332] ppid=24264 vsize=150904 CPUtime=207.4
/proc/24265/task/24332/stat : 24332 (pb_cplex) S 24264 24264 23951 0 -1 4202560 18761 0 0 0 20566 174 0 0 15 0 8 0 146783737 154525696 15652 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 0 0 0 0
[pid=24265/tid=24333] ppid=24264 vsize=150904 CPUtime=205.34
/proc/24265/task/24333/stat : 24333 (pb_cplex) S 24264 24264 23951 0 -1 4202560 21260 0 0 0 20358 176 0 0 15 0 8 0 146783737 154525696 15652 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 6 0 0 0
[pid=24265/tid=24334] ppid=24264 vsize=150904 CPUtime=204.76
/proc/24265/task/24334/stat : 24334 (pb_cplex) S 24264 24264 23951 0 -1 4202560 30117 0 0 0 20301 175 0 0 15 0 8 0 146783737 154525696 15652 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 6 0 0 0
[pid=24265/tid=24335] ppid=24264 vsize=150904 CPUtime=209.75
/proc/24265/task/24335/stat : 24335 (pb_cplex) S 24264 24264 23951 0 -1 4202560 23503 0 0 0 20793 182 0 0 15 0 8 0 146783737 154525696 15652 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 0 0 0 0
[pid=24265/tid=24336] ppid=24264 vsize=150904 CPUtime=205.09
/proc/24265/task/24336/stat : 24336 (pb_cplex) S 24264 24264 23951 0 -1 4202560 22351 0 0 0 20333 176 0 0 16 0 8 0 146783737 154525696 15652 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 0 0 0 0
[pid=24265/tid=24337] ppid=24264 vsize=150904 CPUtime=209.14
/proc/24265/task/24337/stat : 24337 (pb_cplex) R 24264 24264 23951 0 -1 4202560 30454 0 0 0 20728 186 0 0 15 0 8 0 146783737 154525696 15652 4089446400 134512640 146736320 4294026784 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 1757.9
Current children cumulated vsize (KiB) 159604



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+473.003 s]
/proc/loadavg: 9.67 7.41 3.69 9/195 24545
/proc/meminfo: memFree=31198896/32951124 swapFree=67111528/67111528
[pid=24264] ppid=24262 vsize=8700 CPUtime=0
/proc/24264/stat : 24264 (run) S 24262 24264 23951 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146781304 8908800 248 4089446400 4194304 4922060 140733755896608 18446744073709551615 228095270021 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/24264/statm: 2175 248 208 178 0 68 0
[pid=24265] ppid=24264 vsize=150904 CPUtime=1800.14
/proc/24265/stat : 24265 (pb_cplex) R 24264 24264 23951 0 -1 4202496 275133 0 0 0 178465 1549 0 0 16 0 8 0 146781304 154525696 15655 4089446400 134512640 146736320 4294026784 18446744073709551615 139023048 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/24265/statm: 37726 15655 1104 2985 0 34268 0
[pid=24265/tid=24331] ppid=24264 vsize=150904 CPUtime=210.45
/proc/24265/task/24331/stat : 24331 (pb_cplex) R 24264 24264 23951 0 -1 4202560 25633 0 0 0 20868 177 0 0 15 0 8 0 146783737 154525696 15655 4089446400 134512640 146736320 4294026784 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=24265/tid=24332] ppid=24264 vsize=150904 CPUtime=212.7
/proc/24265/task/24332/stat : 24332 (pb_cplex) R 24264 24264 23951 0 -1 4202560 19765 0 0 0 21090 180 0 0 15 0 8 0 146783737 154525696 15655 4089446400 134512640 146736320 4294026784 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=24265/tid=24333] ppid=24264 vsize=150904 CPUtime=210.6
/proc/24265/task/24333/stat : 24333 (pb_cplex) R 24264 24264 23951 0 -1 4202560 21275 0 0 0 20880 180 0 0 15 0 8 0 146783737 154525696 15655 4089446400 134512640 146736320 4294026784 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=24265/tid=24334] ppid=24264 vsize=150904 CPUtime=210.01
/proc/24265/task/24334/stat : 24334 (pb_cplex) R 24264 24264 23951 0 -1 4202560 30279 0 0 0 20820 181 0 0 15 0 8 0 146783737 154525696 15655 4089446400 134512640 146736320 4294026784 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=24265/tid=24335] ppid=24264 vsize=150904 CPUtime=215.02
/proc/24265/task/24335/stat : 24335 (pb_cplex) R 24264 24264 23951 0 -1 4202560 23585 0 0 0 21314 188 0 0 15 0 8 0 146783737 154525696 15655 4089446400 134512640 146736320 4294026784 18446744073709551615 135234618 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=24265/tid=24336] ppid=24264 vsize=150904 CPUtime=210.29
/proc/24265/task/24336/stat : 24336 (pb_cplex) R 24264 24264 23951 0 -1 4202560 22636 0 0 0 20849 180 0 0 15 0 8 0 146783737 154525696 15655 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=24265/tid=24337] ppid=24264 vsize=150904 CPUtime=214.24
/proc/24265/task/24337/stat : 24337 (pb_cplex) S 24264 24264 23951 0 -1 4202560 30640 0 0 0 21232 192 0 0 15 0 8 0 146783737 154525696 15655 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 1800.14
Current children cumulated vsize (KiB) 159604

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+473.003 s]
/proc/loadavg: 9.67 7.41 3.69 9/195 24545
/proc/meminfo: memFree=31198896/32951124 swapFree=67111528/67111528
[pid=24264] ppid=24262 vsize=8700 CPUtime=0
/proc/24264/stat : 24264 (run) S 24262 24264 23951 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 146781304 8908800 248 4089446400 4194304 4922060 140733755896608 18446744073709551615 228095270021 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/24264/statm: 2175 248 208 178 0 68 0
[pid=24265] ppid=24264 vsize=150904 CPUtime=1800.14
/proc/24265/stat : 24265 (pb_cplex) R 24264 24264 23951 0 -1 4202496 275133 0 0 0 178465 1549 0 0 16 0 8 0 146781304 154525696 15655 4089446400 134512640 146736320 4294026784 18446744073709551615 139023048 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/24265/statm: 37726 15655 1104 2985 0 34268 0
[pid=24265/tid=24331] ppid=24264 vsize=150904 CPUtime=210.45
/proc/24265/task/24331/stat : 24331 (pb_cplex) R 24264 24264 23951 0 -1 4202560 25633 0 0 0 20868 177 0 0 15 0 8 0 146783737 154525696 15655 4089446400 134512640 146736320 4294026784 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=24265/tid=24332] ppid=24264 vsize=150904 CPUtime=212.7
/proc/24265/task/24332/stat : 24332 (pb_cplex) R 24264 24264 23951 0 -1 4202560 19765 0 0 0 21090 180 0 0 15 0 8 0 146783737 154525696 15655 4089446400 134512640 146736320 4294026784 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=24265/tid=24333] ppid=24264 vsize=150904 CPUtime=210.6
/proc/24265/task/24333/stat : 24333 (pb_cplex) R 24264 24264 23951 0 -1 4202560 21275 0 0 0 20880 180 0 0 15 0 8 0 146783737 154525696 15655 4089446400 134512640 146736320 4294026784 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=24265/tid=24334] ppid=24264 vsize=150904 CPUtime=210.01
/proc/24265/task/24334/stat : 24334 (pb_cplex) R 24264 24264 23951 0 -1 4202560 30279 0 0 0 20820 181 0 0 15 0 8 0 146783737 154525696 15655 4089446400 134512640 146736320 4294026784 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=24265/tid=24335] ppid=24264 vsize=150904 CPUtime=215.02
/proc/24265/task/24335/stat : 24335 (pb_cplex) R 24264 24264 23951 0 -1 4202560 23585 0 0 0 21314 188 0 0 15 0 8 0 146783737 154525696 15655 4089446400 134512640 146736320 4294026784 18446744073709551615 135234618 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=24265/tid=24336] ppid=24264 vsize=150904 CPUtime=210.29
/proc/24265/task/24336/stat : 24336 (pb_cplex) R 24264 24264 23951 0 -1 4202560 22636 0 0 0 20849 180 0 0 15 0 8 0 146783737 154525696 15655 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=24265/tid=24337] ppid=24264 vsize=150904 CPUtime=214.24
/proc/24265/task/24337/stat : 24337 (pb_cplex) S 24264 24264 23951 0 -1 4202560 30640 0 0 0 21232 192 0 0 15 0 8 0 146783737 154525696 15655 4089446400 134512640 146736320 4294026784 18446744073709551615 144580073 0 2 4096 16387 18446604470108601024 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 1800.14
Current children cumulated vsize (KiB) 159604

Child status: 0
Real time (s): 473.025
CPU time (s): 1800.2
CPU user time (s): 1784.7
CPU system time (s): 15.4966
CPU usage (%): 380.572
Max. virtual memory (cumulated for all children) (KiB): 164724

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1784.7
system time used= 15.4966
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 275507
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 331860
involuntary context switches= 24212

runsolver used 0.910861 second user time and 1.89571 second system time

The end

Launcher Data 

Begin job on node142 at 2010-07-02 09:14:27
IDJOB=2697883
IDBENCH=78929
IDSOLVER=1210
FILE ID=node142/2697883-1278054866
PBS_JOBID= 11198897
Free space on /tmp= 74192 MiB

SOLVER NAME= pb_cplex 2010-07-01
BENCH NAME= PB10/normalized-PB10/PARTIAL-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/tsp/normalized-t3002.11tsp11.1900576151--soft-66-100-0.wbo
COMMAND LINE= run BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-2697883-1278054866/watcher-2697883-1278054866 -o /tmp/evaluation-result-2697883-1278054866/solver-2697883-1278054866 -C 1800 -W 2000 -M 3800  run HOME/instance-2697883-1278054866.wbo

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 3800 MiB
MAX NB THREAD= 0

MD5SUM BENCH= e60bec5d313c91289af640357d00991f
RANDOM SEED=203732205

node142.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.827
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.827
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.16
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.827
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.827
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.827
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.827
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.827
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.827
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      31282152 kB
Buffers:        220672 kB
Cached:        1207312 kB
SwapCached:          0 kB
Active:         571132 kB
Inactive:       948188 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      31282152 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           33700 kB
Writeback:           0 kB
AnonPages:       91672 kB
Mapped:          18780 kB
Slab:            84536 kB
PageTables:       5240 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   281676 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 74176 MiB
End job on node142 at 2010-07-02 09:22:20