Trace number 2697903

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
pb_cplex 2010-07-01? (TO) 1800.14 469.742

General information on the benchmark

Name/PARTIAL-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/tsp/
normalized-t3002.11tsp11.1900573137--soft-33-100-0.wbo
MD5SUM34c2ccfe3c5c18b062797c1b5794b2d4
Bench CategoryPARTIAL-SMALLINT-LIN (both soft and hard constraints, small integers, linear constraints)
Best result obtained on this benchmarkMOPT
Best cost obtained on this benchmark1
Best CPU time to get the best result obtained on this benchmark254.587
Max-Satisfiable
Max-(Un)Satisfiability was proved
Best value of the cost
Optimality of the best cost was proved
Number of variables231
Total number of constraints2707
Number of soft constraints1788
Number of constraints which are clauses2684
Number of constraints which are cardinality constraints (but not clauses)22
Number of constraints which are nor clauses,nor cardinality constraints1
Minimum length of a constraint2
Maximum length of a constraint110
Top cost 89002
Min constraint cost 1
Max constraint cost 100
Sum of constraints costs 89001
Biggest number in a constraint 25
Number of bits of the biggest number in a constraint 5
Biggest sum of numbers in a constraint 693
Number of bits of the biggest sum of numbers10
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c pb_cplex 1.0 - 20100701
0.00/0.00	c Parsing wbo file format. File HOME/instance-2697903-1278054867.wbo.
0.00/0.00	c Instance file HOME/instance-2697903-1278054867.wbo
0.00/0.00	c File size is 92231 bytes.
0.00/0.64	c Highest Coefficient sum: 693
0.00/0.64	c  IBM ILOG License Manager: "IBM ILOG Optimization Suite for Academic Initiative" is accessing CPLEX 12 with option(s): "e m b q ".
0.00/0.64	c ============================[ Problem Statistics ]=============================
0.00/0.64	c |                                                                             |
0.00/0.64	c |  Parsing time:          0.64         s                                      |
0.00/0.64	c |  Number Variables:      2019                                                |
0.00/0.64	c |  Number Constraints:    2708                                                |
0.00/0.64	c ===============================================================================
1800.06/469.73	c Remaining time 1799.36 sec.
1800.06/469.73	c 
1800.06/469.73	c *** INTERRUPTED ***
1800.06/469.73	s UNKNOWN
1800.06/469.73	c Total CPU time              : 1784.67 s
1800.06/469.73	c Memory used           : 137.84 MB
1800.06/469.73	c CPU time              : 1784.67 s
1800.06/469.73	c 
1800.06/469.73	c *** INTERRUPTED ***

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver version 3.2.9 (svn:492) (c) roussel@cril.univ-artois.fr

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-2697903-1278054867/watcher-2697903-1278054867 -o /tmp/evaluation-result-2697903-1278054867/solver-2697903-1278054867 -C 1800 -W 2000 -M 3800 run HOME/instance-2697903-1278054867.wbo 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 3891200 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 3942400 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.11 0.03 0.01 3/175 23525
/proc/meminfo: memFree=30803760/32950928 swapFree=67111528/67111528
[pid=23525] ppid=23521 vsize=8700 CPUtime=0
/proc/23525/stat : 23525 (run) S 23521 23525 23098 0 -1 4202496 365 0 0 0 0 0 0 0 18 0 1 0 146781645 8908800 249 4089446400 4194304 4922060 140735208630000 18446744073709551615 254500510853 0 65536 4100 65538 18446744071562232103 0 0 17 0 0 0 0
/proc/23525/statm: 2175 249 208 178 0 68 0

[startup+0.00719105 s]
/proc/loadavg: 0.11 0.03 0.01 3/175 23525
/proc/meminfo: memFree=30803760/32950928 swapFree=67111528/67111528
[pid=23525] ppid=23521 vsize=8700 CPUtime=0
/proc/23525/stat : 23525 (run) S 23521 23525 23098 0 -1 4202496 365 0 0 0 0 0 0 0 18 0 1 0 146781645 8908800 249 4089446400 4194304 4922060 140735208630000 18446744073709551615 254500510853 0 65536 4100 65538 18446744071562232103 0 0 17 0 0 0 0
/proc/23525/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.101208 s]
/proc/loadavg: 0.11 0.03 0.01 3/175 23525
/proc/meminfo: memFree=30803760/32950928 swapFree=67111528/67111528
[pid=23525] ppid=23521 vsize=8700 CPUtime=0
/proc/23525/stat : 23525 (run) S 23521 23525 23098 0 -1 4202496 365 0 0 0 0 0 0 0 18 0 1 0 146781645 8908800 249 4089446400 4194304 4922060 140735208630000 18446744073709551615 254500510853 0 65536 4100 65538 18446744071562232103 0 0 17 0 0 0 0
/proc/23525/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.300209 s]
/proc/loadavg: 0.11 0.03 0.01 3/175 23525
/proc/meminfo: memFree=30803760/32950928 swapFree=67111528/67111528
[pid=23525] ppid=23521 vsize=8700 CPUtime=0
/proc/23525/stat : 23525 (run) S 23521 23525 23098 0 -1 4202496 365 0 0 0 0 0 0 0 18 0 1 0 146781645 8908800 249 4089446400 4194304 4922060 140735208630000 18446744073709551615 254500510853 0 65536 4100 65538 18446744071562232103 0 0 17 0 0 0 0
/proc/23525/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.701019 s]
/proc/loadavg: 0.11 0.03 0.01 3/175 23525
/proc/meminfo: memFree=30803760/32950928 swapFree=67111528/67111528
[pid=23525] ppid=23521 vsize=8700 CPUtime=0
/proc/23525/stat : 23525 (run) S 23521 23525 23098 0 -1 4202496 365 0 0 0 0 0 0 0 18 0 1 0 146781645 8908800 249 4089446400 4194304 4922060 140735208630000 18446744073709551615 254500510853 0 65536 4100 65538 18446744071562232103 0 0 17 0 0 0 0
/proc/23525/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+1.50085 s]
/proc/loadavg: 0.11 0.03 0.01 6/188 23566
/proc/meminfo: memFree=30739548/32950928 swapFree=67111528/67111528
[pid=23525] ppid=23521 vsize=8700 CPUtime=0
/proc/23525/stat : 23525 (run) S 23521 23525 23098 0 -1 4202496 365 0 0 0 0 0 0 0 18 0 1 0 146781645 8908800 249 4089446400 4194304 4922060 140735208630000 18446744073709551615 254500510853 0 65536 4100 65538 18446744071562232103 0 0 17 0 0 0 0
/proc/23525/statm: 2175 249 208 178 0 68 0
[pid=23528] ppid=23525 vsize=88392 CPUtime=2.59
/proc/23528/stat : 23528 (pb_cplex) R 23525 23525 23098 0 -1 4202496 16246 0 0 0 253 6 0 0 17 0 1 0 146781645 90513408 8238 4089446400 134512640 146736320 4288421280 18446744073709551615 136017757 0 0 4096 16387 0 0 0 17 1 0 0 0
/proc/23528/statm: 22098 8238 1069 2985 0 18640 0
Current children cumulated CPU time (s) 2.59
Current children cumulated vsize (KiB) 97092

[startup+3.10054 s]
/proc/loadavg: 0.11 0.03 0.01 3/195 23615
/proc/meminfo: memFree=30731184/32950928 swapFree=67111528/67111528
[pid=23525] ppid=23521 vsize=8700 CPUtime=0
/proc/23525/stat : 23525 (run) S 23521 23525 23098 0 -1 4202496 365 0 0 0 0 0 0 0 18 0 1 0 146781645 8908800 249 4089446400 4194304 4922060 140735208630000 18446744073709551615 254500510853 0 65536 4100 65538 18446744071562232103 0 0 17 0 0 0 0
/proc/23525/statm: 2175 249 208 178 0 68 0
[pid=23528] ppid=23525 vsize=131744 CPUtime=7
/proc/23528/stat : 23528 (pb_cplex) R 23525 23525 23098 0 -1 4202496 25191 0 0 0 690 10 0 0 18 0 8 0 146781645 134905856 11082 4089446400 134512640 146736320 4288421280 18446744073709551615 136403109 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/23528/statm: 32936 11082 1084 2985 0 29478 0
[pid=23528/tid=23609] ppid=23525 vsize=131744 CPUtime=0.47
/proc/23528/task/23609/stat : 23609 (pb_cplex) R 23525 23525 23098 0 -1 4202560 670 0 0 0 47 0 0 0 16 0 8 0 146781849 134905856 11082 4089446400 134512640 146736320 4288421280 18446744073709551615 139044554 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=23528/tid=23610] ppid=23525 vsize=131744 CPUtime=0.39
/proc/23528/task/23610/stat : 23610 (pb_cplex) S 23525 23525 23098 0 -1 4202560 6 0 0 0 39 0 0 0 16 0 8 0 146781849 134905856 11082 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 5 0 0 0
[pid=23528/tid=23611] ppid=23525 vsize=131744 CPUtime=0.34
/proc/23528/task/23611/stat : 23611 (pb_cplex) S 23525 23525 23098 0 -1 4202560 1522 0 0 0 34 0 0 0 16 0 8 0 146781849 134905856 11082 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 0 0 0 0
[pid=23528/tid=23612] ppid=23525 vsize=131744 CPUtime=0.47
/proc/23528/task/23612/stat : 23612 (pb_cplex) R 23525 23525 23098 0 -1 4202560 7 0 0 0 47 0 0 0 16 0 8 0 146781849 134905856 11082 4089446400 134512640 146736320 4288421280 18446744073709551615 136007083 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=23528/tid=23613] ppid=23525 vsize=131744 CPUtime=0.46
/proc/23528/task/23613/stat : 23613 (pb_cplex) R 23525 23525 23098 0 -1 4202560 411 0 0 0 46 0 0 0 16 0 8 0 146781849 134905856 11082 4089446400 134512640 146736320 4288421280 18446744073709551615 136007289 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=23528/tid=23614] ppid=23525 vsize=131744 CPUtime=0.45
/proc/23528/task/23614/stat : 23614 (pb_cplex) R 23525 23525 23098 0 -1 4202560 1257 0 0 0 45 0 0 0 16 0 8 0 146781849 134905856 11082 4089446400 134512640 146736320 4288421280 18446744073709551615 136403109 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=23528/tid=23615] ppid=23525 vsize=131744 CPUtime=0.42
/proc/23528/task/23615/stat : 23615 (pb_cplex) S 23525 23525 23098 0 -1 4202560 23 0 0 0 42 0 0 0 16 0 8 0 146781849 134905856 11082 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 7
Current children cumulated vsize (KiB) 140444

[startup+6.31389 s]
/proc/loadavg: 1.06 0.23 0.07 9/195 23615
/proc/meminfo: memFree=30708368/32950928 swapFree=67111528/67111528
[pid=23525] ppid=23521 vsize=8700 CPUtime=0
/proc/23525/stat : 23525 (run) S 23521 23525 23098 0 -1 4202496 365 0 0 0 0 0 0 0 18 0 1 0 146781645 8908800 249 4089446400 4194304 4922060 140735208630000 18446744073709551615 254500510853 0 65536 4100 65538 18446744071562232103 0 0 17 0 0 0 0
/proc/23525/statm: 2175 249 208 178 0 68 0
[pid=23528] ppid=23525 vsize=136864 CPUtime=16.38
/proc/23528/stat : 23528 (pb_cplex) S 23525 23525 23098 0 -1 4202496 31786 0 0 0 1618 20 0 0 16 0 8 0 146781645 140148736 12108 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/23528/statm: 34216 12108 1078 2985 0 30758 0
[pid=23528/tid=23609] ppid=23525 vsize=136864 CPUtime=1.56
/proc/23528/task/23609/stat : 23609 (pb_cplex) S 23525 23525 23098 0 -1 4202560 710 0 0 0 156 0 0 0 15 0 8 0 146781849 140148736 12108 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 4 0 0 0
[pid=23528/tid=23610] ppid=23525 vsize=136864 CPUtime=1.41
/proc/23528/task/23610/stat : 23610 (pb_cplex) S 23525 23525 23098 0 -1 4202560 461 0 0 0 140 1 0 0 15 0 8 0 146781849 140148736 12108 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 2 0 0 0
[pid=23528/tid=23611] ppid=23525 vsize=136864 CPUtime=1.36
/proc/23528/task/23611/stat : 23611 (pb_cplex) S 23525 23525 23098 0 -1 4202560 2324 0 0 0 136 0 0 0 15 0 8 0 146781849 140148736 12108 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 2 0 0 0
[pid=23528/tid=23612] ppid=23525 vsize=136864 CPUtime=1.5
/proc/23528/task/23612/stat : 23612 (pb_cplex) S 23525 23525 23098 0 -1 4202560 235 0 0 0 149 1 0 0 15 0 8 0 146781849 140148736 12108 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 2 0 0 0
[pid=23528/tid=23613] ppid=23525 vsize=136864 CPUtime=1.82
/proc/23528/task/23613/stat : 23613 (pb_cplex) S 23525 23525 23098 0 -1 4202560 1253 0 0 0 181 1 0 0 15 0 8 0 146781849 140148736 12108 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 2 0 0 0
[pid=23528/tid=23614] ppid=23525 vsize=136864 CPUtime=1.62
/proc/23528/task/23614/stat : 23614 (pb_cplex) R 23525 23525 23098 0 -1 4202560 2983 0 0 0 161 1 0 0 15 0 8 0 146781849 140148736 12108 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=23528/tid=23615] ppid=23525 vsize=136864 CPUtime=1.49
/proc/23528/task/23615/stat : 23615 (pb_cplex) S 23525 23525 23098 0 -1 4202560 85 0 0 0 148 1 0 0 15 0 8 0 146781849 140148736 12108 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 16.38
Current children cumulated vsize (KiB) 145564

[startup+12.7051 s]
/proc/loadavg: 1.62 0.36 0.12 9/195 23615
/proc/meminfo: memFree=30699696/32950928 swapFree=67111528/67111528
[pid=23525] ppid=23521 vsize=8700 CPUtime=0
/proc/23525/stat : 23525 (run) S 23521 23525 23098 0 -1 4202496 365 0 0 0 0 0 0 0 18 0 1 0 146781645 8908800 249 4089446400 4194304 4922060 140735208630000 18446744073709551615 254500510853 0 65536 4100 65538 18446744071562232103 0 0 17 0 0 0 0
/proc/23525/statm: 2175 249 208 178 0 68 0
[pid=23528] ppid=23525 vsize=146080 CPUtime=39.61
/proc/23528/stat : 23528 (pb_cplex) S 23525 23525 23098 0 -1 4202496 36222 0 0 0 3920 41 0 0 15 0 8 0 146781645 149585920 13852 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0
/proc/23528/statm: 36520 13852 1078 2985 0 33062 0
[pid=23528/tid=23609] ppid=23525 vsize=146080 CPUtime=4.51
/proc/23528/task/23609/stat : 23609 (pb_cplex) S 23525 23525 23098 0 -1 4202560 1305 0 0 0 448 3 0 0 15 0 8 0 146781849 149585920 13852 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 0 0 0 0
[pid=23528/tid=23610] ppid=23525 vsize=146080 CPUtime=4.25
/proc/23528/task/23610/stat : 23610 (pb_cplex) S 23525 23525 23098 0 -1 4202560 1047 0 0 0 420 5 0 0 15 0 8 0 146781849 149585920 13852 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 5 0 0 0
[pid=23528/tid=23611] ppid=23525 vsize=146080 CPUtime=4.24
/proc/23528/task/23611/stat : 23611 (pb_cplex) S 23525 23525 23098 0 -1 4202560 3480 0 0 0 420 4 0 0 15 0 8 0 146781849 149585920 13852 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 5 0 0 0
[pid=23528/tid=23612] ppid=23525 vsize=146080 CPUtime=4.37
/proc/23528/task/23612/stat : 23612 (pb_cplex) S 23525 23525 23098 0 -1 4202560 240 0 0 0 434 3 0 0 15 0 8 0 146781849 149585920 13852 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 7 0 0 0
[pid=23528/tid=23613] ppid=23525 vsize=146080 CPUtime=4.82
/proc/23528/task/23613/stat : 23613 (pb_cplex) R 23525 23525 23098 0 -1 4202560 1837 0 0 0 478 4 0 0 15 0 8 0 146781849 149585920 13852 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=23528/tid=23614] ppid=23525 vsize=146080 CPUtime=4.36
/proc/23528/task/23614/stat : 23614 (pb_cplex) S 23525 23525 23098 0 -1 4202560 4045 0 0 0 432 4 0 0 15 0 8 0 146781849 149585920 13852 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 2 0 0 0
[pid=23528/tid=23615] ppid=23525 vsize=146080 CPUtime=4.33
/proc/23528/task/23615/stat : 23615 (pb_cplex) S 23525 23525 23098 0 -1 4202560 436 0 0 0 430 3 0 0 15 0 8 0 146781849 149585920 13852 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39.61
Current children cumulated vsize (KiB) 154780

[startup+25.5041 s]
/proc/loadavg: 3.43 0.81 0.27 5/195 23629
/proc/meminfo: memFree=30685064/32950928 swapFree=67111528/67111528
[pid=23525] ppid=23521 vsize=8700 CPUtime=0
/proc/23525/stat : 23525 (run) S 23521 23525 23098 0 -1 4202496 365 0 0 0 0 0 0 0 18 0 1 0 146781645 8908800 249 4089446400 4194304 4922060 140735208630000 18446744073709551615 254500510853 0 65536 4100 65538 18446744071562232103 0 0 17 0 0 0 0
/proc/23525/statm: 2175 249 208 178 0 68 0
[pid=23528] ppid=23525 vsize=136232 CPUtime=77.97
/proc/23528/stat : 23528 (pb_cplex) R 23525 23525 23098 0 -1 4202496 41239 0 0 0 7723 74 0 0 18 0 8 0 146781645 139501568 12376 4089446400 134512640 146736320 4288421280 18446744073709551615 144580332 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0
/proc/23528/statm: 34058 12376 1082 2985 0 30600 0
[pid=23528/tid=23623] ppid=23525 vsize=136232 CPUtime=1.37
/proc/23528/task/23623/stat : 23623 (pb_cplex) R 23525 23525 23098 0 -1 4202560 184 0 0 0 137 0 0 0 16 0 8 0 146783818 139501568 12376 4089446400 134512640 146736320 4288421280 18446744073709551615 136007274 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=23528/tid=23624] ppid=23525 vsize=136232 CPUtime=1.65
/proc/23528/task/23624/stat : 23624 (pb_cplex) R 23525 23525 23098 0 -1 4202560 55 0 0 0 165 0 0 0 16 0 8 0 146783818 139501568 12376 4089446400 134512640 146736320 4288421280 18446744073709551615 136007602 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=23528/tid=23625] ppid=23525 vsize=136232 CPUtime=1.03
/proc/23528/task/23625/stat : 23625 (pb_cplex) R 23525 23525 23098 0 -1 4202560 565 0 0 0 103 0 0 0 16 0 8 0 146783818 139501568 12376 4089446400 134512640 146736320 4288421280 18446744073709551615 139023034 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=23528/tid=23626] ppid=23525 vsize=136232 CPUtime=0.5
/proc/23528/task/23626/stat : 23626 (pb_cplex) R 23525 23525 23098 0 -1 4202560 175 0 0 0 50 0 0 0 16 0 8 0 146783818 139501568 12376 4089446400 134512640 146736320 4288421280 18446744073709551615 139023034 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=23528/tid=23627] ppid=23525 vsize=136232 CPUtime=1.65
/proc/23528/task/23627/stat : 23627 (pb_cplex) R 23525 23525 23098 0 -1 4202560 344 0 0 0 165 0 0 0 16 0 8 0 146783818 139501568 12376 4089446400 134512640 146736320 4288421280 18446744073709551615 139042910 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=23528/tid=23628] ppid=23525 vsize=136232 CPUtime=1.69
/proc/23528/task/23628/stat : 23628 (pb_cplex) R 23525 23525 23098 0 -1 4202560 143 0 0 0 169 0 0 0 15 0 8 0 146783818 139501568 12376 4089446400 134512640 146736320 4288421280 18446744073709551615 136388624 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=23528/tid=23629] ppid=23525 vsize=136232 CPUtime=1.84
/proc/23528/task/23629/stat : 23629 (pb_cplex) R 23525 23525 23098 0 -1 4202560 731 0 0 0 184 0 0 0 16 0 8 0 146783818 139501568 12376 4089446400 134512640 146736320 4288421280 18446744073709551615 139044542 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 77.97
Current children cumulated vsize (KiB) 144932

[startup+51.1019 s]
/proc/loadavg: 5.09 1.41 0.48 9/195 23644
/proc/meminfo: memFree=30698740/32950928 swapFree=67111528/67111528
[pid=23525] ppid=23521 vsize=8700 CPUtime=0
/proc/23525/stat : 23525 (run) S 23521 23525 23098 0 -1 4202496 365 0 0 0 0 0 0 0 18 0 1 0 146781645 8908800 249 4089446400 4194304 4922060 140735208630000 18446744073709551615 254500510853 0 65536 4100 65538 18446744071562232103 0 0 17 0 0 0 0
/proc/23525/statm: 2175 249 208 178 0 68 0
[pid=23528] ppid=23525 vsize=136232 CPUtime=176.23
/proc/23528/stat : 23528 (pb_cplex) S 23525 23525 23098 0 -1 4202496 53125 0 0 0 17470 153 0 0 16 0 8 0 146781645 139501568 12474 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/23528/statm: 34058 12474 1082 2985 0 30600 0
[pid=23528/tid=23623] ppid=23525 vsize=136232 CPUtime=13.54
/proc/23528/task/23623/stat : 23623 (pb_cplex) R 23525 23525 23098 0 -1 4202560 2687 0 0 0 1344 10 0 0 15 0 8 0 146783818 139501568 12474 4089446400 134512640 146736320 4288421280 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=23528/tid=23624] ppid=23525 vsize=136232 CPUtime=14.08
/proc/23528/task/23624/stat : 23624 (pb_cplex) S 23525 23525 23098 0 -1 4202560 1659 0 0 0 1398 10 0 0 15 0 8 0 146783818 139501568 12474 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 3 0 0 0
[pid=23528/tid=23625] ppid=23525 vsize=136232 CPUtime=13.22
/proc/23528/task/23625/stat : 23625 (pb_cplex) S 23525 23525 23098 0 -1 4202560 1293 0 0 0 1313 9 0 0 15 0 8 0 146783818 139501568 12474 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 0 0 0 0
[pid=23528/tid=23626] ppid=23525 vsize=136232 CPUtime=12.67
/proc/23528/task/23626/stat : 23626 (pb_cplex) S 23525 23525 23098 0 -1 4202560 1297 0 0 0 1257 10 0 0 15 0 8 0 146783818 139501568 12474 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 2 0 0 0
[pid=23528/tid=23627] ppid=23525 vsize=136232 CPUtime=13.98
/proc/23528/task/23627/stat : 23627 (pb_cplex) S 23525 23525 23098 0 -1 4202560 1530 0 0 0 1389 9 0 0 15 0 8 0 146783818 139501568 12474 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 7 0 0 0
[pid=23528/tid=23628] ppid=23525 vsize=136232 CPUtime=13.61
/proc/23528/task/23628/stat : 23628 (pb_cplex) S 23525 23525 23098 0 -1 4202560 1007 0 0 0 1353 8 0 0 15 0 8 0 146783818 139501568 12474 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 6 0 0 0
[pid=23528/tid=23629] ppid=23525 vsize=136232 CPUtime=13.8
/proc/23528/task/23629/stat : 23629 (pb_cplex) R 23525 23525 23098 0 -1 4202560 3271 0 0 0 1369 11 0 0 15 0 8 0 146783818 139501568 12474 4089446400 134512640 146736320 4288421280 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 176.23
Current children cumulated vsize (KiB) 144932

[startup+102.307 s]
/proc/loadavg: 7.46 2.64 0.95 9/195 23645
/proc/meminfo: memFree=30697924/32950928 swapFree=67111528/67111528

################
# More data... #
################

/proc/23528/task/23627/stat : 23627 (pb_cplex) R 23525 23525 23098 0 -1 4202560 9230 0 0 0 9563 76 0 0 16 0 8 0 146783818 145793024 13722 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=23528/tid=23628] ppid=23525 vsize=142376 CPUtime=95.97
/proc/23528/task/23628/stat : 23628 (pb_cplex) S 23525 23525 23098 0 -1 4202560 7011 0 0 0 9521 76 0 0 15 0 8 0 146783818 145793024 13722 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 5 0 0 0
[pid=23528/tid=23629] ppid=23525 vsize=142376 CPUtime=96.37
/proc/23528/task/23629/stat : 23629 (pb_cplex) S 23525 23525 23098 0 -1 4202560 10011 0 0 0 9556 81 0 0 15 0 8 0 146783818 145793024 13722 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 839.97
Current children cumulated vsize (KiB) 151076

[startup+282.301 s]
/proc/loadavg: 8.73 5.67 2.43 9/195 23721
/proc/meminfo: memFree=30698332/32950928 swapFree=67111528/67111528
[pid=23525] ppid=23521 vsize=8700 CPUtime=0
/proc/23525/stat : 23525 (run) S 23521 23525 23098 0 -1 4202496 365 0 0 0 0 0 0 0 18 0 1 0 146781645 8908800 249 4089446400 4194304 4922060 140735208630000 18446744073709551615 254500510853 0 65536 4100 65538 18446744071562232103 0 0 17 0 0 0 0
/proc/23525/statm: 2175 249 208 178 0 68 0
[pid=23528] ppid=23525 vsize=143400 CPUtime=1084.17
/proc/23528/stat : 23528 (pb_cplex) S 23525 23525 23098 0 -1 4202496 114252 0 0 0 107492 925 0 0 15 0 8 0 146781645 146841600 13766 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0
/proc/23528/statm: 35850 13766 1082 2985 0 32392 0
[pid=23528/tid=23623] ppid=23525 vsize=143400 CPUtime=126.18
/proc/23528/task/23623/stat : 23623 (pb_cplex) S 23525 23525 23098 0 -1 4202560 9225 0 0 0 12510 108 0 0 15 0 8 0 146783818 146841600 13766 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 4 0 0 0
[pid=23528/tid=23624] ppid=23525 vsize=143400 CPUtime=125.84
/proc/23528/task/23624/stat : 23624 (pb_cplex) S 23525 23525 23098 0 -1 4202560 10142 0 0 0 12478 106 0 0 15 0 8 0 146783818 146841600 13766 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 3 0 0 0
[pid=23528/tid=23625] ppid=23525 vsize=143400 CPUtime=126.28
/proc/23528/task/23625/stat : 23625 (pb_cplex) S 23525 23525 23098 0 -1 4202560 8183 0 0 0 12524 104 0 0 16 0 8 0 146783818 146841600 13766 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 5 0 0 0
[pid=23528/tid=23626] ppid=23525 vsize=143400 CPUtime=124.73
/proc/23528/task/23626/stat : 23626 (pb_cplex) S 23525 23525 23098 0 -1 4202560 7704 0 0 0 12365 108 0 0 15 0 8 0 146783818 146841600 13766 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 6 0 0 0
[pid=23528/tid=23627] ppid=23525 vsize=143400 CPUtime=126.67
/proc/23528/task/23627/stat : 23627 (pb_cplex) S 23525 23525 23098 0 -1 4202560 10897 0 0 0 12564 103 0 0 15 0 8 0 146783818 146841600 13766 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 7 0 0 0
[pid=23528/tid=23628] ppid=23525 vsize=143400 CPUtime=126.13
/proc/23528/task/23628/stat : 23628 (pb_cplex) S 23525 23525 23098 0 -1 4202560 8829 0 0 0 12509 104 0 0 15 0 8 0 146783818 146841600 13766 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 0 0 0 0
[pid=23528/tid=23629] ppid=23525 vsize=143400 CPUtime=126.86
/proc/23528/task/23629/stat : 23629 (pb_cplex) R 23525 23525 23098 0 -1 4202560 11045 0 0 0 12577 109 0 0 15 0 8 0 146783818 146841600 13766 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 1084.17
Current children cumulated vsize (KiB) 152100

[startup+342.307 s]
/proc/loadavg: 8.82 6.28 2.84 8/195 23736
/proc/meminfo: memFree=30699636/32950928 swapFree=67111528/67111528
[pid=23525] ppid=23521 vsize=8700 CPUtime=0
/proc/23525/stat : 23525 (run) S 23521 23525 23098 0 -1 4202496 365 0 0 0 0 0 0 0 18 0 1 0 146781645 8908800 249 4089446400 4194304 4922060 140735208630000 18446744073709551615 254500510853 0 65536 4100 65538 18446744071562232103 0 0 17 0 0 0 0
/proc/23525/statm: 2175 249 208 178 0 68 0
[pid=23528] ppid=23525 vsize=142376 CPUtime=1296.62
/proc/23528/stat : 23528 (pb_cplex) S 23525 23525 23098 0 -1 4202496 118964 0 0 0 128558 1104 0 0 16 0 8 0 146781645 145793024 13776 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/23528/statm: 35594 13776 1082 2985 0 32136 0
[pid=23528/tid=23623] ppid=23525 vsize=142376 CPUtime=152.26
/proc/23528/task/23623/stat : 23623 (pb_cplex) S 23525 23525 23098 0 -1 4202560 9438 0 0 0 15096 130 0 0 15 0 8 0 146783818 145793024 13776 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 0 0 0 0
[pid=23528/tid=23624] ppid=23525 vsize=142376 CPUtime=151.97
/proc/23528/task/23624/stat : 23624 (pb_cplex) S 23525 23525 23098 0 -1 4202560 11106 0 0 0 15068 129 0 0 15 0 8 0 146783818 145793024 13776 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 0 0 0 0
[pid=23528/tid=23625] ppid=23525 vsize=142376 CPUtime=152.74
/proc/23528/task/23625/stat : 23625 (pb_cplex) S 23525 23525 23098 0 -1 4202560 8770 0 0 0 15148 126 0 0 15 0 8 0 146783818 145793024 13776 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 0 0 0 0
[pid=23528/tid=23626] ppid=23525 vsize=142376 CPUtime=150.87
/proc/23528/task/23626/stat : 23626 (pb_cplex) S 23525 23525 23098 0 -1 4202560 8124 0 0 0 14956 131 0 0 15 0 8 0 146783818 145793024 13776 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 0 0 0 0
[pid=23528/tid=23627] ppid=23525 vsize=142376 CPUtime=152.83
/proc/23528/task/23627/stat : 23627 (pb_cplex) R 23525 23525 23098 0 -1 4202560 11387 0 0 0 15157 126 0 0 16 0 8 0 146783818 145793024 13776 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=23528/tid=23628] ppid=23525 vsize=142376 CPUtime=152.45
/proc/23528/task/23628/stat : 23628 (pb_cplex) R 23525 23525 23098 0 -1 4202560 9974 0 0 0 15119 126 0 0 15 0 8 0 146783818 145793024 13776 4089446400 134512640 146736320 4288421280 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=23528/tid=23629] ppid=23525 vsize=142376 CPUtime=153.28
/proc/23528/task/23629/stat : 23629 (pb_cplex) R 23525 23525 23098 0 -1 4202560 11381 0 0 0 15193 135 0 0 15 0 8 0 146783818 145793024 13776 4089446400 134512640 146736320 4288421280 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 1296.62
Current children cumulated vsize (KiB) 151076

[startup+402.3 s]
/proc/loadavg: 9.06 6.83 3.25 10/195 23737
/proc/meminfo: memFree=30699348/32950928 swapFree=67111528/67111528
[pid=23525] ppid=23521 vsize=8700 CPUtime=0
/proc/23525/stat : 23525 (run) S 23521 23525 23098 0 -1 4202496 365 0 0 0 0 0 0 0 18 0 1 0 146781645 8908800 249 4089446400 4194304 4922060 140735208630000 18446744073709551615 254500510853 0 65536 4100 65538 18446744071562232103 0 0 17 0 0 0 0
/proc/23525/statm: 2175 249 208 178 0 68 0
[pid=23528] ppid=23525 vsize=140996 CPUtime=1527.96
/proc/23528/stat : 23528 (pb_cplex) R 23525 23525 23098 0 -1 4202496 126419 0 0 0 151496 1300 0 0 16 0 8 0 146781645 144379904 13459 4089446400 134512640 146736320 4288421280 18446744073709551615 136000480 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0
/proc/23528/statm: 35249 13459 1100 2985 0 31791 0
[pid=23528/tid=23623] ppid=23525 vsize=140996 CPUtime=181
/proc/23528/task/23623/stat : 23623 (pb_cplex) R 23525 23525 23098 0 -1 4202560 10292 0 0 0 17948 152 0 0 15 0 8 0 146783818 144379904 13459 4089446400 134512640 146736320 4288421280 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=23528/tid=23624] ppid=23525 vsize=140996 CPUtime=180.73
/proc/23528/task/23624/stat : 23624 (pb_cplex) R 23525 23525 23098 0 -1 4202560 11764 0 0 0 17919 154 0 0 15 0 8 0 146783818 144379904 13459 4089446400 134512640 146736320 4288421280 18446744073709551615 136017481 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=23528/tid=23625] ppid=23525 vsize=140996 CPUtime=181.63
/proc/23528/task/23625/stat : 23625 (pb_cplex) R 23525 23525 23098 0 -1 4202560 9942 0 0 0 18015 148 0 0 15 0 8 0 146783818 144379904 13459 4089446400 134512640 146736320 4288421280 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=23528/tid=23626] ppid=23525 vsize=140996 CPUtime=179.55
/proc/23528/task/23626/stat : 23626 (pb_cplex) R 23525 23525 23098 0 -1 4202560 9284 0 0 0 17800 155 0 0 15 0 8 0 146783818 144379904 13459 4089446400 134512640 146736320 4288421280 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=23528/tid=23627] ppid=23525 vsize=140996 CPUtime=181.3
/proc/23528/task/23627/stat : 23627 (pb_cplex) R 23525 23525 23098 0 -1 4202560 12127 0 0 0 17978 152 0 0 15 0 8 0 146783818 144379904 13459 4089446400 134512640 146736320 4288421280 18446744073709551615 136403112 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=23528/tid=23628] ppid=23525 vsize=140996 CPUtime=181.06
/proc/23528/task/23628/stat : 23628 (pb_cplex) R 23525 23525 23098 0 -1 4202560 10589 0 0 0 17955 151 0 0 15 0 8 0 146783818 144379904 13459 4089446400 134512640 146736320 4288421280 18446744073709551615 135526881 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=23528/tid=23629] ppid=23525 vsize=140996 CPUtime=181.84
/proc/23528/task/23629/stat : 23629 (pb_cplex) R 23525 23525 23098 0 -1 4202560 12635 0 0 0 18026 158 0 0 15 0 8 0 146783818 144379904 13459 4089446400 134512640 146736320 4288421280 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 1527.96
Current children cumulated vsize (KiB) 149696

[startup+462.361 s]
/proc/loadavg: 8.73 7.17 3.59 8/195 23739
/proc/meminfo: memFree=30698312/32950928 swapFree=67111528/67111528
[pid=23525] ppid=23521 vsize=8700 CPUtime=0
/proc/23525/stat : 23525 (run) S 23521 23525 23098 0 -1 4202496 365 0 0 0 0 0 0 0 18 0 1 0 146781645 8908800 249 4089446400 4194304 4922060 140735208630000 18446744073709551615 254500510853 0 65536 4100 65538 18446744071562232103 0 0 17 0 0 0 0
/proc/23525/statm: 2175 249 208 178 0 68 0
[pid=23528] ppid=23525 vsize=141140 CPUtime=1770.28
/proc/23528/stat : 23528 (pb_cplex) S 23525 23525 23098 0 -1 4202496 137185 0 0 0 175510 1518 0 0 15 0 8 0 146781645 144527360 13460 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0
/proc/23528/statm: 35285 13460 1100 2985 0 31827 0
[pid=23528/tid=23623] ppid=23525 vsize=141140 CPUtime=211.17
/proc/23528/task/23623/stat : 23623 (pb_cplex) R 23525 23525 23098 0 -1 4202560 11254 0 0 0 20940 177 0 0 15 0 8 0 146783818 144527360 13460 4089446400 134512640 146736320 4288421280 18446744073709551615 136017491 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=23528/tid=23624] ppid=23525 vsize=141140 CPUtime=210.93
/proc/23528/task/23624/stat : 23624 (pb_cplex) R 23525 23525 23098 0 -1 4202560 12843 0 0 0 20912 181 0 0 15 0 8 0 146783818 144527360 13460 4089446400 134512640 146736320 4288421280 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=23528/tid=23625] ppid=23525 vsize=141140 CPUtime=211.76
/proc/23528/task/23625/stat : 23625 (pb_cplex) R 23525 23525 23098 0 -1 4202560 11871 0 0 0 21001 175 0 0 15 0 8 0 146783818 144527360 13460 4089446400 134512640 146736320 4288421280 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=23528/tid=23626] ppid=23525 vsize=141140 CPUtime=209.4
/proc/23528/task/23626/stat : 23626 (pb_cplex) R 23525 23525 23098 0 -1 4202560 10893 0 0 0 20756 184 0 0 15 0 8 0 146783818 144527360 13460 4089446400 134512640 146736320 4288421280 18446744073709551615 136403116 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=23528/tid=23627] ppid=23525 vsize=141140 CPUtime=211.55
/proc/23528/task/23627/stat : 23627 (pb_cplex) R 23525 23525 23098 0 -1 4202560 12957 0 0 0 20975 180 0 0 15 0 8 0 146783818 144527360 13460 4089446400 134512640 146736320 4288421280 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=23528/tid=23628] ppid=23525 vsize=141140 CPUtime=211.32
/proc/23528/task/23628/stat : 23628 (pb_cplex) R 23525 23525 23098 0 -1 4202560 11224 0 0 0 20951 181 0 0 15 0 8 0 146783818 144527360 13460 4089446400 134512640 146736320 4288421280 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=23528/tid=23629] ppid=23525 vsize=141140 CPUtime=211.7
/proc/23528/task/23629/stat : 23629 (pb_cplex) R 23525 23525 23098 0 -1 4202560 13988 0 0 0 20985 185 0 0 15 0 8 0 146783818 144527360 13460 4089446400 134512640 146736320 4288421280 18446744073709551615 136000338 0 2 4096 16387 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 1770.28
Current children cumulated vsize (KiB) 149840



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+469.723 s]
/proc/loadavg: 8.76 7.23 3.65 11/195 23739
/proc/meminfo: memFree=30697068/32950928 swapFree=67111528/67111528
[pid=23525] ppid=23521 vsize=8700 CPUtime=0
/proc/23525/stat : 23525 (run) S 23521 23525 23098 0 -1 4202496 365 0 0 0 0 0 0 0 18 0 1 0 146781645 8908800 249 4089446400 4194304 4922060 140735208630000 18446744073709551615 254500510853 0 65536 4100 65538 18446744071562232103 0 0 17 0 0 0 0
/proc/23525/statm: 2175 249 208 178 0 68 0
[pid=23528] ppid=23525 vsize=141140 CPUtime=1800.06
/proc/23528/stat : 23528 (pb_cplex) R 23525 23525 23098 0 -1 4202496 138820 0 0 0 178461 1545 0 0 16 0 8 0 146781645 144527360 13460 4089446400 134512640 146736320 4288421280 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0
/proc/23528/statm: 35285 13460 1100 2985 0 31827 0
[pid=23528/tid=23623] ppid=23525 vsize=141140 CPUtime=214.78
/proc/23528/task/23623/stat : 23623 (pb_cplex) R 23525 23525 23098 0 -1 4202560 11257 0 0 0 21297 181 0 0 16 0 8 0 146783818 144527360 13460 4089446400 134512640 146736320 4288421280 18446744073709551615 139043566 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=23528/tid=23624] ppid=23525 vsize=141140 CPUtime=214.67
/proc/23528/task/23624/stat : 23624 (pb_cplex) R 23525 23525 23098 0 -1 4202560 12969 0 0 0 21281 186 0 0 15 0 8 0 146783818 144527360 13460 4089446400 134512640 146736320 4288421280 18446744073709551615 136017748 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=23528/tid=23625] ppid=23525 vsize=141140 CPUtime=215.46
/proc/23528/task/23625/stat : 23625 (pb_cplex) R 23525 23525 23098 0 -1 4202560 12079 0 0 0 21368 178 0 0 15 0 8 0 146783818 144527360 13460 4089446400 134512640 146736320 4288421280 18446744073709551615 136389777 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=23528/tid=23626] ppid=23525 vsize=141140 CPUtime=213.01
/proc/23528/task/23626/stat : 23626 (pb_cplex) R 23525 23525 23098 0 -1 4202560 11409 0 0 0 21115 186 0 0 15 0 8 0 146783818 144527360 13460 4089446400 134512640 146736320 4288421280 18446744073709551615 136475822 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=23528/tid=23627] ppid=23525 vsize=141140 CPUtime=215.3
/proc/23528/task/23627/stat : 23627 (pb_cplex) R 23525 23525 23098 0 -1 4202560 13003 0 0 0 21348 182 0 0 15 0 8 0 146783818 144527360 13460 4089446400 134512640 146736320 4288421280 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=23528/tid=23628] ppid=23525 vsize=141140 CPUtime=215.09
/proc/23528/task/23628/stat : 23628 (pb_cplex) R 23525 23525 23098 0 -1 4202560 11639 0 0 0 21324 185 0 0 15 0 8 0 146783818 144527360 13460 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=23528/tid=23629] ppid=23525 vsize=141140 CPUtime=215.43
/proc/23528/task/23629/stat : 23629 (pb_cplex) S 23525 23525 23098 0 -1 4202560 14222 0 0 0 21354 189 0 0 16 0 8 0 146783818 144527360 13460 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 1800.06
Current children cumulated vsize (KiB) 149840

Sending SIGTERM to process tree (bottom up)


[startup+t ended. D469.723 s]
/proc/loadavg: 8.76 7.23 3.65 11/195 23739
/proc/meminfo: memFree=30697068/32950928 swapFree=67111528/67111528
[pid=23525] ppid=23521 vsize=8700 CPUtime=0
/proc/23525/stat : 23525 (run) S 23521 23525 23098 0 -1 4202496 365 0 0 0 0 0 0 0 18 0 1 0 146781645 8908800 249 4089446400 4194304 4922060 140735208630000 18446744073709551615 254500510853 0 65536 4100 65538 18446744071562232103 0 0 17 0 0 0 0
/proc/23525/statm: 2175 249 208 178 0 68 0
[pid=23528] ppid=23525 vsize=141140 CPUtime=1800.06
/proc/23528/stat : 23528 (pb_cplex) R 23525 23525 23098 0 -1 4202496 138820 0 0 0 178461 1545 0 0 16 0 8 0 146781645 144527360 13460 4089446400 134512640 146736320 4288421280 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0
/proc/23528/statm: 35285 13460 1100 2985 0 31827 0
[pid=23528/tid=23623] ppid=23525 vsize=141140 CPUtime=214.78
/proc/23528/task/23623/stat : 23623 (pb_cplex) R 23525 23525 23098 0 -1 4202560 11257 0 0 0 21297 181 0 0 16 0 8 0 146783818 144527360 13460 4089446400 134512640 146736320 4288421280 18446744073709551615 139043566 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=23528/tid=23624] ppid=23525 vsize=141140 CPUtime=214.67
/proc/23528/task/23624/stat : 23624 (pb_cplex) R 23525 23525 23098 0 -1 4202560 12969 0 0 0 21281 186 0 0 15 0 8 0 146783818 144527360 13460 4089446400 134512640 146736320 4288421280 18446744073709551615 136017748 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=23528/tid=23625] ppid=23525 vsize=141140 CPUtime=215.46
/proc/23528/task/23625/stat : 23625 (pb_cplex) R 23525 23525 23098 0 -1 4202560 12079 0 0 0 21368 178 0 0 15 0 8 0 146783818 144527360 13460 4089446400 134512640 146736320 4288421280 18446744073709551615 136389777 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=23528/tid=23626] ppid=23525 vsize=141140 CPUtime=213.01
/proc/23528/task/23626/stat : 23626 (pb_cplex) R 23525 23525 23098 0 -1 4202560 11409 0 0 0 21115 186 0 0 15 0 8 0 146783818 144527360 13460 4089446400 134512640 146736320 4288421280 18446744073709551615 136475822 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=23528/tid=23627] ppid=23525 vsize=141140 CPUtime=215.3
/proc/23528/task/23627/stat : 23627 (pb_cplex) R 23525 23525 23098 0 -1 4202560 13003 0 0 0 21348 182 0 0 15 0 8 0 146783818 144527360 13460 4089446400 134512640 146736320 4288421280 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=23528/tid=23628] ppid=23525 vsize=141140 CPUtime=215.09
/proc/23528/task/23628/stat : 23628 (pb_cplex) R 23525 23525 23098 0 -1 4202560 11639 0 0 0 21324 185 0 0 15 0 8 0 146783818 144527360 13460 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=23528/tid=23629] ppid=23525 vsize=141140 CPUtime=215.43
/proc/23528/task/23629/stat : 23629 (pb_cplex) S 23525 23525 23098 0 -1 4202560 14222 0 0 0 21354 189 0 0 16 0 8 0 146783818 144527360 13460 4089446400 134512640 146736320 4288421280 18446744073709551615 144580073 0 2 4096 16387 18446604453967458624 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 1800.06
Current children cumulated vsize (KiB) 149840

Child status: 0
Real time (s): 469.742
CPU time (s): 1800.14
CPU user time (s): 1784.67
CPU system time (s): 15.4646
CPU usage (%): 383.218
Max. virtual memory (cumulated for all children) (KiB): 157220

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1784.67
system time used= 15.4646
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 139193
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 313002
involuntary context switches= 28151

runsolver used 0.907861 second user time and 1.85972 second system time

The end

Launcher Data 

Begin job on node149 at 2010-07-02 09:14:27
IDJOB=2697903
IDBENCH=78939
IDSOLVER=1210
FILE ID=node149/2697903-1278054867
PBS_JOBID= 11198884
Free space on /tmp= 74024 MiB

SOLVER NAME= pb_cplex 2010-07-01
BENCH NAME= PB10/normalized-PB10/PARTIAL-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/tsp/normalized-t3002.11tsp11.1900573137--soft-33-100-0.wbo
COMMAND LINE= run BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-2697903-1278054867/watcher-2697903-1278054867 -o /tmp/evaluation-result-2697903-1278054867/solver-2697903-1278054867 -C 1800 -W 2000 -M 3800  run HOME/instance-2697903-1278054867.wbo

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 3800 MiB
MAX NB THREAD= 0

MD5SUM BENCH= 34c2ccfe3c5c18b062797c1b5794b2d4
RANDOM SEED=1097187445

node149.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      30804552 kB
Buffers:        225436 kB
Cached:        1702824 kB
SwapCached:          0 kB
Active:         686536 kB
Inactive:      1304612 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      30804552 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           33828 kB
Writeback:           0 kB
AnonPages:       62356 kB
Mapped:          14416 kB
Slab:            91248 kB
PageTables:       4504 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   184196 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73988 MiB
End job on node149 at 2010-07-02 09:22:17