Trace number 2697897

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
pb_cplex 2010-07-01? (TO) 1800.49 488.737

General information on the benchmark

Name/PARTIAL-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/tsp/
normalized-t3002.11tsp11.1900563250--soft-66-100-0.wbo
MD5SUM48850acf7a795604147e976fcd80db7d
Bench CategoryPARTIAL-SMALLINT-LIN (both soft and hard constraints, small integers, linear constraints)
Best result obtained on this benchmarkMOPT
Best cost obtained on this benchmark1
Best CPU time to get the best result obtained on this benchmark336.106
Max-Satisfiable
Max-(Un)Satisfiability was proved
Best value of the cost
Optimality of the best cost was proved
Number of variables231
Total number of constraints2707
Number of soft constraints892
Number of constraints which are clauses2684
Number of constraints which are cardinality constraints (but not clauses)22
Number of constraints which are nor clauses,nor cardinality constraints1
Minimum length of a constraint2
Maximum length of a constraint110
Top cost 45060
Min constraint cost 1
Max constraint cost 100
Sum of constraints costs 45059
Biggest number in a constraint 25
Number of bits of the biggest number in a constraint 5
Biggest sum of numbers in a constraint 665
Number of bits of the biggest sum of numbers10
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c pb_cplex 1.0 - 20100701
0.00/0.00	c Parsing wbo file format. File HOME/instance-2697897-1278054867.wbo.
0.00/0.00	c Instance file HOME/instance-2697897-1278054867.wbo
0.00/0.00	c File size is 87844 bytes.
0.00/0.18	c Highest Coefficient sum: 665
0.00/0.18	c  IBM ILOG License Manager: "IBM ILOG Optimization Suite for Academic Initiative" is accessing CPLEX 12 with option(s): "e m b q ".
0.00/0.19	c ============================[ Problem Statistics ]=============================
0.00/0.19	c |                                                                             |
0.00/0.19	c |  Parsing time:          0.17         s                                      |
0.00/0.19	c |  Number Variables:      1123                                                |
0.00/0.19	c |  Number Constraints:    2708                                                |
0.00/0.19	c ===============================================================================
1800.47/488.73	c Remaining time 1799.83 sec.
1800.47/488.73	c 
1800.47/488.73	c *** INTERRUPTED ***
1800.47/488.73	s UNKNOWN
1800.47/488.73	c Total CPU time              : 1784.29 s
1800.47/488.73	c Memory used           : 150.12 MB
1800.47/488.73	c CPU time              : 1784.29 s
1800.47/488.73	c 
1800.47/488.73	c *** INTERRUPTED ***

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver version 3.2.9 (svn:492) (c) roussel@cril.univ-artois.fr

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-2697897-1278054867/watcher-2697897-1278054867 -o /tmp/evaluation-result-2697897-1278054867/solver-2697897-1278054867 -C 1800 -W 2000 -M 3800 run HOME/instance-2697897-1278054867.wbo 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 3891200 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 3942400 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.14 0.03 0.01 4/179 25990
/proc/meminfo: memFree=30882532/32951124 swapFree=67111528/67111528
[pid=25990] ppid=25988 vsize=8700 CPUtime=0
/proc/25990/stat : 25990 (run) S 25988 25990 25677 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782213 8908800 249 4089446400 4194304 4922060 140735399037328 18446744073709551615 212687494277 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25990/statm: 2175 249 208 178 0 68 0

[startup+0.0967599 s]
/proc/loadavg: 0.14 0.03 0.01 4/179 25990
/proc/meminfo: memFree=30882532/32951124 swapFree=67111528/67111528
[pid=25990] ppid=25988 vsize=8700 CPUtime=0
/proc/25990/stat : 25990 (run) S 25988 25990 25677 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782213 8908800 249 4089446400 4194304 4922060 140735399037328 18446744073709551615 212687494277 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25990/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.10107 s]
/proc/loadavg: 0.14 0.03 0.01 4/179 25990
/proc/meminfo: memFree=30882532/32951124 swapFree=67111528/67111528
[pid=25990] ppid=25988 vsize=8700 CPUtime=0
/proc/25990/stat : 25990 (run) S 25988 25990 25677 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782213 8908800 249 4089446400 4194304 4922060 140735399037328 18446744073709551615 212687494277 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25990/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.301044 s]
/proc/loadavg: 0.14 0.03 0.01 4/179 25990
/proc/meminfo: memFree=30882532/32951124 swapFree=67111528/67111528
[pid=25990] ppid=25988 vsize=8700 CPUtime=0
/proc/25990/stat : 25990 (run) S 25988 25990 25677 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782213 8908800 249 4089446400 4194304 4922060 140735399037328 18446744073709551615 212687494277 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25990/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.700909 s]
/proc/loadavg: 0.14 0.03 0.01 4/179 25990
/proc/meminfo: memFree=30882532/32951124 swapFree=67111528/67111528
[pid=25990] ppid=25988 vsize=8700 CPUtime=0
/proc/25990/stat : 25990 (run) S 25988 25990 25677 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782213 8908800 249 4089446400 4194304 4922060 140735399037328 18446744073709551615 212687494277 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25990/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+1.50086 s]
/proc/loadavg: 0.14 0.03 0.01 10/188 26027
/proc/meminfo: memFree=30831908/32951124 swapFree=67111528/67111528
[pid=25990] ppid=25988 vsize=8700 CPUtime=0
/proc/25990/stat : 25990 (run) S 25988 25990 25677 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782213 8908800 249 4089446400 4194304 4922060 140735399037328 18446744073709551615 212687494277 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25990/statm: 2175 249 208 178 0 68 0
[pid=25991] ppid=25990 vsize=88776 CPUtime=2.79
/proc/25991/stat : 25991 (pb_cplex) R 25990 25990 25677 0 -1 4202496 15161 0 0 0 273 6 0 0 16 0 1 0 146782213 90906624 8068 4089446400 134512640 146736320 4291930848 18446744073709551615 136462525 0 0 4096 16387 0 0 0 17 5 0 0 0
/proc/25991/statm: 22194 8068 1073 2985 0 18736 0
Current children cumulated CPU time (s) 2.79
Current children cumulated vsize (KiB) 97476

[startup+3.10047 s]
/proc/loadavg: 0.14 0.03 0.01 3/195 26069
/proc/meminfo: memFree=30827760/32951124 swapFree=67111528/67111528
[pid=25990] ppid=25988 vsize=8700 CPUtime=0
/proc/25990/stat : 25990 (run) S 25988 25990 25677 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782213 8908800 249 4089446400 4194304 4922060 140735399037328 18446744073709551615 212687494277 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25990/statm: 2175 249 208 178 0 68 0
[pid=25991] ppid=25990 vsize=132872 CPUtime=6.52
/proc/25991/stat : 25991 (pb_cplex) R 25990 25990 25677 0 -1 4202496 23503 0 0 0 644 8 0 0 18 0 8 0 146782213 136060928 10778 4089446400 134512640 146736320 4291930848 18446744073709551615 135854592 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0
/proc/25991/statm: 33218 10778 1081 2985 0 29760 0
[pid=25991/tid=26042] ppid=25990 vsize=132872 CPUtime=0.13
/proc/25991/task/26042/stat : 26042 (pb_cplex) R 25990 25990 25677 0 -1 4202560 3 0 0 0 13 0 0 0 18 0 8 0 146782381 136060928 10778 4089446400 134512640 146736320 4291930848 18446744073709551615 139044580 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=25991/tid=26043] ppid=25990 vsize=132872 CPUtime=0.57
/proc/25991/task/26043/stat : 26043 (pb_cplex) S 25990 25990 25677 0 -1 4202560 4 0 0 0 57 0 0 0 18 0 8 0 146782381 136060928 10778 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 7 0 0 0
[pid=25991/tid=26044] ppid=25990 vsize=132872 CPUtime=0.4
/proc/25991/task/26044/stat : 26044 (pb_cplex) S 25990 25990 25677 0 -1 4202560 465 0 0 0 40 0 0 0 18 0 8 0 146782381 136060928 10778 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 3 0 0 0
[pid=25991/tid=26045] ppid=25990 vsize=132872 CPUtime=0.21
/proc/25991/task/26045/stat : 26045 (pb_cplex) S 25990 25990 25677 0 -1 4202560 706 0 0 0 21 0 0 0 16 0 8 0 146782381 136060928 10778 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 1 0 0 0
[pid=25991/tid=26046] ppid=25990 vsize=132872 CPUtime=0.24
/proc/25991/task/26046/stat : 26046 (pb_cplex) R 25990 25990 25677 0 -1 4202560 1634 0 0 0 24 0 0 0 18 0 8 0 146782381 136060928 10778 4089446400 134512640 146736320 4291930848 18446744073709551615 135850332 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=25991/tid=26047] ppid=25990 vsize=132872 CPUtime=0.25
/proc/25991/task/26047/stat : 26047 (pb_cplex) R 25990 25990 25677 0 -1 4202560 439 0 0 0 25 0 0 0 18 0 8 0 146782381 136060928 10778 4089446400 134512640 146736320 4291930848 18446744073709551615 139044505 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=25991/tid=26048] ppid=25990 vsize=132872 CPUtime=0.34
/proc/25991/task/26048/stat : 26048 (pb_cplex) R 25990 25990 25677 0 -1 4202560 642 0 0 0 34 0 0 0 18 0 8 0 146782381 136060928 10778 4089446400 134512640 146736320 4291930848 18446744073709551615 136008176 0 2 4096 16387 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 6.52
Current children cumulated vsize (KiB) 141572

[startup+6.30089 s]
/proc/loadavg: 1.25 0.26 0.08 11/195 26069
/proc/meminfo: memFree=30800612/32951124 swapFree=67111528/67111528
[pid=25990] ppid=25988 vsize=8700 CPUtime=0
/proc/25990/stat : 25990 (run) S 25988 25990 25677 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782213 8908800 249 4089446400 4194304 4922060 140735399037328 18446744073709551615 212687494277 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25990/statm: 2175 249 208 178 0 68 0
[pid=25991] ppid=25990 vsize=131848 CPUtime=15.92
/proc/25991/stat : 25991 (pb_cplex) R 25990 25990 25677 0 -1 4202496 27169 0 0 0 1573 19 0 0 16 0 8 0 146782213 135012352 10706 4089446400 134512640 146736320 4291930848 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0
/proc/25991/statm: 32962 10706 1080 2985 0 29504 0
[pid=25991/tid=26042] ppid=25990 vsize=131848 CPUtime=1.32
/proc/25991/task/26042/stat : 26042 (pb_cplex) R 25990 25990 25677 0 -1 4202560 162 0 0 0 131 1 0 0 15 0 8 0 146782381 135012352 10706 4089446400 134512640 146736320 4291930848 18446744073709551615 136001923 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=25991/tid=26043] ppid=25990 vsize=131848 CPUtime=1.61
/proc/25991/task/26043/stat : 26043 (pb_cplex) R 25990 25990 25677 0 -1 4202560 240 0 0 0 160 1 0 0 15 0 8 0 146782381 135012352 10706 4089446400 134512640 146736320 4291930848 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=25991/tid=26044] ppid=25990 vsize=131848 CPUtime=1.46
/proc/25991/task/26044/stat : 26044 (pb_cplex) R 25990 25990 25677 0 -1 4202560 901 0 0 0 145 1 0 0 15 0 8 0 146782381 135012352 10706 4089446400 134512640 146736320 4291930848 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=25991/tid=26045] ppid=25990 vsize=131848 CPUtime=1.29
/proc/25991/task/26045/stat : 26045 (pb_cplex) R 25990 25990 25677 0 -1 4202560 1131 0 0 0 127 2 0 0 15 0 8 0 146782381 135012352 10706 4089446400 134512640 146736320 4291930848 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=25991/tid=26046] ppid=25990 vsize=131848 CPUtime=1.41
/proc/25991/task/26046/stat : 26046 (pb_cplex) R 25990 25990 25677 0 -1 4202560 3583 0 0 0 140 1 0 0 15 0 8 0 146782381 135012352 10706 4089446400 134512640 146736320 4291930848 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=25991/tid=26047] ppid=25990 vsize=131848 CPUtime=1.32
/proc/25991/task/26047/stat : 26047 (pb_cplex) R 25990 25990 25677 0 -1 4202560 651 0 0 0 131 1 0 0 15 0 8 0 146782381 135012352 10706 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=25991/tid=26048] ppid=25990 vsize=131848 CPUtime=1.75
/proc/25991/task/26048/stat : 26048 (pb_cplex) S 25990 25990 25677 0 -1 4202560 689 0 0 0 174 1 0 0 16 0 8 0 146782381 135012352 10706 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 15.92
Current children cumulated vsize (KiB) 140548

[startup+12.7008 s]
/proc/loadavg: 1.87 0.41 0.13 10/195 26069
/proc/meminfo: memFree=30792268/32951124 swapFree=67111528/67111528
[pid=25990] ppid=25988 vsize=8700 CPUtime=0
/proc/25990/stat : 25990 (run) S 25988 25990 25677 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782213 8908800 249 4089446400 4194304 4922060 140735399037328 18446744073709551615 212687494277 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25990/statm: 2175 249 208 178 0 68 0
[pid=25991] ppid=25990 vsize=134920 CPUtime=39.89
/proc/25991/stat : 25991 (pb_cplex) S 25990 25990 25677 0 -1 4202496 38117 0 0 0 3948 41 0 0 15 0 8 0 146782213 138158080 11466 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0
/proc/25991/statm: 33730 11466 1080 2985 0 30272 0
[pid=25991/tid=26042] ppid=25990 vsize=134920 CPUtime=4.4
/proc/25991/task/26042/stat : 26042 (pb_cplex) S 25990 25990 25677 0 -1 4202560 2257 0 0 0 437 3 0 0 16 0 8 0 146782381 138158080 11466 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 2 0 0 0
[pid=25991/tid=26043] ppid=25990 vsize=134920 CPUtime=4.53
/proc/25991/task/26043/stat : 26043 (pb_cplex) S 25990 25990 25677 0 -1 4202560 1039 0 0 0 449 4 0 0 15 0 8 0 146782381 138158080 11466 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 4 0 0 0
[pid=25991/tid=26044] ppid=25990 vsize=134920 CPUtime=4.46
/proc/25991/task/26044/stat : 26044 (pb_cplex) S 25990 25990 25677 0 -1 4202560 967 0 0 0 442 4 0 0 15 0 8 0 146782381 138158080 11466 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 0 0 0 0
[pid=25991/tid=26045] ppid=25990 vsize=134920 CPUtime=4.22
/proc/25991/task/26045/stat : 26045 (pb_cplex) S 25990 25990 25677 0 -1 4202560 2530 0 0 0 418 4 0 0 15 0 8 0 146782381 138158080 11466 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 0 0 0 0
[pid=25991/tid=26046] ppid=25990 vsize=134920 CPUtime=4.35
/proc/25991/task/26046/stat : 26046 (pb_cplex) S 25990 25990 25677 0 -1 4202560 4459 0 0 0 431 4 0 0 15 0 8 0 146782381 138158080 11466 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 0 0 0 0
[pid=25991/tid=26047] ppid=25990 vsize=134920 CPUtime=4.2
/proc/25991/task/26047/stat : 26047 (pb_cplex) S 25990 25990 25677 0 -1 4202560 3243 0 0 0 417 3 0 0 15 0 8 0 146782381 138158080 11466 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 0 0 0 0
[pid=25991/tid=26048] ppid=25990 vsize=134920 CPUtime=4.79
/proc/25991/task/26048/stat : 26048 (pb_cplex) R 25990 25990 25677 0 -1 4202560 2836 0 0 0 474 5 0 0 15 0 8 0 146782381 138158080 11466 4089446400 134512640 146736320 4291930848 18446744073709551615 139042902 0 2 4096 16387 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39.89
Current children cumulated vsize (KiB) 143620

[startup+25.5284 s]
/proc/loadavg: 3.46 0.83 0.27 14/195 26098
/proc/meminfo: memFree=30766724/32951124 swapFree=67111528/67111528
[pid=25990] ppid=25988 vsize=8700 CPUtime=0
/proc/25990/stat : 25990 (run) S 25988 25990 25677 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782213 8908800 249 4089446400 4194304 4922060 140735399037328 18446744073709551615 212687494277 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25990/statm: 2175 249 208 178 0 68 0
[pid=25991] ppid=25990 vsize=150292 CPUtime=83.49
/proc/25991/stat : 25991 (pb_cplex) R 25990 25990 25677 0 -1 4202496 62302 0 0 0 8265 84 0 0 15 0 8 0 146782213 153899008 14759 4089446400 134512640 146736320 4291930848 18446744073709551615 137046145 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0
/proc/25991/statm: 37573 14759 1083 2985 0 34115 0
[pid=25991/tid=26077] ppid=25990 vsize=150292 CPUtime=2.54
/proc/25991/task/26077/stat : 26077 (pb_cplex) S 25990 25990 25677 0 -1 4202560 463 0 0 0 253 1 0 0 15 0 8 0 146784068 153899008 14759 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 2 0 0 0
[pid=25991/tid=26078] ppid=25990 vsize=150292 CPUtime=3.01
/proc/25991/task/26078/stat : 26078 (pb_cplex) R 25990 25990 25677 0 -1 4202560 1235 0 0 0 300 1 0 0 16 0 8 0 146784068 153899008 14759 4089446400 134512640 146736320 4291930848 18446744073709551615 136017491 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=25991/tid=26079] ppid=25990 vsize=150292 CPUtime=1.81
/proc/25991/task/26079/stat : 26079 (pb_cplex) R 25990 25990 25677 0 -1 4202560 579 0 0 0 180 1 0 0 16 0 8 0 146784068 153899008 14759 4089446400 134512640 146736320 4291930848 18446744073709551615 135854502 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=25991/tid=26080] ppid=25990 vsize=150292 CPUtime=4.33
/proc/25991/task/26080/stat : 26080 (pb_cplex) R 25990 25990 25677 0 -1 4202560 2132 0 0 0 432 1 0 0 17 0 8 0 146784068 153899008 14759 4089446400 134512640 146736320 4291930848 18446744073709551615 144580332 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=25991/tid=26081] ppid=25990 vsize=150292 CPUtime=3.96
/proc/25991/task/26081/stat : 26081 (pb_cplex) R 25990 25990 25677 0 -1 4202560 1746 0 0 0 394 2 0 0 15 0 8 0 146784068 153899008 14759 4089446400 134512640 146736320 4291930848 18446744073709551615 137343493 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=25991/tid=26082] ppid=25990 vsize=150292 CPUtime=2.72
/proc/25991/task/26082/stat : 26082 (pb_cplex) R 25990 25990 25677 0 -1 4202560 4699 0 0 0 271 1 0 0 15 0 8 0 146784068 153899008 14759 4089446400 134512640 146736320 4291930848 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=25991/tid=26083] ppid=25990 vsize=150292 CPUtime=3.01
/proc/25991/task/26083/stat : 26083 (pb_cplex) R 25990 25990 25677 0 -1 4202560 2216 0 0 0 300 1 0 0 15 0 8 0 146784068 153899008 14759 4089446400 134512640 146736320 4291930848 18446744073709551615 136017512 0 2 4096 16387 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 83.49
Current children cumulated vsize (KiB) 158992

[startup+51.1067 s]
/proc/loadavg: 5.67 1.56 0.53 10/195 26098
/proc/meminfo: memFree=30758584/32951124 swapFree=67111528/67111528
[pid=25990] ppid=25988 vsize=8700 CPUtime=0
/proc/25990/stat : 25990 (run) S 25988 25990 25677 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782213 8908800 249 4089446400 4194304 4922060 140735399037328 18446744073709551615 212687494277 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25990/statm: 2175 249 208 178 0 68 0
[pid=25991] ppid=25990 vsize=153364 CPUtime=178.7
/proc/25991/stat : 25991 (pb_cplex) S 25990 25990 25677 0 -1 4202496 76358 0 0 0 17702 168 0 0 15 0 8 0 146782213 157044736 15428 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0
/proc/25991/statm: 38341 15428 1083 2985 0 34883 0
[pid=25991/tid=26077] ppid=25990 vsize=153364 CPUtime=14.18
/proc/25991/task/26077/stat : 26077 (pb_cplex) S 25990 25990 25677 0 -1 4202560 1213 0 0 0 1406 12 0 0 15 0 8 0 146784068 157044736 15428 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 5 0 0 0
[pid=25991/tid=26078] ppid=25990 vsize=153364 CPUtime=14.62
/proc/25991/task/26078/stat : 26078 (pb_cplex) S 25990 25990 25677 0 -1 4202560 2748 0 0 0 1450 12 0 0 15 0 8 0 146784068 157044736 15428 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 5 0 0 0
[pid=25991/tid=26079] ppid=25990 vsize=153364 CPUtime=13.55
/proc/25991/task/26079/stat : 26079 (pb_cplex) S 25990 25990 25677 0 -1 4202560 1591 0 0 0 1345 10 0 0 15 0 8 0 146784068 157044736 15428 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 2 0 0 0
[pid=25991/tid=26080] ppid=25990 vsize=153364 CPUtime=16.03
/proc/25991/task/26080/stat : 26080 (pb_cplex) S 25990 25990 25677 0 -1 4202560 8667 0 0 0 1591 12 0 0 15 0 8 0 146784068 157044736 15428 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 2 0 0 0
[pid=25991/tid=26081] ppid=25990 vsize=153364 CPUtime=15.84
/proc/25991/task/26081/stat : 26081 (pb_cplex) S 25990 25990 25677 0 -1 4202560 3474 0 0 0 1572 12 0 0 15 0 8 0 146784068 157044736 15428 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 7 0 0 0
[pid=25991/tid=26082] ppid=25990 vsize=153364 CPUtime=14.6
/proc/25991/task/26082/stat : 26082 (pb_cplex) R 25990 25990 25677 0 -1 4202560 4737 0 0 0 1449 11 0 0 15 0 8 0 146784068 157044736 15428 4089446400 134512640 146736320 4291930848 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=25991/tid=26083] ppid=25990 vsize=153364 CPUtime=14.76
/proc/25991/task/26083/stat : 26083 (pb_cplex) S 25990 25990 25677 0 -1 4202560 4570 0 0 0 1463 13 0 0 15 0 8 0 146784068 157044736 15428 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 178.7
Current children cumulated vsize (KiB) 162064

[startup+102.339 s]
/proc/loadavg: 7.42 2.68 0.96 8/195 26100
/proc/meminfo: memFree=30758716/32951124 swapFree=67111528/67111528

################
# More data... #
################

[pid=25991/tid=26082] ppid=25990 vsize=153724 CPUtime=93.38
/proc/25991/task/26082/stat : 26082 (pb_cplex) S 25990 25990 25677 0 -1 4202560 8417 0 0 0 9254 84 0 0 16 0 8 0 146784068 157413376 15581 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 2 0 0 0
[pid=25991/tid=26083] ppid=25990 vsize=153724 CPUtime=93.64
/proc/25991/task/26083/stat : 26083 (pb_cplex) S 25990 25990 25677 0 -1 4202560 9809 0 0 0 9278 86 0 0 15 0 8 0 146784068 157413376 15581 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 815.12
Current children cumulated vsize (KiB) 162424

[startup+282.301 s]
/proc/loadavg: 9.23 5.69 2.43 11/195 26104
/proc/meminfo: memFree=30758128/32951124 swapFree=67111528/67111528
[pid=25990] ppid=25988 vsize=8700 CPUtime=0
/proc/25990/stat : 25990 (run) S 25988 25990 25677 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782213 8908800 249 4089446400 4194304 4922060 140735399037328 18446744073709551615 212687494277 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25990/statm: 2175 249 208 178 0 68 0
[pid=25991] ppid=25990 vsize=153724 CPUtime=1035.28
/proc/25991/stat : 25991 (pb_cplex) S 25990 25990 25677 0 -1 4202496 117963 0 0 0 102601 927 0 0 16 0 8 0 146782213 157413376 15585 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0
/proc/25991/statm: 38431 15585 1083 2985 0 34973 0
[pid=25991/tid=26077] ppid=25990 vsize=153724 CPUtime=120.23
/proc/25991/task/26077/stat : 26077 (pb_cplex) S 25990 25990 25677 0 -1 4202560 5003 0 0 0 11922 101 0 0 16 0 8 0 146784068 157413376 15585 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 0 0 0 0
[pid=25991/tid=26078] ppid=25990 vsize=153724 CPUtime=120.12
/proc/25991/task/26078/stat : 26078 (pb_cplex) S 25990 25990 25677 0 -1 4202560 11412 0 0 0 11907 105 0 0 15 0 8 0 146784068 157413376 15585 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 0 0 0 0
[pid=25991/tid=26079] ppid=25990 vsize=153724 CPUtime=119.39
/proc/25991/task/26079/stat : 26079 (pb_cplex) S 25990 25990 25677 0 -1 4202560 6938 0 0 0 11840 99 0 0 15 0 8 0 146784068 157413376 15585 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 0 0 0 0
[pid=25991/tid=26080] ppid=25990 vsize=153724 CPUtime=121.5
/proc/25991/task/26080/stat : 26080 (pb_cplex) S 25990 25990 25677 0 -1 4202560 14674 0 0 0 12036 114 0 0 15 0 8 0 146784068 157413376 15585 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 0 0 0 0
[pid=25991/tid=26081] ppid=25990 vsize=153724 CPUtime=121.54
/proc/25991/task/26081/stat : 26081 (pb_cplex) S 25990 25990 25677 0 -1 4202560 7635 0 0 0 12046 108 0 0 15 0 8 0 146784068 157413376 15585 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 5 0 0 0
[pid=25991/tid=26082] ppid=25990 vsize=153724 CPUtime=120.88
/proc/25991/task/26082/stat : 26082 (pb_cplex) S 25990 25990 25677 0 -1 4202560 9012 0 0 0 11979 109 0 0 15 0 8 0 146784068 157413376 15585 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 6 0 0 0
[pid=25991/tid=26083] ppid=25990 vsize=153724 CPUtime=120.97
/proc/25991/task/26083/stat : 26083 (pb_cplex) R 25990 25990 25677 0 -1 4202560 10086 0 0 0 11984 113 0 0 16 0 8 0 146784068 157413376 15585 4089446400 134512640 146736320 4291930848 18446744073709551615 136017784 0 2 4096 16387 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 1035.28
Current children cumulated vsize (KiB) 162424

[startup+342.311 s]
/proc/loadavg: 8.92 6.23 2.82 14/195 26106
/proc/meminfo: memFree=30759400/32951124 swapFree=67111528/67111528
[pid=25990] ppid=25988 vsize=8700 CPUtime=0
/proc/25990/stat : 25990 (run) S 25988 25990 25677 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782213 8908800 249 4089446400 4194304 4922060 140735399037328 18446744073709551615 212687494277 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25990/statm: 2175 249 208 178 0 68 0
[pid=25991] ppid=25990 vsize=153724 CPUtime=1256.93
/proc/25991/stat : 25991 (pb_cplex) S 25990 25990 25677 0 -1 4202496 119540 0 0 0 124557 1136 0 0 15 0 8 0 146782213 157413376 15585 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/25991/statm: 38431 15585 1083 2985 0 34973 0
[pid=25991/tid=26077] ppid=25990 vsize=153724 CPUtime=147.74
/proc/25991/task/26077/stat : 26077 (pb_cplex) R 25990 25990 25677 0 -1 4202560 5081 0 0 0 14648 126 0 0 15 0 8 0 146784068 157413376 15585 4089446400 134512640 146736320 4291930848 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=25991/tid=26078] ppid=25990 vsize=153724 CPUtime=147.43
/proc/25991/task/26078/stat : 26078 (pb_cplex) S 25990 25990 25677 0 -1 4202560 11683 0 0 0 14614 129 0 0 15 0 8 0 146784068 157413376 15585 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 2 0 0 0
[pid=25991/tid=26079] ppid=25990 vsize=153724 CPUtime=146.97
/proc/25991/task/26079/stat : 26079 (pb_cplex) R 25990 25990 25677 0 -1 4202560 6982 0 0 0 14573 124 0 0 15 0 8 0 146784068 157413376 15585 4089446400 134512640 146736320 4291930848 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=25991/tid=26080] ppid=25990 vsize=153724 CPUtime=148.85
/proc/25991/task/26080/stat : 26080 (pb_cplex) R 25990 25990 25677 0 -1 4202560 14732 0 0 0 14746 139 0 0 15 0 8 0 146784068 157413376 15585 4089446400 134512640 146736320 4291930848 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=25991/tid=26081] ppid=25990 vsize=153724 CPUtime=148.85
/proc/25991/task/26081/stat : 26081 (pb_cplex) R 25990 25990 25677 0 -1 4202560 7856 0 0 0 14751 134 0 0 15 0 8 0 146784068 157413376 15585 4089446400 134512640 146736320 4291930848 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=25991/tid=26082] ppid=25990 vsize=153724 CPUtime=148.25
/proc/25991/task/26082/stat : 26082 (pb_cplex) R 25990 25990 25677 0 -1 4202560 9747 0 0 0 14688 137 0 0 15 0 8 0 146784068 157413376 15585 4089446400 134512640 146736320 4291930848 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=25991/tid=26083] ppid=25990 vsize=153724 CPUtime=148.43
/proc/25991/task/26083/stat : 26083 (pb_cplex) R 25990 25990 25677 0 -1 4202560 10214 0 0 0 14701 142 0 0 15 0 8 0 146784068 157413376 15585 4089446400 134512640 146736320 4291930848 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 1256.93
Current children cumulated vsize (KiB) 162424

[startup+402.311 s]
/proc/loadavg: 9.13 6.80 3.23 10/195 26108
/proc/meminfo: memFree=30758840/32951124 swapFree=67111528/67111528
[pid=25990] ppid=25988 vsize=8700 CPUtime=0
/proc/25990/stat : 25990 (run) S 25988 25990 25677 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782213 8908800 249 4089446400 4194304 4922060 140735399037328 18446744073709551615 212687494277 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25990/statm: 2175 249 208 178 0 68 0
[pid=25991] ppid=25990 vsize=153724 CPUtime=1477.67
/proc/25991/stat : 25991 (pb_cplex) S 25990 25990 25677 0 -1 4202496 120836 0 0 0 146435 1332 0 0 16 0 8 0 146782213 157413376 15591 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0
/proc/25991/statm: 38431 15591 1083 2985 0 34973 0
[pid=25991/tid=26077] ppid=25990 vsize=153724 CPUtime=174.85
/proc/25991/task/26077/stat : 26077 (pb_cplex) S 25990 25990 25677 0 -1 4202560 5216 0 0 0 17335 150 0 0 16 0 8 0 146784068 157413376 15591 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 3 0 0 0
[pid=25991/tid=26078] ppid=25990 vsize=153724 CPUtime=174.8
/proc/25991/task/26078/stat : 26078 (pb_cplex) R 25990 25990 25677 0 -1 4202560 12002 0 0 0 17325 155 0 0 15 0 8 0 146784068 157413376 15591 4089446400 134512640 146736320 4291930848 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=25991/tid=26079] ppid=25990 vsize=153724 CPUtime=174.34
/proc/25991/task/26079/stat : 26079 (pb_cplex) S 25990 25990 25677 0 -1 4202560 7278 0 0 0 17286 148 0 0 15 0 8 0 146784068 157413376 15591 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 7 0 0 0
[pid=25991/tid=26080] ppid=25990 vsize=153724 CPUtime=176.09
/proc/25991/task/26080/stat : 26080 (pb_cplex) R 25990 25990 25677 0 -1 4202560 14752 0 0 0 17446 163 0 0 15 0 8 0 146784068 157413376 15591 4089446400 134512640 146736320 4291930848 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=25991/tid=26081] ppid=25990 vsize=153724 CPUtime=176.14
/proc/25991/task/26081/stat : 26081 (pb_cplex) R 25990 25990 25677 0 -1 4202560 7954 0 0 0 17455 159 0 0 15 0 8 0 146784068 157413376 15591 4089446400 134512640 146736320 4291930848 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=25991/tid=26082] ppid=25990 vsize=153724 CPUtime=175.68
/proc/25991/task/26082/stat : 26082 (pb_cplex) R 25990 25990 25677 0 -1 4202560 9957 0 0 0 17406 162 0 0 15 0 8 0 146784068 157413376 15591 4089446400 134512640 146736320 4291930848 18446744073709551615 136384883 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=25991/tid=26083] ppid=25990 vsize=153724 CPUtime=175.93
/proc/25991/task/26083/stat : 26083 (pb_cplex) R 25990 25990 25677 0 -1 4202560 10409 0 0 0 17424 169 0 0 15 0 8 0 146784068 157413376 15591 4089446400 134512640 146736320 4291930848 18446744073709551615 135450809 0 2 4096 16387 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 1477.67
Current children cumulated vsize (KiB) 162424

[startup+462.302 s]
/proc/loadavg: 9.09 7.19 3.58 9/195 26109
/proc/meminfo: memFree=30758764/32951124 swapFree=67111528/67111528
[pid=25990] ppid=25988 vsize=8700 CPUtime=0
/proc/25990/stat : 25990 (run) S 25988 25990 25677 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782213 8908800 249 4089446400 4194304 4922060 140735399037328 18446744073709551615 212687494277 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25990/statm: 2175 249 208 178 0 68 0
[pid=25991] ppid=25990 vsize=153724 CPUtime=1697.33
/proc/25991/stat : 25991 (pb_cplex) R 25990 25990 25677 0 -1 4202496 123450 0 0 0 168213 1520 0 0 15 0 8 0 146782213 157413376 15612 4089446400 134512640 146736320 4291930848 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0
/proc/25991/statm: 38431 15612 1100 2985 0 34973 0
[pid=25991/tid=26077] ppid=25990 vsize=153724 CPUtime=202.06
/proc/25991/task/26077/stat : 26077 (pb_cplex) R 25990 25990 25677 0 -1 4202560 5479 0 0 0 20032 174 0 0 15 0 8 0 146784068 157413376 15612 4089446400 134512640 146736320 4291930848 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=25991/tid=26078] ppid=25990 vsize=153724 CPUtime=201.69
/proc/25991/task/26078/stat : 26078 (pb_cplex) R 25990 25990 25677 0 -1 4202560 12086 0 0 0 19991 178 0 0 15 0 8 0 146784068 157413376 15612 4089446400 134512640 146736320 4291930848 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=25991/tid=26079] ppid=25990 vsize=153724 CPUtime=201.47
/proc/25991/task/26079/stat : 26079 (pb_cplex) R 25990 25990 25677 0 -1 4202560 7612 0 0 0 19978 169 0 0 15 0 8 0 146784068 157413376 15612 4089446400 134512640 146736320 4291930848 18446744073709551615 137341786 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=25991/tid=26080] ppid=25990 vsize=153724 CPUtime=203
/proc/25991/task/26080/stat : 26080 (pb_cplex) R 25990 25990 25677 0 -1 4202560 15151 0 0 0 20114 186 0 0 15 0 8 0 146784068 157413376 15612 4089446400 134512640 146736320 4291930848 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=25991/tid=26081] ppid=25990 vsize=153724 CPUtime=203.13
/proc/25991/task/26081/stat : 26081 (pb_cplex) R 25990 25990 25677 0 -1 4202560 8307 0 0 0 20131 182 0 0 15 0 8 0 146784068 157413376 15612 4089446400 134512640 146736320 4291930848 18446744073709551615 137342827 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=25991/tid=26082] ppid=25990 vsize=153724 CPUtime=202.95
/proc/25991/task/26082/stat : 26082 (pb_cplex) R 25990 25990 25677 0 -1 4202560 10301 0 0 0 20108 187 0 0 15 0 8 0 146784068 157413376 15612 4089446400 134512640 146736320 4291930848 18446744073709551615 137366937 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=25991/tid=26083] ppid=25990 vsize=153724 CPUtime=203.29
/proc/25991/task/26083/stat : 26083 (pb_cplex) R 25990 25990 25677 0 -1 4202560 10849 0 0 0 20135 194 0 0 15 0 8 0 146784068 157413376 15612 4089446400 134512640 146736320 4291930848 18446744073709551615 135450810 0 2 4096 16387 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 1697.33
Current children cumulated vsize (KiB) 162424



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+488.723 s]
/proc/loadavg: 8.63 7.24 3.69 9/188 26139
/proc/meminfo: memFree=30828332/32951124 swapFree=67111528/67111528
[pid=25990] ppid=25988 vsize=8700 CPUtime=0
/proc/25990/stat : 25990 (run) S 25988 25990 25677 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782213 8908800 249 4089446400 4194304 4922060 140735399037328 18446744073709551615 212687494277 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25990/statm: 2175 249 208 178 0 68 0
[pid=25991] ppid=25990 vsize=153724 CPUtime=1800.47
/proc/25991/stat : 25991 (pb_cplex) S 25990 25990 25677 0 -1 4202496 126900 0 0 0 178428 1619 0 0 15 0 8 0 146782213 157413376 15612 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0
/proc/25991/statm: 38431 15612 1100 2985 0 34973 0
[pid=25991/tid=26077] ppid=25990 vsize=153724 CPUtime=214.62
/proc/25991/task/26077/stat : 26077 (pb_cplex) S 25990 25990 25677 0 -1 4202560 5889 0 0 0 21276 186 0 0 15 0 8 0 146784068 157413376 15612 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 2 0 0 0
[pid=25991/tid=26078] ppid=25990 vsize=153724 CPUtime=214.34
/proc/25991/task/26078/stat : 26078 (pb_cplex) S 25990 25990 25677 0 -1 4202560 12143 0 0 0 21245 189 0 0 16 0 8 0 146784068 157413376 15612 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 5 0 0 0
[pid=25991/tid=26079] ppid=25990 vsize=153724 CPUtime=214.23
/proc/25991/task/26079/stat : 26079 (pb_cplex) S 25990 25990 25677 0 -1 4202560 7912 0 0 0 21243 180 0 0 16 0 8 0 146784068 157413376 15612 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 0 0 0 0
[pid=25991/tid=26080] ppid=25990 vsize=153724 CPUtime=215.73
/proc/25991/task/26080/stat : 26080 (pb_cplex) R 25990 25990 25677 0 -1 4202560 15629 0 0 0 21375 198 0 0 16 0 8 0 146784068 157413376 15612 4089446400 134512640 146736320 4291930848 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=25991/tid=26081] ppid=25990 vsize=153724 CPUtime=216.1
/proc/25991/task/26081/stat : 26081 (pb_cplex) S 25990 25990 25677 0 -1 4202560 8316 0 0 0 21413 197 0 0 15 0 8 0 146784068 157413376 15612 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 1 0 0 0
[pid=25991/tid=26082] ppid=25990 vsize=153724 CPUtime=215.76
/proc/25991/task/26082/stat : 26082 (pb_cplex) S 25990 25990 25677 0 -1 4202560 10831 0 0 0 21376 200 0 0 15 0 8 0 146784068 157413376 15612 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 7 0 0 0
[pid=25991/tid=26083] ppid=25990 vsize=153724 CPUtime=216.19
/proc/25991/task/26083/stat : 26083 (pb_cplex) S 25990 25990 25677 0 -1 4202560 11583 0 0 0 21413 206 0 0 15 0 8 0 146784068 157413376 15612 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 1800.47
Current children cumulated vsize (KiB) 162424

Sending SIGTERM to process tree (bottom up)

[startup+t ended. Dt ended. Dumping a history of the last processes samples

[startup+488.723 s]
/proc/loadavg: 8.63 7.24 3.69 9/188 26139
/proc/meminfo: memFree=30828332/32951124 swapFree=67111528/67111528
[pid=25990] ppid=25988 vsize=8700 CPUtime=0
/proc/25990/stat : 25990 (run) S 25988 25990 25677 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782213 8908800 249 4089446400 4194304 4922060 140735399037328 18446744073709551615 212687494277 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/25990/statm: 2175 249 208 178 0 68 0
[pid=25991] ppid=25990 vsize=153724 CPUtime=1800.47
/proc/25991/stat : 25991 (pb_cplex) S 25990 25990 25677 0 -1 4202496 126900 0 0 0 178428 1619 0 0 15 0 8 0 146782213 157413376 15612 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0
/proc/25991/statm: 38431 15612 1100 2985 0 34973 0
[pid=25991/tid=26077] ppid=25990 vsize=153724 CPUtime=214.62
/proc/25991/task/26077/stat : 26077 (pb_cplex) S 25990 25990 25677 0 -1 4202560 5889 0 0 0 21276 186 0 0 15 0 8 0 146784068 157413376 15612 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 2 0 0 0
[pid=25991/tid=26078] ppid=25990 vsize=153724 CPUtime=214.34
/proc/25991/task/26078/stat : 26078 (pb_cplex) S 25990 25990 25677 0 -1 4202560 12143 0 0 0 21245 189 0 0 16 0 8 0 146784068 157413376 15612 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 5 0 0 0
[pid=25991/tid=26079] ppid=25990 vsize=153724 CPUtime=214.23
/proc/25991/task/26079/stat : 26079 (pb_cplex) S 25990 25990 25677 0 -1 4202560 7912 0 0 0 21243 180 0 0 16 0 8 0 146784068 157413376 15612 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 0 0 0 0
[pid=25991/tid=26080] ppid=25990 vsize=153724 CPUtime=215.73
/proc/25991/task/26080/stat : 26080 (pb_cplex) R 25990 25990 25677 0 -1 4202560 15629 0 0 0 21375 198 0 0 16 0 8 0 146784068 157413376 15612 4089446400 134512640 146736320 4291930848 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=25991/tid=26081] ppid=25990 vsize=153724 CPUtime=216.1
/proc/25991/task/26081/stat : 26081 (pb_cplex) S 25990 25990 25677 0 -1 4202560 8316 0 0 0 21413 197 0 0 15 0 8 0 146784068 157413376 15612 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 1 0 0 0
[pid=25991/tid=26082] ppid=25990 vsize=153724 CPUtime=215.76
/proc/25991/task/26082/stat : 26082 (pb_cplex) S 25990 25990 25677 0 -1 4202560 10831 0 0 0 21376 200 0 0 15 0 8 0 146784068 157413376 15612 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 7 0 0 0
[pid=25991/tid=26083] ppid=25990 vsize=153724 CPUtime=216.19
/proc/25991/task/26083/stat : 26083 (pb_cplex) S 25990 25990 25677 0 -1 4202560 11583 0 0 0 21413 206 0 0 15 0 8 0 146784068 157413376 15612 4089446400 134512640 146736320 4291930848 18446744073709551615 144580073 0 2 4096 16387 18446604471113077696 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 1800.47
Current children cumulated vsize (KiB) 162424

Child status: 0
Real time (s): 488.737
CPU time (s): 1800.49
CPU user time (s): 1784.29
CPU system time (s): 16.1965
CPU usage (%): 368.396
Max. virtual memory (cumulated for all children) (KiB): 165496

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1784.29
system time used= 16.1965
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 127273
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 349427
involuntary context switches= 26543

runsolver used 0.979851 second user time and 1.88471 second system time

The end

Launcher Data 

Begin job on node139 at 2010-07-02 09:14:27
IDJOB=2697897
IDBENCH=78936
IDSOLVER=1210
FILE ID=node139/2697897-1278054867
PBS_JOBID= 11198903
Free space on /tmp= 74140 MiB

SOLVER NAME= pb_cplex 2010-07-01
BENCH NAME= PB10/normalized-PB10/PARTIAL-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/tsp/normalized-t3002.11tsp11.1900563250--soft-66-100-0.wbo
COMMAND LINE= run BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-2697897-1278054867/watcher-2697897-1278054867 -o /tmp/evaluation-result-2697897-1278054867/solver-2697897-1278054867 -C 1800 -W 2000 -M 3800  run HOME/instance-2697897-1278054867.wbo

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 3800 MiB
MAX NB THREAD= 0

MD5SUM BENCH= 48850acf7a795604147e976fcd80db7d
RANDOM SEED=558558953

node139.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.840
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.840
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.840
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.840
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.840
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.78
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.840
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.840
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.840
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      30883308 kB
Buffers:        225240 kB
Cached:        1618768 kB
SwapCached:          0 kB
Active:         849088 kB
Inactive:      1066744 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      30883308 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           33668 kB
Writeback:           0 kB
AnonPages:       71648 kB
Mapped:          18308 kB
Slab:            87876 kB
PageTables:       4880 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   221448 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 74120 MiB
End job on node139 at 2010-07-02 09:22:36