Trace number 3736402

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
PB10: pb_cplex 2010-06-29? (TO) 1800.1 553.237

General information on the benchmark

Namenormalized-PB07/OPT-SMALLINT-NLC/submittedPB07/roussel/factor-mod-B/
factor-mod-size=7-P0=107-P1=13-P2=17-P3=127-P4=13-P5=53-P6=73-P7=59-B.opb
MD5SUMc07fae5b4c60769a268376cf318d4faa
Bench CategoryOPT-SMALLINT-NLC (optimisation, small integers, non linear constraints)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark3
Best CPU time to get the best result obtained on this benchmark0.062989
Has Objective FunctionYES
SatisfiableYES
(Un)Satisfiability was provedYES
Best value of the objective function 3
Optimality of the best value was proved YES
Number of variables147
Total number of constraints15
Number of constraints which are clauses0
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints15
Minimum length of a constraint7
Maximum length of a constraint63
Number of terms in the objective function 7
Biggest coefficient in the objective function 64
Number of bits for the biggest coefficient in the objective function 7
Sum of the numbers in the objective function 127
Number of bits of the sum of numbers in the objective function 7
Biggest number in a constraint 8192
Number of bits of the biggest number in a constraint 14
Biggest sum of numbers in a constraint 32512
Number of bits of the biggest sum of numbers15
Number of products (including duplicates)343
Sum of products size (including duplicates)686
Number of different products343
Sum of products size686

Solver Data 

0.00/0.00	c pb_cplex 1.0 - 20100701
0.00/0.00	c Parsing opb file format. File HOME/instance-3736402-1339248096.opb.
0.00/0.00	c Instance file HOME/instance-3736402-1339248096.opb
0.00/0.00	c File size is 25525 bytes.
0.00/0.00	c Highest Coefficient sum: 32512
0.00/0.00	c  IBM ILOG License Manager: "IBM ILOG Optimization Suite for Academic Initiative" is accessing CPLEX 12 with option(s): "e m b q ".
0.00/0.01	c ============================[ Problem Statistics ]=============================
0.00/0.01	c |                                                                             |
0.00/0.01	c |  Parsing time:          0.01         s                                      |
0.00/0.01	c |  Number Variables:      490                                                 |
0.00/0.01	c |  Number Constraints:    701                                                 |
0.00/0.01	c ===============================================================================

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3736402-1339248096/watcher-3736402-1339248096 -o /tmp/evaluation-result-3736402-1339248096/solver-3736402-1339248096 -C 1800 -W 1900 -M 15500 run HOME/instance-3736402-1339248096.opb 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.96 8.39 6.16 10/184 1890
/proc/meminfo: memFree=31161448/32873844 swapFree=7120/7120
[pid=1890] ppid=1888 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1890/stat : 1890 (run) S 1888 1890 1177 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390934537 9367552 270 33554432000 4194304 5064252 140736317580624 140736317579536 233220761166 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1890/statm: 2287 270 229 213 0 72 0

[startup+0.099587 s]
/proc/loadavg: 7.96 8.39 6.16 10/184 1890
/proc/meminfo: memFree=31161448/32873844 swapFree=7120/7120
[pid=1890] ppid=1888 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1890/stat : 1890 (run) S 1888 1890 1177 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390934537 9367552 270 33554432000 4194304 5064252 140736317580624 140736317579536 233220761166 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1890/statm: 2287 270 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.100253 s]
/proc/loadavg: 7.96 8.39 6.16 10/184 1890
/proc/meminfo: memFree=31161448/32873844 swapFree=7120/7120
[pid=1890] ppid=1888 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1890/stat : 1890 (run) S 1888 1890 1177 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390934537 9367552 270 33554432000 4194304 5064252 140736317580624 140736317579536 233220761166 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1890/statm: 2287 270 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.300226 s]
/proc/loadavg: 7.96 8.39 6.16 10/184 1890
/proc/meminfo: memFree=31161448/32873844 swapFree=7120/7120
[pid=1890] ppid=1888 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1890/stat : 1890 (run) S 1888 1890 1177 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390934537 9367552 270 33554432000 4194304 5064252 140736317580624 140736317579536 233220761166 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1890/statm: 2287 270 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.700225 s]
/proc/loadavg: 7.96 8.39 6.16 10/184 1890
/proc/meminfo: memFree=31161448/32873844 swapFree=7120/7120
[pid=1890] ppid=1888 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1890/stat : 1890 (run) S 1888 1890 1177 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390934537 9367552 270 33554432000 4194304 5064252 140736317580624 140736317579536 233220761166 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1890/statm: 2287 270 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+1.50048 s]
/proc/loadavg: 7.96 8.39 6.16 12/193 1913
/proc/meminfo: memFree=31132344/32873844 swapFree=7120/7120
[pid=1890] ppid=1888 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1890/stat : 1890 (run) S 1888 1890 1177 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390934537 9367552 270 33554432000 4194304 5064252 140736317580624 140736317579536 233220761166 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1890/statm: 2287 270 229 213 0 72 0
[pid=1891] ppid=1890 vsize=131004 CPUtime=4.07 cores=1,3,5,7
/proc/1891/stat : 1891 (pb_cplex) R 1890 1890 1177 0 -1 4202496 28275 0 0 0 386 21 0 0 20 0 8 0 390934537 134148096 11028 33554432000 134512640 146736320 4293764128 4293757008 136008069 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 0 0 0
/proc/1891/statm: 32751 11028 968 2985 0 29685 0
[pid=1891/tid=1907] ppid=1890 vsize=131004 CPUtime=0.48 cores=1,3,5,7
/proc/1891/task/1907/stat : 1907 (pb_cplex) S 1890 1890 1177 0 -1 4202560 2561 0 0 0 46 2 0 0 20 0 8 0 390934571 134148096 11028 33554432000 134512640 146736320 4293764128 4131352208 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=1891/tid=1908] ppid=1890 vsize=131004 CPUtime=0.42 cores=1,3,5,7
/proc/1891/task/1908/stat : 1908 (pb_cplex) R 1890 1890 1177 0 -1 4202560 1446 0 0 0 40 2 0 0 20 0 8 0 390934571 134148096 11028 33554432000 134512640 146736320 4293764128 4141834476 135467871 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=1891/tid=1909] ppid=1890 vsize=131004 CPUtime=0.35 cores=1,3,5,7
/proc/1891/task/1909/stat : 1909 (pb_cplex) S 1890 1890 1177 0 -1 4202560 1065 0 0 0 33 2 0 0 20 0 8 0 390934571 134148096 11028 33554432000 134512640 146736320 4293764128 4152323728 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=1891/tid=1910] ppid=1890 vsize=131004 CPUtime=0.43 cores=1,3,5,7
/proc/1891/task/1910/stat : 1910 (pb_cplex) S 1890 1890 1177 0 -1 4202560 3568 0 0 0 40 3 0 0 20 0 8 0 390934571 134148096 11028 33554432000 134512640 146736320 4293764128 4120354448 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=1891/tid=1911] ppid=1890 vsize=131004 CPUtime=0.44 cores=1,3,5,7
/proc/1891/task/1911/stat : 1911 (pb_cplex) R 1890 1890 1177 0 -1 4202560 3244 0 0 0 42 2 0 0 20 0 8 0 390934571 134148096 11028 33554432000 134512640 146736320 4293764128 4109865168 136008259 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=1891/tid=1912] ppid=1890 vsize=131004 CPUtime=0.48 cores=1,3,5,7
/proc/1891/task/1912/stat : 1912 (pb_cplex) S 1890 1890 1177 0 -1 4202560 3929 0 0 0 46 2 0 0 20 0 8 0 390934571 134148096 11028 33554432000 134512640 146736320 4293764128 4099382928 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=1891/tid=1913] ppid=1890 vsize=131004 CPUtime=0.39 cores=1,3,5,7
/proc/1891/task/1913/stat : 1913 (pb_cplex) S 1890 1890 1177 0 -1 4202560 4090 0 0 0 37 2 0 0 20 0 8 0 390934571 134148096 11028 33554432000 134512640 146736320 4293764128 4088897168 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 4.07
Current children cumulated vsize (KiB) 140152

[startup+3.10059 s]
/proc/loadavg: 8.05 8.40 6.18 15/193 1913
/proc/meminfo: memFree=31113356/32873844 swapFree=7120/7120
[pid=1890] ppid=1888 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1890/stat : 1890 (run) S 1888 1890 1177 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390934537 9367552 270 33554432000 4194304 5064252 140736317580624 140736317579536 233220761166 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1890/statm: 2287 270 229 213 0 72 0
[pid=1891] ppid=1890 vsize=153660 CPUtime=9.04 cores=1,3,5,7
/proc/1891/stat : 1891 (pb_cplex) S 1890 1890 1177 0 -1 4202496 46184 0 0 0 872 32 0 0 20 0 8 0 390934537 157347840 16300 33554432000 134512640 146736320 4293764128 4293760528 144580073 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/1891/statm: 38415 16300 971 2985 0 35349 0
[pid=1891/tid=1907] ppid=1890 vsize=153660 CPUtime=1.13 cores=1,3,5,7
/proc/1891/task/1907/stat : 1907 (pb_cplex) R 1890 1890 1177 0 -1 4202560 3354 0 0 0 110 3 0 0 20 0 8 0 390934571 157347840 16300 33554432000 134512640 146736320 4293764128 4131348688 136008450 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=1891/tid=1908] ppid=1890 vsize=153660 CPUtime=1.05 cores=1,3,5,7
/proc/1891/task/1908/stat : 1908 (pb_cplex) S 1890 1890 1177 0 -1 4202560 2666 0 0 0 102 3 0 0 20 0 8 0 390934571 157347840 16300 33554432000 134512640 146736320 4293764128 4141837968 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=1891/tid=1909] ppid=1890 vsize=153660 CPUtime=0.88 cores=1,3,5,7
/proc/1891/task/1909/stat : 1909 (pb_cplex) S 1890 1890 1177 0 -1 4202560 2338 0 0 0 85 3 0 0 20 0 8 0 390934571 157347840 16300 33554432000 134512640 146736320 4293764128 4152323728 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=1891/tid=1910] ppid=1890 vsize=153660 CPUtime=1.04 cores=1,3,5,7
/proc/1891/task/1910/stat : 1910 (pb_cplex) S 1890 1890 1177 0 -1 4202560 6434 0 0 0 100 4 0 0 20 0 8 0 390934571 157347840 16300 33554432000 134512640 146736320 4293764128 4120354448 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=1891/tid=1911] ppid=1890 vsize=153660 CPUtime=1.04 cores=1,3,5,7
/proc/1891/task/1911/stat : 1911 (pb_cplex) S 1890 1890 1177 0 -1 4202560 5166 0 0 0 101 3 0 0 20 0 8 0 390934571 157347840 16300 33554432000 134512640 146736320 4293764128 4109868688 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=1891/tid=1912] ppid=1890 vsize=153660 CPUtime=1.12 cores=1,3,5,7
/proc/1891/task/1912/stat : 1912 (pb_cplex) S 1890 1890 1177 0 -1 4202560 7832 0 0 0 109 3 0 0 20 0 8 0 390934571 157347840 16300 33554432000 134512640 146736320 4293764128 4099382928 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=1891/tid=1913] ppid=1890 vsize=153660 CPUtime=0.94 cores=1,3,5,7
/proc/1891/task/1913/stat : 1913 (pb_cplex) S 1890 1890 1177 0 -1 4202560 7461 0 0 0 91 3 0 0 20 0 8 0 390934571 157347840 16300 33554432000 134512640 146736320 4293764128 4088897168 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 9.04
Current children cumulated vsize (KiB) 162808

[startup+6.30047 s]
/proc/loadavg: 8.05 8.40 6.18 10/193 1913
/proc/meminfo: memFree=31082612/32873844 swapFree=7120/7120
[pid=1890] ppid=1888 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1890/stat : 1890 (run) S 1888 1890 1177 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390934537 9367552 270 33554432000 4194304 5064252 140736317580624 140736317579536 233220761166 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1890/statm: 2287 270 229 213 0 72 0
[pid=1891] ppid=1890 vsize=181468 CPUtime=19.62 cores=1,3,5,7
/proc/1891/stat : 1891 (pb_cplex) R 1890 1890 1177 0 -1 4202496 71129 0 0 0 1911 51 0 0 20 0 8 0 390934537 185823232 22479 33554432000 134512640 146736320 4293764128 4293757256 136000532 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 0 0 0
/proc/1891/statm: 45367 22479 971 2985 0 42301 0
[pid=1891/tid=1907] ppid=1890 vsize=181468 CPUtime=2.54 cores=1,3,5,7
/proc/1891/task/1907/stat : 1907 (pb_cplex) S 1890 1890 1177 0 -1 4202560 6220 0 0 0 250 4 0 0 20 0 8 0 390934571 185823232 22479 33554432000 134512640 146736320 4293764128 4131352208 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=1891/tid=1908] ppid=1890 vsize=181468 CPUtime=2.32 cores=1,3,5,7
/proc/1891/task/1908/stat : 1908 (pb_cplex) R 1890 1890 1177 0 -1 4202560 3973 0 0 0 227 5 0 0 20 0 8 0 390934571 185823232 22479 33554432000 134512640 146736320 4293764128 4141834448 136008065 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=1891/tid=1909] ppid=1890 vsize=181468 CPUtime=2.21 cores=1,3,5,7
/proc/1891/task/1909/stat : 1909 (pb_cplex) R 1890 1890 1177 0 -1 4202560 4217 0 0 0 218 3 0 0 20 0 8 0 390934571 185823232 22479 33554432000 134512640 146736320 4293764128 4152319148 135454093 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=1891/tid=1910] ppid=1890 vsize=181468 CPUtime=2.26 cores=1,3,5,7
/proc/1891/task/1910/stat : 1910 (pb_cplex) S 1890 1890 1177 0 -1 4202560 12341 0 0 0 219 7 0 0 20 0 8 0 390934571 185823232 22479 33554432000 134512640 146736320 4293764128 4120354448 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=1891/tid=1911] ppid=1890 vsize=181468 CPUtime=2.42 cores=1,3,5,7
/proc/1891/task/1911/stat : 1911 (pb_cplex) R 1890 1890 1177 0 -1 4202560 10023 0 0 0 236 6 0 0 20 0 8 0 390934571 185823232 22479 33554432000 134512640 146736320 4293764128 4109864956 135526884 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=1891/tid=1912] ppid=1890 vsize=181468 CPUtime=2.43 cores=1,3,5,7
/proc/1891/task/1912/stat : 1912 (pb_cplex) R 1890 1890 1177 0 -1 4202560 12216 0 0 0 237 6 0 0 20 0 8 0 390934571 185823232 22479 33554432000 134512640 146736320 4293764128 4099379408 136008450 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=1891/tid=1913] ppid=1890 vsize=181468 CPUtime=2.33 cores=1,3,5,7
/proc/1891/task/1913/stat : 1913 (pb_cplex) R 1890 1890 1177 0 -1 4202560 9317 0 0 0 227 6 0 0 20 0 8 0 390934571 185823232 22479 33554432000 134512640 146736320 4293764128 4088893564 135526785 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 19.62
Current children cumulated vsize (KiB) 190616

[startup+12.7005 s]
/proc/loadavg: 8.36 8.45 6.22 14/193 1913
/proc/meminfo: memFree=31069720/32873844 swapFree=7120/7120
[pid=1890] ppid=1888 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1890/stat : 1890 (run) S 1888 1890 1177 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390934537 9367552 270 33554432000 4194304 5064252 140736317580624 140736317579536 233220761166 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1890/statm: 2287 270 229 213 0 72 0
[pid=1891] ppid=1890 vsize=193940 CPUtime=38.99 cores=1,3,5,7
/proc/1891/stat : 1891 (pb_cplex) R 1890 1890 1177 0 -1 4202496 118574 0 0 0 3822 77 0 0 20 0 8 0 390934537 198594560 25764 33554432000 134512640 146736320 4293764128 4293757008 136008039 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/1891/statm: 48485 25764 971 2985 0 45419 0
[pid=1891/tid=1907] ppid=1890 vsize=193940 CPUtime=4.46 cores=1,3,5,7
/proc/1891/task/1907/stat : 1907 (pb_cplex) R 1890 1890 1177 0 -1 4202560 9170 0 0 0 439 7 0 0 20 0 8 0 390934571 198594560 25764 33554432000 134512640 146736320 4293764128 4131348688 136008309 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=1891/tid=1908] ppid=1890 vsize=193940 CPUtime=4.44 cores=1,3,5,7
/proc/1891/task/1908/stat : 1908 (pb_cplex) R 1890 1890 1177 0 -1 4202560 6209 0 0 0 438 6 0 0 20 0 8 0 390934571 198594560 25764 33554432000 134512640 146736320 4293764128 4141835140 135850322 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=1891/tid=1909] ppid=1890 vsize=193940 CPUtime=4.67 cores=1,3,5,7
/proc/1891/task/1909/stat : 1909 (pb_cplex) R 1890 1890 1177 0 -1 4202560 10891 0 0 0 461 6 0 0 20 0 8 0 390934571 198594560 25764 33554432000 134512640 146736320 4293764128 4152320208 136008266 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=1891/tid=1910] ppid=1890 vsize=193940 CPUtime=5.5 cores=1,3,5,7
/proc/1891/task/1910/stat : 1910 (pb_cplex) R 1890 1890 1177 0 -1 4202560 21218 0 0 0 538 12 0 0 20 0 8 0 390934571 198594560 25764 33554432000 134512640 146736320 4293764128 4120351584 135845830 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=1891/tid=1911] ppid=1890 vsize=193940 CPUtime=5.06 cores=1,3,5,7
/proc/1891/task/1911/stat : 1911 (pb_cplex) R 1890 1890 1177 0 -1 4202560 16154 0 0 0 497 9 0 0 20 0 8 0 390934571 198594560 25764 33554432000 134512640 146736320 4293764128 4109865168 136008050 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=1891/tid=1912] ppid=1890 vsize=193940 CPUtime=4.72 cores=1,3,5,7
/proc/1891/task/1912/stat : 1912 (pb_cplex) R 1890 1890 1177 0 -1 4202560 20423 0 0 0 463 9 0 0 20 0 8 0 390934571 198594560 25764 33554432000 134512640 146736320 4293764128 4099381108 135964351 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=1891/tid=1913] ppid=1890 vsize=193940 CPUtime=4.56 cores=1,3,5,7
/proc/1891/task/1913/stat : 1913 (pb_cplex) R 1890 1890 1177 0 -1 4202560 15701 0 0 0 447 9 0 0 20 0 8 0 390934571 198594560 25764 33554432000 134512640 146736320 4293764128 4088894440 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 38.99
Current children cumulated vsize (KiB) 203088

[startup+25.5007 s]
/proc/loadavg: 8.38 8.45 6.24 7/193 1913
/proc/meminfo: memFree=31033616/32873844 swapFree=7120/7120
[pid=1890] ppid=1888 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1890/stat : 1890 (run) S 1888 1890 1177 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390934537 9367552 270 33554432000 4194304 5064252 140736317580624 140736317579536 233220761166 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1890/statm: 2287 270 229 213 0 72 0
[pid=1891] ppid=1890 vsize=224588 CPUtime=72.41 cores=1,3,5,7
/proc/1891/stat : 1891 (pb_cplex) S 1890 1890 1177 0 -1 4202496 182745 0 0 0 7128 113 0 0 20 0 8 0 390934537 229978112 33131 33554432000 134512640 146736320 4293764128 4293760528 144580073 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/1891/statm: 56147 33131 979 2985 0 53081 0
[pid=1891/tid=1907] ppid=1890 vsize=224588 CPUtime=8.14 cores=1,3,5,7
/proc/1891/task/1907/stat : 1907 (pb_cplex) S 1890 1890 1177 0 -1 4202560 14570 0 0 0 804 10 0 0 20 0 8 0 390934571 229978112 33131 33554432000 134512640 146736320 4293764128 4131352208 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=1891/tid=1908] ppid=1890 vsize=224588 CPUtime=8.68 cores=1,3,5,7
/proc/1891/task/1908/stat : 1908 (pb_cplex) R 1890 1890 1177 0 -1 4202560 10283 0 0 0 858 10 0 0 20 0 8 0 390934571 229978112 33131 33554432000 134512640 146736320 4293764128 4141833404 135526778 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=1891/tid=1909] ppid=1890 vsize=224588 CPUtime=8.37 cores=1,3,5,7
/proc/1891/task/1909/stat : 1909 (pb_cplex) S 1890 1890 1177 0 -1 4202560 18656 0 0 0 826 11 0 0 20 0 8 0 390934571 229978112 33131 33554432000 134512640 146736320 4293764128 4152323728 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=1891/tid=1910] ppid=1890 vsize=224588 CPUtime=9.28 cores=1,3,5,7
/proc/1891/task/1910/stat : 1910 (pb_cplex) S 1890 1890 1177 0 -1 4202560 31160 0 0 0 911 17 0 0 20 0 8 0 390934571 229978112 33131 33554432000 134512640 146736320 4293764128 4120354448 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=1891/tid=1911] ppid=1890 vsize=224588 CPUtime=8.39 cores=1,3,5,7
/proc/1891/task/1911/stat : 1911 (pb_cplex) R 1890 1890 1177 0 -1 4202560 20742 0 0 0 825 14 0 0 20 0 8 0 390934571 229978112 33131 33554432000 134512640 146736320 4293764128 4109865024 135521326 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=1891/tid=1912] ppid=1890 vsize=224588 CPUtime=9.25 cores=1,3,5,7
/proc/1891/task/1912/stat : 1912 (pb_cplex) S 1890 1890 1177 0 -1 4202560 32073 0 0 0 909 16 0 0 20 0 8 0 390934571 229978112 33131 33554432000 134512640 146736320 4293764128 4099382928 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 1 0 0 0 0 0
[pid=1891/tid=1913] ppid=1890 vsize=224588 CPUtime=8.85 cores=1,3,5,7
/proc/1891/task/1913/stat : 1913 (pb_cplex) R 1890 1890 1177 0 -1 4202560 28133 0 0 0 870 15 0 0 20 0 8 0 390934571 229978112 33131 33554432000 134512640 146736320 4293764128 4088894032 136403109 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 72.41
Current children cumulated vsize (KiB) 233736

[startup+51.1006 s]
/proc/loadavg: 8.29 8.42 6.29 14/193 1913
/proc/meminfo: memFree=31030552/32873844 swapFree=7120/7120
[pid=1890] ppid=1888 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1890/stat : 1890 (run) S 1888 1890 1177 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390934537 9367552 270 33554432000 4194304 5064252 140736317580624 140736317579536 233220761166 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1890/statm: 2287 270 229 213 0 72 0

################
# More data... #
################

/proc/1891/statm: 56828 34069 979 2985 0 53762 0
[pid=1891/tid=1907] ppid=1890 vsize=227312 CPUtime=103.33 cores=1,3,5,7
/proc/1891/task/1907/stat : 1907 (pb_cplex) R 1890 1890 1177 0 -1 4202560 126179 0 0 0 10114 219 0 0 20 0 8 0 390934571 232767488 34069 33554432000 134512640 146736320 4293764128 4131349480 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=1891/tid=1908] ppid=1890 vsize=227312 CPUtime=108.43 cores=1,3,5,7
/proc/1891/task/1908/stat : 1908 (pb_cplex) R 1890 1890 1177 0 -1 4202560 43416 0 0 0 10664 179 0 0 20 0 8 0 390934571 232767488 34069 33554432000 134512640 146736320 4293764128 4141832956 135526778 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=1891/tid=1909] ppid=1890 vsize=227312 CPUtime=110.09 cores=1,3,5,7
/proc/1891/task/1909/stat : 1909 (pb_cplex) R 1890 1890 1177 0 -1 4202560 126562 0 0 0 10798 211 0 0 20 0 8 0 390934571 232767488 34069 33554432000 134512640 146736320 4293764128 4152320900 135851007 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=1891/tid=1910] ppid=1890 vsize=227312 CPUtime=111.32 cores=1,3,5,7
/proc/1891/task/1910/stat : 1910 (pb_cplex) R 1890 1890 1177 0 -1 4202560 105767 0 0 0 10923 209 0 0 20 0 8 0 390934571 232767488 34069 33554432000 134512640 146736320 4293764128 4120350720 135468511 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=1891/tid=1911] ppid=1890 vsize=227312 CPUtime=109.54 cores=1,3,5,7
/proc/1891/task/1911/stat : 1911 (pb_cplex) R 1890 1890 1177 0 -1 4202560 69751 0 0 0 10755 199 0 0 20 0 8 0 390934571 232767488 34069 33554432000 134512640 146736320 4293764128 4109863668 138462824 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=1891/tid=1912] ppid=1890 vsize=227312 CPUtime=112.39 cores=1,3,5,7
/proc/1891/task/1912/stat : 1912 (pb_cplex) R 1890 1890 1177 0 -1 4202560 99033 0 0 0 11038 201 0 0 20 0 8 0 390934571 232767488 34069 33554432000 134512640 146736320 4293764128 4099379016 135526330 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=1891/tid=1913] ppid=1890 vsize=227312 CPUtime=112.94 cores=1,3,5,7
/proc/1891/task/1913/stat : 1913 (pb_cplex) R 1890 1890 1177 0 -1 4202560 115239 0 0 0 11081 213 0 0 20 0 8 0 390934571 232767488 34069 33554432000 134512640 146736320 4293764128 4088896156 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 894.29
Current children cumulated vsize (KiB) 236460

[startup+342.3 s]
/proc/loadavg: 6.89 8.78 7.15 3/186 2024
/proc/meminfo: memFree=31171304/32873844 swapFree=7120/7120
[pid=1890] ppid=1888 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1890/stat : 1890 (run) S 1888 1890 1177 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390934537 9367552 270 33554432000 4194304 5064252 140736317580624 140736317579536 233220761166 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1890/statm: 2287 270 229 213 0 72 0
[pid=1891] ppid=1890 vsize=228336 CPUtime=1099.6 cores=1,3,5,7
/proc/1891/stat : 1891 (pb_cplex) S 1890 1890 1177 0 -1 4202496 916784 0 0 0 107307 2653 0 0 20 0 8 0 390934537 233816064 34311 33554432000 134512640 146736320 4293764128 4293760528 144580073 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/1891/statm: 57084 34311 979 2985 0 54018 0
[pid=1891/tid=1907] ppid=1890 vsize=228336 CPUtime=127.36 cores=1,3,5,7
/proc/1891/task/1907/stat : 1907 (pb_cplex) R 1890 1890 1177 0 -1 4202560 149080 0 0 0 12387 349 0 0 20 0 8 0 390934571 233816064 34311 33554432000 134512640 146736320 4293764128 4131348476 135453303 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=1891/tid=1908] ppid=1890 vsize=228336 CPUtime=133.95 cores=1,3,5,7
/proc/1891/task/1908/stat : 1908 (pb_cplex) R 1890 1890 1177 0 -1 4202560 55910 0 0 0 13098 297 0 0 20 0 8 0 390934571 233816064 34311 33554432000 134512640 146736320 4293764128 4141834348 135454063 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=1891/tid=1909] ppid=1890 vsize=228336 CPUtime=135.43 cores=1,3,5,7
/proc/1891/task/1909/stat : 1909 (pb_cplex) R 1890 1890 1177 0 -1 4202560 141478 0 0 0 13218 325 0 0 20 0 8 0 390934571 233816064 34311 33554432000 134512640 146736320 4293764128 4152319980 135454093 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=1891/tid=1910] ppid=1890 vsize=228336 CPUtime=137.06 cores=1,3,5,7
/proc/1891/task/1910/stat : 1910 (pb_cplex) R 1890 1890 1177 0 -1 4202560 138235 0 0 0 13354 352 0 0 20 0 8 0 390934571 233816064 34311 33554432000 134512640 146736320 4293764128 4120351620 135850322 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=1891/tid=1911] ppid=1890 vsize=228336 CPUtime=135.18 cores=1,3,5,7
/proc/1891/task/1911/stat : 1911 (pb_cplex) R 1890 1890 1177 0 -1 4202560 78880 0 0 0 13199 319 0 0 20 0 8 0 390934571 233816064 34311 33554432000 134512640 146736320 4293764128 4109863660 135454096 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=1891/tid=1912] ppid=1890 vsize=228336 CPUtime=138.23 cores=1,3,5,7
/proc/1891/task/1912/stat : 1912 (pb_cplex) R 1890 1890 1177 0 -1 4202560 135418 0 0 0 13483 340 0 0 20 0 8 0 390934571 233816064 34311 33554432000 134512640 146736320 4293764128 4099377916 135526703 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=1891/tid=1913] ppid=1890 vsize=228336 CPUtime=138.35 cores=1,3,5,7
/proc/1891/task/1913/stat : 1913 (pb_cplex) R 1890 1890 1177 0 -1 4202560 124147 0 0 0 13501 334 0 0 20 0 8 0 390934571 233816064 34311 33554432000 134512640 146736320 4293764128 4088894340 135850304 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1099.6
Current children cumulated vsize (KiB) 237484

[startup+402.3 s]
/proc/loadavg: 5.92 8.16 7.03 7/186 2024
/proc/meminfo: memFree=31165992/32873844 swapFree=7120/7120
[pid=1890] ppid=1888 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1890/stat : 1890 (run) S 1888 1890 1177 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390934537 9367552 270 33554432000 4194304 5064252 140736317580624 140736317579536 233220761166 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1890/statm: 2287 270 229 213 0 72 0
[pid=1891] ppid=1890 vsize=228336 CPUtime=1303.78 cores=1,3,5,7
/proc/1891/stat : 1891 (pb_cplex) R 1890 1890 1177 0 -1 4202496 1037541 0 0 0 126671 3707 0 0 20 0 8 0 390934537 233816064 34286 33554432000 134512640 146736320 4293764128 4293756796 135526870 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/1891/statm: 57084 34286 979 2985 0 54018 0
[pid=1891/tid=1907] ppid=1890 vsize=228336 CPUtime=152.56 cores=1,3,5,7
/proc/1891/task/1907/stat : 1907 (pb_cplex) S 1890 1890 1177 0 -1 4202560 162126 0 0 0 14775 481 0 0 20 0 8 0 390934571 233816064 34286 33554432000 134512640 146736320 4293764128 4131352208 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=1891/tid=1908] ppid=1890 vsize=228336 CPUtime=159.66 cores=1,3,5,7
/proc/1891/task/1908/stat : 1908 (pb_cplex) R 1890 1890 1177 0 -1 4202560 60134 0 0 0 15564 402 0 0 20 0 8 0 390934571 233816064 34286 33554432000 134512640 146736320 4293764128 4141834220 135454079 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=1891/tid=1909] ppid=1890 vsize=228336 CPUtime=160.8 cores=1,3,5,7
/proc/1891/task/1909/stat : 1909 (pb_cplex) R 1890 1890 1177 0 -1 4202560 149354 0 0 0 15628 452 0 0 20 0 8 0 390934571 233816064 34286 33554432000 134512640 146736320 4293764128 4152319980 135454096 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=1891/tid=1910] ppid=1890 vsize=228336 CPUtime=162.67 cores=1,3,5,7
/proc/1891/task/1910/stat : 1910 (pb_cplex) S 1890 1890 1177 0 -1 4202560 176091 0 0 0 15763 504 0 0 20 0 8 0 390934571 233816064 34286 33554432000 134512640 146736320 4293764128 4120354448 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=1891/tid=1911] ppid=1890 vsize=228336 CPUtime=160.38 cores=1,3,5,7
/proc/1891/task/1911/stat : 1911 (pb_cplex) R 1890 1890 1177 0 -1 4202560 99205 0 0 0 15589 449 0 0 20 0 8 0 390934571 233816064 34286 33554432000 134512640 146736320 4293764128 4109865976 136364176 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=1891/tid=1912] ppid=1890 vsize=228336 CPUtime=163.33 cores=1,3,5,7
/proc/1891/task/1912/stat : 1912 (pb_cplex) R 1890 1890 1177 0 -1 4202560 151745 0 0 0 15860 473 0 0 20 0 8 0 390934571 233816064 34286 33554432000 134512640 146736320 4293764128 4099380100 135850322 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=1891/tid=1913] ppid=1890 vsize=228336 CPUtime=163.52 cores=1,3,5,7
/proc/1891/task/1913/stat : 1913 (pb_cplex) R 1890 1890 1177 0 -1 4202560 131254 0 0 0 15883 469 0 0 20 0 8 0 390934571 233816064 34286 33554432000 134512640 146736320 4293764128 4088894440 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 1303.78
Current children cumulated vsize (KiB) 237484

[startup+462.301 s]
/proc/loadavg: 8.02 8.23 7.12 17/193 2031
/proc/meminfo: memFree=31046512/32873844 swapFree=7120/7120
[pid=1890] ppid=1888 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1890/stat : 1890 (run) S 1888 1890 1177 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390934537 9367552 270 33554432000 4194304 5064252 140736317580624 140736317579536 233220761166 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1890/statm: 2287 270 229 213 0 72 0
[pid=1891] ppid=1890 vsize=228336 CPUtime=1500.54 cores=1,3,5,7
/proc/1891/stat : 1891 (pb_cplex) R 1890 1890 1177 0 -1 4202496 1140472 0 0 0 145568 4486 0 0 20 0 8 0 390934537 233816064 34372 33554432000 134512640 146736320 4293764128 4293757700 135850319 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/1891/statm: 57084 34372 979 2985 0 54018 0
[pid=1891/tid=1907] ppid=1890 vsize=228336 CPUtime=176.63 cores=1,3,5,7
/proc/1891/task/1907/stat : 1907 (pb_cplex) R 1890 1890 1177 0 -1 4202560 190380 0 0 0 17080 583 0 0 20 0 8 0 390934571 233816064 34372 33554432000 134512640 146736320 4293764128 4131348480 135468677 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=1891/tid=1908] ppid=1890 vsize=228336 CPUtime=184.74 cores=1,3,5,7
/proc/1891/task/1908/stat : 1908 (pb_cplex) S 1890 1890 1177 0 -1 4202560 77508 0 0 0 17972 502 0 0 20 0 8 0 390934571 233816064 34372 33554432000 134512640 146736320 4293764128 4141837968 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=1891/tid=1909] ppid=1890 vsize=228336 CPUtime=185.4 cores=1,3,5,7
/proc/1891/task/1909/stat : 1909 (pb_cplex) R 1890 1890 1177 0 -1 4202560 155292 0 0 0 17998 542 0 0 20 0 8 0 390934571 233816064 34372 33554432000 134512640 146736320 4293764128 4152322716 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=1891/tid=1910] ppid=1890 vsize=228336 CPUtime=187.54 cores=1,3,5,7
/proc/1891/task/1910/stat : 1910 (pb_cplex) R 1890 1890 1177 0 -1 4202560 192326 0 0 0 18153 601 0 0 20 0 8 0 390934571 233816064 34372 33554432000 134512640 146736320 4293764128 4120350720 135468684 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=1891/tid=1911] ppid=1890 vsize=228336 CPUtime=184.11 cores=1,3,5,7
/proc/1891/task/1911/stat : 1911 (pb_cplex) R 1890 1890 1177 0 -1 4202560 104296 0 0 0 17867 544 0 0 20 0 8 0 390934571 233816064 34372 33554432000 134512640 146736320 4293764128 4109864960 135468700 0 2 4096 16387 0 0 0 -1 3 0 0 1 0 0
[pid=1891/tid=1912] ppid=1890 vsize=228336 CPUtime=187.46 cores=1,3,5,7
/proc/1891/task/1912/stat : 1912 (pb_cplex) R 1890 1890 1177 0 -1 4202560 172581 0 0 0 18163 583 0 0 20 0 8 0 390934571 233816064 34372 33554432000 134512640 146736320 4293764128 4099379196 135526778 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=1891/tid=1913] ppid=1890 vsize=228336 CPUtime=187.27 cores=1,3,5,7
/proc/1891/task/1913/stat : 1913 (pb_cplex) R 1890 1890 1177 0 -1 4202560 134092 0 0 0 18160 567 0 0 20 0 8 0 390934571 233816064 34372 33554432000 134512640 146736320 4293764128 4088893420 135454096 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 1500.54
Current children cumulated vsize (KiB) 237484

[startup+522.3 s]
/proc/loadavg: 9.68 8.69 7.34 10/194 2034
/proc/meminfo: memFree=31044932/32873844 swapFree=7120/7120
[pid=1890] ppid=1888 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1890/stat : 1890 (run) S 1888 1890 1177 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390934537 9367552 270 33554432000 4194304 5064252 140736317580624 140736317579536 233220761166 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1890/statm: 2287 270 229 213 0 72 0
[pid=1891] ppid=1890 vsize=228336 CPUtime=1694.34 cores=1,3,5,7
/proc/1891/stat : 1891 (pb_cplex) R 1890 1890 1177 0 -1 4202496 1214490 0 0 0 164712 4722 0 0 20 0 8 0 390934537 233816064 34246 33554432000 134512640 146736320 4293764128 4293760528 144580073 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/1891/statm: 57084 34246 979 2985 0 54018 0
[pid=1891/tid=1907] ppid=1890 vsize=228336 CPUtime=200.23 cores=1,3,5,7
/proc/1891/task/1907/stat : 1907 (pb_cplex) R 1890 1890 1177 0 -1 4202560 206928 0 0 0 19410 613 0 0 20 0 8 0 390934571 233816064 34246 33554432000 134512640 146736320 4293764128 4131349312 135231478 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=1891/tid=1908] ppid=1890 vsize=228336 CPUtime=208.92 cores=1,3,5,7
/proc/1891/task/1908/stat : 1908 (pb_cplex) R 1890 1890 1177 0 -1 4202560 84586 0 0 0 20362 530 0 0 20 0 8 0 390934571 233816064 34246 33554432000 134512640 146736320 4293764128 4141834236 135453309 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=1891/tid=1909] ppid=1890 vsize=228336 CPUtime=209.71 cores=1,3,5,7
/proc/1891/task/1909/stat : 1909 (pb_cplex) R 1890 1890 1177 0 -1 4202560 179648 0 0 0 20382 589 0 0 20 0 8 0 390934571 233816064 34246 33554432000 134512640 146736320 4293764128 4152320040 139041192 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=1891/tid=1910] ppid=1890 vsize=228336 CPUtime=211.46 cores=1,3,5,7
/proc/1891/task/1910/stat : 1910 (pb_cplex) R 1890 1890 1177 0 -1 4202560 195597 0 0 0 20521 625 0 0 20 0 8 0 390934571 233816064 34246 33554432000 134512640 146736320 4293764128 4120350716 135526785 0 2 4096 16387 0 0 0 -1 3 0 0 1 0 0
[pid=1891/tid=1911] ppid=1890 vsize=228336 CPUtime=207.98 cores=1,3,5,7
/proc/1891/task/1911/stat : 1911 (pb_cplex) R 1890 1890 1177 0 -1 4202560 106192 0 0 0 20227 571 0 0 20 0 8 0 390934571 233816064 34246 33554432000 134512640 146736320 4293764128 4109864956 135526870 0 2 4096 16387 0 0 0 -1 1 0 0 1 0 0
[pid=1891/tid=1912] ppid=1890 vsize=228336 CPUtime=211.36 cores=1,3,5,7
/proc/1891/task/1912/stat : 1912 (pb_cplex) R 1890 1890 1177 0 -1 4202560 185365 0 0 0 20516 620 0 0 20 0 8 0 390934571 233816064 34246 33554432000 134512640 146736320 4293764128 4099380052 135983229 0 2 4096 16387 0 0 0 -1 7 0 0 1 0 0
[pid=1891/tid=1913] ppid=1890 vsize=228336 CPUtime=209.86 cores=1,3,5,7
/proc/1891/task/1913/stat : 1913 (pb_cplex) R 1890 1890 1177 0 -1 4202560 137169 0 0 0 20395 591 0 0 20 0 8 0 390934571 233816064 34246 33554432000 134512640 146736320 4293764128 4088894440 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
Current children cumulated CPU time (s) 1694.34
Current children cumulated vsize (KiB) 237484



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+553.201 s]
/proc/loadavg: 9.16 8.66 7.38 10/194 2035
/proc/meminfo: memFree=31027784/32873844 swapFree=7120/7120
[pid=1890] ppid=1888 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/1890/stat : 1890 (run) S 1888 1890 1177 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390934537 9367552 270 33554432000 4194304 5064252 140736317580624 140736317579536 233220761166 0 65536 4100 65538 18446744071579302229 0 0 17 5 0 0 0 0 0
/proc/1890/statm: 2287 270 229 213 0 72 0
[pid=1891] ppid=1890 vsize=228336 CPUtime=1800.1 cores=1,3,5,7
/proc/1891/stat : 1891 (pb_cplex) R 1890 1890 1177 0 -1 4202496 1261097 0 0 0 174946 5064 0 0 20 0 8 0 390934537 233816064 34254 33554432000 134512640 146736320 4293764128 4293757400 136402277 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/1891/statm: 57084 34254 979 2985 0 54018 0
[pid=1891/tid=1907] ppid=1890 vsize=228336 CPUtime=213.25 cores=1,3,5,7
/proc/1891/task/1907/stat : 1907 (pb_cplex) R 1890 1890 1177 0 -1 4202560 214233 0 0 0 20668 657 0 0 20 0 8 0 390934571 233816064 34254 33554432000 134512640 146736320 4293764128 4131349380 135850735 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=1891/tid=1908] ppid=1890 vsize=228336 CPUtime=222.32 cores=1,3,5,7
/proc/1891/task/1908/stat : 1908 (pb_cplex) R 1890 1890 1177 0 -1 4202560 93447 0 0 0 21661 571 0 0 20 0 8 0 390934571 233816064 34254 33554432000 134512640 146736320 4293764128 4141834240 135468697 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=1891/tid=1909] ppid=1890 vsize=228336 CPUtime=223.01 cores=1,3,5,7
/proc/1891/task/1909/stat : 1909 (pb_cplex) R 1890 1890 1177 0 -1 4202560 200662 0 0 0 21659 642 0 0 20 0 8 0 390934571 233816064 34254 33554432000 134512640 146736320 4293764128 4152320672 135982692 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=1891/tid=1910] ppid=1890 vsize=228336 CPUtime=224.65 cores=1,3,5,7
/proc/1891/task/1910/stat : 1910 (pb_cplex) R 1890 1890 1177 0 -1 4202560 198582 0 0 0 21792 673 0 0 20 0 8 0 390934571 233816064 34254 33554432000 134512640 146736320 4293764128 4120354476 144580332 0 2 4096 16387 0 0 0 -1 5 0 0 1 0 0
[pid=1891/tid=1911] ppid=1890 vsize=228336 CPUtime=221.03 cores=1,3,5,7
/proc/1891/task/1911/stat : 1911 (pb_cplex) R 1890 1890 1177 0 -1 4202560 107197 0 0 0 21490 613 0 0 20 0 8 0 390934571 233816064 34254 33554432000 134512640 146736320 4293764128 4109865884 139028294 0 2 4096 16387 0 0 0 -1 5 0 0 1 0 0
[pid=1891/tid=1912] ppid=1890 vsize=228336 CPUtime=224.28 cores=1,3,5,7
/proc/1891/task/1912/stat : 1912 (pb_cplex) R 1890 1890 1177 0 -1 4202560 187077 0 0 0 21763 665 0 0 20 0 8 0 390934571 233816064 34254 33554432000 134512640 146736320 4293764128 4099379196 135526785 0 2 4096 16387 0 0 0 -1 3 0 0 1 0 0
[pid=1891/tid=1913] ppid=1890 vsize=228336 CPUtime=222.52 cores=1,3,5,7
/proc/1891/task/1913/stat : 1913 (pb_cplex) R 1890 1890 1177 0 -1 4202560 138474 0 0 0 21624 628 0 0 20 0 8 0 390934571 233816064 34254 33554432000 134512640 146736320 4293764128 4088894440 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 1800.1
Current children cumulated vsize (KiB) 237484

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 1890 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=0
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=999
# CPU time returned by wait4() is 0.000999
# while last known CPU time is 1800.1
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 553.237
CPU time (s): 1800.1
CPU user time (s): 1749.46
CPU system time (s): 50.64
CPU usage (%): 325.376
Max. virtual memory (cumulated for all children) (KiB): 259324

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0
system time used= 0.000999
maximum resident set size= 1080
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 356
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2
involuntary context switches= 1

runsolver used 1.64075 second user time and 4.47532 second system time

The end

Launcher Data 

Begin job on node142 at 2012-06-09 15:21:36
IDJOB=3736402
IDBENCH=47909
IDSOLVER=2325
FILE ID=node142/3736402-1339248096
RUNJOBID= node142-1339247525-1194
PBS_JOBID= 14644380
Free space on /tmp= 71448 MiB

SOLVER NAME= PB10: pb_cplex 2010-06-29
BENCH NAME= PB07/normalized-PB07/OPT-SMALLINT-NLC/submittedPB07/roussel/factor-mod-B/factor-mod-size=7-P0=107-P1=13-P2=17-P3=127-P4=13-P5=53-P6=73-P7=59-B.opb
COMMAND LINE= run BENCHNAME
CONVERSION SCRIPT= PBconversionToLinear BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3736402-1339248096/watcher-3736402-1339248096 -o /tmp/evaluation-result-3736402-1339248096/solver-3736402-1339248096 -C 1800 -W 1900 -M 15500  run HOME/instance-3736402-1339248096.opb

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= c07fae5b4c60769a268376cf318d4faa
RANDOM SEED=1464849357

node142.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.776
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.776
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.776
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5391.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.776
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.776
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.776
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.776
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.776
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        31163820 kB
Buffers:          333828 kB
Cached:           225856 kB
SwapCached:         1676 kB
Active:           764212 kB
Inactive:          45740 kB
Active(anon):     248556 kB
Inactive(anon):     2184 kB
Active(file):     515656 kB
Inactive(file):    43556 kB
Unevictable:        7120 kB
Mlocked:            7120 kB
SwapTotal:      67108856 kB
SwapFree:       67100604 kB
Dirty:             16404 kB
Writeback:             0 kB
AnonPages:        253512 kB
Mapped:            12940 kB
Shmem:                 4 kB
Slab:             758912 kB
SReclaimable:     200356 kB
SUnreclaim:       558556 kB
KernelStack:        1512 kB
PageTables:         4304 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     326652 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:     55296 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

CONVERSION COMMAND LINE= PBconversionToLinear HOME/instance-3736402-1339248096.opb

Free space on /tmp at the end= 71432 MiB
End job on node142 at 2012-06-09 15:30:50