Trace number 3737150

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
PB10: pb_cplex 2010-06-29? (TO) 1800.03 519.417

General information on the benchmark

Namenormalized-PB07/OPT-SMALLINT-NLC/submittedPB07/
manquinho/mis/normalized-mis_500_10_3.opb
MD5SUM0247ef4de041bbff376811a7e6bf0408
Bench CategoryOPT-SMALLINT-NLC (optimisation, small integers, non linear constraints)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark-115
Best CPU time to get the best result obtained on this benchmark1800.13
Has Objective FunctionYES
SatisfiableYES
(Un)Satisfiability was provedYES
Best value of the objective function -110
Optimality of the best value was proved NO
Number of variables500
Total number of constraints500
Number of constraints which are clauses0
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints500
Minimum length of a constraint10
Maximum length of a constraint22
Number of terms in the objective function 500
Biggest coefficient in the objective function 1
Number of bits for the biggest coefficient in the objective function 1
Sum of the numbers in the objective function 500
Number of bits of the sum of numbers in the objective function 9
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 500
Number of bits of the biggest sum of numbers9
Number of products (including duplicates)6316
Sum of products size (including duplicates)12632
Number of different products3158
Sum of products size6316

Solver Data 

0.00/0.00	c pb_cplex 1.0 - 20100701
0.00/0.00	c Parsing opb file format. File HOME/instance-3737150-1339248535.opb.
0.00/0.00	c Instance file HOME/instance-3737150-1339248535.opb
0.00/0.00	c File size is 270643 bytes.
0.00/0.06	c Highest Coefficient sum: 22
0.00/0.06	c  IBM ILOG License Manager: "IBM ILOG Optimization Suite for Academic Initiative" is accessing CPLEX 12 with option(s): "e m b q ".
0.00/0.07	c ============================[ Problem Statistics ]=============================
0.00/0.07	c |                                                                             |
0.00/0.07	c |  Parsing time:          0.07         s                                      |
0.00/0.07	c |  Number Variables:      3658                                                |
0.00/0.07	c |  Number Constraints:    6816                                                |
0.00/0.07	c ===============================================================================
1800.03/519.41	c Remaining time 1799.93 sec.
1800.03/519.41	c 
1800.03/519.41	c *** INTERRUPTED ***
1800.03/519.41	s UNKNOWN
1800.03/519.41	c Total CPU time              : 1796.41 s
1800.03/519.41	c Memory used           : 211.95 MB
1800.03/519.41	c CPU time              : 1796.41 s
1800.03/519.41	c 
1800.03/519.41	c *** INTERRUPTED ***

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3737150-1339248535/watcher-3737150-1339248535 -o /tmp/evaluation-result-3737150-1339248535/solver-3737150-1339248535 -C 1800 -W 1900 -M 15500 run HOME/instance-3737150-1339248535.opb 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.56 6.95 5.22 3/177 7030
/proc/meminfo: memFree=28033336/32873844 swapFree=7900/7900
[pid=7030] ppid=7028 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/7030/stat : 7030 (run) S 7028 7030 6357 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390978030 9367552 270 33554432000 4194304 5064252 140733990401536 140733990400448 225444521550 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/7030/statm: 2287 270 229 213 0 72 0

[startup+0.0725181 s]
/proc/loadavg: 7.56 6.95 5.22 3/177 7030
/proc/meminfo: memFree=28033336/32873844 swapFree=7900/7900
[pid=7030] ppid=7028 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/7030/stat : 7030 (run) S 7028 7030 6357 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390978030 9367552 270 33554432000 4194304 5064252 140733990401536 140733990400448 225444521550 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/7030/statm: 2287 270 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.100284 s]
/proc/loadavg: 7.56 6.95 5.22 3/177 7030
/proc/meminfo: memFree=28033336/32873844 swapFree=7900/7900
[pid=7030] ppid=7028 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/7030/stat : 7030 (run) S 7028 7030 6357 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390978030 9367552 270 33554432000 4194304 5064252 140733990401536 140733990400448 225444521550 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/7030/statm: 2287 270 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.300224 s]
/proc/loadavg: 7.56 6.95 5.22 3/177 7030
/proc/meminfo: memFree=28033336/32873844 swapFree=7900/7900
[pid=7030] ppid=7028 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/7030/stat : 7030 (run) S 7028 7030 6357 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390978030 9367552 270 33554432000 4194304 5064252 140733990401536 140733990400448 225444521550 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/7030/statm: 2287 270 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.700226 s]
/proc/loadavg: 7.56 6.95 5.22 3/177 7030
/proc/meminfo: memFree=28033336/32873844 swapFree=7900/7900
[pid=7030] ppid=7028 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/7030/stat : 7030 (run) S 7028 7030 6357 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390978030 9367552 270 33554432000 4194304 5064252 140733990401536 140733990400448 225444521550 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/7030/statm: 2287 270 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+1.50027 s]
/proc/loadavg: 7.56 6.95 5.22 3/179 7032
/proc/meminfo: memFree=28023912/32873844 swapFree=7900/7900
[pid=7030] ppid=7028 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/7030/stat : 7030 (run) S 7028 7030 6357 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390978030 9367552 270 33554432000 4194304 5064252 140733990401536 140733990400448 225444521550 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/7030/statm: 2287 270 229 213 0 72 0
[pid=7031] ppid=7030 vsize=23452 CPUtime=1.48 cores=1,3,5,7
/proc/7031/stat : 7031 (pb_cplex) R 7030 7030 6357 0 -1 4202496 5646 0 0 0 148 0 0 0 20 0 1 0 390978030 24014848 3178 33554432000 134512640 146736320 4286581600 4286576208 136403116 0 0 4096 16387 0 0 0 17 5 0 0 0 0 0
/proc/7031/statm: 5863 3178 896 2985 0 2797 0
Current children cumulated CPU time (s) 1.48
Current children cumulated vsize (KiB) 32600

[startup+3.10027 s]
/proc/loadavg: 7.11 6.87 5.20 3/179 7032
/proc/meminfo: memFree=28023292/32873844 swapFree=7900/7900
[pid=7030] ppid=7028 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/7030/stat : 7030 (run) S 7028 7030 6357 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390978030 9367552 270 33554432000 4194304 5064252 140733990401536 140733990400448 225444521550 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/7030/statm: 2287 270 229 213 0 72 0
[pid=7031] ppid=7030 vsize=25392 CPUtime=3.09 cores=1,3,5,7
/proc/7031/stat : 7031 (pb_cplex) R 7030 7030 6357 0 -1 4202496 7388 0 0 0 308 1 0 0 20 0 1 0 390978030 26001408 3588 33554432000 134512640 146736320 4286581600 4286577436 136406496 0 0 4096 16387 0 0 0 17 5 0 0 0 0 0
/proc/7031/statm: 6348 3588 896 2985 0 3282 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 34540

[startup+6.30027 s]
/proc/loadavg: 7.11 6.87 5.20 3/179 7032
/proc/meminfo: memFree=28021928/32873844 swapFree=7900/7900
[pid=7030] ppid=7028 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/7030/stat : 7030 (run) S 7028 7030 6357 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390978030 9367552 270 33554432000 4194304 5064252 140733990401536 140733990400448 225444521550 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/7030/statm: 2287 270 229 213 0 72 0
[pid=7031] ppid=7030 vsize=26452 CPUtime=6.29 cores=1,3,5,7
/proc/7031/stat : 7031 (pb_cplex) R 7030 7030 6357 0 -1 4202496 12872 0 0 0 627 2 0 0 20 0 1 0 390978030 27086848 3909 33554432000 134512640 146736320 4286581600 4286577424 137698804 0 0 4096 16387 0 0 0 17 5 0 0 0 0 0
/proc/7031/statm: 6613 3909 897 2985 0 3547 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 35600

[startup+12.7004 s]
/proc/loadavg: 6.48 6.74 5.18 10/186 7039
/proc/meminfo: memFree=28004472/32873844 swapFree=7900/7900
[pid=7030] ppid=7028 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/7030/stat : 7030 (run) S 7028 7030 6357 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390978030 9367552 270 33554432000 4194304 5064252 140733990401536 140733990400448 225444521550 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/7030/statm: 2287 270 229 213 0 72 0
[pid=7031] ppid=7030 vsize=125216 CPUtime=14.91 cores=1,3,5,7
/proc/7031/stat : 7031 (pb_cplex) R 7030 7030 6357 0 -1 4202496 19924 0 0 0 1487 4 0 0 20 0 8 0 390978030 128221184 8816 33554432000 134512640 146736320 4286581600 4286575272 144757825 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/7031/statm: 31304 8816 927 2985 0 28238 0
[pid=7031/tid=7033] ppid=7030 vsize=125216 CPUtime=0.22 cores=1,3,5,7
/proc/7031/task/7033/stat : 7033 (pb_cplex) R 7030 7030 6357 0 -1 4202560 410 0 0 0 22 0 0 0 20 0 8 0 390979038 128221184 8816 33554432000 134512640 146736320 4286581600 4148826704 136403116 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=7031/tid=7034] ppid=7030 vsize=125216 CPUtime=0.34 cores=1,3,5,7
/proc/7031/task/7034/stat : 7034 (pb_cplex) R 7030 7030 6357 0 -1 4202560 476 0 0 0 34 0 0 0 20 0 8 0 390979038 128221184 8816 33554432000 134512640 146736320 4286581600 4138340560 136008042 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=7031/tid=7035] ppid=7030 vsize=125216 CPUtime=0.4 cores=1,3,5,7
/proc/7031/task/7035/stat : 7035 (pb_cplex) R 7030 7030 6357 0 -1 4202560 418 0 0 0 40 0 0 0 20 0 8 0 390979038 128221184 8816 33554432000 134512640 146736320 4286581600 4127853504 136018931 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=7031/tid=7036] ppid=7030 vsize=125216 CPUtime=0.37 cores=1,3,5,7
/proc/7031/task/7036/stat : 7036 (pb_cplex) R 7030 7030 6357 0 -1 4202560 470 0 0 0 37 0 0 0 20 0 8 0 390979038 128221184 8816 33554432000 134512640 146736320 4286581600 4117368880 136384861 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=7031/tid=7037] ppid=7030 vsize=125216 CPUtime=0.35 cores=1,3,5,7
/proc/7031/task/7037/stat : 7037 (pb_cplex) R 7030 7030 6357 0 -1 4202560 530 0 0 0 35 0 0 0 20 0 8 0 390979038 128221184 8816 33554432000 134512640 146736320 4286581600 4106883280 136008042 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=7031/tid=7038] ppid=7030 vsize=125216 CPUtime=0.46 cores=1,3,5,7
/proc/7031/task/7038/stat : 7038 (pb_cplex) R 7030 7030 6357 0 -1 4202560 574 0 0 0 46 0 0 0 20 0 8 0 390979038 128221184 8816 33554432000 134512640 146736320 4286581600 4096397520 136008050 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=7031/tid=7039] ppid=7030 vsize=125216 CPUtime=0.27 cores=1,3,5,7
/proc/7031/task/7039/stat : 7039 (pb_cplex) R 7030 7030 6357 0 -1 4202560 509 0 0 0 27 0 0 0 20 0 8 0 390979038 128221184 8816 33554432000 134512640 146736320 4286581600 4085911720 135983182 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 14.91
Current children cumulated vsize (KiB) 134364

[startup+25.5003 s]
/proc/loadavg: 6.79 6.80 5.21 3/179 7039
/proc/meminfo: memFree=28005852/32873844 swapFree=7900/7900
[pid=7030] ppid=7028 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/7030/stat : 7030 (run) S 7028 7030 6357 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390978030 9367552 270 33554432000 4194304 5064252 140733990401536 140733990400448 225444521550 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/7030/statm: 2287 270 229 213 0 72 0
[pid=7031] ppid=7030 vsize=86168 CPUtime=55.65 cores=1,3,5,7
/proc/7031/stat : 7031 (pb_cplex) R 7030 7030 6357 0 -1 4202496 21154 0 0 0 5552 13 0 0 20 0 1 0 390978030 88236032 7951 33554432000 134512640 146736320 4286581600 4286575048 135525911 0 0 4096 16387 0 0 0 17 1 0 0 0 0 0
/proc/7031/statm: 21542 7951 929 2985 0 18476 0
Current children cumulated CPU time (s) 55.65
Current children cumulated vsize (KiB) 95316

[startup+51.1004 s]
/proc/loadavg: 6.81 6.79 5.25 10/186 7053
/proc/meminfo: memFree=27954856/32873844 swapFree=7900/7900
[pid=7030] ppid=7028 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/7030/stat : 7030 (run) S 7028 7030 6357 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390978030 9367552 270 33554432000 4194304 5064252 140733990401536 140733990400448 225444521550 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/7030/statm: 2287 270 229 213 0 72 0
[pid=7031] ppid=7030 vsize=179600 CPUtime=136.95 cores=1,3,5,7
/proc/7031/stat : 7031 (pb_cplex) R 7030 7030 6357 0 -1 4202496 55220 0 0 0 13661 34 0 0 20 0 8 0 390978030 183910400 20361 33554432000 134512640 146736320 4286581600 4286573392 136008356 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 0 0 0
/proc/7031/statm: 44900 20361 966 2985 0 41834 0
[pid=7031/tid=7047] ppid=7030 vsize=179600 CPUtime=9.74 cores=1,3,5,7
/proc/7031/task/7047/stat : 7047 (pb_cplex) R 7030 7030 6357 0 -1 4202560 2958 0 0 0 972 2 0 0 20 0 8 0 390980655 183910400 20361 33554432000 134512640 146736320 4286581600 4069515336 139042901 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=7031/tid=7048] ppid=7030 vsize=179600 CPUtime=9.2 cores=1,3,5,7
/proc/7031/task/7048/stat : 7048 (pb_cplex) R 7030 7030 6357 0 -1 4202560 2841 0 0 0 920 0 0 0 20 0 8 0 390980655 183910400 20361 33554432000 134512640 146736320 4286581600 4136890408 139044585 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=7031/tid=7049] ppid=7030 vsize=179600 CPUtime=9.73 cores=1,3,5,7
/proc/7031/task/7049/stat : 7049 (pb_cplex) R 7030 7030 6357 0 -1 4202560 3383 0 0 0 971 2 0 0 20 0 8 0 390980655 183910400 20361 33554432000 134512640 146736320 4286581600 4147376584 136001974 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=7031/tid=7050] ppid=7030 vsize=179600 CPUtime=8.02 cores=1,3,5,7
/proc/7031/task/7050/stat : 7050 (pb_cplex) R 7030 7030 6357 0 -1 4202560 3876 0 0 0 800 2 0 0 20 0 8 0 390980655 183910400 20361 33554432000 134512640 146736320 4286581600 4117367836 135854505 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=7031/tid=7051] ppid=7030 vsize=179600 CPUtime=9.11 cores=1,3,5,7
/proc/7031/task/7051/stat : 7051 (pb_cplex) R 7030 7030 6357 0 -1 4202560 3819 0 0 0 909 2 0 0 20 0 8 0 390980655 183910400 20361 33554432000 134512640 146736320 4286581600 4059029596 135854543 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=7031/tid=7052] ppid=7030 vsize=179600 CPUtime=9.72 cores=1,3,5,7
/proc/7031/task/7052/stat : 7052 (pb_cplex) R 7030 7030 6357 0 -1 4202560 1742 0 0 0 970 2 0 0 20 0 8 0 390980655 183910400 20361 33554432000 134512640 146736320 4286581600 4048543952 136008450 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=7031/tid=7053] ppid=7030 vsize=179600 CPUtime=9.2 cores=1,3,5,7
/proc/7031/task/7053/stat : 7053 (pb_cplex) R 7030 7030 6357 0 -1 4202560 3973 0 0 0 918 2 0 0 20 0 8 0 390980655 183910400 20361 33554432000 134512640 146736320 4286581600 4038058576 136403103 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 136.95
Current children cumulated vsize (KiB) 188748

[startup+102.307 s]
/proc/loadavg: 7.97 7.12 5.45 3/187 7056
/proc/meminfo: memFree=27940748/32873844 swapFree=7900/7900
[pid=7030] ppid=7028 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/7030/stat : 7030 (run) S 7028 7030 6357 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390978030 9367552 270 33554432000 4194304 5064252 140733990401536 140733990400448 225444521550 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/7030/statm: 2287 270 229 213 0 72 0
[pid=7031] ppid=7030 vsize=184040 CPUtime=323.19 cores=1,3,5,7
/proc/7031/stat : 7031 (pb_cplex) S 7030 7030 6357 0 -1 4202496 89597 0 0 0 32252 67 0 0 20 0 8 0 390978030 188456960 23778 33554432000 134512640 146736320 4286581600 4286578000 144580073 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/7031/statm: 46010 23778 966 2985 0 42944 0
[pid=7031/tid=7047] ppid=7030 vsize=184040 CPUtime=34.02 cores=1,3,5,7
/proc/7031/task/7047/stat : 7047 (pb_cplex) S 7030 7030 6357 0 -1 4202560 5180 0 0 0 3397 5 0 0 20 0 8 0 390980655 188456960 23778 33554432000 134512640 146736320 4286581600 4069518992 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=7031/tid=7048] ppid=7030 vsize=184040 CPUtime=32.89 cores=1,3,5,7
/proc/7031/task/7048/stat : 7048 (pb_cplex) S 7030 7030 6357 0 -1 4202560 5513 0 0 0 3284 5 0 0 20 0 8 0 390980655 188456960 23778 33554432000 134512640 146736320 4286581600 4136894096 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=7031/tid=7049] ppid=7030 vsize=184040 CPUtime=33.88 cores=1,3,5,7
/proc/7031/task/7049/stat : 7049 (pb_cplex) S 7030 7030 6357 0 -1 4202560 6464 0 0 0 3382 6 0 0 20 0 8 0 390980655 188456960 23778 33554432000 134512640 146736320 4286581600 4147379856 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=7031/tid=7050] ppid=7030 vsize=184040 CPUtime=30.75 cores=1,3,5,7
/proc/7031/task/7050/stat : 7050 (pb_cplex) S 7030 7030 6357 0 -1 4202560 10361 0 0 0 3068 7 0 0 20 0 8 0 390980655 188456960 23778 33554432000 134512640 146736320 4286581600 4117372560 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
[pid=7031/tid=7051] ppid=7030 vsize=184040 CPUtime=31.59 cores=1,3,5,7
/proc/7031/task/7051/stat : 7051 (pb_cplex) R 7030 7030 6357 0 -1 4202560 11024 0 0 0 3151 8 0 0 20 0 8 0 390980655 188456960 23778 33554432000 134512640 146736320 4286581600 4059029416 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=7031/tid=7052] ppid=7030 vsize=184040 CPUtime=32.29 cores=1,3,5,7
/proc/7031/task/7052/stat : 7052 (pb_cplex) S 7030 7030 6357 0 -1 4202560 1758 0 0 0 3224 5 0 0 20 0 8 0 390980655 188456960 23778 33554432000 134512640 146736320 4286581600 4048547472 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=7031/tid=7053] ppid=7030 vsize=184040 CPUtime=32.78 cores=1,3,5,7
/proc/7031/task/7053/stat : 7053 (pb_cplex) S 7030 7030 6357 0 -1 4202560 9993 0 0 0 3272 6 0 0 20 0 8 0 390980655 188456960 23778 33554432000 134512640 146736320 4286581600 4038061712 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 7 0 0 0 0 0
Current children cumulated CPU time (s) 323.19
Current children cumulated vsize (KiB) 193188

[startup+162.3 s]
/proc/loadavg: 6.81 6.96 5.50 10/186 7056
/proc/meminfo: memFree=27932324/32873844 swapFree=7900/7900
[pid=7030] ppid=7028 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/7030/stat : 7030 (run) S 7028 7030 6357 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390978030 9367552 270 33554432000 4194304 5064252 140733990401536 140733990400448 225444521550 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/7030/statm: 2287 270 229 213 0 72 0
[pid=7031] ppid=7030 vsize=219568 CPUtime=529.07 cores=1,3,5,7
/proc/7031/stat : 7031 (pb_cplex) R 7030 7030 6357 0 -1 4202496 110822 0 0 0 52789 118 0 0 20 0 8 0 390978030 224837632 30511 33554432000 134512640 146736320 4286581600 4286571656 136000412 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0 0 0
/proc/7031/statm: 54892 30511 970 2985 0 51826 0
[pid=7031/tid=7047] ppid=7030 vsize=219568 CPUtime=61.04 cores=1,3,5,7

################
# More data... #
################

/proc/7031/statm: 57201 33313 977 2985 0 54135 0
[pid=7031/tid=7047] ppid=7030 vsize=228804 CPUtime=90.76 cores=1,3,5,7
/proc/7031/task/7047/stat : 7047 (pb_cplex) R 7030 7030 6357 0 -1 4202560 12558 0 0 0 9062 14 0 0 20 0 8 0 390980655 234295296 33313 33554432000 134512640 146736320 4286581600 4069510544 136403112 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=7031/tid=7048] ppid=7030 vsize=228804 CPUtime=88.58 cores=1,3,5,7
/proc/7031/task/7048/stat : 7048 (pb_cplex) R 7030 7030 6357 0 -1 4202560 16443 0 0 0 8840 18 0 0 20 0 8 0 390980655 234295296 33313 33554432000 134512640 146736320 4286581600 4136886348 136388877 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=7031/tid=7049] ppid=7030 vsize=228804 CPUtime=89.61 cores=1,3,5,7
/proc/7031/task/7049/stat : 7049 (pb_cplex) R 7030 7030 6357 0 -1 4202560 18406 0 0 0 8942 19 0 0 20 0 8 0 390980655 234295296 33313 33554432000 134512640 146736320 4286581600 4147371976 139042902 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=7031/tid=7050] ppid=7030 vsize=228804 CPUtime=86.28 cores=1,3,5,7
/proc/7031/task/7050/stat : 7050 (pb_cplex) R 7030 7030 6357 0 -1 4202560 15595 0 0 0 8610 18 0 0 20 0 8 0 390980655 234295296 33313 33554432000 134512640 146736320 4286581600 4117364816 136008039 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=7031/tid=7051] ppid=7030 vsize=228804 CPUtime=87.57 cores=1,3,5,7
/proc/7031/task/7051/stat : 7051 (pb_cplex) R 7030 7030 6357 0 -1 4202560 16900 0 0 0 8739 18 0 0 20 0 8 0 390980655 234295296 33313 33554432000 134512640 146736320 4286581600 4059025960 139044552 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=7031/tid=7052] ppid=7030 vsize=228804 CPUtime=87.21 cores=1,3,5,7
/proc/7031/task/7052/stat : 7052 (pb_cplex) R 7030 7030 6357 0 -1 4202560 4202 0 0 0 8705 16 0 0 20 0 8 0 390980655 234295296 33313 33554432000 134512640 146736320 4286581600 4048542528 137701306 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=7031/tid=7053] ppid=7030 vsize=228804 CPUtime=88.78 cores=1,3,5,7
/proc/7031/task/7053/stat : 7053 (pb_cplex) R 7030 7030 6357 0 -1 4202560 15626 0 0 0 8863 15 0 0 20 0 8 0 390980655 234295296 33313 33554432000 134512640 146736320 4286581600 4038054352 136403117 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
Current children cumulated CPU time (s) 768.09
Current children cumulated vsize (KiB) 237952

[startup+282.3 s]
/proc/loadavg: 6.03 6.93 5.67 10/186 7057
/proc/meminfo: memFree=27915084/32873844 swapFree=7900/7900
[pid=7030] ppid=7028 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/7030/stat : 7030 (run) S 7028 7030 6357 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390978030 9367552 270 33554432000 4194304 5064252 140733990401536 140733990400448 225444521550 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/7030/statm: 2287 270 229 213 0 72 0
[pid=7031] ppid=7030 vsize=207920 CPUtime=951.62 cores=1,3,5,7
/proc/7031/stat : 7031 (pb_cplex) R 7030 7030 6357 0 -1 4202496 163311 0 0 0 94987 175 0 0 20 0 8 0 390978030 212910080 29438 33554432000 134512640 146736320 4286581600 4286574312 139044581 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0 0 0
/proc/7031/statm: 51980 29438 977 2985 0 48914 0
[pid=7031/tid=7047] ppid=7030 vsize=207920 CPUtime=141.42 cores=1,3,5,7
/proc/7031/task/7047/stat : 7047 (pb_cplex) R 7030 7030 6357 0 -1 4202560 13830 0 0 0 14122 20 0 0 20 0 8 0 390980655 212910080 29438 33554432000 134512640 146736320 4286581600 4069515472 136008230 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=7031/tid=7048] ppid=7030 vsize=207920 CPUtime=104.89 cores=1,3,5,7
/proc/7031/task/7048/stat : 7048 (pb_cplex) R 7030 7030 6357 0 -1 4202560 19657 0 0 0 10468 21 0 0 20 0 8 0 390980655 212910080 29438 33554432000 134512640 146736320 4286581600 4136891368 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=7031/tid=7049] ppid=7030 vsize=207920 CPUtime=96.92 cores=1,3,5,7
/proc/7031/task/7049/stat : 7049 (pb_cplex) R 7030 7030 6357 0 -1 4202560 18462 0 0 0 9671 21 0 0 20 0 8 0 390980655 212910080 29438 33554432000 134512640 146736320 4286581600 4147377128 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=7031/tid=7050] ppid=7030 vsize=207920 CPUtime=90.36 cores=1,3,5,7
/proc/7031/task/7050/stat : 7050 (pb_cplex) R 7030 7030 6357 0 -1 4202560 15660 0 0 0 9017 19 0 0 20 0 8 0 390980655 212910080 29438 33554432000 134512640 146736320 4286581600 4117369040 136008356 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=7031/tid=7051] ppid=7030 vsize=207920 CPUtime=130.71 cores=1,3,5,7
/proc/7031/task/7051/stat : 7051 (pb_cplex) R 7030 7030 6357 0 -1 4202560 20316 0 0 0 13047 24 0 0 20 0 8 0 390980655 212910080 29438 33554432000 134512640 146736320 4286581600 4059030504 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=7031/tid=7052] ppid=7030 vsize=207920 CPUtime=105.2 cores=1,3,5,7
/proc/7031/task/7052/stat : 7052 (pb_cplex) R 7030 7030 6357 0 -1 4202560 4253 0 0 0 10502 18 0 0 20 0 8 0 390980655 212910080 29438 33554432000 134512640 146736320 4286581600 4048544336 136403109 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=7031/tid=7053] ppid=7030 vsize=207920 CPUtime=125.86 cores=1,3,5,7
/proc/7031/task/7053/stat : 7053 (pb_cplex) R 7030 7030 6357 0 -1 4202560 23766 0 0 0 12564 22 0 0 20 0 8 0 390980655 212910080 29438 33554432000 134512640 146736320 4286581600 4038058376 136388078 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 951.62
Current children cumulated vsize (KiB) 217068

[startup+342.3 s]
/proc/loadavg: 6.04 6.83 5.73 10/186 7057
/proc/meminfo: memFree=27912708/32873844 swapFree=7900/7900
[pid=7030] ppid=7028 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/7030/stat : 7030 (run) S 7028 7030 6357 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390978030 9367552 270 33554432000 4194304 5064252 140733990401536 140733990400448 225444521550 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/7030/statm: 2287 270 229 213 0 72 0
[pid=7031] ppid=7030 vsize=208136 CPUtime=1163.09 cores=1,3,5,7
/proc/7031/stat : 7031 (pb_cplex) S 7030 7030 6357 0 -1 4202496 167779 0 0 0 116085 224 0 0 20 0 8 0 390978030 213131264 29487 33554432000 134512640 146736320 4286581600 4286578000 144580073 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 0 0 0
/proc/7031/statm: 52034 29487 977 2985 0 48968 0
[pid=7031/tid=7047] ppid=7030 vsize=208136 CPUtime=168.61 cores=1,3,5,7
/proc/7031/task/7047/stat : 7047 (pb_cplex) R 7030 7030 6357 0 -1 4202560 13831 0 0 0 16835 26 0 0 20 0 8 0 390980655 213131264 29487 33554432000 134512640 146736320 4286581600 4069516264 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=7031/tid=7048] ppid=7030 vsize=208136 CPUtime=130.96 cores=1,3,5,7
/proc/7031/task/7048/stat : 7048 (pb_cplex) R 7030 7030 6357 0 -1 4202560 20468 0 0 0 13069 27 0 0 20 0 8 0 390980655 213131264 29487 33554432000 134512640 146736320 4286581600 4136890460 135854509 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=7031/tid=7049] ppid=7030 vsize=208136 CPUtime=123.39 cores=1,3,5,7
/proc/7031/task/7049/stat : 7049 (pb_cplex) S 7030 7030 6357 0 -1 4202560 18781 0 0 0 12313 26 0 0 20 0 8 0 390980655 213131264 29487 33554432000 134512640 146736320 4286581600 4147379856 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=7031/tid=7050] ppid=7030 vsize=208136 CPUtime=117.5 cores=1,3,5,7
/proc/7031/task/7050/stat : 7050 (pb_cplex) R 7030 7030 6357 0 -1 4202560 15661 0 0 0 11724 26 0 0 20 0 8 0 390980655 213131264 29487 33554432000 134512640 146736320 4286581600 4117368872 139044552 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=7031/tid=7051] ppid=7030 vsize=208136 CPUtime=156.78 cores=1,3,5,7
/proc/7031/task/7051/stat : 7051 (pb_cplex) R 7030 7030 6357 0 -1 4202560 20391 0 0 0 15647 31 0 0 20 0 8 0 390980655 213131264 29487 33554432000 134512640 146736320 4286581600 4059030504 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=7031/tid=7052] ppid=7030 vsize=208136 CPUtime=131.42 cores=1,3,5,7
/proc/7031/task/7052/stat : 7052 (pb_cplex) R 7030 7030 6357 0 -1 4202560 4254 0 0 0 13117 25 0 0 20 0 8 0 390980655 213131264 29487 33554432000 134512640 146736320 4286581600 4048544744 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=7031/tid=7053] ppid=7030 vsize=208136 CPUtime=152 cores=1,3,5,7
/proc/7031/task/7053/stat : 7053 (pb_cplex) R 7030 7030 6357 0 -1 4202560 25279 0 0 0 15172 28 0 0 20 0 8 0 390980655 213131264 29487 33554432000 134512640 146736320 4286581600 4038058192 136008438 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 1163.09
Current children cumulated vsize (KiB) 217284

[startup+402.3 s]
/proc/loadavg: 7.23 7.05 5.86 10/186 7057
/proc/meminfo: memFree=27906752/32873844 swapFree=7900/7900
[pid=7030] ppid=7028 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/7030/stat : 7030 (run) S 7028 7030 6357 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390978030 9367552 270 33554432000 4194304 5064252 140733990401536 140733990400448 225444521550 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/7030/statm: 2287 270 229 213 0 72 0
[pid=7031] ppid=7030 vsize=213184 CPUtime=1380.03 cores=1,3,5,7
/proc/7031/stat : 7031 (pb_cplex) S 7030 7030 6357 0 -1 4202496 178920 0 0 0 137731 272 0 0 20 0 8 0 390978030 218300416 30589 33554432000 134512640 146736320 4286581600 4286578000 144580073 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/7031/statm: 53296 30589 977 2985 0 50230 0
[pid=7031/tid=7047] ppid=7030 vsize=213184 CPUtime=195.62 cores=1,3,5,7
/proc/7031/task/7047/stat : 7047 (pb_cplex) R 7030 7030 6357 0 -1 4202560 13831 0 0 0 19530 32 0 0 20 0 8 0 390980655 218300416 30589 33554432000 134512640 146736320 4286581600 4069515472 136008450 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=7031/tid=7048] ppid=7030 vsize=213184 CPUtime=158.71 cores=1,3,5,7
/proc/7031/task/7048/stat : 7048 (pb_cplex) S 7030 7030 6357 0 -1 4202560 23123 0 0 0 15837 34 0 0 20 0 8 0 390980655 218300416 30589 33554432000 134512640 146736320 4286581600 4136894096 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 5 0 0 0 0 0
[pid=7031/tid=7049] ppid=7030 vsize=213184 CPUtime=150.36 cores=1,3,5,7
/proc/7031/task/7049/stat : 7049 (pb_cplex) R 7030 7030 6357 0 -1 4202560 20298 0 0 0 15003 33 0 0 20 0 8 0 390980655 218300416 30589 33554432000 134512640 146736320 4286581600 4147376336 136008061 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=7031/tid=7050] ppid=7030 vsize=213184 CPUtime=145.07 cores=1,3,5,7
/proc/7031/task/7050/stat : 7050 (pb_cplex) R 7030 7030 6357 0 -1 4202560 15756 0 0 0 14473 34 0 0 20 0 8 0 390980655 218300416 30589 33554432000 134512640 146736320 4286581600 4117369424 136403112 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=7031/tid=7051] ppid=7030 vsize=213184 CPUtime=183.91 cores=1,3,5,7
/proc/7031/task/7051/stat : 7051 (pb_cplex) R 7030 7030 6357 0 -1 4202560 20391 0 0 0 18354 37 0 0 20 0 8 0 390980655 218300416 30589 33554432000 134512640 146736320 4286581600 4059029712 136008065 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=7031/tid=7052] ppid=7030 vsize=213184 CPUtime=158.26 cores=1,3,5,7
/proc/7031/task/7052/stat : 7052 (pb_cplex) R 7030 7030 6357 0 -1 4202560 4959 0 0 0 15796 30 0 0 20 0 8 0 390980655 218300416 30589 33554432000 134512640 146736320 4286581600 4048543636 136387645 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=7031/tid=7053] ppid=7030 vsize=213184 CPUtime=178.89 cores=1,3,5,7
/proc/7031/task/7053/stat : 7053 (pb_cplex) R 7030 7030 6357 0 -1 4202560 26083 0 0 0 17855 34 0 0 20 0 8 0 390980655 218300416 30589 33554432000 134512640 146736320 4286581600 4038057800 135525953 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 1380.03
Current children cumulated vsize (KiB) 222332

[startup+462.3 s]
/proc/loadavg: 7.64 7.20 6.00 3/186 7057
/proc/meminfo: memFree=27901844/32873844 swapFree=7900/7900
[pid=7030] ppid=7028 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/7030/stat : 7030 (run) S 7028 7030 6357 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390978030 9367552 270 33554432000 4194304 5064252 140733990401536 140733990400448 225444521550 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/7030/statm: 2287 270 229 213 0 72 0
[pid=7031] ppid=7030 vsize=214120 CPUtime=1598.23 cores=1,3,5,7
/proc/7031/stat : 7031 (pb_cplex) R 7030 7030 6357 0 -1 4202496 185069 0 0 0 159504 319 0 0 20 0 8 0 390978030 219258880 30907 33554432000 134512640 146736320 4286581600 4286574344 139042974 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0 0 0
/proc/7031/statm: 53530 30907 977 2985 0 50464 0
[pid=7031/tid=7047] ppid=7030 vsize=214120 CPUtime=223.38 cores=1,3,5,7
/proc/7031/task/7047/stat : 7047 (pb_cplex) R 7030 7030 6357 0 -1 4202560 13831 0 0 0 22300 38 0 0 20 0 8 0 390980655 219258880 30907 33554432000 134512640 146736320 4286581600 4069515856 136403112 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
[pid=7031/tid=7048] ppid=7030 vsize=214120 CPUtime=186.29 cores=1,3,5,7
/proc/7031/task/7048/stat : 7048 (pb_cplex) R 7030 7030 6357 0 -1 4202560 24628 0 0 0 18587 42 0 0 20 0 8 0 390980655 219258880 30907 33554432000 134512640 146736320 4286581600 4136891368 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=7031/tid=7049] ppid=7030 vsize=214120 CPUtime=177.76 cores=1,3,5,7
/proc/7031/task/7049/stat : 7049 (pb_cplex) R 7030 7030 6357 0 -1 4202560 20323 0 0 0 17738 38 0 0 20 0 8 0 390980655 219258880 30907 33554432000 134512640 146736320 4286581600 4147376584 136000488 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=7031/tid=7050] ppid=7030 vsize=214120 CPUtime=171.99 cores=1,3,5,7
/proc/7031/task/7050/stat : 7050 (pb_cplex) S 7030 7030 6357 0 -1 4202560 17467 0 0 0 17159 40 0 0 20 0 8 0 390980655 219258880 30907 33554432000 134512640 146736320 4286581600 4117372560 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 3 0 0 0 0 0
[pid=7031/tid=7051] ppid=7030 vsize=214120 CPUtime=211.38 cores=1,3,5,7
/proc/7031/task/7051/stat : 7051 (pb_cplex) R 7030 7030 6357 0 -1 4202560 20440 0 0 0 21093 45 0 0 20 0 8 0 390980655 219258880 30907 33554432000 134512640 146736320 4286581600 4059029704 136388550 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=7031/tid=7052] ppid=7030 vsize=214120 CPUtime=185.69 cores=1,3,5,7
/proc/7031/task/7052/stat : 7052 (pb_cplex) R 7030 7030 6357 0 -1 4202560 4979 0 0 0 18534 35 0 0 20 0 8 0 390980655 219258880 30907 33554432000 134512640 146736320 4286581600 4048543112 136000333 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=7031/tid=7053] ppid=7030 vsize=214120 CPUtime=205.82 cores=1,3,5,7
/proc/7031/task/7053/stat : 7053 (pb_cplex) R 7030 7030 6357 0 -1 4202560 26090 0 0 0 20542 40 0 0 20 0 8 0 390980655 219258880 30907 33554432000 134512640 146736320 4286581600 4038058192 136008309 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
Current children cumulated CPU time (s) 1598.23
Current children cumulated vsize (KiB) 223268



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+519.4 s]
/proc/loadavg: 7.04 7.09 6.02 10/186 7057
/proc/meminfo: memFree=27899488/32873844 swapFree=7900/7900
[pid=7030] ppid=7028 vsize=9148 CPUtime=0 cores=1,3,5,7
/proc/7030/stat : 7030 (run) S 7028 7030 6357 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390978030 9367552 270 33554432000 4194304 5064252 140733990401536 140733990400448 225444521550 0 65536 4100 65538 18446744071579302229 0 0 17 1 0 0 0 0 0
/proc/7030/statm: 2287 270 229 213 0 72 0
[pid=7031] ppid=7030 vsize=217036 CPUtime=1800.03 cores=1,3,5,7
/proc/7031/stat : 7031 (pb_cplex) R 7030 7030 6357 0 -1 4202496 192865 0 0 0 179638 365 0 0 20 0 8 0 390978030 222244864 31580 33554432000 134512640 146736320 4286581600 4286574864 136403112 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 0 0 0
/proc/7031/statm: 54259 31580 977 2985 0 51193 0
[pid=7031/tid=7047] ppid=7030 vsize=217036 CPUtime=248.3 cores=1,3,5,7
/proc/7031/task/7047/stat : 7047 (pb_cplex) R 7030 7030 6357 0 -1 4202560 13831 0 0 0 24787 43 0 0 20 0 8 0 390980655 222244864 31580 33554432000 134512640 146736320 4286581600 4069515856 136403109 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=7031/tid=7048] ppid=7030 vsize=217036 CPUtime=211.76 cores=1,3,5,7
/proc/7031/task/7048/stat : 7048 (pb_cplex) R 7030 7030 6357 0 -1 4202560 28223 0 0 0 21128 48 0 0 20 0 8 0 390980655 222244864 31580 33554432000 134512640 146736320 4286581600 4136889536 135468687 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=7031/tid=7049] ppid=7030 vsize=217036 CPUtime=203.18 cores=1,3,5,7
/proc/7031/task/7049/stat : 7049 (pb_cplex) R 7030 7030 6357 0 -1 4202560 20486 0 0 0 20273 45 0 0 20 0 8 0 390980655 222244864 31580 33554432000 134512640 146736320 4286581600 4147377128 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0 0 0
[pid=7031/tid=7050] ppid=7030 vsize=217036 CPUtime=196.94 cores=1,3,5,7
/proc/7031/task/7050/stat : 7050 (pb_cplex) R 7030 7030 6357 0 -1 4202560 17467 0 0 0 19649 45 0 0 20 0 8 0 390980655 222244864 31580 33554432000 134512640 146736320 4286581600 4117368924 135854509 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=7031/tid=7051] ppid=7030 vsize=217036 CPUtime=235.98 cores=1,3,5,7
/proc/7031/task/7051/stat : 7051 (pb_cplex) R 7030 7030 6357 0 -1 4202560 21462 0 0 0 23547 51 0 0 20 0 8 0 390980655 222244864 31580 33554432000 134512640 146736320 4286581600 4059029896 136388081 0 2 4096 16387 0 0 0 -1 7 0 0 0 0 0
[pid=7031/tid=7052] ppid=7030 vsize=217036 CPUtime=211.92 cores=1,3,5,7
/proc/7031/task/7052/stat : 7052 (pb_cplex) R 7030 7030 6357 0 -1 4202560 4979 0 0 0 21150 42 0 0 20 0 8 0 390980655 222244864 31580 33554432000 134512640 146736320 4286581600 4048543952 136008441 0 2 4096 16387 0 0 0 -1 5 0 0 0 0 0
[pid=7031/tid=7053] ppid=7030 vsize=217036 CPUtime=231.3 cores=1,3,5,7
/proc/7031/task/7053/stat : 7053 (pb_cplex) R 7030 7030 6357 0 -1 4202560 26096 0 0 0 23083 47 0 0 20 0 8 0 390980655 222244864 31580 33554432000 134512640 146736320 4286581600 4038058984 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0 0 0
Current children cumulated CPU time (s) 1800.03
Current children cumulated vsize (KiB) 226184

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 7030 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=0
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=999
# CPU time returned by wait4() is 0.000999
# while last known CPU time is 1800.03
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 519.417
CPU time (s): 1800.03
CPU user time (s): 1796.38
CPU system time (s): 3.65
CPU usage (%): 346.548
Max. virtual memory (cumulated for all children) (KiB): 242100

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0
system time used= 0.000999
maximum resident set size= 1080
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 356
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2
involuntary context switches= 1

runsolver used 1.50977 second user time and 3.72743 second system time

The end

Launcher Data 

Begin job on node137 at 2012-06-09 15:28:55
IDJOB=3737150
IDBENCH=48187
IDSOLVER=2325
FILE ID=node137/3737150-1339248535
RUNJOBID= node137-1339246832-6374
PBS_JOBID= 14644344
Free space on /tmp= 70880 MiB

SOLVER NAME= PB10: pb_cplex 2010-06-29
BENCH NAME= PB07/normalized-PB07/OPT-SMALLINT-NLC/submittedPB07/manquinho/mis/normalized-mis_500_10_3.opb
COMMAND LINE= run BENCHNAME
CONVERSION SCRIPT= PBconversionToLinear BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3737150-1339248535/watcher-3737150-1339248535 -o /tmp/evaluation-result-3737150-1339248535/solver-3737150-1339248535 -C 1800 -W 1900 -M 15500  run HOME/instance-3737150-1339248535.opb

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 0247ef4de041bbff376811a7e6bf0408
RANDOM SEED=1696303406

node137.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.600
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.20
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.600
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.600
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.600
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.600
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.600
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.600
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.600
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        28034012 kB
Buffers:          575736 kB
Cached:          2750508 kB
SwapCached:         4640 kB
Active:          2134480 kB
Inactive:        1794780 kB
Active(anon):     589960 kB
Inactive(anon):     5856 kB
Active(file):    1544520 kB
Inactive(file):  1788924 kB
Unevictable:        7900 kB
Mlocked:            7900 kB
SwapTotal:      67108856 kB
SwapFree:       67097908 kB
Dirty:             16396 kB
Writeback:             0 kB
AnonPages:        600308 kB
Mapped:            13212 kB
Shmem:                 0 kB
Slab:             765528 kB
SReclaimable:     206220 kB
SUnreclaim:       559308 kB
KernelStack:        1544 kB
PageTables:         5436 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     844156 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:    518144 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

CONVERSION COMMAND LINE= PBconversionToLinear HOME/instance-3737150-1339248535.opb

Free space on /tmp at the end= 70864 MiB
End job on node137 at 2012-06-09 15:37:35