Trace number 3733974

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
PB10: pb_cplex 2010-06-29? (TO) 1800.27 578.317

General information on the benchmark

Namenormalized-PB06/OPT-SMALLINT/submitted-PB06/
manquiho/golomb-rulers/normalized-OGR_20.opb
MD5SUM6a25ee38adc43f783b18badd1224270c
Bench CategoryOPT-SMALLINT (optimisation, small integers)
Best result obtained on this benchmark
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Has Objective FunctionYES
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables13480
Total number of constraints26619
Number of constraints which are clauses0
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints26619
Minimum length of a constraint18
Maximum length of a constraint37
Number of terms in the objective function 9
Biggest coefficient in the objective function 256
Number of bits for the biggest coefficient in the objective function 9
Sum of the numbers in the objective function 511
Number of bits of the sum of numbers in the objective function 9
Biggest number in a constraint 512
Number of bits of the biggest number in a constraint 10
Biggest sum of numbers in a constraint 2843
Number of bits of the biggest sum of numbers12
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c pb_cplex 1.0 - 20100701
0.00/0.00	c Parsing opb file format. File HOME/instance-3733974-1339247497.opb.
0.00/0.00	c Instance file HOME/instance-3733974-1339247497.opb
0.00/0.00	c File size is 7804237 bytes.
3.69/3.77	c Highest Coefficient sum: 2843
3.69/3.77	c  IBM ILOG License Manager: "IBM ILOG Optimization Suite for Academic Initiative" is accessing CPLEX 12 with option(s): "e m b q ".
3.79/3.82	c ============================[ Problem Statistics ]=============================
3.79/3.82	c |                                                                             |
3.79/3.82	c |  Parsing time:          3.80         s                                      |
3.79/3.82	c |  Number Variables:      13480                                               |
3.79/3.85	c |  Number Constraints:    26619                                               |
3.79/3.85	c ===============================================================================
1800.27/578.31	c Remaining time 1796.18 sec.
1800.27/578.31	c 
1800.27/578.31	c *** INTERRUPTED ***
1800.27/578.31	s UNKNOWN
1800.27/578.31	c Total CPU time              : 1797.85 s
1800.27/578.31	c Memory used           : 1626.91 MB
1800.27/578.31	c CPU time              : 1797.85 s
1800.27/578.31	c 
1800.27/578.31	c *** INTERRUPTED ***

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3733974-1339247497/watcher-3733974-1339247497 -o /tmp/evaluation-result-3733974-1339247497/solver-3733974-1339247497 -C 1800 -W 1900 -M 15500 run HOME/instance-3733974-1339247497.opb 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 10.67 9.74 5.54 10/184 26702
/proc/meminfo: memFree=24987272/32873844 swapFree=7896/7896
[pid=26702] ppid=26700 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/26702/stat : 26702 (run) S 26700 26702 25626 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390875204 9367552 270 33554432000 4194304 5064252 140733971490288 140733971489200 239872927310 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/26702/statm: 2287 270 229 213 0 72 0

[startup+0.083499 s]
/proc/loadavg: 10.67 9.74 5.54 10/184 26702
/proc/meminfo: memFree=24987272/32873844 swapFree=7896/7896
[pid=26702] ppid=26700 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/26702/stat : 26702 (run) S 26700 26702 25626 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390875204 9367552 270 33554432000 4194304 5064252 140733971490288 140733971489200 239872927310 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/26702/statm: 2287 270 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.100254 s]
/proc/loadavg: 10.67 9.74 5.54 10/184 26702
/proc/meminfo: memFree=24987272/32873844 swapFree=7896/7896
[pid=26702] ppid=26700 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/26702/stat : 26702 (run) S 26700 26702 25626 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390875204 9367552 270 33554432000 4194304 5064252 140733971490288 140733971489200 239872927310 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/26702/statm: 2287 270 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.30023 s]
/proc/loadavg: 10.67 9.74 5.54 10/184 26702
/proc/meminfo: memFree=24987272/32873844 swapFree=7896/7896
[pid=26702] ppid=26700 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/26702/stat : 26702 (run) S 26700 26702 25626 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390875204 9367552 270 33554432000 4194304 5064252 140733971490288 140733971489200 239872927310 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/26702/statm: 2287 270 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.700228 s]
/proc/loadavg: 10.67 9.74 5.54 10/184 26702
/proc/meminfo: memFree=24987272/32873844 swapFree=7896/7896
[pid=26702] ppid=26700 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/26702/stat : 26702 (run) S 26700 26702 25626 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390875204 9367552 270 33554432000 4194304 5064252 140733971490288 140733971489200 239872927310 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/26702/statm: 2287 270 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+1.50026 s]
/proc/loadavg: 10.67 9.74 5.54 10/186 26704
/proc/meminfo: memFree=24986432/32873844 swapFree=7896/7896
[pid=26702] ppid=26700 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/26702/stat : 26702 (run) S 26700 26702 25626 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390875204 9367552 270 33554432000 4194304 5064252 140733971490288 140733971489200 239872927310 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/26702/statm: 2287 270 229 213 0 72 0
[pid=26703] ppid=26702 vsize=28260 CPUtime=1.49 cores=0,2,4,6
/proc/26703/stat : 26703 (pb_cplex) R 26702 26702 25626 0 -1 4202496 3105 0 0 0 149 0 0 0 20 0 1 0 390875204 28938240 3059 33554432000 134512640 146736320 4293612720 4293610672 134542843 0 0 4096 16387 0 0 0 17 2 0 0 0 0 0
/proc/26703/statm: 7065 3059 1035 2985 0 2093 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 37408

[startup+3.10031 s]
/proc/loadavg: 10.67 9.74 5.54 10/186 26704
/proc/meminfo: memFree=24976636/32873844 swapFree=7896/7896
[pid=26702] ppid=26700 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/26702/stat : 26702 (run) S 26700 26702 25626 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390875204 9367552 270 33554432000 4194304 5064252 140733971490288 140733971489200 239872927310 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/26702/statm: 2287 270 229 213 0 72 0
[pid=26703] ppid=26702 vsize=36796 CPUtime=3.09 cores=0,2,4,6
/proc/26703/stat : 26703 (pb_cplex) R 26702 26702 25626 0 -1 4202496 6008 0 0 0 309 0 0 0 20 0 1 0 390875204 37679104 5929 33554432000 134512640 146736320 4293612720 4293610520 144716523 0 0 4096 16387 0 0 0 17 2 0 0 0 0 0
/proc/26703/statm: 9199 5929 1810 2985 0 4227 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 45944

[startup+6.3003 s]
/proc/loadavg: 10.53 9.72 5.56 10/186 26704
/proc/meminfo: memFree=24884348/32873844 swapFree=7896/7896
[pid=26702] ppid=26700 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/26702/stat : 26702 (run) S 26700 26702 25626 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390875204 9367552 270 33554432000 4194304 5064252 140733971490288 140733971489200 239872927310 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/26702/statm: 2287 270 229 213 0 72 0
[pid=26703] ppid=26702 vsize=150272 CPUtime=6.28 cores=0,2,4,6
/proc/26703/stat : 26703 (pb_cplex) R 26702 26702 25626 0 -1 4202496 34849 0 0 0 622 6 0 0 20 0 1 0 390875204 153878528 26593 33554432000 134512640 146736320 4293612720 4293607504 136008364 0 0 4096 16387 0 0 0 17 4 0 0 0 0 0
/proc/26703/statm: 37568 26593 788 2985 0 34502 0
Current children cumulated CPU time (s) 6.28
Current children cumulated vsize (KiB) 159420

[startup+12.7003 s]
/proc/loadavg: 10.33 9.70 5.58 9/186 26704
/proc/meminfo: memFree=24846660/32873844 swapFree=7896/7896
[pid=26702] ppid=26700 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/26702/stat : 26702 (run) S 26700 26702 25626 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390875204 9367552 270 33554432000 4194304 5064252 140733971490288 140733971489200 239872927310 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/26702/statm: 2287 270 229 213 0 72 0
[pid=26703] ppid=26702 vsize=160964 CPUtime=12.68 cores=0,2,4,6
/proc/26703/stat : 26703 (pb_cplex) R 26702 26702 25626 0 -1 4202496 43413 0 0 0 1260 8 0 0 20 0 1 0 390875204 164827136 29116 33554432000 134512640 146736320 4293612720 4293608376 137176693 0 0 4096 16387 0 0 0 17 4 0 0 0 0 0
/proc/26703/statm: 40241 29116 858 2985 0 37175 0
Current children cumulated CPU time (s) 12.68
Current children cumulated vsize (KiB) 170112

[startup+25.5003 s]
/proc/loadavg: 9.81 9.61 5.61 9/186 26704
/proc/meminfo: memFree=24856272/32873844 swapFree=7896/7896
[pid=26702] ppid=26700 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/26702/stat : 26702 (run) S 26700 26702 25626 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390875204 9367552 270 33554432000 4194304 5064252 140733971490288 140733971489200 239872927310 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/26702/statm: 2287 270 229 213 0 72 0
[pid=26703] ppid=26702 vsize=146284 CPUtime=25.48 cores=0,2,4,6
/proc/26703/stat : 26703 (pb_cplex) R 26702 26702 25626 0 -1 4202496 81849 0 0 0 2531 17 0 0 20 0 1 0 390875204 149794816 27538 33554432000 134512640 146736320 4293612720 4293609120 136207929 0 0 4096 16387 0 0 0 17 4 0 0 0 0 0
/proc/26703/statm: 36571 27538 888 2985 0 33505 0
Current children cumulated CPU time (s) 25.48
Current children cumulated vsize (KiB) 155432

[startup+51.1003 s]
/proc/loadavg: 8.96 9.43 5.66 7/186 26705
/proc/meminfo: memFree=24796712/32873844 swapFree=7896/7896
[pid=26702] ppid=26700 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/26702/stat : 26702 (run) S 26700 26702 25626 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390875204 9367552 270 33554432000 4194304 5064252 140733971490288 140733971489200 239872927310 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/26702/statm: 2287 270 229 213 0 72 0
[pid=26703] ppid=26702 vsize=207860 CPUtime=51.07 cores=0,2,4,6
/proc/26703/stat : 26703 (pb_cplex) R 26702 26702 25626 0 -1 4202496 117070 0 0 0 5080 27 0 0 20 0 1 0 390875204 212848640 41716 33554432000 134512640 146736320 4293612720 4293607032 136015570 0 0 4096 16387 0 0 0 17 4 0 0 0 0 0
/proc/26703/statm: 51965 41716 888 2985 0 48899 0
Current children cumulated CPU time (s) 51.07
Current children cumulated vsize (KiB) 217008

[startup+102.307 s]
/proc/loadavg: 6.27 8.64 5.58 4/186 26705
/proc/meminfo: memFree=24823300/32873844 swapFree=7896/7896
[pid=26702] ppid=26700 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/26702/stat : 26702 (run) S 26700 26702 25626 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390875204 9367552 270 33554432000 4194304 5064252 140733971490288 140733971489200 239872927310 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/26702/statm: 2287 270 229 213 0 72 0
[pid=26703] ppid=26702 vsize=217240 CPUtime=102.26 cores=0,2,4,6
/proc/26703/stat : 26703 (pb_cplex) R 26702 26702 25626 0 -1 4202496 177899 0 0 0 10179 47 0 0 20 0 1 0 390875204 222453760 43190 33554432000 134512640 146736320 4293612720 4293607208 136015572 0 0 4096 16387 0 0 0 17 4 0 0 0 0 0
/proc/26703/statm: 54310 43190 931 2985 0 51244 0
Current children cumulated CPU time (s) 102.26
Current children cumulated vsize (KiB) 226388

[startup+162.301 s]
/proc/loadavg: 5.46 7.87 5.51 13/193 26712
/proc/meminfo: memFree=24328280/32873844 swapFree=7896/7896
[pid=26702] ppid=26700 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/26702/stat : 26702 (run) S 26700 26702 25626 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390875204 9367552 270 33554432000 4194304 5064252 140733971490288 140733971489200 239872927310 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/26702/statm: 2287 270 229 213 0 72 0
[pid=26703] ppid=26702 vsize=972992 CPUtime=164.91 cores=0,2,4,6
/proc/26703/stat : 26703 (pb_cplex) R 26702 26702 25626 0 -1 4202496 276789 0 0 0 16419 72 0 0 20 0 8 0 390875204 996343808 185856 33554432000 134512640 146736320 4293612720 4293605584 136008039 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/26703/statm: 243248 185856 963 2985 0 240182 0
[pid=26703/tid=26706] ppid=26702 vsize=972992 CPUtime=0.56 cores=0,2,4,6
/proc/26703/task/26706/stat : 26706 (pb_cplex) R 26702 26702 25626 0 -1 4202560 4835 0 0 0 54 2 0 0 20 0 8 0 390887066 996343808 185856 33554432000 134512640 146736320 4293612720 3650060360 139042977 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=26703/tid=26707] ppid=26702 vsize=972992 CPUtime=0 cores=0,2,4,6
/proc/26703/task/26707/stat : 26707 (pb_cplex) S 26702 26702 25626 0 -1 4202560 263 0 0 0 0 0 0 0 20 0 8 0 390887066 996343808 185856 33554432000 134512640 146736320 4293612720 3639578256 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=26703/tid=26708] ppid=26702 vsize=972992 CPUtime=0.53 cores=0,2,4,6
/proc/26703/task/26708/stat : 26708 (pb_cplex) R 26702 26702 25626 0 -1 4202560 4081 0 0 0 51 2 0 0 20 0 8 0 390887066 996343808 185856 33554432000 134512640 146736320 4293612720 3629088976 136008450 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=26703/tid=26709] ppid=26702 vsize=972992 CPUtime=0.56 cores=0,2,4,6
/proc/26703/task/26709/stat : 26709 (pb_cplex) R 26702 26702 25626 0 -1 4202560 4258 0 0 0 54 2 0 0 20 0 8 0 390887066 996343808 185856 33554432000 134512640 146736320 4293612720 3618603216 136008065 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=26703/tid=26710] ppid=26702 vsize=972992 CPUtime=0.56 cores=0,2,4,6
/proc/26703/task/26710/stat : 26710 (pb_cplex) R 26702 26702 25626 0 -1 4202560 3621 0 0 0 54 2 0 0 20 0 8 0 390887066 996343808 185856 33554432000 134512640 146736320 4293612720 3608117372 135526778 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=26703/tid=26711] ppid=26702 vsize=972992 CPUtime=0 cores=0,2,4,6
/proc/26703/task/26711/stat : 26711 (pb_cplex) S 26702 26702 25626 0 -1 4202560 263 0 0 0 0 0 0 0 20 0 8 0 390887066 996343808 185856 33554432000 134512640 146736320 4293612720 3597635216 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=26703/tid=26712] ppid=26702 vsize=972992 CPUtime=0.59 cores=0,2,4,6
/proc/26703/task/26712/stat : 26712 (pb_cplex) R 26702 26702 25626 0 -1 4202560 4081 0 0 0 58 1 0 0 20 0 8 0 390887066 996343808 185856 33554432000 134512640 146736320 4293612720 3587145936 136008050 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
Current children cumulated CPU time (s) 164.91
Current children cumulated vsize (KiB) 982140

[startup+222.3 s]
/proc/loadavg: 8.53 8.34 5.82 15/193 26712
/proc/meminfo: memFree=24272636/32873844 swapFree=7896/7896
[pid=26702] ppid=26700 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/26702/stat : 26702 (run) S 26700 26702 25626 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390875204 9367552 270 33554432000 4194304 5064252 140733971490288 140733971489200 239872927310 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/26702/statm: 2287 270 229 213 0 72 0
[pid=26703] ppid=26702 vsize=1005284 CPUtime=404.2 cores=0,2,4,6
/proc/26703/stat : 26703 (pb_cplex) R 26702 26702 25626 0 -1 4202496 337043 0 0 0 40317 103 0 0 20 0 8 0 390875204 1029410816 193393 33554432000 134512640 146736320 4293612720 4293605968 136403109 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/26703/statm: 251321 193393 963 2985 0 248255 0
[pid=26703/tid=26706] ppid=26702 vsize=1005284 CPUtime=39.9 cores=0,2,4,6
/proc/26703/task/26706/stat : 26706 (pb_cplex) R 26702 26702 25626 0 -1 4202560 14220 0 0 0 3984 6 0 0 20 0 8 0 390887066 1029410816 193393 33554432000 134512640 146736320 4293612720 3650059792 136403116 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=26703/tid=26707] ppid=26702 vsize=1005284 CPUtime=0 cores=0,2,4,6
/proc/26703/task/26707/stat : 26707 (pb_cplex) S 26702 26702 25626 0 -1 4202560 263 0 0 0 0 0 0 0 20 0 8 0 390887066 1029410816 193393 33554432000 134512640 146736320 4293612720 3639578256 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=26703/tid=26708] ppid=26702 vsize=1005284 CPUtime=40.83 cores=0,2,4,6
/proc/26703/task/26708/stat : 26708 (pb_cplex) R 26702 26702 25626 0 -1 4202560 19464 0 0 0 4075 8 0 0 20 0 8 0 390887066 1029410816 193393 33554432000 134512640 146736320 4293612720 3629088976 136008242 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=26703/tid=26709] ppid=26702 vsize=1005284 CPUtime=40.9 cores=0,2,4,6
/proc/26703/task/26709/stat : 26709 (pb_cplex) R 26702 26702 25626 0 -1 4202560 11721 0 0 0 4084 6 0 0 20 0 8 0 390887066 1029410816 193393 33554432000 134512640 146736320 4293612720 3618603216 136008056 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=26703/tid=26710] ppid=26702 vsize=1005284 CPUtime=40.33 cores=0,2,4,6
/proc/26703/task/26710/stat : 26710 (pb_cplex) R 26702 26702 25626 0 -1 4202560 11941 0 0 0 4027 6 0 0 20 0 8 0 390887066 1029410816 193393 33554432000 134512640 146736320 4293612720 3608117288 139044505 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=26703/tid=26711] ppid=26702 vsize=1005284 CPUtime=0 cores=0,2,4,6
/proc/26703/task/26711/stat : 26711 (pb_cplex) S 26702 26702 25626 0 -1 4202560 263 0 0 0 0 0 0 0 20 0 8 0 390887066 1029410816 193393 33554432000 134512640 146736320 4293612720 3597635216 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=26703/tid=26712] ppid=26702 vsize=1005284 CPUtime=40.12 cores=0,2,4,6
/proc/26703/task/26712/stat : 26712 (pb_cplex) R 26702 26702 25626 0 -1 4202560 16856 0 0 0 4005 7 0 0 20 0 8 0 390887066 1029410816 193393 33554432000 134512640 146736320 4293612720 3587145768 139044550 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
Current children cumulated CPU time (s) 404.2
Current children cumulated vsize (KiB) 1014432


################
# More data... #
################

/proc/26703/statm: 409428 324020 963 2985 0 406362 0
[pid=26703/tid=26706] ppid=26702 vsize=1637712 CPUtime=118.43 cores=0,2,4,6
/proc/26703/task/26706/stat : 26706 (pb_cplex) R 26702 26702 25626 0 -1 4202560 50368 0 0 0 11820 23 0 0 20 0 8 0 390887066 1677017088 324020 33554432000 134512640 146736320 4293612720 3650060496 136008061 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=26703/tid=26707] ppid=26702 vsize=1637712 CPUtime=25.21 cores=0,2,4,6
/proc/26703/task/26707/stat : 26707 (pb_cplex) R 26702 26702 25626 0 -1 4202560 13541 0 0 0 2514 7 0 0 20 0 8 0 390887066 1677017088 324020 33554432000 134512640 146736320 4293612720 3639574736 136008454 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=26703/tid=26708] ppid=26702 vsize=1637712 CPUtime=118.27 cores=0,2,4,6
/proc/26703/task/26708/stat : 26708 (pb_cplex) R 26702 26702 25626 0 -1 4202560 46555 0 0 0 11803 24 0 0 20 0 8 0 390887066 1677017088 324020 33554432000 134512640 146736320 4293612720 3629088976 136008065 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=26703/tid=26709] ppid=26702 vsize=1637712 CPUtime=100.74 cores=0,2,4,6
/proc/26703/task/26709/stat : 26709 (pb_cplex) R 26702 26702 25626 0 -1 4202560 28527 0 0 0 10054 20 0 0 20 0 8 0 390887066 1677017088 324020 33554432000 134512640 146736320 4293612720 3618603216 136008065 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
[pid=26703/tid=26710] ppid=26702 vsize=1637712 CPUtime=92.41 cores=0,2,4,6
/proc/26703/task/26710/stat : 26710 (pb_cplex) R 26702 26702 25626 0 -1 4202560 31598 0 0 0 9223 18 0 0 20 0 8 0 390887066 1677017088 324020 33554432000 134512640 146736320 4293612720 3608117456 136008065 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=26703/tid=26711] ppid=26702 vsize=1637712 CPUtime=25.35 cores=0,2,4,6
/proc/26703/task/26711/stat : 26711 (pb_cplex) R 26702 26702 25626 0 -1 4202560 16257 0 0 0 2527 8 0 0 20 0 8 0 390887066 1677017088 324020 33554432000 134512640 146736320 4293612720 3597631696 136008047 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=26703/tid=26712] ppid=26702 vsize=1637712 CPUtime=113.39 cores=0,2,4,6
/proc/26703/task/26712/stat : 26712 (pb_cplex) R 26702 26702 25626 0 -1 4202560 39953 0 0 0 11316 23 0 0 20 0 8 0 390887066 1677017088 324020 33554432000 134512640 146736320 4293612720 3587145840 136384895 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 858.11
Current children cumulated vsize (KiB) 1646860

[startup+402.301 s]
/proc/loadavg: 14.41 10.85 7.21 17/193 26713
/proc/meminfo: memFree=23662580/32873844 swapFree=7896/7896
[pid=26702] ppid=26700 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/26702/stat : 26702 (run) S 26700 26702 25626 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390875204 9367552 270 33554432000 4194304 5064252 140733971490288 140733971489200 239872927310 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/26702/statm: 2287 270 229 213 0 72 0
[pid=26703] ppid=26702 vsize=1652616 CPUtime=1097.59 cores=0,2,4,6
/proc/26703/stat : 26703 (pb_cplex) R 26702 26702 25626 0 -1 4202496 525380 0 0 0 109548 211 0 0 20 0 8 0 390875204 1692278784 334141 33554432000 134512640 146736320 4293612720 4293605488 136384892 0 0 4096 16387 18446744073709551615 0 0 17 6 0 0 0 0 0
/proc/26703/statm: 413154 334141 963 2985 0 410088 0
[pid=26703/tid=26706] ppid=26702 vsize=1652616 CPUtime=148.19 cores=0,2,4,6
/proc/26703/task/26706/stat : 26706 (pb_cplex) R 26702 26702 25626 0 -1 4202560 52093 0 0 0 14793 26 0 0 20 0 8 0 390887066 1692278784 334141 33554432000 134512640 146736320 4293612720 3650060680 136388084 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=26703/tid=26707] ppid=26702 vsize=1652616 CPUtime=54.99 cores=0,2,4,6
/proc/26703/task/26707/stat : 26707 (pb_cplex) R 26702 26702 25626 0 -1 4202560 15593 0 0 0 5491 8 0 0 20 0 8 0 390887066 1692278784 334141 33554432000 134512640 146736320 4293612720 3639574736 136008042 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=26703/tid=26708] ppid=26702 vsize=1652616 CPUtime=148.41 cores=0,2,4,6
/proc/26703/task/26708/stat : 26708 (pb_cplex) R 26702 26702 25626 0 -1 4202560 48573 0 0 0 14814 27 0 0 20 0 8 0 390887066 1692278784 334141 33554432000 134512640 146736320 4293612720 3629089360 136403109 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=26703/tid=26709] ppid=26702 vsize=1652616 CPUtime=130.14 cores=0,2,4,6
/proc/26703/task/26709/stat : 26709 (pb_cplex) R 26702 26702 25626 0 -1 4202560 33692 0 0 0 12992 22 0 0 20 0 8 0 390887066 1692278784 334141 33554432000 134512640 146736320 4293612720 3618601860 135523433 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
[pid=26703/tid=26710] ppid=26702 vsize=1652616 CPUtime=122.22 cores=0,2,4,6
/proc/26703/task/26710/stat : 26710 (pb_cplex) R 26702 26702 25626 0 -1 4202560 33856 0 0 0 12202 20 0 0 20 0 8 0 390887066 1692278784 334141 33554432000 134512640 146736320 4293612720 3608117456 136008042 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=26703/tid=26711] ppid=26702 vsize=1652616 CPUtime=55.39 cores=0,2,4,6
/proc/26703/task/26711/stat : 26711 (pb_cplex) R 26702 26702 25626 0 -1 4202560 18336 0 0 0 5530 9 0 0 20 0 8 0 390887066 1692278784 334141 33554432000 134512640 146736320 4293612720 3597631712 139028313 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=26703/tid=26712] ppid=26702 vsize=1652616 CPUtime=143.42 cores=0,2,4,6
/proc/26703/task/26712/stat : 26712 (pb_cplex) R 26702 26702 25626 0 -1 4202560 46708 0 0 0 14315 27 0 0 20 0 8 0 390887066 1692278784 334141 33554432000 134512640 146736320 4293612720 3587145936 136008065 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 1097.59
Current children cumulated vsize (KiB) 1661764

[startup+462.3 s]
/proc/loadavg: 11.83 10.82 7.42 10/186 26763
/proc/meminfo: memFree=23297684/32873844 swapFree=7896/7896
[pid=26702] ppid=26700 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/26702/stat : 26702 (run) S 26700 26702 25626 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390875204 9367552 270 33554432000 4194304 5064252 140733971490288 140733971489200 239872927310 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/26702/statm: 2287 270 229 213 0 72 0
[pid=26703] ppid=26702 vsize=1660124 CPUtime=1337.1 cores=0,2,4,6
/proc/26703/stat : 26703 (pb_cplex) R 26702 26702 25626 0 -1 4202496 543769 0 0 0 133487 223 0 0 20 0 8 0 390875204 1699966976 343552 33554432000 134512640 146736320 4293612720 4293605584 136008450 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/26703/statm: 415031 343552 963 2985 0 411965 0
[pid=26703/tid=26706] ppid=26702 vsize=1660124 CPUtime=178.05 cores=0,2,4,6
/proc/26703/task/26706/stat : 26706 (pb_cplex) R 26702 26702 25626 0 -1 4202560 53946 0 0 0 17777 28 0 0 20 0 8 0 390887066 1699966976 343552 33554432000 134512640 146736320 4293612720 3650060400 136384890 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=26703/tid=26707] ppid=26702 vsize=1660124 CPUtime=85.17 cores=0,2,4,6
/proc/26703/task/26707/stat : 26707 (pb_cplex) R 26702 26702 25626 0 -1 4202560 17512 0 0 0 8507 10 0 0 20 0 8 0 390887066 1699966976 343552 33554432000 134512640 146736320 4293612720 3639574344 135526137 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
[pid=26703/tid=26708] ppid=26702 vsize=1660124 CPUtime=178.17 cores=0,2,4,6
/proc/26703/task/26708/stat : 26708 (pb_cplex) R 26702 26702 25626 0 -1 4202560 49907 0 0 0 17788 29 0 0 20 0 8 0 390887066 1699966976 343552 33554432000 134512640 146736320 4293612720 3629088768 136018927 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=26703/tid=26709] ppid=26702 vsize=1660124 CPUtime=159.87 cores=0,2,4,6
/proc/26703/task/26709/stat : 26709 (pb_cplex) R 26702 26702 25626 0 -1 4202560 36309 0 0 0 15963 24 0 0 20 0 8 0 390887066 1699966976 343552 33554432000 134512640 146736320 4293612720 3618602128 136008065 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=26703/tid=26710] ppid=26702 vsize=1660124 CPUtime=152.01 cores=0,2,4,6
/proc/26703/task/26710/stat : 26710 (pb_cplex) R 26702 26702 25626 0 -1 4202560 35793 0 0 0 15179 22 0 0 20 0 8 0 390887066 1699966976 343552 33554432000 134512640 146736320 4293612720 3608117456 136008065 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=26703/tid=26711] ppid=26702 vsize=1660124 CPUtime=84.89 cores=0,2,4,6
/proc/26703/task/26711/stat : 26711 (pb_cplex) R 26702 26702 25626 0 -1 4202560 23249 0 0 0 8477 12 0 0 20 0 8 0 390887066 1699966976 343552 33554432000 134512640 146736320 4293612720 3597631696 136008065 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=26703/tid=26712] ppid=26702 vsize=1660124 CPUtime=173.38 cores=0,2,4,6
/proc/26703/task/26712/stat : 26712 (pb_cplex) R 26702 26702 25626 0 -1 4202560 48692 0 0 0 17310 28 0 0 20 0 8 0 390887066 1699966976 343552 33554432000 134512640 146736320 4293612720 3587145936 136008441 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
Current children cumulated CPU time (s) 1337.1
Current children cumulated vsize (KiB) 1669272

[startup+522.301 s]
/proc/loadavg: 10.11 10.50 7.52 10/187 26766
/proc/meminfo: memFree=23290580/32873844 swapFree=7896/7896
[pid=26702] ppid=26700 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/26702/stat : 26702 (run) S 26700 26702 25626 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390875204 9367552 270 33554432000 4194304 5064252 140733971490288 140733971489200 239872927310 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/26702/statm: 2287 270 229 213 0 72 0
[pid=26703] ppid=26702 vsize=1662972 CPUtime=1576.66 cores=0,2,4,6
/proc/26703/stat : 26703 (pb_cplex) R 26702 26702 25626 0 -1 4202496 569312 0 0 0 157433 233 0 0 20 0 8 0 390875204 1702883328 345243 33554432000 134512640 146736320 4293612720 4293605584 136008050 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/26703/statm: 415743 345243 963 2985 0 412677 0
[pid=26703/tid=26706] ppid=26702 vsize=1662972 CPUtime=207.86 cores=0,2,4,6
/proc/26703/task/26706/stat : 26706 (pb_cplex) R 26702 26702 25626 0 -1 4202560 59231 0 0 0 20755 31 0 0 20 0 8 0 390887066 1702883328 345243 33554432000 134512640 146736320 4293612720 3650060496 136008454 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
[pid=26703/tid=26707] ppid=26702 vsize=1662972 CPUtime=114.67 cores=0,2,4,6
/proc/26703/task/26707/stat : 26707 (pb_cplex) R 26702 26702 25626 0 -1 4202560 19682 0 0 0 11456 11 0 0 20 0 8 0 390887066 1702883328 345243 33554432000 134512640 146736320 4293612720 3639574528 136018931 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=26703/tid=26708] ppid=26702 vsize=1662972 CPUtime=207.9 cores=0,2,4,6
/proc/26703/task/26708/stat : 26708 (pb_cplex) R 26702 26702 25626 0 -1 4202560 51994 0 0 0 20759 31 0 0 20 0 8 0 390887066 1702883328 345243 33554432000 134512640 146736320 4293612720 3629088976 136008050 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=26703/tid=26709] ppid=26702 vsize=1662972 CPUtime=190.29 cores=0,2,4,6
/proc/26703/task/26709/stat : 26709 (pb_cplex) R 26702 26702 25626 0 -1 4202560 39634 0 0 0 19003 26 0 0 20 0 8 0 390887066 1702883328 345243 33554432000 134512640 146736320 4293612720 3618603216 136008441 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=26703/tid=26710] ppid=26702 vsize=1662972 CPUtime=182 cores=0,2,4,6
/proc/26703/task/26710/stat : 26710 (pb_cplex) R 26702 26702 25626 0 -1 4202560 37966 0 0 0 18178 22 0 0 20 0 8 0 390887066 1702883328 345243 33554432000 134512640 146736320 4293612720 3608117456 136008450 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=26703/tid=26711] ppid=26702 vsize=1662972 CPUtime=114.49 cores=0,2,4,6
/proc/26703/task/26711/stat : 26711 (pb_cplex) R 26702 26702 25626 0 -1 4202560 29129 0 0 0 11435 14 0 0 20 0 8 0 390887066 1702883328 345243 33554432000 134512640 146736320 4293612720 3597631696 136008178 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
[pid=26703/tid=26712] ppid=26702 vsize=1662972 CPUtime=203.34 cores=0,2,4,6
/proc/26703/task/26712/stat : 26712 (pb_cplex) R 26702 26702 25626 0 -1 4202560 51421 0 0 0 20304 30 0 0 20 0 8 0 390887066 1702883328 345243 33554432000 134512640 146736320 4293612720 3587146320 136403109 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1576.66
Current children cumulated vsize (KiB) 1672120



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+578.3 s]
/proc/loadavg: 9.52 10.26 7.61 10/186 26766
/proc/meminfo: memFree=23289064/32873844 swapFree=7896/7896
[pid=26702] ppid=26700 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/26702/stat : 26702 (run) S 26700 26702 25626 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390875204 9367552 270 33554432000 4194304 5064252 140733971490288 140733971489200 239872927310 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/26702/statm: 2287 270 229 213 0 72 0
[pid=26703] ppid=26702 vsize=1665952 CPUtime=1800.27 cores=0,2,4,6
/proc/26703/stat : 26703 (pb_cplex) R 26702 26702 25626 0 -1 4202496 588990 0 0 0 179783 244 0 0 20 0 8 0 390875204 1705934848 345539 33554432000 134512640 146736320 4293612720 4293605416 139044581 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/26703/statm: 416488 345539 963 2985 0 413422 0
[pid=26703/tid=26706] ppid=26702 vsize=1665952 CPUtime=235.5 cores=0,2,4,6
/proc/26703/task/26706/stat : 26706 (pb_cplex) R 26702 26702 25626 0 -1 4202560 60994 0 0 0 23517 33 0 0 20 0 8 0 390887066 1705934848 345539 33554432000 134512640 146736320 4293612720 3650060496 136008039 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=26703/tid=26707] ppid=26702 vsize=1665952 CPUtime=142.59 cores=0,2,4,6
/proc/26703/task/26707/stat : 26707 (pb_cplex) R 26702 26702 25626 0 -1 4202560 21541 0 0 0 14246 13 0 0 20 0 8 0 390887066 1705934848 345539 33554432000 134512640 146736320 4293612720 3639574736 136008167 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=26703/tid=26708] ppid=26702 vsize=1665952 CPUtime=235.7 cores=0,2,4,6
/proc/26703/task/26708/stat : 26708 (pb_cplex) R 26702 26702 25626 0 -1 4202560 53734 0 0 0 23538 32 0 0 20 0 8 0 390887066 1705934848 345539 33554432000 134512640 146736320 4293612720 3629089408 139046496 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
[pid=26703/tid=26709] ppid=26702 vsize=1665952 CPUtime=218.13 cores=0,2,4,6
/proc/26703/task/26709/stat : 26709 (pb_cplex) R 26702 26702 25626 0 -1 4202560 42141 0 0 0 21784 29 0 0 20 0 8 0 390887066 1705934848 345539 33554432000 134512640 146736320 4293612720 3618603216 136008042 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=26703/tid=26710] ppid=26702 vsize=1665952 CPUtime=209.92 cores=0,2,4,6
/proc/26703/task/26710/stat : 26710 (pb_cplex) R 26702 26702 25626 0 -1 4202560 40154 0 0 0 20967 25 0 0 20 0 8 0 390887066 1705934848 345539 33554432000 134512640 146736320 4293612720 3608117456 136008050 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=26703/tid=26711] ppid=26702 vsize=1665952 CPUtime=142.16 cores=0,2,4,6
/proc/26703/task/26711/stat : 26711 (pb_cplex) R 26702 26702 25626 0 -1 4202560 34249 0 0 0 14199 17 0 0 20 0 8 0 390887066 1705934848 345539 33554432000 134512640 146736320 4293612720 3597630472 139042890 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=26703/tid=26712] ppid=26702 vsize=1665952 CPUtime=231.23 cores=0,2,4,6
/proc/26703/task/26712/stat : 26712 (pb_cplex) R 26702 26702 25626 0 -1 4202560 53985 0 0 0 23092 31 0 0 20 0 8 0 390887066 1705934848 345539 33554432000 134512640 146736320 4293612720 3587145936 136008441 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
Current children cumulated CPU time (s) 1800.27
Current children cumulated vsize (KiB) 1675100

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+578.3 s]
/proc/loadavg: 9.52 10.26 7.61 10/186 26766
/proc/meminfo: memFree=23289064/32873844 swapFree=7896/7896
[pid=26702] ppid=26700 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/26702/stat : 26702 (run) S 26700 26702 25626 0 -1 4202496 356 0 0 0 0 0 0 0 20 0 1 0 390875204 9367552 270 33554432000 4194304 5064252 140733971490288 140733971489200 239872927310 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/26702/statm: 2287 270 229 213 0 72 0
[pid=26703] ppid=26702 vsize=1665952 CPUtime=1800.27 cores=0,2,4,6
/proc/26703/stat : 26703 (pb_cplex) R 26702 26702 25626 0 -1 4202496 588990 0 0 0 179783 244 0 0 20 0 8 0 390875204 1705934848 345539 33554432000 134512640 146736320 4293612720 4293605416 139044581 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/26703/statm: 416488 345539 963 2985 0 413422 0
[pid=26703/tid=26706] ppid=26702 vsize=1665952 CPUtime=235.5 cores=0,2,4,6
/proc/26703/task/26706/stat : 26706 (pb_cplex) R 26702 26702 25626 0 -1 4202560 60994 0 0 0 23517 33 0 0 20 0 8 0 390887066 1705934848 345539 33554432000 134512640 146736320 4293612720 3650060496 136008039 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=26703/tid=26707] ppid=26702 vsize=1665952 CPUtime=142.59 cores=0,2,4,6
/proc/26703/task/26707/stat : 26707 (pb_cplex) R 26702 26702 25626 0 -1 4202560 21541 0 0 0 14246 13 0 0 20 0 8 0 390887066 1705934848 345539 33554432000 134512640 146736320 4293612720 3639574736 136008167 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=26703/tid=26708] ppid=26702 vsize=1665952 CPUtime=235.7 cores=0,2,4,6
/proc/26703/task/26708/stat : 26708 (pb_cplex) R 26702 26702 25626 0 -1 4202560 53734 0 0 0 23538 32 0 0 20 0 8 0 390887066 1705934848 345539 33554432000 134512640 146736320 4293612720 3629089408 139046496 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
[pid=26703/tid=26709] ppid=26702 vsize=1665952 CPUtime=218.13 cores=0,2,4,6
/proc/26703/task/26709/stat : 26709 (pb_cplex) R 26702 26702 25626 0 -1 4202560 42141 0 0 0 21784 29 0 0 20 0 8 0 390887066 1705934848 345539 33554432000 134512640 146736320 4293612720 3618603216 136008042 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=26703/tid=26710] ppid=26702 vsize=1665952 CPUtime=209.92 cores=0,2,4,6
/proc/26703/task/26710/stat : 26710 (pb_cplex) R 26702 26702 25626 0 -1 4202560 40154 0 0 0 20967 25 0 0 20 0 8 0 390887066 1705934848 345539 33554432000 134512640 146736320 4293612720 3608117456 136008050 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=26703/tid=26711] ppid=26702 vsize=1665952 CPUtime=142.16 cores=0,2,4,6
/proc/26703/task/26711/stat : 26711 (pb_cplex) R 26702 26702 25626 0 -1 4202560 34249 0 0 0 14199 17 0 0 20 0 8 0 390887066 1705934848 345539 33554432000 134512640 146736320 4293612720 3597630472 139042890 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=26703/tid=26712] ppid=26702 vsize=1665952 CPUtime=231.23 cores=0,2,4,6
/proc/26703/task/26712/stat : 26712 (pb_cplex) R 26702 26702 25626 0 -1 4202560 53985 0 0 0 23092 31 0 0 20 0 8 0 390887066 1705934848 345539 33554432000 134512640 146736320 4293612720 3587145936 136008441 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
Current children cumulated CPU time (s) 1800.27
Current children cumulated vsize (KiB) 1675100

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 26702 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=0
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=999
# CPU time returned by wait4() is 0.000999
# while last known CPU time is 1800.27
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 578.317
CPU time (s): 1800.27
CPU user time (s): 1797.83
CPU system time (s): 2.44
CPU usage (%): 311.295
Max. virtual memory (cumulated for all children) (KiB): 1683748

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0
system time used= 0.000999
maximum resident set size= 1080
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 356
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2
involuntary context switches= 1

runsolver used 1.54276 second user time and 4.25335 second system time

The end

Launcher Data 

Begin job on node136 at 2012-06-09 15:11:37
IDJOB=3733974
IDBENCH=2639
IDSOLVER=2325
FILE ID=node136/3733974-1339247497
RUNJOBID= node136-1339246834-25642
PBS_JOBID= 14644345
Free space on /tmp= 65544 MiB

SOLVER NAME= PB10: pb_cplex 2010-06-29
BENCH NAME= PB06/final/normalized-PB06/OPT-SMALLINT/submitted-PB06/manquiho/golomb-rulers/normalized-OGR_20.opb
COMMAND LINE= run BENCHNAME
CONVERSION SCRIPT= PBconversionToLinear BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3733974-1339247497/watcher-3733974-1339247497 -o /tmp/evaluation-result-3733974-1339247497/solver-3733974-1339247497 -C 1800 -W 1900 -M 15500  run HOME/instance-3733974-1339247497.opb

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 6a25ee38adc43f783b18badd1224270c
RANDOM SEED=1789438571

node136.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.247
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.247
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.247
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.94
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.247
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.247
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.247
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.247
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.247
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        24988652 kB
Buffers:          734340 kB
Cached:          5707144 kB
SwapCached:        20520 kB
Active:          5715644 kB
Inactive:        1165272 kB
Active(anon):     417316 kB
Inactive(anon):    23236 kB
Active(file):    5298328 kB
Inactive(file):  1142036 kB
Unevictable:        7896 kB
Mlocked:            7896 kB
SwapTotal:      67108856 kB
SwapFree:       67083288 kB
Dirty:             24924 kB
Writeback:             0 kB
AnonPages:        426336 kB
Mapped:            12848 kB
Shmem:                 0 kB
Slab:             862364 kB
SReclaimable:     302996 kB
SUnreclaim:       559368 kB
KernelStack:        1504 kB
PageTables:         5052 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     548136 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:    151552 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

CONVERSION COMMAND LINE= PBconversionToLinear HOME/instance-3733974-1339247497.opb

Free space on /tmp at the end= 65476 MiB
End job on node136 at 2012-06-09 15:21:18