Trace number 3736952

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
PB10: pb_cplex 2010-06-29? (TO) 1800.41 535.916

General information on the benchmark

Namenormalized-PB07/OPT-SMALLINT-NLC/submittedPB07/
manquinho/mds/normalized-mds_200_10_2.opb
MD5SUM54e50bd6cbfbb2a5705269c41241de9a
Bench CategoryOPT-SMALLINT-NLC (optimisation, small integers, non linear constraints)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark21
Best CPU time to get the best result obtained on this benchmark1789.78
Has Objective FunctionYES
SatisfiableYES
(Un)Satisfiability was provedYES
Best value of the objective function 25
Optimality of the best value was proved NO
Number of variables200
Total number of constraints200
Number of constraints which are clauses0
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints200
Minimum length of a constraint11
Maximum length of a constraint20
Number of terms in the objective function 200
Biggest coefficient in the objective function 1
Number of bits for the biggest coefficient in the objective function 1
Sum of the numbers in the objective function 200
Number of bits of the sum of numbers in the objective function 8
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 200
Number of bits of the biggest sum of numbers8
Number of products (including duplicates)2492
Sum of products size (including duplicates)4984
Number of different products2492
Sum of products size4984

Solver Data 

0.00/0.00	c pb_cplex 1.0 - 20100701
0.00/0.00	c Parsing opb file format. File HOME/instance-3736952-1339248410.opb.
0.00/0.00	c Instance file HOME/instance-3736952-1339248410.opb
0.00/0.00	c File size is 185648 bytes.
0.00/0.03	c Highest Coefficient sum: 21
0.00/0.03	c  IBM ILOG License Manager: "IBM ILOG Optimization Suite for Academic Initiative" is accessing CPLEX 12 with option(s): "e m b q ".
0.00/0.03	c ============================[ Problem Statistics ]=============================
0.00/0.03	c |                                                                             |
0.00/0.03	c |  Parsing time:          0.03         s                                      |
0.00/0.03	c |  Number Variables:      2692                                                |
0.00/0.03	c |  Number Constraints:    5184                                                |
0.00/0.03	c ===============================================================================
1800.41/535.91	c Remaining time 1799.97 sec.
1800.41/535.91	c 
1800.41/535.91	c *** INTERRUPTED ***
1800.41/535.91	s UNKNOWN
1800.41/535.91	c Total CPU time              : 1797.15 s
1800.41/535.91	c Memory used           : 266.50 MB
1800.41/535.91	c CPU time              : 1797.15 s
1800.41/535.91	c 
1800.41/535.91	c *** INTERRUPTED ***

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3736952-1339248410/watcher-3736952-1339248410 -o /tmp/evaluation-result-3736952-1339248410/solver-3736952-1339248410 -C 1800 -W 1900 -M 15500 run HOME/instance-3736952-1339248410.opb 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.61 7.29 5.96 3/185 3276
/proc/meminfo: memFree=24230132/32873660 swapFree=7120/7120
[pid=3276] ppid=3274 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/3276/stat : 3276 (run) S 3274 3276 2263 0 -1 4202496 354 0 0 0 0 0 0 0 20 0 1 0 390966792 9367552 269 33554432000 4194304 5064252 140734767505968 140734767504880 270329865806 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/3276/statm: 2287 269 229 213 0 72 0

[startup+0.0832221 s]
/proc/loadavg: 5.61 7.29 5.96 3/185 3276
/proc/meminfo: memFree=24230132/32873660 swapFree=7120/7120
[pid=3276] ppid=3274 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/3276/stat : 3276 (run) S 3274 3276 2263 0 -1 4202496 354 0 0 0 0 0 0 0 20 0 1 0 390966792 9367552 269 33554432000 4194304 5064252 140734767505968 140734767504880 270329865806 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/3276/statm: 2287 269 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.10025 s]
/proc/loadavg: 5.61 7.29 5.96 3/185 3276
/proc/meminfo: memFree=24230132/32873660 swapFree=7120/7120
[pid=3276] ppid=3274 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/3276/stat : 3276 (run) S 3274 3276 2263 0 -1 4202496 354 0 0 0 0 0 0 0 20 0 1 0 390966792 9367552 269 33554432000 4194304 5064252 140734767505968 140734767504880 270329865806 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/3276/statm: 2287 269 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.300261 s]
/proc/loadavg: 5.61 7.29 5.96 3/185 3276
/proc/meminfo: memFree=24230132/32873660 swapFree=7120/7120
[pid=3276] ppid=3274 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/3276/stat : 3276 (run) S 3274 3276 2263 0 -1 4202496 354 0 0 0 0 0 0 0 20 0 1 0 390966792 9367552 269 33554432000 4194304 5064252 140734767505968 140734767504880 270329865806 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/3276/statm: 2287 269 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.700173 s]
/proc/loadavg: 5.61 7.29 5.96 3/185 3276
/proc/meminfo: memFree=24230132/32873660 swapFree=7120/7120
[pid=3276] ppid=3274 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/3276/stat : 3276 (run) S 3274 3276 2263 0 -1 4202496 354 0 0 0 0 0 0 0 20 0 1 0 390966792 9367552 269 33554432000 4194304 5064252 140734767505968 140734767504880 270329865806 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/3276/statm: 2287 269 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+1.50043 s]
/proc/loadavg: 5.61 7.29 5.96 10/194 3285
/proc/meminfo: memFree=24148484/32873660 swapFree=7120/7120
[pid=3276] ppid=3274 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/3276/stat : 3276 (run) S 3274 3276 2263 0 -1 4202496 354 0 0 0 0 0 0 0 20 0 1 0 390966792 9367552 269 33554432000 4194304 5064252 140734767505968 140734767504880 270329865806 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/3276/statm: 2287 269 229 213 0 72 0
[pid=3277] ppid=3276 vsize=195436 CPUtime=4.04 cores=0,2,4,6
/proc/3277/stat : 3277 (pb_cplex) R 3276 3276 2263 0 -1 4202496 33071 0 0 0 397 7 0 0 20 0 8 0 390966792 200126464 23223 33554432000 134512640 146736320 4293985856 4293978496 135468707 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/3277/statm: 48859 23223 928 2985 0 45793 0
[pid=3277/tid=3279] ppid=3276 vsize=195436 CPUtime=0.46 cores=0,2,4,6
/proc/3277/task/3279/stat : 3279 (pb_cplex) R 3276 3276 2263 0 -1 4202560 2608 0 0 0 46 0 0 0 20 0 8 0 390966835 200126464 23223 33554432000 134512640 146736320 4293985856 4142707520 139028313 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=3277/tid=3280] ppid=3276 vsize=195436 CPUtime=0.47 cores=0,2,4,6
/proc/3277/task/3280/stat : 3280 (pb_cplex) R 3276 3276 2263 0 -1 4202560 2559 0 0 0 47 0 0 0 20 0 8 0 390966835 200126464 23223 33554432000 134512640 146736320 4293985856 4132219932 135854186 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
[pid=3277/tid=3281] ppid=3276 vsize=195436 CPUtime=0.42 cores=0,2,4,6
/proc/3277/task/3281/stat : 3281 (pb_cplex) R 3276 3276 2263 0 -1 4202560 2086 0 0 0 42 0 0 0 20 0 8 0 390966835 200126464 23223 33554432000 134512640 146736320 4293985856 4121735168 138432847 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=3277/tid=3282] ppid=3276 vsize=195436 CPUtime=0.53 cores=0,2,4,6
/proc/3277/task/3282/stat : 3282 (pb_cplex) R 3276 3276 2263 0 -1 4202560 3701 0 0 0 53 0 0 0 20 0 8 0 390966835 200126464 23223 33554432000 134512640 146736320 4293985856 4111251140 135850315 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
[pid=3277/tid=3283] ppid=3276 vsize=195436 CPUtime=0.4 cores=0,2,4,6
/proc/3277/task/3283/stat : 3283 (pb_cplex) R 3276 3276 2263 0 -1 4202560 2502 0 0 0 40 0 0 0 20 0 8 0 390966835 200126464 23223 33554432000 134512640 146736320 4293985856 4100762712 136007626 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=3277/tid=3284] ppid=3276 vsize=195436 CPUtime=0.34 cores=0,2,4,6
/proc/3277/task/3284/stat : 3284 (pb_cplex) R 3276 3276 2263 0 -1 4202560 2307 0 0 0 34 0 0 0 20 0 8 0 390966835 200126464 23223 33554432000 134512640 146736320 4293985856 4090278888 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=3277/tid=3285] ppid=3276 vsize=195436 CPUtime=0.4 cores=0,2,4,6
/proc/3277/task/3285/stat : 3285 (pb_cplex) R 3276 3276 2263 0 -1 4202560 2248 0 0 0 40 0 0 0 20 0 8 0 390966835 200126464 23223 33554432000 134512640 146736320 4293985856 4079793128 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 4.04
Current children cumulated vsize (KiB) 204584

[startup+3.10021 s]
/proc/loadavg: 5.61 7.29 5.96 3/187 3285
/proc/meminfo: memFree=24149476/32873660 swapFree=7120/7120
[pid=3276] ppid=3274 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/3276/stat : 3276 (run) S 3274 3276 2263 0 -1 4202496 354 0 0 0 0 0 0 0 20 0 1 0 390966792 9367552 269 33554432000 4194304 5064252 140734767505968 140734767504880 270329865806 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/3276/statm: 2287 269 229 213 0 72 0
[pid=3277] ppid=3276 vsize=103968 CPUtime=7.14 cores=0,2,4,6
/proc/3277/stat : 3277 (pb_cplex) R 3276 3276 2263 0 -1 4202496 34070 0 0 0 702 12 0 0 20 0 1 0 390966792 106463232 11879 33554432000 134512640 146736320 4293985856 4293979840 136019045 0 0 4096 16387 0 0 0 17 0 0 0 0 0 0
/proc/3277/statm: 25992 11879 940 2985 0 22926 0
Current children cumulated CPU time (s) 7.14
Current children cumulated vsize (KiB) 113116

[startup+6.30021 s]
/proc/loadavg: 5.32 7.21 5.94 3/187 3285
/proc/meminfo: memFree=24183392/32873660 swapFree=7120/7120
[pid=3276] ppid=3274 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/3276/stat : 3276 (run) S 3274 3276 2263 0 -1 4202496 354 0 0 0 0 0 0 0 20 0 1 0 390966792 9367552 269 33554432000 4194304 5064252 140734767505968 140734767504880 270329865806 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/3276/statm: 2287 269 229 213 0 72 0
[pid=3277] ppid=3276 vsize=103968 CPUtime=10.34 cores=0,2,4,6
/proc/3277/stat : 3277 (pb_cplex) R 3276 3276 2263 0 -1 4202496 34205 0 0 0 1022 12 0 0 20 0 1 0 390966792 106463232 12014 33554432000 134512640 146736320 4293985856 4293981056 136469951 0 0 4096 16387 0 0 0 17 0 0 0 0 0 0
/proc/3277/statm: 25992 12014 941 2985 0 22926 0
Current children cumulated CPU time (s) 10.34
Current children cumulated vsize (KiB) 113116

[startup+12.7003 s]
/proc/loadavg: 5.06 7.12 5.92 10/187 3285
/proc/meminfo: memFree=23995116/32873660 swapFree=7120/7120
[pid=3276] ppid=3274 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/3276/stat : 3276 (run) S 3274 3276 2263 0 -1 4202496 354 0 0 0 0 0 0 0 20 0 1 0 390966792 9367552 269 33554432000 4194304 5064252 140734767505968 140734767504880 270329865806 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/3276/statm: 2287 269 229 213 0 72 0
[pid=3277] ppid=3276 vsize=103968 CPUtime=16.74 cores=0,2,4,6
/proc/3277/stat : 3277 (pb_cplex) R 3276 3276 2263 0 -1 4202496 34360 0 0 0 1662 12 0 0 20 0 1 0 390966792 106463232 12131 33554432000 134512640 146736320 4293985856 4293981648 137698804 0 0 4096 16387 0 0 0 17 0 0 0 0 0 0
/proc/3277/statm: 25992 12131 941 2985 0 22926 0
Current children cumulated CPU time (s) 16.74
Current children cumulated vsize (KiB) 113116

[startup+25.5004 s]
/proc/loadavg: 5.93 7.21 5.97 10/194 3299
/proc/meminfo: memFree=23856672/32873660 swapFree=7120/7120
[pid=3276] ppid=3274 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/3276/stat : 3276 (run) S 3274 3276 2263 0 -1 4202496 354 0 0 0 0 0 0 0 20 0 1 0 390966792 9367552 269 33554432000 4194304 5064252 140734767505968 140734767504880 270329865806 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/3276/statm: 2287 269 229 213 0 72 0
[pid=3277] ppid=3276 vsize=185428 CPUtime=30.56 cores=0,2,4,6
/proc/3277/stat : 3277 (pb_cplex) R 3276 3276 2263 0 -1 4202496 60957 0 0 0 3033 23 0 0 20 0 8 0 390966792 189878272 20813 33554432000 134512640 146736320 4293985856 4293977864 135999691 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/3277/statm: 46357 20813 978 2985 0 43291 0
[pid=3277/tid=3293] ppid=3276 vsize=185428 CPUtime=0 cores=0,2,4,6
/proc/3277/task/3293/stat : 3293 (pb_cplex) S 3276 3276 2263 0 -1 4202560 0 0 0 0 0 0 0 0 20 0 8 0 390968872 189878272 20813 33554432000 134512640 146736320 4293985856 4130397840 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=3277/tid=3294] ppid=3276 vsize=185428 CPUtime=0 cores=0,2,4,6
/proc/3277/task/3294/stat : 3294 (pb_cplex) S 3276 3276 2263 0 -1 4202560 0 0 0 0 0 0 0 0 20 0 8 0 390968872 189878272 20813 33554432000 134512640 146736320 4293985856 4140883600 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=3277/tid=3295] ppid=3276 vsize=185428 CPUtime=0 cores=0,2,4,6
/proc/3277/task/3295/stat : 3295 (pb_cplex) S 3276 3276 2263 0 -1 4202560 0 0 0 0 0 0 0 0 20 0 8 0 390968872 189878272 20813 33554432000 134512640 146736320 4293985856 4151369360 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=3277/tid=3296] ppid=3276 vsize=185428 CPUtime=0 cores=0,2,4,6
/proc/3277/task/3296/stat : 3296 (pb_cplex) S 3276 3276 2263 0 -1 4202560 0 0 0 0 0 0 0 0 20 0 8 0 390968872 189878272 20813 33554432000 134512640 146736320 4293985856 4111253136 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=3277/tid=3297] ppid=3276 vsize=185428 CPUtime=0 cores=0,2,4,6
/proc/3277/task/3297/stat : 3297 (pb_cplex) S 3276 3276 2263 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 8 0 390968872 189878272 20813 33554432000 134512640 146736320 4293985856 4056936080 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=3277/tid=3298] ppid=3276 vsize=185428 CPUtime=0 cores=0,2,4,6
/proc/3277/task/3298/stat : 3298 (pb_cplex) S 3276 3276 2263 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 8 0 390968872 189878272 20813 33554432000 134512640 146736320 4293985856 4046450320 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=3277/tid=3299] ppid=3276 vsize=185428 CPUtime=0 cores=0,2,4,6
/proc/3277/task/3299/stat : 3299 (pb_cplex) S 3276 3276 2263 0 -1 4202560 2 0 0 0 0 0 0 0 20 0 8 0 390968872 189878272 20813 33554432000 134512640 146736320 4293985856 4035964560 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 30.56
Current children cumulated vsize (KiB) 194576

[startup+51.1004 s]
/proc/loadavg: 8.96 7.81 6.20 17/193 3299
/proc/meminfo: memFree=23780424/32873660 swapFree=7120/7120
[pid=3276] ppid=3274 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/3276/stat : 3276 (run) S 3274 3276 2263 0 -1 4202496 354 0 0 0 0 0 0 0 20 0 1 0 390966792 9367552 269 33554432000 4194304 5064252 140734767505968 140734767504880 270329865806 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/3276/statm: 2287 269 229 213 0 72 0
[pid=3277] ppid=3276 vsize=237620 CPUtime=111.44 cores=0,2,4,6
/proc/3277/stat : 3277 (pb_cplex) R 3276 3276 2263 0 -1 4202496 103024 0 0 0 11105 39 0 0 20 0 8 0 390966792 243322880 31736 33554432000 134512640 146736320 4293985856 4293977864 136000404 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/3277/statm: 59405 31736 978 2985 0 56339 0
[pid=3277/tid=3293] ppid=3276 vsize=237620 CPUtime=9.17 cores=0,2,4,6
/proc/3277/task/3293/stat : 3293 (pb_cplex) R 3276 3276 2263 0 -1 4202560 2334 0 0 0 916 1 0 0 20 0 8 0 390968872 243322880 31736 33554432000 134512640 146736320 4293985856 4130393240 136008924 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
[pid=3277/tid=3294] ppid=3276 vsize=237620 CPUtime=9.44 cores=0,2,4,6
/proc/3277/task/3294/stat : 3294 (pb_cplex) R 3276 3276 2263 0 -1 4202560 8626 0 0 0 941 3 0 0 20 0 8 0 390968872 243322880 31736 33554432000 134512640 146736320 4293985856 4140878876 135855165 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=3277/tid=3295] ppid=3276 vsize=237620 CPUtime=9.06 cores=0,2,4,6
/proc/3277/task/3295/stat : 3295 (pb_cplex) R 3276 3276 2263 0 -1 4202560 255 0 0 0 906 0 0 0 20 0 8 0 390968872 243322880 31736 33554432000 134512640 146736320 4293985856 4151364544 136018922 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=3277/tid=3296] ppid=3276 vsize=237620 CPUtime=8.96 cores=0,2,4,6
/proc/3277/task/3296/stat : 3296 (pb_cplex) R 3276 3276 2263 0 -1 4202560 7134 0 0 0 895 1 0 0 20 0 8 0 390968872 243322880 31736 33554432000 134512640 146736320 4293985856 4111248912 136403109 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=3277/tid=3297] ppid=3276 vsize=237620 CPUtime=9.38 cores=0,2,4,6
/proc/3277/task/3297/stat : 3297 (pb_cplex) R 3276 3276 2263 0 -1 4202560 2456 0 0 0 937 1 0 0 20 0 8 0 390968872 243322880 31736 33554432000 134512640 146736320 4293985856 4056932768 139023058 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=3277/tid=3298] ppid=3276 vsize=237620 CPUtime=8.9 cores=0,2,4,6
/proc/3277/task/3298/stat : 3298 (pb_cplex) R 3276 3276 2263 0 -1 4202560 7997 0 0 0 888 2 0 0 20 0 8 0 390968872 243322880 31736 33554432000 134512640 146736320 4293985856 4046447036 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=3277/tid=3299] ppid=3276 vsize=237620 CPUtime=9.37 cores=0,2,4,6
/proc/3277/task/3299/stat : 3299 (pb_cplex) R 3276 3276 2263 0 -1 4202560 5175 0 0 0 935 2 0 0 20 0 8 0 390968872 243322880 31736 33554432000 134512640 146736320 4293985856 4035959856 136384895 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 111.44
Current children cumulated vsize (KiB) 246768

[startup+102.307 s]
/proc/loadavg: 11.72 8.77 6.61 17/193 3299
/proc/meminfo: memFree=23745676/32873660 swapFree=7120/7120
[pid=3276] ppid=3274 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/3276/stat : 3276 (run) S 3274 3276 2263 0 -1 4202496 354 0 0 0 0 0 0 0 20 0 1 0 390966792 9367552 269 33554432000 4194304 5064252 140734767505968 140734767504880 270329865806 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/3276/statm: 2287 269 229 213 0 72 0
[pid=3277] ppid=3276 vsize=229876 CPUtime=303.22 cores=0,2,4,6
/proc/3277/stat : 3277 (pb_cplex) R 3276 3276 2263 0 -1 4202496 165197 0 0 0 30257 65 0 0 20 0 8 0 390966792 235393024 32092 33554432000 134512640 146736320 4293985856 4293978704 136008044 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/3277/statm: 57469 32092 978 2985 0 54403 0
[pid=3277/tid=3293] ppid=3276 vsize=229876 CPUtime=37.72 cores=0,2,4,6
/proc/3277/task/3293/stat : 3293 (pb_cplex) R 3276 3276 2263 0 -1 4202560 9001 0 0 0 3769 3 0 0 20 0 8 0 390968872 235393024 32092 33554432000 134512640 146736320 4293985856 4130393600 136001203 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=3277/tid=3294] ppid=3276 vsize=229876 CPUtime=32.44 cores=0,2,4,6
/proc/3277/task/3294/stat : 3294 (pb_cplex) R 3276 3276 2263 0 -1 4202560 20100 0 0 0 3237 7 0 0 20 0 8 0 390968872 235393024 32092 33554432000 134512640 146736320 4293985856 4140880464 136403112 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=3277/tid=3295] ppid=3276 vsize=229876 CPUtime=29.93 cores=0,2,4,6
/proc/3277/task/3295/stat : 3295 (pb_cplex) R 3276 3276 2263 0 -1 4202560 7044 0 0 0 2990 3 0 0 20 0 8 0 390968872 235393024 32092 33554432000 134512640 146736320 4293985856 4151365136 136403116 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=3277/tid=3296] ppid=3276 vsize=229876 CPUtime=30.82 cores=0,2,4,6
/proc/3277/task/3296/stat : 3296 (pb_cplex) R 3276 3276 2263 0 -1 4202560 13173 0 0 0 3078 4 0 0 20 0 8 0 390968872 235393024 32092 33554432000 134512640 146736320 4293985856 4111248528 136008042 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=3277/tid=3297] ppid=3276 vsize=229876 CPUtime=35.86 cores=0,2,4,6
/proc/3277/task/3297/stat : 3297 (pb_cplex) R 3276 3276 2263 0 -1 4202560 6008 0 0 0 3582 4 0 0 20 0 8 0 390968872 235393024 32092 33554432000 134512640 146736320 4293985856 4056933352 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=3277/tid=3298] ppid=3276 vsize=229876 CPUtime=28.11 cores=0,2,4,6
/proc/3277/task/3298/stat : 3298 (pb_cplex) R 3276 3276 2263 0 -1 4202560 20234 0 0 0 2805 6 0 0 20 0 8 0 390968872 235393024 32092 33554432000 134512640 146736320 4293985856 4046446800 136008441 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=3277/tid=3299] ppid=3276 vsize=229876 CPUtime=32.18 cores=0,2,4,6

################
# More data... #
################

/proc/3277/statm: 60728 36026 978 2985 0 57662 0
[pid=3277/tid=3293] ppid=3276 vsize=242912 CPUtime=137.35 cores=0,2,4,6
/proc/3277/task/3293/stat : 3293 (pb_cplex) R 3276 3276 2263 0 -1 4202560 66180 0 0 0 13707 28 0 0 20 0 8 0 390968872 248741888 36026 33554432000 134512640 146736320 4293985856 4130395112 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=3277/tid=3294] ppid=3276 vsize=242912 CPUtime=131.46 cores=0,2,4,6
/proc/3277/task/3294/stat : 3294 (pb_cplex) R 3276 3276 2263 0 -1 4202560 50598 0 0 0 13121 25 0 0 20 0 8 0 390968872 248741888 36026 33554432000 134512640 146736320 4293985856 4140880080 136008423 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=3277/tid=3295] ppid=3276 vsize=242912 CPUtime=132.89 cores=0,2,4,6
/proc/3277/task/3295/stat : 3295 (pb_cplex) R 3276 3276 2263 0 -1 4202560 7758 0 0 0 13274 15 0 0 20 0 8 0 390968872 248741888 36026 33554432000 134512640 146736320 4293985856 4151365744 136384883 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
[pid=3277/tid=3296] ppid=3276 vsize=242912 CPUtime=131.35 cores=0,2,4,6
/proc/3277/task/3296/stat : 3296 (pb_cplex) R 3276 3276 2263 0 -1 4202560 44980 0 0 0 13113 22 0 0 20 0 8 0 390968872 248741888 36026 33554432000 134512640 146736320 4293985856 4111247484 135453309 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=3277/tid=3297] ppid=3276 vsize=242912 CPUtime=138.63 cores=0,2,4,6
/proc/3277/task/3297/stat : 3297 (pb_cplex) R 3276 3276 2263 0 -1 4202560 48890 0 0 0 13839 24 0 0 20 0 8 0 390968872 248741888 36026 33554432000 134512640 146736320 4293985856 4056932560 136008039 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=3277/tid=3298] ppid=3276 vsize=242912 CPUtime=129.92 cores=0,2,4,6
/proc/3277/task/3298/stat : 3298 (pb_cplex) R 3276 3276 2263 0 -1 4202560 83340 0 0 0 12963 29 0 0 20 0 8 0 390968872 248741888 36026 33554432000 134512640 146736320 4293985856 4046446664 136474929 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
[pid=3277/tid=3299] ppid=3276 vsize=242912 CPUtime=129.15 cores=0,2,4,6
/proc/3277/task/3299/stat : 3299 (pb_cplex) R 3276 3276 2263 0 -1 4202560 40363 0 0 0 12895 20 0 0 20 0 8 0 390968872 248741888 36026 33554432000 134512640 146736320 4293985856 4035959980 135454093 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1109.71
Current children cumulated vsize (KiB) 252060

[startup+402.3 s]
/proc/loadavg: 11.36 10.73 8.09 17/193 3404
/proc/meminfo: memFree=24859260/32873660 swapFree=7120/7120
[pid=3276] ppid=3274 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/3276/stat : 3276 (run) S 3274 3276 2263 0 -1 4202496 354 0 0 0 0 0 0 0 20 0 1 0 390966792 9367552 269 33554432000 4194304 5064252 140734767505968 140734767504880 270329865806 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/3276/statm: 2287 269 229 213 0 72 0
[pid=3277] ppid=3276 vsize=248032 CPUtime=1313.62 cores=0,2,4,6
/proc/3277/stat : 3277 (pb_cplex) S 3276 3276 2263 0 -1 4202496 468108 0 0 0 131140 222 0 0 20 0 8 0 390966792 253984768 37339 33554432000 134512640 146736320 4293985856 4293982224 144580073 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/3277/statm: 62008 37339 978 2985 0 58942 0
[pid=3277/tid=3293] ppid=3276 vsize=248032 CPUtime=163.52 cores=0,2,4,6
/proc/3277/task/3293/stat : 3293 (pb_cplex) S 3276 3276 2263 0 -1 4202560 72131 0 0 0 16320 32 0 0 20 0 8 0 390968872 253984768 37339 33554432000 134512640 146736320 4293985856 4130397840 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=3277/tid=3294] ppid=3276 vsize=248032 CPUtime=157.34 cores=0,2,4,6
/proc/3277/task/3294/stat : 3294 (pb_cplex) R 3276 3276 2263 0 -1 4202560 51944 0 0 0 15705 29 0 0 20 0 8 0 390968872 253984768 37339 33554432000 134512640 146736320 4293985856 4140879964 135855174 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
[pid=3277/tid=3295] ppid=3276 vsize=248032 CPUtime=156.3 cores=0,2,4,6
/proc/3277/task/3295/stat : 3295 (pb_cplex) S 3276 3276 2263 0 -1 4202560 7758 0 0 0 15613 17 0 0 20 0 8 0 390968872 253984768 37339 33554432000 134512640 146736320 4293985856 4151369360 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=3277/tid=3296] ppid=3276 vsize=248032 CPUtime=157.49 cores=0,2,4,6
/proc/3277/task/3296/stat : 3296 (pb_cplex) R 3276 3276 2263 0 -1 4202560 46324 0 0 0 15723 26 0 0 20 0 8 0 390968872 253984768 37339 33554432000 134512640 146736320 4293985856 4111249624 136008924 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=3277/tid=3297] ppid=3276 vsize=248032 CPUtime=162.95 cores=0,2,4,6
/proc/3277/task/3297/stat : 3297 (pb_cplex) R 3276 3276 2263 0 -1 4202560 53340 0 0 0 16268 27 0 0 20 0 8 0 390968872 253984768 37339 33554432000 134512640 146736320 4293985856 4056931472 136008168 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=3277/tid=3298] ppid=3276 vsize=248032 CPUtime=156.3 cores=0,2,4,6
/proc/3277/task/3298/stat : 3298 (pb_cplex) R 3276 3276 2263 0 -1 4202560 111900 0 0 0 15592 38 0 0 20 0 8 0 390968872 253984768 37339 33554432000 134512640 146736320 4293985856 4046446660 136385177 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=3277/tid=3299] ppid=3276 vsize=248032 CPUtime=154.26 cores=0,2,4,6
/proc/3277/task/3299/stat : 3299 (pb_cplex) R 3276 3276 2263 0 -1 4202560 48250 0 0 0 15401 25 0 0 20 0 8 0 390968872 253984768 37339 33554432000 134512640 146736320 4293985856 4035961832 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1313.62
Current children cumulated vsize (KiB) 257180

[startup+462.3 s]
/proc/loadavg: 12.03 10.99 8.34 10/193 3404
/proc/meminfo: memFree=24825964/32873660 swapFree=7120/7120
[pid=3276] ppid=3274 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/3276/stat : 3276 (run) S 3274 3276 2263 0 -1 4202496 354 0 0 0 0 0 0 0 20 0 1 0 390966792 9367552 269 33554432000 4194304 5064252 140734767505968 140734767504880 270329865806 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/3276/statm: 2287 269 229 213 0 72 0
[pid=3277] ppid=3276 vsize=283964 CPUtime=1520.95 cores=0,2,4,6
/proc/3277/stat : 3277 (pb_cplex) R 3276 3276 2263 0 -1 4202496 537820 0 0 0 151837 258 0 0 20 0 8 0 390966792 290779136 44173 33554432000 134512640 146736320 4293985856 4293974728 136000313 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/3277/statm: 70991 44173 983 2985 0 67925 0
[pid=3277/tid=3293] ppid=3276 vsize=283964 CPUtime=189.68 cores=0,2,4,6
/proc/3277/task/3293/stat : 3293 (pb_cplex) R 3276 3276 2263 0 -1 4202560 77269 0 0 0 18933 35 0 0 20 0 8 0 390968872 290779136 44173 33554432000 134512640 146736320 4293985856 4130394860 144785853 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=3277/tid=3294] ppid=3276 vsize=283964 CPUtime=183.22 cores=0,2,4,6
/proc/3277/task/3294/stat : 3294 (pb_cplex) R 3276 3276 2263 0 -1 4202560 56765 0 0 0 18288 34 0 0 20 0 8 0 390968872 290779136 44173 33554432000 134512640 146736320 4293985856 4140876736 136400570 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
[pid=3277/tid=3295] ppid=3276 vsize=283964 CPUtime=180.97 cores=0,2,4,6
/proc/3277/task/3295/stat : 3295 (pb_cplex) R 3276 3276 2263 0 -1 4202560 9641 0 0 0 18077 20 0 0 20 0 8 0 390968872 290779136 44173 33554432000 134512640 146736320 4293985856 4151360912 136403109 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=3277/tid=3296] ppid=3276 vsize=283964 CPUtime=182.56 cores=0,2,4,6
/proc/3277/task/3296/stat : 3296 (pb_cplex) R 3276 3276 2263 0 -1 4202560 50661 0 0 0 18226 30 0 0 20 0 8 0 390968872 290779136 44173 33554432000 134512640 146736320 4293985856 4111245632 144785853 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=3277/tid=3297] ppid=3276 vsize=283964 CPUtime=188.52 cores=0,2,4,6
/proc/3277/task/3297/stat : 3297 (pb_cplex) R 3276 3276 2263 0 -1 4202560 85418 0 0 0 18815 37 0 0 20 0 8 0 390968872 290779136 44173 33554432000 134512640 146736320 4293985856 4056928720 136403109 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=3277/tid=3298] ppid=3276 vsize=283964 CPUtime=181.78 cores=0,2,4,6
/proc/3277/task/3298/stat : 3298 (pb_cplex) R 3276 3276 2263 0 -1 4202560 122278 0 0 0 18135 43 0 0 20 0 8 0 390968872 290779136 44173 33554432000 134512640 146736320 4293985856 4046441348 136385184 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
[pid=3277/tid=3299] ppid=3276 vsize=283964 CPUtime=181.52 cores=0,2,4,6
/proc/3277/task/3299/stat : 3299 (pb_cplex) R 3276 3276 2263 0 -1 4202560 58625 0 0 0 18121 31 0 0 20 0 8 0 390968872 290779136 44173 33554432000 134512640 146736320 4293985856 4035955336 135525908 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 1520.95
Current children cumulated vsize (KiB) 293112

[startup+522.3 s]
/proc/loadavg: 13.21 11.55 8.70 13/193 3404
/proc/meminfo: memFree=24808844/32873660 swapFree=7120/7120
[pid=3276] ppid=3274 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/3276/stat : 3276 (run) S 3274 3276 2263 0 -1 4202496 354 0 0 0 0 0 0 0 20 0 1 0 390966792 9367552 269 33554432000 4194304 5064252 140734767505968 140734767504880 270329865806 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/3276/statm: 2287 269 229 213 0 72 0
[pid=3277] ppid=3276 vsize=293324 CPUtime=1760.58 cores=0,2,4,6
/proc/3277/stat : 3277 (pb_cplex) R 3276 3276 2263 0 -1 4202496 597004 0 0 0 175743 315 0 0 20 0 8 0 390966792 300363776 47415 33554432000 134512640 146736320 4293985856 4293972924 135526863 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/3277/statm: 73331 47415 983 2985 0 70265 0
[pid=3277/tid=3293] ppid=3276 vsize=293324 CPUtime=212.99 cores=0,2,4,6
/proc/3277/task/3293/stat : 3293 (pb_cplex) S 3276 3276 2263 0 -1 4202560 84476 0 0 0 21256 43 0 0 20 0 8 0 390968872 300363776 47415 33554432000 134512640 146736320 4293985856 4130397840 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=3277/tid=3294] ppid=3276 vsize=293324 CPUtime=207.77 cores=0,2,4,6
/proc/3277/task/3294/stat : 3294 (pb_cplex) S 3276 3276 2263 0 -1 4202560 59384 0 0 0 20736 41 0 0 20 0 8 0 390968872 300363776 47415 33554432000 134512640 146736320 4293985856 4140883600 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=3277/tid=3295] ppid=3276 vsize=293324 CPUtime=213.12 cores=0,2,4,6
/proc/3277/task/3295/stat : 3295 (pb_cplex) R 3276 3276 2263 0 -1 4202560 15919 0 0 0 21284 28 0 0 20 0 8 0 390968872 300363776 47415 33554432000 134512640 146736320 4293985856 4151359848 139044580 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=3277/tid=3296] ppid=3276 vsize=293324 CPUtime=214.32 cores=0,2,4,6
/proc/3277/task/3296/stat : 3296 (pb_cplex) R 3276 3276 2263 0 -1 4202560 56465 0 0 0 21395 37 0 0 20 0 8 0 390968872 300363776 47415 33554432000 134512640 146736320 4293985856 4111244880 136008172 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=3277/tid=3297] ppid=3276 vsize=293324 CPUtime=220.01 cores=0,2,4,6
/proc/3277/task/3297/stat : 3297 (pb_cplex) R 3276 3276 2263 0 -1 4202560 90868 0 0 0 21958 43 0 0 20 0 8 0 390968872 300363776 47415 33554432000 134512640 146736320 4293985856 4056928208 136403109 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
[pid=3277/tid=3298] ppid=3276 vsize=293324 CPUtime=213.76 cores=0,2,4,6
/proc/3277/task/3298/stat : 3298 (pb_cplex) R 3276 3276 2263 0 -1 4202560 127110 0 0 0 21326 50 0 0 20 0 8 0 390968872 300363776 47415 33554432000 134512640 146736320 4293985856 4046441480 136002881 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=3277/tid=3299] ppid=3276 vsize=293324 CPUtime=213.24 cores=0,2,4,6
/proc/3277/task/3299/stat : 3299 (pb_cplex) R 3276 3276 2263 0 -1 4202560 85001 0 0 0 21279 45 0 0 20 0 8 0 390968872 300363776 47415 33554432000 134512640 146736320 4293985856 4035962200 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 1760.58
Current children cumulated vsize (KiB) 302472



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+535.9 s]
/proc/loadavg: 12.31 11.43 8.70 11/193 3404
/proc/meminfo: memFree=24824716/32873660 swapFree=7120/7120
[pid=3276] ppid=3274 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/3276/stat : 3276 (run) S 3274 3276 2263 0 -1 4202496 354 0 0 0 0 0 0 0 20 0 1 0 390966792 9367552 269 33554432000 4194304 5064252 140734767505968 140734767504880 270329865806 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/3276/statm: 2287 269 229 213 0 72 0
[pid=3277] ppid=3276 vsize=272888 CPUtime=1800.41 cores=0,2,4,6
/proc/3277/stat : 3277 (pb_cplex) S 3276 3276 2263 0 -1 4202496 598895 0 0 0 179713 328 0 0 20 0 8 0 390966792 279437312 43217 33554432000 134512640 146736320 4293985856 4293982224 144580073 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/3277/statm: 68222 43217 991 2985 0 65156 0
[pid=3277/tid=3293] ppid=3276 vsize=272888 CPUtime=215.48 cores=0,2,4,6
/proc/3277/task/3293/stat : 3293 (pb_cplex) R 3276 3276 2263 0 -1 4202560 84476 0 0 0 21505 43 0 0 20 0 8 0 390968872 279437312 43217 33554432000 134512640 146736320 4293985856 4130395112 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=3277/tid=3294] ppid=3276 vsize=272888 CPUtime=210.33 cores=0,2,4,6
/proc/3277/task/3294/stat : 3294 (pb_cplex) R 3276 3276 2263 0 -1 4202560 59384 0 0 0 20991 42 0 0 20 0 8 0 390968872 279437312 43217 33554432000 134512640 146736320 4293985856 4140880080 136008172 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
[pid=3277/tid=3295] ppid=3276 vsize=272888 CPUtime=217.99 cores=0,2,4,6
/proc/3277/task/3295/stat : 3295 (pb_cplex) S 3276 3276 2263 0 -1 4202560 16639 0 0 0 21769 30 0 0 20 0 8 0 390968872 279437312 43217 33554432000 134512640 146736320 4293985856 4151369360 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=3277/tid=3296] ppid=3276 vsize=272888 CPUtime=217.75 cores=0,2,4,6
/proc/3277/task/3296/stat : 3296 (pb_cplex) R 3276 3276 2263 0 -1 4202560 56505 0 0 0 21736 39 0 0 20 0 8 0 390968872 279437312 43217 33554432000 134512640 146736320 4293985856 4111249688 137056672 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=3277/tid=3297] ppid=3276 vsize=272888 CPUtime=227.22 cores=0,2,4,6
/proc/3277/task/3297/stat : 3297 (pb_cplex) R 3276 3276 2263 0 -1 4202560 90910 0 0 0 22676 46 0 0 20 0 8 0 390968872 279437312 43217 33554432000 134512640 146736320 4293985856 4056932560 136008450 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=3277/tid=3298] ppid=3276 vsize=272888 CPUtime=223.22 cores=0,2,4,6
/proc/3277/task/3298/stat : 3298 (pb_cplex) R 3276 3276 2263 0 -1 4202560 128143 0 0 0 22269 53 0 0 20 0 8 0 390968872 279437312 43217 33554432000 134512640 146736320 4293985856 4046447592 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=3277/tid=3299] ppid=3276 vsize=272888 CPUtime=215.42 cores=0,2,4,6
/proc/3277/task/3299/stat : 3299 (pb_cplex) R 3276 3276 2263 0 -1 4202560 85017 0 0 0 21497 45 0 0 20 0 8 0 390968872 279437312 43217 33554432000 134512640 146736320 4293985856 4035960832 136019021 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1800.41
Current children cumulated vsize (KiB) 282036

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+535.9 s]
/proc/loadavg: 12.31 11.43 8.70 11/193 3404
/proc/meminfo: memFree=24824716/32873660 swapFree=7120/7120
[pid=3276] ppid=3274 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/3276/stat : 3276 (run) S 3274 3276 2263 0 -1 4202496 354 0 0 0 0 0 0 0 20 0 1 0 390966792 9367552 269 33554432000 4194304 5064252 140734767505968 140734767504880 270329865806 0 65536 4100 65538 18446744071579302229 0 0 17 4 0 0 0 0 0
/proc/3276/statm: 2287 269 229 213 0 72 0
[pid=3277] ppid=3276 vsize=272888 CPUtime=1800.41 cores=0,2,4,6
/proc/3277/stat : 3277 (pb_cplex) S 3276 3276 2263 0 -1 4202496 598895 0 0 0 179713 328 0 0 20 0 8 0 390966792 279437312 43217 33554432000 134512640 146736320 4293985856 4293982224 144580073 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/3277/statm: 68222 43217 991 2985 0 65156 0
[pid=3277/tid=3293] ppid=3276 vsize=272888 CPUtime=215.48 cores=0,2,4,6
/proc/3277/task/3293/stat : 3293 (pb_cplex) R 3276 3276 2263 0 -1 4202560 84476 0 0 0 21505 43 0 0 20 0 8 0 390968872 279437312 43217 33554432000 134512640 146736320 4293985856 4130395112 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=3277/tid=3294] ppid=3276 vsize=272888 CPUtime=210.33 cores=0,2,4,6
/proc/3277/task/3294/stat : 3294 (pb_cplex) R 3276 3276 2263 0 -1 4202560 59384 0 0 0 20991 42 0 0 20 0 8 0 390968872 279437312 43217 33554432000 134512640 146736320 4293985856 4140880080 136008172 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
[pid=3277/tid=3295] ppid=3276 vsize=272888 CPUtime=217.99 cores=0,2,4,6
/proc/3277/task/3295/stat : 3295 (pb_cplex) S 3276 3276 2263 0 -1 4202560 16639 0 0 0 21769 30 0 0 20 0 8 0 390968872 279437312 43217 33554432000 134512640 146736320 4293985856 4151369360 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=3277/tid=3296] ppid=3276 vsize=272888 CPUtime=217.75 cores=0,2,4,6
/proc/3277/task/3296/stat : 3296 (pb_cplex) R 3276 3276 2263 0 -1 4202560 56505 0 0 0 21736 39 0 0 20 0 8 0 390968872 279437312 43217 33554432000 134512640 146736320 4293985856 4111249688 137056672 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=3277/tid=3297] ppid=3276 vsize=272888 CPUtime=227.22 cores=0,2,4,6
/proc/3277/task/3297/stat : 3297 (pb_cplex) R 3276 3276 2263 0 -1 4202560 90910 0 0 0 22676 46 0 0 20 0 8 0 390968872 279437312 43217 33554432000 134512640 146736320 4293985856 4056932560 136008450 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=3277/tid=3298] ppid=3276 vsize=272888 CPUtime=223.22 cores=0,2,4,6
/proc/3277/task/3298/stat : 3298 (pb_cplex) R 3276 3276 2263 0 -1 4202560 128143 0 0 0 22269 53 0 0 20 0 8 0 390968872 279437312 43217 33554432000 134512640 146736320 4293985856 4046447592 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=3277/tid=3299] ppid=3276 vsize=272888 CPUtime=215.42 cores=0,2,4,6
/proc/3277/task/3299/stat : 3299 (pb_cplex) R 3276 3276 2263 0 -1 4202560 85017 0 0 0 21497 45 0 0 20 0 8 0 390968872 279437312 43217 33554432000 134512640 146736320 4293985856 4035960832 136019021 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1800.41
Current children cumulated vsize (KiB) 282036

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 3276 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=0
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=1999
# CPU time returned by wait4() is 0.001999
# while last known CPU time is 1800.41
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 535.916
CPU time (s): 1800.41
CPU user time (s): 1797.13
CPU system time (s): 3.28
CPU usage (%): 335.95
Max. virtual memory (cumulated for all children) (KiB): 315784

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0
system time used= 0.001999
maximum resident set size= 1076
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 354
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2
involuntary context switches= 2

runsolver used 1.46078 second user time and 3.49547 second system time

The end

Launcher Data 

Begin job on node117 at 2012-06-09 15:26:50
IDJOB=3736952
IDBENCH=48151
IDSOLVER=2325
FILE ID=node117/3736952-1339248410
RUNJOBID= node117-1339246834-2279
PBS_JOBID= 14644364
Free space on /tmp= 68524 MiB

SOLVER NAME= PB10: pb_cplex 2010-06-29
BENCH NAME= PB07/normalized-PB07/OPT-SMALLINT-NLC/submittedPB07/manquinho/mds/normalized-mds_200_10_2.opb
COMMAND LINE= run BENCHNAME
CONVERSION SCRIPT= PBconversionToLinear BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3736952-1339248410/watcher-3736952-1339248410 -o /tmp/evaluation-result-3736952-1339248410/solver-3736952-1339248410 -C 1800 -W 1900 -M 15500  run HOME/instance-3736952-1339248410.opb

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 54e50bd6cbfbb2a5705269c41241de9a
RANDOM SEED=788574415

node117.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.148
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.29
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.148
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.148
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.148
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.95
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.148
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.148
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.148
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.148
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873660 kB
MemFree:        24231040 kB
Buffers:          863168 kB
Cached:          5908620 kB
SwapCached:          620 kB
Active:          5659232 kB
Inactive:        1999920 kB
Active(anon):     875040 kB
Inactive(anon):    10028 kB
Active(file):    4784192 kB
Inactive(file):  1989892 kB
Unevictable:        7120 kB
Mlocked:            7120 kB
SwapTotal:      67108856 kB
SwapFree:       67106956 kB
Dirty:             16736 kB
Writeback:             0 kB
AnonPages:        889196 kB
Mapped:            16116 kB
Shmem:                80 kB
Slab:             842204 kB
SReclaimable:     286244 kB
SUnreclaim:       555960 kB
KernelStack:        1504 kB
PageTables:         6000 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545684 kB
Committed_AS:    1123244 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346400 kB
VmallocChunk:   34341988052 kB
HardwareCorrupted:     0 kB
AnonHugePages:    634880 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7680 kB
DirectMap2M:    33538048 kB

CONVERSION COMMAND LINE= PBconversionToLinear HOME/instance-3736952-1339248410.opb

Free space on /tmp at the end= 68516 MiB
End job on node117 at 2012-06-09 15:35:48