Trace number 3737524

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
PB10: pb_cplex 2010-06-29? (TO) 1800.14 542.323

General information on the benchmark

Namenormalized-PB07/OPT-SMALLINT-NLC/submittedPB07/
manquinho/bsg/normalized-bsg_100_10_1.opb
MD5SUM8c398fbb12ba7dbf676dd8375a76e0de
Bench CategoryOPT-SMALLINT-NLC (optimisation, small integers, non linear constraints)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark-28
Best CPU time to get the best result obtained on this benchmark1789.76
Has Objective FunctionYES
SatisfiableYES
(Un)Satisfiability was provedYES
Best value of the objective function -30
Optimality of the best value was proved NO
Number of variables200
Total number of constraints301
Number of constraints which are clauses100
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints201
Minimum length of a constraint2
Maximum length of a constraint200
Number of terms in the objective function 100
Biggest coefficient in the objective function 1
Number of bits for the biggest coefficient in the objective function 1
Sum of the numbers in the objective function 100
Number of bits of the sum of numbers in the objective function 7
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 200
Number of bits of the biggest sum of numbers8
Number of products (including duplicates)2496
Sum of products size (including duplicates)4992
Number of different products1248
Sum of products size2496

Solver Data 

0.00/0.00	c pb_cplex 1.0 - 20100701
0.00/0.00	c Parsing opb file format. File HOME/instance-3737524-1339248680.opb.
0.00/0.00	c Instance file HOME/instance-3737524-1339248680.opb
0.00/0.00	c File size is 106648 bytes.
0.00/0.02	c Highest Coefficient sum: 200
0.00/0.02	c  IBM ILOG License Manager: "IBM ILOG Optimization Suite for Academic Initiative" is accessing CPLEX 12 with option(s): "e m b q ".
0.00/0.02	c ============================[ Problem Statistics ]=============================
0.00/0.02	c |                                                                             |
0.00/0.02	c |  Parsing time:          0.02         s                                      |
0.00/0.02	c |  Number Variables:      1448                                                |
0.00/0.02	c |  Number Constraints:    2797                                                |
0.00/0.02	c ===============================================================================
1798.89/542.31	c Remaining time 1799.98 sec.
1798.89/542.31	c 
1798.89/542.31	c *** INTERRUPTED ***
1798.89/542.31	s UNKNOWN
1798.89/542.31	c Total CPU time              : 1794.72 s
1798.89/542.31	c Memory used           : 250.91 MB
1798.89/542.31	c CPU time              : 1794.72 s
1798.89/542.31	c 
1798.89/542.31	c *** INTERRUPTED ***

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3737524-1339248680/watcher-3737524-1339248680 -o /tmp/evaluation-result-3737524-1339248680/solver-3737524-1339248680 -C 1800 -W 1900 -M 15500 run HOME/instance-3737524-1339248680.opb 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 15.51 10.85 8.23 10/183 5372
/proc/meminfo: memFree=29312880/32873844 swapFree=7628/7628
[pid=5372] ppid=5370 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/5372/stat : 5372 (run) S 5370 5372 4561 0 -1 4202496 353 0 0 0 0 0 0 0 20 0 1 0 390992965 9367552 269 33554432000 4194304 5064252 140736509946816 140736509945728 240564987470 0 65536 4100 65538 18446744071579302229 0 0 17 0 0 0 0 0 0
/proc/5372/statm: 2287 269 229 213 0 72 0

[startup+0.0515761 s]
/proc/loadavg: 15.51 10.85 8.23 10/183 5372
/proc/meminfo: memFree=29312880/32873844 swapFree=7628/7628
[pid=5372] ppid=5370 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/5372/stat : 5372 (run) S 5370 5372 4561 0 -1 4202496 353 0 0 0 0 0 0 0 20 0 1 0 390992965 9367552 269 33554432000 4194304 5064252 140736509946816 140736509945728 240564987470 0 65536 4100 65538 18446744071579302229 0 0 17 0 0 0 0 0 0
/proc/5372/statm: 2287 269 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.100292 s]
/proc/loadavg: 15.51 10.85 8.23 10/183 5372
/proc/meminfo: memFree=29312880/32873844 swapFree=7628/7628
[pid=5372] ppid=5370 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/5372/stat : 5372 (run) S 5370 5372 4561 0 -1 4202496 353 0 0 0 0 0 0 0 20 0 1 0 390992965 9367552 269 33554432000 4194304 5064252 140736509946816 140736509945728 240564987470 0 65536 4100 65538 18446744071579302229 0 0 17 0 0 0 0 0 0
/proc/5372/statm: 2287 269 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.300246 s]
/proc/loadavg: 15.51 10.85 8.23 10/183 5372
/proc/meminfo: memFree=29312880/32873844 swapFree=7628/7628
[pid=5372] ppid=5370 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/5372/stat : 5372 (run) S 5370 5372 4561 0 -1 4202496 353 0 0 0 0 0 0 0 20 0 1 0 390992965 9367552 269 33554432000 4194304 5064252 140736509946816 140736509945728 240564987470 0 65536 4100 65538 18446744071579302229 0 0 17 0 0 0 0 0 0
/proc/5372/statm: 2287 269 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+0.700247 s]
/proc/loadavg: 15.51 10.85 8.23 10/183 5372
/proc/meminfo: memFree=29312880/32873844 swapFree=7628/7628
[pid=5372] ppid=5370 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/5372/stat : 5372 (run) S 5370 5372 4561 0 -1 4202496 353 0 0 0 0 0 0 0 20 0 1 0 390992965 9367552 269 33554432000 4194304 5064252 140736509946816 140736509945728 240564987470 0 65536 4100 65538 18446744071579302229 0 0 17 0 0 0 0 0 0
/proc/5372/statm: 2287 269 229 213 0 72 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9148

[startup+1.50027 s]
/proc/loadavg: 15.51 10.85 8.23 10/185 5374
/proc/meminfo: memFree=29303720/32873844 swapFree=7628/7628
[pid=5372] ppid=5370 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/5372/stat : 5372 (run) S 5370 5372 4561 0 -1 4202496 353 0 0 0 0 0 0 0 20 0 1 0 390992965 9367552 269 33554432000 4194304 5064252 140736509946816 140736509945728 240564987470 0 65536 4100 65538 18446744071579302229 0 0 17 0 0 0 0 0 0
/proc/5372/statm: 2287 269 229 213 0 72 0
[pid=5373] ppid=5372 vsize=22540 CPUtime=1.49 cores=0,2,4,6
/proc/5373/stat : 5373 (pb_cplex) R 5372 5372 4561 0 -1 4202496 5218 0 0 0 148 1 0 0 20 0 1 0 390992965 23080960 2872 33554432000 134512640 146736320 4287342640 4287336848 136403109 0 0 4096 16387 0 0 0 17 4 0 0 0 0 0
/proc/5373/statm: 5635 2872 903 2985 0 2569 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 31688

[startup+3.10027 s]
/proc/loadavg: 15.51 10.85 8.23 10/185 5374
/proc/meminfo: memFree=29304340/32873844 swapFree=7628/7628
[pid=5372] ppid=5370 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/5372/stat : 5372 (run) S 5370 5372 4561 0 -1 4202496 353 0 0 0 0 0 0 0 20 0 1 0 390992965 9367552 269 33554432000 4194304 5064252 140736509946816 140736509945728 240564987470 0 65536 4100 65538 18446744071579302229 0 0 17 0 0 0 0 0 0
/proc/5372/statm: 2287 269 229 213 0 72 0
[pid=5373] ppid=5372 vsize=23264 CPUtime=3.09 cores=0,2,4,6
/proc/5373/stat : 5373 (pb_cplex) R 5372 5372 4561 0 -1 4202496 5343 0 0 0 308 1 0 0 20 0 1 0 390992965 23822336 2997 33554432000 134512640 146736320 4287342640 4287336568 136015725 0 0 4096 16387 0 0 0 17 4 0 0 0 0 0
/proc/5373/statm: 5816 2997 910 2985 0 2750 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 32412

[startup+6.30045 s]
/proc/loadavg: 14.99 10.82 8.23 10/193 5396
/proc/meminfo: memFree=29289248/32873844 swapFree=7628/7628
[pid=5372] ppid=5370 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/5372/stat : 5372 (run) S 5370 5372 4561 0 -1 4202496 353 0 0 0 0 0 0 0 20 0 1 0 390992965 9367552 269 33554432000 4194304 5064252 140736509946816 140736509945728 240564987470 0 65536 4100 65538 18446744071579302229 0 0 17 0 0 0 0 0 0
/proc/5372/statm: 2287 269 229 213 0 72 0
[pid=5373] ppid=5372 vsize=116400 CPUtime=7.79 cores=0,2,4,6
/proc/5373/stat : 5373 (pb_cplex) R 5372 5372 4561 0 -1 4202496 17131 0 0 0 762 17 0 0 20 0 8 0 390992965 119193600 6547 33554432000 134512640 146736320 4287342640 4287335364 136385179 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0 0 0
/proc/5373/statm: 29100 6547 965 2985 0 26034 0
[pid=5373/tid=5390] ppid=5372 vsize=116400 CPUtime=0 cores=0,2,4,6
/proc/5373/task/5390/stat : 5390 (pb_cplex) S 5372 5372 4561 0 -1 4202560 0 0 0 0 0 0 0 0 20 0 8 0 390993455 119193600 6547 33554432000 134512640 146736320 4287342640 4067421840 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=5373/tid=5391] ppid=5372 vsize=116400 CPUtime=0 cores=0,2,4,6
/proc/5373/task/5391/stat : 5391 (pb_cplex) S 5372 5372 4561 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 8 0 390993455 119193600 6547 33554432000 134512640 146736320 4287342640 4130336400 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=5373/tid=5392] ppid=5372 vsize=116400 CPUtime=0 cores=0,2,4,6
/proc/5373/task/5392/stat : 5392 (pb_cplex) S 5372 5372 4561 0 -1 4202560 0 0 0 0 0 0 0 0 20 0 8 0 390993455 119193600 6547 33554432000 134512640 146736320 4287342640 4140822160 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=5373/tid=5393] ppid=5372 vsize=116400 CPUtime=0 cores=0,2,4,6
/proc/5373/task/5393/stat : 5393 (pb_cplex) S 5372 5372 4561 0 -1 4202560 2 0 0 0 0 0 0 0 20 0 8 0 390993455 119193600 6547 33554432000 134512640 146736320 4287342640 4117323408 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=5373/tid=5394] ppid=5372 vsize=116400 CPUtime=0 cores=0,2,4,6
/proc/5373/task/5394/stat : 5394 (pb_cplex) S 5372 5372 4561 0 -1 4202560 3 0 0 0 0 0 0 0 20 0 8 0 390993455 119193600 6547 33554432000 134512640 146736320 4287342640 4106837648 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=5373/tid=5395] ppid=5372 vsize=116400 CPUtime=0 cores=0,2,4,6
/proc/5373/task/5395/stat : 5395 (pb_cplex) S 5372 5372 4561 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 8 0 390993455 119193600 6547 33554432000 134512640 146736320 4287342640 4096351888 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=5373/tid=5396] ppid=5372 vsize=116400 CPUtime=0 cores=0,2,4,6
/proc/5373/task/5396/stat : 5396 (pb_cplex) S 5372 5372 4561 0 -1 4202560 2 0 0 0 0 0 0 0 20 0 8 0 390993455 119193600 6547 33554432000 134512640 146736320 4287342640 4085866128 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 7.79
Current children cumulated vsize (KiB) 125548

[startup+12.7005 s]
/proc/loadavg: 14.51 10.79 8.23 17/193 5396
/proc/meminfo: memFree=29275796/32873844 swapFree=7628/7628
[pid=5372] ppid=5370 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/5372/stat : 5372 (run) S 5370 5372 4561 0 -1 4202496 353 0 0 0 0 0 0 0 20 0 1 0 390992965 9367552 269 33554432000 4194304 5064252 140736509946816 140736509945728 240564987470 0 65536 4100 65538 18446744071579302229 0 0 17 0 0 0 0 0 0
/proc/5372/statm: 2287 269 229 213 0 72 0
[pid=5373] ppid=5372 vsize=134436 CPUtime=17.51 cores=0,2,4,6
/proc/5373/stat : 5373 (pb_cplex) R 5372 5372 4561 0 -1 4202496 21525 0 0 0 1732 19 0 0 20 0 8 0 390992965 137662464 10437 33554432000 134512640 146736320 4287342640 4287335936 139046460 0 0 4096 16387 18446744073709551615 0 0 17 6 0 0 0 0 0
/proc/5373/statm: 33609 10437 965 2985 0 30543 0
[pid=5373/tid=5390] ppid=5372 vsize=134436 CPUtime=0.56 cores=0,2,4,6
/proc/5373/task/5390/stat : 5390 (pb_cplex) R 5372 5372 4561 0 -1 4202560 685 0 0 0 56 0 0 0 20 0 8 0 390993455 137662464 10437 33554432000 134512640 146736320 4287342640 4067420224 136688079 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=5373/tid=5391] ppid=5372 vsize=134436 CPUtime=0.67 cores=0,2,4,6
/proc/5373/task/5391/stat : 5391 (pb_cplex) R 5372 5372 4561 0 -1 4202560 445 0 0 0 67 0 0 0 20 0 8 0 390993455 137662464 10437 33554432000 134512640 146736320 4287342640 4130332176 136403109 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
[pid=5373/tid=5392] ppid=5372 vsize=134436 CPUtime=0.39 cores=0,2,4,6
/proc/5373/task/5392/stat : 5392 (pb_cplex) R 5372 5372 4561 0 -1 4202560 519 0 0 0 39 0 0 0 20 0 8 0 390993455 137662464 10437 33554432000 134512640 146736320 4287342640 4140817112 135466439 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=5373/tid=5393] ppid=5372 vsize=134436 CPUtime=0.62 cores=0,2,4,6
/proc/5373/task/5393/stat : 5393 (pb_cplex) R 5372 5372 4561 0 -1 4202560 742 0 0 0 62 0 0 0 20 0 8 0 390993455 137662464 10437 33554432000 134512640 146736320 4287342640 4117318828 135454141 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=5373/tid=5394] ppid=5372 vsize=134436 CPUtime=0.5 cores=0,2,4,6
/proc/5373/task/5394/stat : 5394 (pb_cplex) R 5372 5372 4561 0 -1 4202560 579 0 0 0 50 0 0 0 20 0 8 0 390993455 137662464 10437 33554432000 134512640 146736320 4287342640 4106833040 136008245 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=5373/tid=5395] ppid=5372 vsize=134436 CPUtime=0.46 cores=0,2,4,6
/proc/5373/task/5395/stat : 5395 (pb_cplex) R 5372 5372 4561 0 -1 4202560 611 0 0 0 46 0 0 0 20 0 8 0 390993455 137662464 10435 33554432000 134512640 146736320 4287342640 4096348368 136008299 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
[pid=5373/tid=5396] ppid=5372 vsize=134436 CPUtime=0.52 cores=0,2,4,6
/proc/5373/task/5396/stat : 5396 (pb_cplex) R 5372 5372 4561 0 -1 4202560 661 0 0 0 52 0 0 0 20 0 8 0 390993455 137662464 10435 33554432000 134512640 146736320 4287342640 4085861904 136403112 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 17.51
Current children cumulated vsize (KiB) 143584

[startup+25.5004 s]
/proc/loadavg: 14.84 11.05 8.36 17/193 5396
/proc/meminfo: memFree=29265924/32873844 swapFree=7628/7628
[pid=5372] ppid=5370 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/5372/stat : 5372 (run) S 5370 5372 4561 0 -1 4202496 353 0 0 0 0 0 0 0 20 0 1 0 390992965 9367552 269 33554432000 4194304 5064252 140736509946816 140736509945728 240564987470 0 65536 4100 65538 18446744071579302229 0 0 17 0 0 0 0 0 0
/proc/5372/statm: 2287 269 229 213 0 72 0
[pid=5373] ppid=5372 vsize=143088 CPUtime=66.19 cores=0,2,4,6
/proc/5373/stat : 5373 (pb_cplex) R 5372 5372 4561 0 -1 4202496 25969 0 0 0 6594 25 0 0 20 0 8 0 390992965 146522112 12649 33554432000 134512640 146736320 4287342640 4287335504 136008309 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/5373/statm: 35772 12649 965 2985 0 32706 0
[pid=5373/tid=5390] ppid=5372 vsize=143088 CPUtime=6.43 cores=0,2,4,6
/proc/5373/task/5390/stat : 5390 (pb_cplex) R 5372 5372 4561 0 -1 4202560 1229 0 0 0 642 1 0 0 20 0 8 0 390993455 146522112 12649 33554432000 134512640 146736320 4287342640 4067417176 136007062 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=5373/tid=5391] ppid=5372 vsize=143088 CPUtime=5.53 cores=0,2,4,6
/proc/5373/task/5391/stat : 5391 (pb_cplex) R 5372 5372 4561 0 -1 4202560 852 0 0 0 553 0 0 0 20 0 8 0 390993455 146522112 12649 33554432000 134512640 146736320 4287342640 4130332744 139042902 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=5373/tid=5392] ppid=5372 vsize=143088 CPUtime=6.37 cores=0,2,4,6
/proc/5373/task/5392/stat : 5392 (pb_cplex) R 5372 5372 4561 0 -1 4202560 1197 0 0 0 636 1 0 0 20 0 8 0 390993455 146522112 12649 33554432000 134512640 146736320 4287342640 4140819024 136403129 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=5373/tid=5393] ppid=5372 vsize=143088 CPUtime=7.17 cores=0,2,4,6
/proc/5373/task/5393/stat : 5393 (pb_cplex) R 5372 5372 4561 0 -1 4202560 1281 0 0 0 717 0 0 0 20 0 8 0 390993455 146522112 12649 33554432000 134512640 146736320 4287342640 4117320272 136403112 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
[pid=5373/tid=5394] ppid=5372 vsize=143088 CPUtime=7.7 cores=0,2,4,6
/proc/5373/task/5394/stat : 5394 (pb_cplex) R 5372 5372 4561 0 -1 4202560 869 0 0 0 769 1 0 0 20 0 8 0 390993455 146522112 12649 33554432000 134512640 146736320 4287342640 4106831616 138463243 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=5373/tid=5395] ppid=5372 vsize=143088 CPUtime=5.85 cores=0,2,4,6
/proc/5373/task/5395/stat : 5395 (pb_cplex) R 5372 5372 4561 0 -1 4202560 1307 0 0 0 585 0 0 0 20 0 8 0 390993455 146522112 12649 33554432000 134512640 146736320 4287342640 4096348752 136403112 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=5373/tid=5396] ppid=5372 vsize=143088 CPUtime=7.56 cores=0,2,4,6
/proc/5373/task/5396/stat : 5396 (pb_cplex) R 5372 5372 4561 0 -1 4202560 1654 0 0 0 754 2 0 0 20 0 8 0 390993455 146522112 12649 33554432000 134512640 146736320 4287342640 4085862992 136403129 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
Current children cumulated CPU time (s) 66.19
Current children cumulated vsize (KiB) 152236

############
# WARNING:
# current cumulated CPU time (120.86 s) is less than in the last sample (120.87 s)
# The time of a child was probably not reported to its father.
# (see the two samples below)
# Adding the difference (0.0100021 s) to the 'lost time'.

[startup+40.9005 s]
/proc/loadavg: 14.69 11.19 8.45 10/193 5396
/proc/meminfo: memFree=29263000/32873844 swapFree=7628/7628
[pid=5372] ppid=5370 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/5372/stat : 5372 (run) S 5370 5372 4561 0 -1 4202496 353 0 0 0 0 0 0 0 20 0 1 0 390992965 9367552 269 33554432000 4194304 5064252 140736509946816 140736509945728 240564987470 0 65536 4100 65538 18446744071579302229 0 0 17 0 0 0 0 0 0
/proc/5372/statm: 2287 269 229 213 0 72 0
[pid=5373] ppid=5372 vsize=143868 CPUtime=120.87 cores=0,2,4,6
/proc/5373/stat : 5373 (pb_cplex) S 5372 5372 4561 0 -1 4202496 26829 0 0 0 12054 33 0 0 20 0 8 0 390992965 147320832 13149 33554432000 134512640 146736320 4287342640 4287339024 144580073 0 0 4096 16387 18446744073709551615 0 0 17 6 0 0 0 0 0
/proc/5373/statm: 35967 13149 965 2985 0 32901 0
[pid=5373/tid=5390] ppid=5372 vsize=143868 CPUtime=12.9 cores=0,2,4,6
/proc/5373/task/5390/stat : 5390 (pb_cplex) S 5372 5372 4561 0 -1 4202560 1234 0 0 0 1289 1 0 0 20 0 8 0 390993455 147320832 13149 33554432000 134512640 146736320 4287342640 4067421840 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 0 0 0 0 0 0
[pid=5373/tid=5391] ppid=5372 vsize=143868 CPUtime=12.08 cores=0,2,4,6
/proc/5373/task/5391/stat : 5391 (pb_cplex) S 5372 5372 4561 0 -1 4202560 913 0 0 0 1207 1 0 0 20 0 8 0 390993455 147320832 13149 33554432000 134512640 146736320 4287342640 4130336400 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=5373/tid=5392] ppid=5372 vsize=143868 CPUtime=12.32 cores=0,2,4,6
/proc/5373/task/5392/stat : 5392 (pb_cplex) S 5372 5372 4561 0 -1 4202560 1211 0 0 0 1230 2 0 0 20 0 8 0 390993455 147320832 13149 33554432000 134512640 146736320 4287342640 4140822160 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=5373/tid=5393] ppid=5372 vsize=143868 CPUtime=14.57 cores=0,2,4,6
/proc/5373/task/5393/stat : 5393 (pb_cplex) S 5372 5372 4561 0 -1 4202560 1381 0 0 0 1455 2 0 0 20 0 8 0 390993455 147320832 13149 33554432000 134512640 146736320 4287342640 4117323408 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=5373/tid=5394] ppid=5372 vsize=143868 CPUtime=14.67 cores=0,2,4,6
/proc/5373/task/5394/stat : 5394 (pb_cplex) S 5372 5372 4561 0 -1 4202560 900 0 0 0 1465 2 0 0 20 0 8 0 390993455 147320832 13149 33554432000 134512640 146736320 4287342640 4106837648 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=5373/tid=5395] ppid=5372 vsize=143868 CPUtime=13.6 cores=0,2,4,6
/proc/5373/task/5395/stat : 5395 (pb_cplex) R 5372 5372 4561 0 -1 4202560 1783 0 0 0 1358 2 0 0 20 0 8 0 390993455 147320832 13149 33554432000 134512640 146736320 4287342640 4096348160 136018922 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=5373/tid=5396] ppid=5372 vsize=143868 CPUtime=13.96 cores=0,2,4,6
/proc/5373/task/5396/stat : 5396 (pb_cplex) S 5372 5372 4561 0 -1 4202560 1737 0 0 0 1394 2 0 0 20 0 8 0 390993455 147320832 13149 33554432000 134512640 146736320 4287342640 4085866128 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 120.87
Current children cumulated vsize (KiB) 153016

[startup+41.0003 s]

################
# More data... #
################

/proc/5373/task/5409/stat : 5409 (pb_cplex) S 5372 5372 4561 0 -1 4202560 17023 0 0 0 16482 29 0 0 20 0 8 0 390998643 232595456 33160 33554432000 134512640 146736320 4287342640 4056936080 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=5373/tid=5410] ppid=5372 vsize=227144 CPUtime=163.32 cores=0,2,4,6
/proc/5373/task/5410/stat : 5410 (pb_cplex) R 5372 5372 4561 0 -1 4202560 59412 0 0 0 16289 43 0 0 20 0 8 0 390998643 232595456 33160 33554432000 134512640 146736320 4287342640 4046450320 144580073 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
Current children cumulated CPU time (s) 1488.11
Current children cumulated vsize (KiB) 236292

[startup+454.2 s]
/proc/loadavg: 6.81 8.94 8.52 10/186 5478
/proc/meminfo: memFree=29631540/32873844 swapFree=7628/7628
[pid=5372] ppid=5370 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/5372/stat : 5372 (run) S 5370 5372 4561 0 -1 4202496 353 0 0 0 0 0 0 0 20 0 1 0 390992965 9367552 269 33554432000 4194304 5064252 140736509946816 140736509945728 240564987470 0 65536 4100 65538 18446744071579302229 0 0 17 0 0 0 0 0 0
/proc/5372/statm: 2287 269 229 213 0 72 0
[pid=5373] ppid=5372 vsize=236600 CPUtime=1486.87 cores=0,2,4,6
/proc/5373/stat : 5373 (pb_cplex) R 5372 5372 4561 0 -1 4202496 357679 0 0 0 148346 341 0 0 20 0 8 0 390992965 242278400 35226 33554432000 134512640 146736320 4287342640 4287335368 139042974 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/5373/statm: 59150 35226 981 2985 0 56084 0
[pid=5373/tid=5472] ppid=5372 vsize=236600 CPUtime=0.01 cores=0,2,4,6
/proc/5373/task/5472/stat : 5472 (pb_cplex) R 5372 5372 4561 0 -1 4202560 46 0 0 0 1 0 0 0 20 0 8 0 391038380 242278400 35226 33554432000 134512640 146736320 4287342640 4046446660 136385183 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
[pid=5373/tid=5473] ppid=5372 vsize=236600 CPUtime=0.02 cores=0,2,4,6
/proc/5373/task/5473/stat : 5473 (pb_cplex) R 5372 5372 4561 0 -1 4202560 20 0 0 0 2 0 0 0 20 0 8 0 391038380 242278400 35226 33554432000 134512640 146736320 4287342640 4056932944 136403116 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=5373/tid=5474] ppid=5372 vsize=236600 CPUtime=0.02 cores=0,2,4,6
/proc/5373/task/5474/stat : 5474 (pb_cplex) R 5372 5372 4561 0 -1 4202560 791 0 0 0 2 0 0 0 20 0 8 0 391038380 242278400 35226 33554432000 134512640 146736320 4287342640 4067418184 139042974 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=5373/tid=5475] ppid=5372 vsize=236600 CPUtime=0.01 cores=0,2,4,6
/proc/5373/task/5475/stat : 5475 (pb_cplex) R 5372 5372 4561 0 -1 4202560 45 0 0 0 1 0 0 0 20 0 8 0 391038380 242278400 35226 33554432000 134512640 146736320 4287342640 4117320680 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=5373/tid=5476] ppid=5372 vsize=236600 CPUtime=0 cores=0,2,4,6
/proc/5373/task/5476/stat : 5476 (pb_cplex) R 5372 5372 4561 0 -1 4202560 165 0 0 0 0 0 0 0 20 0 8 0 391038380 242278400 35226 33554432000 134512640 146736320 4287342640 4150211016 136000488 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
[pid=5373/tid=5477] ppid=5372 vsize=236600 CPUtime=0 cores=0,2,4,6
/proc/5373/task/5477/stat : 5477 (pb_cplex) R 5372 5372 4561 0 -1 4202560 699 0 0 0 0 0 0 0 20 0 8 0 391038380 242278400 35226 33554432000 134512640 146736320 4287342640 4139724856 136475864 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=5373/tid=5478] ppid=5372 vsize=236600 CPUtime=0 cores=0,2,4,6
/proc/5373/task/5478/stat : 5478 (pb_cplex) R 5372 5372 4561 0 -1 4202560 709 0 0 0 0 0 0 0 20 0 8 0 391038380 242278400 35226 33554432000 134512640 146736320 4287342640 3936346704 136403109 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
Current children cumulated CPU time (s) 1486.87
Current children cumulated vsize (KiB) 245748
#
############


[startup+462.3 s]
/proc/loadavg: 6.83 8.91 8.51 10/186 5478
/proc/meminfo: memFree=29612560/32873844 swapFree=7628/7628
[pid=5372] ppid=5370 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/5372/stat : 5372 (run) S 5370 5372 4561 0 -1 4202496 353 0 0 0 0 0 0 0 20 0 1 0 390992965 9367552 269 33554432000 4194304 5064252 140736509946816 140736509945728 240564987470 0 65536 4100 65538 18446744071579302229 0 0 17 0 0 0 0 0 0
/proc/5372/statm: 2287 269 229 213 0 72 0
[pid=5373] ppid=5372 vsize=250484 CPUtime=1518.4 cores=0,2,4,6
/proc/5373/stat : 5373 (pb_cplex) R 5372 5372 4561 0 -1 4202496 372166 0 0 0 151492 348 0 0 20 0 8 0 390992965 256495616 39358 33554432000 134512640 146736320 4287342640 4287335484 135453265 0 0 4096 16387 18446744073709551615 0 0 17 6 0 0 0 0 0
/proc/5373/statm: 62621 39358 981 2985 0 59555 0
[pid=5373/tid=5472] ppid=5372 vsize=250484 CPUtime=3.75 cores=0,2,4,6
/proc/5373/task/5472/stat : 5472 (pb_cplex) S 5372 5372 4561 0 -1 4202560 1919 0 0 0 375 0 0 0 20 0 8 0 391038380 256495616 39358 33554432000 134512640 146736320 4287342640 4046450320 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 4 0 0 0 0 0
[pid=5373/tid=5473] ppid=5372 vsize=250484 CPUtime=3.91 cores=0,2,4,6
/proc/5373/task/5473/stat : 5473 (pb_cplex) R 5372 5372 4561 0 -1 4202560 2308 0 0 0 391 0 0 0 20 0 8 0 391038380 256495616 39358 33554432000 134512640 146736320 4287342640 4056932168 135526330 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=5373/tid=5474] ppid=5372 vsize=250484 CPUtime=4.24 cores=0,2,4,6
/proc/5373/task/5474/stat : 5474 (pb_cplex) S 5372 5372 4561 0 -1 4202560 2975 0 0 0 422 2 0 0 20 0 8 0 391038380 256495616 39358 33554432000 134512640 146736320 4287342640 4067421840 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 6 0 0 0 0 0
[pid=5373/tid=5475] ppid=5372 vsize=250484 CPUtime=3.74 cores=0,2,4,6
/proc/5373/task/5475/stat : 5475 (pb_cplex) R 5372 5372 4561 0 -1 4202560 1880 0 0 0 374 0 0 0 20 0 8 0 391038380 256495616 39358 33554432000 134512640 146736320 4287342640 4117319720 139044585 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
[pid=5373/tid=5476] ppid=5372 vsize=250484 CPUtime=3.97 cores=0,2,4,6
/proc/5373/task/5476/stat : 5476 (pb_cplex) R 5372 5372 4561 0 -1 4202560 1915 0 0 0 395 2 0 0 20 0 8 0 391038380 256495616 39358 33554432000 134512640 146736320 4287342640 4150210776 136008673 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=5373/tid=5477] ppid=5372 vsize=250484 CPUtime=4.29 cores=0,2,4,6
/proc/5373/task/5477/stat : 5477 (pb_cplex) R 5372 5372 4561 0 -1 4202560 2204 0 0 0 428 1 0 0 20 0 8 0 391038380 256495616 39358 33554432000 134512640 146736320 4287342640 4139724892 135855165 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=5373/tid=5478] ppid=5372 vsize=250484 CPUtime=3.96 cores=0,2,4,6
/proc/5373/task/5478/stat : 5478 (pb_cplex) R 5372 5372 4561 0 -1 4202560 2242 0 0 0 396 0 0 0 20 0 8 0 391038380 256495616 39358 33554432000 134512640 146736320 4287342640 3936345276 135453303 0 2 4096 16387 0 0 0 -1 0 0 0 0 0 0
Current children cumulated CPU time (s) 1518.4
Current children cumulated vsize (KiB) 259632

[startup+522.3 s]
/proc/loadavg: 7.26 8.69 8.46 10/187 5481
/proc/meminfo: memFree=29604832/32873844 swapFree=7628/7628
[pid=5372] ppid=5370 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/5372/stat : 5372 (run) S 5370 5372 4561 0 -1 4202496 353 0 0 0 0 0 0 0 20 0 1 0 390992965 9367552 269 33554432000 4194304 5064252 140736509946816 140736509945728 240564987470 0 65536 4100 65538 18446744071579302229 0 0 17 0 0 0 0 0 0
/proc/5372/statm: 2287 269 229 213 0 72 0
[pid=5373] ppid=5372 vsize=257636 CPUtime=1731.63 cores=0,2,4,6
/proc/5373/stat : 5373 (pb_cplex) R 5372 5372 4561 0 -1 4202496 393292 0 0 0 172766 397 0 0 20 0 8 0 390992965 263819264 40842 33554432000 134512640 146736320 4287342640 4287335408 136384892 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/5373/statm: 64409 40842 981 2985 0 61343 0
[pid=5373/tid=5472] ppid=5372 vsize=257636 CPUtime=30.76 cores=0,2,4,6
/proc/5373/task/5472/stat : 5472 (pb_cplex) S 5372 5372 4561 0 -1 4202560 1947 0 0 0 3071 5 0 0 20 0 8 0 391038380 263819264 40842 33554432000 134512640 146736320 4287342640 4046450320 144580073 0 2 4096 16387 18446744071579510697 0 0 -1 2 0 0 0 0 0
[pid=5373/tid=5473] ppid=5372 vsize=257636 CPUtime=29.37 cores=0,2,4,6
/proc/5373/task/5473/stat : 5473 (pb_cplex) R 5372 5372 4561 0 -1 4202560 4978 0 0 0 2932 5 0 0 20 0 8 0 391038380 263819264 40842 33554432000 134512640 146736320 4287342640 4056932944 136403109 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
[pid=5373/tid=5474] ppid=5372 vsize=257636 CPUtime=30.19 cores=0,2,4,6
/proc/5373/task/5474/stat : 5474 (pb_cplex) R 5372 5372 4561 0 -1 4202560 4226 0 0 0 3012 7 0 0 20 0 8 0 391038380 263819264 40842 33554432000 134512640 146736320 4287342640 4067418320 136008450 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=5373/tid=5475] ppid=5372 vsize=257636 CPUtime=31.27 cores=0,2,4,6
/proc/5373/task/5475/stat : 5475 (pb_cplex) R 5372 5372 4561 0 -1 4202560 2118 0 0 0 3121 6 0 0 20 0 8 0 391038380 263819264 40842 33554432000 134512640 146736320 4287342640 4117320136 136000506 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
[pid=5373/tid=5476] ppid=5372 vsize=257636 CPUtime=31.56 cores=0,2,4,6
/proc/5373/task/5476/stat : 5476 (pb_cplex) R 5372 5372 4561 0 -1 4202560 5435 0 0 0 3147 9 0 0 20 0 8 0 391038380 263819264 40842 33554432000 134512640 146736320 4287342640 4150210768 136008069 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=5373/tid=5477] ppid=5372 vsize=257636 CPUtime=29.93 cores=0,2,4,6
/proc/5373/task/5477/stat : 5477 (pb_cplex) R 5372 5372 4561 0 -1 4202560 6933 0 0 0 2987 6 0 0 20 0 8 0 391038380 263819264 40842 33554432000 134512640 146736320 4287342640 4139723964 135526870 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=5373/tid=5478] ppid=5372 vsize=257636 CPUtime=29.44 cores=0,2,4,6
/proc/5373/task/5478/stat : 5478 (pb_cplex) R 5372 5372 4561 0 -1 4202560 6124 0 0 0 2937 7 0 0 20 0 8 0 391038380 263819264 40842 33554432000 134512640 146736320 4287342640 3936347112 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
Current children cumulated CPU time (s) 1731.63
Current children cumulated vsize (KiB) 266784



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+542.307 s]
/proc/loadavg: 7.79 8.70 8.47 10/187 5481
/proc/meminfo: memFree=29606264/32873844 swapFree=7628/7628
[pid=5372] ppid=5370 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/5372/stat : 5372 (run) S 5370 5372 4561 0 -1 4202496 353 0 0 0 0 0 0 0 20 0 1 0 390992965 9367552 269 33554432000 4194304 5064252 140736509946816 140736509945728 240564987470 0 65536 4100 65538 18446744071579302229 0 0 17 0 0 0 0 0 0
/proc/5372/statm: 2287 269 229 213 0 72 0
[pid=5373] ppid=5372 vsize=256928 CPUtime=1798.89 cores=0,2,4,6
/proc/5373/stat : 5373 (pb_cplex) R 5372 5372 4561 0 -1 4202496 401526 0 0 0 179469 420 0 0 20 0 8 0 390992965 263094272 41197 33554432000 134512640 146736320 4287342640 4287335888 136403109 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/5373/statm: 64232 41197 981 2985 0 61166 0
[pid=5373/tid=5472] ppid=5372 vsize=256928 CPUtime=38.12 cores=0,2,4,6
/proc/5373/task/5472/stat : 5472 (pb_cplex) R 5372 5372 4561 0 -1 4202560 1947 0 0 0 3805 7 0 0 20 0 8 0 391038380 263094272 41197 33554432000 134512640 146736320 4287342640 4046445740 135454093 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=5373/tid=5473] ppid=5372 vsize=256928 CPUtime=37.42 cores=0,2,4,6
/proc/5373/task/5473/stat : 5473 (pb_cplex) R 5372 5372 4561 0 -1 4202560 5536 0 0 0 3735 7 0 0 20 0 8 0 391038380 263094272 41197 33554432000 134512640 146736320 4287342640 4056931516 135526859 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
[pid=5373/tid=5474] ppid=5372 vsize=256928 CPUtime=39.35 cores=0,2,4,6
/proc/5373/task/5474/stat : 5474 (pb_cplex) R 5372 5372 4561 0 -1 4202560 4226 0 0 0 3921 14 0 0 20 0 8 0 391038380 263094272 41197 33554432000 134512640 146736320 4287342640 4067418316 136388879 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=5373/tid=5475] ppid=5372 vsize=256928 CPUtime=39.24 cores=0,2,4,6
/proc/5373/task/5475/stat : 5475 (pb_cplex) R 5372 5372 4561 0 -1 4202560 2118 0 0 0 3916 8 0 0 20 0 8 0 391038380 263094272 41197 33554432000 134512640 146736320 4287342640 4117319888 136008167 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=5373/tid=5476] ppid=5372 vsize=256928 CPUtime=40.11 cores=0,2,4,6
/proc/5373/task/5476/stat : 5476 (pb_cplex) R 5372 5372 4561 0 -1 4202560 8192 0 0 0 4000 11 0 0 20 0 8 0 391038380 263094272 41197 33554432000 134512640 146736320 4287342640 4150210632 139042981 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=5373/tid=5477] ppid=5372 vsize=256928 CPUtime=39.19 cores=0,2,4,6
/proc/5373/task/5477/stat : 5477 (pb_cplex) R 5372 5372 4561 0 -1 4202560 9235 0 0 0 3910 9 0 0 20 0 8 0 391038380 263094272 41197 33554432000 134512640 146736320 4287342640 4139725392 136403109 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=5373/tid=5478] ppid=5372 vsize=256928 CPUtime=38.1 cores=0,2,4,6
/proc/5373/task/5478/stat : 5478 (pb_cplex) R 5372 5372 4561 0 -1 4202560 8741 0 0 0 3801 9 0 0 20 0 8 0 391038380 263094272 41197 33554432000 134512640 146736320 4287342640 3936346752 139046527 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
Current children cumulated CPU time (s) 1798.89
Current children cumulated vsize (KiB) 266076

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+542.307 s]
/proc/loadavg: 7.79 8.70 8.47 10/187 5481
/proc/meminfo: memFree=29606264/32873844 swapFree=7628/7628
[pid=5372] ppid=5370 vsize=9148 CPUtime=0 cores=0,2,4,6
/proc/5372/stat : 5372 (run) S 5370 5372 4561 0 -1 4202496 353 0 0 0 0 0 0 0 20 0 1 0 390992965 9367552 269 33554432000 4194304 5064252 140736509946816 140736509945728 240564987470 0 65536 4100 65538 18446744071579302229 0 0 17 0 0 0 0 0 0
/proc/5372/statm: 2287 269 229 213 0 72 0
[pid=5373] ppid=5372 vsize=256928 CPUtime=1798.89 cores=0,2,4,6
/proc/5373/stat : 5373 (pb_cplex) R 5372 5372 4561 0 -1 4202496 401526 0 0 0 179469 420 0 0 20 0 8 0 390992965 263094272 41197 33554432000 134512640 146736320 4287342640 4287335888 136403109 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0 0 0
/proc/5373/statm: 64232 41197 981 2985 0 61166 0
[pid=5373/tid=5472] ppid=5372 vsize=256928 CPUtime=38.12 cores=0,2,4,6
/proc/5373/task/5472/stat : 5472 (pb_cplex) R 5372 5372 4561 0 -1 4202560 1947 0 0 0 3805 7 0 0 20 0 8 0 391038380 263094272 41197 33554432000 134512640 146736320 4287342640 4046445740 135454093 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=5373/tid=5473] ppid=5372 vsize=256928 CPUtime=37.42 cores=0,2,4,6
/proc/5373/task/5473/stat : 5473 (pb_cplex) R 5372 5372 4561 0 -1 4202560 5536 0 0 0 3735 7 0 0 20 0 8 0 391038380 263094272 41197 33554432000 134512640 146736320 4287342640 4056931516 135526859 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
[pid=5373/tid=5474] ppid=5372 vsize=256928 CPUtime=39.35 cores=0,2,4,6
/proc/5373/task/5474/stat : 5474 (pb_cplex) R 5372 5372 4561 0 -1 4202560 4226 0 0 0 3921 14 0 0 20 0 8 0 391038380 263094272 41197 33554432000 134512640 146736320 4287342640 4067418316 136388879 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=5373/tid=5475] ppid=5372 vsize=256928 CPUtime=39.24 cores=0,2,4,6
/proc/5373/task/5475/stat : 5475 (pb_cplex) R 5372 5372 4561 0 -1 4202560 2118 0 0 0 3916 8 0 0 20 0 8 0 391038380 263094272 41197 33554432000 134512640 146736320 4287342640 4117319888 136008167 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=5373/tid=5476] ppid=5372 vsize=256928 CPUtime=40.11 cores=0,2,4,6
/proc/5373/task/5476/stat : 5476 (pb_cplex) R 5372 5372 4561 0 -1 4202560 8192 0 0 0 4000 11 0 0 20 0 8 0 391038380 263094272 41197 33554432000 134512640 146736320 4287342640 4150210632 139042981 0 2 4096 16387 0 0 0 -1 2 0 0 0 0 0
[pid=5373/tid=5477] ppid=5372 vsize=256928 CPUtime=39.19 cores=0,2,4,6
/proc/5373/task/5477/stat : 5477 (pb_cplex) R 5372 5372 4561 0 -1 4202560 9235 0 0 0 3910 9 0 0 20 0 8 0 391038380 263094272 41197 33554432000 134512640 146736320 4287342640 4139725392 136403109 0 2 4096 16387 0 0 0 -1 4 0 0 0 0 0
[pid=5373/tid=5478] ppid=5372 vsize=256928 CPUtime=38.1 cores=0,2,4,6
/proc/5373/task/5478/stat : 5478 (pb_cplex) R 5372 5372 4561 0 -1 4202560 8741 0 0 0 3801 9 0 0 20 0 8 0 391038380 263094272 41197 33554432000 134512640 146736320 4287342640 3936346752 139046527 0 2 4096 16387 0 0 0 -1 6 0 0 0 0 0
Current children cumulated CPU time (s) 1798.89
Current children cumulated vsize (KiB) 266076

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 5372 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=0
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=999
# CPU time returned by wait4() is 0.000999
# while last known CPU time is 1798.89
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...


# WARNING:
# The CPU time of some children was not reported to their father
# (probably because of a missing or aborted wait()).
# This 'lost CPU time' is added to the watched process CPU time.
#  lost CPU time (s): 1.24999
#  lost CPU user time (s): 1.49007
#  lost CPU system time (s): -0.24

Real time (s): 542.323
CPU time (s): 1800.14
CPU user time (s): 1796.18
CPU system time (s): 3.96
CPU usage (%): 331.931
Max. virtual memory (cumulated for all children) (KiB): 273620

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0
system time used= 0.000999
maximum resident set size= 1076
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 353
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2
involuntary context switches= 1

runsolver used 1.37079 second user time and 3.71543 second system time

The end

Launcher Data 

Begin job on node146 at 2012-06-09 15:31:20
IDJOB=3737524
IDBENCH=48251
IDSOLVER=2325
FILE ID=node146/3737524-1339248680
RUNJOBID= node146-1339246834-4577
PBS_JOBID= 14644335
Free space on /tmp= 70756 MiB

SOLVER NAME= PB10: pb_cplex 2010-06-29
BENCH NAME= PB07/normalized-PB07/OPT-SMALLINT-NLC/submittedPB07/manquinho/bsg/normalized-bsg_100_10_1.opb
COMMAND LINE= run BENCHNAME
CONVERSION SCRIPT= PBconversionToLinear BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3737524-1339248680/watcher-3737524-1339248680 -o /tmp/evaluation-result-3737524-1339248680/solver-3737524-1339248680 -C 1800 -W 1900 -M 15500  run HOME/instance-3737524-1339248680.opb

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 8c398fbb12ba7dbf676dd8375a76e0de
RANDOM SEED=1834452469

node146.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.034
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        29313540 kB
Buffers:          500172 kB
Cached:          1574532 kB
SwapCached:         2168 kB
Active:          2537860 kB
Inactive:          84424 kB
Active(anon):     546160 kB
Inactive(anon):     3272 kB
Active(file):    1991700 kB
Inactive(file):    81152 kB
Unevictable:        7628 kB
Mlocked:            7628 kB
SwapTotal:      67108856 kB
SwapFree:       67099984 kB
Dirty:             16684 kB
Writeback:             0 kB
AnonPages:        588204 kB
Mapped:            12828 kB
Shmem:                 0 kB
Slab:             795448 kB
SReclaimable:     235440 kB
SUnreclaim:       560008 kB
KernelStack:        1592 kB
PageTables:         4928 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     780932 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:    399360 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

CONVERSION COMMAND LINE= PBconversionToLinear HOME/instance-3737524-1339248680.opb

Free space on /tmp at the end= 70752 MiB
End job on node146 at 2012-06-09 15:40:25