Trace number 2697869

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
pb_cplex 2010-07-01? (TO) 1800.47 457.924

General information on the benchmark

Name/PARTIAL-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/tsp/
normalized-t3002.11tsp11.1900561288--soft-33-100-0.wbo
MD5SUM42cd11e454ef36c6d0d0a1a4a75ff12f
Bench CategoryPARTIAL-SMALLINT-LIN (both soft and hard constraints, small integers, linear constraints)
Best result obtained on this benchmarkMOPT
Best cost obtained on this benchmark1
Best CPU time to get the best result obtained on this benchmark411.329
Max-Satisfiable
Max-(Un)Satisfiability was proved
Best value of the cost
Optimality of the best cost was proved
Number of variables231
Total number of constraints2707
Number of soft constraints1857
Number of constraints which are clauses2684
Number of constraints which are cardinality constraints (but not clauses)22
Number of constraints which are nor clauses,nor cardinality constraints1
Minimum length of a constraint2
Maximum length of a constraint110
Top cost 93863
Min constraint cost 1
Max constraint cost 100
Sum of constraints costs 93862
Biggest number in a constraint 25
Number of bits of the biggest number in a constraint 5
Biggest sum of numbers in a constraint 625
Number of bits of the biggest sum of numbers10
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c pb_cplex 1.0 - 20100701
0.00/0.00	c Parsing wbo file format. File HOME/instance-2697869-1278054866.wbo.
0.00/0.00	c Instance file HOME/instance-2697869-1278054866.wbo
0.00/0.00	c File size is 92575 bytes.
0.00/0.68	c Highest Coefficient sum: 625
0.00/0.68	c  IBM ILOG License Manager: "IBM ILOG Optimization Suite for Academic Initiative" is accessing CPLEX 12 with option(s): "e m b q ".
0.00/0.70	c ============================[ Problem Statistics ]=============================
0.00/0.70	c |                                                                             |
0.00/0.70	c |  Parsing time:          0.69         s                                      |
0.00/0.70	c |  Number Variables:      2088                                                |
0.00/0.70	c |  Number Constraints:    2708                                                |
0.00/0.70	c ===============================================================================
1800.37/457.91	c Remaining time 1799.31 sec.
1800.37/457.91	c 
1800.37/457.91	c *** INTERRUPTED ***
1800.37/457.91	s UNKNOWN
1800.37/457.91	c Total CPU time              : 1784.77 s
1800.37/457.91	c Memory used           : 149.19 MB
1800.37/457.91	c CPU time              : 1784.77 s
1800.37/457.91	c 
1800.37/457.91	c *** INTERRUPTED ***

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver version 3.2.9 (svn:492) (c) roussel@cril.univ-artois.fr

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-2697869-1278054866/watcher-2697869-1278054866 -o /tmp/evaluation-result-2697869-1278054866/solver-2697869-1278054866 -C 1800 -W 2000 -M 3800 run HOME/instance-2697869-1278054866.wbo 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 3891200 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 3942400 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.00 0.00 0.00 2/173 32493
/proc/meminfo: memFree=31323784/32950928 swapFree=67111528/67111528
[pid=32493] ppid=32491 vsize=8700 CPUtime=0
/proc/32493/stat : 32493 (run) S 32491 32493 32226 0 -1 4202496 366 0 0 0 0 0 0 0 20 0 1 0 179078373 8908800 249 4089446400 4194304 4922060 140735422952784 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/32493/statm: 2175 249 208 178 0 68 0

[startup+0.0367659 s]
/proc/loadavg: 0.00 0.00 0.00 2/173 32493
/proc/meminfo: memFree=31323784/32950928 swapFree=67111528/67111528
[pid=32493] ppid=32491 vsize=8700 CPUtime=0
/proc/32493/stat : 32493 (run) S 32491 32493 32226 0 -1 4202496 366 0 0 0 0 0 0 0 20 0 1 0 179078373 8908800 249 4089446400 4194304 4922060 140735422952784 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/32493/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.100742 s]
/proc/loadavg: 0.00 0.00 0.00 2/173 32493
/proc/meminfo: memFree=31323784/32950928 swapFree=67111528/67111528
[pid=32493] ppid=32491 vsize=8700 CPUtime=0
/proc/32493/stat : 32493 (run) S 32491 32493 32226 0 -1 4202496 366 0 0 0 0 0 0 0 20 0 1 0 179078373 8908800 249 4089446400 4194304 4922060 140735422952784 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/32493/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.300702 s]
/proc/loadavg: 0.00 0.00 0.00 2/173 32493
/proc/meminfo: memFree=31323784/32950928 swapFree=67111528/67111528
[pid=32493] ppid=32491 vsize=8700 CPUtime=0
/proc/32493/stat : 32493 (run) S 32491 32493 32226 0 -1 4202496 366 0 0 0 0 0 0 0 20 0 1 0 179078373 8908800 249 4089446400 4194304 4922060 140735422952784 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/32493/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.700554 s]
/proc/loadavg: 0.00 0.00 0.00 2/173 32493
/proc/meminfo: memFree=31323784/32950928 swapFree=67111528/67111528
[pid=32493] ppid=32491 vsize=8700 CPUtime=0
/proc/32493/stat : 32493 (run) S 32491 32493 32226 0 -1 4202496 366 0 0 0 0 0 0 0 20 0 1 0 179078373 8908800 249 4089446400 4194304 4922060 140735422952784 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/32493/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+1.50041 s]
/proc/loadavg: 0.00 0.00 0.00 3/182 32502
/proc/meminfo: memFree=31297424/32950928 swapFree=67111528/67111528
[pid=32493] ppid=32491 vsize=8700 CPUtime=0
/proc/32493/stat : 32493 (run) S 32491 32493 32226 0 -1 4202496 366 0 0 0 0 0 0 0 20 0 1 0 179078373 8908800 249 4089446400 4194304 4922060 140735422952784 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/32493/statm: 2175 249 208 178 0 68 0
[pid=32494] ppid=32493 vsize=141896 CPUtime=3.62
/proc/32494/stat : 32494 (pb_cplex) S 32493 32493 32226 0 -1 4202496 19148 0 0 0 354 8 0 0 18 0 8 0 179078373 145301504 12657 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/32494/statm: 35474 12657 1070 2985 0 32016 0
Current children cumulated CPU time (s) 3.62
Current children cumulated vsize (KiB) 150596

[startup+3.10024 s]
/proc/loadavg: 0.00 0.00 0.00 7/188 32534
/proc/meminfo: memFree=31236740/32950928 swapFree=67111528/67111528
[pid=32493] ppid=32491 vsize=8700 CPUtime=0
/proc/32493/stat : 32493 (run) S 32491 32493 32226 0 -1 4202496 366 0 0 0 0 0 0 0 20 0 1 0 179078373 8908800 249 4089446400 4194304 4922060 140735422952784 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/32493/statm: 2175 249 208 178 0 68 0
[pid=32494] ppid=32493 vsize=122420 CPUtime=5.52
/proc/32494/stat : 32494 (pb_cplex) R 32493 32493 32226 0 -1 4202496 26657 0 0 0 542 10 0 0 19 0 8 0 179078373 125358080 8391 4089446400 134512640 146736320 4288086928 18446744073709551615 136008226 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 0
/proc/32494/statm: 30605 8391 1078 2985 0 27147 0
Current children cumulated CPU time (s) 5.52
Current children cumulated vsize (KiB) 131120

[startup+6.30053 s]
/proc/loadavg: 0.96 0.20 0.06 18/195 32577
/proc/meminfo: memFree=31214440/32950928 swapFree=67111528/67111528
[pid=32493] ppid=32491 vsize=8700 CPUtime=0
/proc/32493/stat : 32493 (run) S 32491 32493 32226 0 -1 4202496 366 0 0 0 0 0 0 0 20 0 1 0 179078373 8908800 249 4089446400 4194304 4922060 140735422952784 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/32493/statm: 2175 249 208 178 0 68 0
[pid=32494] ppid=32493 vsize=134760 CPUtime=17.95
/proc/32494/stat : 32494 (pb_cplex) R 32493 32493 32226 0 -1 4202496 38646 0 0 0 1773 22 0 0 21 0 8 0 179078373 137994240 11493 4089446400 134512640 146736320 4288086928 18446744073709551615 136485910 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0
/proc/32494/statm: 33690 11493 1080 2985 0 30232 0
[pid=32494/tid=32549] ppid=32493 vsize=134760 CPUtime=1.47
/proc/32494/task/32549/stat : 32549 (pb_cplex) S 32493 32493 32226 0 -1 4202560 174 0 0 0 147 0 0 0 15 0 8 0 179078599 137994240 11493 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 7 0 0 0
[pid=32494/tid=32550] ppid=32493 vsize=134760 CPUtime=1.42
/proc/32494/task/32550/stat : 32550 (pb_cplex) S 32493 32493 32226 0 -1 4202560 1094 0 0 0 141 1 0 0 15 0 8 0 179078599 137994240 11493 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 2 0 0 0
[pid=32494/tid=32551] ppid=32493 vsize=134760 CPUtime=1.47
/proc/32494/task/32551/stat : 32551 (pb_cplex) S 32493 32493 32226 0 -1 4202560 801 0 0 0 147 0 0 0 15 0 8 0 179078599 137994240 11493 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 4 0 0 0
[pid=32494/tid=32552] ppid=32493 vsize=134760 CPUtime=1.37
/proc/32494/task/32552/stat : 32552 (pb_cplex) S 32493 32493 32226 0 -1 4202560 1237 0 0 0 136 1 0 0 15 0 8 0 179078599 137994240 11493 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 2 0 0 0
[pid=32494/tid=32553] ppid=32493 vsize=134760 CPUtime=1.55
/proc/32494/task/32553/stat : 32553 (pb_cplex) S 32493 32493 32226 0 -1 4202560 1936 0 0 0 154 1 0 0 15 0 8 0 179078599 137994240 11493 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 2 0 0 0
[pid=32494/tid=32554] ppid=32493 vsize=134760 CPUtime=1.78
/proc/32494/task/32554/stat : 32554 (pb_cplex) S 32493 32493 32226 0 -1 4202560 4424 0 0 0 177 1 0 0 15 0 8 0 179078599 137994240 11493 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 5 0 0 0
[pid=32494/tid=32555] ppid=32493 vsize=134760 CPUtime=1.42
/proc/32494/task/32555/stat : 32555 (pb_cplex) S 32493 32493 32226 0 -1 4202560 1338 0 0 0 140 2 0 0 15 0 8 0 179078599 137994240 11493 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 17.95
Current children cumulated vsize (KiB) 143460

[startup+12.7154 s]
/proc/loadavg: 1.60 0.35 0.11 12/195 32577
/proc/meminfo: memFree=31210976/32950928 swapFree=67111528/67111528
[pid=32493] ppid=32491 vsize=8700 CPUtime=0
/proc/32493/stat : 32493 (run) S 32491 32493 32226 0 -1 4202496 366 0 0 0 0 0 0 0 20 0 1 0 179078373 8908800 249 4089446400 4194304 4922060 140735422952784 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/32493/statm: 2175 249 208 178 0 68 0
[pid=32494] ppid=32493 vsize=137832 CPUtime=39.12
/proc/32494/stat : 32494 (pb_cplex) R 32493 32493 32226 0 -1 4202496 42340 0 0 0 3873 39 0 0 16 0 8 0 179078373 141139968 12034 4089446400 134512640 146736320 4288086928 18446744073709551615 135468681 0 0 4096 16387 18446744073709551615 0 0 17 6 0 0 0
/proc/32494/statm: 34458 12034 1080 2985 0 31000 0
[pid=32494/tid=32549] ppid=32493 vsize=137832 CPUtime=4.24
/proc/32494/task/32549/stat : 32549 (pb_cplex) R 32493 32493 32226 0 -1 4202560 248 0 0 0 422 2 0 0 15 0 8 0 179078599 141139968 12034 4089446400 134512640 146736320 4288086928 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=32494/tid=32550] ppid=32493 vsize=137832 CPUtime=4.01
/proc/32494/task/32550/stat : 32550 (pb_cplex) R 32493 32493 32226 0 -1 4202560 1100 0 0 0 398 3 0 0 15 0 8 0 179078599 141139968 12034 4089446400 134512640 146736320 4288086928 18446744073709551615 135854446 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=32494/tid=32551] ppid=32493 vsize=137832 CPUtime=4.1
/proc/32494/task/32551/stat : 32551 (pb_cplex) R 32493 32493 32226 0 -1 4202560 3358 0 0 0 407 3 0 0 15 0 8 0 179078599 141139968 12034 4089446400 134512640 146736320 4288086928 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=32494/tid=32552] ppid=32493 vsize=137832 CPUtime=3.99
/proc/32494/task/32552/stat : 32552 (pb_cplex) R 32493 32493 32226 0 -1 4202560 1259 0 0 0 396 3 0 0 15 0 8 0 179078599 141139968 12034 4089446400 134512640 146736320 4288086928 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=32494/tid=32553] ppid=32493 vsize=137832 CPUtime=4.14
/proc/32494/task/32553/stat : 32553 (pb_cplex) R 32493 32493 32226 0 -1 4202560 2071 0 0 0 411 3 0 0 15 0 8 0 179078599 141139968 12034 4089446400 134512640 146736320 4288086928 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=32494/tid=32554] ppid=32493 vsize=137832 CPUtime=4.39
/proc/32494/task/32554/stat : 32554 (pb_cplex) R 32493 32493 32226 0 -1 4202560 5103 0 0 0 436 3 0 0 15 0 8 0 179078599 141139968 12034 4089446400 134512640 146736320 4288086928 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=32494/tid=32555] ppid=32493 vsize=137832 CPUtime=3.98
/proc/32494/task/32555/stat : 32555 (pb_cplex) R 32493 32493 32226 0 -1 4202560 1555 0 0 0 394 4 0 0 15 0 8 0 179078599 141139968 12034 4089446400 134512640 146736320 4288086928 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 39.12
Current children cumulated vsize (KiB) 146532

[startup+25.501 s]
/proc/loadavg: 3.05 0.72 0.24 16/195 32591
/proc/meminfo: memFree=31192128/32950928 swapFree=67111528/67111528
[pid=32493] ppid=32491 vsize=8700 CPUtime=0
/proc/32493/stat : 32493 (run) S 32491 32493 32226 0 -1 4202496 366 0 0 0 0 0 0 0 20 0 1 0 179078373 8908800 249 4089446400 4194304 4922060 140735422952784 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/32493/statm: 2175 249 208 178 0 68 0
[pid=32494] ppid=32493 vsize=133204 CPUtime=74.56
/proc/32494/stat : 32494 (pb_cplex) R 32493 32493 32226 0 -1 4202496 50276 0 0 0 7381 75 0 0 21 0 8 0 179078373 136400896 10999 4089446400 134512640 146736320 4288086928 18446744073709551615 136008024 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0
/proc/32494/statm: 33301 10999 1086 2985 0 29843 0
[pid=32494/tid=32585] ppid=32493 vsize=133204 CPUtime=0.32
/proc/32494/task/32585/stat : 32585 (pb_cplex) S 32493 32493 32226 0 -1 4202560 266 0 0 0 32 0 0 0 16 0 8 0 179080676 136400896 10999 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 4 0 0 0
[pid=32494/tid=32586] ppid=32493 vsize=133204 CPUtime=0.64
/proc/32494/task/32586/stat : 32586 (pb_cplex) R 32493 32493 32226 0 -1 4202560 673 0 0 0 64 0 0 0 16 0 8 0 179080676 136400896 10999 4089446400 134512640 146736320 4288086928 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=32494/tid=32587] ppid=32493 vsize=133204 CPUtime=0.16
/proc/32494/task/32587/stat : 32587 (pb_cplex) S 32493 32493 32226 0 -1 4202560 727 0 0 0 16 0 0 0 16 0 8 0 179080676 136400896 10999 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 1 0 0 0
[pid=32494/tid=32588] ppid=32493 vsize=133204 CPUtime=0.51
/proc/32494/task/32588/stat : 32588 (pb_cplex) R 32493 32493 32226 0 -1 4202560 599 0 0 0 51 0 0 0 16 0 8 0 179080676 136400896 10999 4089446400 134512640 146736320 4288086928 18446744073709551615 136005000 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=32494/tid=32589] ppid=32493 vsize=133204 CPUtime=0.55
/proc/32494/task/32589/stat : 32589 (pb_cplex) R 32493 32493 32226 0 -1 4202560 980 0 0 0 55 0 0 0 16 0 8 0 179080684 136400896 10999 4089446400 134512640 146736320 4288086928 18446744073709551615 136007668 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=32494/tid=32590] ppid=32493 vsize=133204 CPUtime=0.52
/proc/32494/task/32590/stat : 32590 (pb_cplex) R 32493 32493 32226 0 -1 4202560 824 0 0 0 52 0 0 0 16 0 8 0 179080718 136400896 10999 4089446400 134512640 146736320 4288086928 18446744073709551615 136007027 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=32494/tid=32591] ppid=32493 vsize=133204 CPUtime=0.48
/proc/32494/task/32591/stat : 32591 (pb_cplex) R 32493 32493 32226 0 -1 4202560 443 0 0 0 48 0 0 0 16 0 8 0 179080726 136400896 10999 4089446400 134512640 146736320 4288086928 18446744073709551615 136007654 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 74.56
Current children cumulated vsize (KiB) 141904

[startup+51.1013 s]
/proc/loadavg: 5.89 1.58 0.53 10/195 32613
/proc/meminfo: memFree=31207076/32950928 swapFree=67111528/67111528
[pid=32493] ppid=32491 vsize=8700 CPUtime=0
/proc/32493/stat : 32493 (run) S 32491 32493 32226 0 -1 4202496 366 0 0 0 0 0 0 0 20 0 1 0 179078373 8908800 249 4089446400 4194304 4922060 140735422952784 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/32493/statm: 2175 249 208 178 0 68 0
[pid=32494] ppid=32493 vsize=141632 CPUtime=173.58
/proc/32494/stat : 32494 (pb_cplex) S 32493 32493 32226 0 -1 4202496 73819 0 0 0 17205 153 0 0 16 0 8 0 179078373 145031168 13143 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0
/proc/32494/statm: 35408 13143 1087 2985 0 31950 0
[pid=32494/tid=32585] ppid=32493 vsize=141632 CPUtime=11.03
/proc/32494/task/32585/stat : 32585 (pb_cplex) S 32493 32493 32226 0 -1 4202560 2751 0 0 0 1095 8 0 0 15 0 8 0 179080676 145031168 13143 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 1 0 0 0
[pid=32494/tid=32586] ppid=32493 vsize=141632 CPUtime=14.24
/proc/32494/task/32586/stat : 32586 (pb_cplex) S 32493 32493 32226 0 -1 4202560 3125 0 0 0 1415 9 0 0 15 0 8 0 179080676 145031168 13143 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 7 0 0 0
[pid=32494/tid=32587] ppid=32493 vsize=141632 CPUtime=10.79
/proc/32494/task/32587/stat : 32587 (pb_cplex) R 32493 32493 32226 0 -1 4202560 5228 0 0 0 1070 9 0 0 15 0 8 0 179080676 145031168 13143 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=32494/tid=32588] ppid=32493 vsize=141632 CPUtime=14.3
/proc/32494/task/32588/stat : 32588 (pb_cplex) S 32493 32493 32226 0 -1 4202560 3918 0 0 0 1419 11 0 0 15 0 8 0 179080676 145031168 13143 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 5 0 0 0
[pid=32494/tid=32589] ppid=32493 vsize=141632 CPUtime=13.62
/proc/32494/task/32589/stat : 32589 (pb_cplex) S 32493 32493 32226 0 -1 4202560 3861 0 0 0 1352 10 0 0 15 0 8 0 179080684 145031168 13143 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 3 0 0 0
[pid=32494/tid=32590] ppid=32493 vsize=141632 CPUtime=12.31
/proc/32494/task/32590/stat : 32590 (pb_cplex) S 32493 32493 32226 0 -1 4202560 4649 0 0 0 1221 10 0 0 15 0 8 0 179080718 145031168 13143 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 4 0 0 0
[pid=32494/tid=32591] ppid=32493 vsize=141632 CPUtime=12.55
/proc/32494/task/32591/stat : 32591 (pb_cplex) R 32493 32493 32226 0 -1 4202560 3366 0 0 0 1246 9 0 0 15 0 8 0 179080726 145031168 13143 4089446400 134512640 146736320 4288086928 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 173.58
Current children cumulated vsize (KiB) 150332

[startup+102.302 s]
/proc/loadavg: 7.57 2.69 0.96 11/195 32614
/proc/meminfo: memFree=31201560/32950928 swapFree=67111528/67111528
[pid=32493] ppid=32491 vsize=8700 CPUtime=0
/proc/32493/stat : 32493 (run) S 32491 32493 32226 0 -1 4202496 366 0 0 0 0 0 0 0 20 0 1 0 179078373 8908800 249 4089446400 4194304 4922060 140735422952784 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/32493/statm: 2175 249 208 178 0 68 0
[pid=32494] ppid=32493 vsize=151208 CPUtime=375.4
/proc/32494/stat : 32494 (pb_cplex) R 32493 32493 32226 0 -1 4202496 86709 0 0 0 37222 318 0 0 16 0 8 0 179078373 154836992 15216 4089446400 134512640 146736320 4288086928 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0
/proc/32494/statm: 37802 15216 1087 2985 0 34344 0
[pid=32494/tid=32585] ppid=32493 vsize=151208 CPUtime=36.12
/proc/32494/task/32585/stat : 32585 (pb_cplex) R 32493 32493 32226 0 -1 4202560 4422 0 0 0 3584 28 0 0 15 0 8 0 179080676 154836992 15216 4089446400 134512640 146736320 4288086928 18446744073709551615 136000315 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=32494/tid=32586] ppid=32493 vsize=151208 CPUtime=39.23
/proc/32494/task/32586/stat : 32586 (pb_cplex) R 32493 32493 32226 0 -1 4202560 3938 0 0 0 3894 29 0 0 15 0 8 0 179080676 154836992 15216 4089446400 134512640 146736320 4288086928 18446744073709551615 135526845 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=32494/tid=32587] ppid=32493 vsize=151208 CPUtime=35.88
/proc/32494/task/32587/stat : 32587 (pb_cplex) R 32493 32493 32226 0 -1 4202560 6524 0 0 0 3557 31 0 0 15 0 8 0 179080676 154836992 15216 4089446400 134512640 146736320 4288086928 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=32494/tid=32588] ppid=32493 vsize=151208 CPUtime=39.4
/proc/32494/task/32588/stat : 32588 (pb_cplex) R 32493 32493 32226 0 -1 4202560 5598 0 0 0 3908 32 0 0 15 0 8 0 179080676 154836992 15216 4089446400 134512640 146736320 4288086928 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0

################
# More data... #
################

/proc/32494/task/32590/stat : 32590 (pb_cplex) S 32493 32493 32226 0 -1 4202560 6902 0 0 0 3728 29 0 0 15 0 8 0 179080718 154836992 15216 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 7 0 0 0
[pid=32494/tid=32591] ppid=32493 vsize=151208 CPUtime=37.59
/proc/32494/task/32591/stat : 32591 (pb_cplex) S 32493 32493 32226 0 -1 4202560 4519 0 0 0 3730 29 0 0 16 0 8 0 179080726 154836992 15216 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 375.4
Current children cumulated vsize (KiB) 159908

[startup+162.338 s]
/proc/loadavg: 8.75 3.92 1.50 8/195 32616
/proc/meminfo: memFree=31197584/32950928 swapFree=67111528/67111528
[pid=32493] ppid=32491 vsize=8700 CPUtime=0
/proc/32493/stat : 32493 (run) S 32491 32493 32226 0 -1 4202496 366 0 0 0 0 0 0 0 20 0 1 0 179078373 8908800 249 4089446400 4194304 4922060 140735422952784 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/32493/statm: 2175 249 208 178 0 68 0
[pid=32494] ppid=32493 vsize=151388 CPUtime=613.75
/proc/32494/stat : 32494 (pb_cplex) R 32493 32493 32226 0 -1 4202496 89804 0 0 0 60858 517 0 0 15 0 8 0 179078373 155021312 15346 4089446400 134512640 146736320 4288086928 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0
/proc/32494/statm: 37847 15346 1087 2985 0 34389 0
[pid=32494/tid=32585] ppid=32493 vsize=151388 CPUtime=65.81
/proc/32494/task/32585/stat : 32585 (pb_cplex) S 32493 32493 32226 0 -1 4202560 4704 0 0 0 6530 51 0 0 15 0 8 0 179080676 155021312 15346 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 6 0 0 0
[pid=32494/tid=32586] ppid=32493 vsize=151388 CPUtime=68.63
/proc/32494/task/32586/stat : 32586 (pb_cplex) R 32493 32493 32226 0 -1 4202560 4413 0 0 0 6808 55 0 0 15 0 8 0 179080676 155021312 15346 4089446400 134512640 146736320 4288086928 18446744073709551615 135854505 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=32494/tid=32587] ppid=32493 vsize=151388 CPUtime=65.32
/proc/32494/task/32587/stat : 32587 (pb_cplex) R 32493 32493 32226 0 -1 4202560 7039 0 0 0 6478 54 0 0 15 0 8 0 179080676 155021312 15346 4089446400 134512640 146736320 4288086928 18446744073709551615 135854511 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=32494/tid=32588] ppid=32493 vsize=151388 CPUtime=69.06
/proc/32494/task/32588/stat : 32588 (pb_cplex) R 32493 32493 32226 0 -1 4202560 5821 0 0 0 6852 54 0 0 15 0 8 0 179080676 155021312 15346 4089446400 134512640 146736320 4288086928 18446744073709551615 136403116 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=32494/tid=32589] ppid=32493 vsize=151388 CPUtime=68.51
/proc/32494/task/32589/stat : 32589 (pb_cplex) R 32493 32493 32226 0 -1 4202560 6714 0 0 0 6791 60 0 0 15 0 8 0 179080684 155021312 15346 4089446400 134512640 146736320 4288086928 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=32494/tid=32590] ppid=32493 vsize=151388 CPUtime=67.3
/proc/32494/task/32590/stat : 32590 (pb_cplex) R 32493 32493 32226 0 -1 4202560 7441 0 0 0 6675 55 0 0 15 0 8 0 179080718 155021312 15346 4089446400 134512640 146736320 4288086928 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=32494/tid=32591] ppid=32493 vsize=151388 CPUtime=67.28
/proc/32494/task/32591/stat : 32591 (pb_cplex) R 32493 32493 32226 0 -1 4202560 4664 0 0 0 6675 53 0 0 15 0 8 0 179080726 155021312 15346 4089446400 134512640 146736320 4288086928 18446744073709551615 136017763 0 2 4096 16387 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 613.75
Current children cumulated vsize (KiB) 160088

[startup+222.307 s]
/proc/loadavg: 9.26 4.94 1.99 11/195 32617
/proc/meminfo: memFree=31197628/32950928 swapFree=67111528/67111528
[pid=32493] ppid=32491 vsize=8700 CPUtime=0
/proc/32493/stat : 32493 (run) S 32491 32493 32226 0 -1 4202496 366 0 0 0 0 0 0 0 20 0 1 0 179078373 8908800 249 4089446400 4194304 4922060 140735422952784 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/32493/statm: 2175 249 208 178 0 68 0
[pid=32494] ppid=32493 vsize=152280 CPUtime=852.5
/proc/32494/stat : 32494 (pb_cplex) R 32493 32493 32226 0 -1 4202496 92794 0 0 0 84534 716 0 0 16 0 8 0 179078373 155934720 15627 4089446400 134512640 146736320 4288086928 18446744073709551615 135468687 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0
/proc/32494/statm: 38070 15627 1103 2985 0 34612 0
[pid=32494/tid=32585] ppid=32493 vsize=152280 CPUtime=95.37
/proc/32494/task/32585/stat : 32585 (pb_cplex) R 32493 32493 32226 0 -1 4202560 5003 0 0 0 9462 75 0 0 15 0 8 0 179080676 155934720 15627 4089446400 134512640 146736320 4288086928 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=32494/tid=32586] ppid=32493 vsize=152280 CPUtime=98.19
/proc/32494/task/32586/stat : 32586 (pb_cplex) R 32493 32493 32226 0 -1 4202560 4625 0 0 0 9738 81 0 0 15 0 8 0 179080676 155934720 15627 4089446400 134512640 146736320 4288086928 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=32494/tid=32587] ppid=32493 vsize=152280 CPUtime=94.79
/proc/32494/task/32587/stat : 32587 (pb_cplex) R 32493 32493 32226 0 -1 4202560 7202 0 0 0 9400 79 0 0 15 0 8 0 179080676 155934720 15627 4089446400 134512640 146736320 4288086928 18446744073709551615 137345114 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=32494/tid=32588] ppid=32493 vsize=152280 CPUtime=98.53
/proc/32494/task/32588/stat : 32588 (pb_cplex) R 32493 32493 32226 0 -1 4202560 5964 0 0 0 9774 79 0 0 15 0 8 0 179080676 155934720 15627 4089446400 134512640 146736320 4288086928 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=32494/tid=32589] ppid=32493 vsize=152280 CPUtime=98.07
/proc/32494/task/32589/stat : 32589 (pb_cplex) R 32493 32493 32226 0 -1 4202560 7868 0 0 0 9723 84 0 0 15 0 8 0 179080684 155934720 15627 4089446400 134512640 146736320 4288086928 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=32494/tid=32590] ppid=32493 vsize=152280 CPUtime=96.91
/proc/32494/task/32590/stat : 32590 (pb_cplex) R 32493 32493 32226 0 -1 4202560 7783 0 0 0 9612 79 0 0 15 0 8 0 179080718 155934720 15627 4089446400 134512640 146736320 4288086928 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=32494/tid=32591] ppid=32493 vsize=152280 CPUtime=96.98
/proc/32494/task/32591/stat : 32591 (pb_cplex) R 32493 32493 32226 0 -1 4202560 5217 0 0 0 9619 79 0 0 15 0 8 0 179080726 155934720 15627 4089446400 134512640 146736320 4288086928 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 852.5
Current children cumulated vsize (KiB) 160980

[startup+282.309 s]
/proc/loadavg: 9.58 5.80 2.47 10/195 32682
/proc/meminfo: memFree=31197380/32950928 swapFree=67111528/67111528
[pid=32493] ppid=32491 vsize=8700 CPUtime=0
/proc/32493/stat : 32493 (run) S 32491 32493 32226 0 -1 4202496 366 0 0 0 0 0 0 0 20 0 1 0 179078373 8908800 249 4089446400 4194304 4922060 140735422952784 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/32493/statm: 2175 249 208 178 0 68 0
[pid=32494] ppid=32493 vsize=153056 CPUtime=1093.68
/proc/32494/stat : 32494 (pb_cplex) S 32493 32493 32226 0 -1 4202496 95483 0 0 0 108436 932 0 0 15 0 8 0 179078373 156729344 15831 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/32494/statm: 38264 15831 1105 2985 0 34806 0
[pid=32494/tid=32585] ppid=32493 vsize=153056 CPUtime=125.3
/proc/32494/task/32585/stat : 32585 (pb_cplex) S 32493 32493 32226 0 -1 4202560 5096 0 0 0 12426 104 0 0 15 0 8 0 179080676 156729344 15831 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 2 0 0 0
[pid=32494/tid=32586] ppid=32493 vsize=153056 CPUtime=128.01
/proc/32494/task/32586/stat : 32586 (pb_cplex) S 32493 32493 32226 0 -1 4202560 4983 0 0 0 12695 106 0 0 16 0 8 0 179080676 156729344 15831 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 6 0 0 0
[pid=32494/tid=32587] ppid=32493 vsize=153056 CPUtime=124.69
/proc/32494/task/32587/stat : 32587 (pb_cplex) S 32493 32493 32226 0 -1 4202560 7722 0 0 0 12361 108 0 0 15 0 8 0 179080676 156729344 15831 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 3 0 0 0
[pid=32494/tid=32588] ppid=32493 vsize=153056 CPUtime=128.33
/proc/32494/task/32588/stat : 32588 (pb_cplex) S 32493 32493 32226 0 -1 4202560 6114 0 0 0 12730 103 0 0 15 0 8 0 179080676 156729344 15831 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 4 0 0 0
[pid=32494/tid=32589] ppid=32493 vsize=153056 CPUtime=128.19
/proc/32494/task/32589/stat : 32589 (pb_cplex) S 32493 32493 32226 0 -1 4202560 8701 0 0 0 12708 111 0 0 15 0 8 0 179080684 156729344 15831 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 7 0 0 0
[pid=32494/tid=32590] ppid=32493 vsize=153056 CPUtime=126.8
/proc/32494/task/32590/stat : 32590 (pb_cplex) S 32493 32493 32226 0 -1 4202560 7827 0 0 0 12573 107 0 0 16 0 8 0 179080718 156729344 15831 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 7 0 0 0
[pid=32494/tid=32591] ppid=32493 vsize=153056 CPUtime=127.07
/proc/32494/task/32591/stat : 32591 (pb_cplex) R 32493 32493 32226 0 -1 4202560 5831 0 0 0 12603 104 0 0 15 0 8 0 179080726 156729344 15831 4089446400 134512640 146736320 4288086928 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1093.68
Current children cumulated vsize (KiB) 161756

[startup+342.303 s]
/proc/loadavg: 10.03 6.64 2.97 9/195 32705
/proc/meminfo: memFree=31188736/32950928 swapFree=67111528/67111528
[pid=32493] ppid=32491 vsize=8700 CPUtime=0
/proc/32493/stat : 32493 (run) S 32491 32493 32226 0 -1 4202496 366 0 0 0 0 0 0 0 20 0 1 0 179078373 8908800 249 4089446400 4194304 4922060 140735422952784 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/32493/statm: 2175 249 208 178 0 68 0
[pid=32494] ppid=32493 vsize=152768 CPUtime=1371.99
/proc/32494/stat : 32494 (pb_cplex) R 32493 32493 32226 0 -1 4202496 96435 0 0 0 136006 1193 0 0 16 0 8 0 179078373 156434432 15811 4089446400 134512640 146736320 4288086928 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0
/proc/32494/statm: 38192 15811 1105 2985 0 34734 0
[pid=32494/tid=32585] ppid=32493 vsize=152768 CPUtime=159.74
/proc/32494/task/32585/stat : 32585 (pb_cplex) S 32493 32493 32226 0 -1 4202560 5126 0 0 0 15838 136 0 0 15 0 8 0 179080676 156434432 15811 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 3 0 0 0
[pid=32494/tid=32586] ppid=32493 vsize=152768 CPUtime=162.59
/proc/32494/task/32586/stat : 32586 (pb_cplex) S 32493 32493 32226 0 -1 4202560 5003 0 0 0 16123 136 0 0 15 0 8 0 179080676 156434432 15811 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 3 0 0 0
[pid=32494/tid=32587] ppid=32493 vsize=152768 CPUtime=159.17
/proc/32494/task/32587/stat : 32587 (pb_cplex) S 32493 32493 32226 0 -1 4202560 8141 0 0 0 15775 142 0 0 15 0 8 0 179080676 156434432 15811 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 5 0 0 0
[pid=32494/tid=32588] ppid=32493 vsize=152768 CPUtime=163.06
/proc/32494/task/32588/stat : 32588 (pb_cplex) S 32493 32493 32226 0 -1 4202560 6167 0 0 0 16169 137 0 0 15 0 8 0 179080676 156434432 15811 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 6 0 0 0
[pid=32494/tid=32589] ppid=32493 vsize=152768 CPUtime=162.8
/proc/32494/task/32589/stat : 32589 (pb_cplex) S 32493 32493 32226 0 -1 4202560 8743 0 0 0 16140 140 0 0 16 0 8 0 179080684 156434432 15811 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 2 0 0 0
[pid=32494/tid=32590] ppid=32493 vsize=152768 CPUtime=161.36
/proc/32494/task/32590/stat : 32590 (pb_cplex) S 32493 32493 32226 0 -1 4202560 7860 0 0 0 15996 140 0 0 15 0 8 0 179080718 156434432 15811 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 4 0 0 0
[pid=32494/tid=32591] ppid=32493 vsize=152768 CPUtime=161.87
/proc/32494/task/32591/stat : 32591 (pb_cplex) S 32493 32493 32226 0 -1 4202560 5931 0 0 0 16048 139 0 0 15 0 8 0 179080726 156434432 15811 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 1371.99
Current children cumulated vsize (KiB) 161468

[startup+402.301 s]
/proc/loadavg: 9.49 7.11 3.36 8/195 32706
/proc/meminfo: memFree=31188428/32950928 swapFree=67111528/67111528
[pid=32493] ppid=32491 vsize=8700 CPUtime=0
/proc/32493/stat : 32493 (run) S 32491 32493 32226 0 -1 4202496 366 0 0 0 0 0 0 0 20 0 1 0 179078373 8908800 249 4089446400 4194304 4922060 140735422952784 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/32493/statm: 2175 249 208 178 0 68 0
[pid=32494] ppid=32493 vsize=152768 CPUtime=1589.53
/proc/32494/stat : 32494 (pb_cplex) S 32493 32493 32226 0 -1 4202496 97632 0 0 0 157568 1385 0 0 16 0 8 0 179078373 156434432 15819 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 0
/proc/32494/statm: 38192 15819 1105 2985 0 34734 0
[pid=32494/tid=32585] ppid=32493 vsize=152768 CPUtime=186.72
/proc/32494/task/32585/stat : 32585 (pb_cplex) S 32493 32493 32226 0 -1 4202560 5453 0 0 0 18510 162 0 0 15 0 8 0 179080676 156434432 15819 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 7 0 0 0
[pid=32494/tid=32586] ppid=32493 vsize=152768 CPUtime=189.44
/proc/32494/task/32586/stat : 32586 (pb_cplex) S 32493 32493 32226 0 -1 4202560 5137 0 0 0 18786 158 0 0 15 0 8 0 179080676 156434432 15819 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 7 0 0 0
[pid=32494/tid=32587] ppid=32493 vsize=152768 CPUtime=186.07
/proc/32494/task/32587/stat : 32587 (pb_cplex) S 32493 32493 32226 0 -1 4202560 8225 0 0 0 18440 167 0 0 15 0 8 0 179080676 156434432 15819 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 0 0 0 0
[pid=32494/tid=32588] ppid=32493 vsize=152768 CPUtime=189.75
/proc/32494/task/32588/stat : 32588 (pb_cplex) S 32493 32493 32226 0 -1 4202560 6444 0 0 0 18815 160 0 0 15 0 8 0 179080676 156434432 15819 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 2 0 0 0
[pid=32494/tid=32589] ppid=32493 vsize=152768 CPUtime=189.6
/proc/32494/task/32589/stat : 32589 (pb_cplex) S 32493 32493 32226 0 -1 4202560 8857 0 0 0 18795 165 0 0 15 0 8 0 179080684 156434432 15819 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 0 0 0 0
[pid=32494/tid=32590] ppid=32493 vsize=152768 CPUtime=188.16
/proc/32494/task/32590/stat : 32590 (pb_cplex) S 32493 32493 32226 0 -1 4202560 7985 0 0 0 18654 162 0 0 15 0 8 0 179080718 156434432 15819 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 18446604471102440704 0 0 -1 7 0 0 0
[pid=32494/tid=32591] ppid=32493 vsize=152768 CPUtime=189.04
/proc/32494/task/32591/stat : 32591 (pb_cplex) R 32493 32493 32226 0 -1 4202560 5997 0 0 0 18739 165 0 0 15 0 8 0 179080726 156434432 15819 4089446400 134512640 146736320 4288086928 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 1589.53
Current children cumulated vsize (KiB) 161468



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+457.902 s]
/proc/loadavg: 9.55 7.52 3.71 10/195 32708
/proc/meminfo: memFree=31188164/32950928 swapFree=67111528/67111528
[pid=32493] ppid=32491 vsize=8700 CPUtime=0
/proc/32493/stat : 32493 (run) S 32491 32493 32226 0 -1 4202496 366 0 0 0 0 0 0 0 20 0 1 0 179078373 8908800 249 4089446400 4194304 4922060 140735422952784 18446744073709551615 260905213061 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/32493/statm: 2175 249 208 178 0 68 0
[pid=32494] ppid=32493 vsize=152768 CPUtime=1800.37
/proc/32494/stat : 32494 (pb_cplex) R 32493 32493 32226 0 -1 4202496 98000 0 0 0 178468 1569 0 0 16 0 8 0 179078373 156434432 15830 4089446400 134512640 146736320 4288086928 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/32494/statm: 38192 15830 1105 2985 0 34734 0
[pid=32494/tid=32585] ppid=32493 vsize=152768 CPUtime=212.81
/proc/32494/task/32585/stat : 32585 (pb_cplex) R 32493 32493 32226 0 -1 4202560 5476 0 0 0 21098 183 0 0 15 0 8 0 179080676 156434432 15830 4089446400 134512640 146736320 4288086928 18446744073709551615 136403112 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=32494/tid=32586] ppid=32493 vsize=152768 CPUtime=215.56
/proc/32494/task/32586/stat : 32586 (pb_cplex) R 32493 32493 32226 0 -1 4202560 5222 0 0 0 21376 180 0 0 15 0 8 0 179080676 156434432 15830 4089446400 134512640 146736320 4288086928 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=32494/tid=32587] ppid=32493 vsize=152768 CPUtime=212.23
/proc/32494/task/32587/stat : 32587 (pb_cplex) R 32493 32493 32226 0 -1 4202560 8245 0 0 0 21034 189 0 0 15 0 8 0 179080676 156434432 15830 4089446400 134512640 146736320 4288086928 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=32494/tid=32588] ppid=32493 vsize=152768 CPUtime=215.89
/proc/32494/task/32588/stat : 32588 (pb_cplex) R 32493 32493 32226 0 -1 4202560 6493 0 0 0 21407 182 0 0 15 0 8 0 179080676 156434432 15830 4089446400 134512640 146736320 4288086928 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=32494/tid=32589] ppid=32493 vsize=152768 CPUtime=215.59
/proc/32494/task/32589/stat : 32589 (pb_cplex) R 32493 32493 32226 0 -1 4202560 8883 0 0 0 21369 190 0 0 15 0 8 0 179080684 156434432 15830 4089446400 134512640 146736320 4288086928 18446744073709551615 136400416 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=32494/tid=32590] ppid=32493 vsize=152768 CPUtime=214.14
/proc/32494/task/32590/stat : 32590 (pb_cplex) R 32493 32493 32226 0 -1 4202560 8083 0 0 0 21229 185 0 0 15 0 8 0 179080718 156434432 15830 4089446400 134512640 146736320 4288086928 18446744073709551615 136017845 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=32494/tid=32591] ppid=32493 vsize=152768 CPUtime=215.11
/proc/32494/task/32591/stat : 32591 (pb_cplex) R 32493 32493 32226 0 -1 4202560 6047 0 0 0 21322 189 0 0 15 0 8 0 179080726 156434432 15830 4089446400 134512640 146736320 4288086928 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 1800.37
Current children cumulated vsize (KiB) 161468

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child status: 0
Real time (s): 457.924
CPU time (s): 1800.47
CPU user time (s): 1784.77
CPU system time (s): 15.6946
CPU usage (%): 393.18
Max. virtual memory (cumulated for all children) (KiB): 162404

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1784.77
system time used= 15.6946
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 98373
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 320415
involuntary context switches= 25422

runsolver used 0.882865 second user time and 1.71174 second system time

The end

Launcher Data 

Begin job on node116 at 2010-07-02 09:14:26
IDJOB=2697869
IDBENCH=78922
IDSOLVER=1210
FILE ID=node116/2697869-1278054866
PBS_JOBID= 11198949
Free space on /tmp= 74192 MiB

SOLVER NAME= pb_cplex 2010-07-01
BENCH NAME= PB10/normalized-PB10/PARTIAL-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/tsp/normalized-t3002.11tsp11.1900561288--soft-33-100-0.wbo
COMMAND LINE= run BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-2697869-1278054866/watcher-2697869-1278054866 -o /tmp/evaluation-result-2697869-1278054866/solver-2697869-1278054866 -C 1800 -W 2000 -M 3800  run HOME/instance-2697869-1278054866.wbo

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 3800 MiB
MAX NB THREAD= 0

MD5SUM BENCH= 42cd11e454ef36c6d0d0a1a4a75ff12f
RANDOM SEED=893539541

node116.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5381.00
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      31324188 kB
Buffers:        217720 kB
Cached:        1198052 kB
SwapCached:          0 kB
Active:         598276 kB
Inactive:       881676 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      31324188 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           16980 kB
Writeback:           0 kB
AnonPages:       64072 kB
Mapped:          14620 kB
Slab:            83152 kB
PageTables:       4496 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   186768 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 74160 MiB
End job on node116 at 2010-07-02 09:22:04