Trace number 2697828

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
pb_cplex 2010-07-01? (TO) 1800.15 485.526

General information on the benchmark

Name/PARTIAL-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/tsp/
normalized-t3002.11tsp11.1900575442--soft-33-100-0.wbo
MD5SUMc8fdbd1e89585cb1f1efe2070d2e6c87
Bench CategoryPARTIAL-SMALLINT-LIN (both soft and hard constraints, small integers, linear constraints)
Best result obtained on this benchmarkMOPT
Best cost obtained on this benchmark1
Best CPU time to get the best result obtained on this benchmark35.7656
Max-Satisfiable
Max-(Un)Satisfiability was proved
Best value of the cost
Optimality of the best cost was proved
Number of variables231
Total number of constraints2707
Number of soft constraints1816
Number of constraints which are clauses2684
Number of constraints which are cardinality constraints (but not clauses)22
Number of constraints which are nor clauses,nor cardinality constraints1
Minimum length of a constraint2
Maximum length of a constraint110
Top cost 92461
Min constraint cost 1
Max constraint cost 100
Sum of constraints costs 92460
Biggest number in a constraint 25
Number of bits of the biggest number in a constraint 5
Biggest sum of numbers in a constraint 667
Number of bits of the biggest sum of numbers10
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c pb_cplex 1.0 - 20100701
0.00/0.00	c Parsing wbo file format. File HOME/instance-2697828-1278054864.wbo.
0.00/0.00	c Instance file HOME/instance-2697828-1278054864.wbo
0.00/0.00	c File size is 92392 bytes.
0.00/0.66	c Highest Coefficient sum: 667
0.00/0.66	c  IBM ILOG License Manager: "IBM ILOG Optimization Suite for Academic Initiative" is accessing CPLEX 12 with option(s): "e m b q ".
0.00/0.66	c ============================[ Problem Statistics ]=============================
0.00/0.66	c |                                                                             |
0.00/0.66	c |  Parsing time:          0.66         s                                      |
0.00/0.66	c |  Number Variables:      2047                                                |
0.00/0.66	c |  Number Constraints:    2708                                                |
0.00/0.66	c ===============================================================================
1800.06/485.52	c Remaining time 1799.34 sec.
1800.06/485.52	c 
1800.06/485.52	c *** INTERRUPTED ***
1800.06/485.52	s UNKNOWN
1800.06/485.52	c Total CPU time              : 1783.84 s
1800.06/485.52	c Memory used           : 152.59 MB
1800.06/485.52	c CPU time              : 1783.84 s
1800.06/485.52	c 
1800.06/485.52	c *** INTERRUPTED ***

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver version 3.2.9 (svn:492) (c) roussel@cril.univ-artois.fr

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-2697828-1278054864/watcher-2697828-1278054864 -o /tmp/evaluation-result-2697828-1278054864/solver-2697828-1278054864 -C 1800 -W 2000 -M 3800 run HOME/instance-2697828-1278054864.wbo 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 3891200 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 3942400 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.08 0.02 0.01 3/179 20101
/proc/meminfo: memFree=31286484/32950928 swapFree=67111528/67111528
[pid=20101] ppid=20099 vsize=8700 CPUtime=0
/proc/20101/stat : 20101 (run) S 20099 20101 19846 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782827 8908800 249 4089446400 4194304 4922060 140733229877728 18446744073709551615 251814059141 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/20101/statm: 2175 249 208 178 0 68 0

[startup+0.03391 s]
/proc/loadavg: 0.08 0.02 0.01 3/179 20101
/proc/meminfo: memFree=31286484/32950928 swapFree=67111528/67111528
[pid=20101] ppid=20099 vsize=8700 CPUtime=0
/proc/20101/stat : 20101 (run) S 20099 20101 19846 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782827 8908800 249 4089446400 4194304 4922060 140733229877728 18446744073709551615 251814059141 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/20101/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.100898 s]
/proc/loadavg: 0.08 0.02 0.01 3/179 20101
/proc/meminfo: memFree=31286484/32950928 swapFree=67111528/67111528
[pid=20101] ppid=20099 vsize=8700 CPUtime=0
/proc/20101/stat : 20101 (run) S 20099 20101 19846 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782827 8908800 249 4089446400 4194304 4922060 140733229877728 18446744073709551615 251814059141 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/20101/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.300864 s]
/proc/loadavg: 0.08 0.02 0.01 3/179 20101
/proc/meminfo: memFree=31286484/32950928 swapFree=67111528/67111528
[pid=20101] ppid=20099 vsize=8700 CPUtime=0
/proc/20101/stat : 20101 (run) S 20099 20101 19846 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782827 8908800 249 4089446400 4194304 4922060 140733229877728 18446744073709551615 251814059141 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/20101/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.700859 s]
/proc/loadavg: 0.08 0.02 0.01 3/179 20101
/proc/meminfo: memFree=31286484/32950928 swapFree=67111528/67111528
[pid=20101] ppid=20099 vsize=8700 CPUtime=0
/proc/20101/stat : 20101 (run) S 20099 20101 19846 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782827 8908800 249 4089446400 4194304 4922060 140733229877728 18446744073709551615 251814059141 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/20101/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+1.50067 s]
/proc/loadavg: 0.08 0.02 0.01 3/188 20124
/proc/meminfo: memFree=31281700/32950928 swapFree=67111528/67111528
[pid=20101] ppid=20099 vsize=8700 CPUtime=0
/proc/20101/stat : 20101 (run) S 20099 20101 19846 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782827 8908800 249 4089446400 4194304 4922060 140733229877728 18446744073709551615 251814059141 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/20101/statm: 2175 249 208 178 0 68 0
[pid=20102] ppid=20101 vsize=136900 CPUtime=2.57
/proc/20102/stat : 20102 (pb_cplex) R 20101 20101 19846 0 -1 4202496 19784 0 0 0 252 5 0 0 19 0 8 0 146782828 140185600 11616 4089446400 134512640 146736320 4293778320 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0
/proc/20102/statm: 34225 11616 1074 2985 0 30767 0
Current children cumulated CPU time (s) 2.57
Current children cumulated vsize (KiB) 145600

[startup+3.1004 s]
/proc/loadavg: 0.08 0.02 0.01 11/188 20145
/proc/meminfo: memFree=31247476/32950928 swapFree=67111528/67111528
[pid=20101] ppid=20099 vsize=8700 CPUtime=0
/proc/20101/stat : 20101 (run) S 20099 20101 19846 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782827 8908800 249 4089446400 4194304 4922060 140733229877728 18446744073709551615 251814059141 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/20101/statm: 2175 249 208 178 0 68 0
[pid=20102] ppid=20101 vsize=126048 CPUtime=5.11
/proc/20102/stat : 20102 (pb_cplex) R 20101 20101 19846 0 -1 4202496 29653 0 0 0 503 8 0 0 21 0 8 0 146782828 129073152 9270 4089446400 134512640 146736320 4293778320 18446744073709551615 136007605 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 0
/proc/20102/statm: 31512 9270 1080 2985 0 28054 0
Current children cumulated CPU time (s) 5.11
Current children cumulated vsize (KiB) 134748

[startup+6.33646 s]
/proc/loadavg: 1.03 0.22 0.07 5/195 20166
/proc/meminfo: memFree=31224248/32950928 swapFree=67111528/67111528
[pid=20101] ppid=20099 vsize=8700 CPUtime=0
/proc/20101/stat : 20101 (run) S 20099 20101 19846 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782827 8908800 249 4089446400 4194304 4922060 140733229877728 18446744073709551615 251814059141 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/20101/statm: 2175 249 208 178 0 68 0
[pid=20102] ppid=20101 vsize=140856 CPUtime=19.04
/proc/20102/stat : 20102 (pb_cplex) R 20101 20101 19846 0 -1 4202496 42224 0 0 0 1884 20 0 0 16 0 8 0 146782828 144236544 12741 4089446400 134512640 146736320 4293778320 18446744073709551615 136000487 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0
/proc/20102/statm: 35214 12741 1080 2985 0 31756 0
[pid=20102/tid=20160] ppid=20101 vsize=140856 CPUtime=1.6
/proc/20102/task/20160/stat : 20160 (pb_cplex) R 20101 20101 19846 0 -1 4202560 935 0 0 0 159 1 0 0 15 0 8 0 146783087 144236544 12741 4089446400 134512640 146736320 4293778320 18446744073709551615 135854524 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=20102/tid=20161] ppid=20101 vsize=140856 CPUtime=1.72
/proc/20102/task/20161/stat : 20161 (pb_cplex) S 20101 20101 19846 0 -1 4202560 873 0 0 0 171 1 0 0 15 0 8 0 146783087 144236544 12741 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 1 0 0 0
[pid=20102/tid=20162] ppid=20101 vsize=140856 CPUtime=1.94
/proc/20102/task/20162/stat : 20162 (pb_cplex) R 20101 20101 19846 0 -1 4202560 926 0 0 0 194 0 0 0 15 0 8 0 146783087 144236544 12741 4089446400 134512640 146736320 4293778320 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=20102/tid=20163] ppid=20101 vsize=140856 CPUtime=1.65
/proc/20102/task/20163/stat : 20163 (pb_cplex) R 20101 20101 19846 0 -1 4202560 2560 0 0 0 164 1 0 0 15 0 8 0 146783087 144236544 12741 4089446400 134512640 146736320 4293778320 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=20102/tid=20164] ppid=20101 vsize=140856 CPUtime=1.94
/proc/20102/task/20164/stat : 20164 (pb_cplex) R 20101 20101 19846 0 -1 4202560 626 0 0 0 192 2 0 0 15 0 8 0 146783087 144236544 12741 4089446400 134512640 146736320 4293778320 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=20102/tid=20165] ppid=20101 vsize=140856 CPUtime=1.45
/proc/20102/task/20165/stat : 20165 (pb_cplex) R 20101 20101 19846 0 -1 4202560 578 0 0 0 145 0 0 0 15 0 8 0 146783087 144236544 12741 4089446400 134512640 146736320 4293778320 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=20102/tid=20166] ppid=20101 vsize=140856 CPUtime=1.63
/proc/20102/task/20166/stat : 20166 (pb_cplex) R 20101 20101 19846 0 -1 4202560 899 0 0 0 162 1 0 0 15 0 8 0 146783087 144236544 12741 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 19.04
Current children cumulated vsize (KiB) 149556

[startup+12.7148 s]
/proc/loadavg: 1.83 0.39 0.13 7/195 20166
/proc/meminfo: memFree=31217692/32950928 swapFree=67111528/67111528
[pid=20101] ppid=20099 vsize=8700 CPUtime=0
/proc/20101/stat : 20101 (run) S 20099 20101 19846 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782827 8908800 249 4089446400 4194304 4922060 140733229877728 18446744073709551615 251814059141 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/20101/statm: 2175 249 208 178 0 68 0
[pid=20102] ppid=20101 vsize=143928 CPUtime=41.17
/proc/20102/stat : 20102 (pb_cplex) R 20101 20101 19846 0 -1 4202496 47495 0 0 0 4074 43 0 0 16 0 8 0 146782828 147382272 13563 4089446400 134512640 146736320 4293778320 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0
/proc/20102/statm: 35982 13563 1081 2985 0 32524 0
[pid=20102/tid=20160] ppid=20101 vsize=143928 CPUtime=4.3
/proc/20102/task/20160/stat : 20160 (pb_cplex) R 20101 20101 19846 0 -1 4202560 987 0 0 0 427 3 0 0 15 0 8 0 146783087 147382272 13563 4089446400 134512640 146736320 4293778320 18446744073709551615 136017496 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=20102/tid=20161] ppid=20101 vsize=143928 CPUtime=4.43
/proc/20102/task/20161/stat : 20161 (pb_cplex) R 20101 20101 19846 0 -1 4202560 1340 0 0 0 439 4 0 0 15 0 8 0 146783087 147382272 13563 4089446400 134512640 146736320 4293778320 18446744073709551615 139046527 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=20102/tid=20162] ppid=20101 vsize=143928 CPUtime=4.79
/proc/20102/task/20162/stat : 20162 (pb_cplex) R 20101 20101 19846 0 -1 4202560 1347 0 0 0 476 3 0 0 15 0 8 0 146783087 147382272 13563 4089446400 134512640 146736320 4293778320 18446744073709551615 139028333 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=20102/tid=20163] ppid=20101 vsize=143928 CPUtime=4.4
/proc/20102/task/20163/stat : 20163 (pb_cplex) R 20101 20101 19846 0 -1 4202560 4038 0 0 0 436 4 0 0 15 0 8 0 146783087 147382272 13563 4089446400 134512640 146736320 4293778320 18446744073709551615 135451622 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=20102/tid=20164] ppid=20101 vsize=143928 CPUtime=4.73
/proc/20102/task/20164/stat : 20164 (pb_cplex) R 20101 20101 19846 0 -1 4202560 1913 0 0 0 468 5 0 0 15 0 8 0 146783087 147382272 13563 4089446400 134512640 146736320 4293778320 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=20102/tid=20165] ppid=20101 vsize=143928 CPUtime=4.11
/proc/20102/task/20165/stat : 20165 (pb_cplex) R 20101 20101 19846 0 -1 4202560 1092 0 0 0 408 3 0 0 15 0 8 0 146783087 147382272 13563 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=20102/tid=20166] ppid=20101 vsize=143928 CPUtime=4.37
/proc/20102/task/20166/stat : 20166 (pb_cplex) S 20101 20101 19846 0 -1 4202560 1882 0 0 0 433 4 0 0 15 0 8 0 146783087 147382272 13563 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 41.17
Current children cumulated vsize (KiB) 152628

[startup+25.5006 s]
/proc/loadavg: 4.56 1.06 0.35 10/195 20166
/proc/meminfo: memFree=31185824/32950928 swapFree=67111528/67111528
[pid=20101] ppid=20099 vsize=8700 CPUtime=0
/proc/20101/stat : 20101 (run) S 20099 20101 19846 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782827 8908800 249 4089446400 4194304 4922060 140733229877728 18446744073709551615 251814059141 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/20101/statm: 2175 249 208 178 0 68 0
[pid=20102] ppid=20101 vsize=160684 CPUtime=86.99
/proc/20102/stat : 20102 (pb_cplex) S 20101 20101 19846 0 -1 4202496 65125 0 0 0 8617 82 0 0 20 0 8 0 146782828 164540416 17523 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0
/proc/20102/statm: 40171 17523 1088 2985 0 36713 0
[pid=20102/tid=20160] ppid=20101 vsize=160684 CPUtime=10.23
/proc/20102/task/20160/stat : 20160 (pb_cplex) R 20101 20101 19846 0 -1 4202560 4030 0 0 0 1016 7 0 0 20 0 8 0 146783087 164540416 17523 4089446400 134512640 146736320 4293778320 18446744073709551615 136403109 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=20102/tid=20161] ppid=20101 vsize=160684 CPUtime=8.57
/proc/20102/task/20161/stat : 20161 (pb_cplex) S 20101 20101 19846 0 -1 4202560 2834 0 0 0 848 9 0 0 17 0 8 0 146783087 164540416 17523 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 4 0 0 0
[pid=20102/tid=20162] ppid=20101 vsize=160684 CPUtime=10.93
/proc/20102/task/20162/stat : 20162 (pb_cplex) R 20101 20101 19846 0 -1 4202560 2578 0 0 0 1085 8 0 0 22 0 8 0 146783087 164540416 17523 4089446400 134512640 146736320 4293778320 18446744073709551615 144785853 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=20102/tid=20163] ppid=20101 vsize=160684 CPUtime=10.45
/proc/20102/task/20163/stat : 20163 (pb_cplex) R 20101 20101 19846 0 -1 4202560 7788 0 0 0 1038 7 0 0 21 0 8 0 146783087 164540416 17523 4089446400 134512640 146736320 4293778320 18446744073709551615 136008309 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=20102/tid=20164] ppid=20101 vsize=160684 CPUtime=11.57
/proc/20102/task/20164/stat : 20164 (pb_cplex) S 20101 20101 19846 0 -1 4202560 4848 0 0 0 1146 11 0 0 21 0 8 0 146783087 164540416 17523 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 7 0 0 0
[pid=20102/tid=20165] ppid=20101 vsize=160684 CPUtime=8.48
/proc/20102/task/20165/stat : 20165 (pb_cplex) S 20101 20101 19846 0 -1 4202560 2358 0 0 0 840 8 0 0 18 0 8 0 146783087 164540416 17523 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 3 0 0 0
[pid=20102/tid=20166] ppid=20101 vsize=160684 CPUtime=10.42
/proc/20102/task/20166/stat : 20166 (pb_cplex) S 20101 20101 19846 0 -1 4202560 3817 0 0 0 1032 10 0 0 20 0 8 0 146783087 164540416 17523 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 86.99
Current children cumulated vsize (KiB) 169384

[startup+51.1197 s]
/proc/loadavg: 5.81 1.62 0.55 12/195 20194
/proc/meminfo: memFree=31215620/32950928 swapFree=67111528/67111528
[pid=20101] ppid=20099 vsize=8700 CPUtime=0
/proc/20101/stat : 20101 (run) S 20099 20101 19846 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782827 8908800 249 4089446400 4194304 4922060 140733229877728 18446744073709551615 251814059141 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/20101/statm: 2175 249 208 178 0 68 0
[pid=20102] ppid=20101 vsize=140036 CPUtime=172.1
/proc/20102/stat : 20102 (pb_cplex) S 20101 20101 19846 0 -1 4202496 91926 0 0 0 17075 135 0 0 16 0 8 0 146782828 143396864 13244 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0
/proc/20102/statm: 35009 13244 1089 2985 0 31551 0
[pid=20102/tid=20174] ppid=20101 vsize=140036 CPUtime=9.76
/proc/20102/task/20174/stat : 20174 (pb_cplex) S 20101 20101 19846 0 -1 4202560 1058 0 0 0 971 5 0 0 15 0 8 0 146785931 143396864 13244 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 6 0 0 0
[pid=20102/tid=20175] ppid=20101 vsize=140036 CPUtime=9.22
/proc/20102/task/20175/stat : 20175 (pb_cplex) S 20101 20101 19846 0 -1 4202560 2614 0 0 0 917 5 0 0 15 0 8 0 146785931 143396864 13244 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 2 0 0 0
[pid=20102/tid=20176] ppid=20101 vsize=140036 CPUtime=9.86
/proc/20102/task/20176/stat : 20176 (pb_cplex) R 20101 20101 19846 0 -1 4202560 3304 0 0 0 979 7 0 0 15 0 8 0 146785931 143396864 13244 4089446400 134512640 146736320 4293778320 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=20102/tid=20177] ppid=20101 vsize=140036 CPUtime=7.73
/proc/20102/task/20177/stat : 20177 (pb_cplex) R 20101 20101 19846 0 -1 4202560 5813 0 0 0 769 4 0 0 15 0 8 0 146785931 143396864 13244 4089446400 134512640 146736320 4293778320 18446744073709551615 136403109 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=20102/tid=20178] ppid=20101 vsize=140036 CPUtime=7.75
/proc/20102/task/20178/stat : 20178 (pb_cplex) R 20101 20101 19846 0 -1 4202560 719 0 0 0 770 5 0 0 15 0 8 0 146785931 143396864 13244 4089446400 134512640 146736320 4293778320 18446744073709551615 139044564 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=20102/tid=20179] ppid=20101 vsize=140036 CPUtime=10.66
/proc/20102/task/20179/stat : 20179 (pb_cplex) R 20101 20101 19846 0 -1 4202560 6574 0 0 0 1060 6 0 0 15 0 8 0 146785931 143396864 13244 4089446400 134512640 146736320 4293778320 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=20102/tid=20180] ppid=20101 vsize=140036 CPUtime=8.1
/proc/20102/task/20180/stat : 20180 (pb_cplex) R 20101 20101 19846 0 -1 4202560 1795 0 0 0 806 4 0 0 15 0 8 0 146785931 143396864 13244 4089446400 134512640 146736320 4293778320 18446744073709551615 136007627 0 2 4096 16387 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 172.1
Current children cumulated vsize (KiB) 148736

[startup+102.308 s]
/proc/loadavg: 8.18 2.91 1.05 13/195 20196
/proc/meminfo: memFree=31212968/32950928 swapFree=67111528/67111528
[pid=20101] ppid=20099 vsize=8700 CPUtime=0
/proc/20101/stat : 20101 (run) S 20099 20101 19846 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782827 8908800 249 4089446400 4194304 4922060 140733229877728 18446744073709551615 251814059141 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/20101/statm: 2175 249 208 178 0 68 0
[pid=20102] ppid=20101 vsize=139708 CPUtime=369.9
/proc/20102/stat : 20102 (pb_cplex) S 20101 20101 19846 0 -1 4202496 111422 0 0 0 36684 306 0 0 16 0 8 0 146782828 143060992 13140 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 6 0 0 0
/proc/20102/statm: 34927 13140 1107 2985 0 31469 0
[pid=20102/tid=20174] ppid=20101 vsize=139708 CPUtime=34.47
/proc/20102/task/20174/stat : 20174 (pb_cplex) S 20101 20101 19846 0 -1 4202560 2836 0 0 0 3419 28 0 0 15 0 8 0 146785931 143060992 13140 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 4 0 0 0
[pid=20102/tid=20175] ppid=20101 vsize=139708 CPUtime=33.5
/proc/20102/task/20175/stat : 20175 (pb_cplex) S 20101 20101 19846 0 -1 4202560 5417 0 0 0 3321 29 0 0 15 0 8 0 146785931 143060992 13140 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 4 0 0 0
[pid=20102/tid=20176] ppid=20101 vsize=139708 CPUtime=34.34
/proc/20102/task/20176/stat : 20176 (pb_cplex) S 20101 20101 19846 0 -1 4202560 5683 0 0 0 3405 29 0 0 15 0 8 0 146785931 143060992 13140 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 4 0 0 0
[pid=20102/tid=20177] ppid=20101 vsize=139708 CPUtime=32.09
/proc/20102/task/20177/stat : 20177 (pb_cplex) S 20101 20101 19846 0 -1 4202560 7792 0 0 0 3186 23 0 0 15 0 8 0 146785931 143060992 13140 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 4 0 0 0

################
# More data... #
################

/proc/20102/task/20178/stat : 20178 (pb_cplex) R 20101 20101 19846 0 -1 4202560 5942 0 0 0 8893 77 0 0 15 0 8 0 146785931 146255872 13697 4089446400 134512640 146736320 4293778320 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=20102/tid=20179] ppid=20101 vsize=142828 CPUtime=92.39
/proc/20102/task/20179/stat : 20179 (pb_cplex) R 20101 20101 19846 0 -1 4202560 12797 0 0 0 9162 77 0 0 15 0 8 0 146785931 146255872 13697 4089446400 134512640 146736320 4293778320 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=20102/tid=20180] ppid=20101 vsize=142828 CPUtime=89.68
/proc/20102/task/20180/stat : 20180 (pb_cplex) R 20101 20101 19846 0 -1 4202560 5857 0 0 0 8889 79 0 0 15 0 8 0 146785931 146255872 13697 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 831.44
Current children cumulated vsize (KiB) 151528

[startup+282.303 s]
/proc/loadavg: 9.30 5.83 2.50 9/195 20270
/proc/meminfo: memFree=31218956/32950928 swapFree=67111528/67111528
[pid=20101] ppid=20099 vsize=8700 CPUtime=0
/proc/20101/stat : 20101 (run) S 20099 20101 19846 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782827 8908800 249 4089446400 4194304 4922060 140733229877728 18446744073709551615 251814059141 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/20101/statm: 2175 249 208 178 0 68 0
[pid=20102] ppid=20101 vsize=142828 CPUtime=1070.98
/proc/20102/stat : 20102 (pb_cplex) R 20101 20101 19846 0 -1 4202496 137364 0 0 0 106156 942 0 0 15 0 8 0 146782828 146255872 13847 4089446400 134512640 146736320 4293778320 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0
/proc/20102/statm: 35707 13847 1107 2985 0 32249 0
[pid=20102/tid=20174] ppid=20101 vsize=142828 CPUtime=121.63
/proc/20102/task/20174/stat : 20174 (pb_cplex) R 20101 20101 19846 0 -1 4202560 6173 0 0 0 12061 102 0 0 15 0 8 0 146785931 146255872 13847 4089446400 134512640 146736320 4293778320 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=20102/tid=20175] ppid=20101 vsize=142828 CPUtime=120.46
/proc/20102/task/20175/stat : 20175 (pb_cplex) R 20101 20101 19846 0 -1 4202560 8833 0 0 0 11935 111 0 0 15 0 8 0 146785931 146255872 13847 4089446400 134512640 146736320 4293778320 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=20102/tid=20176] ppid=20101 vsize=142828 CPUtime=121.06
/proc/20102/task/20176/stat : 20176 (pb_cplex) R 20101 20101 19846 0 -1 4202560 8734 0 0 0 11997 109 0 0 15 0 8 0 146785931 146255872 13847 4089446400 134512640 146736320 4293778320 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=20102/tid=20177] ppid=20101 vsize=142828 CPUtime=119.06
/proc/20102/task/20177/stat : 20177 (pb_cplex) R 20101 20101 19846 0 -1 4202560 11416 0 0 0 11805 101 0 0 15 0 8 0 146785931 146255872 13847 4089446400 134512640 146736320 4293778320 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=20102/tid=20178] ppid=20101 vsize=142828 CPUtime=119.58
/proc/20102/task/20178/stat : 20178 (pb_cplex) R 20101 20101 19846 0 -1 4202560 6854 0 0 0 11853 105 0 0 15 0 8 0 146785931 146255872 13847 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=20102/tid=20179] ppid=20101 vsize=142828 CPUtime=122.21
/proc/20102/task/20179/stat : 20179 (pb_cplex) S 20101 20101 19846 0 -1 4202560 12974 0 0 0 12117 104 0 0 15 0 8 0 146785931 146255872 13847 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 3 0 0 0
[pid=20102/tid=20180] ppid=20101 vsize=142828 CPUtime=119.37
/proc/20102/task/20180/stat : 20180 (pb_cplex) S 20101 20101 19846 0 -1 4202560 6201 0 0 0 11828 109 0 0 15 0 8 0 146785931 146255872 13847 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 1070.98
Current children cumulated vsize (KiB) 151528

[startup+342.31 s]
/proc/loadavg: 9.99 6.66 3.00 11/195 20293
/proc/meminfo: memFree=31212252/32950928 swapFree=67111528/67111528
[pid=20101] ppid=20099 vsize=8700 CPUtime=0
/proc/20101/stat : 20101 (run) S 20099 20101 19846 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782827 8908800 249 4089446400 4194304 4922060 140733229877728 18446744073709551615 251814059141 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/20101/statm: 2175 249 208 178 0 68 0
[pid=20102] ppid=20101 vsize=143852 CPUtime=1292.4
/proc/20102/stat : 20102 (pb_cplex) S 20101 20101 19846 0 -1 4202496 142058 0 0 0 128083 1157 0 0 16 0 8 0 146782828 147304448 14024 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 3 0 0 0
/proc/20102/statm: 35963 14024 1107 2985 0 32505 0
[pid=20102/tid=20174] ppid=20101 vsize=143852 CPUtime=148.9
/proc/20102/task/20174/stat : 20174 (pb_cplex) R 20101 20101 19846 0 -1 4202560 7335 0 0 0 14757 133 0 0 15 0 8 0 146785931 147304448 14024 4089446400 134512640 146736320 4293778320 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=20102/tid=20175] ppid=20101 vsize=143852 CPUtime=147.8
/proc/20102/task/20175/stat : 20175 (pb_cplex) R 20101 20101 19846 0 -1 4202560 9171 0 0 0 14641 139 0 0 15 0 8 0 146785931 147304448 14024 4089446400 134512640 146736320 4293778320 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=20102/tid=20176] ppid=20101 vsize=143852 CPUtime=148.51
/proc/20102/task/20176/stat : 20176 (pb_cplex) S 20101 20101 19846 0 -1 4202560 9575 0 0 0 14714 137 0 0 15 0 8 0 146785931 147304448 14024 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 5 0 0 0
[pid=20102/tid=20177] ppid=20101 vsize=143852 CPUtime=146.24
/proc/20102/task/20177/stat : 20177 (pb_cplex) R 20101 20101 19846 0 -1 4202560 11832 0 0 0 14500 124 0 0 15 0 8 0 146785931 147304448 14024 4089446400 134512640 146736320 4293778320 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=20102/tid=20178] ppid=20101 vsize=143852 CPUtime=146.75
/proc/20102/task/20178/stat : 20178 (pb_cplex) R 20101 20101 19846 0 -1 4202560 7520 0 0 0 14543 132 0 0 15 0 8 0 146785931 147304448 14024 4089446400 134512640 146736320 4293778320 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=20102/tid=20179] ppid=20101 vsize=143852 CPUtime=149.66
/proc/20102/task/20179/stat : 20179 (pb_cplex) R 20101 20101 19846 0 -1 4202560 13679 0 0 0 14839 127 0 0 15 0 8 0 146785931 147304448 14024 4089446400 134512640 146736320 4293778320 18446744073709551615 136017757 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=20102/tid=20180] ppid=20101 vsize=143852 CPUtime=146.75
/proc/20102/task/20180/stat : 20180 (pb_cplex) R 20101 20101 19846 0 -1 4202560 6275 0 0 0 14540 135 0 0 15 0 8 0 146785931 147304448 14024 4089446400 134512640 146736320 4293778320 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1292.4
Current children cumulated vsize (KiB) 152552

[startup+402.36 s]
/proc/loadavg: 9.56 7.13 3.38 7/195 20295
/proc/meminfo: memFree=31213520/32950928 swapFree=67111528/67111528
[pid=20101] ppid=20099 vsize=8700 CPUtime=0
/proc/20101/stat : 20101 (run) S 20099 20101 19846 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782827 8908800 249 4089446400 4194304 4922060 140733229877728 18446744073709551615 251814059141 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/20101/statm: 2175 249 208 178 0 68 0
[pid=20102] ppid=20101 vsize=143852 CPUtime=1504.5
/proc/20102/stat : 20102 (pb_cplex) R 20101 20101 19846 0 -1 4202496 144102 0 0 0 149086 1364 0 0 16 0 8 0 146782828 147304448 14028 4089446400 134512640 146736320 4293778320 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0
/proc/20102/statm: 35963 14028 1107 2985 0 32505 0
[pid=20102/tid=20174] ppid=20101 vsize=143852 CPUtime=175.25
/proc/20102/task/20174/stat : 20174 (pb_cplex) R 20101 20101 19846 0 -1 4202560 7896 0 0 0 17366 159 0 0 15 0 8 0 146785931 147304448 14028 4089446400 134512640 146736320 4293778320 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=20102/tid=20175] ppid=20101 vsize=143852 CPUtime=173.81
/proc/20102/task/20175/stat : 20175 (pb_cplex) R 20101 20101 19846 0 -1 4202560 9442 0 0 0 17215 166 0 0 15 0 8 0 146785931 147304448 14028 4089446400 134512640 146736320 4293778320 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=20102/tid=20176] ppid=20101 vsize=143852 CPUtime=174.48
/proc/20102/task/20176/stat : 20176 (pb_cplex) R 20101 20101 19846 0 -1 4202560 9743 0 0 0 17288 160 0 0 15 0 8 0 146785931 147304448 14028 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=20102/tid=20177] ppid=20101 vsize=143852 CPUtime=172.46
/proc/20102/task/20177/stat : 20177 (pb_cplex) S 20101 20101 19846 0 -1 4202560 12158 0 0 0 17097 149 0 0 15 0 8 0 146785931 147304448 14028 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 3 0 0 0
[pid=20102/tid=20178] ppid=20101 vsize=143852 CPUtime=173.19
/proc/20102/task/20178/stat : 20178 (pb_cplex) S 20101 20101 19846 0 -1 4202560 7800 0 0 0 17162 157 0 0 15 0 8 0 146785931 147304448 14028 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 3 0 0 0
[pid=20102/tid=20179] ppid=20101 vsize=143852 CPUtime=175.72
/proc/20102/task/20179/stat : 20179 (pb_cplex) S 20101 20101 19846 0 -1 4202560 13739 0 0 0 17417 155 0 0 15 0 8 0 146785931 147304448 14028 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 3 0 0 0
[pid=20102/tid=20180] ppid=20101 vsize=143852 CPUtime=173.21
/proc/20102/task/20180/stat : 20180 (pb_cplex) S 20101 20101 19846 0 -1 4202560 6510 0 0 0 17159 162 0 0 15 0 8 0 146785931 147304448 14028 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 1504.5
Current children cumulated vsize (KiB) 152552

[startup+462.321 s]
/proc/loadavg: 8.67 7.32 3.68 10/195 20304
/proc/meminfo: memFree=31203704/32950928 swapFree=67111528/67111528
[pid=20101] ppid=20099 vsize=8700 CPUtime=0
/proc/20101/stat : 20101 (run) S 20099 20101 19846 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782827 8908800 249 4089446400 4194304 4922060 140733229877728 18446744073709551615 251814059141 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/20101/statm: 2175 249 208 178 0 68 0
[pid=20102] ppid=20101 vsize=155224 CPUtime=1718.02
/proc/20102/stat : 20102 (pb_cplex) S 20101 20101 19846 0 -1 4202496 155389 0 0 0 170244 1558 0 0 15 0 8 0 146782828 158949376 16532 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 0
/proc/20102/statm: 38806 16532 1115 2985 0 35348 0
[pid=20102/tid=20296] ppid=20101 vsize=155224 CPUtime=24.68
/proc/20102/task/20296/stat : 20296 (pb_cplex) S 20101 20101 19846 0 -1 4202560 1568 0 0 0 2447 21 0 0 15 0 8 0 146823450 158949376 16532 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 7 0 0 0
[pid=20102/tid=20297] ppid=20101 vsize=155224 CPUtime=24.82
/proc/20102/task/20297/stat : 20297 (pb_cplex) S 20101 20101 19846 0 -1 4202560 258 0 0 0 2461 21 0 0 15 0 8 0 146823450 158949376 16532 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 7 0 0 0
[pid=20102/tid=20298] ppid=20101 vsize=155224 CPUtime=24.89
/proc/20102/task/20298/stat : 20298 (pb_cplex) S 20101 20101 19846 0 -1 4202560 1006 0 0 0 2466 23 0 0 15 0 8 0 146823450 158949376 16532 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 1 0 0 0
[pid=20102/tid=20299] ppid=20101 vsize=155224 CPUtime=24.93
/proc/20102/task/20299/stat : 20299 (pb_cplex) R 20101 20101 19846 0 -1 4202560 995 0 0 0 2471 22 0 0 15 0 8 0 146823450 158949376 16532 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=20102/tid=20300] ppid=20101 vsize=155224 CPUtime=24.61
/proc/20102/task/20300/stat : 20300 (pb_cplex) S 20101 20101 19846 0 -1 4202560 1663 0 0 0 2439 22 0 0 15 0 8 0 146823450 158949376 16532 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 0 0 0 0
[pid=20102/tid=20301] ppid=20101 vsize=155224 CPUtime=25
/proc/20102/task/20301/stat : 20301 (pb_cplex) S 20101 20101 19846 0 -1 4202560 1639 0 0 0 2476 24 0 0 15 0 8 0 146823450 158949376 16532 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 6 0 0 0
[pid=20102/tid=20302] ppid=20101 vsize=155224 CPUtime=24.79
/proc/20102/task/20302/stat : 20302 (pb_cplex) R 20101 20101 19846 0 -1 4202560 840 0 0 0 2457 22 0 0 15 0 8 0 146823450 158949376 16532 4089446400 134512640 146736320 4293778320 18446744073709551615 135983214 0 2 4096 16387 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1718.02
Current children cumulated vsize (KiB) 163924



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+485.501 s]
/proc/loadavg: 8.64 7.42 3.81 7/195 20304
/proc/meminfo: memFree=31201904/32950928 swapFree=67111528/67111528
[pid=20101] ppid=20099 vsize=8700 CPUtime=0
/proc/20101/stat : 20101 (run) S 20099 20101 19846 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782827 8908800 249 4089446400 4194304 4922060 140733229877728 18446744073709551615 251814059141 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/20101/statm: 2175 249 208 178 0 68 0
[pid=20102] ppid=20101 vsize=156248 CPUtime=1800.06
/proc/20102/stat : 20102 (pb_cplex) R 20101 20101 19846 0 -1 4202496 157138 0 0 0 178376 1630 0 0 16 0 8 0 146782828 159997952 16863 4089446400 134512640 146736320 4293778320 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 0
/proc/20102/statm: 39062 16863 1129 2985 0 35604 0
[pid=20102/tid=20296] ppid=20101 vsize=156248 CPUtime=34.99
/proc/20102/task/20296/stat : 20296 (pb_cplex) R 20101 20101 19846 0 -1 4202560 2088 0 0 0 3471 28 0 0 15 0 8 0 146823450 159997952 16863 4089446400 134512640 146736320 4293778320 18446744073709551615 137341780 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=20102/tid=20297] ppid=20101 vsize=156248 CPUtime=34.94
/proc/20102/task/20297/stat : 20297 (pb_cplex) R 20101 20101 19846 0 -1 4202560 403 0 0 0 3463 31 0 0 15 0 8 0 146823450 159997952 16863 4089446400 134512640 146736320 4293778320 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=20102/tid=20298] ppid=20101 vsize=156248 CPUtime=34.98
/proc/20102/task/20298/stat : 20298 (pb_cplex) R 20101 20101 19846 0 -1 4202560 1170 0 0 0 3465 33 0 0 15 0 8 0 146823450 159997952 16863 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=20102/tid=20299] ppid=20101 vsize=156248 CPUtime=35.08
/proc/20102/task/20299/stat : 20299 (pb_cplex) S 20101 20101 19846 0 -1 4202560 1324 0 0 0 3478 30 0 0 15 0 8 0 146823450 159997952 16863 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 0 0 0 0
[pid=20102/tid=20300] ppid=20101 vsize=156248 CPUtime=34.59
/proc/20102/task/20300/stat : 20300 (pb_cplex) S 20101 20101 19846 0 -1 4202560 1775 0 0 0 3429 30 0 0 15 0 8 0 146823450 159997952 16863 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 4 0 0 0
[pid=20102/tid=20301] ppid=20101 vsize=156248 CPUtime=35.14
/proc/20102/task/20301/stat : 20301 (pb_cplex) S 20101 20101 19846 0 -1 4202560 1653 0 0 0 3481 33 0 0 15 0 8 0 146823450 159997952 16863 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 0 0 0 0
[pid=20102/tid=20302] ppid=20101 vsize=156248 CPUtime=35.05
/proc/20102/task/20302/stat : 20302 (pb_cplex) S 20101 20101 19846 0 -1 4202560 884 0 0 0 3475 30 0 0 15 0 8 0 146823450 159997952 16863 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 1800.06
Current children cumulated vsize (KiB) 164948

Sending SIGTERM to process tree (bottom up)
SSolver just ended. Dumping a history of the last processes samples

[startup+485.501 s]
/proc/loadavg: 8.64 7.42 3.81 7/195 20304
/proc/meminfo: memFree=31201904/32950928 swapFree=67111528/67111528
[pid=20101] ppid=20099 vsize=8700 CPUtime=0
/proc/20101/stat : 20101 (run) S 20099 20101 19846 0 -1 4202496 365 0 0 0 0 0 0 0 21 0 1 0 146782827 8908800 249 4089446400 4194304 4922060 140733229877728 18446744073709551615 251814059141 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/20101/statm: 2175 249 208 178 0 68 0
[pid=20102] ppid=20101 vsize=156248 CPUtime=1800.06
/proc/20102/stat : 20102 (pb_cplex) R 20101 20101 19846 0 -1 4202496 157138 0 0 0 178376 1630 0 0 16 0 8 0 146782828 159997952 16863 4089446400 134512640 146736320 4293778320 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 0
/proc/20102/statm: 39062 16863 1129 2985 0 35604 0
[pid=20102/tid=20296] ppid=20101 vsize=156248 CPUtime=34.99
/proc/20102/task/20296/stat : 20296 (pb_cplex) R 20101 20101 19846 0 -1 4202560 2088 0 0 0 3471 28 0 0 15 0 8 0 146823450 159997952 16863 4089446400 134512640 146736320 4293778320 18446744073709551615 137341780 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=20102/tid=20297] ppid=20101 vsize=156248 CPUtime=34.94
/proc/20102/task/20297/stat : 20297 (pb_cplex) R 20101 20101 19846 0 -1 4202560 403 0 0 0 3463 31 0 0 15 0 8 0 146823450 159997952 16863 4089446400 134512640 146736320 4293778320 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=20102/tid=20298] ppid=20101 vsize=156248 CPUtime=34.98
/proc/20102/task/20298/stat : 20298 (pb_cplex) R 20101 20101 19846 0 -1 4202560 1170 0 0 0 3465 33 0 0 15 0 8 0 146823450 159997952 16863 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=20102/tid=20299] ppid=20101 vsize=156248 CPUtime=35.08
/proc/20102/task/20299/stat : 20299 (pb_cplex) S 20101 20101 19846 0 -1 4202560 1324 0 0 0 3478 30 0 0 15 0 8 0 146823450 159997952 16863 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 0 0 0 0
[pid=20102/tid=20300] ppid=20101 vsize=156248 CPUtime=34.59
/proc/20102/task/20300/stat : 20300 (pb_cplex) S 20101 20101 19846 0 -1 4202560 1775 0 0 0 3429 30 0 0 15 0 8 0 146823450 159997952 16863 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 4 0 0 0
[pid=20102/tid=20301] ppid=20101 vsize=156248 CPUtime=35.14
/proc/20102/task/20301/stat : 20301 (pb_cplex) S 20101 20101 19846 0 -1 4202560 1653 0 0 0 3481 33 0 0 15 0 8 0 146823450 159997952 16863 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 0 0 0 0
[pid=20102/tid=20302] ppid=20101 vsize=156248 CPUtime=35.05
/proc/20102/task/20302/stat : 20302 (pb_cplex) S 20101 20101 19846 0 -1 4202560 884 0 0 0 3475 30 0 0 15 0 8 0 146823450 159997952 16863 4089446400 134512640 146736320 4293778320 18446744073709551615 144580073 0 2 4096 16387 18446604471108963264 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 1800.06
Current children cumulated vsize (KiB) 164948

Child status: 0
Real time (s): 485.526
CPU time (s): 1800.15
CPU user time (s): 1783.84
CPU system time (s): 16.3065
CPU usage (%): 370.762
Max. virtual memory (cumulated for all children) (KiB): 176960

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1783.84
system time used= 16.3065
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 157556
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 331784
involuntary context switches= 29103

runsolver used 0.963853 second user time and 1.9677 second system time

The end

Launcher Data 

Begin job on node126 at 2010-07-02 09:14:25
IDJOB=2697828
IDBENCH=78902
IDSOLVER=1210
FILE ID=node126/2697828-1278054864
PBS_JOBID= 11198930
Free space on /tmp= 74168 MiB

SOLVER NAME= pb_cplex 2010-07-01
BENCH NAME= PB10/normalized-PB10/PARTIAL-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/tsp/normalized-t3002.11tsp11.1900575442--soft-33-100-0.wbo
COMMAND LINE= run BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-2697828-1278054864/watcher-2697828-1278054864 -o /tmp/evaluation-result-2697828-1278054864/solver-2697828-1278054864 -C 1800 -W 2000 -M 3800  run HOME/instance-2697828-1278054864.wbo

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 3800 MiB
MAX NB THREAD= 0

MD5SUM BENCH= c8fdbd1e89585cb1f1efe2070d2e6c87
RANDOM SEED=1934137626

node126.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.835
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.835
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.835
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.835
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.835
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.72
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.835
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.835
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.835
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      31287012 kB
Buffers:        214808 kB
Cached:        1204940 kB
SwapCached:          0 kB
Active:         624500 kB
Inactive:       889560 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      31287012 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           33684 kB
Writeback:           0 kB
AnonPages:       95120 kB
Mapped:          18696 kB
Slab:            83520 kB
PageTables:       4924 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   277568 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 74152 MiB
End job on node126 at 2010-07-02 09:22:30