Trace number 2697411

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
pb_cplex 2010-07-01? (TO) 1800.27 554.756

General information on the benchmark

Name/SOFT-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/
tsp/normalized-t3002.11tsp11.1900573137--soft-0-100-0.wbo
MD5SUM337bb89bbfc43fe8a5ddb591a5c00326
Bench CategorySOFT-SMALLINT-LIN (only soft constraints, small integers, linear constraints)
Best result obtained on this benchmarkMOPT
Best cost obtained on this benchmark1
Best CPU time to get the best result obtained on this benchmark205.929
Max-Satisfiable
Max-(Un)Satisfiability was proved
Best value of the cost
Optimality of the best cost was proved
Number of variables231
Total number of constraints2707
Number of soft constraints2707
Number of constraints which are clauses2684
Number of constraints which are cardinality constraints (but not clauses)22
Number of constraints which are nor clauses,nor cardinality constraints1
Minimum length of a constraint2
Maximum length of a constraint110
Top cost 136625
Min constraint cost 1
Max constraint cost 100
Sum of constraints costs 136624
Biggest number in a constraint 25
Number of bits of the biggest number in a constraint 5
Biggest sum of numbers in a constraint 693
Number of bits of the biggest sum of numbers10
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c pb_cplex 1.0 - 20100701
0.00/0.00	c Parsing wbo file format. File HOME/instance-2697411-1278055626.wbo.
0.00/0.00	c Instance file HOME/instance-2697411-1278055626.wbo
0.00/0.00	c File size is 96756 bytes.
1.39/1.43	c Highest Coefficient sum: 693
1.39/1.43	c  IBM ILOG License Manager: "IBM ILOG Optimization Suite for Academic Initiative" is accessing CPLEX 12 with option(s): "e m b q ".
1.39/1.43	c ============================[ Problem Statistics ]=============================
1.39/1.43	c |                                                                             |
1.39/1.43	c |  Parsing time:          1.43         s                                      |
1.39/1.43	c |  Number Variables:      2938                                                |
1.39/1.43	c |  Number Constraints:    2708                                                |
1.39/1.43	c ===============================================================================
1800.22/554.51	c Remaining time 1798.56 sec.
1800.22/554.51	c 
1800.22/554.51	c *** INTERRUPTED ***
1800.22/554.51	s UNKNOWN
1800.22/554.51	c Total CPU time              : 1784.86 s
1800.22/554.51	c Memory used           : 141.68 MB
1800.22/554.51	c CPU time              : 1784.86 s
1800.22/554.51	c 
1800.22/554.51	c *** INTERRUPTED ***

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver version 3.2.9 (svn:492) (c) roussel@cril.univ-artois.fr

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-2697411-1278055626/watcher-2697411-1278055626 -o /tmp/evaluation-result-2697411-1278055626/solver-2697411-1278055626 -C 1800 -W 2000 -M 3800 run HOME/instance-2697411-1278055626.wbo 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 3891200 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 3942400 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 10.64 8.73 5.25 4/190 22745
/proc/meminfo: memFree=31126584/32951124 swapFree=67111528/67111528
[pid=22745] ppid=22743 vsize=8700 CPUtime=0
/proc/22745/stat : 22745 (run) S 22743 22745 22164 0 -1 4202496 366 0 0 0 0 0 0 0 25 0 1 0 146858780 8908800 249 4089446400 4194304 4922060 140733380883456 18446744073709551615 232335711365 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/22745/statm: 2175 249 208 178 0 68 0

[startup+0.0585379 s]
/proc/loadavg: 10.64 8.73 5.25 4/190 22745
/proc/meminfo: memFree=31126584/32951124 swapFree=67111528/67111528
[pid=22745] ppid=22743 vsize=8700 CPUtime=0
/proc/22745/stat : 22745 (run) S 22743 22745 22164 0 -1 4202496 366 0 0 0 0 0 0 0 25 0 1 0 146858780 8908800 249 4089446400 4194304 4922060 140733380883456 18446744073709551615 232335711365 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/22745/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.100533 s]
/proc/loadavg: 10.64 8.73 5.25 4/190 22745
/proc/meminfo: memFree=31126584/32951124 swapFree=67111528/67111528
[pid=22745] ppid=22743 vsize=8700 CPUtime=0
/proc/22745/stat : 22745 (run) S 22743 22745 22164 0 -1 4202496 366 0 0 0 0 0 0 0 25 0 1 0 146858780 8908800 249 4089446400 4194304 4922060 140733380883456 18446744073709551615 232335711365 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/22745/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.300506 s]
/proc/loadavg: 10.64 8.73 5.25 4/190 22745
/proc/meminfo: memFree=31126584/32951124 swapFree=67111528/67111528
[pid=22745] ppid=22743 vsize=8700 CPUtime=0
/proc/22745/stat : 22745 (run) S 22743 22745 22164 0 -1 4202496 366 0 0 0 0 0 0 0 25 0 1 0 146858780 8908800 249 4089446400 4194304 4922060 140733380883456 18446744073709551615 232335711365 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/22745/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.700439 s]
/proc/loadavg: 10.64 8.73 5.25 4/190 22745
/proc/meminfo: memFree=31126584/32951124 swapFree=67111528/67111528
[pid=22745] ppid=22743 vsize=8700 CPUtime=0
/proc/22745/stat : 22745 (run) S 22743 22745 22164 0 -1 4202496 366 0 0 0 0 0 0 0 25 0 1 0 146858780 8908800 249 4089446400 4194304 4922060 140733380883456 18446744073709551615 232335711365 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/22745/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+1.50029 s]
/proc/loadavg: 9.95 8.62 5.23 3/192 22747
/proc/meminfo: memFree=31125824/32951124 swapFree=67111528/67111528
[pid=22745] ppid=22743 vsize=8700 CPUtime=0
/proc/22745/stat : 22745 (run) S 22743 22745 22164 0 -1 4202496 366 0 0 0 0 0 0 0 25 0 1 0 146858780 8908800 249 4089446400 4194304 4922060 140733380883456 18446744073709551615 232335711365 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/22745/statm: 2175 249 208 178 0 68 0
[pid=22746] ppid=22745 vsize=27052 CPUtime=1.49
/proc/22746/stat : 22746 (pb_cplex) R 22745 22745 22164 0 -1 4202496 3592 0 0 0 149 0 0 0 25 0 1 0 146858780 27701248 3118 4089446400 134512640 146736320 4289056880 18446744073709551615 136935116 0 0 4096 16387 0 0 0 17 2 0 0 0
/proc/22746/statm: 6763 3118 975 2985 0 3305 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 35752

[startup+3.10106 s]
/proc/loadavg: 9.95 8.62 5.23 8/199 22768
/proc/meminfo: memFree=31095752/32951124 swapFree=67111528/67111528
[pid=22745] ppid=22743 vsize=8700 CPUtime=0
/proc/22745/stat : 22745 (run) S 22743 22745 22164 0 -1 4202496 366 0 0 0 0 0 0 0 25 0 1 0 146858780 8908800 249 4089446400 4194304 4922060 140733380883456 18446744073709551615 232335711365 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/22745/statm: 2175 249 208 178 0 68 0
[pid=22746] ppid=22745 vsize=94904 CPUtime=4.47
/proc/22746/stat : 22746 (pb_cplex) R 22745 22745 22164 0 -1 4202496 21478 0 0 0 441 6 0 0 16 0 1 0 146858780 97181696 9372 4089446400 134512640 146736320 4289056880 18446744073709551615 136017496 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/22746/statm: 23726 9372 1073 2985 0 20268 0
Current children cumulated CPU time (s) 4.47
Current children cumulated vsize (KiB) 103604

[startup+6.3074 s]
/proc/loadavg: 9.95 8.62 5.23 16/199 22796
/proc/meminfo: memFree=31111748/32951124 swapFree=67111528/67111528
[pid=22745] ppid=22743 vsize=8700 CPUtime=0
/proc/22745/stat : 22745 (run) S 22743 22745 22164 0 -1 4202496 366 0 0 0 0 0 0 0 25 0 1 0 146858780 8908800 249 4089446400 4194304 4922060 140733380883456 18446744073709551615 232335711365 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/22745/statm: 2175 249 208 178 0 68 0
[pid=22746] ppid=22745 vsize=138628 CPUtime=14.65
/proc/22746/stat : 22746 (pb_cplex) R 22745 22745 22164 0 -1 4202496 33056 0 0 0 1451 14 0 0 15 0 8 0 146858780 141955072 12517 4089446400 134512640 146736320 4289056880 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0
/proc/22746/statm: 34657 12517 1080 2985 0 31199 0
[pid=22746/tid=22783] ppid=22745 vsize=138628 CPUtime=0.97
/proc/22746/task/22783/stat : 22783 (pb_cplex) R 22745 22745 22164 0 -1 4202560 783 0 0 0 97 0 0 0 15 0 8 0 146859127 141955072 12517 4089446400 134512640 146736320 4289056880 18446744073709551615 136017902 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=22746/tid=22784] ppid=22745 vsize=138628 CPUtime=1.23
/proc/22746/task/22784/stat : 22784 (pb_cplex) R 22745 22745 22164 0 -1 4202560 1714 0 0 0 123 0 0 0 19 0 8 0 146859127 141955072 12517 4089446400 134512640 146736320 4289056880 18446744073709551615 137371522 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=22746/tid=22785] ppid=22745 vsize=138628 CPUtime=0.99
/proc/22746/task/22785/stat : 22785 (pb_cplex) R 22745 22745 22164 0 -1 4202560 656 0 0 0 99 0 0 0 15 0 8 0 146859127 141955072 12517 4089446400 134512640 146736320 4289056880 18446744073709551615 136384857 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=22746/tid=22786] ppid=22745 vsize=138628 CPUtime=0.94
/proc/22746/task/22786/stat : 22786 (pb_cplex) R 22745 22745 22164 0 -1 4202560 190 0 0 0 94 0 0 0 15 0 8 0 146859127 141955072 12517 4089446400 134512640 146736320 4289056880 18446744073709551615 136475791 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=22746/tid=22787] ppid=22745 vsize=138628 CPUtime=1.1
/proc/22746/task/22787/stat : 22787 (pb_cplex) R 22745 22745 22164 0 -1 4202560 311 0 0 0 110 0 0 0 15 0 8 0 146859127 141955072 12517 4089446400 134512640 146736320 4289056880 18446744073709551615 136384857 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=22746/tid=22788] ppid=22745 vsize=138628 CPUtime=0.86
/proc/22746/task/22788/stat : 22788 (pb_cplex) R 22745 22745 22164 0 -1 4202560 1157 0 0 0 85 1 0 0 15 0 8 0 146859127 141955072 12517 4089446400 134512640 146736320 4289056880 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=22746/tid=22789] ppid=22745 vsize=138628 CPUtime=1.08
/proc/22746/task/22789/stat : 22789 (pb_cplex) R 22745 22745 22164 0 -1 4202560 681 0 0 0 108 0 0 0 15 0 8 0 146859127 141955072 12517 4089446400 134512640 146736320 4289056880 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 14.65
Current children cumulated vsize (KiB) 147328

[startup+12.7013 s]
/proc/loadavg: 10.34 8.75 5.31 8/199 22810
/proc/meminfo: memFree=31097992/32951124 swapFree=67111528/67111528
[pid=22745] ppid=22743 vsize=8700 CPUtime=0
/proc/22745/stat : 22745 (run) S 22743 22745 22164 0 -1 4202496 366 0 0 0 0 0 0 0 25 0 1 0 146858780 8908800 249 4089446400 4194304 4922060 140733380883456 18446744073709551615 232335711365 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/22745/statm: 2175 249 208 178 0 68 0
[pid=22746] ppid=22745 vsize=144772 CPUtime=34.06
/proc/22746/stat : 22746 (pb_cplex) R 22745 22745 22164 0 -1 4202496 38863 0 0 0 3372 34 0 0 16 0 8 0 146858780 148246528 13656 4089446400 134512640 146736320 4289056880 18446744073709551615 136018931 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/22746/statm: 36193 13656 1081 2985 0 32735 0
[pid=22746/tid=22783] ppid=22745 vsize=144772 CPUtime=3.4
/proc/22746/task/22783/stat : 22783 (pb_cplex) S 22745 22745 22164 0 -1 4202560 1528 0 0 0 338 2 0 0 15 0 8 0 146859127 148246528 13656 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 5 0 0 0
[pid=22746/tid=22784] ppid=22745 vsize=144772 CPUtime=3.78
/proc/22746/task/22784/stat : 22784 (pb_cplex) R 22745 22745 22164 0 -1 4202560 1901 0 0 0 376 2 0 0 15 0 8 0 146859127 148246528 13656 4089446400 134512640 146736320 4289056880 18446744073709551615 139042904 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=22746/tid=22785] ppid=22745 vsize=144772 CPUtime=3.44
/proc/22746/task/22785/stat : 22785 (pb_cplex) R 22745 22745 22164 0 -1 4202560 2569 0 0 0 341 3 0 0 15 0 8 0 146859127 148246528 13656 4089446400 134512640 146736320 4289056880 18446744073709551615 136403109 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=22746/tid=22786] ppid=22745 vsize=144772 CPUtime=3.28
/proc/22746/task/22786/stat : 22786 (pb_cplex) S 22745 22745 22164 0 -1 4202560 438 0 0 0 325 3 0 0 15 0 8 0 146859127 148246528 13656 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 3 0 0 0
[pid=22746/tid=22787] ppid=22745 vsize=144772 CPUtime=3.49
/proc/22746/task/22787/stat : 22787 (pb_cplex) R 22745 22745 22164 0 -1 4202560 1573 0 0 0 348 1 0 0 15 0 8 0 146859127 148246528 13656 4089446400 134512640 146736320 4289056880 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=22746/tid=22788] ppid=22745 vsize=144772 CPUtime=3.28
/proc/22746/task/22788/stat : 22788 (pb_cplex) R 22745 22745 22164 0 -1 4202560 2295 0 0 0 324 4 0 0 15 0 8 0 146859127 148246528 13656 4089446400 134512640 146736320 4289056880 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=22746/tid=22789] ppid=22745 vsize=144772 CPUtime=3.37
/proc/22746/task/22789/stat : 22789 (pb_cplex) R 22745 22745 22164 0 -1 4202560 684 0 0 0 333 4 0 0 15 0 8 0 146859127 148246528 13656 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 34.06
Current children cumulated vsize (KiB) 153472

[startup+25.501 s]
/proc/loadavg: 10.13 8.76 5.34 11/199 22811
/proc/meminfo: memFree=31083616/32951124 swapFree=67111528/67111528
[pid=22745] ppid=22743 vsize=8700 CPUtime=0
/proc/22745/stat : 22745 (run) S 22743 22745 22164 0 -1 4202496 366 0 0 0 0 0 0 0 25 0 1 0 146858780 8908800 249 4089446400 4194304 4922060 140733380883456 18446744073709551615 232335711365 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/22745/statm: 2175 249 208 178 0 68 0
[pid=22746] ppid=22745 vsize=162972 CPUtime=75.59
/proc/22746/stat : 22746 (pb_cplex) R 22745 22745 22164 0 -1 4202496 55714 0 0 0 7484 75 0 0 18 0 8 0 146858780 166883328 17752 4089446400 134512640 146736320 4289056880 18446744073709551615 138736300 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0
/proc/22746/statm: 40743 17752 1092 2985 0 37285 0
[pid=22746/tid=22783] ppid=22745 vsize=162972 CPUtime=8.89
/proc/22746/task/22783/stat : 22783 (pb_cplex) R 22745 22745 22164 0 -1 4202560 3779 0 0 0 881 8 0 0 18 0 8 0 146859127 166883328 17752 4089446400 134512640 146736320 4289056880 18446744073709551615 136385184 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=22746/tid=22784] ppid=22745 vsize=162972 CPUtime=8.84
/proc/22746/task/22784/stat : 22784 (pb_cplex) R 22745 22745 22164 0 -1 4202560 2452 0 0 0 876 8 0 0 18 0 8 0 146859127 166883328 17752 4089446400 134512640 146736320 4289056880 18446744073709551615 136389777 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=22746/tid=22785] ppid=22745 vsize=162972 CPUtime=7.67
/proc/22746/task/22785/stat : 22785 (pb_cplex) S 22745 22745 22164 0 -1 4202560 3792 0 0 0 758 9 0 0 17 0 8 0 146859127 166883328 17752 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 3 0 0 0
[pid=22746/tid=22786] ppid=22745 vsize=162972 CPUtime=8.12
/proc/22746/task/22786/stat : 22786 (pb_cplex) R 22745 22745 22164 0 -1 4202560 2745 0 0 0 806 6 0 0 18 0 8 0 146859127 166883328 17752 4089446400 134512640 146736320 4289056880 18446744073709551615 136008226 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=22746/tid=22787] ppid=22745 vsize=162972 CPUtime=8.54
/proc/22746/task/22787/stat : 22787 (pb_cplex) R 22745 22745 22164 0 -1 4202560 4142 0 0 0 848 6 0 0 18 0 8 0 146859127 166883328 17752 4089446400 134512640 146736320 4289056880 18446744073709551615 144785853 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=22746/tid=22788] ppid=22745 vsize=162972 CPUtime=9.13
/proc/22746/task/22788/stat : 22788 (pb_cplex) R 22745 22745 22164 0 -1 4202560 6805 0 0 0 904 9 0 0 19 0 8 0 146859127 166883328 17752 4089446400 134512640 146736320 4289056880 18446744073709551615 135850295 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=22746/tid=22789] ppid=22745 vsize=162972 CPUtime=8.46
/proc/22746/task/22789/stat : 22789 (pb_cplex) R 22745 22745 22164 0 -1 4202560 3554 0 0 0 837 9 0 0 17 0 8 0 146859127 166883328 17752 4089446400 134512640 146736320 4289056880 18446744073709551615 136000487 0 2 4096 16387 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 75.59
Current children cumulated vsize (KiB) 171672

[startup+51.1015 s]
/proc/loadavg: 9.70 8.79 5.46 10/199 22825
/proc/meminfo: memFree=31098252/32951124 swapFree=67111528/67111528
[pid=22745] ppid=22743 vsize=8700 CPUtime=0
/proc/22745/stat : 22745 (run) S 22743 22745 22164 0 -1 4202496 366 0 0 0 0 0 0 0 25 0 1 0 146858780 8908800 249 4089446400 4194304 4922060 140733380883456 18446744073709551615 232335711365 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/22745/statm: 2175 249 208 178 0 68 0
[pid=22746] ppid=22745 vsize=143924 CPUtime=146.57
/proc/22746/stat : 22746 (pb_cplex) S 22745 22745 22164 0 -1 4202496 90026 0 0 0 14526 131 0 0 16 0 8 0 146858780 147378176 13690 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0
/proc/22746/statm: 35981 13690 1089 2985 0 32523 0
[pid=22746/tid=22819] ppid=22745 vsize=143924 CPUtime=6.02
/proc/22746/task/22819/stat : 22819 (pb_cplex) S 22745 22745 22164 0 -1 4202560 771 0 0 0 598 4 0 0 15 0 8 0 146862181 147378176 13690 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 5 0 0 0
[pid=22746/tid=22820] ppid=22745 vsize=143924 CPUtime=6.2
/proc/22746/task/22820/stat : 22820 (pb_cplex) S 22745 22745 22164 0 -1 4202560 2283 0 0 0 614 6 0 0 15 0 8 0 146862181 147378176 13690 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 5 0 0 0
[pid=22746/tid=22821] ppid=22745 vsize=143924 CPUtime=6.27
/proc/22746/task/22821/stat : 22821 (pb_cplex) S 22745 22745 22164 0 -1 4202560 3619 0 0 0 623 4 0 0 15 0 8 0 146862181 147378176 13690 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 6 0 0 0
[pid=22746/tid=22822] ppid=22745 vsize=143924 CPUtime=7.59
/proc/22746/task/22822/stat : 22822 (pb_cplex) R 22745 22745 22164 0 -1 4202560 379 0 0 0 754 5 0 0 15 0 8 0 146862181 147378176 13690 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=22746/tid=22823] ppid=22745 vsize=143924 CPUtime=6.02
/proc/22746/task/22823/stat : 22823 (pb_cplex) S 22745 22745 22164 0 -1 4202560 976 0 0 0 598 4 0 0 15 0 8 0 146862181 147378176 13690 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 4 0 0 0
[pid=22746/tid=22824] ppid=22745 vsize=143924 CPUtime=7.7
/proc/22746/task/22824/stat : 22824 (pb_cplex) S 22745 22745 22164 0 -1 4202560 4718 0 0 0 767 3 0 0 15 0 8 0 146862181 147378176 13690 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 6 0 0 0
[pid=22746/tid=22825] ppid=22745 vsize=143924 CPUtime=6.07
/proc/22746/task/22825/stat : 22825 (pb_cplex) R 22745 22745 22164 0 -1 4202560 1654 0 0 0 603 4 0 0 15 0 8 0 146862181 147378176 13690 4089446400 134512640 146736320 4289056880 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 146.57
Current children cumulated vsize (KiB) 152624

[startup+102.324 s]
/proc/loadavg: 9.98 9.02 5.72 11/199 22827
/proc/meminfo: memFree=31096064/32951124 swapFree=67111528/67111528
[pid=22745] ppid=22743 vsize=8700 CPUtime=0
/proc/22745/stat : 22745 (run) S 22743 22745 22164 0 -1 4202496 366 0 0 0 0 0 0 0 25 0 1 0 146858780 8908800 249 4089446400 4194304 4922060 140733380883456 18446744073709551615 232335711365 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/22745/statm: 2175 249 208 178 0 68 0
[pid=22746] ppid=22745 vsize=146996 CPUtime=315.43
/proc/22746/stat : 22746 (pb_cplex) S 22745 22745 22164 0 -1 4202496 102980 0 0 0 31270 273 0 0 15 0 8 0 146858780 150523904 14453 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0
/proc/22746/statm: 36749 14453 1089 2985 0 33291 0
[pid=22746/tid=22819] ppid=22745 vsize=146996 CPUtime=26.8
/proc/22746/task/22819/stat : 22819 (pb_cplex) S 22745 22745 22164 0 -1 4202560 3545 0 0 0 2657 23 0 0 15 0 8 0 146862181 150523904 14453 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 6 0 0 0
[pid=22746/tid=22820] ppid=22745 vsize=146996 CPUtime=26.97
/proc/22746/task/22820/stat : 22820 (pb_cplex) R 22745 22745 22164 0 -1 4202560 2886 0 0 0 2677 20 0 0 15 0 8 0 146862181 150523904 14453 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=22746/tid=22821] ppid=22745 vsize=146996 CPUtime=26.98
/proc/22746/task/22821/stat : 22821 (pb_cplex) S 22745 22745 22164 0 -1 4202560 4746 0 0 0 2677 21 0 0 16 0 8 0 146862181 150523904 14453 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 1 0 0 0
[pid=22746/tid=22822] ppid=22745 vsize=146996 CPUtime=28.52
/proc/22746/task/22822/stat : 22822 (pb_cplex) S 22745 22745 22164 0 -1 4202560 1312 0 0 0 2829 23 0 0 15 0 8 0 146862181 150523904 14453 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 3 0 0 0

################
# More data... #
################

/proc/22746/task/22820/stat : 22820 (pb_cplex) R 22745 22745 22164 0 -1 4202560 6720 0 0 0 9941 88 0 0 15 0 8 0 146862181 149475328 14440 4089446400 134512640 146736320 4289056880 18446744073709551615 136403109 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=22746/tid=22821] ppid=22745 vsize=145972 CPUtime=100.53
/proc/22746/task/22821/stat : 22821 (pb_cplex) R 22745 22745 22164 0 -1 4202560 11194 0 0 0 9965 88 0 0 15 0 8 0 146862181 149475328 14440 4089446400 134512640 146736320 4289056880 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=22746/tid=22822] ppid=22745 vsize=145972 CPUtime=101.88
/proc/22746/task/22822/stat : 22822 (pb_cplex) S 22745 22745 22164 0 -1 4202560 6932 0 0 0 10105 83 0 0 15 0 8 0 146862181 149475328 14440 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 6 0 0 0
[pid=22746/tid=22823] ppid=22745 vsize=145972 CPUtime=100.81
/proc/22746/task/22823/stat : 22823 (pb_cplex) R 22745 22745 22164 0 -1 4202560 6276 0 0 0 10000 81 0 0 15 0 8 0 146862181 149475328 14440 4089446400 134512640 146736320 4289056880 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=22746/tid=22824] ppid=22745 vsize=145972 CPUtime=102.12
/proc/22746/task/22824/stat : 22824 (pb_cplex) R 22745 22745 22164 0 -1 4202560 10524 0 0 0 10130 82 0 0 15 0 8 0 146862181 149475328 14440 4089446400 134512640 146736320 4289056880 18446744073709551615 136366866 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=22746/tid=22825] ppid=22745 vsize=145972 CPUtime=101
/proc/22746/task/22825/stat : 22825 (pb_cplex) R 22745 22745 22164 0 -1 4202560 7196 0 0 0 10012 88 0 0 15 0 8 0 146862181 149475328 14440 4089446400 134512640 146736320 4289056880 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 915.14
Current children cumulated vsize (KiB) 154672

[startup+342.328 s]
/proc/loadavg: 9.11 9.20 6.55 10/199 22892
/proc/meminfo: memFree=30984012/32951124 swapFree=67111528/67111528
[pid=22745] ppid=22743 vsize=8700 CPUtime=0
/proc/22745/stat : 22745 (run) S 22743 22745 22164 0 -1 4202496 366 0 0 0 0 0 0 0 25 0 1 0 146858780 8908800 249 4089446400 4194304 4922060 140733380883456 18446744073709551615 232335711365 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/22745/statm: 2175 249 208 178 0 68 0
[pid=22746] ppid=22745 vsize=146420 CPUtime=1143.27
/proc/22746/stat : 22746 (pb_cplex) R 22745 22745 22164 0 -1 4202496 145801 0 0 0 113341 986 0 0 15 0 8 0 146858780 149934080 14474 4089446400 134512640 146736320 4289056880 18446744073709551615 139044550 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0
/proc/22746/statm: 36605 14474 1089 2985 0 33147 0
[pid=22746/tid=22819] ppid=22745 vsize=146420 CPUtime=129.16
/proc/22746/task/22819/stat : 22819 (pb_cplex) R 22745 22745 22164 0 -1 4202560 7185 0 0 0 12808 108 0 0 15 0 8 0 146862181 149934080 14474 4089446400 134512640 146736320 4289056880 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=22746/tid=22820] ppid=22745 vsize=146420 CPUtime=128.33
/proc/22746/task/22820/stat : 22820 (pb_cplex) R 22745 22745 22164 0 -1 4202560 7932 0 0 0 12719 114 0 0 15 0 8 0 146862181 149934080 14474 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=22746/tid=22821] ppid=22745 vsize=146420 CPUtime=128.71
/proc/22746/task/22821/stat : 22821 (pb_cplex) S 22745 22745 22164 0 -1 4202560 11706 0 0 0 12754 117 0 0 16 0 8 0 146862181 149934080 14474 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 7 0 0 0
[pid=22746/tid=22822] ppid=22745 vsize=146420 CPUtime=129.93
/proc/22746/task/22822/stat : 22822 (pb_cplex) S 22745 22745 22164 0 -1 4202560 8455 0 0 0 12885 108 0 0 15 0 8 0 146862181 149934080 14474 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 7 0 0 0
[pid=22746/tid=22823] ppid=22745 vsize=146420 CPUtime=129.15
/proc/22746/task/22823/stat : 22823 (pb_cplex) S 22745 22745 22164 0 -1 4202560 8029 0 0 0 12809 106 0 0 16 0 8 0 146862181 149934080 14474 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 7 0 0 0
[pid=22746/tid=22824] ppid=22745 vsize=146420 CPUtime=130.42
/proc/22746/task/22824/stat : 22824 (pb_cplex) S 22745 22745 22164 0 -1 4202560 11719 0 0 0 12937 105 0 0 15 0 8 0 146862181 149934080 14474 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 7 0 0 0
[pid=22746/tid=22825] ppid=22745 vsize=146420 CPUtime=129.18
/proc/22746/task/22825/stat : 22825 (pb_cplex) S 22745 22745 22164 0 -1 4202560 8058 0 0 0 12802 116 0 0 15 0 8 0 146862181 149934080 14474 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 1143.27
Current children cumulated vsize (KiB) 155120

[startup+402.302 s]
/proc/loadavg: 9.66 9.28 6.74 13/199 22908
/proc/meminfo: memFree=30976076/32951124 swapFree=67111528/67111528
[pid=22745] ppid=22743 vsize=8700 CPUtime=0
/proc/22745/stat : 22745 (run) S 22743 22745 22164 0 -1 4202496 366 0 0 0 0 0 0 0 25 0 1 0 146858780 8908800 249 4089446400 4194304 4922060 140733380883456 18446744073709551615 232335711365 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/22745/statm: 2175 249 208 178 0 68 0
[pid=22746] ppid=22745 vsize=146420 CPUtime=1333.32
/proc/22746/stat : 22746 (pb_cplex) S 22745 22745 22164 0 -1 4202496 153743 0 0 0 132176 1156 0 0 15 0 8 0 146858780 149934080 14474 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 6 0 0 0
/proc/22746/statm: 36605 14474 1089 2985 0 33147 0
[pid=22746/tid=22819] ppid=22745 vsize=146420 CPUtime=152.48
/proc/22746/task/22819/stat : 22819 (pb_cplex) S 22745 22745 22164 0 -1 4202560 8493 0 0 0 15117 131 0 0 15 0 8 0 146862181 149934080 14474 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 6 0 0 0
[pid=22746/tid=22820] ppid=22745 vsize=146420 CPUtime=151.21
/proc/22746/task/22820/stat : 22820 (pb_cplex) S 22745 22745 22164 0 -1 4202560 8957 0 0 0 14988 133 0 0 15 0 8 0 146862181 149934080 14474 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 3 0 0 0
[pid=22746/tid=22821] ppid=22745 vsize=146420 CPUtime=151.81
/proc/22746/task/22821/stat : 22821 (pb_cplex) S 22745 22745 22164 0 -1 4202560 12376 0 0 0 15042 139 0 0 15 0 8 0 146862181 149934080 14474 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 2 0 0 0
[pid=22746/tid=22822] ppid=22745 vsize=146420 CPUtime=152.92
/proc/22746/task/22822/stat : 22822 (pb_cplex) R 22745 22745 22164 0 -1 4202560 9507 0 0 0 15163 129 0 0 15 0 8 0 146862181 149934080 14474 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=22746/tid=22823] ppid=22745 vsize=146420 CPUtime=152.52
/proc/22746/task/22823/stat : 22823 (pb_cplex) S 22745 22745 22164 0 -1 4202560 9113 0 0 0 15123 129 0 0 16 0 8 0 146862181 149934080 14474 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 0 0 0 0
[pid=22746/tid=22824] ppid=22745 vsize=146420 CPUtime=153.93
/proc/22746/task/22824/stat : 22824 (pb_cplex) S 22745 22745 22164 0 -1 4202560 12717 0 0 0 15269 124 0 0 15 0 8 0 146862181 149934080 14474 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 5 0 0 0
[pid=22746/tid=22825] ppid=22745 vsize=146420 CPUtime=152.32
/proc/22746/task/22825/stat : 22825 (pb_cplex) S 22745 22745 22164 0 -1 4202560 8442 0 0 0 15095 137 0 0 15 0 8 0 146862181 149934080 14474 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 1333.32
Current children cumulated vsize (KiB) 155120

[startup+462.301 s]
/proc/loadavg: 10.62 9.61 7.01 4/199 22909
/proc/meminfo: memFree=30936768/32951124 swapFree=67111528/67111528
[pid=22745] ppid=22743 vsize=8700 CPUtime=0
/proc/22745/stat : 22745 (run) S 22743 22745 22164 0 -1 4202496 366 0 0 0 0 0 0 0 25 0 1 0 146858780 8908800 249 4089446400 4194304 4922060 140733380883456 18446744073709551615 232335711365 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/22745/statm: 2175 249 208 178 0 68 0
[pid=22746] ppid=22745 vsize=146420 CPUtime=1511.15
/proc/22746/stat : 22746 (pb_cplex) R 22745 22745 22164 0 -1 4202496 159772 0 0 0 149814 1301 0 0 15 0 8 0 146858780 149934080 14475 4089446400 134512640 146736320 4289056880 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 0
/proc/22746/statm: 36605 14475 1089 2985 0 33147 0
[pid=22746/tid=22819] ppid=22745 vsize=146420 CPUtime=174.17
/proc/22746/task/22819/stat : 22819 (pb_cplex) R 22745 22745 22164 0 -1 4202560 9579 0 0 0 17269 148 0 0 15 0 8 0 146862181 149934080 14475 4089446400 134512640 146736320 4289056880 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=22746/tid=22820] ppid=22745 vsize=146420 CPUtime=172.66
/proc/22746/task/22820/stat : 22820 (pb_cplex) R 22745 22745 22164 0 -1 4202560 9596 0 0 0 17115 151 0 0 15 0 8 0 146862181 149934080 14475 4089446400 134512640 146736320 4289056880 18446744073709551615 139043577 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=22746/tid=22821] ppid=22745 vsize=146420 CPUtime=173.15
/proc/22746/task/22821/stat : 22821 (pb_cplex) R 22745 22745 22164 0 -1 4202560 13221 0 0 0 17159 156 0 0 15 0 8 0 146862181 149934080 14475 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=22746/tid=22822] ppid=22745 vsize=146420 CPUtime=174.47
/proc/22746/task/22822/stat : 22822 (pb_cplex) S 22745 22745 22164 0 -1 4202560 10214 0 0 0 17300 147 0 0 15 0 8 0 146862181 149934080 14475 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 1 0 0 0
[pid=22746/tid=22823] ppid=22745 vsize=146420 CPUtime=174.04
/proc/22746/task/22823/stat : 22823 (pb_cplex) S 22745 22745 22164 0 -1 4202560 10148 0 0 0 17258 146 0 0 15 0 8 0 146862181 149934080 14475 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 1 0 0 0
[pid=22746/tid=22824] ppid=22745 vsize=146420 CPUtime=175.32
/proc/22746/task/22824/stat : 22824 (pb_cplex) S 22745 22745 22164 0 -1 4202560 13077 0 0 0 17389 143 0 0 15 0 8 0 146862181 149934080 14475 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 6 0 0 0
[pid=22746/tid=22825] ppid=22745 vsize=146420 CPUtime=173.78
/proc/22746/task/22825/stat : 22825 (pb_cplex) S 22745 22745 22164 0 -1 4202560 9230 0 0 0 17220 158 0 0 15 0 8 0 146862181 149934080 14475 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 1511.15
Current children cumulated vsize (KiB) 155120

[startup+522.307 s]
/proc/loadavg: 10.94 9.84 7.25 5/199 22925
/proc/meminfo: memFree=30943832/32951124 swapFree=67111528/67111528
[pid=22745] ppid=22743 vsize=8700 CPUtime=0
/proc/22745/stat : 22745 (run) S 22743 22745 22164 0 -1 4202496 366 0 0 0 0 0 0 0 25 0 1 0 146858780 8908800 249 4089446400 4194304 4922060 140733380883456 18446744073709551615 232335711365 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/22745/statm: 2175 249 208 178 0 68 0
[pid=22746] ppid=22745 vsize=137912 CPUtime=1690
/proc/22746/stat : 22746 (pb_cplex) S 22745 22745 22164 0 -1 4202496 180360 0 0 0 167550 1450 0 0 21 0 8 0 146858780 141221888 12517 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 6 0 0 0
/proc/22746/statm: 34478 12517 1089 2985 0 31020 0
[pid=22746/tid=22918] ppid=22745 vsize=137912 CPUtime=0.27
/proc/22746/task/22918/stat : 22918 (pb_cplex) S 22745 22745 22164 0 -1 4202560 1148 0 0 0 27 0 0 0 18 0 8 0 146910408 141221888 12517 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 6 0 0 0
[pid=22746/tid=22919] ppid=22745 vsize=137912 CPUtime=1.07
/proc/22746/task/22919/stat : 22919 (pb_cplex) S 22745 22745 22164 0 -1 4202560 35 0 0 0 107 0 0 0 18 0 8 0 146910408 141221888 12517 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 6 0 0 0
[pid=22746/tid=22920] ppid=22745 vsize=137912 CPUtime=2.64
/proc/22746/task/22920/stat : 22920 (pb_cplex) R 22745 22745 22164 0 -1 4202560 1021 0 0 0 264 0 0 0 22 0 8 0 146910408 141221888 12517 4089446400 134512640 146736320 4289056880 18446744073709551615 139044581 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=22746/tid=22921] ppid=22745 vsize=137912 CPUtime=0.55
/proc/22746/task/22921/stat : 22921 (pb_cplex) S 22745 22745 22164 0 -1 4202560 1 0 0 0 55 0 0 0 18 0 8 0 146910408 141221888 12517 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 0 0 0 0
[pid=22746/tid=22922] ppid=22745 vsize=137912 CPUtime=0.02
/proc/22746/task/22922/stat : 22922 (pb_cplex) S 22745 22745 22164 0 -1 4202560 2 0 0 0 2 0 0 0 17 0 8 0 146910408 141221888 12517 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 2 0 0 0
[pid=22746/tid=22923] ppid=22745 vsize=137912 CPUtime=0.01
/proc/22746/task/22923/stat : 22923 (pb_cplex) S 22745 22745 22164 0 -1 4202560 2 0 0 0 1 0 0 0 17 0 8 0 146910408 141221888 12517 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 6 0 0 0
[pid=22746/tid=22924] ppid=22745 vsize=137912 CPUtime=0.88
/proc/22746/task/22924/stat : 22924 (pb_cplex) S 22745 22745 22164 0 -1 4202560 35 0 0 0 88 0 0 0 18 0 8 0 146910408 141221888 12517 4089446400 134512640 146736320 4289056880 18446744073709551615 144580073 0 2 4096 16387 18446604471136041024 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 1690
Current children cumulated vsize (KiB) 146612



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+554.501 s]
/proc/loadavg: 10.14 9.73 7.29 18/199 22926
/proc/meminfo: memFree=30933416/32951124 swapFree=67111528/67111528
[pid=22745] ppid=22743 vsize=8700 CPUtime=0
/proc/22745/stat : 22745 (run) S 22743 22745 22164 0 -1 4202496 366 0 0 0 0 0 0 0 25 0 1 0 146858780 8908800 249 4089446400 4194304 4922060 140733380883456 18446744073709551615 232335711365 0 65536 4100 65538 18446744071562232103 0 0 17 4 0 0 0
/proc/22745/statm: 2175 249 208 178 0 68 0
[pid=22746] ppid=22745 vsize=145080 CPUtime=1800.22
/proc/22746/stat : 22746 (pb_cplex) R 22745 22745 22164 0 -1 4202496 234674 0 0 0 178482 1540 0 0 15 0 8 0 146858780 148561920 14229 4089446400 134512640 146736320 4289056880 18446744073709551615 136017763 0 0 4096 16387 18446744073709551615 0 0 17 2 0 0 0
/proc/22746/statm: 36270 14229 1089 2985 0 32812 0
[pid=22746/tid=22918] ppid=22745 vsize=145080 CPUtime=13.69
/proc/22746/task/22918/stat : 22918 (pb_cplex) R 22745 22745 22164 0 -1 4202560 11281 0 0 0 1358 11 0 0 15 0 8 0 146910408 148561920 14229 4089446400 134512640 146736320 4289056880 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=22746/tid=22919] ppid=22745 vsize=145080 CPUtime=14.39
/proc/22746/task/22919/stat : 22919 (pb_cplex) R 22745 22745 22164 0 -1 4202560 4146 0 0 0 1429 10 0 0 15 0 8 0 146910408 148561920 14229 4089446400 134512640 146736320 4289056880 18446744073709551615 138433215 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=22746/tid=22920] ppid=22745 vsize=145080 CPUtime=16.39
/proc/22746/task/22920/stat : 22920 (pb_cplex) R 22745 22745 22164 0 -1 4202560 8779 0 0 0 1630 9 0 0 15 0 8 0 146910408 148561920 14229 4089446400 134512640 146736320 4289056880 18446744073709551615 136403112 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=22746/tid=22921] ppid=22745 vsize=145080 CPUtime=13.77
/proc/22746/task/22921/stat : 22921 (pb_cplex) R 22745 22745 22164 0 -1 4202560 1591 0 0 0 1367 10 0 0 15 0 8 0 146910408 148561920 14229 4089446400 134512640 146736320 4289056880 18446744073709551615 137334220 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=22746/tid=22922] ppid=22745 vsize=145080 CPUtime=13.29
/proc/22746/task/22922/stat : 22922 (pb_cplex) R 22745 22745 22164 0 -1 4202560 5527 0 0 0 1317 12 0 0 15 0 8 0 146910408 148561920 14229 4089446400 134512640 146736320 4289056880 18446744073709551615 136403109 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=22746/tid=22923] ppid=22745 vsize=145080 CPUtime=13.3
/proc/22746/task/22923/stat : 22923 (pb_cplex) R 22745 22745 22164 0 -1 4202560 4083 0 0 0 1320 10 0 0 15 0 8 0 146910408 148561920 14229 4089446400 134512640 146736320 4289056880 18446744073709551615 136475822 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=22746/tid=22924] ppid=22745 vsize=145080 CPUtime=14.15
/proc/22746/task/22924/stat : 22924 (pb_cplex) R 22745 22745 22164 0 -1 4202560 8174 0 0 0 1404 11 0 0 15 0 8 0 146910408 148561920 14229 4089446400 134512640 146736320 4289056880 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1800.22
Current children cumulated vsize (KiB) 153780

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+554.617 s]
/proc/loadavg: 10.14 9.73 7.29 11/192 22927
/proc/meminfo: memFree=30985552/32951124 swapFree=67111528/67111528
[pid=22745] ppid=22743 vsize=0 CPUtime=1800.26
/proc/22745/stat : 22745 (run) R 22743 22745 22164 0 -1 4203524 366 234677 0 0 0 0 178486 1540 18 0 1 0 146858780 0 0 4089446400 0 0 0 0 0 0 65536 4100 65538 18446744073709551615 0 0 17 4 0 0 0
/proc/22745/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 1800.26
Current children cumulated vsize (KiB) 0

[startup+554.707 s]
/proc/loadavg: 10.14 9.73 7.29 11/192 22927
/proc/meminfo: memFree=30985552/32951124 swapFree=67111528/67111528
[pid=22745] ppid=22743 vsize=0 CPUtime=1800.26
/proc/22745/stat : 22745 (run) R 22743 22745 22164 0 -1 4203524 366 234677 0 0 0 0 178486 1540 18 0 1 0 146858780 0 0 4089446400 0 0 0 0 0 0 65536 4100 65538 18446744073709551615 0 0 17 4 0 0 0
/proc/22745/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 1800.26
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 554.756
CPU time (s): 1800.27
CPU user time (s): 1784.86
CPU system time (s): 15.4047
CPU usage (%): 324.515
Max. virtual memory (cumulated for all children) (KiB): 175112

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1784.86
system time used= 15.4047
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 235043
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 314867
involuntary context switches= 33738

runsolver used 1.07584 second user time and 2.16667 second system time

The end

Launcher Data 

Begin job on node132 at 2010-07-02 09:27:06
IDJOB=2697411
IDBENCH=79279
IDSOLVER=1210
FILE ID=node132/2697411-1278055626
PBS_JOBID= 11198917
Free space on /tmp= 74172 MiB

SOLVER NAME= pb_cplex 2010-07-01
BENCH NAME= PB10/normalized-PB10/SOFT-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/tsp/normalized-t3002.11tsp11.1900573137--soft-0-100-0.wbo
COMMAND LINE= run BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-2697411-1278055626/watcher-2697411-1278055626 -o /tmp/evaluation-result-2697411-1278055626/solver-2697411-1278055626 -C 1800 -W 2000 -M 3800  run HOME/instance-2697411-1278055626.wbo

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 3800 MiB
MAX NB THREAD= 0

MD5SUM BENCH= 337bb89bbfc43fe8a5ddb591a5c00326
RANDOM SEED=2067324590

node132.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.810
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.62
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.810
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.810
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.810
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.810
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.810
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.810
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.810
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      31126988 kB
Buffers:        219428 kB
Cached:        1310660 kB
SwapCached:          0 kB
Active:         681380 kB
Inactive:       992344 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      31126988 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           25188 kB
Writeback:           0 kB
AnonPages:      144148 kB
Mapped:          18984 kB
Slab:            84392 kB
PageTables:       4720 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   367684 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 74156 MiB
End job on node132 at 2010-07-02 09:36:23