Trace number 2697919

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
pb_cplex 2010-07-01OPTIMUM 1267.12 349.985

General information on the benchmark

Name/PARTIAL-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/tsp/
normalized-t3002.11tsp11.1900573901--soft-66-100-0.wbo
MD5SUMa057c56aaeadccffd1b9b5ae1c954d95
Bench CategoryPARTIAL-SMALLINT-LIN (both soft and hard constraints, small integers, linear constraints)
Best result obtained on this benchmarkMOPT
Best cost obtained on this benchmark1
Best CPU time to get the best result obtained on this benchmark132.069
Max-Satisfiable
Max-(Un)Satisfiability was proved
Best value of the cost
Optimality of the best cost was proved
Number of variables231
Total number of constraints2707
Number of soft constraints892
Number of constraints which are clauses2684
Number of constraints which are cardinality constraints (but not clauses)22
Number of constraints which are nor clauses,nor cardinality constraints1
Minimum length of a constraint2
Maximum length of a constraint110
Top cost 45060
Min constraint cost 1
Max constraint cost 100
Sum of constraints costs 45059
Biggest number in a constraint 25
Number of bits of the biggest number in a constraint 5
Biggest sum of numbers in a constraint 701
Number of bits of the biggest sum of numbers10
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c pb_cplex 1.0 - 20100701
0.00/0.00	c Parsing wbo file format. File HOME/instance-2697919-1278054868.wbo.
0.00/0.00	c Instance file HOME/instance-2697919-1278054868.wbo
0.00/0.00	c File size is 87848 bytes.
0.00/0.17	c Highest Coefficient sum: 701
0.00/0.17	c  IBM ILOG License Manager: "IBM ILOG Optimization Suite for Academic Initiative" is accessing CPLEX 12 with option(s): "e m b q ".
0.00/0.17	c ============================[ Problem Statistics ]=============================
0.00/0.17	c |                                                                             |
0.00/0.17	c |  Parsing time:          0.17         s                                      |
0.00/0.17	c |  Number Variables:      1123                                                |
0.00/0.17	c |  Number Constraints:    2708                                                |
0.00/0.17	c ===============================================================================
1266.88/349.98	c Remaining time 1799.83 sec.
1266.88/349.98	s OPTIMUM FOUND
1266.88/349.98	c Solution value = 1.00
1266.88/349.98	v x1 -x2 -x3 -x4 -x5 -x6 -x7 -x8 -x9 -x10 -x11 -x12 -x13 -x14 -x15 -x16 -x17 x18 -x19 -x20 -x21 -x22 -x23 -x24 -x25 x26 -x27 -x28 -x29 -x30 -x31 -x32 -x33 -x34 -x35 -x36 -x37 -x38 -x39 -x40 -x41 -x42 x43 -x44 -x45 -x46 -x47 -x48 -x49 -x50 -x51 x52 -x53 -x54 -x55 -x56 -x57 x58 -x59 -x60 -x61 -x62 -x63 -x64 -x65 -x66 -x67 x68 -x69 -x70 -x71 -x72 -x73 -x74 -x75 -x76 -x77 -x78 -x79 -x80 -x81 -x82 -x83 x84 -x85 -x86 -x87 -x88 -x89 -x90 -x91 -x92 -x93 x94 -x95 -x96 -x97 -x98 -x99 -x100 -x101 -x102 -x103 x104 -x105 -x106 -x107 -x108 -x109 -x110 -x111 -x112 -x113 -x114 -x115 -x116 -x117 -x118 -x119 -x120 x121 -x122 -x123 -x124 -x125 -x126 -x127 -x128 -x129 -x130 x131 -x132 x133 -x134 -x135 -x136 -x137 -x138 -x139 -x140 -x141 -x142 -x143 -x144 -x145 -x146 -x147 x148 -x149 -x150 -x151 -x152 -x153 -x154 -x155 -x156 -x157 -x158 x159 -x160 -x161 -x162 -x163 -x164 -x165 x166 -x167 -x168 -x169 -x170 -x171 -x172 -x173 -x174 -x175 -x176 -x177 -x178 -x179 -x180 -x181 -x182 x183 -x184 -x185 -x186 -x187 -x188 -x189 -x190 -x191 -x192 -x193 -x194 -x195 -x196 -x197 -x198 -x199 x200 -x201 x202 -x203 -x204 -x205 -x206 -x207 -x208 -x209 -x210 -x211 -x212 -x213 -x214 x215 -x216 -x217 -x218 -x219 -x220 -x221 -x222 -x223 -x224 -x225 x226 -x227 -x228 -x229 -x230 -x231
1266.88/349.98	c Total CPU time              : 1255.91 s

Verifier Data 

OK	1

Watcher Data 

runsolver version 3.2.9 (svn:492) (c) roussel@cril.univ-artois.fr

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-2697919-1278054868/watcher-2697919-1278054868 -o /tmp/evaluation-result-2697919-1278054868/solver-2697919-1278054868 -C 1800 -W 2000 -M 3800 run HOME/instance-2697919-1278054868.wbo 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 3891200 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 3942400 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.32 0.07 0.02 4/190 21071
/proc/meminfo: memFree=31322840/32951124 swapFree=67111528/67111528
[pid=21071] ppid=21069 vsize=8700 CPUtime=0
/proc/21071/stat : 21071 (run) S 21069 21071 20807 0 -1 4202496 366 0 0 0 0 0 0 0 19 0 1 0 146782418 8908800 249 4089446400 4194304 4922060 140735476331360 18446744073709551615 271550356613 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21071/statm: 2175 249 208 178 0 68 0

[startup+0.047375 s]
/proc/loadavg: 0.32 0.07 0.02 4/190 21071
/proc/meminfo: memFree=31322840/32951124 swapFree=67111528/67111528
[pid=21071] ppid=21069 vsize=8700 CPUtime=0
/proc/21071/stat : 21071 (run) S 21069 21071 20807 0 -1 4202496 366 0 0 0 0 0 0 0 19 0 1 0 146782418 8908800 249 4089446400 4194304 4922060 140735476331360 18446744073709551615 271550356613 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21071/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.10036 s]
/proc/loadavg: 0.32 0.07 0.02 4/190 21071
/proc/meminfo: memFree=31322840/32951124 swapFree=67111528/67111528
[pid=21071] ppid=21069 vsize=8700 CPUtime=0
/proc/21071/stat : 21071 (run) S 21069 21071 20807 0 -1 4202496 366 0 0 0 0 0 0 0 19 0 1 0 146782418 8908800 249 4089446400 4194304 4922060 140735476331360 18446744073709551615 271550356613 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21071/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.30032 s]
/proc/loadavg: 0.32 0.07 0.02 4/190 21071
/proc/meminfo: memFree=31322840/32951124 swapFree=67111528/67111528
[pid=21071] ppid=21069 vsize=8700 CPUtime=0
/proc/21071/stat : 21071 (run) S 21069 21071 20807 0 -1 4202496 366 0 0 0 0 0 0 0 19 0 1 0 146782418 8908800 249 4089446400 4194304 4922060 140735476331360 18446744073709551615 271550356613 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21071/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.700143 s]
/proc/loadavg: 0.32 0.07 0.02 4/190 21071
/proc/meminfo: memFree=31322840/32951124 swapFree=67111528/67111528
[pid=21071] ppid=21069 vsize=8700 CPUtime=0
/proc/21071/stat : 21071 (run) S 21069 21071 20807 0 -1 4202496 366 0 0 0 0 0 0 0 19 0 1 0 146782418 8908800 249 4089446400 4194304 4922060 140735476331360 18446744073709551615 271550356613 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21071/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+1.501 s]
/proc/loadavg: 0.32 0.07 0.02 13/199 21094
/proc/meminfo: memFree=31275772/32951124 swapFree=67111528/67111528
[pid=21071] ppid=21069 vsize=8700 CPUtime=0
/proc/21071/stat : 21071 (run) S 21069 21071 20807 0 -1 4202496 366 0 0 0 0 0 0 0 19 0 1 0 146782418 8908800 249 4089446400 4194304 4922060 140735476331360 18446744073709551615 271550356613 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21071/statm: 2175 249 208 178 0 68 0
[pid=21072] ppid=21071 vsize=88572 CPUtime=2.92
/proc/21072/stat : 21072 (pb_cplex) R 21071 21071 20807 0 -1 4202496 18643 0 0 0 285 7 0 0 16 0 1 0 146782418 90697728 8008 4089446400 134512640 146736320 4291139040 18446744073709551615 137702864 0 0 4096 16387 0 0 0 17 2 0 0 0
/proc/21072/statm: 22143 8008 1067 2985 0 18685 0
Current children cumulated CPU time (s) 2.92
Current children cumulated vsize (KiB) 97272

[startup+3.10171 s]
/proc/loadavg: 0.32 0.07 0.02 3/199 21115
/proc/meminfo: memFree=31278996/32951124 swapFree=67111528/67111528
[pid=21071] ppid=21069 vsize=8700 CPUtime=0
/proc/21071/stat : 21071 (run) S 21069 21071 20807 0 -1 4202496 366 0 0 0 0 0 0 0 19 0 1 0 146782418 8908800 249 4089446400 4194304 4922060 140735476331360 18446744073709551615 271550356613 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21071/statm: 2175 249 208 178 0 68 0
[pid=21072] ppid=21071 vsize=123388 CPUtime=4.13
/proc/21072/stat : 21072 (pb_cplex) R 21071 21071 20807 0 -1 4202496 23817 0 0 0 404 9 0 0 18 0 8 0 146782418 126349312 8932 4089446400 134512640 146736320 4291139040 18446744073709551615 136385174 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 0
/proc/21072/statm: 30847 8932 1078 2985 0 27389 0
[pid=21072/tid=21109] ppid=21071 vsize=123388 CPUtime=0
/proc/21072/task/21109/stat : 21109 (pb_cplex) S 21071 21071 20807 0 -1 4202560 0 0 0 0 0 0 0 0 20 0 8 0 146782605 126349312 8932 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 2 4096 16387 18446604471082982400 0 0 -1 5 0 0 0
[pid=21072/tid=21110] ppid=21071 vsize=123388 CPUtime=0
/proc/21072/task/21110/stat : 21110 (pb_cplex) S 21071 21071 20807 0 -1 4202560 0 0 0 0 0 0 0 0 18 0 8 0 146782605 126349312 8932 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 2 4096 16387 18446604471082982400 0 0 -1 5 0 0 0
[pid=21072/tid=21111] ppid=21071 vsize=123388 CPUtime=0
/proc/21072/task/21111/stat : 21111 (pb_cplex) S 21071 21071 20807 0 -1 4202560 0 0 0 0 0 0 0 0 20 0 8 0 146782605 126349312 8932 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 2 4096 16387 18446604471082982400 0 0 -1 5 0 0 0
[pid=21072/tid=21112] ppid=21071 vsize=123388 CPUtime=0
/proc/21072/task/21112/stat : 21112 (pb_cplex) S 21071 21071 20807 0 -1 4202560 0 0 0 0 0 0 0 0 20 0 8 0 146782605 126349312 8932 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 2 4096 16387 18446604471082982400 0 0 -1 5 0 0 0
[pid=21072/tid=21113] ppid=21071 vsize=123388 CPUtime=0
/proc/21072/task/21113/stat : 21113 (pb_cplex) S 21071 21071 20807 0 -1 4202560 1 0 0 0 0 0 0 0 18 0 8 0 146782605 126349312 8932 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 2 4096 16387 18446604471082982400 0 0 -1 5 0 0 0
[pid=21072/tid=21114] ppid=21071 vsize=123388 CPUtime=0
/proc/21072/task/21114/stat : 21114 (pb_cplex) S 21071 21071 20807 0 -1 4202560 1 0 0 0 0 0 0 0 21 0 8 0 146782605 126349312 8932 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 2 4096 16387 18446604471082982400 0 0 -1 5 0 0 0
[pid=21072/tid=21115] ppid=21071 vsize=123388 CPUtime=0
/proc/21072/task/21115/stat : 21115 (pb_cplex) S 21071 21071 20807 0 -1 4202560 1 0 0 0 0 0 0 0 18 0 8 0 146782605 126349312 8932 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 2 4096 16387 18446604471082982400 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4.13
Current children cumulated vsize (KiB) 132088

[startup+6.35289 s]
/proc/loadavg: 1.18 0.25 0.08 10/199 21115
/proc/meminfo: memFree=31267100/32951124 swapFree=67111528/67111528
[pid=21071] ppid=21069 vsize=8700 CPUtime=0
/proc/21071/stat : 21071 (run) S 21069 21071 20807 0 -1 4202496 366 0 0 0 0 0 0 0 19 0 1 0 146782418 8908800 249 4089446400 4194304 4922060 140735476331360 18446744073709551615 271550356613 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21071/statm: 2175 249 208 178 0 68 0
[pid=21072] ppid=21071 vsize=133928 CPUtime=11.45
/proc/21072/stat : 21072 (pb_cplex) R 21071 21071 20807 0 -1 4202496 34500 0 0 0 1131 14 0 0 15 0 8 0 146782418 137142272 11394 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/21072/statm: 33482 11394 1083 2985 0 30024 0
[pid=21072/tid=21109] ppid=21071 vsize=133928 CPUtime=0.86
/proc/21072/task/21109/stat : 21109 (pb_cplex) R 21071 21071 20807 0 -1 4202560 346 0 0 0 86 0 0 0 15 0 8 0 146782605 137142272 11394 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=21072/tid=21110] ppid=21071 vsize=133928 CPUtime=0.59
/proc/21072/task/21110/stat : 21110 (pb_cplex) R 21071 21071 20807 0 -1 4202560 614 0 0 0 59 0 0 0 15 0 8 0 146782605 137142272 11394 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=21072/tid=21111] ppid=21071 vsize=133928 CPUtime=0.53
/proc/21072/task/21111/stat : 21111 (pb_cplex) R 21071 21071 20807 0 -1 4202560 145 0 0 0 53 0 0 0 15 0 8 0 146782605 137142272 11394 4089446400 134512640 146736320 4291139040 18446744073709551615 135237157 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=21072/tid=21112] ppid=21071 vsize=133928 CPUtime=0.69
/proc/21072/task/21112/stat : 21112 (pb_cplex) R 21071 21071 20807 0 -1 4202560 1573 0 0 0 69 0 0 0 15 0 8 0 146782605 137142272 11394 4089446400 134512640 146736320 4291139040 18446744073709551615 136384857 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=21072/tid=21113] ppid=21071 vsize=133928 CPUtime=0.96
/proc/21072/task/21113/stat : 21113 (pb_cplex) R 21071 21071 20807 0 -1 4202560 915 0 0 0 96 0 0 0 15 0 8 0 146782605 137142272 11394 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=21072/tid=21114] ppid=21071 vsize=133928 CPUtime=1.06
/proc/21072/task/21114/stat : 21114 (pb_cplex) S 21071 21071 20807 0 -1 4202560 63 0 0 0 106 0 0 0 15 0 8 0 146782605 137142272 11394 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 2 4096 16387 18446604471082982400 0 0 -1 2 0 0 0
[pid=21072/tid=21115] ppid=21071 vsize=133928 CPUtime=0.83
/proc/21072/task/21115/stat : 21115 (pb_cplex) S 21071 21071 20807 0 -1 4202560 1219 0 0 0 83 0 0 0 16 0 8 0 146782605 137142272 11394 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 2 4096 16387 18446604471082982400 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 11.45
Current children cumulated vsize (KiB) 142628

[startup+12.703 s]
/proc/loadavg: 2.12 0.46 0.15 9/199 21115
/proc/meminfo: memFree=31257052/32951124 swapFree=67111528/67111528
[pid=21071] ppid=21069 vsize=8700 CPUtime=0
/proc/21071/stat : 21071 (run) S 21069 21071 20807 0 -1 4202496 366 0 0 0 0 0 0 0 19 0 1 0 146782418 8908800 249 4089446400 4194304 4922060 140735476331360 18446744073709551615 271550356613 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21071/statm: 2175 249 208 178 0 68 0
[pid=21072] ppid=21071 vsize=139048 CPUtime=38.43
/proc/21072/stat : 21072 (pb_cplex) R 21071 21071 20807 0 -1 4202496 44940 0 0 0 3806 37 0 0 16 0 8 0 146782418 142385152 12524 4089446400 134512640 146736320 4291139040 18446744073709551615 144580332 0 0 4096 16387 18446744073709551615 0 0 17 6 0 0 0
/proc/21072/statm: 34762 12524 1080 2985 0 31304 0
[pid=21072/tid=21109] ppid=21071 vsize=139048 CPUtime=4.2
/proc/21072/task/21109/stat : 21109 (pb_cplex) R 21071 21071 20807 0 -1 4202560 545 0 0 0 418 2 0 0 15 0 8 0 146782605 142385152 12524 4089446400 134512640 146736320 4291139040 18446744073709551615 136388075 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=21072/tid=21110] ppid=21071 vsize=139048 CPUtime=3.93
/proc/21072/task/21110/stat : 21110 (pb_cplex) R 21071 21071 20807 0 -1 4202560 635 0 0 0 390 3 0 0 15 0 8 0 146782605 142385152 12524 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=21072/tid=21111] ppid=21071 vsize=139048 CPUtime=3.9
/proc/21072/task/21111/stat : 21111 (pb_cplex) R 21071 21071 20807 0 -1 4202560 569 0 0 0 387 3 0 0 15 0 8 0 146782605 142385152 12524 4089446400 134512640 146736320 4291139040 18446744073709551615 136000534 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=21072/tid=21112] ppid=21071 vsize=139048 CPUtime=4.09
/proc/21072/task/21112/stat : 21112 (pb_cplex) R 21071 21071 20807 0 -1 4202560 4532 0 0 0 406 3 0 0 15 0 8 0 146782605 142385152 12524 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=21072/tid=21113] ppid=21071 vsize=139048 CPUtime=4.32
/proc/21072/task/21113/stat : 21113 (pb_cplex) R 21071 21071 20807 0 -1 4202560 4128 0 0 0 429 3 0 0 15 0 8 0 146782605 142385152 12524 4089446400 134512640 146736320 4291139040 18446744073709551615 135450684 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=21072/tid=21114] ppid=21071 vsize=139048 CPUtime=4.39
/proc/21072/task/21114/stat : 21114 (pb_cplex) R 21071 21071 20807 0 -1 4202560 567 0 0 0 436 3 0 0 15 0 8 0 146782605 142385152 12524 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=21072/tid=21115] ppid=21071 vsize=139048 CPUtime=4.26
/proc/21072/task/21115/stat : 21115 (pb_cplex) S 21071 21071 20807 0 -1 4202560 1479 0 0 0 423 3 0 0 15 0 8 0 146782605 142385152 12524 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 2 4096 16387 18446604471082982400 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38.43
Current children cumulated vsize (KiB) 147748

[startup+25.5006 s]
/proc/loadavg: 4.86 1.14 0.38 14/199 21116
/proc/meminfo: memFree=31224684/32951124 swapFree=67111528/67111528
[pid=21071] ppid=21069 vsize=8700 CPUtime=0
/proc/21071/stat : 21071 (run) S 21069 21071 20807 0 -1 4202496 366 0 0 0 0 0 0 0 19 0 1 0 146782418 8908800 249 4089446400 4194304 4922060 140735476331360 18446744073709551615 271550356613 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21071/statm: 2175 249 208 178 0 68 0
[pid=21072] ppid=21071 vsize=156444 CPUtime=95.97
/proc/21072/stat : 21072 (pb_cplex) R 21071 21071 20807 0 -1 4202496 65939 0 0 0 9510 87 0 0 17 0 8 0 146782418 160198656 16376 4089446400 134512640 146736320 4291139040 18446744073709551615 136475808 0 0 4096 16387 18446744073709551615 0 0 17 6 0 0 0
/proc/21072/statm: 39111 16376 1088 2985 0 35653 0
[pid=21072/tid=21109] ppid=21071 vsize=156444 CPUtime=11.91
/proc/21072/task/21109/stat : 21109 (pb_cplex) R 21071 21071 20807 0 -1 4202560 4775 0 0 0 1182 9 0 0 19 0 8 0 146782605 160198656 16376 4089446400 134512640 146736320 4291139040 18446744073709551615 144785853 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=21072/tid=21110] ppid=21071 vsize=156444 CPUtime=11.19
/proc/21072/task/21110/stat : 21110 (pb_cplex) R 21071 21071 20807 0 -1 4202560 2365 0 0 0 1110 9 0 0 19 0 8 0 146782605 160198656 16376 4089446400 134512640 146736320 4291139040 18446744073709551615 136000488 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=21072/tid=21111] ppid=21071 vsize=156444 CPUtime=11.57
/proc/21072/task/21111/stat : 21111 (pb_cplex) R 21071 21071 20807 0 -1 4202560 2515 0 0 0 1150 7 0 0 19 0 8 0 146782605 160198656 16376 4089446400 134512640 146736320 4291139040 18446744073709551615 136008050 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=21072/tid=21112] ppid=21071 vsize=156444 CPUtime=11.48
/proc/21072/task/21112/stat : 21112 (pb_cplex) R 21071 21071 20807 0 -1 4202560 6590 0 0 0 1138 10 0 0 19 0 8 0 146782605 160198656 16376 4089446400 134512640 146736320 4291139040 18446744073709551615 136403109 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=21072/tid=21113] ppid=21071 vsize=156444 CPUtime=11.74
/proc/21072/task/21113/stat : 21113 (pb_cplex) R 21071 21071 20807 0 -1 4202560 8520 0 0 0 1165 9 0 0 19 0 8 0 146782605 160198656 16376 4089446400 134512640 146736320 4291139040 18446744073709551615 135450320 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=21072/tid=21114] ppid=21071 vsize=156444 CPUtime=12.04
/proc/21072/task/21114/stat : 21114 (pb_cplex) R 21071 21071 20807 0 -1 4202560 2615 0 0 0 1195 9 0 0 19 0 8 0 146782605 160198656 16376 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=21072/tid=21115] ppid=21071 vsize=156444 CPUtime=11.11
/proc/21072/task/21115/stat : 21115 (pb_cplex) R 21071 21071 20807 0 -1 4202560 2850 0 0 0 1102 9 0 0 17 0 8 0 146782605 160198656 16376 4089446400 134512640 146736320 4291139040 18446744073709551615 136008299 0 2 4096 16387 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 95.97
Current children cumulated vsize (KiB) 165144

[startup+51.1228 s]
/proc/loadavg: 6.54 1.82 0.62 9/199 21151
/proc/meminfo: memFree=31258040/32951124 swapFree=67111528/67111528
[pid=21071] ppid=21069 vsize=8700 CPUtime=0
/proc/21071/stat : 21071 (run) S 21069 21071 20807 0 -1 4202496 366 0 0 0 0 0 0 0 19 0 1 0 146782418 8908800 249 4089446400 4194304 4922060 140735476331360 18446744073709551615 271550356613 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21071/statm: 2175 249 208 178 0 68 0
[pid=21072] ppid=21071 vsize=140396 CPUtime=185.92
/proc/21072/stat : 21072 (pb_cplex) R 21071 21071 20807 0 -1 4202496 102196 0 0 0 18440 152 0 0 16 0 8 0 146782418 143765504 12816 4089446400 134512640 146736320 4291139040 18446744073709551615 144580332 0 0 4096 16387 18446744073709551615 0 0 17 4 0 0 0
/proc/21072/statm: 35099 12816 1089 2985 0 31641 0
[pid=21072/tid=21124] ppid=21071 vsize=140396 CPUtime=8.77
/proc/21072/task/21124/stat : 21124 (pb_cplex) R 21071 21071 20807 0 -1 4202560 3344 0 0 0 872 5 0 0 15 0 8 0 146785342 143765504 12816 4089446400 134512640 146736320 4291139040 18446744073709551615 136017665 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=21072/tid=21125] ppid=21071 vsize=140396 CPUtime=9.32
/proc/21072/task/21125/stat : 21125 (pb_cplex) R 21071 21071 20807 0 -1 4202560 1599 0 0 0 925 7 0 0 15 0 8 0 146785342 143765504 12816 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=21072/tid=21126] ppid=21071 vsize=140396 CPUtime=9.1
/proc/21072/task/21126/stat : 21126 (pb_cplex) R 21071 21071 20807 0 -1 4202560 5815 0 0 0 903 7 0 0 15 0 8 0 146785342 143765504 12816 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=21072/tid=21127] ppid=21071 vsize=140396 CPUtime=11.31
/proc/21072/task/21127/stat : 21127 (pb_cplex) R 21071 21071 20807 0 -1 4202560 4677 0 0 0 1124 7 0 0 15 0 8 0 146785342 143765504 12816 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=21072/tid=21128] ppid=21071 vsize=140396 CPUtime=8.88
/proc/21072/task/21128/stat : 21128 (pb_cplex) S 21071 21071 20807 0 -1 4202560 2293 0 0 0 883 5 0 0 15 0 8 0 146785342 143765504 12816 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 2 4096 16387 18446604471082982400 0 0 -1 4 0 0 0
[pid=21072/tid=21129] ppid=21071 vsize=140396 CPUtime=9.54
/proc/21072/task/21129/stat : 21129 (pb_cplex) S 21071 21071 20807 0 -1 4202560 4767 0 0 0 946 8 0 0 15 0 8 0 146785342 143765504 12816 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 2 4096 16387 18446604471082982400 0 0 -1 4 0 0 0
[pid=21072/tid=21130] ppid=21071 vsize=140396 CPUtime=9.94
/proc/21072/task/21130/stat : 21130 (pb_cplex) S 21071 21071 20807 0 -1 4202560 1909 0 0 0 988 6 0 0 15 0 8 0 146785342 143765504 12816 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 2 4096 16387 18446604471082982400 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 185.92
Current children cumulated vsize (KiB) 149096

[startup+102.307 s]
/proc/loadavg: 8.09 2.98 1.08 12/199 21153
/proc/meminfo: memFree=31254320/32951124 swapFree=67111528/67111528

################
# More data... #
################

[pid=21072/tid=21126] ppid=21071 vsize=141420 CPUtime=108.79
/proc/21072/task/21126/stat : 21126 (pb_cplex) R 21071 21071 20807 0 -1 4202560 13473 0 0 0 10779 100 0 0 15 0 8 0 146785342 144814080 13149 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=21072/tid=21127] ppid=21071 vsize=141420 CPUtime=112.14
/proc/21072/task/21127/stat : 21127 (pb_cplex) R 21071 21071 20807 0 -1 4202560 14823 0 0 0 11120 94 0 0 15 0 8 0 146785342 144814080 13149 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=21072/tid=21128] ppid=21071 vsize=141420 CPUtime=108.97
/proc/21072/task/21128/stat : 21128 (pb_cplex) R 21071 21071 20807 0 -1 4202560 11071 0 0 0 10802 95 0 0 15 0 8 0 146785342 144814080 13149 4089446400 134512640 146736320 4291139040 18446744073709551615 139023093 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=21072/tid=21129] ppid=21071 vsize=141420 CPUtime=109.63
/proc/21072/task/21129/stat : 21129 (pb_cplex) R 21071 21071 20807 0 -1 4202560 14472 0 0 0 10867 96 0 0 15 0 8 0 146785342 144814080 13149 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=21072/tid=21130] ppid=21071 vsize=141420 CPUtime=110.55
/proc/21072/task/21130/stat : 21130 (pb_cplex) R 21071 21071 20807 0 -1 4202560 16804 0 0 0 10948 107 0 0 15 0 8 0 146785342 144814080 13149 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 997.79
Current children cumulated vsize (KiB) 150120

[startup+342.301 s]
/proc/loadavg: 9.64 6.52 2.95 9/199 21159
/proc/meminfo: memFree=31253156/32951124 swapFree=67111528/67111528
[pid=21071] ppid=21069 vsize=8700 CPUtime=0
/proc/21071/stat : 21071 (run) S 21069 21071 20807 0 -1 4202496 366 0 0 0 0 0 0 0 19 0 1 0 146782418 8908800 249 4089446400 4194304 4922060 140735476331360 18446744073709551615 271550356613 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21071/statm: 2175 249 208 178 0 68 0
[pid=21072] ppid=21071 vsize=141420 CPUtime=1229.22
/proc/21072/stat : 21072 (pb_cplex) S 21071 21071 20807 0 -1 4202496 206901 0 0 0 121836 1086 0 0 16 0 8 0 146782418 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 0
/proc/21072/statm: 35355 13215 1107 2985 0 31897 0
[pid=21072/tid=21124] ppid=21071 vsize=141420 CPUtime=137.76
/proc/21072/task/21124/stat : 21124 (pb_cplex) S 21071 21071 20807 0 -1 4202560 13715 0 0 0 13653 123 0 0 15 0 8 0 146785342 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 2 4096 16387 18446604471082982400 0 0 -1 3 0 0 0
[pid=21072/tid=21125] ppid=21071 vsize=141420 CPUtime=137.57
/proc/21072/task/21125/stat : 21125 (pb_cplex) S 21071 21071 20807 0 -1 4202560 19038 0 0 0 13632 125 0 0 15 0 8 0 146785342 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 2 4096 16387 18446604471082982400 0 0 -1 2 0 0 0
[pid=21072/tid=21126] ppid=21071 vsize=141420 CPUtime=137.43
/proc/21072/task/21126/stat : 21126 (pb_cplex) S 21071 21071 20807 0 -1 4202560 16815 0 0 0 13619 124 0 0 16 0 8 0 146785342 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 2 4096 16387 18446604471082982400 0 0 -1 6 0 0 0
[pid=21072/tid=21127] ppid=21071 vsize=141420 CPUtime=140.64
/proc/21072/task/21127/stat : 21127 (pb_cplex) S 21071 21071 20807 0 -1 4202560 16843 0 0 0 13945 119 0 0 15 0 8 0 146785342 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 2 4096 16387 18446604471082982400 0 0 -1 1 0 0 0
[pid=21072/tid=21128] ppid=21071 vsize=141420 CPUtime=137.71
/proc/21072/task/21128/stat : 21128 (pb_cplex) S 21071 21071 20807 0 -1 4202560 13030 0 0 0 13648 123 0 0 15 0 8 0 146785342 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 2 4096 16387 18446604471082982400 0 0 -1 6 0 0 0
[pid=21072/tid=21129] ppid=21071 vsize=141420 CPUtime=138.09
/proc/21072/task/21129/stat : 21129 (pb_cplex) R 21071 21071 20807 0 -1 4202560 14773 0 0 0 13689 120 0 0 15 0 8 0 146785342 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=21072/tid=21130] ppid=21071 vsize=141420 CPUtime=139.35
/proc/21072/task/21130/stat : 21130 (pb_cplex) R 21071 21071 20807 0 -1 4202560 20882 0 0 0 13803 132 0 0 15 0 8 0 146785342 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1229.22
Current children cumulated vsize (KiB) 150120

Solver just ended. Dumping a history of the last processes samples

[startup+345.531 s]
/proc/loadavg: 9.51 6.55 2.98 9/199 21159
/proc/meminfo: memFree=31255452/32951124 swapFree=67111528/67111528
[pid=21071] ppid=21069 vsize=8700 CPUtime=0
/proc/21071/stat : 21071 (run) S 21069 21071 20807 0 -1 4202496 366 0 0 0 0 0 0 0 19 0 1 0 146782418 8908800 249 4089446400 4194304 4922060 140735476331360 18446744073709551615 271550356613 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21071/statm: 2175 249 208 178 0 68 0
[pid=21072] ppid=21071 vsize=141420 CPUtime=1241.85
/proc/21072/stat : 21072 (pb_cplex) R 21071 21071 20807 0 -1 4202496 207376 0 0 0 123090 1095 0 0 15 0 8 0 146782418 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0
/proc/21072/statm: 35355 13215 1107 2985 0 31897 0
[pid=21072/tid=21124] ppid=21071 vsize=141420 CPUtime=139.31
/proc/21072/task/21124/stat : 21124 (pb_cplex) R 21071 21071 20807 0 -1 4202560 13734 0 0 0 13807 124 0 0 15 0 8 0 146785342 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 135523293 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=21072/tid=21125] ppid=21071 vsize=141420 CPUtime=139.12
/proc/21072/task/21125/stat : 21125 (pb_cplex) R 21071 21071 20807 0 -1 4202560 19272 0 0 0 13785 127 0 0 15 0 8 0 146785342 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0
[pid=21072/tid=21126] ppid=21071 vsize=141420 CPUtime=139.08
/proc/21072/task/21126/stat : 21126 (pb_cplex) R 21071 21071 20807 0 -1 4202560 16954 0 0 0 13783 125 0 0 16 0 8 0 146785342 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=21072/tid=21127] ppid=21071 vsize=141420 CPUtime=142.25
/proc/21072/task/21127/stat : 21127 (pb_cplex) S 21071 21071 20807 0 -1 4202560 16860 0 0 0 14105 120 0 0 15 0 8 0 146785342 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 2 4096 16387 18446604471082982400 0 0 -1 7 0 0 0
[pid=21072/tid=21128] ppid=21071 vsize=141420 CPUtime=139.28
/proc/21072/task/21128/stat : 21128 (pb_cplex) S 21071 21071 20807 0 -1 4202560 13056 0 0 0 13804 124 0 0 15 0 8 0 146785342 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 2 4096 16387 18446604471082982400 0 0 -1 3 0 0 0
[pid=21072/tid=21129] ppid=21071 vsize=141420 CPUtime=139.63
/proc/21072/task/21129/stat : 21129 (pb_cplex) R 21071 21071 20807 0 -1 4202560 14809 0 0 0 13842 121 0 0 16 0 8 0 146785342 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=21072/tid=21130] ppid=21071 vsize=141420 CPUtime=140.86
/proc/21072/task/21130/stat : 21130 (pb_cplex) R 21071 21071 20807 0 -1 4202560 20882 0 0 0 13953 133 0 0 15 0 8 0 146785342 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 1241.85
Current children cumulated vsize (KiB) 150120

[startup+347.101 s]
/proc/loadavg: 9.51 6.55 2.98 6/186 21181
/proc/meminfo: memFree=31314944/32951124 swapFree=67111528/67111528
[pid=21071] ppid=21069 vsize=8700 CPUtime=0
/proc/21071/stat : 21071 (run) S 21069 21071 20807 0 -1 4202496 366 0 0 0 0 0 0 0 19 0 1 0 146782418 8908800 249 4089446400 4194304 4922060 140735476331360 18446744073709551615 271550356613 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21071/statm: 2175 249 208 178 0 68 0
[pid=21072] ppid=21071 vsize=141420 CPUtime=1251.86
/proc/21072/stat : 21072 (pb_cplex) R 21071 21071 20807 0 -1 4202496 208263 0 0 0 124081 1105 0 0 15 0 8 0 146782418 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 136017763 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0
/proc/21072/statm: 35355 13215 1107 2985 0 31897 0
[pid=21072/tid=21124] ppid=21071 vsize=141420 CPUtime=140.59
/proc/21072/task/21124/stat : 21124 (pb_cplex) R 21071 21071 20807 0 -1 4202560 14098 0 0 0 13934 125 0 0 15 0 8 0 146785342 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=21072/tid=21125] ppid=21071 vsize=141420 CPUtime=140.35
/proc/21072/task/21125/stat : 21125 (pb_cplex) R 21071 21071 20807 0 -1 4202560 19281 0 0 0 13907 128 0 0 15 0 8 0 146785342 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=21072/tid=21126] ppid=21071 vsize=141420 CPUtime=140.33
/proc/21072/task/21126/stat : 21126 (pb_cplex) R 21071 21071 20807 0 -1 4202560 16956 0 0 0 13907 126 0 0 16 0 8 0 146785342 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=21072/tid=21127] ppid=21071 vsize=141420 CPUtime=143.47
/proc/21072/task/21127/stat : 21127 (pb_cplex) R 21071 21071 20807 0 -1 4202560 16860 0 0 0 14226 121 0 0 15 0 8 0 146785342 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=21072/tid=21128] ppid=21071 vsize=141420 CPUtime=140.47
/proc/21072/task/21128/stat : 21128 (pb_cplex) R 21071 21071 20807 0 -1 4202560 13564 0 0 0 13921 126 0 0 15 0 8 0 146785342 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 136017496 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=21072/tid=21129] ppid=21071 vsize=141420 CPUtime=140.92
/proc/21072/task/21129/stat : 21129 (pb_cplex) R 21071 21071 20807 0 -1 4202560 14809 0 0 0 13969 123 0 0 16 0 8 0 146785342 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 136403117 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=21072/tid=21130] ppid=21071 vsize=141420 CPUtime=142.19
/proc/21072/task/21130/stat : 21130 (pb_cplex) R 21071 21071 20807 0 -1 4202560 20883 0 0 0 14084 135 0 0 16 0 8 0 146785342 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 135534640 0 2 4096 16387 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 1251.86
Current children cumulated vsize (KiB) 150120

[startup+348.707 s]
/proc/loadavg: 9.39 6.57 3.00 3/192 21187
/proc/meminfo: memFree=31306012/32951124 swapFree=67111528/67111528
[pid=21071] ppid=21069 vsize=8700 CPUtime=0
/proc/21071/stat : 21071 (run) S 21069 21071 20807 0 -1 4202496 366 0 0 0 0 0 0 0 19 0 1 0 146782418 8908800 249 4089446400 4194304 4922060 140735476331360 18446744073709551615 271550356613 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21071/statm: 2175 249 208 178 0 68 0
[pid=21072] ppid=21071 vsize=141420 CPUtime=1261.34
/proc/21072/stat : 21072 (pb_cplex) R 21071 21071 20807 0 -1 4202496 208304 0 0 0 125020 1114 0 0 15 0 8 0 146782418 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 137342007 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0
/proc/21072/statm: 35355 13215 1107 2985 0 31897 0
[pid=21072/tid=21124] ppid=21071 vsize=141420 CPUtime=141.78
/proc/21072/task/21124/stat : 21124 (pb_cplex) R 21071 21071 20807 0 -1 4202560 14099 0 0 0 14051 127 0 0 16 0 8 0 146785342 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=21072/tid=21125] ppid=21071 vsize=141420 CPUtime=141.5
/proc/21072/task/21125/stat : 21125 (pb_cplex) R 21071 21071 20807 0 -1 4202560 19300 0 0 0 14020 130 0 0 16 0 8 0 146785342 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=21072/tid=21126] ppid=21071 vsize=141420 CPUtime=141.48
/proc/21072/task/21126/stat : 21126 (pb_cplex) R 21071 21071 20807 0 -1 4202560 16956 0 0 0 14021 127 0 0 16 0 8 0 146785342 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=21072/tid=21127] ppid=21071 vsize=141420 CPUtime=144.63
/proc/21072/task/21127/stat : 21127 (pb_cplex) R 21071 21071 20807 0 -1 4202560 16862 0 0 0 14341 122 0 0 16 0 8 0 146785342 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=21072/tid=21128] ppid=21071 vsize=141420 CPUtime=141.69
/proc/21072/task/21128/stat : 21128 (pb_cplex) R 21071 21071 20807 0 -1 4202560 13565 0 0 0 14043 126 0 0 15 0 8 0 146785342 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=21072/tid=21129] ppid=21071 vsize=141420 CPUtime=142.13
/proc/21072/task/21129/stat : 21129 (pb_cplex) R 21071 21071 20807 0 -1 4202560 14817 0 0 0 14089 124 0 0 16 0 8 0 146785342 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 136017512 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=21072/tid=21130] ppid=21071 vsize=141420 CPUtime=143.36
/proc/21072/task/21130/stat : 21130 (pb_cplex) R 21071 21071 20807 0 -1 4202560 20892 0 0 0 14200 136 0 0 16 0 8 0 146785342 144814080 13215 4089446400 134512640 146736320 4291139040 18446744073709551615 137342686 0 2 4096 16387 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 1261.34
Current children cumulated vsize (KiB) 150120

[startup+349.501 s]
/proc/loadavg: 9.39 6.57 3.00 3/192 21187
/proc/meminfo: memFree=31306012/32951124 swapFree=67111528/67111528
[pid=21071] ppid=21069 vsize=8700 CPUtime=0
/proc/21071/stat : 21071 (run) S 21069 21071 20807 0 -1 4202496 366 0 0 0 0 0 0 0 19 0 1 0 146782418 8908800 249 4089446400 4194304 4922060 140735476331360 18446744073709551615 271550356613 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21071/statm: 2175 249 208 178 0 68 0
[pid=21072] ppid=21071 vsize=140396 CPUtime=1265.3
/proc/21072/stat : 21072 (pb_cplex) R 21071 21071 20807 0 -1 4202496 208519 0 0 0 125412 1118 0 0 15 0 8 0 146782418 143765504 13007 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 5 0 0 0
/proc/21072/statm: 35099 13007 1107 2985 0 31641 0
[pid=21072/tid=21124] ppid=21071 vsize=140396 CPUtime=142.29
/proc/21072/task/21124/stat : 21124 (pb_cplex) R 21071 21071 20807 0 -1 4202560 14099 0 0 0 14102 127 0 0 15 0 8 0 146785342 143765504 13007 4089446400 134512640 146736320 4291139040 18446744073709551615 136403112 0 2 4096 16387 0 0 0 -1 1 0 0 0
[pid=21072/tid=21125] ppid=21071 vsize=140396 CPUtime=142.01
/proc/21072/task/21125/stat : 21125 (pb_cplex) R 21071 21071 20807 0 -1 4202560 19487 0 0 0 14071 130 0 0 15 0 8 0 146785342 143765504 13007 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=21072/tid=21126] ppid=21071 vsize=140396 CPUtime=141.97
/proc/21072/task/21126/stat : 21126 (pb_cplex) R 21071 21071 20807 0 -1 4202560 16956 0 0 0 14069 128 0 0 15 0 8 0 146785342 143765504 13007 4089446400 134512640 146736320 4291139040 18446744073709551615 136001203 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=21072/tid=21127] ppid=21071 vsize=140396 CPUtime=145.09
/proc/21072/task/21127/stat : 21127 (pb_cplex) R 21071 21071 20807 0 -1 4202560 16863 0 0 0 14386 123 0 0 16 0 8 0 146785342 143765504 13007 4089446400 134512640 146736320 4291139040 18446744073709551615 136384857 0 2 4096 16387 0 0 0 -1 6 0 0 0
[pid=21072/tid=21128] ppid=21071 vsize=140396 CPUtime=142.22
/proc/21072/task/21128/stat : 21128 (pb_cplex) R 21071 21071 20807 0 -1 4202560 13591 0 0 0 14095 127 0 0 15 0 8 0 146785342 143765504 13007 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0
[pid=21072/tid=21129] ppid=21071 vsize=140396 CPUtime=142.61
/proc/21072/task/21129/stat : 21129 (pb_cplex) R 21071 21071 20807 0 -1 4202560 14817 0 0 0 14137 124 0 0 15 0 8 0 146785342 143765504 13007 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 7 0 0 0
[pid=21072/tid=21130] ppid=21071 vsize=140396 CPUtime=143.84
/proc/21072/task/21130/stat : 21130 (pb_cplex) R 21071 21071 20807 0 -1 4202560 20892 0 0 0 14248 136 0 0 15 0 8 0 146785342 143765504 13007 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 1265.3
Current children cumulated vsize (KiB) 149096

[startup+349.9 s]
/proc/loadavg: 9.39 6.57 3.00 10/199 21201
/proc/meminfo: memFree=31278924/32951124 swapFree=67111528/67111528
[pid=21071] ppid=21069 vsize=8700 CPUtime=0
/proc/21071/stat : 21071 (run) S 21069 21071 20807 0 -1 4202496 366 0 0 0 0 0 0 0 19 0 1 0 146782418 8908800 249 4089446400 4194304 4922060 140735476331360 18446744073709551615 271550356613 0 65536 4100 65538 18446744071562232103 0 0 17 6 0 0 0
/proc/21071/statm: 2175 249 208 178 0 68 0
[pid=21072] ppid=21071 vsize=140396 CPUtime=1266.88
/proc/21072/stat : 21072 (pb_cplex) S 21071 21071 20807 0 -1 4202496 208519 0 0 0 125568 1120 0 0 15 0 8 0 146782418 143765504 13007 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 7 0 0 0
/proc/21072/statm: 35099 13007 1107 2985 0 31641 0
[pid=21072/tid=21124] ppid=21071 vsize=140396 CPUtime=142.47
/proc/21072/task/21124/stat : 21124 (pb_cplex) S 21071 21071 20807 0 -1 4202560 14099 0 0 0 14120 127 0 0 15 0 8 0 146785342 143765504 13007 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 2 4096 16387 18446604471082982400 0 0 -1 1 0 0 0
[pid=21072/tid=21125] ppid=21071 vsize=140396 CPUtime=142.18
/proc/21072/task/21125/stat : 21125 (pb_cplex) S 21071 21071 20807 0 -1 4202560 19487 0 0 0 14088 130 0 0 15 0 8 0 146785342 143765504 13007 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 2 4096 16387 18446604471082982400 0 0 -1 7 0 0 0
[pid=21072/tid=21126] ppid=21071 vsize=140396 CPUtime=142.2
/proc/21072/task/21126/stat : 21126 (pb_cplex) S 21071 21071 20807 0 -1 4202560 16956 0 0 0 14092 128 0 0 15 0 8 0 146785342 143765504 13007 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 2 4096 16387 18446604471082982400 0 0 -1 6 0 0 0
[pid=21072/tid=21127] ppid=21071 vsize=140396 CPUtime=145.33
/proc/21072/task/21127/stat : 21127 (pb_cplex) R 21071 21071 20807 0 -1 4202560 16863 0 0 0 14410 123 0 0 16 0 8 0 146785342 143765504 13007 4089446400 134512640 146736320 4291139040 18446744073709551615 144580073 0 2 4096 16387 0 0 0 -1 4 0 0 0
[pid=21072/tid=21128] ppid=21071 vsize=140396 CPUtime=142.44
/proc/21072/task/21128/stat : 21128 (pb_cplex) R 21071 21071 20807 0 -1 4202560 13591 0 0 0 14117 127 0 0 15 0 8 0 146785342 143765504 13007 4089446400 134512640 146736320 4291139040 18446744073709551615 135453303 0 2 4096 16387 0 0 0 -1 0 0 0 0
[pid=21072/tid=21129] ppid=21071 vsize=140396 CPUtime=142.83
/proc/21072/task/21129/stat : 21129 (pb_cplex) R 21071 21071 20807 0 -1 4202560 14817 0 0 0 14158 125 0 0 15 0 8 0 146785342 143765504 13007 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 5 0 0 0
[pid=21072/tid=21130] ppid=21071 vsize=140396 CPUtime=144.04
/proc/21072/task/21130/stat : 21130 (pb_cplex) R 21071 21071 20807 0 -1 4202560 20892 0 0 0 14267 137 0 0 15 0 8 0 146785342 143765504 13007 4089446400 134512640 146736320 4291139040 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 1266.88
Current children cumulated vsize (KiB) 149096

Child status: 0
Real time (s): 349.985
CPU time (s): 1267.12
CPU user time (s): 1255.91
CPU system time (s): 11.2103
CPU usage (%): 362.049
Max. virtual memory (cumulated for all children) (KiB): 166428

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1255.91
system time used= 11.2103
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 208915
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 252299
involuntary context switches= 23308

runsolver used 0.688895 second user time and 1.37079 second system time

The end

Launcher Data 

Begin job on node133 at 2010-07-02 09:14:28
IDJOB=2697919
IDBENCH=78947
IDSOLVER=1210
FILE ID=node133/2697919-1278054868
PBS_JOBID= 11198915
Free space on /tmp= 74188 MiB

SOLVER NAME= pb_cplex 2010-07-01
BENCH NAME= PB10/normalized-PB10/PARTIAL-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/tsp/normalized-t3002.11tsp11.1900573901--soft-66-100-0.wbo
COMMAND LINE= run BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-2697919-1278054868/watcher-2697919-1278054868 -o /tmp/evaluation-result-2697919-1278054868/solver-2697919-1278054868 -C 1800 -W 2000 -M 3800  run HOME/instance-2697919-1278054868.wbo

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 3800 MiB
MAX NB THREAD= 0

MD5SUM BENCH= a057c56aaeadccffd1b9b5ae1c954d95
RANDOM SEED=1244874401

node133.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.821
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.821
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.821
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.821
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 7296.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.821
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.821
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.821
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.821
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      31323740 kB
Buffers:        217256 kB
Cached:        1155864 kB
SwapCached:          0 kB
Active:         700948 kB
Inactive:       779580 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      31323740 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           33680 kB
Writeback:           0 kB
AnonPages:      106852 kB
Mapped:          18792 kB
Slab:            83680 kB
PageTables:       4592 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   324384 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 74172 MiB
End job on node133 at 2010-07-02 09:20:18