Trace number 2696237

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
pb_cplex 2010-06-29? (TO) 1800.17 973.625

General information on the benchmark

Namenormalized-PB06/SATUNSAT-SMALLINT/submitted-PB06/
namasivayam/tsp/normalized-t3002.11tsp11.1900576151.opb
MD5SUM8485c5b622ae9efcb86cc93d1fd8b600
Bench CategoryDEC-SMALLINT (no optimisation, small integers)
Best result obtained on this benchmarkUNSAT
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark203.568
Has Objective FunctionNO
SatisfiableNO
(Un)Satisfiability was provedYES
Best value of the objective function
Optimality of the best value was proved
Number of variables231
Total number of constraints2707
Number of constraints which are clauses2684
Number of constraints which are cardinality constraints (but not clauses)22
Number of constraints which are nor clauses,nor cardinality constraints1
Minimum length of a constraint2
Maximum length of a constraint110
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 25
Number of bits of the biggest number in a constraint 5
Biggest sum of numbers in a constraint 697
Number of bits of the biggest sum of numbers10
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c pb_cplex 1.0 - 20100701
0.00/0.00	c Parsing opb file format. File HOME/instance-2696237-1278015176.opb.
0.00/0.00	c Instance file HOME/instance-2696237-1278015176.opb
0.00/0.00	c File size is 83377 bytes.
0.00/0.03	c Highest Coefficient sum: 697
0.00/0.03	c  IBM ILOG License Manager: "IBM ILOG Optimization Suite for Academic Initiative" is accessing CPLEX 12 with option(s): "e m b q ".
0.00/0.04	c ============================[ Problem Statistics ]=============================
0.00/0.04	c |                                                                             |
0.00/0.04	c |  Parsing time:          0.03         s                                      |
0.00/0.04	c |  Number Variables:      231                                                 |
0.00/0.04	c |  Number Constraints:    2707                                                |
0.00/0.04	c ===============================================================================
1800.15/973.62	c Remaining time 1799.96 sec.
1800.15/973.62	c 
1800.15/973.62	c *** INTERRUPTED ***
1800.15/973.62	s UNKNOWN
1800.15/973.62	c Total CPU time              : 1789.21 s
1800.15/973.62	c Memory used           : 39.52 MB
1800.15/973.62	c CPU time              : 1789.21 s
1800.15/973.62	c 
1800.15/973.62	c *** INTERRUPTED ***

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver version 3.2.9 (svn:492) (c) roussel@cril.univ-artois.fr

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-2696237-1278015176/watcher-2696237-1278015176 -o /tmp/evaluation-result-2696237-1278015176/solver-2696237-1278015176 -C 1800 -W 2000 -M 1800 run HOME/instance-2696237-1278015176.opb 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 1843200 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 1894400 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.38 0.09 0.03 3/106 24169
/proc/meminfo: memFree=1776320/2059040 swapFree=4139352/4192956
[pid=24169] ppid=24167 vsize=8700 CPUtime=0
/proc/24169/stat : 24169 (run) S 24167 24169 24051 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 175173927 8908800 249 1992294400 4194304 4922060 140734147825936 18446744073709551615 271797820549 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/24169/statm: 2175 249 208 178 0 68 0

[startup+0.111418 s]
/proc/loadavg: 0.38 0.09 0.03 3/106 24169
/proc/meminfo: memFree=1776320/2059040 swapFree=4139352/4192956
[pid=24169] ppid=24167 vsize=8700 CPUtime=0
/proc/24169/stat : 24169 (run) S 24167 24169 24051 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 175173927 8908800 249 1992294400 4194304 4922060 140734147825936 18446744073709551615 271797820549 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/24169/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.200417 s]
/proc/loadavg: 0.38 0.09 0.03 3/106 24169
/proc/meminfo: memFree=1776320/2059040 swapFree=4139352/4192956
[pid=24169] ppid=24167 vsize=8700 CPUtime=0
/proc/24169/stat : 24169 (run) S 24167 24169 24051 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 175173927 8908800 249 1992294400 4194304 4922060 140734147825936 18446744073709551615 271797820549 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/24169/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.300416 s]
/proc/loadavg: 0.38 0.09 0.03 3/106 24169
/proc/meminfo: memFree=1776320/2059040 swapFree=4139352/4192956
[pid=24169] ppid=24167 vsize=8700 CPUtime=0
/proc/24169/stat : 24169 (run) S 24167 24169 24051 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 175173927 8908800 249 1992294400 4194304 4922060 140734147825936 18446744073709551615 271797820549 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/24169/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.700412 s]
/proc/loadavg: 0.38 0.09 0.03 3/106 24169
/proc/meminfo: memFree=1776320/2059040 swapFree=4139352/4192956
[pid=24169] ppid=24167 vsize=8700 CPUtime=0
/proc/24169/stat : 24169 (run) S 24167 24169 24051 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 175173927 8908800 249 1992294400 4194304 4922060 140734147825936 18446744073709551615 271797820549 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/24169/statm: 2175 249 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+1.5014 s]
/proc/loadavg: 0.38 0.09 0.03 3/109 24172
/proc/meminfo: memFree=1765136/2059040 swapFree=4139352/4192956
[pid=24169] ppid=24167 vsize=8700 CPUtime=0
/proc/24169/stat : 24169 (run) S 24167 24169 24051 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 175173927 8908800 249 1992294400 4194304 4922060 140734147825936 18446744073709551615 271797820549 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/24169/statm: 2175 249 208 178 0 68 0
[pid=24170] ppid=24169 vsize=40052 CPUtime=1.79
/proc/24170/stat : 24170 (pb_cplex) R 24169 24169 24051 0 -1 4202496 5514 0 0 0 177 2 0 0 17 0 2 0 175173927 41013248 3979 1992294400 134512640 146736320 4290072656 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/24170/statm: 10013 3979 987 2985 0 6555 0
[pid=24170/tid=24172] ppid=24169 vsize=40052 CPUtime=0.42
/proc/24170/task/24172/stat : 24172 (pb_cplex) R 24169 24169 24051 0 -1 4202560 612 0 0 0 42 0 0 0 17 0 2 0 175174025 41013248 3979 1992294400 134512640 146736320 4290072656 18446744073709551615 136403112 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1.79
Current children cumulated vsize (KiB) 48752

[startup+3.10039 s]
/proc/loadavg: 0.38 0.09 0.03 3/109 24172
/proc/meminfo: memFree=1762780/2059040 swapFree=4139352/4192956
[pid=24169] ppid=24167 vsize=8700 CPUtime=0
/proc/24169/stat : 24169 (run) S 24167 24169 24051 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 175173927 8908800 249 1992294400 4194304 4922060 140734147825936 18446744073709551615 271797820549 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/24169/statm: 2175 249 208 178 0 68 0
[pid=24170] ppid=24169 vsize=35904 CPUtime=4.63
/proc/24170/stat : 24170 (pb_cplex) R 24169 24169 24051 0 -1 4202496 6333 0 0 0 458 5 0 0 17 0 1 0 175173927 36765696 3416 1992294400 134512640 146736320 4290072656 18446744073709551615 138380924 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/24170/statm: 8976 3416 991 2985 0 5518 0
Current children cumulated CPU time (s) 4.63
Current children cumulated vsize (KiB) 44604

[startup+6.30135 s]
/proc/loadavg: 0.51 0.12 0.04 3/109 24174
/proc/meminfo: memFree=1762904/2059040 swapFree=4139352/4192956
[pid=24169] ppid=24167 vsize=8700 CPUtime=0
/proc/24169/stat : 24169 (run) S 24167 24169 24051 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 175173927 8908800 249 1992294400 4194304 4922060 140734147825936 18446744073709551615 271797820549 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/24169/statm: 2175 249 208 178 0 68 0
[pid=24170] ppid=24169 vsize=38632 CPUtime=8.42
/proc/24170/stat : 24170 (pb_cplex) S 24169 24169 24051 0 -1 4202496 8944 0 0 0 835 7 0 0 19 0 2 0 175173927 39559168 4017 1992294400 134512640 146736320 4290072656 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0
/proc/24170/statm: 9658 4017 1000 2985 0 6200 0
[pid=24170/tid=24174] ppid=24169 vsize=38632 CPUtime=0.92
/proc/24170/task/24174/stat : 24174 (pb_cplex) R 24169 24169 24051 0 -1 4202560 920 0 0 0 92 0 0 0 18 0 2 0 175174417 39559168 4017 1992294400 134512640 146736320 4290072656 18446744073709551615 136400557 0 2 4096 16387 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 8.42
Current children cumulated vsize (KiB) 47332

[startup+12.7003 s]
/proc/loadavg: 0.63 0.16 0.05 3/109 24174
/proc/meminfo: memFree=1761664/2059040 swapFree=4139352/4192956
[pid=24169] ppid=24167 vsize=8700 CPUtime=0
/proc/24169/stat : 24169 (run) S 24167 24169 24051 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 175173927 8908800 249 1992294400 4194304 4922060 140734147825936 18446744073709551615 271797820549 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/24169/statm: 2175 249 208 178 0 68 0
[pid=24170] ppid=24169 vsize=39380 CPUtime=20.15
/proc/24170/stat : 24170 (pb_cplex) R 24169 24169 24051 0 -1 4202496 18007 0 0 0 1998 17 0 0 18 0 2 0 175173927 40325120 4256 1992294400 134512640 146736320 4290072656 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0
/proc/24170/statm: 9845 4256 1000 2985 0 6387 0
[pid=24170/tid=24174] ppid=24169 vsize=39380 CPUtime=6.62
/proc/24170/task/24174/stat : 24174 (pb_cplex) R 24169 24169 24051 0 -1 4202560 8060 0 0 0 656 6 0 0 18 0 2 0 175174417 40325120 4256 1992294400 134512640 146736320 4290072656 18446744073709551615 137083064 0 2 4096 16387 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 20.15
Current children cumulated vsize (KiB) 48080

[startup+25.5012 s]
/proc/loadavg: 0.93 0.25 0.08 3/109 24174
/proc/meminfo: memFree=1761044/2059040 swapFree=4139352/4192956
[pid=24169] ppid=24167 vsize=8700 CPUtime=0
/proc/24169/stat : 24169 (run) S 24167 24169 24051 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 175173927 8908800 249 1992294400 4194304 4922060 140734147825936 18446744073709551615 271797820549 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/24169/statm: 2175 249 208 178 0 68 0
[pid=24170] ppid=24169 vsize=39724 CPUtime=43.75
/proc/24170/stat : 24170 (pb_cplex) R 24169 24169 24051 0 -1 4202496 25191 0 0 0 4342 33 0 0 16 0 2 0 175173927 40677376 4405 1992294400 134512640 146736320 4290072656 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/24170/statm: 9931 4405 1000 2985 0 6473 0
[pid=24170/tid=24174] ppid=24169 vsize=39724 CPUtime=18.58
/proc/24170/task/24174/stat : 24174 (pb_cplex) R 24169 24169 24051 0 -1 4202560 14459 0 0 0 1844 14 0 0 18 0 2 0 175174417 40677376 4405 1992294400 134512640 146736320 4290072656 18446744073709551615 136388115 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 43.75
Current children cumulated vsize (KiB) 48424

[startup+51.1019 s]
/proc/loadavg: 1.23 0.37 0.12 2/109 24176
/proc/meminfo: memFree=1760920/2059040 swapFree=4139352/4192956
[pid=24169] ppid=24167 vsize=8700 CPUtime=0
/proc/24169/stat : 24169 (run) S 24167 24169 24051 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 175173927 8908800 249 1992294400 4194304 4922060 140734147825936 18446744073709551615 271797820549 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/24169/statm: 2175 249 208 178 0 68 0
[pid=24170] ppid=24169 vsize=39724 CPUtime=91.26
/proc/24170/stat : 24170 (pb_cplex) R 24169 24169 24051 0 -1 4202496 35811 0 0 0 9061 65 0 0 19 0 2 0 175173927 40677376 4456 1992294400 134512640 146736320 4290072656 18446744073709551615 136007303 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0
/proc/24170/statm: 9931 4456 1000 2985 0 6473 0
[pid=24170/tid=24174] ppid=24169 vsize=39724 CPUtime=42.38
/proc/24170/task/24174/stat : 24174 (pb_cplex) R 24169 24169 24051 0 -1 4202560 25066 0 0 0 4205 33 0 0 18 0 2 0 175174417 40677376 4456 1992294400 134512640 146736320 4290072656 18446744073709551615 136007083 0 2 4096 16387 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 91.26
Current children cumulated vsize (KiB) 48424

[startup+102.301 s]
/proc/loadavg: 1.46 0.57 0.20 3/109 24176
/proc/meminfo: memFree=1760796/2059040 swapFree=4139352/4192956
[pid=24169] ppid=24167 vsize=8700 CPUtime=0
/proc/24169/stat : 24169 (run) S 24167 24169 24051 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 175173927 8908800 249 1992294400 4194304 4922060 140734147825936 18446744073709551615 271797820549 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/24169/statm: 2175 249 208 178 0 68 0
[pid=24170] ppid=24169 vsize=39872 CPUtime=169.9
/proc/24170/stat : 24170 (pb_cplex) R 24169 24169 24051 0 -1 4202496 61548 0 0 0 16876 114 0 0 17 0 2 0 175173927 40828928 4525 1992294400 134512640 146736320 4290072656 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/24170/statm: 9968 4525 1007 2985 0 6510 0
[pid=24170/tid=24174] ppid=24169 vsize=39872 CPUtime=72.74
/proc/24170/task/24174/stat : 24174 (pb_cplex) R 24169 24169 24051 0 -1 4202560 26146 0 0 0 7222 52 0 0 18 0 2 0 175174417 40828928 4525 1992294400 134512640 146736320 4290072656 18446744073709551615 136403109 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 169.9
Current children cumulated vsize (KiB) 48572

[startup+162.301 s]
/proc/loadavg: 1.86 0.85 0.32 4/109 24178
/proc/meminfo: memFree=1760548/2059040 swapFree=4139352/4192956
[pid=24169] ppid=24167 vsize=8700 CPUtime=0
/proc/24169/stat : 24169 (run) S 24167 24169 24051 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 175173927 8908800 249 1992294400 4194304 4922060 140734147825936 18446744073709551615 271797820549 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/24169/statm: 2175 249 208 178 0 68 0
[pid=24170] ppid=24169 vsize=40012 CPUtime=281.8
/proc/24170/stat : 24170 (pb_cplex) R 24169 24169 24051 0 -1 4202496 75828 0 0 0 27996 184 0 0 17 0 2 0 175173927 40972288 4581 1992294400 134512640 146736320 4290072656 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/24170/statm: 10003 4581 1007 2985 0 6545 0
[pid=24170/tid=24174] ppid=24169 vsize=40012 CPUtime=130.43
/proc/24170/task/24174/stat : 24174 (pb_cplex) R 24169 24169 24051 0 -1 4202560 26671 0 0 0 12958 85 0 0 17 0 2 0 175174417 40972288 4581 1992294400 134512640 146736320 4290072656 18446744073709551615 135526124 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 281.8
Current children cumulated vsize (KiB) 48712

[startup+222.308 s]
/proc/loadavg: 1.90 1.05 0.42 3/109 24180
/proc/meminfo: memFree=1759804/2059040 swapFree=4139352/4192956
[pid=24169] ppid=24167 vsize=8700 CPUtime=0
/proc/24169/stat : 24169 (run) S 24167 24169 24051 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 175173927 8908800 249 1992294400 4194304 4922060 140734147825936 18446744073709551615 271797820549 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/24169/statm: 2175 249 208 178 0 68 0
[pid=24170] ppid=24169 vsize=41036 CPUtime=394.19
/proc/24170/stat : 24170 (pb_cplex) R 24169 24169 24051 0 -1 4202496 85975 0 0 0 39168 251 0 0 17 0 2 0 175173927 42020864 4659 1992294400 134512640 146736320 4290072656 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/24170/statm: 10259 4659 1007 2985 0 6801 0
[pid=24170/tid=24174] ppid=24169 vsize=41036 CPUtime=188.55
/proc/24170/task/24174/stat : 24174 (pb_cplex) R 24169 24169 24051 0 -1 4202560 27936 0 0 0 18740 115 0 0 18 0 2 0 175174417 42020864 4659 1992294400 134512640 146736320 4290072656 18446744073709551615 136403132 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 394.19
Current children cumulated vsize (KiB) 49736

[startup+282.301 s]
/proc/loadavg: 1.86 1.19 0.51 4/109 24182
/proc/meminfo: memFree=1760300/2059040 swapFree=4139352/4192956
[pid=24169] ppid=24167 vsize=8700 CPUtime=0
/proc/24169/stat : 24169 (run) S 24167 24169 24051 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 175173927 8908800 249 1992294400 4194304 4922060 140734147825936 18446744073709551615 271797820549 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/24169/statm: 2175 249 208 178 0 68 0
[pid=24170] ppid=24169 vsize=41040 CPUtime=506.27
/proc/24170/stat : 24170 (pb_cplex) R 24169 24169 24051 0 -1 4202496 95086 0 0 0 50312 315 0 0 17 0 2 0 175173927 42024960 4616 1992294400 134512640 146736320 4290072656 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/24170/statm: 10260 4616 1007 2985 0 6802 0
[pid=24170/tid=24174] ppid=24169 vsize=41040 CPUtime=246.34
/proc/24170/task/24174/stat : 24174 (pb_cplex) R 24169 24169 24051 0 -1 4202560 35111 0 0 0 24486 148 0 0 18 0 2 0 175174417 42024960 4616 1992294400 134512640 146736320 4290072656 18446744073709551615 136384781 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 506.27
Current children cumulated vsize (KiB) 49740

[startup+342.303 s]
/proc/loadavg: 2.03 1.36 0.61 3/109 24182
/proc/meminfo: memFree=1760300/2059040 swapFree=4139352/4192956
[pid=24169] ppid=24167 vsize=8700 CPUtime=0
/proc/24169/stat : 24169 (run) S 24167 24169 24051 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 175173927 8908800 249 1992294400 4194304 4922060 140734147825936 18446744073709551615 271797820549 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/24169/statm: 2175 249 208 178 0 68 0
[pid=24170] ppid=24169 vsize=40464 CPUtime=618.46

################
# More data... #
################

[pid=24170] ppid=24169 vsize=40464 CPUtime=730.66
/proc/24170/stat : 24170 (pb_cplex) R 24169 24169 24051 0 -1 4202496 119232 0 0 0 72611 455 0 0 18 0 2 0 175173927 41435136 4564 1992294400 134512640 146736320 4290072656 18446744073709551615 136403109 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/24170/statm: 10116 4564 1007 2985 0 6658 0
[pid=24170/tid=24174] ppid=24169 vsize=40464 CPUtime=361.67
/proc/24170/task/24174/stat : 24174 (pb_cplex) R 24169 24169 24051 0 -1 4202560 39453 0 0 0 35954 213 0 0 18 0 2 0 175174417 41435136 4564 1992294400 134512640 146736320 4290072656 18446744073709551615 136008050 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 730.66
Current children cumulated vsize (KiB) 49164

[startup+462.301 s]
/proc/loadavg: 2.07 1.59 0.78 3/109 24186
/proc/meminfo: memFree=1760672/2059040 swapFree=4139352/4192956
[pid=24169] ppid=24167 vsize=8700 CPUtime=0
/proc/24169/stat : 24169 (run) S 24167 24169 24051 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 175173927 8908800 249 1992294400 4194304 4922060 140734147825936 18446744073709551615 271797820549 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/24169/statm: 2175 249 208 178 0 68 0
[pid=24170] ppid=24169 vsize=40464 CPUtime=843.38
/proc/24170/stat : 24170 (pb_cplex) R 24169 24169 24051 0 -1 4202496 134353 0 0 0 83811 527 0 0 16 0 2 0 175173927 41435136 4581 1992294400 134512640 146736320 4290072656 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/24170/statm: 10116 4581 1007 2985 0 6658 0
[pid=24170/tid=24174] ppid=24169 vsize=40464 CPUtime=419.82
/proc/24170/task/24174/stat : 24174 (pb_cplex) R 24169 24169 24051 0 -1 4202560 46213 0 0 0 41735 247 0 0 18 0 2 0 175174417 41435136 4581 1992294400 134512640 146736320 4290072656 18446744073709551615 136400564 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 843.38
Current children cumulated vsize (KiB) 49164

[startup+522.3 s]
/proc/loadavg: 1.99 1.65 0.85 2/109 24188
/proc/meminfo: memFree=1760052/2059040 swapFree=4139352/4192956
[pid=24169] ppid=24167 vsize=8700 CPUtime=0
/proc/24169/stat : 24169 (run) S 24167 24169 24051 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 175173927 8908800 249 1992294400 4194304 4922060 140734147825936 18446744073709551615 271797820549 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/24169/statm: 2175 249 208 178 0 68 0
[pid=24170] ppid=24169 vsize=40464 CPUtime=955.66
/proc/24170/stat : 24170 (pb_cplex) R 24169 24169 24051 0 -1 4202496 146055 0 0 0 94967 599 0 0 18 0 2 0 175173927 41435136 4581 1992294400 134512640 146736320 4290072656 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/24170/statm: 10116 4581 1007 2985 0 6658 0
[pid=24170/tid=24174] ppid=24169 vsize=40464 CPUtime=475.45
/proc/24170/task/24174/stat : 24174 (pb_cplex) R 24169 24169 24051 0 -1 4202560 52044 0 0 0 47265 280 0 0 18 0 2 0 175174417 41435136 4581 1992294400 134512640 146736320 4290072656 18446744073709551615 135854186 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 955.66
Current children cumulated vsize (KiB) 49164

[startup+582.3 s]
/proc/loadavg: 2.05 1.73 0.92 4/109 24190
/proc/meminfo: memFree=1760052/2059040 swapFree=4139352/4192956
[pid=24169] ppid=24167 vsize=8700 CPUtime=0
/proc/24169/stat : 24169 (run) S 24167 24169 24051 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 175173927 8908800 249 1992294400 4194304 4922060 140734147825936 18446744073709551615 271797820549 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/24169/statm: 2175 249 208 178 0 68 0
[pid=24170] ppid=24169 vsize=40464 CPUtime=1068.02
/proc/24170/stat : 24170 (pb_cplex) R 24169 24169 24051 0 -1 4202496 155257 0 0 0 106138 664 0 0 16 0 2 0 175173927 41435136 4582 1992294400 134512640 146736320 4290072656 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/24170/statm: 10116 4582 1007 2985 0 6658 0
[pid=24170/tid=24174] ppid=24169 vsize=40464 CPUtime=533.66
/proc/24170/task/24174/stat : 24174 (pb_cplex) R 24169 24169 24051 0 -1 4202560 57607 0 0 0 53054 312 0 0 18 0 2 0 175174417 41435136 4582 1992294400 134512640 146736320 4290072656 18446744073709551615 136007303 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1068.02
Current children cumulated vsize (KiB) 49164

[startup+642.301 s]
/proc/loadavg: 2.10 1.80 1.00 2/109 24190
/proc/meminfo: memFree=1759928/2059040 swapFree=4139352/4192956
[pid=24169] ppid=24167 vsize=8700 CPUtime=0
/proc/24169/stat : 24169 (run) S 24167 24169 24051 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 175173927 8908800 249 1992294400 4194304 4922060 140734147825936 18446744073709551615 271797820549 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/24169/statm: 2175 249 208 178 0 68 0
[pid=24170] ppid=24169 vsize=40464 CPUtime=1180.41
/proc/24170/stat : 24170 (pb_cplex) R 24169 24169 24051 0 -1 4202496 166016 0 0 0 117312 729 0 0 16 0 2 0 175173927 41435136 4582 1992294400 134512640 146736320 4290072656 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/24170/statm: 10116 4582 1007 2985 0 6658 0
[pid=24170/tid=24174] ppid=24169 vsize=40464 CPUtime=592.02
/proc/24170/task/24174/stat : 24174 (pb_cplex) R 24169 24169 24051 0 -1 4202560 63876 0 0 0 58856 346 0 0 18 0 2 0 175174417 41435136 4582 1992294400 134512640 146736320 4290072656 18446744073709551615 135999252 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1180.41
Current children cumulated vsize (KiB) 49164

[startup+702.301 s]
/proc/loadavg: 2.11 1.85 1.06 3/109 24192
/proc/meminfo: memFree=1759804/2059040 swapFree=4139352/4192956
[pid=24169] ppid=24167 vsize=8700 CPUtime=0
/proc/24169/stat : 24169 (run) S 24167 24169 24051 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 175173927 8908800 249 1992294400 4194304 4922060 140734147825936 18446744073709551615 271797820549 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/24169/statm: 2175 249 208 178 0 68 0
[pid=24170] ppid=24169 vsize=40464 CPUtime=1293.29
/proc/24170/stat : 24170 (pb_cplex) R 24169 24169 24051 0 -1 4202496 175556 0 0 0 128535 794 0 0 17 0 2 0 175173927 41435136 4581 1992294400 134512640 146736320 4290072656 18446744073709551615 136403116 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/24170/statm: 10116 4581 1007 2985 0 6658 0
[pid=24170/tid=24174] ppid=24169 vsize=40464 CPUtime=650.29
/proc/24170/task/24174/stat : 24174 (pb_cplex) R 24169 24169 24051 0 -1 4202560 68494 0 0 0 64652 377 0 0 18 0 2 0 175174417 41435136 4581 1992294400 134512640 146736320 4290072656 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1293.29
Current children cumulated vsize (KiB) 49164

[startup+762.302 s]
/proc/loadavg: 2.05 1.89 1.12 3/109 24194
/proc/meminfo: memFree=1759928/2059040 swapFree=4139352/4192956
[pid=24169] ppid=24167 vsize=8700 CPUtime=0
/proc/24169/stat : 24169 (run) S 24167 24169 24051 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 175173927 8908800 249 1992294400 4194304 4922060 140734147825936 18446744073709551615 271797820549 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/24169/statm: 2175 249 208 178 0 68 0
[pid=24170] ppid=24169 vsize=40464 CPUtime=1405.79
/proc/24170/stat : 24170 (pb_cplex) R 24169 24169 24051 0 -1 4202496 185147 0 0 0 139714 865 0 0 17 0 2 0 175173927 41435136 4581 1992294400 134512640 146736320 4290072656 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/24170/statm: 10116 4581 1007 2985 0 6658 0
[pid=24170/tid=24174] ppid=24169 vsize=40464 CPUtime=707.68
/proc/24170/task/24174/stat : 24174 (pb_cplex) R 24169 24169 24051 0 -1 4202560 72172 0 0 0 70355 413 0 0 19 0 2 0 175174417 41435136 4581 1992294400 134512640 146736320 4290072656 18446744073709551615 136007612 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1405.79
Current children cumulated vsize (KiB) 49164

[startup+822.301 s]
/proc/loadavg: 1.92 1.88 1.17 3/109 24196
/proc/meminfo: memFree=1759804/2059040 swapFree=4139352/4192956
[pid=24169] ppid=24167 vsize=8700 CPUtime=0
/proc/24169/stat : 24169 (run) S 24167 24169 24051 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 175173927 8908800 249 1992294400 4194304 4922060 140734147825936 18446744073709551615 271797820549 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/24169/statm: 2175 249 208 178 0 68 0
[pid=24170] ppid=24169 vsize=40464 CPUtime=1518.42
/proc/24170/stat : 24170 (pb_cplex) R 24169 24169 24051 0 -1 4202496 193784 0 0 0 150913 929 0 0 17 0 2 0 175173927 41435136 4581 1992294400 134512640 146736320 4290072656 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/24170/statm: 10116 4581 1007 2985 0 6658 0
[pid=24170/tid=24174] ppid=24169 vsize=40464 CPUtime=764.56
/proc/24170/task/24174/stat : 24174 (pb_cplex) R 24169 24169 24051 0 -1 4202560 77470 0 0 0 76010 446 0 0 18 0 2 0 175174417 41435136 4581 1992294400 134512640 146736320 4290072656 18446744073709551615 136007289 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1518.42
Current children cumulated vsize (KiB) 49164

[startup+882.308 s]
/proc/loadavg: 2.14 1.94 1.23 2/109 24198
/proc/meminfo: memFree=1759680/2059040 swapFree=4139352/4192956
[pid=24169] ppid=24167 vsize=8700 CPUtime=0
/proc/24169/stat : 24169 (run) S 24167 24169 24051 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 175173927 8908800 249 1992294400 4194304 4922060 140734147825936 18446744073709551615 271797820549 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/24169/statm: 2175 249 208 178 0 68 0
[pid=24170] ppid=24169 vsize=40464 CPUtime=1630.14
/proc/24170/stat : 24170 (pb_cplex) R 24169 24169 24051 0 -1 4202496 202693 0 0 0 162018 996 0 0 17 0 2 0 175173927 41435136 4581 1992294400 134512640 146736320 4290072656 18446744073709551615 135454118 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/24170/statm: 10116 4581 1007 2985 0 6658 0
[pid=24170/tid=24174] ppid=24169 vsize=40464 CPUtime=822.29
/proc/24170/task/24174/stat : 24174 (pb_cplex) R 24169 24169 24051 0 -1 4202560 81807 0 0 0 81748 481 0 0 18 0 2 0 175174417 41435136 4581 1992294400 134512640 146736320 4290072656 18446744073709551615 136008216 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1630.14
Current children cumulated vsize (KiB) 49164

[startup+942.301 s]
/proc/loadavg: 2.05 1.95 1.28 3/109 24198
/proc/meminfo: memFree=1759680/2059040 swapFree=4139352/4192956
[pid=24169] ppid=24167 vsize=8700 CPUtime=0
/proc/24169/stat : 24169 (run) S 24167 24169 24051 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 175173927 8908800 249 1992294400 4194304 4922060 140734147825936 18446744073709551615 271797820549 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/24169/statm: 2175 249 208 178 0 68 0
[pid=24170] ppid=24169 vsize=40464 CPUtime=1741.89
/proc/24170/stat : 24170 (pb_cplex) R 24169 24169 24051 0 -1 4202496 209802 0 0 0 173128 1061 0 0 18 0 2 0 175173927 41435136 4581 1992294400 134512640 146736320 4290072656 18446744073709551615 139042897 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/24170/statm: 10116 4581 1007 2985 0 6658 0
[pid=24170/tid=24174] ppid=24169 vsize=40464 CPUtime=878.18
/proc/24170/task/24174/stat : 24174 (pb_cplex) R 24169 24169 24051 0 -1 4202560 84813 0 0 0 87305 513 0 0 18 0 2 0 175174417 41435136 4581 1992294400 134512640 146736320 4290072656 18446744073709551615 136007622 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1741.89
Current children cumulated vsize (KiB) 49164



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+973.608 s]
/proc/loadavg: 2.10 1.97 1.31 3/109 24200
/proc/meminfo: memFree=1759556/2059040 swapFree=4139352/4192956
[pid=24169] ppid=24167 vsize=8700 CPUtime=0
/proc/24169/stat : 24169 (run) S 24167 24169 24051 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 175173927 8908800 249 1992294400 4194304 4922060 140734147825936 18446744073709551615 271797820549 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/24169/statm: 2175 249 208 178 0 68 0
[pid=24170] ppid=24169 vsize=40464 CPUtime=1800.15
/proc/24170/stat : 24170 (pb_cplex) R 24169 24169 24051 0 -1 4202496 211911 0 0 0 178919 1096 0 0 18 0 2 0 175173927 41435136 4581 1992294400 134512640 146736320 4290072656 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/24170/statm: 10116 4581 1007 2985 0 6658 0
[pid=24170/tid=24174] ppid=24169 vsize=40464 CPUtime=908.25
/proc/24170/task/24174/stat : 24174 (pb_cplex) R 24169 24169 24051 0 -1 4202560 85549 0 0 0 90295 530 0 0 18 0 2 0 175174417 41435136 4581 1992294400 134512640 146736320 4290072656 18446744073709551615 135196483 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1800.15
Current children cumulated vsize (KiB) 49164

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+973.608 s]
/proc/loadavg: 2.10 1.97 1.31 3/109 24200
/proc/meminfo: memFree=1759556/2059040 swapFree=4139352/4192956
[pid=24169] ppid=24167 vsize=8700 CPUtime=0
/proc/24169/stat : 24169 (run) S 24167 24169 24051 0 -1 4202496 364 0 0 0 0 0 0 0 19 0 1 0 175173927 8908800 249 1992294400 4194304 4922060 140734147825936 18446744073709551615 271797820549 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/24169/statm: 2175 249 208 178 0 68 0
[pid=24170] ppid=24169 vsize=40464 CPUtime=1800.15
/proc/24170/stat : 24170 (pb_cplex) R 24169 24169 24051 0 -1 4202496 211911 0 0 0 178919 1096 0 0 18 0 2 0 175173927 41435136 4581 1992294400 134512640 146736320 4290072656 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/24170/statm: 10116 4581 1007 2985 0 6658 0
[pid=24170/tid=24174] ppid=24169 vsize=40464 CPUtime=908.25
/proc/24170/task/24174/stat : 24174 (pb_cplex) R 24169 24169 24051 0 -1 4202560 85549 0 0 0 90295 530 0 0 18 0 2 0 175174417 41435136 4581 1992294400 134512640 146736320 4290072656 18446744073709551615 135196483 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1800.15
Current children cumulated vsize (KiB) 49164

Child status: 0
Real time (s): 973.625
CPU time (s): 1800.17
CPU user time (s): 1789.21
CPU system time (s): 10.9663
CPU usage (%): 184.894
Max. virtual memory (cumulated for all children) (KiB): 60212

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1789.21
system time used= 10.9663
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 212285
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 147907
involuntary context switches= 31998

runsolver used 1.9927 second user time and 5.10522 second system time

The end

Launcher Data 

Begin job on node019 at 2010-07-01 22:12:56
IDJOB=2696237
IDBENCH=1327
IDSOLVER=1209
FILE ID=node019/2696237-1278015176
PBS_JOBID= 11198737
Free space on /tmp= 62048 MiB

SOLVER NAME= pb_cplex 2010-06-29
BENCH NAME= PB06//final/normalized-PB06/SATUNSAT-SMALLINT/submitted-PB06/namasivayam/tsp/normalized-t3002.11tsp11.1900576151.opb
COMMAND LINE= run BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-2696237-1278015176/watcher-2696237-1278015176 -o /tmp/evaluation-result-2696237-1278015176/solver-2696237-1278015176 -C 1800 -W 2000 -M 1800  run HOME/instance-2696237-1278015176.opb

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 1800 MiB
MAX NB THREAD= 0

MD5SUM BENCH= 8485c5b622ae9efcb86cc93d1fd8b600
RANDOM SEED=528556170

node019.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.264
cache size	: 2048 KB
physical id	: 0
siblings	: 1
core id		: 0
cpu cores	: 1
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc pni monitor ds_cpl cid cx16 xtpr
bogomips	: 6000.52
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.264
cache size	: 2048 KB
physical id	: 3
siblings	: 1
core id		: 0
cpu cores	: 1
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5599.54
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:      2059040 kB
MemFree:       1776732 kB
Buffers:         57112 kB
Cached:         125056 kB
SwapCached:       6180 kB
Active:         104376 kB
Inactive:        91500 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:      2059040 kB
LowFree:       1776732 kB
SwapTotal:     4192956 kB
SwapFree:      4139352 kB
Dirty:           16992 kB
Writeback:           0 kB
AnonPages:       12308 kB
Mapped:          11860 kB
Slab:            64464 kB
PageTables:       4184 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   5222476 kB
Committed_AS:   182584 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    264948 kB
VmallocChunk: 34359471699 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 62032 MiB
End job on node019 at 2010-07-01 22:29:10