Trace number 2696271

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
pb_cplex 2010-06-29? (TO) 1800.15 988.928

General information on the benchmark

Namenormalized-PB06/SATUNSAT-SMALLINT/submitted-PB06/
namasivayam/tsp/normalized-t3002.11tsp11.1900550720.opb
MD5SUM76a48b8845ca731b7df3e7e82d604015
Bench CategoryDEC-SMALLINT (no optimisation, small integers)
Best result obtained on this benchmarkUNSAT
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark63.3934
Has Objective FunctionNO
SatisfiableNO
(Un)Satisfiability was provedYES
Best value of the objective function
Optimality of the best value was proved
Number of variables231
Total number of constraints2707
Number of constraints which are clauses2684
Number of constraints which are cardinality constraints (but not clauses)22
Number of constraints which are nor clauses,nor cardinality constraints1
Minimum length of a constraint2
Maximum length of a constraint110
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 25
Number of bits of the biggest number in a constraint 5
Biggest sum of numbers in a constraint 687
Number of bits of the biggest sum of numbers10
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c pb_cplex 1.0 - 20100701
0.00/0.00	c Parsing opb file format. File HOME/instance-2696271-1278015736.opb.
0.00/0.00	c Instance file HOME/instance-2696271-1278015736.opb
0.00/0.00	c File size is 83375 bytes.
0.00/0.03	c Highest Coefficient sum: 687
0.00/0.03	c  IBM ILOG License Manager: "IBM ILOG Optimization Suite for Academic Initiative" is accessing CPLEX 12 with option(s): "e m b q ".
0.00/0.04	c ============================[ Problem Statistics ]=============================
0.00/0.04	c |                                                                             |
0.00/0.04	c |  Parsing time:          0.03         s                                      |
0.00/0.04	c |  Number Variables:      231                                                 |
0.00/0.04	c |  Number Constraints:    2707                                                |
0.00/0.04	c ===============================================================================
1800.13/988.92	c Remaining time 1799.97 sec.
1800.13/988.92	c 
1800.13/988.92	c *** INTERRUPTED ***
1800.13/988.92	s UNKNOWN
1800.13/988.92	c Total CPU time              : 1792.43 s
1800.13/988.92	c Memory used           : 39.93 MB
1800.13/988.92	c CPU time              : 1792.43 s
1800.13/988.92	c 
1800.13/988.92	c *** INTERRUPTED ***

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver version 3.2.9 (svn:492) (c) roussel@cril.univ-artois.fr

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-2696271-1278015736/watcher-2696271-1278015736 -o /tmp/evaluation-result-2696271-1278015736/solver-2696271-1278015736 -C 1800 -W 2000 -M 1800 run HOME/instance-2696271-1278015736.opb 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 1843200 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 1894400 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.85 1.68 0.91 3/106 23352
/proc/meminfo: memFree=1583624/2059040 swapFree=4139332/4192956
[pid=23352] ppid=23350 vsize=8700 CPUtime=0
/proc/23352/stat : 23352 (run) S 23350 23352 23192 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 175229972 8908800 248 1992294400 4194304 4922060 140735328128128 18446744073709551615 273567816837 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/23352/statm: 2175 248 208 178 0 68 0

[startup+0.070853 s]
/proc/loadavg: 1.85 1.68 0.91 3/106 23352
/proc/meminfo: memFree=1583624/2059040 swapFree=4139332/4192956
[pid=23352] ppid=23350 vsize=8700 CPUtime=0
/proc/23352/stat : 23352 (run) S 23350 23352 23192 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 175229972 8908800 248 1992294400 4194304 4922060 140735328128128 18446744073709551615 273567816837 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/23352/statm: 2175 248 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.100852 s]
/proc/loadavg: 1.85 1.68 0.91 3/106 23352
/proc/meminfo: memFree=1583624/2059040 swapFree=4139332/4192956
[pid=23352] ppid=23350 vsize=8700 CPUtime=0
/proc/23352/stat : 23352 (run) S 23350 23352 23192 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 175229972 8908800 248 1992294400 4194304 4922060 140735328128128 18446744073709551615 273567816837 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/23352/statm: 2175 248 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.300848 s]
/proc/loadavg: 1.85 1.68 0.91 3/106 23352
/proc/meminfo: memFree=1583624/2059040 swapFree=4139332/4192956
[pid=23352] ppid=23350 vsize=8700 CPUtime=0
/proc/23352/stat : 23352 (run) S 23350 23352 23192 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 175229972 8908800 248 1992294400 4194304 4922060 140735328128128 18446744073709551615 273567816837 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/23352/statm: 2175 248 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.70084 s]
/proc/loadavg: 1.85 1.68 0.91 3/106 23352
/proc/meminfo: memFree=1583624/2059040 swapFree=4139332/4192956
[pid=23352] ppid=23350 vsize=8700 CPUtime=0
/proc/23352/stat : 23352 (run) S 23350 23352 23192 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 175229972 8908800 248 1992294400 4194304 4922060 140735328128128 18446744073709551615 273567816837 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/23352/statm: 2175 248 208 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+1.50082 s]
/proc/loadavg: 1.85 1.68 0.91 3/109 23355
/proc/meminfo: memFree=1572440/2059040 swapFree=4139332/4192956
[pid=23352] ppid=23350 vsize=8700 CPUtime=0
/proc/23352/stat : 23352 (run) S 23350 23352 23192 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 175229972 8908800 248 1992294400 4194304 4922060 140735328128128 18446744073709551615 273567816837 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/23352/statm: 2175 248 208 178 0 68 0
[pid=23353] ppid=23352 vsize=40592 CPUtime=2.02
/proc/23353/stat : 23353 (pb_cplex) S 23352 23352 23192 0 -1 4202496 5468 0 0 0 199 3 0 0 17 0 2 0 175229972 41566208 3996 1992294400 134512640 146736320 4287765328 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/23353/statm: 10148 3996 987 2985 0 6690 0
[pid=23353/tid=23355] ppid=23352 vsize=40592 CPUtime=0.6
/proc/23353/task/23355/stat : 23355 (pb_cplex) R 23352 23352 23192 0 -1 4202560 743 0 0 0 60 0 0 0 17 0 2 0 175230048 41566208 3996 1992294400 134512640 146736320 4287765328 18446744073709551615 136400557 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 2.02
Current children cumulated vsize (KiB) 49292

[startup+3.10079 s]
/proc/loadavg: 1.85 1.68 0.91 3/109 23355
/proc/meminfo: memFree=1571324/2059040 swapFree=4139332/4192956
[pid=23352] ppid=23350 vsize=8700 CPUtime=0
/proc/23352/stat : 23352 (run) S 23350 23352 23192 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 175229972 8908800 248 1992294400 4194304 4922060 140735328128128 18446744073709551615 273567816837 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/23352/statm: 2175 248 208 178 0 68 0
[pid=23353] ppid=23352 vsize=34560 CPUtime=4.64
/proc/23353/stat : 23353 (pb_cplex) R 23352 23352 23192 0 -1 4202496 6217 0 0 0 460 4 0 0 18 0 1 0 175229972 35389440 3117 1992294400 134512640 146736320 4287765328 18446744073709551615 135999691 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/23353/statm: 8640 3117 991 2985 0 5182 0
Current children cumulated CPU time (s) 4.64
Current children cumulated vsize (KiB) 43260

[startup+6.30072 s]
/proc/loadavg: 1.78 1.66 0.91 2/109 23357
/proc/meminfo: memFree=1571820/2059040 swapFree=4139332/4192956
[pid=23352] ppid=23350 vsize=8700 CPUtime=0
/proc/23352/stat : 23352 (run) S 23350 23352 23192 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 175229972 8908800 248 1992294400 4194304 4922060 140735328128128 18446744073709551615 273567816837 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/23352/statm: 2175 248 208 178 0 68 0
[pid=23353] ppid=23352 vsize=40744 CPUtime=8.7
/proc/23353/stat : 23353 (pb_cplex) R 23352 23352 23192 0 -1 4202496 8937 0 0 0 863 7 0 0 25 0 2 0 175229972 41721856 4443 1992294400 134512640 146736320 4287765328 18446744073709551615 135468677 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0
/proc/23353/statm: 10186 4443 1000 2985 0 6728 0
[pid=23353/tid=23357] ppid=23352 vsize=40744 CPUtime=0.73
/proc/23353/task/23357/stat : 23357 (pb_cplex) R 23352 23352 23192 0 -1 4202560 611 0 0 0 73 0 0 0 16 0 2 0 175230452 41721856 4443 1992294400 134512640 146736320 4287765328 18446744073709551615 136384849 0 2 4096 16387 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 8.7
Current children cumulated vsize (KiB) 49444

[startup+12.7016 s]
/proc/loadavg: 1.72 1.65 0.91 3/109 23357
/proc/meminfo: memFree=1568472/2059040 swapFree=4139332/4192956
[pid=23352] ppid=23350 vsize=8700 CPUtime=0
/proc/23352/stat : 23352 (run) S 23350 23352 23192 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 175229972 8908800 248 1992294400 4194304 4922060 140735328128128 18446744073709551615 273567816837 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/23352/statm: 2175 248 208 178 0 68 0
[pid=23353] ppid=23352 vsize=39860 CPUtime=18.74
/proc/23353/stat : 23353 (pb_cplex) R 23352 23352 23192 0 -1 4202496 9871 0 0 0 1865 9 0 0 25 0 2 0 175229972 40816640 4357 1992294400 134512640 146736320 4287765328 18446744073709551615 137412871 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0
/proc/23353/statm: 9965 4357 1000 2985 0 6507 0
[pid=23353/tid=23357] ppid=23352 vsize=39860 CPUtime=4.41
/proc/23353/task/23357/stat : 23357 (pb_cplex) R 23352 23352 23192 0 -1 4202560 1249 0 0 0 440 1 0 0 17 0 2 0 175230452 40816640 4357 1992294400 134512640 146736320 4287765328 18446744073709551615 136403116 0 2 4096 16387 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 18.74
Current children cumulated vsize (KiB) 48560

[startup+25.5003 s]
/proc/loadavg: 1.78 1.67 0.93 3/109 23357
/proc/meminfo: memFree=1568224/2059040 swapFree=4139332/4192956
[pid=23352] ppid=23350 vsize=8700 CPUtime=0
/proc/23352/stat : 23352 (run) S 23350 23352 23192 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 175229972 8908800 248 1992294400 4194304 4922060 140735328128128 18446744073709551615 273567816837 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/23352/statm: 2175 248 208 178 0 68 0
[pid=23353] ppid=23352 vsize=40336 CPUtime=42.36
/proc/23353/stat : 23353 (pb_cplex) R 23352 23352 23192 0 -1 4202496 17623 0 0 0 4216 20 0 0 17 0 2 0 175229972 41304064 4444 1992294400 134512640 146736320 4287765328 18446744073709551615 136017787 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0
/proc/23353/statm: 10084 4444 1000 2985 0 6626 0
[pid=23353/tid=23357] ppid=23352 vsize=40336 CPUtime=15.8
/proc/23353/task/23357/stat : 23357 (pb_cplex) S 23352 23352 23192 0 -1 4202560 6815 0 0 0 1573 7 0 0 18 0 2 0 175230452 41304064 4444 1992294400 134512640 146736320 4287765328 18446744073709551615 144580073 0 2 4096 16387 18446604437845402048 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 42.36
Current children cumulated vsize (KiB) 49036

[startup+51.1008 s]
/proc/loadavg: 1.92 1.71 0.96 3/109 23357
/proc/meminfo: memFree=1568348/2059040 swapFree=4139332/4192956
[pid=23352] ppid=23350 vsize=8700 CPUtime=0
/proc/23352/stat : 23352 (run) S 23350 23352 23192 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 175229972 8908800 248 1992294400 4194304 4922060 140735328128128 18446744073709551615 273567816837 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/23352/statm: 2175 248 208 178 0 68 0
[pid=23353] ppid=23352 vsize=39860 CPUtime=89.79
/proc/23353/stat : 23353 (pb_cplex) R 23352 23352 23192 0 -1 4202496 33405 0 0 0 8938 41 0 0 16 0 2 0 175229972 40816640 4422 1992294400 134512640 146736320 4287765328 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/23353/statm: 9965 4422 1000 2985 0 6507 0
[pid=23353/tid=23357] ppid=23352 vsize=39860 CPUtime=40.32
/proc/23353/task/23357/stat : 23357 (pb_cplex) R 23352 23352 23192 0 -1 4202560 21492 0 0 0 4012 20 0 0 19 0 2 0 175230452 40816640 4422 1992294400 134512640 146736320 4287765328 18446744073709551615 136400484 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 89.79
Current children cumulated vsize (KiB) 48560

[startup+102.309 s]
/proc/loadavg: 2.04 1.77 1.02 4/109 23359
/proc/meminfo: memFree=1568224/2059040 swapFree=4139332/4192956
[pid=23352] ppid=23350 vsize=8700 CPUtime=0
/proc/23352/stat : 23352 (run) S 23350 23352 23192 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 175229972 8908800 248 1992294400 4194304 4922060 140735328128128 18446744073709551615 273567816837 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/23352/statm: 2175 248 208 178 0 68 0
[pid=23353] ppid=23352 vsize=40260 CPUtime=184.45
/proc/23353/stat : 23353 (pb_cplex) R 23352 23352 23192 0 -1 4202496 59931 0 0 0 18364 81 0 0 16 0 2 0 175229972 41226240 4481 1992294400 134512640 146736320 4287765328 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/23353/statm: 10065 4481 1000 2985 0 6607 0
[pid=23353/tid=23357] ppid=23352 vsize=40260 CPUtime=89.39
/proc/23353/task/23357/stat : 23357 (pb_cplex) R 23352 23352 23192 0 -1 4202560 39236 0 0 0 8899 40 0 0 19 0 2 0 175230452 41226240 4481 1992294400 134512640 146736320 4287765328 18446744073709551615 139042908 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 184.45
Current children cumulated vsize (KiB) 48960

[startup+162.3 s]
/proc/loadavg: 2.16 1.85 1.09 3/109 23361
/proc/meminfo: memFree=1567976/2059040 swapFree=4139332/4192956
[pid=23352] ppid=23350 vsize=8700 CPUtime=0
/proc/23352/stat : 23352 (run) S 23350 23352 23192 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 175229972 8908800 248 1992294400 4194304 4922060 140735328128128 18446744073709551615 273567816837 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/23352/statm: 2175 248 208 178 0 68 0
[pid=23353] ppid=23352 vsize=40068 CPUtime=296.06
/proc/23353/stat : 23353 (pb_cplex) R 23352 23352 23192 0 -1 4202496 94789 0 0 0 29476 130 0 0 18 0 2 0 175229972 41029632 4440 1992294400 134512640 146736320 4287765328 18446744073709551615 139046545 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/23353/statm: 10017 4440 1000 2985 0 6559 0
[pid=23353/tid=23357] ppid=23352 vsize=40068 CPUtime=147
/proc/23353/task/23357/stat : 23357 (pb_cplex) R 23352 23352 23192 0 -1 4202560 42404 0 0 0 14642 58 0 0 19 0 2 0 175230452 41029632 4440 1992294400 134512640 146736320 4287765328 18446744073709551615 139023058 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 296.06
Current children cumulated vsize (KiB) 48768

[startup+222.3 s]
/proc/loadavg: 1.98 1.84 1.13 3/109 23363
/proc/meminfo: memFree=1568100/2059040 swapFree=4139332/4192956
[pid=23352] ppid=23350 vsize=8700 CPUtime=0
/proc/23352/stat : 23352 (run) S 23350 23352 23192 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 175229972 8908800 248 1992294400 4194304 4922060 140735328128128 18446744073709551615 273567816837 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/23352/statm: 2175 248 208 178 0 68 0
[pid=23353] ppid=23352 vsize=40068 CPUtime=389.41
/proc/23353/stat : 23353 (pb_cplex) R 23352 23352 23192 0 -1 4202496 131063 0 0 0 38766 175 0 0 16 0 2 0 175229972 41029632 4500 1992294400 134512640 146736320 4287765328 18446744073709551615 136018931 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/23353/statm: 10017 4500 1007 2985 0 6559 0
[pid=23353/tid=23357] ppid=23352 vsize=40068 CPUtime=185.51
/proc/23353/task/23357/stat : 23357 (pb_cplex) R 23352 23352 23192 0 -1 4202560 44001 0 0 0 18479 72 0 0 25 0 2 0 175230452 41029632 4500 1992294400 134512640 146736320 4287765328 18446744073709551615 136015707 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 389.41
Current children cumulated vsize (KiB) 48768

[startup+282.301 s]
/proc/loadavg: 2.14 1.90 1.19 3/109 23363
/proc/meminfo: memFree=1567728/2059040 swapFree=4139332/4192956
[pid=23352] ppid=23350 vsize=8700 CPUtime=0
/proc/23352/stat : 23352 (run) S 23350 23352 23192 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 175229972 8908800 248 1992294400 4194304 4922060 140735328128128 18446744073709551615 273567816837 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/23352/statm: 2175 248 208 178 0 68 0
[pid=23353] ppid=23352 vsize=40072 CPUtime=499.67
/proc/23353/stat : 23353 (pb_cplex) R 23352 23352 23192 0 -1 4202496 162005 0 0 0 49745 222 0 0 16 0 2 0 175229972 41033728 4499 1992294400 134512640 146736320 4287765328 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/23353/statm: 10018 4499 1007 2985 0 6560 0
[pid=23353/tid=23357] ppid=23352 vsize=40072 CPUtime=242.85
/proc/23353/task/23357/stat : 23357 (pb_cplex) R 23352 23352 23192 0 -1 4202560 44783 0 0 0 24195 90 0 0 18 0 2 0 175230452 41033728 4499 1992294400 134512640 146736320 4287765328 18446744073709551615 136008065 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 499.67
Current children cumulated vsize (KiB) 48772

[startup+342.301 s]
/proc/loadavg: 2.23 1.96 1.25 2/109 23365
/proc/meminfo: memFree=1567728/2059040 swapFree=4139332/4192956
[pid=23352] ppid=23350 vsize=8700 CPUtime=0
/proc/23352/stat : 23352 (run) S 23350 23352 23192 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 175229972 8908800 248 1992294400 4194304 4922060 140735328128128 18446744073709551615 273567816837 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/23352/statm: 2175 248 208 178 0 68 0
[pid=23353] ppid=23352 vsize=40072 CPUtime=610.48

################
# More data... #
################

[pid=23353] ppid=23352 vsize=40072 CPUtime=721.07
/proc/23353/stat : 23353 (pb_cplex) S 23352 23352 23192 0 -1 4202496 221028 0 0 0 71801 306 0 0 16 0 2 0 175229972 41033728 4530 1992294400 134512640 146736320 4287765328 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/23353/statm: 10018 4530 1007 2985 0 6560 0
[pid=23353/tid=23357] ppid=23352 vsize=40072 CPUtime=357.16
/proc/23353/task/23357/stat : 23357 (pb_cplex) R 23352 23352 23192 0 -1 4202560 86696 0 0 0 35577 139 0 0 19 0 2 0 175230452 41033728 4530 1992294400 134512640 146736320 4287765328 18446744073709551615 136382497 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 721.07
Current children cumulated vsize (KiB) 48772

[startup+462.3 s]
/proc/loadavg: 2.01 1.99 1.35 3/109 23369
/proc/meminfo: memFree=1567728/2059040 swapFree=4139332/4192956
[pid=23352] ppid=23350 vsize=8700 CPUtime=0
/proc/23352/stat : 23352 (run) S 23350 23352 23192 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 175229972 8908800 248 1992294400 4194304 4922060 140735328128128 18446744073709551615 273567816837 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/23352/statm: 2175 248 208 178 0 68 0
[pid=23353] ppid=23352 vsize=40072 CPUtime=831.99
/proc/23353/stat : 23353 (pb_cplex) R 23352 23352 23192 0 -1 4202496 253069 0 0 0 82847 352 0 0 17 0 2 0 175229972 41033728 4526 1992294400 134512640 146736320 4287765328 18446744073709551615 136008042 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/23353/statm: 10018 4526 1007 2985 0 6560 0
[pid=23353/tid=23357] ppid=23352 vsize=40072 CPUtime=415.19
/proc/23353/task/23357/stat : 23357 (pb_cplex) R 23352 23352 23192 0 -1 4202560 88571 0 0 0 41364 155 0 0 19 0 2 0 175230452 41033728 4526 1992294400 134512640 146736320 4287765328 18446744073709551615 135443527 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 831.99
Current children cumulated vsize (KiB) 48772

[startup+522.301 s]
/proc/loadavg: 2.11 2.01 1.39 3/109 23369
/proc/meminfo: memFree=1567728/2059040 swapFree=4139332/4192956
[pid=23352] ppid=23350 vsize=8700 CPUtime=0
/proc/23352/stat : 23352 (run) S 23350 23352 23192 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 175229972 8908800 248 1992294400 4194304 4922060 140735328128128 18446744073709551615 273567816837 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/23352/statm: 2175 248 208 178 0 68 0
[pid=23353] ppid=23352 vsize=40028 CPUtime=942.77
/proc/23353/stat : 23353 (pb_cplex) S 23352 23352 23192 0 -1 4202496 274976 0 0 0 93890 387 0 0 16 0 2 0 175229972 40988672 4542 1992294400 134512640 146736320 4287765328 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/23353/statm: 10007 4542 1028 2985 0 6549 0
[pid=23353/tid=23357] ppid=23352 vsize=40028 CPUtime=472.15
/proc/23353/task/23357/stat : 23357 (pb_cplex) R 23352 23352 23192 0 -1 4202560 88818 0 0 0 47046 169 0 0 22 0 2 0 175230452 40988672 4542 1992294400 134512640 146736320 4287765328 18446744073709551615 135982862 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 942.77
Current children cumulated vsize (KiB) 48728

[startup+582.301 s]
/proc/loadavg: 1.94 1.97 1.42 3/109 23371
/proc/meminfo: memFree=1567604/2059040 swapFree=4139332/4192956
[pid=23352] ppid=23350 vsize=8700 CPUtime=0
/proc/23352/stat : 23352 (run) S 23350 23352 23192 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 175229972 8908800 248 1992294400 4194304 4922060 140735328128128 18446744073709551615 273567816837 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/23352/statm: 2175 248 208 178 0 68 0
[pid=23353] ppid=23352 vsize=40136 CPUtime=1053.46
/proc/23353/stat : 23353 (pb_cplex) R 23352 23352 23192 0 -1 4202496 301769 0 0 0 104917 429 0 0 18 0 2 0 175229972 41099264 4533 1992294400 134512640 146736320 4287765328 18446744073709551615 136008266 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/23353/statm: 10034 4533 1028 2985 0 6576 0
[pid=23353/tid=23357] ppid=23352 vsize=40136 CPUtime=528.25
/proc/23353/task/23357/stat : 23357 (pb_cplex) R 23352 23352 23192 0 -1 4202560 105007 0 0 0 52633 192 0 0 20 0 2 0 175230452 41099264 4533 1992294400 134512640 146736320 4287765328 18446744073709551615 136008245 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1053.46
Current children cumulated vsize (KiB) 48836

[startup+642.3 s]
/proc/loadavg: 2.22 2.02 1.47 3/109 23373
/proc/meminfo: memFree=1567604/2059040 swapFree=4139332/4192956
[pid=23352] ppid=23350 vsize=8700 CPUtime=0
/proc/23352/stat : 23352 (run) S 23350 23352 23192 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 175229972 8908800 248 1992294400 4194304 4922060 140735328128128 18446744073709551615 273567816837 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/23352/statm: 2175 248 208 178 0 68 0
[pid=23353] ppid=23352 vsize=40136 CPUtime=1163.99
/proc/23353/stat : 23353 (pb_cplex) R 23352 23352 23192 0 -1 4202496 340101 0 0 0 115924 475 0 0 16 0 2 0 175229972 41099264 4551 1992294400 134512640 146736320 4287765328 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/23353/statm: 10034 4551 1028 2985 0 6576 0
[pid=23353/tid=23357] ppid=23352 vsize=40136 CPUtime=586.06
/proc/23353/task/23357/stat : 23357 (pb_cplex) R 23352 23352 23192 0 -1 4202560 129746 0 0 0 58388 218 0 0 18 0 2 0 175230452 41099264 4551 1992294400 134512640 146736320 4287765328 18446744073709551615 136403114 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1163.99
Current children cumulated vsize (KiB) 48836

[startup+702.3 s]
/proc/loadavg: 2.26 2.06 1.51 4/109 23375
/proc/meminfo: memFree=1567108/2059040 swapFree=4139332/4192956
[pid=23352] ppid=23350 vsize=8700 CPUtime=0
/proc/23352/stat : 23352 (run) S 23350 23352 23192 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 175229972 8908800 248 1992294400 4194304 4922060 140735328128128 18446744073709551615 273567816837 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/23352/statm: 2175 248 208 178 0 68 0
[pid=23353] ppid=23352 vsize=41160 CPUtime=1273.23
/proc/23353/stat : 23353 (pb_cplex) R 23352 23352 23192 0 -1 4202496 360474 0 0 0 126795 528 0 0 16 0 2 0 175229972 42147840 4705 1992294400 134512640 146736320 4287765328 18446744073709551615 144757825 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/23353/statm: 10290 4705 1028 2985 0 6832 0
[pid=23353/tid=23357] ppid=23352 vsize=41160 CPUtime=644.03
/proc/23353/task/23357/stat : 23357 (pb_cplex) R 23352 23352 23192 0 -1 4202560 143035 0 0 0 64158 245 0 0 18 0 2 0 175230452 42147840 4705 1992294400 134512640 146736320 4287765328 18446744073709551615 136019069 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1273.23
Current children cumulated vsize (KiB) 49860

[startup+762.309 s]
/proc/loadavg: 2.18 2.08 1.55 2/109 23377
/proc/meminfo: memFree=1566984/2059040 swapFree=4139332/4192956
[pid=23352] ppid=23350 vsize=8700 CPUtime=0
/proc/23352/stat : 23352 (run) S 23350 23352 23192 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 175229972 8908800 248 1992294400 4194304 4922060 140735328128128 18446744073709551615 273567816837 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/23352/statm: 2175 248 208 178 0 68 0
[pid=23353] ppid=23352 vsize=40884 CPUtime=1383.72
/proc/23353/stat : 23353 (pb_cplex) S 23352 23352 23192 0 -1 4202496 393575 0 0 0 137788 584 0 0 17 0 2 0 175229972 41865216 4687 1992294400 134512640 146736320 4287765328 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/23353/statm: 10221 4687 1028 2985 0 6763 0
[pid=23353/tid=23357] ppid=23352 vsize=40884 CPUtime=701.38
/proc/23353/task/23357/stat : 23357 (pb_cplex) R 23352 23352 23192 0 -1 4202560 152746 0 0 0 69868 270 0 0 19 0 2 0 175230452 41865216 4687 1992294400 134512640 146736320 4287765328 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1383.72
Current children cumulated vsize (KiB) 49584

[startup+822.3 s]
/proc/loadavg: 2.12 2.08 1.58 3/109 23377
/proc/meminfo: memFree=1566860/2059040 swapFree=4139332/4192956
[pid=23352] ppid=23350 vsize=8700 CPUtime=0
/proc/23352/stat : 23352 (run) S 23350 23352 23192 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 175229972 8908800 248 1992294400 4194304 4922060 140735328128128 18446744073709551615 273567816837 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/23352/statm: 2175 248 208 178 0 68 0
[pid=23353] ppid=23352 vsize=40884 CPUtime=1494.14
/proc/23353/stat : 23353 (pb_cplex) R 23352 23352 23192 0 -1 4202496 427261 0 0 0 148782 632 0 0 19 0 2 0 175229972 41865216 4701 1992294400 134512640 146736320 4287765328 18446744073709551615 136017495 0 0 4096 16387 18446744073709551615 0 0 17 1 0 0 0
/proc/23353/statm: 10221 4701 1028 2985 0 6763 0
[pid=23353/tid=23357] ppid=23352 vsize=40884 CPUtime=756.16
/proc/23353/task/23357/stat : 23357 (pb_cplex) R 23352 23352 23192 0 -1 4202560 153952 0 0 0 75327 289 0 0 16 0 2 0 175230452 41865216 4701 1992294400 134512640 146736320 4287765328 18446744073709551615 136403109 0 2 4096 16387 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1494.14
Current children cumulated vsize (KiB) 49584

[startup+882.3 s]
/proc/loadavg: 2.11 2.08 1.61 3/109 23379
/proc/meminfo: memFree=1566736/2059040 swapFree=4139332/4192956
[pid=23352] ppid=23350 vsize=8700 CPUtime=0
/proc/23352/stat : 23352 (run) S 23350 23352 23192 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 175229972 8908800 248 1992294400 4194304 4922060 140735328128128 18446744073709551615 273567816837 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/23352/statm: 2175 248 208 178 0 68 0
[pid=23353] ppid=23352 vsize=40884 CPUtime=1604.53
/proc/23353/stat : 23353 (pb_cplex) R 23352 23352 23192 0 -1 4202496 456953 0 0 0 159770 683 0 0 17 0 2 0 175229972 41865216 4735 1992294400 134512640 146736320 4287765328 18446744073709551615 136403114 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/23353/statm: 10221 4735 1028 2985 0 6763 0
[pid=23353/tid=23357] ppid=23352 vsize=40884 CPUtime=810.02
/proc/23353/task/23357/stat : 23357 (pb_cplex) R 23352 23352 23192 0 -1 4202560 155737 0 0 0 80692 310 0 0 17 0 2 0 175230452 41865216 4735 1992294400 134512640 146736320 4287765328 18446744073709551615 135526785 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1604.53
Current children cumulated vsize (KiB) 49584

[startup+942.301 s]
/proc/loadavg: 2.08 2.08 1.64 3/109 23381
/proc/meminfo: memFree=1566736/2059040 swapFree=4139332/4192956
[pid=23352] ppid=23350 vsize=8700 CPUtime=0
/proc/23352/stat : 23352 (run) S 23350 23352 23192 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 175229972 8908800 248 1992294400 4194304 4922060 140735328128128 18446744073709551615 273567816837 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/23352/statm: 2175 248 208 178 0 68 0
[pid=23353] ppid=23352 vsize=40884 CPUtime=1714.75
/proc/23353/stat : 23353 (pb_cplex) R 23352 23352 23192 0 -1 4202496 489672 0 0 0 170744 731 0 0 16 0 2 0 175229972 41865216 4743 1992294400 134512640 146736320 4287765328 18446744073709551615 136008230 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/23353/statm: 10221 4743 1028 2985 0 6763 0
[pid=23353/tid=23357] ppid=23352 vsize=40884 CPUtime=867.03
/proc/23353/task/23357/stat : 23357 (pb_cplex) R 23352 23352 23192 0 -1 4202560 163176 0 0 0 86374 329 0 0 16 0 2 0 175230452 41865216 4743 1992294400 134512640 146736320 4287765328 18446744073709551615 136403116 0 2 4096 16387 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1714.75
Current children cumulated vsize (KiB) 49584



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+988.91 s]
/proc/loadavg: 2.10 2.09 1.66 2/109 23386
/proc/meminfo: memFree=1566488/2059040 swapFree=4139332/4192956
[pid=23352] ppid=23350 vsize=8700 CPUtime=0
/proc/23352/stat : 23352 (run) S 23350 23352 23192 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 175229972 8908800 248 1992294400 4194304 4922060 140735328128128 18446744073709551615 273567816837 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/23352/statm: 2175 248 208 178 0 68 0
[pid=23353] ppid=23352 vsize=40884 CPUtime=1800.13
/proc/23353/stat : 23353 (pb_cplex) S 23352 23352 23192 0 -1 4202496 515611 0 0 0 179242 771 0 0 16 0 2 0 175229972 41865216 4755 1992294400 134512640 146736320 4287765328 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/23353/statm: 10221 4755 1028 2985 0 6763 0
[pid=23353/tid=23357] ppid=23352 vsize=40884 CPUtime=911.26
/proc/23353/task/23357/stat : 23357 (pb_cplex) R 23352 23352 23192 0 -1 4202560 174457 0 0 0 90779 347 0 0 19 0 2 0 175230452 41865216 4755 1992294400 134512640 146736320 4287765328 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1800.13
Current children cumulated vsize (KiB) 49584

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+988.91 s]
/proc/loadavg: 2.10 2.09 1.66 2/109 23386
/proc/meminfo: memFree=1566488/2059040 swapFree=4139332/4192956
[pid=23352] ppid=23350 vsize=8700 CPUtime=0
/proc/23352/stat : 23352 (run) S 23350 23352 23192 0 -1 4202496 364 0 0 0 0 0 0 0 21 0 1 0 175229972 8908800 248 1992294400 4194304 4922060 140735328128128 18446744073709551615 273567816837 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/23352/statm: 2175 248 208 178 0 68 0
[pid=23353] ppid=23352 vsize=40884 CPUtime=1800.13
/proc/23353/stat : 23353 (pb_cplex) S 23352 23352 23192 0 -1 4202496 515611 0 0 0 179242 771 0 0 16 0 2 0 175229972 41865216 4755 1992294400 134512640 146736320 4287765328 18446744073709551615 144580073 0 0 4096 16387 18446744073709551615 0 0 17 0 0 0 0
/proc/23353/statm: 10221 4755 1028 2985 0 6763 0
[pid=23353/tid=23357] ppid=23352 vsize=40884 CPUtime=911.26
/proc/23353/task/23357/stat : 23357 (pb_cplex) R 23352 23352 23192 0 -1 4202560 174457 0 0 0 90779 347 0 0 19 0 2 0 175230452 41865216 4755 1992294400 134512640 146736320 4287765328 18446744073709551615 144757825 0 2 4096 16387 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1800.13
Current children cumulated vsize (KiB) 49584

Child status: 0
Real time (s): 988.928
CPU time (s): 1800.15
CPU user time (s): 1792.43
CPU system time (s): 7.71783
CPU usage (%): 182.031
Max. virtual memory (cumulated for all children) (KiB): 58980

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1792.43
system time used= 7.71783
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 515985
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 67195
involuntary context switches= 29295

runsolver used 1.80773 second user time and 5.40518 second system time

The end

Launcher Data 

Begin job on node018 at 2010-07-01 22:22:16
IDJOB=2696271
IDBENCH=1365
IDSOLVER=1209
FILE ID=node018/2696271-1278015736
PBS_JOBID= 11198738
Free space on /tmp= 62436 MiB

SOLVER NAME= pb_cplex 2010-06-29
BENCH NAME= PB06//final/normalized-PB06/SATUNSAT-SMALLINT/submitted-PB06/namasivayam/tsp/normalized-t3002.11tsp11.1900550720.opb
COMMAND LINE= run BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-2696271-1278015736/watcher-2696271-1278015736 -o /tmp/evaluation-result-2696271-1278015736/solver-2696271-1278015736 -C 1800 -W 2000 -M 1800  run HOME/instance-2696271-1278015736.opb

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 1800 MiB
MAX NB THREAD= 0

MD5SUM BENCH= 76a48b8845ca731b7df3e7e82d604015
RANDOM SEED=828175314

node018.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.248
cache size	: 2048 KB
physical id	: 0
siblings	: 1
core id		: 0
cpu cores	: 1
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc pni monitor ds_cpl cid cx16 xtpr
bogomips	: 6000.49
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.248
cache size	: 2048 KB
physical id	: 3
siblings	: 1
core id		: 0
cpu cores	: 1
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5599.47
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:      2059040 kB
MemFree:       1584152 kB
Buffers:        117164 kB
Cached:         256128 kB
SwapCached:       6180 kB
Active:         165656 kB
Inactive:       220868 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:      2059040 kB
LowFree:       1584152 kB
SwapTotal:     4192956 kB
SwapFree:      4139332 kB
Dirty:           16624 kB
Writeback:           0 kB
AnonPages:       11696 kB
Mapped:          11880 kB
Slab:            66484 kB
PageTables:       4168 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   5222476 kB
Committed_AS:   181976 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    264948 kB
VmallocChunk: 34359471699 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 62420 MiB
End job on node018 at 2010-07-01 22:38:45