Trace number 1724461

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
HydraSAT 2009-03-22-BaseUNSAT 277.404 277.5

General information on the benchmark

Namemanol-pipe-c6nidw_i.cnf
MD5SUM15c4413e436aecbd4fcb1d7037e18abf
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark44.8372
Satisfiable
(Un)Satisfiability was proved
Number of variables95863
Number of clauses283363
Sum of the clauses size661179
Maximum clause length3
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 2188908
Number of clauses of size 394454
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

277.29/277.48	
277.29/277.48	c decisions : 5338171
277.29/277.48	c conflicts : 634054
277.29/277.48	c Memory limit 1620 MiB
277.29/277.48	c RANDOM SEED: 934436822
277.29/277.48	s UNSATISFIABLE

Verifier Data 

No possible verification on an UNSAT instance

Watcher Data 

runsolver version 3.2.9 (svn:492) (c) roussel@cril.univ-artois.fr

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-1724461-1240696331/watcher-1724461-1240696331 -o /tmp/evaluation-result-1724461-1240696331/solver-1724461-1240696331 -C 1200 -W 1800 -M 1800 --output-limit 1,15 HOME/hydrasat HOME/instance-1724461-1240696331.cnf -rs 934436822 -co memory_limit 1800 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1200 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1230 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 1843200 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 1894400 KiB
Solver output will be limited to a maximum of 15728640 bytes. The first 1048576 bytes and the last 14680064 bytes will be preserved
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.91 1.84 1.47 3/64 21560
/proc/meminfo: memFree=1680288/2055920 swapFree=4181092/4192956
[pid=21560] ppid=21558 vsize=2876 CPUtime=0
/proc/21560/stat : 21560 (hydrasat) R 21558 21560 21241 0 -1 4194304 222 0 0 0 0 0 0 0 18 0 1 0 97666627 2945024 206 1992294400 134512640 135997824 4294956160 18446744073709551615 135431166 0 0 4096 1 0 0 0 17 1 0 0
/proc/21560/statm: 719 207 151 362 0 354 0

[startup+0.0176511 s]
/proc/loadavg: 1.91 1.84 1.47 3/64 21560
/proc/meminfo: memFree=1680288/2055920 swapFree=4181092/4192956
[pid=21560] ppid=21558 vsize=3272 CPUtime=0.01
/proc/21560/stat : 21560 (hydrasat) R 21558 21560 21241 0 -1 4194304 346 0 0 0 1 0 0 0 18 0 1 0 97666627 3350528 330 1992294400 134512640 135997824 4294956160 18446744073709551615 135371350 0 0 4096 1 0 0 0 17 1 0 0
/proc/21560/statm: 818 331 151 362 0 453 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 3272

[startup+0.101661 s]
/proc/loadavg: 1.91 1.84 1.47 3/64 21560
/proc/meminfo: memFree=1680288/2055920 swapFree=4181092/4192956
[pid=21560] ppid=21558 vsize=6176 CPUtime=0.09
/proc/21560/stat : 21560 (hydrasat) R 21558 21560 21241 0 -1 4194304 1140 0 0 0 9 0 0 0 18 0 1 0 97666627 6324224 1124 1992294400 134512640 135997824 4294956160 18446744073709551615 135371350 0 0 4096 1 0 0 0 17 1 0 0
/proc/21560/statm: 1544 1125 151 362 0 1179 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 6176

[startup+0.301687 s]
/proc/loadavg: 1.91 1.84 1.47 3/64 21560
/proc/meminfo: memFree=1680288/2055920 swapFree=4181092/4192956
[pid=21560] ppid=21558 vsize=13040 CPUtime=0.29
/proc/21560/stat : 21560 (hydrasat) R 21558 21560 21241 0 -1 4194304 3030 0 0 0 28 1 0 0 19 0 1 0 97666627 13352960 3014 1992294400 134512640 135997824 4294956160 18446744073709551615 135371350 0 0 4096 1 0 0 0 17 1 0 0
/proc/21560/statm: 3260 3014 151 362 0 2895 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 13040

[startup+0.701739 s]
/proc/loadavg: 1.91 1.84 1.47 3/64 21560
/proc/meminfo: memFree=1680288/2055920 swapFree=4181092/4192956
[pid=21560] ppid=21558 vsize=26556 CPUtime=0.69
/proc/21560/stat : 21560 (hydrasat) R 21558 21560 21241 0 -1 4194304 6619 0 0 0 66 3 0 0 22 0 1 0 97666627 27193344 6044 1992294400 134512640 135997824 4294956160 18446744073709551615 135371350 0 0 4096 1 0 0 0 17 1 0 0
/proc/21560/statm: 6639 6044 159 362 0 6274 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 26556

[startup+1.50286 s]
/proc/loadavg: 1.91 1.84 1.47 2/65 21561
/proc/meminfo: memFree=1650456/2055920 swapFree=4181092/4192956
[pid=21560] ppid=21558 vsize=40488 CPUtime=1.49
/proc/21560/stat : 21560 (hydrasat) R 21558 21560 21241 0 -1 4194304 10473 0 0 0 143 6 0 0 25 0 1 0 97666627 41459712 9512 1992294400 134512640 135997824 4294956160 18446744073709551615 135371350 0 0 4096 1 0 0 0 17 1 0 0
/proc/21560/statm: 10122 9512 159 362 0 9757 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 40488

[startup+3.10105 s]
/proc/loadavg: 1.91 1.84 1.47 2/65 21561
/proc/meminfo: memFree=1639640/2055920 swapFree=4181092/4192956
[pid=21560] ppid=21558 vsize=63900 CPUtime=3.08
/proc/21560/stat : 21560 (hydrasat) R 21558 21560 21241 0 -1 4194304 17276 0 0 0 299 9 0 0 25 0 1 0 97666627 65433600 15145 1992294400 134512640 135997824 4294956160 18446744073709551615 134590879 0 0 4096 1 0 0 0 17 1 0 0
/proc/21560/statm: 15975 15145 174 362 0 15610 0
Current children cumulated CPU time (s) 3.08
Current children cumulated vsize (KiB) 63900

[startup+6.30148 s]
/proc/loadavg: 1.83 1.83 1.46 2/65 21561
/proc/meminfo: memFree=1618584/2055920 swapFree=4181092/4192956
[pid=21560] ppid=21558 vsize=65380 CPUtime=6.29
/proc/21560/stat : 21560 (hydrasat) R 21558 21560 21241 0 -1 4194304 17657 0 0 0 619 10 0 0 25 0 1 0 97666627 66949120 15526 1992294400 134512640 135997824 4294956160 18446744073709551615 134757664 0 0 4096 1 0 0 0 17 1 0 0
/proc/21560/statm: 16345 15526 174 362 0 15980 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 65380

[startup+12.7013 s]
/proc/loadavg: 1.77 1.81 1.46 2/65 21561
/proc/meminfo: memFree=1617624/2055920 swapFree=4181092/4192956
[pid=21560] ppid=21558 vsize=66452 CPUtime=12.68
/proc/21560/stat : 21560 (hydrasat) R 21558 21560 21241 0 -1 4194304 17909 0 0 0 1258 10 0 0 25 0 1 0 97666627 68046848 15778 1992294400 134512640 135997824 4294956160 18446744073709551615 134850374 0 0 4096 1 0 0 0 17 1 0 0
/proc/21560/statm: 16613 15778 174 362 0 16248 0
Current children cumulated CPU time (s) 12.68
Current children cumulated vsize (KiB) 66452

[startup+25.502 s]
/proc/loadavg: 1.60 1.77 1.45 2/65 21561
/proc/meminfo: memFree=1615832/2055920 swapFree=4181092/4192956
[pid=21560] ppid=21558 vsize=68236 CPUtime=25.48
/proc/21560/stat : 21560 (hydrasat) R 21558 21560 21241 0 -1 4194304 18347 0 0 0 2537 11 0 0 25 0 1 0 97666627 69873664 16216 1992294400 134512640 135997824 4294956160 18446744073709551615 134843936 0 0 4096 1 0 0 0 17 1 0 0
/proc/21560/statm: 17059 16216 174 362 0 16694 0
Current children cumulated CPU time (s) 25.48
Current children cumulated vsize (KiB) 68236

[startup+51.1014 s]
/proc/loadavg: 1.39 1.71 1.44 2/65 21561
/proc/meminfo: memFree=1610392/2055920 swapFree=4181092/4192956
[pid=21560] ppid=21558 vsize=74020 CPUtime=51.06
/proc/21560/stat : 21560 (hydrasat) R 21558 21560 21241 0 -1 4194304 19787 0 0 0 5094 12 0 0 25 0 1 0 97666627 75796480 17656 1992294400 134512640 135997824 4294956160 18446744073709551615 134843469 0 0 4096 1 0 0 0 17 1 0 0
/proc/21560/statm: 18505 17656 174 362 0 18140 0
Current children cumulated CPU time (s) 51.06
Current children cumulated vsize (KiB) 74020

[startup+102.305 s]
/proc/loadavg: 1.17 1.60 1.41 2/65 21561
/proc/meminfo: memFree=1606552/2055920 swapFree=4181092/4192956
[pid=21560] ppid=21558 vsize=77840 CPUtime=102.25
/proc/21560/stat : 21560 (hydrasat) R 21558 21560 21241 0 -1 4194304 20691 0 0 0 10212 13 0 0 25 0 1 0 97666627 79708160 18560 1992294400 134512640 135997824 4294956160 18446744073709551615 134757372 0 0 4096 1 0 0 0 17 1 0 0
/proc/21560/statm: 19460 18560 174 362 0 19095 0
Current children cumulated CPU time (s) 102.25
Current children cumulated vsize (KiB) 77840

[startup+162.302 s]
/proc/loadavg: 1.06 1.49 1.39 2/65 21561
/proc/meminfo: memFree=1604504/2055920 swapFree=4181092/4192956
[pid=21560] ppid=21558 vsize=79952 CPUtime=162.23
/proc/21560/stat : 21560 (hydrasat) R 21558 21560 21241 0 -1 4194304 21185 0 0 0 16209 14 0 0 25 0 1 0 97666627 81870848 19054 1992294400 134512640 135997824 4294956160 18446744073709551615 134817232 0 0 4096 1 0 0 0 17 1 0 0
/proc/21560/statm: 19988 19054 174 362 0 19623 0
Current children cumulated CPU time (s) 162.23
Current children cumulated vsize (KiB) 79952

[startup+222.302 s]
/proc/loadavg: 1.02 1.39 1.36 2/65 21561
/proc/meminfo: memFree=1597720/2055920 swapFree=4181092/4192956
[pid=21560] ppid=21558 vsize=86792 CPUtime=222.22
/proc/21560/stat : 21560 (hydrasat) R 21558 21560 21241 0 -1 4194304 22868 0 0 0 22206 16 0 0 25 0 1 0 97666627 88875008 20737 1992294400 134512640 135997824 4294956160 18446744073709551615 134664906 0 0 4096 1 0 0 0 17 1 0 0
/proc/21560/statm: 21698 20737 174 362 0 21333 0
Current children cumulated CPU time (s) 222.22
Current children cumulated vsize (KiB) 86792

Solver just ended. Dumping a history of the last processes samples

[startup+230.302 s]
/proc/loadavg: 1.02 1.38 1.35 2/65 21561
/proc/meminfo: memFree=1597464/2055920 swapFree=4181092/4192956
[pid=21560] ppid=21558 vsize=87056 CPUtime=230.21
/proc/21560/stat : 21560 (hydrasat) R 21558 21560 21241 0 -1 4194304 22938 0 0 0 23005 16 0 0 25 0 1 0 97666627 89145344 20807 1992294400 134512640 135997824 4294956160 18446744073709551615 134816983 0 0 4096 1 0 0 0 17 1 0 0
/proc/21560/statm: 21764 20807 174 362 0 21399 0
Current children cumulated CPU time (s) 230.21
Current children cumulated vsize (KiB) 87056

[startup+255.901 s]
/proc/loadavg: 1.01 1.35 1.34 2/65 21561
/proc/meminfo: memFree=1596632/2055920 swapFree=4181092/4192956
[pid=21560] ppid=21558 vsize=87856 CPUtime=255.8
/proc/21560/stat : 21560 (hydrasat) R 21558 21560 21241 0 -1 4194304 23138 0 0 0 25564 16 0 0 25 0 1 0 97666627 89964544 21007 1992294400 134512640 135997824 4294956160 18446744073709551615 134843428 0 0 4096 1 0 0 0 17 1 0 0
/proc/21560/statm: 21964 21007 174 362 0 21599 0
Current children cumulated CPU time (s) 255.8
Current children cumulated vsize (KiB) 87856

[startup+268.702 s]
/proc/loadavg: 1.01 1.33 1.34 2/65 21561
/proc/meminfo: memFree=1596312/2055920 swapFree=4181092/4192956
[pid=21560] ppid=21558 vsize=88120 CPUtime=268.6
/proc/21560/stat : 21560 (hydrasat) R 21558 21560 21241 0 -1 4194304 23214 0 0 0 26844 16 0 0 25 0 1 0 97666627 90234880 21083 1992294400 134512640 135997824 4294956160 18446744073709551615 134757582 0 0 4096 1 0 0 0 17 1 0 0
/proc/21560/statm: 22030 21083 174 362 0 21665 0
Current children cumulated CPU time (s) 268.6
Current children cumulated vsize (KiB) 88120

[startup+271.901 s]
/proc/loadavg: 1.01 1.33 1.34 2/65 21561
/proc/meminfo: memFree=1596248/2055920 swapFree=4181092/4192956
[pid=21560] ppid=21558 vsize=88252 CPUtime=271.79
/proc/21560/stat : 21560 (hydrasat) R 21558 21560 21241 0 -1 4194304 23230 0 0 0 27163 16 0 0 25 0 1 0 97666627 90370048 21099 1992294400 134512640 135997824 4294956160 18446744073709551615 134590879 0 0 4096 1 0 0 0 17 1 0 0
/proc/21560/statm: 22063 21099 174 362 0 21698 0
Current children cumulated CPU time (s) 271.79
Current children cumulated vsize (KiB) 88252

[startup+275.103 s]
/proc/loadavg: 1.01 1.33 1.34 2/65 21561
/proc/meminfo: memFree=1596184/2055920 swapFree=4181092/4192956
[pid=21560] ppid=21558 vsize=88252 CPUtime=275
/proc/21560/stat : 21560 (hydrasat) R 21558 21560 21241 0 -1 4194304 23246 0 0 0 27484 16 0 0 25 0 1 0 97666627 90370048 21115 1992294400 134512640 135997824 4294956160 18446744073709551615 134757372 0 0 4096 1 0 0 0 17 1 0 0
/proc/21560/statm: 22063 21115 174 362 0 21698 0
Current children cumulated CPU time (s) 275
Current children cumulated vsize (KiB) 88252

[startup+276.701 s]
/proc/loadavg: 1.01 1.33 1.34 2/65 21561
/proc/meminfo: memFree=1596120/2055920 swapFree=4181092/4192956
[pid=21560] ppid=21558 vsize=88252 CPUtime=276.59
/proc/21560/stat : 21560 (hydrasat) R 21558 21560 21241 0 -1 4194304 23258 0 0 0 27643 16 0 0 25 0 1 0 97666627 90370048 21127 1992294400 134512640 135997824 4294956160 18446744073709551615 134682699 0 0 4096 1 0 0 0 17 1 0 0
/proc/21560/statm: 22063 21127 174 362 0 21698 0
Current children cumulated CPU time (s) 276.59
Current children cumulated vsize (KiB) 88252

[startup+277.101 s]
/proc/loadavg: 1.01 1.33 1.34 2/65 21561
/proc/meminfo: memFree=1596120/2055920 swapFree=4181092/4192956
[pid=21560] ppid=21558 vsize=88252 CPUtime=277
/proc/21560/stat : 21560 (hydrasat) R 21558 21560 21241 0 -1 4194304 23258 0 0 0 27684 16 0 0 25 0 1 0 97666627 90370048 21127 1992294400 134512640 135997824 4294956160 18446744073709551615 134757627 0 0 4096 1 0 0 0 17 1 0 0
/proc/21560/statm: 22063 21127 174 362 0 21698 0
Current children cumulated CPU time (s) 277
Current children cumulated vsize (KiB) 88252

[startup+277.301 s]
/proc/loadavg: 1.01 1.33 1.34 2/65 21561
/proc/meminfo: memFree=1596120/2055920 swapFree=4181092/4192956
[pid=21560] ppid=21558 vsize=88252 CPUtime=277.19
/proc/21560/stat : 21560 (hydrasat) R 21558 21560 21241 0 -1 4194304 23258 0 0 0 27703 16 0 0 25 0 1 0 97666627 90370048 21127 1992294400 134512640 135997824 4294956160 18446744073709551615 134817019 0 0 4096 1 0 0 0 17 1 0 0
/proc/21560/statm: 22063 21127 174 362 0 21698 0
Current children cumulated CPU time (s) 277.19
Current children cumulated vsize (KiB) 88252

[startup+277.401 s]
/proc/loadavg: 1.01 1.33 1.34 2/65 21561
/proc/meminfo: memFree=1596120/2055920 swapFree=4181092/4192956
[pid=21560] ppid=21558 vsize=88252 CPUtime=277.29
/proc/21560/stat : 21560 (hydrasat) R 21558 21560 21241 0 -1 4194304 23258 0 0 0 27713 16 0 0 25 0 1 0 97666627 90370048 21127 1992294400 134512640 135997824 4294956160 18446744073709551615 134757315 0 0 4096 1 0 0 0 17 1 0 0
/proc/21560/statm: 22063 21127 174 362 0 21698 0
Current children cumulated CPU time (s) 277.29
Current children cumulated vsize (KiB) 88252

Child status: 20
Real time (s): 277.5
CPU time (s): 277.404
CPU user time (s): 277.223
CPU system time (s): 0.180972
CPU usage (%): 99.9653
Max. virtual memory (cumulated for all children) (KiB): 88252

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 277.223
system time used= 0.180972
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 23450
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2
involuntary context switches= 740

runsolver used 0.270958 second user time and 0.501923 second system time

The end

Launcher Data 

Begin job on node13 at 2009-04-25 23:52:11
IDJOB=1724461
IDBENCH=71780
IDSOLVER=520
FILE ID=node13/1724461-1240696331
PBS_JOBID= 9186720
Free space on /tmp= 66228 MiB

SOLVER NAME= HydraSAT 2009-03-22-Base
BENCH NAME= SAT_RACE06/manol-pipe-c6nidw_i.cnf
COMMAND LINE= DIR/hydrasat BENCHNAME -rs RANDOMSEED -co memory_limit MEMLIMIT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-1724461-1240696331/watcher-1724461-1240696331 -o /tmp/evaluation-result-1724461-1240696331/solver-1724461-1240696331 -C 1200 -W 1800 -M 1800 --output-limit 1,15  HOME/hydrasat HOME/instance-1724461-1240696331.cnf -rs 934436822 -co memory_limit 1800

TIME LIMIT= 1200 seconds
MEMORY LIMIT= 1800 MiB
MAX NB THREAD= 0

MD5SUM BENCH= 15c4413e436aecbd4fcb1d7037e18abf
RANDOM SEED=934436822

node13.alineos.net Linux 2.6.9-22.EL.rootsmp #1 SMP Mon Oct 3 08:59:52 CEST 2005

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.227
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5914.62
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.227
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5586.94
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:      2055920 kB
MemFree:       1680704 kB
Buffers:         41484 kB
Cached:         260160 kB
SwapCached:       5580 kB
Active:          98312 kB
Inactive:       211324 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:      2055920 kB
LowFree:       1680704 kB
SwapTotal:     4192956 kB
SwapFree:      4181092 kB
Dirty:            6140 kB
Writeback:           0 kB
Mapped:          13468 kB
Slab:            51524 kB
Committed_AS:   311536 kB
PageTables:       1460 kB
VmallocTotal: 536870911 kB
VmallocUsed:    264952 kB
VmallocChunk: 536605679 kB
HugePages_Total:     0
HugePages_Free:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 66228 MiB
End job on node13 at 2009-04-25 23:56:49