Trace number 3483233

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
MiniUnsat2 2011-02-22UNSAT 1.44278 1.44562

General information on the benchmark

NameSAT03/industrial/kukula/addm_bench/
am_4_4.shuffled-as.sat03-360.cnf
MD5SUM3bb259ba186116af9b777352e56b1069
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark0.353945
Satisfiable
(Un)Satisfiability was proved
Number of variables433
Number of clauses1458
Sum of the clauses size3954
Maximum clause length3
Minimum clause length1
Number of clauses of size 12
Number of clauses of size 2416
Number of clauses of size 31040
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data

1.39/1.44	c WARNING: The input formula was NOT proven UNSATISFIABLE, it has simply been ASSUMED to be.
1.39/1.44	s UNSATISFIABLE
1.39/1.44	v 1433 144 223 231 314 326 344 427 428 430 433 482 489 493 496 530 659 684 775 798 838 857 887 979 981 988 1018 1141 1217 1225 1231 1329 1349 2 5 11 21 31 34 36 38 44 45 51 53 63 72 77 82 84 85 94 99 101 106 108 112 115 116 126 127 141 146 147 151 153 160 165 167 174 178 184 196 197 198 199 200 221 222 229 235 238 239 241 245 253 261 265 268 278 286 298 312 318 324 332 333 334 336 341 345 349 351 354 365 369 375 380 381 384 385 388 390 396 397 400 403 407 414 422 431 436 437 438 439 440 446 447 449 453 456 464 468 470 477 486 488 490 492 494 495 498 503 507 510 517 532 535 542 544 547 550 551 554 564 570 576 595 599 601 619 625 627 628 640 644 646 649 661 662 666 669 670 687 689 692 710 716 727 729 736 738 743 751 753 757 759 766 772 773 790 791 793 797 802 807 811 817 824 828 832 836 844 848 851 858 860 861 862 863 870 871 878 881 885 892 893 905 912 915 917 931 949 955 956 959 982 990 993 996 1000 1001 1004 1010 1020 1036 1043 1047 1054 1079 1080 1083 1091 1095 1100 1111 1128 1140 1145 1171 1176 1187 1190 1196 1199 1203 1205 1220 1224 1233 1239 1246 1257 1260 1263 1266 1268 1273 1276 1281 1287 1293 1331 1336 1357 1362 1367 1398 1399 1400 1402 1407 1430 1449 64 425 12 17 26 32 61 175 185 191 213 220 335 339 342 355 460 472 484 497 578 589 593 594 616 660 667 683 688 725 756 776 805 843 880 891 1028 1064 1099 1120 1126 1137 1191 1258 1284 1290 1322 1353 1382 1401 1418 1455 6 86 124 202 226 227 262 276 306 315 352 491 499 555 559 581 665 682 696 835 853 879 902 903 914 930 934 970 1014 1038 1051 1096 1134 1150 1289 1305 1338 13 15 22 27 30 35 41 47 48 52 65 70 76 90 96 107 121 125 129 132 143 152 158 164 168 169 170 177 180 187 188 189 190 192 193 201 204 212 218 224 230 232 233 251 252 256 259 260 263 285 291 293 296 299 300 301 309 311 319 330 362 378 382 383 392 394 398 413 421 434 454 457 462 476 478 480 485 501 516 519 522 534 541 552 557 567 569 575 579 580 583 602 608 610 618 623 626 641 643 651 654 656 657 663 664 668 671 672 675 676 686 694 699 700 702 705 707 708 714 722 728 737 740 741 744 749 752 760 761 762 779 782 789 794 795 803 806 813 818 821 822 823 829 830 837 846 850 856 864 872 883 889 898 911 919 920 922 926 927 932 938 946 950 951 953 957 985 986 987 992 994 1003 1009 1011 1012 1015 1022 1027 1029 1034 1040 1044 1049 1053 1058 1063 1070 1072 1073 1074 1075 1076 1078 1082 1089 1098 1102 1104 1123 1125 1127 1129 1138 1153 1156 1161 1163 1164 1168 1177 1178 1181 1182 1189 1200 1201 1202 1204 1215 1222 1223 1228 1234 1245 1256 1264 1285 1292 1295 1296 1297 1302 1306 1314 1341 1343 1351 1358 1360 1361 1366 1371 1377 1379 1386 1394 1419 1420 1424 1426 1441 1443 1446 1450 1452 3 7 9 24 42 49 71 78 79 89 93 97 104 105 110 114 131 134 137 139 140 149 157 162 172 182 183 194 195 208 215 217 225 248 250 254 264 269 270 279 287 297 302 310 316 317 322 323 328 329 348 353 357 367 371 372 379 402 406 415 423 429 435 445 461 463 469 475 481 504 506 508 514 536 537 546 563 572 573 582 588 592 596 603 606 607 611 614 617 629 630 647 652 680 690 704 720 721 723 733 734 774 781 787 788 792 799 804 809 831 839 849 854 868 873 877 896 901 906 908 921 925 935 941 942 944 947 966 978 991 995 1002 1016 1024 1041 1062 1065 1068 1085 1094 1101 1105 1106 1110 1122 1131 1147 1160 1170 1192 1251 1255 1259 1265 1275 1286 1303 1312 1315 1317 1319 1320 1321 1323 1326 1334 1346 1365 1368 1375 1408 1409 1414 1417 1436 1440 92 145 166 340 612 771 814 819 1227 1380 1391 1405 1413 1457 33 130 133 387 529 560 605 639 712 827 847 875 971 975 1017 1026 1033 1037 1088 1159 1339 1421 271 305 399 416 424 548 556 631 746 748 758 929 1135 1148 1173 1431 87 0

Verifier Data

OK	857

Watcher Data

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 988)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3483233-1307800703/watcher-3483233-1307800703 -o /tmp/evaluation-result-3483233-1307800703/solver-3483233-1307800703 -C 1800 -W 1850 -M 7680 munsat2_release -rnd-seed=737525513 -verb=0 HOME/instance-3483233-1307800703.cnf 

running on 2 cores: 5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1850 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 7864320 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 7915520 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.19 0.82 1.42 4/186 22994
/proc/meminfo: memFree=31749696/32951124 swapFree=67054712/67111528
[pid=22994] ppid=22992 vsize=14988 CPUtime=0 cores=5,7
/proc/22994/stat : 22994 (munsat2_release) R 22992 22994 22928 0 -1 4202496 427 0 0 0 0 0 0 0 22 0 1 0 970300973 15347712 359 33554432000 4194304 4258824 140733417976208 18446744073709551615 216734839493 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/22994/statm: 3747 359 230 16 0 389 0

[startup+0.08728 s]
/proc/loadavg: 0.19 0.82 1.42 4/186 22994
/proc/meminfo: memFree=31749696/32951124 swapFree=67054712/67111528
[pid=22994] ppid=22992 vsize=19496 CPUtime=0.08 cores=5,7
/proc/22994/stat : 22994 (munsat2_release) R 22992 22994 22928 0 -1 4202496 473 0 0 0 8 0 0 0 22 0 1 0 970300973 19963904 401 33554432000 4194304 4258824 140733417976208 18446744073709551615 4218832 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/22994/statm: 4874 401 239 16 0 1516 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 19496

[startup+0.100264 s]
/proc/loadavg: 0.19 0.82 1.42 4/186 22994
/proc/meminfo: memFree=31749696/32951124 swapFree=67054712/67111528
[pid=22994] ppid=22992 vsize=19496 CPUtime=0.09 cores=5,7
/proc/22994/stat : 22994 (munsat2_release) R 22992 22994 22928 0 -1 4202496 473 0 0 0 9 0 0 0 22 0 1 0 970300973 19963904 401 33554432000 4194304 4258824 140733417976208 18446744073709551615 4221133 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/22994/statm: 4874 401 239 16 0 1516 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 19496

[startup+0.300227 s]
/proc/loadavg: 0.19 0.82 1.42 4/186 22994
/proc/meminfo: memFree=31749696/32951124 swapFree=67054712/67111528
[pid=22994] ppid=22992 vsize=19496 CPUtime=0.29 cores=5,7
/proc/22994/stat : 22994 (munsat2_release) R 22992 22994 22928 0 -1 4202496 489 0 0 0 29 0 0 0 23 0 1 0 970300973 19963904 416 33554432000 4194304 4258824 140733417976208 18446744073709551615 4225519 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/22994/statm: 4874 416 239 16 0 1516 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 19496

[startup+0.70016 s]
/proc/loadavg: 0.19 0.82 1.42 4/186 22994
/proc/meminfo: memFree=31749696/32951124 swapFree=67054712/67111528
[pid=22994] ppid=22992 vsize=19628 CPUtime=0.69 cores=5,7
/proc/22994/stat : 22994 (munsat2_release) R 22992 22994 22928 0 -1 4202496 519 0 0 0 69 0 0 0 25 0 1 0 970300973 20099072 446 33554432000 4194304 4258824 140733417976208 18446744073709551615 4221193 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/22994/statm: 4907 446 239 16 0 1549 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 19628

Solver just ended. Dumping a history of the last processes samples

[startup+1.10609 s]
/proc/loadavg: 0.19 0.82 1.42 5/190 23005
/proc/meminfo: memFree=31742372/32951124 swapFree=67054712/67111528
[pid=22994] ppid=22992 vsize=19760 CPUtime=1.1 cores=5,7
/proc/22994/stat : 22994 (munsat2_release) R 22992 22994 22928 0 -1 4202496 566 0 0 0 110 0 0 0 25 0 1 0 970300973 20234240 490 33554432000 4194304 4258824 140733417976208 18446744073709551615 4221157 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/22994/statm: 4940 490 240 16 0 1582 0
Current children cumulated CPU time (s) 1.1
Current children cumulated vsize (KiB) 19760

[startup+1.30105 s]
/proc/loadavg: 0.19 0.82 1.42 5/190 23005
/proc/meminfo: memFree=31742372/32951124 swapFree=67054712/67111528
[pid=22994] ppid=22992 vsize=19760 CPUtime=1.29 cores=5,7
/proc/22994/stat : 22994 (munsat2_release) R 22992 22994 22928 0 -1 4202496 576 0 0 0 129 0 0 0 25 0 1 0 970300973 20234240 500 33554432000 4194304 4258824 140733417976208 18446744073709551615 4218485 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/22994/statm: 4940 500 240 16 0 1582 0
Current children cumulated CPU time (s) 1.29
Current children cumulated vsize (KiB) 19760

[startup+1.40104 s]
/proc/loadavg: 0.19 0.82 1.42 5/190 23005
/proc/meminfo: memFree=31742372/32951124 swapFree=67054712/67111528
[pid=22994] ppid=22992 vsize=19760 CPUtime=1.39 cores=5,7
/proc/22994/stat : 22994 (munsat2_release) R 22992 22994 22928 0 -1 4202496 577 0 0 0 139 0 0 0 25 0 1 0 970300973 20234240 501 33554432000 4194304 4258824 140733417976208 18446744073709551615 4221037 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/22994/statm: 4940 501 240 16 0 1582 0
Current children cumulated CPU time (s) 1.39
Current children cumulated vsize (KiB) 19760

Child status: 0
Real time (s): 1.44562
CPU time (s): 1.44278
CPU user time (s): 1.44278
CPU system time (s): 0
CPU usage (%): 99.8035
Max. virtual memory (cumulated for all children) (KiB): 19760

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1.44278
system time used= 0
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 586
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 4
involuntary context switches= 1

runsolver used 0.003999 second user time and 0.006998 second system time

The end

Launcher Data

Begin job on node119 at 2011-06-11 15:58:23
IDJOB=3483233
IDBENCH=84235
IDSOLVER=1973
FILE ID=node119/3483233-1307800703
RUNJOBID= node119-1307800702-22949
PBS_JOBID= 13551059
Free space on /tmp= 74076 MiB

SOLVER NAME= MiniUnsat2 2011-02-22
BENCH NAME= SAT03/industrial/kukula/addm_bench/am_4_4.shuffled-as.sat03-360.cnf
COMMAND LINE= munsat2_release -rnd-seed=RANDOMSEED -verb=0 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3483233-1307800703/watcher-3483233-1307800703 -o /tmp/evaluation-result-3483233-1307800703/solver-3483233-1307800703 -C 1800 -W 1850 -M 7680  munsat2_release -rnd-seed=737525513 -verb=0 HOME/instance-3483233-1307800703.cnf

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 7680 MiB
NBCORE= 2

MD5SUM BENCH= 3bb259ba186116af9b777352e56b1069
RANDOM SEED=737525513

node119.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.63
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      31750464 kB
Buffers:        272340 kB
Cached:         676648 kB
SwapCached:      11012 kB
Active:         493760 kB
Inactive:       490288 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      31750464 kB
SwapTotal:    67111528 kB
SwapFree:     67054712 kB
Dirty:            2372 kB
Writeback:           0 kB
AnonPages:       32772 kB
Mapped:          13272 kB
Slab:           150348 kB
PageTables:       5980 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   253708 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 74072 MiB
End job on node119 at 2011-06-11 15:58:25