Trace number 3380876

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-industrial (fixed)? 39494.8 5000.75

General information on the benchmark

Nameapplication/jarvisalo/HWMCC10-combination-timeframe-expansion/
hwmcc10-timeframe-expansion-k45-pdtvisns3p02-tseitin.cnf
MD5SUMa0ec1baa890da62d0c6df7f1e62c6af3
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark2133.18
Satisfiable
(Un)Satisfiability was proved
Number of variables163622
Number of clauses488120
Sum of the clauses size1138944
Maximum clause length3
Minimum clause length1
Number of clauses of size 12
Number of clauses of size 2325412
Number of clauses of size 3162706
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.38/0.66	c INFO: loaded portfolio from HOME/etc/borg-mix+class.industrial.pickle
0.38/0.66	c INFO: solving HOME/instance-3380876-1305494107.cnf
0.38/0.66	c INFO: computing task features
1.86/2.13	c INFO: computed variable-clause graph statistics
1.86/2.19	c INFO: computed clause balance statistics
9.45/9.72	c INFO: computed variable balance statistics
14.95/15.29	c INFO: computed small-clause counts
15.07/15.40	c INFO: computed Horn-clause counts
45.97/46.23	c INFO: computed variable graph statistics
45.97/46.23	c INFO: collected features for HOME/instance-3380876-1305494107.cnf in 45.52 s
45.97/46.25	c INFO: running lingeling-276@0 for 97 with 4954 remaining (b = 0.51)
45.97/46.28	c INFO: running lingeling-276@0 for 1261 with 4954 remaining (b = 0.54)
46.05/46.30	c INFO: running cryptominisat-2.9.0@0 for 485 with 4954 remaining (b = 0.39)
46.05/46.33	c INFO: running SApperloT@0 for 97 with 4954 remaining (b = 0.03)
46.05/46.36	c INFO: running lingeling-276@0 for 1261 with 4954 remaining (b = 0.43)
46.05/46.38	c INFO: running cryptominisat-2.9.0@0 for 1649 with 4954 remaining (b = 0.54)
46.14/46.41	c INFO: running lingeling-276@0 for 1067 with 4954 remaining (b = 0.32)
46.14/46.44	c INFO: running precosat-570@0 for 388 with 4954 remaining (b = 0.23)
846.26/146.65	c INFO: running cryptominisat-2.9.0@0 for 1940 with 4854 remaining (b = 0.49)
847.01/146.71	c INFO: running lingeling-276@0 for 97 with 4853 remaining (b = 0.05)
1607.08/242.08	c INFO: running lingeling-276@94 for 873 with 4758 remaining (b = 0.20)
3200.25/441.89	c INFO: running precosat-570@100 for 97 with 4558 remaining (b = 0.10)
3937.07/534.23	c INFO: running precosat-570@0 for 776 with 4466 remaining (b = 0.33)
4008.81/543.27	c INFO: running cryptominisat-2.9.0@0 for 97 with 4457 remaining (b = 0.07)
4819.77/644.92	c INFO: running precosat-570@585 for 97 with 4355 remaining (b = 0.05)
5596.49/742.30	c INFO: running cryptominisat-2.9.0@0 for 388 with 4258 remaining (b = 0.18)
8657.01/1126.19	c INFO: running lingeling-276@0 for 97 with 3874 remaining (b = 0.04)
8673.76/1128.24	c INFO: running cryptominisat-2.9.0@98 for 1843 with 3872 remaining (b = 0.39)
8723.98/1134.51	c INFO: running precosat-570@682 for 97 with 3866 remaining (b = 0.05)
9474.48/1228.66	c INFO: running precosat-570@0 for 388 with 3772 remaining (b = 0.13)
9499.87/1231.90	c INFO: running cryptominisat-2.9.0@0 for 485 with 3768 remaining (b = 0.14)
10113.03/1308.73	c INFO: running precosat-570@784 for 485 with 3691 remaining (b = 0.12)
10113.03/1308.77	c INFO: running lingeling-276@1068 for 194 with 3691 remaining (b = 0.05)
10284.37/1330.20	c INFO: running precosat-570@0 for 97 with 3670 remaining (b = 0.06)
11100.62/1432.69	c INFO: running cryptominisat-2.9.0@0 for 485 with 3567 remaining (b = 0.11)
11642.20/1500.64	c INFO: running precosat-570@98 for 97 with 3500 remaining (b = 0.03)
12417.90/1598.04	c INFO: running precosat-570@196 for 582 with 3402 remaining (b = 0.13)
12625.77/1624.10	c INFO: running precosat-570@98 for 97 with 3376 remaining (b = 0.03)
13294.76/1708.16	c INFO: running lingeling-276@0 for 97 with 3292 remaining (b = 0.03)
13440.44/1726.47	c INFO: running cryptominisat-2.9.0@1262 for 97 with 3274 remaining (b = 0.03)
13465.17/1729.59	c INFO: running cryptominisat-2.9.0@875 for 1067 with 3271 remaining (b = 0.18)
13975.63/1793.69	c INFO: running precosat-570@2047 for 97 with 3206 remaining (b = 0.03)
14126.96/1812.65	c INFO: running precosat-570@97 for 679 with 3188 remaining (b = 0.12)
14224.23/1824.89	c INFO: running cryptominisat-2.9.0@1364 for 97 with 3175 remaining (b = 0.03)
14783.14/1895.09	c INFO: running precosat-570@2150 for 97 with 3105 remaining (b = 0.02)
14995.94/1921.76	c INFO: running cryptominisat-2.9.0@1464 for 194 with 3078 remaining (b = 0.04)
15027.64/1925.76	c INFO: running precosat-570@0 for 776 with 3074 remaining (b = 0.13)
15542.86/1990.48	c INFO: running cryptominisat-2.9.0@0 for 97 with 3010 remaining (b = 0.02)
16359.94/2093.08	c INFO: running precosat-570@1752 for 485 with 2907 remaining (b = 0.07)
16425.98/2101.37	c INFO: running lingeling-276@1849 for 97 with 2899 remaining (b = 0.01)
16540.67/2115.74	c INFO: running lingeling-276@486 for 97 with 2884 remaining (b = 0.01)
17102.08/2186.21	c INFO: running precosat-570@586 for 194 with 2814 remaining (b = 0.03)
17202.51/2198.82	c INFO: running lingeling-276@2041 for 97 with 2801 remaining (b = 0.02)
17316.42/2213.14	c INFO: running lingeling-276@1068 for 194 with 2787 remaining (b = 0.03)
18009.93/2300.25	c INFO: running glucose@0 for 776 with 2700 remaining (b = 0.09)
18628.64/2377.94	c INFO: running precosat-570@683 for 97 with 2622 remaining (b = 0.02)
18891.40/2410.91	c INFO: running lingeling-276@1166 for 97 with 2589 remaining (b = 0.01)
19436.17/2479.34	c INFO: running precosat-570@876 for 291 with 2521 remaining (b = 0.04)
19538.73/2492.20	c INFO: running precosat-570@195 for 194 with 2508 remaining (b = 0.03)
19690.17/2511.32	c INFO: running lingeling-276@0 for 97 with 2489 remaining (b = 0.01)
20181.51/2572.93	c INFO: running lingeling-276@0 for 194 with 2427 remaining (b = 0.03)
20466.46/2608.71	c INFO: running lingeling-276@479 for 97 with 2391 remaining (b = 0.02)
21160.88/2695.97	c INFO: running lingeling-276@489 for 776 with 2304 remaining (b = 0.10)
21273.90/2710.11	c INFO: running precosat-570@293 for 97 with 2290 remaining (b = 0.02)
21288.17/2711.93	c INFO: running precosat-570@93 for 97 with 2288 remaining (b = 0.02)
21772.29/2772.70	c INFO: running precosat-570@3698 for 291 with 2227 remaining (b = 0.04)
21777.85/2773.44	c INFO: running precosat-570@0 for 291 with 2227 remaining (b = 0.04)
22049.38/2807.51	c INFO: running precosat-570@1073 for 97 with 2193 remaining (b = 0.03)
22055.89/2808.39	c INFO: running cryptominisat-2.9.0@1356 for 97 with 2192 remaining (b = 0.02)
22087.58/2812.35	c INFO: running precosat-570@292 for 97 with 2188 remaining (b = 0.02)
22843.34/2907.26	c INFO: running cryptominisat-2.9.0@1456 for 194 with 2093 remaining (b = 0.05)
22847.18/2907.72	c INFO: running cryptominisat-2.9.0@0 for 1843 with 2092 remaining (b = 0.19)
22856.73/2908.91	c INFO: running precosat-570@2143 for 97 with 2091 remaining (b = 0.01)
23402.15/2977.53	c INFO: running precosat-570@390 for 388 with 2023 remaining (b = 0.05)
23614.76/3004.29	c INFO: running precosat-570@0 for 97 with 1996 remaining (b = 0.01)
24129.07/3068.83	c INFO: running cryptominisat-2.9.0@191 for 1931 with 1931 remaining (b = 0.19)
24158.54/3072.58	c INFO: running cryptominisat-2.9.0@97 for 388 with 1928 remaining (b = 0.04)
24227.09/3081.15	c INFO: running glucose@189 for 97 with 1919 remaining (b = 0.02)
24408.41/3104.00	c INFO: running cryptominisat-2.9.0@1550 for 194 with 1896 remaining (b = 0.04)
24413.97/3104.70	c INFO: running precosat-570@295 for 97 with 1895 remaining (b = 0.01)
25033.86/3182.55	c INFO: running glucosER@0 for 97 with 1818 remaining (b = 0.01)
25172.87/3200.07	c INFO: running lingeling-276@676 for 679 with 1800 remaining (b = 0.07)
25832.28/3282.97	c INFO: running lingeling-276@0 for 194 with 1717 remaining (b = 0.03)
25988.18/3302.55	c INFO: running cryptominisat-2.9.0@1650 for 291 with 1698 remaining (b = 0.06)
26478.95/3364.24	c INFO: running cryptominisat-2.9.0@0 for 97 with 1636 remaining (b = 0.01)
27261.59/3462.67	c INFO: running cryptominisat-2.9.0@487 for 1538 with 1538 remaining (b = 0.13)
27333.95/3471.76	c INFO: running cryptominisat-2.9.0@874 for 1067 with 1528 remaining (b = 0.12)
27343.49/3472.93	c INFO: running lingeling-276@589 for 679 with 1527 remaining (b = 0.07)
27413.46/3481.75	c INFO: running lingeling-276@198 for 1067 with 1518 remaining (b = 0.10)
28305.87/3593.99	c INFO: running lingeling-276@96 for 194 with 1406 remaining (b = 0.02)
29839.39/3786.76	c INFO: running lingeling-276@285 for 97 with 1213 remaining (b = 0.01)
30637.94/3887.15	c INFO: running glucose@379 for 582 with 1113 remaining (b = 0.07)
30857.41/3914.76	c INFO: running precosat-570@485 for 291 with 1085 remaining (b = 0.03)
32748.11/4152.43	c INFO: running lingeling-276@776 for 97 with 848 remaining (b = 0.01)
33205.36/4209.90	c INFO: running precosat-570@1161 for 97 with 790 remaining (b = 0.02)
33522.68/4249.82	c INFO: running lingeling-276@1071 for 97 with 750 remaining (b = 0.01)
34011.03/4311.30	c INFO: running lingeling-276@787 for 97 with 689 remaining (b = 0.01)
34329.20/4351.21	c INFO: running lingeling-276@1169 for 97 with 649 remaining (b = 0.02)
34785.73/4408.69	c INFO: running lingeling-276@887 for 97 with 591 remaining (b = 0.01)
35134.88/4452.60	c INFO: running precosat-570@396 for 388 with 548 remaining (b = 0.04)
35292.36/4472.33	c INFO: running glucose@1060 for 194 with 528 remaining (b = 0.04)
35592.32/4510.08	c INFO: running lingeling-276@984 for 291 with 490 remaining (b = 0.04)
35848.27/4542.26	c INFO: running cryptominisat-2.9.0@1650 for 291 with 458 remaining (b = 0.04)
35931.03/4552.68	c INFO: running lingeling-276@385 for 97 with 447 remaining (b = 0.01)
36697.73/4649.10	c INFO: running glucose@0 for 194 with 351 remaining (b = 0.02)
36817.13/4664.05	c INFO: running glucose@1159 for 97 with 336 remaining (b = 0.03)
37553.48/4756.69	c INFO: running lingeling-276@0 for 243 with 243 remaining (b = 0.02)
37575.74/4759.42	c INFO: running lingeling-276@488 for 97 with 241 remaining (b = 0.01)
38019.55/4815.28	c INFO: running lingeling-276@2052 for 185 with 185 remaining (b = 0.01)
38155.64/4832.38	c INFO: running cryptominisat-2.9.0@2716 for 168 with 168 remaining (b = 0.01)
38241.45/4843.10	c INFO: running precosat-570@979 for 157 with 157 remaining (b = 0.01)
38262.91/4845.85	c INFO: running glucose@191 for 154 with 154 remaining (b = 0.01)
38350.38/4856.81	c INFO: running lingeling-276@779 for 143 with 143 remaining (b = 0.01)
39494.72/5000.70	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data

Sorry, but this trace is too large and cannot be obtained.

Launcher Data 

Begin job on node126 at 2011-05-15 23:15:07
IDJOB=3380876
IDBENCH=82468
IDSOLVER=1864
FILE ID=node126/3380876-1305494107
RUNJOBID= node126-1305487557-12920
PBS_JOBID= 13325392
Free space on /tmp= 73304 MiB

SOLVER NAME= borg-sat 11.03.02-par-industrial (fixed)
BENCH NAME= SAT11/application/jarvisalo/HWMCC10-combination-timeframe-expansion/hwmcc10-timeframe-expansion-k45-pdtvisns3p02-tseitin.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.industrial.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3380876-1305494107/watcher-3380876-1305494107 -o /tmp/evaluation-result-3380876-1305494107/solver-3380876-1305494107 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3380876-1305494107.cnf -seed 115668301 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= a0ec1baa890da62d0c6df7f1e62c6af3
RANDOM SEED=115668301

node126.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      29383540 kB
Buffers:        196192 kB
Cached:        1899844 kB
SwapCached:      49688 kB
Active:         995296 kB
Inactive:      2299580 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      29383540 kB
SwapTotal:    67111528 kB
SwapFree:     41934240 kB
Dirty:           40272 kB
Writeback:           0 kB
AnonPages:     1162584 kB
Mapped:          17252 kB
Slab:           113648 kB
PageTables:      87316 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 133952524 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73268 MiB
End job on node126 at 2011-05-16 00:38:28