Trace number 2695015

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PB/CT 0.1MSAT (TO) 1800.05 1800.01

General information on the benchmark

Name/SOFT-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/
tsp/normalized-t3002.11tsp11.1900670234--soft-0-100-0.wbo
MD5SUM68083219a398085883a8cf7e46688b9f
Bench CategorySOFT-SMALLINT-LIN (only soft constraints, small integers, linear constraints)
Best result obtained on this benchmarkMOPT
Best cost obtained on this benchmark1
Best CPU time to get the best result obtained on this benchmark488.736
Max-Satisfiable
Max-(Un)Satisfiability was proved
Best value of the cost
Optimality of the best cost was proved
Number of variables231
Total number of constraints2707
Number of soft constraints2707
Number of constraints which are clauses2684
Number of constraints which are cardinality constraints (but not clauses)22
Number of constraints which are nor clauses,nor cardinality constraints1
Minimum length of a constraint2
Maximum length of a constraint110
Top cost 134314
Min constraint cost 1
Max constraint cost 100
Sum of constraints costs 134313
Biggest number in a constraint 25
Number of bits of the biggest number in a constraint 5
Biggest sum of numbers in a constraint 641
Number of bits of the biggest sum of numbers10
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Quality of the solution as a function of time


Download the above graph as a PDF file / as an EPS file

Solver Data 

0.00/0.00	c PB/CT 0.1.1 http://www.residual.se/pbct
0.00/0.00	c Based on OpenSMT/CT 20091015
0.00/0.01	c constraints 0/23 clauses 0/2684 units 0
0.00/0.01	c vars 231
0.09/0.11	c 0.094985 20 11 13578
0.09/0.11	o 813
0.09/0.17	c 0.104984 20.2344 103 38489
0.09/0.17	o 511
0.09/0.19	c 0.105983 20.4688 130 43716
0.09/0.19	o 459
0.29/0.35	c 0.122981 20.4688 311 51166
0.29/0.35	o 411
0.39/0.44	c 0.135979 20.4688 446 54996
0.39/0.44	o 282
0.39/0.46	c 0.137979 20.4688 471 56086
0.39/0.46	o 259
0.39/0.49	c 0.141978 20.4688 494 57780
0.39/0.49	o 249
0.59/0.61	c 0.156976 20.4688 650 59555
0.59/0.61	o 226
0.59/0.68	c 0.163975 20.4688 711 59888
0.59/0.68	o 224
1.29/1.34	c 0.299954 20.4688 1392 62457
1.29/1.34	o 220
1.49/1.52	c 0.373943 20.4688 922 64039
1.49/1.52	o 203
1.59/1.63	c 0.407937 20.4688 1107 64849
1.59/1.63	o 201
1.59/1.64	c 0.411937 20.4688 1118 65031
1.59/1.64	o 197
2.39/2.41	c 0.642902 20.4688 893 69620
2.39/2.41	o 190
2.39/2.48	c 0.650901 20.4688 975 70354
2.39/2.48	o 181
2.49/2.57	c 0.662899 20.4688 1088 71129
2.49/2.57	o 171
2.69/2.71	c 0.687895 20.4688 1270 71864
2.69/2.71	o 165
2.89/2.91	c 0.724889 20.4688 894 73188
2.89/2.91	o 152
3.49/3.50	c 0.839872 20.4688 1065 75394
3.49/3.50	o 151
3.59/3.61	c 0.859869 20.4688 615 75969
3.59/3.61	o 147
3.69/3.78	c 0.901862 20.4688 830 77359
3.69/3.78	o 131
4.19/4.25	c 0.98085 20.4688 1350 78859
4.19/4.25	o 130
4.29/4.38	c 1.01185 20.4688 860 79766
4.29/4.38	o 120
4.69/4.78	c 1.09383 20.4688 694 81130
4.69/4.78	o 116
4.89/4.92	c 1.11383 20.4688 874 81660
4.89/4.92	o 114
6.60/6.63	c 1.39879 20.5977 1103 86702
6.60/6.63	o 113
6.99/7.02	c 1.46978 20.5977 931 87943
6.99/7.02	o 110
6.99/7.09	c 1.48377 20.5977 1035 88440
6.99/7.09	o 107
7.09/7.10	c 1.48677 20.5977 1050 88664
7.09/7.10	o 102
8.39/8.49	c 1.88071 20.5977 1037 93666
8.39/8.49	o 101
9.20/9.20	c 2.09168 20.5977 798 96552
9.20/9.20	o 79
9.99/10.04	c 2.27965 20.5977 1103 100015
9.99/10.04	o 69
10.50/10.50	c 2.42163 20.7266 1278 102369
10.50/10.50	o 64
11.29/11.32	c 2.6266 20.7266 1846 105312
11.29/11.32	o 58
11.49/11.55	c 2.69759 20.7266 1156 106123
11.49/11.55	o 56
12.00/12.05	c 2.86256 20.8555 1265 109328
12.00/12.05	o 26
12.50/12.58	c 3.13552 21.2422 2636 113639
12.50/12.58	o 16
12.79/12.86	c 3.3075 21.2422 3313 116518
12.79/12.86	o 14
18.49/18.57	c 7.1989 21.3711 2034 183578
18.49/18.57	o 9
23.00/23.01	c 10.5984 22.1445 3890 240097
23.00/23.01	o 8
36.99/37.02	c 21.0238 22.1445 4008 426452
36.99/37.02	o 5
57.89/57.97	c 37.4463 22.5312 2978 724288
57.89/57.97	o 3
71.19/71.29	c 48.2417 22.5312 3811 912779
71.19/71.29	o 1
1800.03/1800.00	s SATISFIABLE
1800.03/1800.00	v -x1 -x2 x3 -x4 -x5 x6 -x7 -x8 -x9 -x10 -x11 -x12 -x13 -x14 -x15 -x16 -x17 -x18 -x19 -x20 -x21 -x22 -x23 -x24 -x25 -x26 -x27 -x28 -x29 -x30 -x31 -x32 x33 x34 -x35 -x36 -x37 -x38 -x39 -x40 -x41 -x42 -x43 -x44 -x45 -x46 -x47 -x48 -x49 x50 -x51 -x52 -x53 -x54 -x55 -x56 -x57 -x58 x59 -x60 -x61 -x62 -x63 -x64 -x65 -x66 -x67 -x68 -x69 -x70 -x71 -x72 x73 -x74 -x75 -x76 -x77 -x78 -x79 -x80 -x81 -x82 -x83 -x84 -x85 x86 -x87 -x88 -x89 -x90 -x91 -x92 -x93 -x94 -x95 -x96 -x97 x98 -x99 -x100 -x101 -x102 -x103 x104 -x105 -x106 -x107 -x108 -x109 -x110 -x111 -x112 -x113 -x114 -x115 -x116 -x117 x118 -x119 -x120 -x121 -x122 -x123 -x124 -x125 -x126 -x127 -x128 -x129 -x130 x131 -x132 -x133 -x134 -x135 -x136 -x137 x138 -x139 -x140 -x141 -x142 x143 -x144 -x145 -x146 -x147 -x148 -x149 -x150 -x151 -x152 -x153 -x154 -x155 -x156 -x157 -x158 -x159 -x160 -x161 -x162 -x163 -x164 -x165 -x166 -x167 -x168 -x169 x170 -x171 x172 -x173 -x174 -x175 -x176 -x177 -x178 -x179 -x180 -x181 -x182 -x183 -x184 x185 -x186 -x187 -x188 -x189 -x190 -x191 -x192 -x193 -x194 -x195 x196 -x197 -x198 -x199 -x200 -x201 -x202 -x203 -x204 -x205 -x206 -x207 x208 -x209 -x210 -x211 -x212 -x213 -x214 -x215 -x216 -x217 -x218 -x219 x220 -x221 -x222 -x223 x224 -x225 -x226 -x227 -x228 -x229 -x230 -x231

Verifier Data 

OK	1

Watcher Data 

runsolver version 3.2.9 (svn:492) (c) roussel@cril.univ-artois.fr

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-2695015-1277926435/watcher-2695015-1277926435 -o /tmp/evaluation-result-2695015-1277926435/solver-2695015-1277926435 -C 1800 -W 2000 -M 3800 HOME/pbct --model HOME/instance-2695015-1277926435.wbo 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 3891200 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 3942400 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.92 1.98 1.99 3/178 16378
/proc/meminfo: memFree=31413668/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=2136 CPUtime=0
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 254 0 0 0 0 0 0 0 21 0 1 0 133939814 2187264 173 4089446400 134512640 136421960 4291025744 18446744073709551615 135811817 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/16378/statm: 534 173 144 467 0 65 0

[startup+0.06267 s]
/proc/loadavg: 1.92 1.98 1.99 3/178 16378
/proc/meminfo: memFree=31413668/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=14884 CPUtime=0.05
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 3399 0 0 0 5 0 0 0 21 0 1 0 133939814 15241216 3318 4089446400 134512640 136421960 4291025744 18446744073709551615 134706180 0 0 4096 16387 0 0 0 17 3 0 0 0
/proc/16378/statm: 3770 3328 193 467 0 3301 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 14884

[startup+0.100665 s]
/proc/loadavg: 1.92 1.98 1.99 3/178 16378
/proc/meminfo: memFree=31413668/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=20476 CPUtime=0.09
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 4754 0 0 0 9 0 0 0 21 0 1 0 133939814 20967424 4640 4089446400 134512640 136421960 4291025744 18446744073709551615 134868555 0 0 4096 16387 0 0 0 17 3 0 0 0
/proc/16378/statm: 5119 4640 213 467 0 4650 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 20476

[startup+0.300624 s]
/proc/loadavg: 1.92 1.98 1.99 3/178 16378
/proc/meminfo: memFree=31413668/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=20956 CPUtime=0.29
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 4846 0 0 0 11 18 0 0 21 0 1 0 133939814 21458944 4729 4089446400 134512640 136421960 4291025744 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 3 0 0 0
/proc/16378/statm: 5239 4729 236 467 0 4770 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 20956

[startup+0.700553 s]
/proc/loadavg: 1.92 1.98 1.99 3/178 16378
/proc/meminfo: memFree=31413668/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=20956 CPUtime=0.69
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 4853 0 0 0 16 53 0 0 22 0 1 0 133939814 21458944 4730 4089446400 134512640 136421960 4291025744 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 3 0 0 0
/proc/16378/statm: 5239 4730 237 467 0 4770 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 20956

[startup+1.50041 s]
/proc/loadavg: 1.92 1.98 1.99 3/179 16379
/proc/meminfo: memFree=31395556/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=20956 CPUtime=1.49
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 4864 0 0 0 36 113 0 0 25 0 1 0 133939814 21458944 4740 4089446400 134512640 136421960 4291025744 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 3 0 0 0
/proc/16378/statm: 5239 4740 237 467 0 4770 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 20956

[startup+3.10013 s]
/proc/loadavg: 1.92 1.98 1.99 3/179 16379
/proc/meminfo: memFree=31395184/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=20956 CPUtime=3.09
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 4874 0 0 0 75 234 0 0 18 0 1 0 133939814 21458944 4742 4089446400 134512640 136421960 4291025744 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 3 0 0 0
/proc/16378/statm: 5239 4742 237 467 0 4770 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 20956

[startup+6.30056 s]
/proc/loadavg: 1.93 1.98 1.99 3/179 16379
/proc/meminfo: memFree=31395668/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=21088 CPUtime=6.29
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 4905 0 0 0 133 496 0 0 25 0 1 0 133939814 21594112 4766 4089446400 134512640 136421960 4291025744 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 3 0 0 0
/proc/16378/statm: 5272 4766 237 467 0 4803 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 21088

[startup+12.7004 s]
/proc/loadavg: 1.93 1.98 1.99 3/179 16380
/proc/meminfo: memFree=31396400/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=21748 CPUtime=12.69
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 5077 0 0 0 320 949 0 0 25 0 1 0 133939814 22269952 4926 4089446400 134512640 136421960 4291025744 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 3 0 0 0
/proc/16378/statm: 5437 4926 237 467 0 4968 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 21748

[startup+25.5002 s]
/proc/loadavg: 1.95 1.98 1.99 3/179 16380
/proc/meminfo: memFree=31396136/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=22672 CPUtime=25.49
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 5335 0 0 0 1245 1304 0 0 25 0 1 0 133939814 23216128 5181 4089446400 134512640 136421960 4291025744 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 3 0 0 0
/proc/16378/statm: 5668 5181 237 467 0 5199 0
Current children cumulated CPU time (s) 25.49
Current children cumulated vsize (KiB) 22672

[startup+51.1007 s]
/proc/loadavg: 1.96 1.98 1.99 3/179 16381
/proc/meminfo: memFree=31395696/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=23068 CPUtime=51.1
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 5419 0 0 0 3216 1894 0 0 25 0 1 0 133939814 23621632 5264 4089446400 134512640 136421960 4291025744 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 3 0 0 0
/proc/16378/statm: 5767 5264 237 467 0 5298 0
Current children cumulated CPU time (s) 51.1
Current children cumulated vsize (KiB) 23068

[startup+102.306 s]
/proc/loadavg: 1.98 1.98 1.99 3/179 16382
/proc/meminfo: memFree=31395180/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=23068 CPUtime=102.3
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 5438 0 0 0 7361 2869 0 0 25 0 1 0 133939814 23621632 5281 4089446400 134512640 136421960 4291025744 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 3 0 0 0
/proc/16378/statm: 5767 5281 237 467 0 5298 0
Current children cumulated CPU time (s) 102.3
Current children cumulated vsize (KiB) 23068

[startup+162.301 s]
/proc/loadavg: 1.99 1.98 1.99 3/179 16384
/proc/meminfo: memFree=31394512/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=23464 CPUtime=162.3
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 5510 0 0 0 12215 4015 0 0 25 0 1 0 133939814 24027136 5353 4089446400 134512640 136421960 4291025744 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 3 0 0 0
/proc/16378/statm: 5866 5353 237 467 0 5397 0
Current children cumulated CPU time (s) 162.3
Current children cumulated vsize (KiB) 23464

[startup+222.3 s]
/proc/loadavg: 1.99 1.98 1.99 3/179 16385
/proc/meminfo: memFree=31394232/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=23596 CPUtime=222.3
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 5570 0 0 0 17085 5145 0 0 25 0 1 0 133939814 24162304 5413 4089446400 134512640 136421960 4291025744 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 3 0 0 0
/proc/16378/statm: 5899 5413 237 467 0 5430 0
Current children cumulated CPU time (s) 222.3
Current children cumulated vsize (KiB) 23596

[startup+282.301 s]
/proc/loadavg: 1.99 1.98 1.99 3/179 16387
/proc/meminfo: memFree=31394064/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=23860 CPUtime=282.3
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 5630 0 0 0 21989 6241 0 0 25 0 1 0 133939814 24432640 5473 4089446400 134512640 136421960 4291025744 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 3 0 0 0
/proc/16378/statm: 5965 5473 237 467 0 5496 0
Current children cumulated CPU time (s) 282.3
Current children cumulated vsize (KiB) 23860

[startup+342.3 s]
/proc/loadavg: 1.99 1.98 1.99 3/179 16389
/proc/meminfo: memFree=31393892/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=23992 CPUtime=342.3
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 5659 0 0 0 26866 7364 0 0 25 0 1 0 133939814 24567808 5502 4089446400 134512640 136421960 4291025744 18446744073709551615 135026402 0 0 4096 16387 0 0 0 17 3 0 0 0
/proc/16378/statm: 5998 5502 237 467 0 5529 0
Current children cumulated CPU time (s) 342.3
Current children cumulated vsize (KiB) 23992

[startup+402.301 s]
/proc/loadavg: 2.04 2.00 2.00 3/179 16417
/proc/meminfo: memFree=31354872/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=24124 CPUtime=402.3
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 5684 0 0 0 31966 8264 0 0 25 0 1 0 133939814 24702976 5527 4089446400 134512640 136421960 4291025744 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/16378/statm: 6031 5527 237 467 0 5562 0
Current children cumulated CPU time (s) 402.3
Current children cumulated vsize (KiB) 24124

[startup+462.301 s]
/proc/loadavg: 2.01 2.00 2.00 3/179 16419
/proc/meminfo: memFree=31167080/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=24124 CPUtime=462.31
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 5696 0 0 0 37519 8712 0 0 25 0 1 0 133939814 24702976 5539 4089446400 134512640 136421960 4291025744 18446744073709551615 135006147 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/16378/statm: 6031 5539 237 467 0 5562 0
Current children cumulated CPU time (s) 462.31
Current children cumulated vsize (KiB) 24124

[startup+522.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/179 16420
/proc/meminfo: memFree=31000768/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=24240 CPUtime=522.31
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 5743 0 0 0 43061 9170 0 0 25 0 1 0 133939814 24821760 5577 4089446400 134512640 136421960 4291025744 18446744073709551615 134861890 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/16378/statm: 6060 5577 237 467 0 5591 0
Current children cumulated CPU time (s) 522.31
Current children cumulated vsize (KiB) 24240

[startup+582.301 s]
/proc/loadavg: 2.00 2.00 2.00 3/197 16467
/proc/meminfo: memFree=31296980/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=24496 CPUtime=582.3
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 5824 0 0 0 48229 10001 0 0 25 0 1 0 133939814 25083904 5633 4089446400 134512640 136421960 4291025744 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 7 0 0 0
/proc/16378/statm: 6124 5633 237 467 0 5655 0
Current children cumulated CPU time (s) 582.3
Current children cumulated vsize (KiB) 24496

[startup+642.301 s]
/proc/loadavg: 2.00 2.00 2.00 3/197 16469
/proc/meminfo: memFree=31295676/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=24496 CPUtime=642.3
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 5824 0 0 0 53347 10883 0 0 25 0 1 0 133939814 25083904 5633 4089446400 134512640 136421960 4291025744 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 7 0 0 0
/proc/16378/statm: 6124 5633 237 467 0 5655 0
Current children cumulated CPU time (s) 642.3
Current children cumulated vsize (KiB) 24496

[startup+702.3 s]

################
# More data... #
################

[startup+822.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/197 16473
/proc/meminfo: memFree=31293348/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=24760 CPUtime=822.31
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 5875 0 0 0 68513 13718 0 0 25 0 1 0 133939814 25354240 5684 4089446400 134512640 136421960 4291025744 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 7 0 0 0
/proc/16378/statm: 6190 5684 237 467 0 5721 0
Current children cumulated CPU time (s) 822.31
Current children cumulated vsize (KiB) 24760

[startup+882.301 s]
/proc/loadavg: 2.00 2.00 2.00 3/197 16475
/proc/meminfo: memFree=31293012/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=24860 CPUtime=882.31
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 5934 0 0 0 73431 14800 0 0 25 0 1 0 133939814 25456640 5732 4089446400 134512640 136421960 4291025744 18446744073709551615 134861958 0 0 4096 16387 0 0 0 17 7 0 0 0
/proc/16378/statm: 6215 5732 237 467 0 5746 0
Current children cumulated CPU time (s) 882.31
Current children cumulated vsize (KiB) 24860

[startup+942.301 s]
/proc/loadavg: 2.07 2.02 2.00 3/197 16477
/proc/meminfo: memFree=31292452/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=24860 CPUtime=942.31
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 5934 0 0 0 78401 15830 0 0 25 0 1 0 133939814 25456640 5732 4089446400 134512640 136421960 4291025744 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 7 0 0 0
/proc/16378/statm: 6215 5732 237 467 0 5746 0
Current children cumulated CPU time (s) 942.31
Current children cumulated vsize (KiB) 24860

[startup+1002.3 s]
/proc/loadavg: 2.02 2.01 2.00 3/197 16478
/proc/meminfo: memFree=31292124/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=24860 CPUtime=1002.31
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 5934 0 0 0 83296 16935 0 0 25 0 1 0 133939814 25456640 5732 4089446400 134512640 136421960 4291025744 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 7 0 0 0
/proc/16378/statm: 6215 5732 237 467 0 5746 0
Current children cumulated CPU time (s) 1002.31
Current children cumulated vsize (KiB) 24860

[startup+1062.3 s]
/proc/loadavg: 2.01 2.00 2.00 3/179 16508
/proc/meminfo: memFree=31395360/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=24860 CPUtime=1062.31
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 5934 0 0 0 88382 17849 0 0 25 0 1 0 133939814 25456640 5732 4089446400 134512640 136421960 4291025744 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 7 0 0 0
/proc/16378/statm: 6215 5732 237 467 0 5746 0
Current children cumulated CPU time (s) 1062.31
Current children cumulated vsize (KiB) 24860

[startup+1122.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/179 16509
/proc/meminfo: memFree=31393532/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=24992 CPUtime=1122.31
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 5964 0 0 0 93472 18759 0 0 25 0 1 0 133939814 25591808 5762 4089446400 134512640 136421960 4291025744 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 7 0 0 0
/proc/16378/statm: 6248 5762 237 467 0 5779 0
Current children cumulated CPU time (s) 1122.31
Current children cumulated vsize (KiB) 24992

[startup+1182.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/179 16511
/proc/meminfo: memFree=31393248/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=25124 CPUtime=1182.32
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 5997 0 0 0 98576 19656 0 0 25 0 1 0 133939814 25726976 5795 4089446400 134512640 136421960 4291025744 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 7 0 0 0
/proc/16378/statm: 6281 5795 237 467 0 5812 0
Current children cumulated CPU time (s) 1182.32
Current children cumulated vsize (KiB) 25124

[startup+1242.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/179 16513
/proc/meminfo: memFree=31392708/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=25112 CPUtime=1242.31
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 6011 0 0 0 103686 20545 0 0 25 0 1 0 133939814 25714688 5795 4089446400 134512640 136421960 4291025744 18446744073709551615 135020642 0 0 4096 16387 0 0 0 17 7 0 0 0
/proc/16378/statm: 6278 5795 237 467 0 5809 0
Current children cumulated CPU time (s) 1242.31
Current children cumulated vsize (KiB) 25112

[startup+1302.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/179 16514
/proc/meminfo: memFree=31392796/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=25244 CPUtime=1302.32
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 6039 0 0 0 108762 21470 0 0 25 0 1 0 133939814 25849856 5823 4089446400 134512640 136421960 4291025744 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 7 0 0 0
/proc/16378/statm: 6311 5823 237 467 0 5842 0
Current children cumulated CPU time (s) 1302.32
Current children cumulated vsize (KiB) 25244

[startup+1362.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/179 16516
/proc/meminfo: memFree=31392748/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=25244 CPUtime=1362.32
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 6039 0 0 0 113881 22351 0 0 25 0 1 0 133939814 25849856 5823 4089446400 134512640 136421960 4291025744 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 7 0 0 0
/proc/16378/statm: 6311 5823 237 467 0 5842 0
Current children cumulated CPU time (s) 1362.32
Current children cumulated vsize (KiB) 25244

[startup+1422.31 s]
/proc/loadavg: 2.00 2.00 2.00 3/179 16517
/proc/meminfo: memFree=31392468/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=25376 CPUtime=1422.32
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 6047 0 0 0 118975 23257 0 0 25 0 1 0 133939814 25985024 5831 4089446400 134512640 136421960 4291025744 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 7 0 0 0
/proc/16378/statm: 6344 5831 237 467 0 5875 0
Current children cumulated CPU time (s) 1422.32
Current children cumulated vsize (KiB) 25376

[startup+1482.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/179 16519
/proc/meminfo: memFree=31392316/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=25364 CPUtime=1482.32
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 6063 0 0 0 124079 24153 0 0 25 0 1 0 133939814 25972736 5841 4089446400 134512640 136421960 4291025744 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 7 0 0 0
/proc/16378/statm: 6341 5841 237 467 0 5872 0
Current children cumulated CPU time (s) 1482.32
Current children cumulated vsize (KiB) 25364

[startup+1542.3 s]
/proc/loadavg: 2.03 2.01 2.00 3/179 16521
/proc/meminfo: memFree=31392404/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=25364 CPUtime=1542.32
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 6063 0 0 0 129147 25085 0 0 25 0 1 0 133939814 25972736 5841 4089446400 134512640 136421960 4291025744 18446744073709551615 134862312 0 0 4096 16387 0 0 0 17 7 0 0 0
/proc/16378/statm: 6341 5841 237 467 0 5872 0
Current children cumulated CPU time (s) 1542.32
Current children cumulated vsize (KiB) 25364

[startup+1602.3 s]
/proc/loadavg: 2.01 2.01 2.00 3/179 16522
/proc/meminfo: memFree=31392128/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=25496 CPUtime=1602.33
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 6082 0 0 0 134244 25989 0 0 25 0 1 0 133939814 26107904 5860 4089446400 134512640 136421960 4291025744 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 7 0 0 0
/proc/16378/statm: 6374 5860 237 467 0 5905 0
Current children cumulated CPU time (s) 1602.33
Current children cumulated vsize (KiB) 25496

[startup+1662.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/179 16533
/proc/meminfo: memFree=31391368/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=25496 CPUtime=1662.33
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 6099 0 0 0 139343 26890 0 0 25 0 1 0 133939814 26107904 5877 4089446400 134512640 136421960 4291025744 18446744073709551615 135032319 0 0 4096 16387 0 0 0 17 7 0 0 0
/proc/16378/statm: 6374 5877 237 467 0 5905 0
Current children cumulated CPU time (s) 1662.33
Current children cumulated vsize (KiB) 25496

[startup+1722.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/179 16534
/proc/meminfo: memFree=31391440/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=25496 CPUtime=1722.33
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 6099 0 0 0 144442 27791 0 0 25 0 1 0 133939814 26107904 5877 4089446400 134512640 136421960 4291025744 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 7 0 0 0
/proc/16378/statm: 6374 5877 237 467 0 5905 0
Current children cumulated CPU time (s) 1722.33
Current children cumulated vsize (KiB) 25496

[startup+1782.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/179 16536
/proc/meminfo: memFree=31391284/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=25628 CPUtime=1782.33
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 6131 0 0 0 149533 28700 0 0 25 0 1 0 133939814 26243072 5909 4089446400 134512640 136421960 4291025744 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 7 0 0 0
/proc/16378/statm: 6407 5909 237 467 0 5938 0
Current children cumulated CPU time (s) 1782.33
Current children cumulated vsize (KiB) 25628



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1800 s]
/proc/loadavg: 2.00 2.00 2.00 3/179 16536
/proc/meminfo: memFree=31391152/32950928 swapFree=4192956/4192956
[pid=16378] ppid=16376 vsize=25628 CPUtime=1800.03
/proc/16378/stat : 16378 (pbct) R 16376 16378 14276 0 -1 4202496 6131 0 0 0 151013 28990 0 0 25 0 1 0 133939814 26243072 5909 4089446400 134512640 136421960 4291025744 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 7 0 0 0
/proc/16378/statm: 6407 5909 237 467 0 5938 0
Current children cumulated CPU time (s) 1800.03
Current children cumulated vsize (KiB) 25628

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child status: 0
Real time (s): 1800.01
CPU time (s): 1800.05
CPU user time (s): 1510.14
CPU system time (s): 289.911
CPU usage (%): 100.002
Max. virtual memory (cumulated for all children) (KiB): 25628

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1510.14
system time used= 289.911
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 6132
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2
involuntary context switches= 4714

runsolver used 2.53761 second user time and 5.67514 second system time

The end

Launcher Data 

Begin job on node124 at 2010-06-30 21:33:55
IDJOB=2695015
IDBENCH=79269
IDSOLVER=1208
FILE ID=node124/2695015-1277926435
PBS_JOBID= 11197343
Free space on /tmp= 130620 MiB

SOLVER NAME= PB/CT 0.1
BENCH NAME= PB10/normalized-PB10/SOFT-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/tsp/normalized-t3002.11tsp11.1900670234--soft-0-100-0.wbo
COMMAND LINE= HOME/pbct --model BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-2695015-1277926435/watcher-2695015-1277926435 -o /tmp/evaluation-result-2695015-1277926435/solver-2695015-1277926435 -C 1800 -W 2000 -M 3800  HOME/pbct --model HOME/instance-2695015-1277926435.wbo

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 3800 MiB
MAX NB THREAD= 0

MD5SUM BENCH= 68083219a398085883a8cf7e46688b9f
RANDOM SEED=881514806

node124.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      31414436 kB
Buffers:        213756 kB
Cached:        1122312 kB
SwapCached:          0 kB
Active:         575612 kB
Inactive:       845924 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      31414436 kB
SwapTotal:     4192956 kB
SwapFree:      4192956 kB
Dirty:            1996 kB
Writeback:           0 kB
AnonPages:       85812 kB
Mapped:          15456 kB
Slab:            81004 kB
PageTables:       4848 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  20668420 kB
Committed_AS:   232104 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    265212 kB
VmallocChunk: 34359471699 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 130620 MiB
End job on node124 at 2010-06-30 22:03:57