Trace number 2694967

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PB/CT 0.1MSAT (TO) 1800.05 1800.01

General information on the benchmark

Name/PARTIAL-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/tsp/
normalized-t3002.11tsp11.1900577940--soft-66-100-0.wbo
MD5SUM95aa4a8f0430d7fc4dd41a3f578cfb3e
Bench CategoryPARTIAL-SMALLINT-LIN (both soft and hard constraints, small integers, linear constraints)
Best result obtained on this benchmarkMOPT
Best cost obtained on this benchmark1
Best CPU time to get the best result obtained on this benchmark267.837
Max-Satisfiable
Max-(Un)Satisfiability was proved
Best value of the cost
Optimality of the best cost was proved
Number of variables231
Total number of constraints2707
Number of soft constraints937
Number of constraints which are clauses2684
Number of constraints which are cardinality constraints (but not clauses)22
Number of constraints which are nor clauses,nor cardinality constraints1
Minimum length of a constraint2
Maximum length of a constraint110
Top cost 46894
Min constraint cost 1
Max constraint cost 100
Sum of constraints costs 46893
Biggest number in a constraint 25
Number of bits of the biggest number in a constraint 5
Biggest sum of numbers in a constraint 661
Number of bits of the biggest sum of numbers10
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Quality of the solution as a function of time


Download the above graph as a PDF file / as an EPS file

Solver Data 

0.00/0.00	c PB/CT 0.1.1 http://www.residual.se/pbct
0.00/0.00	c Based on OpenSMT/CT 20091015
0.00/0.02	c constraints 14/9 clauses 1756/928 units 0
0.00/0.02	c vars 231
0.02/0.06	c 0.065989 13.3867 23 2964
0.02/0.06	o 339
0.02/0.07	c 0.071989 13.5234 36 3555
0.02/0.07	o 328
6.99/7.07	c 4.33634 13.7695 1612 1387727
6.99/7.07	o 320
89.10/89.14	c 53.8458 13.8984 1349 18493082
89.10/89.14	o 281
89.89/89.90	c 54.2997 13.8984 750 18642378
89.89/89.90	o 274
95.70/95.76	c 57.7852 13.8984 802 19312379
95.70/95.76	o 271
100.90/100.92	c 60.9357 13.8984 914 20362348
100.90/100.92	o 264
100.90/100.95	c 60.9527 13.8984 1071 20363487
100.90/100.95	o 178
100.90/100.99	c 60.9797 13.8984 628 20363800
100.90/100.99	o 174
100.99/101.01	c 60.9987 13.8984 813 20364178
100.99/101.01	o 166
100.99/101.03	c 61.0107 13.8984 902 20364618
100.99/101.03	o 137
100.99/101.04	c 61.0197 13.8984 956 20364851
100.99/101.04	o 124
100.99/101.05	c 61.0257 13.8984 964 20365011
100.99/101.05	o 113
102.59/102.63	c 62.0116 14.0273 1198 20426394
102.59/102.63	o 112
104.79/104.87	c 63.3294 14.0273 727 20554966
104.79/104.87	o 103
104.89/104.91	c 63.3584 14.0273 913 20555416
104.89/104.91	o 93
104.89/104.97	c 63.3954 14.0273 1194 20556127
104.89/104.97	o 67
105.00/105.05	c 63.4634 14.0273 1229 20557111
105.00/105.05	o 66
105.09/105.17	c 63.5603 14.0273 1405 20558416
105.09/105.17	o 65
105.19/105.20	c 63.5773 14.0273 762 20558840
105.19/105.20	o 43
105.39/105.50	c 63.8303 14.0273 1364 20562054
105.39/105.50	o 39
105.50/105.52	c 63.8543 14.0273 1573 20562426
105.50/105.52	o 35
106.30/106.36	c 64.5632 14.1562 1332 20573032
106.30/106.36	o 31
106.30/106.39	c 64.5922 14.1562 1670 20573691
106.30/106.39	o 25
106.70/106.78	c 64.9151 14.1562 1310 20579558
106.70/106.78	o 24
106.99/107.04	c 65.1391 14.1562 1659 20582957
106.99/107.04	o 13
107.09/107.15	c 65.2461 14.1562 1583 20584408
107.09/107.15	o 9
107.20/107.21	c 65.3031 14.1562 1087 20585304
107.20/107.21	o 8
108.10/108.10	c 66.1179 14.1562 1440 20597221
108.10/108.10	o 3
138.20/138.25	c 93.4038 14.7734 1835 21004669
138.20/138.25	o 2
149.50/149.57	c 103.624 14.7734 1732 21158535
149.50/149.57	o 1
1800.04/1800.00	s SATISFIABLE
1800.04/1800.00	v -x1 -x2 -x3 -x4 -x5 -x6 x7 -x8 -x9 -x10 -x11 -x12 -x13 -x14 -x15 x16 -x17 -x18 -x19 -x20 -x21 -x22 x23 -x24 -x25 -x26 -x27 -x28 -x29 -x30 -x31 -x32 -x33 -x34 -x35 -x36 -x37 -x38 -x39 -x40 -x41 -x42 x43 -x44 -x45 -x46 -x47 -x48 x49 -x50 -x51 -x52 -x53 -x54 -x55 -x56 -x57 x58 -x59 -x60 -x61 -x62 -x63 -x64 -x65 -x66 -x67 -x68 -x69 -x70 -x71 -x72 -x73 -x74 -x75 -x76 x77 -x78 x79 -x80 -x81 -x82 -x83 -x84 -x85 -x86 -x87 -x88 -x89 -x90 -x91 -x92 -x93 -x94 x95 -x96 -x97 -x98 -x99 -x100 -x101 -x102 -x103 -x104 -x105 -x106 -x107 x108 -x109 -x110 -x111 -x112 -x113 -x114 -x115 x116 -x117 -x118 -x119 -x120 -x121 -x122 -x123 -x124 -x125 -x126 -x127 x128 -x129 -x130 -x131 -x132 -x133 -x134 -x135 -x136 -x137 -x138 -x139 -x140 x141 -x142 -x143 -x144 x145 -x146 -x147 -x148 -x149 -x150 -x151 -x152 -x153 -x154 -x155 x156 -x157 -x158 -x159 -x160 -x161 -x162 -x163 -x164 -x165 -x166 -x167 -x168 -x169 x170 -x171 -x172 -x173 -x174 -x175 -x176 -x177 -x178 x179 -x180 -x181 -x182 x183 -x184 -x185 -x186 -x187 -x188 -x189 -x190 -x191 -x192 -x193 -x194 -x195 -x196 -x197 -x198 -x199 -x200 -x201 -x202 -x203 -x204 -x205 -x206 -x207 x208 -x209 -x210 -x211 x212 -x213 -x214 -x215 -x216 -x217 -x218 -x219 -x220 -x221 -x222 -x223 x224 -x225 -x226 -x227 -x228 -x229 -x230 -x231

Verifier Data 

OK	1

Watcher Data 

runsolver version 3.2.9 (svn:492) (c) roussel@cril.univ-artois.fr

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-2694967-1277911787/watcher-2694967-1277911787 -o /tmp/evaluation-result-2694967-1277911787/solver-2694967-1277911787 -C 1800 -W 2000 -M 3800 HOME/pbct --model HOME/instance-2694967-1277911787.wbo 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 3891200 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 3942400 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.58 2.71 2.47 4/178 26175
/proc/meminfo: memFree=30638764/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=2136 CPUtime=0
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 252 0 0 0 0 0 0 0 23 0 1 0 166782256 2187264 171 4089446400 134512640 136421960 4293243376 18446744073709551615 135873694 0 0 4096 0 0 0 0 17 0 0 0 0
/proc/26175/statm: 534 171 144 467 0 65 0

[startup+0.0216949 s]
/proc/loadavg: 2.58 2.71 2.47 4/178 26175
/proc/meminfo: memFree=30638764/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=3444 CPUtime=0.02
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 573 0 0 0 2 0 0 0 23 0 1 0 166782256 3526656 492 4089446400 134512640 136421960 4293243376 18446744073709551615 135810640 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 861 494 187 467 0 392 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 3444

[startup+0.100685 s]
/proc/loadavg: 2.58 2.71 2.47 4/178 26175
/proc/meminfo: memFree=30638764/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=13844 CPUtime=0.09
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3112 0 0 0 8 1 0 0 23 0 1 0 166782256 14176256 3029 4089446400 134512640 136421960 4293243376 18446744073709551615 135028864 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 3461 3029 236 467 0 2992 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 13844

[startup+0.300649 s]
/proc/loadavg: 2.58 2.71 2.47 4/178 26175
/proc/meminfo: memFree=30638764/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=13844 CPUtime=0.29
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3112 0 0 0 19 10 0 0 23 0 1 0 166782256 14176256 3029 4089446400 134512640 136421960 4293243376 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 3461 3029 236 467 0 2992 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 13844

[startup+0.700574 s]
/proc/loadavg: 2.58 2.71 2.47 4/178 26175
/proc/meminfo: memFree=30638764/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=13844 CPUtime=0.69
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3125 0 0 0 42 27 0 0 25 0 1 0 166782256 14176256 3042 4089446400 134512640 136421960 4293243376 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 3461 3042 236 467 0 2992 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 13844

[startup+1.50043 s]
/proc/loadavg: 2.53 2.69 2.46 3/179 26176
/proc/meminfo: memFree=30627224/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=13968 CPUtime=1.49
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3153 0 0 0 89 60 0 0 25 0 1 0 166782256 14303232 3070 4089446400 134512640 136421960 4293243376 18446744073709551615 134855961 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 3492 3070 236 467 0 3023 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 13968

[startup+3.10015 s]
/proc/loadavg: 2.53 2.69 2.46 3/179 26176
/proc/meminfo: memFree=30627100/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=13968 CPUtime=3.09
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3159 0 0 0 185 124 0 0 25 0 1 0 166782256 14303232 3076 4089446400 134512640 136421960 4293243376 18446744073709551615 135032470 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 3492 3076 236 467 0 3023 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 13968

[startup+6.30059 s]
/proc/loadavg: 2.53 2.69 2.46 3/179 26176
/proc/meminfo: memFree=30627220/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=14096 CPUtime=6.29
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3192 0 0 0 383 246 0 0 25 0 1 0 166782256 14434304 3109 4089446400 134512640 136421960 4293243376 18446744073709551615 134872614 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 3524 3109 236 467 0 3055 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 14096

[startup+12.7005 s]
/proc/loadavg: 2.45 2.67 2.46 3/179 26176
/proc/meminfo: memFree=30627472/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=14096 CPUtime=12.69
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3194 0 0 0 782 487 0 0 25 0 1 0 166782256 14434304 3110 4089446400 134512640 136421960 4293243376 18446744073709551615 134861902 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 3524 3110 237 467 0 3055 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 14096

[startup+25.5002 s]
/proc/loadavg: 2.38 2.65 2.45 3/179 26176
/proc/meminfo: memFree=30627996/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=14096 CPUtime=25.49
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3194 0 0 0 1550 999 0 0 25 0 1 0 166782256 14434304 3110 4089446400 134512640 136421960 4293243376 18446744073709551615 135005960 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 3524 3110 237 467 0 3055 0
Current children cumulated CPU time (s) 25.49
Current children cumulated vsize (KiB) 14096

[startup+51.1007 s]
/proc/loadavg: 2.25 2.59 2.44 3/179 26177
/proc/meminfo: memFree=30627568/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=14096 CPUtime=51.1
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3202 0 0 0 3069 2041 0 0 25 0 1 0 166782256 14434304 3118 4089446400 134512640 136421960 4293243376 18446744073709551615 135006405 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 3524 3118 237 467 0 3055 0
Current children cumulated CPU time (s) 51.1
Current children cumulated vsize (KiB) 14096

[startup+102.307 s]
/proc/loadavg: 2.15 2.51 2.42 3/179 26206
/proc/meminfo: memFree=30538844/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=14228 CPUtime=102.3
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3230 0 0 0 6180 4050 0 0 20 0 1 0 166782256 14569472 3136 4089446400 134512640 136421960 4293243376 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 3557 3136 237 467 0 3088 0
Current children cumulated CPU time (s) 102.3
Current children cumulated vsize (KiB) 14228

[startup+162.3 s]
/proc/loadavg: 2.05 2.41 2.39 3/179 26208
/proc/meminfo: memFree=30505312/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=15124 CPUtime=162.29
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3466 0 0 0 11520 4709 0 0 25 0 1 0 166782256 15486976 3354 4089446400 134512640 136421960 4293243376 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 3781 3354 237 467 0 3312 0
Current children cumulated CPU time (s) 162.29
Current children cumulated vsize (KiB) 15124

[startup+222.301 s]
/proc/loadavg: 2.02 2.33 2.36 3/179 26210
/proc/meminfo: memFree=30487548/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=15124 CPUtime=222.29
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3485 0 0 0 16954 5275 0 0 25 0 1 0 166782256 15486976 3373 4089446400 134512640 136421960 4293243376 18446744073709551615 135028872 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 3781 3373 237 467 0 3312 0
Current children cumulated CPU time (s) 222.29
Current children cumulated vsize (KiB) 15124

[startup+282.3 s]
/proc/loadavg: 2.00 2.27 2.34 3/179 26212
/proc/meminfo: memFree=30475240/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=15228 CPUtime=282.3
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3504 0 0 0 22385 5845 0 0 25 0 1 0 166782256 15593472 3392 4089446400 134512640 136421960 4293243376 18446744073709551615 134861902 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 3807 3392 237 467 0 3338 0
Current children cumulated CPU time (s) 282.3
Current children cumulated vsize (KiB) 15228

[startup+342.301 s]
/proc/loadavg: 2.07 2.24 2.32 3/179 26213
/proc/meminfo: memFree=30470504/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=15368 CPUtime=342.3
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3540 0 0 0 27827 6403 0 0 25 0 1 0 166782256 15736832 3428 4089446400 134512640 136421960 4293243376 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 3842 3428 237 467 0 3373 0
Current children cumulated CPU time (s) 342.3
Current children cumulated vsize (KiB) 15368

[startup+402.301 s]
/proc/loadavg: 2.02 2.19 2.29 3/179 26214
/proc/meminfo: memFree=30468856/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=15500 CPUtime=402.31
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3561 0 0 0 33240 6991 0 0 25 0 1 0 166782256 15872000 3449 4089446400 134512640 136421960 4293243376 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 3875 3449 237 467 0 3406 0
Current children cumulated CPU time (s) 402.31
Current children cumulated vsize (KiB) 15500

[startup+462.301 s]
/proc/loadavg: 2.01 2.15 2.27 3/179 26216
/proc/meminfo: memFree=30463984/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=15500 CPUtime=462.3
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3578 0 0 0 38677 7553 0 0 25 0 1 0 166782256 15872000 3466 4089446400 134512640 136421960 4293243376 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 3875 3466 237 467 0 3406 0
Current children cumulated CPU time (s) 462.3
Current children cumulated vsize (KiB) 15500

[startup+522.301 s]
/proc/loadavg: 2.00 2.12 2.25 3/179 26218
/proc/meminfo: memFree=30462072/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=15624 CPUtime=522.3
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3597 0 0 0 44115 8115 0 0 25 0 1 0 166782256 15998976 3485 4089446400 134512640 136421960 4293243376 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 3906 3485 237 467 0 3437 0
Current children cumulated CPU time (s) 522.3
Current children cumulated vsize (KiB) 15624

[startup+582.3 s]
/proc/loadavg: 2.00 2.10 2.24 3/179 26220
/proc/meminfo: memFree=30461780/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=15624 CPUtime=582.3
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3609 0 0 0 49568 8662 0 0 25 0 1 0 166782256 15998976 3497 4089446400 134512640 136421960 4293243376 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 3906 3497 237 467 0 3437 0
Current children cumulated CPU time (s) 582.3
Current children cumulated vsize (KiB) 15624

[startup+642.301 s]
/proc/loadavg: 2.00 2.08 2.22 3/179 26221
/proc/meminfo: memFree=30460364/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=15776 CPUtime=642.31
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3632 0 0 0 55019 9212 0 0 25 0 1 0 166782256 16154624 3520 4089446400 134512640 136421960 4293243376 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 3944 3520 237 467 0 3475 0
Current children cumulated CPU time (s) 642.31
Current children cumulated vsize (KiB) 15776

[startup+702.3 s]

################
# More data... #
################

[startup+822.301 s]
/proc/loadavg: 2.00 2.04 2.17 3/179 26226
/proc/meminfo: memFree=30459624/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=15904 CPUtime=822.31
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3668 0 0 0 71407 10824 0 0 25 0 1 0 166782256 16285696 3556 4089446400 134512640 136421960 4293243376 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 3976 3556 237 467 0 3507 0
Current children cumulated CPU time (s) 822.31
Current children cumulated vsize (KiB) 15904

[startup+882.301 s]
/proc/loadavg: 2.00 2.03 2.16 3/179 26228
/proc/meminfo: memFree=30459076/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=15904 CPUtime=882.31
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3679 0 0 0 76870 11361 0 0 25 0 1 0 166782256 16285696 3567 4089446400 134512640 136421960 4293243376 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 3976 3567 237 467 0 3507 0
Current children cumulated CPU time (s) 882.31
Current children cumulated vsize (KiB) 15904

[startup+942.301 s]
/proc/loadavg: 2.04 2.03 2.15 3/179 26229
/proc/meminfo: memFree=30457184/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=15904 CPUtime=942.31
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3683 0 0 0 82314 11917 0 0 25 0 1 0 166782256 16285696 3571 4089446400 134512640 136421960 4293243376 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 3976 3571 237 467 0 3507 0
Current children cumulated CPU time (s) 942.31
Current children cumulated vsize (KiB) 15904

[startup+1002.3 s]
/proc/loadavg: 2.01 2.03 2.14 3/179 26230
/proc/meminfo: memFree=30454912/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=15904 CPUtime=1002.32
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3689 0 0 0 87767 12465 0 0 25 0 1 0 166782256 16285696 3577 4089446400 134512640 136421960 4293243376 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 3976 3577 237 467 0 3507 0
Current children cumulated CPU time (s) 1002.32
Current children cumulated vsize (KiB) 15904

[startup+1062.3 s]
/proc/loadavg: 2.00 2.02 2.13 3/179 26232
/proc/meminfo: memFree=30453248/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=16036 CPUtime=1062.31
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3690 0 0 0 93221 13010 0 0 25 0 1 0 166782256 16420864 3578 4089446400 134512640 136421960 4293243376 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 4009 3578 237 467 0 3540 0
Current children cumulated CPU time (s) 1062.31
Current children cumulated vsize (KiB) 16036

[startup+1122.3 s]
/proc/loadavg: 2.00 2.01 2.12 3/179 26234
/proc/meminfo: memFree=30451360/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=16036 CPUtime=1122.32
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3690 0 0 0 98630 13602 0 0 25 0 1 0 166782256 16420864 3578 4089446400 134512640 136421960 4293243376 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 4009 3578 237 467 0 3540 0
Current children cumulated CPU time (s) 1122.32
Current children cumulated vsize (KiB) 16036

[startup+1182.3 s]
/proc/loadavg: 2.00 2.01 2.11 3/179 26236
/proc/meminfo: memFree=30449836/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=16036 CPUtime=1182.32
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3710 0 0 0 104063 14169 0 0 25 0 1 0 166782256 16420864 3598 4089446400 134512640 136421960 4293243376 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 4009 3598 237 467 0 3540 0
Current children cumulated CPU time (s) 1182.32
Current children cumulated vsize (KiB) 16036

[startup+1242.3 s]
/proc/loadavg: 2.00 2.00 2.09 3/179 26237
/proc/meminfo: memFree=30448684/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=16168 CPUtime=1242.32
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3727 0 0 0 109491 14741 0 0 25 0 1 0 166782256 16556032 3615 4089446400 134512640 136421960 4293243376 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 4042 3615 237 467 0 3573 0
Current children cumulated CPU time (s) 1242.32
Current children cumulated vsize (KiB) 16168

[startup+1302.3 s]
/proc/loadavg: 2.00 2.00 2.09 3/179 26238
/proc/meminfo: memFree=30448152/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=16168 CPUtime=1302.32
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3735 0 0 0 114960 15272 0 0 25 0 1 0 166782256 16556032 3623 4089446400 134512640 136421960 4293243376 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 4042 3623 237 467 0 3573 0
Current children cumulated CPU time (s) 1302.32
Current children cumulated vsize (KiB) 16168

[startup+1362.3 s]
/proc/loadavg: 2.00 2.00 2.08 3/179 26240
/proc/meminfo: memFree=30447632/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=16304 CPUtime=1362.32
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3745 0 0 0 120416 15816 0 0 25 0 1 0 166782256 16695296 3633 4089446400 134512640 136421960 4293243376 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 4076 3633 237 467 0 3607 0
Current children cumulated CPU time (s) 1362.32
Current children cumulated vsize (KiB) 16304

[startup+1422.31 s]
/proc/loadavg: 2.00 2.00 2.07 3/179 26242
/proc/meminfo: memFree=30447336/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=16304 CPUtime=1422.33
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3756 0 0 0 125889 16344 0 0 25 0 1 0 166782256 16695296 3644 4089446400 134512640 136421960 4293243376 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 4076 3644 237 467 0 3607 0
Current children cumulated CPU time (s) 1422.33
Current children cumulated vsize (KiB) 16304

[startup+1482.3 s]
/proc/loadavg: 2.05 2.01 2.07 3/179 26244
/proc/meminfo: memFree=30447284/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=16304 CPUtime=1482.33
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3767 0 0 0 131353 16880 0 0 25 0 1 0 166782256 16695296 3655 4089446400 134512640 136421960 4293243376 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 4076 3655 237 467 0 3607 0
Current children cumulated CPU time (s) 1482.33
Current children cumulated vsize (KiB) 16304

[startup+1542.3 s]
/proc/loadavg: 2.02 2.01 2.07 3/179 26245
/proc/meminfo: memFree=30447376/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=16436 CPUtime=1542.32
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3785 0 0 0 136824 17408 0 0 25 0 1 0 166782256 16830464 3673 4089446400 134512640 136421960 4293243376 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 4109 3673 237 467 0 3640 0
Current children cumulated CPU time (s) 1542.32
Current children cumulated vsize (KiB) 16436

[startup+1602.3 s]
/proc/loadavg: 2.00 2.00 2.06 3/179 26246
/proc/meminfo: memFree=30446968/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=16436 CPUtime=1602.33
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3794 0 0 0 142278 17955 0 0 25 0 1 0 166782256 16830464 3682 4089446400 134512640 136421960 4293243376 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 4109 3682 237 467 0 3640 0
Current children cumulated CPU time (s) 1602.33
Current children cumulated vsize (KiB) 16436

[startup+1662.3 s]
/proc/loadavg: 2.00 2.00 2.06 3/179 26248
/proc/meminfo: memFree=30446820/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=16436 CPUtime=1662.33
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3800 0 0 0 147757 18476 0 0 25 0 1 0 166782256 16830464 3688 4089446400 134512640 136421960 4293243376 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 4109 3688 237 467 0 3640 0
Current children cumulated CPU time (s) 1662.33
Current children cumulated vsize (KiB) 16436

[startup+1722.3 s]
/proc/loadavg: 2.00 2.00 2.05 3/179 26250
/proc/meminfo: memFree=30446904/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=16436 CPUtime=1722.33
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3801 0 0 0 153188 19045 0 0 25 0 1 0 166782256 16830464 3689 4089446400 134512640 136421960 4293243376 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 4109 3689 237 467 0 3640 0
Current children cumulated CPU time (s) 1722.33
Current children cumulated vsize (KiB) 16436

[startup+1782.3 s]
/proc/loadavg: 2.00 2.00 2.04 3/179 26252
/proc/meminfo: memFree=30446496/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=16436 CPUtime=1782.33
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3807 0 0 0 158656 19577 0 0 25 0 1 0 166782256 16830464 3695 4089446400 134512640 136421960 4293243376 18446744073709551615 135874657 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 4109 3695 237 467 0 3640 0
Current children cumulated CPU time (s) 1782.33
Current children cumulated vsize (KiB) 16436



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1800 s]
/proc/loadavg: 2.00 2.00 2.04 3/179 26252
/proc/meminfo: memFree=30446240/32950928 swapFree=67111528/67111528
[pid=26175] ppid=26173 vsize=16436 CPUtime=1800.04
/proc/26175/stat : 26175 (pbct) R 26173 26175 26150 0 -1 4202496 3813 0 0 0 160263 19741 0 0 25 0 1 0 166782256 16830464 3701 4089446400 134512640 136421960 4293243376 18446744073709551615 135032116 0 0 4096 16387 0 0 0 17 0 0 0 0
/proc/26175/statm: 4109 3701 237 467 0 3640 0
Current children cumulated CPU time (s) 1800.04
Current children cumulated vsize (KiB) 16436

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child status: 0
Real time (s): 1800.01
CPU time (s): 1800.05
CPU user time (s): 1602.64
CPU system time (s): 197.412
CPU usage (%): 100.002
Max. virtual memory (cumulated for all children) (KiB): 16436

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1602.64
system time used= 197.412
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3814
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3
involuntary context switches= 3647

runsolver used 2.88156 second user time and 6.88395 second system time

The end

Launcher Data 

Begin job on node112 at 2010-06-30 17:29:47
IDJOB=2694967
IDBENCH=78990
IDSOLVER=1208
FILE ID=node112/2694967-1277911787
PBS_JOBID= 11197281
Free space on /tmp= 74208 MiB

SOLVER NAME= PB/CT 0.1
BENCH NAME= PB10/normalized-PB10/PARTIAL-SMALLINT-LIN/PB06/submitted-PB06/namasivayam/tsp/normalized-t3002.11tsp11.1900577940--soft-66-100-0.wbo
COMMAND LINE= HOME/pbct --model BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-2694967-1277911787/watcher-2694967-1277911787 -o /tmp/evaluation-result-2694967-1277911787/solver-2694967-1277911787 -C 1800 -W 2000 -M 3800  HOME/pbct --model HOME/instance-2694967-1277911787.wbo

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 3800 MiB
MAX NB THREAD= 0

MD5SUM BENCH= 95aa4a8f0430d7fc4dd41a3f578cfb3e
RANDOM SEED=225564125

node112.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.827
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.827
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.827
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.827
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.73
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.827
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.827
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.827
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.827
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      30639284 kB
Buffers:        193596 kB
Cached:        1891828 kB
SwapCached:          0 kB
Active:         609204 kB
Inactive:      1558396 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      30639284 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:            1952 kB
Writeback:           0 kB
AnonPages:       82164 kB
Mapped:          15452 kB
Slab:            80844 kB
PageTables:       5076 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   226696 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 74204 MiB
End job on node112 at 2010-06-30 17:59:50