Trace number 3719991

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
PB07: Pueblo 1.4? (TO) 1782.59 1900.09

General information on the benchmark

Namenormalized-PB06/OPT-SMALLINT/submitted-PB06/liu/
wnq/normalized-wnq_n100_mw99_12.opb.PB06.opb
MD5SUM35ffbede57b8807d38d913370f4c4486
Bench CategoryOPT-SMALLINT (optimisation, small integers)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark448
Best CPU time to get the best result obtained on this benchmark1793.17
Has Objective FunctionYES
SatisfiableYES
(Un)Satisfiability was provedYES
Best value of the objective function 419
Optimality of the best value was proved NO
Number of variables10000
Total number of constraints656900
Number of constraints which are clauses656900
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint2
Maximum length of a constraint100
Number of terms in the objective function 10000
Biggest coefficient in the objective function 99
Number of bits for the biggest coefficient in the objective function 7
Sum of the numbers in the objective function 495618
Number of bits of the sum of numbers in the objective function 19
Biggest number in a constraint 99
Number of bits of the biggest number in a constraint 7
Biggest sum of numbers in a constraint 495618
Number of bits of the biggest sum of numbers19
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Quality of the solution as a function of time


Download the above graph as a PDF file / as an EPS file

Solver Data 

0.00/0.00	c Pueblo version 1.4 (September 2005)
0.00/0.00	c Developed @ University of Michigan, Ann Arbor, MI
0.00/0.00	c  by Hossein Sheini
15.68/15.79	c starting to Solve
15.68/15.79	c #variables read: 10000 - #constraints read: 656901
18.79/18.82	o 6846
18.88/18.91	o 6826
18.98/19.01	o 6757
19.18/19.21	o 6607
19.28/19.30	o 6560
19.38/19.41	o 6545
19.48/19.50	o 6537
20.08/20.15	o 6488
20.18/20.24	o 6485
20.38/20.41	o 6435
20.48/20.50	o 6419
20.48/20.58	o 6410
20.58/20.68	o 6406
20.68/20.78	o 6391
20.78/20.88	o 6361
20.89/20.97	o 6319
20.99/21.06	o 6313
21.48/21.52	o 6292
21.58/21.64	o 6277
21.68/21.73	o 6271
21.78/21.82	o 6260
21.88/21.90	o 6254
22.98/23.06	o 6239
23.78/23.81	o 6237
23.88/23.90	o 6204
32.88/32.98	o 6150
47.37/47.45	o 5976
47.47/47.54	o 5962
47.67/47.78	o 5954
47.77/47.86	o 5940
48.67/48.71	o 5916
56.26/56.30	o 5904
56.36/56.41	o 5860
59.36/59.47	o 5854
60.57/60.62	o 5823
60.66/60.72	o 5794
61.86/61.98	o 5751
61.96/62.07	o 5750
62.06/62.18	o 5743
62.26/62.32	o 5738
64.46/64.55	o 5623
64.56/64.63	o 5575
64.66/64.72	o 5559
64.76/64.80	o 5511
106.75/106.82	o 5509
106.84/106.92	o 5470
106.94/107.02	o 5451
107.24/107.31	o 5426
107.34/107.41	o 5420
107.44/107.53	o 5400
108.54/108.70	o 5399
108.74/108.90	o 5389
109.35/109.40	o 5385
109.75/109.81	o 5375
109.85/109.95	o 5365
109.95/110.07	o 5352
110.05/110.16	o 5346
110.15/110.25	o 5336
110.55/110.60	o 5334
1778.80/1779.46	Pueblo: Global.h:70: T* xrealloc(T*, unsigned int) [with T = Lit]: Assertion `size == 0 || tmp != __null' failed.

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3719991-1338506036/watcher-3719991-1338506036 -o /tmp/evaluation-result-3719991-1338506036/solver-3719991-1338506036 -C 1800 -W 1900 -M 15500 Pueblo HOME/instance-3719991-1338506036.opb 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.68 0.16 0.05 3/176 17415
/proc/meminfo: memFree=31632944/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=3104 CPUtime=0 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 318 0 0 0 0 0 0 0 20 0 1 0 316730607 3178496 255 33554432000 134512640 134579096 4289264736 4289131920 1582375 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 776 255 215 17 0 82 0

[startup+0.104718 s]
/proc/loadavg: 0.68 0.16 0.05 3/176 17415
/proc/meminfo: memFree=31632944/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=6300 CPUtime=0.09 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 1122 0 0 0 9 0 0 0 20 0 1 0 316730607 6451200 1059 33554432000 134512640 134579096 4289264736 4289132196 2331455 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 1575 1061 221 17 0 881 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 6300

[startup+0.200281 s]
/proc/loadavg: 0.68 0.16 0.05 3/176 17415
/proc/meminfo: memFree=31632944/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=8832 CPUtime=0.19 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 1755 0 0 0 19 0 0 0 20 0 1 0 316730607 9043968 1692 33554432000 134512640 134579096 4289264736 4289132196 2331455 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 2208 1692 221 17 0 1514 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 8832

[startup+0.300271 s]
/proc/loadavg: 0.68 0.16 0.05 3/176 17415
/proc/meminfo: memFree=31632944/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=11364 CPUtime=0.29 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 2405 0 0 0 29 0 0 0 20 0 1 0 316730607 11636736 2342 33554432000 134512640 134579096 4289264736 4289132012 7513136 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 2841 2344 221 17 0 2147 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 11364

[startup+0.700235 s]
/proc/loadavg: 0.68 0.16 0.05 3/176 17415
/proc/meminfo: memFree=31632944/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=21752 CPUtime=0.68 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 4981 0 0 0 68 0 0 0 20 0 1 0 316730607 22274048 4918 33554432000 134512640 134579096 4289264736 4289132132 2331440 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 5438 4918 221 17 0 4744 0
Current children cumulated CPU time (s) 0.68
Current children cumulated vsize (KiB) 21752

[startup+1.50025 s]
/proc/loadavg: 0.68 0.16 0.05 3/177 17416
/proc/meminfo: memFree=31603416/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=42132 CPUtime=1.48 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 10080 0 0 0 147 1 0 0 20 0 1 0 316730607 43143168 10017 33554432000 134512640 134579096 4289264736 4289132148 2331460 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 10533 10017 221 17 0 9839 0
Current children cumulated CPU time (s) 1.48
Current children cumulated vsize (KiB) 42132

[startup+3.10024 s]
/proc/loadavg: 0.68 0.16 0.05 3/177 17416
/proc/meminfo: memFree=31574152/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=83920 CPUtime=3.09 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 20431 0 0 0 306 3 0 0 20 0 1 0 316730607 85934080 20337 33554432000 134512640 134579096 4289264736 4289260016 134533472 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 20980 20337 227 17 0 20286 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 83920

[startup+6.30028 s]
/proc/loadavg: 0.79 0.19 0.06 3/177 17416
/proc/meminfo: memFree=31545384/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=84944 CPUtime=6.29 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 20626 0 0 0 626 3 0 0 20 0 1 0 316730607 86982656 20532 33554432000 134512640 134579096 4289264736 4289263696 134532448 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 21236 20532 228 17 0 20542 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 84944

[startup+12.7002 s]
/proc/loadavg: 0.88 0.22 0.07 3/177 17416
/proc/meminfo: memFree=31534276/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=86760 CPUtime=12.69 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 21147 0 0 0 1266 3 0 0 20 0 1 0 316730607 88842240 21053 33554432000 134512640 134579096 4289264736 4289263440 1582375 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 21690 21053 228 17 0 20996 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 86760

[startup+25.5003 s]
/proc/loadavg: 1.13 0.31 0.10 3/177 17416
/proc/meminfo: memFree=31472080/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=96072 CPUtime=25.48 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 23096 0 0 0 2545 3 0 0 20 0 1 0 316730607 98377728 23002 33554432000 134512640 134579096 4289264736 4289263568 134517903 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 24018 23002 230 17 0 23324 0
Current children cumulated CPU time (s) 25.48
Current children cumulated vsize (KiB) 96072

[startup+51.1003 s]
/proc/loadavg: 1.43 0.44 0.15 3/177 17464
/proc/meminfo: memFree=31959968/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=115224 CPUtime=51.07 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 27913 0 0 0 5102 5 0 0 20 0 1 0 316730607 117989376 27806 33554432000 134512640 134579096 4289264736 4289263568 134517896 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 28806 27806 230 17 0 28112 0
Current children cumulated CPU time (s) 51.07
Current children cumulated vsize (KiB) 115224

[startup+102.3 s]
/proc/loadavg: 1.77 0.70 0.26 3/177 17464
/proc/meminfo: memFree=31808128/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=236412 CPUtime=102.25 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 58234 0 0 0 10212 13 0 0 20 0 1 0 316730607 242085888 58135 33554432000 134512640 134579096 4289264736 4289263568 134517908 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 59103 58135 230 17 0 58409 0
Current children cumulated CPU time (s) 102.25
Current children cumulated vsize (KiB) 236412

[startup+162.3 s]
/proc/loadavg: 1.91 0.92 0.35 3/177 17464
/proc/meminfo: memFree=31641996/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=401424 CPUtime=162.2 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 99465 0 0 0 16192 28 0 0 20 0 1 0 316730607 411058176 99344 33554432000 134512640 134579096 4289264736 4289263536 134528560 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 100356 99344 230 17 0 99662 0
Current children cumulated CPU time (s) 162.2
Current children cumulated vsize (KiB) 401424

[startup+222.307 s]
/proc/loadavg: 1.96 1.11 0.45 3/177 17464
/proc/meminfo: memFree=31468672/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=566396 CPUtime=222.19 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 140614 0 0 0 22178 41 0 0 20 0 1 0 316730607 579989504 140487 33554432000 134512640 134579096 4289264736 4289263708 7513136 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 141599 140487 230 17 0 140905 0
Current children cumulated CPU time (s) 222.19
Current children cumulated vsize (KiB) 566396

[startup+282.3 s]
/proc/loadavg: 1.98 1.27 0.55 3/177 17465
/proc/meminfo: memFree=31297332/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=732504 CPUtime=282.15 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 182211 0 0 0 28158 57 0 0 20 0 1 0 316730607 750084096 182089 33554432000 134512640 134579096 4289264736 4289263708 7513136 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 183126 182089 230 17 0 182432 0
Current children cumulated CPU time (s) 282.15
Current children cumulated vsize (KiB) 732504

[startup+342.3 s]
/proc/loadavg: 1.99 1.40 0.64 3/177 17465
/proc/meminfo: memFree=31108880/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=919556 CPUtime=342.13 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 228732 0 0 0 34141 72 0 0 20 0 1 0 316730607 941625344 228625 33554432000 134512640 134579096 4289264736 4289263568 134517903 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 229889 228625 230 17 0 229195 0
Current children cumulated CPU time (s) 342.13
Current children cumulated vsize (KiB) 919556

[startup+402.3 s]
/proc/loadavg: 1.99 1.51 0.72 3/178 17468
/proc/meminfo: memFree=30937292/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=1092196 CPUtime=402.12 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 271940 0 0 0 40124 88 0 0 20 0 1 0 316730607 1118408704 271832 33554432000 134512640 134579096 4289264736 4289263708 7513136 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 273049 271832 230 17 0 272355 0
Current children cumulated CPU time (s) 402.12
Current children cumulated vsize (KiB) 1092196

[startup+462.3 s]
/proc/loadavg: 1.99 1.59 0.80 3/177 17468
/proc/meminfo: memFree=30789272/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=1235140 CPUtime=462.1 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 307665 0 0 0 46108 102 0 0 20 0 1 0 316730607 1264783360 307562 33554432000 134512640 134579096 4289264736 4289263568 134517908 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 308785 307562 230 17 0 308091 0
Current children cumulated CPU time (s) 462.1
Current children cumulated vsize (KiB) 1235140

[startup+522.3 s]
/proc/loadavg: 1.99 1.67 0.87 3/177 17468
/proc/meminfo: memFree=30661332/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=1362888 CPUtime=522.08 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 339553 0 0 0 52096 112 0 0 20 0 1 0 316730607 1395597312 339450 33554432000 134512640 134579096 4289264736 4289263708 7513136 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 340722 339450 230 17 0 340028 0
Current children cumulated CPU time (s) 522.08
Current children cumulated vsize (KiB) 1362888

[startup+582.3 s]
/proc/loadavg: 1.99 1.72 0.94 3/177 17469
/proc/meminfo: memFree=30529424/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=1494168 CPUtime=582.07 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 371836 0 0 0 58080 127 0 0 20 0 1 0 316730607 1530028032 371764 33554432000 134512640 134579096 4289264736 4289263448 1588641 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 373542 371764 230 17 0 372848 0
Current children cumulated CPU time (s) 582.07
Current children cumulated vsize (KiB) 1494168

[startup+642.3 s]

################
# More data... #
################

/proc/loadavg: 2.00 1.91 1.27 3/177 17470
/proc/meminfo: memFree=29743928/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=2271820 CPUtime=941.95 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 566217 0 0 0 93997 198 0 0 20 0 1 0 316730607 2326343680 566316 33554432000 134512640 134579096 4289264736 4289263584 134527765 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 567955 566316 230 17 0 567261 0
Current children cumulated CPU time (s) 941.95
Current children cumulated vsize (KiB) 2271820

[startup+1002.3 s]
/proc/loadavg: 2.00 1.93 1.31 3/178 17473
/proc/meminfo: memFree=29571712/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=2441484 CPUtime=1001.93 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 608555 0 0 0 99975 218 0 0 20 0 1 0 316730607 2500079616 608658 33554432000 134512640 134579096 4289264736 4289263708 7513136 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 610371 608658 230 17 0 609677 0
Current children cumulated CPU time (s) 1001.93
Current children cumulated vsize (KiB) 2441484

[startup+1062.3 s]
/proc/loadavg: 2.00 1.94 1.35 3/177 17473
/proc/meminfo: memFree=29415260/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=2598724 CPUtime=1061.92 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 647954 0 0 0 105956 236 0 0 20 0 1 0 316730607 2661093376 648085 33554432000 134512640 134579096 4289264736 4289263664 134559533 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 649681 648085 230 17 0 648987 0
Current children cumulated CPU time (s) 1061.92
Current children cumulated vsize (KiB) 2598724

[startup+1122.3 s]
/proc/loadavg: 2.00 1.95 1.39 3/177 17473
/proc/meminfo: memFree=29274052/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=2738536 CPUtime=1121.9 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 682874 0 0 0 111936 254 0 0 20 0 1 0 316730607 2804260864 683055 33554432000 134512640 134579096 4289264736 4289263708 7513136 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 684634 683055 230 17 0 683940 0
Current children cumulated CPU time (s) 1121.9
Current children cumulated vsize (KiB) 2738536

[startup+1182.3 s]
/proc/loadavg: 2.05 1.97 1.43 3/177 17474
/proc/meminfo: memFree=29127636/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=2888388 CPUtime=1181.89 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 719262 0 0 0 117920 269 0 0 20 0 1 0 316730607 2957709312 719487 33554432000 134512640 134579096 4289264736 4289263536 134528569 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 722097 719487 230 17 0 721403 0
Current children cumulated CPU time (s) 1181.89
Current children cumulated vsize (KiB) 2888388

[startup+1242.3 s]
/proc/loadavg: 2.05 1.98 1.47 3/177 17474
/proc/meminfo: memFree=28969564/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=3044312 CPUtime=1241.87 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 758280 0 0 0 123901 286 0 0 20 0 1 0 316730607 3117375488 758553 33554432000 134512640 134579096 4289264736 4289263708 7513136 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 761078 758553 230 17 0 760384 0
Current children cumulated CPU time (s) 1241.87
Current children cumulated vsize (KiB) 3044312

[startup+1302.3 s]
/proc/loadavg: 2.02 1.98 1.50 3/178 17475
/proc/meminfo: memFree=28822892/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=3191944 CPUtime=1301.85 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 795159 0 0 0 129882 303 0 0 20 0 1 0 316730607 3268550656 795433 33554432000 134512640 134579096 4289264736 4289263664 134559533 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 797986 795433 230 17 0 797292 0
Current children cumulated CPU time (s) 1301.85
Current children cumulated vsize (KiB) 3191944

[startup+1362.3 s]
/proc/loadavg: 2.00 1.98 1.53 3/177 17475
/proc/meminfo: memFree=28679584/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=3333172 CPUtime=1361.82 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 830378 0 0 0 135864 318 0 0 20 0 1 0 316730607 3413168128 830652 33554432000 134512640 134579096 4289264736 4289263568 134517893 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 833293 830652 230 17 0 832599 0
Current children cumulated CPU time (s) 1361.82
Current children cumulated vsize (KiB) 3333172

[startup+1422.3 s]
/proc/loadavg: 2.00 1.98 1.55 3/177 17475
/proc/meminfo: memFree=28542468/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=3469780 CPUtime=1421.81 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 864584 0 0 0 141850 331 0 0 20 0 1 0 316730607 3553054720 864864 33554432000 134512640 134579096 4289264736 4289263644 134519884 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 867445 864864 230 17 0 866751 0
Current children cumulated CPU time (s) 1421.81
Current children cumulated vsize (KiB) 3469780

[startup+1482.3 s]
/proc/loadavg: 2.00 1.98 1.58 3/177 17476
/proc/meminfo: memFree=28410932/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=3601656 CPUtime=1481.79 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 897497 0 0 0 147832 347 0 0 20 0 1 0 316730607 3688095744 897791 33554432000 134512640 134579096 4289264736 4289263536 134528544 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 900414 897791 230 17 0 899720 0
Current children cumulated CPU time (s) 1481.79
Current children cumulated vsize (KiB) 3601656

[startup+1542.31 s]
/proc/loadavg: 2.00 1.98 1.60 3/177 17476
/proc/meminfo: memFree=28298864/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=3712064 CPUtime=1541.78 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 925054 0 0 0 153815 363 0 0 20 0 1 0 316730607 3801153536 925370 33554432000 134512640 134579096 4289264736 4289263600 134519973 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 928016 925370 230 17 0 927322 0
Current children cumulated CPU time (s) 1541.78
Current children cumulated vsize (KiB) 3712064

[startup+1602.3 s]
/proc/loadavg: 2.00 1.98 1.62 3/178 17479
/proc/meminfo: memFree=28171040/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=3839852 CPUtime=1601.75 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 957052 0 0 0 159797 378 0 0 20 0 1 0 316730607 3932008448 957376 33554432000 134512640 134579096 4289264736 4289263568 134517908 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 959963 957376 230 17 0 959269 0
Current children cumulated CPU time (s) 1601.75
Current children cumulated vsize (KiB) 3839852

[startup+1662.3 s]
/proc/loadavg: 2.00 1.98 1.64 3/177 17479
/proc/meminfo: memFree=28068836/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=3937168 CPUtime=1661.74 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 981162 0 0 0 165784 390 0 0 20 0 1 0 316730607 4031660032 982006 33554432000 134512640 134579096 4289264736 4289263584 134527740 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 984292 982006 230 17 0 983598 0
Current children cumulated CPU time (s) 1661.74
Current children cumulated vsize (KiB) 3937168

[startup+1722.3 s]
/proc/loadavg: 2.00 1.98 1.66 3/177 17479
/proc/meminfo: memFree=27965572/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=4042128 CPUtime=1721.72 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) R 17413 17415 17309 0 -1 4202496 1007077 0 0 0 171771 401 0 0 20 0 1 0 316730607 4139139072 1008070 33554432000 134512640 134579096 4289264736 4289263568 134517908 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 1010532 1008070 230 17 0 1009838 0
Current children cumulated CPU time (s) 1721.72
Current children cumulated vsize (KiB) 4042128

[startup+1782.3 s]
/proc/loadavg: 2.03 2.00 1.68 4/178 17481
/proc/meminfo: memFree=26569344/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=4188560 CPUtime=1779.57 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) S 17413 17415 17309 0 -1 4204032 1043784 0 0 0 177469 488 0 0 20 0 1 0 316730607 4289085440 1044774 33554432000 134512640 134579096 4289264736 4289263168 7513136 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 1047140 1044774 236 17 0 1046446 0
Current children cumulated CPU time (s) 1779.57
Current children cumulated vsize (KiB) 4188560

[startup+1842.3 s]
/proc/loadavg: 3.66 2.46 1.86 2/178 17529
/proc/meminfo: memFree=19733608/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=4188560 CPUtime=1782.31 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) S 17413 17415 17309 0 -1 4204032 1043784 0 0 0 177469 762 0 0 20 0 1 0 316730607 4289085440 1044774 33554432000 134512640 134579096 4289264736 4289263168 7513136 0 0 4096 0 18446744071580380219 0 0 17 3 0 0 0 0 0
/proc/17415/statm: 1047140 1044774 236 17 0 1046446 0
Current children cumulated CPU time (s) 1782.31
Current children cumulated vsize (KiB) 4188560



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+1900 s]
/proc/loadavg: 4.44 2.89 2.04 2/178 17529
/proc/meminfo: memFree=19036160/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=4188560 CPUtime=1782.5 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) S 17413 17415 17309 0 -1 4204032 1043784 0 0 0 177469 781 0 0 20 0 1 0 316730607 4289085440 1044785 33554432000 134512640 134579096 4289264736 4289263168 7513136 0 0 4096 0 18446744071580380219 0 0 17 5 0 0 0 0 0
/proc/17415/statm: 1047140 1044785 236 17 0 1046446 0
Current children cumulated CPU time (s) 1782.5
Current children cumulated vsize (KiB) 4188560

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1900 s]
/proc/loadavg: 4.44 2.89 2.04 2/178 17529
/proc/meminfo: memFree=19036160/32873844 swapFree=7116/7116
[pid=17415] ppid=17413 vsize=4188560 CPUtime=1782.5 cores=1,3,5,7
/proc/17415/stat : 17415 (Pueblo) S 17413 17415 17309 0 -1 4204032 1043784 0 0 0 177469 781 0 0 20 0 1 0 316730607 4289085440 1044785 33554432000 134512640 134579096 4289264736 4289263168 7513136 0 0 4096 0 18446744071580380219 0 0 17 5 0 0 0 0 0
/proc/17415/statm: 1047140 1044785 236 17 0 1046446 0
Current children cumulated CPU time (s) 1782.5
Current children cumulated vsize (KiB) 4188560

Child ended because it received signal 6 (SIGABRT)
Child dumped core
Real time (s): 1900.09
CPU time (s): 1782.59
CPU user time (s): 1774.7
CPU system time (s): 7.8908
CPU usage (%): 93.8161
Max. virtual memory (cumulated for all children) (KiB): 4188560

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1774.7
system time used= 7.8908
maximum resident set size= 4179140
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1043784
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1010992
involuntary context switches= 1801

runsolver used 4.09238 second user time and 9.52455 second system time

The end

Launcher Data 

Begin job on node111 at 2012-06-01 01:13:56
IDJOB=3719991
IDBENCH=1964
IDSOLVER=2305
FILE ID=node111/3719991-1338506036
RUNJOBID= node111-1338506011-17326
PBS_JOBID= 14635780
Free space on /tmp= 67292 MiB

SOLVER NAME= Pueblo 1.4
BENCH NAME= PB06//final/normalized-PB06/OPT-SMALLINT/submitted-PB06/liu/wnq/normalized-wnq_n100_mw99_12.opb.PB06.opb
COMMAND LINE= Pueblo BENCHNAME
CONVERSION SCRIPT= PBconversionToLinear BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3719991-1338506036/watcher-3719991-1338506036 -o /tmp/evaluation-result-3719991-1338506036/solver-3719991-1338506036 -C 1800 -W 1900 -M 15500  Pueblo HOME/instance-3719991-1338506036.opb

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 35ffbede57b8807d38d913370f4c4486
RANDOM SEED=1506028433

node111.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.152
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.30
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.152
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.152
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.152
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.152
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.152
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.152
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.152
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        31632092 kB
Buffers:            5444 kB
Cached:           156512 kB
SwapCached:         1300 kB
Active:           479136 kB
Inactive:          38372 kB
Active(anon):     357176 kB
Inactive(anon):     1056 kB
Active(file):     121960 kB
Inactive(file):    37316 kB
Unevictable:        7116 kB
Mlocked:            7116 kB
SwapTotal:      67108856 kB
SwapFree:       67100036 kB
Dirty:            110008 kB
Writeback:             0 kB
AnonPages:        362012 kB
Mapped:             8524 kB
Shmem:                 0 kB
Slab:             585796 kB
SReclaimable:      27720 kB
SUnreclaim:       558076 kB
KernelStack:        1456 kB
PageTables:         4556 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     478188 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:     16384 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

CONVERSION COMMAND LINE= PBconversionToLinear HOME/instance-3719991-1338506036.opb

Free space on /tmp at the end= 63292 MiB
End job on node111 at 2012-06-01 01:45:38