Trace number 3742555

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PB07: bsolo 3.0.17? (TO) 1800.13 1800.66

General information on the benchmark

Name/DEC-SMALLINT-LIN/sroussel/
ShortestPathNG/normalized-NG_K168.opb
MD5SUM4a002b4e44699cf4cd35a469f7223c94
Bench CategoryDEC-SMALLINT-LIN (no optimisation, small integers, linear constraints)
Best result obtained on this benchmark
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Has Objective FunctionNO
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables19992
Total number of constraints20330
Number of constraints which are clauses20162
Number of constraints which are cardinality constraints (but not clauses)168
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint1
Maximum length of a constraint168
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 169
Number of bits of the biggest sum of numbers8
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c bsolo beta version 3.0.17 - 27/04/2007 : 1600 GMT
0.00/0.00	c Developed by Vasco Manquinho IST/UTL - INESC-ID
0.00/0.00	c type "bsolo -h" for help
0.00/0.00	c Time Limit set via environment variable PBTIMEOUT to 1800
0.00/0.00	c Memory Limit set to 15500 MB
0.00/0.00	c Instance file HOME/instance-3742555-1338772192.opb
0.00/0.00	c File size is 1377299 bytes.
0.08/0.13	c Highest Coefficient sum: 169
0.08/0.13	c Parsing was ok!!
0.08/0.13	c Total parsing time: 0.128 s
0.08/0.14	c Var: 19992 Constr: 20330 20162/168/0 Lit: 130786 Watch. Lit: 60314
0.08/0.14	c Obj. Vars: 0 (0 % of total variables)
9.18/9.27	c Pre-processing Time: 9.264 s
9.18/9.27	c    Confl   Vars     Ctrs     Lits  LPC   W.Lits     Max Learnt Conf/s Time
9.18/9.27	c        0  18445    18889   124316    6    57012  170673      0      0 9.26
9.18/9.27	c Switching off LPR mode.
1800.07/1800.61	c      100  18445    18989   129978    6    57212  187741    100     10 9.31
1800.07/1800.61	c      250  18445    19139   140193    7    57512  206516    250     26 9.44
1800.07/1800.61	c      475  18445    19364   158643    8    57962  227168    475     49 9.62
1800.07/1800.61	c      813  18445    19702   180323    9    58638  249885    813     81 9.97
1800.07/1800.61	c     1321  18445    20210   226381   11    59654  274874   1321    125 10.55
1800.07/1800.61	c     2084  18445    20973   291344   13    61180  302362   2084    183 11.35
1800.07/1800.61	c     3230  18445    22119   364338   16    63472  332599   3230    252 12.80
1800.07/1800.61	c     4951  18445    23840   560645   23    66914  365859   4951    327 15.10
1800.07/1800.61	c     7536  18445    26425   833549   31    72084  402445   7536    410 18.35
1800.07/1800.61	c    11416  18445    30305  1442569   47    79844  442690  11416    497 22.95
1800.07/1800.61	c    17238  18445    36127  2277139   63    91488  486960  17238    565 30.50
1800.07/1800.61	c    25975  18445    44864  3497978   77   108962  535657  25975    568 45.67
1800.07/1800.61	c    39086  18445    57975  6404814  110   135184  589223  39086    573 68.15
1800.07/1800.61	c    58759  18445    77648 12181346  156   174530  648146  58759    506 115.96
1800.07/1800.61	c    88269  18445   107158 22672629  211   233550  712961  88269    414 212.88
1800.07/1800.61	c   132540  18445   151429 41158090  271   322092  784258 132540    315 420.39
1800.07/1800.61	c   198951  18445   217840 68171739  312   454914  862684 198951    233 853.54
1800.07/1800.61	c   298574  18445   317463 133564890  420   654160  948953 298574    176 1694.63
1800.07/1800.61	c Caught signal: 15
1800.07/1800.61	c  Rst    Confl    NCB      Dec     Ctrs     Lits Learnt Conf/s Time
1800.07/1800.61	c   18   308786 196266 18401265   327675 137296304 308786    171 1800.08
1800.07/1800.61	s UNKNOWN
1800.07/1800.61	c Total time: 1800.08 s

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3742555-1338772192/watcher-3742555-1338772192 -o /tmp/evaluation-result-3742555-1338772192/solver-3742555-1338772192 -C 1800 -W 1900 -M 15500 bsolo3.0.17 -m15500 HOME/instance-3742555-1338772192.opb 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 14414
/proc/meminfo: memFree=26718868/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=4472 CPUtime=0 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4194304 115 0 0 0 0 0 0 0 20 0 1 0 343343665 4579328 67 33554432000 134512640 135840783 4289352848 4289352036 10788213 0 0 4096 0 0 0 0 17 0 0 0 0 0 0
/proc/14414/statm: 1119 76 60 325 0 39 0

[startup+0.0869899 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 14414
/proc/meminfo: memFree=26718868/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=9328 CPUtime=0.07 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 1513 0 0 0 7 0 0 0 20 0 1 0 343343665 9551872 1450 33554432000 134512640 135840783 4289352848 4289351536 11330855 0 0 4096 16384 0 0 0 17 0 0 0 0 0 0
/proc/14414/statm: 2332 1450 535 325 0 931 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 9328

[startup+0.100266 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 14414
/proc/meminfo: memFree=26718868/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=9736 CPUtime=0.08 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 1646 0 0 0 8 0 0 0 20 0 1 0 343343665 9969664 1583 33554432000 134512640 135840783 4289352848 4289351792 134524175 0 0 4096 16384 0 0 0 17 0 0 0 0 0 0
/proc/14414/statm: 2434 1583 572 325 0 1033 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 9736

[startup+0.300201 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 14414
/proc/meminfo: memFree=26718868/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=12100 CPUtime=0.28 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 2627 0 0 0 28 0 0 0 20 0 1 0 343343665 12390400 2227 33554432000 134512640 135840783 4289352848 4289111920 134550902 0 0 4096 16384 0 0 0 17 0 0 0 0 0 0
/proc/14414/statm: 3025 2227 342 325 0 1961 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 12100

[startup+0.700186 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 14414
/proc/meminfo: memFree=26718868/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=12100 CPUtime=0.68 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 2628 0 0 0 68 0 0 0 20 0 1 0 343343665 12390400 2228 33554432000 134512640 135840783 4289352848 4289111696 11328414 0 0 4096 16384 0 0 0 17 0 0 0 0 0 0
/proc/14414/statm: 3025 2228 342 325 0 1961 0
Current children cumulated CPU time (s) 0.68
Current children cumulated vsize (KiB) 12100

[startup+1.50018 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 14415
/proc/meminfo: memFree=26711048/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=12100 CPUtime=1.48 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 2629 0 0 0 148 0 0 0 20 0 1 0 343343665 12390400 2229 33554432000 134512640 135840783 4289352848 4289111912 3631276 0 0 4096 16384 0 0 0 17 0 0 0 0 0 0
/proc/14414/statm: 3025 2229 342 325 0 1961 0
Current children cumulated CPU time (s) 1.48
Current children cumulated vsize (KiB) 12100

[startup+3.10019 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 14415
/proc/meminfo: memFree=26707940/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=12100 CPUtime=3.09 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 2630 0 0 0 308 1 0 0 20 0 1 0 343343665 12390400 2230 33554432000 134512640 135840783 4289352848 4289111920 134551266 0 0 4096 16384 0 0 0 17 0 0 0 0 0 0
/proc/14414/statm: 3025 2230 342 325 0 1961 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 12100

[startup+6.30019 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 14415
/proc/meminfo: memFree=26703724/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=12100 CPUtime=6.29 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 2631 0 0 0 628 1 0 0 20 0 1 0 343343665 12390400 2231 33554432000 134512640 135840783 4289352848 4289111920 134551052 0 0 4096 16384 0 0 0 17 0 0 0 0 0 0
/proc/14414/statm: 3025 2231 342 325 0 1961 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 12100

[startup+12.7002 s]
/proc/loadavg: 2.00 2.00 2.00 3/179 14418
/proc/meminfo: memFree=26696136/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=15428 CPUtime=12.68 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 3684 0 0 0 1266 2 0 0 20 0 1 0 343343665 15798272 3086 33554432000 134512640 135840783 4289352848 4289352144 134572899 0 0 4096 16384 0 0 0 17 0 0 0 0 0 0
/proc/14414/statm: 3857 3086 347 325 0 2793 0
Current children cumulated CPU time (s) 12.68
Current children cumulated vsize (KiB) 15428

[startup+25.5002 s]
/proc/loadavg: 2.00 2.00 2.00 3/179 14418
/proc/meminfo: memFree=26668120/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=34616 CPUtime=25.48 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 8462 0 0 0 2545 3 0 0 20 0 1 0 343343665 35446784 7864 33554432000 134512640 135840783 4289352848 4289352208 134551033 0 0 4096 16384 0 0 0 17 0 0 0 0 0 0
/proc/14414/statm: 8654 7864 347 325 0 7590 0
Current children cumulated CPU time (s) 25.48
Current children cumulated vsize (KiB) 34616

[startup+51.1003 s]
/proc/loadavg: 2.00 2.00 2.00 3/179 14418
/proc/meminfo: memFree=26622944/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=70000 CPUtime=51.07 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 17329 0 0 0 5102 5 0 0 20 0 1 0 343343665 71680000 16731 33554432000 134512640 135840783 4289352848 4289352144 134571853 0 0 4096 16384 0 0 0 17 2 0 0 0 0 0
/proc/14414/statm: 17500 16731 347 325 0 16436 0
Current children cumulated CPU time (s) 51.07
Current children cumulated vsize (KiB) 70000

[startup+102.306 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 14418
/proc/meminfo: memFree=26498128/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=143328 CPUtime=102.27 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 35661 0 0 0 10218 9 0 0 20 0 1 0 343343665 146767872 35063 33554432000 134512640 135840783 4289352848 4289352208 134550764 0 0 4096 16384 0 0 0 17 2 0 0 0 0 0
/proc/14414/statm: 35832 35063 347 325 0 34768 0
Current children cumulated CPU time (s) 102.27
Current children cumulated vsize (KiB) 143328

[startup+162.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 14418
/proc/meminfo: memFree=26375768/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=226948 CPUtime=162.26 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 56579 0 0 0 16212 14 0 0 20 0 1 0 343343665 232394752 55981 33554432000 134512640 135840783 4289352848 4289352208 134550782 0 0 4096 16384 0 0 0 17 2 0 0 0 0 0
/proc/14414/statm: 56737 55981 347 325 0 55673 0
Current children cumulated CPU time (s) 162.26
Current children cumulated vsize (KiB) 226948

[startup+222.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 14419
/proc/meminfo: memFree=26238156/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=314120 CPUtime=222.24 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 78360 0 0 0 22206 18 0 0 20 0 1 0 343343665 321658880 77762 33554432000 134512640 135840783 4289352848 4289352208 134550804 0 0 4096 16384 0 0 0 17 2 0 0 0 0 0
/proc/14414/statm: 78530 77762 347 325 0 77466 0
Current children cumulated CPU time (s) 222.24
Current children cumulated vsize (KiB) 314120

[startup+282.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 14419
/proc/meminfo: memFree=26141588/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=367764 CPUtime=282.23 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 91782 0 0 0 28201 22 0 0 20 0 1 0 343343665 376590336 91184 33554432000 134512640 135840783 4289352848 4289352200 3631289 0 0 4096 16384 0 0 0 17 2 0 0 0 0 0
/proc/14414/statm: 91941 91184 347 325 0 90877 0
Current children cumulated CPU time (s) 282.23
Current children cumulated vsize (KiB) 367764

[startup+342.3 s]
/proc/loadavg: 2.05 2.01 2.00 3/178 14621
/proc/meminfo: memFree=26007572/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=444316 CPUtime=342.23 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 110908 0 0 0 34196 27 0 0 20 0 1 0 343343665 454979584 110310 33554432000 134512640 135840783 4289352848 4289352208 134551324 0 0 4096 16384 0 0 0 17 2 0 0 0 0 0
/proc/14414/statm: 111079 110310 347 325 0 110015 0
Current children cumulated CPU time (s) 342.23
Current children cumulated vsize (KiB) 444316

[startup+402.3 s]
/proc/loadavg: 2.02 2.01 2.00 3/177 14621
/proc/meminfo: memFree=25893652/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=506760 CPUtime=402.22 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 126523 0 0 0 40191 31 0 0 20 0 1 0 343343665 518922240 125925 33554432000 134512640 135840783 4289352848 4289352208 134551104 0 0 4096 16384 0 0 0 17 2 0 0 0 0 0
/proc/14414/statm: 126690 125925 347 325 0 125626 0
Current children cumulated CPU time (s) 402.22
Current children cumulated vsize (KiB) 506760

[startup+462.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 14621
/proc/meminfo: memFree=25793984/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=567032 CPUtime=462.21 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 141575 0 0 0 46186 35 0 0 20 0 1 0 343343665 580640768 140977 33554432000 134512640 135840783 4289352848 4289352368 134551977 0 0 4096 16384 0 0 0 17 2 0 0 0 0 0
/proc/14414/statm: 141758 140977 347 325 0 140694 0
Current children cumulated CPU time (s) 462.21
Current children cumulated vsize (KiB) 567032

[startup+522.3 s]
/proc/loadavg: 2.04 2.01 2.00 3/177 14622
/proc/meminfo: memFree=25717256/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=611628 CPUtime=522.2 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 152727 0 0 0 52183 37 0 0 20 0 1 0 343343665 626307072 152129 33554432000 134512640 135840783 4289352848 4289352208 134550764 0 0 4096 16384 0 0 0 17 2 0 0 0 0 0
/proc/14414/statm: 152907 152129 347 325 0 151843 0
Current children cumulated CPU time (s) 522.2
Current children cumulated vsize (KiB) 611628

[startup+582.3 s]
/proc/loadavg: 2.01 2.01 2.00 3/177 14622
/proc/meminfo: memFree=25652432/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=647576 CPUtime=582.19 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 161734 0 0 0 58179 40 0 0 20 0 1 0 343343665 663117824 161136 33554432000 134512640 135840783 4289352848 4289352208 134551071 0 0 4096 16384 0 0 0 17 2 0 0 0 0 0
/proc/14414/statm: 161894 161136 347 325 0 160830 0
Current children cumulated CPU time (s) 582.19
Current children cumulated vsize (KiB) 647576

[startup+642.3 s]

################
# More data... #
################

[startup+822.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 14626
/proc/meminfo: memFree=25036140/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=870760 CPUtime=822.15 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 217527 0 0 0 82161 54 0 0 20 0 1 0 343343665 891658240 216929 33554432000 134512640 135840783 4289352848 4289352208 134551048 0 0 4096 16384 0 0 0 17 2 0 0 0 0 0
/proc/14414/statm: 217690 216929 347 325 0 216626 0
Current children cumulated CPU time (s) 822.15
Current children cumulated vsize (KiB) 870760

[startup+882.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 14626
/proc/meminfo: memFree=24810736/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=913456 CPUtime=882.14 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 228183 0 0 0 88157 57 0 0 20 0 1 0 343343665 935378944 227585 33554432000 134512640 135840783 4289352848 4289346408 3631312 0 0 4096 16384 0 0 0 17 2 0 0 0 0 0
/proc/14414/statm: 228364 227585 347 325 0 227300 0
Current children cumulated CPU time (s) 882.14
Current children cumulated vsize (KiB) 913456

[startup+942.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 14626
/proc/meminfo: memFree=24587812/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=966376 CPUtime=942.13 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 241421 0 0 0 94153 60 0 0 20 0 1 0 343343665 989569024 240823 33554432000 134512640 135840783 4289352848 4289352208 134551076 0 0 4096 16384 0 0 0 17 2 0 0 0 0 0
/proc/14414/statm: 241594 240823 347 325 0 240530 0
Current children cumulated CPU time (s) 942.13
Current children cumulated vsize (KiB) 966376

[startup+1002.3 s]
/proc/loadavg: 2.05 2.01 2.00 3/177 14626
/proc/meminfo: memFree=24363524/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=1016392 CPUtime=1002.12 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 253922 0 0 0 100149 63 0 0 20 0 1 0 343343665 1040785408 253324 33554432000 134512640 135840783 4289352848 4289352208 134551330 0 0 4096 16384 0 0 0 17 2 0 0 0 0 0
/proc/14414/statm: 254098 253324 347 325 0 253034 0
Current children cumulated CPU time (s) 1002.12
Current children cumulated vsize (KiB) 1016392

[startup+1062.3 s]
/proc/loadavg: 2.02 2.01 2.00 3/177 14626
/proc/meminfo: memFree=24162052/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=1062956 CPUtime=1062.11 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 265556 0 0 0 106146 65 0 0 20 0 1 0 343343665 1088466944 264958 33554432000 134512640 135840783 4289352848 4289352208 134550779 0 0 4096 16384 0 0 0 17 2 0 0 0 0 0
/proc/14414/statm: 265739 264958 347 325 0 264675 0
Current children cumulated CPU time (s) 1062.11
Current children cumulated vsize (KiB) 1062956

[startup+1122.3 s]
/proc/loadavg: 2.07 2.02 2.00 3/177 14666
/proc/meminfo: memFree=26214512/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=1147084 CPUtime=1122.1 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 286606 0 0 0 112139 71 0 0 20 0 1 0 343343665 1174614016 286008 33554432000 134512640 135840783 4289352848 4289352200 3631276 0 0 4096 16384 0 0 0 17 2 0 0 0 0 0
/proc/14414/statm: 286771 286008 347 325 0 285707 0
Current children cumulated CPU time (s) 1122.1
Current children cumulated vsize (KiB) 1147084

[startup+1182.3 s]
/proc/loadavg: 2.02 2.01 2.00 3/177 14666
/proc/meminfo: memFree=26026148/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=1194996 CPUtime=1182.07 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 298636 0 0 0 118133 74 0 0 20 0 1 0 343343665 1223675904 297990 33554432000 134512640 135840783 4289352848 4289352208 134551084 0 0 4096 16384 0 0 0 17 2 0 0 0 0 0
/proc/14414/statm: 298749 297990 347 325 0 297685 0
Current children cumulated CPU time (s) 1182.07
Current children cumulated vsize (KiB) 1194996

[startup+1242.3 s]
/proc/loadavg: 2.01 2.01 2.00 3/178 14669
/proc/meminfo: memFree=25869680/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=1255660 CPUtime=1242.03 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 313849 0 0 0 124125 78 0 0 20 0 1 0 343343665 1285795840 313162 33554432000 134512640 135840783 4289352848 4289352084 11325412 0 0 4096 16384 0 0 0 17 2 0 0 0 0 0
/proc/14414/statm: 313915 313162 347 325 0 312851 0
Current children cumulated CPU time (s) 1242.03
Current children cumulated vsize (KiB) 1255660

[startup+1302.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 14669
/proc/meminfo: memFree=25772624/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=1276172 CPUtime=1302.01 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 318998 0 0 0 130121 80 0 0 20 0 1 0 343343665 1306800128 318276 33554432000 134512640 135840783 4289352848 4289352200 3631312 0 0 4096 16384 0 0 0 17 2 0 0 0 0 0
/proc/14414/statm: 319043 318276 347 325 0 317979 0
Current children cumulated CPU time (s) 1302.01
Current children cumulated vsize (KiB) 1276172

[startup+1362.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 14669
/proc/meminfo: memFree=25688332/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=1309612 CPUtime=1361.97 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 327397 0 0 0 136115 82 0 0 20 0 1 0 343343665 1341042688 326636 33554432000 134512640 135840783 4289352848 4289352160 134551186 0 0 4096 16384 0 0 0 17 2 0 0 0 0 0
/proc/14414/statm: 327403 326636 347 325 0 326339 0
Current children cumulated CPU time (s) 1361.97
Current children cumulated vsize (KiB) 1309612

[startup+1422.31 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 14670
/proc/meminfo: memFree=25592880/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=1338308 CPUtime=1421.95 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 334600 0 0 0 142110 85 0 0 20 0 1 0 343343665 1370427392 333800 33554432000 134512640 135840783 4289352848 4289352208 134550764 0 0 4096 16384 0 0 0 17 2 0 0 0 0 0
/proc/14414/statm: 334577 333800 347 325 0 333513 0
Current children cumulated CPU time (s) 1421.95
Current children cumulated vsize (KiB) 1338308

[startup+1482.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 14670
/proc/meminfo: memFree=25483292/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=1402776 CPUtime=1481.92 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 350759 0 0 0 148103 89 0 0 20 0 1 0 343343665 1436442624 349928 33554432000 134512640 135840783 4289352848 4289351888 11328755 0 0 4096 16384 0 0 0 17 2 0 0 0 0 0
/proc/14414/statm: 350694 349928 347 325 0 349630 0
Current children cumulated CPU time (s) 1481.92
Current children cumulated vsize (KiB) 1402776

[startup+1542.3 s]
/proc/loadavg: 2.04 2.01 2.00 3/177 14670
/proc/meminfo: memFree=25352252/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=1483452 CPUtime=1541.88 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 370974 0 0 0 154095 93 0 0 20 0 1 0 343343665 1519054848 370104 33554432000 134512640 135840783 4289352848 4289351984 11328414 0 0 4096 16384 0 0 0 17 2 0 0 0 0 0
/proc/14414/statm: 370863 370104 347 325 0 369799 0
Current children cumulated CPU time (s) 1541.88
Current children cumulated vsize (KiB) 1483452

[startup+1602.3 s]
/proc/loadavg: 2.01 2.01 2.00 3/177 14670
/proc/meminfo: memFree=25238448/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=1555068 CPUtime=1601.86 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 388888 0 0 0 160088 98 0 0 20 0 1 0 343343665 1592389632 387990 33554432000 134512640 135840783 4289352848 4289352208 134551084 0 0 4096 16384 0 0 0 17 2 0 0 0 0 0
/proc/14414/statm: 388767 387990 347 325 0 387703 0
Current children cumulated CPU time (s) 1601.86
Current children cumulated vsize (KiB) 1555068

[startup+1662.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 14670
/proc/meminfo: memFree=25063388/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=1689348 CPUtime=1661.82 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 422487 0 0 0 166080 102 0 0 20 0 1 0 343343665 1729892352 421562 33554432000 134512640 135840783 4289352848 4289352208 134551084 0 0 4096 16384 0 0 0 17 2 0 0 0 0 0
/proc/14414/statm: 422337 421562 347 325 0 421273 0
Current children cumulated CPU time (s) 1661.82
Current children cumulated vsize (KiB) 1689348

[startup+1722.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 14671
/proc/meminfo: memFree=24965828/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=1746864 CPUtime=1721.8 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 436905 0 0 0 172075 105 0 0 20 0 1 0 343343665 1788788736 435944 33554432000 134512640 135840783 4289352848 4289352208 134550779 0 0 4096 16384 0 0 0 17 2 0 0 0 0 0
/proc/14414/statm: 436716 435944 347 325 0 435652 0
Current children cumulated CPU time (s) 1721.8
Current children cumulated vsize (KiB) 1746864

[startup+1782.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 14671
/proc/meminfo: memFree=24898152/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=1782192 CPUtime=1781.77 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 445778 0 0 0 178069 108 0 0 20 0 1 0 343343665 1824964608 444781 33554432000 134512640 135840783 4289352848 4289352208 134551324 0 0 4096 16384 0 0 0 17 2 0 0 0 0 0
/proc/14414/statm: 445548 444781 347 325 0 444484 0
Current children cumulated CPU time (s) 1781.77
Current children cumulated vsize (KiB) 1782192



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1800.6 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 14671
/proc/meminfo: memFree=24880512/32873844 swapFree=6608/6608
[pid=14414] ppid=14412 vsize=1789516 CPUtime=1800.07 cores=0,2,4,6
/proc/14414/stat : 14414 (bsolo3.0.17) R 14412 14414 12501 0 -1 4202496 447616 0 0 0 179899 108 0 0 20 0 1 0 343343665 1832464384 446609 33554432000 134512640 135840783 4289352848 4289352208 134551098 0 0 4096 16384 0 0 0 17 2 0 0 0 0 0
/proc/14414/statm: 447379 446609 347 325 0 446315 0
Current children cumulated CPU time (s) 1800.07
Current children cumulated vsize (KiB) 1789516

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child status: 15
Real time (s): 1800.66
CPU time (s): 1800.13
CPU user time (s): 1798.99
CPU system time (s): 1.13783
CPU usage (%): 99.9705
Max. virtual memory (cumulated for all children) (KiB): 1789516

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1798.99
system time used= 1.13783
maximum resident set size= 1786436
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 447619
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 440
involuntary context switches= 2167

runsolver used 2.21066 second user time and 9.98348 second system time

The end

Launcher Data 

Begin job on node140 at 2012-06-04 03:09:52
IDJOB=3742555
IDBENCH=90204
IDSOLVER=2318
FILE ID=node140/3742555-1338772192
RUNJOBID= node140-1338749243-12517
PBS_JOBID= 14636796
Free space on /tmp= 70608 MiB

SOLVER NAME= PB07: bsolo 3.0.17
BENCH NAME= PB12/normalized-PB12/DEC-SMALLINT-LIN/sroussel/ShortestPathNG/normalized-NG_K168.opb
COMMAND LINE= bsolo3.0.17 -mMEMLIMIT BENCHNAME            
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3742555-1338772192/watcher-3742555-1338772192 -o /tmp/evaluation-result-3742555-1338772192/solver-3742555-1338772192 -C 1800 -W 1900 -M 15500  bsolo3.0.17 -m15500 HOME/instance-3742555-1338772192.opb            

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 4a002b4e44699cf4cd35a469f7223c94
RANDOM SEED=374912118

node140.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.88
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.094
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        26719512 kB
Buffers:          519388 kB
Cached:          4018168 kB
SwapCached:         1620 kB
Active:          2220508 kB
Inactive:        3000548 kB
Active(anon):     676036 kB
Inactive(anon):     9476 kB
Active(file):    1544472 kB
Inactive(file):  2991072 kB
Unevictable:        6608 kB
Mlocked:            6608 kB
SwapTotal:      67108856 kB
SwapFree:       67105264 kB
Dirty:              7776 kB
Writeback:             0 kB
AnonPages:        688572 kB
Mapped:            11772 kB
Shmem:                 8 kB
Slab:             791592 kB
SReclaimable:     233644 kB
SUnreclaim:       557948 kB
KernelStack:        1472 kB
PageTables:         5508 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     799280 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:     16384 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70604 MiB
End job on node140 at 2012-06-04 03:39:55