Trace number 3742334

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PB07: bsolo 3.0.17? (TO) 1800.12 1800.83

General information on the benchmark

Name/DEC-SMALLINT-LIN/sroussel/
ShortestPathNG/normalized-NG_K129.opb
MD5SUM1a22ba01765c232fe0c8623cfb5d750f
Bench CategoryDEC-SMALLINT-LIN (no optimisation, small integers, linear constraints)
Best result obtained on this benchmark
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Has Objective FunctionNO
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables15351
Total number of constraints15611
Number of constraints which are clauses15482
Number of constraints which are cardinality constraints (but not clauses)129
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint1
Maximum length of a constraint129
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 130
Number of bits of the biggest sum of numbers8
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data

0.00/0.00	c bsolo beta version 3.0.17 - 27/04/2007 : 1600 GMT
0.00/0.00	c Developed by Vasco Manquinho IST/UTL - INESC-ID
0.00/0.00	c type "bsolo -h" for help
0.00/0.00	c Time Limit set via environment variable PBTIMEOUT to 1800
0.00/0.00	c Memory Limit set to 15500 MB
0.00/0.00	c Instance file HOME/instance-3742334-1338769972.opb
0.00/0.00	c File size is 1040150 bytes.
0.09/0.10	c Highest Coefficient sum: 130
0.09/0.10	c Parsing was ok!!
0.09/0.10	c Total parsing time: 0.096 s
0.09/0.10	c Var: 15351 Constr: 15611 15482/129/0 Lit: 100327 Watch. Lit: 46313
0.09/0.10	c Obj. Vars: 0 (0 % of total variables)
6.28/6.31	c Pre-processing Time: 6.306 s
6.28/6.31	c    Confl   Vars     Ctrs     Lits  LPC   W.Lits     Max Learnt Conf/s Time
6.28/6.31	c        0  13843    14209    94130    6    43089  130126      0      0 6.31
6.28/6.31	c Switching off LPR mode.
1800.10/1800.81	c      107  13843    14316   101464    7    43303  143139    107     16 6.33
1800.10/1800.81	c      258  13843    14467   144800   10    43605  157453    258     40 6.40
1800.10/1800.81	c      483  13843    14692   168757   11    44055  173199    483     73 6.58
1800.10/1800.81	c      821  13843    15030   195360   12    44731  190519    821    119 6.90
1800.10/1800.81	c     1329  13843    15538   218932   14    45747  209571   1329    180 7.35
1800.10/1800.81	c     2092  13843    16301   298705   18    47273  230529   2092    265 7.87
1800.10/1800.81	c     3238  13843    17447   432696   24    49565  253582   3238    367 8.81
1800.10/1800.81	c     4960  13843    19169   616406   32    53009  278941   4960    485 10.22
1800.10/1800.81	c     7544  13843    21753  1310973   60    58177  306836   7544    591 12.75
1800.10/1800.81	c    11426  13843    25635  1980409   77    65941  337520  11426    660 17.30
1800.10/1800.81	c    17248  13843    31457  2897606   92    77585  371273  17248    697 24.72
1800.10/1800.81	c    25985  13843    40194  5364656  133    95059  408401  25985    704 36.89
1800.10/1800.81	c    39098  13843    53307 10792563  202   121285  449242  39098    631 61.93
1800.10/1800.81	c    58768  13843    72977 17780772  243   160625  494167  58768    562 104.40
1800.10/1800.81	c    88281  13843   102490 44213056  431   219651  543584  88281    345 255.55
1800.10/1800.81	c   132553  13843   146762 97988964  667   308195  597943 132553    223 593.82
1800.10/1800.81	c   198963  13843   213172 149445538  701   441015  657738 198963    148 1337.23
1800.10/1800.81	c Caught signal: 15
1800.10/1800.81	c  Rst    Confl    NCB      Dec     Ctrs     Lits Learnt Conf/s Time
1800.10/1800.81	c   17   227715 130062  5233637   241924 182819998 227715    126 1800.10
1800.10/1800.81	s UNKNOWN
1800.10/1800.81	c Total time: 1800.1 s

Verifier Data

ERROR: no interpretation found !

Watcher Data

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3742334-1338769972/watcher-3742334-1338769972 -o /tmp/evaluation-result-3742334-1338769972/solver-3742334-1338769972 -C 1800 -W 1900 -M 15500 bsolo3.0.17 -m15500 HOME/instance-3742334-1338769972.opb 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.02 2.01 2.00 3/176 23121
/proc/meminfo: memFree=26446044/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=2660 CPUtime=0 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) D 23119 23121 20253 0 -1 4194304 95 0 0 0 0 0 0 0 20 0 1 0 343124047 2723840 48 33554432000 134512640 135840783 4293822384 4293820420 11203524 0 0 4096 0 18446744072100609316 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 665 48 37 325 0 32 0

[startup+0.073092 s]
/proc/loadavg: 2.02 2.01 2.00 3/176 23121
/proc/meminfo: memFree=26446044/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=8480 CPUtime=0.06 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 1333 0 0 0 6 0 0 0 20 0 1 0 343124047 8683520 1270 33554432000 134512640 135840783 4293822384 4293821216 2315559 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 2120 1270 501 325 0 802 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 8480

[startup+0.100275 s]
/proc/loadavg: 2.02 2.01 2.00 3/176 23121
/proc/meminfo: memFree=26446044/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=8872 CPUtime=0.09 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 1509 0 0 0 9 0 0 0 20 0 1 0 343124047 9084928 1446 33554432000 134512640 135840783 4293822384 4293821376 134525286 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 2218 1447 568 325 0 900 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 8872

[startup+0.300232 s]
/proc/loadavg: 2.02 2.01 2.00 3/176 23121
/proc/meminfo: memFree=26446044/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=10408 CPUtime=0.28 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 2131 0 0 0 28 0 0 0 20 0 1 0 343124047 10657792 1814 33554432000 134512640 135840783 4293822384 4293637276 7623728 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 2602 1814 344 325 0 1538 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 10408

[startup+0.700226 s]
/proc/loadavg: 2.02 2.01 2.00 3/176 23121
/proc/meminfo: memFree=26446044/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=10408 CPUtime=0.68 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 2133 0 0 0 68 0 0 0 20 0 1 0 343124047 10657792 1816 33554432000 134512640 135840783 4293822384 4293637276 7623728 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 2602 1816 344 325 0 1538 0
Current children cumulated CPU time (s) 0.68
Current children cumulated vsize (KiB) 10408

[startup+1.50022 s]
/proc/loadavg: 2.02 2.01 2.00 3/177 23122
/proc/meminfo: memFree=26440224/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=10408 CPUtime=1.48 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 2133 0 0 0 148 0 0 0 20 0 1 0 343124047 10657792 1816 33554432000 134512640 135840783 4293822384 4293637276 7623728 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 2602 1816 344 325 0 1538 0
Current children cumulated CPU time (s) 1.48
Current children cumulated vsize (KiB) 10408

[startup+3.10022 s]
/proc/loadavg: 2.02 2.01 2.00 3/177 23122
/proc/meminfo: memFree=26439464/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=10408 CPUtime=3.08 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 2134 0 0 0 308 0 0 0 20 0 1 0 343124047 10657792 1817 33554432000 134512640 135840783 4293822384 4293637276 7623728 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 2602 1817 344 325 0 1538 0
Current children cumulated CPU time (s) 3.08
Current children cumulated vsize (KiB) 10408

[startup+6.30023 s]
/proc/loadavg: 2.02 2.01 2.00 3/177 23122
/proc/meminfo: memFree=26437136/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=11548 CPUtime=6.28 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 2560 0 0 0 627 1 0 0 20 0 1 0 343124047 11825152 2123 33554432000 134512640 135840783 4293822384 4293621676 7623728 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 2887 2123 346 325 0 1823 0
Current children cumulated CPU time (s) 6.28
Current children cumulated vsize (KiB) 11548

[startup+12.7003 s]
/proc/loadavg: 2.02 2.01 2.00 3/177 23122
/proc/meminfo: memFree=26422172/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=25984 CPUtime=12.69 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 6181 0 0 0 1266 3 0 0 20 0 1 0 343124047 26607616 5744 33554432000 134512640 135840783 4293822384 4293821744 134551312 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 6496 5744 349 325 0 5432 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 25984

[startup+25.5003 s]
/proc/loadavg: 2.01 2.01 2.00 3/177 23122
/proc/meminfo: memFree=26391188/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=48644 CPUtime=25.48 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 11841 0 0 0 2544 4 0 0 20 0 1 0 343124047 49811456 11404 33554432000 134512640 135840783 4293822384 4293821744 134550670 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 12161 11404 349 325 0 11097 0
Current children cumulated CPU time (s) 25.48
Current children cumulated vsize (KiB) 48644

[startup+51.1003 s]
/proc/loadavg: 2.07 2.02 2.00 3/177 23122
/proc/meminfo: memFree=26320584/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=108196 CPUtime=51.08 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 26725 0 0 0 5102 6 0 0 20 0 1 0 343124047 110792704 26288 33554432000 134512640 135840783 4293822384 4293806944 134586828 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 27049 26288 349 325 0 25985 0
Current children cumulated CPU time (s) 51.08
Current children cumulated vsize (KiB) 108196

[startup+102.3 s]
/proc/loadavg: 2.03 2.02 2.00 3/177 23122
/proc/meminfo: memFree=26142704/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=232780 CPUtime=102.27 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 57879 0 0 0 10214 13 0 0 20 0 1 0 343124047 238366720 57442 33554432000 134512640 135840783 4293822384 4293821696 134551093 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 58195 57442 349 325 0 57131 0
Current children cumulated CPU time (s) 102.27
Current children cumulated vsize (KiB) 232780

[startup+162.3 s]
/proc/loadavg: 2.01 2.01 2.00 3/177 23123
/proc/meminfo: memFree=25946068/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=387964 CPUtime=162.26 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 96657 0 0 0 16204 22 0 0 20 0 1 0 343124047 397275136 96220 33554432000 134512640 135840783 4293822384 4293821744 134551093 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 96991 96220 349 325 0 95927 0
Current children cumulated CPU time (s) 162.26
Current children cumulated vsize (KiB) 387964

[startup+222.307 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 23123
/proc/meminfo: memFree=25768656/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=520632 CPUtime=222.26 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 129839 0 0 0 22198 28 0 0 20 0 1 0 343124047 533127168 129402 33554432000 134512640 135840783 4293822384 4293821744 134550893 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 130158 129402 349 325 0 129094 0
Current children cumulated CPU time (s) 222.26
Current children cumulated vsize (KiB) 520632

[startup+282.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 23123
/proc/meminfo: memFree=25611204/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=617752 CPUtime=282.24 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 154120 0 0 0 28190 34 0 0 20 0 1 0 343124047 632578048 153683 33554432000 134512640 135840783 4293822384 4293821744 134550893 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 154438 153683 349 325 0 153374 0
Current children cumulated CPU time (s) 282.24
Current children cumulated vsize (KiB) 617752

[startup+342.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 23123
/proc/meminfo: memFree=25462808/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=728232 CPUtime=342.23 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 181756 0 0 0 34182 41 0 0 20 0 1 0 343124047 745709568 181319 33554432000 134512640 135840783 4293822384 4293821744 134551071 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 182058 181319 349 325 0 180994 0
Current children cumulated CPU time (s) 342.23
Current children cumulated vsize (KiB) 728232

[startup+402.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 23123
/proc/meminfo: memFree=25295188/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=870864 CPUtime=402.22 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 217397 0 0 0 40172 50 0 0 20 0 1 0 343124047 891764736 216960 33554432000 134512640 135840783 4293822384 4293787984 134586828 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 217716 216960 349 325 0 216652 0
Current children cumulated CPU time (s) 402.22
Current children cumulated vsize (KiB) 870864

[startup+462.3 s]
/proc/loadavg: 1.97 1.98 1.99 3/178 23166
/proc/meminfo: memFree=26829192/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=1019496 CPUtime=462.19 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 254600 0 0 0 46161 58 0 0 20 0 1 0 343124047 1043963904 254148 33554432000 134512640 135840783 4293822384 4293821696 134551084 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 254874 254148 349 325 0 253810 0
Current children cumulated CPU time (s) 462.19
Current children cumulated vsize (KiB) 1019496

[startup+522.3 s]
/proc/loadavg: 2.04 2.00 2.00 3/177 23166
/proc/meminfo: memFree=26635016/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=1138004 CPUtime=522.16 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 284271 0 0 0 52153 63 0 0 20 0 1 0 343124047 1165316096 283780 33554432000 134512640 135840783 4293822384 4293821744 134551071 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 284501 283780 349 325 0 283437 0
Current children cumulated CPU time (s) 522.16
Current children cumulated vsize (KiB) 1138004

[startup+582.3 s]
/proc/loadavg: 2.01 2.00 2.00 3/177 23166
/proc/meminfo: memFree=26427232/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=1238172 CPUtime=582.13 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 309340 0 0 0 58146 67 0 0 20 0 1 0 343124047 1267888128 308811 33554432000 134512640 135840783 4293822384 4293821744 134550913 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 309543 308811 349 325 0 308479 0
Current children cumulated CPU time (s) 582.13
Current children cumulated vsize (KiB) 1238172

[startup+642.3 s]

################
# More data... #
################

[startup+882.307 s]
/proc/loadavg: 2.02 2.01 2.00 3/177 23167
/proc/meminfo: memFree=25738428/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=1539848 CPUtime=882.01 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 384944 0 0 0 88114 87 0 0 20 0 1 0 343124047 1576804352 384247 33554432000 134512640 135840783 4293822384 4293821744 134550804 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 384962 384247 349 325 0 383898 0
Current children cumulated CPU time (s) 882.01
Current children cumulated vsize (KiB) 1539848

[startup+942.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 23167
/proc/meminfo: memFree=25612348/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=1602396 CPUtime=941.97 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 400589 0 0 0 94107 90 0 0 20 0 1 0 343124047 1640853504 399861 33554432000 134512640 135840783 4293822384 4293821744 134551186 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 400599 399861 349 325 0 399535 0
Current children cumulated CPU time (s) 941.97
Current children cumulated vsize (KiB) 1602396

[startup+1002.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 23167
/proc/meminfo: memFree=25510448/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=1653740 CPUtime=1001.95 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 413477 0 0 0 100102 93 0 0 20 0 1 0 343124047 1693429760 412717 33554432000 134512640 135840783 4293822384 4293821664 2317966 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 413435 412717 349 325 0 412371 0
Current children cumulated CPU time (s) 1001.95
Current children cumulated vsize (KiB) 1653740

[startup+1062.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 23171
/proc/meminfo: memFree=25399048/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=1723352 CPUtime=1061.92 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 430901 0 0 0 106093 99 0 0 20 0 1 0 343124047 1764712448 430113 33554432000 134512640 135840783 4293822384 4293821744 134551084 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 430838 430113 349 325 0 429774 0
Current children cumulated CPU time (s) 1061.92
Current children cumulated vsize (KiB) 1723352

[startup+1122.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 23171
/proc/meminfo: memFree=25282028/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=1767060 CPUtime=1121.9 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 441842 0 0 0 112087 103 0 0 20 0 1 0 343124047 1809469440 441022 33554432000 134512640 135840783 4293822384 4293821744 134551330 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 441765 441022 349 325 0 440701 0
Current children cumulated CPU time (s) 1121.9
Current children cumulated vsize (KiB) 1767060

[startup+1182.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 23171
/proc/meminfo: memFree=25191288/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=1803412 CPUtime=1181.87 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 450980 0 0 0 118082 105 0 0 20 0 1 0 343124047 1846693888 450131 33554432000 134512640 135840783 4293822384 4293821744 134550872 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 450853 450131 349 325 0 449789 0
Current children cumulated CPU time (s) 1181.87
Current children cumulated vsize (KiB) 1803412

[startup+1242.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 23171
/proc/meminfo: memFree=25102408/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=1838248 CPUtime=1241.85 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 459710 0 0 0 124077 108 0 0 20 0 1 0 343124047 1882365952 458827 33554432000 134512640 135840783 4293822384 4293821744 134551071 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 459562 458827 349 325 0 458498 0
Current children cumulated CPU time (s) 1241.85
Current children cumulated vsize (KiB) 1838248

[startup+1302.3 s]
/proc/loadavg: 2.06 2.02 2.00 3/177 23171
/proc/meminfo: memFree=24970996/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=1901444 CPUtime=1301.82 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 475557 0 0 0 130069 113 0 0 20 0 1 0 343124047 1947078656 474637 33554432000 134512640 135840783 4293822384 4293821744 134550893 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 475361 474637 349 325 0 474297 0
Current children cumulated CPU time (s) 1301.82
Current children cumulated vsize (KiB) 1901444

[startup+1362.3 s]
/proc/loadavg: 2.02 2.01 2.00 3/176 23171
/proc/meminfo: memFree=24863648/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=1952748 CPUtime=1361.79 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 488428 0 0 0 136062 117 0 0 20 0 1 0 343124047 1999613952 487467 33554432000 134512640 135840783 4293822384 4293821744 134551741 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 488187 487467 349 325 0 487123 0
Current children cumulated CPU time (s) 1361.79
Current children cumulated vsize (KiB) 1952748

[startup+1422.3 s]
/proc/loadavg: 2.01 2.00 2.00 3/177 23172
/proc/meminfo: memFree=24726648/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=2042868 CPUtime=1421.76 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 510992 0 0 0 142055 121 0 0 20 0 1 0 343124047 2091896832 509996 33554432000 134512640 135840783 4293822384 4293821744 134551084 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 510717 509996 349 325 0 509653 0
Current children cumulated CPU time (s) 1421.76
Current children cumulated vsize (KiB) 2042868

[startup+1482.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 23172
/proc/meminfo: memFree=24604288/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=2096484 CPUtime=1481.73 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 524427 0 0 0 148048 125 0 0 20 0 1 0 343124047 2146799616 523400 33554432000 134512640 135840783 4293822384 4293821744 134550764 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 524121 523400 349 325 0 523057 0
Current children cumulated CPU time (s) 1481.73
Current children cumulated vsize (KiB) 2096484

[startup+1542.31 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 23172
/proc/meminfo: memFree=24472504/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=2163404 CPUtime=1541.71 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 541180 0 0 0 154042 129 0 0 20 0 1 0 343124047 2215325696 540113 33554432000 134512640 135840783 4293822384 4293821744 134551048 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 540851 540113 349 325 0 539787 0
Current children cumulated CPU time (s) 1541.71
Current children cumulated vsize (KiB) 2163404

[startup+1602.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 23172
/proc/meminfo: memFree=24357100/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=2220712 CPUtime=1601.68 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 555540 0 0 0 160036 132 0 0 20 0 1 0 343124047 2274009088 554435 33554432000 134512640 135840783 4293822384 4293821744 134551063 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 555178 554435 349 325 0 554114 0
Current children cumulated CPU time (s) 1601.68
Current children cumulated vsize (KiB) 2220712

[startup+1662.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 23175
/proc/meminfo: memFree=24247448/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=2258596 CPUtime=1661.65 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 565061 0 0 0 166031 134 0 0 20 0 1 0 343124047 2312802304 563918 33554432000 134512640 135840783 4293822384 4293821744 134550779 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 564649 563918 349 325 0 563585 0
Current children cumulated CPU time (s) 1661.65
Current children cumulated vsize (KiB) 2258596

[startup+1722.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/179 23199
/proc/meminfo: memFree=24151164/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=2290136 CPUtime=1721.61 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 572981 0 0 0 172025 136 0 0 20 0 1 0 343124047 2345099264 571802 33554432000 134512640 135840783 4293822384 4293821744 134550764 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 572534 571802 349 325 0 571470 0
Current children cumulated CPU time (s) 1721.61
Current children cumulated vsize (KiB) 2290136

[startup+1782.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 23199
/proc/meminfo: memFree=24034920/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=2347104 CPUtime=1781.59 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 587269 0 0 0 178018 141 0 0 20 0 1 0 343124047 2403434496 586057 33554432000 134512640 135840783 4293822384 4293821744 134550779 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 586776 586057 349 325 0 585712 0
Current children cumulated CPU time (s) 1781.59
Current children cumulated vsize (KiB) 2347104



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1800.81 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 23199
/proc/meminfo: memFree=23978356/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=2356452 CPUtime=1800.1 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 589621 0 0 0 179868 142 0 0 20 0 1 0 343124047 2413006848 588396 33554432000 134512640 135840783 4293822384 4293821744 134550893 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 589113 588396 349 325 0 588049 0
Current children cumulated CPU time (s) 1800.1
Current children cumulated vsize (KiB) 2356452

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1800.81 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 23199
/proc/meminfo: memFree=23978356/32873648 swapFree=6864/6864
[pid=23121] ppid=23119 vsize=2356452 CPUtime=1800.1 cores=1,3,5,7
/proc/23121/stat : 23121 (bsolo3.0.17) R 23119 23121 20253 0 -1 4202496 589621 0 0 0 179868 142 0 0 20 0 1 0 343124047 2413006848 588396 33554432000 134512640 135840783 4293822384 4293821744 134550893 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/23121/statm: 589113 588396 349 325 0 588049 0
Current children cumulated CPU time (s) 1800.1
Current children cumulated vsize (KiB) 2356452

Child status: 15
Real time (s): 1800.83
CPU time (s): 1800.12
CPU user time (s): 1798.68
CPU system time (s): 1.43378
CPU usage (%): 99.9605
Max. virtual memory (cumulated for all children) (KiB): 2356452

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1798.68
system time used= 1.43378
maximum resident set size= 2353584
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 589624
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 858
involuntary context switches= 1936

runsolver used 3.43248 second user time and 9.34158 second system time

The end

Launcher Data

Begin job on node113 at 2012-06-04 02:32:52
IDJOB=3742334
IDBENCH=90161
IDSOLVER=2318
FILE ID=node113/3742334-1338769972
RUNJOBID= node113-1338748314-20270
PBS_JOBID= 14636785
Free space on /tmp= 70972 MiB

SOLVER NAME= PB07: bsolo 3.0.17
BENCH NAME= PB12/normalized-PB12/DEC-SMALLINT-LIN/sroussel/ShortestPathNG/normalized-NG_K129.opb
COMMAND LINE= bsolo3.0.17 -mMEMLIMIT BENCHNAME            
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3742334-1338769972/watcher-3742334-1338769972 -o /tmp/evaluation-result-3742334-1338769972/solver-3742334-1338769972 -C 1800 -W 1900 -M 15500  bsolo3.0.17 -m15500 HOME/instance-3742334-1338769972.opb            

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 1a22ba01765c232fe0c8623cfb5d750f
RANDOM SEED=191729497

node113.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.026
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.026
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.026
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5438.54
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.026
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.026
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.026
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.026
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.026
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873648 kB
MemFree:        26446688 kB
Buffers:          538656 kB
Cached:          3499472 kB
SwapCached:         2028 kB
Active:          3237672 kB
Inactive:        2287100 kB
Active(anon):    1485908 kB
Inactive(anon):     1912 kB
Active(file):    1751764 kB
Inactive(file):  2285188 kB
Unevictable:        6864 kB
Mlocked:            6864 kB
SwapTotal:      67108856 kB
SwapFree:       67099708 kB
Dirty:              7788 kB
Writeback:             0 kB
AnonPages:       1490100 kB
Mapped:            12576 kB
Shmem:                 0 kB
Slab:             763680 kB
SReclaimable:     207736 kB
SUnreclaim:       555944 kB
KernelStack:        1472 kB
PageTables:         7092 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545680 kB
Committed_AS:    1606100 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346400 kB
VmallocChunk:   34341988052 kB
HardwareCorrupted:     0 kB
AnonHugePages:     16384 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7680 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70964 MiB
End job on node113 at 2012-06-04 03:02:55