Trace number 3740907

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PB07: bsolo 3.0.17? (TO) 1800.07 1800.51

General information on the benchmark

Name/DEC-SMALLINT-LIN/lopes/
normalized-101.opb
MD5SUMa3ed9477da365050d631dd031519efe8
Bench CategoryDEC-SMALLINT-LIN (no optimisation, small integers, linear constraints)
Best result obtained on this benchmarkUNSAT
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark79.04
Has Objective FunctionNO
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables7936
Total number of constraints176837
Number of constraints which are clauses158928
Number of constraints which are cardinality constraints (but not clauses)563
Number of constraints which are nor clauses,nor cardinality constraints17346
Minimum length of a constraint2
Maximum length of a constraint195
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 3
Number of bits of the biggest number in a constraint 2
Biggest sum of numbers in a constraint 196
Number of bits of the biggest sum of numbers8
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c bsolo beta version 3.0.17 - 27/04/2007 : 1600 GMT
0.00/0.00	c Developed by Vasco Manquinho IST/UTL - INESC-ID
0.00/0.00	c type "bsolo -h" for help
0.00/0.00	c Time Limit set via environment variable PBTIMEOUT to 1800
0.00/0.00	c Memory Limit set to 15500 MB
0.00/0.00	c Instance file HOME/instance-3740907-1338753927.opb
0.00/0.00	c File size is 7426750 bytes.
0.48/0.57	c Highest Coefficient sum: 196
0.48/0.57	c Parsing was ok!!
0.48/0.57	c Total parsing time: 0.563 s
0.58/0.60	c Var: 7936 Constr: 177125 159215/564/17346 Lit: 683558 Watch. Lit: 357405
0.58/0.60	c Obj. Vars: 0 (0 % of total variables)
6.18/6.20	c Pre-processing Time: 6.193 s
6.18/6.20	c    Confl   Vars     Ctrs     Lits  LPC   W.Lits     Max Learnt Conf/s Time
6.18/6.20	c        0   7936   174624   671054    3   369749  237455      0      0 6.19
6.18/6.20	c Switching off LPR mode.
1800.05/1800.50	c      101   7936   174725   684979    3   367013  261201    101     16 6.24
1800.05/1800.50	c      251   7936   174875   690093    3   365625  287322    251     39 6.28
1800.05/1800.50	c      478   7936   175102   697041    3   366258  316055    478     74 6.39
1800.05/1800.50	c      818   7936   175442   707819    4   366812  347661    818    125 6.53
1800.05/1800.50	c     1326   7936   175950   718695    4   367829  382428   1326    192 6.89
1800.05/1800.50	c     2090   7936   176714   745532    4   369341  420671   2090    294 7.10
1800.05/1800.50	c     3236   7936   177860   777464    4   371861  462739   3236    426 7.58
1800.05/1800.50	c     4957   7936   179581   833106    4   375588  509013   4957    596 8.31
1800.05/1800.50	c     7543   7936   182167   956221    5   380678  559915   7543    793 9.50
1800.05/1800.50	c    11423   7936   186047  1148910    6   388765  615907  11423   1017 11.22
1800.05/1800.50	c    17246   7936   191870  1434122    7   400206  677498  17246   1059 16.29
1800.05/1800.50	c    25983   7936   200607  2109436   10   418034  745248  25983   1199 21.66
1800.05/1800.50	c    39093   7936   213717  2748534   12   444106  819773  39093    880 44.40
1800.05/1800.50	c    58763   7936   233387  6190866   26   483233  901751  58763    888 66.17
1800.05/1800.50	c    88274   7936   262898  7525903   28   541715  991927  88274    501 175.94
1800.05/1800.50	c   132544   7933   307168  9208477   29   625700  500000 132544    342 387.50
1800.05/1800.50	c   198954   7932   373578 13057820   34   759317  550001 198954    218 911.59
1800.05/1800.50	c   298574   7931   473198 21966577   46   960685  605002 298574    229 1299.92
1800.05/1800.50	c Caught signal: 15
1800.05/1800.50	c  Rst    Confl    NCB      Dec     Ctrs     Lits Learnt Conf/s Time
1800.05/1800.50	c   18   417195 187051  3844407   591819 35999839 417195    231 1800.07
1800.05/1800.50	s UNKNOWN
1800.05/1800.50	c Total time: 1800.07 s

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3740907-1338753927/watcher-3740907-1338753927 -o /tmp/evaluation-result-3740907-1338753927/solver-3740907-1338753927 -C 1800 -W 1900 -M 15500 bsolo3.0.17 -m15500 HOME/instance-3740907-1338753927.opb 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.92 1.98 1.96 5/176 22027
/proc/meminfo: memFree=27287744/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=2660 CPUtime=0 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) D 22025 22027 20253 0 -1 4194304 95 0 0 0 0 0 0 0 20 0 1 0 341519548 2723840 47 33554432000 134512640 135840783 4289349760 4289347796 11203524 0 0 4096 0 18446744072100609316 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 665 47 37 325 0 32 0

[startup+0.0872819 s]
/proc/loadavg: 1.92 1.98 1.96 5/176 22027
/proc/meminfo: memFree=27287744/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=15220 CPUtime=0.07 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 1569 0 0 0 7 0 0 0 20 0 1 0 341519548 15585280 1505 33554432000 134512640 135840783 4289349760 4289348448 1692967 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 3805 1505 590 325 0 927 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 15220

[startup+0.100286 s]
/proc/loadavg: 1.92 1.98 1.96 5/176 22027
/proc/meminfo: memFree=27287744/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=15772 CPUtime=0.09 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 1754 0 0 0 9 0 0 0 20 0 1 0 341519548 16150528 1690 33554432000 134512640 135840783 4289349760 4289348960 134528207 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 3943 1690 636 325 0 1065 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 15772

[startup+0.300223 s]
/proc/loadavg: 1.92 1.98 1.96 5/176 22027
/proc/meminfo: memFree=27287744/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=23684 CPUtime=0.28 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 4366 0 0 0 28 0 0 0 20 0 1 0 341519548 24252416 4302 33554432000 134512640 135840783 4289349760 4289348416 1692967 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 5921 4302 1286 325 0 3043 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 23684

[startup+0.700288 s]
/proc/loadavg: 1.92 1.98 1.96 5/176 22027
/proc/meminfo: memFree=27287744/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=36200 CPUtime=0.68 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 10158 0 0 0 67 1 0 0 20 0 1 0 341519548 37068800 8280 33554432000 134512640 135840783 4289349760 4289253644 5547056 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 9050 8280 344 325 0 7986 0
Current children cumulated CPU time (s) 0.68
Current children cumulated vsize (KiB) 36200

[startup+1.50027 s]
/proc/loadavg: 1.92 1.98 1.96 3/177 22028
/proc/meminfo: memFree=27244808/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=47072 CPUtime=1.48 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 12909 0 0 0 146 2 0 0 20 0 1 0 341519548 48201728 11031 33554432000 134512640 135840783 4289349760 4289253468 5547056 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 11768 11031 346 325 0 10704 0
Current children cumulated CPU time (s) 1.48
Current children cumulated vsize (KiB) 47072

[startup+3.10028 s]
/proc/loadavg: 1.92 1.98 1.96 3/177 22028
/proc/meminfo: memFree=27243420/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=47204 CPUtime=3.08 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 12912 0 0 0 304 4 0 0 20 0 1 0 341519548 48336896 11034 33554432000 134512640 135840783 4289349760 4289253468 5547056 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 11801 11034 346 325 0 10737 0
Current children cumulated CPU time (s) 3.08
Current children cumulated vsize (KiB) 47204

[startup+6.3003 s]
/proc/loadavg: 1.93 1.98 1.96 3/177 22028
/proc/meminfo: memFree=27241932/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=36348 CPUtime=6.28 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 15438 0 0 0 622 6 0 0 20 0 1 0 341519548 37220352 8353 33554432000 134512640 135840783 4289349760 4289349120 134551081 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 9087 8353 349 325 0 8023 0
Current children cumulated CPU time (s) 6.28
Current children cumulated vsize (KiB) 36348

[startup+12.7003 s]
/proc/loadavg: 1.93 1.98 1.96 3/176 22028
/proc/meminfo: memFree=27243700/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=44440 CPUtime=12.68 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 17441 0 0 0 1261 7 0 0 20 0 1 0 341519548 45506560 10356 33554432000 134512640 135840783 4289349760 4289349120 134551048 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 11110 10356 349 325 0 10046 0
Current children cumulated CPU time (s) 12.68
Current children cumulated vsize (KiB) 44440

[startup+25.5003 s]
/proc/loadavg: 1.95 1.98 1.96 3/177 22029
/proc/meminfo: memFree=27224116/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=58904 CPUtime=25.48 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 21070 0 0 0 2540 8 0 0 20 0 1 0 341519548 60317696 13985 33554432000 134512640 135840783 4289349760 4289349120 134551081 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 14726 13985 349 325 0 13662 0
Current children cumulated CPU time (s) 25.48
Current children cumulated vsize (KiB) 58904

[startup+51.1003 s]
/proc/loadavg: 1.96 1.98 1.96 3/177 22029
/proc/meminfo: memFree=27193612/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=79496 CPUtime=51.08 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 26222 0 0 0 5098 10 0 0 20 0 1 0 341519548 81403904 19137 33554432000 134512640 135840783 4289349760 4289349120 134551312 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 19874 19137 349 325 0 18810 0
Current children cumulated CPU time (s) 51.08
Current children cumulated vsize (KiB) 79496

[startup+102.306 s]
/proc/loadavg: 1.98 1.98 1.96 3/177 22029
/proc/meminfo: memFree=27122812/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=120776 CPUtime=102.27 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 36545 0 0 0 10215 12 0 0 20 0 1 0 341519548 123674624 29460 33554432000 134512640 135840783 4289349760 4289349120 134551048 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 30194 29460 349 325 0 29130 0
Current children cumulated CPU time (s) 102.27
Current children cumulated vsize (KiB) 120776

[startup+162.3 s]
/proc/loadavg: 2.03 2.00 1.96 3/177 22029
/proc/meminfo: memFree=27075692/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=131000 CPUtime=162.26 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 39083 0 0 0 16211 15 0 0 20 0 1 0 341519548 134144000 31998 33554432000 134512640 135840783 4289349760 4289348848 1692967 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 32750 31998 349 325 0 31686 0
Current children cumulated CPU time (s) 162.26
Current children cumulated vsize (KiB) 131000

[startup+222.3 s]
/proc/loadavg: 2.01 2.00 1.96 3/177 22029
/proc/meminfo: memFree=27025844/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=141628 CPUtime=222.25 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 41737 0 0 0 22208 17 0 0 20 0 1 0 341519548 145027072 34652 33554432000 134512640 135840783 4289349760 4289349120 134551186 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 35407 34652 349 325 0 34343 0
Current children cumulated CPU time (s) 222.25
Current children cumulated vsize (KiB) 141628

[startup+282.3 s]
/proc/loadavg: 2.00 2.00 1.96 3/178 22032
/proc/meminfo: memFree=26991232/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=148672 CPUtime=282.24 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 43505 0 0 0 28206 18 0 0 20 0 1 0 341519548 152240128 36420 33554432000 134512640 135840783 4289349760 4289349120 134551048 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 37168 36420 349 325 0 36104 0
Current children cumulated CPU time (s) 282.24
Current children cumulated vsize (KiB) 148672

[startup+342.3 s]
/proc/loadavg: 2.05 2.01 1.97 3/177 22033
/proc/meminfo: memFree=26960008/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=155452 CPUtime=342.23 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 45199 0 0 0 34204 19 0 0 20 0 1 0 341519548 159182848 38114 33554432000 134512640 135840783 4289349760 4289349120 134551058 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 38863 38114 349 325 0 37799 0
Current children cumulated CPU time (s) 342.23
Current children cumulated vsize (KiB) 155452

[startup+402.3 s]
/proc/loadavg: 2.07 2.02 1.97 3/177 22072
/proc/meminfo: memFree=27754332/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=160232 CPUtime=402.22 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 46426 0 0 0 40202 20 0 0 20 0 1 0 341519548 164077568 39337 33554432000 134512640 135840783 4289349760 4289349120 134550764 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 40058 39337 349 325 0 38994 0
Current children cumulated CPU time (s) 402.22
Current children cumulated vsize (KiB) 160232

[startup+462.3 s]
/proc/loadavg: 2.06 2.03 1.98 4/177 22072
/proc/meminfo: memFree=27719380/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=168600 CPUtime=462.17 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 48549 0 0 0 46197 20 0 0 20 0 1 0 341519548 172646400 41415 33554432000 134512640 135840783 4289349760 4289349120 134551065 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 42150 41415 349 325 0 41086 0
Current children cumulated CPU time (s) 462.17
Current children cumulated vsize (KiB) 168600

[startup+522.3 s]
/proc/loadavg: 2.02 2.02 1.98 4/177 22072
/proc/meminfo: memFree=27701400/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=173640 CPUtime=522.15 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 49817 0 0 0 52194 21 0 0 20 0 1 0 341519548 177807360 42668 33554432000 134512640 135840783 4289349760 4289349120 134551182 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 43410 42668 349 325 0 42346 0
Current children cumulated CPU time (s) 522.15
Current children cumulated vsize (KiB) 173640

[startup+582.3 s]
/proc/loadavg: 2.00 2.02 1.98 4/177 22072
/proc/meminfo: memFree=27689228/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=178680 CPUtime=582.14 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 51094 0 0 0 58192 22 0 0 20 0 1 0 341519548 182968320 43945 33554432000 134512640 135840783 4289349760 4289349120 134550764 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 44670 43945 349 325 0 43606 0
Current children cumulated CPU time (s) 582.14
Current children cumulated vsize (KiB) 178680

[startup+642.3 s]

################
# More data... #
################

[startup+882.3 s]
/proc/loadavg: 2.00 2.00 1.98 4/178 22076
/proc/meminfo: memFree=27590336/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=207456 CPUtime=882.08 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 58309 0 0 0 88181 27 0 0 20 0 1 0 341519548 212434944 51147 33554432000 134512640 135840783 4289349760 4289349072 134550804 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 51864 51147 349 325 0 50800 0
Current children cumulated CPU time (s) 882.08
Current children cumulated vsize (KiB) 207456

[startup+942.3 s]
/proc/loadavg: 2.00 2.00 1.98 4/177 22077
/proc/meminfo: memFree=27544524/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=222156 CPUtime=942.08 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 61969 0 0 0 94179 29 0 0 20 0 1 0 341519548 227487744 54807 33554432000 134512640 135840783 4289349760 4289349120 134550893 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 55539 54807 349 325 0 54475 0
Current children cumulated CPU time (s) 942.08
Current children cumulated vsize (KiB) 222156

[startup+1002.3 s]
/proc/loadavg: 2.00 2.00 1.98 4/177 22077
/proc/meminfo: memFree=27507224/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=251440 CPUtime=1002.06 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 69293 0 0 0 100174 32 0 0 20 0 1 0 341519548 257474560 62131 33554432000 134512640 135840783 4289349760 4289349120 134550764 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 62860 62131 349 325 0 61796 0
Current children cumulated CPU time (s) 1002.06
Current children cumulated vsize (KiB) 251440

[startup+1062.3 s]
/proc/loadavg: 2.00 2.00 1.98 4/177 22077
/proc/meminfo: memFree=27481932/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=271968 CPUtime=1062.06 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 74414 0 0 0 106171 35 0 0 20 0 1 0 341519548 278495232 67252 33554432000 134512640 135840783 4289349760 4289349120 134550924 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 67992 67252 349 325 0 66928 0
Current children cumulated CPU time (s) 1062.06
Current children cumulated vsize (KiB) 271968

[startup+1122.3 s]
/proc/loadavg: 2.00 2.00 1.98 4/177 22077
/proc/meminfo: memFree=27439544/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=308400 CPUtime=1122.03 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 83560 0 0 0 112164 39 0 0 20 0 1 0 341519548 315801600 76369 33554432000 134512640 135840783 4289349760 4289349120 134551365 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 77100 76369 349 325 0 76036 0
Current children cumulated CPU time (s) 1122.03
Current children cumulated vsize (KiB) 308400

[startup+1182.3 s]
/proc/loadavg: 2.00 2.00 1.98 4/176 22077
/proc/meminfo: memFree=27424040/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=319656 CPUtime=1181.99 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 86411 0 0 0 118158 41 0 0 20 0 1 0 341519548 327327744 79191 33554432000 134512640 135840783 4289349760 4289349072 134550764 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 79914 79191 349 325 0 78850 0
Current children cumulated CPU time (s) 1181.99
Current children cumulated vsize (KiB) 319656

[startup+1242.3 s]
/proc/loadavg: 2.00 2.00 1.98 4/177 22079
/proc/meminfo: memFree=27413708/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=325752 CPUtime=1241.99 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 87940 0 0 0 124157 42 0 0 20 0 1 0 341519548 333570048 80720 33554432000 134512640 135840783 4289349760 4289349120 134550804 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 81438 80720 349 325 0 80374 0
Current children cumulated CPU time (s) 1241.99
Current children cumulated vsize (KiB) 325752

[startup+1302.3 s]
/proc/loadavg: 2.00 2.00 1.98 4/177 22079
/proc/meminfo: memFree=27393908/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=340600 CPUtime=1301.99 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 91625 0 0 0 130155 44 0 0 20 0 1 0 341519548 348774400 84405 33554432000 134512640 135840783 4289349760 4289349120 134550884 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 85150 84405 349 325 0 84086 0
Current children cumulated CPU time (s) 1301.99
Current children cumulated vsize (KiB) 340600

[startup+1362.3 s]
/proc/loadavg: 2.00 2.00 1.98 4/177 22079
/proc/meminfo: memFree=27372292/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=359284 CPUtime=1361.97 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 96319 0 0 0 136152 45 0 0 20 0 1 0 341519548 367906816 89099 33554432000 134512640 135840783 4289349760 4289348896 1690254 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 89821 89099 349 325 0 88757 0
Current children cumulated CPU time (s) 1361.97
Current children cumulated vsize (KiB) 359284

[startup+1422.31 s]
/proc/loadavg: 2.06 2.02 1.98 4/177 22079
/proc/meminfo: memFree=27343896/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=383152 CPUtime=1421.96 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 102266 0 0 0 142148 48 0 0 20 0 1 0 341519548 392347648 95046 33554432000 134512640 135840783 4289349760 4289349120 134550779 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 95788 95046 349 325 0 94724 0
Current children cumulated CPU time (s) 1421.96
Current children cumulated vsize (KiB) 383152

[startup+1482.3 s]
/proc/loadavg: 2.02 2.01 1.98 4/178 22082
/proc/meminfo: memFree=27305364/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=418492 CPUtime=1481.95 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 111109 0 0 0 148142 53 0 0 20 0 1 0 341519548 428535808 103889 33554432000 134512640 135840783 4289349760 4289349072 134551048 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 104623 103889 349 325 0 103559 0
Current children cumulated CPU time (s) 1481.95
Current children cumulated vsize (KiB) 418492

[startup+1542.3 s]
/proc/loadavg: 2.01 2.00 1.98 4/177 22083
/proc/meminfo: memFree=27278464/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=440600 CPUtime=1541.9 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 116695 0 0 0 154135 55 0 0 20 0 1 0 341519548 451174400 109435 33554432000 134512640 135840783 4289349760 4289349120 134551330 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 110150 109435 349 325 0 109086 0
Current children cumulated CPU time (s) 1541.9
Current children cumulated vsize (KiB) 440600

[startup+1602.3 s]
/proc/loadavg: 2.00 2.00 1.98 4/177 22083
/proc/meminfo: memFree=27260124/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=456224 CPUtime=1601.88 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 120612 0 0 0 160130 58 0 0 20 0 1 0 341519548 467173376 113328 33554432000 134512640 135840783 4289349760 4289349120 134550764 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 114056 113328 349 325 0 112992 0
Current children cumulated CPU time (s) 1601.88
Current children cumulated vsize (KiB) 456224

[startup+1662.3 s]
/proc/loadavg: 2.00 2.00 1.98 4/177 22083
/proc/meminfo: memFree=27233836/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=477312 CPUtime=1661.87 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 125892 0 0 0 166126 61 0 0 20 0 1 0 341519548 488767488 118608 33554432000 134512640 135840783 4289349760 4289349120 134550764 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 119328 118608 349 325 0 118264 0
Current children cumulated CPU time (s) 1661.87
Current children cumulated vsize (KiB) 477312

[startup+1722.3 s]
/proc/loadavg: 2.00 2.00 1.98 4/177 22083
/proc/meminfo: memFree=27202340/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=505556 CPUtime=1721.86 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 132954 0 0 0 172122 64 0 0 20 0 1 0 341519548 517689344 125670 33554432000 134512640 135840783 4289349760 4289349120 134550872 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 126389 125670 349 325 0 125325 0
Current children cumulated CPU time (s) 1721.86
Current children cumulated vsize (KiB) 505556

[startup+1782.3 s]
/proc/loadavg: 2.00 2.00 1.98 4/177 22083
/proc/meminfo: memFree=27178904/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=525668 CPUtime=1781.85 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 137963 0 0 0 178119 66 0 0 20 0 1 0 341519548 538284032 130679 33554432000 134512640 135840783 4289349760 4289349060 1698326 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 131417 130679 349 325 0 130353 0
Current children cumulated CPU time (s) 1781.85
Current children cumulated vsize (KiB) 525668



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1800.5 s]
/proc/loadavg: 2.00 2.00 1.98 3/177 22083
/proc/meminfo: memFree=27171092/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=532288 CPUtime=1800.05 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 139638 0 0 0 179939 66 0 0 20 0 1 0 341519548 545062912 132354 33554432000 134512640 135840783 4289349760 4289349120 134550779 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 133072 132354 349 325 0 132008 0
Current children cumulated CPU time (s) 1800.05
Current children cumulated vsize (KiB) 532288

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1800.5 s]
/proc/loadavg: 2.00 2.00 1.98 3/177 22083
/proc/meminfo: memFree=27171092/32873648 swapFree=6864/6864
[pid=22027] ppid=22025 vsize=532288 CPUtime=1800.05 cores=0,2,4,6
/proc/22027/stat : 22027 (bsolo3.0.17) R 22025 22027 20253 0 -1 4202496 139638 0 0 0 179939 66 0 0 20 0 1 0 341519548 545062912 132354 33554432000 134512640 135840783 4289349760 4289349120 134550779 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/22027/statm: 133072 132354 349 325 0 132008 0
Current children cumulated CPU time (s) 1800.05
Current children cumulated vsize (KiB) 532288

Child status: 15
Real time (s): 1800.51
CPU time (s): 1800.07
CPU user time (s): 1799.4
CPU system time (s): 0.674897
CPU usage (%): 99.9756
Max. virtual memory (cumulated for all children) (KiB): 532288

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1799.4
system time used= 0.674897
maximum resident set size= 529416
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 139641
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 221
involuntary context switches= 1880

runsolver used 3.46947 second user time and 9.05762 second system time

The end

Launcher Data 

Begin job on node113 at 2012-06-03 22:05:27
IDJOB=3740907
IDBENCH=89675
IDSOLVER=2318
FILE ID=node113/3740907-1338753927
RUNJOBID= node113-1338748314-20269
PBS_JOBID= 14636785
Free space on /tmp= 70960 MiB

SOLVER NAME= PB07: bsolo 3.0.17
BENCH NAME= PB11/normalized-PB11/DEC-SMALLINT-LIN/lopes/normalized-101.opb
COMMAND LINE= bsolo3.0.17 -mMEMLIMIT BENCHNAME            
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3740907-1338753927/watcher-3740907-1338753927 -o /tmp/evaluation-result-3740907-1338753927/solver-3740907-1338753927 -C 1800 -W 1900 -M 15500  bsolo3.0.17 -m15500 HOME/instance-3740907-1338753927.opb            

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= a3ed9477da365050d631dd031519efe8
RANDOM SEED=2013555692

node113.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.026
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.026
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.026
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5438.54
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.026
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.026
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.026
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.026
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.026
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873648 kB
MemFree:        27287744 kB
Buffers:          519272 kB
Cached:          3495536 kB
SwapCached:         2028 kB
Active:          2418768 kB
Inactive:        2278356 kB
Active(anon):     681452 kB
Inactive(anon):     1912 kB
Active(file):    1737316 kB
Inactive(file):  2276444 kB
Unevictable:        6864 kB
Mlocked:            6864 kB
SwapTotal:      67108856 kB
SwapFree:       67099708 kB
Dirty:             14152 kB
Writeback:             0 kB
AnonPages:        686292 kB
Mapped:            12656 kB
Shmem:                 0 kB
Slab:             751788 kB
SReclaimable:     195744 kB
SUnreclaim:       556044 kB
KernelStack:        1448 kB
PageTables:         5356 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545680 kB
Committed_AS:     802312 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346400 kB
VmallocChunk:   34341988052 kB
HardwareCorrupted:     0 kB
AnonHugePages:    653312 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7680 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70952 MiB
End job on node113 at 2012-06-03 22:35:30