Trace number 3741037

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PB07: bsolo 3.0.17? (TO) 1800.08 1800.62

General information on the benchmark

Name/DEC-SMALLINT-LIN/lopes/
normalized-99.opb
MD5SUMa0ef64dc4a665e7cf00a752d7a124006
Bench CategoryDEC-SMALLINT-LIN (no optimisation, small integers, linear constraints)
Best result obtained on this benchmarkUNSAT
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark28.0957
Has Objective FunctionNO
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables7903
Total number of constraints176726
Number of constraints which are clauses158806
Number of constraints which are cardinality constraints (but not clauses)574
Number of constraints which are nor clauses,nor cardinality constraints17346
Minimum length of a constraint1
Maximum length of a constraint195
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 3
Number of bits of the biggest number in a constraint 2
Biggest sum of numbers in a constraint 196
Number of bits of the biggest sum of numbers8
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c bsolo beta version 3.0.17 - 27/04/2007 : 1600 GMT
0.00/0.00	c Developed by Vasco Manquinho IST/UTL - INESC-ID
0.00/0.00	c type "bsolo -h" for help
0.00/0.00	c Time Limit set via environment variable PBTIMEOUT to 1800
0.00/0.00	c Memory Limit set to 15500 MB
0.00/0.00	c Instance file HOME/instance-3741037-1338754863.opb
0.00/0.00	c File size is 7410770 bytes.
0.48/0.55	c Highest Coefficient sum: 196
0.48/0.55	c Parsing was ok!!
0.48/0.55	c Total parsing time: 0.543 s
0.48/0.58	c Var: 7903 Constr: 177025 159116/563/17346 Lit: 683308 Watch. Lit: 357159
0.48/0.58	c Obj. Vars: 0 (0 % of total variables)
5.89/5.96	c Pre-processing Time: 5.953 s
5.89/5.96	c    Confl   Vars     Ctrs     Lits  LPC   W.Lits     Max Learnt Conf/s Time
5.89/5.96	c        0   7571   164060   631340    3   348391  232510      0      0 5.95
5.89/5.96	c Switching off LPR mode.
1800.07/1800.61	c      103   7571   164163   637201    3   346705  255762    103     17 5.98
1800.07/1800.61	c      255   7571   164315   643449    3   345331  281339    255     42 6.03
1800.07/1800.61	c      480   7571   164540   648324    3   346143  309473    480     78 6.15
1800.07/1800.61	c      818   7571   164878   653965    3   346237  340421    818    128 6.36
1800.07/1800.61	c     1326   7571   165386   665769    4   347101  374464   1326    200 6.60
1800.07/1800.61	c     2089   7571   166149   709802    4   349060  411911   2089    305 6.84
1800.07/1800.61	c     3235   7571   167295   738085    4   349703  453103   3235    436 7.42
1800.07/1800.61	c     4956   7571   169016   812366    4   353099  498414   4956    602 8.22
1800.07/1800.61	c     7540   7571   171600  1019185    5   358955  548256   7540    794 9.49
1800.07/1800.61	c    11419   7571   175479  1257406    7   366396  603082  11419    981 11.63
1800.07/1800.61	c    17244   7571   181304  1513029    8   377978  663391  17244   1082 15.94
1800.07/1800.61	c    25981   7571   190041  2401851   12   396263  729731  25981   1220 21.29
1800.07/1800.61	c    39093   7571   203153  3722544   18   422250  802705  39093   1258 31.06
1800.07/1800.61	c    58765   7571   222825  6963853   31   460351  882976  58765   1190 49.35
1800.07/1800.61	c    88276   7571   252336  8348275   33   518946  971274  88276    731 120.69
1800.07/1800.61	c   132547   7570   296607 10542360   35   605764  500000 132547    411 321.93
1800.07/1800.61	c   198958   7569   363018 16006098   44   740425  550001 198958    367 541.13
1800.07/1800.61	c   298582   7562   462642 25550964   55   938022  605002 298582    266 1119.47
1800.07/1800.61	c Caught signal: 15
1800.07/1800.61	c  Rst    Confl    NCB      Dec     Ctrs     Lits Learnt Conf/s Time
1800.07/1800.61	c   18   410455 160035  3071330   574515 41247893 410455    228 1800.07
1800.07/1800.61	s UNKNOWN
1800.07/1800.61	c Total time: 1800.07 s

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3741037-1338754863/watcher-3741037-1338754863 -o /tmp/evaluation-result-3741037-1338754863/solver-3741037-1338754863 -C 1800 -W 1900 -M 15500 bsolo3.0.17 -m15500 HOME/instance-3741037-1338754863.opb 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.93 1.99 1.99 3/173 9850
/proc/meminfo: memFree=31240000/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=4476 CPUtime=0 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4194304 207 0 0 0 0 0 0 0 20 0 1 0 341614857 4583424 157 33554432000 134512640 135840783 4292944528 4292943868 1541464 0 0 4096 0 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 1119 157 136 325 0 39 0

[startup+0.0118781 s]
/proc/loadavg: 1.93 1.99 1.99 3/173 9850
/proc/meminfo: memFree=31240000/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=12064 CPUtime=0 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 519 0 0 0 0 0 0 0 20 0 1 0 341614857 12353536 455 33554432000 134512640 135840783 4292944528 4292943184 2102567 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 3016 455 340 325 0 142 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 12064

[startup+0.100303 s]
/proc/loadavg: 1.93 1.99 1.99 3/173 9850
/proc/meminfo: memFree=31240000/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=16016 CPUtime=0.09 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 1815 0 0 0 9 0 0 0 20 0 1 0 341614857 16400384 1751 33554432000 134512640 135840783 4292944528 4292943232 2102567 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 4004 1752 649 325 0 1130 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 16016

[startup+0.30029 s]
/proc/loadavg: 1.93 1.99 1.99 3/173 9850
/proc/meminfo: memFree=31240000/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=24068 CPUtime=0.28 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 4489 0 0 0 28 0 0 0 20 0 1 0 341614857 24645632 4425 33554432000 134512640 135840783 4292944528 4292942704 1960422 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 6017 4426 1314 325 0 3143 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 24068

[startup+0.700323 s]
/proc/loadavg: 1.93 1.99 1.99 3/173 9850
/proc/meminfo: memFree=31240000/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=36184 CPUtime=0.68 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 10149 0 0 0 67 1 0 0 20 0 1 0 341614857 37052416 8275 33554432000 134512640 135840783 4292944528 4292848796 16143408 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 9046 8275 342 325 0 7982 0
Current children cumulated CPU time (s) 0.68
Current children cumulated vsize (KiB) 36184

[startup+1.50027 s]
/proc/loadavg: 1.93 1.99 1.99 3/177 9861
/proc/meminfo: memFree=31155728/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=44652 CPUtime=1.49 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 12290 0 0 0 146 3 0 0 20 0 1 0 341614857 45723648 10416 33554432000 134512640 135840783 4292944528 4292848620 16143408 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 11163 10416 344 325 0 10099 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 44652

[startup+3.10032 s]
/proc/loadavg: 1.93 1.99 1.99 3/177 9861
/proc/meminfo: memFree=31111924/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=44652 CPUtime=3.08 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 12294 0 0 0 304 4 0 0 20 0 1 0 341614857 45723648 10420 33554432000 134512640 135840783 4292944528 4292848620 16143408 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 11163 10420 344 325 0 10099 0
Current children cumulated CPU time (s) 3.08
Current children cumulated vsize (KiB) 44652

[startup+6.30032 s]
/proc/loadavg: 1.93 1.99 1.99 3/176 9861
/proc/meminfo: memFree=30964620/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=43720 CPUtime=6.29 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 14185 0 0 0 622 7 0 0 20 0 1 0 341614857 44769280 10196 33554432000 134512640 135840783 4292944528 4292943660 1727231 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 10930 10196 347 325 0 9866 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 43720

[startup+12.7003 s]
/proc/loadavg: 1.94 1.99 1.99 3/177 9862
/proc/meminfo: memFree=30894128/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=44252 CPUtime=12.69 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 14307 0 0 0 1262 7 0 0 20 0 1 0 341614857 45314048 10318 33554432000 134512640 135840783 4292944528 4292943888 134551065 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 11063 10318 347 325 0 9999 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 44252

[startup+25.5003 s]
/proc/loadavg: 1.95 1.99 1.99 3/177 9862
/proc/meminfo: memFree=30872312/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=66660 CPUtime=25.47 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 19929 0 0 0 2539 8 0 0 20 0 1 0 341614857 68259840 15936 33554432000 134512640 135840783 4292944528 4292943888 134550764 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 16665 15936 347 325 0 15601 0
Current children cumulated CPU time (s) 25.47
Current children cumulated vsize (KiB) 66660

[startup+51.1003 s]
/proc/loadavg: 1.97 1.99 1.99 3/177 9862
/proc/meminfo: memFree=30816836/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=121820 CPUtime=51.05 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 33735 0 0 0 5094 11 0 0 20 0 1 0 341614857 124743680 29726 33554432000 134512640 135840783 4292944528 4292943888 134551339 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 30455 29726 347 325 0 29391 0
Current children cumulated CPU time (s) 51.05
Current children cumulated vsize (KiB) 121820

[startup+102.306 s]
/proc/loadavg: 1.98 1.99 1.99 3/177 9862
/proc/meminfo: memFree=30799148/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=138148 CPUtime=102.22 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 37832 0 0 0 10209 13 0 0 20 0 1 0 341614857 141463552 33793 33554432000 134512640 135840783 4292944528 4292943888 134551084 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 34537 33793 347 325 0 33473 0
Current children cumulated CPU time (s) 102.22
Current children cumulated vsize (KiB) 138148

[startup+162.3 s]
/proc/loadavg: 1.99 1.99 1.99 3/177 9862
/proc/meminfo: memFree=30787892/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=149692 CPUtime=162.2 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 40721 0 0 0 16205 15 0 0 20 0 1 0 341614857 153284608 36682 33554432000 134512640 135840783 4292944528 4292943888 134550939 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 37423 36682 347 325 0 36359 0
Current children cumulated CPU time (s) 162.2
Current children cumulated vsize (KiB) 149692

[startup+222.3 s]
/proc/loadavg: 1.99 1.99 1.99 3/177 9862
/proc/meminfo: memFree=30779488/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=160572 CPUtime=222.19 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 43443 0 0 0 22203 16 0 0 20 0 1 0 341614857 164425728 39404 33554432000 134512640 135840783 4292944528 4292943808 2104930 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 40143 39404 347 325 0 39079 0
Current children cumulated CPU time (s) 222.19
Current children cumulated vsize (KiB) 160572

[startup+282.3 s]
/proc/loadavg: 1.99 1.99 1.99 3/177 9862
/proc/meminfo: memFree=30746396/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=169728 CPUtime=282.18 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 45750 0 0 0 28201 17 0 0 20 0 1 0 341614857 173801472 41711 33554432000 134512640 135840783 4292944528 4292943888 134550808 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 42432 41711 347 325 0 41368 0
Current children cumulated CPU time (s) 282.18
Current children cumulated vsize (KiB) 169728

[startup+342.3 s]
/proc/loadavg: 1.99 1.99 1.99 3/177 9863
/proc/meminfo: memFree=30679804/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=194396 CPUtime=342.17 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 51917 0 0 0 34197 20 0 0 20 0 1 0 341614857 199061504 47876 33554432000 134512640 135840783 4292944528 4292943888 134550872 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 48599 47876 347 325 0 47535 0
Current children cumulated CPU time (s) 342.17
Current children cumulated vsize (KiB) 194396

[startup+402.3 s]
/proc/loadavg: 1.99 1.99 1.99 3/177 9863
/proc/meminfo: memFree=30637400/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=202620 CPUtime=402.13 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 53993 0 0 0 40192 21 0 0 20 0 1 0 341614857 207482880 49923 33554432000 134512640 135840783 4292944528 4292943888 134550797 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 50655 49923 347 325 0 49591 0
Current children cumulated CPU time (s) 402.13
Current children cumulated vsize (KiB) 202620

[startup+462.3 s]
/proc/loadavg: 2.05 2.00 2.00 3/177 9863
/proc/meminfo: memFree=30591272/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=219168 CPUtime=462.12 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 58132 0 0 0 46189 23 0 0 20 0 1 0 341614857 224428032 54062 33554432000 134512640 135840783 4292944528 4292943888 134550804 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 54792 54062 347 325 0 53728 0
Current children cumulated CPU time (s) 462.12
Current children cumulated vsize (KiB) 219168

[startup+522.3 s]
/proc/loadavg: 2.02 2.00 2.00 3/177 9863
/proc/meminfo: memFree=30543160/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=243120 CPUtime=522.11 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 64131 0 0 0 52186 25 0 0 20 0 1 0 341614857 248954880 60061 33554432000 134512640 135840783 4292944528 4292943888 134551324 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 60780 60061 347 325 0 59716 0
Current children cumulated CPU time (s) 522.11
Current children cumulated vsize (KiB) 243120

[startup+582.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 9866
/proc/meminfo: memFree=30507244/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=256240 CPUtime=582.1 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 67400 0 0 0 58183 27 0 0 20 0 1 0 341614857 262389760 63330 33554432000 134512640 135840783 4292944528 4292943888 134550779 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 64060 63330 347 325 0 62996 0
Current children cumulated CPU time (s) 582.1
Current children cumulated vsize (KiB) 256240

[startup+642.3 s]

################
# More data... #
################

[startup+822.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 9867
/proc/meminfo: memFree=30368508/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=322116 CPUtime=822.04 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 83885 0 0 0 82171 33 0 0 20 0 1 0 341614857 329846784 79791 33554432000 134512640 135840783 4292944528 4292943888 134550779 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 80529 79791 347 325 0 79465 0
Current children cumulated CPU time (s) 822.04
Current children cumulated vsize (KiB) 322116

[startup+882.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 9867
/proc/meminfo: memFree=30346916/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=328092 CPUtime=881.99 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 85415 0 0 0 88165 34 0 0 20 0 1 0 341614857 335966208 81283 33554432000 134512640 135840783 4292944528 4292943888 134550779 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 82023 81283 347 325 0 80959 0
Current children cumulated CPU time (s) 881.99
Current children cumulated vsize (KiB) 328092

[startup+942.3 s]
/proc/loadavg: 2.10 2.03 2.01 3/177 9868
/proc/meminfo: memFree=30319760/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=339348 CPUtime=941.99 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 88223 0 0 0 94163 36 0 0 20 0 1 0 341614857 347492352 84091 33554432000 134512640 135840783 4292944528 4292943888 134550779 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 84837 84091 347 325 0 83773 0
Current children cumulated CPU time (s) 941.99
Current children cumulated vsize (KiB) 339348

[startup+1002.3 s]
/proc/loadavg: 2.04 2.02 2.00 3/177 9868
/proc/meminfo: memFree=30288636/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=355492 CPUtime=1001.98 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 92278 0 0 0 100159 39 0 0 20 0 1 0 341614857 364023808 88146 33554432000 134512640 135840783 4292944528 4292943888 134551084 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 88873 88146 347 325 0 87809 0
Current children cumulated CPU time (s) 1001.98
Current children cumulated vsize (KiB) 355492

[startup+1062.3 s]
/proc/loadavg: 2.01 2.02 2.00 3/177 9868
/proc/meminfo: memFree=30262720/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=369644 CPUtime=1061.97 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 95825 0 0 0 106157 40 0 0 20 0 1 0 341614857 378515456 91693 33554432000 134512640 135840783 4292944528 4292943880 5318828 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 92411 91693 347 325 0 91347 0
Current children cumulated CPU time (s) 1061.97
Current children cumulated vsize (KiB) 369644

[startup+1122.3 s]
/proc/loadavg: 2.00 2.01 2.00 3/177 9868
/proc/meminfo: memFree=30230976/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=384992 CPUtime=1121.96 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 99632 0 0 0 112154 42 0 0 20 0 1 0 341614857 394231808 95500 33554432000 134512640 135840783 4292944528 4292943888 134550779 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 96248 95500 347 325 0 95184 0
Current children cumulated CPU time (s) 1121.96
Current children cumulated vsize (KiB) 384992

[startup+1182.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 9871
/proc/meminfo: memFree=30189256/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=412276 CPUtime=1181.95 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 106470 0 0 0 118151 44 0 0 20 0 1 0 341614857 422170624 102338 33554432000 134512640 135840783 4292944528 4292943852 2107920 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 103069 102338 347 325 0 102005 0
Current children cumulated CPU time (s) 1181.95
Current children cumulated vsize (KiB) 412276

[startup+1242.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 9872
/proc/meminfo: memFree=30139320/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=445152 CPUtime=1241.94 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 114679 0 0 0 124147 47 0 0 20 0 1 0 341614857 455835648 110547 33554432000 134512640 135840783 4292944528 4292943888 134550804 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 111288 110547 347 325 0 110224 0
Current children cumulated CPU time (s) 1241.94
Current children cumulated vsize (KiB) 445152

[startup+1302.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 9872
/proc/meminfo: memFree=30101744/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=467964 CPUtime=1301.94 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 120390 0 0 0 130144 50 0 0 20 0 1 0 341614857 479195136 116258 33554432000 134512640 135840783 4292944528 4292943824 134571866 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 116991 116258 347 325 0 115927 0
Current children cumulated CPU time (s) 1301.94
Current children cumulated vsize (KiB) 467964

[startup+1362.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 9872
/proc/meminfo: memFree=30068636/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=489436 CPUtime=1361.93 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 125753 0 0 0 136141 52 0 0 20 0 1 0 341614857 501182464 121621 33554432000 134512640 135840783 4292944528 4292943888 134550902 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 122359 121621 347 325 0 121295 0
Current children cumulated CPU time (s) 1361.93
Current children cumulated vsize (KiB) 489436

[startup+1422.31 s]
/proc/loadavg: 2.04 2.01 2.00 3/177 9872
/proc/meminfo: memFree=30039744/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=503064 CPUtime=1421.92 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 129181 0 0 0 142138 54 0 0 20 0 1 0 341614857 515137536 125049 33554432000 134512640 135840783 4292944528 4292943888 134551186 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 125766 125049 347 325 0 124702 0
Current children cumulated CPU time (s) 1421.92
Current children cumulated vsize (KiB) 503064

[startup+1482.3 s]
/proc/loadavg: 2.01 2.01 2.00 3/177 9872
/proc/meminfo: memFree=30006716/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=522516 CPUtime=1481.88 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 134064 0 0 0 148132 56 0 0 20 0 1 0 341614857 535056384 129910 33554432000 134512640 135840783 4292944528 4292943888 134550779 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 130629 129910 347 325 0 129565 0
Current children cumulated CPU time (s) 1481.88
Current children cumulated vsize (KiB) 522516

[startup+1542.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 9873
/proc/meminfo: memFree=29966816/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=542152 CPUtime=1541.83 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 139020 0 0 0 154124 59 0 0 20 0 1 0 341614857 555163648 134819 33554432000 134512640 135840783 4292944528 4292943888 134551646 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 135538 134819 347 325 0 134474 0
Current children cumulated CPU time (s) 1541.83
Current children cumulated vsize (KiB) 542152

[startup+1602.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 9873
/proc/meminfo: memFree=29943680/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=545740 CPUtime=1601.78 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 139928 0 0 0 160119 59 0 0 20 0 1 0 341614857 558837760 135692 33554432000 134512640 135840783 4292944528 4292943880 5318864 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 136435 135692 347 325 0 135371 0
Current children cumulated CPU time (s) 1601.78
Current children cumulated vsize (KiB) 545740

[startup+1662.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 9873
/proc/meminfo: memFree=29912556/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=556728 CPUtime=1661.77 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 142691 0 0 0 166116 61 0 0 20 0 1 0 341614857 570089472 138455 33554432000 134512640 135840783 4292944528 4292943888 134550779 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 139182 138455 347 325 0 138118 0
Current children cumulated CPU time (s) 1661.77
Current children cumulated vsize (KiB) 556728

[startup+1722.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 9873
/proc/meminfo: memFree=29878580/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=575108 CPUtime=1721.76 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 147272 0 0 0 172113 63 0 0 20 0 1 0 341614857 588910592 143036 33554432000 134512640 135840783 4292944528 4292943888 134550764 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 143777 143036 347 325 0 142713 0
Current children cumulated CPU time (s) 1721.76
Current children cumulated vsize (KiB) 575108

[startup+1782.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 9876
/proc/meminfo: memFree=29845960/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=592172 CPUtime=1781.76 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 151551 0 0 0 178110 66 0 0 20 0 1 0 341614857 606384128 147315 33554432000 134512640 135840783 4292944528 4292943888 134550779 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 148043 147315 347 325 0 146979 0
Current children cumulated CPU time (s) 1781.76
Current children cumulated vsize (KiB) 592172



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1800.6 s]
/proc/loadavg: 2.00 2.00 2.00 4/177 9876
/proc/meminfo: memFree=29836668/32873844 swapFree=8656/8656
[pid=9850] ppid=9848 vsize=596936 CPUtime=1800.07 cores=0,2,4,6
/proc/9850/stat : 9850 (bsolo3.0.17) R 9848 9850 9396 0 -1 4202496 152736 0 0 0 179940 67 0 0 20 0 1 0 341614857 611262464 148500 33554432000 134512640 135840783 4292944528 4292943888 134550764 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/9850/statm: 149234 148500 347 325 0 148170 0
Current children cumulated CPU time (s) 1800.07
Current children cumulated vsize (KiB) 596936

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child status: 15
Real time (s): 1800.62
CPU time (s): 1800.08
CPU user time (s): 1799.4
CPU system time (s): 0.678896
CPU usage (%): 99.9701
Max. virtual memory (cumulated for all children) (KiB): 596936

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1799.4
system time used= 0.678896
maximum resident set size= 594000
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 152739
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 273
involuntary context switches= 1979

runsolver used 4.94925 second user time and 7.38488 second system time

The end

Launcher Data 

Begin job on node102 at 2012-06-03 22:21:03
IDJOB=3741037
IDBENCH=89738
IDSOLVER=2318
FILE ID=node102/3741037-1338754863
RUNJOBID= node102-1338747648-9412
PBS_JOBID= 14636781
Free space on /tmp= 71152 MiB

SOLVER NAME= PB07: bsolo 3.0.17
BENCH NAME= PB11/normalized-PB11/DEC-SMALLINT-LIN/lopes/normalized-99.opb
COMMAND LINE= bsolo3.0.17 -mMEMLIMIT BENCHNAME            
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3741037-1338754863/watcher-3741037-1338754863 -o /tmp/evaluation-result-3741037-1338754863/solver-3741037-1338754863 -C 1800 -W 1900 -M 15500  bsolo3.0.17 -m15500 HOME/instance-3741037-1338754863.opb            

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= a0ef64dc4a665e7cf00a752d7a124006
RANDOM SEED=1927749164

node102.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.178
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.35
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.178
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.178
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.178
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.178
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.178
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5438.54
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.178
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.178
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        31241024 kB
Buffers:          196356 kB
Cached:           661900 kB
SwapCached:         5520 kB
Active:           225684 kB
Inactive:         644404 kB
Active(anon):       8440 kB
Inactive(anon):     5604 kB
Active(file):     217244 kB
Inactive(file):   638800 kB
Unevictable:        8656 kB
Mlocked:            8656 kB
SwapTotal:      67108856 kB
SwapFree:       67097276 kB
Dirty:             31468 kB
Writeback:             0 kB
AnonPages:         13008 kB
Mapped:             8584 kB
Shmem:                 0 kB
Slab:             620684 kB
SReclaimable:      61752 kB
SUnreclaim:       558932 kB
KernelStack:        1456 kB
PageTables:         4192 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     108684 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 71108 MiB
End job on node102 at 2012-06-03 22:51:06