Trace number 3729003

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
PB07: bsolo 3.0.17? (TO) 1800.04 1800.41

General information on the benchmark

Namenormalized-PB06/SATUNSAT-SMALLINT/submitted-PB06/
prestwich/armies/normalized-army13.24bt.opb
MD5SUMf8964c43535341fcc69e672675167b77
Bench CategoryDEC-SMALLINT (no optimisation, small integers)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark0
Best CPU time to get the best result obtained on this benchmark23.1595
Has Objective FunctionNO
SatisfiableYES
(Un)Satisfiability was provedYES
Best value of the objective function
Optimality of the best value was proved NO
Number of variables490
Total number of constraints1085
Number of constraints which are clauses738
Number of constraints which are cardinality constraints (but not clauses)2
Number of constraints which are nor clauses,nor cardinality constraints345
Minimum length of a constraint1
Maximum length of a constraint169
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 32
Number of bits of the biggest number in a constraint 6
Biggest sum of numbers in a constraint 193
Number of bits of the biggest sum of numbers8
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.04	c bsolo beta version 3.0.17 - 27/04/2007 : 1600 GMT
0.00/0.04	c Developed by Vasco Manquinho IST/UTL - INESC-ID
0.00/0.04	c type "bsolo -h" for help
0.00/0.04	c Time Limit set via environment variable PBTIMEOUT to 1800
0.00/0.04	c Memory Limit set to 15500 MB
0.00/0.04	c Instance file HOME/instance-3729003-1338678621.opb
0.00/0.04	c File size is 55178 bytes.
0.00/0.04	c Highest Coefficient sum: 193
0.00/0.04	c Parsing was ok!!
0.00/0.04	c Total parsing time: 0.006 s
0.00/0.04	c Var: 490 Constr: 1085 738/2/345 Lit: 6010 Watch. Lit: 3271
0.00/0.04	c Obj. Vars: 0 (0 % of total variables)
0.01/0.06	c Pre-processing Time: 0.018 s
0.01/0.06	c    Confl   Vars     Ctrs     Lits  LPC   W.Lits     Max Learnt Conf/s Time
0.01/0.06	c        0    457     1023     5787    5     3146    5638      0      0 0.02
0.01/0.06	c Switching off LPR mode.
1800.02/1800.40	c      103    457     1126    17057   15     3324    6202    103   3322 0.03
1800.02/1800.40	c      254    457     1277    44551   34     3618    6823    254   5080 0.05
1800.02/1800.40	c      480    457     1503    88020   58     4070    7506    480   6153 0.08
1800.02/1800.40	c      818    457     1841   146020   79     4744    8257    818   6816 0.12
1800.02/1800.40	c     1328    457     2351   238378  101     5766    9083   1328   6775 0.20
1800.02/1800.40	c     2091    457     3114   359356  115     7289    9992   2091   6811 0.31
1800.02/1800.40	c     3238    457     4261   559819  131     9588   10992   3238   6226 0.52
1800.02/1800.40	c     4959    457     5982   861814  144    13023   12092   4959   5086 0.97
1800.02/1800.40	c     7546    457     8569  1289614  150    18202   13302   7546   4107 1.84
1800.02/1800.40	c    11427    457    12450  1953164  156    25967   14633  11427   3047 3.75
1800.02/1800.40	c    17251    457    10646  1658984  155    22359   16097   9623   2484 6.94
1800.02/1800.40	c    25988    457    10967  1720975  156    22997   17707   9944   2202 11.80
1800.02/1800.40	c    39100    457    14940  2348831  157    30947   19478  13917   1833 21.32
1800.02/1800.40	c    58772    457    14513  2342131  161    30093   21426  13490   1742 33.73
1800.02/1800.40	c    88282    457    21917  3576037  163    44901   23569  20894   1593 55.38
1800.02/1800.40	c   132555    457    15827  2534886  160    32715   25926  14804   1474 89.89
1800.02/1800.40	c   198966    457    23082  3791120  164    47228   28519  22059   1461 136.16
1800.02/1800.40	c   298587    457    18734  3064737  163    38532   31371  17711   1422 209.93
1800.02/1800.40	c   448025    457    28011  4783923  170    57089   34509  26988   1361 329.14
1800.02/1800.40	c   672183    457    22622  4007368  177    46309   37960  21599   1276 526.73
1800.02/1800.40	c  1008435    457    19917  3070186  154    40897   41757  18894   1149 876.98
1800.02/1800.40	c  1512806    457    17180  2892789  168    35422   45933  16157   1074 1408.31
1800.02/1800.40	c Caught signal: 15
1800.02/1800.40	c  Rst    Confl    NCB      Dec     Ctrs     Lits Learnt Conf/s Time
1800.02/1800.40	c   22  1838869 145358  2094450    39722  7103596  38699   1021 1800.03
1800.02/1800.40	s UNKNOWN
1800.02/1800.40	c Total time: 1800.03 s

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3729003-1338678621/watcher-3729003-1338678621 -o /tmp/evaluation-result-3729003-1338678621/solver-3729003-1338678621 -C 1800 -W 1900 -M 15500 bsolo3.0.17 -m15500 HOME/instance-3729003-1338678621.opb 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.00 0.19 0.96 2/172 31574
/proc/meminfo: memFree=28730908/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=1532 CPUtime=0 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) D 31572 31574 31541 0 -1 4194304 74 0 0 0 0 0 0 0 20 0 1 0 333987778 1568768 28 33554432000 134512640 135840783 4293479440 4293477532 5489604 0 0 4096 0 18446744072100609316 0 0 17 0 0 0 0 0 0
/proc/31574/statm: 383 28 22 325 0 25 0

[startup+0.0585919 s]
/proc/loadavg: 0.00 0.19 0.96 2/172 31574
/proc/meminfo: memFree=28730908/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=4956 CPUtime=0.01 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 568 0 4 0 1 0 0 0 20 0 1 0 333987778 5074944 494 33554432000 134512640 135840783 4293479440 4293472544 134557472 0 0 4096 16384 0 0 0 17 0 0 0 2 0 0
/proc/31574/statm: 1239 494 344 325 0 175 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 4956

[startup+0.100261 s]
/proc/loadavg: 0.00 0.19 0.96 2/172 31574
/proc/meminfo: memFree=28730908/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=5596 CPUtime=0.05 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 716 0 4 0 5 0 0 0 20 0 1 0 333987778 5730304 642 33554432000 134512640 135840783 4293479440 4293478736 134574219 0 0 4096 16384 0 0 0 17 0 0 0 2 0 0
/proc/31574/statm: 1399 642 347 325 0 335 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 5596

[startup+0.300176 s]
/proc/loadavg: 0.00 0.19 0.96 2/172 31574
/proc/meminfo: memFree=28730908/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=8948 CPUtime=0.25 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 1555 0 4 0 25 0 0 0 20 0 1 0 333987778 9162752 1481 33554432000 134512640 135840783 4293479440 4293478672 5626626 0 0 4096 16384 0 0 0 17 0 0 0 2 0 0
/proc/31574/statm: 2237 1481 347 325 0 1173 0
Current children cumulated CPU time (s) 0.25
Current children cumulated vsize (KiB) 8948

[startup+0.70018 s]
/proc/loadavg: 0.00 0.19 0.96 2/172 31574
/proc/meminfo: memFree=28730908/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=13580 CPUtime=0.65 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 2743 0 4 0 65 0 0 0 20 0 1 0 333987778 13905920 2669 33554432000 134512640 135840783 4293479440 4293478800 134551326 0 0 4096 16384 0 0 0 17 0 0 0 2 0 0
/proc/31574/statm: 3395 2669 347 325 0 2331 0
Current children cumulated CPU time (s) 0.65
Current children cumulated vsize (KiB) 13580

[startup+1.50018 s]
/proc/loadavg: 0.16 0.22 0.96 3/178 31589
/proc/meminfo: memFree=28702160/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=19768 CPUtime=1.44 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 4285 0 4 0 144 0 0 0 20 0 1 0 333987778 20242432 4211 33554432000 134512640 135840783 4293479440 4293478800 134550913 0 0 4096 16384 0 0 0 17 0 0 0 2 0 0
/proc/31574/statm: 4942 4211 347 325 0 3878 0
Current children cumulated CPU time (s) 1.44
Current children cumulated vsize (KiB) 19768

[startup+3.10026 s]
/proc/loadavg: 0.16 0.22 0.96 3/178 31589
/proc/meminfo: memFree=28689264/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=28136 CPUtime=3.05 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 6369 0 4 0 304 1 0 0 20 0 1 0 333987778 28811264 6295 33554432000 134512640 135840783 4293479440 4293478800 134551093 0 0 4096 16384 0 0 0 17 0 0 0 2 0 0
/proc/31574/statm: 7034 6295 347 325 0 5970 0
Current children cumulated CPU time (s) 3.05
Current children cumulated vsize (KiB) 28136

[startup+6.30025 s]
/proc/loadavg: 0.39 0.26 0.97 3/178 31589
/proc/meminfo: memFree=28672276/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=39052 CPUtime=6.25 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 9094 0 4 0 623 2 0 0 20 0 1 0 333987778 39989248 9020 33554432000 134512640 135840783 4293479440 4293478800 134550893 0 0 4096 16384 0 0 0 17 0 0 0 2 0 0
/proc/31574/statm: 9763 9020 347 325 0 8699 0
Current children cumulated CPU time (s) 6.25
Current children cumulated vsize (KiB) 39052

[startup+12.7003 s]
/proc/loadavg: 0.52 0.29 0.98 3/178 31589
/proc/meminfo: memFree=28661316/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=42784 CPUtime=12.65 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 10034 0 4 0 1263 2 0 0 20 0 1 0 333987778 43810816 9959 33554432000 134512640 135840783 4293479440 4293478800 134551071 0 0 4096 16384 0 0 0 17 0 0 0 2 0 0
/proc/31574/statm: 10696 9959 347 325 0 9632 0
Current children cumulated CPU time (s) 12.65
Current children cumulated vsize (KiB) 42784

[startup+25.5003 s]
/proc/loadavg: 0.84 0.38 0.99 3/178 31589
/proc/meminfo: memFree=28652404/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=50116 CPUtime=25.44 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 11874 0 4 0 2541 3 0 0 20 0 1 0 333987778 51318784 11791 33554432000 134512640 135840783 4293479440 4293478800 134551093 0 0 4096 16384 0 0 0 17 2 0 0 2 0 0
/proc/31574/statm: 12529 11791 347 325 0 11465 0
Current children cumulated CPU time (s) 25.44
Current children cumulated vsize (KiB) 50116

[startup+51.1002 s]
/proc/loadavg: 1.24 0.50 1.02 3/177 31589
/proc/meminfo: memFree=28641088/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=56056 CPUtime=51.02 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 13376 0 4 0 5099 3 0 0 20 0 1 0 333987778 57401344 13278 33554432000 134512640 135840783 4293479440 4293478752 134550804 0 0 4096 16384 0 0 0 17 2 0 0 2 0 0
/proc/31574/statm: 14014 13278 347 325 0 12950 0
Current children cumulated CPU time (s) 51.02
Current children cumulated vsize (KiB) 56056

[startup+102.306 s]
/proc/loadavg: 1.67 0.73 1.07 3/177 31589
/proc/meminfo: memFree=28627556/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=64780 CPUtime=102.22 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 15731 0 4 0 10218 4 0 0 20 0 1 0 333987778 66334720 15466 33554432000 134512640 135840783 4293479440 4293478800 134550921 0 0 4096 16384 0 0 0 17 2 0 0 2 0 0
/proc/31574/statm: 16195 15466 347 325 0 15131 0
Current children cumulated CPU time (s) 102.22
Current children cumulated vsize (KiB) 64780

[startup+162.3 s]
/proc/loadavg: 1.88 0.96 1.12 3/177 31589
/proc/meminfo: memFree=28617248/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=70564 CPUtime=162.19 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 17181 0 4 0 16215 4 0 0 20 0 1 0 333987778 72257536 16912 33554432000 134512640 135840783 4293479440 4293478800 134550804 0 0 4096 16384 0 0 0 17 2 0 0 2 0 0
/proc/31574/statm: 17641 16912 347 325 0 16577 0
Current children cumulated CPU time (s) 162.19
Current children cumulated vsize (KiB) 70564

[startup+222.3 s]
/proc/loadavg: 2.01 1.16 1.18 3/177 31589
/proc/meminfo: memFree=28604032/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=79308 CPUtime=222.18 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 19373 0 4 0 22213 5 0 0 20 0 1 0 333987778 81211392 19099 33554432000 134512640 135840783 4293479440 4293478752 10461718 0 0 4096 16384 0 0 0 17 2 0 0 2 0 0
/proc/31574/statm: 19827 19099 347 325 0 18763 0
Current children cumulated CPU time (s) 222.18
Current children cumulated vsize (KiB) 79308

[startup+282.3 s]
/proc/loadavg: 2.00 1.31 1.23 3/177 31589
/proc/meminfo: memFree=28601356/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=79272 CPUtime=282.17 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 19373 0 4 0 28212 5 0 0 20 0 1 0 333987778 81174528 19091 33554432000 134512640 135840783 4293479440 4293478676 5996674 0 0 4096 16384 0 0 0 17 2 0 0 2 0 0
/proc/31574/statm: 19818 19091 347 325 0 18754 0
Current children cumulated CPU time (s) 282.17
Current children cumulated vsize (KiB) 79272

[startup+342.3 s]
/proc/loadavg: 2.00 1.43 1.28 3/177 31590
/proc/meminfo: memFree=28593228/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=87300 CPUtime=342.16 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 21419 0 4 0 34210 6 0 0 20 0 1 0 333987778 89395200 21094 33554432000 134512640 135840783 4293479440 4293478700 5999443 0 0 4096 16384 0 0 0 17 2 0 0 2 0 0
/proc/31574/statm: 21825 21094 347 325 0 20761 0
Current children cumulated CPU time (s) 342.16
Current children cumulated vsize (KiB) 87300

[startup+402.3 s]
/proc/loadavg: 2.00 1.54 1.32 3/177 31590
/proc/meminfo: memFree=28593228/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=87300 CPUtime=402.15 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 21420 0 4 0 40209 6 0 0 20 0 1 0 333987778 89395200 21095 33554432000 134512640 135840783 4293479440 4293478768 10452972 0 0 4096 16384 0 0 0 17 2 0 0 2 0 0
/proc/31574/statm: 21825 21095 347 325 0 20761 0
Current children cumulated CPU time (s) 402.15
Current children cumulated vsize (KiB) 87300

[startup+462.3 s]
/proc/loadavg: 2.00 1.62 1.36 3/177 31590
/proc/meminfo: memFree=28593036/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=87300 CPUtime=462.14 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 21423 0 4 0 46208 6 0 0 20 0 1 0 333987778 89395200 21095 33554432000 134512640 135840783 4293479440 4293478792 10461704 0 0 4096 16384 0 0 0 17 2 0 0 2 0 0
/proc/31574/statm: 21825 21095 347 325 0 20761 0
Current children cumulated CPU time (s) 462.14
Current children cumulated vsize (KiB) 87300

[startup+522.3 s]
/proc/loadavg: 2.00 1.68 1.39 3/177 31590
/proc/meminfo: memFree=28586272/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=92236 CPUtime=522.13 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 22667 0 4 0 52206 7 0 0 20 0 1 0 333987778 94449664 22301 33554432000 134512640 135840783 4293479440 4293478800 134551318 0 0 4096 16384 0 0 0 17 2 0 0 2 0 0
/proc/31574/statm: 23059 22301 347 325 0 21995 0
Current children cumulated CPU time (s) 522.13
Current children cumulated vsize (KiB) 92236

[startup+582.3 s]
/proc/loadavg: 2.00 1.74 1.43 3/178 31593
/proc/meminfo: memFree=28581364/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=96868 CPUtime=582.12 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 23876 0 4 0 58205 7 0 0 20 0 1 0 333987778 99192832 23472 33554432000 134512640 135840783 4293479440 4293478800 134550779 0 0 4096 16384 0 0 0 17 2 0 0 2 0 0
/proc/31574/statm: 24217 23472 347 325 0 23153 0
Current children cumulated CPU time (s) 582.12
Current children cumulated vsize (KiB) 96868

[startup+642.3 s]

################
# More data... #
################

[startup+822.3 s]
/proc/loadavg: 2.01 1.89 1.55 3/177 31594
/proc/meminfo: memFree=28570204/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=99984 CPUtime=822.07 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 24645 0 4 0 82200 7 0 0 20 0 1 0 333987778 102383616 24238 33554432000 134512640 135840783 4293479440 4293478800 134550804 0 0 4096 16384 0 0 0 17 2 0 0 2 0 0
/proc/31574/statm: 24996 24238 347 325 0 23932 0
Current children cumulated CPU time (s) 822.07
Current children cumulated vsize (KiB) 99984

[startup+882.3 s]
/proc/loadavg: 2.00 1.90 1.58 3/177 31594
/proc/meminfo: memFree=28570136/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=99552 CPUtime=882.07 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 24645 0 4 0 88199 8 0 0 20 0 1 0 333987778 101941248 24162 33554432000 134512640 135840783 4293479440 4293478800 134550910 0 0 4096 16384 0 0 0 17 2 0 0 2 0 0
/proc/31574/statm: 24888 24162 347 325 0 23824 0
Current children cumulated CPU time (s) 882.07
Current children cumulated vsize (KiB) 99552

[startup+942.3 s]
/proc/loadavg: 2.00 1.92 1.60 3/177 31595
/proc/meminfo: memFree=28567780/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=102500 CPUtime=942.06 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 25392 0 4 0 94198 8 0 0 20 0 1 0 333987778 104960000 24867 33554432000 134512640 135840783 4293479440 4293478800 134550804 0 0 4096 16384 0 0 0 17 2 0 0 2 0 0
/proc/31574/statm: 25625 24867 347 325 0 24561 0
Current children cumulated CPU time (s) 942.06
Current children cumulated vsize (KiB) 102500

[startup+1002.3 s]
/proc/loadavg: 2.00 1.93 1.62 3/178 31596
/proc/meminfo: memFree=28560696/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=109200 CPUtime=1002.04 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 27071 0 4 0 100196 8 0 0 20 0 1 0 333987778 111820800 26542 33554432000 134512640 135840783 4293479440 4293478752 134550804 0 0 4096 16384 0 0 0 17 2 0 0 2 0 0
/proc/31574/statm: 27300 26542 347 325 0 26236 0
Current children cumulated CPU time (s) 1002.04
Current children cumulated vsize (KiB) 109200

[startup+1062.3 s]
/proc/loadavg: 2.00 1.94 1.65 3/177 31596
/proc/meminfo: memFree=28559092/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=110852 CPUtime=1062.04 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 27484 0 4 0 106195 9 0 0 20 0 1 0 333987778 113512448 26955 33554432000 134512640 135840783 4293479440 4293478800 134550764 0 0 4096 16384 0 0 0 17 2 0 0 2 0 0
/proc/31574/statm: 27713 26955 347 325 0 26649 0
Current children cumulated CPU time (s) 1062.04
Current children cumulated vsize (KiB) 110852

[startup+1122.3 s]
/proc/loadavg: 2.00 1.95 1.66 3/177 31596
/proc/meminfo: memFree=28556980/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=112912 CPUtime=1122.03 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 28026 0 4 0 112194 9 0 0 20 0 1 0 333987778 115621888 27470 33554432000 134512640 135840783 4293479440 4293478672 6004347 0 0 4096 16384 0 0 0 17 2 0 0 2 0 0
/proc/31574/statm: 28228 27470 347 325 0 27164 0
Current children cumulated CPU time (s) 1122.03
Current children cumulated vsize (KiB) 112912

[startup+1182.3 s]
/proc/loadavg: 2.03 1.97 1.69 3/178 31599
/proc/meminfo: memFree=28555112/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=112684 CPUtime=1182.02 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 28026 0 4 0 118193 9 0 0 20 0 1 0 333987778 115388416 27445 33554432000 134512640 135840783 4293479440 4293478572 5626623 0 0 4096 16384 0 0 0 17 2 0 0 2 0 0
/proc/31574/statm: 28171 27445 347 325 0 27107 0
Current children cumulated CPU time (s) 1182.02
Current children cumulated vsize (KiB) 112684

[startup+1242.3 s]
/proc/loadavg: 2.01 1.97 1.70 3/177 31600
/proc/meminfo: memFree=28555240/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=112684 CPUtime=1242.01 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 28027 0 4 0 124192 9 0 0 20 0 1 0 333987778 115388416 27445 33554432000 134512640 135840783 4293479440 4293478800 134550804 0 0 4096 16384 0 0 0 17 2 0 0 2 0 0
/proc/31574/statm: 28171 27445 347 325 0 27107 0
Current children cumulated CPU time (s) 1242.01
Current children cumulated vsize (KiB) 112684

[startup+1302.3 s]
/proc/loadavg: 2.00 1.97 1.72 3/177 31600
/proc/meminfo: memFree=28554372/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=112684 CPUtime=1302 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 28027 0 4 0 130191 9 0 0 20 0 1 0 333987778 115388416 27445 33554432000 134512640 135840783 4293479440 4293478800 134551058 0 0 4096 16384 0 0 0 17 2 0 0 2 0 0
/proc/31574/statm: 28171 27445 347 325 0 27107 0
Current children cumulated CPU time (s) 1302
Current children cumulated vsize (KiB) 112684

[startup+1362.3 s]
/proc/loadavg: 2.00 1.97 1.74 3/177 31600
/proc/meminfo: memFree=28553504/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=112684 CPUtime=1361.99 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 28027 0 4 0 136190 9 0 0 20 0 1 0 333987778 115388416 27445 33554432000 134512640 135840783 4293479440 4293478800 134551063 0 0 4096 16384 0 0 0 17 2 0 0 2 0 0
/proc/31574/statm: 28171 27445 347 325 0 27107 0
Current children cumulated CPU time (s) 1361.99
Current children cumulated vsize (KiB) 112684

[startup+1422.31 s]
/proc/loadavg: 2.00 1.97 1.75 3/176 31600
/proc/meminfo: memFree=28551216/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=115832 CPUtime=1421.98 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 28782 0 4 0 142189 9 0 0 20 0 1 0 333987778 118611968 28200 33554432000 134512640 135840783 4293479440 4293478800 134551065 0 0 4096 16384 0 0 0 17 2 0 0 2 0 0
/proc/31574/statm: 28958 28200 347 325 0 27894 0
Current children cumulated CPU time (s) 1421.98
Current children cumulated vsize (KiB) 115832

[startup+1482.3 s]
/proc/loadavg: 2.00 1.97 1.76 3/177 31601
/proc/meminfo: memFree=28546804/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=115848 CPUtime=1481.97 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 28787 0 4 0 148188 9 0 0 20 0 1 0 333987778 118628352 28204 33554432000 134512640 135840783 4293479440 4293478800 134551033 0 0 4096 16384 0 0 0 17 2 0 0 2 0 0
/proc/31574/statm: 28962 28204 347 325 0 27898 0
Current children cumulated CPU time (s) 1481.97
Current children cumulated vsize (KiB) 115848

[startup+1542.3 s]
/proc/loadavg: 2.00 1.97 1.77 3/177 31602
/proc/meminfo: memFree=28543328/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=119448 CPUtime=1541.96 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 29733 0 4 0 154186 10 0 0 20 0 1 0 333987778 122314752 29104 33554432000 134512640 135840783 4293479440 4293478800 134550808 0 0 4096 16384 0 0 0 17 2 0 0 2 0 0
/proc/31574/statm: 29862 29104 347 325 0 28798 0
Current children cumulated CPU time (s) 1541.96
Current children cumulated vsize (KiB) 119448

[startup+1602.3 s]
/proc/loadavg: 2.00 1.97 1.78 3/177 31602
/proc/meminfo: memFree=28543076/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=119448 CPUtime=1601.95 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 29735 0 4 0 160185 10 0 0 20 0 1 0 333987778 122314752 29104 33554432000 134512640 135840783 4293479440 4293478800 134551048 0 0 4096 16384 0 0 0 17 2 0 0 2 0 0
/proc/31574/statm: 29862 29104 347 325 0 28798 0
Current children cumulated CPU time (s) 1601.95
Current children cumulated vsize (KiB) 119448

[startup+1662.3 s]
/proc/loadavg: 2.03 1.99 1.80 3/177 31602
/proc/meminfo: memFree=28543820/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=119448 CPUtime=1661.94 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 29735 0 4 0 166184 10 0 0 20 0 1 0 333987778 122314752 29104 33554432000 134512640 135840783 4293479440 4293478800 134551048 0 0 4096 16384 0 0 0 17 2 0 0 2 0 0
/proc/31574/statm: 29862 29104 347 325 0 28798 0
Current children cumulated CPU time (s) 1661.94
Current children cumulated vsize (KiB) 119448

[startup+1722.3 s]
/proc/loadavg: 2.08 2.00 1.82 3/177 31602
/proc/meminfo: memFree=28542580/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=120720 CPUtime=1721.93 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 30053 0 4 0 172183 10 0 0 20 0 1 0 333987778 123617280 29422 33554432000 134512640 135840783 4293479440 4293478800 134551048 0 0 4096 16384 0 0 0 17 2 0 0 2 0 0
/proc/31574/statm: 30180 29422 347 325 0 29116 0
Current children cumulated CPU time (s) 1721.93
Current children cumulated vsize (KiB) 120720

[startup+1782.3 s]
/proc/loadavg: 2.03 2.00 1.82 3/178 31605
/proc/meminfo: memFree=28541492/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=120720 CPUtime=1781.92 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 30054 0 4 0 178182 10 0 0 20 0 1 0 333987778 123617280 29422 33554432000 134512640 135840783 4293479440 4293478800 134551475 0 0 4096 16384 0 0 0 17 2 0 0 2 0 0
/proc/31574/statm: 30180 29422 347 325 0 29116 0
Current children cumulated CPU time (s) 1781.92
Current children cumulated vsize (KiB) 120720



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1800.4 s]
/proc/loadavg: 2.02 2.00 1.82 3/178 31605
/proc/meminfo: memFree=28541888/32873844 swapFree=7112/7112
[pid=31574] ppid=31572 vsize=120720 CPUtime=1800.02 cores=0,2,4,6
/proc/31574/stat : 31574 (bsolo3.0.17) R 31572 31574 31541 0 -1 4202496 30054 0 4 0 179992 10 0 0 20 0 1 0 333987778 123617280 29422 33554432000 134512640 135840783 4293479440 4293478800 134550779 0 0 4096 16384 0 0 0 17 2 0 0 2 0 0
/proc/31574/statm: 30180 29422 347 325 0 29116 0
Current children cumulated CPU time (s) 1800.02
Current children cumulated vsize (KiB) 120720

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child status: 15
Real time (s): 1800.41
CPU time (s): 1800.04
CPU user time (s): 1799.93
CPU system time (s): 0.107983
CPU usage (%): 99.9793
Max. virtual memory (cumulated for all children) (KiB): 120720

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1799.93
system time used= 0.107983
maximum resident set size= 117688
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 30057
page faults= 4
swaps= 0
block input operations= 2416
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 73
involuntary context switches= 2058

runsolver used 2.84257 second user time and 9.57154 second system time

The end

Launcher Data 

Begin job on node119 at 2012-06-03 01:10:21
IDJOB=3729003
IDBENCH=1160
IDSOLVER=2318
FILE ID=node119/3729003-1338678621
RUNJOBID= node119-1338678620-31560
PBS_JOBID= 14636694
Free space on /tmp= 70900 MiB

SOLVER NAME= bsolo 3.0.17
BENCH NAME= PB06//final/normalized-PB06/SATUNSAT-SMALLINT/submitted-PB06/prestwich/armies/normalized-army13.24bt.opb
COMMAND LINE= bsolo3.0.17 -mMEMLIMIT BENCHNAME            
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3729003-1338678621/watcher-3729003-1338678621 -o /tmp/evaluation-result-3729003-1338678621/solver-3729003-1338678621 -C 1800 -W 1900 -M 15500  bsolo3.0.17 -m15500 HOME/instance-3729003-1338678621.opb            

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= f8964c43535341fcc69e672675167b77
RANDOM SEED=402869727

node119.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.943
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.88
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.943
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.943
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.93
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.943
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.943
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.93
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.943
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.943
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.943
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        28731436 kB
Buffers:          389792 kB
Cached:          2926420 kB
SwapCached:         1936 kB
Active:          1436864 kB
Inactive:        1885824 kB
Active(anon):       2000 kB
Inactive(anon):     6636 kB
Active(file):    1434864 kB
Inactive(file):  1879188 kB
Unevictable:        7112 kB
Mlocked:            7112 kB
SwapTotal:      67108856 kB
SwapFree:       67100224 kB
Dirty:              6636 kB
Writeback:             0 kB
AnonPages:         11964 kB
Mapped:            11148 kB
Shmem:                 0 kB
Slab:             681580 kB
SReclaimable:     123772 kB
SUnreclaim:       557808 kB
KernelStack:        1408 kB
PageTables:         3872 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     104136 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:      2048 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70888 MiB
End job on node119 at 2012-06-03 01:40:23