Trace number 3741740

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
PB09: SCIPspx SCIP 1.1.0.7 with SoPLEX 1.4.1(24.4.2009)? (TO) 1802.6 1802.92

General information on the benchmark

Name/OPT-SMALLINT-LIN/
kullmann/normalized-sbox_8_shg.opb
MD5SUM3cb4e48d7e295175417c2e8c2155aba3
Bench CategoryOPT-SMALLINT-LIN (optimisation, small integers, linear constraints)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark655
Best CPU time to get the best result obtained on this benchmark1800.93
Has Objective FunctionYES
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables136253
Total number of constraints65280
Number of constraints which are clauses65280
Number of constraints which are cardinality constraints (but not clauses)0
Number of constraints which are nor clauses,nor cardinality constraints0
Minimum length of a constraint170
Maximum length of a constraint1517
Number of terms in the objective function 136253
Biggest coefficient in the objective function 1
Number of bits for the biggest coefficient in the objective function 1
Sum of the numbers in the objective function 136253
Number of bits of the sum of numbers in the objective function 18
Biggest number in a constraint 1
Number of bits of the biggest number in a constraint 1
Biggest sum of numbers in a constraint 136253
Number of bits of the biggest sum of numbers18
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

49.26/49.34	c SCIP version 1.1.0.7
49.26/49.34	c LP-Solver SoPlex 1.4.1
49.26/49.34	c user parameter file <scip.set> not found - using default parameters
49.26/49.34	c read problem <HOME/instance-3741740-1338773363.opb>
49.26/49.34	c original problem has 136253 variables (136253 bin, 0 int, 0 impl, 0 cont) and 65280 constraints
65.35/65.49	c start presolving problem
65.35/65.49	c presolving:
93.75/93.87	c (round 1) 0 del vars, 0 del conss, 0 chg bounds, 0 chg sides, 0 chg coeffs, 65280 upgd conss, 0 impls, 0 clqs
97.64/97.77	c    (46.4s) probing: 101/136253 (0.1%) - 0 fixings, 0 aggregations, 0 implications, 0 bound changes
97.64/97.77	c    (46.4s) probing aborted: 100/100 successive totally useless probings
97.64/97.78	c presolving (2 rounds):
97.64/97.78	c  0 deleted vars, 0 deleted constraints, 0 tightened bounds, 0 added holes, 0 changed sides, 0 changed coefficients
97.64/97.78	c  0 implications, 0 cliques
97.64/97.78	c presolved problem has 136253 variables (136253 bin, 0 int, 0 impl, 0 cont) and 65280 constraints
97.64/97.78	c   65280 constraints of type <logicor>
97.64/97.78	c transformed objective value is always integral (scale: 1)
97.64/97.78	c Presolving Time: 32.28
97.75/97.82	c strange?!, PURESAT with obj
97.75/97.82	c -----------------------------------------------------------------------------------------------
97.75/97.82	c # maximal memory usage in MB; reported memory usage is lower than real memory usage!
97.75/97.82	c # [type: real, range: [0,1.79769313486232e+308], default: 1e+20]
97.75/97.82	c limits/memory = c 13950
97.75/97.82	c # maximal time in seconds to run
97.75/97.82	c # [type: real, range: [0,1.79769313486232e+308], default: 1e+20]
97.75/97.82	c limits/time = c 1791
97.75/97.82	c # frequency for displaying node information lines
97.75/97.82	c # [type: int, range: [-1,2147483647], default: 100]
97.75/97.82	c display/freq = c 10000
97.75/97.82	c -----------------------------------------------------------------------------------------------
97.75/97.82	c start solving problem
97.75/97.82	c

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3741740-1338773363/watcher-3741740-1338773363 -o /tmp/evaluation-result-3741740-1338773363/solver-3741740-1338773363 -C 1800 -W 1900 -M 15500 HOME/pbscip.linux.x86_64.gnu.opt-gcc42.spx -t 1800 -m 15500 -f HOME/instance-3741740-1338773363.opb 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.00 2.00 2.00 3/176 1732
/proc/meminfo: memFree=27779680/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=6124 CPUtime=0 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 921 0 0 0 0 0 0 0 20 0 1 0 343466669 6270976 874 33554432000 4194304 7823515 140737078289568 140735546443224 6957168 0 0 4096 0 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 1531 874 330 887 0 641 0

[startup+0.054535 s]
/proc/loadavg: 2.00 2.00 2.00 3/176 1732
/proc/meminfo: memFree=27779680/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=32848 CPUtime=0.04 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 7424 0 0 0 4 0 0 0 20 0 1 0 343466669 33636352 7373 33554432000 4194304 7823515 140737078289568 140737078221752 5544052 0 0 4096 0 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 8212 7373 349 887 0 7322 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 32848

[startup+0.100331 s]
/proc/loadavg: 2.00 2.00 2.00 3/176 1732
/proc/meminfo: memFree=27779680/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=54092 CPUtime=0.09 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 12763 0 0 0 8 1 0 0 20 0 1 0 343466669 55390208 12713 33554432000 4194304 7823515 140737078289568 140737078221752 5878822 0 0 4096 0 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 13523 12747 349 887 0 12633 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 54092

[startup+0.300288 s]
/proc/loadavg: 2.00 2.00 2.00 3/176 1732
/proc/meminfo: memFree=27779680/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=106608 CPUtime=0.29 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 25226 0 0 0 26 3 0 0 20 0 1 0 343466669 109166592 25175 33554432000 4194304 7823515 140737078289568 140737078221752 6957168 0 0 4096 0 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 26652 25175 354 887 0 25762 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 106608

[startup+0.700257 s]
/proc/loadavg: 2.00 2.00 2.00 3/176 1732
/proc/meminfo: memFree=27779680/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=118396 CPUtime=0.69 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 27660 0 0 0 66 3 0 0 20 0 1 0 343466669 121237504 27609 33554432000 4194304 7823515 140737078289568 140737078221752 6957168 0 0 4096 0 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 29599 27609 354 887 0 28709 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 118396

[startup+1.50026 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 1733
/proc/meminfo: memFree=27662972/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=141252 CPUtime=1.48 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 32516 0 0 0 144 4 0 0 20 0 1 0 343466669 144642048 32465 33554432000 4194304 7823515 140737078289568 140737078155672 6957168 0 0 4096 0 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 35313 32465 354 887 0 34423 0
Current children cumulated CPU time (s) 1.48
Current children cumulated vsize (KiB) 141252

[startup+3.10028 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 1733
/proc/meminfo: memFree=27633088/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=186072 CPUtime=3.09 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 42100 0 0 0 302 7 0 0 20 0 1 0 343466669 190537728 42049 33554432000 4194304 7823515 140737078289568 140737078155672 4526122 0 0 4096 0 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 46518 42049 354 887 0 45628 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 186072

[startup+6.30026 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 1733
/proc/meminfo: memFree=27560532/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=272240 CPUtime=6.28 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 60579 0 0 0 617 11 0 0 20 0 1 0 343466669 278773760 60562 33554432000 4194304 7823515 140737078289568 140737078155672 6957168 0 0 4096 0 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 68060 60562 354 887 0 67170 0
Current children cumulated CPU time (s) 6.28
Current children cumulated vsize (KiB) 272240

[startup+12.7003 s]
/proc/loadavg: 2.08 2.02 2.01 3/177 1733
/proc/meminfo: memFree=27404044/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=458056 CPUtime=12.68 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 99798 0 0 0 1248 20 0 0 20 0 1 0 343466669 469049344 99781 33554432000 4194304 7823515 140737078289568 140737078155672 6957168 0 0 4096 0 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 114514 99781 354 887 0 113624 0
Current children cumulated CPU time (s) 12.68
Current children cumulated vsize (KiB) 458056

[startup+25.5003 s]
/proc/loadavg: 2.06 2.02 2.00 3/177 1736
/proc/meminfo: memFree=27084076/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=813264 CPUtime=25.47 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 176185 0 0 0 2512 35 0 0 20 0 1 0 343466669 832782336 176171 33554432000 4194304 7823515 140737078289568 140737078155672 6830519 0 0 4096 0 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 203316 176171 354 887 0 202426 0
Current children cumulated CPU time (s) 25.47
Current children cumulated vsize (KiB) 813264

[startup+51.1003 s]
/proc/loadavg: 2.04 2.02 2.00 3/178 1737
/proc/meminfo: memFree=26025216/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=2279728 CPUtime=51.06 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 504691 0 0 0 5012 94 0 0 20 0 1 0 343466669 2334441472 504324 33554432000 4194304 7823515 140737078289568 140737078285672 6911387 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 569932 504327 370 887 0 569042 0
Current children cumulated CPU time (s) 51.06
Current children cumulated vsize (KiB) 2279728

[startup+102.307 s]
/proc/loadavg: 2.02 2.01 2.00 3/177 1737
/proc/meminfo: memFree=20212644/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=7736352 CPUtime=102.25 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 1895522 0 0 0 9933 292 0 0 20 0 1 0 343466669 7922024448 1894545 33554432000 4194304 7823515 140737078289568 140737078285672 4298277 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 1934088 1894545 470 887 0 1933198 0
Current children cumulated CPU time (s) 102.25
Current children cumulated vsize (KiB) 7736352

[startup+162.3 s]
/proc/loadavg: 2.07 2.02 2.00 3/177 1737
/proc/meminfo: memFree=15059296/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=13997668 CPUtime=162.22 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 2537443 0 0 0 15744 478 0 0 20 0 1 0 343466669 14333612032 3173155 33554432000 4194304 7823515 140737078289568 140737078285368 6209770 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 3499417 3173155 504 887 0 3498527 0
Current children cumulated CPU time (s) 162.22
Current children cumulated vsize (KiB) 13997668

[startup+222.3 s]
/proc/loadavg: 2.02 2.02 2.00 3/177 1737
/proc/meminfo: memFree=15006636/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=14047480 CPUtime=222.22 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 2540020 0 0 0 21739 483 0 0 20 0 1 0 343466669 14384619520 3185556 33554432000 4194304 7823515 140737078289568 140737078284904 6237405 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 3511870 3185556 504 887 0 3510980 0
Current children cumulated CPU time (s) 222.22
Current children cumulated vsize (KiB) 14047480

[startup+282.3 s]
/proc/loadavg: 2.01 2.01 2.00 3/177 1737
/proc/meminfo: memFree=15012688/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=14055732 CPUtime=282.21 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 2541310 0 0 0 27735 486 0 0 20 0 1 0 343466669 14393069568 3183432 33554432000 4194304 7823515 140737078289568 140737078284904 6267016 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 3513933 3183432 504 887 0 3513043 0
Current children cumulated CPU time (s) 282.21
Current children cumulated vsize (KiB) 14055732

[startup+342.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 1738
/proc/meminfo: memFree=15000144/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=14060832 CPUtime=342.19 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 2542026 0 0 0 33731 488 0 0 20 0 1 0 343466669 14398291968 3185534 33554432000 4194304 7823515 140737078289568 140737078284008 6254631 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 3515208 3185534 504 887 0 3514318 0
Current children cumulated CPU time (s) 342.19
Current children cumulated vsize (KiB) 14060832

[startup+402.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/176 1738
/proc/meminfo: memFree=14963680/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=14131008 CPUtime=402.19 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 2543491 0 0 0 39726 493 0 0 20 0 1 0 343466669 14470152192 3194257 33554432000 4194304 7823515 140737078289568 140737078284904 6212430 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 3532752 3194257 504 887 0 3531862 0
Current children cumulated CPU time (s) 402.19
Current children cumulated vsize (KiB) 14131008

[startup+462.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 1739
/proc/meminfo: memFree=14933820/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=14132312 CPUtime=462.18 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 2544006 0 0 0 45722 496 0 0 20 0 1 0 343466669 14471487488 3200913 33554432000 4194304 7823515 140737078289568 140737078284904 6211688 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 3533078 3200913 504 887 0 3532188 0
Current children cumulated CPU time (s) 462.18
Current children cumulated vsize (KiB) 14132312

[startup+522.3 s]
/proc/loadavg: 2.06 2.02 2.00 3/177 1739
/proc/meminfo: memFree=14925140/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=14133372 CPUtime=522.16 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 2544365 0 0 0 51718 498 0 0 20 0 1 0 343466669 14472572928 3202449 33554432000 4194304 7823515 140737078289568 140737078284008 6254145 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 3533343 3202449 504 887 0 3532453 0
Current children cumulated CPU time (s) 522.16
Current children cumulated vsize (KiB) 14133372

[startup+582.3 s]
/proc/loadavg: 2.02 2.01 2.00 3/177 1739
/proc/meminfo: memFree=14948480/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=14134708 CPUtime=582.15 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 2544882 0 0 0 57714 501 0 0 20 0 1 0 343466669 14473940992 3195793 33554432000 4194304 7823515 140737078289568 140737078284008 6254076 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 3533677 3195793 504 887 0 3532787 0
Current children cumulated CPU time (s) 582.15
Current children cumulated vsize (KiB) 14134708

[startup+642.3 s]

################
# More data... #
################

/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 2550478 0 0 0 111684 523 0 0 20 0 1 0 343466669 14499229696 3204349 33554432000 4194304 7823515 140737078289568 140737078284904 6215022 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 3539851 3204349 504 887 0 3538961 0
Current children cumulated CPU time (s) 1122.07
Current children cumulated vsize (KiB) 14159404

[startup+1182.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 1744
/proc/meminfo: memFree=14886032/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=14160076 CPUtime=1182.06 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 2550911 0 0 0 117681 525 0 0 20 0 1 0 343466669 14499917824 3203328 33554432000 4194304 7823515 140737078289568 140737078284520 6180619 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 3540019 3203328 504 887 0 3539129 0
Current children cumulated CPU time (s) 1182.06
Current children cumulated vsize (KiB) 14160076

[startup+1242.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 1788
/proc/meminfo: memFree=14849632/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=14160536 CPUtime=1242.05 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 2551368 0 0 0 123677 528 0 0 20 0 1 0 343466669 14500388864 3214152 33554432000 4194304 7823515 140737078289568 140737078284232 6254611 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 3540134 3214152 504 887 0 3539244 0
Current children cumulated CPU time (s) 1242.05
Current children cumulated vsize (KiB) 14160536

[startup+1302.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 1789
/proc/meminfo: memFree=13987504/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=14161336 CPUtime=1302.05 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 2551807 0 0 0 129675 530 0 0 20 0 1 0 343466669 14501208064 3207551 33554432000 4194304 7823515 140737078289568 140737078284904 6234850 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 3540334 3207551 504 887 0 3539444 0
Current children cumulated CPU time (s) 1302.05
Current children cumulated vsize (KiB) 14161336

[startup+1362.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 1789
/proc/meminfo: memFree=14016372/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=14161844 CPUtime=1362.03 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 2552238 0 0 0 135672 531 0 0 20 0 1 0 343466669 14501728256 3204536 33554432000 4194304 7823515 140737078289568 140737078284904 6180642 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 3540461 3204536 504 887 0 3539571 0
Current children cumulated CPU time (s) 1362.03
Current children cumulated vsize (KiB) 14161844

[startup+1422.31 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 1789
/proc/meminfo: memFree=13948372/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=14162216 CPUtime=1422.03 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 2552672 0 0 0 141670 533 0 0 20 0 1 0 343466669 14502109184 3205618 33554432000 4194304 7823515 140737078289568 140737078284904 6206197 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 3540554 3205618 504 887 0 3539664 0
Current children cumulated CPU time (s) 1422.03
Current children cumulated vsize (KiB) 14162216

[startup+1482.3 s]
/proc/loadavg: 2.04 2.01 2.00 3/177 1789
/proc/meminfo: memFree=13994808/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=14162756 CPUtime=1482.02 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 2552883 0 0 0 147668 534 0 0 20 0 1 0 343466669 14502662144 3203713 33554432000 4194304 7823515 140737078289568 140737078284008 6254072 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 3540689 3203713 504 887 0 3539799 0
Current children cumulated CPU time (s) 1482.02
Current children cumulated vsize (KiB) 14162756

[startup+1542.3 s]
/proc/loadavg: 2.01 2.01 2.00 3/177 1789
/proc/meminfo: memFree=14018912/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=14163144 CPUtime=1542.01 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 2553311 0 0 0 153665 536 0 0 20 0 1 0 343466669 14503059456 3204331 33554432000 4194304 7823515 140737078289568 140737078284008 6233498 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 3540786 3204331 504 887 0 3539896 0
Current children cumulated CPU time (s) 1542.01
Current children cumulated vsize (KiB) 14163144

[startup+1602.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 1791
/proc/meminfo: memFree=13951860/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=14163404 CPUtime=1602 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 2553762 0 0 0 159662 538 0 0 20 0 1 0 343466669 14503325696 3216738 33554432000 4194304 7823515 140737078289568 140737078284232 6254289 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 3540851 3216738 504 887 0 3539961 0
Current children cumulated CPU time (s) 1602
Current children cumulated vsize (KiB) 14163404

[startup+1662.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 1791
/proc/meminfo: memFree=13922528/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=14163556 CPUtime=1661.99 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 2554209 0 0 0 165658 541 0 0 20 0 1 0 343466669 14503481344 3212327 33554432000 4194304 7823515 140737078289568 140737078284008 6254272 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 3540889 3212327 504 887 0 3539999 0
Current children cumulated CPU time (s) 1661.99
Current children cumulated vsize (KiB) 14163556

[startup+1722.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 1791
/proc/meminfo: memFree=13979868/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=14163736 CPUtime=1721.97 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 2554646 0 0 0 171655 542 0 0 20 0 1 0 343466669 14503665664 3207915 33554432000 4194304 7823515 140737078289568 140737078284008 6254094 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 3540934 3207915 504 887 0 3540044 0
Current children cumulated CPU time (s) 1721.97
Current children cumulated vsize (KiB) 14163736

[startup+1782.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 1791
/proc/meminfo: memFree=14001712/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=14163924 CPUtime=1781.97 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 2555074 0 0 0 177653 544 0 0 20 0 1 0 343466669 14503858176 3206988 33554432000 4194304 7823515 140737078289568 140737078284904 6180619 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 3540981 3206988 504 887 0 3540091 0
Current children cumulated CPU time (s) 1781.97
Current children cumulated vsize (KiB) 14163924



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1800.4 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 1791
/proc/meminfo: memFree=13966804/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=14164076 CPUtime=1800.07 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 2555297 0 0 0 179462 545 0 0 20 0 1 0 343466669 14504013824 3211094 33554432000 4194304 7823515 140737078289568 140737078284232 6254696 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 3541019 3211094 504 887 0 3540129 0
Current children cumulated CPU time (s) 1800.07
Current children cumulated vsize (KiB) 14164076

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)

Sending SIGKILL to -1732

Solver just ended. Dumping a history of the last processes samples

[startup+1801.4 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 1792
/proc/meminfo: memFree=13966804/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=14164076 CPUtime=1801.07 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 2555297 0 0 0 179562 545 0 0 20 0 1 0 343466669 14504013824 3211094 33554432000 4194304 7823515 140737078289568 140737078282752 6254140 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 3541019 3211094 504 887 0 3540129 0
Current children cumulated CPU time (s) 1801.07
Current children cumulated vsize (KiB) 14164076

[startup+1802.2 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 1792
/proc/meminfo: memFree=13940864/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=14164076 CPUtime=1801.87 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4202496 2555297 0 0 0 179642 545 0 0 20 0 1 0 343466669 14504013824 3211094 33554432000 4194304 7823515 140737078289568 140737078282752 6254097 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 3541019 3211094 504 887 0 3540129 0
Current children cumulated CPU time (s) 1801.87
Current children cumulated vsize (KiB) 14164076

[startup+1802.6 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 1792
/proc/meminfo: memFree=13940864/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=0 CPUtime=1802.26 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4203524 2555297 0 0 0 179662 564 0 0 20 0 1 0 343466669 0 0 33554432000 0 0 0 0 0 0 0 4096 16384 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 1802.26
Current children cumulated vsize (KiB) 0

[startup+1802.8 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 1792
/proc/meminfo: memFree=23714156/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=0 CPUtime=1802.47 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4203524 2555297 0 0 0 179662 585 0 0 20 0 1 0 343466669 0 0 33554432000 0 0 0 0 0 0 0 4096 16384 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 1802.47
Current children cumulated vsize (KiB) 0

[startup+1802.91 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 1792
/proc/meminfo: memFree=23714156/32873844 swapFree=7372/7372
[pid=1732] ppid=1730 vsize=0 CPUtime=1802.57 cores=0,2,4,6
/proc/1732/stat : 1732 (pbscip.linux.x8) R 1730 1732 32091 0 -1 4203524 2555297 0 0 0 179662 595 0 0 20 0 1 0 343466669 0 0 33554432000 0 0 0 0 0 0 0 4096 16384 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/1732/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 1802.57
Current children cumulated vsize (KiB) 0

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1802.92
CPU time (s): 1802.6
CPU user time (s): 1796.63
CPU system time (s): 5.97009
CPU usage (%): 99.982
Max. virtual memory (cumulated for all children) (KiB): 15833072

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1796.63
system time used= 5.97009
maximum resident set size= 13528752
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2555297
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 74
involuntary context switches= 1871

runsolver used 2.41663 second user time and 7.44087 second system time

The end

Launcher Data 

Begin job on node109 at 2012-06-04 03:29:41
IDJOB=3741740
IDBENCH=89906
IDSOLVER=2321
FILE ID=node109/3741740-1338773363
RUNJOBID= node109-1338747536-32107
PBS_JOBID= 14636780
Free space on /tmp= 70128 MiB

SOLVER NAME= PB09: SCIPspx SCIP 1.1.0.7 with SoPLEX 1.4.1(24.4.2009)
BENCH NAME= PB11/normalized-PB11/OPT-SMALLINT-LIN/kullmann/normalized-sbox_8_shg.opb
COMMAND LINE= HOME/pbscip.linux.x86_64.gnu.opt-gcc42.spx -t TIMEOUT -m MEMLIMIT -f BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3741740-1338773363/watcher-3741740-1338773363 -o /tmp/evaluation-result-3741740-1338773363/solver-3741740-1338773363 -C 1800 -W 1900 -M 15500  HOME/pbscip.linux.x86_64.gnu.opt-gcc42.spx -t 1800 -m 15500 -f HOME/instance-3741740-1338773363.opb

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 3cb4e48d7e295175417c2e8c2155aba3
RANDOM SEED=444769382

node109.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.264
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.52
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.264
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.264
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.264
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.264
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.264
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.264
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.264
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.92
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873844 kB
MemFree:        27779936 kB
Buffers:         1867352 kB
Cached:          2279384 kB
SwapCached:          988 kB
Active:          3083364 kB
Inactive:        1121904 kB
Active(anon):      49296 kB
Inactive(anon):     7876 kB
Active(file):    3034068 kB
Inactive(file):  1114028 kB
Unevictable:        7372 kB
Mlocked:            7372 kB
SwapTotal:      67108856 kB
SwapFree:       67105540 kB
Dirty:            598908 kB
Writeback:             0 kB
AnonPages:         61284 kB
Mapped:            12316 kB
Shmem:                 4 kB
Slab:             749812 kB
SReclaimable:     190948 kB
SUnreclaim:       558864 kB
KernelStack:        1440 kB
PageTables:         4012 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545776 kB
Committed_AS:     171036 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346368 kB
VmallocChunk:   34341987984 kB
HardwareCorrupted:     0 kB
AnonHugePages:     38912 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7872 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 70088 MiB
End job on node109 at 2012-06-04 03:59:44