Trace number 3693713

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
SCIP spx standard SCIP 2.1.1.4. with SoPlex 1.6.0.3 standard fixed? (TO) 1802.08 1802.37

General information on the benchmark

Name/OPT-SMALLINT-LIN/sroussel/NG/
normalized-NG.dot_luby_20null0mast_rapportOE2.0_TMAX300_K192_cod2.opb
MD5SUMcf9128b870ef9b100d6538eb54f0c456
Bench CategoryOPT-SMALLINT-LIN (optimisation, small integers, linear constraints)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark-451
Best CPU time to get the best result obtained on this benchmark1800.75
Has Objective FunctionYES
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables400896
Total number of constraints402763
Number of constraints which are clauses400785
Number of constraints which are cardinality constraints (but not clauses)194
Number of constraints which are nor clauses,nor cardinality constraints1784
Minimum length of a constraint2
Maximum length of a constraint58023
Number of terms in the objective function 1427
Biggest coefficient in the objective function 512
Number of bits for the biggest coefficient in the objective function 10
Sum of the numbers in the objective function 6807
Number of bits of the sum of numbers in the objective function 13
Biggest number in a constraint 512
Number of bits of the biggest number in a constraint 10
Biggest sum of numbers in a constraint 65956
Number of bits of the biggest sum of numbers17
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c SCIP version 2.1.1.4 [precision: 8 byte] [memory: block] [mode: optimized] [LP solver: SoPlex 1.6.0.3] [GitHash: a3bf3a4-dirty]
0.00/0.00	c Copyright (c) 2002-2012 Konrad-Zuse-Zentrum fuer Informationstechnik Berlin (ZIB)
0.00/0.00	c 
0.00/0.00	c user parameter file <scip.set> not found - using default parameters
0.00/0.00	c reading problem <HOME/instance-3693713-1338047805.opb>
1.99/2.08	c original problem has 400896 variables (400896 bin, 0 int, 0 impl, 0 cont) and 402763 constraints
1.99/2.08	c problem read in 2.08
4.58/4.67	c presolving:
8.38/8.46	c (round 1) 0 del vars, 1977 del conss, 0 add conss, 0 chg bounds, 0 chg sides, 0 chg coeffs, 398809 upgd conss, 67806 impls, 1977 clqs
9.48/9.56	c (round 2) 0 del vars, 1977 del conss, 0 add conss, 0 chg bounds, 0 chg sides, 0 chg coeffs, 400786 upgd conss, 67806 impls, 1977 clqs

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3693713-1338047805/watcher-3693713-1338047805 -o /tmp/evaluation-result-3693713-1338047805/solver-3693713-1338047805 -C 1800 -W 1900 -M 15500 HOME/scip-2.1.1.4sfix.linux.x86_64.gnu.opt.spx -f HOME/instance-3693713-1338047805.opb -t 1800 -m 15500 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.09 2.04 2.00 3/176 5743
/proc/meminfo: memFree=26666260/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=9152 CPUtime=0 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 1067 0 0 0 0 0 0 0 20 0 1 0 270906333 9371648 1022 33554432000 4194304 10169799 140734861610512 140736064043368 8844157 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 2288 1033 387 1459 0 820 0

[startup+0.09442 s]
/proc/loadavg: 2.09 2.04 2.00 3/176 5743
/proc/meminfo: memFree=26666260/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=42344 CPUtime=0.08 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 9346 0 0 0 7 1 0 0 20 0 1 0 270906333 43360256 9300 33554432000 4194304 10169799 140734861610512 140736064043368 7543003 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 10586 9300 412 1459 0 9118 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 42344

[startup+0.100223 s]
/proc/loadavg: 2.09 2.04 2.00 3/176 5743
/proc/meminfo: memFree=26666260/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=43536 CPUtime=0.09 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 9643 0 0 0 8 1 0 0 20 0 1 0 270906333 44580864 9597 33554432000 4194304 10169799 140734861610512 140736064043368 5477139 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 10884 9597 412 1459 0 9416 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 43536

[startup+0.300237 s]
/proc/loadavg: 2.09 2.04 2.00 3/176 5743
/proc/meminfo: memFree=26666260/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=84500 CPUtime=0.29 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 19825 0 0 0 27 2 0 0 20 0 1 0 270906333 86528000 19779 33554432000 4194304 10169799 140734861610512 140736064043368 7169630 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 21318 19789 412 1459 0 19850 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 84500

[startup+0.700284 s]
/proc/loadavg: 2.09 2.04 2.00 3/176 5743
/proc/meminfo: memFree=26666260/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=190504 CPUtime=0.68 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 46275 0 0 0 63 5 0 0 20 0 1 0 270906333 195076096 46229 33554432000 4194304 10169799 140734861610512 140734861476840 8744515 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 47626 46229 412 1459 0 46158 0
Current children cumulated CPU time (s) 0.68
Current children cumulated vsize (KiB) 190504

[startup+1.5003 s]
/proc/loadavg: 2.09 2.04 2.00 3/177 5744
/proc/meminfo: memFree=26406316/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=388780 CPUtime=1.48 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 95811 0 0 0 136 12 0 0 20 0 1 0 270906333 398110720 95765 33554432000 4194304 10169799 140734861610512 140734861476168 5281244 0 0 4096 0 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 97195 95765 412 1459 0 95727 0
Current children cumulated CPU time (s) 1.48
Current children cumulated vsize (KiB) 388780

[startup+3.10032 s]
/proc/loadavg: 2.09 2.04 2.00 3/177 5744
/proc/meminfo: memFree=26233212/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=935584 CPUtime=3.09 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 231512 0 0 0 280 29 0 0 20 0 1 0 270906333 958038016 232362 33554432000 4194304 10169799 140734861610512 140734861605576 5290857 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 233896 232362 489 1459 0 232428 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 935584

[startup+6.30024 s]
/proc/loadavg: 2.17 2.06 2.01 3/177 5744
/proc/meminfo: memFree=25428300/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=1346764 CPUtime=6.29 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 356655 0 0 0 566 63 0 0 20 0 1 0 270906333 1379086336 333957 33554432000 4194304 10169799 140734861610512 140734861608664 8961143 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 336691 333957 554 1459 0 335223 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 1346764

[startup+12.7002 s]
/proc/loadavg: 2.15 2.06 2.01 3/177 5744
/proc/meminfo: memFree=24760768/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=1902856 CPUtime=12.69 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 507238 0 0 0 1143 126 0 0 20 0 1 0 270906333 1948524544 471476 33554432000 4194304 10169799 140734861610512 140734861607160 8922295 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 475714 471476 608 1459 0 474246 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 1902856

[startup+25.5003 s]
/proc/loadavg: 2.12 2.06 2.01 3/177 5745
/proc/meminfo: memFree=24755260/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=1910216 CPUtime=25.48 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 508399 0 0 0 2422 126 0 0 20 0 1 0 270906333 1956061184 473148 33554432000 4194304 10169799 140734861610512 140734861607032 8922310 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 477554 473148 608 1459 0 476086 0
Current children cumulated CPU time (s) 25.48
Current children cumulated vsize (KiB) 1910216

[startup+51.1002 s]
/proc/loadavg: 2.08 2.05 2.00 3/177 5745
/proc/meminfo: memFree=24751336/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=1914124 CPUtime=51.08 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 509454 0 0 0 4982 126 0 0 20 0 1 0 270906333 1960062976 474203 33554432000 4194304 10169799 140734861610512 140734861608040 8922310 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 478531 474203 608 1459 0 477063 0
Current children cumulated CPU time (s) 51.08
Current children cumulated vsize (KiB) 1914124

[startup+102.307 s]
/proc/loadavg: 2.03 2.04 2.00 3/177 5745
/proc/meminfo: memFree=24713668/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=1948932 CPUtime=102.27 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 518387 0 0 0 10100 127 0 0 20 0 1 0 270906333 1995706368 482641 33554432000 4194304 10169799 140734861610512 140734861608040 4936832 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 487233 482641 608 1459 0 485765 0
Current children cumulated CPU time (s) 102.27
Current children cumulated vsize (KiB) 1948932

[startup+162.3 s]
/proc/loadavg: 2.01 2.03 2.00 3/177 5745
/proc/meminfo: memFree=23386384/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=3308376 CPUtime=162.26 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 829724 0 0 0 16058 168 0 0 20 0 1 0 270906333 3387777024 813412 33554432000 4194304 10169799 140734861610512 140734861606776 8922280 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 827094 813412 608 1459 0 825626 0
Current children cumulated CPU time (s) 162.26
Current children cumulated vsize (KiB) 3308376

[startup+222.3 s]
/proc/loadavg: 2.04 2.04 2.00 3/178 5748
/proc/meminfo: memFree=23384812/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=3309400 CPUtime=222.26 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 830192 0 0 0 22057 169 0 0 20 0 1 0 270906333 3388825600 813880 33554432000 4194304 10169799 140734861610512 140734861606776 8922305 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 827350 813880 608 1459 0 825882 0
Current children cumulated CPU time (s) 222.26
Current children cumulated vsize (KiB) 3309400

[startup+282.3 s]
/proc/loadavg: 2.05 2.04 2.00 3/178 5809
/proc/meminfo: memFree=23735512/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=3311256 CPUtime=282.25 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 830866 0 0 0 28056 169 0 0 20 0 1 0 270906333 3390726144 814615 33554432000 4194304 10169799 140734861610512 140734861606776 8922305 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 827814 814615 608 1459 0 826346 0
Current children cumulated CPU time (s) 282.25
Current children cumulated vsize (KiB) 3311256

[startup+342.3 s]
/proc/loadavg: 2.02 2.03 2.00 3/177 5810
/proc/meminfo: memFree=22375084/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=3313304 CPUtime=342.24 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 831564 0 0 0 34055 169 0 0 20 0 1 0 270906333 3392823296 815313 33554432000 4194304 10169799 140734861610512 140734861606776 8922285 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 828326 815313 608 1459 0 826858 0
Current children cumulated CPU time (s) 342.24
Current children cumulated vsize (KiB) 3313304

[startup+402.3 s]
/proc/loadavg: 2.00 2.02 2.00 3/177 5810
/proc/meminfo: memFree=22371648/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=3313304 CPUtime=402.23 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 831776 0 0 0 40054 169 0 0 20 0 1 0 270906333 3392823296 815525 33554432000 4194304 10169799 140734861610512 140734861606776 8922280 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 828326 815525 608 1459 0 826858 0
Current children cumulated CPU time (s) 402.23
Current children cumulated vsize (KiB) 3313304

[startup+462.3 s]
/proc/loadavg: 2.00 2.02 2.00 3/177 5810
/proc/meminfo: memFree=22369348/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=3314196 CPUtime=462.21 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 832244 0 0 0 46052 169 0 0 20 0 1 0 270906333 3393736704 815960 33554432000 4194304 10169799 140734861610512 140734861607304 8922310 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 828549 815960 608 1459 0 827081 0
Current children cumulated CPU time (s) 462.21
Current children cumulated vsize (KiB) 3314196

[startup+522.3 s]
/proc/loadavg: 2.00 2.01 2.00 3/177 5810
/proc/meminfo: memFree=22361820/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=3316052 CPUtime=522.2 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 832856 0 0 0 52051 169 0 0 20 0 1 0 270906333 3395637248 816573 33554432000 4194304 10169799 140734861610512 140734861606776 8922295 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 829013 816573 608 1459 0 827545 0
Current children cumulated CPU time (s) 522.2
Current children cumulated vsize (KiB) 3316052

[startup+582.3 s]
/proc/loadavg: 2.00 2.01 2.00 3/177 5810
/proc/meminfo: memFree=22359492/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=3316052 CPUtime=582.2 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 833076 0 0 0 58050 170 0 0 20 0 1 0 270906333 3395637248 816961 33554432000 4194304 10169799 140734861610512 140734861606776 8922290 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 829013 816961 608 1459 0 827545 0
Current children cumulated CPU time (s) 582.2
Current children cumulated vsize (KiB) 3316052

[startup+642.3 s]

################
# More data... #
################

/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 837107 0 0 0 106041 172 0 0 20 0 1 0 270906333 3405131776 824996 33554432000 4194304 10169799 140734861610512 140734861606776 8922361 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 831331 824996 608 1459 0 829863 0
Current children cumulated CPU time (s) 1062.13
Current children cumulated vsize (KiB) 3325324

[startup+1122.3 s]
/proc/loadavg: 2.08 2.02 2.01 3/177 5815
/proc/meminfo: memFree=22298936/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=3326348 CPUtime=1122.11 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 837487 0 0 0 112039 172 0 0 20 0 1 0 270906333 3406180352 827036 33554432000 4194304 10169799 140734861610512 140734861606776 8922295 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 831587 827036 608 1459 0 830119 0
Current children cumulated CPU time (s) 1122.11
Current children cumulated vsize (KiB) 3326348

[startup+1182.3 s]
/proc/loadavg: 2.03 2.01 2.00 3/177 5815
/proc/meminfo: memFree=22292860/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=3328204 CPUtime=1182.1 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 838019 0 0 0 118038 172 0 0 20 0 1 0 270906333 3408080896 828426 33554432000 4194304 10169799 140734861610512 140734861606776 8922316 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 832051 828426 608 1459 0 830583 0
Current children cumulated CPU time (s) 1182.1
Current children cumulated vsize (KiB) 3328204

[startup+1242.3 s]
/proc/loadavg: 2.01 2.00 2.00 3/177 5815
/proc/meminfo: memFree=22290560/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=3328204 CPUtime=1242.09 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 838059 0 0 0 124037 172 0 0 20 0 1 0 270906333 3408080896 828466 33554432000 4194304 10169799 140734861610512 140734861606776 8922285 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 832051 828466 608 1459 0 830583 0
Current children cumulated CPU time (s) 1242.09
Current children cumulated vsize (KiB) 3328204

[startup+1302.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 5816
/proc/meminfo: memFree=22285944/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=3330252 CPUtime=1302.09 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 838612 0 0 0 130037 172 0 0 20 0 1 0 270906333 3410178048 829019 33554432000 4194304 10169799 140734861610512 140734861606776 8922305 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 832563 829019 608 1459 0 831095 0
Current children cumulated CPU time (s) 1302.09
Current children cumulated vsize (KiB) 3330252

[startup+1362.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 5816
/proc/meminfo: memFree=22284520/32873660 swapFree=7112/7112
[pid=5743] ppid=5741 vsize=3331276 CPUtime=1362.07 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 838908 0 0 0 136035 172 0 0 20 0 1 0 270906333 3411226624 829315 33554432000 4194304 10169799 140734861610512 140734861606776 8922295 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 832819 829315 608 1459 0 831351 0
Current children cumulated CPU time (s) 1362.07
Current children cumulated vsize (KiB) 3331276

[startup+1422.31 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 5819
/proc/meminfo: memFree=22279900/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=3333132 CPUtime=1422.07 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 839413 0 0 0 142034 173 0 0 20 0 1 0 270906333 3413127168 829820 33554432000 4194304 10169799 140734861610512 140734861606776 8922295 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 833283 829820 608 1459 0 831815 0
Current children cumulated CPU time (s) 1422.07
Current children cumulated vsize (KiB) 3333132

[startup+1482.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 5819
/proc/meminfo: memFree=22278548/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=3333132 CPUtime=1482.06 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 839453 0 0 0 148033 173 0 0 20 0 1 0 270906333 3413127168 830018 33554432000 4194304 10169799 140734861610512 140734861606776 8922295 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 833283 830018 608 1459 0 831815 0
Current children cumulated CPU time (s) 1482.06
Current children cumulated vsize (KiB) 3333132

[startup+1542.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 5819
/proc/meminfo: memFree=22274000/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=3334024 CPUtime=1542.05 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 839750 0 0 0 154032 173 0 0 20 0 1 0 270906333 3414040576 830600 33554432000 4194304 10169799 140734861610512 140734861606776 8922300 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 833506 830600 608 1459 0 832038 0
Current children cumulated CPU time (s) 1542.05
Current children cumulated vsize (KiB) 3334024

[startup+1602.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 5820
/proc/meminfo: memFree=22269000/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=3335048 CPUtime=1602.04 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 840048 0 0 0 160031 173 0 0 20 0 1 0 270906333 3415089152 831390 33554432000 4194304 10169799 140734861610512 140734861606776 8922280 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 833762 831390 608 1459 0 832294 0
Current children cumulated CPU time (s) 1602.04
Current children cumulated vsize (KiB) 3335048

[startup+1662.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 5820
/proc/meminfo: memFree=22267168/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=3336072 CPUtime=1662.03 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 840334 0 0 0 166030 173 0 0 20 0 1 0 270906333 3416137728 831676 33554432000 4194304 10169799 140734861610512 140734861606776 8922285 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 834018 831676 608 1459 0 832550 0
Current children cumulated CPU time (s) 1662.03
Current children cumulated vsize (KiB) 3336072

[startup+1722.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 5820
/proc/meminfo: memFree=22264716/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=3336904 CPUtime=1722.01 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 840582 0 0 0 172028 173 0 0 20 0 1 0 270906333 3416989696 831923 33554432000 4194304 10169799 140734861610512 140734861606776 8922290 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 834226 831923 608 1459 0 832758 0
Current children cumulated CPU time (s) 1722.01
Current children cumulated vsize (KiB) 3336904

[startup+1782.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 5820
/proc/meminfo: memFree=22260592/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=3338952 CPUtime=1782 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 841138 0 0 0 178027 173 0 0 20 0 1 0 270906333 3419086848 832518 33554432000 4194304 10169799 140734861610512 140734861606776 8922310 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 834738 832518 608 1459 0 833270 0
Current children cumulated CPU time (s) 1782
Current children cumulated vsize (KiB) 3338952



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1800.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 5821
/proc/meminfo: memFree=22260800/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=3338952 CPUtime=1800.01 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 841152 0 0 0 179828 173 0 0 20 0 1 0 270906333 3419086848 832532 33554432000 4194304 10169799 140734861610512 140734861606776 8922290 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 834738 832532 608 1459 0 833270 0
Current children cumulated CPU time (s) 1800.01
Current children cumulated vsize (KiB) 3338952

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)

Sending SIGKILL to -5743

Solver just ended. Dumping a history of the last processes samples

[startup+1801.4 s]
/proc/loadavg: 2.00 2.00 2.00 3/179 5822
/proc/meminfo: memFree=22260668/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=3338952 CPUtime=1801.1 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 841152 0 0 0 179937 173 0 0 20 0 1 0 270906333 3419086848 832532 33554432000 4194304 10169799 140734861610512 140734861606208 8922280 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 834738 832532 608 1459 0 833270 0
Current children cumulated CPU time (s) 1801.1
Current children cumulated vsize (KiB) 3338952

[startup+1801.81 s]
/proc/loadavg: 2.00 2.00 2.00 3/179 5822
/proc/meminfo: memFree=22260420/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=3338952 CPUtime=1801.51 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 841152 0 0 0 179978 173 0 0 20 0 1 0 270906333 3419086848 832532 33554432000 4194304 10169799 140734861610512 140734861606208 8922285 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 834738 832532 608 1459 0 833270 0
Current children cumulated CPU time (s) 1801.51
Current children cumulated vsize (KiB) 3338952

[startup+1802.2 s]
/proc/loadavg: 2.00 2.00 2.00 3/179 5822
/proc/meminfo: memFree=22260420/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=3338952 CPUtime=1801.9 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 841152 0 0 0 180017 173 0 0 20 0 1 0 270906333 3419086848 832532 33554432000 4194304 10169799 140734861610512 140734861606208 8922290 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 834738 832532 608 1459 0 833270 0
Current children cumulated CPU time (s) 1801.9
Current children cumulated vsize (KiB) 3338952

[startup+1802.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/179 5822
/proc/meminfo: memFree=22260420/32873660 swapFree=7108/7108
[pid=5743] ppid=5741 vsize=3338952 CPUtime=1802 cores=1,3,5,7
/proc/5743/stat : 5743 (scip-2.1.1.4sfi) R 5741 5743 3226 0 -1 4202496 841152 0 0 0 180027 173 0 0 20 0 1 0 270906333 3419086848 832532 33554432000 4194304 10169799 140734861610512 140734861606208 8922300 0 0 4096 16384 0 0 0 17 3 0 0 0 0 0
/proc/5743/statm: 834738 832532 608 1459 0 833270 0
Current children cumulated CPU time (s) 1802
Current children cumulated vsize (KiB) 3338952

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1802.37
CPU time (s): 1802.08
CPU user time (s): 1800.28
CPU system time (s): 1.79773
CPU usage (%): 99.984
Max. virtual memory (cumulated for all children) (KiB): 3338952

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1800.28
system time used= 1.79773
maximum resident set size= 3330128
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 841152
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 21
involuntary context switches= 1853

runsolver used 2.96755 second user time and 8.63269 second system time

The end

Launcher Data 

Begin job on node117 at 2012-05-26 17:56:46
IDJOB=3693713
IDBENCH=91092
IDSOLVER=2272
FILE ID=node117/3693713-1338047805
RUNJOBID= node117-1338029730-4453
PBS_JOBID= 14624354
Free space on /tmp= 71468 MiB

SOLVER NAME= SCIP spx standard SCIP 2.1.1.4. with SoPlex 1.6.0.3 standard fixed
BENCH NAME= PB12/normalized-PB12/OPT-SMALLINT-LIN/sroussel/NG/normalized-NG.dot_luby_20null0mast_rapportOE2.0_TMAX300_K192_cod2.opb
COMMAND LINE= HOME/scip-2.1.1.4sfix.linux.x86_64.gnu.opt.spx -f BENCHNAME -t TIMEOUT -m MEMLIMIT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3693713-1338047805/watcher-3693713-1338047805 -o /tmp/evaluation-result-3693713-1338047805/solver-3693713-1338047805 -C 1800 -W 1900 -M 15500  HOME/scip-2.1.1.4sfix.linux.x86_64.gnu.opt.spx -f HOME/instance-3693713-1338047805.opb -t 1800 -m 15500

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= cf9128b870ef9b100d6538eb54f0c456
RANDOM SEED=360827610

node117.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.148
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.29
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.148
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.148
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.148
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.95
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.148
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.148
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.148
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.148
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5331.91
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873660 kB
MemFree:        26666136 kB
Buffers:          271828 kB
Cached:          2876936 kB
SwapCached:            0 kB
Active:          2874476 kB
Inactive:        2541020 kB
Active(anon):    2269248 kB
Inactive(anon):        0 kB
Active(file):     605228 kB
Inactive(file):  2541020 kB
Unevictable:        7108 kB
Mlocked:            7108 kB
SwapTotal:      67108856 kB
SwapFree:       67108856 kB
Dirty:             41064 kB
Writeback:             0 kB
AnonPages:       2416876 kB
Mapped:            17748 kB
Shmem:               200 kB
Slab:             645788 kB
SReclaimable:      90040 kB
SUnreclaim:       555748 kB
KernelStack:        1448 kB
PageTables:         8632 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545684 kB
Committed_AS:    2431804 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346400 kB
VmallocChunk:   34341988052 kB
HardwareCorrupted:     0 kB
AnonHugePages:    780288 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7680 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 71432 MiB
End job on node117 at 2012-05-26 18:26:48