Trace number 3693717

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
SCIP spx standard SCIP 2.1.1.4. with SoPlex 1.6.0.3 standard fixed? (TO) 1802.2 1802.47

General information on the benchmark

Name/OPT-SMALLINT-LIN/sroussel/NG/
normalized-NG.dot_luby_50null20mast_rapportOE2.0_TMAX60_K192_cod2.opb
MD5SUM6f3eb6421078a0de6a5b5f585e0aca61
Bench CategoryOPT-SMALLINT-LIN (optimisation, small integers, linear constraints)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark-5280
Best CPU time to get the best result obtained on this benchmark1781.2
Has Objective FunctionYES
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables400896
Total number of constraints402763
Number of constraints which are clauses400785
Number of constraints which are cardinality constraints (but not clauses)194
Number of constraints which are nor clauses,nor cardinality constraints1784
Minimum length of a constraint2
Maximum length of a constraint58023
Number of terms in the objective function 892
Biggest coefficient in the objective function 1024
Number of bits for the biggest coefficient in the objective function 11
Sum of the numbers in the objective function 367542
Number of bits of the sum of numbers in the objective function 19
Biggest number in a constraint 1024
Number of bits of the biggest number in a constraint 11
Biggest sum of numbers in a constraint 367542
Number of bits of the biggest sum of numbers19
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.00/0.00	c SCIP version 2.1.1.4 [precision: 8 byte] [memory: block] [mode: optimized] [LP solver: SoPlex 1.6.0.3] [GitHash: a3bf3a4-dirty]
0.00/0.00	c Copyright (c) 2002-2012 Konrad-Zuse-Zentrum fuer Informationstechnik Berlin (ZIB)
0.00/0.00	c 
0.00/0.00	c user parameter file <scip.set> not found - using default parameters
0.00/0.00	c reading problem <HOME/instance-3693717-1338047229.opb>
1.99/2.09	c original problem has 400896 variables (400896 bin, 0 int, 0 impl, 0 cont) and 402763 constraints
1.99/2.09	c problem read in 2.09
4.59/4.69	c presolving:
8.39/8.48	c (round 1) 0 del vars, 1977 del conss, 0 add conss, 0 chg bounds, 0 chg sides, 0 chg coeffs, 398809 upgd conss, 67806 impls, 1977 clqs
9.49/9.57	c (round 2) 0 del vars, 1977 del conss, 0 add conss, 0 chg bounds, 0 chg sides, 0 chg coeffs, 400786 upgd conss, 67806 impls, 1977 clqs

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3693717-1338047229/watcher-3693717-1338047229 -o /tmp/evaluation-result-3693717-1338047229/solver-3693717-1338047229 -C 1800 -W 1900 -M 15500 HOME/scip-2.1.1.4sfix.linux.x86_64.gnu.opt.spx -f HOME/instance-3693717-1338047229.opb -t 1800 -m 15500 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.92 1.98 1.99 2/176 2190
/proc/meminfo: memFree=27290600/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=11132 CPUtime=0 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 1540 0 0 0 0 0 0 0 20 0 1 0 270848820 11399168 1495 33554432000 4194304 10169799 140735078376448 140735078241144 7543003 0 0 4096 0 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 2783 1495 412 1459 0 1315 0

[startup+0.0525121 s]
/proc/loadavg: 1.92 1.98 1.99 2/176 2190
/proc/meminfo: memFree=27290600/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=30836 CPUtime=0.04 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 6466 0 0 0 4 0 0 0 20 0 1 0 270848820 31576064 6421 33554432000 4194304 10169799 140735078376448 140735078241144 7543003 0 0 4096 0 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 7709 6421 412 1459 0 6241 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 30836

[startup+0.100286 s]
/proc/loadavg: 1.92 1.98 1.99 2/176 2190
/proc/meminfo: memFree=27290600/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=43580 CPUtime=0.09 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 9634 0 0 0 8 1 0 0 20 0 1 0 270848820 44625920 9589 33554432000 4194304 10169799 140735078376448 140735078241144 8982256 0 0 4096 0 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 10895 9589 412 1459 0 9427 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 43580

[startup+0.300284 s]
/proc/loadavg: 1.92 1.98 1.99 2/176 2190
/proc/meminfo: memFree=27290600/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=85120 CPUtime=0.29 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 20010 0 0 0 27 2 0 0 20 0 1 0 270848820 87162880 19965 33554432000 4194304 10169799 140735078376448 140735078241144 7172001 0 0 4096 0 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 21280 19965 412 1459 0 19812 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 85120

[startup+0.700281 s]
/proc/loadavg: 1.92 1.98 1.99 2/176 2190
/proc/meminfo: memFree=27290600/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=190472 CPUtime=0.69 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 46264 0 0 0 63 6 0 0 20 0 1 0 270848820 195043328 46219 33554432000 4194304 10169799 140735078376448 140735078242184 8982256 0 0 4096 0 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 47618 46219 412 1459 0 46150 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 190472

[startup+1.50035 s]
/proc/loadavg: 1.92 1.98 1.99 3/177 2191
/proc/meminfo: memFree=26735164/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=387716 CPUtime=1.49 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 95540 0 0 0 136 13 0 0 20 0 1 0 270848820 397021184 95495 33554432000 4194304 10169799 140735078376448 140735078242776 7543003 0 0 4096 0 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 96929 95496 412 1459 0 95461 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 387716

[startup+3.10026 s]
/proc/loadavg: 1.92 1.98 1.99 3/177 2191
/proc/meminfo: memFree=26239288/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=935660 CPUtime=3.09 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 231968 0 0 0 277 32 0 0 20 0 1 0 270848820 958115840 232382 33554432000 4194304 10169799 140735078376448 140735078373800 5757893 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 233915 232382 490 1459 0 232447 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 935660

[startup+6.3003 s]
/proc/loadavg: 1.93 1.98 1.99 3/177 2191
/proc/meminfo: memFree=24736392/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=1339212 CPUtime=6.29 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 356295 0 0 0 562 67 0 0 20 0 1 0 270848820 1371353088 332110 33554432000 4194304 10169799 140735078376448 140735078374600 8961143 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 334803 332110 555 1459 0 333335 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 1339212

[startup+12.7003 s]
/proc/loadavg: 1.93 1.98 1.99 3/177 2191
/proc/meminfo: memFree=24080680/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=1903132 CPUtime=12.69 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 509177 0 0 0 1143 126 0 0 20 0 1 0 270848820 1948807168 471367 33554432000 4194304 10169799 140735078376448 140735078373096 8922305 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 475783 471367 609 1459 0 474315 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 1903132

[startup+25.5004 s]
/proc/loadavg: 1.95 1.98 1.99 3/177 2191
/proc/meminfo: memFree=24078952/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=1906012 CPUtime=25.49 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 509955 0 0 0 2423 126 0 0 20 0 1 0 270848820 1951756288 472145 33554432000 4194304 10169799 140735078376448 140735078373000 8922295 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 476503 472145 609 1459 0 475035 0
Current children cumulated CPU time (s) 25.49
Current children cumulated vsize (KiB) 1906012

[startup+51.1002 s]
/proc/loadavg: 2.03 2.00 2.00 3/177 2191
/proc/meminfo: memFree=24068176/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=1914652 CPUtime=51.09 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 511496 0 0 0 4982 127 0 0 20 0 1 0 270848820 1960603648 474197 33554432000 4194304 10169799 140735078376448 140735078373976 8922254 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 478663 474197 609 1459 0 477195 0
Current children cumulated CPU time (s) 51.09
Current children cumulated vsize (KiB) 1914652

[startup+102.306 s]
/proc/loadavg: 2.01 2.00 2.00 3/177 2191
/proc/meminfo: memFree=24039700/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=1942028 CPUtime=102.28 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 518629 0 0 0 10100 128 0 0 20 0 1 0 270848820 1988636672 480769 33554432000 4194304 10169799 140735078376448 140735078373976 8922365 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 485507 480769 609 1459 0 484039 0
Current children cumulated CPU time (s) 102.28
Current children cumulated vsize (KiB) 1942028

[startup+162.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 2191
/proc/meminfo: memFree=22704304/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=3308280 CPUtime=162.27 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 831393 0 0 0 16057 170 0 0 20 0 1 0 270848820 3387678720 813478 33554432000 4194304 10169799 140735078376448 140735078372712 8922280 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 827070 813478 609 1459 0 825602 0
Current children cumulated CPU time (s) 162.27
Current children cumulated vsize (KiB) 3308280

[startup+222.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 2195
/proc/meminfo: memFree=22699988/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=3309304 CPUtime=222.26 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 831857 0 0 0 22056 170 0 0 20 0 1 0 270848820 3388727296 813942 33554432000 4194304 10169799 140735078376448 140735078372712 8922310 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 827326 813942 609 1459 0 825858 0
Current children cumulated CPU time (s) 222.26
Current children cumulated vsize (KiB) 3309304

[startup+282.301 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 2195
/proc/meminfo: memFree=22694692/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=3313208 CPUtime=282.25 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 833042 0 0 0 28055 170 0 0 20 0 1 0 270848820 3392724992 815127 33554432000 4194304 10169799 140735078376448 140735078372712 8922295 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 828302 815127 609 1459 0 826834 0
Current children cumulated CPU time (s) 282.25
Current children cumulated vsize (KiB) 3313208

[startup+342.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 2195
/proc/meminfo: memFree=22688768/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=3313208 CPUtime=342.24 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 833259 0 0 0 34054 170 0 0 20 0 1 0 270848820 3392724992 815344 33554432000 4194304 10169799 140735078376448 140735078372712 8922285 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 828302 815344 609 1459 0 826834 0
Current children cumulated CPU time (s) 342.24
Current children cumulated vsize (KiB) 3313208

[startup+402.301 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 2195
/proc/meminfo: memFree=22685820/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=3314100 CPUtime=402.23 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 833732 0 0 0 40053 170 0 0 20 0 1 0 270848820 3393638400 815784 33554432000 4194304 10169799 140735078376448 140735078373240 8922290 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 828525 815784 609 1459 0 827057 0
Current children cumulated CPU time (s) 402.23
Current children cumulated vsize (KiB) 3314100

[startup+462.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 2195
/proc/meminfo: memFree=22671836/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=3315956 CPUtime=462.22 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 834412 0 0 0 46051 171 0 0 20 0 1 0 270848820 3395538944 816464 33554432000 4194304 10169799 140735078376448 140735078372712 8922300 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 828989 816464 609 1459 0 827521 0
Current children cumulated CPU time (s) 462.22
Current children cumulated vsize (KiB) 3315956

[startup+522.301 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 2196
/proc/meminfo: memFree=22665168/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=3315956 CPUtime=522.21 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 834628 0 0 0 52050 171 0 0 20 0 1 0 270848820 3395538944 816680 33554432000 4194304 10169799 140735078376448 140735078372712 8922351 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 828989 816680 609 1459 0 827521 0
Current children cumulated CPU time (s) 522.21
Current children cumulated vsize (KiB) 3315956

[startup+582.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 2196
/proc/meminfo: memFree=22660980/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=3316980 CPUtime=582.2 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 835132 0 0 0 58049 171 0 0 20 0 1 0 270848820 3396587520 817184 33554432000 4194304 10169799 140735078376448 140735078372712 8922254 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 829245 817184 609 1459 0 827777 0
Current children cumulated CPU time (s) 582.2
Current children cumulated vsize (KiB) 3316980

[startup+642.3 s]

################
# More data... #
################

/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 838671 0 0 0 100042 172 0 0 20 0 1 0 270848820 3405033472 820657 33554432000 4194304 10169799 140735078376448 140735078372712 8922285 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 831307 820657 609 1459 0 829839 0
Current children cumulated CPU time (s) 1002.14
Current children cumulated vsize (KiB) 3325228

[startup+1062.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 2223
/proc/meminfo: memFree=22617376/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=3326252 CPUtime=1062.13 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 839046 0 0 0 106041 172 0 0 20 0 1 0 270848820 3406082048 821032 33554432000 4194304 10169799 140735078376448 140735078372712 8922280 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 831563 821032 609 1459 0 830095 0
Current children cumulated CPU time (s) 1062.13
Current children cumulated vsize (KiB) 3326252

[startup+1122.3 s]
/proc/loadavg: 2.03 2.01 2.00 3/177 2224
/proc/meminfo: memFree=22612760/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=3328108 CPUtime=1122.12 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 839657 0 0 0 112040 172 0 0 20 0 1 0 270848820 3407982592 821643 33554432000 4194304 10169799 140735078376448 140735078372712 8922285 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 832027 821643 609 1459 0 830559 0
Current children cumulated CPU time (s) 1122.12
Current children cumulated vsize (KiB) 3328108

[startup+1182.3 s]
/proc/loadavg: 2.01 2.01 2.00 3/177 2224
/proc/meminfo: memFree=22610356/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=3328108 CPUtime=1182.11 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 839847 0 0 0 118039 172 0 0 20 0 1 0 270848820 3407982592 821833 33554432000 4194304 10169799 140735078376448 140735078372712 8922310 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 832027 821833 609 1459 0 830559 0
Current children cumulated CPU time (s) 1182.11
Current children cumulated vsize (KiB) 3328108

[startup+1242.3 s]
/proc/loadavg: 2.07 2.02 2.00 3/177 2224
/proc/meminfo: memFree=22607104/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=3330156 CPUtime=1242.1 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 840481 0 0 0 124037 173 0 0 20 0 1 0 270848820 3410079744 822467 33554432000 4194304 10169799 140735078376448 140735078372712 8922316 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 832539 822467 609 1459 0 831071 0
Current children cumulated CPU time (s) 1242.1
Current children cumulated vsize (KiB) 3330156

[startup+1302.3 s]
/proc/loadavg: 2.02 2.01 2.00 3/177 2224
/proc/meminfo: memFree=22604880/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=3330156 CPUtime=1302.1 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 840625 0 0 0 130037 173 0 0 20 0 1 0 270848820 3410079744 822611 33554432000 4194304 10169799 140735078376448 140735078372712 8922295 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 832539 822611 609 1459 0 831071 0
Current children cumulated CPU time (s) 1302.1
Current children cumulated vsize (KiB) 3330156

[startup+1362.3 s]
/proc/loadavg: 2.01 2.00 2.00 3/177 2224
/proc/meminfo: memFree=22598612/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=3331180 CPUtime=1362.09 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 841071 0 0 0 136035 174 0 0 20 0 1 0 270848820 3411128320 823057 33554432000 4194304 10169799 140735078376448 140735078372712 8922280 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 832795 823057 609 1459 0 831327 0
Current children cumulated CPU time (s) 1362.09
Current children cumulated vsize (KiB) 3331180

[startup+1422.31 s]
/proc/loadavg: 2.00 2.00 2.00 3/178 2228
/proc/meminfo: memFree=22594912/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=3333036 CPUtime=1422.08 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 841657 0 0 0 142034 174 0 0 20 0 1 0 270848820 3413028864 823643 33554432000 4194304 10169799 140735078376448 140735078372712 8922290 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 833259 823643 609 1459 0 831791 0
Current children cumulated CPU time (s) 1422.08
Current children cumulated vsize (KiB) 3333036

[startup+1482.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 2228
/proc/meminfo: memFree=22593924/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=3333036 CPUtime=1482.07 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 841807 0 0 0 148033 174 0 0 20 0 1 0 270848820 3413028864 823793 33554432000 4194304 10169799 140735078376448 140735078372712 8922316 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 833259 823793 609 1459 0 831791 0
Current children cumulated CPU time (s) 1482.07
Current children cumulated vsize (KiB) 3333036

[startup+1542.3 s]
/proc/loadavg: 2.00 2.00 2.00 3/177 2228
/proc/meminfo: memFree=22590108/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=3333928 CPUtime=1542.06 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 842262 0 0 0 154032 174 0 0 20 0 1 0 270848820 3413942272 824215 33554432000 4194304 10169799 140735078376448 140735078372712 8922310 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 833482 824215 609 1459 0 832014 0
Current children cumulated CPU time (s) 1542.06
Current children cumulated vsize (KiB) 3333928

[startup+1602.3 s]
/proc/loadavg: 2.03 2.01 2.00 3/177 2228
/proc/meminfo: memFree=22587968/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=3334952 CPUtime=1602.05 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 842647 0 0 0 160031 174 0 0 20 0 1 0 270848820 3414990848 824600 33554432000 4194304 10169799 140735078376448 140735078372712 8922310 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 833738 824600 609 1459 0 832270 0
Current children cumulated CPU time (s) 1602.05
Current children cumulated vsize (KiB) 3334952

[startup+1662.3 s]
/proc/loadavg: 2.01 2.01 2.00 3/177 2228
/proc/meminfo: memFree=22586516/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=3335976 CPUtime=1662.03 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 843055 0 0 0 166029 174 0 0 20 0 1 0 270848820 3416039424 825008 33554432000 4194304 10169799 140735078376448 140735078372712 8922280 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 833994 825008 609 1459 0 832526 0
Current children cumulated CPU time (s) 1662.03
Current children cumulated vsize (KiB) 3335976

[startup+1722.3 s]
/proc/loadavg: 1.72 1.94 1.98 2/171 2254
/proc/meminfo: memFree=25959316/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=3336808 CPUtime=1722.03 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 843463 0 0 0 172028 175 0 0 20 0 1 0 270848820 3416891392 825454 33554432000 4194304 10169799 140735078376448 140735078372712 8922280 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 834202 825454 609 1459 0 832734 0
Current children cumulated CPU time (s) 1722.03
Current children cumulated vsize (KiB) 3336808

[startup+1782.3 s]
/proc/loadavg: 1.26 1.76 1.91 2/171 2254
/proc/meminfo: memFree=25957904/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=3338856 CPUtime=1782.02 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 844216 0 0 0 178027 175 0 0 20 0 1 0 270848820 3418988544 826206 33554432000 4194304 10169799 140735078376448 140735078372712 8922305 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 834714 826206 609 1459 0 833246 0
Current children cumulated CPU time (s) 1782.02
Current children cumulated vsize (KiB) 3338856



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1800.3 s]
/proc/loadavg: 1.19 1.71 1.89 2/171 2254
/proc/meminfo: memFree=25957748/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=3338856 CPUtime=1800.02 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 844258 0 0 0 179827 175 0 0 20 0 1 0 270848820 3418988544 826248 33554432000 4194304 10169799 140735078376448 140735078372712 8922254 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 834714 826248 609 1459 0 833246 0
Current children cumulated CPU time (s) 1800.02
Current children cumulated vsize (KiB) 3338856

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)

Sending SIGKILL to -2190

Solver just ended. Dumping a history of the last processes samples

[startup+1801.4 s]
/proc/loadavg: 1.19 1.71 1.89 2/172 2255
/proc/meminfo: memFree=25957740/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=3338856 CPUtime=1801.12 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 844258 0 0 0 179937 175 0 0 20 0 1 0 270848820 3418988544 826248 33554432000 4194304 10169799 140735078376448 140735078372160 8922290 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 834714 826248 609 1459 0 833246 0
Current children cumulated CPU time (s) 1801.12
Current children cumulated vsize (KiB) 3338856

[startup+1802.2 s]
/proc/loadavg: 1.19 1.71 1.89 2/172 2255
/proc/meminfo: memFree=25957740/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=3338856 CPUtime=1801.92 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4202496 844264 0 0 0 180017 175 0 0 20 0 1 0 270848820 3418988544 826254 33554432000 4194304 10169799 140735078376448 140735078372160 8922290 0 0 4096 16384 0 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 834714 826254 609 1459 0 833246 0
Current children cumulated CPU time (s) 1801.92
Current children cumulated vsize (KiB) 3338856

[startup+1802.4 s]
/proc/loadavg: 1.19 1.71 1.89 2/172 2255
/proc/meminfo: memFree=25957740/32873648 swapFree=6844/6844
[pid=2190] ppid=2188 vsize=0 CPUtime=1802.11 cores=0,2,4,6
/proc/2190/stat : 2190 (scip-2.1.1.4sfi) R 2188 2190 1406 0 -1 4203524 844264 0 0 0 180027 184 0 0 20 0 1 0 270848820 0 0 33554432000 0 0 0 0 0 0 0 4096 16384 18446744073709551615 0 0 17 4 0 0 0 0 0
/proc/2190/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 1802.11
Current children cumulated vsize (KiB) 0

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1802.47
CPU time (s): 1802.2
CPU user time (s): 1800.28
CPU system time (s): 1.91871
CPU usage (%): 99.9846
Max. virtual memory (cumulated for all children) (KiB): 3338856

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1800.28
system time used= 1.91871
maximum resident set size= 3305016
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 844264
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 5
involuntary context switches= 1916

runsolver used 3.12252 second user time and 8.88865 second system time

The end

Launcher Data 

Begin job on node124 at 2012-05-26 17:47:09
IDJOB=3693717
IDBENCH=90934
IDSOLVER=2272
FILE ID=node124/3693717-1338047229
RUNJOBID= node124-1338036340-1423
PBS_JOBID= 14624372
Free space on /tmp= 71476 MiB

SOLVER NAME= SCIP spx standard SCIP 2.1.1.4. with SoPlex 1.6.0.3 standard fixed
BENCH NAME= PB12/normalized-PB12/OPT-SMALLINT-LIN/sroussel/NG/normalized-NG.dot_luby_50null20mast_rapportOE2.0_TMAX60_K192_cod2.opb
COMMAND LINE= HOME/scip-2.1.1.4sfix.linux.x86_64.gnu.opt.spx -f BENCHNAME -t TIMEOUT -m MEMLIMIT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3693717-1338047229/watcher-3693717-1338047229 -o /tmp/evaluation-result-3693717-1338047229/solver-3693717-1338047229 -C 1800 -W 1900 -M 15500  HOME/scip-2.1.1.4sfix.linux.x86_64.gnu.opt.spx -f HOME/instance-3693717-1338047229.opb -t 1800 -m 15500

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 6f3eb6421078a0de6a5b5f585e0aca61
RANDOM SEED=1543799151

node124.alineos.net Linux 2.6.32-71.29.1.el6.x86_64 #1 SMP Mon Jun 27 19:49:27 BST 2011

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.325
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.325
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5385.03
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.325
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5385.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.325
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5357.53
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.325
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5385.03
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.325
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5385.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.325
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5385.03
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2667.325
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5385.03
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32873648 kB
MemFree:        27291244 kB
Buffers:          273264 kB
Cached:          2580816 kB
SwapCached:            0 kB
Active:          2718200 kB
Inactive:        2080916 kB
Active(anon):    1947508 kB
Inactive(anon):        0 kB
Active(file):     770692 kB
Inactive(file):  2080916 kB
Unevictable:        6844 kB
Mlocked:            6844 kB
SwapTotal:      67108856 kB
SwapFree:       67108856 kB
Dirty:             41024 kB
Writeback:             0 kB
AnonPages:       1954056 kB
Mapped:            16792 kB
Shmem:               200 kB
Slab:             641084 kB
SReclaimable:      85428 kB
SUnreclaim:       555656 kB
KernelStack:        1448 kB
PageTables:         8284 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83545680 kB
Committed_AS:    2071636 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      346400 kB
VmallocChunk:   34341988052 kB
HardwareCorrupted:     0 kB
AnonHugePages:    108544 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:        7680 kB
DirectMap2M:    33538048 kB

Free space on /tmp at the end= 71504 MiB
End job on node124 at 2012-05-26 18:17:12