Trace number 3482892

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg pb-dec-11.04.03? (MO) 1261.04 1260.97

General information on the benchmark

Name/DEC-SMALLINT-LIN/lopes/
normalized-182.opb
MD5SUM378c188d79de101f4114d358aff291a2
Bench CategoryDEC-SMALLINT-LIN (no optimisation, small integers, linear constraints)
Best result obtained on this benchmark
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Has Objective FunctionNO
Satisfiable
(Un)Satisfiability was proved
Best value of the objective function
Optimality of the best value was proved
Number of variables599527
Total number of constraints14097016
Number of constraints which are clauses12663693
Number of constraints which are cardinality constraints (but not clauses)44152
Number of constraints which are nor clauses,nor cardinality constraints1389171
Minimum length of a constraint2
Maximum length of a constraint3822
Number of terms in the objective function 0
Biggest coefficient in the objective function 0
Number of bits for the biggest coefficient in the objective function 0
Sum of the numbers in the objective function 0
Number of bits of the sum of numbers in the objective function 0
Biggest number in a constraint 3
Number of bits of the biggest number in a constraint 2
Biggest sum of numbers in a constraint 3823
Number of bits of the biggest sum of numbers12
Number of products (including duplicates)0
Sum of products size (including duplicates)0
Number of different products0
Sum of products size0

Solver Data 

0.31/0.48	c INFO: loaded portfolio from HOME/etc/borg-mix+class.dec.pickle
0.31/0.48	c INFO: solving HOME/instance-3482892-1307578358.opb
99.69/99.86	c INFO: parsing took 97.45 s
279.87/280.04	c INFO: features timed out after 179.71 CPU seconds
279.87/280.06	c INFO: running clasp-1.3.7@0 for 45 with 1522 remaining (b = 0.71)
328.41/328.43	c INFO: running scip-2.0.1-spx@0 for 45 with 1475 remaining (b = 0.38)
1260.33/1260.19	c INFO: wrote minion input file to HOME/tmpaqhpBV.minion
1260.33/1260.19	c INFO: running minion-0.12@0 for 45 with 1428 remaining (b = 0.11)

Verifier Data 

ERROR: no interpretation found !

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3482892-1307578358/watcher-3482892-1307578358 -o /tmp/evaluation-result-3482892-1307578358/solver-3482892-1307578358 -C 1800 -W 1900 -M 15500 python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3482892-1307578358.opb -seed 2097467692 -budget 1800 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1900 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.58 2.20 1.85 2/600 1853
/proc/meminfo: memFree=24602396/32951124 swapFree=58212620/67111528
[pid=1853] ppid=1851 vsize=15184 CPUtime=0 cores=1,3,5,7
/proc/1853/stat : 1853 (python) R 1851 1853 758 0 -1 4202496 734 0 0 0 0 0 0 0 23 0 1 0 869435672 15548416 622 33554432000 4194304 5683644 140736202612192 18446744073709551615 228244346880 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/1853/statm: 3796 622 322 364 0 283 0

[startup+0.116215 s]
/proc/loadavg: 2.58 2.20 1.85 2/600 1853
/proc/meminfo: memFree=24602396/32951124 swapFree=58212620/67111528
[pid=1853] ppid=1851 vsize=96128 CPUtime=0.09 cores=1,3,5,7
/proc/1853/stat : 1853 (python) S 1851 1853 758 0 -1 4202496 3246 928 1 0 7 2 0 0 18 0 1 0 869435672 98435072 2484 33554432000 4194304 5683644 140736202612192 18446744073709551615 228244346880 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 0
/proc/1853/statm: 24032 2484 792 364 0 1627 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 96128

[startup+0.200726 s]
/proc/loadavg: 2.58 2.20 1.85 2/600 1853
/proc/meminfo: memFree=24602396/32951124 swapFree=58212620/67111528
[pid=1853] ppid=1851 vsize=123744 CPUtime=0.14 cores=1,3,5,7
/proc/1853/stat : 1853 (python) S 1851 1853 758 0 -1 4202496 5357 928 1 0 11 3 0 0 17 0 1 0 869435672 126713856 3744 33554432000 4194304 5683644 140736202612192 18446744073709551615 228244346880 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 0
/proc/1853/statm: 30936 3744 987 364 0 2768 0
Current children cumulated CPU time (s) 0.14
Current children cumulated vsize (KiB) 123744

[startup+0.301526 s]
/proc/loadavg: 2.58 2.20 1.85 2/600 1853
/proc/meminfo: memFree=24602396/32951124 swapFree=58212620/67111528
[pid=1853] ppid=1851 vsize=200780 CPUtime=0.23 cores=1,3,5,7
/proc/1853/stat : 1853 (python) S 1851 1853 758 0 -1 4202496 6648 928 1 0 17 6 0 0 17 0 1 0 869435672 205598720 4936 33554432000 4194304 5683644 140736202612192 18446744073709551615 228244346880 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 0
/proc/1853/statm: 50195 4936 1465 364 0 3677 0
Current children cumulated CPU time (s) 0.23
Current children cumulated vsize (KiB) 200780

[startup+0.700361 s]
/proc/loadavg: 2.58 2.20 1.85 2/600 1853
/proc/meminfo: memFree=24602396/32951124 swapFree=58212620/67111528
[pid=1853] ppid=1851 vsize=962904 CPUtime=0.59 cores=1,3,5,7
/proc/1853/stat : 1853 (python) R 1851 1853 758 0 -1 4202496 127153 928 1 0 29 30 0 0 17 0 1 0 869435672 986013696 125015 33554432000 4194304 5683644 140736202612192 18446744073709551615 228244348672 0 0 16781312 2 0 0 0 17 7 0 0 0
/proc/1853/statm: 240726 125015 1776 364 0 175619 0
Current children cumulated CPU time (s) 0.59
Current children cumulated vsize (KiB) 962904

[startup+1.50025 s]
/proc/loadavg: 2.58 2.20 1.85 3/601 1858
/proc/meminfo: memFree=23887108/32951124 swapFree=58212620/67111528
[pid=1853] ppid=1851 vsize=986792 CPUtime=1.39 cores=1,3,5,7
/proc/1853/stat : 1853 (python) R 1851 1853 758 0 -1 4202496 185059 928 1 0 99 40 0 0 18 0 1 0 869435672 1010475008 182921 33554432000 4194304 5683644 140736202612192 18446744073709551615 5114678 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/1853/statm: 246698 182922 1781 364 0 181591 0
Current children cumulated CPU time (s) 1.39
Current children cumulated vsize (KiB) 986792

[startup+3.10097 s]
/proc/loadavg: 2.58 2.20 1.85 3/601 1858
/proc/meminfo: memFree=23847428/32951124 swapFree=58212620/67111528
[pid=1853] ppid=1851 vsize=1042008 CPUtime=2.99 cores=1,3,5,7
/proc/1853/stat : 1853 (python) R 1851 1853 758 0 -1 4202496 198667 928 1 0 257 42 0 0 22 0 1 0 869435672 1067016192 196529 33554432000 4194304 5683644 140736202612192 18446744073709551615 4603074 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/1853/statm: 260502 196529 1781 364 0 195395 0
Current children cumulated CPU time (s) 2.99
Current children cumulated vsize (KiB) 1042008

[startup+6.3004 s]
/proc/loadavg: 2.70 2.23 1.86 3/601 1858
/proc/meminfo: memFree=23727328/32951124 swapFree=58212620/67111528
[pid=1853] ppid=1851 vsize=1155912 CPUtime=6.19 cores=1,3,5,7
/proc/1853/stat : 1853 (python) R 1851 1853 758 0 -1 4202496 225864 928 1 0 573 46 0 0 25 0 1 0 869435672 1183653888 223726 33554432000 4194304 5683644 140736202612192 18446744073709551615 4602349 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/1853/statm: 288978 223727 1781 364 0 223871 0
Current children cumulated CPU time (s) 6.19
Current children cumulated vsize (KiB) 1155912

[startup+12.7003 s]
/proc/loadavg: 2.64 2.22 1.86 3/601 1858
/proc/meminfo: memFree=23495828/32951124 swapFree=58212620/67111528
[pid=1853] ppid=1851 vsize=1375716 CPUtime=12.6 cores=1,3,5,7
/proc/1853/stat : 1853 (python) R 1851 1853 758 0 -1 4202496 280391 928 1 0 1206 54 0 0 25 0 1 0 869435672 1408733184 278253 33554432000 4194304 5683644 140736202612192 18446744073709551615 4445115 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/1853/statm: 343929 278253 1781 364 0 278822 0
Current children cumulated CPU time (s) 12.6
Current children cumulated vsize (KiB) 1375716

[startup+25.501 s]
/proc/loadavg: 2.50 2.21 1.86 3/605 1862
/proc/meminfo: memFree=22756132/32951124 swapFree=58212620/67111528
[pid=1853] ppid=1851 vsize=1810912 CPUtime=25.4 cores=1,3,5,7
/proc/1853/stat : 1853 (python) R 1851 1853 758 0 -1 4202496 388868 928 1 0 2470 70 0 0 25 0 1 0 869435672 1854373888 386730 33554432000 4194304 5683644 140736202612192 18446744073709551615 228244009669 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/1853/statm: 452728 386730 1781 364 0 387621 0
Current children cumulated CPU time (s) 25.4
Current children cumulated vsize (KiB) 1810912

[startup+51.1005 s]
/proc/loadavg: 2.39 2.21 1.87 3/605 1863
/proc/meminfo: memFree=21838248/32951124 swapFree=58212620/67111528
[pid=1853] ppid=1851 vsize=2685744 CPUtime=50.99 cores=1,3,5,7
/proc/1853/stat : 1853 (python) R 1851 1853 758 0 -1 4202496 605630 928 1 0 4998 101 0 0 25 0 1 0 869435672 2750201856 603492 33554432000 4194304 5683644 140736202612192 18446744073709551615 4445115 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/1853/statm: 671436 603492 1781 364 0 606329 0
Current children cumulated CPU time (s) 50.99
Current children cumulated vsize (KiB) 2685744

[startup+102.301 s]
/proc/loadavg: 2.17 2.17 1.87 3/606 1865
/proc/meminfo: memFree=20414392/32951124 swapFree=58212620/67111528
[pid=1853] ppid=1851 vsize=4017532 CPUtime=102.19 cores=1,3,5,7
/proc/1853/stat : 1853 (python) R 1851 1853 758 0 -1 4202496 1677432 928 1 0 9979 240 0 0 25 0 1 0 869435672 4113952768 940730 33554432000 4194304 5683644 140736202612192 18446744073709551615 4662302 0 0 16781312 33554434 0 0 0 17 5 0 0 0
/proc/1853/statm: 1004383 940730 1815 364 0 939276 0
Current children cumulated CPU time (s) 102.19
Current children cumulated vsize (KiB) 4017532

[startup+162.301 s]
/proc/loadavg: 2.06 2.14 1.88 3/609 1870
/proc/meminfo: memFree=20349868/32951124 swapFree=58212620/67111528
[pid=1853] ppid=1851 vsize=4017532 CPUtime=162.19 cores=1,3,5,7
/proc/1853/stat : 1853 (python) R 1851 1853 758 0 -1 4202496 1677432 928 1 0 15979 240 0 0 25 0 1 0 869435672 4113952768 940730 33554432000 4194304 5683644 140736202612192 18446744073709551615 47549766156079 0 0 16781312 33554434 0 0 0 17 5 0 0 0
/proc/1853/statm: 1004383 940730 1815 364 0 939276 0
Current children cumulated CPU time (s) 162.19
Current children cumulated vsize (KiB) 4017532

[startup+222.3 s]
/proc/loadavg: 2.18 2.15 1.90 3/610 1872
/proc/meminfo: memFree=20225088/32951124 swapFree=58212620/67111528
[pid=1853] ppid=1851 vsize=4127668 CPUtime=222.18 cores=1,3,5,7
/proc/1853/stat : 1853 (python) R 1851 1853 758 0 -1 4202496 1704967 928 1 0 21976 242 0 0 25 0 1 0 869435672 4226732032 968265 33554432000 4194304 5683644 140736202612192 18446744073709551615 228247757493 0 0 16781312 33554434 0 0 0 17 5 0 0 0
/proc/1853/statm: 1031917 968265 1816 364 0 966810 0
Current children cumulated CPU time (s) 222.18
Current children cumulated vsize (KiB) 4127668

[startup+282.301 s]
/proc/loadavg: 2.14 2.13 1.91 5/613 1877
/proc/meminfo: memFree=21124420/32951124 swapFree=58212620/67111528
[pid=1853] ppid=1851 vsize=3075144 CPUtime=279.98 cores=1,3,5,7
/proc/1853/stat : 1853 (python) S 1851 1853 758 0 -1 4202496 1705399 928 1 0 27746 252 0 0 25 0 2 0 869435672 3148947456 702619 33554432000 4194304 5683644 140736202612192 18446744073709551615 228256175563 0 0 16781312 33554434 18446744073709551615 0 0 17 5 0 0 0
/proc/1853/statm: 768786 702619 1848 364 0 703673 0
[pid=1875] ppid=1853 vsize=3065988 CPUtime=0.04 cores=1,3,5,7
/proc/1875/stat : 1875 (python) S 1853 1853 758 0 -1 4202560 726 0 0 0 0 4 0 0 18 0 1 0 869463679 3139571712 701121 33554432000 4194304 5683644 140736202612192 18446744073709551615 228244368911 0 0 16781312 33554946 13004593388064833797 0 0 17 5 0 0 0
/proc/1875/statm: 766497 701121 353 364 0 701384 0
[pid=1877] ppid=1875 vsize=198468 CPUtime=2.12 cores=1,3,5,7
/proc/1877/stat : 1877 (clasp-1.3.7-x86) R 1875 1877 1877 0 -1 4202496 55094 0 0 0 198 14 0 0 25 0 1 0 869463683 203231232 47649 33554432000 134512640 136139019 4290441664 18446744073709551615 135529895 0 0 16781312 16386 0 0 0 17 5 0 0 0
/proc/1877/statm: 49617 47649 164 398 0 49217 0
[pid=1853/tid=1876] ppid=1851 vsize=3075144 CPUtime=0 cores=1,3,5,7
/proc/1853/task/1876/stat : 1876 (python) S 1851 1853 758 0 -1 4202560 27 928 0 0 0 0 0 0 25 0 2 0 869463683 3148947456 702619 33554432000 4194304 5683644 140736202612192 18446744073709551615 228256172465 0 0 16781312 33554434 18446604457588720832 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 282.14
Current children cumulated vsize (KiB) 6339600

[startup+342.301 s]
/proc/loadavg: 2.05 2.11 1.91 3/638 1904
/proc/meminfo: memFree=19018712/32951124 swapFree=58212620/67111528
[pid=1853] ppid=1851 vsize=3085968 CPUtime=280.05 cores=1,3,5,7
/proc/1853/stat : 1853 (python) S 1851 1853 758 0 -1 4202496 1706291 928 1 0 27748 257 0 0 20 0 3 0 869435672 3160031232 702752 33554432000 4194304 5683644 140736202612192 18446744073709551615 228256175563 0 0 16781312 33554434 18446744073709551615 0 0 17 7 0 0 0
/proc/1853/statm: 771492 702752 1851 364 0 706376 0
[pid=1875] ppid=1853 vsize=3076232 CPUtime=0.26 cores=1,3,5,7
/proc/1875/stat : 1875 (python) S 1853 1853 758 0 -1 4202560 1159 0 0 0 10 16 0 0 19 0 2 0 869463679 3150061568 701185 33554432000 4194304 5683644 140736202612192 18446744073709551615 228256175563 0 0 16781312 33554946 18446744073709551615 0 0 17 5 0 0 0
/proc/1875/statm: 769058 701185 412 364 0 703945 0
[pid=1877] ppid=1875 vsize=1164868 CPUtime=48.2 cores=1,3,5,7
/proc/1877/stat : 1877 (clasp-1.3.7-x86) T 1875 1877 1877 0 -1 4202496 408297 0 0 0 4744 76 0 0 25 0 1 0 869463683 1192824832 287505 33554432000 134512640 136139019 4290441664 18446744073709551615 134640592 0 0 16781312 16386 18446744071562691360 0 0 17 7 0 0 0
/proc/1877/statm: 291217 287505 184 398 0 290817 0
[pid=1875/tid=1900] ppid=1853 vsize=3076232 CPUtime=0 cores=1,3,5,7
/proc/1875/task/1900/stat : 1900 (python) S 1853 1853 758 0 -1 4202560 43 0 0 0 0 0 0 0 19 0 2 0 869468514 3150061568 701185 33554432000 4194304 5683644 140736202612192 18446744073709551615 228256172465 0 0 16781312 33554946 18446604459085539328 0 0 -1 7 0 0 0
[pid=1901] ppid=1853 vsize=3076120 CPUtime=0.11 cores=1,3,5,7
/proc/1901/stat : 1901 (python) S 1853 1853 758 0 -1 4202560 923 0 0 0 3 8 0 0 18 0 1 0 869468516 3149946880 701272 33554432000 4194304 5683644 140736202612192 18446744073709551615 228244368966 0 0 16781312 33554946 5623753486446373695 0 0 17 7 0 0 0
/proc/1901/statm: 769030 701272 374 364 0 703914 0
[pid=1903] ppid=1901 vsize=779364 CPUtime=13.77 cores=1,3,5,7
/proc/1903/stat : 1903 (scip-2.0.1.linu) R 1901 1903 1903 0 -1 4202496 192512 0 0 0 1347 30 0 0 25 0 1 0 869468520 798068736 192235 33554432000 4194304 13684479 140733552664224 18446744073709551615 6733904 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/1903/statm: 194841 192235 510 2317 0 192519 0
[pid=1853/tid=1876] ppid=1851 vsize=3085968 CPUtime=0 cores=1,3,5,7
/proc/1853/task/1876/stat : 1876 (python) S 1851 1853 758 0 -1 4202560 27 928 0 0 0 0 0 0 25 0 3 0 869463683 3160031232 702752 33554432000 4194304 5683644 140736202612192 18446744073709551615 228256172465 0 0 16781312 33554434 18446604457588720832 0 0 -1 5 0 0 0
[pid=1853/tid=1902] ppid=1851 vsize=3085968 CPUtime=0 cores=1,3,5,7
/proc/1853/task/1902/stat : 1902 (python) S 1851 1853 758 0 -1 4202560 25 928 0 0 0 0 0 0 20 0 3 0 869468520 3160031232 702752 33554432000 4194304 5683644 140736202612192 18446744073709551615 228256172465 0 0 16781312 33554434 18446604457588720832 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 342.39
Current children cumulated vsize (KiB) 11182552

[startup+402.301 s]
/proc/loadavg: 2.02 2.08 1.91 3/622 1964
/proc/meminfo: memFree=17147664/32951124 swapFree=58212620/67111528
[pid=1853] ppid=1851 vsize=3085972 CPUtime=306.54 cores=1,3,5,7
/proc/1853/stat : 1853 (python) R 1851 1853 758 0 -1 4202496 1706857 928 1 0 30392 262 0 0 25 0 3 0 869435672 3160035328 702761 33554432000 4194304 5683644 140736202612192 18446744073709551615 4509251 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0
/proc/1853/statm: 771493 702761 1859 364 0 706377 0
[pid=1875] ppid=1853 vsize=3076232 CPUtime=0.26 cores=1,3,5,7
/proc/1875/stat : 1875 (python) S 1853 1853 758 0 -1 4202560 1159 0 0 0 10 16 0 0 19 0 2 0 869463679 3150061568 701185 33554432000 4194304 5683644 140736202612192 18446744073709551615 228256175563 0 0 16781312 33554946 18446744073709551615 0 0 17 5 0 0 0
/proc/1875/statm: 769058 701185 412 364 0 703945 0
[pid=1877] ppid=1875 vsize=1164868 CPUtime=48.2 cores=1,3,5,7
/proc/1877/stat : 1877 (clasp-1.3.7-x86) T 1875 1877 1877 0 -1 4202496 408297 0 0 0 4744 76 0 0 25 0 1 0 869463683 1192824832 287505 33554432000 134512640 136139019 4290441664 18446744073709551615 134640592 0 0 16781312 16386 18446744071562691360 0 0 17 7 0 0 0

################
# More data... #
################

Current children cumulated CPU time (s) 1242.53
Current children cumulated vsize (KiB) 12984492



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+1260.8 s]
/proc/loadavg: 1.00 1.22 1.55 3/640 2070
/proc/meminfo: memFree=15734296/32951124 swapFree=58212620/67111528
[pid=1853] ppid=1851 vsize=3096224 CPUtime=1164.46 cores=1,3,5,7
/proc/1853/stat : 1853 (python) S 1851 1853 758 0 -1 4202496 1707248 928 1 0 115971 475 0 0 25 0 4 0 869435672 3170533376 702766 33554432000 4194304 5683644 140736202612192 18446744073709551615 228256175563 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0
/proc/1853/statm: 774056 702766 1861 364 0 708937 0
[pid=1875] ppid=1853 vsize=3076232 CPUtime=0.26 cores=1,3,5,7
/proc/1875/stat : 1875 (python) S 1853 1853 758 0 -1 4202560 1159 0 0 0 10 16 0 0 19 0 2 0 869463679 3150061568 701185 33554432000 4194304 5683644 140736202612192 18446744073709551615 228256175563 0 0 16781312 33554946 18446744073709551615 0 0 17 5 0 0 0
/proc/1875/statm: 769058 701185 412 364 0 703945 0
[pid=1877] ppid=1875 vsize=1164868 CPUtime=48.2 cores=1,3,5,7
/proc/1877/stat : 1877 (clasp-1.3.7-x86) T 1875 1877 1877 0 -1 4202496 408297 0 0 0 4744 76 0 0 25 0 1 0 869463683 1192824832 287505 33554432000 134512640 136139019 4290441664 18446744073709551615 134640592 0 0 16781312 16386 18446744071562691360 0 0 17 7 0 0 0
/proc/1877/statm: 291217 287505 184 398 0 290817 0
[pid=1875/tid=1900] ppid=1853 vsize=3076232 CPUtime=0 cores=1,3,5,7
/proc/1875/task/1900/stat : 1900 (python) S 1853 1853 758 0 -1 4202560 43 0 0 0 0 0 0 0 19 0 2 0 869468514 3150061568 701185 33554432000 4194304 5683644 140736202612192 18446744073709551615 228256172465 0 0 16781312 33554946 18446604459085539328 0 0 -1 7 0 0 0
[pid=1901] ppid=1853 vsize=3076120 CPUtime=0.27 cores=1,3,5,7
/proc/1901/stat : 1901 (python) S 1853 1853 758 0 -1 4202560 1109 0 0 0 11 16 0 0 19 0 2 0 869468516 3149946880 701296 33554432000 4194304 5683644 140736202612192 18446744073709551615 228256175563 0 0 16781312 33554946 18446744073709551615 0 0 17 7 0 0 0
/proc/1901/statm: 769030 701296 397 364 0 703914 0
[pid=1903] ppid=1901 vsize=2581300 CPUtime=47.28 cores=1,3,5,7
/proc/1903/stat : 1903 (scip-2.0.1.linu) T 1901 1903 1903 0 -1 4202496 641376 0 0 0 4642 86 0 0 25 0 1 0 869468520 2643251200 641099 33554432000 4194304 13684479 140733552664224 18446744073709551615 11824617 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/1903/statm: 645325 641099 510 2317 0 643003 0
[pid=1901/tid=1907] ppid=1853 vsize=3076120 CPUtime=0 cores=1,3,5,7
/proc/1901/task/1907/stat : 1907 (python) S 1853 1853 758 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 869473253 3149946880 701296 33554432000 4194304 5683644 140736202612192 18446744073709551615 228256172465 0 0 16781312 33554946 18446604455339136128 0 0 -1 1 0 0 0
[pid=2068] ppid=1853 vsize=3086308 CPUtime=0.04 cores=1,3,5,7
/proc/2068/stat : 2068 (python) S 1853 1853 758 0 -1 4202560 761 0 0 0 0 4 0 0 25 0 1 0 869561691 3160379392 701252 33554432000 4194304 5683644 140736202612192 18446744073709551615 228244368966 0 0 16781312 33554946 4527234836695318789 0 0 17 1 0 0 0
/proc/2068/statm: 771577 701252 350 364 0 706458 0
[pid=2070] ppid=2068 vsize=554920 CPUtime=0.53 cores=1,3,5,7
/proc/2070/stat : 2070 (minion) R 2068 2070 2070 0 -1 4202496 218168 0 0 0 11 42 0 0 25 0 1 0 869561696 568238080 86864 33554432000 134512640 165565024 4294526400 18446744073709551615 159737071 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/2070/statm: 138730 86870 226 7582 0 131145 0
[pid=1853/tid=1876] ppid=1851 vsize=3096224 CPUtime=0 cores=1,3,5,7
/proc/1853/task/1876/stat : 1876 (python) S 1851 1853 758 0 -1 4202560 27 928 0 0 0 0 0 0 25 0 4 0 869463683 3170533376 702766 33554432000 4194304 5683644 140736202612192 18446744073709551615 228256172465 0 0 16781312 33554434 18446604457588720832 0 0 -1 5 0 0 0
[pid=1853/tid=1902] ppid=1851 vsize=3096224 CPUtime=0 cores=1,3,5,7
/proc/1853/task/1902/stat : 1902 (python) S 1851 1853 758 0 -1 4202560 25 928 0 0 0 0 0 0 20 0 4 0 869468520 3170533376 702766 33554432000 4194304 5683644 140736202612192 18446744073709551615 228256172465 0 0 16781312 33554434 18446604457588720832 0 0 -1 5 0 0 0
[pid=1853/tid=2069] ppid=1851 vsize=3096224 CPUtime=0 cores=1,3,5,7
/proc/1853/task/2069/stat : 2069 (python) S 1851 1853 758 0 -1 4202560 31 928 0 0 0 0 0 0 25 0 4 0 869561696 3170533376 702766 33554432000 4194304 5683644 140736202612192 18446744073709551615 228256172465 0 0 16781312 33554434 18446604457588720832 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1261.04
Current children cumulated vsize (KiB) 16635972

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

############
# WARNING:
# current cumulated CPU time (1260.72 s) is less than in the last sample (1261.04 s)
# The time of a child was probably not reported to its father.
# (see the two samples below)
# Adding the difference (0.320068 s) to the 'lost time'.

[startup+1260.8 s]
/proc/loadavg: 1.00 1.22 1.55 3/640 2070
/proc/meminfo: memFree=15734296/32951124 swapFree=58212620/67111528
[pid=1853] ppid=1851 vsize=3096224 CPUtime=1164.46 cores=1,3,5,7
/proc/1853/stat : 1853 (python) S 1851 1853 758 0 -1 4202496 1707248 928 1 0 115971 475 0 0 25 0 4 0 869435672 3170533376 702766 33554432000 4194304 5683644 140736202612192 18446744073709551615 228256175563 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0
/proc/1853/statm: 774056 702766 1861 364 0 708937 0
[pid=1875] ppid=1853 vsize=3076232 CPUtime=0.26 cores=1,3,5,7
/proc/1875/stat : 1875 (python) S 1853 1853 758 0 -1 4202560 1159 0 0 0 10 16 0 0 19 0 2 0 869463679 3150061568 701185 33554432000 4194304 5683644 140736202612192 18446744073709551615 228256175563 0 0 16781312 33554946 18446744073709551615 0 0 17 5 0 0 0
/proc/1875/statm: 769058 701185 412 364 0 703945 0
[pid=1877] ppid=1875 vsize=1164868 CPUtime=48.2 cores=1,3,5,7
/proc/1877/stat : 1877 (clasp-1.3.7-x86) T 1875 1877 1877 0 -1 4202496 408297 0 0 0 4744 76 0 0 25 0 1 0 869463683 1192824832 287505 33554432000 134512640 136139019 4290441664 18446744073709551615 134640592 0 0 16781312 16386 18446744071562691360 0 0 17 7 0 0 0
/proc/1877/statm: 291217 287505 184 398 0 290817 0
[pid=1875/tid=1900] ppid=1853 vsize=3076232 CPUtime=0 cores=1,3,5,7
/proc/1875/task/1900/stat : 1900 (python) S 1853 1853 758 0 -1 4202560 43 0 0 0 0 0 0 0 19 0 2 0 869468514 3150061568 701185 33554432000 4194304 5683644 140736202612192 18446744073709551615 228256172465 0 0 16781312 33554946 18446604459085539328 0 0 -1 7 0 0 0
[pid=1901] ppid=1853 vsize=3076120 CPUtime=0.27 cores=1,3,5,7
/proc/1901/stat : 1901 (python) S 1853 1853 758 0 -1 4202560 1109 0 0 0 11 16 0 0 19 0 2 0 869468516 3149946880 701296 33554432000 4194304 5683644 140736202612192 18446744073709551615 228256175563 0 0 16781312 33554946 18446744073709551615 0 0 17 7 0 0 0
/proc/1901/statm: 769030 701296 397 364 0 703914 0
[pid=1903] ppid=1901 vsize=2581300 CPUtime=47.28 cores=1,3,5,7
/proc/1903/stat : 1903 (scip-2.0.1.linu) T 1901 1903 1903 0 -1 4202496 641376 0 0 0 4642 86 0 0 25 0 1 0 869468520 2643251200 641099 33554432000 4194304 13684479 140733552664224 18446744073709551615 11824617 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/1903/statm: 645325 641099 510 2317 0 643003 0
[pid=1901/tid=1907] ppid=1853 vsize=3076120 CPUtime=0 cores=1,3,5,7
/proc/1901/task/1907/stat : 1907 (python) S 1853 1853 758 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 869473253 3149946880 701296 33554432000 4194304 5683644 140736202612192 18446744073709551615 228256172465 0 0 16781312 33554946 18446604455339136128 0 0 -1 1 0 0 0
[pid=2068] ppid=1853 vsize=3086308 CPUtime=0.04 cores=1,3,5,7
/proc/2068/stat : 2068 (python) S 1853 1853 758 0 -1 4202560 761 0 0 0 0 4 0 0 25 0 1 0 869561691 3160379392 701252 33554432000 4194304 5683644 140736202612192 18446744073709551615 228244368966 0 0 16781312 33554946 4527234836695318789 0 0 17 1 0 0 0
/proc/2068/statm: 771577 701252 350 364 0 706458 0
[pid=2070] ppid=2068 vsize=554920 CPUtime=0.53 cores=1,3,5,7
/proc/2070/stat : 2070 (minion) R 2068 2070 2070 0 -1 4202496 218168 0 0 0 11 42 0 0 25 0 1 0 869561696 568238080 86864 33554432000 134512640 165565024 4294526400 18446744073709551615 159737071 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/2070/statm: 138730 86870 226 7582 0 131145 0
[pid=1853/tid=1876] ppid=1851 vsize=3096224 CPUtime=0 cores=1,3,5,7
/proc/1853/task/1876/stat : 1876 (python) S 1851 1853 758 0 -1 4202560 27 928 0 0 0 0 0 0 25 0 4 0 869463683 3170533376 702766 33554432000 4194304 5683644 140736202612192 18446744073709551615 228256172465 0 0 16781312 33554434 18446604457588720832 0 0 -1 5 0 0 0
[pid=1853/tid=1902] ppid=1851 vsize=3096224 CPUtime=0 cores=1,3,5,7
/proc/1853/task/1902/stat : 1902 (python) S 1851 1853 758 0 -1 4202560 25 928 0 0 0 0 0 0 20 0 4 0 869468520 3170533376 702766 33554432000 4194304 5683644 140736202612192 18446744073709551615 228256172465 0 0 16781312 33554434 18446604457588720832 0 0 -1 5 0 0 0
[pid=1853/tid=2069] ppid=1851 vsize=3096224 CPUtime=0 cores=1,3,5,7
/proc/1853/task/2069/stat : 2069 (python) S 1851 1853 758 0 -1 4202560 31 928 0 0 0 0 0 0 25 0 4 0 869561696 3170533376 702766 33554432000 4194304 5683644 140736202612192 18446744073709551615 228256172465 0 0 16781312 33554434 18446604457588720832 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1261.04
Current children cumulated vsize (KiB) 16635972

[startup+1260.96 s]
/proc/loadavg: 1.00 1.22 1.55 3/640 2070
/proc/meminfo: memFree=15734296/32951124 swapFree=58212620/67111528
[pid=1853] ppid=1851 vsize=0 CPUtime=1164.55 cores=1,3,5,7
/proc/1853/stat : 1853 (python) Z 1851 1853 758 0 -1 4203548 1707248 928 1 0 115971 484 0 0 19 0 2 0 869435672 0 0 33554432000 0 0 0 0 0 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0
/proc/1853/statm: 0 0 0 0 0 0 0
[pid=1875] ppid=1853 vsize=0 CPUtime=0.3 cores=1,3,5,7
/proc/1875/stat : 1875 (python) Z 1853 1853 758 0 -1 4203612 1159 0 0 0 10 20 0 0 18 0 1 0 869463679 0 0 33554432000 0 0 0 0 0 0 0 16781312 33554946 18446744073709551615 0 0 17 5 0 0 0
/proc/1875/statm: 0 0 0 0 0 0 0
[pid=1877] ppid=1 vsize=1164868 CPUtime=48.2 cores=1,3,5,7
/proc/1877/stat : 1877 (clasp-1.3.7-x86) T 1 1877 1877 0 -1 4202496 408297 0 0 0 4744 76 0 0 25 0 1 0 869463683 1192824832 287505 33554432000 134512640 136139019 4290441664 18446744073709551615 134640592 0 0 16781312 16386 18446744071562691360 0 0 17 7 0 0 0
/proc/1877/statm: 291217 287505 184 398 0 290817 0
[pid=1901] ppid=1853 vsize=0 CPUtime=0.31 cores=1,3,5,7
/proc/1901/stat : 1901 (python) Z 1853 1853 758 0 -1 4203612 1109 0 0 0 11 20 0 0 18 0 1 0 869468516 0 0 33554432000 0 0 0 0 0 0 0 16781312 33554946 18446744073709551615 0 0 17 7 0 0 0
/proc/1901/statm: 0 0 0 0 0 0 0
[pid=1903] ppid=1 vsize=2581300 CPUtime=47.28 cores=1,3,5,7
/proc/1903/stat : 1903 (scip-2.0.1.linu) T 1 1903 1903 0 -1 4202496 641376 0 0 0 4642 86 0 0 25 0 1 0 869468520 2643251200 641099 33554432000 4194304 13684479 140733552664224 18446744073709551615 11824617 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/1903/statm: 645325 641099 510 2317 0 643003 0
[pid=2068] ppid=1853 vsize=0 CPUtime=0.08 cores=1,3,5,7
/proc/2068/stat : 2068 (python) Z 1853 1853 758 0 -1 4203612 761 0 0 0 0 8 0 0 20 0 1 0 869561691 0 0 33554432000 0 0 0 0 0 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0
/proc/2068/statm: 0 0 0 0 0 0 0
[pid=1853/tid=1876] ppid=1851 vsize=0 CPUtime=0.09 cores=1,3,5,7
/proc/1853/task/1876/stat : 1876 (python) R 1851 1853 758 0 -1 4203588 27 928 0 0 0 9 0 0 18 0 2 0 869463683 0 0 33554432000 0 0 0 0 0 0 0 16781312 33554434 18446744073709551615 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 1260.72
Current children cumulated vsize (KiB) 3746168
#
############


Solver just ended. Dumping a history of the last processes samples

[startup+1260.96 s]
/proc/loadavg: 1.00 1.22 1.55 3/640 2070
/proc/meminfo: memFree=15734296/32951124 swapFree=58212620/67111528
[pid=1853] ppid=1851 vsize=0 CPUtime=1164.55 cores=1,3,5,7
/proc/1853/stat : 1853 (python) Z 1851 1853 758 0 -1 4203548 1707248 928 1 0 115971 484 0 0 19 0 2 0 869435672 0 0 33554432000 0 0 0 0 0 0 0 16781312 33554434 18446744073709551615 0 0 17 1 0 0 0
/proc/1853/statm: 0 0 0 0 0 0 0
[pid=1875] ppid=1853 vsize=0 CPUtime=0.3 cores=1,3,5,7
/proc/1875/stat : 1875 (python) Z 1853 1853 758 0 -1 4203612 1159 0 0 0 10 20 0 0 18 0 1 0 869463679 0 0 33554432000 0 0 0 0 0 0 0 16781312 33554946 18446744073709551615 0 0 17 5 0 0 0
/proc/1875/statm: 0 0 0 0 0 0 0
[pid=1877] ppid=1 vsize=1164868 CPUtime=48.2 cores=1,3,5,7
/proc/1877/stat : 1877 (clasp-1.3.7-x86) T 1 1877 1877 0 -1 4202496 408297 0 0 0 4744 76 0 0 25 0 1 0 869463683 1192824832 287505 33554432000 134512640 136139019 4290441664 18446744073709551615 134640592 0 0 16781312 16386 18446744071562691360 0 0 17 7 0 0 0
/proc/1877/statm: 291217 287505 184 398 0 290817 0
[pid=1901] ppid=1853 vsize=0 CPUtime=0.31 cores=1,3,5,7
/proc/1901/stat : 1901 (python) Z 1853 1853 758 0 -1 4203612 1109 0 0 0 11 20 0 0 18 0 1 0 869468516 0 0 33554432000 0 0 0 0 0 0 0 16781312 33554946 18446744073709551615 0 0 17 7 0 0 0
/proc/1901/statm: 0 0 0 0 0 0 0
[pid=1903] ppid=1 vsize=2581300 CPUtime=47.28 cores=1,3,5,7
/proc/1903/stat : 1903 (scip-2.0.1.linu) T 1 1903 1903 0 -1 4202496 641376 0 0 0 4642 86 0 0 25 0 1 0 869468520 2643251200 641099 33554432000 4194304 13684479 140733552664224 18446744073709551615 11824617 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/1903/statm: 645325 641099 510 2317 0 643003 0
[pid=2068] ppid=1853 vsize=0 CPUtime=0.08 cores=1,3,5,7
/proc/2068/stat : 2068 (python) Z 1853 1853 758 0 -1 4203612 761 0 0 0 0 8 0 0 20 0 1 0 869561691 0 0 33554432000 0 0 0 0 0 0 0 16781312 33554946 18446744073709551615 0 0 17 1 0 0 0
/proc/2068/statm: 0 0 0 0 0 0 0
[pid=1853/tid=1876] ppid=1851 vsize=0 CPUtime=0.09 cores=1,3,5,7
/proc/1853/task/1876/stat : 1876 (python) R 1851 1853 758 0 -1 4203588 27 928 0 0 0 9 0 0 18 0 2 0 869463683 0 0 33554432000 0 0 0 0 0 0 0 16781312 33554434 18446744073709551615 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 1260.72
Current children cumulated vsize (KiB) 3746168

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 1853 and gives
#  childrusage.ru_utime.tv_sec=1159
#  childrusage.ru_utime.tv_usec=717696
#  childrusage.ru_stime.tv_sec=4
#  childrusage.ru_stime.tv_usec=857261
# CPU time returned by wait4() is 1164.57
# while last known CPU time is 1260.72
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...


# WARNING:
# The CPU time of some children was not reported to their father
# (probably because of a missing or aborted wait()).
# This 'lost CPU time' is added to the watched process CPU time.
#  lost CPU time (s): 0.320068
#  lost CPU user time (s): 0.109985
#  lost CPU system time (s): 0.21

Real time (s): 1260.97
CPU time (s): 1261.04
CPU user time (s): 1253.89
CPU system time (s): 7.15
CPU usage (%): 100.005
Max. virtual memory (cumulated for all children) (KiB): 16635972

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1159.72
system time used= 4.85726
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1708176
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1022
involuntary context switches= 4068

runsolver used 6.62299 second user time and 16.4365 second system time

The end

Launcher Data 

Begin job on node136 at 2011-06-09 02:12:54
IDJOB=3482892
IDBENCH=89647
IDSOLVER=1971
FILE ID=node136/3482892-1307578358
RUNJOBID= node136-1307576355-777
PBS_JOBID= 13547663
Free space on /tmp= 72988 MiB

SOLVER NAME= borg pb-dec-11.04.03
BENCH NAME= PB11/normalized-PB11/DEC-SMALLINT-LIN/lopes/normalized-182.opb
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.dec.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3482892-1307578358/watcher-3482892-1307578358 -o /tmp/evaluation-result-3482892-1307578358/solver-3482892-1307578358 -C 1800 -W 1900 -M 15500  python HOME/solve HOME/etc/borg-mix+class.dec.pickle HOME/instance-3482892-1307578358.opb -seed 2097467692 -budget 1800

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 378c188d79de101f4114d358aff291a2
RANDOM SEED=2097467692

node136.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      24602196 kB
Buffers:        162980 kB
Cached:        1370116 kB
SwapCached:       8824 kB
Active:        7821476 kB
Inactive:       215340 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      24602196 kB
SwapTotal:    67111528 kB
SwapFree:     58212620 kB
Dirty:          789372 kB
Writeback:           0 kB
AnonPages:     6501144 kB
Mapped:          64552 kB
Slab:           159892 kB
PageTables:      80068 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 174077848 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71888 MiB
End job on node136 at 2011-06-09 02:34:00