Trace number 3284016

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 4800.21 1201.76

General information on the benchmark

NameAPPLICATIONS/satComp09_BioInstances/
ndhf_xits_19_UNKNOWN.cnf
MD5SUM4cf2a693df1b6e5a084b88e84ec5f2af
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark14.5608
Satisfiable
(Un)Satisfiability was proved
Number of variables4020
Number of clauses466486
Sum of the clauses size1393798
Maximum clause length20
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 26187
Number of clauses of size 3460268
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 531

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 4 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] lingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	[5] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 1
0.00/0.00	Starting clasp 1.3.6 on core(s) 5
0.00/0.00	Starting TNM 2009 on core(s) 7
0.00/0.00	Starting lingeling 276-6264d55-100731 on core(s) 3
0.00/0.00	Starting march_hi 2009 on core(s) 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3284016-1304033817/watcher-3284016-1304033817 -o /tmp/evaluation-result-3284016-1304033817/solver-3284016-1304033817 -C 4800 -W 1300 -M 15500 ppfolio HOME/instance-3284016-1304033817.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 10.22 10.15 9.35 6/186 2839
/proc/meminfo: memFree=24717740/32951124 swapFree=67111352/67111528
[pid=2839] ppid=2837 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2839/stat : 2839 (ppfolio) S 2837 2839 2100 0 -1 4202496 523 0 0 0 0 0 0 0 25 0 1 0 514979969 18763776 390 33554432000 4194304 4287625 140734618231952 18446744073709551615 246516854287 0 0 4096 65536 2826088480825 0 0 17 5 0 0 0
/proc/2839/statm: 4581 390 341 23 0 91 0

[startup+0.092568 s]
/proc/loadavg: 10.22 10.15 9.35 6/186 2839
/proc/meminfo: memFree=24717740/32951124 swapFree=67111352/67111528
[pid=2839] ppid=2837 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2839/stat : 2839 (ppfolio) R 2837 2839 2100 0 -1 4202496 523 0 0 0 0 0 0 0 25 0 1 0 514979969 18763776 390 33554432000 4194304 4287625 140734618231952 18446744073709551615 246516854287 0 0 4096 65536 0 0 0 17 5 0 0 0
/proc/2839/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100591 s]
/proc/loadavg: 10.22 10.15 9.35 6/186 2839
/proc/meminfo: memFree=24717740/32951124 swapFree=67111352/67111528
[pid=2839] ppid=2837 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2839/stat : 2839 (ppfolio) R 2837 2839 2100 0 -1 4202496 523 0 0 0 0 0 0 0 25 0 1 0 514979969 18763776 390 33554432000 4194304 4287625 140734618231952 18446744073709551615 246516854287 0 0 4096 65536 0 0 0 17 5 0 0 0
/proc/2839/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.30053 s]
/proc/loadavg: 10.22 10.15 9.35 6/186 2839
/proc/meminfo: memFree=24717740/32951124 swapFree=67111352/67111528
[pid=2839] ppid=2837 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2839/stat : 2839 (ppfolio) S 2837 2839 2100 0 -1 4202496 524 0 0 0 0 0 0 0 25 0 1 0 514979969 18763776 390 33554432000 4194304 4287625 140734618231952 18446744073709551615 246516854287 0 0 4096 65536 4404854966276063231 0 0 17 5 0 0 0
/proc/2839/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700459 s]
/proc/loadavg: 10.22 10.15 9.35 6/186 2839
/proc/meminfo: memFree=24717740/32951124 swapFree=67111352/67111528
[pid=2839] ppid=2837 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2839/stat : 2839 (ppfolio) S 2837 2839 2100 0 -1 4202496 524 0 0 0 0 0 0 0 20 0 1 0 514979969 18763776 390 33554432000 4194304 4287625 140734618231952 18446744073709551615 246516854287 0 0 4096 65536 4404854966276063231 0 0 17 5 0 0 0
/proc/2839/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50131 s]
/proc/loadavg: 10.22 10.15 9.35 11/192 2845
/proc/meminfo: memFree=24486460/32951124 swapFree=67111352/67111528
[pid=2839] ppid=2837 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2839/stat : 2839 (ppfolio) S 2837 2839 2100 0 -1 4202496 524 0 0 0 0 0 0 0 19 0 1 0 514979969 18763776 390 33554432000 4194304 4287625 140734618231952 18446744073709551615 246516854287 0 0 4096 65536 4404854966276063231 0 0 17 5 0 0 0
/proc/2839/statm: 4581 390 341 23 0 91 0
[pid=2840] ppid=2839 vsize=143528 CPUtime=1.49 cores=1
/proc/2840/stat : 2840 (cryptominisat) R 2839 2839 2100 0 -1 4202496 26392 0 0 0 145 4 0 0 25 0 1 0 514979970 146972672 23613 33554432000 4194304 4587354 140736981477968 18446744073709551615 4388277 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2840/statm: 35882 23613 317 96 0 32444 0
[pid=2841] ppid=2839 vsize=49396 CPUtime=1.49 cores=5
/proc/2841/stat : 2841 (clasp) R 2839 2839 2100 0 -1 4202496 11590 0 0 0 147 2 0 0 25 0 1 0 514979970 50581504 9685 33554432000 4194304 4785740 140737466685840 18446744073709551615 4495597 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2841/statm: 12349 9685 314 145 0 9394 0
[pid=2842] ppid=2839 vsize=482764 CPUtime=0.74 cores=7
/proc/2842/stat : 2842 (TNM) R 2839 2839 2100 0 -1 4202496 9803 0 0 0 73 1 0 0 25 0 1 0 514979970 494350336 9699 33554432000 134512640 134534564 4287287376 18446744073709551615 134524541 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2842/statm: 120691 9699 91 6 0 119357 0
[pid=2843] ppid=2839 vsize=49440 CPUtime=1.48 cores=3
/proc/2843/stat : 2843 (lingeling) R 2839 2839 2100 0 -1 4202496 16479 0 0 0 146 2 0 0 25 0 1 0 514979970 50626560 10531 33554432000 4194304 4352236 140734773883328 18446744073709551615 4207619 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2843/statm: 12360 10531 124 39 0 11440 0
[pid=2844] ppid=2839 vsize=54832 CPUtime=0.73 cores=7
/proc/2844/stat : 2844 (march_hi) R 2839 2839 2100 0 -1 4194304 16110 0 0 0 70 3 0 0 25 0 1 0 514979970 56147968 7688 33554432000 4194304 4331988 140734209786144 18446744073709551615 4257024 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2844/statm: 13708 7688 129 34 0 12150 0
Current children cumulated CPU time (s) 5.93
Current children cumulated vsize (KiB) 798284

[startup+3.10102 s]
/proc/loadavg: 10.22 10.15 9.35 11/192 2845
/proc/meminfo: memFree=24491204/32951124 swapFree=67111352/67111528
[pid=2839] ppid=2837 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2839/stat : 2839 (ppfolio) S 2837 2839 2100 0 -1 4202496 524 0 0 0 0 0 0 0 19 0 1 0 514979969 18763776 390 33554432000 4194304 4287625 140734618231952 18446744073709551615 246516854287 0 0 4096 65536 4404854966276063231 0 0 17 5 0 0 0
/proc/2839/statm: 4581 390 341 23 0 91 0
[pid=2840] ppid=2839 vsize=171964 CPUtime=3.09 cores=1
/proc/2840/stat : 2840 (cryptominisat) R 2839 2839 2100 0 -1 4202496 36090 0 0 0 304 5 0 0 25 0 1 0 514979970 176091136 28894 33554432000 4194304 4587354 140736981477968 18446744073709551615 4388248 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2840/statm: 42991 28894 317 96 0 39553 0
[pid=2841] ppid=2839 vsize=49396 CPUtime=3.09 cores=5
/proc/2841/stat : 2841 (clasp) R 2839 2839 2100 0 -1 4202496 11590 0 0 0 307 2 0 0 25 0 1 0 514979970 50581504 9685 33554432000 4194304 4785740 140737466685840 18446744073709551615 4495638 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2841/statm: 12349 9685 314 145 0 9394 0
[pid=2842] ppid=2839 vsize=482764 CPUtime=1.54 cores=7
/proc/2842/stat : 2842 (TNM) R 2839 2839 2100 0 -1 4202496 9803 0 0 0 153 1 0 0 25 0 1 0 514979970 494350336 9699 33554432000 134512640 134534564 4287287376 18446744073709551615 134524582 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2842/statm: 120691 9699 91 6 0 119357 0
[pid=2843] ppid=2839 vsize=49440 CPUtime=3.08 cores=3
/proc/2843/stat : 2843 (lingeling) R 2839 2839 2100 0 -1 4202496 16481 0 0 0 306 2 0 0 25 0 1 0 514979970 50626560 10533 33554432000 4194304 4352236 140734773883328 18446744073709551615 4329043 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2843/statm: 12360 10533 126 39 0 11440 0
[pid=2844] ppid=2839 vsize=60396 CPUtime=1.53 cores=7
/proc/2844/stat : 2844 (march_hi) R 2839 2839 2100 0 -1 4202496 36905 0 0 0 147 6 0 0 25 0 1 0 514979970 61845504 9095 33554432000 4194304 4331988 140734209786144 18446744073709551615 4252215 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2844/statm: 15099 9095 148 34 0 13541 0
Current children cumulated CPU time (s) 12.33
Current children cumulated vsize (KiB) 832284

[startup+6.30043 s]
/proc/loadavg: 10.20 10.15 9.36 11/192 2845
/proc/meminfo: memFree=24397348/32951124 swapFree=67111352/67111528
[pid=2839] ppid=2837 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2839/stat : 2839 (ppfolio) S 2837 2839 2100 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 514979969 18763776 390 33554432000 4194304 4287625 140734618231952 18446744073709551615 246516854287 0 0 4096 65536 4404854966276063231 0 0 17 5 0 0 0
/proc/2839/statm: 4581 390 341 23 0 91 0
[pid=2840] ppid=2839 vsize=156352 CPUtime=6.29 cores=1
/proc/2840/stat : 2840 (cryptominisat) R 2839 2839 2100 0 -1 4202496 37678 0 0 0 623 6 0 0 25 0 1 0 514979970 160104448 27340 33554432000 4194304 4587354 140736981477968 18446744073709551615 4238611 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2840/statm: 39088 27340 327 96 0 35650 0
[pid=2841] ppid=2839 vsize=53804 CPUtime=6.29 cores=5
/proc/2841/stat : 2841 (clasp) R 2839 2839 2100 0 -1 4202496 12684 0 0 0 627 2 0 0 25 0 1 0 514979970 55095296 10779 33554432000 4194304 4785740 140737466685840 18446744073709551615 4482821 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2841/statm: 13451 10779 316 145 0 10496 0
[pid=2842] ppid=2839 vsize=482764 CPUtime=3.14 cores=7
/proc/2842/stat : 2842 (TNM) R 2839 2839 2100 0 -1 4202496 9803 0 0 0 313 1 0 0 25 0 1 0 514979970 494350336 9699 33554432000 134512640 134534564 4287287376 18446744073709551615 134524568 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2842/statm: 120691 9699 91 6 0 119357 0
[pid=2843] ppid=2839 vsize=66232 CPUtime=6.29 cores=3
/proc/2843/stat : 2843 (lingeling) R 2839 2839 2100 0 -1 4202496 21670 0 0 0 626 3 0 0 25 0 1 0 514979970 67821568 15722 33554432000 4194304 4352236 140734773883328 18446744073709551615 4236203 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2843/statm: 16558 15722 126 39 0 15638 0
[pid=2844] ppid=2839 vsize=125932 CPUtime=3.1 cores=7
/proc/2844/stat : 2844 (march_hi) R 2839 2839 2100 0 -1 4202496 64215 0 0 0 300 10 0 0 25 0 1 0 514979970 128954368 22570 33554432000 4194304 4331988 140734209786144 18446744073709551615 4271154 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2844/statm: 31483 22570 151 34 0 29925 0
Current children cumulated CPU time (s) 25.11
Current children cumulated vsize (KiB) 903408

[startup+12.7013 s]
/proc/loadavg: 10.18 10.15 9.36 12/192 2845
/proc/meminfo: memFree=24219920/32951124 swapFree=67111352/67111528
[pid=2839] ppid=2837 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2839/stat : 2839 (ppfolio) S 2837 2839 2100 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 514979969 18763776 390 33554432000 4194304 4287625 140734618231952 18446744073709551615 246516854287 0 0 4096 65536 4404854966276063231 0 0 17 5 0 0 0
/proc/2839/statm: 4581 390 341 23 0 91 0
[pid=2840] ppid=2839 vsize=156352 CPUtime=12.69 cores=1
/proc/2840/stat : 2840 (cryptominisat) R 2839 2839 2100 0 -1 4202496 40900 0 0 0 1262 7 0 0 25 0 1 0 514979970 160104448 30546 33554432000 4194304 4587354 140736981477968 18446744073709551615 246516530833 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2840/statm: 39088 30546 331 96 0 35650 0
[pid=2841] ppid=2839 vsize=61620 CPUtime=12.68 cores=5
/proc/2841/stat : 2841 (clasp) R 2839 2839 2100 0 -1 4202496 15522 0 0 0 1266 2 0 0 25 0 1 0 514979970 63098880 12705 33554432000 4194304 4785740 140737466685840 18446744073709551615 4553010 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2841/statm: 15405 12705 340 145 0 12450 0
[pid=2842] ppid=2839 vsize=482764 CPUtime=6.3 cores=7
/proc/2842/stat : 2842 (TNM) R 2839 2839 2100 0 -1 4202496 9803 0 0 0 629 1 0 0 25 0 1 0 514979970 494350336 9699 33554432000 134512640 134534564 4287287376 18446744073709551615 134524491 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2842/statm: 120691 9699 91 6 0 119357 0
[pid=2843] ppid=2839 vsize=68280 CPUtime=12.69 cores=3
/proc/2843/stat : 2843 (lingeling) R 2839 2839 2100 0 -1 4202496 22182 0 0 0 1266 3 0 0 25 0 1 0 514979970 69918720 16234 33554432000 4194304 4352236 140734773883328 18446744073709551615 4328194 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2843/statm: 17070 16234 126 39 0 16150 0
[pid=2844] ppid=2839 vsize=307504 CPUtime=6.3 cores=7
/proc/2844/stat : 2844 (march_hi) R 2839 2839 2100 0 -1 4202496 101283 0 0 0 614 16 0 0 25 0 1 0 514979970 314884096 59638 33554432000 4194304 4331988 140734209786144 18446744073709551615 4271136 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2844/statm: 76876 59638 151 34 0 75318 0
Current children cumulated CPU time (s) 50.66
Current children cumulated vsize (KiB) 1094844

[startup+25.5009 s]
/proc/loadavg: 10.22 10.16 9.37 11/192 2845
/proc/meminfo: memFree=23846252/32951124 swapFree=67111352/67111528
[pid=2839] ppid=2837 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2839/stat : 2839 (ppfolio) S 2837 2839 2100 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 514979969 18763776 390 33554432000 4194304 4287625 140734618231952 18446744073709551615 246516854287 0 0 4096 65536 4404854966276063231 0 0 17 5 0 0 0
/proc/2839/statm: 4581 390 341 23 0 91 0
[pid=2840] ppid=2839 vsize=209952 CPUtime=25.49 cores=1
/proc/2840/stat : 2840 (cryptominisat) R 2839 2839 2100 0 -1 4202496 63592 0 0 0 2539 10 0 0 25 0 1 0 514979970 214990848 44272 33554432000 4194304 4587354 140736981477968 18446744073709551615 4388336 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2840/statm: 52488 44272 337 96 0 49050 0
[pid=2841] ppid=2839 vsize=76080 CPUtime=25.49 cores=5
/proc/2841/stat : 2841 (clasp) R 2839 2839 2100 0 -1 4202496 19065 0 0 0 2545 4 0 0 25 0 1 0 514979970 77905920 16248 33554432000 4194304 4785740 140737466685840 18446744073709551615 4553046 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2841/statm: 19020 16248 340 145 0 16065 0
[pid=2842] ppid=2839 vsize=482764 CPUtime=12.63 cores=7
/proc/2842/stat : 2842 (TNM) R 2839 2839 2100 0 -1 4202496 9803 0 0 0 1262 1 0 0 25 0 1 0 514979970 494350336 9699 33554432000 134512640 134534564 4287287376 18446744073709551615 134524464 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2842/statm: 120691 9699 91 6 0 119357 0
[pid=2843] ppid=2839 vsize=72376 CPUtime=25.48 cores=3
/proc/2843/stat : 2843 (lingeling) R 2839 2839 2100 0 -1 4202496 23206 0 0 0 2545 3 0 0 25 0 1 0 514979970 74113024 17258 33554432000 4194304 4352236 140734773883328 18446744073709551615 4267799 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2843/statm: 18094 17258 126 39 0 17174 0
[pid=2844] ppid=2839 vsize=619516 CPUtime=12.7 cores=7
/proc/2844/stat : 2844 (march_hi) R 2839 2839 2100 0 -1 4202496 174726 0 0 0 1240 30 0 0 25 0 1 0 514979970 634384384 133081 33554432000 4194304 4331988 140734209786144 18446744073709551615 4271154 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2844/statm: 154879 133081 151 34 0 153321 0
Current children cumulated CPU time (s) 101.79
Current children cumulated vsize (KiB) 1479012

[startup+51.1013 s]
/proc/loadavg: 10.39 10.21 9.41 11/192 2846
/proc/meminfo: memFree=23259376/32951124 swapFree=67111352/67111528
[pid=2839] ppid=2837 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2839/stat : 2839 (ppfolio) S 2837 2839 2100 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 514979969 18763776 390 33554432000 4194304 4287625 140734618231952 18446744073709551615 246516854287 0 0 4096 65536 4404854966276063231 0 0 17 5 0 0 0
/proc/2839/statm: 4581 390 341 23 0 91 0
[pid=2840] ppid=2839 vsize=142724 CPUtime=51.09 cores=1
/proc/2840/stat : 2840 (cryptominisat) R 2839 2839 2100 0 -1 4202496 68372 0 0 0 5098 11 0 0 25 0 1 0 514979970 146149376 29449 33554432000 4194304 4587354 140736981477968 18446744073709551615 4241286 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2840/statm: 35681 29449 337 96 0 32243 0
[pid=2841] ppid=2839 vsize=126856 CPUtime=51.09 cores=5
/proc/2841/stat : 2841 (clasp) R 2839 2839 2100 0 -1 4202496 31395 0 0 0 5101 8 0 0 25 0 1 0 514979970 129900544 28578 33554432000 4194304 4785740 140737466685840 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2841/statm: 31714 28578 340 145 0 28759 0
[pid=2842] ppid=2839 vsize=485536 CPUtime=25.41 cores=7
/proc/2842/stat : 2842 (TNM) R 2839 2839 2100 0 -1 4202496 12884 0 0 0 2539 2 0 0 25 0 1 0 514979970 497188864 12780 33554432000 134512640 134534564 4287287376 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2842/statm: 121384 12780 95 6 0 120050 0
[pid=2843] ppid=2839 vsize=78008 CPUtime=51.08 cores=3

################
# More data... #
################

/proc/2844/statm: 222472 215968 175 34 0 220914 0
Current children cumulated CPU time (s) 3764.02
Current children cumulated vsize (KiB) 3062608

[startup+1002.3 s]
/proc/loadavg: 10.43 10.31 9.95 11/192 2911
/proc/meminfo: memFree=22993820/32951124 swapFree=67111352/67111528
[pid=2839] ppid=2837 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2839/stat : 2839 (ppfolio) S 2837 2839 2100 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 514979969 18763776 390 33554432000 4194304 4287625 140734618231952 18446744073709551615 246516854287 0 0 4096 65536 4404854966276063231 0 0 17 5 0 0 0
/proc/2839/statm: 4581 390 341 23 0 91 0
[pid=2840] ppid=2839 vsize=853352 CPUtime=1002.32 cores=1
/proc/2840/stat : 2840 (cryptominisat) R 2839 2839 2100 0 -1 4202496 840210 0 0 0 99986 246 0 0 25 0 1 0 514979970 873832448 34919 33554432000 4194304 4587354 140736981477968 18446744073709551615 4238595 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2840/statm: 213338 34919 337 96 0 209900 0
[pid=2841] ppid=2839 vsize=421588 CPUtime=1002.32 cores=5
/proc/2841/stat : 2841 (clasp) R 2839 2839 2100 0 -1 4202496 100481 0 0 0 100200 32 0 0 25 0 1 0 514979970 431706112 97664 33554432000 4194304 4785740 140737466685840 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2841/statm: 105397 97664 340 145 0 102442 0
[pid=2842] ppid=2839 vsize=485536 CPUtime=498.42 cores=7
/proc/2842/stat : 2842 (TNM) R 2839 2839 2100 0 -1 4202496 12884 0 0 0 49840 2 0 0 25 0 1 0 514979970 497188864 12780 33554432000 134512640 134534564 4287287376 18446744073709551615 134530606 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2842/statm: 121384 12780 95 6 0 120050 0
[pid=2843] ppid=2839 vsize=137696 CPUtime=1002.21 cores=3
/proc/2843/stat : 2843 (lingeling) R 2839 2839 2100 0 -1 4202496 39538 0 0 0 100215 6 0 0 25 0 1 0 514979970 141000704 33590 33554432000 4194304 4352236 140734773883328 18446744073709551615 4328000 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2843/statm: 34424 33590 126 39 0 33504 0
[pid=2844] ppid=2839 vsize=889888 CPUtime=498.43 cores=7
/proc/2844/stat : 2844 (march_hi) R 2839 2839 2100 0 -1 4202496 538494 0 0 0 49754 89 0 0 25 0 1 0 514979970 911245312 215968 33554432000 4194304 4331988 140734209786144 18446744073709551615 4299294 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2844/statm: 222472 215968 175 34 0 220914 0
Current children cumulated CPU time (s) 4003.7
Current children cumulated vsize (KiB) 2806384

[startup+1062.3 s]
/proc/loadavg: 10.37 10.30 9.97 12/192 2913
/proc/meminfo: memFree=22997156/32951124 swapFree=67111352/67111528
[pid=2839] ppid=2837 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2839/stat : 2839 (ppfolio) S 2837 2839 2100 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 514979969 18763776 390 33554432000 4194304 4287625 140734618231952 18446744073709551615 246516854287 0 0 4096 65536 4404854966276063231 0 0 17 5 0 0 0
/proc/2839/statm: 4581 390 341 23 0 91 0
[pid=2840] ppid=2839 vsize=852252 CPUtime=1062.32 cores=1
/proc/2840/stat : 2840 (cryptominisat) R 2839 2839 2100 0 -1 4202496 881884 0 0 0 105971 261 0 0 25 0 1 0 514979970 872706048 31285 33554432000 4194304 4587354 140736981477968 18446744073709551615 4241215 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2840/statm: 213063 31285 337 96 0 209625 0
[pid=2841] ppid=2839 vsize=433556 CPUtime=1062.32 cores=5
/proc/2841/stat : 2841 (clasp) R 2839 2839 2100 0 -1 4202496 103339 0 0 0 106199 33 0 0 25 0 1 0 514979970 443961344 100522 33554432000 4194304 4785740 140737466685840 18446744073709551615 4287017 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2841/statm: 108389 100522 340 145 0 105434 0
[pid=2842] ppid=2839 vsize=485536 CPUtime=528.26 cores=7
/proc/2842/stat : 2842 (TNM) R 2839 2839 2100 0 -1 4202496 12884 0 0 0 52824 2 0 0 25 0 1 0 514979970 497188864 12780 33554432000 134512640 134534564 4287287376 18446744073709551615 134528968 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2842/statm: 121384 12780 95 6 0 120050 0
[pid=2843] ppid=2839 vsize=137696 CPUtime=1062.21 cores=3
/proc/2843/stat : 2843 (lingeling) R 2839 2839 2100 0 -1 4202496 39538 0 0 0 106215 6 0 0 25 0 1 0 514979970 141000704 33590 33554432000 4194304 4352236 140734773883328 18446744073709551615 4328194 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2843/statm: 34424 33590 126 39 0 33504 0
[pid=2844] ppid=2839 vsize=889888 CPUtime=528.26 cores=7
/proc/2844/stat : 2844 (march_hi) R 2839 2839 2100 0 -1 4202496 538501 0 0 0 52737 89 0 0 25 0 1 0 514979970 911245312 215975 33554432000 4194304 4331988 140734209786144 18446744073709551615 4299269 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2844/statm: 222472 215975 175 34 0 220914 0
Current children cumulated CPU time (s) 4243.37
Current children cumulated vsize (KiB) 2817252

[startup+1122.3 s]
/proc/loadavg: 10.39 10.31 9.99 11/192 2914
/proc/meminfo: memFree=22980752/32951124 swapFree=67111352/67111528
[pid=2839] ppid=2837 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2839/stat : 2839 (ppfolio) S 2837 2839 2100 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 514979969 18763776 390 33554432000 4194304 4287625 140734618231952 18446744073709551615 246516854287 0 0 4096 65536 4404854966276063231 0 0 17 5 0 0 0
/proc/2839/statm: 4581 390 341 23 0 91 0
[pid=2840] ppid=2839 vsize=851136 CPUtime=1122.32 cores=1
/proc/2840/stat : 2840 (cryptominisat) R 2839 2839 2100 0 -1 4202496 915454 0 0 0 111957 275 0 0 25 0 1 0 514979970 871563264 34450 33554432000 4194304 4587354 140736981477968 18446744073709551615 4237686 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2840/statm: 212784 34450 337 96 0 209346 0
[pid=2841] ppid=2839 vsize=442908 CPUtime=1122.32 cores=5
/proc/2841/stat : 2841 (clasp) R 2839 2839 2100 0 -1 4202496 105533 0 0 0 112199 33 0 0 25 0 1 0 514979970 453537792 102716 33554432000 4194304 4785740 140737466685840 18446744073709551615 4553423 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2841/statm: 110727 102716 340 145 0 107772 0
[pid=2842] ppid=2839 vsize=485536 CPUtime=558.08 cores=7
/proc/2842/stat : 2842 (TNM) R 2839 2839 2100 0 -1 4202496 12884 0 0 0 55806 2 0 0 25 0 1 0 514979970 497188864 12780 33554432000 134512640 134534564 4287287376 18446744073709551615 134528726 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2842/statm: 121384 12780 95 6 0 120050 0
[pid=2843] ppid=2839 vsize=137696 CPUtime=1122.2 cores=3
/proc/2843/stat : 2843 (lingeling) R 2839 2839 2100 0 -1 4202496 39538 0 0 0 112214 6 0 0 25 0 1 0 514979970 141000704 33590 33554432000 4194304 4352236 140734773883328 18446744073709551615 4328068 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2843/statm: 34424 33590 126 39 0 33504 0
[pid=2844] ppid=2839 vsize=889888 CPUtime=558.12 cores=7
/proc/2844/stat : 2844 (march_hi) R 2839 2839 2100 0 -1 4202496 538502 0 0 0 55723 89 0 0 25 0 1 0 514979970 911245312 215976 33554432000 4194304 4331988 140734209786144 18446744073709551615 4228243 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2844/statm: 222472 215976 175 34 0 220914 0
Current children cumulated CPU time (s) 4483.04
Current children cumulated vsize (KiB) 2825488

[startup+1182.3 s]
/proc/loadavg: 10.39 10.31 10.01 11/192 2916
/proc/meminfo: memFree=22988288/32951124 swapFree=67111352/67111528
[pid=2839] ppid=2837 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2839/stat : 2839 (ppfolio) S 2837 2839 2100 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 514979969 18763776 390 33554432000 4194304 4287625 140734618231952 18446744073709551615 246516854287 0 0 4096 65536 4404854966276063231 0 0 17 5 0 0 0
/proc/2839/statm: 4581 390 341 23 0 91 0
[pid=2840] ppid=2839 vsize=918752 CPUtime=1182.32 cores=1
/proc/2840/stat : 2840 (cryptominisat) R 2839 2839 2100 0 -1 4202496 958215 0 0 0 117939 293 0 0 25 0 1 0 514979970 940802048 30920 33554432000 4194304 4587354 140736981477968 18446744073709551615 246516530840 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2840/statm: 229688 30920 337 96 0 226250 0
[pid=2841] ppid=2839 vsize=448988 CPUtime=1182.32 cores=5
/proc/2841/stat : 2841 (clasp) R 2839 2839 2100 0 -1 4202496 106857 0 0 0 118199 33 0 0 25 0 1 0 514979970 459763712 104040 33554432000 4194304 4785740 140737466685840 18446744073709551615 4553411 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2841/statm: 112247 104040 340 145 0 109292 0
[pid=2842] ppid=2839 vsize=485536 CPUtime=587.95 cores=7
/proc/2842/stat : 2842 (TNM) R 2839 2839 2100 0 -1 4202496 12884 0 0 0 58793 2 0 0 25 0 1 0 514979970 497188864 12780 33554432000 134512640 134534564 4287287376 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2842/statm: 121384 12780 95 6 0 120050 0
[pid=2843] ppid=2839 vsize=137696 CPUtime=1182.2 cores=3
/proc/2843/stat : 2843 (lingeling) R 2839 2839 2100 0 -1 4202496 39538 0 0 0 118214 6 0 0 25 0 1 0 514979970 141000704 33590 33554432000 4194304 4352236 140734773883328 18446744073709551615 4329185 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2843/statm: 34424 33590 126 39 0 33504 0
[pid=2844] ppid=2839 vsize=889888 CPUtime=587.91 cores=7
/proc/2844/stat : 2844 (march_hi) R 2839 2839 2100 0 -1 4202496 538502 0 0 0 58702 89 0 0 25 0 1 0 514979970 911245312 215976 33554432000 4194304 4331988 140734209786144 18446744073709551615 4315818 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2844/statm: 222472 215976 175 34 0 220914 0
Current children cumulated CPU time (s) 4722.7
Current children cumulated vsize (KiB) 2899184



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 10.28 10.29 10.01 12/192 2917
/proc/meminfo: memFree=22952428/32951124 swapFree=67111352/67111528
[pid=2839] ppid=2837 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2839/stat : 2839 (ppfolio) S 2837 2839 2100 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 514979969 18763776 390 33554432000 4194304 4287625 140734618231952 18446744073709551615 246516854287 0 0 4096 65536 4404854966276063231 0 0 17 5 0 0 0
/proc/2839/statm: 4581 390 341 23 0 91 0
[pid=2840] ppid=2839 vsize=918752 CPUtime=1201.73 cores=1
/proc/2840/stat : 2840 (cryptominisat) R 2839 2839 2100 0 -1 4202496 964038 0 0 0 119876 297 0 0 25 0 1 0 514979970 940802048 36697 33554432000 4194304 4587354 140736981477968 18446744073709551615 4238752 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2840/statm: 229688 36697 337 96 0 226250 0
[pid=2841] ppid=2839 vsize=451068 CPUtime=1201.72 cores=5
/proc/2841/stat : 2841 (clasp) R 2839 2839 2100 0 -1 4202496 107372 0 0 0 120139 33 0 0 25 0 1 0 514979970 461893632 104555 33554432000 4194304 4785740 140737466685840 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2841/statm: 112767 104555 340 145 0 109812 0
[pid=2842] ppid=2839 vsize=485536 CPUtime=597.55 cores=7
/proc/2842/stat : 2842 (TNM) R 2839 2839 2100 0 -1 4202496 12884 0 0 0 59753 2 0 0 25 0 1 0 514979970 497188864 12780 33554432000 134512640 134534564 4287287376 18446744073709551615 134528957 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2842/statm: 121384 12780 95 6 0 120050 0
[pid=2843] ppid=2839 vsize=137696 CPUtime=1201.6 cores=3
/proc/2843/stat : 2843 (lingeling) R 2839 2839 2100 0 -1 4202496 39538 0 0 0 120154 6 0 0 25 0 1 0 514979970 141000704 33590 33554432000 4194304 4352236 140734773883328 18446744073709551615 4328456 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2843/statm: 34424 33590 126 39 0 33504 0
[pid=2844] ppid=2839 vsize=889888 CPUtime=597.61 cores=7
/proc/2844/stat : 2844 (march_hi) R 2839 2839 2100 0 -1 4202496 538625 0 0 0 59672 89 0 0 25 0 1 0 514979970 911245312 216099 33554432000 4194304 4331988 140734209786144 18446744073709551615 4226474 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2844/statm: 222472 216099 175 34 0 220914 0
Current children cumulated CPU time (s) 4800.21
Current children cumulated vsize (KiB) 2901264

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.7 s]
/proc/loadavg: 10.28 10.29 10.01 12/192 2917
/proc/meminfo: memFree=22952428/32951124 swapFree=67111352/67111528
[pid=2839] ppid=2837 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/2839/stat : 2839 (ppfolio) S 2837 2839 2100 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 514979969 18763776 390 33554432000 4194304 4287625 140734618231952 18446744073709551615 246516854287 0 0 4096 65536 4404854966276063231 0 0 17 5 0 0 0
/proc/2839/statm: 4581 390 341 23 0 91 0
[pid=2840] ppid=2839 vsize=918752 CPUtime=1201.73 cores=1
/proc/2840/stat : 2840 (cryptominisat) R 2839 2839 2100 0 -1 4202496 964038 0 0 0 119876 297 0 0 25 0 1 0 514979970 940802048 36697 33554432000 4194304 4587354 140736981477968 18446744073709551615 4238752 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/2840/statm: 229688 36697 337 96 0 226250 0
[pid=2841] ppid=2839 vsize=451068 CPUtime=1201.72 cores=5
/proc/2841/stat : 2841 (clasp) R 2839 2839 2100 0 -1 4202496 107372 0 0 0 120139 33 0 0 25 0 1 0 514979970 461893632 104555 33554432000 4194304 4785740 140737466685840 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/2841/statm: 112767 104555 340 145 0 109812 0
[pid=2842] ppid=2839 vsize=485536 CPUtime=597.55 cores=7
/proc/2842/stat : 2842 (TNM) R 2839 2839 2100 0 -1 4202496 12884 0 0 0 59753 2 0 0 25 0 1 0 514979970 497188864 12780 33554432000 134512640 134534564 4287287376 18446744073709551615 134528957 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2842/statm: 121384 12780 95 6 0 120050 0
[pid=2843] ppid=2839 vsize=137696 CPUtime=1201.6 cores=3
/proc/2843/stat : 2843 (lingeling) R 2839 2839 2100 0 -1 4202496 39538 0 0 0 120154 6 0 0 25 0 1 0 514979970 141000704 33590 33554432000 4194304 4352236 140734773883328 18446744073709551615 4328456 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/2843/statm: 34424 33590 126 39 0 33504 0
[pid=2844] ppid=2839 vsize=889888 CPUtime=597.61 cores=7
/proc/2844/stat : 2844 (march_hi) R 2839 2839 2100 0 -1 4202496 538625 0 0 0 59672 89 0 0 25 0 1 0 514979970 911245312 216099 33554432000 4194304 4331988 140734209786144 18446744073709551615 4226474 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/2844/statm: 222472 216099 175 34 0 220914 0
Current children cumulated CPU time (s) 4800.21
Current children cumulated vsize (KiB) 2901264

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 2839 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=1999
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=1999
# CPU time returned by wait4() is 0.003998
# while last known CPU time is 4800.21
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1201.76
CPU time (s): 4800.21
CPU user time (s): 4795.94
CPU system time (s): 4.27
CPU usage (%): 399.433
Max. virtual memory (cumulated for all children) (KiB): 3650132

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.001999
system time used= 0.001999
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 524
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 266
involuntary context switches= 7

runsolver used 1.79073 second user time and 4.77427 second system time

The end

Launcher Data 

Begin job on node134 at 2011-04-29 01:36:58
IDJOB=3284016
IDBENCH=71095
IDSOLVER=1612
FILE ID=node134/3284016-1304033817
RUNJOBID= node134-1304032574-2555
PBS_JOBID= 13170318
Free space on /tmp= 71564 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT09/APPLICATIONS/satComp09_BioInstances/ndhf_xits_19_UNKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3284016-1304033817/watcher-3284016-1304033817 -o /tmp/evaluation-result-3284016-1304033817/solver-3284016-1304033817 -C 4800 -W 1300 -M 15500  ppfolio HOME/instance-3284016-1304033817.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 4cf2a693df1b6e5a084b88e84ec5f2af
RANDOM SEED=260726132

node134.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 7186.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      24718020 kB
Buffers:       1468716 kB
Cached:        5783240 kB
SwapCached:         64 kB
Active:        1796912 kB
Inactive:      5914136 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      24718020 kB
SwapTotal:    67111528 kB
SwapFree:     67111352 kB
Dirty:           17752 kB
Writeback:           4 kB
AnonPages:      458956 kB
Mapped:          16952 kB
Slab:           458840 kB
PageTables:       6336 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:  1890924 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71552 MiB
End job on node134 at 2011-04-29 01:57:01