Trace number 3251215

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 4800.15 1201.61

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m47-q841.5-n45-p30-s84954709.cnf
MD5SUM556c58061360a1b195d9b2f28888af77
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark2.03469
Satisfiable
(Un)Satisfiability was proved
Number of variables2115
Number of clauses356340
Sum of the clauses size714705
Maximum clause length47
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2356295
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 545

Solver Data 

0.00/0.06	This is pico-portfolio, a naive parallel portfolio
0.00/0.06	Copyright (C) 2011 Olivier Roussel
0.00/0.06	Using 4 processing units
0.00/0.06	This portfolio uses the following solvers:
0.00/0.06	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.06	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.06	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.06	[4] lingeling 276-6264d55-100731 (Armin Biere)
0.00/0.06	[5] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.06	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.06	Starting clasp 1.3.6 on core(s) 4
0.00/0.06	Starting TNM 2009 on core(s) 6
0.00/0.06	Starting lingeling 276-6264d55-100731 on core(s) 2
0.00/0.06	Starting march_hi 2009 on core(s) 6

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3251215-1304099267/watcher-3251215-1304099267 -o /tmp/evaluation-result-3251215-1304099267/solver-3251215-1304099267 -C 4800 -W 1300 -M 15500 ppfolio HOME/instance-3251215-1304099267.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.06 4.12 4.49 2/171 15753
/proc/meminfo: memFree=23082248/32951124 swapFree=67111524/67111528
[pid=15753] ppid=15751 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/15753/stat : 15753 (ppfolio) S 15751 15753 14150 0 -1 4202496 519 0 0 0 0 0 0 0 25 0 1 0 521527173 18763776 389 33554432000 4194304 4287625 140737424367184 18446744073709551615 244446965263 0 0 4096 65536 17970514393372295167 0 0 17 2 0 0 0
/proc/15753/statm: 4581 389 341 23 0 91 0

[startup+0.104536 s]
/proc/loadavg: 4.06 4.12 4.49 2/171 15753
/proc/meminfo: memFree=23082248/32951124 swapFree=67111524/67111528
[pid=15753] ppid=15751 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/15753/stat : 15753 (ppfolio) R 15751 15753 14150 0 -1 4202496 519 0 0 0 0 0 0 0 25 0 1 0 521527173 18763776 389 33554432000 4194304 4287625 140737424367184 18446744073709551615 244446965263 0 0 4096 65536 0 0 0 17 2 0 0 0
/proc/15753/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.20052 s]
/proc/loadavg: 4.06 4.12 4.49 2/171 15753
/proc/meminfo: memFree=23082248/32951124 swapFree=67111524/67111528
[pid=15753] ppid=15751 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/15753/stat : 15753 (ppfolio) S 15751 15753 14150 0 -1 4202496 519 0 0 0 0 0 0 0 24 0 1 0 521527173 18763776 389 33554432000 4194304 4287625 140737424367184 18446744073709551615 244446965263 0 0 4096 65536 17970514393372295167 0 0 17 2 0 0 0
/proc/15753/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300499 s]
/proc/loadavg: 4.06 4.12 4.49 2/171 15753
/proc/meminfo: memFree=23082248/32951124 swapFree=67111524/67111528
[pid=15753] ppid=15751 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/15753/stat : 15753 (ppfolio) S 15751 15753 14150 0 -1 4202496 519 0 0 0 0 0 0 0 24 0 1 0 521527173 18763776 389 33554432000 4194304 4287625 140737424367184 18446744073709551615 244446965263 0 0 4096 65536 17970514393372295167 0 0 17 2 0 0 0
/proc/15753/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700426 s]
/proc/loadavg: 4.06 4.12 4.49 2/171 15753
/proc/meminfo: memFree=23082248/32951124 swapFree=67111524/67111528
[pid=15753] ppid=15751 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/15753/stat : 15753 (ppfolio) S 15751 15753 14150 0 -1 4202496 519 0 0 0 0 0 0 0 24 0 1 0 521527173 18763776 389 33554432000 4194304 4287625 140737424367184 18446744073709551615 244446965263 0 0 4096 65536 17970514393372295167 0 0 17 2 0 0 0
/proc/15753/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50128 s]
/proc/loadavg: 4.06 4.12 4.49 6/177 15759
/proc/meminfo: memFree=22942700/32951124 swapFree=67111524/67111528
[pid=15753] ppid=15751 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/15753/stat : 15753 (ppfolio) S 15751 15753 14150 0 -1 4202496 519 0 0 0 0 0 0 0 16 0 1 0 521527173 18763776 389 33554432000 4194304 4287625 140737424367184 18446744073709551615 244446965263 0 0 4096 65536 17970514393372295167 0 0 17 2 0 0 0
/proc/15753/statm: 4581 389 341 23 0 91 0
[pid=15754] ppid=15753 vsize=77464 CPUtime=1.49 cores=0
/proc/15754/stat : 15754 (cryptominisat) R 15753 15753 14150 0 -1 4202496 18588 0 0 0 147 2 0 0 25 0 1 0 521527173 79323136 13606 33554432000 4194304 4587354 140735462241248 18446744073709551615 244446641862 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/15754/statm: 19366 13606 326 96 0 15928 0
[pid=15755] ppid=15753 vsize=35424 CPUtime=1.49 cores=4
/proc/15755/stat : 15755 (clasp) R 15753 15753 14150 0 -1 4202496 7731 0 0 0 149 0 0 0 25 0 1 0 521527173 36274176 5845 33554432000 4194304 4785740 140735380872560 18446744073709551615 4482898 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/15755/statm: 8856 5845 316 145 0 5901 0
[pid=15756] ppid=15753 vsize=468628 CPUtime=0.77 cores=6
/proc/15756/stat : 15756 (TNM) R 15753 15753 14150 0 -1 4202496 5821 0 0 0 77 0 0 0 25 0 1 0 521527173 479875072 5718 33554432000 134512640 134534564 4293628896 18446744073709551615 134524478 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/15756/statm: 117157 5718 91 6 0 115823 0
[pid=15757] ppid=15753 vsize=16964 CPUtime=1.48 cores=2
/proc/15757/stat : 15757 (lingeling) R 15753 15753 14150 0 -1 4202496 5314 0 0 0 148 0 0 0 25 0 1 0 521527173 17371136 3129 33554432000 4194304 4352236 140734604546832 18446744073709551615 4208191 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/15757/statm: 4241 3129 123 39 0 3321 0
[pid=15758] ppid=15753 vsize=41792 CPUtime=0.7 cores=6
/proc/15758/stat : 15758 (march_hi) R 15753 15753 14150 0 -1 4202496 21708 0 0 0 68 2 0 0 25 0 1 0 521527173 42795008 5533 33554432000 4194304 4331988 140733926804352 18446744073709551615 4257036 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/15758/statm: 10448 5533 150 34 0 8890 0
Current children cumulated CPU time (s) 5.93
Current children cumulated vsize (KiB) 658596

[startup+3.10098 s]
/proc/loadavg: 4.06 4.12 4.49 6/177 15759
/proc/meminfo: memFree=22943816/32951124 swapFree=67111524/67111528
[pid=15753] ppid=15751 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/15753/stat : 15753 (ppfolio) S 15751 15753 14150 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 521527173 18763776 389 33554432000 4194304 4287625 140737424367184 18446744073709551615 244446965263 0 0 4096 65536 17970514393372295167 0 0 17 2 0 0 0
/proc/15753/statm: 4581 389 341 23 0 91 0
[pid=15754] ppid=15753 vsize=77464 CPUtime=3.09 cores=0
/proc/15754/stat : 15754 (cryptominisat) R 15753 15753 14150 0 -1 4202496 19513 0 0 0 307 2 0 0 25 0 1 0 521527173 79323136 14518 33554432000 4194304 4587354 140735462241248 18446744073709551615 4236860 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/15754/statm: 19366 14518 326 96 0 15928 0
[pid=15755] ppid=15753 vsize=33888 CPUtime=3.09 cores=4
/proc/15755/stat : 15755 (clasp) R 15753 15753 14150 0 -1 4202496 8297 0 0 0 308 1 0 0 25 0 1 0 521527173 34701312 5715 33554432000 4194304 4785740 140735380872560 18446744073709551615 4287028 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/15755/statm: 8472 5715 340 145 0 5517 0
[pid=15756] ppid=15753 vsize=468628 CPUtime=1.56 cores=6
/proc/15756/stat : 15756 (TNM) R 15753 15753 14150 0 -1 4202496 5821 0 0 0 156 0 0 0 25 0 1 0 521527173 479875072 5718 33554432000 134512640 134534564 4293628896 18446744073709551615 134524691 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/15756/statm: 117157 5718 91 6 0 115823 0
[pid=15757] ppid=15753 vsize=19084 CPUtime=3.09 cores=2
/proc/15757/stat : 15757 (lingeling) R 15753 15753 14150 0 -1 4202496 6116 0 0 0 308 1 0 0 25 0 1 0 521527173 19542016 3931 33554432000 4194304 4352236 140734604546832 18446744073709551615 4327845 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/15757/statm: 4771 3931 126 39 0 3851 0
[pid=15758] ppid=15753 vsize=54304 CPUtime=1.5 cores=6
/proc/15758/stat : 15758 (march_hi) R 15753 15753 14150 0 -1 4202496 28081 0 0 0 147 3 0 0 25 0 1 0 521527173 55607296 6172 33554432000 4194304 4331988 140733926804352 18446744073709551615 4228247 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/15758/statm: 13576 6172 171 34 0 12018 0
Current children cumulated CPU time (s) 12.33
Current children cumulated vsize (KiB) 671692

[startup+6.30038 s]
/proc/loadavg: 4.22 4.15 4.50 6/177 15759
/proc/meminfo: memFree=22918764/32951124 swapFree=67111524/67111528
[pid=15753] ppid=15751 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/15753/stat : 15753 (ppfolio) S 15751 15753 14150 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 521527173 18763776 389 33554432000 4194304 4287625 140737424367184 18446744073709551615 244446965263 0 0 4096 65536 17970514393372295167 0 0 17 2 0 0 0
/proc/15753/statm: 4581 389 341 23 0 91 0
[pid=15754] ppid=15753 vsize=1437212 CPUtime=6.28 cores=0
/proc/15754/stat : 15754 (cryptominisat) R 15753 15753 14150 0 -1 4202496 21506 0 0 0 626 2 0 0 25 0 1 0 521527173 1471705088 14664 33554432000 4194304 4587354 140735462241248 18446744073709551615 4365216 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/15754/statm: 359303 14664 337 96 0 355865 0
[pid=15755] ppid=15753 vsize=53480 CPUtime=6.29 cores=4
/proc/15755/stat : 15755 (clasp) R 15753 15753 14150 0 -1 4202496 13262 0 0 0 627 2 0 0 25 0 1 0 521527173 54763520 10680 33554432000 4194304 4785740 140735380872560 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/15755/statm: 13370 10680 340 145 0 10415 0
[pid=15756] ppid=15753 vsize=471400 CPUtime=3.14 cores=6
/proc/15756/stat : 15756 (TNM) R 15753 15753 14150 0 -1 4202496 8419 0 0 0 314 0 0 0 25 0 1 0 521527173 482713600 8316 33554432000 134512640 134534564 4293628896 18446744073709551615 134528944 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/15756/statm: 117850 8316 95 6 0 116516 0
[pid=15757] ppid=15753 vsize=27788 CPUtime=6.29 cores=2
/proc/15757/stat : 15757 (lingeling) R 15753 15753 14150 0 -1 4202496 7781 0 0 0 628 1 0 0 25 0 1 0 521527173 28454912 5596 33554432000 4194304 4352236 140734604546832 18446744073709551615 4328184 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/15757/statm: 6947 5596 126 39 0 6027 0
[pid=15758] ppid=15753 vsize=54304 CPUtime=3.1 cores=6
/proc/15758/stat : 15758 (march_hi) R 15753 15753 14150 0 -1 4202496 28105 0 0 0 307 3 0 0 25 0 1 0 521527173 55607296 6196 33554432000 4194304 4331988 140733926804352 18446744073709551615 4226645 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/15758/statm: 13576 6196 171 34 0 12018 0
Current children cumulated CPU time (s) 25.1
Current children cumulated vsize (KiB) 2062508

[startup+12.7012 s]
/proc/loadavg: 4.28 4.16 4.50 6/177 15759
/proc/meminfo: memFree=22868084/32951124 swapFree=67111524/67111528
[pid=15753] ppid=15751 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/15753/stat : 15753 (ppfolio) S 15751 15753 14150 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 521527173 18763776 389 33554432000 4194304 4287625 140737424367184 18446744073709551615 244446965263 0 0 4096 65536 17970514393372295167 0 0 17 2 0 0 0
/proc/15753/statm: 4581 389 341 23 0 91 0
[pid=15754] ppid=15753 vsize=1437212 CPUtime=12.68 cores=0
/proc/15754/stat : 15754 (cryptominisat) R 15753 15753 14150 0 -1 4202496 21506 0 0 0 1266 2 0 0 25 0 1 0 521527173 1471705088 14664 33554432000 4194304 4587354 140735462241248 18446744073709551615 4365241 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/15754/statm: 359303 14664 337 96 0 355865 0
[pid=15755] ppid=15753 vsize=86228 CPUtime=12.69 cores=4
/proc/15755/stat : 15755 (clasp) R 15753 15753 14150 0 -1 4202496 21341 0 0 0 1266 3 0 0 25 0 1 0 521527173 88297472 18759 33554432000 4194304 4785740 140735380872560 18446744073709551615 4286912 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/15755/statm: 21557 18759 340 145 0 18602 0
[pid=15756] ppid=15753 vsize=471400 CPUtime=6.33 cores=6
/proc/15756/stat : 15756 (TNM) R 15753 15753 14150 0 -1 4202496 8419 0 0 0 633 0 0 0 25 0 1 0 521527173 482713600 8316 33554432000 134512640 134534564 4293628896 18446744073709551615 134529042 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/15756/statm: 117850 8316 95 6 0 116516 0
[pid=15757] ppid=15753 vsize=31884 CPUtime=12.69 cores=2
/proc/15757/stat : 15757 (lingeling) R 15753 15753 14150 0 -1 4202496 9061 0 0 0 1268 1 0 0 25 0 1 0 521527173 32649216 6876 33554432000 4194304 4352236 140734604546832 18446744073709551615 4207631 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/15757/statm: 7971 6876 126 39 0 7051 0
[pid=15758] ppid=15753 vsize=54900 CPUtime=6.28 cores=6
/proc/15758/stat : 15758 (march_hi) R 15753 15753 14150 0 -1 4202496 28259 0 0 0 625 3 0 0 25 0 1 0 521527173 56217600 6350 33554432000 4194304 4331988 140733926804352 18446744073709551615 4226400 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/15758/statm: 13725 6350 174 34 0 12167 0
Current children cumulated CPU time (s) 50.67
Current children cumulated vsize (KiB) 2099948

[startup+25.5008 s]
/proc/loadavg: 4.44 4.20 4.51 6/177 15759
/proc/meminfo: memFree=22784988/32951124 swapFree=67111524/67111528
[pid=15753] ppid=15751 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/15753/stat : 15753 (ppfolio) S 15751 15753 14150 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 521527173 18763776 389 33554432000 4194304 4287625 140737424367184 18446744073709551615 244446965263 0 0 4096 65536 17970514393372295167 0 0 17 2 0 0 0
/proc/15753/statm: 4581 389 341 23 0 91 0
[pid=15754] ppid=15753 vsize=1468360 CPUtime=25.48 cores=0
/proc/15754/stat : 15754 (cryptominisat) R 15753 15753 14150 0 -1 4202496 28908 0 0 0 2545 3 0 0 25 0 1 0 521527173 1503600640 22066 33554432000 4194304 4587354 140735462241248 18446744073709551615 4388277 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/15754/statm: 367090 22066 337 96 0 363652 0
[pid=15755] ppid=15753 vsize=130364 CPUtime=25.49 cores=4
/proc/15755/stat : 15755 (clasp) R 15753 15753 14150 0 -1 4202496 32160 0 0 0 2542 7 0 0 25 0 1 0 521527173 133492736 29578 33554432000 4194304 4785740 140735380872560 18446744073709551615 4287237 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/15755/statm: 32591 29578 340 145 0 29636 0
[pid=15756] ppid=15753 vsize=471400 CPUtime=12.73 cores=6
/proc/15756/stat : 15756 (TNM) R 15753 15753 14150 0 -1 4202496 8419 0 0 0 1273 0 0 0 25 0 1 0 521527173 482713600 8316 33554432000 134512640 134534564 4293628896 18446744073709551615 134528302 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/15756/statm: 117850 8316 95 6 0 116516 0
[pid=15757] ppid=15753 vsize=40240 CPUtime=25.48 cores=2
/proc/15757/stat : 15757 (lingeling) R 15753 15753 14150 0 -1 4202496 10894 0 0 0 2547 1 0 0 25 0 1 0 521527173 41205760 8709 33554432000 4194304 4352236 140734604546832 18446744073709551615 4328102 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/15757/statm: 10060 8709 126 39 0 9140 0
[pid=15758] ppid=15753 vsize=54904 CPUtime=12.61 cores=6
/proc/15758/stat : 15758 (march_hi) R 15753 15753 14150 0 -1 4202496 28267 0 0 0 1258 3 0 0 25 0 1 0 521527173 56221696 6358 33554432000 4194304 4331988 140733926804352 18446744073709551615 4228247 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/15758/statm: 13726 6358 174 34 0 12168 0
Current children cumulated CPU time (s) 101.79
Current children cumulated vsize (KiB) 2183592

[startup+51.101 s]
/proc/loadavg: 4.63 4.27 4.52 6/177 15760
/proc/meminfo: memFree=22655812/32951124 swapFree=67111524/67111528
[pid=15753] ppid=15751 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/15753/stat : 15753 (ppfolio) S 15751 15753 14150 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 521527173 18763776 389 33554432000 4194304 4287625 140737424367184 18446744073709551615 244446965263 0 0 4096 65536 17970514393372295167 0 0 17 2 0 0 0
/proc/15753/statm: 4581 389 341 23 0 91 0
[pid=15754] ppid=15753 vsize=1501408 CPUtime=51.08 cores=0
/proc/15754/stat : 15754 (cryptominisat) R 15753 15753 14150 0 -1 4202496 45062 0 0 0 5103 5 0 0 25 0 1 0 521527173 1537441792 31126 33554432000 4194304 4587354 140735462241248 18446744073709551615 4388277 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/15754/statm: 375352 31126 337 96 0 371914 0
[pid=15755] ppid=15753 vsize=192316 CPUtime=51.09 cores=4
/proc/15755/stat : 15755 (clasp) R 15753 15753 14150 0 -1 4202496 47251 0 0 0 5096 13 0 0 25 0 1 0 521527173 196931584 44669 33554432000 4194304 4785740 140735380872560 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/15755/statm: 48079 44669 340 145 0 45124 0
[pid=15756] ppid=15753 vsize=471400 CPUtime=25.43 cores=6
/proc/15756/stat : 15756 (TNM) R 15753 15753 14150 0 -1 4202496 8419 0 0 0 2543 0 0 0 25 0 1 0 521527173 482713600 8316 33554432000 134512640 134534564 4293628896 18446744073709551615 134530573 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/15756/statm: 117850 8316 95 6 0 116516 0
[pid=15757] ppid=15753 vsize=70092 CPUtime=51.09 cores=2

################
# More data... #
################

/proc/15758/statm: 13792 6403 174 34 0 12234 0
Current children cumulated CPU time (s) 3764.29
Current children cumulated vsize (KiB) 2778564

[startup+1002.3 s]
/proc/loadavg: 5.08 5.05 4.86 6/177 15797
/proc/meminfo: memFree=22161644/32951124 swapFree=67111524/67111528
[pid=15753] ppid=15751 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/15753/stat : 15753 (ppfolio) S 15751 15753 14150 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 521527173 18763776 389 33554432000 4194304 4287625 140737424367184 18446744073709551615 244446965263 0 0 4096 65536 17970514393372295167 0 0 17 2 0 0 0
/proc/15753/statm: 4581 389 341 23 0 91 0
[pid=15754] ppid=15753 vsize=1607020 CPUtime=1002.31 cores=0
/proc/15754/stat : 15754 (cryptominisat) R 15753 15753 14150 0 -1 4202496 286883 0 0 0 100127 104 0 0 25 0 1 0 521527173 1645588480 63399 33554432000 4194304 4587354 140735462241248 18446744073709551615 4232256 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/15754/statm: 401755 63399 337 96 0 398317 0
[pid=15755] ppid=15753 vsize=485312 CPUtime=1002.32 cores=4
/proc/15755/stat : 15755 (clasp) R 15753 15753 14150 0 -1 4202496 116530 0 0 0 100198 34 0 0 25 0 1 0 521527173 496959488 113948 33554432000 4194304 4785740 140735380872560 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/15755/statm: 121328 113948 340 145 0 118373 0
[pid=15756] ppid=15753 vsize=471400 CPUtime=498.56 cores=6
/proc/15756/stat : 15756 (TNM) R 15753 15753 14150 0 -1 4202496 8419 0 0 0 49855 1 0 0 25 0 1 0 521527173 482713600 8316 33554432000 134512640 134534564 4293628896 18446744073709551615 134530367 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/15756/statm: 117850 8316 95 6 0 116516 0
[pid=15757] ppid=15753 vsize=156128 CPUtime=1002.32 cores=2
/proc/15757/stat : 15757 (lingeling) R 15753 15753 14150 0 -1 4202496 137657 0 0 0 100213 19 0 0 25 0 1 0 521527173 159875072 38192 33554432000 4194304 4352236 140734604546832 18446744073709551615 4328194 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/15757/statm: 39032 38192 126 39 0 38112 0
[pid=15758] ppid=15753 vsize=55168 CPUtime=498.45 cores=6
/proc/15758/stat : 15758 (march_hi) R 15753 15753 14150 0 -1 4202496 28312 0 0 0 49841 4 0 0 25 0 1 0 521527173 56492032 6403 33554432000 4194304 4331988 140733926804352 18446744073709551615 4228247 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/15758/statm: 13792 6403 174 34 0 12234 0
Current children cumulated CPU time (s) 4003.96
Current children cumulated vsize (KiB) 2793352

[startup+1062.3 s]
/proc/loadavg: 5.11 5.07 4.88 6/177 15799
/proc/meminfo: memFree=22144860/32951124 swapFree=67111524/67111528
[pid=15753] ppid=15751 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/15753/stat : 15753 (ppfolio) S 15751 15753 14150 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 521527173 18763776 389 33554432000 4194304 4287625 140737424367184 18446744073709551615 244446965263 0 0 4096 65536 17970514393372295167 0 0 17 2 0 0 0
/proc/15753/statm: 4581 389 341 23 0 91 0
[pid=15754] ppid=15753 vsize=1606968 CPUtime=1062.31 cores=0
/proc/15754/stat : 15754 (cryptominisat) R 15753 15753 14150 0 -1 4202496 306753 0 0 0 106121 110 0 0 25 0 1 0 521527173 1645535232 61400 33554432000 4194304 4587354 140735462241248 18446744073709551615 4238658 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/15754/statm: 401742 61400 337 96 0 398304 0
[pid=15755] ppid=15753 vsize=522160 CPUtime=1062.32 cores=4
/proc/15755/stat : 15755 (clasp) R 15753 15753 14150 0 -1 4202496 125122 0 0 0 106196 36 0 0 25 0 1 0 521527173 534691840 122540 33554432000 4194304 4785740 140735380872560 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/15755/statm: 130540 122540 340 145 0 127585 0
[pid=15756] ppid=15753 vsize=471400 CPUtime=528.37 cores=6
/proc/15756/stat : 15756 (TNM) R 15753 15753 14150 0 -1 4202496 8419 0 0 0 52836 1 0 0 25 0 1 0 521527173 482713600 8316 33554432000 134512640 134534564 4293628896 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/15756/statm: 117850 8316 95 6 0 116516 0
[pid=15757] ppid=15753 vsize=147936 CPUtime=1062.33 cores=2
/proc/15757/stat : 15757 (lingeling) R 15753 15753 14150 0 -1 4202496 145337 0 0 0 106212 21 0 0 25 0 1 0 521527173 151486464 36144 33554432000 4194304 4352236 140734604546832 18446744073709551615 4328194 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/15757/statm: 36984 36144 126 39 0 36064 0
[pid=15758] ppid=15753 vsize=55168 CPUtime=528.32 cores=6
/proc/15758/stat : 15758 (march_hi) R 15753 15753 14150 0 -1 4202496 28313 0 0 0 52828 4 0 0 25 0 1 0 521527173 56492032 6404 33554432000 4194304 4331988 140733926804352 18446744073709551615 4228311 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/15758/statm: 13792 6404 174 34 0 12234 0
Current children cumulated CPU time (s) 4243.65
Current children cumulated vsize (KiB) 2821956

[startup+1122.3 s]
/proc/loadavg: 5.07 5.07 4.89 6/177 15801
/proc/meminfo: memFree=22111360/32951124 swapFree=67111524/67111528
[pid=15753] ppid=15751 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/15753/stat : 15753 (ppfolio) S 15751 15753 14150 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 521527173 18763776 389 33554432000 4194304 4287625 140737424367184 18446744073709551615 244446965263 0 0 4096 65536 17970514393372295167 0 0 17 2 0 0 0
/proc/15753/statm: 4581 389 341 23 0 91 0
[pid=15754] ppid=15753 vsize=1607020 CPUtime=1122.32 cores=0
/proc/15754/stat : 15754 (cryptominisat) R 15753 15753 14150 0 -1 4202496 324721 0 0 0 112113 119 0 0 25 0 1 0 521527173 1645588480 64189 33554432000 4194304 4587354 140735462241248 18446744073709551615 4238658 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/15754/statm: 401755 64189 337 96 0 398317 0
[pid=15755] ppid=15753 vsize=523300 CPUtime=1122.33 cores=4
/proc/15755/stat : 15755 (clasp) R 15753 15753 14150 0 -1 4202496 125366 0 0 0 112196 37 0 0 25 0 1 0 521527173 535859200 122784 33554432000 4194304 4785740 140735380872560 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/15755/statm: 130825 122784 340 145 0 127870 0
[pid=15756] ppid=15753 vsize=471400 CPUtime=558.25 cores=6
/proc/15756/stat : 15756 (TNM) R 15753 15753 14150 0 -1 4202496 8419 0 0 0 55824 1 0 0 25 0 1 0 521527173 482713600 8316 33554432000 134512640 134534564 4293628896 18446744073709551615 134528250 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/15756/statm: 117850 8316 95 6 0 116516 0
[pid=15757] ppid=15753 vsize=168416 CPUtime=1122.33 cores=2
/proc/15757/stat : 15757 (lingeling) R 15753 15753 14150 0 -1 4202496 157625 0 0 0 112211 22 0 0 25 0 1 0 521527173 172457984 41264 33554432000 4194304 4352236 140734604546832 18446744073709551615 4328100 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/15757/statm: 42104 41264 126 39 0 41184 0
[pid=15758] ppid=15753 vsize=55168 CPUtime=558.12 cores=6
/proc/15758/stat : 15758 (march_hi) R 15753 15753 14150 0 -1 4202496 28313 0 0 0 55808 4 0 0 25 0 1 0 521527173 56492032 6404 33554432000 4194304 4331988 140733926804352 18446744073709551615 4228247 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/15758/statm: 13792 6404 174 34 0 12234 0
Current children cumulated CPU time (s) 4483.35
Current children cumulated vsize (KiB) 2843628

[startup+1182.3 s]
/proc/loadavg: 5.06 5.07 4.90 6/177 15802
/proc/meminfo: memFree=22119772/32951124 swapFree=67111524/67111528
[pid=15753] ppid=15751 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/15753/stat : 15753 (ppfolio) S 15751 15753 14150 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 521527173 18763776 389 33554432000 4194304 4287625 140737424367184 18446744073709551615 244446965263 0 0 4096 65536 17970514393372295167 0 0 17 2 0 0 0
/proc/15753/statm: 4581 389 341 23 0 91 0
[pid=15754] ppid=15753 vsize=1606968 CPUtime=1182.32 cores=0
/proc/15754/stat : 15754 (cryptominisat) R 15753 15753 14150 0 -1 4202496 340938 0 0 0 118105 127 0 0 25 0 1 0 521527173 1645535232 59316 33554432000 4194304 4587354 140735462241248 18446744073709551615 4238593 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/15754/statm: 401742 59316 337 96 0 398304 0
[pid=15755] ppid=15753 vsize=525220 CPUtime=1182.33 cores=4
/proc/15755/stat : 15755 (clasp) R 15753 15753 14150 0 -1 4202496 125821 0 0 0 118196 37 0 0 25 0 1 0 521527173 537825280 123239 33554432000 4194304 4785740 140735380872560 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/15755/statm: 131305 123239 340 145 0 128350 0
[pid=15756] ppid=15753 vsize=471400 CPUtime=588.05 cores=6
/proc/15756/stat : 15756 (TNM) R 15753 15753 14150 0 -1 4202496 8419 0 0 0 58804 1 0 0 25 0 1 0 521527173 482713600 8316 33554432000 134512640 134534564 4293628896 18446744073709551615 134528288 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/15756/statm: 117850 8316 95 6 0 116516 0
[pid=15757] ppid=15753 vsize=156128 CPUtime=1182.33 cores=2
/proc/15757/stat : 15757 (lingeling) R 15753 15753 14150 0 -1 4202496 162233 0 0 0 118210 23 0 0 25 0 1 0 521527173 159875072 38192 33554432000 4194304 4352236 140734604546832 18446744073709551615 4328308 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/15757/statm: 39032 38192 126 39 0 38112 0
[pid=15758] ppid=15753 vsize=55168 CPUtime=588 cores=6
/proc/15758/stat : 15758 (march_hi) R 15753 15753 14150 0 -1 4202496 28313 0 0 0 58796 4 0 0 25 0 1 0 521527173 56492032 6404 33554432000 4194304 4331988 140733926804352 18446744073709551615 4226450 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/15758/statm: 13792 6404 174 34 0 12234 0
Current children cumulated CPU time (s) 4723.03
Current children cumulated vsize (KiB) 2833208



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.6 s]
/proc/loadavg: 5.12 5.08 4.91 6/177 15803
/proc/meminfo: memFree=22096324/32951124 swapFree=67111524/67111528
[pid=15753] ppid=15751 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/15753/stat : 15753 (ppfolio) S 15751 15753 14150 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 521527173 18763776 389 33554432000 4194304 4287625 140737424367184 18446744073709551615 244446965263 0 0 4096 65536 17970514393372295167 0 0 17 2 0 0 0
/proc/15753/statm: 4581 389 341 23 0 91 0
[pid=15754] ppid=15753 vsize=1606968 CPUtime=1201.63 cores=0
/proc/15754/stat : 15754 (cryptominisat) R 15753 15753 14150 0 -1 4202496 346603 0 0 0 120034 129 0 0 25 0 1 0 521527173 1645535232 64921 33554432000 4194304 4587354 140735462241248 18446744073709551615 4228754 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/15754/statm: 401742 64921 337 96 0 398304 0
[pid=15755] ppid=15753 vsize=527296 CPUtime=1201.64 cores=4
/proc/15755/stat : 15755 (clasp) R 15753 15753 14150 0 -1 4202496 126223 0 0 0 120127 37 0 0 25 0 1 0 521527173 539951104 123641 33554432000 4194304 4785740 140735380872560 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/15755/statm: 131824 123641 340 145 0 128869 0
[pid=15756] ppid=15753 vsize=471400 CPUtime=597.65 cores=6
/proc/15756/stat : 15756 (TNM) R 15753 15753 14150 0 -1 4202496 8419 0 0 0 59764 1 0 0 25 0 1 0 521527173 482713600 8316 33554432000 134512640 134534564 4293628896 18446744073709551615 134530360 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/15756/statm: 117850 8316 95 6 0 116516 0
[pid=15757] ppid=15753 vsize=176608 CPUtime=1201.64 cores=2
/proc/15757/stat : 15757 (lingeling) R 15753 15753 14150 0 -1 4202496 167353 0 0 0 120140 24 0 0 25 0 1 0 521527173 180846592 43312 33554432000 4194304 4352236 140734604546832 18446744073709551615 4329060 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/15757/statm: 44152 43312 126 39 0 43232 0
[pid=15758] ppid=15753 vsize=55168 CPUtime=597.59 cores=6
/proc/15758/stat : 15758 (march_hi) R 15753 15753 14150 0 -1 4202496 28313 0 0 0 59755 4 0 0 25 0 1 0 521527173 56492032 6404 33554432000 4194304 4331988 140733926804352 18446744073709551615 4286069 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/15758/statm: 13792 6404 174 34 0 12234 0
Current children cumulated CPU time (s) 4800.15
Current children cumulated vsize (KiB) 2855764

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.6 s]
/proc/loadavg: 5.12 5.08 4.91 6/177 15803
/proc/meminfo: memFree=22096324/32951124 swapFree=67111524/67111528
[pid=15753] ppid=15751 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/15753/stat : 15753 (ppfolio) S 15751 15753 14150 0 -1 4202496 520 0 0 0 0 0 0 0 15 0 1 0 521527173 18763776 389 33554432000 4194304 4287625 140737424367184 18446744073709551615 244446965263 0 0 4096 65536 17970514393372295167 0 0 17 2 0 0 0
/proc/15753/statm: 4581 389 341 23 0 91 0
[pid=15754] ppid=15753 vsize=1606968 CPUtime=1201.63 cores=0
/proc/15754/stat : 15754 (cryptominisat) R 15753 15753 14150 0 -1 4202496 346603 0 0 0 120034 129 0 0 25 0 1 0 521527173 1645535232 64921 33554432000 4194304 4587354 140735462241248 18446744073709551615 4228754 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/15754/statm: 401742 64921 337 96 0 398304 0
[pid=15755] ppid=15753 vsize=527296 CPUtime=1201.64 cores=4
/proc/15755/stat : 15755 (clasp) R 15753 15753 14150 0 -1 4202496 126223 0 0 0 120127 37 0 0 25 0 1 0 521527173 539951104 123641 33554432000 4194304 4785740 140735380872560 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/15755/statm: 131824 123641 340 145 0 128869 0
[pid=15756] ppid=15753 vsize=471400 CPUtime=597.65 cores=6
/proc/15756/stat : 15756 (TNM) R 15753 15753 14150 0 -1 4202496 8419 0 0 0 59764 1 0 0 25 0 1 0 521527173 482713600 8316 33554432000 134512640 134534564 4293628896 18446744073709551615 134530360 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/15756/statm: 117850 8316 95 6 0 116516 0
[pid=15757] ppid=15753 vsize=176608 CPUtime=1201.64 cores=2
/proc/15757/stat : 15757 (lingeling) R 15753 15753 14150 0 -1 4202496 167353 0 0 0 120140 24 0 0 25 0 1 0 521527173 180846592 43312 33554432000 4194304 4352236 140734604546832 18446744073709551615 4329060 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/15757/statm: 44152 43312 126 39 0 43232 0
[pid=15758] ppid=15753 vsize=55168 CPUtime=597.59 cores=6
/proc/15758/stat : 15758 (march_hi) R 15753 15753 14150 0 -1 4202496 28313 0 0 0 59755 4 0 0 25 0 1 0 521527173 56492032 6404 33554432000 4194304 4331988 140733926804352 18446744073709551615 4286069 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/15758/statm: 13792 6404 174 34 0 12234 0
Current children cumulated CPU time (s) 4800.15
Current children cumulated vsize (KiB) 2855764

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 15753 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=1999
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=999
# CPU time returned by wait4() is 0.002998
# while last known CPU time is 4800.15
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1201.61
CPU time (s): 4800.15
CPU user time (s): 4798.2
CPU system time (s): 1.95
CPU usage (%): 399.476
Max. virtual memory (cumulated for all children) (KiB): 4198788

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.001999
system time used= 0.000999
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 520
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 275
involuntary context switches= 6

runsolver used 2.17567 second user time and 4.21036 second system time

The end

Launcher Data 

Begin job on node105 at 2011-04-29 19:47:47
IDJOB=3251215
IDBENCH=88216
IDSOLVER=1612
FILE ID=node105/3251215-1304099267
RUNJOBID= node105-1304087809-14168
PBS_JOBID= 13170561
Free space on /tmp= 71936 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m47-q841.5-n45-p30-s84954709.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3251215-1304099267/watcher-3251215-1304099267 -o /tmp/evaluation-result-3251215-1304099267/solver-3251215-1304099267 -C 4800 -W 1300 -M 15500  ppfolio HOME/instance-3251215-1304099267.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 556c58061360a1b195d9b2f28888af77
RANDOM SEED=1141636884

node105.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5410.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      23082404 kB
Buffers:       2157360 kB
Cached:        7102096 kB
SwapCached:          4 kB
Active:         555200 kB
Inactive:      8767320 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      23082404 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:           14484 kB
Writeback:           0 kB
AnonPages:       62680 kB
Mapped:          14320 kB
Slab:           483748 kB
PageTables:       3828 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   184624 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71924 MiB
End job on node105 at 2011-04-29 20:07:51