Trace number 3293677

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio seq? (TO) 1298.15 1300.01

General information on the benchmark

NameSAT-Race-2010-CNF/hardware-verification/
velev/2dlx_ca_bp_f_liveness.cnf
MD5SUM99cd8de7f06ce8c087a5c7fb299dfb62
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables202253
Number of clauses4313014
Sum of the clauses size12562188
Maximum clause length651
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 23935640
Number of clauses of size 3199653
Number of clauses of size 410363
Number of clauses of size 55846
Number of clauses of size over 5161511

Solver Data 

0.00/1.88	This is pico-portfolio, a naive parallel portfolio
0.00/1.88	Copyright (C) 2011 Olivier Roussel
0.00/1.88	Using 1 processing units
0.00/1.88	This portfolio uses the following solvers:
0.00/1.88	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/1.88	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/1.88	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/1.88	Starting cryptominisat 2.7.1 on core(s) 5
0.00/1.88	Starting clasp 1.3.6 on core(s) 5
0.00/1.88	Starting TNM 2009 on core(s) 5

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3293677-1303787650/watcher-3293677-1303787650 -o /tmp/evaluation-result-3293677-1303787650/solver-3293677-1303787650 -C 2400 -W 1300 -M 7680 ppfolio --nbcore=1 HOME/instance-3293677-1303787650.cnf 

running on 2 cores: 5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 7864320 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 7915520 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.62 3.96 3.97 2/180 26874
/proc/meminfo: memFree=27328680/32950928 swapFree=67111364/67111528
[pid=26874] ppid=26872 vsize=11800 CPUtime=0 cores=5,7
/proc/26874/stat : 26874 (runsolver) S 26872 26874 24832 0 -1 4202560 42 0 0 0 0 0 0 0 17 0 1 0 490364685 12083200 51 33554432000 4194304 5434984 140736961371776 18446744073709551615 4881607 0 2147483391 4096 24578 18446744071699968823 0 0 17 5 0 0 0
/proc/26874/statm: 2950 51 31 303 0 2644 0

[startup+0.0202031 s]
/proc/loadavg: 3.62 3.96 3.97 2/180 26874
/proc/meminfo: memFree=27328680/32950928 swapFree=67111364/67111528
[pid=26874] ppid=26872 vsize=2712 CPUtime=0 cores=5,7
/proc/26874/stat : 26874 (ppfolio) S 26872 26874 24832 0 -1 4194304 84 0 0 0 0 0 0 0 18 0 1 0 490364685 2777088 37 33554432000 4194304 4287625 140735109517120 18446744073709551615 208075308679 0 2147483391 4096 0 18446744071699953811 0 0 17 7 0 0 0
/proc/26874/statm: 678 37 27 23 0 23 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 2712

[startup+0.100246 s]
/proc/loadavg: 3.62 3.96 3.97 2/180 26874
/proc/meminfo: memFree=27328680/32950928 swapFree=67111364/67111528
[pid=26874] ppid=26872 vsize=2712 CPUtime=0 cores=5,7
/proc/26874/stat : 26874 (ppfolio) S 26872 26874 24832 0 -1 4194304 84 0 0 0 0 0 0 0 18 0 1 0 490364685 2777088 37 33554432000 4194304 4287625 140735109517120 18446744073709551615 208075308679 0 2147483391 4096 0 18446744071699953811 0 0 17 7 0 0 0
/proc/26874/statm: 678 37 27 23 0 23 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 2712

[startup+0.30021 s]
/proc/loadavg: 3.62 3.96 3.97 2/180 26874
/proc/meminfo: memFree=27328680/32950928 swapFree=67111364/67111528
[pid=26874] ppid=26872 vsize=2712 CPUtime=0 cores=5,7
/proc/26874/stat : 26874 (ppfolio) S 26872 26874 24832 0 -1 4194304 84 0 0 0 0 0 0 0 18 0 1 0 490364685 2777088 37 33554432000 4194304 4287625 140735109517120 18446744073709551615 208075308679 0 2147483391 4096 0 18446744071699953811 0 0 17 7 0 0 0
/proc/26874/statm: 678 37 27 23 0 23 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 2712

[startup+0.701075 s]
/proc/loadavg: 3.62 3.96 3.97 2/180 26874
/proc/meminfo: memFree=27328680/32950928 swapFree=67111364/67111528
[pid=26874] ppid=26872 vsize=2712 CPUtime=0 cores=5,7
/proc/26874/stat : 26874 (ppfolio) S 26872 26874 24832 0 -1 4194304 84 0 0 0 0 0 0 0 18 0 1 0 490364685 2777088 37 33554432000 4194304 4287625 140735109517120 18446744073709551615 208075308679 0 2147483391 4096 0 18446744071699953811 0 0 17 7 0 0 0
/proc/26874/statm: 678 37 27 23 0 23 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 2712

[startup+1.50092 s]
/proc/loadavg: 3.62 3.96 3.97 5/188 26884
/proc/meminfo: memFree=27040716/32950928 swapFree=67111364/67111528
[pid=26874] ppid=26872 vsize=2712 CPUtime=0 cores=5,7
/proc/26874/stat : 26874 (ppfolio) S 26872 26874 24832 0 -1 4194304 84 0 0 0 0 0 0 0 18 0 1 0 490364685 2777088 37 33554432000 4194304 4287625 140735109517120 18446744073709551615 208075308679 0 2147483391 4096 0 18446744071699953811 0 0 17 7 0 0 0
/proc/26874/statm: 678 37 27 23 0 23 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 2712

[startup+3.10062 s]
/proc/loadavg: 3.62 3.96 3.97 9/201 26899
/proc/meminfo: memFree=26641952/32950928 swapFree=67111364/67111528
[pid=26874] ppid=26872 vsize=18324 CPUtime=0 cores=5,7
/proc/26874/stat : 26874 (ppfolio) S 26872 26874 24832 0 -1 4202496 500 0 0 0 0 0 0 0 20 0 1 0 490364685 18763776 394 33554432000 4194304 4287625 140735109517120 18446744073709551615 208080252431 0 0 4096 65536 0 0 0 17 7 0 0 0
/proc/26874/statm: 4581 394 345 23 0 91 0
[pid=26891] ppid=26874 vsize=215920 CPUtime=0.79 cores=5
/proc/26891/stat : 26891 (cryptominisat) R 26874 26874 24832 0 -1 4202496 40678 0 0 0 74 5 0 0 18 0 1 0 490364874 221102080 40310 33554432000 4194304 4587354 140733368448384 18446744073709551615 4298550 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/26891/statm: 53980 40310 287 96 0 50542 0
[pid=26894] ppid=26874 vsize=65488 CPUtime=0.4 cores=5
/proc/26894/stat : 26894 (clasp) R 26874 26874 24832 0 -1 4202496 14894 0 0 0 38 2 0 0 16 0 1 0 490364874 67059712 13302 33554432000 4194304 4785740 140737361410384 18446744073709551615 208079893189 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/26894/statm: 16372 13302 308 145 0 13417 0
[pid=26895] ppid=26874 vsize=4 CPUtime=0 cores=5
/proc/26895/stat : 26895 (TNM) R 26874 26874 24832 0 -1 4194304 93 0 0 0 0 0 0 0 21 0 1 0 490364874 4096 1 33554432000 0 0 0 18446744073709551615 208080053239 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/26895/statm: 1 1 0 0 0 1 0
Current children cumulated CPU time (s) 1.19
Current children cumulated vsize (KiB) 299736
heavy processes:

[startup+6.30101 s]
/proc/loadavg: 4.13 4.06 4.01 11/200 26900
/proc/meminfo: memFree=24981172/32950928 swapFree=67111364/67111528
[pid=26874] ppid=26872 vsize=18324 CPUtime=0 cores=5,7
/proc/26874/stat : 26874 (ppfolio) S 26872 26874 24832 0 -1 4202496 500 0 0 0 0 0 0 0 20 0 1 0 490364685 18763776 394 33554432000 4194304 4287625 140735109517120 18446744073709551615 208080252431 0 0 4096 65536 0 0 0 17 7 0 0 0
/proc/26874/statm: 4581 394 345 23 0 91 0
[pid=26891] ppid=26874 vsize=296676 CPUtime=1.24 cores=5
/proc/26891/stat : 26891 (cryptominisat) R 26874 26874 24832 0 -1 4202496 61506 0 0 0 115 9 0 0 19 0 1 0 490364874 303796224 60946 33554432000 4194304 4587354 140733368448384 18446744073709551615 208079890797 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/26891/statm: 74169 60946 287 96 0 70731 0
[pid=26894] ppid=26874 vsize=139552 CPUtime=2.81 cores=5
/proc/26894/stat : 26894 (clasp) R 26874 26874 24832 0 -1 4202496 37180 0 0 0 274 7 0 0 19 0 1 0 490364874 142901248 31587 33554432000 4194304 4785740 140737361410384 18446744073709551615 4550418 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/26894/statm: 34888 31587 336 145 0 31933 0
[pid=26895] ppid=26874 vsize=459920 CPUtime=0.33 cores=5
/proc/26895/stat : 26895 (TNM) R 26874 26874 24832 0 -1 4202496 3505 0 0 0 33 0 0 0 22 0 1 0 490364874 470958080 3403 33554432000 134512640 134534564 4289387536 18446744073709551615 4294960144 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/26895/statm: 114980 3404 90 6 0 113646 0
Current children cumulated CPU time (s) 4.38
Current children cumulated vsize (KiB) 914472
heavy processes:

[startup+12.7008 s]
/proc/loadavg: 4.84 4.21 4.05 11/200 26900
/proc/meminfo: memFree=23667804/32950928 swapFree=67111364/67111528
[pid=26874] ppid=26872 vsize=18324 CPUtime=0 cores=5,7
/proc/26874/stat : 26874 (ppfolio) S 26872 26874 24832 0 -1 4202496 500 0 0 0 0 0 0 0 15 0 1 0 490364685 18763776 394 33554432000 4194304 4287625 140735109517120 18446744073709551615 208080252431 0 0 4096 65536 14872755963978317824 0 0 17 7 0 0 0
/proc/26874/statm: 4581 394 345 23 0 91 0
[pid=26891] ppid=26874 vsize=509804 CPUtime=3.34 cores=5
/proc/26891/stat : 26891 (cryptominisat) R 26874 26874 24832 0 -1 4202496 113464 0 0 0 318 16 0 0 25 0 1 0 490364874 522039296 112717 33554432000 4194304 4587354 140733368448384 18446744073709551615 4363792 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/26891/statm: 127451 112717 309 96 0 124013 0
[pid=26894] ppid=26874 vsize=147348 CPUtime=4.95 cores=5
/proc/26894/stat : 26894 (clasp) R 26874 26874 24832 0 -1 4202496 39114 0 0 0 487 8 0 0 25 0 1 0 490364874 150884352 33521 33554432000 4194304 4785740 140737361410384 18446744073709551615 4552618 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/26894/statm: 36837 33521 336 145 0 33882 0
[pid=26895] ppid=26874 vsize=507836 CPUtime=2.5 cores=5
/proc/26895/stat : 26895 (TNM) R 26874 26874 24832 0 -1 4202496 22750 0 0 0 244 6 0 0 25 0 1 0 490364874 520024064 22648 33554432000 134512640 134534564 4289387536 18446744073709551615 8169633 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/26895/statm: 126959 22648 90 6 0 125625 0
Current children cumulated CPU time (s) 10.79
Current children cumulated vsize (KiB) 1183312

[startup+25.5004 s]
/proc/loadavg: 5.98 4.49 4.15 11/200 26900
/proc/meminfo: memFree=23203196/32950928 swapFree=67111364/67111528
[pid=26874] ppid=26872 vsize=18324 CPUtime=0 cores=5,7
/proc/26874/stat : 26874 (ppfolio) S 26872 26874 24832 0 -1 4202496 500 0 0 0 0 0 0 0 15 0 1 0 490364685 18763776 394 33554432000 4194304 4287625 140735109517120 18446744073709551615 208080252431 0 0 4096 65536 14872756101417271296 0 0 17 7 0 0 0
/proc/26874/statm: 4581 394 345 23 0 91 0
[pid=26891] ppid=26874 vsize=510376 CPUtime=7.64 cores=5
/proc/26891/stat : 26891 (cryptominisat) R 26874 26874 24832 0 -1 4202496 113598 0 0 0 748 16 0 0 25 0 1 0 490364874 522625024 112851 33554432000 4194304 4587354 140733368448384 18446744073709551615 4421069 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/26891/statm: 127594 112851 310 96 0 124156 0
[pid=26894] ppid=26874 vsize=155416 CPUtime=9.21 cores=5
/proc/26894/stat : 26894 (clasp) R 26874 26874 24832 0 -1 4202496 41024 0 0 0 913 8 0 0 25 0 1 0 490364874 159145984 35431 33554432000 4194304 4785740 140737361410384 18446744073709551615 4553003 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/26894/statm: 38854 35431 336 145 0 35899 0
[pid=26895] ppid=26874 vsize=720184 CPUtime=6.73 cores=5
/proc/26895/stat : 26895 (TNM) R 26874 26874 24832 0 -1 4202496 85171 0 0 0 660 13 0 0 25 0 1 0 490364874 737468416 85068 33554432000 134512640 134534564 4289387536 18446744073709551615 134524745 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/26895/statm: 180046 85068 91 6 0 178712 0
Current children cumulated CPU time (s) 23.58
Current children cumulated vsize (KiB) 1404300

[startup+51.1005 s]
/proc/loadavg: 7.43 4.95 4.31 12/201 26930
/proc/meminfo: memFree=23971928/32950928 swapFree=67111364/67111528
[pid=26874] ppid=26872 vsize=18324 CPUtime=0 cores=5,7
/proc/26874/stat : 26874 (ppfolio) S 26872 26874 24832 0 -1 4202496 500 0 0 0 0 0 0 0 15 0 1 0 490364685 18763776 394 33554432000 4194304 4287625 140735109517120 18446744073709551615 208080252431 0 0 4096 65536 0 0 0 17 7 0 0 0
/proc/26874/statm: 4581 394 345 23 0 91 0
[pid=26891] ppid=26874 vsize=803656 CPUtime=16.14 cores=5
/proc/26891/stat : 26891 (cryptominisat) R 26874 26874 24832 0 -1 4202496 283978 0 0 0 1576 38 0 0 25 0 1 0 490364874 822943744 177221 33554432000 4194304 4587354 140733368448384 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/26891/statm: 200914 177221 327 96 0 197476 0
[pid=26894] ppid=26874 vsize=167692 CPUtime=17.74 cores=5
/proc/26894/stat : 26894 (clasp) R 26874 26874 24832 0 -1 4202496 44062 0 0 0 1765 9 0 0 25 0 1 0 490364874 171716608 38469 33554432000 4194304 4785740 140737361410384 18446744073709551615 4552863 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/26894/statm: 41923 38469 336 145 0 38968 0
[pid=26895] ppid=26874 vsize=720184 CPUtime=15.3 cores=5
/proc/26895/stat : 26895 (TNM) R 26874 26874 24832 0 -1 4202496 85171 0 0 0 1516 14 0 0 25 0 1 0 490364874 737468416 85068 33554432000 134512640 134534564 4289387536 18446744073709551615 134524554 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/26895/statm: 180046 85068 91 6 0 178712 0
Current children cumulated CPU time (s) 49.18
Current children cumulated vsize (KiB) 1709856

[startup+102.301 s]
/proc/loadavg: 8.33 5.61 4.57 9/192 26977
/proc/meminfo: memFree=23816596/32950928 swapFree=67111364/67111528
[pid=26874] ppid=26872 vsize=18324 CPUtime=0 cores=5,7
/proc/26874/stat : 26874 (ppfolio) S 26872 26874 24832 0 -1 4202496 500 0 0 0 0 0 0 0 15 0 1 0 490364685 18763776 394 33554432000 4194304 4287625 140735109517120 18446744073709551615 208080252431 0 0 4096 65536 14872757338367852544 0 0 17 7 0 0 0
/proc/26874/statm: 4581 394 345 23 0 91 0
[pid=26891] ppid=26874 vsize=803656 CPUtime=33.23 cores=5
/proc/26891/stat : 26891 (cryptominisat) R 26874 26874 24832 0 -1 4202496 285153 0 0 0 3285 38 0 0 25 0 1 0 490364874 822943744 178386 33554432000 4194304 4587354 140733368448384 18446744073709551615 4237467 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/26891/statm: 200914 178386 327 96 0 197476 0
[pid=26894] ppid=26874 vsize=197056 CPUtime=34.8 cores=5
/proc/26894/stat : 26894 (clasp) R 26874 26874 24832 0 -1 4202496 51273 0 0 0 3469 11 0 0 25 0 1 0 490364874 201785344 45680 33554432000 4194304 4785740 140737361410384 18446744073709551615 4552550 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/26894/statm: 49264 45680 336 145 0 46309 0
[pid=26895] ppid=26874 vsize=720184 CPUtime=32.35 cores=5
/proc/26895/stat : 26895 (TNM) R 26874 26874 24832 0 -1 4202496 85171 0 0 0 3221 14 0 0 25 0 1 0 490364874 737468416 85068 33554432000 134512640 134534564 4289387536 18446744073709551615 134524712 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/26895/statm: 180046 85068 91 6 0 178712 0
Current children cumulated CPU time (s) 100.38
Current children cumulated vsize (KiB) 1739220

[startup+162.3 s]
/proc/loadavg: 8.12 6.04 4.78 9/192 26979
/proc/meminfo: memFree=22953636/32950928 swapFree=67111364/67111528
[pid=26874] ppid=26872 vsize=18324 CPUtime=0 cores=5,7
/proc/26874/stat : 26874 (ppfolio) S 26872 26874 24832 0 -1 4202496 500 0 0 0 0 0 0 0 15 0 1 0 490364685 18763776 394 33554432000 4194304 4287625 140735109517120 18446744073709551615 208080252431 0 0 4096 65536 14872756238856224768 0 0 17 7 0 0 0
/proc/26874/statm: 4581 394 345 23 0 91 0
[pid=26891] ppid=26874 vsize=803656 CPUtime=53.23 cores=5
/proc/26891/stat : 26891 (cryptominisat) R 26874 26874 24832 0 -1 4202496 286478 0 0 0 5284 39 0 0 25 0 1 0 490364874 822943744 179701 33554432000 4194304 4587354 140733368448384 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/26891/statm: 200914 179701 338 96 0 197476 0
[pid=26894] ppid=26874 vsize=242424 CPUtime=54.81 cores=5
/proc/26894/stat : 26894 (clasp) R 26874 26874 24832 0 -1 4202496 62552 0 0 0 5466 15 0 0 25 0 1 0 490364874 248242176 56959 33554432000 4194304 4785740 140737361410384 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/26894/statm: 60606 56959 336 145 0 57651 0
[pid=26895] ppid=26874 vsize=720184 CPUtime=52.36 cores=5
/proc/26895/stat : 26895 (TNM) R 26874 26874 24832 0 -1 4202496 85171 0 0 0 5222 14 0 0 25 0 1 0 490364874 737468416 85068 33554432000 134512640 134534564 4289387536 18446744073709551615 134524508 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/26895/statm: 180046 85068 91 6 0 178712 0
Current children cumulated CPU time (s) 160.4
Current children cumulated vsize (KiB) 1784588

################
# More data... #
################

/proc/26895/statm: 180046 85068 91 6 0 178712 0
Current children cumulated CPU time (s) 880.42
Current children cumulated vsize (KiB) 4358632

[startup+942.307 s]
/proc/loadavg: 8.00 7.85 6.59 9/192 27000
/proc/meminfo: memFree=20887108/32950928 swapFree=67111364/67111528
[pid=26874] ppid=26872 vsize=18324 CPUtime=0 cores=5,7
/proc/26874/stat : 26874 (ppfolio) S 26872 26874 24832 0 -1 4202496 500 0 0 0 0 0 0 0 15 0 1 0 490364685 18763776 394 33554432000 4194304 4287625 140735109517120 18446744073709551615 208080252431 0 0 4096 65536 14872755963978317824 0 0 17 7 0 0 0
/proc/26874/statm: 4581 394 345 23 0 91 0
[pid=26891] ppid=26874 vsize=2949272 CPUtime=313.2 cores=5
/proc/26891/stat : 26891 (cryptominisat) R 26874 26874 24832 0 -1 4202496 647703 0 0 0 31233 87 0 0 25 0 1 0 490364874 3020054528 182937 33554432000 4194304 4587354 140733368448384 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/26891/statm: 737318 182937 338 96 0 733880 0
[pid=26894] ppid=26874 vsize=612352 CPUtime=314.86 cores=5
/proc/26894/stat : 26894 (clasp) R 26874 26874 24832 0 -1 4202496 154486 0 0 0 31447 39 0 0 25 0 1 0 490364874 627048448 148893 33554432000 4194304 4785740 140737361410384 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/26894/statm: 153088 148893 336 145 0 150133 0
[pid=26895] ppid=26874 vsize=720184 CPUtime=312.36 cores=5
/proc/26895/stat : 26895 (TNM) R 26874 26874 24832 0 -1 4202496 85171 0 0 0 31222 14 0 0 25 0 1 0 490364874 737468416 85068 33554432000 134512640 134534564 4289387536 18446744073709551615 134524552 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/26895/statm: 180046 85068 91 6 0 178712 0
Current children cumulated CPU time (s) 940.42
Current children cumulated vsize (KiB) 4300132

[startup+1002.3 s]
/proc/loadavg: 8.00 7.88 6.68 10/192 27001
/proc/meminfo: memFree=20456776/32950928 swapFree=67111364/67111528
[pid=26874] ppid=26872 vsize=18324 CPUtime=0 cores=5,7
/proc/26874/stat : 26874 (ppfolio) S 26872 26874 24832 0 -1 4202496 500 0 0 0 0 0 0 0 15 0 1 0 490364685 18763776 394 33554432000 4194304 4287625 140735109517120 18446744073709551615 208080252431 0 0 4096 65536 14872756238856224768 0 0 17 7 0 0 0
/proc/26874/statm: 4581 394 345 23 0 91 0
[pid=26891] ppid=26874 vsize=2949272 CPUtime=333.21 cores=5
/proc/26891/stat : 26891 (cryptominisat) R 26874 26874 24832 0 -1 4202496 649201 0 0 0 33233 88 0 0 25 0 1 0 490364874 3020054528 184421 33554432000 4194304 4587354 140733368448384 18446744073709551615 4237695 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/26891/statm: 737318 184421 338 96 0 733880 0
[pid=26894] ppid=26874 vsize=645648 CPUtime=334.86 cores=5
/proc/26894/stat : 26894 (clasp) R 26874 26874 24832 0 -1 4202496 162775 0 0 0 33445 41 0 0 25 0 1 0 490364874 661143552 157182 33554432000 4194304 4785740 140737361410384 18446744073709551615 4552993 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/26894/statm: 161412 157182 336 145 0 158457 0
[pid=26895] ppid=26874 vsize=720184 CPUtime=332.36 cores=5
/proc/26895/stat : 26895 (TNM) R 26874 26874 24832 0 -1 4202496 85171 0 0 0 33222 14 0 0 25 0 1 0 490364874 737468416 85068 33554432000 134512640 134534564 4289387536 18446744073709551615 134524664 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/26895/statm: 180046 85068 91 6 0 178712 0
Current children cumulated CPU time (s) 1000.43
Current children cumulated vsize (KiB) 4333428

[startup+1062.3 s]
/proc/loadavg: 8.04 7.91 6.76 9/192 27003
/proc/meminfo: memFree=19932824/32950928 swapFree=67111364/67111528
[pid=26874] ppid=26872 vsize=18324 CPUtime=0 cores=5,7
/proc/26874/stat : 26874 (ppfolio) S 26872 26874 24832 0 -1 4202496 500 0 0 0 0 0 0 0 15 0 1 0 490364685 18763776 394 33554432000 4194304 4287625 140735109517120 18446744073709551615 208080252431 0 0 4096 65536 14872756926050992128 0 0 17 7 0 0 0
/proc/26874/statm: 4581 394 345 23 0 91 0
[pid=26891] ppid=26874 vsize=2949272 CPUtime=353.22 cores=5
/proc/26891/stat : 26891 (cryptominisat) R 26874 26874 24832 0 -1 4202496 649964 0 0 0 35233 89 0 0 25 0 1 0 490364874 3020054528 185179 33554432000 4194304 4587354 140733368448384 18446744073709551615 4449070 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/26891/statm: 737318 185179 338 96 0 733880 0
[pid=26894] ppid=26874 vsize=697632 CPUtime=354.86 cores=5
/proc/26894/stat : 26894 (clasp) R 26874 26874 24832 0 -1 4202496 175580 0 0 0 35441 45 0 0 25 0 1 0 490364874 714375168 169987 33554432000 4194304 4785740 140737361410384 18446744073709551615 4552628 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/26894/statm: 174408 169987 336 145 0 171453 0
[pid=26895] ppid=26874 vsize=720184 CPUtime=352.35 cores=5
/proc/26895/stat : 26895 (TNM) R 26874 26874 24832 0 -1 4202496 85171 0 0 0 35221 14 0 0 25 0 1 0 490364874 737468416 85068 33554432000 134512640 134534564 4289387536 18446744073709551615 134524712 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/26895/statm: 180046 85068 91 6 0 178712 0
Current children cumulated CPU time (s) 1060.43
Current children cumulated vsize (KiB) 4385412

[startup+1122.3 s]
/proc/loadavg: 8.01 7.92 6.84 9/192 27004
/proc/meminfo: memFree=19358052/32950928 swapFree=67111364/67111528
[pid=26874] ppid=26872 vsize=18324 CPUtime=0 cores=5,7
/proc/26874/stat : 26874 (ppfolio) S 26872 26874 24832 0 -1 4202496 500 0 0 0 0 0 0 0 15 0 1 0 490364685 18763776 394 33554432000 4194304 4287625 140735109517120 18446744073709551615 208080252431 0 0 4096 65536 14872755414222503936 0 0 17 7 0 0 0
/proc/26874/statm: 4581 394 345 23 0 91 0
[pid=26891] ppid=26874 vsize=2949272 CPUtime=373.23 cores=5
/proc/26891/stat : 26891 (cryptominisat) R 26874 26874 24832 0 -1 4202496 650050 0 0 0 37234 89 0 0 25 0 1 0 490364874 3020054528 185265 33554432000 4194304 4587354 140733368448384 18446744073709551615 4430380 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/26891/statm: 737318 185265 338 96 0 733880 0
[pid=26894] ppid=26874 vsize=765220 CPUtime=374.85 cores=5
/proc/26894/stat : 26894 (clasp) R 26874 26874 24832 0 -1 4202496 192489 0 0 0 37436 49 0 0 25 0 1 0 490364874 783585280 186896 33554432000 4194304 4785740 140737361410384 18446744073709551615 4553293 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/26894/statm: 191305 186896 336 145 0 188350 0
[pid=26895] ppid=26874 vsize=720184 CPUtime=372.35 cores=5
/proc/26895/stat : 26895 (TNM) R 26874 26874 24832 0 -1 4202496 85171 0 0 0 37221 14 0 0 25 0 1 0 490364874 737468416 85068 33554432000 134512640 134534564 4289387536 18446744073709551615 134524568 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/26895/statm: 180046 85068 91 6 0 178712 0
Current children cumulated CPU time (s) 1120.43
Current children cumulated vsize (KiB) 4453000

[startup+1182.3 s]
/proc/loadavg: 8.08 7.95 6.91 9/192 27006
/proc/meminfo: memFree=19358512/32950928 swapFree=67111364/67111528
[pid=26874] ppid=26872 vsize=18324 CPUtime=0 cores=5,7
/proc/26874/stat : 26874 (ppfolio) S 26872 26874 24832 0 -1 4202496 500 0 0 0 0 0 0 0 15 0 1 0 490364685 18763776 394 33554432000 4194304 4287625 140735109517120 18446744073709551615 208080252431 0 0 4096 65536 14872756376295178240 0 0 17 7 0 0 0
/proc/26874/statm: 4581 394 345 23 0 91 0
[pid=26891] ppid=26874 vsize=820012 CPUtime=393.23 cores=5
/proc/26891/stat : 26891 (cryptominisat) R 26874 26874 24832 0 -1 4202496 729105 0 0 0 39224 99 0 0 25 0 1 0 490364874 839692288 182931 33554432000 4194304 4587354 140733368448384 18446744073709551615 4230185 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/26891/statm: 205003 182931 338 96 0 201565 0
[pid=26894] ppid=26874 vsize=846684 CPUtime=394.85 cores=5
/proc/26894/stat : 26894 (clasp) R 26874 26874 24832 0 -1 4202496 212811 0 0 0 39431 54 0 0 25 0 1 0 490364874 867004416 207218 33554432000 4194304 4785740 140737361410384 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/26894/statm: 211671 207218 336 145 0 208716 0
[pid=26895] ppid=26874 vsize=720184 CPUtime=392.35 cores=5
/proc/26895/stat : 26895 (TNM) R 26874 26874 24832 0 -1 4202496 85171 0 0 0 39221 14 0 0 25 0 1 0 490364874 737468416 85068 33554432000 134512640 134534564 4289387536 18446744073709551615 134524491 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/26895/statm: 180046 85068 91 6 0 178712 0
Current children cumulated CPU time (s) 1180.43
Current children cumulated vsize (KiB) 2405204

[startup+1242.3 s]
/proc/loadavg: 8.03 7.96 6.98 9/192 27007
/proc/meminfo: memFree=19158452/32950928 swapFree=67111364/67111528
[pid=26874] ppid=26872 vsize=18324 CPUtime=0 cores=5,7
/proc/26874/stat : 26874 (ppfolio) S 26872 26874 24832 0 -1 4202496 500 0 0 0 0 0 0 0 15 0 1 0 490364685 18763776 394 33554432000 4194304 4287625 140735109517120 18446744073709551615 208080252431 0 0 4096 65536 14872756651173085184 0 0 17 7 0 0 0
/proc/26874/statm: 4581 394 345 23 0 91 0
[pid=26891] ppid=26874 vsize=986500 CPUtime=413.24 cores=5
/proc/26891/stat : 26891 (cryptominisat) R 26874 26874 24832 0 -1 4202496 758347 0 0 0 41221 103 0 0 25 0 1 0 490364874 1010176000 212148 33554432000 4194304 4587354 140733368448384 18446744073709551615 4309583 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/26891/statm: 246625 212148 338 96 0 243187 0
[pid=26894] ppid=26874 vsize=900932 CPUtime=414.85 cores=5
/proc/26894/stat : 26894 (clasp) R 26874 26874 24832 0 -1 4202496 226350 0 0 0 41428 57 0 0 25 0 1 0 490364874 922554368 220757 33554432000 4194304 4785740 140737361410384 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/26894/statm: 225233 220757 336 145 0 222278 0
[pid=26895] ppid=26874 vsize=720184 CPUtime=412.34 cores=5
/proc/26895/stat : 26895 (TNM) R 26874 26874 24832 0 -1 4202496 85171 0 0 0 41220 14 0 0 25 0 1 0 490364874 737468416 85068 33554432000 134512640 134534564 4289387536 18446744073709551615 134524712 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/26895/statm: 180046 85068 91 6 0 178712 0
Current children cumulated CPU time (s) 1240.43
Current children cumulated vsize (KiB) 2625940



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+1300 s]
/proc/loadavg: 8.01 7.96 7.04 8/190 27010
/proc/meminfo: memFree=19606532/32950928 swapFree=67111364/67111528
[pid=26874] ppid=26872 vsize=18324 CPUtime=0 cores=5,7
/proc/26874/stat : 26874 (ppfolio) S 26872 26874 24832 0 -1 4202496 500 0 0 0 0 0 0 0 15 0 1 0 490364685 18763776 394 33554432000 4194304 4287625 140735109517120 18446744073709551615 208080252431 0 0 4096 65536 14872756926050992128 0 0 17 7 0 0 0
/proc/26874/statm: 4581 394 345 23 0 91 0
[pid=26891] ppid=26874 vsize=820012 CPUtime=432.48 cores=5
/proc/26891/stat : 26891 (cryptominisat) R 26874 26874 24832 0 -1 4202496 759885 0 0 0 43144 104 0 0 25 0 1 0 490364874 839692288 185913 33554432000 4194304 4587354 140733368448384 18446744073709551615 4237679 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/26891/statm: 205003 185913 338 96 0 201565 0
[pid=26894] ppid=26874 vsize=970312 CPUtime=434.05 cores=5
/proc/26894/stat : 26894 (clasp) R 26874 26874 24832 0 -1 4202496 243673 0 0 0 43345 60 0 0 25 0 1 0 490364874 993599488 238080 33554432000 4194304 4785740 140737361410384 18446744073709551615 4416798 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/26894/statm: 242578 238080 336 145 0 239623 0
[pid=26895] ppid=26874 vsize=720184 CPUtime=431.62 cores=5
/proc/26895/stat : 26895 (TNM) R 26874 26874 24832 0 -1 4202496 85171 0 0 0 43148 14 0 0 25 0 1 0 490364874 737468416 85068 33554432000 134512640 134534564 4289387536 18446744073709551615 134524552 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/26895/statm: 180046 85068 91 6 0 178712 0
Current children cumulated CPU time (s) 1298.15
Current children cumulated vsize (KiB) 2528832

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1300 s]
/proc/loadavg: 8.01 7.96 7.04 8/190 27010
/proc/meminfo: memFree=19606532/32950928 swapFree=67111364/67111528
[pid=26874] ppid=26872 vsize=18324 CPUtime=0 cores=5,7
/proc/26874/stat : 26874 (ppfolio) S 26872 26874 24832 0 -1 4202496 500 0 0 0 0 0 0 0 15 0 1 0 490364685 18763776 394 33554432000 4194304 4287625 140735109517120 18446744073709551615 208080252431 0 0 4096 65536 14872756926050992128 0 0 17 7 0 0 0
/proc/26874/statm: 4581 394 345 23 0 91 0
[pid=26891] ppid=26874 vsize=820012 CPUtime=432.48 cores=5
/proc/26891/stat : 26891 (cryptominisat) R 26874 26874 24832 0 -1 4202496 759885 0 0 0 43144 104 0 0 25 0 1 0 490364874 839692288 185913 33554432000 4194304 4587354 140733368448384 18446744073709551615 4237679 0 0 4096 2 0 0 0 17 5 0 0 0
/proc/26891/statm: 205003 185913 338 96 0 201565 0
[pid=26894] ppid=26874 vsize=970312 CPUtime=434.05 cores=5
/proc/26894/stat : 26894 (clasp) R 26874 26874 24832 0 -1 4202496 243673 0 0 0 43345 60 0 0 25 0 1 0 490364874 993599488 238080 33554432000 4194304 4785740 140737361410384 18446744073709551615 4416798 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/26894/statm: 242578 238080 336 145 0 239623 0
[pid=26895] ppid=26874 vsize=720184 CPUtime=431.62 cores=5
/proc/26895/stat : 26895 (TNM) R 26874 26874 24832 0 -1 4202496 85171 0 0 0 43148 14 0 0 25 0 1 0 490364874 737468416 85068 33554432000 134512640 134534564 4289387536 18446744073709551615 134524552 0 0 4096 0 0 0 0 17 5 0 0 0
/proc/26895/statm: 180046 85068 91 6 0 178712 0
Current children cumulated CPU time (s) 1298.15
Current children cumulated vsize (KiB) 2528832

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 26874 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=999
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=2999
# CPU time returned by wait4() is 0.003998
# while last known CPU time is 1298.15
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1300.01
CPU time (s): 1298.15
CPU user time (s): 1296.37
CPU system time (s): 1.78
CPU usage (%): 99.8568
Max. virtual memory (cumulated for all children) (KiB): 6542884

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.000999
system time used= 0.002999
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 500
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 144
involuntary context switches= 5

runsolver used 2.10868 second user time and 5.00224 second system time

The end

Launcher Data 

Begin job on node127 at 2011-04-26 05:14:10
IDJOB=3293677
IDBENCH=88165
IDSOLVER=1613
FILE ID=node127/3293677-1303787650
RUNJOBID= node127-1303787650-26851
PBS_JOBID= 13157213
Free space on /tmp= 71636 MiB

SOLVER NAME= ppfolio seq
BENCH NAME= SAT-Race-2010-CNF/hardware-verification/velev/2dlx_ca_bp_f_liveness.cnf
COMMAND LINE= ppfolio --nbcore=1 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3293677-1303787650/watcher-3293677-1303787650 -o /tmp/evaluation-result-3293677-1303787650/solver-3293677-1303787650 -C 2400 -W 1300 -M 7680  ppfolio --nbcore=1 HOME/instance-3293677-1303787650.cnf

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 7680 MiB
NBCORE= 2

MD5SUM BENCH= 99cd8de7f06ce8c087a5c7fb299dfb62
RANDOM SEED=1979554680

node127.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.60
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      27330912 kB
Buffers:       1285148 kB
Cached:        3787048 kB
SwapCached:          0 kB
Active:        1231552 kB
Inactive:      3905084 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      27330912 kB
SwapTotal:    67111528 kB
SwapFree:     67111364 kB
Dirty:          338872 kB
Writeback:           0 kB
AnonPages:       64344 kB
Mapped:          14872 kB
Slab:           418120 kB
PageTables:       4732 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   212488 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71644 MiB
End job on node127 at 2011-04-26 05:35:52