Trace number 3190879

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 4800.29 1201.71

General information on the benchmark

Namerandom/large/
unif-k3-r4.2-v35000-c147000-S970100151-015-UNKNOWN.cnf
MD5SUM1ba3e47e885ea3862e5726ce64a96a9c
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark136.94
Satisfiable
(Un)Satisfiability was proved
Number of variables35000
Number of clauses147000
Sum of the clauses size441000
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3147000
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 4 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] lingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	[5] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 1
0.00/0.00	Starting clasp 1.3.6 on core(s) 5
0.00/0.00	Starting TNM 2009 on core(s) 7
0.00/0.00	Starting lingeling 276-6264d55-100731 on core(s) 3
0.00/0.00	Starting march_hi 2009 on core(s) 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3190879-1304040941/watcher-3190879-1304040941 -o /tmp/evaluation-result-3190879-1304040941/solver-3190879-1304040941 -C 4800 -W 1300 -M 15500 ppfolio HOME/instance-3190879-1304040941.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 9.78 10.07 9.81 6/182 11173
/proc/meminfo: memFree=13702880/32950928 swapFree=67111528/67111528
[pid=11173] ppid=11171 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/11173/stat : 11173 (ppfolio) R 11171 11173 10696 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 515693953 18763776 390 33554432000 4194304 4287625 140736732386832 18446744073709551615 248089718287 0 0 4096 65536 0 0 0 17 1 0 0 0
/proc/11173/statm: 4581 390 341 23 0 91 0

[startup+0.311301 s]
/proc/loadavg: 9.78 10.07 9.81 6/182 11173
/proc/meminfo: memFree=13702880/32950928 swapFree=67111528/67111528
[pid=11173] ppid=11171 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/11173/stat : 11173 (ppfolio) S 11171 11173 10696 0 -1 4202496 524 0 0 0 0 0 0 0 23 0 1 0 515693953 18763776 390 33554432000 4194304 4287625 140736732386832 18446744073709551615 248089718287 0 0 4096 65536 4607534541693779967 0 0 17 1 0 0 0
/proc/11173/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.400289 s]
/proc/loadavg: 9.78 10.07 9.81 6/182 11173
/proc/meminfo: memFree=13702880/32950928 swapFree=67111528/67111528
[pid=11173] ppid=11171 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/11173/stat : 11173 (ppfolio) S 11171 11173 10696 0 -1 4202496 524 0 0 0 0 0 0 0 22 0 1 0 515693953 18763776 390 33554432000 4194304 4287625 140736732386832 18446744073709551615 248089718287 0 0 4096 65536 4607534541693779967 0 0 17 1 0 0 0
/proc/11173/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.500273 s]
/proc/loadavg: 9.78 10.07 9.81 6/182 11173
/proc/meminfo: memFree=13702880/32950928 swapFree=67111528/67111528
[pid=11173] ppid=11171 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/11173/stat : 11173 (ppfolio) S 11171 11173 10696 0 -1 4202496 524 0 0 0 0 0 0 0 22 0 1 0 515693953 18763776 390 33554432000 4194304 4287625 140736732386832 18446744073709551615 248089718287 0 0 4096 65536 4607534541693779967 0 0 17 1 0 0 0
/proc/11173/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700237 s]
/proc/loadavg: 9.78 10.07 9.81 6/182 11173
/proc/meminfo: memFree=13702880/32950928 swapFree=67111528/67111528
[pid=11173] ppid=11171 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/11173/stat : 11173 (ppfolio) S 11171 11173 10696 0 -1 4202496 524 0 0 0 0 0 0 0 19 0 1 0 515693953 18763776 390 33554432000 4194304 4287625 140736732386832 18446744073709551615 248089718287 0 0 4096 65536 4607534541693779967 0 0 17 1 0 0 0
/proc/11173/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50109 s]
/proc/loadavg: 9.78 10.07 9.81 11/188 11179
/proc/meminfo: memFree=13586024/32950928 swapFree=67111528/67111528
[pid=11173] ppid=11171 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/11173/stat : 11173 (ppfolio) S 11171 11173 10696 0 -1 4202496 524 0 0 0 0 0 0 0 19 0 1 0 515693953 18763776 390 33554432000 4194304 4287625 140736732386832 18446744073709551615 248089718287 0 0 4096 65536 4607534541693779967 0 0 17 1 0 0 0
/proc/11173/statm: 4581 390 341 23 0 91 0
[pid=11174] ppid=11173 vsize=57136 CPUtime=1.48 cores=1
/proc/11174/stat : 11174 (cryptominisat) R 11173 11173 10696 0 -1 4202496 12295 0 0 0 147 1 0 0 25 0 1 0 515693954 58507264 9493 33554432000 4194304 4587354 140735784621376 18446744073709551615 4238258 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/11174/statm: 14284 9493 326 96 0 10846 0
[pid=11175] ppid=11173 vsize=31156 CPUtime=1.49 cores=5
/proc/11175/stat : 11175 (clasp) R 11173 11173 10696 0 -1 4202496 6923 0 0 0 148 1 0 0 25 0 1 0 515693954 31903744 5011 33554432000 4194304 4785740 140735593305088 18446744073709551615 4553032 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/11175/statm: 7789 5011 340 145 0 4834 0
[pid=11176] ppid=11173 vsize=465988 CPUtime=0.68 cores=7
/proc/11176/stat : 11176 (TNM) R 11173 11173 10696 0 -1 4202496 5914 0 0 0 67 1 0 0 25 0 1 0 515693954 477171712 5810 33554432000 134512640 134534564 4288832608 18446744073709551615 134529352 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/11176/statm: 116497 5810 95 6 0 115163 0
[pid=11177] ppid=11173 vsize=19360 CPUtime=1.49 cores=3
/proc/11177/stat : 11177 (lingeling) R 11173 11173 10696 0 -1 4202496 6378 0 0 0 148 1 0 0 25 0 1 0 515693954 19824640 3495 33554432000 4194304 4352236 140736715569184 18446744073709551615 4328447 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/11177/statm: 4840 3495 127 39 0 3920 0
[pid=11178] ppid=11173 vsize=33184 CPUtime=0.79 cores=7
/proc/11178/stat : 11178 (march_hi) R 11173 11173 10696 0 -1 4202496 17801 0 0 0 76 3 0 0 25 0 1 0 515693954 33980416 5423 33554432000 4194304 4331988 140736138775712 18446744073709551615 4252197 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/11178/statm: 8296 5423 141 34 0 6738 0
Current children cumulated CPU time (s) 5.93
Current children cumulated vsize (KiB) 625148

[startup+3.10081 s]
/proc/loadavg: 9.78 10.07 9.81 11/188 11179
/proc/meminfo: memFree=13581212/32950928 swapFree=67111528/67111528
[pid=11173] ppid=11171 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/11173/stat : 11173 (ppfolio) S 11171 11173 10696 0 -1 4202496 524 0 0 0 0 0 0 0 18 0 1 0 515693953 18763776 390 33554432000 4194304 4287625 140736732386832 18446744073709551615 248089718287 0 0 4096 65536 4607534541693779967 0 0 17 1 0 0 0
/proc/11173/statm: 4581 390 341 23 0 91 0
[pid=11174] ppid=11173 vsize=94640 CPUtime=3.09 cores=1
/proc/11174/stat : 11174 (cryptominisat) R 11173 11173 10696 0 -1 4202496 13213 0 0 0 307 2 0 0 25 0 1 0 515693954 96911360 10409 33554432000 4194304 4587354 140735784621376 18446744073709551615 4238736 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/11174/statm: 23660 10409 326 96 0 20222 0
[pid=11175] ppid=11173 vsize=34508 CPUtime=3.09 cores=5
/proc/11175/stat : 11175 (clasp) R 11173 11173 10696 0 -1 4202496 7807 0 0 0 308 1 0 0 25 0 1 0 515693954 35336192 5895 33554432000 4194304 4785740 140735593305088 18446744073709551615 4553003 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/11175/statm: 8627 5895 340 145 0 5672 0
[pid=11176] ppid=11173 vsize=465988 CPUtime=1.48 cores=7
/proc/11176/stat : 11176 (TNM) R 11173 11173 10696 0 -1 4202496 5914 0 0 0 147 1 0 0 25 0 1 0 515693954 477171712 5810 33554432000 134512640 134534564 4288832608 18446744073709551615 134530268 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/11176/statm: 116497 5810 95 6 0 115163 0
[pid=11177] ppid=11173 vsize=19360 CPUtime=3.09 cores=3
/proc/11177/stat : 11177 (lingeling) R 11173 11173 10696 0 -1 4202496 6378 0 0 0 308 1 0 0 25 0 1 0 515693954 19824640 3495 33554432000 4194304 4352236 140736715569184 18446744073709551615 4328126 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/11177/statm: 4840 3495 127 39 0 3920 0
[pid=11178] ppid=11173 vsize=33216 CPUtime=1.59 cores=7
/proc/11178/stat : 11178 (march_hi) R 11173 11173 10696 0 -1 4202496 31601 0 0 0 155 4 0 0 25 0 1 0 515693954 34013184 5428 33554432000 4194304 4331988 140736138775712 18446744073709551615 4252204 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/11178/statm: 8304 5428 144 34 0 6746 0
Current children cumulated CPU time (s) 12.34
Current children cumulated vsize (KiB) 666036

[startup+6.30125 s]
/proc/loadavg: 9.80 10.07 9.81 11/188 11179
/proc/meminfo: memFree=13566460/32950928 swapFree=67111528/67111528
[pid=11173] ppid=11171 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/11173/stat : 11173 (ppfolio) S 11171 11173 10696 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 515693953 18763776 390 33554432000 4194304 4287625 140736732386832 18446744073709551615 248089718287 0 0 4096 65536 4607534541693779967 0 0 17 1 0 0 0
/proc/11173/statm: 4581 390 341 23 0 91 0
[pid=11174] ppid=11173 vsize=94640 CPUtime=6.28 cores=1
/proc/11174/stat : 11174 (cryptominisat) R 11173 11173 10696 0 -1 4202496 14893 0 0 0 626 2 0 0 25 0 1 0 515693954 96911360 12083 33554432000 4194304 4587354 140735784621376 18446744073709551615 4238294 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/11174/statm: 23660 12083 326 96 0 20222 0
[pid=11175] ppid=11173 vsize=46260 CPUtime=6.29 cores=5
/proc/11175/stat : 11175 (clasp) R 11173 11173 10696 0 -1 4202496 10741 0 0 0 627 2 0 0 25 0 1 0 515693954 47370240 8829 33554432000 4194304 4785740 140735593305088 18446744073709551615 4279309 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/11175/statm: 11565 8829 340 145 0 8610 0
[pid=11176] ppid=11173 vsize=465988 CPUtime=3.06 cores=7
/proc/11176/stat : 11176 (TNM) R 11173 11173 10696 0 -1 4202496 5914 0 0 0 305 1 0 0 25 0 1 0 515693954 477171712 5810 33554432000 134512640 134534564 4288832608 18446744073709551615 134528504 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/11176/statm: 116497 5810 95 6 0 115163 0
[pid=11177] ppid=11173 vsize=19360 CPUtime=6.29 cores=3
/proc/11177/stat : 11177 (lingeling) R 11173 11173 10696 0 -1 4202496 6378 0 0 0 628 1 0 0 25 0 1 0 515693954 19824640 3495 33554432000 4194304 4352236 140736715569184 18446744073709551615 4328114 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/11177/statm: 4840 3495 127 39 0 3920 0
[pid=11178] ppid=11173 vsize=76864 CPUtime=3.19 cores=7
/proc/11178/stat : 11178 (march_hi) R 11173 11173 10696 0 -1 4202496 48512 0 0 0 312 7 0 0 25 0 1 0 515693954 78708736 15992 33554432000 4194304 4331988 140736138775712 18446744073709551615 4229017 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/11178/statm: 19216 15992 173 34 0 17658 0
Current children cumulated CPU time (s) 25.11
Current children cumulated vsize (KiB) 721436

[startup+12.7011 s]
/proc/loadavg: 9.81 10.07 9.81 11/188 11180
/proc/meminfo: memFree=13535596/32950928 swapFree=67111528/67111528
[pid=11173] ppid=11171 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/11173/stat : 11173 (ppfolio) S 11171 11173 10696 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 515693953 18763776 390 33554432000 4194304 4287625 140736732386832 18446744073709551615 248089718287 0 0 4096 65536 4607534541693779967 0 0 17 1 0 0 0
/proc/11173/statm: 4581 390 341 23 0 91 0
[pid=11174] ppid=11173 vsize=64648 CPUtime=12.68 cores=1
/proc/11174/stat : 11174 (cryptominisat) R 11173 11173 10696 0 -1 4202496 19906 0 0 0 1264 4 0 0 25 0 1 0 515693954 66199552 10671 33554432000 4194304 4587354 140735784621376 18446744073709551615 4238658 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/11174/statm: 16162 10671 327 96 0 12724 0
[pid=11175] ppid=11173 vsize=69388 CPUtime=12.69 cores=5
/proc/11175/stat : 11175 (clasp) R 11173 11173 10696 0 -1 4202496 16505 0 0 0 1265 4 0 0 25 0 1 0 515693954 71053312 14593 33554432000 4194304 4785740 140735593305088 18446744073709551615 248089359045 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/11175/statm: 17347 14593 340 145 0 14392 0
[pid=11176] ppid=11173 vsize=465988 CPUtime=6.22 cores=7
/proc/11176/stat : 11176 (TNM) R 11173 11173 10696 0 -1 4202496 5914 0 0 0 621 1 0 0 25 0 1 0 515693954 477171712 5810 33554432000 134512640 134534564 4288832608 18446744073709551615 134528726 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/11176/statm: 116497 5810 95 6 0 115163 0
[pid=11177] ppid=11173 vsize=23416 CPUtime=12.69 cores=3
/proc/11177/stat : 11177 (lingeling) R 11173 11173 10696 0 -1 4202496 7903 0 0 0 1268 1 0 0 25 0 1 0 515693954 23977984 5020 33554432000 4194304 4352236 140736715569184 18446744073709551615 4231337 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/11177/statm: 5854 5020 127 39 0 4934 0
[pid=11178] ppid=11173 vsize=79364 CPUtime=6.39 cores=7
/proc/11178/stat : 11178 (march_hi) R 11173 11173 10696 0 -1 4202496 49004 0 0 0 632 7 0 0 25 0 1 0 515693954 81268736 16484 33554432000 4194304 4331988 140736138775712 18446744073709551615 4228957 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/11178/statm: 19841 16484 173 34 0 18283 0
Current children cumulated CPU time (s) 50.67
Current children cumulated vsize (KiB) 721128

[startup+25.5009 s]
/proc/loadavg: 9.85 10.07 9.81 11/188 11180
/proc/meminfo: memFree=13464936/32950928 swapFree=67111528/67111528
[pid=11173] ppid=11171 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/11173/stat : 11173 (ppfolio) S 11171 11173 10696 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 515693953 18763776 390 33554432000 4194304 4287625 140736732386832 18446744073709551615 248089718287 0 0 4096 65536 4607534541693779967 0 0 17 1 0 0 0
/proc/11173/statm: 4581 390 341 23 0 91 0
[pid=11174] ppid=11173 vsize=67060 CPUtime=25.49 cores=1
/proc/11174/stat : 11174 (cryptominisat) R 11173 11173 10696 0 -1 4202496 48097 0 0 0 2538 11 0 0 25 0 1 0 515693954 68669440 10216 33554432000 4194304 4587354 140735784621376 18446744073709551615 4230288 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/11174/statm: 16765 10216 337 96 0 13327 0
[pid=11175] ppid=11173 vsize=108608 CPUtime=25.5 cores=5
/proc/11175/stat : 11175 (clasp) R 11173 11173 10696 0 -1 4202496 26290 0 0 0 2542 8 0 0 25 0 1 0 515693954 111214592 24378 33554432000 4194304 4785740 140735593305088 18446744073709551615 4286953 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/11175/statm: 27152 24378 340 145 0 24197 0
[pid=11176] ppid=11173 vsize=465988 CPUtime=12.6 cores=7
/proc/11176/stat : 11176 (TNM) R 11173 11173 10696 0 -1 4202496 5914 0 0 0 1259 1 0 0 25 0 1 0 515693954 477171712 5810 33554432000 134512640 134534564 4288832608 18446744073709551615 134530151 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/11176/statm: 116497 5810 95 6 0 115163 0
[pid=11177] ppid=11173 vsize=25116 CPUtime=25.49 cores=3
/proc/11177/stat : 11177 (lingeling) R 11173 11173 10696 0 -1 4202496 8328 0 0 0 2548 1 0 0 25 0 1 0 515693954 25718784 5445 33554432000 4194304 4352236 140736715569184 18446744073709551615 4331541 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/11177/statm: 6279 5445 127 39 0 5359 0
[pid=11178] ppid=11173 vsize=85140 CPUtime=12.76 cores=7
/proc/11178/stat : 11178 (march_hi) R 11173 11173 10696 0 -1 4202496 50548 0 0 0 1268 8 0 0 25 0 1 0 515693954 87183360 18028 33554432000 4194304 4331988 140736138775712 18446744073709551615 4240424 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/11178/statm: 21285 18028 173 34 0 19727 0
Current children cumulated CPU time (s) 101.84
Current children cumulated vsize (KiB) 770236

[startup+51.1003 s]
/proc/loadavg: 9.98 10.08 9.82 11/188 11181
/proc/meminfo: memFree=13387780/32950928 swapFree=67111528/67111528
[pid=11173] ppid=11171 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/11173/stat : 11173 (ppfolio) S 11171 11173 10696 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 515693953 18763776 390 33554432000 4194304 4287625 140736732386832 18446744073709551615 248089718287 0 0 4096 65536 4607534541693779967 0 0 17 1 0 0 0
/proc/11173/statm: 4581 390 341 23 0 91 0
[pid=11174] ppid=11173 vsize=166744 CPUtime=51.09 cores=1
/proc/11174/stat : 11174 (cryptominisat) R 11173 11173 10696 0 -1 4202496 86543 0 0 0 5087 22 0 0 25 0 1 0 515693954 170745856 21265 33554432000 4194304 4587354 140735784621376 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/11174/statm: 41686 21265 337 96 0 38248 0
[pid=11175] ppid=11173 vsize=137652 CPUtime=51.08 cores=5
/proc/11175/stat : 11175 (clasp) R 11173 11173 10696 0 -1 4202496 33307 0 0 0 5099 9 0 0 25 0 1 0 515693954 140955648 31395 33554432000 4194304 4785740 140735593305088 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/11175/statm: 34413 31395 340 145 0 31458 0
[pid=11176] ppid=11173 vsize=465988 CPUtime=25.32 cores=7
/proc/11176/stat : 11176 (TNM) R 11173 11173 10696 0 -1 4202496 5914 0 0 0 2531 1 0 0 25 0 1 0 515693954 477171712 5810 33554432000 134512640 134534564 4288832608 18446744073709551615 134528877 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/11176/statm: 116497 5810 95 6 0 115163 0
[pid=11177] ppid=11173 vsize=28224 CPUtime=51.08 cores=3

################
# More data... #
################

/proc/11178/statm: 22869 19614 173 34 0 21311 0
Current children cumulated CPU time (s) 3524.39
Current children cumulated vsize (KiB) 2068204

[startup+942.301 s]
/proc/loadavg: 10.24 10.16 10.01 12/188 11237
/proc/meminfo: memFree=12979080/32950928 swapFree=67111528/67111528
[pid=11173] ppid=11171 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/11173/stat : 11173 (ppfolio) S 11171 11173 10696 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 515693953 18763776 390 33554432000 4194304 4287625 140736732386832 18446744073709551615 248089718287 0 0 4096 65536 4607534541693779967 0 0 17 1 0 0 0
/proc/11173/statm: 4581 390 341 23 0 91 0
[pid=11174] ppid=11173 vsize=1053728 CPUtime=942.27 cores=1
/proc/11174/stat : 11174 (cryptominisat) R 11173 11173 10696 0 -1 4202496 572709 0 0 0 94006 221 0 0 25 0 1 0 515693954 1079017472 36320 33554432000 4194304 4587354 140735784621376 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/11174/statm: 263432 36320 337 96 0 259994 0
[pid=11175] ppid=11173 vsize=407172 CPUtime=942.31 cores=5
/proc/11175/stat : 11175 (clasp) R 11173 11173 10696 0 -1 4202496 99790 0 0 0 94198 33 0 0 25 0 1 0 515693954 416944128 97878 33554432000 4194304 4785740 140735593305088 18446744073709551615 4552550 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/11175/statm: 101793 97878 340 145 0 98838 0
[pid=11176] ppid=11173 vsize=465988 CPUtime=468.53 cores=7
/proc/11176/stat : 11176 (TNM) R 11173 11173 10696 0 -1 4202496 5914 0 0 0 46852 1 0 0 25 0 1 0 515693954 477171712 5810 33554432000 134512640 134534564 4288832608 18446744073709551615 134525956 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/11176/statm: 116497 5810 95 6 0 115163 0
[pid=11177] ppid=11173 vsize=31296 CPUtime=942.31 cores=3
/proc/11177/stat : 11177 (lingeling) R 11173 11173 10696 0 -1 4202496 9873 0 0 0 94229 2 0 0 25 0 1 0 515693954 32047104 6990 33554432000 4194304 4352236 140736715569184 18446744073709551615 4328013 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/11177/statm: 7824 6990 127 39 0 6904 0
[pid=11178] ppid=11173 vsize=91608 CPUtime=468.63 cores=7
/proc/11178/stat : 11178 (march_hi) R 11173 11173 10696 0 -1 4202496 52171 0 0 0 46855 8 0 0 25 0 1 0 515693954 93806592 19651 33554432000 4194304 4331988 140736138775712 18446744073709551615 4263120 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/11178/statm: 22902 19651 173 34 0 21344 0
Current children cumulated CPU time (s) 3764.05
Current children cumulated vsize (KiB) 2068116

[startup+1002.3 s]
/proc/loadavg: 10.28 10.18 10.02 12/188 11239
/proc/meminfo: memFree=12950536/32950928 swapFree=67111528/67111528
[pid=11173] ppid=11171 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/11173/stat : 11173 (ppfolio) S 11171 11173 10696 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 515693953 18763776 390 33554432000 4194304 4287625 140736732386832 18446744073709551615 248089718287 0 0 4096 65536 4607534541693779967 0 0 17 1 0 0 0
/proc/11173/statm: 4581 390 341 23 0 91 0
[pid=11174] ppid=11173 vsize=1044588 CPUtime=1002.28 cores=1
/proc/11174/stat : 11174 (cryptominisat) R 11173 11173 10696 0 -1 4202496 591655 0 0 0 99995 233 0 0 25 0 1 0 515693954 1069658112 31598 33554432000 4194304 4587354 140735784621376 18446744073709551615 4230288 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/11174/statm: 261147 31598 337 96 0 257709 0
[pid=11175] ppid=11173 vsize=423796 CPUtime=1002.32 cores=5
/proc/11175/stat : 11175 (clasp) R 11173 11173 10696 0 -1 4202496 103934 0 0 0 100197 35 0 0 25 0 1 0 515693954 433967104 102022 33554432000 4194304 4785740 140735593305088 18446744073709551615 4287204 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/11175/statm: 105949 102022 340 145 0 102994 0
[pid=11176] ppid=11173 vsize=465988 CPUtime=498.33 cores=7
/proc/11176/stat : 11176 (TNM) R 11173 11173 10696 0 -1 4202496 5914 0 0 0 49832 1 0 0 25 0 1 0 515693954 477171712 5810 33554432000 134512640 134534564 4288832608 18446744073709551615 134530367 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/11176/statm: 116497 5810 95 6 0 115163 0
[pid=11177] ppid=11173 vsize=31296 CPUtime=1002.32 cores=3
/proc/11177/stat : 11177 (lingeling) R 11173 11173 10696 0 -1 4202496 9873 0 0 0 100230 2 0 0 25 0 1 0 515693954 32047104 6990 33554432000 4194304 4352236 140736715569184 18446744073709551615 4328194 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/11177/statm: 7824 6990 127 39 0 6904 0
[pid=11178] ppid=11173 vsize=91740 CPUtime=498.51 cores=7
/proc/11178/stat : 11178 (march_hi) R 11173 11173 10696 0 -1 4202496 52203 0 0 0 49843 8 0 0 25 0 1 0 515693954 93941760 19683 33554432000 4194304 4331988 140736138775712 18446744073709551615 4263143 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/11178/statm: 22935 19683 173 34 0 21377 0
Current children cumulated CPU time (s) 4003.76
Current children cumulated vsize (KiB) 2075732

[startup+1062.3 s]
/proc/loadavg: 10.29 10.19 10.03 12/188 11240
/proc/meminfo: memFree=12924224/32950928 swapFree=67111528/67111528
[pid=11173] ppid=11171 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/11173/stat : 11173 (ppfolio) S 11171 11173 10696 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 515693953 18763776 390 33554432000 4194304 4287625 140736732386832 18446744073709551615 248089718287 0 0 4096 65536 4607534541693779967 0 0 17 1 0 0 0
/proc/11173/statm: 4581 390 341 23 0 91 0
[pid=11174] ppid=11173 vsize=1039620 CPUtime=1062.28 cores=1
/proc/11174/stat : 11174 (cryptominisat) R 11173 11173 10696 0 -1 4202496 612118 0 0 0 105984 244 0 0 25 0 1 0 515693954 1064570880 27157 33554432000 4194304 4587354 140735784621376 18446744073709551615 4244609 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/11174/statm: 259905 27157 337 96 0 256467 0
[pid=11175] ppid=11173 vsize=461100 CPUtime=1062.32 cores=5
/proc/11175/stat : 11175 (clasp) R 11173 11173 10696 0 -1 4202496 113113 0 0 0 106193 39 0 0 25 0 1 0 515693954 472166400 111201 33554432000 4194304 4785740 140735593305088 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/11175/statm: 115275 111201 340 145 0 112320 0
[pid=11176] ppid=11173 vsize=465988 CPUtime=528.2 cores=7
/proc/11176/stat : 11176 (TNM) R 11173 11173 10696 0 -1 4202496 5914 0 0 0 52819 1 0 0 25 0 1 0 515693954 477171712 5810 33554432000 134512640 134534564 4288832608 18446744073709551615 134528737 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/11176/statm: 116497 5810 95 6 0 115163 0
[pid=11177] ppid=11173 vsize=31296 CPUtime=1062.32 cores=3
/proc/11177/stat : 11177 (lingeling) R 11173 11173 10696 0 -1 4202496 9873 0 0 0 106230 2 0 0 25 0 1 0 515693954 32047104 6990 33554432000 4194304 4352236 140736715569184 18446744073709551615 4328126 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/11177/statm: 7824 6990 127 39 0 6904 0
[pid=11178] ppid=11173 vsize=91872 CPUtime=528.31 cores=7
/proc/11178/stat : 11178 (march_hi) R 11173 11173 10696 0 -1 4202496 52231 0 0 0 52823 8 0 0 25 0 1 0 515693954 94076928 19711 33554432000 4194304 4331988 140736138775712 18446744073709551615 4228970 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/11178/statm: 22968 19711 173 34 0 21410 0
Current children cumulated CPU time (s) 4243.43
Current children cumulated vsize (KiB) 2108200

[startup+1122.31 s]
/proc/loadavg: 10.27 10.20 10.04 12/188 11242
/proc/meminfo: memFree=12882912/32950928 swapFree=67111528/67111528
[pid=11173] ppid=11171 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/11173/stat : 11173 (ppfolio) S 11171 11173 10696 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 515693953 18763776 390 33554432000 4194304 4287625 140736732386832 18446744073709551615 248089718287 0 0 4096 65536 4607534541693779967 0 0 17 1 0 0 0
/proc/11173/statm: 4581 390 341 23 0 91 0
[pid=11174] ppid=11173 vsize=1053376 CPUtime=1122.28 cores=1
/proc/11174/stat : 11174 (cryptominisat) R 11173 11173 10696 0 -1 4202496 631385 0 0 0 111974 254 0 0 25 0 1 0 515693954 1078657024 33472 33554432000 4194304 4587354 140735784621376 18446744073709551615 4236606 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/11174/statm: 263344 33472 337 96 0 259906 0
[pid=11175] ppid=11173 vsize=461648 CPUtime=1122.32 cores=5
/proc/11175/stat : 11175 (clasp) R 11173 11173 10696 0 -1 4202496 113200 0 0 0 112193 39 0 0 25 0 1 0 515693954 472727552 111288 33554432000 4194304 4785740 140735593305088 18446744073709551615 4546305 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/11175/statm: 115412 111288 340 145 0 112457 0
[pid=11176] ppid=11173 vsize=465988 CPUtime=558.01 cores=7
/proc/11176/stat : 11176 (TNM) R 11173 11173 10696 0 -1 4202496 5914 0 0 0 55800 1 0 0 25 0 1 0 515693954 477171712 5810 33554432000 134512640 134534564 4288832608 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/11176/statm: 116497 5810 95 6 0 115163 0
[pid=11177] ppid=11173 vsize=31296 CPUtime=1122.32 cores=3
/proc/11177/stat : 11177 (lingeling) R 11173 11173 10696 0 -1 4202496 9873 0 0 0 112230 2 0 0 25 0 1 0 515693954 32047104 6990 33554432000 4194304 4352236 140736715569184 18446744073709551615 4328105 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/11177/statm: 7824 6990 127 39 0 6904 0
[pid=11178] ppid=11173 vsize=92004 CPUtime=558.18 cores=7
/proc/11178/stat : 11178 (march_hi) R 11173 11173 10696 0 -1 4202496 52250 0 0 0 55809 9 0 0 25 0 1 0 515693954 94212096 19730 33554432000 4194304 4331988 140736138775712 18446744073709551615 4228970 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/11178/statm: 23001 19730 173 34 0 21443 0
Current children cumulated CPU time (s) 4483.11
Current children cumulated vsize (KiB) 2122636

[startup+1182.3 s]
/proc/loadavg: 10.20 10.19 10.05 12/188 11244
/proc/meminfo: memFree=12938320/32950928 swapFree=67111528/67111528
[pid=11173] ppid=11171 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/11173/stat : 11173 (ppfolio) S 11171 11173 10696 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 515693953 18763776 390 33554432000 4194304 4287625 140736732386832 18446744073709551615 248089718287 0 0 4096 65536 4607534541693779967 0 0 17 1 0 0 0
/proc/11173/statm: 4581 390 341 23 0 91 0
[pid=11174] ppid=11173 vsize=1046244 CPUtime=1182.27 cores=1
/proc/11174/stat : 11174 (cryptominisat) R 11173 11173 10696 0 -1 4202496 651582 0 0 0 117960 267 0 0 25 0 1 0 515693954 1071353856 27254 33554432000 4194304 4587354 140735784621376 18446744073709551615 4238658 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/11174/statm: 261561 27254 337 96 0 258123 0
[pid=11175] ppid=11173 vsize=462108 CPUtime=1182.32 cores=5
/proc/11175/stat : 11175 (clasp) R 11173 11173 10696 0 -1 4202496 113287 0 0 0 118193 39 0 0 25 0 1 0 515693954 473198592 111375 33554432000 4194304 4785740 140735593305088 18446744073709551615 4553402 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/11175/statm: 115527 111375 340 145 0 112572 0
[pid=11176] ppid=11173 vsize=465988 CPUtime=587.86 cores=7
/proc/11176/stat : 11176 (TNM) R 11173 11173 10696 0 -1 4202496 5914 0 0 0 58785 1 0 0 25 0 1 0 515693954 477171712 5810 33554432000 134512640 134534564 4288832608 18446744073709551615 134528569 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/11176/statm: 116497 5810 95 6 0 115163 0
[pid=11177] ppid=11173 vsize=31296 CPUtime=1182.32 cores=3
/proc/11177/stat : 11177 (lingeling) R 11173 11173 10696 0 -1 4202496 9873 0 0 0 118230 2 0 0 25 0 1 0 515693954 32047104 6990 33554432000 4194304 4352236 140736715569184 18446744073709551615 4280960 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/11177/statm: 7824 6990 127 39 0 6904 0
[pid=11178] ppid=11173 vsize=92004 CPUtime=588.01 cores=7
/proc/11178/stat : 11178 (march_hi) R 11173 11173 10696 0 -1 4202496 52272 0 0 0 58792 9 0 0 25 0 1 0 515693954 94212096 19752 33554432000 4194304 4331988 140736138775712 18446744073709551615 4292232 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/11178/statm: 23001 19752 173 34 0 21443 0
Current children cumulated CPU time (s) 4722.78
Current children cumulated vsize (KiB) 2115964



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 10.14 10.18 10.04 11/188 11244
/proc/meminfo: memFree=12924184/32950928 swapFree=67111528/67111528
[pid=11173] ppid=11171 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/11173/stat : 11173 (ppfolio) S 11171 11173 10696 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 515693953 18763776 390 33554432000 4194304 4287625 140736732386832 18446744073709551615 248089718287 0 0 4096 65536 4607534541693779967 0 0 17 1 0 0 0
/proc/11173/statm: 4581 390 341 23 0 91 0
[pid=11174] ppid=11173 vsize=1009940 CPUtime=1201.68 cores=1
/proc/11174/stat : 11174 (cryptominisat) R 11173 11173 10696 0 -1 4202496 657978 0 0 0 119897 271 0 0 25 0 1 0 515693954 1034178560 22798 33554432000 4194304 4587354 140735784621376 18446744073709551615 4238712 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/11174/statm: 252485 22798 337 96 0 249047 0
[pid=11175] ppid=11173 vsize=462108 CPUtime=1201.72 cores=5
/proc/11175/stat : 11175 (clasp) R 11173 11173 10696 0 -1 4202496 113287 0 0 0 120133 39 0 0 25 0 1 0 515693954 473198592 111375 33554432000 4194304 4785740 140735593305088 18446744073709551615 4553032 0 0 4096 16386 0 0 0 17 5 0 0 0
/proc/11175/statm: 115527 111375 340 145 0 112572 0
[pid=11176] ppid=11173 vsize=465988 CPUtime=597.51 cores=7
/proc/11176/stat : 11176 (TNM) R 11173 11173 10696 0 -1 4202496 5914 0 0 0 59750 1 0 0 25 0 1 0 515693954 477171712 5810 33554432000 134512640 134534564 4288832608 18446744073709551615 134528913 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/11176/statm: 116497 5810 95 6 0 115163 0
[pid=11177] ppid=11173 vsize=31296 CPUtime=1201.73 cores=3
/proc/11177/stat : 11177 (lingeling) R 11173 11173 10696 0 -1 4202496 9873 0 0 0 120171 2 0 0 25 0 1 0 515693954 32047104 6990 33554432000 4194304 4352236 140736715569184 18446744073709551615 4328479 0 0 4096 17506 0 0 0 17 3 0 0 0
/proc/11177/statm: 7824 6990 127 39 0 6904 0
[pid=11178] ppid=11173 vsize=92004 CPUtime=597.65 cores=7
/proc/11178/stat : 11178 (march_hi) R 11173 11173 10696 0 -1 4202496 52278 0 0 0 59756 9 0 0 25 0 1 0 515693954 94212096 19758 33554432000 4194304 4331988 140736138775712 18446744073709551615 4287256 0 0 4096 0 0 0 0 17 7 0 0 0
/proc/11178/statm: 23001 19758 173 34 0 21443 0
Current children cumulated CPU time (s) 4800.29
Current children cumulated vsize (KiB) 2079660

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 11173 and gives
#  childrusage.ru_utime.tv_sec=1201
#  childrusage.ru_utime.tv_usec=713311
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=27995
# CPU time returned by wait4() is 1201.74
# while last known CPU time is 4800.29
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1201.71
CPU time (s): 4800.29
CPU user time (s): 4797.07
CPU system time (s): 3.22
CPU usage (%): 399.454
Max. virtual memory (cumulated for all children) (KiB): 3039212

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1201.71
system time used= 0.027995
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 10399
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 270
involuntary context switches= 13809

runsolver used 1.77873 second user time and 4.78427 second system time

The end

Launcher Data 

Begin job on node123 at 2011-04-29 03:35:42
IDJOB=3190879
IDBENCH=82610
IDSOLVER=1612
FILE ID=node123/3190879-1304040941
RUNJOBID= node123-1304037296-10715
PBS_JOBID= 13170370
Free space on /tmp= 72804 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/random/large/unif-k3-r4.2-v35000-c147000-S970100151-015-UNKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3190879-1304040941/watcher-3190879-1304040941 -o /tmp/evaluation-result-3190879-1304040941/solver-3190879-1304040941 -C 4800 -W 1300 -M 15500  ppfolio HOME/instance-3190879-1304040941.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 1ba3e47e885ea3862e5726ce64a96a9c
RANDOM SEED=1241021494

node123.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      13703160 kB
Buffers:       2191616 kB
Cached:       16025936 kB
SwapCached:          0 kB
Active:        9520092 kB
Inactive:      9178308 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      13703160 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           12840 kB
Writeback:           0 kB
AnonPages:      480164 kB
Mapped:          16716 kB
Slab:           486232 kB
PageTables:       5944 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:  1901260 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72792 MiB
End job on node123 at 2011-04-29 03:55:45