Trace number 3251347

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 4800.17 1201.71

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m62-q1504.5-n60-p30-s88600538.cnf
MD5SUM76c30bc0971bc865c2b988ddaca89385
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark5.29319
Satisfiable
(Un)Satisfiability was proved
Number of variables3651
Number of clauses926696
Sum of the clauses size1856923
Maximum clause length62
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2926636
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 560

Solver Data 

0.00/0.01	This is pico-portfolio, a naive parallel portfolio
0.00/0.01	Copyright (C) 2011 Olivier Roussel
0.00/0.01	Using 4 processing units
0.00/0.01	This portfolio uses the following solvers:
0.00/0.01	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.01	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.01	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.01	[4] lingeling 276-6264d55-100731 (Armin Biere)
0.00/0.01	[5] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.01	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.01	Starting clasp 1.3.6 on core(s) 4
0.00/0.01	Starting TNM 2009 on core(s) 6
0.00/0.01	Starting lingeling 276-6264d55-100731 on core(s) 2
0.00/0.01	Starting march_hi 2009 on core(s) 6

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3251347-1304099362/watcher-3251347-1304099362 -o /tmp/evaluation-result-3251347-1304099362/solver-3251347-1304099362 -C 4800 -W 1300 -M 15500 ppfolio HOME/instance-3251347-1304099362.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 9.72 9.58 9.42 7/182 29750
/proc/meminfo: memFree=17120252/32950940 swapFree=67111528/67111528
[pid=29750] ppid=29748 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/29750/stat : 29750 (ppfolio) S 29748 29750 28861 0 -1 4202496 524 0 0 0 0 0 0 0 25 0 1 0 600166996 18763776 390 33554432000 4194304 4287625 140734454502448 18446744073709551615 215988612623 0 0 4096 65536 4578965106435555327 0 0 17 2 0 0 0
/proc/29750/statm: 4581 390 341 23 0 91 0

[startup+0.235274 s]
/proc/loadavg: 9.72 9.58 9.42 7/182 29750
/proc/meminfo: memFree=17120252/32950940 swapFree=67111528/67111528
[pid=29750] ppid=29748 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/29750/stat : 29750 (ppfolio) S 29748 29750 28861 0 -1 4202496 524 0 0 0 0 0 0 0 23 0 1 0 600166996 18763776 390 33554432000 4194304 4287625 140734454502448 18446744073709551615 215988612623 0 0 4096 65536 4578965106435555327 0 0 17 2 0 0 0
/proc/29750/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300262 s]
/proc/loadavg: 9.72 9.58 9.42 7/182 29750
/proc/meminfo: memFree=17120252/32950940 swapFree=67111528/67111528
[pid=29750] ppid=29748 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/29750/stat : 29750 (ppfolio) S 29748 29750 28861 0 -1 4202496 524 0 0 0 0 0 0 0 23 0 1 0 600166996 18763776 390 33554432000 4194304 4287625 140734454502448 18446744073709551615 215988612623 0 0 4096 65536 4578965106435555327 0 0 17 2 0 0 0
/proc/29750/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.400245 s]
/proc/loadavg: 9.72 9.58 9.42 7/182 29750
/proc/meminfo: memFree=17120252/32950940 swapFree=67111528/67111528
[pid=29750] ppid=29748 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/29750/stat : 29750 (ppfolio) S 29748 29750 28861 0 -1 4202496 524 0 0 0 0 0 0 0 23 0 1 0 600166996 18763776 390 33554432000 4194304 4287625 140734454502448 18446744073709551615 215988612623 0 0 4096 65536 4578965106435555327 0 0 17 2 0 0 0
/proc/29750/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700194 s]
/proc/loadavg: 9.72 9.58 9.42 7/182 29750
/proc/meminfo: memFree=17120252/32950940 swapFree=67111528/67111528
[pid=29750] ppid=29748 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/29750/stat : 29750 (ppfolio) S 29748 29750 28861 0 -1 4202496 524 0 0 0 0 0 0 0 21 0 1 0 600166996 18763776 390 33554432000 4194304 4287625 140734454502448 18446744073709551615 215988612623 0 0 4096 65536 4578965106435555327 0 0 17 2 0 0 0
/proc/29750/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50105 s]
/proc/loadavg: 9.72 9.58 9.42 12/188 29756
/proc/meminfo: memFree=16805372/32950940 swapFree=67111528/67111528
[pid=29750] ppid=29748 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/29750/stat : 29750 (ppfolio) S 29748 29750 28861 0 -1 4202496 524 0 0 0 0 0 0 0 21 0 1 0 600166996 18763776 390 33554432000 4194304 4287625 140734454502448 18446744073709551615 215988612623 0 0 4096 65536 4578965106435555327 0 0 17 2 0 0 0
/proc/29750/statm: 4581 390 341 23 0 91 0
[pid=29751] ppid=29750 vsize=139004 CPUtime=1.48 cores=0
/proc/29751/stat : 29751 (cryptominisat) R 29750 29750 28861 0 -1 4202496 29070 0 0 0 145 3 0 0 25 0 1 0 600166997 142340096 28955 33554432000 4194304 4587354 140733237865968 18446744073709551615 4388368 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/29751/statm: 34751 28955 314 96 0 31313 0
[pid=29752] ppid=29750 vsize=68572 CPUtime=1.49 cores=4
/proc/29752/stat : 29752 (clasp) R 29750 29750 28861 0 -1 4202496 18296 0 0 0 147 2 0 0 25 0 1 0 600166997 70217728 14114 33554432000 4194304 4785740 140734299932624 18446744073709551615 4495691 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/29752/statm: 17143 14114 313 145 0 14188 0
[pid=29753] ppid=29750 vsize=495424 CPUtime=0.71 cores=6
/proc/29753/stat : 29753 (TNM) R 29750 29750 28861 0 -1 4202496 14756 0 0 0 70 1 0 0 25 0 1 0 600166997 507314176 14652 33554432000 134512640 134534564 4289823632 18446744073709551615 134524478 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/29753/statm: 123856 14652 91 6 0 122522 0
[pid=29754] ppid=29750 vsize=42528 CPUtime=1.48 cores=2
/proc/29754/stat : 29754 (lingeling) R 29750 29750 28861 0 -1 4202496 13793 0 0 0 147 1 0 0 25 0 1 0 600166997 43548672 9559 33554432000 4194304 4352236 140735566239776 18446744073709551615 4267780 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/29754/statm: 10632 9559 123 39 0 9712 0
[pid=29755] ppid=29750 vsize=105352 CPUtime=0.76 cores=6
/proc/29755/stat : 29755 (march_hi) R 29750 29750 28861 0 -1 4194304 20504 0 0 0 73 3 0 0 25 0 1 0 600166997 107880448 15833 33554432000 4194304 4331988 140734883440608 18446744073709551615 4252200 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/29755/statm: 26338 15833 127 34 0 24780 0
Current children cumulated CPU time (s) 5.92
Current children cumulated vsize (KiB) 869204

[startup+3.10077 s]
/proc/loadavg: 9.72 9.58 9.42 12/188 29756
/proc/meminfo: memFree=16788384/32950940 swapFree=67111528/67111528
[pid=29750] ppid=29748 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/29750/stat : 29750 (ppfolio) S 29748 29750 28861 0 -1 4202496 524 0 0 0 0 0 0 0 21 0 1 0 600166996 18763776 390 33554432000 4194304 4287625 140734454502448 18446744073709551615 215988612623 0 0 4096 65536 4578965106435555327 0 0 17 2 0 0 0
/proc/29750/statm: 4581 390 341 23 0 91 0
[pid=29751] ppid=29750 vsize=151752 CPUtime=3.09 cores=0
/proc/29751/stat : 29751 (cryptominisat) R 29750 29750 28861 0 -1 4202496 35832 0 0 0 304 5 0 0 25 0 1 0 600166997 155394048 30271 33554432000 4194304 4587354 140733237865968 18446744073709551615 4375171 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/29751/statm: 37938 30271 315 96 0 34500 0
[pid=29752] ppid=29750 vsize=68572 CPUtime=3.09 cores=4
/proc/29752/stat : 29752 (clasp) R 29750 29750 28861 0 -1 4202496 18296 0 0 0 307 2 0 0 25 0 1 0 600166997 70217728 14114 33554432000 4194304 4785740 140734299932624 18446744073709551615 4495695 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/29752/statm: 17143 14114 313 145 0 14188 0
[pid=29753] ppid=29750 vsize=495424 CPUtime=1.51 cores=6
/proc/29753/stat : 29753 (TNM) R 29750 29750 28861 0 -1 4202496 14756 0 0 0 150 1 0 0 25 0 1 0 600166997 507314176 14652 33554432000 134512640 134534564 4289823632 18446744073709551615 134524508 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/29753/statm: 123856 14652 91 6 0 122522 0
[pid=29754] ppid=29750 vsize=45312 CPUtime=3.07 cores=2
/proc/29754/stat : 29754 (lingeling) R 29750 29750 28861 0 -1 4202496 14257 0 0 0 306 1 0 0 25 0 1 0 600166997 46399488 10023 33554432000 4194304 4352236 140735566239776 18446744073709551615 4207645 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/29754/statm: 11328 10023 123 39 0 10408 0
[pid=29755] ppid=29750 vsize=97820 CPUtime=1.55 cores=6
/proc/29755/stat : 29755 (march_hi) R 29750 29750 28861 0 -1 4202496 48329 0 0 0 148 7 0 0 25 0 1 0 600166997 100167680 13986 33554432000 4194304 4331988 140734883440608 18446744073709551615 4257057 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/29755/statm: 24455 13986 149 34 0 22897 0
Current children cumulated CPU time (s) 12.31
Current children cumulated vsize (KiB) 877204

[startup+6.30121 s]
/proc/loadavg: 9.74 9.59 9.43 12/188 29756
/proc/meminfo: memFree=16739404/32950940 swapFree=67111528/67111528
[pid=29750] ppid=29748 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/29750/stat : 29750 (ppfolio) S 29748 29750 28861 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 600166996 18763776 390 33554432000 4194304 4287625 140734454502448 18446744073709551615 215988612623 0 0 4096 65536 4578965106435555327 0 0 17 2 0 0 0
/proc/29750/statm: 4581 390 341 23 0 91 0
[pid=29751] ppid=29750 vsize=198024 CPUtime=6.29 cores=0
/proc/29751/stat : 29751 (cryptominisat) R 29750 29750 28861 0 -1 4202496 53801 0 0 0 622 7 0 0 25 0 1 0 600166997 202776576 34814 33554432000 4194304 4587354 140733237865968 18446744073709551615 4238593 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/29751/statm: 49506 34814 326 96 0 46068 0
[pid=29752] ppid=29750 vsize=69496 CPUtime=6.29 cores=4
/proc/29752/stat : 29752 (clasp) R 29750 29750 28861 0 -1 4202496 18530 0 0 0 627 2 0 0 25 0 1 0 600166997 71163904 14348 33554432000 4194304 4785740 140734299932624 18446744073709551615 4482895 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/29752/statm: 17374 14348 316 145 0 14419 0
[pid=29753] ppid=29750 vsize=495424 CPUtime=3.11 cores=6
/proc/29753/stat : 29753 (TNM) R 29750 29750 28861 0 -1 4202496 14756 0 0 0 310 1 0 0 25 0 1 0 600166997 507314176 14652 33554432000 134512640 134534564 4289823632 18446744073709551615 134524483 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/29753/statm: 123856 14652 91 6 0 122522 0
[pid=29754] ppid=29750 vsize=75200 CPUtime=6.28 cores=2
/proc/29754/stat : 29754 (lingeling) R 29750 29750 28861 0 -1 4202496 20148 0 0 0 626 2 0 0 25 0 1 0 600166997 77004800 15914 33554432000 4194304 4352236 140735566239776 18446744073709551615 4328059 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/29754/statm: 18800 15914 126 39 0 17880 0
[pid=29755] ppid=29750 vsize=128032 CPUtime=3.13 cores=6
/proc/29755/stat : 29755 (march_hi) R 29750 29750 28861 0 -1 4202496 71990 0 0 0 303 10 0 0 25 0 1 0 600166997 131104768 15381 33554432000 4194304 4331988 140734883440608 18446744073709551615 4226499 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/29755/statm: 32008 15381 170 34 0 30450 0
Current children cumulated CPU time (s) 25.1
Current children cumulated vsize (KiB) 984500

[startup+12.7011 s]
/proc/loadavg: 9.76 9.60 9.43 12/188 29756
/proc/meminfo: memFree=16732112/32950940 swapFree=67111528/67111528
[pid=29750] ppid=29748 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/29750/stat : 29750 (ppfolio) S 29748 29750 28861 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 600166996 18763776 390 33554432000 4194304 4287625 140734454502448 18446744073709551615 215988612623 0 0 4096 65536 4578965106435555327 0 0 17 2 0 0 0
/proc/29750/statm: 4581 390 341 23 0 91 0
[pid=29751] ppid=29750 vsize=1447684 CPUtime=12.68 cores=0
/proc/29751/stat : 29751 (cryptominisat) R 29750 29750 28861 0 -1 4202496 57712 0 0 0 1260 8 0 0 25 0 1 0 600166997 1482428416 35148 33554432000 4194304 4587354 140733237865968 18446744073709551615 4365254 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/29751/statm: 361921 35148 337 96 0 358483 0
[pid=29752] ppid=29750 vsize=66976 CPUtime=12.68 cores=4
/proc/29752/stat : 29752 (clasp) R 29750 29750 28861 0 -1 4202496 19961 0 0 0 1266 2 0 0 25 0 1 0 600166997 68583424 13969 33554432000 4194304 4785740 140734299932624 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/29752/statm: 16744 13969 340 145 0 13789 0
[pid=29753] ppid=29750 vsize=495424 CPUtime=6.31 cores=6
/proc/29753/stat : 29753 (TNM) R 29750 29750 28861 0 -1 4202496 14756 0 0 0 630 1 0 0 25 0 1 0 600166997 507314176 14652 33554432000 134512640 134534564 4289823632 18446744073709551615 134524712 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/29753/statm: 123856 14652 91 6 0 122522 0
[pid=29754] ppid=29750 vsize=75200 CPUtime=12.68 cores=2
/proc/29754/stat : 29754 (lingeling) R 29750 29750 28861 0 -1 4202496 21428 0 0 0 1266 2 0 0 25 0 1 0 600166997 77004800 17194 33554432000 4194304 4352236 140735566239776 18446744073709551615 4267565 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/29754/statm: 18800 17194 126 39 0 17880 0
[pid=29755] ppid=29750 vsize=128164 CPUtime=6.3 cores=6
/proc/29755/stat : 29755 (march_hi) R 29750 29750 28861 0 -1 4202496 72008 0 0 0 620 10 0 0 25 0 1 0 600166997 131239936 15399 33554432000 4194304 4331988 140734883440608 18446744073709551615 4228235 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/29755/statm: 32041 15399 171 34 0 30483 0
Current children cumulated CPU time (s) 50.65
Current children cumulated vsize (KiB) 2231772

[startup+25.5069 s]
/proc/loadavg: 9.95 9.65 9.45 12/188 29757
/proc/meminfo: memFree=16646044/32950940 swapFree=67111528/67111528
[pid=29750] ppid=29748 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/29750/stat : 29750 (ppfolio) S 29748 29750 28861 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 600166996 18763776 390 33554432000 4194304 4287625 140734454502448 18446744073709551615 215988612623 0 0 4096 65536 4578965106435555327 0 0 17 2 0 0 0
/proc/29750/statm: 4581 390 341 23 0 91 0
[pid=29751] ppid=29750 vsize=1447684 CPUtime=25.49 cores=0
/proc/29751/stat : 29751 (cryptominisat) R 29750 29750 28861 0 -1 4202496 57712 0 0 0 2541 8 0 0 25 0 1 0 600166997 1482428416 35148 33554432000 4194304 4587354 140733237865968 18446744073709551615 4365356 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/29751/statm: 361921 35148 337 96 0 358483 0
[pid=29752] ppid=29750 vsize=136496 CPUtime=25.49 cores=4
/proc/29752/stat : 29752 (clasp) R 29750 29750 28861 0 -1 4202496 37157 0 0 0 2543 6 0 0 25 0 1 0 600166997 139771904 31165 33554432000 4194304 4785740 140734299932624 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/29752/statm: 34124 31165 340 145 0 31169 0
[pid=29753] ppid=29750 vsize=495424 CPUtime=12.65 cores=6
/proc/29753/stat : 29753 (TNM) R 29750 29750 28861 0 -1 4202496 14756 0 0 0 1264 1 0 0 25 0 1 0 600166997 507314176 14652 33554432000 134512640 134534564 4289823632 18446744073709551615 134524491 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/29753/statm: 123856 14652 91 6 0 122522 0
[pid=29754] ppid=29750 vsize=85252 CPUtime=25.48 cores=2
/proc/29754/stat : 29754 (lingeling) R 29750 29750 28861 0 -1 4202496 23685 0 0 0 2546 2 0 0 25 0 1 0 600166997 87298048 19451 33554432000 4194304 4352236 140735566239776 18446744073709551615 4329060 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/29754/statm: 21313 19451 126 39 0 20393 0
[pid=29755] ppid=29750 vsize=128264 CPUtime=12.69 cores=6
/proc/29755/stat : 29755 (march_hi) R 29750 29750 28861 0 -1 4202496 72059 0 0 0 1259 10 0 0 25 0 1 0 600166997 131342336 15450 33554432000 4194304 4331988 140734883440608 18446744073709551615 4226263 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/29755/statm: 32066 15450 171 34 0 30508 0
Current children cumulated CPU time (s) 101.8
Current children cumulated vsize (KiB) 2311444

[startup+51.1004 s]
/proc/loadavg: 9.97 9.68 9.46 12/188 29757
/proc/meminfo: memFree=16498464/32950940 swapFree=67111528/67111528
[pid=29750] ppid=29748 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/29750/stat : 29750 (ppfolio) S 29748 29750 28861 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 600166996 18763776 390 33554432000 4194304 4287625 140734454502448 18446744073709551615 215988612623 0 0 4096 65536 4578965106435555327 0 0 17 2 0 0 0
/proc/29750/statm: 4581 390 341 23 0 91 0
[pid=29751] ppid=29750 vsize=1447684 CPUtime=51.08 cores=0
/proc/29751/stat : 29751 (cryptominisat) R 29750 29750 28861 0 -1 4202496 57712 0 0 0 5100 8 0 0 25 0 1 0 600166997 1482428416 35148 33554432000 4194304 4587354 140733237865968 18446744073709551615 4365356 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/29751/statm: 361921 35148 337 96 0 358483 0
[pid=29752] ppid=29750 vsize=235816 CPUtime=51.09 cores=4
/proc/29752/stat : 29752 (clasp) R 29750 29750 28861 0 -1 4202496 61658 0 0 0 5096 13 0 0 25 0 1 0 600166997 241475584 55666 33554432000 4194304 4785740 140734299932624 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/29752/statm: 58954 55666 340 145 0 55999 0
[pid=29753] ppid=29750 vsize=502684 CPUtime=25.41 cores=6
/proc/29753/stat : 29753 (TNM) R 29750 29750 28861 0 -1 4202496 21432 0 0 0 2539 2 0 0 25 0 1 0 600166997 514748416 21328 33554432000 134512640 134534564 4289823632 18446744073709551615 134528295 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/29753/statm: 125671 21328 95 6 0 124337 0
[pid=29754] ppid=29750 vsize=102148 CPUtime=51.08 cores=2

################
# More data... #
################

/proc/29755/statm: 43481 23504 175 34 0 41923 0
Current children cumulated CPU time (s) 3763.94
Current children cumulated vsize (KiB) 3607852

[startup+1002.31 s]
/proc/loadavg: 10.11 10.14 9.90 12/188 29792
/proc/meminfo: memFree=15170148/32950940 swapFree=67111528/67111528
[pid=29750] ppid=29748 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/29750/stat : 29750 (ppfolio) S 29748 29750 28861 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 600166996 18763776 390 33554432000 4194304 4287625 140734454502448 18446744073709551615 215988612623 0 0 4096 65536 4578965106435555327 0 0 17 2 0 0 0
/proc/29750/statm: 4581 390 341 23 0 91 0
[pid=29751] ppid=29750 vsize=1697152 CPUtime=1002.3 cores=0
/proc/29751/stat : 29751 (cryptominisat) R 29750 29750 28861 0 -1 4202496 277398 0 0 0 100171 59 0 0 25 0 1 0 600166997 1737883648 88127 33554432000 4194304 4587354 140733237865968 18446744073709551615 4238477 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/29751/statm: 424288 88127 337 96 0 420850 0
[pid=29752] ppid=29750 vsize=985404 CPUtime=1002.31 cores=4
/proc/29752/stat : 29752 (clasp) R 29750 29750 28861 0 -1 4202496 243567 0 0 0 100167 64 0 0 25 0 1 0 600166997 1009053696 237575 33554432000 4194304 4785740 140734299932624 18446744073709551615 4287241 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/29752/statm: 246351 237575 340 145 0 243396 0
[pid=29753] ppid=29750 vsize=502684 CPUtime=498.42 cores=6
/proc/29753/stat : 29753 (TNM) R 29750 29750 28861 0 -1 4202496 21432 0 0 0 49840 2 0 0 25 0 1 0 600166997 514748416 21328 33554432000 134512640 134534564 4289823632 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/29753/statm: 125671 21328 95 6 0 124337 0
[pid=29754] ppid=29750 vsize=206748 CPUtime=1002.17 cores=2
/proc/29754/stat : 29754 (lingeling) R 29750 29750 28861 0 -1 4202496 159018 0 0 0 100200 17 0 0 25 0 1 0 600166997 211709952 50848 33554432000 4194304 4352236 140735566239776 18446744073709551615 4222713 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/29754/statm: 51687 50848 126 39 0 50767 0
[pid=29755] ppid=29750 vsize=173924 CPUtime=498.42 cores=6
/proc/29755/stat : 29755 (march_hi) R 29750 29750 28861 0 -1 4202496 80558 0 0 0 49828 14 0 0 25 0 1 0 600166997 178098176 23949 33554432000 4194304 4331988 140734883440608 18446744073709551615 4287882 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/29755/statm: 43481 23949 175 34 0 41923 0
Current children cumulated CPU time (s) 4003.62
Current children cumulated vsize (KiB) 3584236

[startup+1062.3 s]
/proc/loadavg: 10.23 10.17 9.92 12/188 29793
/proc/meminfo: memFree=15049572/32950940 swapFree=67111528/67111528
[pid=29750] ppid=29748 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/29750/stat : 29750 (ppfolio) S 29748 29750 28861 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 600166996 18763776 390 33554432000 4194304 4287625 140734454502448 18446744073709551615 215988612623 0 0 4096 65536 4578965106435555327 0 0 17 2 0 0 0
/proc/29750/statm: 4581 390 341 23 0 91 0
[pid=29751] ppid=29750 vsize=1726636 CPUtime=1062.3 cores=0
/proc/29751/stat : 29751 (cryptominisat) R 29750 29750 28861 0 -1 4202496 290381 0 0 0 106166 64 0 0 25 0 1 0 600166997 1768075264 84320 33554432000 4194304 4587354 140733237865968 18446744073709551615 4365241 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/29751/statm: 431659 84320 337 96 0 428221 0
[pid=29752] ppid=29750 vsize=1037192 CPUtime=1062.31 cores=4
/proc/29752/stat : 29752 (clasp) R 29750 29750 28861 0 -1 4202496 256126 0 0 0 106162 69 0 0 25 0 1 0 600166997 1062084608 250134 33554432000 4194304 4785740 140734299932624 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/29752/statm: 259298 250134 340 145 0 256343 0
[pid=29753] ppid=29750 vsize=502684 CPUtime=528.23 cores=6
/proc/29753/stat : 29753 (TNM) R 29750 29750 28861 0 -1 4202496 21432 0 0 0 52821 2 0 0 25 0 1 0 600166997 514748416 21328 33554432000 134512640 134534564 4289823632 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/29753/statm: 125671 21328 95 6 0 124337 0
[pid=29754] ppid=29750 vsize=276380 CPUtime=1062.17 cores=2
/proc/29754/stat : 29754 (lingeling) R 29750 29750 28861 0 -1 4202496 181546 0 0 0 106197 20 0 0 25 0 1 0 600166997 283013120 68256 33554432000 4194304 4352236 140735566239776 18446744073709551615 4208047 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/29754/statm: 69095 68256 126 39 0 68175 0
[pid=29755] ppid=29750 vsize=173924 CPUtime=528.28 cores=6
/proc/29755/stat : 29755 (march_hi) R 29750 29750 28861 0 -1 4202496 80993 0 0 0 52813 15 0 0 25 0 1 0 600166997 178098176 24384 33554432000 4194304 4331988 140734883440608 18446744073709551615 4284315 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/29755/statm: 43481 24384 175 34 0 41923 0
Current children cumulated CPU time (s) 4243.29
Current children cumulated vsize (KiB) 3735140

[startup+1122.3 s]
/proc/loadavg: 10.08 10.14 9.92 12/188 29795
/proc/meminfo: memFree=15042712/32950940 swapFree=67111528/67111528
[pid=29750] ppid=29748 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/29750/stat : 29750 (ppfolio) S 29748 29750 28861 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 600166996 18763776 390 33554432000 4194304 4287625 140734454502448 18446744073709551615 215988612623 0 0 4096 65536 4578965106435555327 0 0 17 2 0 0 0
/proc/29750/statm: 4581 390 341 23 0 91 0
[pid=29751] ppid=29750 vsize=1669140 CPUtime=1122.3 cores=0
/proc/29751/stat : 29751 (cryptominisat) R 29750 29750 28861 0 -1 4202496 302660 0 0 0 112161 69 0 0 25 0 1 0 600166997 1709199360 82613 33554432000 4194304 4587354 140733237865968 18446744073709551615 4231923 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/29751/statm: 417285 82613 337 96 0 413847 0
[pid=29752] ppid=29750 vsize=1074492 CPUtime=1122.31 cores=4
/proc/29752/stat : 29752 (clasp) R 29750 29750 28861 0 -1 4202496 265247 0 0 0 112160 71 0 0 25 0 1 0 600166997 1100279808 259255 33554432000 4194304 4785740 140734299932624 18446744073709551615 4553399 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/29752/statm: 268623 259255 340 145 0 265668 0
[pid=29753] ppid=29750 vsize=502684 CPUtime=558.07 cores=6
/proc/29753/stat : 29753 (TNM) R 29750 29750 28861 0 -1 4202496 21432 0 0 0 55805 2 0 0 25 0 1 0 600166997 514748416 21328 33554432000 134512640 134534564 4289823632 18446744073709551615 134528957 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/29753/statm: 125671 21328 95 6 0 124337 0
[pid=29754] ppid=29750 vsize=219036 CPUtime=1122.16 cores=2
/proc/29754/stat : 29754 (lingeling) R 29750 29750 28861 0 -1 4202496 182570 0 0 0 112196 20 0 0 25 0 1 0 600166997 224292864 53920 33554432000 4194304 4352236 140735566239776 18446744073709551615 4328126 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/29754/statm: 54759 53920 126 39 0 53839 0
[pid=29755] ppid=29750 vsize=173924 CPUtime=558.11 cores=6
/proc/29755/stat : 29755 (march_hi) R 29750 29750 28861 0 -1 4202496 81437 0 0 0 55796 15 0 0 25 0 1 0 600166997 178098176 24828 33554432000 4194304 4331988 140734883440608 18446744073709551615 4237139 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/29755/statm: 43481 24828 175 34 0 41923 0
Current children cumulated CPU time (s) 4482.95
Current children cumulated vsize (KiB) 3657600

[startup+1182.3 s]
/proc/loadavg: 7.66 9.51 9.72 6/177 29815
/proc/meminfo: memFree=15677340/32950940 swapFree=67111528/67111528
[pid=29750] ppid=29748 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/29750/stat : 29750 (ppfolio) S 29748 29750 28861 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 600166996 18763776 390 33554432000 4194304 4287625 140734454502448 18446744073709551615 215988612623 0 0 4096 65536 4578965106435555327 0 0 17 2 0 0 0
/proc/29750/statm: 4581 390 341 23 0 91 0
[pid=29751] ppid=29750 vsize=1662520 CPUtime=1182.3 cores=0
/proc/29751/stat : 29751 (cryptominisat) R 29750 29750 28861 0 -1 4202496 320647 0 0 0 118153 77 0 0 25 0 1 0 600166997 1702420480 77583 33554432000 4194304 4587354 140733237865968 18446744073709551615 4237665 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/29751/statm: 415630 77583 337 96 0 412192 0
[pid=29752] ppid=29750 vsize=1079332 CPUtime=1182.31 cores=4
/proc/29752/stat : 29752 (clasp) R 29750 29750 28861 0 -1 4202496 266359 0 0 0 118159 72 0 0 25 0 1 0 600166997 1105235968 260367 33554432000 4194304 4785740 140734299932624 18446744073709551615 4287008 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/29752/statm: 269833 260367 340 145 0 266878 0
[pid=29753] ppid=29750 vsize=502684 CPUtime=587.92 cores=6
/proc/29753/stat : 29753 (TNM) R 29750 29750 28861 0 -1 4202496 21432 0 0 0 58790 2 0 0 25 0 1 0 600166997 514748416 21328 33554432000 134512640 134534564 4289823632 18446744073709551615 134530360 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/29753/statm: 125671 21328 95 6 0 124337 0
[pid=29754] ppid=29750 vsize=206748 CPUtime=1182.15 cores=2
/proc/29754/stat : 29754 (lingeling) R 29750 29750 28861 0 -1 4202496 188714 0 0 0 118194 21 0 0 25 0 1 0 600166997 211709952 50848 33554432000 4194304 4352236 140735566239776 18446744073709551615 4329004 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/29754/statm: 51687 50848 126 39 0 50767 0
[pid=29755] ppid=29750 vsize=173924 CPUtime=587.95 cores=6
/proc/29755/stat : 29755 (march_hi) R 29750 29750 28861 0 -1 4202496 81874 0 0 0 58780 15 0 0 25 0 1 0 600166997 178098176 25265 33554432000 4194304 4331988 140734883440608 18446744073709551615 4228247 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/29755/statm: 43481 25265 175 34 0 41923 0
Current children cumulated CPU time (s) 4722.63
Current children cumulated vsize (KiB) 3643532



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 6.97 9.23 9.62 6/177 29815
/proc/meminfo: memFree=15668540/32950940 swapFree=67111528/67111528
[pid=29750] ppid=29748 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/29750/stat : 29750 (ppfolio) S 29748 29750 28861 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 600166996 18763776 390 33554432000 4194304 4287625 140734454502448 18446744073709551615 215988612623 0 0 4096 65536 4578965106435555327 0 0 17 2 0 0 0
/proc/29750/statm: 4581 390 341 23 0 91 0
[pid=29751] ppid=29750 vsize=1708688 CPUtime=1201.7 cores=0
/proc/29751/stat : 29751 (cryptominisat) R 29750 29750 28861 0 -1 4202496 327631 0 0 0 120091 79 0 0 25 0 1 0 600166997 1749696512 79779 33554432000 4194304 4587354 140733237865968 18446744073709551615 4237681 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/29751/statm: 427172 79779 337 96 0 423734 0
[pid=29752] ppid=29750 vsize=1079332 CPUtime=1201.72 cores=4
/proc/29752/stat : 29752 (clasp) R 29750 29750 28861 0 -1 4202496 266374 0 0 0 120100 72 0 0 25 0 1 0 600166997 1105235968 260382 33554432000 4194304 4785740 140734299932624 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/29752/statm: 269833 260382 340 145 0 266878 0
[pid=29753] ppid=29750 vsize=502684 CPUtime=597.58 cores=6
/proc/29753/stat : 29753 (TNM) R 29750 29750 28861 0 -1 4202496 21432 0 0 0 59756 2 0 0 25 0 1 0 600166997 514748416 21328 33554432000 134512640 134534564 4289823632 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/29753/statm: 125671 21328 95 6 0 124337 0
[pid=29754] ppid=29750 vsize=206748 CPUtime=1201.56 cores=2
/proc/29754/stat : 29754 (lingeling) R 29750 29750 28861 0 -1 4202496 191786 0 0 0 120134 22 0 0 25 0 1 0 600166997 211709952 50848 33554432000 4194304 4352236 140735566239776 18446744073709551615 4328026 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/29754/statm: 51687 50848 126 39 0 50767 0
[pid=29755] ppid=29750 vsize=173924 CPUtime=597.61 cores=6
/proc/29755/stat : 29755 (march_hi) R 29750 29750 28861 0 -1 4202496 82019 0 0 0 59745 16 0 0 25 0 1 0 600166997 178098176 25410 33554432000 4194304 4331988 140734883440608 18446744073709551615 4228247 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/29755/statm: 43481 25410 175 34 0 41923 0
Current children cumulated CPU time (s) 4800.17
Current children cumulated vsize (KiB) 3689700

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.7 s]
/proc/loadavg: 6.97 9.23 9.62 6/177 29815
/proc/meminfo: memFree=15668540/32950940 swapFree=67111528/67111528
[pid=29750] ppid=29748 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/29750/stat : 29750 (ppfolio) S 29748 29750 28861 0 -1 4202496 524 0 0 0 0 0 0 0 15 0 1 0 600166996 18763776 390 33554432000 4194304 4287625 140734454502448 18446744073709551615 215988612623 0 0 4096 65536 4578965106435555327 0 0 17 2 0 0 0
/proc/29750/statm: 4581 390 341 23 0 91 0
[pid=29751] ppid=29750 vsize=1708688 CPUtime=1201.7 cores=0
/proc/29751/stat : 29751 (cryptominisat) R 29750 29750 28861 0 -1 4202496 327631 0 0 0 120091 79 0 0 25 0 1 0 600166997 1749696512 79779 33554432000 4194304 4587354 140733237865968 18446744073709551615 4237681 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/29751/statm: 427172 79779 337 96 0 423734 0
[pid=29752] ppid=29750 vsize=1079332 CPUtime=1201.72 cores=4
/proc/29752/stat : 29752 (clasp) R 29750 29750 28861 0 -1 4202496 266374 0 0 0 120100 72 0 0 25 0 1 0 600166997 1105235968 260382 33554432000 4194304 4785740 140734299932624 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/29752/statm: 269833 260382 340 145 0 266878 0
[pid=29753] ppid=29750 vsize=502684 CPUtime=597.58 cores=6
/proc/29753/stat : 29753 (TNM) R 29750 29750 28861 0 -1 4202496 21432 0 0 0 59756 2 0 0 25 0 1 0 600166997 514748416 21328 33554432000 134512640 134534564 4289823632 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/29753/statm: 125671 21328 95 6 0 124337 0
[pid=29754] ppid=29750 vsize=206748 CPUtime=1201.56 cores=2
/proc/29754/stat : 29754 (lingeling) R 29750 29750 28861 0 -1 4202496 191786 0 0 0 120134 22 0 0 25 0 1 0 600166997 211709952 50848 33554432000 4194304 4352236 140735566239776 18446744073709551615 4328026 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/29754/statm: 51687 50848 126 39 0 50767 0
[pid=29755] ppid=29750 vsize=173924 CPUtime=597.61 cores=6
/proc/29755/stat : 29755 (march_hi) R 29750 29750 28861 0 -1 4202496 82019 0 0 0 59745 16 0 0 25 0 1 0 600166997 178098176 25410 33554432000 4194304 4331988 140734883440608 18446744073709551615 4228247 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/29755/statm: 43481 25410 175 34 0 41923 0
Current children cumulated CPU time (s) 4800.17
Current children cumulated vsize (KiB) 3689700

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 29750 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=2999
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=4999
# CPU time returned by wait4() is 0.007998
# while last known CPU time is 4800.17
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1201.71
CPU time (s): 4800.17
CPU user time (s): 4798.26
CPU system time (s): 1.91
CPU usage (%): 399.444
Max. virtual memory (cumulated for all children) (KiB): 5013696

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.002999
system time used= 0.004999
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 524
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 235
involuntary context switches= 6

runsolver used 1.9767 second user time and 4.55031 second system time

The end

Launcher Data 

Begin job on node117 at 2011-04-29 19:49:22
IDJOB=3251347
IDBENCH=88219
IDSOLVER=1612
FILE ID=node117/3251347-1304099362
RUNJOBID= node117-1304095715-28879
PBS_JOBID= 13170599
Free space on /tmp= 71660 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m62-q1504.5-n60-p30-s88600538.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3251347-1304099362/watcher-3251347-1304099362 -o /tmp/evaluation-result-3251347-1304099362/solver-3251347-1304099362 -C 4800 -W 1300 -M 15500  ppfolio HOME/instance-3251347-1304099362.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 76c30bc0971bc865c2b988ddaca89385
RANDOM SEED=2000291654

node117.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5267.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950940 kB
MemFree:      17120408 kB
Buffers:       2381456 kB
Cached:       12514436 kB
SwapCached:          0 kB
Active:        8469700 kB
Inactive:      6777980 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950940 kB
LowFree:      17120408 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           22164 kB
Writeback:           0 kB
AnonPages:      351116 kB
Mapped:          16716 kB
Slab:           519428 kB
PageTables:       5300 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586996 kB
Committed_AS:  1040476 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71660 MiB
End job on node117 at 2011-04-29 20:09:26