Trace number 3219190

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio seq? (TO) 5000 4999.91

General information on the benchmark

Namerandom/large/
unif-k5-r20-v1000-c20000-S72635791-098-UNKNOWN.cnf
MD5SUM93976db0d99d3daab756f06103f77642
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark20.2519
Satisfiable
(Un)Satisfiability was proved
Number of variables1000
Number of clauses20000
Sum of the clauses size100000
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 520000
Number of clauses of size over 50

Solver Data 

0.00/0.01	This is pico-portfolio, a naive parallel portfolio
0.00/0.01	Copyright (C) 2011 Olivier Roussel
0.00/0.01	Using 1 processing units
0.00/0.01	This portfolio uses the following solvers:
0.00/0.01	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.01	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.01	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.01	Starting cryptominisat 2.7.1 on core(s) 1
0.00/0.01	Starting clasp 1.3.6 on core(s) 1
0.00/0.01	Starting TNM 2009 on core(s) 1

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3219190-1303223105/watcher-3219190-1303223105 -o /tmp/evaluation-result-3219190-1303223105/solver-3219190-1303223105 -C 5000 -W 5000 -M 15500 ppfolio --nbcore=1 HOME/instance-3219190-1303223105.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 5030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.08 6.01 5.86 4/180 830
/proc/meminfo: memFree=27616316/32951124 swapFree=67051452/67111528
[pid=830] ppid=828 vsize=18188 CPUtime=0 cores=1,3,5,7
/proc/830/stat : 830 (ppfolio) R 828 830 32262 0 -1 4194304 361 0 0 0 0 0 0 0 25 0 1 0 433913306 18624512 299 33554432000 4194304 4287625 140733400228208 18446744073709551615 47373151496592 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/830/statm: 4547 306 262 23 0 57 0

[startup+0.0573651 s]
/proc/loadavg: 6.08 6.01 5.86 4/180 830
/proc/meminfo: memFree=27616316/32951124 swapFree=67051452/67111528
[pid=830] ppid=828 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/830/stat : 830 (ppfolio) S 828 830 32262 0 -1 4202496 498 0 0 0 0 0 0 0 25 0 1 0 433913306 18763776 393 33554432000 4194304 4287625 140733400228208 18446744073709551615 218370977295 0 0 4096 65536 9440197108536672520 0 0 17 7 0 0 0
/proc/830/statm: 4581 393 345 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100359 s]
/proc/loadavg: 6.08 6.01 5.86 4/180 830
/proc/meminfo: memFree=27616316/32951124 swapFree=67051452/67111528
[pid=830] ppid=828 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/830/stat : 830 (ppfolio) S 828 830 32262 0 -1 4202496 498 0 0 0 0 0 0 0 25 0 1 0 433913306 18763776 393 33554432000 4194304 4287625 140733400228208 18446744073709551615 218370977295 0 0 4096 65536 9440197108536672520 0 0 17 7 0 0 0
/proc/830/statm: 4581 393 345 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300323 s]
/proc/loadavg: 6.08 6.01 5.86 4/180 830
/proc/meminfo: memFree=27616316/32951124 swapFree=67051452/67111528
[pid=830] ppid=828 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/830/stat : 830 (ppfolio) S 828 830 32262 0 -1 4202496 499 0 0 0 0 0 0 0 23 0 1 0 433913306 18763776 393 33554432000 4194304 4287625 140733400228208 18446744073709551615 218370977295 0 0 4096 65536 0 0 0 17 7 0 0 0
/proc/830/statm: 4581 393 345 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700261 s]
/proc/loadavg: 6.08 6.01 5.86 4/180 830
/proc/meminfo: memFree=27616316/32951124 swapFree=67051452/67111528
[pid=830] ppid=828 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/830/stat : 830 (ppfolio) S 828 830 32262 0 -1 4202496 499 0 0 0 0 0 0 0 23 0 1 0 433913306 18763776 393 33554432000 4194304 4287625 140733400228208 18446744073709551615 218370977295 0 0 4096 65536 0 0 0 17 7 0 0 0
/proc/830/statm: 4581 393 345 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50111 s]
/proc/loadavg: 6.08 6.01 5.86 7/184 843
/proc/meminfo: memFree=27597684/32951124 swapFree=67051452/67111528
[pid=830] ppid=828 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/830/stat : 830 (ppfolio) S 828 830 32262 0 -1 4202496 499 0 0 0 0 0 0 0 18 0 1 0 433913306 18763776 393 33554432000 4194304 4287625 140733400228208 18446744073709551615 218370977295 0 0 4096 65536 13905670478724333568 0 0 17 7 0 0 0
/proc/830/statm: 4581 393 345 23 0 91 0
[pid=831] ppid=830 vsize=729284 CPUtime=0.54 cores=1
/proc/831/stat : 831 (cryptominisat) R 830 830 32262 0 -1 4202496 3276 0 0 0 54 0 0 0 25 0 1 0 433913306 746786816 2150 33554432000 4194304 4587354 140735699859392 18446744073709551615 4238595 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/831/statm: 182321 2150 327 96 0 178883 0
[pid=832] ppid=830 vsize=18360 CPUtime=0.42 cores=1
/proc/832/stat : 832 (clasp) R 830 830 32262 0 -1 4202496 2118 0 0 0 42 0 0 0 25 0 1 0 433913306 18800640 1925 33554432000 4194304 4785740 140733243360352 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/832/statm: 4590 1925 340 145 0 1635 0
[pid=833] ppid=830 vsize=458888 CPUtime=0.52 cores=1
/proc/833/stat : 833 (TNM) R 830 830 32262 0 -1 4202496 1274 0 0 0 52 0 0 0 25 0 1 0 433913306 469901312 1171 33554432000 134512640 134534564 4291604304 18446744073709551615 134528726 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/833/statm: 114722 1171 95 6 0 113388 0
Current children cumulated CPU time (s) 1.48
Current children cumulated vsize (KiB) 1224856

[startup+3.10083 s]
/proc/loadavg: 6.08 6.01 5.86 7/184 843
/proc/meminfo: memFree=27592168/32951124 swapFree=67051452/67111528
[pid=830] ppid=828 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/830/stat : 830 (ppfolio) S 828 830 32262 0 -1 4202496 499 0 0 0 0 0 0 0 15 0 1 0 433913306 18763776 393 33554432000 4194304 4287625 140733400228208 18446744073709551615 218370977295 0 0 4096 65536 13905670753602240512 0 0 17 7 0 0 0
/proc/830/statm: 4581 393 345 23 0 91 0
[pid=831] ppid=830 vsize=725932 CPUtime=1.04 cores=1
/proc/831/stat : 831 (cryptominisat) R 830 830 32262 0 -1 4202496 4509 0 0 0 104 0 0 0 25 0 1 0 433913306 743354368 2354 33554432000 4194304 4587354 140735699859392 18446744073709551615 4238755 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/831/statm: 181483 2354 327 96 0 178045 0
[pid=832] ppid=830 vsize=19984 CPUtime=0.93 cores=1
/proc/832/stat : 832 (clasp) R 830 830 32262 0 -1 4202496 2512 0 0 0 93 0 0 0 25 0 1 0 433913306 20463616 2319 33554432000 4194304 4785740 140733243360352 18446744073709551615 4550316 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/832/statm: 4996 2319 340 145 0 2041 0
[pid=833] ppid=830 vsize=458888 CPUtime=1.11 cores=1
/proc/833/stat : 833 (TNM) R 830 830 32262 0 -1 4202496 1274 0 0 0 111 0 0 0 25 0 1 0 433913306 469901312 1171 33554432000 134512640 134534564 4291604304 18446744073709551615 134530632 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/833/statm: 114722 1171 95 6 0 113388 0
Current children cumulated CPU time (s) 3.08
Current children cumulated vsize (KiB) 1223128

[startup+6.30025 s]
/proc/loadavg: 6.07 6.00 5.86 7/184 843
/proc/meminfo: memFree=27586912/32951124 swapFree=67051452/67111528
[pid=830] ppid=828 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/830/stat : 830 (ppfolio) S 828 830 32262 0 -1 4202496 499 0 0 0 0 0 0 0 15 0 1 0 433913306 18763776 393 33554432000 4194304 4287625 140733400228208 18446744073709551615 218370977295 0 0 4096 65536 13905669104334798848 0 0 17 7 0 0 0
/proc/830/statm: 4581 393 345 23 0 91 0
[pid=831] ppid=830 vsize=730160 CPUtime=2.14 cores=1
/proc/831/stat : 831 (cryptominisat) R 830 830 32262 0 -1 4202496 7113 0 0 0 214 0 0 0 25 0 1 0 433913306 747683840 2396 33554432000 4194304 4587354 140735699859392 18446744073709551615 4238755 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/831/statm: 182540 2396 331 96 0 179102 0
[pid=832] ppid=830 vsize=22096 CPUtime=2.02 cores=1
/proc/832/stat : 832 (clasp) R 830 830 32262 0 -1 4202496 3055 0 0 0 202 0 0 0 25 0 1 0 433913306 22626304 2862 33554432000 4194304 4785740 140733243360352 18446744073709551615 4549076 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/832/statm: 5524 2862 340 145 0 2569 0
[pid=833] ppid=830 vsize=458888 CPUtime=2.12 cores=1
/proc/833/stat : 833 (TNM) R 830 830 32262 0 -1 4202496 1274 0 0 0 212 0 0 0 25 0 1 0 433913306 469901312 1171 33554432000 134512640 134534564 4291604304 18446744073709551615 134528944 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/833/statm: 114722 1171 95 6 0 113388 0
Current children cumulated CPU time (s) 6.28
Current children cumulated vsize (KiB) 1229468

[startup+12.7011 s]
/proc/loadavg: 6.07 6.00 5.86 7/184 843
/proc/meminfo: memFree=27563656/32951124 swapFree=67051452/67111528
[pid=830] ppid=828 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/830/stat : 830 (ppfolio) S 828 830 32262 0 -1 4202496 499 0 0 0 0 0 0 0 15 0 1 0 433913306 18763776 393 33554432000 4194304 4287625 140733400228208 18446744073709551615 218370977295 0 0 4096 65536 13905669104334798848 0 0 17 7 0 0 0
/proc/830/statm: 4581 393 345 23 0 91 0
[pid=831] ppid=830 vsize=740028 CPUtime=4.24 cores=1
/proc/831/stat : 831 (cryptominisat) R 830 830 32262 0 -1 4202496 13022 0 0 0 423 1 0 0 25 0 1 0 433913306 757788672 6743 33554432000 4194304 4587354 140735699859392 18446744073709551615 4243968 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/831/statm: 185007 6743 337 96 0 181569 0
[pid=832] ppid=830 vsize=24736 CPUtime=4.13 cores=1
/proc/832/stat : 832 (clasp) R 830 830 32262 0 -1 4202496 3697 0 0 0 413 0 0 0 25 0 1 0 433913306 25329664 3504 33554432000 4194304 4785740 140733243360352 18446744073709551615 4287244 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/832/statm: 6184 3504 340 145 0 3229 0
[pid=833] ppid=830 vsize=458888 CPUtime=4.31 cores=1
/proc/833/stat : 833 (TNM) R 830 830 32262 0 -1 4202496 1274 0 0 0 431 0 0 0 25 0 1 0 433913306 469901312 1171 33554432000 134512640 134534564 4291604304 18446744073709551615 134528913 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/833/statm: 114722 1171 95 6 0 113388 0
Current children cumulated CPU time (s) 12.68
Current children cumulated vsize (KiB) 1241976

[startup+25.5008 s]
/proc/loadavg: 6.05 6.00 5.86 7/184 844
/proc/meminfo: memFree=27548284/32951124 swapFree=67051452/67111528
[pid=830] ppid=828 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/830/stat : 830 (ppfolio) S 828 830 32262 0 -1 4202496 499 0 0 0 0 0 0 0 15 0 1 0 433913306 18763776 393 33554432000 4194304 4287625 140733400228208 18446744073709551615 218370977295 0 0 4096 65536 13905668966895845376 0 0 17 7 0 0 0
/proc/830/statm: 4581 393 345 23 0 91 0
[pid=831] ppid=830 vsize=750764 CPUtime=8.54 cores=1
/proc/831/stat : 831 (cryptominisat) R 830 830 32262 0 -1 4202496 15606 0 0 0 853 1 0 0 25 0 1 0 433913306 768782336 7577 33554432000 4194304 4587354 140735699859392 18446744073709551615 4238690 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/831/statm: 187691 7577 337 96 0 184253 0
[pid=832] ppid=830 vsize=27940 CPUtime=8.41 cores=1
/proc/832/stat : 832 (clasp) R 830 830 32262 0 -1 4202496 4496 0 0 0 841 0 0 0 25 0 1 0 433913306 28610560 4303 33554432000 4194304 4785740 140733243360352 18446744073709551615 4287241 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/832/statm: 6985 4303 340 145 0 4030 0
[pid=833] ppid=830 vsize=458888 CPUtime=8.52 cores=1
/proc/833/stat : 833 (TNM) R 830 830 32262 0 -1 4202496 1274 0 0 0 852 0 0 0 25 0 1 0 433913306 469901312 1171 33554432000 134512640 134534564 4291604304 18446744073709551615 134528829 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/833/statm: 114722 1171 95 6 0 113388 0
Current children cumulated CPU time (s) 25.47
Current children cumulated vsize (KiB) 1255916

[startup+51.1003 s]
/proc/loadavg: 6.03 6.00 5.86 7/184 844
/proc/meminfo: memFree=27506460/32951124 swapFree=67051452/67111528
[pid=830] ppid=828 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/830/stat : 830 (ppfolio) S 828 830 32262 0 -1 4202496 499 0 0 0 0 0 0 0 15 0 1 0 433913306 18763776 393 33554432000 4194304 4287625 140733400228208 18446744073709551615 218370977295 0 0 4096 65536 13905669516651659264 0 0 17 7 0 0 0
/proc/830/statm: 4581 393 345 23 0 91 0
[pid=831] ppid=830 vsize=768832 CPUtime=17.04 cores=1
/proc/831/stat : 831 (cryptominisat) R 830 830 32262 0 -1 4202496 23190 0 0 0 1701 3 0 0 25 0 1 0 433913306 787283968 13137 33554432000 4194304 4587354 140735699859392 18446744073709551615 4364784 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/831/statm: 192208 13137 337 96 0 188770 0
[pid=832] ppid=830 vsize=32576 CPUtime=16.93 cores=1
/proc/832/stat : 832 (clasp) R 830 830 32262 0 -1 4202496 5646 0 0 0 1693 0 0 0 25 0 1 0 433913306 33357824 5453 33554432000 4194304 4785740 140733243360352 18446744073709551615 4287241 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/832/statm: 8144 5453 340 145 0 5189 0
[pid=833] ppid=830 vsize=458888 CPUtime=17.1 cores=1
/proc/833/stat : 833 (TNM) R 830 830 32262 0 -1 4202496 1274 0 0 0 1710 0 0 0 25 0 1 0 433913306 469901312 1171 33554432000 134512640 134534564 4291604304 18446744073709551615 134528726 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/833/statm: 114722 1171 95 6 0 113388 0
Current children cumulated CPU time (s) 51.07
Current children cumulated vsize (KiB) 1278620

[startup+102.306 s]
/proc/loadavg: 6.01 6.00 5.87 7/184 846
/proc/meminfo: memFree=27247628/32951124 swapFree=67051452/67111528
[pid=830] ppid=828 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/830/stat : 830 (ppfolio) S 828 830 32262 0 -1 4202496 499 0 0 0 0 0 0 0 15 0 1 0 433913306 18763776 393 33554432000 4194304 4287625 140733400228208 18446744073709551615 218370977295 0 0 4096 65536 13905670066407473152 0 0 17 7 0 0 0
/proc/830/statm: 4581 393 345 23 0 91 0
[pid=831] ppid=830 vsize=786524 CPUtime=34.13 cores=1
/proc/831/stat : 831 (cryptominisat) R 830 830 32262 0 -1 4202496 44778 0 0 0 3408 5 0 0 25 0 1 0 433913306 805400576 18198 33554432000 4194304 4587354 140735699859392 18446744073709551615 4365356 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/831/statm: 196631 18198 337 96 0 193193 0
[pid=832] ppid=830 vsize=38168 CPUtime=34.01 cores=1
/proc/832/stat : 832 (clasp) R 830 830 32262 0 -1 4202496 7027 0 0 0 3401 0 0 0 25 0 1 0 433913306 39084032 6834 33554432000 4194304 4785740 140733243360352 18446744073709551615 4287241 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/832/statm: 9542 6834 340 145 0 6587 0
[pid=833] ppid=830 vsize=458888 CPUtime=34.14 cores=1
/proc/833/stat : 833 (TNM) R 830 830 32262 0 -1 4202496 1274 0 0 0 3414 0 0 0 25 0 1 0 433913306 469901312 1171 33554432000 134512640 134534564 4291604304 18446744073709551615 8033680 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/833/statm: 114722 1171 95 6 0 113388 0
Current children cumulated CPU time (s) 102.28
Current children cumulated vsize (KiB) 1301904

[startup+162.301 s]
/proc/loadavg: 6.00 6.00 5.87 7/184 847
/proc/meminfo: memFree=27222784/32951124 swapFree=67051452/67111528
[pid=830] ppid=828 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/830/stat : 830 (ppfolio) S 828 830 32262 0 -1 4202496 499 0 0 0 0 0 0 0 15 0 1 0 433913306 18763776 393 33554432000 4194304 4287625 140733400228208 18446744073709551615 218370977295 0 0 4096 65536 13905670616163287040 0 0 17 7 0 0 0
/proc/830/statm: 4581 393 345 23 0 91 0
[pid=831] ppid=830 vsize=799236 CPUtime=54.13 cores=1
/proc/831/stat : 831 (cryptominisat) R 830 830 32262 0 -1 4202496 70035 0 0 0 5404 9 0 0 25 0 1 0 433913306 818417664 19614 33554432000 4194304 4587354 140735699859392 18446744073709551615 4388352 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/831/statm: 199809 19614 337 96 0 196371 0
[pid=832] ppid=830 vsize=44196 CPUtime=54.01 cores=1

################
# More data... #
################

/proc/833/statm: 114722 1171 95 6 0 113388 0
Current children cumulated CPU time (s) 4602.4
Current children cumulated vsize (KiB) 1420424

[startup+4662.3 s]
/proc/loadavg: 6.00 6.00 5.95 8/184 1079
/proc/meminfo: memFree=27078936/32951124 swapFree=67051452/67111528
[pid=830] ppid=828 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/830/stat : 830 (ppfolio) S 828 830 32262 0 -1 4202496 499 0 0 0 0 0 0 0 15 0 1 0 433913306 18763776 393 33554432000 4194304 4287625 140733400228208 18446744073709551615 218370977295 0 0 4096 65536 13905669104334798848 0 0 17 7 0 0 0
/proc/830/statm: 4581 393 345 23 0 91 0
[pid=831] ppid=830 vsize=802748 CPUtime=1554.14 cores=1
/proc/831/stat : 831 (cryptominisat) R 830 830 32262 0 -1 4202496 770163 0 0 0 155263 151 0 0 25 0 1 0 433913306 822013952 27779 33554432000 4194304 4587354 140735699859392 18446744073709551615 4237695 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/831/statm: 200687 27779 337 96 0 197249 0
[pid=832] ppid=830 vsize=141800 CPUtime=1554.08 cores=1
/proc/832/stat : 832 (clasp) R 830 830 32262 0 -1 4202496 31027 0 0 0 155403 5 0 0 25 0 1 0 433913306 145203200 30787 33554432000 4194304 4785740 140733243360352 18446744073709551615 4553344 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/832/statm: 35450 30787 341 145 0 32495 0
[pid=833] ppid=830 vsize=458888 CPUtime=1554.17 cores=1
/proc/833/stat : 833 (TNM) R 830 830 32262 0 -1 4202496 1274 0 0 0 155417 0 0 0 25 0 1 0 433913306 469901312 1171 33554432000 134512640 134534564 4291604304 18446744073709551615 134528957 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/833/statm: 114722 1171 95 6 0 113388 0
Current children cumulated CPU time (s) 4662.39
Current children cumulated vsize (KiB) 1421760

[startup+4722.31 s]
/proc/loadavg: 6.00 6.00 5.95 8/184 1081
/proc/meminfo: memFree=27162572/32951124 swapFree=67051452/67111528
[pid=830] ppid=828 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/830/stat : 830 (ppfolio) S 828 830 32262 0 -1 4202496 499 0 0 0 0 0 0 0 15 0 1 0 433913306 18763776 393 33554432000 4194304 4287625 140733400228208 18446744073709551615 218370977295 0 0 4096 65536 13905669104334798848 0 0 17 7 0 0 0
/proc/830/statm: 4581 393 345 23 0 91 0
[pid=831] ppid=830 vsize=770172 CPUtime=1574.16 cores=1
/proc/831/stat : 831 (cryptominisat) R 830 830 32262 0 -1 4202496 779229 0 0 0 157262 154 0 0 25 0 1 0 433913306 788656128 15321 33554432000 4194304 4587354 140735699859392 18446744073709551615 4238622 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/831/statm: 192543 15321 337 96 0 189105 0
[pid=832] ppid=830 vsize=141988 CPUtime=1574.08 cores=1
/proc/832/stat : 832 (clasp) R 830 830 32262 0 -1 4202496 31081 0 0 0 157403 5 0 0 25 0 1 0 433913306 145395712 30841 33554432000 4194304 4785740 140733243360352 18446744073709551615 4287028 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/832/statm: 35497 30841 341 145 0 32542 0
[pid=833] ppid=830 vsize=458888 CPUtime=1574.17 cores=1
/proc/833/stat : 833 (TNM) R 830 830 32262 0 -1 4202496 1274 0 0 0 157417 0 0 0 25 0 1 0 433913306 469901312 1171 33554432000 134512640 134534564 4291604304 18446744073709551615 134528737 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/833/statm: 114722 1171 95 6 0 113388 0
Current children cumulated CPU time (s) 4722.41
Current children cumulated vsize (KiB) 1389372

[startup+4782.3 s]
/proc/loadavg: 6.00 6.00 5.95 9/184 1083
/proc/meminfo: memFree=27100280/32951124 swapFree=67051452/67111528
[pid=830] ppid=828 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/830/stat : 830 (ppfolio) S 828 830 32262 0 -1 4202496 499 0 0 0 0 0 0 0 15 0 1 0 433913306 18763776 393 33554432000 4194304 4287625 140733400228208 18446744073709551615 218370977295 0 0 4096 65536 13905669104334798848 0 0 17 7 0 0 0
/proc/830/statm: 4581 393 345 23 0 91 0
[pid=831] ppid=830 vsize=770236 CPUtime=1594.16 cores=1
/proc/831/stat : 831 (cryptominisat) R 830 830 32262 0 -1 4202496 786639 0 0 0 159260 156 0 0 25 0 1 0 433913306 788721664 9213 33554432000 4194304 4587354 140735699859392 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/831/statm: 192559 9213 337 96 0 189121 0
[pid=832] ppid=830 vsize=142564 CPUtime=1594.07 cores=1
/proc/832/stat : 832 (clasp) R 830 830 32262 0 -1 4202496 31225 0 0 0 159402 5 0 0 25 0 1 0 433913306 145985536 30985 33554432000 4194304 4785740 140733243360352 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/832/statm: 35641 30985 341 145 0 32686 0
[pid=833] ppid=830 vsize=458888 CPUtime=1594.16 cores=1
/proc/833/stat : 833 (TNM) R 830 830 32262 0 -1 4202496 1274 0 0 0 159416 0 0 0 25 0 1 0 433913306 469901312 1171 33554432000 134512640 134534564 4291604304 18446744073709551615 134530126 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/833/statm: 114722 1171 95 6 0 113388 0
Current children cumulated CPU time (s) 4782.39
Current children cumulated vsize (KiB) 1390012

[startup+4842.3 s]
/proc/loadavg: 6.00 6.00 5.95 8/184 1084
/proc/meminfo: memFree=27151212/32951124 swapFree=67051452/67111528
[pid=830] ppid=828 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/830/stat : 830 (ppfolio) S 828 830 32262 0 -1 4202496 499 0 0 0 0 0 0 0 15 0 1 0 433913306 18763776 393 33554432000 4194304 4287625 140733400228208 18446744073709551615 218370977295 0 0 4096 65536 13905669241773752320 0 0 17 7 0 0 0
/proc/830/statm: 4581 393 345 23 0 91 0
[pid=831] ppid=830 vsize=770236 CPUtime=1614.17 cores=1
/proc/831/stat : 831 (cryptominisat) R 830 830 32262 0 -1 4202496 794486 0 0 0 161259 158 0 0 25 0 1 0 433913306 788721664 17008 33554432000 4194304 4587354 140735699859392 18446744073709551615 4238618 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/831/statm: 192559 17008 337 96 0 189121 0
[pid=832] ppid=830 vsize=143008 CPUtime=1614.07 cores=1
/proc/832/stat : 832 (clasp) R 830 830 32262 0 -1 4202496 31302 0 0 0 161402 5 0 0 25 0 1 0 433913306 146440192 31062 33554432000 4194304 4785740 140733243360352 18446744073709551615 4287042 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/832/statm: 35752 31062 341 145 0 32797 0
[pid=833] ppid=830 vsize=458888 CPUtime=1614.16 cores=1
/proc/833/stat : 833 (TNM) R 830 830 32262 0 -1 4202496 1274 0 0 0 161416 0 0 0 25 0 1 0 433913306 469901312 1171 33554432000 134512640 134534564 4291604304 18446744073709551615 134528918 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/833/statm: 114722 1171 95 6 0 113388 0
Current children cumulated CPU time (s) 4842.4
Current children cumulated vsize (KiB) 1390456

[startup+4902.3 s]
/proc/loadavg: 6.00 6.00 5.95 9/184 1086
/proc/meminfo: memFree=27136040/32951124 swapFree=67051452/67111528
[pid=830] ppid=828 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/830/stat : 830 (ppfolio) S 828 830 32262 0 -1 4202496 499 0 0 0 0 0 0 0 15 0 1 0 433913306 18763776 393 33554432000 4194304 4287625 140733400228208 18446744073709551615 218370977295 0 0 4096 65536 13905669241773752320 0 0 17 7 0 0 0
/proc/830/statm: 4581 393 345 23 0 91 0
[pid=831] ppid=830 vsize=755932 CPUtime=1634.17 cores=1
/proc/831/stat : 831 (cryptominisat) R 830 830 32262 0 -1 4202496 802409 0 0 0 163258 159 0 0 25 0 1 0 433913306 774074368 11282 33554432000 4194304 4587354 140735699859392 18446744073709551615 4238690 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/831/statm: 188983 11282 337 96 0 185545 0
[pid=832] ppid=830 vsize=143596 CPUtime=1634.07 cores=1
/proc/832/stat : 832 (clasp) R 830 830 32262 0 -1 4202496 31428 0 0 0 163402 5 0 0 25 0 1 0 433913306 147042304 31188 33554432000 4194304 4785740 140733243360352 18446744073709551615 4287252 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/832/statm: 35899 31188 341 145 0 32944 0
[pid=833] ppid=830 vsize=458888 CPUtime=1634.16 cores=1
/proc/833/stat : 833 (TNM) R 830 830 32262 0 -1 4202496 1274 0 0 0 163416 0 0 0 25 0 1 0 433913306 469901312 1171 33554432000 134512640 134534564 4291604304 18446744073709551615 134528726 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/833/statm: 114722 1171 95 6 0 113388 0
Current children cumulated CPU time (s) 4902.4
Current children cumulated vsize (KiB) 1376740

[startup+4962.3 s]
/proc/loadavg: 6.08 6.02 5.95 8/184 1087
/proc/meminfo: memFree=27135988/32951124 swapFree=67051452/67111528
[pid=830] ppid=828 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/830/stat : 830 (ppfolio) S 828 830 32262 0 -1 4202496 499 0 0 0 0 0 0 0 15 0 1 0 433913306 18763776 393 33554432000 4194304 4287625 140733400228208 18446744073709551615 218370977295 0 0 4096 65536 13905669241773752320 0 0 17 7 0 0 0
/proc/830/statm: 4581 393 345 23 0 91 0
[pid=831] ppid=830 vsize=755932 CPUtime=1654.18 cores=1
/proc/831/stat : 831 (cryptominisat) R 830 830 32262 0 -1 4202496 809434 0 0 0 165257 161 0 0 25 0 1 0 433913306 774074368 18260 33554432000 4194304 4587354 140735699859392 18446744073709551615 4238525 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/831/statm: 188983 18260 337 96 0 185545 0
[pid=832] ppid=830 vsize=144164 CPUtime=1654.06 cores=1
/proc/832/stat : 832 (clasp) R 830 830 32262 0 -1 4202496 31551 0 0 0 165401 5 0 0 25 0 1 0 433913306 147623936 31311 33554432000 4194304 4785740 140733243360352 18446744073709551615 4287244 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/832/statm: 36041 31311 341 145 0 33086 0
[pid=833] ppid=830 vsize=458888 CPUtime=1654.15 cores=1
/proc/833/stat : 833 (TNM) R 830 830 32262 0 -1 4202496 1274 0 0 0 165415 0 0 0 25 0 1 0 433913306 469901312 1171 33554432000 134512640 134534564 4291604304 18446744073709551615 134528726 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/833/statm: 114722 1171 95 6 0 113388 0
Current children cumulated CPU time (s) 4962.39
Current children cumulated vsize (KiB) 1377308



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+4999.9 s]
/proc/loadavg: 6.04 6.01 5.95 7/184 1088
/proc/meminfo: memFree=27145888/32951124 swapFree=67051452/67111528
[pid=830] ppid=828 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/830/stat : 830 (ppfolio) S 828 830 32262 0 -1 4202496 499 0 0 0 0 0 0 0 15 0 1 0 433913306 18763776 393 33554432000 4194304 4287625 140733400228208 18446744073709551615 218370977295 0 0 4096 65536 13905669241773752320 0 0 17 7 0 0 0
/proc/830/statm: 4581 393 345 23 0 91 0
[pid=831] ppid=830 vsize=770364 CPUtime=1666.69 cores=1
/proc/831/stat : 831 (cryptominisat) R 830 830 32262 0 -1 4202496 814506 0 0 0 166506 163 0 0 25 0 1 0 433913306 788852736 9590 33554432000 4194304 4587354 140735699859392 18446744073709551615 4238582 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/831/statm: 192591 9590 337 96 0 189153 0
[pid=832] ppid=830 vsize=144988 CPUtime=1666.56 cores=1
/proc/832/stat : 832 (clasp) R 830 830 32262 0 -1 4202496 31736 0 0 0 166651 5 0 0 25 0 1 0 433913306 148467712 31496 33554432000 4194304 4785740 140733243360352 18446744073709551615 4287241 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/832/statm: 36247 31496 341 145 0 33292 0
[pid=833] ppid=830 vsize=458888 CPUtime=1666.75 cores=1
/proc/833/stat : 833 (TNM) R 830 830 32262 0 -1 4202496 1274 0 0 0 166675 0 0 0 25 0 1 0 433913306 469901312 1171 33554432000 134512640 134534564 4291604304 18446744073709551615 134530367 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/833/statm: 114722 1171 95 6 0 113388 0
Current children cumulated CPU time (s) 5000
Current children cumulated vsize (KiB) 1392564

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+4999.9 s]
/proc/loadavg: 6.04 6.01 5.95 7/184 1088
/proc/meminfo: memFree=27145888/32951124 swapFree=67051452/67111528
[pid=830] ppid=828 vsize=18324 CPUtime=0 cores=1,3,5,7
/proc/830/stat : 830 (ppfolio) S 828 830 32262 0 -1 4202496 499 0 0 0 0 0 0 0 15 0 1 0 433913306 18763776 393 33554432000 4194304 4287625 140733400228208 18446744073709551615 218370977295 0 0 4096 65536 13905669241773752320 0 0 17 7 0 0 0
/proc/830/statm: 4581 393 345 23 0 91 0
[pid=831] ppid=830 vsize=770364 CPUtime=1666.69 cores=1
/proc/831/stat : 831 (cryptominisat) R 830 830 32262 0 -1 4202496 814506 0 0 0 166506 163 0 0 25 0 1 0 433913306 788852736 9590 33554432000 4194304 4587354 140735699859392 18446744073709551615 4238582 0 0 4096 2 0 0 0 17 1 0 0 0
/proc/831/statm: 192591 9590 337 96 0 189153 0
[pid=832] ppid=830 vsize=144988 CPUtime=1666.56 cores=1
/proc/832/stat : 832 (clasp) R 830 830 32262 0 -1 4202496 31736 0 0 0 166651 5 0 0 25 0 1 0 433913306 148467712 31496 33554432000 4194304 4785740 140733243360352 18446744073709551615 4287241 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/832/statm: 36247 31496 341 145 0 33292 0
[pid=833] ppid=830 vsize=458888 CPUtime=1666.75 cores=1
/proc/833/stat : 833 (TNM) R 830 830 32262 0 -1 4202496 1274 0 0 0 166675 0 0 0 25 0 1 0 433913306 469901312 1171 33554432000 134512640 134534564 4291604304 18446744073709551615 134530367 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/833/statm: 114722 1171 95 6 0 113388 0
Current children cumulated CPU time (s) 5000
Current children cumulated vsize (KiB) 1392564

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 830 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=3999
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=2999
# CPU time returned by wait4() is 0.006998
# while last known CPU time is 5000
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 4999.91
CPU time (s): 5000
CPU user time (s): 4998.32
CPU system time (s): 1.68
CPU usage (%): 100.002
Max. virtual memory (cumulated for all children) (KiB): 2129820

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.003999
system time used= 0.002999
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 499
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 289
involuntary context switches= 5

runsolver used 8.26174 second user time and 18.2532 second system time

The end

Launcher Data 

Begin job on node101 at 2011-04-19 16:25:05
IDJOB=3219190
IDBENCH=82548
IDSOLVER=1639
FILE ID=node101/3219190-1303223105
RUNJOBID= node101-1303203304-32281
PBS_JOBID= 13030636
Free space on /tmp= 72896 MiB

SOLVER NAME= ppfolio seq
BENCH NAME= SAT11/random/large/unif-k5-r20-v1000-c20000-S72635791-098-UNKNOWN.cnf
COMMAND LINE= ppfolio --nbcore=1 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3219190-1303223105/watcher-3219190-1303223105 -o /tmp/evaluation-result-3219190-1303223105/solver-3219190-1303223105 -C 5000 -W 5000 -M 15500  ppfolio --nbcore=1 HOME/instance-3219190-1303223105.cnf

TIME LIMIT= 5000 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 93976db0d99d3daab756f06103f77642
RANDOM SEED=1715172652

node101.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      27616472 kB
Buffers:        256620 kB
Cached:        4643436 kB
SwapCached:      33620 kB
Active:        2484696 kB
Inactive:      2665940 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      27616472 kB
SwapTotal:    67111528 kB
SwapFree:     67051452 kB
Dirty:           10128 kB
Writeback:           0 kB
AnonPages:      238500 kB
Mapped:          10528 kB
Slab:           120436 kB
PageTables:       5312 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:  2890692 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72884 MiB
End job on node101 at 2011-04-19 17:48:27