Trace number 3263097

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 4800.14 1201.71

General information on the benchmark

NameSAT03/handmade/anton/UNSAT/
SGI_30_80_17_90_7-log.shuffled-as.sat03-190.cnf
MD5SUMad30956261e72142648a01bc7c025dba
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables85
Number of clauses29282
Sum of the clauses size292650
Maximum clause length10
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 534
Number of clauses of size over 529248

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 4 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] lingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	[5] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 4
0.00/0.00	Starting TNM 2009 on core(s) 6
0.00/0.00	Starting lingeling 276-6264d55-100731 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 6

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3263097-1304090617/watcher-3263097-1304090617 -o /tmp/evaluation-result-3263097-1304090617/solver-3263097-1304090617 -C 4800 -W 1300 -M 15500 ppfolio HOME/instance-3263097-1304090617.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.39 7.70 7.57 7/181 20649
/proc/meminfo: memFree=18393736/32951124 swapFree=67110396/67111528
[pid=20648] ppid=20643 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/20648/stat : 20648 (ppfolio) R 20643 20648 20607 0 -1 4202496 521 0 0 0 0 0 0 0 25 0 1 0 599292398 18763776 390 33554432000 4194304 4287625 140736111077440 18446744073709551615 224284945935 0 0 4096 65536 0 0 0 17 4 0 0 0
/proc/20648/statm: 4581 390 341 23 0 91 0

[startup+0.074686 s]
/proc/loadavg: 6.39 7.70 7.57 7/181 20649
/proc/meminfo: memFree=18393736/32951124 swapFree=67110396/67111528
[pid=20648] ppid=20643 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/20648/stat : 20648 (ppfolio) R 20643 20648 20607 0 -1 4202496 521 0 0 0 0 0 0 0 25 0 1 0 599292398 18763776 390 33554432000 4194304 4287625 140736111077440 18446744073709551615 224284945935 0 0 4096 65536 0 0 0 17 4 0 0 0
/proc/20648/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100683 s]
/proc/loadavg: 6.39 7.70 7.57 7/181 20649
/proc/meminfo: memFree=18393736/32951124 swapFree=67110396/67111528
[pid=20648] ppid=20643 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/20648/stat : 20648 (ppfolio) R 20643 20648 20607 0 -1 4202496 521 0 0 0 0 0 0 0 25 0 1 0 599292398 18763776 390 33554432000 4194304 4287625 140736111077440 18446744073709551615 224284945935 0 0 4096 65536 0 0 0 17 4 0 0 0
/proc/20648/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300649 s]
/proc/loadavg: 6.39 7.70 7.57 7/181 20649
/proc/meminfo: memFree=18393736/32951124 swapFree=67110396/67111528
[pid=20648] ppid=20643 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/20648/stat : 20648 (ppfolio) S 20643 20648 20607 0 -1 4202496 523 0 0 0 0 0 0 0 25 0 1 0 599292398 18763776 390 33554432000 4194304 4287625 140736111077440 18446744073709551615 224284945935 0 0 4096 65536 2826088513535 0 0 17 4 0 0 0
/proc/20648/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.70058 s]
/proc/loadavg: 6.39 7.70 7.57 7/181 20649
/proc/meminfo: memFree=18393736/32951124 swapFree=67110396/67111528
[pid=20648] ppid=20643 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/20648/stat : 20648 (ppfolio) S 20643 20648 20607 0 -1 4202496 523 0 0 0 0 0 0 0 25 0 1 0 599292398 18763776 390 33554432000 4194304 4287625 140736111077440 18446744073709551615 224284945935 0 0 4096 65536 2826088513535 0 0 17 4 0 0 0
/proc/20648/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50045 s]
/proc/loadavg: 6.39 7.70 7.57 12/188 20656
/proc/meminfo: memFree=18316288/32951124 swapFree=67110396/67111528
[pid=20648] ppid=20643 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/20648/stat : 20648 (ppfolio) S 20643 20648 20607 0 -1 4202496 523 0 0 0 0 0 0 0 25 0 1 0 599292398 18763776 390 33554432000 4194304 4287625 140736111077440 18446744073709551615 224284945935 0 0 4096 65536 2826088513535 0 0 17 4 0 0 0
/proc/20648/statm: 4581 390 341 23 0 91 0
[pid=20651] ppid=20648 vsize=26616 CPUtime=1.49 cores=0
/proc/20651/stat : 20651 (cryptominisat) R 20648 20648 20607 0 -1 4202496 1667 0 0 0 149 0 0 0 20 0 1 0 599292399 27254784 1530 33554432000 4194304 4587354 140733415611616 18446744073709551615 4238690 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/20651/statm: 6654 1530 306 96 0 3216 0
[pid=20652] ppid=20648 vsize=16324 CPUtime=1.49 cores=4
/proc/20652/stat : 20652 (clasp) R 20648 20648 20607 0 -1 4202496 1576 0 0 0 149 0 0 0 25 0 1 0 599292399 16715776 1378 33554432000 4194304 4785740 140735153044848 18446744073709551615 4495520 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/20652/statm: 4081 1378 314 145 0 1126 0
[pid=20653] ppid=20648 vsize=456888 CPUtime=0.74 cores=6
/proc/20653/stat : 20653 (TNM) R 20648 20648 20607 0 -1 4202496 1597 0 0 0 74 0 0 0 25 0 1 0 599292399 467853312 1494 33554432000 134512640 134534564 4286870064 18446744073709551615 134524763 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/20653/statm: 114222 1494 91 6 0 112888 0
[pid=20654] ppid=20648 vsize=11512 CPUtime=1.49 cores=2
/proc/20654/stat : 20654 (lingeling) R 20648 20648 20607 0 -1 4202496 2426 0 0 0 149 0 0 0 25 0 1 0 599292399 11788288 2068 33554432000 4194304 4352236 140733456134320 18446744073709551615 4208180 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/20654/statm: 2878 2068 123 39 0 1958 0
[pid=20655] ppid=20648 vsize=13256 CPUtime=0.73 cores=6
/proc/20655/stat : 20655 (march_hi) R 20648 20648 20607 0 -1 4202496 7640 0 0 0 72 1 0 0 25 0 1 0 599292399 13574144 1607 33554432000 4194304 4331988 140735648669872 18446744073709551615 4226642 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/20655/statm: 3314 1607 150 34 0 1756 0
Current children cumulated CPU time (s) 5.94
Current children cumulated vsize (KiB) 542920

[startup+3.10118 s]
/proc/loadavg: 6.39 7.70 7.57 12/188 20656
/proc/meminfo: memFree=18305872/32951124 swapFree=67110396/67111528
[pid=20648] ppid=20643 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/20648/stat : 20648 (ppfolio) S 20643 20648 20607 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 599292398 18763776 390 33554432000 4194304 4287625 140736111077440 18446744073709551615 224284945935 0 0 4096 65536 5890681516299649023 0 0 17 4 0 0 0
/proc/20648/statm: 4581 390 341 23 0 91 0
[pid=20651] ppid=20648 vsize=30804 CPUtime=3.08 cores=0
/proc/20651/stat : 20651 (cryptominisat) R 20648 20648 20607 0 -1 4202496 6725 0 0 0 308 0 0 0 25 0 1 0 599292399 31543296 2161 33554432000 4194304 4587354 140733415611616 18446744073709551615 4238752 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/20651/statm: 7701 2161 327 96 0 4263 0
[pid=20652] ppid=20648 vsize=18248 CPUtime=3.09 cores=4
/proc/20652/stat : 20652 (clasp) R 20648 20648 20607 0 -1 4202496 2151 0 0 0 309 0 0 0 25 0 1 0 599292399 18685952 1895 33554432000 4194304 4785740 140735153044848 18446744073709551615 224284593119 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/20652/statm: 4562 1895 340 145 0 1607 0
[pid=20653] ppid=20648 vsize=456888 CPUtime=1.54 cores=6
/proc/20653/stat : 20653 (TNM) R 20648 20648 20607 0 -1 4202496 1597 0 0 0 154 0 0 0 25 0 1 0 599292399 467853312 1494 33554432000 134512640 134534564 4286870064 18446744073709551615 134524546 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/20653/statm: 114222 1494 91 6 0 112888 0
[pid=20654] ppid=20648 vsize=8684 CPUtime=3.09 cores=2
/proc/20654/stat : 20654 (lingeling) R 20648 20648 20607 0 -1 4202496 2430 0 0 0 309 0 0 0 25 0 1 0 599292399 8892416 1365 33554432000 4194304 4352236 140733456134320 18446744073709551615 4329060 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/20654/statm: 2171 1365 127 39 0 1251 0
[pid=20655] ppid=20648 vsize=13432 CPUtime=1.52 cores=6
/proc/20655/stat : 20655 (march_hi) R 20648 20648 20607 0 -1 4202496 7651 0 0 0 151 1 0 0 25 0 1 0 599292399 13754368 1618 33554432000 4194304 4331988 140735648669872 18446744073709551615 4271136 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/20655/statm: 3358 1618 150 34 0 1800 0
Current children cumulated CPU time (s) 12.32
Current children cumulated vsize (KiB) 546380

[startup+6.30065 s]
/proc/loadavg: 6.76 7.76 7.59 12/188 20656
/proc/meminfo: memFree=18285708/32951124 swapFree=67110396/67111528
[pid=20648] ppid=20643 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/20648/stat : 20648 (ppfolio) S 20643 20648 20607 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 599292398 18763776 390 33554432000 4194304 4287625 140736111077440 18446744073709551615 224284945935 0 0 4096 65536 5890681516299649023 0 0 17 4 0 0 0
/proc/20648/statm: 4581 390 341 23 0 91 0
[pid=20651] ppid=20648 vsize=966376 CPUtime=6.29 cores=0
/proc/20651/stat : 20651 (cryptominisat) R 20648 20648 20607 0 -1 4202496 10942 0 0 0 628 1 0 0 25 0 1 0 599292399 989569024 4919 33554432000 4194304 4587354 140733415611616 18446744073709551615 4388277 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/20651/statm: 241594 4919 337 96 0 238156 0
[pid=20652] ppid=20648 vsize=20012 CPUtime=6.29 cores=4
/proc/20652/stat : 20652 (clasp) R 20648 20648 20607 0 -1 4202496 2553 0 0 0 629 0 0 0 25 0 1 0 599292399 20492288 2297 33554432000 4194304 4785740 140735153044848 18446744073709551615 4278129 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/20652/statm: 5003 2297 340 145 0 2048 0
[pid=20653] ppid=20648 vsize=456888 CPUtime=3.14 cores=6
/proc/20653/stat : 20653 (TNM) R 20648 20648 20607 0 -1 4202496 1597 0 0 0 314 0 0 0 25 0 1 0 599292399 467853312 1494 33554432000 134512640 134534564 4286870064 18446744073709551615 134524681 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/20653/statm: 114222 1494 91 6 0 112888 0
[pid=20654] ppid=20648 vsize=12644 CPUtime=6.29 cores=2
/proc/20654/stat : 20654 (lingeling) R 20648 20648 20607 0 -1 4202496 3969 0 0 0 628 1 0 0 25 0 1 0 599292399 12947456 2355 33554432000 4194304 4352236 140733456134320 18446744073709551615 4328194 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/20654/statm: 3161 2355 127 39 0 2241 0
[pid=20655] ppid=20648 vsize=13432 CPUtime=3.1 cores=6
/proc/20655/stat : 20655 (march_hi) R 20648 20648 20607 0 -1 4202496 7675 0 0 0 309 1 0 0 25 0 1 0 599292399 13754368 1642 33554432000 4194304 4331988 140735648669872 18446744073709551615 4271184 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/20655/statm: 3358 1642 150 34 0 1800 0
Current children cumulated CPU time (s) 25.11
Current children cumulated vsize (KiB) 1487676

[startup+12.7006 s]
/proc/loadavg: 7.02 7.79 7.60 12/188 20656
/proc/meminfo: memFree=18258716/32951124 swapFree=67110396/67111528
[pid=20648] ppid=20643 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/20648/stat : 20648 (ppfolio) S 20643 20648 20607 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 599292398 18763776 390 33554432000 4194304 4287625 140736111077440 18446744073709551615 224284945935 0 0 4096 65536 5890681516299649023 0 0 17 4 0 0 0
/proc/20648/statm: 4581 390 341 23 0 91 0
[pid=20651] ppid=20648 vsize=3762596 CPUtime=12.68 cores=0
/proc/20651/stat : 20651 (cryptominisat) R 20648 20648 20607 0 -1 4202496 13730 0 0 0 1267 1 0 0 25 0 1 0 599292399 3852898304 7688 33554432000 4194304 4587354 140733415611616 18446744073709551615 4232256 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/20651/statm: 940649 7688 337 96 0 937211 0
[pid=20652] ppid=20648 vsize=22804 CPUtime=12.69 cores=4
/proc/20652/stat : 20652 (clasp) R 20648 20648 20607 0 -1 4202496 3177 0 0 0 1269 0 0 0 25 0 1 0 599292399 23351296 2921 33554432000 4194304 4785740 140735153044848 18446744073709551615 4286941 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/20652/statm: 5701 2921 340 145 0 2746 0
[pid=20653] ppid=20648 vsize=456888 CPUtime=6.31 cores=6
/proc/20653/stat : 20653 (TNM) R 20648 20648 20607 0 -1 4202496 1597 0 0 0 631 0 0 0 25 0 1 0 599292399 467853312 1494 33554432000 134512640 134534564 4286870064 18446744073709551615 134524548 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/20653/statm: 114222 1494 91 6 0 112888 0
[pid=20654] ppid=20648 vsize=12208 CPUtime=12.68 cores=2
/proc/20654/stat : 20654 (lingeling) R 20648 20648 20607 0 -1 4202496 4848 0 0 0 1267 1 0 0 25 0 1 0 599292399 12500992 2246 33554432000 4194304 4352236 140733456134320 18446744073709551615 4324426 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/20654/statm: 3052 2246 127 39 0 2132 0
[pid=20655] ppid=20648 vsize=13956 CPUtime=6.3 cores=6
/proc/20655/stat : 20655 (march_hi) R 20648 20648 20607 0 -1 4202496 7787 0 0 0 629 1 0 0 25 0 1 0 599292399 14290944 1754 33554432000 4194304 4331988 140735648669872 18446744073709551615 4226642 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/20655/statm: 3489 1754 150 34 0 1931 0
Current children cumulated CPU time (s) 50.66
Current children cumulated vsize (KiB) 4286776

[startup+25.5004 s]
/proc/loadavg: 7.82 7.93 7.65 12/188 20656
/proc/meminfo: memFree=18237820/32951124 swapFree=67110396/67111528
[pid=20648] ppid=20643 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/20648/stat : 20648 (ppfolio) S 20643 20648 20607 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 599292398 18763776 390 33554432000 4194304 4287625 140736111077440 18446744073709551615 224284945935 0 0 4096 65536 5890681516299649023 0 0 17 4 0 0 0
/proc/20648/statm: 4581 390 341 23 0 91 0
[pid=20651] ppid=20648 vsize=44176 CPUtime=25.48 cores=0
/proc/20651/stat : 20651 (cryptominisat) R 20648 20648 20607 0 -1 4202496 22311 0 0 0 2546 2 0 0 25 0 1 0 599292399 45236224 5458 33554432000 4194304 4587354 140733415611616 18446744073709551615 4238491 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/20651/statm: 11044 5458 337 96 0 7606 0
[pid=20652] ppid=20648 vsize=25268 CPUtime=25.48 cores=4
/proc/20652/stat : 20652 (clasp) R 20648 20648 20607 0 -1 4202496 3777 0 0 0 2548 0 0 0 25 0 1 0 599292399 25874432 3521 33554432000 4194304 4785740 140735153044848 18446744073709551615 224284623498 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/20652/statm: 6317 3521 340 145 0 3362 0
[pid=20653] ppid=20648 vsize=456888 CPUtime=12.64 cores=6
/proc/20653/stat : 20653 (TNM) R 20648 20648 20607 0 -1 4202496 1597 0 0 0 1264 0 0 0 25 0 1 0 599292399 467853312 1494 33554432000 134512640 134534564 4286870064 18446744073709551615 134524508 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/20653/statm: 114222 1494 91 6 0 112888 0
[pid=20654] ppid=20648 vsize=18972 CPUtime=25.48 cores=2
/proc/20654/stat : 20654 (lingeling) R 20648 20648 20607 0 -1 4202496 8020 0 0 0 2547 1 0 0 25 0 1 0 599292399 19427328 3937 33554432000 4194304 4352236 140733456134320 18446744073709551615 4267912 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/20654/statm: 4743 3937 127 39 0 3823 0
[pid=20655] ppid=20648 vsize=15528 CPUtime=12.7 cores=6
/proc/20655/stat : 20655 (march_hi) R 20648 20648 20607 0 -1 4202496 8138 0 0 0 1269 1 0 0 25 0 1 0 599292399 15900672 2105 33554432000 4194304 4331988 140735648669872 18446744073709551615 4271184 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/20655/statm: 3882 2105 151 34 0 2324 0
Current children cumulated CPU time (s) 101.78
Current children cumulated vsize (KiB) 579156

[startup+51.1012 s]
/proc/loadavg: 8.68 8.13 7.72 12/188 20658
/proc/meminfo: memFree=18189120/32951124 swapFree=67110396/67111528
[pid=20648] ppid=20643 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/20648/stat : 20648 (ppfolio) S 20643 20648 20607 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 599292398 18763776 390 33554432000 4194304 4287625 140736111077440 18446744073709551615 224284945935 0 0 4096 65536 5890681516299649023 0 0 17 4 0 0 0
/proc/20648/statm: 4581 390 341 23 0 91 0
[pid=20651] ppid=20648 vsize=62236 CPUtime=51.09 cores=0
/proc/20651/stat : 20651 (cryptominisat) R 20648 20648 20607 0 -1 4202496 32228 0 0 0 5105 4 0 0 25 0 1 0 599292399 63729664 11874 33554432000 4194304 4587354 140733415611616 18446744073709551615 4231856 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/20651/statm: 15559 11874 337 96 0 12121 0
[pid=20652] ppid=20648 vsize=28744 CPUtime=51.08 cores=4
/proc/20652/stat : 20652 (clasp) R 20648 20648 20607 0 -1 4202496 4522 0 0 0 5108 0 0 0 25 0 1 0 599292399 29433856 4266 33554432000 4194304 4785740 140735153044848 18446744073709551615 4286912 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/20652/statm: 7186 4266 340 145 0 4231 0
[pid=20653] ppid=20648 vsize=460716 CPUtime=25.4 cores=6
/proc/20653/stat : 20653 (TNM) R 20648 20648 20607 0 -1 4202496 1780 0 0 0 2540 0 0 0 25 0 1 0 599292399 471773184 1677 33554432000 134512640 134534564 4286870064 18446744073709551615 134530367 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/20653/statm: 115179 1677 95 6 0 113845 0
[pid=20654] ppid=20648 vsize=19740 CPUtime=51.09 cores=2

################
# More data... #
################

/proc/20655/statm: 6958 4211 175 34 0 5400 0
Current children cumulated CPU time (s) 3763.95
Current children cumulated vsize (KiB) 687568

[startup+1002.3 s]
/proc/loadavg: 10.41 10.19 9.31 12/188 20685
/proc/meminfo: memFree=18024440/32951124 swapFree=67110396/67111528
[pid=20648] ppid=20643 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/20648/stat : 20648 (ppfolio) S 20643 20648 20607 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 599292398 18763776 390 33554432000 4194304 4287625 140736111077440 18446744073709551615 224284945935 0 0 4096 65536 5890681516299649023 0 0 17 4 0 0 0
/proc/20648/statm: 4581 390 341 23 0 91 0
[pid=20651] ppid=20648 vsize=66188 CPUtime=1002.3 cores=0
/proc/20651/stat : 20651 (cryptominisat) R 20648 20648 20607 0 -1 4202496 242395 0 0 0 100145 85 0 0 25 0 1 0 599292399 67776512 11256 33554432000 4194304 4587354 140733415611616 18446744073709551615 4239875 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/20651/statm: 16547 11257 337 96 0 13109 0
[pid=20652] ppid=20648 vsize=57380 CPUtime=1002.2 cores=4
/proc/20652/stat : 20652 (clasp) R 20648 20648 20607 0 -1 4202496 10882 0 0 0 100218 2 0 0 25 0 1 0 599292399 58757120 10626 33554432000 4194304 4785740 140735153044848 18446744073709551615 4553363 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/20652/statm: 14345 10626 340 145 0 11390 0
[pid=20653] ppid=20648 vsize=460716 CPUtime=498.41 cores=6
/proc/20653/stat : 20653 (TNM) R 20648 20648 20607 0 -1 4202496 1780 0 0 0 49841 0 0 0 25 0 1 0 599292399 471773184 1677 33554432000 134512640 134534564 4286870064 18446744073709551615 134528844 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/20653/statm: 115179 1677 95 6 0 113845 0
[pid=20654] ppid=20648 vsize=55012 CPUtime=1002.29 cores=2
/proc/20654/stat : 20654 (lingeling) R 20648 20648 20607 0 -1 4202496 106987 0 0 0 100214 15 0 0 25 0 1 0 599292399 56332288 12947 33554432000 4194304 4352236 140733456134320 18446744073709551615 4328100 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/20654/statm: 13753 12947 127 39 0 12833 0
[pid=20655] ppid=20648 vsize=27832 CPUtime=498.42 cores=6
/proc/20655/stat : 20655 (march_hi) R 20648 20648 20607 0 -1 4202496 16545 0 0 0 49839 3 0 0 25 0 1 0 599292399 28499968 4236 33554432000 4194304 4331988 140735648669872 18446744073709551615 4299269 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/20655/statm: 6958 4236 175 34 0 5400 0
Current children cumulated CPU time (s) 4003.62
Current children cumulated vsize (KiB) 685452

[startup+1062.3 s]
/proc/loadavg: 10.25 10.18 9.36 12/188 20687
/proc/meminfo: memFree=17978024/32951124 swapFree=67110396/67111528
[pid=20648] ppid=20643 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/20648/stat : 20648 (ppfolio) S 20643 20648 20607 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 599292398 18763776 390 33554432000 4194304 4287625 140736111077440 18446744073709551615 224284945935 0 0 4096 65536 5890681516299649023 0 0 17 4 0 0 0
/proc/20648/statm: 4581 390 341 23 0 91 0
[pid=20651] ppid=20648 vsize=131796 CPUtime=1062.31 cores=0
/proc/20651/stat : 20651 (cryptominisat) R 20648 20648 20607 0 -1 4202496 259029 0 0 0 106143 88 0 0 25 0 1 0 599292399 134959104 27839 33554432000 4194304 4587354 140733415611616 18446744073709551615 4388277 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/20651/statm: 32949 27839 337 96 0 29511 0
[pid=20652] ppid=20648 vsize=59092 CPUtime=1062.19 cores=4
/proc/20652/stat : 20652 (clasp) R 20648 20648 20607 0 -1 4202496 11241 0 0 0 106217 2 0 0 25 0 1 0 599292399 60510208 10985 33554432000 4194304 4785740 140735153044848 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/20652/statm: 14773 10985 340 145 0 11818 0
[pid=20653] ppid=20648 vsize=460716 CPUtime=528.26 cores=6
/proc/20653/stat : 20653 (TNM) R 20648 20648 20607 0 -1 4202496 1780 0 0 0 52826 0 0 0 25 0 1 0 599292399 471773184 1677 33554432000 134512640 134534564 4286870064 18446744073709551615 134530320 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/20653/statm: 115179 1677 95 6 0 113845 0
[pid=20654] ppid=20648 vsize=54756 CPUtime=1062.3 cores=2
/proc/20654/stat : 20654 (lingeling) R 20648 20648 20607 0 -1 4202496 110379 0 0 0 106214 16 0 0 25 0 1 0 599292399 56070144 12883 33554432000 4194304 4352236 140733456134320 18446744073709551615 4328194 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/20654/statm: 13689 12883 127 39 0 12769 0
[pid=20655] ppid=20648 vsize=29368 CPUtime=528.25 cores=6
/proc/20655/stat : 20655 (march_hi) R 20648 20648 20607 0 -1 4202496 16821 0 0 0 52822 3 0 0 25 0 1 0 599292399 30072832 4512 33554432000 4194304 4331988 140735648669872 18446744073709551615 4299264 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/20655/statm: 7342 4512 175 34 0 5784 0
Current children cumulated CPU time (s) 4243.31
Current children cumulated vsize (KiB) 754052

[startup+1122.3 s]
/proc/loadavg: 10.28 10.21 9.42 12/188 20689
/proc/meminfo: memFree=17989268/32951124 swapFree=67110396/67111528
[pid=20648] ppid=20643 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/20648/stat : 20648 (ppfolio) S 20643 20648 20607 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 599292398 18763776 390 33554432000 4194304 4287625 140736111077440 18446744073709551615 224284945935 0 0 4096 65536 5890681516299649023 0 0 17 4 0 0 0
/proc/20648/statm: 4581 390 341 23 0 91 0
[pid=20651] ppid=20648 vsize=83224 CPUtime=1122.3 cores=0
/proc/20651/stat : 20651 (cryptominisat) R 20648 20648 20607 0 -1 4202496 263263 0 0 0 112141 89 0 0 25 0 1 0 599292399 85221376 17603 33554432000 4194304 4587354 140733415611616 18446744073709551615 4238525 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/20651/statm: 20806 17603 337 96 0 17368 0
[pid=20652] ppid=20648 vsize=60376 CPUtime=1122.18 cores=4
/proc/20652/stat : 20652 (clasp) R 20648 20648 20607 0 -1 4202496 11519 0 0 0 112216 2 0 0 25 0 1 0 599292399 61825024 11263 33554432000 4194304 4785740 140735153044848 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/20652/statm: 15094 11263 340 145 0 12139 0
[pid=20653] ppid=20648 vsize=460716 CPUtime=558.07 cores=6
/proc/20653/stat : 20653 (TNM) R 20648 20648 20607 0 -1 4202496 1780 0 0 0 55807 0 0 0 25 0 1 0 599292399 471773184 1677 33554432000 134512640 134534564 4286870064 18446744073709551615 134530360 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/20653/statm: 115179 1677 95 6 0 113845 0
[pid=20654] ppid=20648 vsize=56548 CPUtime=1122.29 cores=2
/proc/20654/stat : 20654 (lingeling) R 20648 20648 20607 0 -1 4202496 114923 0 0 0 112213 16 0 0 25 0 1 0 599292399 57905152 13331 33554432000 4194304 4352236 140733456134320 18446744073709551615 4328194 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/20654/statm: 14137 13331 127 39 0 13217 0
[pid=20655] ppid=20648 vsize=29368 CPUtime=558.11 cores=6
/proc/20655/stat : 20655 (march_hi) R 20648 20648 20607 0 -1 4202496 16844 0 0 0 55808 3 0 0 25 0 1 0 599292399 30072832 4535 33554432000 4194304 4331988 140735648669872 18446744073709551615 4299269 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/20655/statm: 7342 4535 175 34 0 5784 0
Current children cumulated CPU time (s) 4482.95
Current children cumulated vsize (KiB) 708556

[startup+1182.3 s]
/proc/loadavg: 10.18 10.20 9.47 12/188 20691
/proc/meminfo: memFree=17995308/32951124 swapFree=67110396/67111528
[pid=20648] ppid=20643 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/20648/stat : 20648 (ppfolio) S 20643 20648 20607 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 599292398 18763776 390 33554432000 4194304 4287625 140736111077440 18446744073709551615 224284945935 0 0 4096 65536 5890681516299649023 0 0 17 4 0 0 0
/proc/20648/statm: 4581 390 341 23 0 91 0
[pid=20651] ppid=20648 vsize=101808 CPUtime=1182.31 cores=0
/proc/20651/stat : 20651 (cryptominisat) R 20648 20648 20607 0 -1 4202496 271261 0 0 0 118138 93 0 0 25 0 1 0 599292399 104251392 20954 33554432000 4194304 4587354 140733415611616 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/20651/statm: 25452 20954 337 96 0 22014 0
[pid=20652] ppid=20648 vsize=61232 CPUtime=1182.18 cores=4
/proc/20652/stat : 20652 (clasp) R 20648 20648 20607 0 -1 4202496 11709 0 0 0 118216 2 0 0 25 0 1 0 599292399 62701568 11453 33554432000 4194304 4785740 140735153044848 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/20652/statm: 15308 11453 340 145 0 12353 0
[pid=20653] ppid=20648 vsize=460716 CPUtime=587.95 cores=6
/proc/20653/stat : 20653 (TNM) R 20648 20648 20607 0 -1 4202496 1780 0 0 0 58795 0 0 0 25 0 1 0 599292399 471773184 1677 33554432000 134512640 134534564 4286870064 18446744073709551615 134530367 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/20653/statm: 115179 1677 95 6 0 113845 0
[pid=20654] ppid=20648 vsize=53856 CPUtime=1182.3 cores=2
/proc/20654/stat : 20654 (lingeling) R 20648 20648 20607 0 -1 4202496 119276 0 0 0 118213 17 0 0 25 0 1 0 599292399 55148544 12658 33554432000 4194304 4352236 140733456134320 18446744073709551615 4328194 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/20654/statm: 13464 12658 127 39 0 12544 0
[pid=20655] ppid=20648 vsize=29368 CPUtime=587.91 cores=6
/proc/20655/stat : 20655 (march_hi) R 20648 20648 20607 0 -1 4202496 16859 0 0 0 58788 3 0 0 25 0 1 0 599292399 30072832 4550 33554432000 4194304 4331988 140735648669872 18446744073709551615 4299264 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/20655/statm: 7342 4550 175 34 0 5784 0
Current children cumulated CPU time (s) 4722.65
Current children cumulated vsize (KiB) 725304



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 10.20 10.20 9.48 11/188 20691
/proc/meminfo: memFree=18012412/32951124 swapFree=67110396/67111528
[pid=20648] ppid=20643 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/20648/stat : 20648 (ppfolio) S 20643 20648 20607 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 599292398 18763776 390 33554432000 4194304 4287625 140736111077440 18446744073709551615 224284945935 0 0 4096 65536 5890681516299649023 0 0 17 4 0 0 0
/proc/20648/statm: 4581 390 341 23 0 91 0
[pid=20651] ppid=20648 vsize=83224 CPUtime=1201.71 cores=0
/proc/20651/stat : 20651 (cryptominisat) R 20648 20648 20607 0 -1 4202496 272444 0 0 0 120077 94 0 0 25 0 1 0 599292399 85221376 17603 33554432000 4194304 4587354 140733415611616 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/20651/statm: 20806 17603 337 96 0 17368 0
[pid=20652] ppid=20648 vsize=61232 CPUtime=1201.58 cores=4
/proc/20652/stat : 20652 (clasp) R 20648 20648 20607 0 -1 4202496 11715 0 0 0 120156 2 0 0 25 0 1 0 599292399 62701568 11459 33554432000 4194304 4785740 140735153044848 18446744073709551615 4287190 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/20652/statm: 15308 11459 340 145 0 12353 0
[pid=20653] ppid=20648 vsize=460716 CPUtime=597.55 cores=6
/proc/20653/stat : 20653 (TNM) R 20648 20648 20607 0 -1 4202496 1780 0 0 0 59755 0 0 0 25 0 1 0 599292399 471773184 1677 33554432000 134512640 134534564 4286870064 18446744073709551615 134530367 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/20653/statm: 115179 1677 95 6 0 113845 0
[pid=20654] ppid=20648 vsize=55904 CPUtime=1201.7 cores=2
/proc/20654/stat : 20654 (lingeling) R 20648 20648 20607 0 -1 4202496 121772 0 0 0 120153 17 0 0 25 0 1 0 599292399 57245696 13170 33554432000 4194304 4352236 140733456134320 18446744073709551615 4328039 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/20654/statm: 13976 13170 127 39 0 13056 0
[pid=20655] ppid=20648 vsize=29368 CPUtime=597.6 cores=6
/proc/20655/stat : 20655 (march_hi) R 20648 20648 20607 0 -1 4202496 16868 0 0 0 59757 3 0 0 25 0 1 0 599292399 30072832 4559 33554432000 4194304 4331988 140735648669872 18446744073709551615 4299264 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/20655/statm: 7342 4559 175 34 0 5784 0
Current children cumulated CPU time (s) 4800.14
Current children cumulated vsize (KiB) 708768

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.7 s]
/proc/loadavg: 10.20 10.20 9.48 11/188 20691
/proc/meminfo: memFree=18012412/32951124 swapFree=67110396/67111528
[pid=20648] ppid=20643 vsize=18324 CPUtime=0 cores=0,2,4,6
/proc/20648/stat : 20648 (ppfolio) S 20643 20648 20607 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 599292398 18763776 390 33554432000 4194304 4287625 140736111077440 18446744073709551615 224284945935 0 0 4096 65536 5890681516299649023 0 0 17 4 0 0 0
/proc/20648/statm: 4581 390 341 23 0 91 0
[pid=20651] ppid=20648 vsize=83224 CPUtime=1201.71 cores=0
/proc/20651/stat : 20651 (cryptominisat) R 20648 20648 20607 0 -1 4202496 272444 0 0 0 120077 94 0 0 25 0 1 0 599292399 85221376 17603 33554432000 4194304 4587354 140733415611616 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/20651/statm: 20806 17603 337 96 0 17368 0
[pid=20652] ppid=20648 vsize=61232 CPUtime=1201.58 cores=4
/proc/20652/stat : 20652 (clasp) R 20648 20648 20607 0 -1 4202496 11715 0 0 0 120156 2 0 0 25 0 1 0 599292399 62701568 11459 33554432000 4194304 4785740 140735153044848 18446744073709551615 4287190 0 0 4096 16386 0 0 0 17 4 0 0 0
/proc/20652/statm: 15308 11459 340 145 0 12353 0
[pid=20653] ppid=20648 vsize=460716 CPUtime=597.55 cores=6
/proc/20653/stat : 20653 (TNM) R 20648 20648 20607 0 -1 4202496 1780 0 0 0 59755 0 0 0 25 0 1 0 599292399 471773184 1677 33554432000 134512640 134534564 4286870064 18446744073709551615 134530367 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/20653/statm: 115179 1677 95 6 0 113845 0
[pid=20654] ppid=20648 vsize=55904 CPUtime=1201.7 cores=2
/proc/20654/stat : 20654 (lingeling) R 20648 20648 20607 0 -1 4202496 121772 0 0 0 120153 17 0 0 25 0 1 0 599292399 57245696 13170 33554432000 4194304 4352236 140733456134320 18446744073709551615 4328039 0 0 4096 17506 0 0 0 17 2 0 0 0
/proc/20654/statm: 13976 13170 127 39 0 13056 0
[pid=20655] ppid=20648 vsize=29368 CPUtime=597.6 cores=6
/proc/20655/stat : 20655 (march_hi) R 20648 20648 20607 0 -1 4202496 16868 0 0 0 59757 3 0 0 25 0 1 0 599292399 30072832 4559 33554432000 4194304 4331988 140735648669872 18446744073709551615 4299264 0 0 4096 0 0 0 0 17 6 0 0 0
/proc/20655/statm: 7342 4559 175 34 0 5784 0
Current children cumulated CPU time (s) 4800.14
Current children cumulated vsize (KiB) 708768

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 20648 and gives
#  childrusage.ru_utime.tv_sec=3000
#  childrusage.ru_utime.tv_usec=655830
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=230964
# CPU time returned by wait4() is 3000.89
# while last known CPU time is 4800.14
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1201.71
CPU time (s): 4800.14
CPU user time (s): 4798.98
CPU system time (s): 1.16
CPU usage (%): 399.442
Max. virtual memory (cumulated for all children) (KiB): 4286776

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 3000.66
system time used= 0.230964
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 135808
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 456
involuntary context switches= 19561

runsolver used 1.87771 second user time and 4.67129 second system time

The end

Launcher Data 

Begin job on node118 at 2011-04-29 17:23:37
IDJOB=3263097
IDBENCH=83924
IDSOLVER=1612
FILE ID=node118/3263097-1304090617
RUNJOBID= node118-1304090617-20625
PBS_JOBID= 13170577
Free space on /tmp= 71612 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT03/handmade/anton/UNSAT/SGI_30_80_17_90_7-log.shuffled-as.sat03-190.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3263097-1304090617/watcher-3263097-1304090617 -o /tmp/evaluation-result-3263097-1304090617/solver-3263097-1304090617 -C 4800 -W 1300 -M 15500  ppfolio HOME/instance-3263097-1304090617.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= ad30956261e72142648a01bc7c025dba
RANDOM SEED=325357574

node118.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5374.59
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      18394924 kB
Buffers:       1902056 kB
Cached:       12050336 kB
SwapCached:        600 kB
Active:        9508300 kB
Inactive:      4509304 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      18394924 kB
SwapTotal:    67111528 kB
SwapFree:     67110396 kB
Dirty:           22956 kB
Writeback:           0 kB
AnonPages:       64264 kB
Mapped:          14516 kB
Slab:           472960 kB
PageTables:       5272 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   195336 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71596 MiB
End job on node118 at 2011-04-29 17:43:41