Trace number 3347463

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 160001 5009.34

General information on the benchmark

Nameapplication/jarvisalo/HWMCC10-combination-timeframe-expansion/
hwmcc10-timeframe-expansion-k45-pdtpmspalu-tseitin.cnf
MD5SUM8dae13531986e2a9bdb6559418f57130
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark70467.6
Satisfiable
(Un)Satisfiability was proved
Number of variables89190
Number of clauses265718
Sum of the clauses size620006
Maximum clause length3
Minimum clause length1
Number of clauses of size 12
Number of clauses of size 2177144
Number of clauses of size 388572
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 32 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3347463-1305187202/watcher-3347463-1305187202 -o /tmp/evaluation-result-3347463-1305187202/solver-3347463-1305187202 -C 160000 -W 5100 -M 266338304 ppfolio HOME/instance-3347463-1305187202.cnf 

running on 32 cores: 0,4,8,12,16,20,24,28,1,5,9,13,17,21,25,29,2,6,10,14,18,22,26,30,3,7,11,15,19,23,27,31

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 160000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 160030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 272730423296 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 272730474496 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 27.21 31.00 31.63 1/413 16685
/proc/meminfo: memFree=255738600/264257624 swapFree=67111528/67111528
[pid=16685] ppid=16683 vsize=18324 CPUtime=0 cores=0-31
/proc/16685/stat : 16685 (ppfolio) S 16683 16685 16659 0 -1 4202496 514 0 0 0 0 0 0 0 20 0 1 0 630206655 18763776 386 268435456000 4194304 4287625 140736075597328 18446744073709551615 258650975999 0 0 4096 65536 9197547811548168191 0 0 17 20 0 0 0
/proc/16685/statm: 4581 386 337 23 0 91 0

[startup+0.101616 s]
/proc/loadavg: 27.21 31.00 31.63 1/413 16685
/proc/meminfo: memFree=255738600/264257624 swapFree=67111528/67111528
[pid=16685] ppid=16683 vsize=18324 CPUtime=0 cores=0-31
/proc/16685/stat : 16685 (ppfolio) S 16683 16685 16659 0 -1 4202496 514 0 0 0 0 0 0 0 20 0 1 0 630206655 18763776 386 268435456000 4194304 4287625 140736075597328 18446744073709551615 258650975999 0 0 4096 65536 9197547811548168191 0 0 17 20 0 0 0
/proc/16685/statm: 4581 386 337 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.200619 s]
/proc/loadavg: 27.21 31.00 31.63 1/413 16685
/proc/meminfo: memFree=255738600/264257624 swapFree=67111528/67111528
[pid=16685] ppid=16683 vsize=18324 CPUtime=0 cores=0-31
/proc/16685/stat : 16685 (ppfolio) S 16683 16685 16659 0 -1 4202496 514 0 0 0 0 0 0 0 20 0 1 0 630206655 18763776 386 268435456000 4194304 4287625 140736075597328 18446744073709551615 258650975999 0 0 4096 65536 9197547811548168191 0 0 17 20 0 0 0
/proc/16685/statm: 4581 386 337 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300642 s]
/proc/loadavg: 27.21 31.00 31.63 1/413 16685
/proc/meminfo: memFree=255738600/264257624 swapFree=67111528/67111528
[pid=16685] ppid=16683 vsize=18324 CPUtime=0 cores=0-31
/proc/16685/stat : 16685 (ppfolio) S 16683 16685 16659 0 -1 4202496 514 0 0 0 0 0 0 0 18 0 1 0 630206655 18763776 386 268435456000 4194304 4287625 140736075597328 18446744073709551615 258650975999 0 0 4096 65536 9197547811548168191 0 0 17 20 0 0 0
/proc/16685/statm: 4581 386 337 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700646 s]
/proc/loadavg: 27.21 31.00 31.63 1/413 16685
/proc/meminfo: memFree=255738600/264257624 swapFree=67111528/67111528
[pid=16685] ppid=16683 vsize=18324 CPUtime=0 cores=0-31
/proc/16685/stat : 16685 (ppfolio) S 16683 16685 16659 0 -1 4202496 514 0 0 0 0 0 0 0 18 0 1 0 630206655 18763776 386 268435456000 4194304 4287625 140736075597328 18446744073709551615 258650975999 0 0 4096 65536 9197547811548168191 0 0 17 20 0 0 0
/proc/16685/statm: 4581 386 337 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50066 s]
/proc/loadavg: 27.21 31.00 31.63 6/419 16696
/proc/meminfo: memFree=255467832/264257624 swapFree=67111528/67111528
[pid=16685] ppid=16683 vsize=18324 CPUtime=0 cores=0-31
/proc/16685/stat : 16685 (ppfolio) S 16683 16685 16659 0 -1 4202496 514 0 0 0 0 0 0 0 15 0 1 0 630206655 18763776 386 268435456000 4194304 4287625 140736075597328 18446744073709551615 258650975999 0 0 4096 65536 9197547811548168191 0 0 17 20 0 0 0
/proc/16685/statm: 4581 386 337 23 0 91 0
[pid=16686] ppid=16685 vsize=86396 CPUtime=1.48 cores=0
/proc/16686/stat : 16686 (cryptominisat) R 16685 16685 16659 0 -1 4202496 16876 0 0 0 144 4 0 0 25 0 1 0 630206656 88469504 15887 268435456000 4194304 4587354 140733603184512 18446744073709551615 4372590 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/16686/statm: 21599 15887 315 96 0 18161 0
[pid=16687] ppid=16685 vsize=67608 CPUtime=1.48 cores=1
/proc/16687/stat : 16687 (clasp) R 16685 16685 16659 0 -1 4202496 17566 0 0 0 143 5 0 0 20 0 1 0 630206656 69230592 13863 268435456000 4194304 4785740 140734899955056 18446744073709551615 4552550 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/16687/statm: 16902 13863 330 145 0 13947 0
[pid=16688] ppid=16685 vsize=470480 CPUtime=1.48 cores=2
/proc/16688/stat : 16688 (TNM) R 16685 16685 16659 0 -1 4202496 9379 0 0 0 144 4 0 0 20 0 1 0 630206656 481771520 9276 268435456000 134512640 134534564 4289708224 18446744073709551615 134529970 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/16688/statm: 117620 9276 95 6 0 116285 0
[pid=16689] ppid=16685 vsize=47440 CPUtime=1.48 cores=3
/proc/16689/stat : 16689 (march_hi) R 16685 16685 16659 0 -1 4202496 46059 0 0 0 138 10 0 0 20 0 1 0 630206656 48578560 7756 268435456000 4194304 4331988 140733853023136 18446744073709551615 4222776 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/16689/statm: 11860 7756 140 34 0 10302 0
[pid=16690] ppid=16685 vsize=119996 CPUtime=1.49 cores=4-31
/proc/16690/stat : 16690 (plingeling) R 16685 16685 16659 0 -1 4202496 28774 1245 0 0 141 7 0 1 21 0 1 0 630206656 122875904 28658 268435456000 4194304 4356188 140734058563888 18446744073709551615 4210544 0 0 4096 17442 0 0 0 17 8 0 0 0
/proc/16690/statm: 29999 28658 129 40 0 28543 0
Current children cumulated CPU time (s) 7.41
Current children cumulated vsize (KiB) 810244

[startup+3.1007 s]
/proc/loadavg: 27.21 31.00 31.63 6/419 16696
/proc/meminfo: memFree=255257096/264257624 swapFree=67111528/67111528
[pid=16685] ppid=16683 vsize=18324 CPUtime=0 cores=0-31
/proc/16685/stat : 16685 (ppfolio) S 16683 16685 16659 0 -1 4202496 514 0 0 0 0 0 0 0 15 0 1 0 630206655 18763776 386 268435456000 4194304 4287625 140736075597328 18446744073709551615 258650975999 0 0 4096 65536 9197547811548168191 0 0 17 20 0 0 0
/proc/16685/statm: 4581 386 337 23 0 91 0
[pid=16686] ppid=16685 vsize=133740 CPUtime=3.08 cores=0
/proc/16686/stat : 16686 (cryptominisat) R 16685 16685 16659 0 -1 4202496 24483 0 0 0 302 6 0 0 25 0 1 0 630206656 136949760 19967 268435456000 4194304 4587354 140733603184512 18446744073709551615 4237485 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/16686/statm: 33435 19967 328 96 0 29997 0
[pid=16687] ppid=16685 vsize=67608 CPUtime=3.08 cores=1
/proc/16687/stat : 16687 (clasp) R 16685 16685 16659 0 -1 4202496 17574 0 0 0 303 5 0 0 25 0 1 0 630206656 69230592 13871 268435456000 4194304 4785740 140734899955056 18446744073709551615 4327568 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/16687/statm: 16902 13871 338 145 0 13947 0
[pid=16688] ppid=16685 vsize=470480 CPUtime=3.08 cores=2
/proc/16688/stat : 16688 (TNM) R 16685 16685 16659 0 -1 4202496 9379 0 0 0 304 4 0 0 25 0 1 0 630206656 481771520 9276 268435456000 134512640 134534564 4289708224 18446744073709551615 134529031 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/16688/statm: 117620 9276 95 6 0 116285 0
[pid=16689] ppid=16685 vsize=174392 CPUtime=3.08 cores=3
/proc/16689/stat : 16689 (march_hi) R 16685 16685 16659 0 -1 4202496 89174 0 0 0 289 19 0 0 25 0 1 0 630206656 178577408 31431 268435456000 4194304 4331988 140733853023136 18446744073709551615 4228847 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/16689/statm: 43598 31435 172 34 0 42040 0
[pid=16690] ppid=16685 vsize=341980 CPUtime=3.08 cores=4-31
/proc/16690/stat : 16690 (plingeling) R 16685 16685 16659 0 -1 4202496 84164 1245 0 0 289 18 0 1 25 0 1 0 630206656 350187520 84046 268435456000 4194304 4356188 140734058563888 18446744073709551615 258650653120 0 0 4096 17442 0 0 0 17 8 0 0 0
/proc/16690/statm: 85495 84051 129 40 0 84039 0
Current children cumulated CPU time (s) 15.4
Current children cumulated vsize (KiB) 1206524

[startup+6.30177 s]
/proc/loadavg: 25.43 30.57 31.49 33/447 16724
/proc/meminfo: memFree=254923952/264257624 swapFree=67111528/67111528
[pid=16685] ppid=16683 vsize=18324 CPUtime=0 cores=0-31
/proc/16685/stat : 16685 (ppfolio) S 16683 16685 16659 0 -1 4202496 514 0 0 0 0 0 0 0 15 0 1 0 630206655 18763776 386 268435456000 4194304 4287625 140736075597328 18446744073709551615 258650975999 0 0 4096 65536 9197547811548168191 0 0 17 20 0 0 0
/proc/16685/statm: 4581 386 337 23 0 91 0
[pid=16686] ppid=16685 vsize=129984 CPUtime=6.28 cores=0
/proc/16686/stat : 16686 (cryptominisat) R 16685 16685 16659 0 -1 4202496 29477 0 0 0 620 8 0 0 25 0 1 0 630206656 133103616 19963 268435456000 4194304 4587354 140733603184512 18446744073709551615 4237679 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/16686/statm: 32496 19963 335 96 0 29058 0
[pid=16687] ppid=16685 vsize=67608 CPUtime=6.28 cores=1
/proc/16687/stat : 16687 (clasp) R 16685 16685 16659 0 -1 4202496 17574 0 0 0 623 5 0 0 25 0 1 0 630206656 69230592 13871 268435456000 4194304 4785740 140734899955056 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/16687/statm: 16902 13871 338 145 0 13947 0
[pid=16688] ppid=16685 vsize=470480 CPUtime=6.28 cores=2
/proc/16688/stat : 16688 (TNM) R 16685 16685 16659 0 -1 4202496 9379 0 0 0 624 4 0 0 25 0 1 0 630206656 481771520 9276 268435456000 134512640 134534564 4289708224 18446744073709551615 134526136 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/16688/statm: 117620 9276 95 6 0 116285 0
[pid=16689] ppid=16685 vsize=180272 CPUtime=6.28 cores=3
/proc/16689/stat : 16689 (march_hi) R 16685 16685 16659 0 -1 4202496 96555 0 0 0 608 20 0 0 25 0 1 0 630206656 184598528 38812 268435456000 4194304 4331988 140733853023136 18446744073709551615 4228977 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/16689/statm: 45068 38812 172 34 0 43510 0
[pid=16690] ppid=16685 vsize=841112 CPUtime=62.27 cores=4-31
/proc/16690/stat : 16690 (plingeling) S 16685 16685 16659 0 -1 4202496 238598 1245 0 0 6163 63 0 1 18 0 29 0 630206656 861298688 118034 268435456000 4194304 4356188 140734058563888 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 30 0 0 0
/proc/16690/statm: 210278 118040 155 40 0 208822 0
[pid=16690/tid=16697] ppid=16685 vsize=841112 CPUtime=2.08 cores=4-31
/proc/16690/task/16697/stat : 16697 (plingeling) R 16685 16685 16659 0 -1 4202560 4153 1245 0 0 207 0 0 1 21 0 29 0 630207075 861298688 118048 268435456000 4194304 4356188 140734058563888 18446744073709551615 4322016 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16690/tid=16698] ppid=16685 vsize=841112 CPUtime=2.08 cores=4-31
/proc/16690/task/16698/stat : 16698 (plingeling) R 16685 16685 16659 0 -1 4202560 3670 1245 0 0 207 0 0 1 25 0 29 0 630207075 861298688 118057 268435456000 4194304 4356188 140734058563888 18446744073709551615 4322106 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16690/tid=16699] ppid=16685 vsize=841112 CPUtime=2.07 cores=4-31
/proc/16690/task/16699/stat : 16699 (plingeling) R 16685 16685 16659 0 -1 4202560 7073 1245 0 0 204 2 0 1 25 0 29 0 630207075 861298688 118066 268435456000 4194304 4356188 140734058563888 18446744073709551615 4322016 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=16690/tid=16700] ppid=16685 vsize=841112 CPUtime=2.07 cores=4-31
/proc/16690/task/16700/stat : 16700 (plingeling) R 16685 16685 16659 0 -1 4202560 5911 1245 0 0 205 1 0 1 25 0 29 0 630207075 861298688 118076 268435456000 4194304 4356188 140734058563888 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=16690/tid=16701] ppid=16685 vsize=841112 CPUtime=2.03 cores=4-31
/proc/16690/task/16701/stat : 16701 (plingeling) R 16685 16685 16659 0 -1 4202560 1123 1245 0 0 202 0 0 1 25 0 29 0 630207075 861298688 118085 268435456000 4194304 4356188 140734058563888 18446744073709551615 4244324 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=16690/tid=16702] ppid=16685 vsize=841112 CPUtime=2.08 cores=4-31
/proc/16690/task/16702/stat : 16702 (plingeling) R 16685 16685 16659 0 -1 4202560 4502 1245 0 0 206 1 0 1 25 0 29 0 630207075 861298688 118095 268435456000 4194304 4356188 140734058563888 18446744073709551615 4321963 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=16690/tid=16703] ppid=16685 vsize=841112 CPUtime=2.09 cores=4-31
/proc/16690/task/16703/stat : 16703 (plingeling) R 16685 16685 16659 0 -1 4202560 1818 1245 0 0 208 0 0 1 21 0 29 0 630207075 861298688 118103 268435456000 4194304 4356188 140734058563888 18446744073709551615 4210557 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16690/tid=16704] ppid=16685 vsize=841112 CPUtime=2.09 cores=4-31
/proc/16690/task/16704/stat : 16704 (plingeling) R 16685 16685 16659 0 -1 4202560 4492 1245 0 0 207 1 0 1 21 0 29 0 630207075 861298688 118112 268435456000 4194304 4356188 140734058563888 18446744073709551615 4322016 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=16690/tid=16705] ppid=16685 vsize=841112 CPUtime=2.08 cores=4-31
/proc/16690/task/16705/stat : 16705 (plingeling) R 16685 16685 16659 0 -1 4202560 4591 1245 0 0 206 1 0 1 25 0 29 0 630207075 861298688 118122 268435456000 4194304 4356188 140734058563888 18446744073709551615 4315078 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=16690/tid=16706] ppid=16685 vsize=841112 CPUtime=2.09 cores=4-31
/proc/16690/task/16706/stat : 16706 (plingeling) R 16685 16685 16659 0 -1 4202560 5366 1245 0 0 207 1 0 1 18 0 29 0 630207075 861298688 118131 268435456000 4194304 4356188 140734058563888 18446744073709551615 4318205 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=16690/tid=16707] ppid=16685 vsize=841112 CPUtime=2.08 cores=4-31
/proc/16690/task/16707/stat : 16707 (plingeling) R 16685 16685 16659 0 -1 4202560 4581 1245 0 0 206 1 0 1 21 0 29 0 630207075 861298688 118140 268435456000 4194304 4356188 140734058563888 18446744073709551615 4322029 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=16690/tid=16708] ppid=16685 vsize=841112 CPUtime=2.08 cores=4-31
/proc/16690/task/16708/stat : 16708 (plingeling) R 16685 16685 16659 0 -1 4202560 5453 1245 0 0 205 2 0 1 25 0 29 0 630207075 861298688 118150 268435456000 4194304 4356188 140734058563888 18446744073709551615 4210587 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=16690/tid=16709] ppid=16685 vsize=841112 CPUtime=2.07 cores=4-31
/proc/16690/task/16709/stat : 16709 (plingeling) R 16685 16685 16659 0 -1 4202560 5149 1245 0 0 204 2 0 1 18 0 29 0 630207075 861298688 118159 268435456000 4194304 4356188 140734058563888 18446744073709551615 4322106 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=16690/tid=16710] ppid=16685 vsize=841112 CPUtime=2.08 cores=4-31
/proc/16690/task/16710/stat : 16710 (plingeling) R 16685 16685 16659 0 -1 4202560 1156 1245 0 0 207 0 0 1 25 0 29 0 630207075 861298688 118168 268435456000 4194304 4356188 140734058563888 18446744073709551615 4244483 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=16690/tid=16711] ppid=16685 vsize=841112 CPUtime=2.09 cores=4-31
/proc/16690/task/16711/stat : 16711 (plingeling) R 16685 16685 16659 0 -1 4202560 6291 1245 0 0 207 1 0 1 25 0 29 0 630207075 861298688 118178 268435456000 4194304 4356188 140734058563888 18446744073709551615 4322106 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=16690/tid=16712] ppid=16685 vsize=841112 CPUtime=2.08 cores=4-31
/proc/16690/task/16712/stat : 16712 (plingeling) R 16685 16685 16659 0 -1 4202560 5177 1245 0 0 207 0 0 1 18 0 29 0 630207075 861298688 118190 268435456000 4194304 4356188 140734058563888 18446744073709551615 258650649908 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=16690/tid=16713] ppid=16685 vsize=841112 CPUtime=2.09 cores=4-31
/proc/16690/task/16713/stat : 16713 (plingeling) R 16685 16685 16659 0 -1 4202560 6171 1245 0 0 207 1 0 1 18 0 29 0 630207075 861298688 118198 268435456000 4194304 4356188 140734058563888 18446744073709551615 4317399 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=16690/tid=16714] ppid=16685 vsize=841112 CPUtime=2.08 cores=4-31
/proc/16690/task/16714/stat : 16714 (plingeling) R 16685 16685 16659 0 -1 4202560 5813 1245 0 0 206 1 0 1 21 0 29 0 630207075 861298688 118208 268435456000 4194304 4356188 140734058563888 18446744073709551615 4317628 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=16690/tid=16715] ppid=16685 vsize=841112 CPUtime=2.08 cores=4-31
/proc/16690/task/16715/stat : 16715 (plingeling) R 16685 16685 16659 0 -1 4202560 3875 1245 0 0 206 1 0 1 18 0 29 0 630207076 861298688 118217 268435456000 4194304 4356188 140734058563888 18446744073709551615 4321963 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=16690/tid=16716] ppid=16685 vsize=841112 CPUtime=2.08 cores=4-31
/proc/16690/task/16716/stat : 16716 (plingeling) R 16685 16685 16659 0 -1 4202560 3997 1245 0 0 207 0 0 1 21 0 29 0 630207076 861298688 118225 268435456000 4194304 4356188 140734058563888 18446744073709551615 4334501 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=16690/tid=16717] ppid=16685 vsize=841112 CPUtime=2.08 cores=4-31
/proc/16690/task/16717/stat : 16717 (plingeling) R 16685 16685 16659 0 -1 4202560 1174 1245 0 0 207 0 0 1 18 0 29 0 630207077 861298688 118235 268435456000 4194304 4356188 140734058563888 18446744073709551615 4244595 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=16690/tid=16718] ppid=16685 vsize=841112 CPUtime=2.08 cores=4-31
/proc/16690/task/16718/stat : 16718 (plingeling) R 16685 16685 16659 0 -1 4202560 2888 1245 0 0 207 0 0 1 21 0 29 0 630207077 861298688 118244 268435456000 4194304 4356188 140734058563888 18446744073709551615 4260739 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=16690/tid=16719] ppid=16685 vsize=841112 CPUtime=2.03 cores=4-31
/proc/16690/task/16719/stat : 16719 (plingeling) R 16685 16685 16659 0 -1 4202560 4784 1245 0 0 202 0 0 1 18 0 29 0 630207077 861298688 118252 268435456000 4194304 4356188 140734058563888 18446744073709551615 4315013 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=16690/tid=16720] ppid=16685 vsize=841112 CPUtime=2.07 cores=4-31
/proc/16690/task/16720/stat : 16720 (plingeling) R 16685 16685 16659 0 -1 4202560 7688 1245 0 0 203 3 0 1 25 0 29 0 630207077 861298688 118262 268435456000 4194304 4356188 140734058563888 18446744073709551615 4322000 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=16690/tid=16721] ppid=16685 vsize=841112 CPUtime=2.04 cores=4-31
/proc/16690/task/16721/stat : 16721 (plingeling) R 16685 16685 16659 0 -1 4202560 3624 1245 0 0 202 1 0 1 18 0 29 0 630207077 861298688 118272 268435456000 4194304 4356188 140734058563888 18446744073709551615 4322077 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=16690/tid=16722] ppid=16685 vsize=841112 CPUtime=2.06 cores=4-31
/proc/16690/task/16722/stat : 16722 (plingeling) R 16685 16685 16659 0 -1 4202560 4814 1245 0 0 204 1 0 1 25 0 29 0 630207077 861298688 118281 268435456000 4194304 4356188 140734058563888 18446744073709551615 4317144 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=16690/tid=16723] ppid=16685 vsize=841112 CPUtime=2.07 cores=4-31
/proc/16690/task/16723/stat : 16723 (plingeling) R 16685 16685 16659 0 -1 4202560 6372 1245 0 0 205 1 0 1 18 0 29 0 630207077 861298688 118290 268435456000 4194304 4356188 140734058563888 18446744073709551615 4321963 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16690/tid=16724] ppid=16685 vsize=841112 CPUtime=2.06 cores=4-31
/proc/16690/task/16724/stat : 16724 (plingeling) R 16685 16685 16659 0 -1 4202560 4005 1245 0 0 204 1 0 1 25 0 29 0 630207077 861298688 118299 268435456000 4194304 4356188 140734058563888 18446744073709551615 4330862 0 0 4096 17442 0 0 0 -1 26 0 0 0
Current children cumulated CPU time (s) 87.39
Current children cumulated vsize (KiB) 1707780

[startup+12.702 s]
/proc/loadavg: 25.96 30.59 31.49 33/447 16724
/proc/meminfo: memFree=254653708/264257624 swapFree=67111528/67111528
[pid=16685] ppid=16683 vsize=18324 CPUtime=0 cores=0-31
/proc/16685/stat : 16685 (ppfolio) S 16683 16685 16659 0 -1 4202496 514 0 0 0 0 0 0 0 15 0 1 0 630206655 18763776 386 268435456000 4194304 4287625 140736075597328 18446744073709551615 258650975999 0 0 4096 65536 9197547811548168191 0 0 17 20 0 0 0
/proc/16685/statm: 4581 386 337 23 0 91 0
[pid=16686] ppid=16685 vsize=138372 CPUtime=12.68 cores=0
/proc/16686/stat : 16686 (cryptominisat) R 16685 16685 16659 0 -1 4202496 34655 0 0 0 1259 9 0 0 25 0 1 0 630206656 141692928 22800 268435456000 4194304 4587354 140733603184512 18446744073709551615 4236352 0 0 4096 2 0 0 0 17 0 0 0 0

################
# More data... #
################

[pid=16688] ppid=16685 vsize=470484 CPUtime=4961.32 cores=2
/proc/16688/stat : 16688 (TNM) R 16685 16685 16659 0 -1 4202496 9388 0 0 0 496128 4 0 0 25 0 1 0 630206656 481775616 9285 268435456000 134512640 134534564 4289708224 18446744073709551615 134526136 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/16688/statm: 117621 9285 103 6 0 116286 0
[pid=16689] ppid=16685 vsize=276304 CPUtime=4961.35 cores=3
/proc/16689/stat : 16689 (march_hi) R 16685 16685 16659 0 -1 4202496 114189 0 0 0 496101 34 0 0 25 0 1 0 630206656 282935296 56372 268435456000 4194304 4331988 140733853023136 18446744073709551615 4263143 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/16689/statm: 69076 56372 172 34 0 67518 0
[pid=16690] ppid=16685 vsize=3426276 CPUtime=138653 cores=4-31
/proc/16690/stat : 16690 (plingeling) S 16685 16685 16659 0 -1 4202496 6764058 1245 0 0 13861041 4245 0 1 18 0 29 0 630206656 3508506624 756362 268435456000 4194304 4356188 140734058563888 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 30 0 0 0
/proc/16690/statm: 856569 756362 159 40 0 855113 0
[pid=16690/tid=16697] ppid=16685 vsize=3426276 CPUtime=4956.82 cores=4-31
/proc/16690/task/16697/stat : 16697 (plingeling) R 16685 16685 16659 0 -1 4202560 212883 1245 0 0 495521 160 0 1 25 0 29 0 630207075 3508506624 756362 268435456000 4194304 4356188 140734058563888 18446744073709551615 4299498 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16690/tid=16698] ppid=16685 vsize=3426276 CPUtime=4956.93 cores=4-31
/proc/16690/task/16698/stat : 16698 (plingeling) R 16685 16685 16659 0 -1 4202560 170905 1245 0 0 495576 116 0 1 25 0 29 0 630207075 3508506624 756362 268435456000 4194304 4356188 140734058563888 18446744073709551615 4330973 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16690/tid=16699] ppid=16685 vsize=3426276 CPUtime=4955.53 cores=4-31
/proc/16690/task/16699/stat : 16699 (plingeling) R 16685 16685 16659 0 -1 4202560 213530 1245 0 0 495437 115 0 1 25 0 29 0 630207075 3508506624 756362 268435456000 4194304 4356188 140734058563888 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=16690/tid=16700] ppid=16685 vsize=3426276 CPUtime=4922.83 cores=4-31
/proc/16690/task/16700/stat : 16700 (plingeling) R 16685 16685 16659 0 -1 4202560 246719 1245 0 0 492170 112 0 1 25 0 29 0 630207075 3508506624 756362 268435456000 4194304 4356188 140734058563888 18446744073709551615 4334515 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=16690/tid=16701] ppid=16685 vsize=3426276 CPUtime=4956.9 cores=4-31
/proc/16690/task/16701/stat : 16701 (plingeling) R 16685 16685 16659 0 -1 4202560 192332 1245 0 0 495598 91 0 1 25 0 29 0 630207075 3508506624 756362 268435456000 4194304 4356188 140734058563888 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=16690/tid=16702] ppid=16685 vsize=3426276 CPUtime=4956.6 cores=4-31
/proc/16690/task/16702/stat : 16702 (plingeling) R 16685 16685 16659 0 -1 4202560 279537 1245 0 0 495539 120 0 1 25 0 29 0 630207075 3508506624 756362 268435456000 4194304 4356188 140734058563888 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=16690/tid=16703] ppid=16685 vsize=3426276 CPUtime=4956.87 cores=4-31
/proc/16690/task/16703/stat : 16703 (plingeling) R 16685 16685 16659 0 -1 4202560 270393 1245 0 0 495440 246 0 1 25 0 29 0 630207075 3508506624 756362 268435456000 4194304 4356188 140734058563888 18446744073709551615 4330827 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=16690/tid=16704] ppid=16685 vsize=3426276 CPUtime=4956.83 cores=4-31
/proc/16690/task/16704/stat : 16704 (plingeling) R 16685 16685 16659 0 -1 4202560 362694 1245 0 0 495406 276 0 1 25 0 29 0 630207075 3508506624 756362 268435456000 4194304 4356188 140734058563888 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=16690/tid=16705] ppid=16685 vsize=3426276 CPUtime=4956.66 cores=4-31
/proc/16690/task/16705/stat : 16705 (plingeling) R 16685 16685 16659 0 -1 4202560 228450 1245 0 0 495525 140 0 1 25 0 29 0 630207075 3508506624 756362 268435456000 4194304 4356188 140734058563888 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=16690/tid=16706] ppid=16685 vsize=3426276 CPUtime=4956.88 cores=4-31
/proc/16690/task/16706/stat : 16706 (plingeling) R 16685 16685 16659 0 -1 4202560 220378 1245 0 0 495488 199 0 1 25 0 29 0 630207075 3508506624 756362 268435456000 4194304 4356188 140734058563888 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=16690/tid=16707] ppid=16685 vsize=3426276 CPUtime=4957.07 cores=4-31
/proc/16690/task/16707/stat : 16707 (plingeling) R 16685 16685 16659 0 -1 4202560 220724 1245 0 0 495511 195 0 1 25 0 29 0 630207075 3508506624 756362 268435456000 4194304 4356188 140734058563888 18446744073709551615 4331350 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=16690/tid=16708] ppid=16685 vsize=3426276 CPUtime=4957 cores=4-31
/proc/16690/task/16708/stat : 16708 (plingeling) R 16685 16685 16659 0 -1 4202560 218823 1245 0 0 495578 121 0 1 25 0 29 0 630207075 3508506624 756362 268435456000 4194304 4356188 140734058563888 18446744073709551615 4210834 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=16690/tid=16709] ppid=16685 vsize=3426276 CPUtime=4956.97 cores=4-31
/proc/16690/task/16709/stat : 16709 (plingeling) R 16685 16685 16659 0 -1 4202560 322762 1245 0 0 495540 156 0 1 25 0 29 0 630207075 3508506624 756362 268435456000 4194304 4356188 140734058563888 18446744073709551615 4331311 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=16690/tid=16710] ppid=16685 vsize=3426276 CPUtime=4937.65 cores=4-31
/proc/16690/task/16710/stat : 16710 (plingeling) R 16685 16685 16659 0 -1 4202560 199669 1245 0 0 493672 92 0 1 25 0 29 0 630207075 3508506624 756362 268435456000 4194304 4356188 140734058563888 18446744073709551615 4281925 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=16690/tid=16711] ppid=16685 vsize=3426276 CPUtime=4956.9 cores=4-31
/proc/16690/task/16711/stat : 16711 (plingeling) R 16685 16685 16659 0 -1 4202560 238330 1245 0 0 495537 152 0 1 25 0 29 0 630207075 3508506624 756362 268435456000 4194304 4356188 140734058563888 18446744073709551615 4331340 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=16690/tid=16712] ppid=16685 vsize=3426276 CPUtime=4925.24 cores=4-31
/proc/16690/task/16712/stat : 16712 (plingeling) R 16685 16685 16659 0 -1 4202560 331116 1245 0 0 492298 225 0 1 25 0 29 0 630207075 3508506624 756362 268435456000 4194304 4356188 140734058563888 18446744073709551615 4331086 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16690/tid=16713] ppid=16685 vsize=3426276 CPUtime=4956.65 cores=4-31
/proc/16690/task/16713/stat : 16713 (plingeling) R 16685 16685 16659 0 -1 4202560 208636 1245 0 0 495554 110 0 1 25 0 29 0 630207075 3508506624 756362 268435456000 4194304 4356188 140734058563888 18446744073709551615 4210615 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=16690/tid=16714] ppid=16685 vsize=3426276 CPUtime=4956.85 cores=4-31
/proc/16690/task/16714/stat : 16714 (plingeling) R 16685 16685 16659 0 -1 4202560 212689 1245 0 0 495540 144 0 1 25 0 29 0 630207075 3508506624 756362 268435456000 4194304 4356188 140734058563888 18446744073709551615 4331296 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=16690/tid=16715] ppid=16685 vsize=3426276 CPUtime=4956.97 cores=4-31
/proc/16690/task/16715/stat : 16715 (plingeling) R 16685 16685 16659 0 -1 4202560 210700 1245 0 0 495580 116 0 1 25 0 29 0 630207076 3508506624 756362 268435456000 4194304 4356188 140734058563888 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=16690/tid=16716] ppid=16685 vsize=3426276 CPUtime=4953.18 cores=4-31
/proc/16690/task/16716/stat : 16716 (plingeling) R 16685 16685 16659 0 -1 4202560 118986 1245 0 0 495243 74 0 1 25 0 29 0 630207076 3508506624 756362 268435456000 4194304 4356188 140734058563888 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=16690/tid=16717] ppid=16685 vsize=3426276 CPUtime=4938.14 cores=4-31
/proc/16690/task/16717/stat : 16717 (plingeling) R 16685 16685 16659 0 -1 4202560 273390 1245 0 0 493689 124 0 1 25 0 29 0 630207077 3508506624 756362 268435456000 4194304 4356188 140734058563888 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16690/tid=16718] ppid=16685 vsize=3426276 CPUtime=4947.79 cores=4-31
/proc/16690/task/16718/stat : 16718 (plingeling) R 16685 16685 16659 0 -1 4202560 258013 1245 0 0 494631 147 0 1 25 0 29 0 630207077 3508506624 756362 268435456000 4194304 4356188 140734058563888 18446744073709551615 4331165 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=16690/tid=16719] ppid=16685 vsize=3426276 CPUtime=4947.87 cores=4-31
/proc/16690/task/16719/stat : 16719 (plingeling) R 16685 16685 16659 0 -1 4202560 194826 1245 0 0 494667 119 0 1 25 0 29 0 630207077 3508506624 756362 268435456000 4194304 4356188 140734058563888 18446744073709551615 4331350 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=16690/tid=16720] ppid=16685 vsize=3426276 CPUtime=4941.04 cores=4-31
/proc/16690/task/16720/stat : 16720 (plingeling) R 16685 16685 16659 0 -1 4202560 331723 1245 0 0 493830 273 0 1 25 0 29 0 630207077 3508506624 756362 268435456000 4194304 4356188 140734058563888 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=16690/tid=16721] ppid=16685 vsize=3426276 CPUtime=4956.63 cores=4-31
/proc/16690/task/16721/stat : 16721 (plingeling) R 16685 16685 16659 0 -1 4202560 266623 1245 0 0 495461 201 0 1 25 0 29 0 630207077 3508506624 756362 268435456000 4194304 4356188 140734058563888 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=16690/tid=16722] ppid=16685 vsize=3426276 CPUtime=4956.82 cores=4-31
/proc/16690/task/16722/stat : 16722 (plingeling) R 16685 16685 16659 0 -1 4202560 221252 1245 0 0 495513 168 0 1 25 0 29 0 630207077 3508506624 756362 268435456000 4194304 4356188 140734058563888 18446744073709551615 4331264 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=16690/tid=16723] ppid=16685 vsize=3426276 CPUtime=4956.02 cores=4-31
/proc/16690/task/16723/stat : 16723 (plingeling) R 16685 16685 16659 0 -1 4202560 220815 1245 0 0 495497 104 0 1 25 0 29 0 630207077 3508506624 756362 268435456000 4194304 4356188 140734058563888 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=16690/tid=16724] ppid=16685 vsize=3426276 CPUtime=4957.09 cores=4-31
/proc/16690/task/16724/stat : 16724 (plingeling) R 16685 16685 16659 0 -1 4202560 204117 1245 0 0 495596 112 0 1 25 0 29 0 630207077 3508506624 756362 268435456000 4194304 4356188 140734058563888 18446744073709551615 4241608 0 0 4096 17442 0 0 0 -1 26 0 0 0
Current children cumulated CPU time (s) 158498
Current children cumulated vsize (KiB) 6223992



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5009.3 s]
/proc/loadavg: 32.12 32.12 32.07 33/447 16881
/proc/meminfo: memFree=251178832/264257624 swapFree=67111528/67111528
[pid=16685] ppid=16683 vsize=18324 CPUtime=0 cores=0-31
/proc/16685/stat : 16685 (ppfolio) S 16683 16685 16659 0 -1 4202496 514 0 0 0 0 0 0 0 15 0 1 0 630206655 18763776 386 268435456000 4194304 4287625 140736075597328 18446744073709551615 258650975999 0 0 4096 65536 9197547811548168191 0 0 17 20 0 0 0
/proc/16685/statm: 4581 386 337 23 0 91 0
[pid=16686] ppid=16685 vsize=1086328 CPUtime=5008.36 cores=0
/proc/16686/stat : 16686 (cryptominisat) R 16685 16685 16659 0 -1 4202496 410047 0 0 0 500657 179 0 0 25 0 1 0 630206656 1112399872 73781 268435456000 4194304 4587354 140733603184512 18446744073709551615 4238690 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/16686/statm: 271582 73781 338 96 0 268144 0
[pid=16687] ppid=16685 vsize=946276 CPUtime=5008.33 cores=1
/proc/16687/stat : 16687 (clasp) R 16685 16685 16659 0 -1 4202496 237267 0 0 0 500735 98 0 0 25 0 1 0 630206656 968986624 231498 268435456000 4194304 4785740 140734899955056 18446744073709551615 4553388 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/16687/statm: 236569 231498 339 145 0 233614 0
[pid=16688] ppid=16685 vsize=470484 CPUtime=5008.34 cores=2
/proc/16688/stat : 16688 (TNM) R 16685 16685 16659 0 -1 4202496 9388 0 0 0 500830 4 0 0 25 0 1 0 630206656 481775616 9285 268435456000 134512640 134534564 4289708224 18446744073709551615 134530126 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/16688/statm: 117621 9285 103 6 0 116286 0
[pid=16689] ppid=16685 vsize=276436 CPUtime=5008.36 cores=3
/proc/16689/stat : 16689 (march_hi) R 16685 16685 16659 0 -1 4202496 114307 0 0 0 500802 34 0 0 25 0 1 0 630206656 283070464 56490 268435456000 4194304 4331988 140733853023136 18446744073709551615 4229451 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/16689/statm: 69109 56490 172 34 0 67551 0
[pid=16690] ppid=16685 vsize=3474408 CPUtime=139968 cores=4-31
/proc/16690/stat : 16690 (plingeling) S 16685 16685 16659 0 -1 4202496 6834639 1245 0 0 13992470 4311 0 1 18 0 29 0 630206656 3557793792 767439 268435456000 4194304 4356188 140734058563888 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 30 0 0 0
/proc/16690/statm: 868602 767439 159 40 0 867146 0
[pid=16690/tid=16697] ppid=16685 vsize=3474408 CPUtime=5003.83 cores=4-31
/proc/16690/task/16697/stat : 16697 (plingeling) R 16685 16685 16659 0 -1 4202560 222099 1245 0 0 500221 161 0 1 25 0 29 0 630207075 3557793792 767439 268435456000 4194304 4356188 140734058563888 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16690/tid=16698] ppid=16685 vsize=3474408 CPUtime=5003.94 cores=4-31
/proc/16690/task/16698/stat : 16698 (plingeling) R 16685 16685 16659 0 -1 4202560 170905 1245 0 0 500277 116 0 1 25 0 29 0 630207075 3557793792 767439 268435456000 4194304 4356188 140734058563888 18446744073709551615 4331111 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16690/tid=16699] ppid=16685 vsize=3474408 CPUtime=5002.53 cores=4-31
/proc/16690/task/16699/stat : 16699 (plingeling) R 16685 16685 16659 0 -1 4202560 216602 1245 0 0 500134 118 0 1 25 0 29 0 630207075 3557793792 767439 268435456000 4194304 4356188 140734058563888 18446744073709551615 4283920 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=16690/tid=16700] ppid=16685 vsize=3474408 CPUtime=4969.85 cores=4-31
/proc/16690/task/16700/stat : 16700 (plingeling) R 16685 16685 16659 0 -1 4202560 250770 1245 0 0 496870 114 0 1 25 0 29 0 630207075 3557793792 767439 268435456000 4194304 4356188 140734058563888 18446744073709551615 4330992 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=16690/tid=16701] ppid=16685 vsize=3474408 CPUtime=5003.91 cores=4-31
/proc/16690/task/16701/stat : 16701 (plingeling) R 16685 16685 16659 0 -1 4202560 193325 1245 0 0 500299 91 0 1 25 0 29 0 630207075 3557793792 767439 268435456000 4194304 4356188 140734058563888 18446744073709551615 4331203 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=16690/tid=16702] ppid=16685 vsize=3474408 CPUtime=5003.6 cores=4-31
/proc/16690/task/16702/stat : 16702 (plingeling) R 16685 16685 16659 0 -1 4202560 281073 1245 0 0 500239 120 0 1 25 0 29 0 630207075 3557793792 767439 268435456000 4194304 4356188 140734058563888 18446744073709551615 4330999 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=16690/tid=16703] ppid=16685 vsize=3474408 CPUtime=5003.87 cores=4-31
/proc/16690/task/16703/stat : 16703 (plingeling) R 16685 16685 16659 0 -1 4202560 280121 1245 0 0 500120 266 0 1 25 0 29 0 630207075 3557793792 767439 268435456000 4194304 4356188 140734058563888 18446744073709551615 4210579 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=16690/tid=16704] ppid=16685 vsize=3474408 CPUtime=5003.85 cores=4-31
/proc/16690/task/16704/stat : 16704 (plingeling) R 16685 16685 16659 0 -1 4202560 362694 1245 0 0 500108 276 0 1 25 0 29 0 630207075 3557793792 767439 268435456000 4194304 4356188 140734058563888 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=16690/tid=16705] ppid=16685 vsize=3474408 CPUtime=5003.66 cores=4-31
/proc/16690/task/16705/stat : 16705 (plingeling) R 16685 16685 16659 0 -1 4202560 229474 1245 0 0 500223 142 0 1 25 0 29 0 630207075 3557793792 767439 268435456000 4194304 4356188 140734058563888 18446744073709551615 4279481 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=16690/tid=16706] ppid=16685 vsize=3474408 CPUtime=5003.88 cores=4-31
/proc/16690/task/16706/stat : 16706 (plingeling) R 16685 16685 16659 0 -1 4202560 221402 1245 0 0 500188 199 0 1 25 0 29 0 630207075 3557793792 767439 268435456000 4194304 4356188 140734058563888 18446744073709551615 4331074 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=16690/tid=16707] ppid=16685 vsize=3474408 CPUtime=5004.09 cores=4-31
/proc/16690/task/16707/stat : 16707 (plingeling) R 16685 16685 16659 0 -1 4202560 224308 1245 0 0 500209 199 0 1 25 0 29 0 630207075 3557793792 767439 268435456000 4194304 4356188 140734058563888 18446744073709551615 4331835 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=16690/tid=16708] ppid=16685 vsize=3474408 CPUtime=5004.02 cores=4-31
/proc/16690/task/16708/stat : 16708 (plingeling) R 16685 16685 16659 0 -1 4202560 219847 1245 0 0 500279 122 0 1 25 0 29 0 630207075 3557793792 767439 268435456000 4194304 4356188 140734058563888 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=16690/tid=16709] ppid=16685 vsize=3474408 CPUtime=5003.98 cores=4-31
/proc/16690/task/16709/stat : 16709 (plingeling) R 16685 16685 16659 0 -1 4202560 324810 1245 0 0 500241 156 0 1 25 0 29 0 630207075 3557793792 767439 268435456000 4194304 4356188 140734058563888 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=16690/tid=16710] ppid=16685 vsize=3474408 CPUtime=4984.66 cores=4-31
/proc/16690/task/16710/stat : 16710 (plingeling) R 16685 16685 16659 0 -1 4202560 200181 1245 0 0 498373 92 0 1 25 0 29 0 630207075 3557793792 767439 268435456000 4194304 4356188 140734058563888 18446744073709551615 4332041 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=16690/tid=16711] ppid=16685 vsize=3474408 CPUtime=5003.91 cores=4-31
/proc/16690/task/16711/stat : 16711 (plingeling) R 16685 16685 16659 0 -1 4202560 239866 1245 0 0 500238 152 0 1 25 0 29 0 630207075 3557793792 767439 268435456000 4194304 4356188 140734058563888 18446744073709551615 4331047 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=16690/tid=16712] ppid=16685 vsize=3474408 CPUtime=4972.26 cores=4-31
/proc/16690/task/16712/stat : 16712 (plingeling) R 16685 16685 16659 0 -1 4202560 333164 1245 0 0 496995 230 0 1 25 0 29 0 630207075 3557793792 767439 268435456000 4194304 4356188 140734058563888 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16690/tid=16713] ppid=16685 vsize=3474408 CPUtime=5003.66 cores=4-31
/proc/16690/task/16713/stat : 16713 (plingeling) R 16685 16685 16659 0 -1 4202560 208636 1245 0 0 500255 110 0 1 25 0 29 0 630207075 3557793792 767439 268435456000 4194304 4356188 140734058563888 18446744073709551615 4331203 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=16690/tid=16714] ppid=16685 vsize=3474408 CPUtime=5003.86 cores=4-31
/proc/16690/task/16714/stat : 16714 (plingeling) R 16685 16685 16659 0 -1 4202560 212689 1245 0 0 500241 144 0 1 25 0 29 0 630207075 3557793792 767439 268435456000 4194304 4356188 140734058563888 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=16690/tid=16715] ppid=16685 vsize=3474408 CPUtime=5003.98 cores=4-31
/proc/16690/task/16715/stat : 16715 (plingeling) R 16685 16685 16659 0 -1 4202560 219404 1245 0 0 500261 136 0 1 25 0 29 0 630207076 3557793792 767439 268435456000 4194304 4356188 140734058563888 18446744073709551615 4280645 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=16690/tid=16716] ppid=16685 vsize=3474408 CPUtime=4998.87 cores=4-31
/proc/16690/task/16716/stat : 16716 (plingeling) R 16685 16685 16659 0 -1 4202560 121034 1245 0 0 499810 76 0 1 25 0 29 0 630207076 3557793792 767439 268435456000 4194304 4356188 140734058563888 18446744073709551615 4333116 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=16690/tid=16717] ppid=16685 vsize=3474408 CPUtime=4985.15 cores=4-31
/proc/16690/task/16717/stat : 16717 (plingeling) R 16685 16685 16659 0 -1 4202560 273390 1245 0 0 498390 124 0 1 25 0 29 0 630207077 3557793792 767439 268435456000 4194304 4356188 140734058563888 18446744073709551615 4331268 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16690/tid=16718] ppid=16685 vsize=3474408 CPUtime=4994.8 cores=4-31
/proc/16690/task/16718/stat : 16718 (plingeling) R 16685 16685 16659 0 -1 4202560 258525 1245 0 0 499332 147 0 1 25 0 29 0 630207077 3557793792 767439 268435456000 4194304 4356188 140734058563888 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=16690/tid=16719] ppid=16685 vsize=3474408 CPUtime=4994.88 cores=4-31
/proc/16690/task/16719/stat : 16719 (plingeling) R 16685 16685 16659 0 -1 4202560 196362 1245 0 0 499368 119 0 1 25 0 29 0 630207077 3557793792 767439 268435456000 4194304 4356188 140734058563888 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=16690/tid=16720] ppid=16685 vsize=3474408 CPUtime=4988.05 cores=4-31
/proc/16690/task/16720/stat : 16720 (plingeling) R 16685 16685 16659 0 -1 4202560 331723 1245 0 0 498531 273 0 1 25 0 29 0 630207077 3557793792 767439 268435456000 4194304 4356188 140734058563888 18446744073709551615 4331799 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=16690/tid=16721] ppid=16685 vsize=3474408 CPUtime=5003.64 cores=4-31
/proc/16690/task/16721/stat : 16721 (plingeling) R 16685 16685 16659 0 -1 4202560 267135 1245 0 0 500162 201 0 1 25 0 29 0 630207077 3557793792 767439 268435456000 4194304 4356188 140734058563888 18446744073709551615 4284032 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=16690/tid=16722] ppid=16685 vsize=3474408 CPUtime=5003.84 cores=4-31
/proc/16690/task/16722/stat : 16722 (plingeling) R 16685 16685 16659 0 -1 4202560 223300 1245 0 0 500214 169 0 1 25 0 29 0 630207077 3557793792 767439 268435456000 4194304 4356188 140734058563888 18446744073709551615 4279656 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=16690/tid=16723] ppid=16685 vsize=3474408 CPUtime=5003 cores=4-31
/proc/16690/task/16723/stat : 16723 (plingeling) R 16685 16685 16659 0 -1 4202560 234128 1245 0 0 500192 107 0 1 25 0 29 0 630207077 3557793792 767439 268435456000 4194304 4356188 140734058563888 18446744073709551615 4332293 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=16690/tid=16724] ppid=16685 vsize=3474408 CPUtime=5004.1 cores=4-31
/proc/16690/task/16724/stat : 16724 (plingeling) R 16685 16685 16659 0 -1 4202560 204629 1245 0 0 500297 112 0 1 25 0 29 0 630207077 3557793792 767439 268435456000 4194304 4356188 140734058563888 18446744073709551615 4330862 0 0 4096 17442 0 0 0 -1 26 0 0 0
Current children cumulated CPU time (s) 160001
Current children cumulated vsize (KiB) 6272256

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 16685 and gives
#  childrusage.ru_utime.tv_sec=5008
#  childrusage.ru_utime.tv_usec=308621
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=52991
# CPU time returned by wait4() is 5008.36
# while last known CPU time is 160001
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5009.34
CPU time (s): 160001
CPU user time (s): 159955
CPU system time (s): 46.27
CPU usage (%): 3194.06
Max. virtual memory (cumulated for all children) (KiB): 7039356

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 5008.31
system time used= 0.052991
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 9903
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 491
involuntary context switches= 27091

runsolver used 27.6978 second user time and 113.633 second system time

The end

Launcher Data 

Begin job on node200 at 2011-05-12 10:00:02
IDJOB=3347463
IDBENCH=82467
IDSOLVER=1808
FILE ID=node200/3347463-1305187202
RUNJOBID= node200-1305187202-16673
PBS_JOBID= 13282080
Free space on /tmp= 62048 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/application/jarvisalo/HWMCC10-combination-timeframe-expansion/hwmcc10-timeframe-expansion-k45-pdtpmspalu-tseitin.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3347463-1305187202/watcher-3347463-1305187202 -o /tmp/evaluation-result-3347463-1305187202/solver-3347463-1305187202 -C 160000 -W 5100 -M 266338304  ppfolio HOME/instance-3347463-1305187202.cnf

TIME LIMIT= 160000 seconds
MEMORY LIMIT= 266338304 MiB
NBCORE= 32

MD5SUM BENCH= 8dae13531986e2a9bdb6559418f57130
RANDOM SEED=1003772363

node200.alineos.net Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 64
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 96
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 66
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 98
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 68
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 100
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 70
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.98
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 102
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 16
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 17
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 48
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 18
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 80
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 19
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 112
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 20
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 21
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 50
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 22
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 82
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 114
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.23
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 24
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 25
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 26
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 84
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 27
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 116
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 28
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 29
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 54
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 30
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 86
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 31
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 118
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     264257624 kB
MemFree:      255739012 kB
Buffers:        426352 kB
Cached:        7689628 kB
SwapCached:          0 kB
Active:        7260196 kB
Inactive:       924300 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     264257624 kB
LowFree:      255739012 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           14536 kB
Writeback:           0 kB
AnonPages:       69980 kB
Mapped:          13484 kB
Slab:           248848 kB
PageTables:       3996 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  199240340 kB
Committed_AS:   188560 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296372 kB
VmallocChunk: 34359440603 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 62036 MiB
End job on node200 at 2011-05-12 11:23:34