Trace number 3472410

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 160014 5021.06

General information on the benchmark

Nameapplication/jarvisalo/smtqfbv-aigs/
smtlib-qfbv-aigs-VS3-benchmark-S2-tseitin.cnf
MD5SUMb967fca11e85ef5b614d3452b1c4b2ad
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables257030
Number of clauses769313
Sum of the clauses size1795061
Maximum clause length3
Minimum clause length1
Number of clauses of size 12
Number of clauses of size 2512874
Number of clauses of size 3256437
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...614621/instance-3472410-1307614621.cnf
2.89/2.94	c Solving...
160004.08/5011.64	
160004.08/5011.64	*** INTERRUPTED! ***
160004.08/5011.64	*** Shutting down threads...

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3472410-1307614621/watcher-3472410-1307614621 -o /tmp/evaluation-result-3472410-1307614621/solver-3472410-1307614621 -C 160000 -W 5100 -M 256000 HOME/claspmt -t32 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3472410-1307614621.cnf 

running on 32 cores: 0,4,8,12,16,20,24,28,1,5,9,13,17,21,25,29,2,6,10,14,18,22,26,30,3,7,11,15,19,23,27,31

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 160000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 160030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 262144000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 262195200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 9.07 17.11 15.92 1/413 28204
/proc/meminfo: memFree=262361132/264257624 swapFree=67111528/67111528
[pid=28204] ppid=28202 vsize=53512 CPUtime=0.02 cores=0-31
/proc/28204/stat : 28204 (claspmt) R 28202 28204 27592 0 -1 4202496 5130 0 0 0 2 0 0 0 20 0 1 0 26849105 54796288 5059 270582939648 4194304 4964800 140735171401392 18446744073709551615 4700412 0 0 4096 18946 0 0 0 17 28 0 0 0
/proc/28204/statm: 13378 5068 373 189 0 8252 0

[startup+0.120018 s]
/proc/loadavg: 9.07 17.11 15.92 1/413 28204
/proc/meminfo: memFree=262361132/264257624 swapFree=67111528/67111528
[pid=28204] ppid=28202 vsize=77196 CPUtime=0.11 cores=0-31
/proc/28204/stat : 28204 (claspmt) R 28202 28204 27592 0 -1 4202496 16810 0 0 0 8 3 0 0 20 0 1 0 26849105 79048704 14258 270582939648 4194304 4964800 140735171401392 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 17 28 0 0 0
/proc/28204/statm: 19332 14260 383 189 0 14206 0
Current children cumulated CPU time (s) 0.11
Current children cumulated vsize (KiB) 77196

[startup+0.201024 s]
/proc/loadavg: 9.07 17.11 15.92 1/413 28204
/proc/meminfo: memFree=262361132/264257624 swapFree=67111528/67111528
[pid=28204] ppid=28202 vsize=88472 CPUtime=0.19 cores=0-31
/proc/28204/stat : 28204 (claspmt) R 28202 28204 27592 0 -1 4202496 20721 0 0 0 15 4 0 0 20 0 1 0 26849105 90595328 16884 270582939648 4194304 4964800 140735171401392 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 17 28 0 0 0
/proc/28204/statm: 22118 16885 383 189 0 16992 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 88472

[startup+0.301052 s]
/proc/loadavg: 9.07 17.11 15.92 1/413 28204
/proc/meminfo: memFree=262361132/264257624 swapFree=67111528/67111528
[pid=28204] ppid=28202 vsize=100460 CPUtime=0.29 cores=0-31
/proc/28204/stat : 28204 (claspmt) R 28202 28204 27592 0 -1 4202496 24715 0 0 0 24 5 0 0 21 0 1 0 26849105 102871040 19966 270582939648 4194304 4964800 140735171401392 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 17 28 0 0 0
/proc/28204/statm: 25115 19967 383 189 0 19989 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 100460

[startup+0.701046 s]
/proc/loadavg: 9.07 17.11 15.92 1/413 28204
/proc/meminfo: memFree=262361132/264257624 swapFree=67111528/67111528
[pid=28204] ppid=28202 vsize=147224 CPUtime=0.69 cores=0-31
/proc/28204/stat : 28204 (claspmt) R 28202 28204 27592 0 -1 4202496 39542 0 0 0 62 7 0 0 22 0 1 0 26849105 150757376 30852 270582939648 4194304 4964800 140735171401392 18446744073709551615 4602317 0 0 4096 18946 0 0 0 17 28 0 0 0
/proc/28204/statm: 36806 30852 401 189 0 31680 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 147224

[startup+1.50107 s]
/proc/loadavg: 9.07 17.11 15.92 2/414 28205
/proc/meminfo: memFree=262229496/264257624 swapFree=67111528/67111528
[pid=28204] ppid=28202 vsize=164252 CPUtime=1.49 cores=0-31
/proc/28204/stat : 28204 (claspmt) R 28202 28204 27592 0 -1 4202496 43810 0 0 0 140 9 0 0 25 0 1 0 26849105 168194048 35120 270582939648 4194304 4964800 140735171401392 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 17 28 0 0 0
/proc/28204/statm: 41063 35120 401 189 0 35937 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 164252

[startup+3.10108 s]
/proc/loadavg: 9.07 17.11 15.92 2/414 28205
/proc/meminfo: memFree=262212632/264257624 swapFree=67111528/67111528
[pid=28204] ppid=28202 vsize=864088 CPUtime=7.03 cores=0-31
/proc/28204/stat : 28204 (claspmt) S 28202 28204 27592 0 -1 4202496 91680 0 0 0 301 402 0 0 18 0 32 0 26849105 884826112 70371 270582939648 4194304 4964800 140735171401392 18446744073709551615 258662772457 0 0 4096 18946 18446744073709551615 0 0 17 16 0 0 0
/proc/28204/statm: 216022 70384 426 189 0 210371 0
Current children cumulated CPU time (s) 7.03
Current children cumulated vsize (KiB) 864088

[startup+6.30213 s]
/proc/loadavg: 10.83 17.34 16.00 5/445 28236
/proc/meminfo: memFree=261262156/264257624 swapFree=67111528/67111528
[pid=28204] ppid=28202 vsize=2641624 CPUtime=37.6 cores=0-31
/proc/28204/stat : 28204 (claspmt) S 28202 28204 27592 0 -1 4202496 370855 0 0 0 1001 2759 0 0 18 0 32 0 26849105 2705022976 295893 270582939648 4194304 4964800 140735171401392 18446744073709551615 258662772457 0 0 4096 18946 18446744073709551615 0 0 17 16 0 0 0
/proc/28204/statm: 660406 295893 439 189 0 654755 0
[pid=28204/tid=28206] ppid=28202 vsize=2641624 CPUtime=1.03 cores=0-31
/proc/28204/task/28206/stat : 28206 (claspmt) S 28202 28204 27592 0 -1 4202560 8487 0 0 0 9 94 0 0 17 0 32 0 26849399 2705022976 295893 270582939648 4194304 4964800 140735171401392 18446744073709551615 258662772457 0 0 4096 18946 18446604498340232672 0 0 -1 25 0 0 0
[pid=28204/tid=28207] ppid=28202 vsize=2641624 CPUtime=1.09 cores=0-31
/proc/28204/task/28207/stat : 28207 (claspmt) S 28202 28204 27592 0 -1 4202560 10926 0 0 0 18 91 0 0 17 0 32 0 26849399 2705022976 295893 270582939648 4194304 4964800 140735171401392 18446744073709551615 258662772457 0 0 4096 18946 18446604500242472416 0 0 -1 3 0 0 0
[pid=28204/tid=28208] ppid=28202 vsize=2641624 CPUtime=1.1 cores=0-31
/proc/28204/task/28208/stat : 28208 (claspmt) S 28202 28204 27592 0 -1 4202560 10884 0 0 0 16 94 0 0 17 0 32 0 26849399 2705022976 295893 270582939648 4194304 4964800 140735171401392 18446744073709551615 258662772457 0 0 4096 18946 18446604500231003616 0 0 -1 31 0 0 0
[pid=28204/tid=28209] ppid=28202 vsize=2641624 CPUtime=0.93 cores=0-31
/proc/28204/task/28209/stat : 28209 (claspmt) S 28202 28204 27592 0 -1 4202560 9398 0 0 0 8 85 0 0 16 0 32 0 26849399 2705022976 295893 270582939648 4194304 4964800 140735171401392 18446744073709551615 258662772457 0 0 4096 18946 18446604498340224480 0 0 -1 24 0 0 0
[pid=28204/tid=28210] ppid=28202 vsize=2641624 CPUtime=1.71 cores=0-31
/proc/28204/task/28210/stat : 28210 (claspmt) R 28202 28204 27592 0 -1 4202560 9920 0 0 0 76 95 0 0 18 0 32 0 26849399 2705022976 295893 270582939648 4194304 4964800 140735171401392 18446744073709551615 4669011 0 0 4096 18946 0 0 0 -1 31 0 0 0
[pid=28204/tid=28211] ppid=28202 vsize=2641624 CPUtime=1.05 cores=0-31
/proc/28204/task/28211/stat : 28211 (claspmt) S 28202 28204 27592 0 -1 4202560 12083 0 0 0 15 90 0 0 17 0 32 0 26849399 2705022976 295893 270582939648 4194304 4964800 140735171401392 18446744073709551615 258662772457 0 0 4096 18946 18446604500398915040 0 0 -1 27 0 0 0
[pid=28204/tid=28212] ppid=28202 vsize=2641624 CPUtime=1.1 cores=0-31
/proc/28204/task/28212/stat : 28212 (claspmt) S 28202 28204 27592 0 -1 4202560 10525 0 0 0 19 91 0 0 17 0 32 0 26849399 2705022976 295893 270582939648 4194304 4964800 140735171401392 18446744073709551615 258662772457 0 0 4096 18946 18446604500999757280 0 0 -1 13 0 0 0
[pid=28204/tid=28213] ppid=28202 vsize=2641624 CPUtime=0.88 cores=0-31
/proc/28204/task/28213/stat : 28213 (claspmt) S 28202 28204 27592 0 -1 4202560 9033 0 0 0 8 80 0 0 17 0 32 0 26849399 2705022976 295893 270582939648 4194304 4964800 140735171401392 18446744073709551615 258662772457 0 0 4096 18946 18446604495315451360 0 0 -1 12 0 0 0
[pid=28204/tid=28214] ppid=28202 vsize=2641624 CPUtime=0.93 cores=0-31
/proc/28204/task/28214/stat : 28214 (claspmt) S 28202 28204 27592 0 -1 4202560 9136 0 0 0 9 84 0 0 17 0 32 0 26849399 2705022976 295894 270582939648 4194304 4964800 140735171401392 18446744073709551615 258662772457 0 0 4096 18946 18446604498333466080 0 0 -1 6 0 0 0
[pid=28204/tid=28215] ppid=28202 vsize=2641624 CPUtime=1 cores=0-31
/proc/28204/task/28215/stat : 28215 (claspmt) S 28202 28204 27592 0 -1 4202560 12254 0 0 0 16 84 0 0 17 0 32 0 26849399 2705022976 295894 270582939648 4194304 4964800 140735171401392 18446744073709551615 258662772457 0 0 4096 18946 18446604504038538720 0 0 -1 7 0 0 0
[pid=28204/tid=28216] ppid=28202 vsize=2641624 CPUtime=1.06 cores=0-31
/proc/28204/task/28216/stat : 28216 (claspmt) S 28202 28204 27592 0 -1 4202560 11768 0 0 0 15 91 0 0 17 0 32 0 26849399 2705022976 295895 270582939648 4194304 4964800 140735171401392 18446744073709551615 258662772457 0 0 4096 18946 18446604504854101472 0 0 -1 23 0 0 0
[pid=28204/tid=28217] ppid=28202 vsize=2641624 CPUtime=0.89 cores=0-31
/proc/28204/task/28217/stat : 28217 (claspmt) S 28202 28204 27592 0 -1 4202560 9136 0 0 0 8 81 0 0 17 0 32 0 26849399 2705022976 295895 270582939648 4194304 4964800 140735171401392 18446744073709551615 258662772457 0 0 4096 18946 18446604558903786976 0 0 -1 8 0 0 0
[pid=28204/tid=28218] ppid=28202 vsize=2641624 CPUtime=1.72 cores=0-31
/proc/28204/task/28218/stat : 28218 (claspmt) R 28202 28204 27592 0 -1 4202560 10368 0 0 0 85 87 0 0 17 0 32 0 26849399 2705022976 295895 270582939648 4194304 4964800 140735171401392 18446744073709551615 4668709 0 0 4096 18946 0 0 0 -1 26 0 0 0
[pid=28204/tid=28219] ppid=28202 vsize=2641624 CPUtime=1.05 cores=0-31
/proc/28204/task/28219/stat : 28219 (claspmt) S 28202 28204 27592 0 -1 4202560 11654 0 0 0 15 90 0 0 17 0 32 0 26849399 2705022976 295895 270582939648 4194304 4964800 140735171401392 18446744073709551615 258662772457 0 0 4096 18946 18446604504280858080 0 0 -1 18 0 0 0
[pid=28204/tid=28220] ppid=28202 vsize=2641624 CPUtime=1.14 cores=0-31
/proc/28204/task/28220/stat : 28220 (claspmt) S 28202 28204 27592 0 -1 4202560 10318 0 0 0 23 91 0 0 17 0 32 0 26849399 2705022976 295895 270582939648 4194304 4964800 140735171401392 18446744073709551615 258662772457 0 0 4096 18946 18446604504854085088 0 0 -1 30 0 0 0
[pid=28204/tid=28221] ppid=28202 vsize=2641624 CPUtime=0.95 cores=0-31
/proc/28204/task/28221/stat : 28221 (claspmt) S 28202 28204 27592 0 -1 4202560 9201 0 0 0 6 89 0 0 17 0 32 0 26849400 2705022976 295895 270582939648 4194304 4964800 140735171401392 18446744073709551615 258662772457 0 0 4096 18946 18446604499575455200 0 0 -1 20 0 0 0
[pid=28204/tid=28222] ppid=28202 vsize=2641624 CPUtime=0.98 cores=0-31
/proc/28204/task/28222/stat : 28222 (claspmt) S 28202 28204 27592 0 -1 4202560 9218 0 0 0 9 89 0 0 17 0 32 0 26849400 2705022976 295895 270582939648 4194304 4964800 140735171401392 18446744073709551615 258662772457 0 0 4096 18946 18446604501722021344 0 0 -1 4 0 0 0
[pid=28204/tid=28223] ppid=28202 vsize=2641624 CPUtime=1.02 cores=0-31
/proc/28204/task/28223/stat : 28223 (claspmt) S 28202 28204 27592 0 -1 4202560 11910 0 0 0 16 86 0 0 17 0 32 0 26849400 2705022976 295895 270582939648 4194304 4964800 140735171401392 18446744073709551615 258662772457 0 0 4096 18946 18446604498340240864 0 0 -1 16 0 0 0
[pid=28204/tid=28224] ppid=28202 vsize=2641624 CPUtime=1.1 cores=0-31
/proc/28204/task/28224/stat : 28224 (claspmt) S 28202 28204 27592 0 -1 4202560 10564 0 0 0 15 95 0 0 18 0 32 0 26849400 2705022976 295895 270582939648 4194304 4964800 140735171401392 18446744073709551615 258662772457 0 0 4096 18946 18446604498333449696 0 0 -1 0 0 0 0
[pid=28204/tid=28225] ppid=28202 vsize=2641624 CPUtime=0.96 cores=0-31
/proc/28204/task/28225/stat : 28225 (claspmt) S 28202 28204 27592 0 -1 4202560 9142 0 0 0 9 87 0 0 16 0 32 0 26849400 2705022976 295895 270582939648 4194304 4964800 140735171401392 18446744073709551615 258662772457 0 0 4096 18946 18446604484878040544 0 0 -1 9 0 0 0
[pid=28204/tid=28226] ppid=28202 vsize=2641624 CPUtime=1.76 cores=0-31
/proc/28204/task/28226/stat : 28226 (claspmt) R 28202 28204 27592 0 -1 4202560 10356 0 0 0 87 89 0 0 17 0 32 0 26849400 2705022976 295895 270582939648 4194304 4964800 140735171401392 18446744073709551615 4669157 0 0 4096 18946 0 0 0 -1 8 0 0 0
[pid=28204/tid=28227] ppid=28202 vsize=2641624 CPUtime=1.06 cores=0-31
/proc/28204/task/28227/stat : 28227 (claspmt) S 28202 28204 27592 0 -1 4202560 11538 0 0 0 15 91 0 0 17 0 32 0 26849400 2705022976 295895 270582939648 4194304 4964800 140735171401392 18446744073709551615 258662772457 0 0 4096 18946 18446604498333457888 0 0 -1 21 0 0 0
[pid=28204/tid=28228] ppid=28202 vsize=2641624 CPUtime=1.1 cores=0-31
/proc/28204/task/28228/stat : 28228 (claspmt) S 28202 28204 27592 0 -1 4202560 10478 0 0 0 27 83 0 0 17 0 32 0 26849400 2705022976 295895 270582939648 4194304 4964800 140735171401392 18446744073709551615 258662772457 0 0 4096 18946 18446604500242480608 0 0 -1 11 0 0 0
[pid=28204/tid=28229] ppid=28202 vsize=2641624 CPUtime=0.93 cores=0-31
/proc/28204/task/28229/stat : 28229 (claspmt) S 28202 28204 27592 0 -1 4202560 9378 0 0 0 8 85 0 0 17 0 32 0 26849400 2705022976 295895 270582939648 4194304 4964800 140735171401392 18446744073709551615 258662772457 0 0 4096 18946 18446604499321920992 0 0 -1 9 0 0 0
[pid=28204/tid=28230] ppid=28202 vsize=2641624 CPUtime=0.86 cores=0-31
/proc/28204/task/28230/stat : 28230 (claspmt) S 28202 28204 27592 0 -1 4202560 9304 0 0 0 8 78 0 0 17 0 32 0 26849400 2705022976 295895 270582939648 4194304 4964800 140735171401392 18446744073709551615 258662772457 0 0 4096 18946 18446604485407432160 0 0 -1 19 0 0 0
[pid=28204/tid=28231] ppid=28202 vsize=2641624 CPUtime=1.07 cores=0-31
/proc/28204/task/28231/stat : 28231 (claspmt) S 28202 28204 27592 0 -1 4202560 11756 0 0 0 14 93 0 0 17 0 32 0 26849400 2705022976 295895 270582939648 4194304 4964800 140735171401392 18446744073709551615 258662772457 0 0 4096 18946 18446604497774640608 0 0 -1 24 0 0 0
[pid=28204/tid=28232] ppid=28202 vsize=2641624 CPUtime=1.06 cores=0-31
/proc/28204/task/28232/stat : 28232 (claspmt) S 28202 28204 27592 0 -1 4202560 12213 0 0 0 15 91 0 0 17 0 32 0 26849400 2705022976 295895 270582939648 4194304 4964800 140735171401392 18446744073709551615 258662772457 0 0 4096 18946 18446604504028511712 0 0 -1 14 0 0 0
[pid=28204/tid=28233] ppid=28202 vsize=2641624 CPUtime=0.94 cores=0-31
/proc/28204/task/28233/stat : 28233 (claspmt) S 28202 28204 27592 0 -1 4202560 9276 0 0 0 8 86 0 0 17 0 32 0 26849400 2705022976 295895 270582939648 4194304 4964800 140735171401392 18446744073709551615 258662772457 0 0 4096 18946 18446604499575463392 0 0 -1 22 0 0 0
[pid=28204/tid=28234] ppid=28202 vsize=2641624 CPUtime=1.75 cores=0-31
/proc/28204/task/28234/stat : 28234 (claspmt) R 28202 28204 27592 0 -1 4202560 10633 0 0 0 87 88 0 0 17 0 32 0 26849400 2705022976 295895 270582939648 4194304 4964800 140735171401392 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 17 0 0 0
[pid=28204/tid=28235] ppid=28202 vsize=2641624 CPUtime=1 cores=0-31
/proc/28204/task/28235/stat : 28235 (claspmt) S 28202 28204 27592 0 -1 4202560 12237 0 0 0 15 85 0 0 17 0 32 0 26849400 2705022976 295895 270582939648 4194304 4964800 140735171401392 18446744073709551615 258662772457 0 0 4096 18946 18446604504280849888 0 0 -1 15 0 0 0
[pid=28204/tid=28236] ppid=28202 vsize=2641624 CPUtime=1.09 cores=0-31
/proc/28204/task/28236/stat : 28236 (claspmt) S 28202 28204 27592 0 -1 4202560 10284 0 0 0 20 89 0 0 17 0 32 0 26849400 2705022976 295895 270582939648 4194304 4964800 140735171401392 18446744073709551615 258662772457 0 0 4096 18946 18446604499942727136 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 37.6
Current children cumulated vsize (KiB) 2641624

[startup+12.7023 s]
/proc/loadavg: 12.76 17.63 16.10 33/445 28236
/proc/meminfo: memFree=261124500/264257624 swapFree=67111528/67111528
[pid=28204] ppid=28202 vsize=2641624 CPUtime=216.96 cores=0-31
/proc/28204/stat : 28204 (claspmt) R 28202 28204 27592 0 -1 4202496 385909 0 0 0 18879 2817 0 0 25 0 32 0 26849105 2705022976 310944 270582939648 4194304 4964800 140735171401392 18446744073709551615 4668987 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/28204/statm: 660406 310945 461 189 0 654755 0
[pid=28204/tid=28206] ppid=28202 vsize=2641624 CPUtime=6.57 cores=0-31
/proc/28204/task/28206/stat : 28206 (claspmt) R 28202 28204 27592 0 -1 4202560 8853 0 0 0 561 96 0 0 25 0 32 0 26849399 2705022976 310945 270582939648 4194304 4964800 140735171401392 18446744073709551615 4669199 0 0 4096 18946 0 0 0 -1 21 0 0 0
[pid=28204/tid=28207] ppid=28202 vsize=2641624 CPUtime=6.61 cores=0-31
/proc/28204/task/28207/stat : 28207 (claspmt) R 28202 28204 27592 0 -1 4202560 11341 0 0 0 568 93 0 0 25 0 32 0 26849399 2705022976 310945 270582939648 4194304 4964800 140735171401392 18446744073709551615 4669542 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=28204/tid=28208] ppid=28202 vsize=2641624 CPUtime=6.62 cores=0-31
/proc/28204/task/28208/stat : 28208 (claspmt) R 28202 28204 27592 0 -1 4202560 11373 0 0 0 566 96 0 0 25 0 32 0 26849399 2705022976 310945 270582939648 4194304 4964800 140735171401392 18446744073709551615 4668712 0 0 4096 18946 0 0 0 -1 30 0 0 0
[pid=28204/tid=28209] ppid=28202 vsize=2641624 CPUtime=6.48 cores=0-31
/proc/28204/task/28209/stat : 28209 (claspmt) R 28202 28204 27592 0 -1 4202560 9930 0 0 0 561 87 0 0 25 0 32 0 26849399 2705022976 310945 270582939648 4194304 4964800 140735171401392 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 24 0 0 0
[pid=28204/tid=28210] ppid=28202 vsize=2641624 CPUtime=7.94 cores=0-31
/proc/28204/task/28210/stat : 28210 (claspmt) R 28202 28204 27592 0 -1 4202560 10232 0 0 0 698 96 0 0 25 0 32 0 26849399 2705022976 310945 270582939648 4194304 4964800 140735171401392 18446744073709551615 4668987 0 0 4096 18946 0 0 0 -1 26 0 0 0
[pid=28204/tid=28211] ppid=28202 vsize=2641624 CPUtime=6.55 cores=0-31
/proc/28204/task/28211/stat : 28211 (claspmt) R 28202 28204 27592 0 -1 4202560 12501 0 0 0 563 92 0 0 25 0 32 0 26849399 2705022976 310945 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 15 0 0 0
[pid=28204/tid=28212] ppid=28202 vsize=2641624 CPUtime=6.64 cores=0-31
/proc/28204/task/28212/stat : 28212 (claspmt) R 28202 28204 27592 0 -1 4202560 11080 0 0 0 571 93 0 0 25 0 32 0 26849399 2705022976 310945 270582939648 4194304 4964800 140735171401392 18446744073709551615 4668801 0 0 4096 18946 0 0 0 -1 13 0 0 0
[pid=28204/tid=28213] ppid=28202 vsize=2641624 CPUtime=6.41 cores=0-31
/proc/28204/task/28213/stat : 28213 (claspmt) R 28202 28204 27592 0 -1 4202560 9605 0 0 0 559 82 0 0 25 0 32 0 26849399 2705022976 310945 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670407 0 0 4096 18946 0 0 0 -1 12 0 0 0
[pid=28204/tid=28214] ppid=28202 vsize=2641624 CPUtime=6.47 cores=0-31
/proc/28204/task/28214/stat : 28214 (claspmt) R 28202 28204 27592 0 -1 4202560 9638 0 0 0 561 86 0 0 18 0 32 0 26849399 2705022976 310945 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670077 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=28204/tid=28215] ppid=28202 vsize=2641624 CPUtime=6.48 cores=0-31
/proc/28204/task/28215/stat : 28215 (claspmt) R 28202 28204 27592 0 -1 4202560 12872 0 0 0 561 87 0 0 25 0 32 0 26849399 2705022976 310945 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=28204/tid=28216] ppid=28202 vsize=2641624 CPUtime=6.57 cores=0-31
/proc/28204/task/28216/stat : 28216 (claspmt) R 28202 28204 27592 0 -1 4202560 12124 0 0 0 565 92 0 0 25 0 32 0 26849399 2705022976 310945 270582939648 4194304 4964800 140735171401392 18446744073709551615 4668773 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=28204/tid=28217] ppid=28202 vsize=2641624 CPUtime=6.43 cores=0-31
/proc/28204/task/28217/stat : 28217 (claspmt) R 28202 28204 27592 0 -1 4202560 9611 0 0 0 559 84 0 0 25 0 32 0 26849399 2705022976 310945 270582939648 4194304 4964800 140735171401392 18446744073709551615 4668734 0 0 4096 18946 0 0 0 -1 27 0 0 0
[pid=28204/tid=28218] ppid=28202 vsize=2641624 CPUtime=7.71 cores=0-31
/proc/28204/task/28218/stat : 28218 (claspmt) R 28202 28204 27592 0 -1 4202560 10656 0 0 0 683 88 0 0 25 0 32 0 26849399 2705022976 310945 270582939648 4194304 4964800 140735171401392 18446744073709551615 4399292 0 0 4096 18946 0 0 0 -1 31 0 0 0
[pid=28204/tid=28219] ppid=28202 vsize=2641624 CPUtime=6.57 cores=0-31
/proc/28204/task/28219/stat : 28219 (claspmt) R 28202 28204 27592 0 -1 4202560 12483 0 0 0 565 92 0 0 25 0 32 0 26849399 2705022976 310945 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=28204/tid=28220] ppid=28202 vsize=2641624 CPUtime=6.66 cores=0-31
/proc/28204/task/28220/stat : 28220 (claspmt) R 28202 28204 27592 0 -1 4202560 10776 0 0 0 573 93 0 0 25 0 32 0 26849399 2705022976 310945 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670361 0 0 4096 18946 0 0 0 -1 14 0 0 0
[pid=28204/tid=28221] ppid=28202 vsize=2641624 CPUtime=6.47 cores=0-31
/proc/28204/task/28221/stat : 28221 (claspmt) R 28202 28204 27592 0 -1 4202560 9586 0 0 0 557 90 0 0 25 0 32 0 26849400 2705022976 310945 270582939648 4194304 4964800 140735171401392 18446744073709551615 4653464 0 0 4096 18946 0 0 0 -1 20 0 0 0
[pid=28204/tid=28222] ppid=28202 vsize=2641624 CPUtime=6.52 cores=0-31
/proc/28204/task/28222/stat : 28222 (claspmt) R 28202 28204 27592 0 -1 4202560 9749 0 0 0 561 91 0 0 25 0 32 0 26849400 2705022976 310945 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=28204/tid=28223] ppid=28202 vsize=2641624 CPUtime=6.55 cores=0-31
/proc/28204/task/28223/stat : 28223 (claspmt) R 28202 28204 27592 0 -1 4202560 12622 0 0 0 566 89 0 0 25 0 32 0 26849400 2705022976 310945 270582939648 4194304 4964800 140735171401392 18446744073709551615 4668672 0 0 4096 18946 0 0 0 -1 8 0 0 0
[pid=28204/tid=28224] ppid=28202 vsize=2641624 CPUtime=6.64 cores=0-31
/proc/28204/task/28224/stat : 28224 (claspmt) R 28202 28204 27592 0 -1 4202560 11000 0 0 0 567 97 0 0 25 0 32 0 26849400 2705022976 310946 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 16 0 0 0
[pid=28204/tid=28225] ppid=28202 vsize=2641624 CPUtime=6.5 cores=0-31
/proc/28204/task/28225/stat : 28225 (claspmt) R 28202 28204 27592 0 -1 4202560 9484 0 0 0 562 88 0 0 18 0 32 0 26849400 2705022976 310946 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 9 0 0 0
[pid=28204/tid=28226] ppid=28202 vsize=2641624 CPUtime=8.16 cores=0-31

################
# More data... #
################

/proc/28204/task/28222/stat : 28222 (claspmt) R 28202 28204 27592 0 -1 4202560 21727 0 0 0 495385 124 0 0 25 0 32 0 26849400 6266208256 1180494 270582939648 4194304 4964800 140735171401392 18446744073709551615 4669011 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=28204/tid=28223] ppid=28202 vsize=6119344 CPUtime=4954.15 cores=0-31
/proc/28204/task/28223/stat : 28223 (claspmt) R 28202 28204 27592 0 -1 4202560 53445 0 0 0 495271 144 0 0 25 0 32 0 26849400 6266208256 1180494 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 8 0 0 0
[pid=28204/tid=28224] ppid=28202 vsize=6119344 CPUtime=4954.78 cores=0-31
/proc/28204/task/28224/stat : 28224 (claspmt) R 28202 28204 27592 0 -1 4202560 22632 0 0 0 495353 125 0 0 25 0 32 0 26849400 6266208256 1180494 270582939648 4194304 4964800 140735171401392 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 12 0 0 0
[pid=28204/tid=28225] ppid=28202 vsize=6119344 CPUtime=4955.06 cores=0-31
/proc/28204/task/28225/stat : 28225 (claspmt) R 28202 28204 27592 0 -1 4202560 29645 0 0 0 495357 149 0 0 25 0 32 0 26849400 6266208256 1180494 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 29 0 0 0
[pid=28204/tid=28226] ppid=28202 vsize=6119344 CPUtime=4956.79 cores=0-31
/proc/28204/task/28226/stat : 28226 (claspmt) R 28202 28204 27592 0 -1 4202560 31804 0 0 0 495538 141 0 0 25 0 32 0 26849400 6266208256 1180494 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 28 0 0 0
[pid=28204/tid=28227] ppid=28202 vsize=6119344 CPUtime=4935.58 cores=0-31
/proc/28204/task/28227/stat : 28227 (claspmt) R 28202 28204 27592 0 -1 4202560 23923 0 0 0 493444 114 0 0 25 0 32 0 26849400 6266208256 1180494 270582939648 4194304 4964800 140735171401392 18446744073709551615 4684107 0 0 4096 18946 0 0 0 -1 22 0 0 0
[pid=28204/tid=28228] ppid=28202 vsize=6119344 CPUtime=4955.19 cores=0-31
/proc/28204/task/28228/stat : 28228 (claspmt) R 28202 28204 27592 0 -1 4202560 108203 0 0 0 494938 581 0 0 25 0 32 0 26849400 6266208256 1180494 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670440 0 0 4096 18946 0 0 0 -1 19 0 0 0
[pid=28204/tid=28229] ppid=28202 vsize=6119344 CPUtime=4954.88 cores=0-31
/proc/28204/task/28229/stat : 28229 (claspmt) R 28202 28204 27592 0 -1 4202560 24258 0 0 0 495334 154 0 0 25 0 32 0 26849400 6266208256 1180494 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=28204/tid=28230] ppid=28202 vsize=6119344 CPUtime=4954.98 cores=0-31
/proc/28204/task/28230/stat : 28230 (claspmt) R 28202 28204 27592 0 -1 4202560 20479 0 0 0 495382 116 0 0 25 0 32 0 26849400 6266208256 1180494 270582939648 4194304 4964800 140735171401392 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 11 0 0 0
[pid=28204/tid=28231] ppid=28202 vsize=6119344 CPUtime=4947.1 cores=0-31
/proc/28204/task/28231/stat : 28231 (claspmt) R 28202 28204 27592 0 -1 4202560 49116 0 0 0 494560 150 0 0 25 0 32 0 26849400 6266208256 1180494 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 31 0 0 0
[pid=28204/tid=28232] ppid=28202 vsize=6119344 CPUtime=4955.17 cores=0-31
/proc/28204/task/28232/stat : 28232 (claspmt) R 28202 28204 27592 0 -1 4202560 24234 0 0 0 495400 117 0 0 25 0 32 0 26849400 6266208256 1180494 270582939648 4194304 4964800 140735171401392 18446744073709551615 4668773 0 0 4096 18946 0 0 0 -1 18 0 0 0
[pid=28204/tid=28233] ppid=28202 vsize=6119344 CPUtime=4955.08 cores=0-31
/proc/28204/task/28233/stat : 28233 (claspmt) R 28202 28204 27592 0 -1 4202560 28517 0 0 0 495371 137 0 0 25 0 32 0 26849400 6266208256 1180494 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 10 0 0 0
[pid=28204/tid=28234] ppid=28202 vsize=6119344 CPUtime=4956.54 cores=0-31
/proc/28204/task/28234/stat : 28234 (claspmt) R 28202 28204 27592 0 -1 4202560 35550 0 0 0 495509 145 0 0 25 0 32 0 26849400 6266208256 1180494 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 17 0 0 0
[pid=28204/tid=28235] ppid=28202 vsize=6119344 CPUtime=4955.1 cores=0-31
/proc/28204/task/28235/stat : 28235 (claspmt) R 28202 28204 27592 0 -1 4202560 25998 0 0 0 495391 119 0 0 25 0 32 0 26849400 6266208256 1180494 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670323 0 0 4096 18946 0 0 0 -1 25 0 0 0
[pid=28204/tid=28236] ppid=28202 vsize=6119344 CPUtime=4915.18 cores=0-31
/proc/28204/task/28236/stat : 28236 (claspmt) R 28202 28204 27592 0 -1 4202560 97554 0 0 0 491007 511 0 0 25 0 32 0 26849400 6266208256 1180494 270582939648 4194304 4964800 140735171401392 18446744073709551615 4669022 0 0 4096 18946 0 0 0 -1 15 0 0 0
Current children cumulated CPU time (s) 158427
Current children cumulated vsize (KiB) 6119344



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5011.6 s]
/proc/loadavg: 32.07 32.07 31.97 34/445 28382
/proc/meminfo: memFree=257621596/264257624 swapFree=67111528/67111528
[pid=28204] ppid=28202 vsize=6119344 CPUtime=160004 cores=0-31
/proc/28204/stat : 28204 (claspmt) R 28202 28204 27592 0 -1 4202496 1273361 0 0 0 15994427 5982 0 0 25 0 32 0 26849105 6266208256 1184915 270582939648 4194304 4964800 140735171401392 18446744073709551615 4370224 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/28204/statm: 1529836 1184915 463 189 0 1524185 0
[pid=28204/tid=28206] ppid=28202 vsize=6119344 CPUtime=5004.48 cores=0-31
/proc/28204/task/28206/stat : 28206 (claspmt) R 28202 28204 27592 0 -1 4202560 18461 0 0 0 500325 123 0 0 25 0 32 0 26849399 6266208256 1184915 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 21 0 0 0
[pid=28204/tid=28207] ppid=28202 vsize=6119344 CPUtime=4959.77 cores=0-31
/proc/28204/task/28207/stat : 28207 (claspmt) R 28202 28204 27592 0 -1 4202560 56761 0 0 0 495813 164 0 0 25 0 32 0 26849399 6266208256 1184915 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 23 0 0 0
[pid=28204/tid=28208] ppid=28202 vsize=6119344 CPUtime=5004.56 cores=0-31
/proc/28204/task/28208/stat : 28208 (claspmt) R 28202 28204 27592 0 -1 4202560 19904 0 0 0 500343 113 0 0 25 0 32 0 26849399 6266208256 1184915 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 30 0 0 0
[pid=28204/tid=28209] ppid=28202 vsize=6119344 CPUtime=5004.42 cores=0-31
/proc/28204/task/28209/stat : 28209 (claspmt) R 28202 28204 27592 0 -1 4202560 31871 0 0 0 500306 136 0 0 25 0 32 0 26849399 6266208256 1184916 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 24 0 0 0
[pid=28204/tid=28210] ppid=28202 vsize=6119344 CPUtime=5005.89 cores=0-31
/proc/28204/task/28210/stat : 28210 (claspmt) R 28202 28204 27592 0 -1 4202560 29071 0 0 0 500446 143 0 0 25 0 32 0 26849399 6266208256 1184916 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 26 0 0 0
[pid=28204/tid=28211] ppid=28202 vsize=6119344 CPUtime=4981.75 cores=0-31
/proc/28204/task/28211/stat : 28211 (claspmt) R 28202 28204 27592 0 -1 4202560 25229 0 0 0 498059 116 0 0 25 0 32 0 26849399 6266208256 1184916 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 9 0 0 0
[pid=28204/tid=28212] ppid=28202 vsize=6119344 CPUtime=5004.57 cores=0-31
/proc/28204/task/28212/stat : 28212 (claspmt) R 28202 28204 27592 0 -1 4202560 109036 0 0 0 499856 601 0 0 25 0 32 0 26849399 6266208256 1184916 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 13 0 0 0
[pid=28204/tid=28213] ppid=28202 vsize=6119344 CPUtime=5001.7 cores=0-31
/proc/28204/task/28213/stat : 28213 (claspmt) R 28202 28204 27592 0 -1 4202560 19965 0 0 0 499993 177 0 0 25 0 32 0 26849399 6266208256 1184916 270582939648 4194304 4964800 140735171401392 18446744073709551615 4669131 0 0 4096 18946 0 0 0 -1 16 0 0 0
[pid=28204/tid=28214] ppid=28202 vsize=6119344 CPUtime=5004.36 cores=0-31
/proc/28204/task/28214/stat : 28214 (claspmt) R 28202 28204 27592 0 -1 4202560 21749 0 0 0 500313 123 0 0 25 0 32 0 26849399 6266208256 1184916 270582939648 4194304 4964800 140735171401392 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=28204/tid=28215] ppid=28202 vsize=6119344 CPUtime=5004.45 cores=0-31
/proc/28204/task/28215/stat : 28215 (claspmt) R 28202 28204 27592 0 -1 4202560 48240 0 0 0 500301 144 0 0 25 0 32 0 26849399 6266208256 1184916 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=28204/tid=28216] ppid=28202 vsize=6119344 CPUtime=5003.41 cores=0-31
/proc/28204/task/28216/stat : 28216 (claspmt) R 28202 28204 27592 0 -1 4202560 26000 0 0 0 500219 122 0 0 25 0 32 0 26849399 6266208256 1184916 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=28204/tid=28217] ppid=28202 vsize=6119344 CPUtime=5004.37 cores=0-31
/proc/28204/task/28217/stat : 28217 (claspmt) R 28202 28204 27592 0 -1 4202560 31493 0 0 0 500294 143 0 0 25 0 32 0 26849399 6266208256 1184916 270582939648 4194304 4964800 140735171401392 18446744073709551615 4370295 0 0 4096 18946 0 0 0 -1 27 0 0 0
[pid=28204/tid=28218] ppid=28202 vsize=6119344 CPUtime=5000.99 cores=0-31
/proc/28204/task/28218/stat : 28218 (claspmt) R 28202 28204 27592 0 -1 4202560 31749 0 0 0 499955 144 0 0 25 0 32 0 26849399 6266208256 1184916 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=28204/tid=28219] ppid=28202 vsize=6119344 CPUtime=5004.48 cores=0-31
/proc/28204/task/28219/stat : 28219 (claspmt) R 28202 28204 27592 0 -1 4202560 22737 0 0 0 500334 114 0 0 25 0 32 0 26849399 6266208256 1184916 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=28204/tid=28220] ppid=28202 vsize=6119344 CPUtime=5004.59 cores=0-31
/proc/28204/task/28220/stat : 28220 (claspmt) R 28202 28204 27592 0 -1 4202560 106950 0 0 0 499869 590 0 0 25 0 32 0 26849399 6266208256 1184916 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 14 0 0 0
[pid=28204/tid=28221] ppid=28202 vsize=6119344 CPUtime=5004.41 cores=0-31
/proc/28204/task/28221/stat : 28221 (claspmt) R 28202 28204 27592 0 -1 4202560 19851 0 0 0 500307 134 0 0 25 0 32 0 26849400 6266208256 1184916 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 20 0 0 0
[pid=28204/tid=28222] ppid=28202 vsize=6119344 CPUtime=5004.42 cores=0-31
/proc/28204/task/28222/stat : 28222 (claspmt) R 28202 28204 27592 0 -1 4202560 21727 0 0 0 500318 124 0 0 25 0 32 0 26849400 6266208256 1184916 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=28204/tid=28223] ppid=28202 vsize=6119344 CPUtime=5003.47 cores=0-31
/proc/28204/task/28223/stat : 28223 (claspmt) R 28202 28204 27592 0 -1 4202560 53618 0 0 0 500202 145 0 0 25 0 32 0 26849400 6266208256 1184916 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 8 0 0 0
[pid=28204/tid=28224] ppid=28202 vsize=6119344 CPUtime=5004.11 cores=0-31
/proc/28204/task/28224/stat : 28224 (claspmt) R 28202 28204 27592 0 -1 4202560 22648 0 0 0 500286 125 0 0 25 0 32 0 26849400 6266208256 1184916 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670343 0 0 4096 18946 0 0 0 -1 12 0 0 0
[pid=28204/tid=28225] ppid=28202 vsize=6119344 CPUtime=5004.4 cores=0-31
/proc/28204/task/28225/stat : 28225 (claspmt) R 28202 28204 27592 0 -1 4202560 30036 0 0 0 500288 152 0 0 25 0 32 0 26849400 6266208256 1184916 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 29 0 0 0
[pid=28204/tid=28226] ppid=28202 vsize=6119344 CPUtime=5006.12 cores=0-31
/proc/28204/task/28226/stat : 28226 (claspmt) R 28202 28204 27592 0 -1 4202560 31840 0 0 0 500471 141 0 0 25 0 32 0 26849400 6266208256 1184916 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 28 0 0 0
[pid=28204/tid=28227] ppid=28202 vsize=6119344 CPUtime=4984.91 cores=0-31
/proc/28204/task/28227/stat : 28227 (claspmt) R 28202 28204 27592 0 -1 4202560 23923 0 0 0 498377 114 0 0 25 0 32 0 26849400 6266208256 1184916 270582939648 4194304 4964800 140735171401392 18446744073709551615 4378499 0 0 4096 18946 0 0 0 -1 22 0 0 0
[pid=28204/tid=28228] ppid=28202 vsize=6119344 CPUtime=5004.52 cores=0-31
/proc/28204/task/28228/stat : 28228 (claspmt) R 28202 28204 27592 0 -1 4202560 108342 0 0 0 499870 582 0 0 25 0 32 0 26849400 6266208256 1184916 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 19 0 0 0
[pid=28204/tid=28229] ppid=28202 vsize=6119344 CPUtime=5004.21 cores=0-31
/proc/28204/task/28229/stat : 28229 (claspmt) R 28202 28204 27592 0 -1 4202560 24258 0 0 0 500267 154 0 0 25 0 32 0 26849400 6266208256 1184916 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=28204/tid=28230] ppid=28202 vsize=6119344 CPUtime=5004.31 cores=0-31
/proc/28204/task/28230/stat : 28230 (claspmt) R 28202 28204 27592 0 -1 4202560 20488 0 0 0 500315 116 0 0 25 0 32 0 26849400 6266208256 1184916 270582939648 4194304 4964800 140735171401392 18446744073709551615 4342157 0 0 4096 18946 0 0 0 -1 11 0 0 0
[pid=28204/tid=28231] ppid=28202 vsize=6119344 CPUtime=4995.02 cores=0-31
/proc/28204/task/28231/stat : 28231 (claspmt) R 28202 28204 27592 0 -1 4202560 49130 0 0 0 499352 150 0 0 25 0 32 0 26849400 6266208256 1184916 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 31 0 0 0
[pid=28204/tid=28232] ppid=28202 vsize=6119344 CPUtime=5004.5 cores=0-31
/proc/28204/task/28232/stat : 28232 (claspmt) R 28202 28204 27592 0 -1 4202560 24234 0 0 0 500333 117 0 0 25 0 32 0 26849400 6266208256 1184916 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 18 0 0 0
[pid=28204/tid=28233] ppid=28202 vsize=6119344 CPUtime=5004.41 cores=0-31
/proc/28204/task/28233/stat : 28233 (claspmt) R 28202 28204 27592 0 -1 4202560 28517 0 0 0 500304 137 0 0 25 0 32 0 26849400 6266208256 1184916 270582939648 4194304 4964800 140735171401392 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 10 0 0 0
[pid=28204/tid=28234] ppid=28202 vsize=6119344 CPUtime=5005.87 cores=0-31
/proc/28204/task/28234/stat : 28234 (claspmt) R 28202 28204 27592 0 -1 4202560 37215 0 0 0 500440 147 0 0 25 0 32 0 26849400 6266208256 1184916 270582939648 4194304 4964800 140735171401392 18446744073709551615 4669011 0 0 4096 18946 0 0 0 -1 17 0 0 0
[pid=28204/tid=28235] ppid=28202 vsize=6119344 CPUtime=5004.43 cores=0-31
/proc/28204/task/28235/stat : 28235 (claspmt) R 28202 28204 27592 0 -1 4202560 25998 0 0 0 500324 119 0 0 25 0 32 0 26849400 6266208256 1184916 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 25 0 0 0
[pid=28204/tid=28236] ppid=28202 vsize=6119344 CPUtime=4964.51 cores=0-31
/proc/28204/task/28236/stat : 28236 (claspmt) R 28202 28204 27592 0 -1 4202560 97722 0 0 0 495940 511 0 0 25 0 32 0 26849400 6266208256 1184916 270582939648 4194304 4964800 140735171401392 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 15 0 0 0
Current children cumulated CPU time (s) 160004
Current children cumulated vsize (KiB) 6119344

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)

Sending SIGKILL to -28204

Solver just ended. Dumping a history of the last processes samples

[startup+5017.5 s]
/proc/loadavg: 29.58 31.55 31.81 2/414 28383
/proc/meminfo: memFree=255090356/264257624 swapFree=67111528/67111528
[pid=28204] ppid=28202 vsize=5948552 CPUtime=160010 cores=0-31
/proc/28204/stat : 28204 (claspmt) R 28202 28204 27592 0 -1 4204032 1273379 0 0 0 15994584 6454 0 0 25 0 1 0 26849105 6091317248 1164575 270582939648 4194304 4964800 140735171401392 18446744073709551615 258650611056 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/28204/statm: 1487138 1164575 474 189 0 1481487 0
Current children cumulated CPU time (s) 160010
Current children cumulated vsize (KiB) 5948552

[startup+5019.1 s]
/proc/loadavg: 29.58 31.55 31.81 2/414 28383
/proc/meminfo: memFree=254413440/264257624 swapFree=67111528/67111528
[pid=28204] ppid=28202 vsize=5948552 CPUtime=160012 cores=0-31
/proc/28204/stat : 28204 (claspmt) R 28202 28204 27592 0 -1 4204032 1273379 0 0 0 15994584 6614 0 0 25 0 1 0 26849105 6091317248 1164575 270582939648 4194304 4964800 140735171401392 18446744073709551615 258650611056 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/28204/statm: 1487138 1164575 474 189 0 1481487 0
Current children cumulated CPU time (s) 160012
Current children cumulated vsize (KiB) 5948552

[startup+5019.9 s]
/proc/loadavg: 27.45 31.08 31.65 2/414 28383
/proc/meminfo: memFree=253733480/264257624 swapFree=67111528/67111528
[pid=28204] ppid=28202 vsize=5948552 CPUtime=160013 cores=0-31
/proc/28204/stat : 28204 (claspmt) R 28202 28204 27592 0 -1 4204032 1273379 0 0 0 15994584 6694 0 0 25 0 1 0 26849105 6091317248 1164575 270582939648 4194304 4964800 140735171401392 18446744073709551615 258650611056 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/28204/statm: 1487138 1164575 474 189 0 1481487 0
Current children cumulated CPU time (s) 160013
Current children cumulated vsize (KiB) 5948552

[startup+5020.7 s]
/proc/loadavg: 27.45 31.08 31.65 2/414 28383
/proc/meminfo: memFree=253057928/264257624 swapFree=67111528/67111528
[pid=28204] ppid=28202 vsize=5948552 CPUtime=160014 cores=0-31
/proc/28204/stat : 28204 (claspmt) R 28202 28204 27592 0 -1 4204032 1273379 0 0 0 15994584 6774 0 0 25 0 1 0 26849105 6091317248 1164575 270582939648 4194304 4964800 140735171401392 18446744073709551615 258650611056 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/28204/statm: 1487138 1164575 474 189 0 1481487 0
Current children cumulated CPU time (s) 160014
Current children cumulated vsize (KiB) 5948552

[startup+5020.9 s]
/proc/loadavg: 27.45 31.08 31.65 2/414 28383
/proc/meminfo: memFree=253057928/264257624 swapFree=67111528/67111528
[pid=28204] ppid=28202 vsize=0 CPUtime=160014 cores=0-31
/proc/28204/stat : 28204 (claspmt) R 28202 28204 27592 0 -1 4204036 1273379 0 0 0 15994584 6794 0 0 25 0 1 0 26849105 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/28204/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 160014
Current children cumulated vsize (KiB) 0

[startup+5021 s]
/proc/loadavg: 27.45 31.08 31.65 2/414 28383
/proc/meminfo: memFree=253057928/264257624 swapFree=67111528/67111528
[pid=28204] ppid=28202 vsize=0 CPUtime=160014 cores=0-31
/proc/28204/stat : 28204 (claspmt) R 28202 28204 27592 0 -1 4204036 1273379 0 0 0 15994584 6804 0 0 25 0 1 0 26849105 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/28204/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 160014
Current children cumulated vsize (KiB) 0

Child ended because it received signal 24 (SIGXCPU)
Child dumped core
Real time (s): 5021.06
CPU time (s): 160014
CPU user time (s): 159946
CPU system time (s): 68.1036
CPU usage (%): 3186.85
Max. virtual memory (cumulated for all children) (KiB): 6119344

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 159946
system time used= 68.1036
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1273379
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 277511
involuntary context switches= 392515

runsolver used 28.0477 second user time and 113.844 second system time

The end

Launcher Data 

Begin job on node200 at 2011-06-09 12:17:02
IDJOB=3472410
IDBENCH=82459
IDSOLVER=1961
FILE ID=node200/3472410-1307614621
RUNJOBID= node200-1307614385-27606
PBS_JOBID= 13547731
Free space on /tmp= 61732 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT11/application/jarvisalo/smtqfbv-aigs/smtlib-qfbv-aigs-VS3-benchmark-S2-tseitin.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3472410-1307614621/watcher-3472410-1307614621 -o /tmp/evaluation-result-3472410-1307614621/solver-3472410-1307614621 -C 160000 -W 5100 -M 256000  HOME/claspmt -t32 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3472410-1307614621.cnf

TIME LIMIT= 160000 seconds
MEMORY LIMIT= 256000 MiB
NBCORE= 32

MD5SUM BENCH= b967fca11e85ef5b614d3452b1c4b2ad
RANDOM SEED=1850194087

node200.alineos.net Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 64
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 96
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 66
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 98
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.99
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 68
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 100
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.03
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 70
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 102
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 16
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 17
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 48
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 18
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 80
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 19
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 112
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.97
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 20
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 21
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 50
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.10
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 22
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 82
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.10
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 114
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 24
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 25
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 26
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 84
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 27
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 116
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 28
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 29
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 54
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 30
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 86
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 31
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 118
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     264257624 kB
MemFree:      262361404 kB
Buffers:        221060 kB
Cached:        1380440 kB
SwapCached:          0 kB
Active:         256500 kB
Inactive:      1412636 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     264257624 kB
LowFree:      262361404 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           15132 kB
Writeback:         112 kB
AnonPages:       68800 kB
Mapped:          14484 kB
Slab:           146728 kB
PageTables:       4168 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  199240340 kB
Committed_AS:   173824 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296372 kB
VmallocChunk: 34359440603 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 57176 MiB
End job on node200 at 2011-06-09 13:40:43