Trace number 3472552

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 155888 5105.22

General information on the benchmark

NameSAT-Race-2010-CNF/hardware-verification/
velev/2dlx_ca_bp_f_liveness.cnf
MD5SUM99cd8de7f06ce8c087a5c7fb299dfb62
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables202253
Number of clauses4313014
Sum of the clauses size12562188
Maximum clause length651
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 23935640
Number of clauses of size 3199653
Number of clauses of size 410363
Number of clauses of size 55846
Number of clauses of size over 5161511

Solver Data 

0.00/0.01	c clasp version 2.0.0-RC2
0.00/0.01	c Reading from ...867405/instance-3472552-1307867405.cnf
1.99/2.02	c Solving...
155882.03/5100.03	
155882.03/5100.03	*** INTERRUPTED! ***
155882.03/5100.03	*** Shutting down threads...

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 988)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3472552-1307867405/watcher-3472552-1307867405 -o /tmp/evaluation-result-3472552-1307867405/solver-3472552-1307867405 -C 160000 -W 5100 -M 256000 HOME/claspmt -t32 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3472552-1307867405.cnf 

running on 32 cores: 0,4,8,12,16,20,24,28,1,5,9,13,17,21,25,29,2,6,10,14,18,22,26,30,3,7,11,15,19,23,27,31

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 160000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 160030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 262144000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 262195200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 25.24 30.54 31.20 1/413 22965
/proc/meminfo: memFree=261689956/264257624 swapFree=67111528/67111528
[pid=22965] ppid=22963 vsize=46560 CPUtime=0.02 cores=0-31
/proc/22965/stat : 22965 (claspmt) R 22963 22965 22935 0 -1 4202496 2467 0 0 0 2 0 0 0 23 0 1 0 52127785 47677440 2397 270582939648 4194304 4964800 140735186988704 18446744073709551615 4700412 0 0 4096 18946 0 0 0 17 16 0 0 0
/proc/22965/statm: 11640 2402 373 189 0 6514 0

[startup+0.122978 s]
/proc/loadavg: 25.24 30.54 31.20 1/413 22965
/proc/meminfo: memFree=261689956/264257624 swapFree=67111528/67111528
[pid=22965] ppid=22963 vsize=61084 CPUtime=0.11 cores=0-31
/proc/22965/stat : 22965 (claspmt) R 22963 22965 22935 0 -1 4202496 11956 0 0 0 9 2 0 0 23 0 1 0 52127785 62550016 10306 270582939648 4194304 4964800 140735186988704 18446744073709551615 258650955760 0 0 4096 18946 0 0 0 17 16 0 0 0
/proc/22965/statm: 15271 10307 385 189 0 10145 0
Current children cumulated CPU time (s) 0.11
Current children cumulated vsize (KiB) 61084

[startup+0.201015 s]
/proc/loadavg: 25.24 30.54 31.20 1/413 22965
/proc/meminfo: memFree=261689956/264257624 swapFree=67111528/67111528
[pid=22965] ppid=22963 vsize=63948 CPUtime=0.19 cores=0-31
/proc/22965/stat : 22965 (claspmt) R 22963 22965 22935 0 -1 4202496 12667 0 0 0 16 3 0 0 23 0 1 0 52127785 65482752 11017 270582939648 4194304 4964800 140735186988704 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 17 16 0 0 0
/proc/22965/statm: 15987 11019 385 189 0 10861 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 63948

[startup+0.300974 s]
/proc/loadavg: 25.24 30.54 31.20 1/413 22965
/proc/meminfo: memFree=261689956/264257624 swapFree=67111528/67111528
[pid=22965] ppid=22963 vsize=67364 CPUtime=0.29 cores=0-31
/proc/22965/stat : 22965 (claspmt) R 22963 22965 22935 0 -1 4202496 13556 0 0 0 26 3 0 0 24 0 1 0 52127785 68980736 11873 270582939648 4194304 4964800 140735186988704 18446744073709551615 258650955760 0 0 4096 18946 0 0 0 17 16 0 0 0
/proc/22965/statm: 16841 11873 385 189 0 11715 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 67364

[startup+0.700999 s]
/proc/loadavg: 25.24 30.54 31.20 1/413 22965
/proc/meminfo: memFree=261689956/264257624 swapFree=67111528/67111528
[pid=22965] ppid=22963 vsize=79876 CPUtime=0.69 cores=0-31
/proc/22965/stat : 22965 (claspmt) R 22963 22965 22935 0 -1 4202496 16809 0 0 0 64 5 0 0 25 0 1 0 52127785 81793024 15001 270582939648 4194304 4964800 140735186988704 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 17 16 0 0 0
/proc/22965/statm: 19969 15001 385 189 0 14843 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 79876

[startup+1.50104 s]
/proc/loadavg: 23.30 30.05 31.04 2/414 22966
/proc/meminfo: memFree=261611660/264257624 swapFree=67111528/67111528
[pid=22965] ppid=22963 vsize=121328 CPUtime=1.49 cores=0-31
/proc/22965/stat : 22965 (claspmt) R 22963 22965 22935 0 -1 4202496 27478 0 0 0 138 11 0 0 25 0 1 0 52127785 124239872 25140 270582939648 4194304 4964800 140735186988704 18446744073709551615 258650955760 0 0 4096 18946 0 0 0 17 16 0 0 0
/proc/22965/statm: 30332 25140 385 189 0 25206 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 121328

[startup+3.103 s]
/proc/loadavg: 23.30 30.05 31.04 32/445 22997
/proc/meminfo: memFree=261428744/264257624 swapFree=67111528/67111528
[pid=22965] ppid=22963 vsize=1962740 CPUtime=20.98 cores=0-31
/proc/22965/stat : 22965 (claspmt) S 22963 22965 22935 0 -1 4202496 211256 0 0 0 301 1797 0 0 18 0 32 0 52127785 2009845760 156292 270582939648 4194304 4964800 140735186988704 18446744073709551615 258662772457 0 0 4096 18946 18446744073709551615 0 0 17 25 0 0 0
/proc/22965/statm: 490685 156293 428 189 0 485034 0
[pid=22965/tid=22967] ppid=22963 vsize=1962740 CPUtime=0.58 cores=0-31
/proc/22965/task/22967/stat : 22967 (claspmt) R 22963 22965 22935 0 -1 4202560 5813 0 0 0 4 54 0 0 17 0 32 0 52127988 2009845760 156306 270582939648 4194304 4964800 140735186988704 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 -1 17 0 0 0
[pid=22965/tid=22968] ppid=22963 vsize=1962740 CPUtime=0.63 cores=0-31
/proc/22965/task/22968/stat : 22968 (claspmt) D 22963 22965 22935 0 -1 4202560 4645 0 0 0 2 61 0 0 18 0 32 0 52127988 2009845760 156307 270582939648 4194304 4964800 140735186988704 18446744073709551615 4378021 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=22965/tid=22969] ppid=22963 vsize=1962740 CPUtime=0.6 cores=0-31
/proc/22965/task/22969/stat : 22969 (claspmt) D 22963 22965 22935 0 -1 4202560 5720 0 0 0 2 58 0 0 17 0 32 0 52127988 2009845760 156307 270582939648 4194304 4964800 140735186988704 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=22965/tid=22970] ppid=22963 vsize=1962740 CPUtime=0.61 cores=0-31
/proc/22965/task/22970/stat : 22970 (claspmt) D 22963 22965 22935 0 -1 4202560 5705 0 0 0 4 57 0 0 17 0 32 0 52127988 2009845760 156312 270582939648 4194304 4964800 140735186988704 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 18 0 0 0
[pid=22965/tid=22971] ppid=22963 vsize=1962740 CPUtime=0.56 cores=0-31
/proc/22965/task/22971/stat : 22971 (claspmt) S 22963 22965 22935 0 -1 4202560 8467 0 0 0 10 46 0 0 17 0 32 0 52127988 2009845760 156313 270582939648 4194304 4964800 140735186988704 18446744073709551615 258651060958 0 0 4096 18946 18446744071566926264 0 0 -1 19 0 0 0
[pid=22965/tid=22972] ppid=22963 vsize=1962740 CPUtime=0.63 cores=0-31
/proc/22965/task/22972/stat : 22972 (claspmt) S 22963 22965 22935 0 -1 4202560 5588 0 0 0 2 61 0 0 17 0 32 0 52127988 2009845760 156319 270582939648 4194304 4964800 140735186988704 18446744073709551615 258651060958 0 0 4096 18946 18446744071566927184 0 0 -1 28 0 0 0
[pid=22965/tid=22973] ppid=22963 vsize=1962740 CPUtime=0.64 cores=0-31
/proc/22965/task/22973/stat : 22973 (claspmt) D 22963 22965 22935 0 -1 4202560 5832 0 0 0 1 63 0 0 18 0 32 0 52127988 2009845760 156324 270582939648 4194304 4964800 140735186988704 18446744073709551615 4685574 0 0 4096 18946 0 0 0 -1 20 0 0 0
[pid=22965/tid=22974] ppid=22963 vsize=1962740 CPUtime=0.54 cores=0-31
/proc/22965/task/22974/stat : 22974 (claspmt) D 22963 22965 22935 0 -1 4202560 6405 0 0 0 6 48 0 0 17 0 32 0 52127988 2009845760 156330 270582939648 4194304 4964800 140735186988704 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=22965/tid=22975] ppid=22963 vsize=1962740 CPUtime=0.56 cores=0-31
/proc/22965/task/22975/stat : 22975 (claspmt) D 22963 22965 22935 0 -1 4202560 4180 0 0 0 2 54 0 0 18 0 32 0 52127988 2009845760 156334 270582939648 4194304 4964800 140735186988704 18446744073709551615 4679094 0 0 4096 18946 0 0 0 -1 27 0 0 0
[pid=22965/tid=22976] ppid=22963 vsize=1962740 CPUtime=0.54 cores=0-31
/proc/22965/task/22976/stat : 22976 (claspmt) S 22963 22965 22935 0 -1 4202560 5854 0 0 0 3 51 0 0 17 0 32 0 52127988 2009845760 156338 270582939648 4194304 4964800 140735186988704 18446744073709551615 258651060958 0 0 4096 18946 18446744071566923904 0 0 -1 23 0 0 0
[pid=22965/tid=22977] ppid=22963 vsize=1962740 CPUtime=0.62 cores=0-31
/proc/22965/task/22977/stat : 22977 (claspmt) D 22963 22965 22935 0 -1 4202560 5583 0 0 0 3 59 0 0 17 0 32 0 52127988 2009845760 156341 270582939648 4194304 4964800 140735186988704 18446744073709551615 4685574 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=22965/tid=22978] ppid=22963 vsize=1962740 CPUtime=0.62 cores=0-31
/proc/22965/task/22978/stat : 22978 (claspmt) R 22963 22965 22935 0 -1 4202560 5362 0 0 0 2 60 0 0 17 0 32 0 52127988 2009845760 156344 270582939648 4194304 4964800 140735186988704 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 12 0 0 0
[pid=22965/tid=22979] ppid=22963 vsize=1962740 CPUtime=0.6 cores=0-31
/proc/22965/task/22979/stat : 22979 (claspmt) S 22963 22965 22935 0 -1 4202560 5132 0 0 0 3 57 0 0 17 0 32 0 52127988 2009845760 156345 270582939648 4194304 4964800 140735186988704 18446744073709551615 258651060958 0 0 4096 18946 18446744071566926984 0 0 -1 11 0 0 0
[pid=22965/tid=22980] ppid=22963 vsize=1962740 CPUtime=0.64 cores=0-31
/proc/22965/task/22980/stat : 22980 (claspmt) S 22963 22965 22935 0 -1 4202560 5684 0 0 0 3 61 0 0 17 0 32 0 52127988 2009845760 156348 270582939648 4194304 4964800 140735186988704 18446744073709551615 258651060958 0 0 4096 18946 18446744071566932544 0 0 -1 30 0 0 0
[pid=22965/tid=22981] ppid=22963 vsize=1962740 CPUtime=0.59 cores=0-31
/proc/22965/task/22981/stat : 22981 (claspmt) D 22963 22965 22935 0 -1 4202560 6400 0 0 0 4 55 0 0 17 0 32 0 52127988 2009845760 156350 270582939648 4194304 4964800 140735186988704 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=22965/tid=22982] ppid=22963 vsize=1962740 CPUtime=0.59 cores=0-31
/proc/22965/task/22982/stat : 22982 (claspmt) D 22963 22965 22935 0 -1 4202560 5516 0 0 0 2 57 0 0 17 0 32 0 52127988 2009845760 156355 270582939648 4194304 4964800 140735186988704 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 -1 8 0 0 0
[pid=22965/tid=22983] ppid=22963 vsize=1962740 CPUtime=0.62 cores=0-31
/proc/22965/task/22983/stat : 22983 (claspmt) D 22963 22965 22935 0 -1 4202560 4027 0 0 0 3 59 0 0 18 0 32 0 52127988 2009845760 156366 270582939648 4194304 4964800 140735186988704 18446744073709551615 4679151 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=22965/tid=22984] ppid=22963 vsize=1962740 CPUtime=0.61 cores=0-31
/proc/22965/task/22984/stat : 22984 (claspmt) D 22963 22965 22935 0 -1 4202560 5761 0 0 0 1 60 0 0 17 0 32 0 52127988 2009845760 156368 270582939648 4194304 4964800 140735186988704 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 24 0 0 0
[pid=22965/tid=22985] ppid=22963 vsize=1962740 CPUtime=0.64 cores=0-31
/proc/22965/task/22985/stat : 22985 (claspmt) D 22963 22965 22935 0 -1 4202560 4644 0 0 0 1 63 0 0 18 0 32 0 52127988 2009845760 156374 270582939648 4194304 4964800 140735186988704 18446744073709551615 4378056 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=22965/tid=22986] ppid=22963 vsize=1962740 CPUtime=0.63 cores=0-31
/proc/22965/task/22986/stat : 22986 (claspmt) D 22963 22965 22935 0 -1 4202560 5353 0 0 0 3 60 0 0 17 0 32 0 52127988 2009845760 156377 270582939648 4194304 4964800 140735186988704 18446744073709551615 4697753 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=22965/tid=22987] ppid=22963 vsize=1962740 CPUtime=0.58 cores=0-31
/proc/22965/task/22987/stat : 22987 (claspmt) D 22963 22965 22935 0 -1 4202560 5799 0 0 0 4 54 0 0 17 0 32 0 52127988 2009845760 156381 270582939648 4194304 4964800 140735186988704 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 15 0 0 0
[pid=22965/tid=22988] ppid=22963 vsize=1962740 CPUtime=0.62 cores=0-31
/proc/22965/task/22988/stat : 22988 (claspmt) S 22963 22965 22935 0 -1 4202560 5689 0 0 0 3 59 0 0 17 0 32 0 52127988 2009845760 156382 270582939648 4194304 4964800 140735186988704 18446744073709551615 258651060958 0 0 4096 18946 18446744071566928704 0 0 -1 16 0 0 0
[pid=22965/tid=22989] ppid=22963 vsize=1962740 CPUtime=0.54 cores=0-31
/proc/22965/task/22989/stat : 22989 (claspmt) S 22963 22965 22935 0 -1 4202560 5918 0 0 0 2 52 0 0 17 0 32 0 52127988 2009845760 156387 270582939648 4194304 4964800 140735186988704 18446744073709551615 258651060958 0 0 4096 18946 18446744071566924864 0 0 -1 26 0 0 0
[pid=22965/tid=22990] ppid=22963 vsize=1962740 CPUtime=0.6 cores=0-31
/proc/22965/task/22990/stat : 22990 (claspmt) S 22963 22965 22935 0 -1 4202560 5473 0 0 0 4 56 0 0 17 0 32 0 52127988 2009845760 156391 270582939648 4194304 4964800 140735186988704 18446744073709551615 258651060958 0 0 4096 18946 18446744071566927184 0 0 -1 13 0 0 0
[pid=22965/tid=22991] ppid=22963 vsize=1962740 CPUtime=0.62 cores=0-31
/proc/22965/task/22991/stat : 22991 (claspmt) D 22963 22965 22935 0 -1 4202560 5492 0 0 0 4 58 0 0 17 0 32 0 52127988 2009845760 156396 270582939648 4194304 4964800 140735186988704 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 10 0 0 0
[pid=22965/tid=22992] ppid=22963 vsize=1962740 CPUtime=0.57 cores=0-31
/proc/22965/task/22992/stat : 22992 (claspmt) S 22963 22965 22935 0 -1 4202560 5582 0 0 0 3 54 0 0 17 0 32 0 52127989 2009845760 156404 270582939648 4194304 4964800 140735186988704 18446744073709551615 258651060958 0 0 4096 18946 18446744071566932384 0 0 -1 29 0 0 0
[pid=22965/tid=22993] ppid=22963 vsize=1962740 CPUtime=0.63 cores=0-31
/proc/22965/task/22993/stat : 22993 (claspmt) D 22963 22965 22935 0 -1 4202560 4921 0 0 0 3 60 0 0 18 0 32 0 52127989 2009845760 156407 270582939648 4194304 4964800 140735186988704 18446744073709551615 4378056 0 0 4096 18946 0 0 0 -1 9 0 0 0
[pid=22965/tid=22994] ppid=22963 vsize=1962740 CPUtime=0.6 cores=0-31
/proc/22965/task/22994/stat : 22994 (claspmt) R 22963 22965 22935 0 -1 4202560 5294 0 0 0 4 56 0 0 17 0 32 0 52127989 2009845760 156411 270582939648 4194304 4964800 140735186988704 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 22 0 0 0
[pid=22965/tid=22995] ppid=22963 vsize=1962740 CPUtime=0.61 cores=0-31
/proc/22965/task/22995/stat : 22995 (claspmt) D 22963 22965 22935 0 -1 4202560 5463 0 0 0 3 58 0 0 17 0 32 0 52127989 2009845760 156417 270582939648 4194304 4964800 140735186988704 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 21 0 0 0
[pid=22965/tid=22996] ppid=22963 vsize=1962740 CPUtime=0.59 cores=0-31
/proc/22965/task/22996/stat : 22996 (claspmt) D 22963 22965 22935 0 -1 4202560 5898 0 0 0 3 56 0 0 17 0 32 0 52127989 2009845760 156417 270582939648 4194304 4964800 140735186988704 18446744073709551615 258650999207 0 0 4096 18946 0 0 0 -1 25 0 0 0
[pid=22965/tid=22997] ppid=22963 vsize=1962740 CPUtime=0.62 cores=0-31
/proc/22965/task/22997/stat : 22997 (claspmt) S 22963 22965 22935 0 -1 4202560 5996 0 0 0 3 59 0 0 17 0 32 0 52127989 2009845760 156419 270582939648 4194304 4964800 140735186988704 18446744073709551615 258651060958 0 0 4096 18946 18446604488308588000 0 0 -1 14 0 0 0
Current children cumulated CPU time (s) 20.98
Current children cumulated vsize (KiB) 1962740

[startup+6.30314 s]
/proc/loadavg: 23.44 29.96 31.01 5/445 22997
/proc/meminfo: memFree=260516240/264257624 swapFree=67111528/67111528
[pid=22965] ppid=22963 vsize=2395868 CPUtime=38.33 cores=0-31
/proc/22965/stat : 22965 (claspmt) S 22963 22965 22935 0 -1 4202496 401583 0 0 0 1299 2534 0 0 18 0 32 0 52127785 2453368832 345500 270582939648 4194304 4964800 140735186988704 18446744073709551615 258662772457 0 0 4096 18946 18446744073709551615 0 0 17 25 0 0 0
/proc/22965/statm: 598967 345504 434 189 0 593316 0
[pid=22965/tid=22967] ppid=22963 vsize=2395868 CPUtime=0.98 cores=0-31
/proc/22965/task/22967/stat : 22967 (claspmt) S 22963 22965 22935 0 -1 4202560 10543 0 0 0 22 76 0 0 15 0 32 0 52127988 2453368832 345505 270582939648 4194304 4964800 140735186988704 18446744073709551615 258662772457 0 0 4096 18946 18446604497008459232 0 0 -1 17 0 0 0
[pid=22965/tid=22968] ppid=22963 vsize=2395868 CPUtime=1.21 cores=0-31
/proc/22965/task/22968/stat : 22968 (claspmt) R 22963 22965 22935 0 -1 4202560 12016 0 0 0 36 85 0 0 16 0 32 0 52127988 2453368832 345507 270582939648 4194304 4964800 140735186988704 18446744073709551615 4652258 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=22965/tid=22969] ppid=22963 vsize=2395868 CPUtime=1.21 cores=0-31
/proc/22965/task/22969/stat : 22969 (claspmt) R 22963 22965 22935 0 -1 4202560 12996 0 0 0 40 81 0 0 16 0 32 0 52127988 2453368832 345509 270582939648 4194304 4964800 140735186988704 18446744073709551615 4375558 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=22965/tid=22970] ppid=22963 vsize=2395868 CPUtime=1.04 cores=0-31
/proc/22965/task/22970/stat : 22970 (claspmt) S 22963 22965 22935 0 -1 4202560 10583 0 0 0 22 82 0 0 15 0 32 0 52127988 2453368832 345510 270582939648 4194304 4964800 140735186988704 18446744073709551615 258662772457 0 0 4096 18946 18446604497008434656 0 0 -1 18 0 0 0
[pid=22965/tid=22971] ppid=22963 vsize=2395868 CPUtime=1.77 cores=0-31
/proc/22965/task/22971/stat : 22971 (claspmt) R 22963 22965 22935 0 -1 4202560 12748 0 0 0 112 65 0 0 16 0 32 0 52127988 2453368832 345512 270582939648 4194304 4964800 140735186988704 18446744073709551615 4670273 0 0 4096 18946 0 0 0 -1 19 0 0 0
[pid=22965/tid=22972] ppid=22963 vsize=2395868 CPUtime=1.23 cores=0-31
/proc/22965/task/22972/stat : 22972 (claspmt) R 22963 22965 22935 0 -1 4202560 12282 0 0 0 38 85 0 0 16 0 32 0 52127988 2453368832 345513 270582939648 4194304 4964800 140735186988704 18446744073709551615 4342208 0 0 4096 18946 0 0 0 -1 28 0 0 0
[pid=22965/tid=22973] ppid=22963 vsize=2395868 CPUtime=1.23 cores=0-31
/proc/22965/task/22973/stat : 22973 (claspmt) R 22963 22965 22935 0 -1 4202560 12106 0 0 0 35 88 0 0 16 0 32 0 52127988 2453368832 345513 270582939648 4194304 4964800 140735186988704 18446744073709551615 4342157 0 0 4096 18946 0 0 0 -1 16 0 0 0
[pid=22965/tid=22974] ppid=22963 vsize=2395868 CPUtime=0.9 cores=0-31
/proc/22965/task/22974/stat : 22974 (claspmt) S 22963 22965 22935 0 -1 4202560 10734 0 0 0 23 67 0 0 15 0 32 0 52127988 2453368832 345516 270582939648 4194304 4964800 140735186988704 18446744073709551615 258662772457 0 0 4096 18946 18446604567354514912 0 0 -1 3 0 0 0
[pid=22965/tid=22975] ppid=22963 vsize=2395868 CPUtime=1.01 cores=0-31
/proc/22965/task/22975/stat : 22975 (claspmt) S 22963 22965 22935 0 -1 4202560 10410 0 0 0 21 80 0 0 16 0 32 0 52127988 2453368832 345518 270582939648 4194304 4964800 140735186988704 18446744073709551615 258662772457 0 0 4096 18946 18446604488816696800 0 0 -1 27 0 0 0
[pid=22965/tid=22976] ppid=22963 vsize=2395868 CPUtime=1.2 cores=0-31
/proc/22965/task/22976/stat : 22976 (claspmt) R 22963 22965 22935 0 -1 4202560 12976 0 0 0 45 75 0 0 16 0 32 0 52127988 2453368832 345520 270582939648 4194304 4964800 140735186988704 18446744073709551615 4652172 0 0 4096 18946 0 0 0 -1 23 0 0 0
[pid=22965/tid=22977] ppid=22963 vsize=2395868 CPUtime=1.23 cores=0-31
/proc/22965/task/22977/stat : 22977 (claspmt) R 22963 22965 22935 0 -1 4202560 12638 0 0 0 40 83 0 0 16 0 32 0 52127988 2453368832 345521 270582939648 4194304 4964800 140735186988704 18446744073709551615 4652178 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=22965/tid=22978] ppid=22963 vsize=2395868 CPUtime=1 cores=0-31
/proc/22965/task/22978/stat : 22978 (claspmt) S 22963 22965 22935 0 -1 4202560 10743 0 0 0 20 80 0 0 15 0 32 0 52127988 2453368832 345523 270582939648 4194304 4964800 140735186988704 18446744073709551615 258662772457 0 0 4096 18946 18446604562910879200 0 0 -1 12 0 0 0
[pid=22965/tid=22979] ppid=22963 vsize=2395868 CPUtime=1.16 cores=0-31
/proc/22965/task/22979/stat : 22979 (claspmt) R 22963 22965 22935 0 -1 4202560 11944 0 0 0 32 84 0 0 16 0 32 0 52127988 2453368832 345525 270582939648 4194304 4964800 140735186988704 18446744073709551615 258650612746 0 0 4096 18946 0 0 0 -1 11 0 0 0
[pid=22965/tid=22980] ppid=22963 vsize=2395868 CPUtime=1.24 cores=0-31
/proc/22965/task/22980/stat : 22980 (claspmt) R 22963 22965 22935 0 -1 4202560 13264 0 0 0 39 85 0 0 16 0 32 0 52127988 2453368832 345530 270582939648 4194304 4964800 140735186988704 18446744073709551615 4375558 0 0 4096 18946 0 0 0 -1 30 0 0 0
[pid=22965/tid=22981] ppid=22963 vsize=2395868 CPUtime=1.31 cores=0-31
/proc/22965/task/22981/stat : 22981 (claspmt) R 22963 22965 22935 0 -1 4202560 12466 0 0 0 48 83 0 0 16 0 32 0 52127988 2453368832 345530 270582939648 4194304 4964800 140735186988704 18446744073709551615 4388685 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=22965/tid=22982] ppid=22963 vsize=2395868 CPUtime=0.98 cores=0-31
/proc/22965/task/22982/stat : 22982 (claspmt) S 22963 22965 22935 0 -1 4202560 10709 0 0 0 18 80 0 0 15 0 32 0 52127988 2453368832 345530 270582939648 4194304 4964800 140735186988704 18446744073709551615 258662772457 0 0 4096 18946 18446604567755832800 0 0 -1 8 0 0 0
[pid=22965/tid=22983] ppid=22963 vsize=2395868 CPUtime=1.03 cores=0-31
/proc/22965/task/22983/stat : 22983 (claspmt) S 22963 22965 22935 0 -1 4202560 10337 0 0 0 22 81 0 0 15 0 32 0 52127988 2453368832 345532 270582939648 4194304 4964800 140735186988704 18446744073709551615 258662772457 0 0 4096 18946 18446604559619907040 0 0 -1 1 0 0 0
[pid=22965/tid=22984] ppid=22963 vsize=2395868 CPUtime=1.26 cores=0-31
/proc/22965/task/22984/stat : 22984 (claspmt) R 22963 22965 22935 0 -1 4202560 12010 0 0 0 42 84 0 0 16 0 32 0 52127988 2453368832 345535 270582939648 4194304 4964800 140735186988704 18446744073709551615 4652162 0 0 4096 18946 0 0 0 -1 24 0 0 0
[pid=22965/tid=22985] ppid=22963 vsize=2395868 CPUtime=1.2 cores=0-31
/proc/22965/task/22985/stat : 22985 (claspmt) R 22963 22965 22935 0 -1 4202560 11646 0 0 0 37 83 0 0 16 0 32 0 52127988 2453368832 345537 270582939648 4194304 4964800 140735186988704 18446744073709551615 4375558 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=22965/tid=22986] ppid=22963 vsize=2395868 CPUtime=1.03 cores=0-31
/proc/22965/task/22986/stat : 22986 (claspmt) S 22963 22965 22935 0 -1 4202560 10589 0 0 0 21 82 0 0 15 0 32 0 52127988 2453368832 345539 270582939648 4194304 4964800 140735186988704 18446744073709551615 258662772457 0 0 4096 18946 18446604501372976608 0 0 -1 4 0 0 0
[pid=22965/tid=22987] ppid=22963 vsize=2395868 CPUtime=1.26 cores=0-31
/proc/22965/task/22987/stat : 22987 (claspmt) R 22963 22965 22935 0 -1 4202560 11384 0 0 0 46 80 0 0 16 0 32 0 52127988 2453368832 345540 270582939648 4194304 4964800 140735186988704 18446744073709551615 258650604460 0 0 4096 18946 0 0 0 -1 15 0 0 0
[pid=22965/tid=22988] ppid=22963 vsize=2395868 CPUtime=1.25 cores=0-31
/proc/22965/task/22988/stat : 22988 (claspmt) R 22963 22965 22935 0 -1 4202560 12921 0 0 0 42 83 0 0 16 0 32 0 52127988 2453368832 345542 270582939648 4194304 4964800 140735186988704 18446744073709551615 4375842 0 0 4096 18946 0 0 0 -1 20 0 0 0
[pid=22965/tid=22989] ppid=22963 vsize=2395868 CPUtime=1.15 cores=0-31
/proc/22965/task/22989/stat : 22989 (claspmt) R 22963 22965 22935 0 -1 4202560 12198 0 0 0 36 79 0 0 16 0 32 0 52127988 2453368832 345545 270582939648 4194304 4964800 140735186988704 18446744073709551615 4342186 0 0 4096 18946 0 0 0 -1 26 0 0 0
[pid=22965/tid=22990] ppid=22963 vsize=2395868 CPUtime=1 cores=0-31
/proc/22965/task/22990/stat : 22990 (claspmt) S 22963 22965 22935 0 -1 4202560 10695 0 0 0 21 79 0 0 16 0 32 0 52127988 2453368832 345546 270582939648 4194304 4964800 140735186988704 18446744073709551615 258662772457 0 0 4096 18946 18446604498597354976 0 0 -1 13 0 0 0
[pid=22965/tid=22991] ppid=22963 vsize=2395868 CPUtime=1.03 cores=0-31
/proc/22965/task/22991/stat : 22991 (claspmt) S 22963 22965 22935 0 -1 4202560 10479 0 0 0 22 81 0 0 15 0 32 0 52127988 2453368832 345547 270582939648 4194304 4964800 140735186988704 18446744073709551615 258662772457 0 0 4096 18946 18446604499161128416 0 0 -1 10 0 0 0

################
# More data... #
################

/proc/22965/task/22978/stat : 22978 (claspmt) R 22963 22965 22935 0 -1 4202560 216581 0 0 0 481402 626 0 0 25 0 32 0 52127988 70972444672 17083833 270582939648 4194304 4964800 140735186988704 18446744073709551615 4658540 0 0 4096 18946 0 0 0 -1 27 0 0 0
[pid=22965/tid=22979] ppid=22963 vsize=69309028 CPUtime=5076.2 cores=0-31
/proc/22965/task/22979/stat : 22979 (claspmt) R 22963 22965 22935 0 -1 4202560 13795 0 0 0 507533 87 0 0 25 0 32 0 52127988 70972444672 17083833 270582939648 4194304 4964800 140735186988704 18446744073709551615 4668987 0 0 4096 18946 0 0 0 -1 21 0 0 0
[pid=22965/tid=22980] ppid=22963 vsize=69309028 CPUtime=4819.62 cores=0-31
/proc/22965/task/22980/stat : 22980 (claspmt) R 22963 22965 22935 0 -1 4202560 176639 0 0 0 481433 529 0 0 25 0 32 0 52127988 70972444672 17083833 270582939648 4194304 4964800 140735186988704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 8 0 0 0
[pid=22965/tid=22981] ppid=22963 vsize=69309028 CPUtime=4823.31 cores=0-31
/proc/22965/task/22981/stat : 22981 (claspmt) R 22963 22965 22935 0 -1 4202560 2039976 0 0 0 479267 3064 0 0 25 0 32 0 52127988 70972444672 17083833 270582939648 4194304 4964800 140735186988704 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 18 0 0 0
[pid=22965/tid=22982] ppid=22963 vsize=69309028 CPUtime=4827.18 cores=0-31
/proc/22965/task/22982/stat : 22982 (claspmt) R 22963 22965 22935 0 -1 4202560 349794 0 0 0 481623 1095 0 0 25 0 32 0 52127988 70972444672 17083833 270582939648 4194304 4964800 140735186988704 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=22965/tid=22983] ppid=22963 vsize=69309028 CPUtime=4827.53 cores=0-31
/proc/22965/task/22983/stat : 22983 (claspmt) R 22963 22965 22935 0 -1 4202560 123857 0 0 0 482274 479 0 0 25 0 32 0 52127988 70972444672 17083833 270582939648 4194304 4964800 140735186988704 18446744073709551615 4669131 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=22965/tid=22984] ppid=22963 vsize=69309028 CPUtime=4820.54 cores=0-31
/proc/22965/task/22984/stat : 22984 (claspmt) R 22963 22965 22935 0 -1 4202560 991829 0 0 0 480174 1880 0 0 25 0 32 0 52127988 70972444672 17083833 270582939648 4194304 4964800 140735186988704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 26 0 0 0
[pid=22965/tid=22985] ppid=22963 vsize=69309028 CPUtime=4813.05 cores=0-31
/proc/22965/task/22985/stat : 22985 (claspmt) R 22963 22965 22935 0 -1 4202560 163716 0 0 0 480784 521 0 0 25 0 32 0 52127988 70972444672 17083833 270582939648 4194304 4964800 140735186988704 18446744073709551615 4669086 0 0 4096 18946 0 0 0 -1 13 0 0 0
[pid=22965/tid=22986] ppid=22963 vsize=69309028 CPUtime=4827.42 cores=0-31
/proc/22965/task/22986/stat : 22986 (claspmt) R 22963 22965 22935 0 -1 4202560 308133 0 0 0 481942 800 0 0 25 0 32 0 52127988 70972444672 17083833 270582939648 4194304 4964800 140735186988704 18446744073709551615 4660495 0 0 4096 18946 0 0 0 -1 16 0 0 0
[pid=22965/tid=22987] ppid=22963 vsize=69309028 CPUtime=5074.61 cores=0-31
/proc/22965/task/22987/stat : 22987 (claspmt) R 22963 22965 22935 0 -1 4202560 13669 0 0 0 507380 81 0 0 25 0 32 0 52127988 70972444672 17083833 270582939648 4194304 4964800 140735186988704 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 15 0 0 0
[pid=22965/tid=22988] ppid=22963 vsize=69309028 CPUtime=4827.97 cores=0-31
/proc/22965/task/22988/stat : 22988 (claspmt) R 22963 22965 22935 0 -1 4202560 180744 0 0 0 482250 547 0 0 25 0 32 0 52127988 70972444672 17083839 270582939648 4194304 4964800 140735186988704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 12 0 0 0
[pid=22965/tid=22989] ppid=22963 vsize=69309028 CPUtime=4825.64 cores=0-31
/proc/22965/task/22989/stat : 22989 (claspmt) R 22963 22965 22935 0 -1 4202560 2054339 0 0 0 479440 3124 0 0 25 0 32 0 52127988 70972444672 17083839 270582939648 4194304 4964800 140735186988704 18446744073709551615 4653207 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=22965/tid=22990] ppid=22963 vsize=69309028 CPUtime=4818.26 cores=0-31
/proc/22965/task/22990/stat : 22990 (claspmt) R 22963 22965 22935 0 -1 4202560 366632 0 0 0 480661 1165 0 0 25 0 32 0 52127988 70972444672 17083839 270582939648 4194304 4964800 140735186988704 18446744073709551615 4669087 0 0 4096 18946 0 0 0 -1 31 0 0 0
[pid=22965/tid=22991] ppid=22963 vsize=69309028 CPUtime=4827.66 cores=0-31
/proc/22965/task/22991/stat : 22991 (claspmt) R 22963 22965 22935 0 -1 4202560 139411 0 0 0 482281 485 0 0 25 0 32 0 52127988 70972444672 17083839 270582939648 4194304 4964800 140735186988704 18446744073709551615 4669509 0 0 4096 18946 0 0 0 -1 10 0 0 0
[pid=22965/tid=22992] ppid=22963 vsize=69309028 CPUtime=4820.26 cores=0-31
/proc/22965/task/22992/stat : 22992 (claspmt) R 22963 22965 22935 0 -1 4202560 1231885 0 0 0 479972 2054 0 0 25 0 32 0 52127989 70972444672 17083839 270582939648 4194304 4964800 140735186988704 18446744073709551615 258650653116 0 0 4096 18946 0 0 0 -1 22 0 0 0
[pid=22965/tid=22993] ppid=22963 vsize=69309028 CPUtime=4820.34 cores=0-31
/proc/22965/task/22993/stat : 22993 (claspmt) R 22963 22965 22935 0 -1 4202560 159858 0 0 0 481501 533 0 0 25 0 32 0 52127989 70972444672 17083839 270582939648 4194304 4964800 140735186988704 18446744073709551615 4659391 0 0 4096 18946 0 0 0 -1 23 0 0 0
[pid=22965/tid=22994] ppid=22963 vsize=69309028 CPUtime=4825.33 cores=0-31
/proc/22965/task/22994/stat : 22994 (claspmt) R 22963 22965 22935 0 -1 4202560 208681 0 0 0 481947 586 0 0 25 0 32 0 52127989 70972444672 17083839 270582939648 4194304 4964800 140735186988704 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=22965/tid=22995] ppid=22963 vsize=69309028 CPUtime=5072.17 cores=0-31
/proc/22965/task/22995/stat : 22995 (claspmt) R 22963 22965 22935 0 -1 4202560 13405 0 0 0 507132 85 0 0 25 0 32 0 52127989 70972444672 17083839 270582939648 4194304 4964800 140735186988704 18446744073709551615 4669140 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=22965/tid=22996] ppid=22963 vsize=69309028 CPUtime=4828.1 cores=0-31
/proc/22965/task/22996/stat : 22996 (claspmt) R 22963 22965 22935 0 -1 4202560 172939 0 0 0 482239 571 0 0 25 0 32 0 52127989 70972444672 17083839 270582939648 4194304 4964800 140735186988704 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 29 0 0 0
[pid=22965/tid=22997] ppid=22963 vsize=69309028 CPUtime=4785.45 cores=0-31
/proc/22965/task/22997/stat : 22997 (claspmt) R 22963 22965 22935 0 -1 4202560 1996527 0 0 0 475350 3195 0 0 25 0 32 0 52127989 70972444672 17083839 270582939648 4194304 4964800 140735186988704 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 155316
Current children cumulated vsize (KiB) 69309024



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5100 s]
/proc/loadavg: 32.08 32.10 32.03 33/445 23148
/proc/meminfo: memFree=193124664/264257624 swapFree=67111528/67111528
[pid=22965] ppid=22963 vsize=69505636 CPUtime=155882 cores=0-31
/proc/22965/stat : 22965 (claspmt) R 22963 22965 22935 0 -1 4202496 17205037 0 0 0 15553552 34651 0 0 25 0 32 0 52127785 71173771264 17111158 270582939648 4194304 4964800 140735186988704 18446744073709551615 4669011 0 0 4096 18946 18446744073709551615 0 0 17 11 0 0 0
/proc/22965/statm: 17376409 17111159 457 189 0 17370758 0
[pid=22965/tid=22967] ppid=22963 vsize=69505636 CPUtime=4845.23 cores=0-31
/proc/22965/task/22967/stat : 22967 (claspmt) R 22963 22965 22935 0 -1 4202560 174210 0 0 0 483935 588 0 0 25 0 32 0 52127988 71173771264 17111159 270582939648 4194304 4964800 140735186988704 18446744073709551615 4321023 0 0 4096 18946 0 0 0 -1 17 0 0 0
[pid=22965/tid=22968] ppid=22963 vsize=69505636 CPUtime=4841.16 cores=0-31
/proc/22965/task/22968/stat : 22968 (claspmt) R 22963 22965 22935 0 -1 4202560 916714 0 0 0 482385 1731 0 0 25 0 32 0 52127988 71173771264 17111159 270582939648 4194304 4964800 140735186988704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 9 0 0 0
[pid=22965/tid=22969] ppid=22963 vsize=69505636 CPUtime=4845.73 cores=0-31
/proc/22965/task/22969/stat : 22969 (claspmt) R 22963 22965 22935 0 -1 4202560 188264 0 0 0 484004 569 0 0 25 0 32 0 52127988 71173771264 17111159 270582939648 4194304 4964800 140735186988704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=22965/tid=22970] ppid=22963 vsize=69505636 CPUtime=4845.21 cores=0-31
/proc/22965/task/22970/stat : 22970 (claspmt) R 22963 22965 22935 0 -1 4202560 312091 0 0 0 483714 807 0 0 25 0 32 0 52127988 71173771264 17111159 270582939648 4194304 4964800 140735186988704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 14 0 0 0
[pid=22965/tid=22971] ppid=22963 vsize=69505636 CPUtime=5089.87 cores=0-31
/proc/22965/task/22971/stat : 22971 (claspmt) R 22963 22965 22935 0 -1 4202560 15294 0 0 0 508918 69 0 0 25 0 32 0 52127988 71173771264 17111159 270582939648 4194304 4964800 140735186988704 18446744073709551615 4668900 0 0 4096 18946 0 0 0 -1 19 0 0 0
[pid=22965/tid=22972] ppid=22963 vsize=69505636 CPUtime=4843.37 cores=0-31
/proc/22965/task/22972/stat : 22972 (claspmt) R 22963 22965 22935 0 -1 4202560 214300 0 0 0 483641 696 0 0 25 0 32 0 52127988 71173771264 17111159 270582939648 4194304 4964800 140735186988704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 24 0 0 0
[pid=22965/tid=22973] ppid=22963 vsize=69505636 CPUtime=4841.8 cores=0-31
/proc/22965/task/22973/stat : 22973 (claspmt) R 22963 22965 22935 0 -1 4202560 2203567 0 0 0 480693 3487 0 0 25 0 32 0 52127988 71173771264 17111159 270582939648 4194304 4964800 140735186988704 18446744073709551615 4668984 0 0 4096 18946 0 0 0 -1 28 0 0 0
[pid=22965/tid=22974] ppid=22963 vsize=69505636 CPUtime=4839.07 cores=0-31
/proc/22965/task/22974/stat : 22974 (claspmt) R 22963 22965 22935 0 -1 4202560 281084 0 0 0 483020 887 0 0 25 0 32 0 52127988 71173771264 17111159 270582939648 4194304 4964800 140735186988704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 20 0 0 0
[pid=22965/tid=22975] ppid=22963 vsize=69505636 CPUtime=4841.92 cores=0-31
/proc/22965/task/22975/stat : 22975 (claspmt) R 22963 22965 22935 0 -1 4202560 127916 0 0 0 483715 477 0 0 25 0 32 0 52127988 71173771264 17111159 270582939648 4194304 4964800 140735186988704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 25 0 0 0
[pid=22965/tid=22976] ppid=22963 vsize=69505636 CPUtime=4841.63 cores=0-31
/proc/22965/task/22976/stat : 22976 (claspmt) R 22963 22965 22935 0 -1 4202560 1269727 0 0 0 482000 2163 0 0 20 0 32 0 52127988 71173771264 17111159 270582939648 4194304 4964800 140735186988704 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 30 0 0 0
[pid=22965/tid=22977] ppid=22963 vsize=69505636 CPUtime=4845.69 cores=0-31
/proc/22965/task/22977/stat : 22977 (claspmt) R 22963 22965 22935 0 -1 4202560 183749 0 0 0 484001 568 0 0 25 0 32 0 52127988 71173771264 17111159 270582939648 4194304 4964800 140735186988704 18446744073709551615 4669011 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=22965/tid=22978] ppid=22963 vsize=69505636 CPUtime=4838 cores=0-31
/proc/22965/task/22978/stat : 22978 (claspmt) R 22963 22965 22935 0 -1 4202560 216596 0 0 0 483174 626 0 0 25 0 32 0 52127988 71173771264 17111160 270582939648 4194304 4964800 140735186988704 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 27 0 0 0
[pid=22965/tid=22979] ppid=22963 vsize=69505636 CPUtime=5093.92 cores=0-31
/proc/22965/task/22979/stat : 22979 (claspmt) R 22963 22965 22935 0 -1 4202560 13795 0 0 0 509305 87 0 0 25 0 32 0 52127988 71173771264 17111160 270582939648 4194304 4964800 140735186988704 18446744073709551615 4399292 0 0 4096 18946 0 0 0 -1 21 0 0 0
[pid=22965/tid=22980] ppid=22963 vsize=69505636 CPUtime=4837.34 cores=0-31
/proc/22965/task/22980/stat : 22980 (claspmt) R 22963 22965 22935 0 -1 4202560 176659 0 0 0 483205 529 0 0 25 0 32 0 52127988 71173771264 17111160 270582939648 4194304 4964800 140735186988704 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 8 0 0 0
[pid=22965/tid=22981] ppid=22963 vsize=69505636 CPUtime=4841.03 cores=0-31
/proc/22965/task/22981/stat : 22981 (claspmt) R 22963 22965 22935 0 -1 4202560 2039976 0 0 0 481039 3064 0 0 25 0 32 0 52127988 71173771264 17111160 270582939648 4194304 4964800 140735186988704 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 18 0 0 0
[pid=22965/tid=22982] ppid=22963 vsize=69505636 CPUtime=4844.89 cores=0-31
/proc/22965/task/22982/stat : 22982 (claspmt) R 22963 22965 22935 0 -1 4202560 352267 0 0 0 483385 1104 0 0 25 0 32 0 52127988 71173771264 17111160 270582939648 4194304 4964800 140735186988704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=22965/tid=22983] ppid=22963 vsize=69505636 CPUtime=4845.25 cores=0-31
/proc/22965/task/22983/stat : 22983 (claspmt) R 22963 22965 22935 0 -1 4202560 123939 0 0 0 484046 479 0 0 25 0 32 0 52127988 71173771264 17111160 270582939648 4194304 4964800 140735186988704 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=22965/tid=22984] ppid=22963 vsize=69505636 CPUtime=4838.26 cores=0-31
/proc/22965/task/22984/stat : 22984 (claspmt) R 22963 22965 22935 0 -1 4202560 991836 0 0 0 481946 1880 0 0 25 0 32 0 52127988 71173771264 17111160 270582939648 4194304 4964800 140735186988704 18446744073709551615 4319903 0 0 4096 18946 0 0 0 -1 26 0 0 0
[pid=22965/tid=22985] ppid=22963 vsize=69505636 CPUtime=4830.77 cores=0-31
/proc/22965/task/22985/stat : 22985 (claspmt) R 22963 22965 22935 0 -1 4202560 163724 0 0 0 482556 521 0 0 25 0 32 0 52127988 71173771264 17111160 270582939648 4194304 4964800 140735186988704 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 13 0 0 0
[pid=22965/tid=22986] ppid=22963 vsize=69505636 CPUtime=4845.14 cores=0-31
/proc/22965/task/22986/stat : 22986 (claspmt) R 22963 22965 22935 0 -1 4202560 308145 0 0 0 483714 800 0 0 25 0 32 0 52127988 71173771264 17111160 270582939648 4194304 4964800 140735186988704 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 16 0 0 0
[pid=22965/tid=22987] ppid=22963 vsize=69505636 CPUtime=5092.33 cores=0-31
/proc/22965/task/22987/stat : 22987 (claspmt) R 22963 22965 22935 0 -1 4202560 13669 0 0 0 509152 81 0 0 25 0 32 0 52127988 71173771264 17111160 270582939648 4194304 4964800 140735186988704 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 15 0 0 0
[pid=22965/tid=22988] ppid=22963 vsize=69505636 CPUtime=4845.68 cores=0-31
/proc/22965/task/22988/stat : 22988 (claspmt) R 22963 22965 22935 0 -1 4202560 180751 0 0 0 484021 547 0 0 25 0 32 0 52127988 71173771264 17111160 270582939648 4194304 4964800 140735186988704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 12 0 0 0
[pid=22965/tid=22989] ppid=22963 vsize=69505636 CPUtime=4843.36 cores=0-31
/proc/22965/task/22989/stat : 22989 (claspmt) R 22963 22965 22935 0 -1 4202560 2054340 0 0 0 481212 3124 0 0 25 0 32 0 52127988 71173771264 17111160 270582939648 4194304 4964800 140735186988704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=22965/tid=22990] ppid=22963 vsize=69505636 CPUtime=4835.45 cores=0-31
/proc/22965/task/22990/stat : 22990 (claspmt) R 22963 22965 22935 0 -1 4202560 366632 0 0 0 482380 1165 0 0 25 0 32 0 52127988 71173771264 17111160 270582939648 4194304 4964800 140735186988704 18446744073709551615 4370381 0 0 4096 18946 0 0 0 -1 31 0 0 0
[pid=22965/tid=22991] ppid=22963 vsize=69505636 CPUtime=4845.38 cores=0-31
/proc/22965/task/22991/stat : 22991 (claspmt) R 22963 22965 22935 0 -1 4202560 139411 0 0 0 484053 485 0 0 25 0 32 0 52127988 71173771264 17111160 270582939648 4194304 4964800 140735186988704 18446744073709551615 4669011 0 0 4096 18946 0 0 0 -1 10 0 0 0
[pid=22965/tid=22992] ppid=22963 vsize=69505636 CPUtime=4837.97 cores=0-31
/proc/22965/task/22992/stat : 22992 (claspmt) R 22963 22965 22935 0 -1 4202560 1236732 0 0 0 481736 2061 0 0 25 0 32 0 52127989 71173771264 17111160 270582939648 4194304 4964800 140735186988704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 22 0 0 0
[pid=22965/tid=22993] ppid=22963 vsize=69505636 CPUtime=4838.06 cores=0-31
/proc/22965/task/22993/stat : 22993 (claspmt) R 22963 22965 22935 0 -1 4202560 159865 0 0 0 483273 533 0 0 25 0 32 0 52127989 71173771264 17111160 270582939648 4194304 4964800 140735186988704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 23 0 0 0
[pid=22965/tid=22994] ppid=22963 vsize=69505636 CPUtime=4843.04 cores=0-31
/proc/22965/task/22994/stat : 22994 (claspmt) R 22963 22965 22935 0 -1 4202560 211395 0 0 0 483711 593 0 0 25 0 32 0 52127989 71173771264 17111160 270582939648 4194304 4964800 140735186988704 18446744073709551615 4670534 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=22965/tid=22995] ppid=22963 vsize=69505636 CPUtime=5089.88 cores=0-31
/proc/22965/task/22995/stat : 22995 (claspmt) R 22963 22965 22935 0 -1 4202560 13405 0 0 0 508903 85 0 0 25 0 32 0 52127989 71173771264 17111160 270582939648 4194304 4964800 140735186988704 18446744073709551615 4669574 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=22965/tid=22996] ppid=22963 vsize=69505636 CPUtime=4845.82 cores=0-31
/proc/22965/task/22996/stat : 22996 (claspmt) R 22963 22965 22935 0 -1 4202560 172978 0 0 0 484010 572 0 0 25 0 32 0 52127989 71173771264 17111160 270582939648 4194304 4964800 140735186988704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 29 0 0 0
[pid=22965/tid=22997] ppid=22963 vsize=69505636 CPUtime=4803.17 cores=0-31
/proc/22965/task/22997/stat : 22997 (claspmt) R 22963 22965 22935 0 -1 4202560 2009652 0 0 0 477110 3207 0 0 25 0 32 0 52127989 71173771264 17111160 270582939648 4194304 4964800 140735186988704 18446744073709551615 4669140 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 155882
Current children cumulated vsize (KiB) 69505632

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)

Sending SIGKILL to -22965

Solver just ended. Dumping a history of the last processes samples

[startup+5100.7 s]
/proc/loadavg: 29.59 31.58 31.86 2/415 23149
/proc/meminfo: memFree=193210340/264257624 swapFree=67111528/67111528
[pid=22965] ppid=22963 vsize=69314932 CPUtime=155883 cores=0-31
/proc/22965/stat : 22965 (claspmt) R 22963 22965 22935 0 -1 4202496 17205110 0 0 0 15553662 34652 0 0 25 0 1 0 52127785 70978490368 17089323 270582939648 4194304 4964800 140735186988704 18446744073709551615 258650604632 0 0 4096 18946 0 0 0 17 11 0 0 0
/proc/22965/statm: 17328733 17089323 468 189 0 17323082 0
Current children cumulated CPU time (s) 155883
Current children cumulated vsize (KiB) 69314928

[startup+5102.3 s]
/proc/loadavg: 29.59 31.58 31.86 2/415 23149
/proc/meminfo: memFree=193238116/264257624 swapFree=67111528/67111528
[pid=22965] ppid=22963 vsize=0 CPUtime=155885 cores=0-31
/proc/22965/stat : 22965 (claspmt) R 22963 22965 22935 0 -1 4203524 17205110 0 0 0 15553791 34679 0 0 25 0 1 0 52127785 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 11 0 0 0
/proc/22965/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 155885
Current children cumulated vsize (KiB) 0

[startup+5103.9 s]
/proc/loadavg: 29.59 31.58 31.86 2/414 23149
/proc/meminfo: memFree=208684588/264257624 swapFree=67111528/67111528
[pid=22965] ppid=22963 vsize=0 CPUtime=155886 cores=0-31
/proc/22965/stat : 22965 (claspmt) R 22963 22965 22935 0 -1 4203524 17205110 0 0 0 15553791 34839 0 0 25 0 1 0 52127785 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 11 0 0 0
/proc/22965/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 155886
Current children cumulated vsize (KiB) 0

[startup+5104.7 s]
/proc/loadavg: 29.59 31.58 31.86 2/414 23149
/proc/meminfo: memFree=233357364/264257624 swapFree=67111528/67111528
[pid=22965] ppid=22963 vsize=0 CPUtime=155887 cores=0-31
/proc/22965/stat : 22965 (claspmt) R 22963 22965 22935 0 -1 4203524 17205110 0 0 0 15553791 34919 0 0 25 0 1 0 52127785 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 11 0 0 0
/proc/22965/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 155887
Current children cumulated vsize (KiB) 0

[startup+5105.1 s]
/proc/loadavg: 29.59 31.58 31.86 2/414 23149
/proc/meminfo: memFree=259746912/264257624 swapFree=67111528/67111528
[pid=22965] ppid=22963 vsize=0 CPUtime=155888 cores=0-31
/proc/22965/stat : 22965 (claspmt) R 22963 22965 22935 0 -1 4203524 17205110 0 0 0 15553791 34961 0 0 25 0 1 0 52127785 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 11 0 0 0
/proc/22965/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 155888
Current children cumulated vsize (KiB) 0

Child ended because it received signal 9 (SIGKILL)
Real time (s): 5105.22
CPU time (s): 155888
CPU user time (s): 155538
CPU system time (s): 349.709
CPU usage (%): 3053.5
Max. virtual memory (cumulated for all children) (KiB): 69505636

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 155538
system time used= 349.709
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 17205110
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1561858
involuntary context switches= 336289

runsolver used 26.577 second user time and 117.353 second system time

The end

Launcher Data 

Begin job on node200 at 2011-06-12 10:30:09
IDJOB=3472552
IDBENCH=88165
IDSOLVER=1961
FILE ID=node200/3472552-1307867405
RUNJOBID= node200-1307867405-22949
PBS_JOBID= 13551116
Free space on /tmp= 61652 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT-Race-2010-CNF/hardware-verification/velev/2dlx_ca_bp_f_liveness.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3472552-1307867405/watcher-3472552-1307867405 -o /tmp/evaluation-result-3472552-1307867405/solver-3472552-1307867405 -C 160000 -W 5100 -M 256000  HOME/claspmt -t32 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3472552-1307867405.cnf

TIME LIMIT= 160000 seconds
MEMORY LIMIT= 256000 MiB
NBCORE= 32

MD5SUM BENCH= 99cd8de7f06ce8c087a5c7fb299dfb62
RANDOM SEED=1807200035

node200.alineos.net Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 64
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 96
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 66
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 98
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.99
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 68
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 100
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.03
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 70
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 102
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 16
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 17
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 48
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 18
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 80
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 19
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 112
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.97
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 20
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 21
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 50
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.10
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 22
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 82
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.10
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 114
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 24
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 25
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 26
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 84
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 27
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 116
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 28
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 29
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 54
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 30
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 86
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 31
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 118
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     264257624 kB
MemFree:      261690368 kB
Buffers:        304336 kB
Cached:        1928016 kB
SwapCached:          0 kB
Active:         448080 kB
Inactive:      1847820 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     264257624 kB
LowFree:      261690368 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           93676 kB
Writeback:           0 kB
AnonPages:       63764 kB
Mapped:          14480 kB
Slab:           186824 kB
PageTables:       4452 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  199240340 kB
Committed_AS:   163792 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296372 kB
VmallocChunk: 34359440603 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 61648 MiB
End job on node200 at 2011-06-12 11:55:14