Trace number 3373256

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 4800.33 1205.18

General information on the benchmark

NameCRAFTED/rbsat/crafted/unforced/
rbsat-v2400c266431g9.cnf
MD5SUM35278b3ce28129761cf8ae7994f37d3f
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2400
Number of clauses266431
Sum of the clauses size535112
Maximum clause length32
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2266356
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 575

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...419772/instance-3373256-1305419772.cnf
1.39/1.48	c Solving...
4800.12/1205.01	
4800.12/1205.01	*** INTERRUPTED! ***
4800.12/1205.01	*** Shutting down threads...
4800.26/1205.17	s UNKNOWN
4800.26/1205.17	
4800.26/1205.17	c INTERRUPTED   : 1
4800.26/1205.17	c Models        : 0+    
4800.26/1205.17	c Time          : 1205.017s (Solving: 1203.53s 1st Model: 0.00s Unsat: 0.00s)
4800.26/1205.17	c CPU Time      : 4800.160s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3373256-1305419772/watcher-3373256-1305419772 -o /tmp/evaluation-result-3373256-1305419772/solver-3373256-1305419772 -C 4800 -W 1300 -M 15500 HOME/claspmt -t4 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3373256-1305419772.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.17 8.10 7.79 5/367 16952
/proc/meminfo: memFree=23272664/32950928 swapFree=65730864/67111528
[pid=16952] ppid=16950 vsize=21964 CPUtime=0 cores=0,2,4,6
/proc/16952/stat : 16952 (claspmt) R 16950 16952 16432 0 -1 4202496 791 0 0 0 0 0 0 0 25 0 1 0 732207841 22491136 703 33554432000 4194304 4964800 140736374656064 18446744073709551615 4599599 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/16952/statm: 5491 703 385 189 0 365 0

[startup+0.073539 s]
/proc/loadavg: 8.17 8.10 7.79 5/367 16952
/proc/meminfo: memFree=23272664/32950928 swapFree=65730864/67111528
[pid=16952] ppid=16950 vsize=34756 CPUtime=0.06 cores=0,2,4,6
/proc/16952/stat : 16952 (claspmt) R 16950 16952 16432 0 -1 4202496 5052 0 0 0 6 0 0 0 25 0 1 0 732207841 35590144 3791 33554432000 4194304 4964800 140736374656064 18446744073709551615 4618467 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/16952/statm: 8689 3791 396 189 0 3563 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 34756

[startup+0.100534 s]
/proc/loadavg: 8.17 8.10 7.79 5/367 16952
/proc/meminfo: memFree=23272664/32950928 swapFree=65730864/67111528
[pid=16952] ppid=16950 vsize=37900 CPUtime=0.09 cores=0,2,4,6
/proc/16952/stat : 16952 (claspmt) R 16950 16952 16432 0 -1 4202496 5841 0 0 0 9 0 0 0 25 0 1 0 732207841 38809600 4580 33554432000 4194304 4964800 140736374656064 18446744073709551615 4607585 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/16952/statm: 9475 4580 398 189 0 4349 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 37900

[startup+0.3005 s]
/proc/loadavg: 8.17 8.10 7.79 5/367 16952
/proc/meminfo: memFree=23272664/32950928 swapFree=65730864/67111528
[pid=16952] ppid=16950 vsize=37900 CPUtime=0.29 cores=0,2,4,6
/proc/16952/stat : 16952 (claspmt) R 16950 16952 16432 0 -1 4202496 5841 0 0 0 29 0 0 0 25 0 1 0 732207841 38809600 4580 33554432000 4194304 4964800 140736374656064 18446744073709551615 4607590 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/16952/statm: 9475 4580 398 189 0 4349 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 37900

[startup+0.700425 s]
/proc/loadavg: 8.17 8.10 7.79 5/367 16952
/proc/meminfo: memFree=23272664/32950928 swapFree=65730864/67111528
[pid=16952] ppid=16950 vsize=37900 CPUtime=0.69 cores=0,2,4,6
/proc/16952/stat : 16952 (claspmt) R 16950 16952 16432 0 -1 4202496 5842 0 0 0 69 0 0 0 25 0 1 0 732207841 38809600 4581 33554432000 4194304 4964800 140736374656064 18446744073709551615 4600392 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/16952/statm: 9475 4581 399 189 0 4349 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 37900

[startup+1.50028 s]
/proc/loadavg: 8.17 8.10 7.79 6/368 16953
/proc/meminfo: memFree=23241596/32950928 swapFree=65730864/67111528
[pid=16952] ppid=16950 vsize=186372 CPUtime=1.5 cores=0,2,4,6
/proc/16952/stat : 16952 (claspmt) R 16950 16952 16432 0 -1 4202496 7314 0 0 0 149 1 0 0 25 0 4 0 732207841 190844928 5522 33554432000 4194304 4964800 140736374656064 18446744073709551615 4652195 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/16952/statm: 46593 5522 442 189 0 40942 0
Current children cumulated CPU time (s) 1.5
Current children cumulated vsize (KiB) 186372

[startup+3.10099 s]
/proc/loadavg: 8.23 8.11 7.80 9/371 16956
/proc/meminfo: memFree=23225452/32950928 swapFree=65730864/67111528
[pid=16952] ppid=16950 vsize=186504 CPUtime=7.81 cores=0,2,4,6
/proc/16952/stat : 16952 (claspmt) R 16950 16952 16432 0 -1 4202496 12352 0 0 0 778 3 0 0 25 0 4 0 732207841 190980096 10560 33554432000 4194304 4964800 140736374656064 18446744073709551615 4362231 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/16952/statm: 46626 10560 458 189 0 40975 0
[pid=16952/tid=16954] ppid=16950 vsize=186504 CPUtime=1.58 cores=0,2,4,6
/proc/16952/task/16954/stat : 16954 (claspmt) R 16950 16952 16432 0 -1 4202560 1096 0 0 0 158 0 0 0 25 0 4 0 732207989 190980096 10560 33554432000 4194304 4964800 140736374656064 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=16952/tid=16955] ppid=16950 vsize=186504 CPUtime=1.53 cores=0,2,4,6
/proc/16952/task/16955/stat : 16955 (claspmt) R 16950 16952 16432 0 -1 4202560 1598 0 0 0 153 0 0 0 25 0 4 0 732207989 190980096 10560 33554432000 4194304 4964800 140736374656064 18446744073709551615 4627248 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=16952/tid=16956] ppid=16950 vsize=186504 CPUtime=1.59 cores=0,2,4,6
/proc/16952/task/16956/stat : 16956 (claspmt) R 16950 16952 16432 0 -1 4202560 1441 0 0 0 159 0 0 0 25 0 4 0 732207989 190980096 10560 33554432000 4194304 4964800 140736374656064 18446744073709551615 4627290 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 7.81
Current children cumulated vsize (KiB) 186504

[startup+6.3004 s]
/proc/loadavg: 8.23 8.11 7.80 9/371 16956
/proc/meminfo: memFree=23183912/32950928 swapFree=65730864/67111528
[pid=16952] ppid=16950 vsize=195964 CPUtime=20.57 cores=0,2,4,6
/proc/16952/stat : 16952 (claspmt) R 16950 16952 16432 0 -1 4202496 19517 0 0 0 2052 5 0 0 25 0 4 0 732207841 200667136 17725 33554432000 4194304 4964800 140736374656064 18446744073709551615 260905437367 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/16952/statm: 48991 17726 460 189 0 43340 0
[pid=16952/tid=16954] ppid=16950 vsize=195964 CPUtime=4.74 cores=0,2,4,6
/proc/16952/task/16954/stat : 16954 (claspmt) R 16950 16952 16432 0 -1 4202560 2926 0 0 0 473 1 0 0 25 0 4 0 732207989 200667136 17726 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670273 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=16952/tid=16955] ppid=16950 vsize=195964 CPUtime=4.74 cores=0,2,4,6
/proc/16952/task/16955/stat : 16955 (claspmt) R 16950 16952 16432 0 -1 4202560 3656 0 0 0 473 1 0 0 25 0 4 0 732207989 200667136 17726 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=16952/tid=16956] ppid=16950 vsize=195964 CPUtime=4.79 cores=0,2,4,6
/proc/16952/task/16956/stat : 16956 (claspmt) R 16950 16952 16432 0 -1 4202560 2971 0 0 0 479 0 0 0 25 0 4 0 732207989 200667136 17726 33554432000 4194304 4964800 140736374656064 18446744073709551615 4653191 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 20.57
Current children cumulated vsize (KiB) 195964

[startup+12.7002 s]
/proc/loadavg: 8.20 8.11 7.80 9/371 16956
/proc/meminfo: memFree=23117120/32950928 swapFree=65730864/67111528
[pid=16952] ppid=16950 vsize=212904 CPUtime=46.08 cores=0,2,4,6
/proc/16952/stat : 16952 (claspmt) R 16950 16952 16432 0 -1 4202496 31710 0 0 0 4597 11 0 0 25 0 4 0 732207841 218013696 29918 33554432000 4194304 4964800 140736374656064 18446744073709551615 4627328 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/16952/statm: 53226 29918 460 189 0 47575 0
[pid=16952/tid=16954] ppid=16950 vsize=212904 CPUtime=11.06 cores=0,2,4,6
/proc/16952/task/16954/stat : 16954 (claspmt) R 16950 16952 16432 0 -1 4202560 6652 0 0 0 1103 3 0 0 25 0 4 0 732207989 218013696 29918 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670419 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=16952/tid=16955] ppid=16950 vsize=212904 CPUtime=11.13 cores=0,2,4,6
/proc/16952/task/16955/stat : 16955 (claspmt) R 16950 16952 16432 0 -1 4202560 6785 0 0 0 1111 2 0 0 25 0 4 0 732207989 218013696 29918 33554432000 4194304 4964800 140736374656064 18446744073709551615 4627272 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=16952/tid=16956] ppid=16950 vsize=212904 CPUtime=11.18 cores=0,2,4,6
/proc/16952/task/16956/stat : 16956 (claspmt) R 16950 16952 16432 0 -1 4202560 5817 0 0 0 1117 1 0 0 25 0 4 0 732207989 218013696 29918 33554432000 4194304 4964800 140736374656064 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 46.08
Current children cumulated vsize (KiB) 212904

[startup+25.5009 s]
/proc/loadavg: 8.17 8.11 7.80 9/371 16956
/proc/meminfo: memFree=23065680/32950928 swapFree=65730864/67111528
[pid=16952] ppid=16950 vsize=225884 CPUtime=97.12 cores=0,2,4,6
/proc/16952/stat : 16952 (claspmt) R 16950 16952 16432 0 -1 4202496 40073 0 0 0 9696 16 0 0 25 0 4 0 732207841 231305216 38281 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670403 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/16952/statm: 56471 38281 460 189 0 50820 0
[pid=16952/tid=16954] ppid=16950 vsize=225884 CPUtime=23.7 cores=0,2,4,6
/proc/16952/task/16954/stat : 16954 (claspmt) R 16950 16952 16432 0 -1 4202560 9317 0 0 0 2366 4 0 0 25 0 4 0 732207989 231305216 38281 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670538 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=16952/tid=16955] ppid=16950 vsize=225884 CPUtime=23.93 cores=0,2,4,6
/proc/16952/task/16955/stat : 16955 (claspmt) R 16950 16952 16432 0 -1 4202560 8857 0 0 0 2390 3 0 0 25 0 4 0 732207989 231305216 38281 33554432000 4194304 4964800 140736374656064 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=16952/tid=16956] ppid=16950 vsize=225884 CPUtime=23.98 cores=0,2,4,6
/proc/16952/task/16956/stat : 16956 (claspmt) R 16950 16952 16432 0 -1 4202560 7402 0 0 0 2396 2 0 0 25 0 4 0 732207989 231305216 38281 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 97.12
Current children cumulated vsize (KiB) 225884

[startup+51.1002 s]
/proc/loadavg: 8.11 8.10 7.81 9/371 16957
/proc/meminfo: memFree=22999692/32950928 swapFree=65730864/67111528
[pid=16952] ppid=16950 vsize=295872 CPUtime=199.21 cores=0,2,4,6
/proc/16952/stat : 16952 (claspmt) R 16950 16952 16432 0 -1 4202496 49626 0 0 0 19900 21 0 0 25 0 4 0 732207841 302972928 47822 33554432000 4194304 4964800 140736374656064 18446744073709551615 4658574 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/16952/statm: 73968 47822 460 189 0 68317 0
[pid=16952/tid=16954] ppid=16950 vsize=295872 CPUtime=48.99 cores=0,2,4,6
/proc/16952/task/16954/stat : 16954 (claspmt) R 16950 16952 16432 0 -1 4202560 12673 0 0 0 4893 6 0 0 25 0 4 0 732207989 302972928 47822 33554432000 4194304 4964800 140736374656064 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=16952/tid=16955] ppid=16950 vsize=295872 CPUtime=49.54 cores=0,2,4,6
/proc/16952/task/16955/stat : 16955 (claspmt) R 16950 16952 16432 0 -1 4202560 12131 0 0 0 4949 5 0 0 25 0 4 0 732207989 302972928 47822 33554432000 4194304 4964800 140736374656064 18446744073709551615 4320966 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=16952/tid=16956] ppid=16950 vsize=295872 CPUtime=49.57 cores=0,2,4,6
/proc/16952/task/16956/stat : 16956 (claspmt) R 16950 16952 16432 0 -1 4202560 8570 0 0 0 4954 3 0 0 25 0 4 0 732207989 302972928 47822 33554432000 4194304 4964800 140736374656064 18446744073709551615 4668984 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 199.21
Current children cumulated vsize (KiB) 295872

[startup+102.314 s]
/proc/loadavg: 8.18 8.12 7.83 9/371 16958
/proc/meminfo: memFree=22879720/32950928 swapFree=65730864/67111528
[pid=16952] ppid=16950 vsize=383744 CPUtime=403.43 cores=0,2,4,6
/proc/16952/stat : 16952 (claspmt) R 16950 16952 16432 0 -1 4202496 72218 0 0 0 40301 42 0 0 25 0 4 0 732207841 392953856 70367 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670538 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/16952/statm: 95936 70367 460 189 0 90285 0
[pid=16952/tid=16954] ppid=16950 vsize=383744 CPUtime=99.58 cores=0,2,4,6
/proc/16952/task/16954/stat : 16954 (claspmt) R 16950 16952 16432 0 -1 4202560 18560 0 0 0 9947 11 0 0 25 0 4 0 732207989 392953856 70367 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670340 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=16952/tid=16955] ppid=16950 vsize=383744 CPUtime=100.74 cores=0,2,4,6
/proc/16952/task/16955/stat : 16955 (claspmt) R 16950 16952 16432 0 -1 4202560 19253 0 0 0 10064 10 0 0 25 0 4 0 732207989 392953856 70367 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=16952/tid=16956] ppid=16950 vsize=383744 CPUtime=100.77 cores=0,2,4,6
/proc/16952/task/16956/stat : 16956 (claspmt) R 16950 16952 16432 0 -1 4202560 15169 0 0 0 10066 11 0 0 25 0 4 0 732207989 392953856 70367 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670499 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 403.43
Current children cumulated vsize (KiB) 383744

[startup+162.301 s]
/proc/loadavg: 8.15 8.13 7.85 9/371 16962
/proc/meminfo: memFree=22783572/32950928 swapFree=65730864/67111528
[pid=16952] ppid=16950 vsize=545156 CPUtime=642.66 cores=0,2,4,6
/proc/16952/stat : 16952 (claspmt) R 16950 16952 16432 0 -1 4202496 92233 0 0 0 64207 59 0 0 25 0 4 0 732207841 558239744 90329 33554432000 4194304 4964800 140736374656064 18446744073709551615 4627252 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/16952/statm: 136289 90329 460 189 0 130638 0
[pid=16952/tid=16954] ppid=16950 vsize=545156 CPUtime=158.87 cores=0,2,4,6
/proc/16952/task/16954/stat : 16954 (claspmt) R 16950 16952 16432 0 -1 4202560 32071 0 0 0 15864 23 0 0 25 0 4 0 732207989 558239744 90329 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670349 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=16952/tid=16955] ppid=16950 vsize=545156 CPUtime=160.74 cores=0,2,4,6
/proc/16952/task/16955/stat : 16955 (claspmt) R 16950 16952 16432 0 -1 4202560 23563 0 0 0 16061 13 0 0 25 0 4 0 732207989 558239744 90329 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=16952/tid=16956] ppid=16950 vsize=545156 CPUtime=160.75 cores=0,2,4,6
/proc/16952/task/16956/stat : 16956 (claspmt) R 16950 16952 16432 0 -1 4202560 16302 0 0 0 16062 13 0 0 25 0 4 0 732207989 558239744 90329 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 642.66
Current children cumulated vsize (KiB) 545156

[startup+222.301 s]
/proc/loadavg: 8.23 8.17 7.88 9/371 16964
/proc/meminfo: memFree=22713232/32950928 swapFree=65730864/67111528
[pid=16952] ppid=16950 vsize=545156 CPUtime=881.12 cores=0,2,4,6
/proc/16952/stat : 16952 (claspmt) R 16950 16952 16432 0 -1 4202496 106183 0 0 0 88039 73 0 0 25 0 4 0 732207841 558239744 104146 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/16952/statm: 136289 104146 460 189 0 130638 0
[pid=16952/tid=16954] ppid=16950 vsize=545156 CPUtime=218.05 cores=0,2,4,6
/proc/16952/task/16954/stat : 16954 (claspmt) R 16950 16952 16432 0 -1 4202560 40851 0 0 0 21773 32 0 0 25 0 4 0 732207989 558239744 104146 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=16952/tid=16955] ppid=16950 vsize=545156 CPUtime=220.73 cores=0,2,4,6
/proc/16952/task/16955/stat : 16955 (claspmt) R 16950 16952 16432 0 -1 4202560 25632 0 0 0 22059 14 0 0 25 0 4 0 732207989 558239744 104146 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=16952/tid=16956] ppid=16950 vsize=545156 CPUtime=220.18 cores=0,2,4,6
/proc/16952/task/16956/stat : 16956 (claspmt) R 16950 16952 16432 0 -1 4202560 18706 0 0 0 22002 16 0 0 25 0 4 0 732207989 558239744 104146 33554432000 4194304 4964800 140736374656064 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 881.12
Current children cumulated vsize (KiB) 545156

[startup+282.301 s]
/proc/loadavg: 8.13 8.15 7.89 9/371 16966
/proc/meminfo: memFree=22691992/32950928 swapFree=65730864/67111528
[pid=16952] ppid=16950 vsize=545156 CPUtime=1120.4 cores=0,2,4,6
/proc/16952/stat : 16952 (claspmt) R 16950 16952 16432 0 -1 4202496 109331 0 0 0 111964 76 0 0 25 0 4 0 732207841 558239744 107191 33554432000 4194304 4964800 140736374656064 18446744073709551615 4321102 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/16952/statm: 136289 107191 460 189 0 130638 0
[pid=16952/tid=16954] ppid=16950 vsize=545156 CPUtime=277.33 cores=0,2,4,6

################
# More data... #
################

Current children cumulated CPU time (s) 2556.09
Current children cumulated vsize (KiB) 831744

[startup+702.301 s]
/proc/loadavg: 8.13 8.14 7.96 9/371 16977
/proc/meminfo: memFree=22363596/32950928 swapFree=65730864/67111528
[pid=16952] ppid=16950 vsize=831744 CPUtime=2795.34 cores=0,2,4,6
/proc/16952/stat : 16952 (claspmt) R 16950 16952 16432 0 -1 4202496 179887 0 0 0 279390 144 0 0 25 0 4 0 732207841 851705856 176191 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/16952/statm: 207936 176191 460 189 0 202285 0
[pid=16952/tid=16954] ppid=16950 vsize=831744 CPUtime=692.34 cores=0,2,4,6
/proc/16952/task/16954/stat : 16954 (claspmt) R 16950 16952 16432 0 -1 4202560 82514 0 0 0 69159 75 0 0 25 0 4 0 732207989 851705856 176191 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=16952/tid=16955] ppid=16950 vsize=831744 CPUtime=700.74 cores=0,2,4,6
/proc/16952/task/16955/stat : 16955 (claspmt) R 16950 16952 16432 0 -1 4202560 34174 0 0 0 70054 20 0 0 25 0 4 0 732207989 851705856 176191 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=16952/tid=16956] ppid=16950 vsize=831744 CPUtime=700.08 cores=0,2,4,6
/proc/16952/task/16956/stat : 16956 (claspmt) R 16950 16952 16432 0 -1 4202560 33363 0 0 0 69980 28 0 0 25 0 4 0 732207989 851705856 176191 33554432000 4194304 4964800 140736374656064 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 2795.34
Current children cumulated vsize (KiB) 831744

[startup+762.314 s]
/proc/loadavg: 8.17 8.14 7.97 9/371 16979
/proc/meminfo: memFree=22281704/32950928 swapFree=65730864/67111528
[pid=16952] ppid=16950 vsize=929836 CPUtime=3034.67 cores=0,2,4,6
/proc/16952/stat : 16952 (claspmt) R 16950 16952 16432 0 -1 4202496 200041 0 0 0 303304 163 0 0 25 0 4 0 732207841 952152064 194977 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/16952/statm: 232459 194977 460 189 0 226808 0
[pid=16952/tid=16954] ppid=16950 vsize=929836 CPUtime=751.63 cores=0,2,4,6
/proc/16952/task/16954/stat : 16954 (claspmt) R 16950 16952 16432 0 -1 4202560 89591 0 0 0 75085 78 0 0 25 0 4 0 732207989 952152064 194977 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=16952/tid=16955] ppid=16950 vsize=929836 CPUtime=760.76 cores=0,2,4,6
/proc/16952/task/16955/stat : 16955 (claspmt) R 16950 16952 16432 0 -1 4202560 36630 0 0 0 76054 22 0 0 25 0 4 0 732207989 952152064 194977 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=16952/tid=16956] ppid=16950 vsize=929836 CPUtime=760.09 cores=0,2,4,6
/proc/16952/task/16956/stat : 16956 (claspmt) R 16950 16952 16432 0 -1 4202560 43612 0 0 0 75967 42 0 0 25 0 4 0 732207989 952152064 194977 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3034.67
Current children cumulated vsize (KiB) 929836

[startup+822.301 s]
/proc/loadavg: 8.07 8.12 7.97 9/371 16980
/proc/meminfo: memFree=22253124/32950928 swapFree=65730864/67111528
[pid=16952] ppid=16950 vsize=943056 CPUtime=3273.91 cores=0,2,4,6
/proc/16952/stat : 16952 (claspmt) R 16950 16952 16432 0 -1 4202496 206212 0 0 0 327224 167 0 0 25 0 4 0 732207841 965689344 200938 33554432000 4194304 4964800 140736374656064 18446744073709551615 4627204 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/16952/statm: 235764 200938 460 189 0 230113 0
[pid=16952/tid=16954] ppid=16950 vsize=943056 CPUtime=810.91 cores=0,2,4,6
/proc/16952/task/16954/stat : 16954 (claspmt) R 16950 16952 16432 0 -1 4202560 90750 0 0 0 81012 79 0 0 25 0 4 0 732207989 965689344 200938 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=16952/tid=16955] ppid=16950 vsize=943056 CPUtime=820.74 cores=0,2,4,6
/proc/16952/task/16955/stat : 16955 (claspmt) R 16950 16952 16432 0 -1 4202560 38372 0 0 0 82051 23 0 0 25 0 4 0 732207989 965689344 200938 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=16952/tid=16956] ppid=16950 vsize=943056 CPUtime=820.07 cores=0,2,4,6
/proc/16952/task/16956/stat : 16956 (claspmt) R 16950 16952 16432 0 -1 4202560 43616 0 0 0 81965 42 0 0 25 0 4 0 732207989 965689344 200938 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670295 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3273.91
Current children cumulated vsize (KiB) 943056

[startup+882.301 s]
/proc/loadavg: 8.13 8.12 7.98 9/371 16982
/proc/meminfo: memFree=22241664/32950928 swapFree=65730864/67111528
[pid=16952] ppid=16950 vsize=1008592 CPUtime=3513.17 cores=0,2,4,6
/proc/16952/stat : 16952 (claspmt) R 16950 16952 16432 0 -1 4202496 207493 0 0 0 351149 168 0 0 25 0 4 0 732207841 1032798208 202219 33554432000 4194304 4964800 140736374656064 18446744073709551615 4320955 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/16952/statm: 252148 202219 460 189 0 246497 0
[pid=16952/tid=16954] ppid=16950 vsize=1008592 CPUtime=870.19 cores=0,2,4,6
/proc/16952/task/16954/stat : 16954 (claspmt) R 16950 16952 16432 0 -1 4202560 91491 0 0 0 86940 79 0 0 25 0 4 0 732207989 1032798208 202219 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=16952/tid=16955] ppid=16950 vsize=1008592 CPUtime=880.75 cores=0,2,4,6
/proc/16952/task/16955/stat : 16955 (claspmt) R 16950 16952 16432 0 -1 4202560 38652 0 0 0 88051 24 0 0 25 0 4 0 732207989 1032798208 202219 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=16952/tid=16956] ppid=16950 vsize=1008592 CPUtime=880.05 cores=0,2,4,6
/proc/16952/task/16956/stat : 16956 (claspmt) R 16950 16952 16432 0 -1 4202560 43876 0 0 0 87963 42 0 0 25 0 4 0 732207989 1032798208 202219 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3513.17
Current children cumulated vsize (KiB) 1008592

[startup+942.301 s]
/proc/loadavg: 8.05 8.10 7.98 9/371 16984
/proc/meminfo: memFree=22208516/32950928 swapFree=65730864/67111528
[pid=16952] ppid=16950 vsize=1008592 CPUtime=3752.47 cores=0,2,4,6
/proc/16952/stat : 16952 (claspmt) R 16950 16952 16432 0 -1 4202496 215489 0 0 0 375069 178 0 0 25 0 4 0 732207841 1032798208 210215 33554432000 4194304 4964800 140736374656064 18446744073709551615 4653477 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/16952/statm: 252148 210215 460 189 0 246497 0
[pid=16952/tid=16954] ppid=16950 vsize=1008592 CPUtime=929.49 cores=0,2,4,6
/proc/16952/task/16954/stat : 16954 (claspmt) R 16950 16952 16432 0 -1 4202560 98417 0 0 0 92861 88 0 0 25 0 4 0 732207989 1032798208 210215 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=16952/tid=16955] ppid=16950 vsize=1008592 CPUtime=940.75 cores=0,2,4,6
/proc/16952/task/16955/stat : 16955 (claspmt) R 16950 16952 16432 0 -1 4202560 38709 0 0 0 94051 24 0 0 25 0 4 0 732207989 1032798208 210215 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=16952/tid=16956] ppid=16950 vsize=1008592 CPUtime=940.05 cores=0,2,4,6
/proc/16952/task/16956/stat : 16956 (claspmt) R 16950 16952 16432 0 -1 4202560 44884 0 0 0 93962 43 0 0 25 0 4 0 732207989 1032798208 210215 33554432000 4194304 4964800 140736374656064 18446744073709551615 4321059 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3752.47
Current children cumulated vsize (KiB) 1008592

[startup+1002.3 s]
/proc/loadavg: 8.11 8.11 8.00 9/371 16985
/proc/meminfo: memFree=22170644/32950928 swapFree=65730864/67111528
[pid=16952] ppid=16950 vsize=1074128 CPUtime=3991.74 cores=0,2,4,6
/proc/16952/stat : 16952 (claspmt) R 16950 16952 16432 0 -1 4202496 224002 0 0 0 398985 189 0 0 25 0 4 0 732207841 1099907072 218493 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/16952/statm: 268532 218493 460 189 0 262881 0
[pid=16952/tid=16954] ppid=16950 vsize=1074128 CPUtime=988.77 cores=0,2,4,6
/proc/16952/task/16954/stat : 16954 (claspmt) R 16950 16952 16432 0 -1 4202560 105878 0 0 0 98779 98 0 0 25 0 4 0 732207989 1099907072 218493 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670403 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=16952/tid=16955] ppid=16950 vsize=1074128 CPUtime=1000.75 cores=0,2,4,6
/proc/16952/task/16955/stat : 16955 (claspmt) R 16950 16952 16432 0 -1 4202560 39048 0 0 0 100051 24 0 0 25 0 4 0 732207989 1099907072 218493 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=16952/tid=16956] ppid=16950 vsize=1074128 CPUtime=1000.05 cores=0,2,4,6
/proc/16952/task/16956/stat : 16956 (claspmt) R 16950 16952 16432 0 -1 4202560 45592 0 0 0 99960 45 0 0 25 0 4 0 732207989 1099907072 218493 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3991.74
Current children cumulated vsize (KiB) 1074128

[startup+1062.3 s]
/proc/loadavg: 8.04 8.09 8.00 9/371 16987
/proc/meminfo: memFree=22078348/32950928 swapFree=65730864/67111528
[pid=16952] ppid=16950 vsize=1164724 CPUtime=4231.03 cores=0,2,4,6
/proc/16952/stat : 16952 (claspmt) R 16950 16952 16432 0 -1 4202496 246664 0 0 0 422888 215 0 0 25 0 4 0 732207841 1192677376 241155 33554432000 4194304 4964800 140736374656064 18446744073709551615 4320981 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/16952/statm: 291181 241155 460 189 0 285530 0
[pid=16952/tid=16954] ppid=16950 vsize=1164724 CPUtime=1048.07 cores=0,2,4,6
/proc/16952/task/16954/stat : 16954 (claspmt) R 16950 16952 16432 0 -1 4202560 122198 0 0 0 104686 121 0 0 25 0 4 0 732207989 1192677376 241155 33554432000 4194304 4964800 140736374656064 18446744073709551615 4668900 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=16952/tid=16955] ppid=16950 vsize=1164724 CPUtime=1060.75 cores=0,2,4,6
/proc/16952/task/16955/stat : 16955 (claspmt) R 16950 16952 16432 0 -1 4202560 39150 0 0 0 106050 25 0 0 25 0 4 0 732207989 1192677376 241155 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=16952/tid=16956] ppid=16950 vsize=1164724 CPUtime=1060.03 cores=0,2,4,6
/proc/16952/task/16956/stat : 16956 (claspmt) R 16950 16952 16432 0 -1 4202560 51796 0 0 0 105956 47 0 0 25 0 4 0 732207989 1192677376 241155 33554432000 4194304 4964800 140736374656064 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4231.03
Current children cumulated vsize (KiB) 1164724

[startup+1122.3 s]
/proc/loadavg: 8.17 8.11 8.01 9/371 16988
/proc/meminfo: memFree=22064896/32950928 swapFree=65730864/67111528
[pid=16952] ppid=16950 vsize=1168992 CPUtime=4470.3 cores=0,2,4,6
/proc/16952/stat : 16952 (claspmt) R 16950 16952 16432 0 -1 4202496 248542 0 0 0 446813 217 0 0 25 0 4 0 732207841 1197047808 243012 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670335 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/16952/statm: 292248 243012 460 189 0 286597 0
[pid=16952/tid=16954] ppid=16950 vsize=1168992 CPUtime=1107.35 cores=0,2,4,6
/proc/16952/task/16954/stat : 16954 (claspmt) R 16950 16952 16432 0 -1 4202560 122923 0 0 0 110613 122 0 0 25 0 4 0 732207989 1197047808 243012 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=16952/tid=16955] ppid=16950 vsize=1168992 CPUtime=1120.75 cores=0,2,4,6
/proc/16952/task/16955/stat : 16955 (claspmt) R 16950 16952 16432 0 -1 4202560 39450 0 0 0 112050 25 0 0 25 0 4 0 732207989 1197047808 243012 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=16952/tid=16956] ppid=16950 vsize=1168992 CPUtime=1120.02 cores=0,2,4,6
/proc/16952/task/16956/stat : 16956 (claspmt) R 16950 16952 16432 0 -1 4202560 52435 0 0 0 111954 48 0 0 25 0 4 0 732207989 1197047808 243012 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670343 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4470.3
Current children cumulated vsize (KiB) 1168992

[startup+1182.3 s]
/proc/loadavg: 8.23 8.14 8.02 9/371 16990
/proc/meminfo: memFree=22035220/32950928 swapFree=65730864/67111528
[pid=16952] ppid=16950 vsize=1168992 CPUtime=4709.58 cores=0,2,4,6
/proc/16952/stat : 16952 (claspmt) R 16950 16952 16432 0 -1 4202496 254278 0 0 0 470735 223 0 0 25 0 4 0 732207841 1197047808 248748 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670328 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/16952/statm: 292248 248748 460 189 0 286597 0
[pid=16952/tid=16954] ppid=16950 vsize=1168992 CPUtime=1166.65 cores=0,2,4,6
/proc/16952/task/16954/stat : 16954 (claspmt) R 16950 16952 16432 0 -1 4202560 128577 0 0 0 116537 128 0 0 25 0 4 0 732207989 1197047808 248748 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670313 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=16952/tid=16955] ppid=16950 vsize=1168992 CPUtime=1180.74 cores=0,2,4,6
/proc/16952/task/16955/stat : 16955 (claspmt) R 16950 16952 16432 0 -1 4202560 39476 0 0 0 118049 25 0 0 25 0 4 0 732207989 1197047808 248748 33554432000 4194304 4964800 140736374656064 18446744073709551615 4379076 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=16952/tid=16956] ppid=16950 vsize=1168992 CPUtime=1180.01 cores=0,2,4,6
/proc/16952/task/16956/stat : 16956 (claspmt) R 16950 16952 16432 0 -1 4202560 52474 0 0 0 117953 48 0 0 25 0 4 0 732207989 1197047808 248748 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4709.58
Current children cumulated vsize (KiB) 1168992



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1205 s]
/proc/loadavg: 8.23 8.14 8.02 5/366 16992
/proc/meminfo: memFree=22258564/32950928 swapFree=65730864/67111528
[pid=16952] ppid=16950 vsize=1168992 CPUtime=4800.12 cores=0,2,4,6
/proc/16952/stat : 16952 (claspmt) R 16950 16952 16432 0 -1 4202496 259414 0 0 0 479780 232 0 0 25 0 4 0 732207841 1197047808 253884 33554432000 4194304 4964800 140736374656064 18446744073709551615 4627204 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/16952/statm: 292248 253884 460 189 0 286597 0
[pid=16952/tid=16954] ppid=16950 vsize=1168992 CPUtime=1189.08 cores=0,2,4,6
/proc/16952/task/16954/stat : 16954 (claspmt) R 16950 16952 16432 0 -1 4202560 133598 0 0 0 118772 136 0 0 25 0 4 0 732207989 1197047808 253884 33554432000 4194304 4964800 140736374656064 18446744073709551615 4342174 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=16952/tid=16955] ppid=16950 vsize=1168992 CPUtime=1203.46 cores=0,2,4,6
/proc/16952/task/16955/stat : 16955 (claspmt) R 16950 16952 16432 0 -1 4202560 39577 0 0 0 120321 25 0 0 25 0 4 0 732207989 1197047808 253884 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=16952/tid=16956] ppid=16950 vsize=1168992 CPUtime=1202.71 cores=0,2,4,6
/proc/16952/task/16956/stat : 16956 (claspmt) R 16950 16952 16432 0 -1 4202560 52478 0 0 0 120223 48 0 0 25 0 4 0 732207989 1197047808 253884 33554432000 4194304 4964800 140736374656064 18446744073709551615 4670273 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4800.12
Current children cumulated vsize (KiB) 1168992

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1205.12 s]
/proc/loadavg: 8.23 8.14 8.02 5/366 16992
/proc/meminfo: memFree=22258564/32950928 swapFree=65730864/67111528
[pid=16952] ppid=16950 vsize=1160484 CPUtime=4800.26 cores=0,2,4,6
/proc/16952/stat : 16952 (claspmt) R 16950 16952 16432 0 -1 4202496 259424 0 0 0 479794 232 0 0 25 0 1 0 732207841 1188335616 252041 33554432000 4194304 4964800 140736374656064 18446744073709551615 260905050267 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/16952/statm: 290121 252041 470 189 0 284470 0
Current children cumulated CPU time (s) 4800.26
Current children cumulated vsize (KiB) 1160484

Child status: 0
Real time (s): 1205.18
CPU time (s): 4800.33
CPU user time (s): 4797.96
CPU system time (s): 2.36364
CPU usage (%): 398.309
Max. virtual memory (cumulated for all children) (KiB): 1168992

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4797.96
system time used= 2.36364
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 259426
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 7851
involuntary context switches= 23306

runsolver used 4.09538 second user time and 10.1635 second system time

The end

Launcher Data 

Begin job on node116 at 2011-05-15 02:36:12
IDJOB=3373256
IDBENCH=71492
IDSOLVER=1853
FILE ID=node116/3373256-1305419772
RUNJOBID= node116-1305418565-16833
PBS_JOBID= 13324555
Free space on /tmp= 73132 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT09/CRAFTED/rbsat/crafted/unforced/rbsat-v2400c266431g9.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3373256-1305419772/watcher-3373256-1305419772 -o /tmp/evaluation-result-3373256-1305419772/solver-3373256-1305419772 -C 4800 -W 1300 -M 15500  HOME/claspmt -t4 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3373256-1305419772.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 35278b3ce28129761cf8ae7994f37d3f
RANDOM SEED=1172086093

node116.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      23273068 kB
Buffers:         55620 kB
Cached:         117048 kB
SwapCached:      12356 kB
Active:        9389232 kB
Inactive:        82132 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      23273068 kB
SwapTotal:    67111528 kB
SwapFree:     65730864 kB
Dirty:            5484 kB
Writeback:           0 kB
AnonPages:     9296736 kB
Mapped:          16372 kB
Slab:            96892 kB
PageTables:      44080 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 82783632 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73132 MiB
End job on node116 at 2011-05-15 02:56:19