Trace number 3322407

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095? (TO) 35743.6 5100.85

General information on the benchmark

Namecrafted/mosoi/289/
289-unsat-17x30.cnf
MD5SUM3a506c7557d86d0d326e3b3cdee85b69
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2040
Number of clauses240210
Sum of the clauses size954720
Maximum clause length4
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 23060
Number of clauses of size 30
Number of clauses of size 4237150
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...383803/instance-3322407-1304383803.cnf
12.38/12.40	c Solving...
35742.69/5100.01	
35742.69/5100.01	*** INTERRUPTED! ***
35742.69/5100.01	*** Shutting down threads...
35743.51/5100.84	s UNKNOWN
35743.51/5100.84	
35743.51/5100.84	c INTERRUPTED   : 1
35743.51/5100.84	c Models        : 0+    
35743.51/5100.84	c Time          : 5100.018s (Solving: 5087.61s 1st Model: 0.00s Unsat: 0.00s)
35743.51/5100.84	c CPU Time      : 35742.740s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3322407-1304383803/watcher-3322407-1304383803 -o /tmp/evaluation-result-3322407-1304383803/solver-3322407-1304383803 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3322407-1304383803.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.00 1.00 1.00 2/379 11968
/proc/meminfo: memFree=4536216/32950928 swapFree=67111448/67111528
[pid=11968] ppid=11966 vsize=23208 CPUtime=0.01 cores=0-7
/proc/11968/stat : 11968 (claspmt) R 11966 11968 9896 0 -1 4202496 1173 0 0 0 1 0 0 0 22 0 1 0 549979049 23764992 997 33554432000 4194304 4964704 140734324619456 18446744073709551615 251813900997 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/11968/statm: 5802 999 386 189 0 676 0

[startup+0.0462381 s]
/proc/loadavg: 1.00 1.00 1.00 2/379 11968
/proc/meminfo: memFree=4536216/32950928 swapFree=67111448/67111528
[pid=11968] ppid=11966 vsize=28720 CPUtime=0.04 cores=0-7
/proc/11968/stat : 11968 (claspmt) R 11966 11968 9896 0 -1 4202496 2963 0 0 0 4 0 0 0 22 0 1 0 549979049 29409280 2404 33554432000 4194304 4964704 140734324619456 18446744073709551615 4567891 0 0 4096 18946 0 0 0 17 7 0 0 0
/proc/11968/statm: 7180 2404 386 189 0 2054 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 28720

[startup+0.100234 s]
/proc/loadavg: 1.00 1.00 1.00 2/379 11968
/proc/meminfo: memFree=4536216/32950928 swapFree=67111448/67111528
[pid=11968] ppid=11966 vsize=39080 CPUtime=0.09 cores=0-7
/proc/11968/stat : 11968 (claspmt) R 11966 11968 9896 0 -1 4202496 6104 0 0 0 9 0 0 0 22 0 1 0 549979049 40017920 4850 33554432000 4194304 4964704 140734324619456 18446744073709551615 4616033 0 0 4096 18946 0 0 0 17 7 0 0 0
/proc/11968/statm: 9770 4850 397 189 0 4644 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 39080

[startup+0.300207 s]
/proc/loadavg: 1.00 1.00 1.00 2/379 11968
/proc/meminfo: memFree=4536216/32950928 swapFree=67111448/67111528
[pid=11968] ppid=11966 vsize=41812 CPUtime=0.29 cores=0-7
/proc/11968/stat : 11968 (claspmt) R 11966 11968 9896 0 -1 4202496 6721 0 0 0 29 0 0 0 23 0 1 0 549979049 42815488 5467 33554432000 4194304 4964704 140734324619456 18446744073709551615 4607472 0 0 4096 18946 0 0 0 17 7 0 0 0
/proc/11968/statm: 10453 5467 399 189 0 5327 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 41812

[startup+0.700127 s]
/proc/loadavg: 1.00 1.00 1.00 2/379 11968
/proc/meminfo: memFree=4536216/32950928 swapFree=67111448/67111528
[pid=11968] ppid=11966 vsize=41812 CPUtime=0.69 cores=0-7
/proc/11968/stat : 11968 (claspmt) R 11966 11968 9896 0 -1 4202496 6721 0 0 0 69 0 0 0 25 0 1 0 549979049 42815488 5467 33554432000 4194304 4964704 140734324619456 18446744073709551615 4607585 0 0 4096 18946 0 0 0 17 7 0 0 0
/proc/11968/statm: 10453 5467 399 189 0 5327 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 41812

[startup+1.501 s]
/proc/loadavg: 1.00 1.00 1.00 3/380 11969
/proc/meminfo: memFree=4515504/32950928 swapFree=67111448/67111528
[pid=11968] ppid=11966 vsize=42076 CPUtime=1.48 cores=0-7
/proc/11968/stat : 11968 (claspmt) R 11966 11968 9896 0 -1 4202496 6788 0 0 0 148 0 0 0 25 0 1 0 549979049 43085824 5534 33554432000 4194304 4964704 140734324619456 18446744073709551615 4600360 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/11968/statm: 10519 5534 400 189 0 5393 0
Current children cumulated CPU time (s) 1.48
Current children cumulated vsize (KiB) 42076

[startup+3.10079 s]
/proc/loadavg: 1.00 1.00 1.00 3/380 11969
/proc/meminfo: memFree=4512280/32950928 swapFree=67111448/67111528
[pid=11968] ppid=11966 vsize=47356 CPUtime=3.08 cores=0-7
/proc/11968/stat : 11968 (claspmt) R 11966 11968 9896 0 -1 4202496 8113 0 0 0 308 0 0 0 25 0 1 0 549979049 48492544 6859 33554432000 4194304 4964704 140734324619456 18446744073709551615 4612288 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/11968/statm: 11839 6859 400 189 0 6713 0
Current children cumulated CPU time (s) 3.08
Current children cumulated vsize (KiB) 47356

[startup+6.30019 s]
/proc/loadavg: 1.08 1.02 1.01 3/380 11970
/proc/meminfo: memFree=4503352/32950928 swapFree=67111448/67111528
[pid=11968] ppid=11966 vsize=55672 CPUtime=6.28 cores=0-7
/proc/11968/stat : 11968 (claspmt) R 11966 11968 9896 0 -1 4202496 10198 0 0 0 627 1 0 0 25 0 1 0 549979049 57008128 8944 33554432000 4194304 4964704 140734324619456 18446744073709551615 4600585 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/11968/statm: 13918 8944 400 189 0 8792 0
Current children cumulated CPU time (s) 6.28
Current children cumulated vsize (KiB) 55672

[startup+12.7011 s]
/proc/loadavg: 1.15 1.03 1.01 3/380 11970
/proc/meminfo: memFree=4497900/32950928 swapFree=67111448/67111528
[pid=11968] ppid=11966 vsize=512828 CPUtime=13.45 cores=0-7
/proc/11968/stat : 11968 (claspmt) R 11966 11968 9896 0 -1 4202496 42316 0 0 0 1327 18 0 0 16 0 8 0 549979049 525135872 40497 33554432000 4194304 4964704 140734324619456 18446744073709551615 4653350 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/11968/statm: 128207 40497 462 189 0 122556 0
Current children cumulated CPU time (s) 13.45
Current children cumulated vsize (KiB) 512828

[startup+25.501 s]
/proc/loadavg: 2.96 1.44 1.14 10/387 11977
/proc/meminfo: memFree=4164216/32950928 swapFree=67111448/67111528
[pid=11968] ppid=11966 vsize=533192 CPUtime=102.47 cores=0-7
/proc/11968/stat : 11968 (claspmt) R 11966 11968 9896 0 -1 4202496 96082 0 0 0 10198 49 0 0 25 0 8 0 549979049 545988608 94263 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/11968/statm: 133298 94263 466 189 0 127647 0
[pid=11968/tid=11971] ppid=11966 vsize=533192 CPUtime=7.89 cores=0-7
/proc/11968/task/11971/stat : 11971 (claspmt) R 11966 11968 9896 0 -1 4202560 8684 0 0 0 786 3 0 0 25 0 8 0 549980290 545988608 94263 33554432000 4194304 4964704 140734324619456 18446744073709551615 4362195 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11968/tid=11972] ppid=11966 vsize=533192 CPUtime=12.24 cores=0-7
/proc/11968/task/11972/stat : 11972 (claspmt) R 11966 11968 9896 0 -1 4202560 11755 0 0 0 1216 8 0 0 25 0 8 0 549980290 545988608 94263 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670193 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11968/tid=11973] ppid=11966 vsize=533192 CPUtime=11.65 cores=0-7
/proc/11968/task/11973/stat : 11973 (claspmt) R 11966 11968 9896 0 -1 4202560 8542 0 0 0 1160 5 0 0 25 0 8 0 549980290 545988608 94263 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11968/tid=11974] ppid=11966 vsize=533192 CPUtime=11.44 cores=0-7
/proc/11968/task/11974/stat : 11974 (claspmt) R 11966 11968 9896 0 -1 4202560 11312 0 0 0 1138 6 0 0 25 0 8 0 549980290 545988608 94263 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670281 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11968/tid=11975] ppid=11966 vsize=533192 CPUtime=12.57 cores=0-7
/proc/11968/task/11975/stat : 11975 (claspmt) R 11966 11968 9896 0 -1 4202560 10952 0 0 0 1251 6 0 0 25 0 8 0 549980290 545988608 94263 33554432000 4194304 4964704 140734324619456 18446744073709551615 4684662 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11968/tid=11976] ppid=11966 vsize=533192 CPUtime=11.24 cores=0-7
/proc/11968/task/11976/stat : 11976 (claspmt) R 11966 11968 9896 0 -1 4202560 8913 0 0 0 1120 4 0 0 25 0 8 0 549980290 545988608 94263 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11968/tid=11977] ppid=11966 vsize=533192 CPUtime=10.72 cores=0-7
/proc/11968/task/11977/stat : 11977 (claspmt) R 11966 11968 9896 0 -1 4202560 13362 0 0 0 1065 7 0 0 25 0 8 0 549980290 545988608 94263 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 102.47
Current children cumulated vsize (KiB) 533192

[startup+51.1007 s]
/proc/loadavg: 5.02 2.04 1.35 10/387 11977
/proc/meminfo: memFree=4012648/32950928 swapFree=67111448/67111528
[pid=11968] ppid=11966 vsize=881732 CPUtime=280.39 cores=0-7
/proc/11968/stat : 11968 (claspmt) R 11966 11968 9896 0 -1 4202496 135827 0 0 0 27961 78 0 0 25 0 8 0 549979049 902893568 134001 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670266 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11968/statm: 220433 134001 466 189 0 214782 0
[pid=11968/tid=11971] ppid=11966 vsize=881732 CPUtime=24.5 cores=0-7
/proc/11968/task/11971/stat : 11971 (claspmt) R 11966 11968 9896 0 -1 4202560 13721 0 0 0 2444 6 0 0 25 0 8 0 549980290 902893568 134001 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11968/tid=11972] ppid=11966 vsize=881732 CPUtime=34.84 cores=0-7
/proc/11968/task/11972/stat : 11972 (claspmt) R 11966 11968 9896 0 -1 4202560 15566 0 0 0 3472 12 0 0 25 0 8 0 549980290 902893568 134001 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670193 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11968/tid=11973] ppid=11966 vsize=881732 CPUtime=36.04 cores=0-7
/proc/11968/task/11973/stat : 11973 (claspmt) R 11966 11968 9896 0 -1 4202560 11332 0 0 0 3597 7 0 0 25 0 8 0 549980290 902893568 134001 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11968/tid=11974] ppid=11966 vsize=881732 CPUtime=35.16 cores=0-7
/proc/11968/task/11974/stat : 11974 (claspmt) R 11966 11968 9896 0 -1 4202560 16846 0 0 0 3504 12 0 0 25 0 8 0 549980290 902893568 134001 33554432000 4194304 4964704 140734324619456 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11968/tid=11975] ppid=11966 vsize=881732 CPUtime=37.9 cores=0-7
/proc/11968/task/11975/stat : 11975 (claspmt) R 11966 11968 9896 0 -1 4202560 14710 0 0 0 3782 8 0 0 25 0 8 0 549980290 902893568 134001 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11968/tid=11976] ppid=11966 vsize=881732 CPUtime=34.15 cores=0-7
/proc/11968/task/11976/stat : 11976 (claspmt) R 11966 11968 9896 0 -1 4202560 13249 0 0 0 3407 8 0 0 25 0 8 0 549980290 902893568 134001 33554432000 4194304 4964704 140734324619456 18446744073709551615 251814345198 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11968/tid=11977] ppid=11966 vsize=881732 CPUtime=35.09 cores=0-7
/proc/11968/task/11977/stat : 11977 (claspmt) R 11966 11968 9896 0 -1 4202560 23872 0 0 0 3494 15 0 0 25 0 8 0 549980290 902893568 134001 33554432000 4194304 4964704 140734324619456 18446744073709551615 4320966 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 280.39
Current children cumulated vsize (KiB) 881732

[startup+102.314 s]
/proc/loadavg: 7.27 3.11 1.75 10/387 11979
/proc/meminfo: memFree=3858752/32950928 swapFree=67111448/67111528
[pid=11968] ppid=11966 vsize=1034088 CPUtime=637.46 cores=0-7
/proc/11968/stat : 11968 (claspmt) R 11966 11968 9896 0 -1 4202496 172564 0 0 0 63634 112 0 0 25 0 8 0 549979049 1058906112 170552 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/11968/statm: 258522 170552 466 189 0 252871 0
[pid=11968/tid=11971] ppid=11966 vsize=1034088 CPUtime=69.08 cores=0-7
/proc/11968/task/11971/stat : 11971 (claspmt) R 11966 11968 9896 0 -1 4202560 19626 0 0 0 6897 11 0 0 25 0 8 0 549980290 1058906112 170552 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670250 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11968/tid=11972] ppid=11966 vsize=1034088 CPUtime=81.12 cores=0-7
/proc/11968/task/11972/stat : 11972 (claspmt) R 11966 11968 9896 0 -1 4202560 25569 0 0 0 8093 19 0 0 25 0 8 0 549980290 1058906112 170552 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11968/tid=11973] ppid=11966 vsize=1034088 CPUtime=79.88 cores=0-7
/proc/11968/task/11973/stat : 11973 (claspmt) R 11966 11968 9896 0 -1 4202560 13185 0 0 0 7979 9 0 0 25 0 8 0 549980290 1058906112 170552 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11968/tid=11974] ppid=11966 vsize=1034088 CPUtime=82.17 cores=0-7
/proc/11968/task/11974/stat : 11974 (claspmt) R 11966 11968 9896 0 -1 4202560 19832 0 0 0 8202 15 0 0 25 0 8 0 549980290 1058906112 170552 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11968/tid=11975] ppid=11966 vsize=1034088 CPUtime=89.11 cores=0-7
/proc/11968/task/11975/stat : 11975 (claspmt) R 11966 11968 9896 0 -1 4202560 16883 0 0 0 8900 11 0 0 25 0 8 0 549980290 1058906112 170552 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11968/tid=11976] ppid=11966 vsize=1034088 CPUtime=81.31 cores=0-7
/proc/11968/task/11976/stat : 11976 (claspmt) R 11966 11968 9896 0 -1 4202560 15526 0 0 0 8120 11 0 0 25 0 8 0 549980290 1058906112 170552 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11968/tid=11977] ppid=11966 vsize=1034088 CPUtime=73.99 cores=0-7
/proc/11968/task/11977/stat : 11977 (claspmt) R 11966 11968 9896 0 -1 4202560 28559 0 0 0 7380 19 0 0 25 0 8 0 549980290 1058906112 170552 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 637.46
Current children cumulated vsize (KiB) 1034088

[startup+162.301 s]
/proc/loadavg: 8.40 4.20 2.20 10/387 11981
/proc/meminfo: memFree=3713012/32950928 swapFree=67111448/67111528
[pid=11968] ppid=11966 vsize=1128440 CPUtime=1056.11 cores=0-7
/proc/11968/stat : 11968 (claspmt) R 11966 11968 9896 0 -1 4202496 209568 0 0 0 105468 143 0 0 25 0 8 0 549979049 1155522560 207372 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/11968/statm: 282110 207372 466 189 0 276459 0
[pid=11968/tid=11971] ppid=11966 vsize=1128440 CPUtime=107.77 cores=0-7
/proc/11968/task/11971/stat : 11971 (claspmt) R 11966 11968 9896 0 -1 4202560 25795 0 0 0 10760 17 0 0 25 0 8 0 549980290 1155522560 207372 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670215 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11968/tid=11972] ppid=11966 vsize=1128440 CPUtime=140.41 cores=0-7
/proc/11968/task/11972/stat : 11972 (claspmt) R 11966 11968 9896 0 -1 4202560 28255 0 0 0 14021 20 0 0 25 0 8 0 549980290 1155522560 207372 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11968/tid=11973] ppid=11966 vsize=1128440 CPUtime=139.57 cores=0-7
/proc/11968/task/11973/stat : 11973 (claspmt) R 11966 11968 9896 0 -1 4202560 14534 0 0 0 13947 10 0 0 25 0 8 0 549980290 1155522560 207372 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11968/tid=11974] ppid=11966 vsize=1128440 CPUtime=134.89 cores=0-7
/proc/11968/task/11974/stat : 11974 (claspmt) R 11966 11968 9896 0 -1 4202560 22934 0 0 0 13471 18 0 0 25 0 8 0 549980290 1155522560 207372 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11968/tid=11975] ppid=11966 vsize=1128440 CPUtime=146.96 cores=0-7
/proc/11968/task/11975/stat : 11975 (claspmt) R 11966 11968 9896 0 -1 4202560 21592 0 0 0 14681 15 0 0 25 0 8 0 549980290 1155522560 207372 33554432000 4194304 4964704 140734324619456 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11968/tid=11976] ppid=11966 vsize=1128440 CPUtime=138.69 cores=0-7
/proc/11968/task/11976/stat : 11976 (claspmt) R 11966 11968 9896 0 -1 4202560 20508 0 0 0 13854 15 0 0 25 0 8 0 549980290 1155522560 207372 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11968/tid=11977] ppid=11966 vsize=1128440 CPUtime=120.24 cores=0-7
/proc/11968/task/11977/stat : 11977 (claspmt) R 11966 11968 9896 0 -1 4202560 36065 0 0 0 11999 25 0 0 25 0 8 0 549980290 1155522560 207372 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 1056.11
Current children cumulated vsize (KiB) 1128440

[startup+222.301 s]
/proc/loadavg: 8.76 5.07 2.63 10/387 11982
/proc/meminfo: memFree=3540988/32950928 swapFree=67111448/67111528
[pid=11968] ppid=11966 vsize=1265116 CPUtime=1473.67 cores=0-7
/proc/11968/stat : 11968 (claspmt) R 11966 11968 9896 0 -1 4202496 252891 0 0 0 147178 189 0 0 25 0 8 0 549979049 1295478784 249760 33554432000 4194304 4964704 140734324619456 18446744073709551615 4320944 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/11968/statm: 316279 249760 466 189 0 310628 0
[pid=11968/tid=11971] ppid=11966 vsize=1265116 CPUtime=156.56 cores=0-7
/proc/11968/task/11971/stat : 11971 (claspmt) R 11966 11968 9896 0 -1 4202560 28328 0 0 0 15635 21 0 0 25 0 8 0 549980290 1295478784 249760 33554432000 4194304 4964704 140734324619456 18446744073709551615 251814345198 0 0 4096 18946 0 0 0 -1 3 0 0 0

################
# More data... #
################

/proc/11968/task/11976/stat : 11976 (claspmt) R 11966 11968 9896 0 -1 4202560 65654 0 0 0 437933 86 0 0 25 0 8 0 549980290 3114819584 692444 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11968/tid=11977] ppid=11966 vsize=3041816 CPUtime=3796.62 cores=0-7
/proc/11968/task/11977/stat : 11977 (claspmt) R 11966 11968 9896 0 -1 4202560 147812 0 0 0 379477 185 0 0 25 0 8 0 549980290 3114819584 692444 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 33517
Current children cumulated vsize (KiB) 3041816

[startup+4842.3 s]
/proc/loadavg: 9.12 9.11 9.01 10/387 12627
/proc/meminfo: memFree=1763636/32950928 swapFree=67111448/67111528
[pid=11968] ppid=11966 vsize=3042456 CPUtime=33936.2 cores=0-7
/proc/11968/stat : 11968 (claspmt) R 11966 11968 9896 0 -1 4202496 704411 0 0 0 3392790 829 0 0 25 0 8 0 549979049 3115474944 692748 33554432000 4194304 4964704 140734324619456 18446744073709551615 4320951 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/11968/statm: 760614 692748 466 189 0 754963 0
[pid=11968/tid=11971] ppid=11966 vsize=3042456 CPUtime=4308.94 cores=0-7
/proc/11968/task/11971/stat : 11971 (claspmt) R 11966 11968 9896 0 -1 4202560 78085 0 0 0 430805 89 0 0 25 0 8 0 549980290 3115474944 692748 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11968/tid=11972] ppid=11966 vsize=3042456 CPUtime=4303.26 cores=0-7
/proc/11968/task/11972/stat : 11972 (claspmt) R 11966 11968 9896 0 -1 4202560 90370 0 0 0 430227 99 0 0 25 0 8 0 549980290 3115474944 692748 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11968/tid=11973] ppid=11966 vsize=3042456 CPUtime=4330.09 cores=0-7
/proc/11968/task/11973/stat : 11973 (claspmt) R 11966 11968 9896 0 -1 4202560 62159 0 0 0 432927 82 0 0 25 0 8 0 549980290 3115474944 692748 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11968/tid=11974] ppid=11966 vsize=3042456 CPUtime=4378.65 cores=0-7
/proc/11968/task/11974/stat : 11974 (claspmt) R 11966 11968 9896 0 -1 4202560 59531 0 0 0 437802 63 0 0 25 0 8 0 549980290 3115474944 692748 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11968/tid=11975] ppid=11966 vsize=3042456 CPUtime=4572.16 cores=0-7
/proc/11968/task/11975/stat : 11975 (claspmt) R 11966 11968 9896 0 -1 4202560 53520 0 0 0 457163 53 0 0 25 0 8 0 549980290 3115474944 692748 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11968/tid=11976] ppid=11966 vsize=3042456 CPUtime=4440.19 cores=0-7
/proc/11968/task/11976/stat : 11976 (claspmt) R 11966 11968 9896 0 -1 4202560 65655 0 0 0 443933 86 0 0 25 0 8 0 549980290 3115474944 692748 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670419 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11968/tid=11977] ppid=11966 vsize=3042456 CPUtime=3856.62 cores=0-7
/proc/11968/task/11977/stat : 11977 (claspmt) R 11966 11968 9896 0 -1 4202560 147813 0 0 0 385477 185 0 0 25 0 8 0 549980290 3115474944 692748 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 33936.2
Current children cumulated vsize (KiB) 3042456

[startup+4902.3 s]
/proc/loadavg: 9.10 9.11 9.01 10/387 12629
/proc/meminfo: memFree=1763464/32950928 swapFree=67111448/67111528
[pid=11968] ppid=11966 vsize=3042456 CPUtime=34355.4 cores=0-7
/proc/11968/stat : 11968 (claspmt) R 11966 11968 9896 0 -1 4202496 704458 0 0 0 3434710 832 0 0 25 0 8 0 549979049 3115474944 692795 33554432000 4194304 4964704 140734324619456 18446744073709551615 4320928 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/11968/statm: 760614 692795 466 189 0 754963 0
[pid=11968/tid=11971] ppid=11966 vsize=3042456 CPUtime=4357.24 cores=0-7
/proc/11968/task/11971/stat : 11971 (claspmt) R 11966 11968 9896 0 -1 4202560 78085 0 0 0 435634 90 0 0 25 0 8 0 549980290 3115474944 692795 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11968/tid=11972] ppid=11966 vsize=3042456 CPUtime=4362.5 cores=0-7
/proc/11968/task/11972/stat : 11972 (claspmt) R 11966 11968 9896 0 -1 4202560 90372 0 0 0 436151 99 0 0 25 0 8 0 549980290 3115474944 692795 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670193 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11968/tid=11973] ppid=11966 vsize=3042456 CPUtime=4368.79 cores=0-7
/proc/11968/task/11973/stat : 11973 (claspmt) R 11966 11968 9896 0 -1 4202560 62160 0 0 0 436797 82 0 0 25 0 8 0 549980290 3115474944 692795 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11968/tid=11974] ppid=11966 vsize=3042456 CPUtime=4435.14 cores=0-7
/proc/11968/task/11974/stat : 11974 (claspmt) R 11966 11968 9896 0 -1 4202560 59536 0 0 0 443451 63 0 0 25 0 8 0 549980290 3115474944 692795 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11968/tid=11975] ppid=11966 vsize=3042456 CPUtime=4632.16 cores=0-7
/proc/11968/task/11975/stat : 11975 (claspmt) R 11966 11968 9896 0 -1 4202560 53520 0 0 0 463163 53 0 0 25 0 8 0 549980290 3115474944 692795 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11968/tid=11976] ppid=11966 vsize=3042456 CPUtime=4500.18 cores=0-7
/proc/11968/task/11976/stat : 11976 (claspmt) R 11966 11968 9896 0 -1 4202560 65666 0 0 0 449932 86 0 0 25 0 8 0 549980290 3115474944 692795 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11968/tid=11977] ppid=11966 vsize=3042456 CPUtime=3916.61 cores=0-7
/proc/11968/task/11977/stat : 11977 (claspmt) R 11966 11968 9896 0 -1 4202560 147841 0 0 0 391476 185 0 0 25 0 8 0 549980290 3115474944 692795 33554432000 4194304 4964704 140734324619456 18446744073709551615 251813937770 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 34355.4
Current children cumulated vsize (KiB) 3042456

[startup+4962.3 s]
/proc/loadavg: 9.09 9.10 9.01 10/387 12631
/proc/meminfo: memFree=1763176/32950928 swapFree=67111448/67111528
[pid=11968] ppid=11966 vsize=3042456 CPUtime=34774.6 cores=0-7
/proc/11968/stat : 11968 (claspmt) R 11966 11968 9896 0 -1 4202496 704637 0 0 0 3476631 832 0 0 25 0 8 0 549979049 3115474944 692974 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/11968/statm: 760614 692974 466 189 0 754963 0
[pid=11968/tid=11971] ppid=11966 vsize=3042456 CPUtime=4395.84 cores=0-7
/proc/11968/task/11971/stat : 11971 (claspmt) R 11966 11968 9896 0 -1 4202560 78150 0 0 0 439494 90 0 0 25 0 8 0 549980290 3115474944 692974 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11968/tid=11972] ppid=11966 vsize=3042456 CPUtime=4421.73 cores=0-7
/proc/11968/task/11972/stat : 11972 (claspmt) R 11966 11968 9896 0 -1 4202560 90469 0 0 0 442074 99 0 0 25 0 8 0 549980290 3115474944 692974 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11968/tid=11973] ppid=11966 vsize=3042456 CPUtime=4420.19 cores=0-7
/proc/11968/task/11973/stat : 11973 (claspmt) R 11966 11968 9896 0 -1 4202560 62163 0 0 0 441937 82 0 0 25 0 8 0 549980290 3115474944 692974 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670255 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11968/tid=11974] ppid=11966 vsize=3042456 CPUtime=4495.12 cores=0-7
/proc/11968/task/11974/stat : 11974 (claspmt) R 11966 11968 9896 0 -1 4202560 59538 0 0 0 449449 63 0 0 25 0 8 0 549980290 3115474944 692974 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670440 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11968/tid=11975] ppid=11966 vsize=3042456 CPUtime=4692.16 cores=0-7
/proc/11968/task/11975/stat : 11975 (claspmt) R 11966 11968 9896 0 -1 4202560 53522 0 0 0 469163 53 0 0 25 0 8 0 549980290 3115474944 692974 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11968/tid=11976] ppid=11966 vsize=3042456 CPUtime=4560.18 cores=0-7
/proc/11968/task/11976/stat : 11976 (claspmt) R 11966 11968 9896 0 -1 4202560 65671 0 0 0 455932 86 0 0 25 0 8 0 549980290 3115474944 692974 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11968/tid=11977] ppid=11966 vsize=3042456 CPUtime=3976.61 cores=0-7
/proc/11968/task/11977/stat : 11977 (claspmt) R 11966 11968 9896 0 -1 4202560 147845 0 0 0 397476 185 0 0 25 0 8 0 549980290 3115474944 692974 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 34774.6
Current children cumulated vsize (KiB) 3042456

[startup+5022.3 s]
/proc/loadavg: 9.08 9.09 9.01 10/387 12632
/proc/meminfo: memFree=1761400/32950928 swapFree=67111448/67111528
[pid=11968] ppid=11966 vsize=3042456 CPUtime=35193.9 cores=0-7
/proc/11968/stat : 11968 (claspmt) R 11966 11968 9896 0 -1 4202496 705197 0 0 0 3518554 832 0 0 25 0 8 0 549979049 3115474944 693460 33554432000 4194304 4964704 140734324619456 18446744073709551615 4684027 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/11968/statm: 760614 693460 466 189 0 754963 0
[pid=11968/tid=11971] ppid=11966 vsize=3042456 CPUtime=4439.24 cores=0-7
/proc/11968/task/11971/stat : 11971 (claspmt) R 11966 11968 9896 0 -1 4202560 78150 0 0 0 443834 90 0 0 25 0 8 0 549980290 3115474944 693460 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11968/tid=11972] ppid=11966 vsize=3042456 CPUtime=4480.98 cores=0-7
/proc/11968/task/11972/stat : 11972 (claspmt) R 11966 11968 9896 0 -1 4202560 90921 0 0 0 447998 100 0 0 25 0 8 0 549980290 3115474944 693460 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=11968/tid=11973] ppid=11966 vsize=3042456 CPUtime=4480.2 cores=0-7
/proc/11968/task/11973/stat : 11973 (claspmt) R 11966 11968 9896 0 -1 4202560 62252 0 0 0 447937 83 0 0 25 0 8 0 549980290 3115474944 693460 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11968/tid=11974] ppid=11966 vsize=3042456 CPUtime=4555.11 cores=0-7
/proc/11968/task/11974/stat : 11974 (claspmt) R 11966 11968 9896 0 -1 4202560 59540 0 0 0 455448 63 0 0 25 0 8 0 549980290 3115474944 693460 33554432000 4194304 4964704 140734324619456 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11968/tid=11975] ppid=11966 vsize=3042456 CPUtime=4739.4 cores=0-7
/proc/11968/task/11975/stat : 11975 (claspmt) R 11966 11968 9896 0 -1 4202560 53523 0 0 0 473887 53 0 0 25 0 8 0 549980290 3115474944 693460 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=11968/tid=11976] ppid=11966 vsize=3042456 CPUtime=4620.18 cores=0-7
/proc/11968/task/11976/stat : 11976 (claspmt) R 11966 11968 9896 0 -1 4202560 65671 0 0 0 461932 86 0 0 25 0 8 0 549980290 3115474944 693460 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670215 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11968/tid=11977] ppid=11966 vsize=3042456 CPUtime=4036.61 cores=0-7
/proc/11968/task/11977/stat : 11977 (claspmt) R 11966 11968 9896 0 -1 4202560 147847 0 0 0 403476 185 0 0 25 0 8 0 549980290 3115474944 693460 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 35193.9
Current children cumulated vsize (KiB) 3042456

[startup+5082.3 s]
/proc/loadavg: 9.03 9.07 9.00 10/387 12634
/proc/meminfo: memFree=1737052/32950928 swapFree=67111448/67111528
[pid=11968] ppid=11966 vsize=3107992 CPUtime=35618.7 cores=0-7
/proc/11968/stat : 11968 (claspmt) R 11966 11968 9896 0 -1 4202496 711391 0 0 0 3561027 844 0 0 25 0 8 0 549979049 3182583808 699654 33554432000 4194304 4964704 140734324619456 18446744073709551615 251813937794 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/11968/statm: 776998 699654 466 189 0 771347 0
[pid=11968/tid=11971] ppid=11966 vsize=3107992 CPUtime=4499.24 cores=0-7
/proc/11968/task/11971/stat : 11971 (claspmt) R 11966 11968 9896 0 -1 4202560 78161 0 0 0 449834 90 0 0 25 0 8 0 549980290 3182583808 699654 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670193 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11968/tid=11972] ppid=11966 vsize=3107992 CPUtime=4521.08 cores=0-7
/proc/11968/task/11972/stat : 11972 (claspmt) R 11966 11968 9896 0 -1 4202560 93714 0 0 0 452005 103 0 0 25 0 8 0 549980290 3182583808 699654 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11968/tid=11973] ppid=11966 vsize=3107992 CPUtime=4540.19 cores=0-7
/proc/11968/task/11973/stat : 11973 (claspmt) R 11966 11968 9896 0 -1 4202560 62309 0 0 0 453936 83 0 0 25 0 8 0 549980290 3182583808 699654 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11968/tid=11974] ppid=11966 vsize=3107992 CPUtime=4615.1 cores=0-7
/proc/11968/task/11974/stat : 11974 (claspmt) R 11966 11968 9896 0 -1 4202560 59540 0 0 0 461447 63 0 0 25 0 8 0 549980290 3182583808 699654 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11968/tid=11975] ppid=11966 vsize=3107992 CPUtime=4789.67 cores=0-7
/proc/11968/task/11975/stat : 11975 (claspmt) R 11966 11968 9896 0 -1 4202560 53525 0 0 0 478914 53 0 0 25 0 8 0 549980290 3182583808 699654 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11968/tid=11976] ppid=11966 vsize=3107992 CPUtime=4678.7 cores=0-7
/proc/11968/task/11976/stat : 11976 (claspmt) R 11966 11968 9896 0 -1 4202560 65698 0 0 0 467784 86 0 0 25 0 8 0 549980290 3182583808 699654 33554432000 4194304 4964704 140734324619456 18446744073709551615 251814345198 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11968/tid=11977] ppid=11966 vsize=3107992 CPUtime=4079.07 cores=0-7
/proc/11968/task/11977/stat : 11977 (claspmt) R 11966 11968 9896 0 -1 4202560 151088 0 0 0 407714 193 0 0 25 0 8 0 549980290 3182583808 699654 33554432000 4194304 4964704 140734324619456 18446744073709551615 4660075 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 35618.7
Current children cumulated vsize (KiB) 3107992



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5100 s]
/proc/loadavg: 9.02 9.07 9.00 10/387 12634
/proc/meminfo: memFree=1736404/32950928 swapFree=67111448/67111528
[pid=11968] ppid=11966 vsize=3107992 CPUtime=35742.7 cores=0-7
/proc/11968/stat : 11968 (claspmt) R 11966 11968 9896 0 -1 4202496 711537 0 0 0 3573424 845 0 0 25 0 8 0 549979049 3182583808 699800 33554432000 4194304 4964704 140734324619456 18446744073709551615 4362248 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/11968/statm: 776998 699800 466 189 0 771347 0
[pid=11968/tid=11971] ppid=11966 vsize=3107992 CPUtime=4516.96 cores=0-7
/proc/11968/task/11971/stat : 11971 (claspmt) R 11966 11968 9896 0 -1 4202560 78161 0 0 0 451606 90 0 0 25 0 8 0 549980290 3182583808 699800 33554432000 4194304 4964704 140734324619456 18446744073709551615 4684027 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=11968/tid=11972] ppid=11966 vsize=3107992 CPUtime=4537.9 cores=0-7
/proc/11968/task/11972/stat : 11972 (claspmt) R 11966 11968 9896 0 -1 4202560 93859 0 0 0 453687 103 0 0 25 0 8 0 549980290 3182583808 699800 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=11968/tid=11973] ppid=11966 vsize=3107992 CPUtime=4557.91 cores=0-7
/proc/11968/task/11973/stat : 11973 (claspmt) R 11966 11968 9896 0 -1 4202560 62310 0 0 0 455708 83 0 0 25 0 8 0 549980290 3182583808 699800 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=11968/tid=11974] ppid=11966 vsize=3107992 CPUtime=4632.81 cores=0-7
/proc/11968/task/11974/stat : 11974 (claspmt) R 11966 11968 9896 0 -1 4202560 59540 0 0 0 463218 63 0 0 25 0 8 0 549980290 3182583808 699800 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=11968/tid=11975] ppid=11966 vsize=3107992 CPUtime=4807.39 cores=0-7
/proc/11968/task/11975/stat : 11975 (claspmt) R 11966 11968 9896 0 -1 4202560 53525 0 0 0 480686 53 0 0 25 0 8 0 549980290 3182583808 699800 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=11968/tid=11976] ppid=11966 vsize=3107992 CPUtime=4686.5 cores=0-7
/proc/11968/task/11976/stat : 11976 (claspmt) R 11966 11968 9896 0 -1 4202560 65698 0 0 0 468564 86 0 0 25 0 8 0 549980290 3182583808 699800 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670243 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=11968/tid=11977] ppid=11966 vsize=3107992 CPUtime=4089.87 cores=0-7
/proc/11968/task/11977/stat : 11977 (claspmt) R 11966 11968 9896 0 -1 4202560 151088 0 0 0 408794 193 0 0 25 0 8 0 549980290 3182583808 699800 33554432000 4194304 4964704 140734324619456 18446744073709551615 4670193 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 35742.7
Current children cumulated vsize (KiB) 3107992

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5100.6 s]
/proc/loadavg: 9.02 9.07 9.00 3/381 12635
/proc/meminfo: memFree=1749720/32950928 swapFree=67111448/67111528
[pid=11968] ppid=11966 vsize=3091328 CPUtime=35743.3 cores=0-7
/proc/11968/stat : 11968 (claspmt) R 11966 11968 9896 0 -1 4202496 711547 0 0 0 3573488 845 0 0 25 0 1 0 549979049 3165519872 695937 33554432000 4194304 4964704 140734324619456 18446744073709551615 251813896753 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/11968/statm: 772832 695937 476 189 0 767181 0
Current children cumulated CPU time (s) 35743.3
Current children cumulated vsize (KiB) 3091328

[startup+5100.8 s]
/proc/loadavg: 9.02 9.07 9.00 3/381 12635
/proc/meminfo: memFree=1749720/32950928 swapFree=67111448/67111528
[pid=11968] ppid=11966 vsize=1499092 CPUtime=35743.5 cores=0-7
/proc/11968/stat : 11968 (claspmt) R 11966 11968 9896 0 -1 4202496 711547 0 0 0 3573501 850 0 0 25 0 1 0 549979049 1535070208 298479 33554432000 4194304 4964704 140734324619456 18446744073709551615 251814283399 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/11968/statm: 374773 298023 476 189 0 369122 0
Current children cumulated CPU time (s) 35743.5
Current children cumulated vsize (KiB) 1499092

Child status: 0
Real time (s): 5100.85
CPU time (s): 35743.6
CPU user time (s): 35735
CPU system time (s): 8.5517
CPU usage (%): 700.737
Max. virtual memory (cumulated for all children) (KiB): 3107992

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 35735
system time used= 8.5517
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 711548
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 137709
involuntary context switches= 262267

runsolver used 18.2452 second user time and 46.6599 second system time

The end

Launcher Data 

Begin job on node126 at 2011-05-03 02:50:04
IDJOB=3322407
IDBENCH=83172
IDSOLVER=1711
FILE ID=node126/3322407-1304383803
RUNJOBID= node126-1304366466-9910
PBS_JOBID= 13181745
Free space on /tmp= 73592 MiB

SOLVER NAME= claspmt 2.0-R4095
BENCH NAME= SAT11/crafted/mosoi/289/289-unsat-17x30.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3322407-1304383803/watcher-3322407-1304383803 -o /tmp/evaluation-result-3322407-1304383803/solver-3322407-1304383803 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3322407-1304383803.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 3a506c7557d86d0d326e3b3cdee85b69
RANDOM SEED=1229813612

node126.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:       4536256 kB
Buffers:        354484 kB
Cached:        2042396 kB
SwapCached:         76 kB
Active:       26135784 kB
Inactive:      1868432 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:       4536256 kB
SwapTotal:    67111528 kB
SwapFree:     67111448 kB
Dirty:            6644 kB
Writeback:           0 kB
AnonPages:    25607972 kB
Mapped:          31204 kB
Slab:           264780 kB
PageTables:      75300 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 88549148 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73744 MiB
End job on node126 at 2011-05-03 04:15:06