Trace number 3373558

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 4800.51 1203.36

General information on the benchmark

Nameapplication/fuhs/slp-synthesis-AES/
slp-synthesis-aes-bottom21.cnf
MD5SUMe6407843d1fbd00b68d4ad600a25dfb2
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables48639
Number of clauses162901
Sum of the clauses size415741
Maximum clause length1542
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 283161
Number of clauses of size 378900
Number of clauses of size 4230
Number of clauses of size 519
Number of clauses of size over 5590

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...422916/instance-3373558-1305422916.cnf
0.39/0.47	c Solving...
4800.32/1203.21	
4800.32/1203.21	*** INTERRUPTED! ***
4800.32/1203.21	*** Shutting down threads...
4800.46/1203.35	s UNKNOWN
4800.46/1203.35	
4800.46/1203.35	c INTERRUPTED   : 1
4800.46/1203.35	c Models        : 0+    
4800.46/1203.35	c Time          : 1203.216s (Solving: 1202.75s 1st Model: 0.00s Unsat: 0.00s)
4800.46/1203.35	c CPU Time      : 4800.360s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3373558-1305422916/watcher-3373558-1305422916 -o /tmp/evaluation-result-3373558-1305422916/solver-3373558-1305422916 -C 4800 -W 1300 -M 15500 HOME/claspmt -t4 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3373558-1305422916.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.98 6.64 7.49 2/295 5403
/proc/meminfo: memFree=15814496/32950928 swapFree=67111364/67111528
[pid=5403] ppid=5401 vsize=27116 CPUtime=0 cores=1,3,5,7
/proc/5403/stat : 5403 (claspmt) R 5401 5403 4917 0 -1 4202496 1990 0 0 0 0 0 0 0 21 0 1 0 109900997 27766784 1921 33554432000 4194304 4964800 140733639622704 18446744073709551615 4700412 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/5403/statm: 6779 1929 375 189 0 1653 0

[startup+0.018676 s]
/proc/loadavg: 3.98 6.64 7.49 2/295 5403
/proc/meminfo: memFree=15814496/32950928 swapFree=67111364/67111528
[pid=5403] ppid=5401 vsize=31860 CPUtime=0.01 cores=1,3,5,7
/proc/5403/stat : 5403 (claspmt) R 5401 5403 4917 0 -1 4202496 3650 0 0 0 1 0 0 0 21 0 1 0 109900997 32624640 3111 33554432000 4194304 4964800 140733639622704 18446744073709551615 218880225989 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/5403/statm: 7965 3113 388 189 0 2839 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 31860

[startup+0.100664 s]
/proc/loadavg: 3.98 6.64 7.49 2/295 5403
/proc/meminfo: memFree=15814496/32950928 swapFree=67111364/67111528
[pid=5403] ppid=5401 vsize=45920 CPUtime=0.09 cores=1,3,5,7
/proc/5403/stat : 5403 (claspmt) R 5401 5403 4917 0 -1 4202496 8132 0 0 0 9 0 0 0 21 0 1 0 109900997 47022080 6464 33554432000 4194304 4964800 140733639622704 18446744073709551615 4607513 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/5403/statm: 11480 6464 402 189 0 6354 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 45920

[startup+0.300634 s]
/proc/loadavg: 3.98 6.64 7.49 2/295 5403
/proc/meminfo: memFree=15814496/32950928 swapFree=67111364/67111528
[pid=5403] ppid=5401 vsize=51332 CPUtime=0.29 cores=1,3,5,7
/proc/5403/stat : 5403 (claspmt) R 5401 5403 4917 0 -1 4202496 9503 0 0 0 28 1 0 0 22 0 1 0 109900997 52563968 7835 33554432000 4194304 4964800 140733639622704 18446744073709551615 4621769 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/5403/statm: 12833 7836 403 189 0 7707 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 51332

[startup+0.700482 s]
/proc/loadavg: 3.98 6.64 7.49 2/295 5403
/proc/meminfo: memFree=15814496/32950928 swapFree=67111364/67111528
[pid=5403] ppid=5401 vsize=206608 CPUtime=1.34 cores=1,3,5,7
/proc/5403/stat : 5403 (claspmt) R 5401 5403 4917 0 -1 4202496 16170 0 0 0 131 3 0 0 24 0 4 0 109900997 211566592 12577 33554432000 4194304 4964800 140733639622704 18446744073709551615 4668773 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/5403/statm: 51652 12577 458 189 0 46001 0
Current children cumulated CPU time (s) 1.34
Current children cumulated vsize (KiB) 206608

[startup+1.50034 s]
/proc/loadavg: 3.98 6.64 7.49 10/302 5410
/proc/meminfo: memFree=15742012/32950928 swapFree=67111364/67111528
[pid=5403] ppid=5401 vsize=206924 CPUtime=4.53 cores=1,3,5,7
/proc/5403/stat : 5403 (claspmt) R 5401 5403 4917 0 -1 4202496 17220 0 0 0 449 4 0 0 25 0 4 0 109900997 211890176 13627 33554432000 4194304 4964800 140733639622704 18446744073709551615 4668568 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/5403/statm: 51731 13627 458 189 0 46080 0
[pid=5403/tid=5408] ppid=5401 vsize=206924 CPUtime=1.01 cores=1,3,5,7
/proc/5403/task/5408/stat : 5408 (claspmt) R 5401 5403 4917 0 -1 4202560 1649 0 0 0 101 0 0 0 19 0 4 0 109901045 211890176 13627 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5403/tid=5409] ppid=5401 vsize=206924 CPUtime=1.01 cores=1,3,5,7
/proc/5403/task/5409/stat : 5409 (claspmt) R 5401 5403 4917 0 -1 4202560 2482 0 0 0 101 0 0 0 19 0 4 0 109901045 211890176 13627 33554432000 4194304 4964800 140733639622704 18446744073709551615 4668814 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5403/tid=5410] ppid=5401 vsize=206924 CPUtime=1.01 cores=1,3,5,7
/proc/5403/task/5410/stat : 5410 (claspmt) R 5401 5403 4917 0 -1 4202560 2613 0 0 0 100 1 0 0 19 0 4 0 109901045 211890176 13627 33554432000 4194304 4964800 140733639622704 18446744073709551615 4668773 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4.53
Current children cumulated vsize (KiB) 206924

[startup+3.10106 s]
/proc/loadavg: 3.98 6.64 7.49 9/302 5410
/proc/meminfo: memFree=15727380/32950928 swapFree=67111364/67111528
[pid=5403] ppid=5401 vsize=210308 CPUtime=10.92 cores=1,3,5,7
/proc/5403/stat : 5403 (claspmt) R 5401 5403 4917 0 -1 4202496 19931 0 0 0 1087 5 0 0 25 0 4 0 109900997 215355392 16338 33554432000 4194304 4964800 140733639622704 18446744073709551615 4668552 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/5403/statm: 52577 16338 458 189 0 46926 0
[pid=5403/tid=5408] ppid=5401 vsize=210308 CPUtime=2.61 cores=1,3,5,7
/proc/5403/task/5408/stat : 5408 (claspmt) R 5401 5403 4917 0 -1 4202560 2089 0 0 0 261 0 0 0 22 0 4 0 109901045 215355392 16338 33554432000 4194304 4964800 140733639622704 18446744073709551615 4668689 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5403/tid=5409] ppid=5401 vsize=210308 CPUtime=2.61 cores=1,3,5,7
/proc/5403/task/5409/stat : 5409 (claspmt) R 5401 5403 4917 0 -1 4202560 3814 0 0 0 261 0 0 0 23 0 4 0 109901045 215355392 16338 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5403/tid=5410] ppid=5401 vsize=210308 CPUtime=2.59 cores=1,3,5,7
/proc/5403/task/5410/stat : 5410 (claspmt) R 5401 5403 4917 0 -1 4202560 3176 0 0 0 258 1 0 0 22 0 4 0 109901045 215355392 16339 33554432000 4194304 4964800 140733639622704 18446744073709551615 4668672 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 10.92
Current children cumulated vsize (KiB) 210308

[startup+6.3005 s]
/proc/loadavg: 4.30 6.66 7.49 9/302 5410
/proc/meminfo: memFree=15692232/32950928 swapFree=67111364/67111528
[pid=5403] ppid=5401 vsize=214108 CPUtime=23.68 cores=1,3,5,7
/proc/5403/stat : 5403 (claspmt) R 5401 5403 4917 0 -1 4202496 23649 0 0 0 2361 7 0 0 25 0 4 0 109900997 219246592 20056 33554432000 4194304 4964800 140733639622704 18446744073709551615 4653316 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/5403/statm: 53527 20056 458 189 0 47876 0
[pid=5403/tid=5408] ppid=5401 vsize=214108 CPUtime=5.81 cores=1,3,5,7
/proc/5403/task/5408/stat : 5408 (claspmt) R 5401 5403 4917 0 -1 4202560 2547 0 0 0 580 1 0 0 25 0 4 0 109901045 219246592 20056 33554432000 4194304 4964800 140733639622704 18446744073709551615 4660449 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5403/tid=5409] ppid=5401 vsize=214108 CPUtime=5.81 cores=1,3,5,7
/proc/5403/task/5409/stat : 5409 (claspmt) R 5401 5403 4917 0 -1 4202560 5309 0 0 0 580 1 0 0 25 0 4 0 109901045 219246592 20056 33554432000 4194304 4964800 140733639622704 18446744073709551615 4668734 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5403/tid=5410] ppid=5401 vsize=214108 CPUtime=5.76 cores=1,3,5,7
/proc/5403/task/5410/stat : 5410 (claspmt) R 5401 5403 4917 0 -1 4202560 4165 0 0 0 574 2 0 0 25 0 4 0 109901045 219246592 20056 33554432000 4194304 4964800 140733639622704 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 23.68
Current children cumulated vsize (KiB) 214108

[startup+12.7004 s]
/proc/loadavg: 4.59 6.68 7.50 9/302 5410
/proc/meminfo: memFree=15661540/32950928 swapFree=67111364/67111528
[pid=5403] ppid=5401 vsize=286408 CPUtime=49.22 cores=1,3,5,7
/proc/5403/stat : 5403 (claspmt) R 5401 5403 4917 0 -1 4202496 27742 0 0 0 4911 11 0 0 25 0 4 0 109900997 293281792 24149 33554432000 4194304 4964800 140733639622704 18446744073709551615 4332355 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/5403/statm: 71602 24149 460 189 0 65951 0
[pid=5403/tid=5408] ppid=5401 vsize=286408 CPUtime=12.21 cores=1,3,5,7
/proc/5403/task/5408/stat : 5408 (claspmt) R 5401 5403 4917 0 -1 4202560 2948 0 0 0 1220 1 0 0 25 0 4 0 109901045 293281792 24149 33554432000 4194304 4964800 140733639622704 18446744073709551615 4653504 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5403/tid=5409] ppid=5401 vsize=286408 CPUtime=12.22 cores=1,3,5,7
/proc/5403/task/5409/stat : 5409 (claspmt) R 5401 5403 4917 0 -1 4202560 7043 0 0 0 1219 3 0 0 25 0 4 0 109901045 293281792 24149 33554432000 4194304 4964800 140733639622704 18446744073709551615 218880225989 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5403/tid=5410] ppid=5401 vsize=286408 CPUtime=12.1 cores=1,3,5,7
/proc/5403/task/5410/stat : 5410 (claspmt) R 5401 5403 4917 0 -1 4202560 4689 0 0 0 1208 2 0 0 25 0 4 0 109901045 293281792 24149 33554432000 4194304 4964800 140733639622704 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 49.22
Current children cumulated vsize (KiB) 286408

[startup+25.5011 s]
/proc/loadavg: 5.35 6.75 7.50 9/302 5410
/proc/meminfo: memFree=15586156/32950928 swapFree=67111364/67111528
[pid=5403] ppid=5401 vsize=292640 CPUtime=100.31 cores=1,3,5,7
/proc/5403/stat : 5403 (claspmt) R 5401 5403 4917 0 -1 4202496 39306 0 0 0 10012 19 0 0 25 0 4 0 109900997 299663360 35713 33554432000 4194304 4964800 140733639622704 18446744073709551615 4342144 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/5403/statm: 73160 35713 460 189 0 67509 0
[pid=5403/tid=5408] ppid=5401 vsize=292640 CPUtime=25.01 cores=1,3,5,7
/proc/5403/task/5408/stat : 5408 (claspmt) R 5401 5403 4917 0 -1 4202560 4197 0 0 0 2499 2 0 0 25 0 4 0 109901045 299663360 35713 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670323 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5403/tid=5409] ppid=5401 vsize=292640 CPUtime=25.02 cores=1,3,5,7
/proc/5403/task/5409/stat : 5409 (claspmt) R 5401 5403 4917 0 -1 4202560 13453 0 0 0 2495 7 0 0 25 0 4 0 109901045 299663360 35713 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5403/tid=5410] ppid=5401 vsize=292640 CPUtime=24.79 cores=1,3,5,7
/proc/5403/task/5410/stat : 5410 (claspmt) R 5401 5403 4917 0 -1 4202560 6839 0 0 0 2475 4 0 0 25 0 4 0 109901045 299663360 35713 33554432000 4194304 4964800 140733639622704 18446744073709551615 4653491 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 100.31
Current children cumulated vsize (KiB) 292640

[startup+51.1007 s]
/proc/loadavg: 6.25 6.85 7.52 9/302 5411
/proc/meminfo: memFree=15504052/32950928 swapFree=67111364/67111528
[pid=5403] ppid=5401 vsize=297612 CPUtime=202.47 cores=1,3,5,7
/proc/5403/stat : 5403 (claspmt) R 5401 5403 4917 0 -1 4202496 46692 0 0 0 20222 25 0 0 25 0 4 0 109900997 304754688 43099 33554432000 4194304 4964800 140733639622704 18446744073709551615 4332324 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/5403/statm: 74403 43099 460 189 0 68752 0
[pid=5403/tid=5408] ppid=5401 vsize=297612 CPUtime=50.61 cores=1,3,5,7
/proc/5403/task/5408/stat : 5408 (claspmt) R 5401 5403 4917 0 -1 4202560 6182 0 0 0 5057 4 0 0 25 0 4 0 109901045 304754688 43099 33554432000 4194304 4964800 140733639622704 18446744073709551615 4668734 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5403/tid=5409] ppid=5401 vsize=297612 CPUtime=50.62 cores=1,3,5,7
/proc/5403/task/5409/stat : 5409 (claspmt) R 5401 5403 4917 0 -1 4202560 16500 0 0 0 5053 9 0 0 25 0 4 0 109901045 304754688 43099 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5403/tid=5410] ppid=5401 vsize=297612 CPUtime=50.16 cores=1,3,5,7
/proc/5403/task/5410/stat : 5410 (claspmt) R 5401 5403 4917 0 -1 4202560 7887 0 0 0 5010 6 0 0 25 0 4 0 109901045 304754688 43099 33554432000 4194304 4964800 140733639622704 18446744073709551615 4668777 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 202.47
Current children cumulated vsize (KiB) 297612

[startup+102.301 s]
/proc/loadavg: 7.29 7.04 7.54 9/302 5412
/proc/meminfo: memFree=15322508/32950928 swapFree=67111364/67111528
[pid=5403] ppid=5401 vsize=414872 CPUtime=406.84 cores=1,3,5,7
/proc/5403/stat : 5403 (claspmt) R 5401 5403 4917 0 -1 4202496 77268 0 0 0 40635 49 0 0 25 0 4 0 109900997 424828928 73670 33554432000 4194304 4964800 140733639622704 18446744073709551615 4321102 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/5403/statm: 103718 73670 460 189 0 98067 0
[pid=5403/tid=5408] ppid=5401 vsize=414872 CPUtime=101.81 cores=1,3,5,7
/proc/5403/task/5408/stat : 5408 (claspmt) R 5401 5403 4917 0 -1 4202560 12122 0 0 0 10172 9 0 0 25 0 4 0 109901045 424828928 73670 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5403/tid=5409] ppid=5401 vsize=414872 CPUtime=101.82 cores=1,3,5,7
/proc/5403/task/5409/stat : 5409 (claspmt) R 5401 5403 4917 0 -1 4202560 31723 0 0 0 10164 18 0 0 25 0 4 0 109901045 424828928 73670 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5403/tid=5410] ppid=5401 vsize=414872 CPUtime=100.89 cores=1,3,5,7
/proc/5403/task/5410/stat : 5410 (claspmt) R 5401 5403 4917 0 -1 4202560 12908 0 0 0 10079 10 0 0 25 0 4 0 109901045 424828928 73670 33554432000 4194304 4964800 140733639622704 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 406.84
Current children cumulated vsize (KiB) 414872

[startup+162.301 s]
/proc/loadavg: 7.79 7.23 7.57 9/302 5414
/proc/meminfo: memFree=15236816/32950928 swapFree=67111364/67111528
[pid=5403] ppid=5401 vsize=480408 CPUtime=646.25 cores=1,3,5,7
/proc/5403/stat : 5403 (claspmt) R 5401 5403 4917 0 -1 4202496 91375 0 0 0 64556 69 0 0 25 0 4 0 109900997 491937792 87777 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/5403/statm: 120102 87777 460 189 0 114451 0
[pid=5403/tid=5408] ppid=5401 vsize=480408 CPUtime=161.81 cores=1,3,5,7
/proc/5403/task/5408/stat : 5408 (claspmt) R 5401 5403 4917 0 -1 4202560 12442 0 0 0 16171 10 0 0 25 0 4 0 109901045 491937792 87777 33554432000 4194304 4964800 140733639622704 18446744073709551615 4668797 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5403/tid=5409] ppid=5401 vsize=480408 CPUtime=161.81 cores=1,3,5,7
/proc/5403/task/5409/stat : 5409 (claspmt) R 5401 5403 4917 0 -1 4202560 43038 0 0 0 16147 34 0 0 25 0 4 0 109901045 491937792 87777 33554432000 4194304 4964800 140733639622704 18446744073709551615 4669574 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5403/tid=5410] ppid=5401 vsize=480408 CPUtime=160.34 cores=1,3,5,7
/proc/5403/task/5410/stat : 5410 (claspmt) R 5401 5403 4917 0 -1 4202560 13360 0 0 0 16023 11 0 0 25 0 4 0 109901045 491937792 87777 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 646.25
Current children cumulated vsize (KiB) 480408

[startup+222.312 s]
/proc/loadavg: 7.97 7.38 7.60 9/302 5416
/proc/meminfo: memFree=15134008/32950928 swapFree=67111364/67111528
[pid=5403] ppid=5401 vsize=612568 CPUtime=885.75 cores=1,3,5,7
/proc/5403/stat : 5403 (claspmt) R 5401 5403 4917 0 -1 4202496 113277 0 0 0 88478 97 0 0 25 0 4 0 109900997 627269632 109499 33554432000 4194304 4964800 140733639622704 18446744073709551615 4668712 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/5403/statm: 153142 109499 460 189 0 147491 0
[pid=5403/tid=5408] ppid=5401 vsize=612568 CPUtime=221.81 cores=1,3,5,7
/proc/5403/task/5408/stat : 5408 (claspmt) R 5401 5403 4917 0 -1 4202560 13172 0 0 0 22171 10 0 0 25 0 4 0 109901045 627269632 109499 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5403/tid=5409] ppid=5401 vsize=612568 CPUtime=221.83 cores=1,3,5,7
/proc/5403/task/5409/stat : 5409 (claspmt) R 5401 5403 4917 0 -1 4202560 60673 0 0 0 22125 58 0 0 25 0 4 0 109901045 627269632 109499 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5403/tid=5410] ppid=5401 vsize=612568 CPUtime=219.8 cores=1,3,5,7
/proc/5403/task/5410/stat : 5410 (claspmt) R 5401 5403 4917 0 -1 4202560 16222 0 0 0 21966 14 0 0 25 0 4 0 109901045 627269632 109499 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 885.75
Current children cumulated vsize (KiB) 612568

[startup+282.3 s]

################
# More data... #
################

Current children cumulated CPU time (s) 2561.86
Current children cumulated vsize (KiB) 800076

[startup+702.301 s]
/proc/loadavg: 8.22 7.98 7.80 9/302 5428
/proc/meminfo: memFree=14825508/32950928 swapFree=67111364/67111528
[pid=5403] ppid=5401 vsize=800076 CPUtime=2801.32 cores=1,3,5,7
/proc/5403/stat : 5403 (claspmt) R 5401 5403 4917 0 -1 4202496 165489 0 0 0 279969 163 0 0 25 0 4 0 109900997 819277824 161263 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670534 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/5403/statm: 200019 161263 460 189 0 194368 0
[pid=5403/tid=5408] ppid=5401 vsize=800076 CPUtime=701.77 cores=1,3,5,7
/proc/5403/task/5408/stat : 5408 (claspmt) R 5401 5403 4917 0 -1 4202560 27939 0 0 0 70148 29 0 0 25 0 4 0 109901045 819277824 161263 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5403/tid=5409] ppid=5401 vsize=800076 CPUtime=701.81 cores=1,3,5,7
/proc/5403/task/5409/stat : 5409 (claspmt) R 5401 5403 4917 0 -1 4202560 74290 0 0 0 70110 71 0 0 25 0 4 0 109901045 819277824 161263 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5403/tid=5410] ppid=5401 vsize=800076 CPUtime=695.42 cores=1,3,5,7
/proc/5403/task/5410/stat : 5410 (claspmt) R 5401 5403 4917 0 -1 4202560 32878 0 0 0 69502 40 0 0 25 0 4 0 109901045 819277824 161263 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2801.32
Current children cumulated vsize (KiB) 800076

[startup+762.301 s]
/proc/loadavg: 8.22 8.01 7.82 9/302 5430
/proc/meminfo: memFree=14813092/32950928 swapFree=67111364/67111528
[pid=5403] ppid=5401 vsize=810460 CPUtime=3040.79 cores=1,3,5,7
/proc/5403/stat : 5403 (claspmt) R 5401 5403 4917 0 -1 4202496 168277 0 0 0 303913 166 0 0 25 0 4 0 109900997 829911040 163902 33554432000 4194304 4964800 140733639622704 18446744073709551615 4668930 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/5403/statm: 202615 163902 460 189 0 196964 0
[pid=5403/tid=5408] ppid=5401 vsize=810460 CPUtime=761.78 cores=1,3,5,7
/proc/5403/task/5408/stat : 5408 (claspmt) R 5401 5403 4917 0 -1 4202560 28139 0 0 0 76148 30 0 0 25 0 4 0 109901045 829911040 163902 33554432000 4194304 4964800 140733639622704 18446744073709551615 4653373 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5403/tid=5409] ppid=5401 vsize=810460 CPUtime=761.83 cores=1,3,5,7
/proc/5403/task/5409/stat : 5409 (claspmt) R 5401 5403 4917 0 -1 4202560 76767 0 0 0 76109 74 0 0 25 0 4 0 109901045 829911040 163902 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5403/tid=5410] ppid=5401 vsize=810460 CPUtime=754.88 cores=1,3,5,7
/proc/5403/task/5410/stat : 5410 (claspmt) R 5401 5403 4917 0 -1 4202560 32988 0 0 0 75448 40 0 0 25 0 4 0 109901045 829911040 163902 33554432000 4194304 4964800 140733639622704 18446744073709551615 4668698 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3040.79
Current children cumulated vsize (KiB) 810460

[startup+822.3 s]
/proc/loadavg: 8.15 8.02 7.83 9/302 5432
/proc/meminfo: memFree=14806256/32950928 swapFree=67111364/67111528
[pid=5403] ppid=5401 vsize=813456 CPUtime=3280.22 cores=1,3,5,7
/proc/5403/stat : 5403 (claspmt) R 5401 5403 4917 0 -1 4202496 169302 0 0 0 327855 167 0 0 25 0 4 0 109900997 832978944 164927 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/5403/statm: 203364 164927 460 189 0 197713 0
[pid=5403/tid=5408] ppid=5401 vsize=813456 CPUtime=821.77 cores=1,3,5,7
/proc/5403/task/5408/stat : 5408 (claspmt) R 5401 5403 4917 0 -1 4202560 28382 0 0 0 82147 30 0 0 25 0 4 0 109901045 832978944 164927 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5403/tid=5409] ppid=5401 vsize=813456 CPUtime=821.82 cores=1,3,5,7
/proc/5403/task/5409/stat : 5409 (claspmt) R 5401 5403 4917 0 -1 4202560 77139 0 0 0 82108 74 0 0 25 0 4 0 109901045 832978944 164927 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5403/tid=5410] ppid=5401 vsize=813456 CPUtime=814.32 cores=1,3,5,7
/proc/5403/task/5410/stat : 5410 (claspmt) R 5401 5403 4917 0 -1 4202560 33189 0 0 0 81392 40 0 0 25 0 4 0 109901045 832978944 164927 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3280.22
Current children cumulated vsize (KiB) 813456

[startup+882.312 s]
/proc/loadavg: 8.17 8.05 7.85 9/302 5433
/proc/meminfo: memFree=14802152/32950928 swapFree=67111364/67111528
[pid=5403] ppid=5401 vsize=813456 CPUtime=3519.7 cores=1,3,5,7
/proc/5403/stat : 5403 (claspmt) R 5401 5403 4917 0 -1 4202496 169674 0 0 0 351802 168 0 0 25 0 4 0 109900997 832978944 165131 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670534 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/5403/statm: 203364 165131 460 189 0 197713 0
[pid=5403/tid=5408] ppid=5401 vsize=813456 CPUtime=881.76 cores=1,3,5,7
/proc/5403/task/5408/stat : 5408 (claspmt) R 5401 5403 4917 0 -1 4202560 28703 0 0 0 88146 30 0 0 25 0 4 0 109901045 832978944 165131 33554432000 4194304 4964800 140733639622704 18446744073709551615 4669987 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5403/tid=5409] ppid=5401 vsize=813456 CPUtime=881.83 cores=1,3,5,7
/proc/5403/task/5409/stat : 5409 (claspmt) R 5401 5403 4917 0 -1 4202560 77190 0 0 0 88109 74 0 0 25 0 4 0 109901045 832978944 165131 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670534 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5403/tid=5410] ppid=5401 vsize=813456 CPUtime=873.78 cores=1,3,5,7
/proc/5403/task/5410/stat : 5410 (claspmt) R 5401 5403 4917 0 -1 4202560 33189 0 0 0 87338 40 0 0 25 0 4 0 109901045 832978944 165131 33554432000 4194304 4964800 140733639622704 18446744073709551615 4668770 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3519.7
Current children cumulated vsize (KiB) 813456

[startup+942.3 s]
/proc/loadavg: 8.17 8.07 7.86 9/302 5435
/proc/meminfo: memFree=14781172/32950928 swapFree=67111364/67111528
[pid=5403] ppid=5401 vsize=878992 CPUtime=3759.11 cores=1,3,5,7
/proc/5403/stat : 5403 (claspmt) R 5401 5403 4917 0 -1 4202496 174505 0 0 0 375735 176 0 0 25 0 4 0 109900997 900087808 169962 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670180 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/5403/statm: 219748 169962 460 189 0 214097 0
[pid=5403/tid=5408] ppid=5401 vsize=878992 CPUtime=941.76 cores=1,3,5,7
/proc/5403/task/5408/stat : 5408 (claspmt) R 5401 5403 4917 0 -1 4202560 31219 0 0 0 94141 35 0 0 25 0 4 0 109901045 900087808 169962 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5403/tid=5409] ppid=5401 vsize=878992 CPUtime=941.82 cores=1,3,5,7
/proc/5403/task/5409/stat : 5409 (claspmt) R 5401 5403 4917 0 -1 4202560 79250 0 0 0 94104 78 0 0 25 0 4 0 109901045 900087808 169962 33554432000 4194304 4964800 140733639622704 18446744073709551615 4660165 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5403/tid=5410] ppid=5401 vsize=878992 CPUtime=933.23 cores=1,3,5,7
/proc/5403/task/5410/stat : 5410 (claspmt) R 5401 5403 4917 0 -1 4202560 33443 0 0 0 93282 41 0 0 25 0 4 0 109901045 900087808 169962 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3759.11
Current children cumulated vsize (KiB) 878992

[startup+1002.3 s]
/proc/loadavg: 8.16 8.08 7.88 9/302 5436
/proc/meminfo: memFree=14772480/32950928 swapFree=67111364/67111528
[pid=5403] ppid=5401 vsize=878992 CPUtime=3998.54 cores=1,3,5,7
/proc/5403/stat : 5403 (claspmt) R 5401 5403 4917 0 -1 4202496 174968 0 0 0 399677 177 0 0 25 0 4 0 109900997 900087808 170425 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/5403/statm: 219748 170425 460 189 0 214097 0
[pid=5403/tid=5408] ppid=5401 vsize=878992 CPUtime=1001.73 cores=1,3,5,7
/proc/5403/task/5408/stat : 5408 (claspmt) R 5401 5403 4917 0 -1 4202560 31223 0 0 0 100138 35 0 0 25 0 4 0 109901045 900087808 170425 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5403/tid=5409] ppid=5401 vsize=878992 CPUtime=1001.82 cores=1,3,5,7
/proc/5403/task/5409/stat : 5409 (claspmt) R 5401 5403 4917 0 -1 4202560 79285 0 0 0 100104 78 0 0 25 0 4 0 109901045 900087808 170425 33554432000 4194304 4964800 140733639622704 18446744073709551615 4668773 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5403/tid=5410] ppid=5401 vsize=878992 CPUtime=992.67 cores=1,3,5,7
/proc/5403/task/5410/stat : 5410 (claspmt) R 5401 5403 4917 0 -1 4202560 33688 0 0 0 99226 41 0 0 25 0 4 0 109901045 900087808 170425 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3998.54
Current children cumulated vsize (KiB) 878992

[startup+1062.3 s]
/proc/loadavg: 8.20 8.11 7.90 9/302 5438
/proc/meminfo: memFree=14768376/32950928 swapFree=67111364/67111528
[pid=5403] ppid=5401 vsize=878992 CPUtime=4237.99 cores=1,3,5,7
/proc/5403/stat : 5403 (claspmt) R 5401 5403 4917 0 -1 4202496 175150 0 0 0 423622 177 0 0 25 0 4 0 109900997 900087808 170568 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/5403/statm: 219748 170568 460 189 0 214097 0
[pid=5403/tid=5408] ppid=5401 vsize=878992 CPUtime=1061.73 cores=1,3,5,7
/proc/5403/task/5408/stat : 5408 (claspmt) R 5401 5403 4917 0 -1 4202560 31357 0 0 0 106138 35 0 0 25 0 4 0 109901045 900087808 170568 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670499 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5403/tid=5409] ppid=5401 vsize=878992 CPUtime=1061.82 cores=1,3,5,7
/proc/5403/task/5409/stat : 5409 (claspmt) R 5401 5403 4917 0 -1 4202560 79321 0 0 0 106104 78 0 0 25 0 4 0 109901045 900087808 170568 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5403/tid=5410] ppid=5401 vsize=878992 CPUtime=1052.14 cores=1,3,5,7
/proc/5403/task/5410/stat : 5410 (claspmt) R 5401 5403 4917 0 -1 4202560 33699 0 0 0 105172 42 0 0 25 0 4 0 109901045 900087808 170568 33554432000 4194304 4964800 140733639622704 18446744073709551615 4320966 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4237.99
Current children cumulated vsize (KiB) 878992

[startup+1122.3 s]
/proc/loadavg: 8.16 8.12 7.91 9/302 5440
/proc/meminfo: memFree=14757948/32950928 swapFree=67111364/67111528
[pid=5403] ppid=5401 vsize=878992 CPUtime=4477.44 cores=1,3,5,7
/proc/5403/stat : 5403 (claspmt) R 5401 5403 4917 0 -1 4202496 175370 0 0 0 447566 178 0 0 25 0 4 0 109900997 900087808 170614 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670357 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/5403/statm: 219748 170614 460 189 0 214097 0
[pid=5403/tid=5408] ppid=5401 vsize=878992 CPUtime=1121.72 cores=1,3,5,7
/proc/5403/task/5408/stat : 5408 (claspmt) R 5401 5403 4917 0 -1 4202560 31571 0 0 0 112137 35 0 0 25 0 4 0 109901045 900087808 170614 33554432000 4194304 4964800 140733639622704 18446744073709551615 4334298 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5403/tid=5409] ppid=5401 vsize=878992 CPUtime=1121.82 cores=1,3,5,7
/proc/5403/task/5409/stat : 5409 (claspmt) R 5401 5403 4917 0 -1 4202560 79323 0 0 0 112104 78 0 0 25 0 4 0 109901045 900087808 170614 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5403/tid=5410] ppid=5401 vsize=878992 CPUtime=1111.59 cores=1,3,5,7
/proc/5403/task/5410/stat : 5410 (claspmt) R 5401 5403 4917 0 -1 4202560 33702 0 0 0 111117 42 0 0 25 0 4 0 109901045 900087808 170614 33554432000 4194304 4964800 140733639622704 18446744073709551615 4320953 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4477.44
Current children cumulated vsize (KiB) 878992

[startup+1182.3 s]
/proc/loadavg: 8.09 8.11 7.92 9/302 5441
/proc/meminfo: memFree=14717384/32950928 swapFree=67111364/67111528
[pid=5403] ppid=5401 vsize=911788 CPUtime=4716.89 cores=1,3,5,7
/proc/5403/stat : 5403 (claspmt) R 5401 5403 4917 0 -1 4202496 183468 0 0 0 471504 185 0 0 25 0 4 0 109900997 933670912 178712 33554432000 4194304 4964800 140733639622704 18446744073709551615 4668568 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/5403/statm: 227947 178712 460 189 0 222296 0
[pid=5403/tid=5408] ppid=5401 vsize=911788 CPUtime=1181.73 cores=1,3,5,7
/proc/5403/task/5408/stat : 5408 (claspmt) R 5401 5403 4917 0 -1 4202560 33268 0 0 0 118136 37 0 0 25 0 4 0 109901045 933670912 178712 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5403/tid=5409] ppid=5401 vsize=911788 CPUtime=1181.82 cores=1,3,5,7
/proc/5403/task/5409/stat : 5409 (claspmt) R 5401 5403 4917 0 -1 4202560 83470 0 0 0 118100 82 0 0 25 0 4 0 109901045 933670912 178713 33554432000 4194304 4964800 140733639622704 18446744073709551615 218880225989 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5403/tid=5410] ppid=5401 vsize=911788 CPUtime=1171.05 cores=1,3,5,7
/proc/5403/task/5410/stat : 5410 (claspmt) R 5401 5403 4917 0 -1 4202560 35178 0 0 0 117062 43 0 0 25 0 4 0 109901045 933670912 178713 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670357 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4716.89
Current children cumulated vsize (KiB) 911788



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.2 s]
/proc/loadavg: 8.20 8.13 7.93 5/298 5443
/proc/meminfo: memFree=15109248/32950928 swapFree=67111364/67111528
[pid=5403] ppid=5401 vsize=916492 CPUtime=4800.32 cores=1,3,5,7
/proc/5403/stat : 5403 (claspmt) R 5401 5403 4917 0 -1 4202496 184623 0 0 0 479846 186 0 0 25 0 4 0 109900997 938487808 179867 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/5403/statm: 229123 179867 460 189 0 223472 0
[pid=5403/tid=5408] ppid=5401 vsize=916492 CPUtime=1202.64 cores=1,3,5,7
/proc/5403/task/5408/stat : 5408 (claspmt) R 5401 5403 4917 0 -1 4202560 33268 0 0 0 120227 37 0 0 25 0 4 0 109901045 938487808 179867 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=5403/tid=5409] ppid=5401 vsize=916492 CPUtime=1202.72 cores=1,3,5,7
/proc/5403/task/5409/stat : 5409 (claspmt) R 5401 5403 4917 0 -1 4202560 84592 0 0 0 120190 82 0 0 25 0 4 0 109901045 938487808 179867 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=5403/tid=5410] ppid=5401 vsize=916492 CPUtime=1191.75 cores=1,3,5,7
/proc/5403/task/5410/stat : 5410 (claspmt) R 5401 5403 4917 0 -1 4202560 35194 0 0 0 119132 43 0 0 25 0 4 0 109901045 938487808 179867 33554432000 4194304 4964800 140733639622704 18446744073709551615 4670546 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4800.32
Current children cumulated vsize (KiB) 916492

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.32 s]
/proc/loadavg: 8.20 8.13 7.93 5/298 5443
/proc/meminfo: memFree=15109248/32950928 swapFree=67111364/67111528
[pid=5403] ppid=5401 vsize=903408 CPUtime=4800.46 cores=1,3,5,7
/proc/5403/stat : 5403 (claspmt) R 5401 5403 4917 0 -1 4202496 184633 0 0 0 479860 186 0 0 25 0 1 0 109900997 925089792 176665 33554432000 4194304 4964800 140733639622704 18446744073709551615 218880220363 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/5403/statm: 225852 176665 470 189 0 220201 0
Current children cumulated CPU time (s) 4800.46
Current children cumulated vsize (KiB) 903408

Child status: 0
Real time (s): 1203.36
CPU time (s): 4800.51
CPU user time (s): 4798.61
CPU system time (s): 1.89371
CPU usage (%): 398.925
Max. virtual memory (cumulated for all children) (KiB): 916492

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4798.61
system time used= 1.89371
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 184635
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 7691
involuntary context switches= 33294

runsolver used 2.98455 second user time and 7.97879 second system time

The end

Launcher Data 

Begin job on node112 at 2011-05-15 03:28:36
IDJOB=3373558
IDBENCH=83135
IDSOLVER=1853
FILE ID=node112/3373558-1305422916
RUNJOBID= node112-1305422915-5384
PBS_JOBID= 13324570
Free space on /tmp= 73560 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT11/application/fuhs/slp-synthesis-AES/slp-synthesis-aes-bottom21.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3373558-1305422916/watcher-3373558-1305422916 -o /tmp/evaluation-result-3373558-1305422916/solver-3373558-1305422916 -C 4800 -W 1300 -M 15500  HOME/claspmt -t4 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3373558-1305422916.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= e6407843d1fbd00b68d4ad600a25dfb2
RANDOM SEED=1557156595

node112.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5180.45
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      15814900 kB
Buffers:        293472 kB
Cached:        2830660 kB
SwapCached:          0 kB
Active:       15114276 kB
Inactive:      1782908 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      15814900 kB
SwapTotal:    67111528 kB
SwapFree:     67111364 kB
Dirty:            6272 kB
Writeback:           0 kB
AnonPages:    13772472 kB
Mapped:          21628 kB
Slab:           133692 kB
PageTables:      42228 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 35024496 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73560 MiB
End job on node112 at 2011-05-15 03:48:41