Trace number 3373579

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 4800.45 1203.44

General information on the benchmark

Nameapplication/kullmann/AES/Benchmarks/
64/aes_64_2_keyfind_1.cnf
MD5SUM1024a3346322fdba0a20b51f99eae4c0
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables1000
Number of clauses5176
Sum of the clauses size20024
Maximum clause length5
Minimum clause length1
Number of clauses of size 1136
Number of clauses of size 2160
Number of clauses of size 32080
Number of clauses of size 4672
Number of clauses of size 52128
Number of clauses of size over 50

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...427927/instance-3373579-1305427927.cnf
0.00/0.01	c Solving...
4800.40/1203.43	
4800.40/1203.43	*** INTERRUPTED! ***
4800.40/1203.43	*** Shutting down threads...
4800.40/1203.44	s UNKNOWN
4800.40/1203.44	
4800.40/1203.44	c INTERRUPTED   : 1
4800.40/1203.44	c Models        : 0+    
4800.40/1203.44	c Time          : 1203.427s (Solving: 1203.42s 1st Model: 0.00s Unsat: 0.00s)
4800.40/1203.44	c CPU Time      : 4800.430s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3373579-1305427927/watcher-3373579-1305427927 -o /tmp/evaluation-result-3373579-1305427927/solver-3373579-1305427927 -C 4800 -W 1300 -M 15500 HOME/claspmt -t4 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3373579-1305427927.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.91 8.09 8.34 5/354 18902
/proc/meminfo: memFree=17561936/32950928 swapFree=67111200/67111528
[pid=18902] ppid=18900 vsize=21612 CPUtime=0 cores=1,3,5,7
/proc/18902/stat : 18902 (claspmt) R 18900 18902 10053 0 -1 4202496 728 0 0 0 0 0 0 0 25 0 1 0 733023397 22130688 643 33554432000 4194304 4964800 140734855717904 18446744073709551615 256415052485 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/18902/statm: 5403 643 403 189 0 277 0

[startup+0.127582 s]
/proc/loadavg: 7.91 8.09 8.34 5/354 18902
/proc/meminfo: memFree=17561936/32950928 swapFree=67111200/67111528
[pid=18902] ppid=18900 vsize=169272 CPUtime=0.44 cores=1,3,5,7
/proc/18902/stat : 18902 (claspmt) R 18900 18902 10053 0 -1 4202496 1540 0 0 0 44 0 0 0 25 0 4 0 733023397 173334528 1423 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670328 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/18902/statm: 42318 1423 458 189 0 36667 0
Current children cumulated CPU time (s) 0.44
Current children cumulated vsize (KiB) 169272

[startup+0.200568 s]
/proc/loadavg: 7.91 8.09 8.34 5/354 18902
/proc/meminfo: memFree=17561936/32950928 swapFree=67111200/67111528
[pid=18902] ppid=18900 vsize=169404 CPUtime=0.73 cores=1,3,5,7
/proc/18902/stat : 18902 (claspmt) R 18900 18902 10053 0 -1 4202496 1635 0 0 0 73 0 0 0 25 0 4 0 733023397 173469696 1518 33554432000 4194304 4964800 140734855717904 18446744073709551615 4320944 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/18902/statm: 42351 1518 458 189 0 36700 0
Current children cumulated CPU time (s) 0.73
Current children cumulated vsize (KiB) 169404

[startup+0.30055 s]
/proc/loadavg: 7.91 8.09 8.34 5/354 18902
/proc/meminfo: memFree=17561936/32950928 swapFree=67111200/67111528
[pid=18902] ppid=18900 vsize=169668 CPUtime=1.13 cores=1,3,5,7
/proc/18902/stat : 18902 (claspmt) R 18900 18902 10053 0 -1 4202496 1802 0 0 0 113 0 0 0 25 0 4 0 733023397 173740032 1685 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670328 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/18902/statm: 42417 1685 458 189 0 36766 0
Current children cumulated CPU time (s) 1.13
Current children cumulated vsize (KiB) 169668

[startup+0.700474 s]
/proc/loadavg: 7.91 8.09 8.34 5/354 18902
/proc/meminfo: memFree=17561936/32950928 swapFree=67111200/67111528
[pid=18902] ppid=18900 vsize=170368 CPUtime=2.73 cores=1,3,5,7
/proc/18902/stat : 18902 (claspmt) R 18900 18902 10053 0 -1 4202496 2224 0 0 0 273 0 0 0 25 0 4 0 733023397 174456832 2107 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/18902/statm: 42592 2107 458 189 0 36941 0
Current children cumulated CPU time (s) 2.73
Current children cumulated vsize (KiB) 170368

[startup+1.50032 s]
/proc/loadavg: 7.91 8.09 8.34 9/358 18906
/proc/meminfo: memFree=17552380/32950928 swapFree=67111200/67111528
[pid=18902] ppid=18900 vsize=171376 CPUtime=5.92 cores=1,3,5,7
/proc/18902/stat : 18902 (claspmt) R 18900 18902 10053 0 -1 4202496 3155 0 0 0 592 0 0 0 25 0 4 0 733023397 175489024 3038 33554432000 4194304 4964800 140734855717904 18446744073709551615 4342232 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/18902/statm: 42844 3038 458 189 0 37193 0
[pid=18902/tid=18904] ppid=18900 vsize=171376 CPUtime=1.47 cores=1,3,5,7
/proc/18902/task/18904/stat : 18904 (claspmt) R 18900 18902 10053 0 -1 4202560 603 0 0 0 147 0 0 0 25 0 4 0 733023398 175489024 3038 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670295 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18902/tid=18905] ppid=18900 vsize=171376 CPUtime=1.47 cores=1,3,5,7
/proc/18902/task/18905/stat : 18905 (claspmt) R 18900 18902 10053 0 -1 4202560 772 0 0 0 147 0 0 0 20 0 4 0 733023398 175489024 3038 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670546 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18902/tid=18906] ppid=18900 vsize=171376 CPUtime=1.46 cores=1,3,5,7
/proc/18902/task/18906/stat : 18906 (claspmt) R 18900 18902 10053 0 -1 4202560 516 0 0 0 146 0 0 0 25 0 4 0 733023398 175489024 3038 33554432000 4194304 4964800 140734855717904 18446744073709551615 4321076 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 5.92
Current children cumulated vsize (KiB) 171376

[startup+3.10102 s]
/proc/loadavg: 7.91 8.09 8.34 9/358 18906
/proc/meminfo: memFree=17549200/32950928 swapFree=67111200/67111528
[pid=18902] ppid=18900 vsize=172576 CPUtime=12.31 cores=1,3,5,7
/proc/18902/stat : 18902 (claspmt) R 18900 18902 10053 0 -1 4202496 3821 0 0 0 1230 1 0 0 25 0 4 0 733023397 176717824 3704 33554432000 4194304 4964800 140734855717904 18446744073709551615 4321069 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/18902/statm: 43144 3704 458 189 0 37493 0
[pid=18902/tid=18904] ppid=18900 vsize=172576 CPUtime=3.07 cores=1,3,5,7
/proc/18902/task/18904/stat : 18904 (claspmt) R 18900 18902 10053 0 -1 4202560 791 0 0 0 307 0 0 0 25 0 4 0 733023398 176717824 3704 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670349 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18902/tid=18905] ppid=18900 vsize=172576 CPUtime=3.07 cores=1,3,5,7
/proc/18902/task/18905/stat : 18905 (claspmt) R 18900 18902 10053 0 -1 4202560 923 0 0 0 307 0 0 0 25 0 4 0 733023398 176717824 3704 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18902/tid=18906] ppid=18900 vsize=172576 CPUtime=3.05 cores=1,3,5,7
/proc/18902/task/18906/stat : 18906 (claspmt) R 18900 18902 10053 0 -1 4202560 675 0 0 0 305 0 0 0 25 0 4 0 733023398 176717824 3704 33554432000 4194304 4964800 140734855717904 18446744073709551615 4668689 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 12.31
Current children cumulated vsize (KiB) 172576

[startup+6.30042 s]
/proc/loadavg: 7.91 8.09 8.34 9/358 18906
/proc/meminfo: memFree=17545356/32950928 swapFree=67111200/67111528
[pid=18902] ppid=18900 vsize=173916 CPUtime=25.07 cores=1,3,5,7
/proc/18902/stat : 18902 (claspmt) R 18900 18902 10053 0 -1 4202496 4648 0 0 0 2506 1 0 0 25 0 4 0 733023397 178089984 4531 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/18902/statm: 43479 4531 458 189 0 37828 0
[pid=18902/tid=18904] ppid=18900 vsize=173916 CPUtime=6.27 cores=1,3,5,7
/proc/18902/task/18904/stat : 18904 (claspmt) R 18900 18902 10053 0 -1 4202560 1122 0 0 0 627 0 0 0 25 0 4 0 733023398 178089984 4531 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670321 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18902/tid=18905] ppid=18900 vsize=173916 CPUtime=6.27 cores=1,3,5,7
/proc/18902/task/18905/stat : 18905 (claspmt) R 18900 18902 10053 0 -1 4202560 1127 0 0 0 627 0 0 0 25 0 4 0 733023398 178089984 4531 33554432000 4194304 4964800 140734855717904 18446744073709551615 4669413 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18902/tid=18906] ppid=18900 vsize=173916 CPUtime=6.22 cores=1,3,5,7
/proc/18902/task/18906/stat : 18906 (claspmt) R 18900 18902 10053 0 -1 4202560 812 0 0 0 622 0 0 0 25 0 4 0 733023398 178089984 4531 33554432000 4194304 4964800 140734855717904 18446744073709551615 4668712 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 25.07
Current children cumulated vsize (KiB) 173916

[startup+12.7002 s]
/proc/loadavg: 7.92 8.09 8.33 9/358 18906
/proc/meminfo: memFree=17540788/32950928 swapFree=67111200/67111528
[pid=18902] ppid=18900 vsize=174580 CPUtime=50.6 cores=1,3,5,7
/proc/18902/stat : 18902 (claspmt) R 18900 18902 10053 0 -1 4202496 5398 0 0 0 5058 2 0 0 25 0 4 0 733023397 178769920 5281 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670137 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/18902/statm: 43645 5281 458 189 0 37994 0
[pid=18902/tid=18904] ppid=18900 vsize=174580 CPUtime=12.67 cores=1,3,5,7
/proc/18902/task/18904/stat : 18904 (claspmt) R 18900 18902 10053 0 -1 4202560 1278 0 0 0 1267 0 0 0 25 0 4 0 733023398 178769920 5281 33554432000 4194304 4964800 140734855717904 18446744073709551615 4668734 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18902/tid=18905] ppid=18900 vsize=174580 CPUtime=12.67 cores=1,3,5,7
/proc/18902/task/18905/stat : 18905 (claspmt) R 18900 18902 10053 0 -1 4202560 1374 0 0 0 1267 0 0 0 25 0 4 0 733023398 178769920 5281 33554432000 4194304 4964800 140734855717904 18446744073709551615 4321023 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18902/tid=18906] ppid=18900 vsize=174580 CPUtime=12.54 cores=1,3,5,7
/proc/18902/task/18906/stat : 18906 (claspmt) R 18900 18902 10053 0 -1 4202560 1014 0 0 0 1254 0 0 0 25 0 4 0 733023398 178769920 5281 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.6
Current children cumulated vsize (KiB) 174580

[startup+25.5008 s]
/proc/loadavg: 8.02 8.10 8.33 9/358 18907
/proc/meminfo: memFree=17532484/32950928 swapFree=67111200/67111528
[pid=18902] ppid=18900 vsize=176268 CPUtime=101.65 cores=1,3,5,7
/proc/18902/stat : 18902 (claspmt) R 18900 18902 10053 0 -1 4202496 6507 0 0 0 10162 3 0 0 25 0 4 0 733023397 180498432 6390 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/18902/statm: 44067 6390 458 189 0 38416 0
[pid=18902/tid=18904] ppid=18900 vsize=176268 CPUtime=25.47 cores=1,3,5,7
/proc/18902/task/18904/stat : 18904 (claspmt) R 18900 18902 10053 0 -1 4202560 1630 0 0 0 2546 1 0 0 25 0 4 0 733023398 180498432 6390 33554432000 4194304 4964800 140734855717904 18446744073709551615 4668681 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18902/tid=18905] ppid=18900 vsize=176268 CPUtime=25.47 cores=1,3,5,7
/proc/18902/task/18905/stat : 18905 (claspmt) R 18900 18902 10053 0 -1 4202560 1719 0 0 0 2547 0 0 0 25 0 4 0 733023398 180498432 6390 33554432000 4194304 4964800 140734855717904 18446744073709551615 4668552 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18902/tid=18906] ppid=18900 vsize=176268 CPUtime=25.2 cores=1,3,5,7
/proc/18902/task/18906/stat : 18906 (claspmt) R 18900 18902 10053 0 -1 4202560 1249 0 0 0 2520 0 0 0 25 0 4 0 733023398 180498432 6390 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.65
Current children cumulated vsize (KiB) 176268

[startup+51.101 s]
/proc/loadavg: 8.01 8.09 8.32 9/358 18907
/proc/meminfo: memFree=17524528/32950928 swapFree=67111200/67111528
[pid=18902] ppid=18900 vsize=178200 CPUtime=203.78 cores=1,3,5,7
/proc/18902/stat : 18902 (claspmt) R 18900 18902 10053 0 -1 4202496 8179 0 0 0 20373 5 0 0 25 0 4 0 733023397 182476800 8062 33554432000 4194304 4964800 140734855717904 18446744073709551615 4320909 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/18902/statm: 44550 8062 458 189 0 38899 0
[pid=18902/tid=18904] ppid=18900 vsize=178200 CPUtime=51.07 cores=1,3,5,7
/proc/18902/task/18904/stat : 18904 (claspmt) R 18900 18902 10053 0 -1 4202560 2176 0 0 0 5106 1 0 0 25 0 4 0 733023398 182476800 8062 33554432000 4194304 4964800 140734855717904 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18902/tid=18905] ppid=18900 vsize=178200 CPUtime=51.07 cores=1,3,5,7
/proc/18902/task/18905/stat : 18905 (claspmt) R 18900 18902 10053 0 -1 4202560 2301 0 0 0 5106 1 0 0 25 0 4 0 733023398 182476800 8062 33554432000 4194304 4964800 140734855717904 18446744073709551615 4320919 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18902/tid=18906] ppid=18900 vsize=178200 CPUtime=50.52 cores=1,3,5,7
/proc/18902/task/18906/stat : 18906 (claspmt) R 18900 18902 10053 0 -1 4202560 1467 0 0 0 5052 0 0 0 25 0 4 0 733023398 182476800 8062 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 203.78
Current children cumulated vsize (KiB) 178200

[startup+102.3 s]
/proc/loadavg: 8.10 8.10 8.31 9/358 18909
/proc/meminfo: memFree=17509380/32950928 swapFree=67111200/67111528
[pid=18902] ppid=18900 vsize=181700 CPUtime=408.04 cores=1,3,5,7
/proc/18902/stat : 18902 (claspmt) R 18900 18902 10053 0 -1 4202496 10603 0 0 0 40797 7 0 0 25 0 4 0 733023397 186060800 10486 33554432000 4194304 4964800 140734855717904 18446744073709551615 4362248 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/18902/statm: 45425 10486 458 189 0 39774 0
[pid=18902/tid=18904] ppid=18900 vsize=181700 CPUtime=102.28 cores=1,3,5,7
/proc/18902/task/18904/stat : 18904 (claspmt) R 18900 18902 10053 0 -1 4202560 3337 0 0 0 10226 2 0 0 25 0 4 0 733023398 186060800 10486 33554432000 4194304 4964800 140734855717904 18446744073709551615 4332144 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18902/tid=18905] ppid=18900 vsize=181700 CPUtime=102.28 cores=1,3,5,7
/proc/18902/task/18905/stat : 18905 (claspmt) R 18900 18902 10053 0 -1 4202560 2950 0 0 0 10227 1 0 0 25 0 4 0 733023398 186060800 10486 33554432000 4194304 4964800 140734855717904 18446744073709551615 4342157 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18902/tid=18906] ppid=18900 vsize=181700 CPUtime=101.17 cores=1,3,5,7
/proc/18902/task/18906/stat : 18906 (claspmt) R 18900 18902 10053 0 -1 4202560 1753 0 0 0 10116 1 0 0 25 0 4 0 733023398 186060800 10486 33554432000 4194304 4964800 140734855717904 18446744073709551615 4668588 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 408.04
Current children cumulated vsize (KiB) 181700

[startup+162.301 s]
/proc/loadavg: 8.18 8.13 8.31 9/358 18910
/proc/meminfo: memFree=17500304/32950928 swapFree=67111200/67111528
[pid=18902] ppid=18900 vsize=183348 CPUtime=647.35 cores=1,3,5,7
/proc/18902/stat : 18902 (claspmt) R 18900 18902 10053 0 -1 4202496 12297 0 0 0 64726 9 0 0 25 0 4 0 733023397 187748352 12180 33554432000 4194304 4964800 140734855717904 18446744073709551615 4669479 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/18902/statm: 45837 12180 458 189 0 40186 0
[pid=18902/tid=18904] ppid=18900 vsize=183348 CPUtime=162.27 cores=1,3,5,7
/proc/18902/task/18904/stat : 18904 (claspmt) R 18900 18902 10053 0 -1 4202560 4069 0 0 0 16225 2 0 0 25 0 4 0 733023398 187748352 12180 33554432000 4194304 4964800 140734855717904 18446744073709551615 4669338 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18902/tid=18905] ppid=18900 vsize=183348 CPUtime=162.27 cores=1,3,5,7
/proc/18902/task/18905/stat : 18905 (claspmt) R 18900 18902 10053 0 -1 4202560 3253 0 0 0 16225 2 0 0 25 0 4 0 733023398 187748352 12180 33554432000 4194304 4964800 140734855717904 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18902/tid=18906] ppid=18900 vsize=183348 CPUtime=160.51 cores=1,3,5,7
/proc/18902/task/18906/stat : 18906 (claspmt) R 18900 18902 10053 0 -1 4202560 1976 0 0 0 16050 1 0 0 25 0 4 0 733023398 187748352 12180 33554432000 4194304 4964800 140734855717904 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 647.35
Current children cumulated vsize (KiB) 183348

[startup+222.314 s]
/proc/loadavg: 8.15 8.13 8.29 9/358 18912
/proc/meminfo: memFree=17491088/32950928 swapFree=67111200/67111528
[pid=18902] ppid=18900 vsize=186140 CPUtime=886.75 cores=1,3,5,7
/proc/18902/stat : 18902 (claspmt) R 18900 18902 10053 0 -1 4202496 14068 0 0 0 88663 12 0 0 25 0 4 0 733023397 190607360 13951 33554432000 4194304 4964800 140734855717904 18446744073709551615 4332025 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/18902/statm: 46535 13951 458 189 0 40884 0
[pid=18902/tid=18904] ppid=18900 vsize=186140 CPUtime=222.27 cores=1,3,5,7
/proc/18902/task/18904/stat : 18904 (claspmt) R 18900 18902 10053 0 -1 4202560 4839 0 0 0 22225 2 0 0 25 0 4 0 733023398 190607360 13951 33554432000 4194304 4964800 140734855717904 18446744073709551615 4660236 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18902/tid=18905] ppid=18900 vsize=186140 CPUtime=222.28 cores=1,3,5,7
/proc/18902/task/18905/stat : 18905 (claspmt) R 18900 18902 10053 0 -1 4202560 3889 0 0 0 22226 2 0 0 25 0 4 0 733023398 190607360 13951 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670332 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18902/tid=18906] ppid=18900 vsize=186140 CPUtime=219.87 cores=1,3,5,7
/proc/18902/task/18906/stat : 18906 (claspmt) R 18900 18902 10053 0 -1 4202560 2146 0 0 0 21985 2 0 0 25 0 4 0 733023398 190607360 13951 33554432000 4194304 4964800 140734855717904 18446744073709551615 4342144 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 886.75
Current children cumulated vsize (KiB) 186140

[startup+282.3 s]

################
# More data... #
################

[pid=18902] ppid=18900 vsize=258352 CPUtime=2801.4 cores=1,3,5,7
/proc/18902/stat : 18902 (claspmt) R 18900 18902 10053 0 -1 4202496 21736 0 0 0 280112 28 0 0 22 0 4 0 733023397 264552448 21470 33554432000 4194304 4964800 140734855717904 18446744073709551615 4669560 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/18902/statm: 64588 21470 458 189 0 58937 0
[pid=18902/tid=18904] ppid=18900 vsize=258352 CPUtime=702.24 cores=1,3,5,7
/proc/18902/task/18904/stat : 18904 (claspmt) R 18900 18902 10053 0 -1 4202560 8313 0 0 0 70217 7 0 0 25 0 4 0 733023398 264552448 21470 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670538 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18902/tid=18905] ppid=18900 vsize=258352 CPUtime=702.27 cores=1,3,5,7
/proc/18902/task/18905/stat : 18905 (claspmt) R 18900 18902 10053 0 -1 4202560 6216 0 0 0 70221 6 0 0 25 0 4 0 733023398 264552448 21470 33554432000 4194304 4964800 140734855717904 18446744073709551615 4668773 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18902/tid=18906] ppid=18900 vsize=258352 CPUtime=694.61 cores=1,3,5,7
/proc/18902/task/18906/stat : 18906 (claspmt) R 18900 18902 10053 0 -1 4202560 3216 0 0 0 69455 6 0 0 25 0 4 0 733023398 264552448 21470 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2801.4
Current children cumulated vsize (KiB) 258352

[startup+762.301 s]
/proc/loadavg: 8.12 8.10 8.19 9/358 18935
/proc/meminfo: memFree=17392908/32950928 swapFree=67111200/67111528
[pid=18902] ppid=18900 vsize=258800 CPUtime=3040.79 cores=1,3,5,7
/proc/18902/stat : 18902 (claspmt) R 18900 18902 10053 0 -1 4202496 22358 0 0 0 304050 29 0 0 25 0 4 0 733023397 265011200 22092 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670534 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/18902/statm: 64700 22092 458 189 0 59049 0
[pid=18902/tid=18904] ppid=18900 vsize=258800 CPUtime=762.26 cores=1,3,5,7
/proc/18902/task/18904/stat : 18904 (claspmt) R 18900 18902 10053 0 -1 4202560 8428 0 0 0 76218 8 0 0 25 0 4 0 733023398 265011200 22092 33554432000 4194304 4964800 140734855717904 18446744073709551615 4668698 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18902/tid=18905] ppid=18900 vsize=258800 CPUtime=762.29 cores=1,3,5,7
/proc/18902/task/18905/stat : 18905 (claspmt) R 18900 18902 10053 0 -1 4202560 6501 0 0 0 76222 7 0 0 25 0 4 0 733023398 265011200 22092 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670437 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18902/tid=18906] ppid=18900 vsize=258800 CPUtime=753.96 cores=1,3,5,7
/proc/18902/task/18906/stat : 18906 (claspmt) R 18900 18902 10053 0 -1 4202560 3400 0 0 0 75390 6 0 0 25 0 4 0 733023398 265011200 22092 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670273 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3040.79
Current children cumulated vsize (KiB) 258800

[startup+822.3 s]
/proc/loadavg: 8.10 8.10 8.18 9/358 18937
/proc/meminfo: memFree=17378516/32950928 swapFree=67111200/67111528
[pid=18902] ppid=18900 vsize=259024 CPUtime=3280.1 cores=1,3,5,7
/proc/18902/stat : 18902 (claspmt) R 18900 18902 10053 0 -1 4202496 22703 0 0 0 327979 31 0 0 25 0 4 0 733023397 265240576 22437 33554432000 4194304 4964800 140734855717904 18446744073709551615 4669410 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/18902/statm: 64756 22437 458 189 0 59105 0
[pid=18902/tid=18904] ppid=18900 vsize=259024 CPUtime=822.24 cores=1,3,5,7
/proc/18902/task/18904/stat : 18904 (claspmt) R 18900 18902 10053 0 -1 4202560 8553 0 0 0 82216 8 0 0 25 0 4 0 733023398 265240576 22437 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18902/tid=18905] ppid=18900 vsize=259024 CPUtime=822.28 cores=1,3,5,7
/proc/18902/task/18905/stat : 18905 (claspmt) R 18900 18902 10053 0 -1 4202560 6663 0 0 0 82221 7 0 0 25 0 4 0 733023398 265240576 22437 33554432000 4194304 4964800 140734855717904 18446744073709551615 4669507 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18902/tid=18906] ppid=18900 vsize=259024 CPUtime=813.3 cores=1,3,5,7
/proc/18902/task/18906/stat : 18906 (claspmt) R 18900 18902 10053 0 -1 4202560 3432 0 0 0 81324 6 0 0 25 0 4 0 733023398 265240576 22437 33554432000 4194304 4964800 140734855717904 18446744073709551615 4669338 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3280.1
Current children cumulated vsize (KiB) 259024

[startup+882.313 s]
/proc/loadavg: 8.14 8.11 8.18 9/358 18939
/proc/meminfo: memFree=17370412/32950928 swapFree=67111200/67111528
[pid=18902] ppid=18900 vsize=259308 CPUtime=3519.48 cores=1,3,5,7
/proc/18902/stat : 18902 (claspmt) R 18900 18902 10053 0 -1 4202496 23043 0 0 0 351916 32 0 0 25 0 4 0 733023397 265531392 22777 33554432000 4194304 4964800 140734855717904 18446744073709551615 4668689 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/18902/statm: 64827 22777 458 189 0 59176 0
[pid=18902/tid=18904] ppid=18900 vsize=259308 CPUtime=882.25 cores=1,3,5,7
/proc/18902/task/18904/stat : 18904 (claspmt) R 18900 18902 10053 0 -1 4202560 8636 0 0 0 88216 9 0 0 20 0 4 0 733023398 265531392 22777 33554432000 4194304 4964800 140734855717904 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18902/tid=18905] ppid=18900 vsize=259308 CPUtime=882.29 cores=1,3,5,7
/proc/18902/task/18905/stat : 18905 (claspmt) R 18900 18902 10053 0 -1 4202560 6870 0 0 0 88222 7 0 0 25 0 4 0 733023398 265531392 22777 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18902/tid=18906] ppid=18900 vsize=259308 CPUtime=872.66 cores=1,3,5,7
/proc/18902/task/18906/stat : 18906 (claspmt) R 18900 18902 10053 0 -1 4202560 3444 0 0 0 87259 7 0 0 25 0 4 0 733023398 265531392 22777 33554432000 4194304 4964800 140734855717904 18446744073709551615 4668598 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3519.48
Current children cumulated vsize (KiB) 259308

[startup+942.301 s]
/proc/loadavg: 5.94 7.52 7.98 5/349 18958
/proc/meminfo: memFree=17655552/32950928 swapFree=67111200/67111528
[pid=18902] ppid=18900 vsize=260012 CPUtime=3758.78 cores=1,3,5,7
/proc/18902/stat : 18902 (claspmt) R 18900 18902 10053 0 -1 4202496 23822 0 0 0 375843 35 0 0 25 0 4 0 733023397 266252288 23556 33554432000 4194304 4964800 140734855717904 18446744073709551615 4669509 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/18902/statm: 65003 23556 458 189 0 59352 0
[pid=18902/tid=18904] ppid=18900 vsize=260012 CPUtime=942.22 cores=1,3,5,7
/proc/18902/task/18904/stat : 18904 (claspmt) R 18900 18902 10053 0 -1 4202560 9075 0 0 0 94213 9 0 0 25 0 4 0 733023398 266252288 23556 33554432000 4194304 4964800 140734855717904 18446744073709551615 4668773 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18902/tid=18905] ppid=18900 vsize=260012 CPUtime=942.28 cores=1,3,5,7
/proc/18902/task/18905/stat : 18905 (claspmt) R 18900 18902 10053 0 -1 4202560 7049 0 0 0 94220 8 0 0 25 0 4 0 733023398 266252288 23556 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18902/tid=18906] ppid=18900 vsize=260012 CPUtime=931.99 cores=1,3,5,7
/proc/18902/task/18906/stat : 18906 (claspmt) R 18900 18902 10053 0 -1 4202560 3479 0 0 0 93192 7 0 0 25 0 4 0 733023398 266252288 23556 33554432000 4194304 4964800 140734855717904 18446744073709551615 4669490 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3758.78
Current children cumulated vsize (KiB) 260012

[startup+1002.3 s]
/proc/loadavg: 4.71 6.88 7.72 5/349 18960
/proc/meminfo: memFree=17652548/32950928 swapFree=67111200/67111528
[pid=18902] ppid=18900 vsize=260604 CPUtime=3998.12 cores=1,3,5,7
/proc/18902/stat : 18902 (claspmt) R 18900 18902 10053 0 -1 4202496 24611 0 0 0 399775 37 0 0 25 0 4 0 733023397 266858496 24326 33554432000 4194304 4964800 140734855717904 18446744073709551615 4653542 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/18902/statm: 65151 24326 458 189 0 59500 0
[pid=18902/tid=18904] ppid=18900 vsize=260604 CPUtime=1002.23 cores=1,3,5,7
/proc/18902/task/18904/stat : 18904 (claspmt) R 18900 18902 10053 0 -1 4202560 9412 0 0 0 100212 11 0 0 25 0 4 0 733023398 266858496 24326 33554432000 4194304 4964800 140734855717904 18446744073709551615 4321013 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18902/tid=18905] ppid=18900 vsize=260604 CPUtime=1002.28 cores=1,3,5,7
/proc/18902/task/18905/stat : 18905 (claspmt) R 18900 18902 10053 0 -1 4202560 7274 0 0 0 100220 8 0 0 25 0 4 0 733023398 266858496 24326 33554432000 4194304 4964800 140734855717904 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18902/tid=18906] ppid=18900 vsize=260604 CPUtime=991.33 cores=1,3,5,7
/proc/18902/task/18906/stat : 18906 (claspmt) R 18900 18902 10053 0 -1 4202560 3651 0 0 0 99125 8 0 0 25 0 4 0 733023398 266858496 24326 33554432000 4194304 4964800 140734855717904 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3998.12
Current children cumulated vsize (KiB) 260604

[startup+1062.3 s]
/proc/loadavg: 4.26 6.35 7.49 5/349 18961
/proc/meminfo: memFree=17650536/32950928 swapFree=67111200/67111528
[pid=18902] ppid=18900 vsize=261496 CPUtime=4237.47 cores=1,3,5,7
/proc/18902/stat : 18902 (claspmt) R 18900 18902 10053 0 -1 4202496 25250 0 0 0 423708 39 0 0 25 0 4 0 733023397 267771904 24932 33554432000 4194304 4964800 140734855717904 18446744073709551615 4321102 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/18902/statm: 65374 24932 458 189 0 59723 0
[pid=18902/tid=18904] ppid=18900 vsize=261496 CPUtime=1062.22 cores=1,3,5,7
/proc/18902/task/18904/stat : 18904 (claspmt) R 18900 18902 10053 0 -1 4202560 9814 0 0 0 106211 11 0 0 25 0 4 0 733023398 267771904 24932 33554432000 4194304 4964800 140734855717904 18446744073709551615 4660463 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18902/tid=18905] ppid=18900 vsize=261496 CPUtime=1062.29 cores=1,3,5,7
/proc/18902/task/18905/stat : 18905 (claspmt) R 18900 18902 10053 0 -1 4202560 7410 0 0 0 106220 9 0 0 25 0 4 0 733023398 267771904 24932 33554432000 4194304 4964800 140734855717904 18446744073709551615 4669429 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18902/tid=18906] ppid=18900 vsize=261496 CPUtime=1050.69 cores=1,3,5,7
/proc/18902/task/18906/stat : 18906 (claspmt) R 18900 18902 10053 0 -1 4202560 3681 0 0 0 105061 8 0 0 25 0 4 0 733023398 267771904 24932 33554432000 4194304 4964800 140734855717904 18446744073709551615 4342157 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4237.47
Current children cumulated vsize (KiB) 261496

[startup+1122.3 s]
/proc/loadavg: 4.09 5.92 7.27 5/349 18963
/proc/meminfo: memFree=17649268/32950928 swapFree=67111200/67111528
[pid=18902] ppid=18900 vsize=261704 CPUtime=4476.81 cores=1,3,5,7
/proc/18902/stat : 18902 (claspmt) R 18900 18902 10053 0 -1 4202496 25545 0 0 0 447639 42 0 0 25 0 4 0 733023397 267984896 25227 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670299 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/18902/statm: 65426 25227 458 189 0 59775 0
[pid=18902/tid=18904] ppid=18900 vsize=261704 CPUtime=1122.22 cores=1,3,5,7
/proc/18902/task/18904/stat : 18904 (claspmt) R 18900 18902 10053 0 -1 4202560 9905 0 0 0 112210 12 0 0 25 0 4 0 733023398 267984896 25227 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18902/tid=18905] ppid=18900 vsize=261704 CPUtime=1122.29 cores=1,3,5,7
/proc/18902/task/18905/stat : 18905 (claspmt) R 18900 18902 10053 0 -1 4202560 7461 0 0 0 112219 10 0 0 25 0 4 0 733023398 267984896 25227 33554432000 4194304 4964800 140734855717904 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18902/tid=18906] ppid=18900 vsize=261704 CPUtime=1110.02 cores=1,3,5,7
/proc/18902/task/18906/stat : 18906 (claspmt) R 18900 18902 10053 0 -1 4202560 3776 0 0 0 110994 8 0 0 25 0 4 0 733023398 267984896 25227 33554432000 4194304 4964800 140734855717904 18446744073709551615 4668797 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4476.81
Current children cumulated vsize (KiB) 261704

[startup+1182.3 s]
/proc/loadavg: 4.03 5.57 7.06 5/349 18965
/proc/meminfo: memFree=17648124/32950928 swapFree=67111200/67111528
[pid=18902] ppid=18900 vsize=261872 CPUtime=4716.16 cores=1,3,5,7
/proc/18902/stat : 18902 (claspmt) R 18900 18902 10053 0 -1 4202496 25811 0 0 0 471573 43 0 0 25 0 4 0 733023397 268156928 25484 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670349 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/18902/statm: 65468 25484 458 189 0 59817 0
[pid=18902/tid=18904] ppid=18900 vsize=261872 CPUtime=1182.22 cores=1,3,5,7
/proc/18902/task/18904/stat : 18904 (claspmt) R 18900 18902 10053 0 -1 4202560 10015 0 0 0 118210 12 0 0 25 0 4 0 733023398 268156928 25484 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18902/tid=18905] ppid=18900 vsize=261872 CPUtime=1182.29 cores=1,3,5,7
/proc/18902/task/18905/stat : 18905 (claspmt) R 18900 18902 10053 0 -1 4202560 7488 0 0 0 118219 10 0 0 25 0 4 0 733023398 268156928 25484 33554432000 4194304 4964800 140734855717904 18446744073709551615 4362938 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18902/tid=18906] ppid=18900 vsize=261872 CPUtime=1169.38 cores=1,3,5,7
/proc/18902/task/18906/stat : 18906 (claspmt) R 18900 18902 10053 0 -1 4202560 3854 0 0 0 116929 9 0 0 25 0 4 0 733023398 268156928 25484 33554432000 4194304 4964800 140734855717904 18446744073709551615 4321097 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4716.16
Current children cumulated vsize (KiB) 261872



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.4 s]
/proc/loadavg: 4.02 5.46 6.99 6/349 18965
/proc/meminfo: memFree=17647616/32950928 swapFree=67111200/67111528
[pid=18902] ppid=18900 vsize=261872 CPUtime=4800.4 cores=1,3,5,7
/proc/18902/stat : 18902 (claspmt) R 18900 18902 10053 0 -1 4202496 25983 0 0 0 479996 44 0 0 25 0 4 0 733023397 268156928 25607 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/18902/statm: 65468 25607 458 189 0 59817 0
[pid=18902/tid=18904] ppid=18900 vsize=261872 CPUtime=1203.34 cores=1,3,5,7
/proc/18902/task/18904/stat : 18904 (claspmt) R 18900 18902 10053 0 -1 4202560 10065 0 0 0 120322 12 0 0 25 0 4 0 733023398 268156928 25607 33554432000 4194304 4964800 140734855717904 18446744073709551615 4320939 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18902/tid=18905] ppid=18900 vsize=261872 CPUtime=1203.41 cores=1,3,5,7
/proc/18902/task/18905/stat : 18905 (claspmt) R 18900 18902 10053 0 -1 4202560 7609 0 0 0 120331 10 0 0 25 0 4 0 733023398 268156928 25607 33554432000 4194304 4964800 140734855717904 18446744073709551615 4658574 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18902/tid=18906] ppid=18900 vsize=261872 CPUtime=1190.25 cores=1,3,5,7
/proc/18902/task/18906/stat : 18906 (claspmt) R 18900 18902 10053 0 -1 4202560 3854 0 0 0 119016 9 0 0 25 0 4 0 733023398 268156928 25607 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670534 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4800.4
Current children cumulated vsize (KiB) 261872

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.4 s]
/proc/loadavg: 4.02 5.46 6.99 6/349 18965
/proc/meminfo: memFree=17647616/32950928 swapFree=67111200/67111528
[pid=18902] ppid=18900 vsize=261872 CPUtime=4800.4 cores=1,3,5,7
/proc/18902/stat : 18902 (claspmt) R 18900 18902 10053 0 -1 4202496 25983 0 0 0 479996 44 0 0 25 0 4 0 733023397 268156928 25607 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/18902/statm: 65468 25607 458 189 0 59817 0
[pid=18902/tid=18904] ppid=18900 vsize=261872 CPUtime=1203.34 cores=1,3,5,7
/proc/18902/task/18904/stat : 18904 (claspmt) R 18900 18902 10053 0 -1 4202560 10065 0 0 0 120322 12 0 0 25 0 4 0 733023398 268156928 25607 33554432000 4194304 4964800 140734855717904 18446744073709551615 4320939 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=18902/tid=18905] ppid=18900 vsize=261872 CPUtime=1203.41 cores=1,3,5,7
/proc/18902/task/18905/stat : 18905 (claspmt) R 18900 18902 10053 0 -1 4202560 7609 0 0 0 120331 10 0 0 25 0 4 0 733023398 268156928 25607 33554432000 4194304 4964800 140734855717904 18446744073709551615 4658574 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=18902/tid=18906] ppid=18900 vsize=261872 CPUtime=1190.25 cores=1,3,5,7
/proc/18902/task/18906/stat : 18906 (claspmt) R 18900 18902 10053 0 -1 4202560 3854 0 0 0 119016 9 0 0 25 0 4 0 733023398 268156928 25607 33554432000 4194304 4964800 140734855717904 18446744073709551615 4670534 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4800.4
Current children cumulated vsize (KiB) 261872

Child status: 0
Real time (s): 1203.44
CPU time (s): 4800.45
CPU user time (s): 4800
CPU system time (s): 0.45393
CPU usage (%): 398.894
Max. virtual memory (cumulated for all children) (KiB): 261872

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4800
system time used= 0.45393
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 25995
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 40883
involuntary context switches= 34203

runsolver used 3.71043 second user time and 9.46456 second system time

The end

Launcher Data 

Begin job on node115 at 2011-05-15 04:52:07
IDJOB=3373579
IDBENCH=83418
IDSOLVER=1853
FILE ID=node115/3373579-1305427927
RUNJOBID= node115-1305426413-29658
PBS_JOBID= 13324588
Free space on /tmp= 73296 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT11/application/kullmann/AES/Benchmarks/64/aes_64_2_keyfind_1.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3373579-1305427927/watcher-3373579-1305427927 -o /tmp/evaluation-result-3373579-1305427927/solver-3373579-1305427927 -C 4800 -W 1300 -M 15500  HOME/claspmt -t4 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3373579-1305427927.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 1024a3346322fdba0a20b51f99eae4c0
RANDOM SEED=1866501385

node115.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.62
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      17562216 kB
Buffers:        297464 kB
Cached:        4563308 kB
SwapCached:          0 kB
Active:       12165256 kB
Inactive:      2907676 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      17562216 kB
SwapTotal:    67111528 kB
SwapFree:     67111200 kB
Dirty:             844 kB
Writeback:           0 kB
AnonPages:    10211560 kB
Mapped:          28832 kB
Slab:           210324 kB
PageTables:      41404 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 69874124 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73296 MiB
End job on node115 at 2011-05-15 05:12:13