Trace number 3373155

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 4800.15 1203.88

General information on the benchmark

NameCRAFTED/rbsat/random/unforced/
rbsat-v945c61409g6.cnf
MD5SUM2094ee8d4e443e860293b947e469bd81
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables945
Number of clauses61409
Sum of the clauses size123673
Maximum clause length21
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 261364
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 545

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...419284/instance-3373155-1305419284.cnf
0.09/0.17	c Solving...
4800.03/1203.81	
4800.03/1203.81	*** INTERRUPTED! ***
4800.03/1203.81	*** Shutting down threads...
4800.03/1203.87	s UNKNOWN
4800.03/1203.87	
4800.03/1203.87	c INTERRUPTED   : 1
4800.03/1203.87	c Models        : 0+    
4800.03/1203.87	c Time          : 1203.812s (Solving: 1203.65s 1st Model: 0.00s Unsat: 0.00s)
4800.03/1203.87	c CPU Time      : 4800.080s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3373155-1305419284/watcher-3373155-1305419284 -o /tmp/evaluation-result-3373155-1305419284/solver-3373155-1305419284 -C 4800 -W 1300 -M 15500 HOME/claspmt -t4 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3373155-1305419284.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.16 8.12 7.83 6/367 1998
/proc/meminfo: memFree=25058284/32951124 swapFree=63083164/67111528
[pid=1998] ppid=1996 vsize=968 CPUtime=0 cores=1,3,5,7
/proc/1998/stat : 1998 (claspmt) R 1996 1998 1566 0 -1 4194304 44 0 0 0 0 0 0 0 25 0 1 0 634343976 991232 2 33554432000 0 0 140734932657857 18446744073709551615 5055511 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/1998/statm: 242 12 7 189 0 22 0

[startup+0.0846301 s]
/proc/loadavg: 8.16 8.12 7.83 6/367 1998
/proc/meminfo: memFree=25058284/32951124 swapFree=63083164/67111528
[pid=1998] ppid=1996 vsize=24784 CPUtime=0.07 cores=1,3,5,7
/proc/1998/stat : 1998 (claspmt) R 1996 1998 1566 0 -1 4202496 1678 0 0 0 7 0 0 0 25 0 1 0 634343976 25378816 1423 33554432000 4194304 4964800 140734932654240 18446744073709551615 4607575 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/1998/statm: 6196 1423 398 189 0 1070 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 24784

[startup+0.100628 s]
/proc/loadavg: 8.16 8.12 7.83 6/367 1998
/proc/meminfo: memFree=25058284/32951124 swapFree=63083164/67111528
[pid=1998] ppid=1996 vsize=24916 CPUtime=0.09 cores=1,3,5,7
/proc/1998/stat : 1998 (claspmt) R 1996 1998 1566 0 -1 4202496 1705 0 0 0 9 0 0 0 25 0 1 0 634343976 25513984 1450 33554432000 4194304 4964800 140734932654240 18446744073709551615 264726065861 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/1998/statm: 6229 1450 399 189 0 1103 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 24916

[startup+0.300554 s]
/proc/loadavg: 8.16 8.12 7.83 6/367 1998
/proc/meminfo: memFree=25058284/32951124 swapFree=63083164/67111528
[pid=1998] ppid=1996 vsize=171796 CPUtime=0.66 cores=1,3,5,7
/proc/1998/stat : 1998 (claspmt) R 1996 1998 1566 0 -1 4202496 2705 0 0 0 66 0 0 0 25 0 4 0 634343976 175919104 2322 33554432000 4194304 4964800 140734932654240 18446744073709551615 4631129 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/1998/statm: 42949 2322 458 189 0 37298 0
Current children cumulated CPU time (s) 0.66
Current children cumulated vsize (KiB) 171796

[startup+0.700481 s]
/proc/loadavg: 8.16 8.12 7.83 6/367 1998
/proc/meminfo: memFree=25058284/32951124 swapFree=63083164/67111528
[pid=1998] ppid=1996 vsize=174272 CPUtime=2.26 cores=1,3,5,7
/proc/1998/stat : 1998 (claspmt) R 1996 1998 1566 0 -1 4202496 4577 0 0 0 225 1 0 0 25 0 4 0 634343976 178454528 4194 33554432000 4194304 4964800 140734932654240 18446744073709551615 4320981 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/1998/statm: 43568 4194 458 189 0 37917 0
Current children cumulated CPU time (s) 2.26
Current children cumulated vsize (KiB) 174272

[startup+1.50033 s]
/proc/loadavg: 8.15 8.12 7.83 9/371 2002
/proc/meminfo: memFree=25040156/32951124 swapFree=63083164/67111528
[pid=1998] ppid=1996 vsize=176324 CPUtime=5.44 cores=1,3,5,7
/proc/1998/stat : 1998 (claspmt) R 1996 1998 1566 0 -1 4202496 6004 0 0 0 543 1 0 0 25 0 4 0 634343976 180555776 5621 33554432000 4194304 4964800 140734932654240 18446744073709551615 4320939 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/1998/statm: 44081 5621 458 189 0 38430 0
[pid=1998/tid=2000] ppid=1996 vsize=176324 CPUtime=1.31 cores=1,3,5,7
/proc/1998/task/2000/stat : 2000 (claspmt) R 1996 1998 1566 0 -1 4202560 926 0 0 0 131 0 0 0 25 0 4 0 634343993 180555776 5621 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670112 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1998/tid=2001] ppid=1996 vsize=176324 CPUtime=1.3 cores=1,3,5,7
/proc/1998/task/2001/stat : 2001 (claspmt) R 1996 1998 1566 0 -1 4202560 1235 0 0 0 130 0 0 0 25 0 4 0 634343993 180555776 5621 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=1998/tid=2002] ppid=1996 vsize=176324 CPUtime=1.32 cores=1,3,5,7
/proc/1998/task/2002/stat : 2002 (claspmt) R 1996 1998 1566 0 -1 4202560 975 0 0 0 132 0 0 0 25 0 4 0 634343993 180555776 5621 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670499 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 5.44
Current children cumulated vsize (KiB) 176324

[startup+3.10103 s]
/proc/loadavg: 8.15 8.12 7.83 9/371 2002
/proc/meminfo: memFree=25032872/32951124 swapFree=63083164/67111528
[pid=1998] ppid=1996 vsize=176732 CPUtime=11.83 cores=1,3,5,7
/proc/1998/stat : 1998 (claspmt) R 1996 1998 1566 0 -1 4202496 6666 0 0 0 1182 1 0 0 25 0 4 0 634343976 180973568 6283 33554432000 4194304 4964800 140734932654240 18446744073709551615 4669033 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/1998/statm: 44183 6283 460 189 0 38532 0
[pid=1998/tid=2000] ppid=1996 vsize=176732 CPUtime=2.91 cores=1,3,5,7
/proc/1998/task/2000/stat : 2000 (claspmt) R 1996 1998 1566 0 -1 4202560 1087 0 0 0 291 0 0 0 25 0 4 0 634343993 180973568 6283 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1998/tid=2001] ppid=1996 vsize=176732 CPUtime=2.89 cores=1,3,5,7
/proc/1998/task/2001/stat : 2001 (claspmt) R 1996 1998 1566 0 -1 4202560 1462 0 0 0 289 0 0 0 25 0 4 0 634343993 180973568 6283 33554432000 4194304 4964800 140734932654240 18446744073709551615 4669136 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=1998/tid=2002] ppid=1996 vsize=176732 CPUtime=2.92 cores=1,3,5,7
/proc/1998/task/2002/stat : 2002 (claspmt) R 1996 1998 1566 0 -1 4202560 1064 0 0 0 292 0 0 0 25 0 4 0 634343993 180973568 6283 33554432000 4194304 4964800 140734932654240 18446744073709551615 4627540 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 11.83
Current children cumulated vsize (KiB) 176732

[startup+6.30043 s]
/proc/loadavg: 8.15 8.12 7.83 9/371 2002
/proc/meminfo: memFree=25022184/32951124 swapFree=63083164/67111528
[pid=1998] ppid=1996 vsize=178588 CPUtime=24.59 cores=1,3,5,7
/proc/1998/stat : 1998 (claspmt) R 1996 1998 1566 0 -1 4202496 8885 0 0 0 2457 2 0 0 25 0 4 0 634343976 182874112 8502 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670499 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/1998/statm: 44647 8502 460 189 0 38996 0
[pid=1998/tid=2000] ppid=1996 vsize=178588 CPUtime=6.11 cores=1,3,5,7
/proc/1998/task/2000/stat : 2000 (claspmt) R 1996 1998 1566 0 -1 4202560 1529 0 0 0 611 0 0 0 25 0 4 0 634343993 182874112 8502 33554432000 4194304 4964800 140734932654240 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1998/tid=2001] ppid=1996 vsize=178588 CPUtime=6.05 cores=1,3,5,7
/proc/1998/task/2001/stat : 2001 (claspmt) R 1996 1998 1566 0 -1 4202560 2199 0 0 0 605 0 0 0 25 0 4 0 634343993 182874112 8502 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=1998/tid=2002] ppid=1996 vsize=178588 CPUtime=6.12 cores=1,3,5,7
/proc/1998/task/2002/stat : 2002 (claspmt) R 1996 1998 1566 0 -1 4202560 1466 0 0 0 612 0 0 0 25 0 4 0 634343993 182874112 8502 33554432000 4194304 4964800 140734932654240 18446744073709551615 4320928 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 24.59
Current children cumulated vsize (KiB) 178588

[startup+12.7002 s]
/proc/loadavg: 8.27 8.15 7.84 9/371 2003
/proc/meminfo: memFree=24996040/32951124 swapFree=63083164/67111528
[pid=1998] ppid=1996 vsize=185580 CPUtime=50.11 cores=1,3,5,7
/proc/1998/stat : 1998 (claspmt) R 1996 1998 1566 0 -1 4202496 13483 0 0 0 5007 4 0 0 25 0 4 0 634343976 190033920 13100 33554432000 4194304 4964800 140734932654240 18446744073709551615 4627658 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/1998/statm: 46395 13100 460 189 0 40744 0
[pid=1998/tid=2000] ppid=1996 vsize=185580 CPUtime=12.51 cores=1,3,5,7
/proc/1998/task/2000/stat : 2000 (claspmt) R 1996 1998 1566 0 -1 4202560 2745 0 0 0 1250 1 0 0 25 0 4 0 634343993 190033920 13100 33554432000 4194304 4964800 140734932654240 18446744073709551615 4627549 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1998/tid=2001] ppid=1996 vsize=185580 CPUtime=12.37 cores=1,3,5,7
/proc/1998/task/2001/stat : 2001 (claspmt) R 1996 1998 1566 0 -1 4202560 3655 0 0 0 1237 0 0 0 25 0 4 0 634343993 190033920 13100 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=1998/tid=2002] ppid=1996 vsize=185580 CPUtime=12.51 cores=1,3,5,7
/proc/1998/task/2002/stat : 2002 (claspmt) R 1996 1998 1566 0 -1 4202560 2155 0 0 0 1251 0 0 0 25 0 4 0 634343993 190033920 13100 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 50.11
Current children cumulated vsize (KiB) 185580

[startup+25.5008 s]
/proc/loadavg: 8.23 8.14 7.85 9/371 2034
/proc/meminfo: memFree=25646036/32951124 swapFree=63083164/67111528
[pid=1998] ppid=1996 vsize=187644 CPUtime=101.14 cores=1,3,5,7
/proc/1998/stat : 1998 (claspmt) R 1996 1998 1566 0 -1 4202496 16107 0 0 0 10109 5 0 0 25 0 4 0 634343976 192147456 15723 33554432000 4194304 4964800 140734932654240 18446744073709551615 4627540 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/1998/statm: 46911 15723 460 189 0 41260 0
[pid=1998/tid=2000] ppid=1996 vsize=187644 CPUtime=25.31 cores=1,3,5,7
/proc/1998/task/2000/stat : 2000 (claspmt) R 1996 1998 1566 0 -1 4202560 3588 0 0 0 2529 2 0 0 25 0 4 0 634343993 192147456 15723 33554432000 4194304 4964800 140734932654240 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1998/tid=2001] ppid=1996 vsize=187644 CPUtime=25.02 cores=1,3,5,7
/proc/1998/task/2001/stat : 2001 (claspmt) R 1996 1998 1566 0 -1 4202560 4472 0 0 0 2501 1 0 0 25 0 4 0 634343993 192147456 15723 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670295 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=1998/tid=2002] ppid=1996 vsize=187644 CPUtime=25.3 cores=1,3,5,7
/proc/1998/task/2002/stat : 2002 (claspmt) R 1996 1998 1566 0 -1 4202560 2633 0 0 0 2530 0 0 0 25 0 4 0 634343993 192147456 15723 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670645 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 101.14
Current children cumulated vsize (KiB) 187644

[startup+51.101 s]
/proc/loadavg: 8.21 8.14 7.85 9/371 2065
/proc/meminfo: memFree=25568176/32951124 swapFree=63083164/67111528
[pid=1998] ppid=1996 vsize=203976 CPUtime=203.24 cores=1,3,5,7
/proc/1998/stat : 1998 (claspmt) R 1996 1998 1566 0 -1 4202496 25969 0 0 0 20312 12 0 0 25 0 4 0 634343976 208871424 25099 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/1998/statm: 50994 25099 460 189 0 45343 0
[pid=1998/tid=2000] ppid=1996 vsize=203976 CPUtime=50.9 cores=1,3,5,7
/proc/1998/task/2000/stat : 2000 (claspmt) R 1996 1998 1566 0 -1 4202560 6238 0 0 0 5086 4 0 0 25 0 4 0 634343993 208871424 25099 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670437 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1998/tid=2001] ppid=1996 vsize=203976 CPUtime=50.32 cores=1,3,5,7
/proc/1998/task/2001/stat : 2001 (claspmt) R 1996 1998 1566 0 -1 4202560 7465 0 0 0 5030 2 0 0 25 0 4 0 634343993 208871424 25099 33554432000 4194304 4964800 140734932654240 18446744073709551615 4660236 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=1998/tid=2002] ppid=1996 vsize=203976 CPUtime=50.91 cores=1,3,5,7
/proc/1998/task/2002/stat : 2002 (claspmt) R 1996 1998 1566 0 -1 4202560 4629 0 0 0 5089 2 0 0 25 0 4 0 634343993 208871424 25099 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670335 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 203.24
Current children cumulated vsize (KiB) 203976

[startup+102.314 s]
/proc/loadavg: 8.23 8.16 7.87 9/371 2066
/proc/meminfo: memFree=25457780/32951124 swapFree=63083164/67111528
[pid=1998] ppid=1996 vsize=211648 CPUtime=407.47 cores=1,3,5,7
/proc/1998/stat : 1998 (claspmt) R 1996 1998 1566 0 -1 4202496 31011 0 0 0 40730 17 0 0 25 0 4 0 634343976 216727552 30026 33554432000 4194304 4964800 140734932654240 18446744073709551615 4334266 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/1998/statm: 52912 30026 460 189 0 47261 0
[pid=1998/tid=2000] ppid=1996 vsize=211648 CPUtime=102.11 cores=1,3,5,7
/proc/1998/task/2000/stat : 2000 (claspmt) R 1996 1998 1566 0 -1 4202560 7768 0 0 0 10205 6 0 0 25 0 4 0 634343993 216727552 30026 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1998/tid=2001] ppid=1996 vsize=211648 CPUtime=100.92 cores=1,3,5,7
/proc/1998/task/2001/stat : 2001 (claspmt) R 1996 1998 1566 0 -1 4202560 9414 0 0 0 10089 3 0 0 25 0 4 0 634343993 216727552 30026 33554432000 4194304 4964800 140734932654240 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=1998/tid=2002] ppid=1996 vsize=211648 CPUtime=102.11 cores=1,3,5,7
/proc/1998/task/2002/stat : 2002 (claspmt) R 1996 1998 1566 0 -1 4202560 5155 0 0 0 10209 2 0 0 25 0 4 0 634343993 216727552 30026 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 407.47
Current children cumulated vsize (KiB) 211648

[startup+162.301 s]
/proc/loadavg: 8.13 8.14 7.89 9/371 2068
/proc/meminfo: memFree=25343528/32951124 swapFree=63083164/67111528
[pid=1998] ppid=1996 vsize=222552 CPUtime=646.68 cores=1,3,5,7
/proc/1998/stat : 1998 (claspmt) R 1996 1998 1566 0 -1 4202496 37592 0 0 0 64646 22 0 0 25 0 4 0 634343976 227893248 36449 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670346 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/1998/statm: 55638 36449 460 189 0 49987 0
[pid=1998/tid=2000] ppid=1996 vsize=222552 CPUtime=162.09 cores=1,3,5,7
/proc/1998/task/2000/stat : 2000 (claspmt) R 1996 1998 1566 0 -1 4202560 9923 0 0 0 16202 7 0 0 25 0 4 0 634343993 227893248 36449 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1998/tid=2001] ppid=1996 vsize=222552 CPUtime=160.2 cores=1,3,5,7
/proc/1998/task/2001/stat : 2001 (claspmt) R 1996 1998 1566 0 -1 4202560 11139 0 0 0 16016 4 0 0 25 0 4 0 634343993 227893248 36449 33554432000 4194304 4964800 140734932654240 18446744073709551615 4321016 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=1998/tid=2002] ppid=1996 vsize=222552 CPUtime=162.08 cores=1,3,5,7
/proc/1998/task/2002/stat : 2002 (claspmt) R 1996 1998 1566 0 -1 4202560 6660 0 0 0 16203 5 0 0 25 0 4 0 634343993 227893248 36449 33554432000 4194304 4964800 140734932654240 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 646.68
Current children cumulated vsize (KiB) 222552

[startup+222.301 s]
/proc/loadavg: 8.13 8.14 7.90 9/371 2069
/proc/meminfo: memFree=25287812/32951124 swapFree=63083164/67111528
[pid=1998] ppid=1996 vsize=225596 CPUtime=885.94 cores=1,3,5,7
/proc/1998/stat : 1998 (claspmt) R 1996 1998 1566 0 -1 4202496 41214 0 0 0 88569 25 0 0 25 0 4 0 634343976 231010304 40032 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670323 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/1998/statm: 56399 40032 460 189 0 50748 0
[pid=1998/tid=2000] ppid=1996 vsize=225596 CPUtime=222.09 cores=1,3,5,7
/proc/1998/task/2000/stat : 2000 (claspmt) R 1996 1998 1566 0 -1 4202560 10672 0 0 0 22201 8 0 0 25 0 4 0 634343993 231010304 40032 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670417 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1998/tid=2001] ppid=1996 vsize=225596 CPUtime=219.49 cores=1,3,5,7
/proc/1998/task/2001/stat : 2001 (claspmt) R 1996 1998 1566 0 -1 4202560 12490 0 0 0 21944 5 0 0 25 0 4 0 634343993 231010304 40032 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=1998/tid=2002] ppid=1996 vsize=225596 CPUtime=222.06 cores=1,3,5,7
/proc/1998/task/2002/stat : 2002 (claspmt) R 1996 1998 1566 0 -1 4202560 7192 0 0 0 22201 5 0 0 25 0 4 0 634343993 231010304 40032 33554432000 4194304 4964800 140734932654240 18446744073709551615 4653247 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 885.94
Current children cumulated vsize (KiB) 225596

[startup+282.3 s]

################
# More data... #
################

[pid=1998] ppid=1996 vsize=382288 CPUtime=2560.86 cores=1,3,5,7
/proc/1998/stat : 1998 (claspmt) R 1996 1998 1566 0 -1 4202496 57478 0 0 0 256048 38 0 0 25 0 4 0 634343976 391462912 55550 33554432000 4194304 4964800 140734932654240 18446744073709551615 4320909 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/1998/statm: 95572 55550 460 189 0 89921 0
[pid=1998/tid=2000] ppid=1996 vsize=382288 CPUtime=642.04 cores=1,3,5,7
/proc/1998/task/2000/stat : 2000 (claspmt) R 1996 1998 1566 0 -1 4202560 14933 0 0 0 64193 11 0 0 25 0 4 0 634343993 391462912 55550 33554432000 4194304 4964800 140734932654240 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1998/tid=2001] ppid=1996 vsize=382288 CPUtime=634.53 cores=1,3,5,7
/proc/1998/task/2001/stat : 2001 (claspmt) R 1996 1998 1566 0 -1 4202560 17284 0 0 0 63447 6 0 0 25 0 4 0 634343993 391462912 55550 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=1998/tid=2002] ppid=1996 vsize=382288 CPUtime=641.99 cores=1,3,5,7
/proc/1998/task/2002/stat : 2002 (claspmt) R 1996 1998 1566 0 -1 4202560 9304 0 0 0 64191 8 0 0 25 0 4 0 634343993 391462912 55550 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 2560.86
Current children cumulated vsize (KiB) 382288

[startup+702.301 s]
/proc/loadavg: 8.12 8.12 7.97 9/371 2082
/proc/meminfo: memFree=25083072/32951124 swapFree=63083164/67111528
[pid=1998] ppid=1996 vsize=384364 CPUtime=2800.12 cores=1,3,5,7
/proc/1998/stat : 1998 (claspmt) R 1996 1998 1566 0 -1 4202496 58924 0 0 0 279973 39 0 0 25 0 4 0 634343976 393588736 56664 33554432000 4194304 4964800 140734932654240 18446744073709551615 4321054 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/1998/statm: 96091 56664 460 189 0 90440 0
[pid=1998/tid=2000] ppid=1996 vsize=384364 CPUtime=702.01 cores=1,3,5,7
/proc/1998/task/2000/stat : 2000 (claspmt) R 1996 1998 1566 0 -1 4202560 15415 0 0 0 70190 11 0 0 25 0 4 0 634343993 393588736 56664 33554432000 4194304 4964800 140734932654240 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1998/tid=2001] ppid=1996 vsize=384364 CPUtime=693.83 cores=1,3,5,7
/proc/1998/task/2001/stat : 2001 (claspmt) R 1996 1998 1566 0 -1 4202560 18243 0 0 0 69376 7 0 0 25 0 4 0 634343993 393588736 56664 33554432000 4194304 4964800 140734932654240 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=1998/tid=2002] ppid=1996 vsize=384364 CPUtime=701.96 cores=1,3,5,7
/proc/1998/task/2002/stat : 2002 (claspmt) R 1996 1998 1566 0 -1 4202560 9304 0 0 0 70188 8 0 0 25 0 4 0 634343993 393588736 56664 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670295 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 2800.12
Current children cumulated vsize (KiB) 384364

[startup+762.315 s]
/proc/loadavg: 8.10 8.11 7.97 9/371 2084
/proc/meminfo: memFree=25046324/32951124 swapFree=63083164/67111528
[pid=1998] ppid=1996 vsize=384888 CPUtime=3039.45 cores=1,3,5,7
/proc/1998/stat : 1998 (claspmt) R 1996 1998 1566 0 -1 4202496 59526 0 0 0 303905 40 0 0 25 0 4 0 634343976 394125312 57253 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/1998/statm: 96222 57253 460 189 0 90571 0
[pid=1998/tid=2000] ppid=1996 vsize=384888 CPUtime=762.03 cores=1,3,5,7
/proc/1998/task/2000/stat : 2000 (claspmt) R 1996 1998 1566 0 -1 4202560 15601 0 0 0 76191 12 0 0 25 0 4 0 634343993 394125312 57253 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670368 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1998/tid=2001] ppid=1996 vsize=384888 CPUtime=753.12 cores=1,3,5,7
/proc/1998/task/2001/stat : 2001 (claspmt) R 1996 1998 1566 0 -1 4202560 18537 0 0 0 75305 7 0 0 25 0 4 0 634343993 394125312 57253 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=1998/tid=2002] ppid=1996 vsize=384888 CPUtime=761.97 cores=1,3,5,7
/proc/1998/task/2002/stat : 2002 (claspmt) R 1996 1998 1566 0 -1 4202560 9386 0 0 0 76189 8 0 0 25 0 4 0 634343993 394125312 57253 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670534 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3039.45
Current children cumulated vsize (KiB) 384888

[startup+822.3 s]
/proc/loadavg: 8.04 8.09 7.97 9/371 2085
/proc/meminfo: memFree=25024448/32951124 swapFree=63083164/67111528
[pid=1998] ppid=1996 vsize=386520 CPUtime=3278.67 cores=1,3,5,7
/proc/1998/stat : 1998 (claspmt) R 1996 1998 1566 0 -1 4202496 61101 0 0 0 327825 42 0 0 25 0 4 0 634343976 395796480 58598 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/1998/statm: 96630 58598 460 189 0 90979 0
[pid=1998/tid=2000] ppid=1996 vsize=386520 CPUtime=822.01 cores=1,3,5,7
/proc/1998/task/2000/stat : 2000 (claspmt) R 1996 1998 1566 0 -1 4202560 16024 0 0 0 82189 12 0 0 25 0 4 0 634343993 395796480 58598 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1998/tid=2001] ppid=1996 vsize=386520 CPUtime=812.4 cores=1,3,5,7
/proc/1998/task/2001/stat : 2001 (claspmt) R 1996 1998 1566 0 -1 4202560 19227 0 0 0 81233 7 0 0 25 0 4 0 634343993 395796480 58598 33554432000 4194304 4964800 140734932654240 18446744073709551615 4321082 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=1998/tid=2002] ppid=1996 vsize=386520 CPUtime=821.95 cores=1,3,5,7
/proc/1998/task/2002/stat : 2002 (claspmt) R 1996 1998 1566 0 -1 4202560 9638 0 0 0 82186 9 0 0 25 0 4 0 634343993 395796480 58598 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670363 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3278.67
Current children cumulated vsize (KiB) 386520

[startup+882.301 s]
/proc/loadavg: 8.12 8.10 7.99 9/371 2087
/proc/meminfo: memFree=25012380/32951124 swapFree=63083164/67111528
[pid=1998] ppid=1996 vsize=387232 CPUtime=3517.92 cores=1,3,5,7
/proc/1998/stat : 1998 (claspmt) R 1996 1998 1566 0 -1 4202496 61994 0 0 0 351749 43 0 0 25 0 4 0 634343976 396525568 59395 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/1998/statm: 96808 59395 460 189 0 91157 0
[pid=1998/tid=2000] ppid=1996 vsize=387232 CPUtime=882.01 cores=1,3,5,7
/proc/1998/task/2000/stat : 2000 (claspmt) R 1996 1998 1566 0 -1 4202560 16369 0 0 0 88188 13 0 0 25 0 4 0 634343993 396525568 59395 33554432000 4194304 4964800 140734932654240 18446744073709551615 4342193 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1998/tid=2001] ppid=1996 vsize=387232 CPUtime=871.69 cores=1,3,5,7
/proc/1998/task/2001/stat : 2001 (claspmt) R 1996 1998 1566 0 -1 4202560 19559 0 0 0 87161 8 0 0 25 0 4 0 634343993 396525568 59395 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670361 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=1998/tid=2002] ppid=1996 vsize=387232 CPUtime=881.92 cores=1,3,5,7
/proc/1998/task/2002/stat : 2002 (claspmt) R 1996 1998 1566 0 -1 4202560 9741 0 0 0 88183 9 0 0 25 0 4 0 634343993 396525568 59395 33554432000 4194304 4964800 140734932654240 18446744073709551615 4628024 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3517.92
Current children cumulated vsize (KiB) 387232

[startup+942.301 s]
/proc/loadavg: 8.09 8.09 7.99 9/371 2088
/proc/meminfo: memFree=24972396/32951124 swapFree=63083164/67111528
[pid=1998] ppid=1996 vsize=392864 CPUtime=3757.21 cores=1,3,5,7
/proc/1998/stat : 1998 (claspmt) R 1996 1998 1566 0 -1 4202496 64523 0 0 0 375675 46 0 0 25 0 4 0 634343976 402292736 61665 33554432000 4194304 4964800 140734932654240 18446744073709551615 264726067668 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/1998/statm: 98216 61665 460 189 0 92565 0
[pid=1998/tid=2000] ppid=1996 vsize=392864 CPUtime=942 cores=1,3,5,7
/proc/1998/task/2000/stat : 2000 (claspmt) R 1996 1998 1566 0 -1 4202560 16720 0 0 0 94187 13 0 0 25 0 4 0 634343993 402292736 61665 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1998/tid=2001] ppid=1996 vsize=392864 CPUtime=930.99 cores=1,3,5,7
/proc/1998/task/2001/stat : 2001 (claspmt) R 1996 1998 1566 0 -1 4202560 20601 0 0 0 93090 9 0 0 25 0 4 0 634343993 402292736 61665 33554432000 4194304 4964800 140734932654240 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=1998/tid=2002] ppid=1996 vsize=392864 CPUtime=941.91 cores=1,3,5,7
/proc/1998/task/2002/stat : 2002 (claspmt) R 1996 1998 1566 0 -1 4202560 9775 0 0 0 94182 9 0 0 25 0 4 0 634343993 402292736 61665 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670361 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3757.21
Current children cumulated vsize (KiB) 392864

[startup+1002.3 s]
/proc/loadavg: 8.12 8.10 8.00 9/371 2090
/proc/meminfo: memFree=24954872/32951124 swapFree=63083164/67111528
[pid=1998] ppid=1996 vsize=393660 CPUtime=3996.47 cores=1,3,5,7
/proc/1998/stat : 1998 (claspmt) R 1996 1998 1566 0 -1 4202496 64909 0 0 0 399601 46 0 0 25 0 4 0 634343976 403107840 62051 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670328 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/1998/statm: 98415 62051 460 189 0 92764 0
[pid=1998/tid=2000] ppid=1996 vsize=393660 CPUtime=1001.98 cores=1,3,5,7
/proc/1998/task/2000/stat : 2000 (claspmt) R 1996 1998 1566 0 -1 4202560 16851 0 0 0 100185 13 0 0 25 0 4 0 634343993 403107840 62051 33554432000 4194304 4964800 140734932654240 18446744073709551615 4321020 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1998/tid=2001] ppid=1996 vsize=393660 CPUtime=990.28 cores=1,3,5,7
/proc/1998/task/2001/stat : 2001 (claspmt) R 1996 1998 1566 0 -1 4202560 20851 0 0 0 99019 9 0 0 25 0 4 0 634343993 403107840 62051 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=1998/tid=2002] ppid=1996 vsize=393660 CPUtime=1001.9 cores=1,3,5,7
/proc/1998/task/2002/stat : 2002 (claspmt) R 1996 1998 1566 0 -1 4202560 9779 0 0 0 100181 9 0 0 25 0 4 0 634343993 403107840 62051 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3996.47
Current children cumulated vsize (KiB) 393660

[startup+1062.3 s]
/proc/loadavg: 8.08 8.10 8.00 9/371 2092
/proc/meminfo: memFree=24951844/32951124 swapFree=63083164/67111528
[pid=1998] ppid=1996 vsize=393660 CPUtime=4235.75 cores=1,3,5,7
/proc/1998/stat : 1998 (claspmt) R 1996 1998 1566 0 -1 4202496 65701 0 0 0 423528 47 0 0 25 0 4 0 634343976 403107840 62651 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670335 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/1998/statm: 98415 62651 460 189 0 92764 0
[pid=1998/tid=2000] ppid=1996 vsize=393660 CPUtime=1061.98 cores=1,3,5,7
/proc/1998/task/2000/stat : 2000 (claspmt) R 1996 1998 1566 0 -1 4202560 17378 0 0 0 106185 13 0 0 25 0 4 0 634343993 403107840 62651 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1998/tid=2001] ppid=1996 vsize=393660 CPUtime=1049.57 cores=1,3,5,7
/proc/1998/task/2001/stat : 2001 (claspmt) R 1996 1998 1566 0 -1 4202560 20953 0 0 0 104948 9 0 0 25 0 4 0 634343993 403107840 62651 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=1998/tid=2002] ppid=1996 vsize=393660 CPUtime=1061.88 cores=1,3,5,7
/proc/1998/task/2002/stat : 2002 (claspmt) R 1996 1998 1566 0 -1 4202560 9863 0 0 0 106179 9 0 0 25 0 4 0 634343993 403107840 62651 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670310 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4235.75
Current children cumulated vsize (KiB) 393660

[startup+1122.3 s]
/proc/loadavg: 8.14 8.12 8.01 9/371 2093
/proc/meminfo: memFree=24948212/32951124 swapFree=63083164/67111528
[pid=1998] ppid=1996 vsize=395128 CPUtime=4475.01 cores=1,3,5,7
/proc/1998/stat : 1998 (claspmt) R 1996 1998 1566 0 -1 4202496 66694 0 0 0 447453 48 0 0 25 0 4 0 634343976 404611072 63548 33554432000 4194304 4964800 140734932654240 18446744073709551615 4320951 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/1998/statm: 98782 63548 460 189 0 93131 0
[pid=1998/tid=2000] ppid=1996 vsize=395128 CPUtime=1121.98 cores=1,3,5,7
/proc/1998/task/2000/stat : 2000 (claspmt) R 1996 1998 1566 0 -1 4202560 17693 0 0 0 112184 14 0 0 25 0 4 0 634343993 404611072 63548 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1998/tid=2001] ppid=1996 vsize=395128 CPUtime=1108.85 cores=1,3,5,7
/proc/1998/task/2001/stat : 2001 (claspmt) R 1996 1998 1566 0 -1 4202560 21611 0 0 0 110876 9 0 0 25 0 4 0 634343993 404611072 63548 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=1998/tid=2002] ppid=1996 vsize=395128 CPUtime=1121.87 cores=1,3,5,7
/proc/1998/task/2002/stat : 2002 (claspmt) R 1996 1998 1566 0 -1 4202560 9883 0 0 0 112178 9 0 0 25 0 4 0 634343993 404611072 63548 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4475.01
Current children cumulated vsize (KiB) 395128

[startup+1182.3 s]
/proc/loadavg: 8.09 8.11 8.01 9/371 2095
/proc/meminfo: memFree=24942344/32951124 swapFree=63083164/67111528
[pid=1998] ppid=1996 vsize=397024 CPUtime=4714.3 cores=1,3,5,7
/proc/1998/stat : 1998 (claspmt) R 1996 1998 1566 0 -1 4202496 68183 0 0 0 471381 49 0 0 25 0 4 0 634343976 406552576 64903 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670328 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/1998/statm: 99256 64903 460 189 0 93605 0
[pid=1998/tid=2000] ppid=1996 vsize=397024 CPUtime=1181.97 cores=1,3,5,7
/proc/1998/task/2000/stat : 2000 (claspmt) R 1996 1998 1566 0 -1 4202560 18095 0 0 0 118183 14 0 0 25 0 4 0 634343993 406552576 64903 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1998/tid=2001] ppid=1996 vsize=397024 CPUtime=1168.15 cores=1,3,5,7
/proc/1998/task/2001/stat : 2001 (claspmt) R 1996 1998 1566 0 -1 4202560 22687 0 0 0 116805 10 0 0 25 0 4 0 634343993 406552576 64903 33554432000 4194304 4964800 140734932654240 18446744073709551615 4669136 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=1998/tid=2002] ppid=1996 vsize=397024 CPUtime=1181.86 cores=1,3,5,7
/proc/1998/task/2002/stat : 2002 (claspmt) R 1996 1998 1566 0 -1 4202560 9883 0 0 0 118177 9 0 0 25 0 4 0 634343993 406552576 64903 33554432000 4194304 4964800 140734932654240 18446744073709551615 4627281 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4714.3
Current children cumulated vsize (KiB) 397024



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.8 s]
/proc/loadavg: 8.14 8.12 8.01 9/371 2095
/proc/meminfo: memFree=24935268/32951124 swapFree=63083164/67111528
[pid=1998] ppid=1996 vsize=397536 CPUtime=4800.03 cores=1,3,5,7
/proc/1998/stat : 1998 (claspmt) R 1996 1998 1566 0 -1 4202496 68315 0 0 0 479954 49 0 0 25 0 4 0 634343976 407076864 65035 33554432000 4194304 4964800 140734932654240 18446744073709551615 4690026 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/1998/statm: 99384 65035 460 189 0 93733 0
[pid=1998/tid=2000] ppid=1996 vsize=397536 CPUtime=1203.48 cores=1,3,5,7
/proc/1998/task/2000/stat : 2000 (claspmt) R 1996 1998 1566 0 -1 4202560 18224 0 0 0 120334 14 0 0 25 0 4 0 634343993 407076864 65035 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=1998/tid=2001] ppid=1996 vsize=397536 CPUtime=1189.39 cores=1,3,5,7
/proc/1998/task/2001/stat : 2001 (claspmt) R 1996 1998 1566 0 -1 4202560 22689 0 0 0 118929 10 0 0 25 0 4 0 634343993 407076864 65035 33554432000 4194304 4964800 140734932654240 18446744073709551615 4660157 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=1998/tid=2002] ppid=1996 vsize=397536 CPUtime=1203.37 cores=1,3,5,7
/proc/1998/task/2002/stat : 2002 (claspmt) R 1996 1998 1566 0 -1 4202560 9884 0 0 0 120328 9 0 0 25 0 4 0 634343993 407076864 65035 33554432000 4194304 4964800 140734932654240 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4800.03
Current children cumulated vsize (KiB) 397536

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child status: 0
Real time (s): 1203.88
CPU time (s): 4800.15
CPU user time (s): 4799.65
CPU system time (s): 0.505923
CPU usage (%): 398.724
Max. virtual memory (cumulated for all children) (KiB): 397536

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4799.65
system time used= 0.505923
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 68327
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 8808
involuntary context switches= 24368

runsolver used 3.82642 second user time and 10.4204 second system time

The end

Launcher Data 

Begin job on node141 at 2011-05-15 02:28:04
IDJOB=3373155
IDBENCH=71300
IDSOLVER=1853
FILE ID=node141/3373155-1305419284
RUNJOBID= node141-1305418075-1880
PBS_JOBID= 13324546
Free space on /tmp= 73288 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT09/CRAFTED/rbsat/random/unforced/rbsat-v945c61409g6.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3373155-1305419284/watcher-3373155-1305419284 -o /tmp/evaluation-result-3373155-1305419284/solver-3373155-1305419284 -C 4800 -W 1300 -M 15500  HOME/claspmt -t4 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3373155-1305419284.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 2094ee8d4e443e860293b947e469bd81
RANDOM SEED=1376144248

node141.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5362.96
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      25058440 kB
Buffers:        238352 kB
Cached:        4789600 kB
SwapCached:      15708 kB
Active:        2652516 kB
Inactive:      5000928 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      25058440 kB
SwapTotal:    67111528 kB
SwapFree:     63083164 kB
Dirty:            1596 kB
Writeback:           0 kB
AnonPages:     2616816 kB
Mapped:          13616 kB
Slab:           135640 kB
PageTables:      38928 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 86935124 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73296 MiB
End job on node141 at 2011-05-15 02:48:09