Trace number 3381166

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 40001 5010.94

General information on the benchmark

NameSAT03/handmade/anton/UNSAT/
SGI_30_80_28_70_8-dir.shuffled-as.sat03-215.cnf
MD5SUM731331ace9dcb7404987f98927f5c914
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables840
Number of clauses72688
Sum of the clauses size146160
Maximum clause length30
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 272660
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 528

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...506590/instance-3381166-1305506590.cnf
0.29/0.33	c Solving...
40000.66/5010.62	
40000.66/5010.62	*** INTERRUPTED! ***
40000.66/5010.62	*** Shutting down threads...
40001.01/5010.94	s UNKNOWN
40001.01/5010.94	
40001.01/5010.94	c INTERRUPTED   : 1
40001.01/5010.94	c Models        : 0+    
40001.01/5010.94	c Time          : 5010.626s (Solving: 5010.29s 1st Model: 0.00s Unsat: 0.00s)
40001.01/5010.94	c CPU Time      : 40000.720s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3381166-1305506590/watcher-3381166-1305506590 -o /tmp/evaluation-result-3381166-1305506590/solver-3381166-1305506590 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3381166-1305506590.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.81 7.80 7.92 1/447 20810
/proc/meminfo: memFree=24976512/32951124 swapFree=53501244/67111528
[pid=20810] ppid=20808 vsize=23688 CPUtime=0.01 cores=0-7
/proc/20810/stat : 20810 (claspmt) R 20808 20810 20784 0 -1 4202496 1459 0 0 0 1 0 0 0 25 0 1 0 662261609 24256512 1091 33554432000 4194304 4964800 140737022621168 18446744073709551615 216634176197 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/20810/statm: 5922 1091 386 189 0 796 0

[startup+0.058705 s]
/proc/loadavg: 6.81 7.80 7.92 1/447 20810
/proc/meminfo: memFree=24976512/32951124 swapFree=53501244/67111528
[pid=20810] ppid=20808 vsize=25728 CPUtime=0.05 cores=0-7
/proc/20810/stat : 20810 (claspmt) R 20808 20810 20784 0 -1 4202496 1971 0 0 0 5 0 0 0 25 0 1 0 662261609 26345472 1596 33554432000 4194304 4964800 140737022621168 18446744073709551615 4607590 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/20810/statm: 6432 1596 399 189 0 1306 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 25728

[startup+0.100709 s]
/proc/loadavg: 6.81 7.80 7.92 1/447 20810
/proc/meminfo: memFree=24976512/32951124 swapFree=53501244/67111528
[pid=20810] ppid=20808 vsize=25728 CPUtime=0.09 cores=0-7
/proc/20810/stat : 20810 (claspmt) R 20808 20810 20784 0 -1 4202496 1971 0 0 0 9 0 0 0 25 0 1 0 662261609 26345472 1596 33554432000 4194304 4964800 140737022621168 18446744073709551615 4607549 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/20810/statm: 6432 1596 399 189 0 1306 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 25728

[startup+0.300662 s]
/proc/loadavg: 6.81 7.80 7.92 1/447 20810
/proc/meminfo: memFree=24976512/32951124 swapFree=53501244/67111528
[pid=20810] ppid=20808 vsize=25992 CPUtime=0.29 cores=0-7
/proc/20810/stat : 20810 (claspmt) R 20808 20810 20784 0 -1 4202496 2068 0 0 0 29 0 0 0 25 0 1 0 662261609 26615808 1693 33554432000 4194304 4964800 140737022621168 18446744073709551615 4600405 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/20810/statm: 6498 1693 400 189 0 1372 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 25992

[startup+0.868351 s]
/proc/loadavg: 6.81 7.80 7.92 1/447 20810
/proc/meminfo: memFree=24976512/32951124 swapFree=53501244/67111528
[pid=20810] ppid=20808 vsize=390796 CPUtime=4.46 cores=0-7
/proc/20810/stat : 20810 (claspmt) R 20808 20810 20784 0 -1 4202496 13311 0 0 0 441 5 0 0 18 0 8 0 662261609 400175104 12769 33554432000 4194304 4964800 140737022621168 18446744073709551615 4320966 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/20810/statm: 98194 14714 464 189 0 92543 0
Current children cumulated CPU time (s) 4.46
Current children cumulated vsize (KiB) 390796

[startup+1.51741 s]
/proc/loadavg: 6.81 7.80 7.92 9/455 20818
/proc/meminfo: memFree=24901064/32951124 swapFree=53501244/67111528
[pid=20810] ppid=20808 vsize=397132 CPUtime=9.44 cores=0-7
/proc/20810/stat : 20810 (claspmt) R 20808 20810 20784 0 -1 4202496 21636 0 0 0 936 8 0 0 19 0 8 0 662261609 406663168 21094 33554432000 4194304 4964800 140737022621168 18446744073709551615 4653482 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/20810/statm: 99283 21094 464 189 0 93632 0
[pid=20810/tid=20812] ppid=20808 vsize=397132 CPUtime=1.13 cores=0-7
/proc/20810/task/20812/stat : 20812 (claspmt) R 20808 20810 20784 0 -1 4202560 2193 0 0 0 112 1 0 0 18 0 8 0 662261643 406663168 21094 33554432000 4194304 4964800 140737022621168 18446744073709551615 4320872 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20810/tid=20813] ppid=20808 vsize=397132 CPUtime=1.14 cores=0-7
/proc/20810/task/20813/stat : 20813 (claspmt) R 20808 20810 20784 0 -1 4202560 1425 0 0 0 114 0 0 0 19 0 8 0 662261643 406663168 21094 33554432000 4194304 4964800 140737022621168 18446744073709551615 4627179 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=20810/tid=20814] ppid=20808 vsize=397132 CPUtime=1.14 cores=0-7
/proc/20810/task/20814/stat : 20814 (claspmt) R 20808 20810 20784 0 -1 4202560 2422 0 0 0 114 0 0 0 19 0 8 0 662261643 406663168 21094 33554432000 4194304 4964800 140737022621168 18446744073709551615 4346000 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20810/tid=20815] ppid=20808 vsize=397132 CPUtime=1.11 cores=0-7
/proc/20810/task/20815/stat : 20815 (claspmt) R 20808 20810 20784 0 -1 4202560 2848 0 0 0 110 1 0 0 18 0 8 0 662261643 406663168 21094 33554432000 4194304 4964800 140737022621168 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20810/tid=20816] ppid=20808 vsize=397132 CPUtime=1.16 cores=0-7
/proc/20810/task/20816/stat : 20816 (claspmt) R 20808 20810 20784 0 -1 4202560 2554 0 0 0 115 1 0 0 19 0 8 0 662261643 406663168 21094 33554432000 4194304 4964800 140737022621168 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20810/tid=20817] ppid=20808 vsize=397132 CPUtime=1.13 cores=0-7
/proc/20810/task/20817/stat : 20817 (claspmt) R 20808 20810 20784 0 -1 4202560 2534 0 0 0 112 1 0 0 19 0 8 0 662261643 406663168 21094 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670273 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20810/tid=20818] ppid=20808 vsize=397132 CPUtime=1.12 cores=0-7
/proc/20810/task/20818/stat : 20818 (claspmt) R 20808 20810 20784 0 -1 4202560 2915 0 0 0 111 1 0 0 19 0 8 0 662261643 406663168 21094 33554432000 4194304 4964800 140737022621168 18446744073709551615 216634175624 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 9.44
Current children cumulated vsize (KiB) 397132

[startup+3.10113 s]
/proc/loadavg: 6.81 7.80 7.92 9/455 20818
/proc/meminfo: memFree=24865600/32951124 swapFree=53501244/67111528
[pid=20810] ppid=20808 vsize=402380 CPUtime=22.09 cores=0-7
/proc/20810/stat : 20810 (claspmt) R 20808 20810 20784 0 -1 4202496 28671 0 0 0 2198 11 0 0 23 0 8 0 662261609 412037120 28129 33554432000 4194304 4964800 140737022621168 18446744073709551615 4342189 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/20810/statm: 100595 28129 466 189 0 94944 0
[pid=20810/tid=20812] ppid=20808 vsize=402380 CPUtime=2.71 cores=0-7
/proc/20810/task/20812/stat : 20812 (claspmt) R 20808 20810 20784 0 -1 4202560 2945 0 0 0 270 1 0 0 20 0 8 0 662261643 412037120 28129 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=20810/tid=20813] ppid=20808 vsize=402380 CPUtime=2.72 cores=0-7
/proc/20810/task/20813/stat : 20813 (claspmt) R 20808 20810 20784 0 -1 4202560 1914 0 0 0 272 0 0 0 23 0 8 0 662261643 412037120 28129 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670332 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=20810/tid=20814] ppid=20808 vsize=402380 CPUtime=2.72 cores=0-7
/proc/20810/task/20814/stat : 20814 (claspmt) R 20808 20810 20784 0 -1 4202560 3193 0 0 0 271 1 0 0 22 0 8 0 662261643 412037120 28129 33554432000 4194304 4964800 140737022621168 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20810/tid=20815] ppid=20808 vsize=402380 CPUtime=2.69 cores=0-7
/proc/20810/task/20815/stat : 20815 (claspmt) R 20808 20810 20784 0 -1 4202560 3790 0 0 0 268 1 0 0 20 0 8 0 662261643 412037120 28129 33554432000 4194304 4964800 140737022621168 18446744073709551615 4627259 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20810/tid=20816] ppid=20808 vsize=402380 CPUtime=2.74 cores=0-7
/proc/20810/task/20816/stat : 20816 (claspmt) R 20808 20810 20784 0 -1 4202560 3650 0 0 0 273 1 0 0 23 0 8 0 662261643 412037120 28129 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670295 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20810/tid=20817] ppid=20808 vsize=402380 CPUtime=2.71 cores=0-7
/proc/20810/task/20817/stat : 20817 (claspmt) R 20808 20810 20784 0 -1 4202560 3440 0 0 0 270 1 0 0 22 0 8 0 662261643 412037120 28129 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20810/tid=20818] ppid=20808 vsize=402380 CPUtime=2.68 cores=0-7
/proc/20810/task/20818/stat : 20818 (claspmt) R 20808 20810 20784 0 -1 4202560 3913 0 0 0 267 1 0 0 22 0 8 0 662261643 412037120 28129 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 22.09
Current children cumulated vsize (KiB) 402380

[startup+6.30057 s]
/proc/loadavg: 6.90 7.80 7.92 9/455 20818
/proc/meminfo: memFree=24837840/32951124 swapFree=53501244/67111528
[pid=20810] ppid=20808 vsize=408712 CPUtime=47.63 cores=0-7
/proc/20810/stat : 20810 (claspmt) R 20808 20810 20784 0 -1 4202496 37602 0 0 0 4747 16 0 0 25 0 8 0 662261609 418521088 37060 33554432000 4194304 4964800 140737022621168 18446744073709551615 4320934 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/20810/statm: 102178 37061 466 189 0 96527 0
[pid=20810/tid=20812] ppid=20808 vsize=408712 CPUtime=5.85 cores=0-7
/proc/20810/task/20812/stat : 20812 (claspmt) R 20808 20810 20784 0 -1 4202560 3555 0 0 0 584 1 0 0 21 0 8 0 662261643 418521088 37061 33554432000 4194304 4964800 140737022621168 18446744073709551615 4342174 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=20810/tid=20813] ppid=20808 vsize=408712 CPUtime=5.92 cores=0-7
/proc/20810/task/20813/stat : 20813 (claspmt) R 20808 20810 20784 0 -1 4202560 2659 0 0 0 592 0 0 0 25 0 8 0 662261643 418521088 37061 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=20810/tid=20814] ppid=20808 vsize=408712 CPUtime=5.92 cores=0-7
/proc/20810/task/20814/stat : 20814 (claspmt) R 20808 20810 20784 0 -1 4202560 4183 0 0 0 591 1 0 0 25 0 8 0 662261643 418521088 37061 33554432000 4194304 4964800 140737022621168 18446744073709551615 4653307 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20810/tid=20815] ppid=20808 vsize=408712 CPUtime=5.89 cores=0-7
/proc/20810/task/20815/stat : 20815 (claspmt) R 20808 20810 20784 0 -1 4202560 5059 0 0 0 588 1 0 0 25 0 8 0 662261643 418521088 37061 33554432000 4194304 4964800 140737022621168 18446744073709551615 216634176197 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20810/tid=20816] ppid=20808 vsize=408712 CPUtime=5.94 cores=0-7
/proc/20810/task/20816/stat : 20816 (claspmt) R 20808 20810 20784 0 -1 4202560 5119 0 0 0 592 2 0 0 25 0 8 0 662261643 418521088 37061 33554432000 4194304 4964800 140737022621168 18446744073709551615 4669140 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20810/tid=20817] ppid=20808 vsize=408712 CPUtime=5.91 cores=0-7
/proc/20810/task/20817/stat : 20817 (claspmt) R 20808 20810 20784 0 -1 4202560 4497 0 0 0 589 2 0 0 25 0 8 0 662261643 418521088 37061 33554432000 4194304 4964800 140737022621168 18446744073709551615 4332499 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20810/tid=20818] ppid=20808 vsize=408712 CPUtime=5.88 cores=0-7
/proc/20810/task/20818/stat : 20818 (claspmt) R 20808 20810 20784 0 -1 4202560 5469 0 0 0 586 2 0 0 25 0 8 0 662261643 418521088 37061 33554432000 4194304 4964800 140737022621168 18446744073709551615 4321016 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 47.63
Current children cumulated vsize (KiB) 408712

[startup+12.7004 s]
/proc/loadavg: 6.99 7.80 7.92 9/455 20819
/proc/meminfo: memFree=24776436/32951124 swapFree=53501244/67111528
[pid=20810] ppid=20808 vsize=478940 CPUtime=98.71 cores=0-7
/proc/20810/stat : 20810 (claspmt) R 20808 20810 20784 0 -1 4202496 51698 0 0 0 9847 24 0 0 25 0 8 0 662261609 490434560 51156 33554432000 4194304 4964800 140737022621168 18446744073709551615 4320909 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/20810/statm: 119735 51156 466 189 0 114084 0
[pid=20810/tid=20812] ppid=20808 vsize=478940 CPUtime=12.16 cores=0-7
/proc/20810/task/20812/stat : 20812 (claspmt) R 20808 20810 20784 0 -1 4202560 5247 0 0 0 1214 2 0 0 25 0 8 0 662261643 490434560 51156 33554432000 4194304 4964800 140737022621168 18446744073709551615 4647578 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=20810/tid=20813] ppid=20808 vsize=478940 CPUtime=12.32 cores=0-7
/proc/20810/task/20813/stat : 20813 (claspmt) R 20808 20810 20784 0 -1 4202560 3429 0 0 0 1231 1 0 0 25 0 8 0 662261643 490434560 51156 33554432000 4194304 4964800 140737022621168 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=20810/tid=20814] ppid=20808 vsize=478940 CPUtime=12.32 cores=0-7
/proc/20810/task/20814/stat : 20814 (claspmt) R 20808 20810 20784 0 -1 4202560 5747 0 0 0 1230 2 0 0 25 0 8 0 662261643 490434560 51156 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20810/tid=20815] ppid=20808 vsize=478940 CPUtime=12.28 cores=0-7
/proc/20810/task/20815/stat : 20815 (claspmt) R 20808 20810 20784 0 -1 4202560 6076 0 0 0 1227 1 0 0 25 0 8 0 662261643 490434560 51156 33554432000 4194304 4964800 140737022621168 18446744073709551615 4653186 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20810/tid=20816] ppid=20808 vsize=478940 CPUtime=12.34 cores=0-7
/proc/20810/task/20816/stat : 20816 (claspmt) R 20808 20810 20784 0 -1 4202560 7196 0 0 0 1231 3 0 0 25 0 8 0 662261643 490434560 51156 33554432000 4194304 4964800 140737022621168 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20810/tid=20817] ppid=20808 vsize=478940 CPUtime=12.3 cores=0-7
/proc/20810/task/20817/stat : 20817 (claspmt) R 20808 20810 20784 0 -1 4202560 6337 0 0 0 1227 3 0 0 25 0 8 0 662261643 490434560 51156 33554432000 4194304 4964800 140737022621168 18446744073709551615 4325338 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20810/tid=20818] ppid=20808 vsize=478940 CPUtime=12.29 cores=0-7
/proc/20810/task/20818/stat : 20818 (claspmt) R 20808 20810 20784 0 -1 4202560 7283 0 0 0 1225 4 0 0 25 0 8 0 662261643 490434560 51156 33554432000 4194304 4964800 140737022621168 18446744073709551615 4321016 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 98.71
Current children cumulated vsize (KiB) 478940

[startup+25.5012 s]
/proc/loadavg: 7.22 7.82 7.93 9/455 20819
/proc/meminfo: memFree=24719900/32951124 swapFree=53501244/67111528
[pid=20810] ppid=20808 vsize=481304 CPUtime=200.89 cores=0-7
/proc/20810/stat : 20810 (claspmt) R 20808 20810 20784 0 -1 4202496 66098 0 0 0 20055 34 0 0 25 0 8 0 662261609 492855296 65556 33554432000 4194304 4964800 140737022621168 18446744073709551615 4320909 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/20810/statm: 120326 65556 466 189 0 114675 0
[pid=20810/tid=20812] ppid=20808 vsize=481304 CPUtime=24.77 cores=0-7
/proc/20810/task/20812/stat : 20812 (claspmt) R 20808 20810 20784 0 -1 4202560 6941 0 0 0 2473 4 0 0 25 0 8 0 662261643 492855296 65556 33554432000 4194304 4964800 140737022621168 18446744073709551615 4653482 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=20810/tid=20813] ppid=20808 vsize=481304 CPUtime=25.12 cores=0-7
/proc/20810/task/20813/stat : 20813 (claspmt) R 20808 20810 20784 0 -1 4202560 4755 0 0 0 2510 2 0 0 25 0 8 0 662261643 492855296 65556 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670520 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=20810/tid=20814] ppid=20808 vsize=481304 CPUtime=25.11 cores=0-7
/proc/20810/task/20814/stat : 20814 (claspmt) R 20808 20810 20784 0 -1 4202560 8498 0 0 0 2508 3 0 0 25 0 8 0 662261643 492855296 65556 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670499 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20810/tid=20815] ppid=20808 vsize=481304 CPUtime=25.08 cores=0-7
/proc/20810/task/20815/stat : 20815 (claspmt) R 20808 20810 20784 0 -1 4202560 7003 0 0 0 2506 2 0 0 25 0 8 0 662261643 492855296 65556 33554432000 4194304 4964800 140737022621168 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20810/tid=20816] ppid=20808 vsize=481304 CPUtime=25.14 cores=0-7
/proc/20810/task/20816/stat : 20816 (claspmt) R 20808 20810 20784 0 -1 4202560 8526 0 0 0 2510 4 0 0 25 0 8 0 662261643 492855296 65556 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670338 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20810/tid=20817] ppid=20808 vsize=481304 CPUtime=25.1 cores=0-7
/proc/20810/task/20817/stat : 20817 (claspmt) R 20808 20810 20784 0 -1 4202560 8776 0 0 0 2504 6 0 0 22 0 8 0 662261643 492855296 65556 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20810/tid=20818] ppid=20808 vsize=481304 CPUtime=25.08 cores=0-7
/proc/20810/task/20818/stat : 20818 (claspmt) R 20808 20810 20784 0 -1 4202560 9102 0 0 0 2503 5 0 0 25 0 8 0 662261643 492855296 65556 33554432000 4194304 4964800 140737022621168 18446744073709551615 4342157 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 200.89
Current children cumulated vsize (KiB) 481304

[startup+51.1007 s]
/proc/loadavg: 7.53 7.84 7.93 10/455 20820
/proc/meminfo: memFree=24624864/32951124 swapFree=53501244/67111528
[pid=20810] ppid=20808 vsize=499136 CPUtime=405.24 cores=0-7
/proc/20810/stat : 20810 (claspmt) R 20808 20810 20784 0 -1 4202496 89217 0 0 0 40474 50 0 0 25 0 8 0 662261609 511115264 88623 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/20810/statm: 124784 88623 466 189 0 119133 0
[pid=20810/tid=20812] ppid=20808 vsize=499136 CPUtime=49.97 cores=0-7
/proc/20810/task/20812/stat : 20812 (claspmt) R 20808 20810 20784 0 -1 4202560 9325 0 0 0 4991 6 0 0 25 0 8 0 662261643 511115264 88623 33554432000 4194304 4964800 140737022621168 18446744073709551615 4644872 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=20810/tid=20813] ppid=20808 vsize=499136 CPUtime=50.71 cores=0-7
/proc/20810/task/20813/stat : 20813 (claspmt) R 20808 20810 20784 0 -1 4202560 6069 0 0 0 5069 2 0 0 25 0 8 0 662261643 511115264 88623 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=20810/tid=20814] ppid=20808 vsize=499136 CPUtime=50.7 cores=0-7
/proc/20810/task/20814/stat : 20814 (claspmt) R 20808 20810 20784 0 -1 4202560 10988 0 0 0 5066 4 0 0 25 0 8 0 662261643 511115264 88623 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20810/tid=20815] ppid=20808 vsize=499136 CPUtime=50.67 cores=0-7
/proc/20810/task/20815/stat : 20815 (claspmt) R 20808 20810 20784 0 -1 4202560 9217 0 0 0 5064 3 0 0 25 0 8 0 662261643 511115264 88623 33554432000 4194304 4964800 140737022621168 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20810/tid=20816] ppid=20808 vsize=499136 CPUtime=50.73 cores=0-7
/proc/20810/task/20816/stat : 20816 (claspmt) R 20808 20810 20784 0 -1 4202560 11774 0 0 0 5066 7 0 0 18 0 8 0 662261643 511115264 88623 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20810/tid=20817] ppid=20808 vsize=499136 CPUtime=50.69 cores=0-7
/proc/20810/task/20817/stat : 20817 (claspmt) R 20808 20810 20784 0 -1 4202560 12488 0 0 0 5061 8 0 0 25 0 8 0 662261643 511115264 88623 33554432000 4194304 4964800 140737022621168 18446744073709551615 4320966 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20810/tid=20818] ppid=20808 vsize=499136 CPUtime=50.68 cores=0-7
/proc/20810/task/20818/stat : 20818 (claspmt) R 20808 20810 20784 0 -1 4202560 13159 0 0 0 5060 8 0 0 18 0 8 0 662261643 511115264 88623 33554432000 4194304 4964800 140737022621168 18446744073709551615 4321023 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 405.24

################
# More data... #
################

/proc/20810/task/20817/stat : 20817 (claspmt) R 20808 20810 20784 0 -1 4202560 50910 0 0 0 465216 75 0 0 25 0 8 0 662261643 2144616448 455521 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=20810/tid=20818] ppid=20808 vsize=2094352 CPUtime=4652.44 cores=0-7
/proc/20810/task/20818/stat : 20818 (claspmt) R 20808 20810 20784 0 -1 4202560 61177 0 0 0 465167 77 0 0 25 0 8 0 662261643 2144616448 455521 33554432000 4194304 4964800 140737022621168 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 37222.8
Current children cumulated vsize (KiB) 2094352

[startup+4722.3 s]
/proc/loadavg: 8.06 8.07 8.00 9/455 20965
/proc/meminfo: memFree=23150588/32951124 swapFree=53501244/67111528
[pid=20810] ppid=20808 vsize=2094352 CPUtime=37701.9 cores=0-7
/proc/20810/stat : 20810 (claspmt) R 20808 20810 20784 0 -1 4202496 466065 0 0 0 3769700 489 0 0 25 0 8 0 662261609 2144616448 455548 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/20810/statm: 523588 455548 466 189 0 517937 0
[pid=20810/tid=20812] ppid=20808 vsize=2094352 CPUtime=4692.54 cores=0-7
/proc/20810/task/20812/stat : 20812 (claspmt) R 20808 20810 20784 0 -1 4202560 43732 0 0 0 469198 56 0 0 25 0 8 0 662261643 2144616448 455548 33554432000 4194304 4964800 140737022621168 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=20810/tid=20813] ppid=20808 vsize=2094352 CPUtime=4710.51 cores=0-7
/proc/20810/task/20813/stat : 20813 (claspmt) R 20808 20810 20784 0 -1 4202560 48398 0 0 0 471029 22 0 0 25 0 8 0 662261643 2144616448 455548 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20810/tid=20814] ppid=20808 vsize=2094352 CPUtime=4720.43 cores=0-7
/proc/20810/task/20814/stat : 20814 (claspmt) R 20808 20810 20784 0 -1 4202560 48592 0 0 0 471990 53 0 0 25 0 8 0 662261643 2144616448 455548 33554432000 4194304 4964800 140737022621168 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20810/tid=20815] ppid=20808 vsize=2094352 CPUtime=4709.63 cores=0-7
/proc/20810/task/20815/stat : 20815 (claspmt) R 20808 20810 20784 0 -1 4202560 35644 0 0 0 470927 36 0 0 25 0 8 0 662261643 2144616448 455548 33554432000 4194304 4964800 140737022621168 18446744073709551615 4320976 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20810/tid=20816] ppid=20808 vsize=2094352 CPUtime=4721.52 cores=0-7
/proc/20810/task/20816/stat : 20816 (claspmt) R 20808 20810 20784 0 -1 4202560 46508 0 0 0 472111 41 0 0 25 0 8 0 662261643 2144616448 455548 33554432000 4194304 4964800 140737022621168 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20810/tid=20817] ppid=20808 vsize=2094352 CPUtime=4712.88 cores=0-7
/proc/20810/task/20817/stat : 20817 (claspmt) R 20808 20810 20784 0 -1 4202560 50919 0 0 0 471213 75 0 0 25 0 8 0 662261643 2144616448 455548 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=20810/tid=20818] ppid=20808 vsize=2094352 CPUtime=4712.45 cores=0-7
/proc/20810/task/20818/stat : 20818 (claspmt) R 20808 20810 20784 0 -1 4202560 61177 0 0 0 471167 78 0 0 25 0 8 0 662261643 2144616448 455548 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 37701.9
Current children cumulated vsize (KiB) 2094352

[startup+4782.3 s]
/proc/loadavg: 8.18 8.11 8.02 9/455 21440
/proc/meminfo: memFree=23147092/32951124 swapFree=53501244/67111528
[pid=20810] ppid=20808 vsize=2096028 CPUtime=38178.5 cores=0-7
/proc/20810/stat : 20810 (claspmt) R 20808 20810 20784 0 -1 4202496 466599 0 0 0 3817357 491 0 0 25 0 8 0 662261609 2146332672 456082 33554432000 4194304 4964800 140737022621168 18446744073709551615 4320951 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/20810/statm: 524007 456082 466 189 0 518356 0
[pid=20810/tid=20812] ppid=20808 vsize=2096028 CPUtime=4752.01 cores=0-7
/proc/20810/task/20812/stat : 20812 (claspmt) R 20808 20810 20784 0 -1 4202560 43956 0 0 0 475144 57 0 0 25 0 8 0 662261643 2146332672 456082 33554432000 4194304 4964800 140737022621168 18446744073709551615 4659391 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=20810/tid=20813] ppid=20808 vsize=2096028 CPUtime=4770.33 cores=0-7
/proc/20810/task/20813/stat : 20813 (claspmt) R 20808 20810 20784 0 -1 4202560 48398 0 0 0 477011 22 0 0 25 0 8 0 662261643 2146332672 456082 33554432000 4194304 4964800 140737022621168 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20810/tid=20814] ppid=20808 vsize=2096028 CPUtime=4780.38 cores=0-7
/proc/20810/task/20814/stat : 20814 (claspmt) R 20808 20810 20784 0 -1 4202560 48666 0 0 0 477984 54 0 0 25 0 8 0 662261643 2146332672 456082 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670335 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20810/tid=20815] ppid=20808 vsize=2096028 CPUtime=4768.87 cores=0-7
/proc/20810/task/20815/stat : 20815 (claspmt) R 20808 20810 20784 0 -1 4202560 35765 0 0 0 476851 36 0 0 25 0 8 0 662261643 2146332672 456082 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=20810/tid=20816] ppid=20808 vsize=2096028 CPUtime=4781.49 cores=0-7
/proc/20810/task/20816/stat : 20816 (claspmt) R 20808 20810 20784 0 -1 4202560 46619 0 0 0 478108 41 0 0 25 0 8 0 662261643 2146332672 456082 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20810/tid=20817] ppid=20808 vsize=2096028 CPUtime=4771.35 cores=0-7
/proc/20810/task/20817/stat : 20817 (claspmt) R 20808 20810 20784 0 -1 4202560 50919 0 0 0 477060 75 0 0 25 0 8 0 662261643 2146332672 456082 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=20810/tid=20818] ppid=20808 vsize=2096028 CPUtime=4772.27 cores=0-7
/proc/20810/task/20818/stat : 20818 (claspmt) R 20808 20810 20784 0 -1 4202560 61181 0 0 0 477149 78 0 0 25 0 8 0 662261643 2146332672 456082 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 38178.5
Current children cumulated vsize (KiB) 2096028

[startup+4842.3 s]
/proc/loadavg: 8.28 8.14 8.03 11/456 21444
/proc/meminfo: memFree=23090912/32951124 swapFree=53501244/67111528
[pid=20810] ppid=20808 vsize=2178256 CPUtime=38657 cores=0-7
/proc/20810/stat : 20810 (claspmt) R 20808 20810 20784 0 -1 4202496 475684 0 0 0 3865200 503 0 0 25 0 8 0 662261609 2230534144 465167 33554432000 4194304 4964800 140737022621168 18446744073709551615 4627252 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/20810/statm: 544564 465167 466 189 0 538913 0
[pid=20810/tid=20812] ppid=20808 vsize=2178256 CPUtime=4812.01 cores=0-7
/proc/20810/task/20812/stat : 20812 (claspmt) R 20808 20810 20784 0 -1 4202560 43968 0 0 0 481144 57 0 0 25 0 8 0 662261643 2230534144 465167 33554432000 4194304 4964800 140737022621168 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=20810/tid=20813] ppid=20808 vsize=2178256 CPUtime=4829.78 cores=0-7
/proc/20810/task/20813/stat : 20813 (claspmt) R 20808 20810 20784 0 -1 4202560 48597 0 0 0 482956 22 0 0 25 0 8 0 662261643 2230534144 465167 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20810/tid=20814] ppid=20808 vsize=2178256 CPUtime=4840.35 cores=0-7
/proc/20810/task/20814/stat : 20814 (claspmt) R 20808 20810 20784 0 -1 4202560 48666 0 0 0 483981 54 0 0 25 0 8 0 662261643 2230534144 465167 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20810/tid=20815] ppid=20808 vsize=2178256 CPUtime=4828.01 cores=0-7
/proc/20810/task/20815/stat : 20815 (claspmt) R 20808 20810 20784 0 -1 4202560 35789 0 0 0 482765 36 0 0 25 0 8 0 662261643 2230534144 465167 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=20810/tid=20816] ppid=20808 vsize=2178256 CPUtime=4841.47 cores=0-7
/proc/20810/task/20816/stat : 20816 (claspmt) R 20808 20810 20784 0 -1 4202560 46936 0 0 0 484106 41 0 0 25 0 8 0 662261643 2230534144 465167 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20810/tid=20817] ppid=20808 vsize=2178256 CPUtime=4831.35 cores=0-7
/proc/20810/task/20817/stat : 20817 (claspmt) R 20808 20810 20784 0 -1 4202560 51024 0 0 0 483060 75 0 0 25 0 8 0 662261643 2230534144 465167 33554432000 4194304 4964800 140737022621168 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=20810/tid=20818] ppid=20808 vsize=2178256 CPUtime=4832.27 cores=0-7
/proc/20810/task/20818/stat : 20818 (claspmt) R 20808 20810 20784 0 -1 4202560 63275 0 0 0 483148 79 0 0 25 0 8 0 662261643 2230534144 465167 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 38657
Current children cumulated vsize (KiB) 2178256

[startup+4902.3 s]
/proc/loadavg: 8.17 8.14 8.04 9/455 21446
/proc/meminfo: memFree=23111656/32951124 swapFree=53501244/67111528
[pid=20810] ppid=20808 vsize=2178256 CPUtime=39135.8 cores=0-7
/proc/20810/stat : 20810 (claspmt) R 20808 20810 20784 0 -1 4202496 475890 0 0 0 3913073 506 0 0 25 0 8 0 662261609 2230534144 465053 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/20810/statm: 544564 465053 466 189 0 538913 0
[pid=20810/tid=20812] ppid=20808 vsize=2178256 CPUtime=4872 cores=0-7
/proc/20810/task/20812/stat : 20812 (claspmt) R 20808 20810 20784 0 -1 4202560 44138 0 0 0 487142 58 0 0 25 0 8 0 662261643 2230534144 465053 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=20810/tid=20813] ppid=20808 vsize=2178256 CPUtime=4889.5 cores=0-7
/proc/20810/task/20813/stat : 20813 (claspmt) R 20808 20810 20784 0 -1 4202560 48604 0 0 0 488928 22 0 0 25 0 8 0 662261643 2230534144 465053 33554432000 4194304 4964800 140737022621168 18446744073709551615 4320966 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20810/tid=20814] ppid=20808 vsize=2178256 CPUtime=4900.33 cores=0-7
/proc/20810/task/20814/stat : 20814 (claspmt) R 20808 20810 20784 0 -1 4202560 48666 0 0 0 489979 54 0 0 25 0 8 0 662261643 2230534144 465053 33554432000 4194304 4964800 140737022621168 18446744073709551615 4321038 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20810/tid=20815] ppid=20808 vsize=2178256 CPUtime=4887.41 cores=0-7
/proc/20810/task/20815/stat : 20815 (claspmt) R 20808 20810 20784 0 -1 4202560 35818 0 0 0 488705 36 0 0 25 0 8 0 662261643 2230534144 465053 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20810/tid=20816] ppid=20808 vsize=2178256 CPUtime=4901.47 cores=0-7
/proc/20810/task/20816/stat : 20816 (claspmt) R 20808 20810 20784 0 -1 4202560 46936 0 0 0 490105 42 0 0 25 0 8 0 662261643 2230534144 465053 33554432000 4194304 4964800 140737022621168 18446744073709551615 4321116 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20810/tid=20817] ppid=20808 vsize=2178256 CPUtime=4891.34 cores=0-7
/proc/20810/task/20817/stat : 20817 (claspmt) R 20808 20810 20784 0 -1 4202560 51024 0 0 0 489059 75 0 0 25 0 8 0 662261643 2230534144 465053 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=20810/tid=20818] ppid=20808 vsize=2178256 CPUtime=4892.27 cores=0-7
/proc/20810/task/20818/stat : 20818 (claspmt) R 20808 20810 20784 0 -1 4202560 63275 0 0 0 489147 80 0 0 25 0 8 0 662261643 2230534144 465053 33554432000 4194304 4964800 140737022621168 18446744073709551615 4321082 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 39135.8
Current children cumulated vsize (KiB) 2178256

[startup+4962.3 s]
/proc/loadavg: 8.12 8.13 8.04 9/455 21448
/proc/meminfo: memFree=23111608/32951124 swapFree=53501244/67111528
[pid=20810] ppid=20808 vsize=2178256 CPUtime=39614.9 cores=0-7
/proc/20810/stat : 20810 (claspmt) R 20808 20810 20784 0 -1 4202496 475935 0 0 0 3960981 508 0 0 25 0 8 0 662261609 2230534144 465098 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670499 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/20810/statm: 544564 465098 466 189 0 538913 0
[pid=20810/tid=20812] ppid=20808 vsize=2178256 CPUtime=4931.99 cores=0-7
/proc/20810/task/20812/stat : 20812 (claspmt) R 20808 20810 20784 0 -1 4202560 44145 0 0 0 493141 58 0 0 25 0 8 0 662261643 2230534144 465098 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=20810/tid=20813] ppid=20808 vsize=2178256 CPUtime=4949.5 cores=0-7
/proc/20810/task/20813/stat : 20813 (claspmt) R 20808 20810 20784 0 -1 4202560 48622 0 0 0 494928 22 0 0 25 0 8 0 662261643 2230534144 465098 33554432000 4194304 4964800 140737022621168 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20810/tid=20814] ppid=20808 vsize=2178256 CPUtime=4960.32 cores=0-7
/proc/20810/task/20814/stat : 20814 (claspmt) R 20808 20810 20784 0 -1 4202560 48666 0 0 0 495977 55 0 0 25 0 8 0 662261643 2230534144 465098 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670520 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20810/tid=20815] ppid=20808 vsize=2178256 CPUtime=4947.42 cores=0-7
/proc/20810/task/20815/stat : 20815 (claspmt) R 20808 20810 20784 0 -1 4202560 35828 0 0 0 494706 36 0 0 25 0 8 0 662261643 2230534144 465098 33554432000 4194304 4964800 140737022621168 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20810/tid=20816] ppid=20808 vsize=2178256 CPUtime=4961.47 cores=0-7
/proc/20810/task/20816/stat : 20816 (claspmt) R 20808 20810 20784 0 -1 4202560 46937 0 0 0 496105 42 0 0 25 0 8 0 662261643 2230534144 465098 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20810/tid=20817] ppid=20808 vsize=2178256 CPUtime=4951.34 cores=0-7
/proc/20810/task/20817/stat : 20817 (claspmt) R 20808 20810 20784 0 -1 4202560 51028 0 0 0 495058 76 0 0 25 0 8 0 662261643 2230534144 465098 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=20810/tid=20818] ppid=20808 vsize=2178256 CPUtime=4952.26 cores=0-7
/proc/20810/task/20818/stat : 20818 (claspmt) R 20808 20810 20784 0 -1 4202560 63275 0 0 0 495146 80 0 0 25 0 8 0 662261643 2230534144 465098 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670295 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 39614.9
Current children cumulated vsize (KiB) 2178256



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5010.6 s]
/proc/loadavg: 8.12 8.12 8.04 9/455 21449
/proc/meminfo: memFree=23106608/32951124 swapFree=53501244/67111528
[pid=20810] ppid=20808 vsize=2178256 CPUtime=40000.7 cores=0-7
/proc/20810/stat : 20810 (claspmt) R 20808 20810 20784 0 -1 4202496 477245 0 0 0 3999556 510 0 0 25 0 8 0 662261609 2230534144 466408 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/20810/statm: 544564 466408 466 189 0 538913 0
[pid=20810/tid=20812] ppid=20808 vsize=2178256 CPUtime=4980.29 cores=0-7
/proc/20810/task/20812/stat : 20812 (claspmt) R 20808 20810 20784 0 -1 4202560 44145 0 0 0 497971 58 0 0 25 0 8 0 662261643 2230534144 466408 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=20810/tid=20813] ppid=20808 vsize=2178256 CPUtime=4997.82 cores=0-7
/proc/20810/task/20813/stat : 20813 (claspmt) R 20808 20810 20784 0 -1 4202560 48625 0 0 0 499760 22 0 0 25 0 8 0 662261643 2230534144 466408 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20810/tid=20814] ppid=20808 vsize=2178256 CPUtime=5008.62 cores=0-7
/proc/20810/task/20814/stat : 20814 (claspmt) R 20808 20810 20784 0 -1 4202560 48666 0 0 0 500807 55 0 0 25 0 8 0 662261643 2230534144 466408 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20810/tid=20815] ppid=20808 vsize=2178256 CPUtime=4995.73 cores=0-7
/proc/20810/task/20815/stat : 20815 (claspmt) R 20808 20810 20784 0 -1 4202560 35847 0 0 0 499537 36 0 0 25 0 8 0 662261643 2230534144 466408 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670335 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20810/tid=20816] ppid=20808 vsize=2178256 CPUtime=5009.79 cores=0-7
/proc/20810/task/20816/stat : 20816 (claspmt) R 20808 20810 20784 0 -1 4202560 46963 0 0 0 500937 42 0 0 25 0 8 0 662261643 2230534144 466408 33554432000 4194304 4964800 140737022621168 18446744073709551615 4321099 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20810/tid=20817] ppid=20808 vsize=2178256 CPUtime=4999.66 cores=0-7
/proc/20810/task/20817/stat : 20817 (claspmt) R 20808 20810 20784 0 -1 4202560 51028 0 0 0 499890 76 0 0 25 0 8 0 662261643 2230534144 466408 33554432000 4194304 4964800 140737022621168 18446744073709551615 4670321 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=20810/tid=20818] ppid=20808 vsize=2178256 CPUtime=5000.58 cores=0-7
/proc/20810/task/20818/stat : 20818 (claspmt) R 20808 20810 20784 0 -1 4202560 63276 0 0 0 499978 80 0 0 25 0 8 0 662261643 2230534144 466408 33554432000 4194304 4964800 140737022621168 18446744073709551615 4321038 0 0 4096 18946 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.7
Current children cumulated vsize (KiB) 2178256

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5010.8 s]
/proc/loadavg: 8.12 8.12 8.04 2/449 21450
/proc/meminfo: memFree=23109508/32951124 swapFree=53501244/67111528
[pid=20810] ppid=20808 vsize=2173392 CPUtime=40000.9 cores=0-7
/proc/20810/stat : 20810 (claspmt) R 20808 20810 20784 0 -1 4202496 477255 0 0 0 3999581 510 0 0 25 0 1 0 662261609 2225553408 465355 33554432000 4194304 4964800 140737022621168 18446744073709551615 216634172404 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/20810/statm: 543348 465355 476 189 0 537697 0
Current children cumulated CPU time (s) 40000.9
Current children cumulated vsize (KiB) 2173392

[startup+5010.92 s]
/proc/loadavg: 8.12 8.12 8.04 2/449 21450
/proc/meminfo: memFree=23109508/32951124 swapFree=53501244/67111528
[pid=20810] ppid=20808 vsize=908220 CPUtime=40001 cores=0-7
/proc/20810/stat : 20810 (claspmt) R 20808 20810 20784 0 -1 4202496 477255 0 0 0 3999586 515 0 0 25 0 1 0 662261609 930017280 153663 33554432000 4194304 4964800 140737022621168 18446744073709551615 216634558599 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/20810/statm: 227055 153151 476 189 0 221404 0
Current children cumulated CPU time (s) 40001
Current children cumulated vsize (KiB) 908220

Child status: 0
Real time (s): 5010.94
CPU time (s): 40001
CPU user time (s): 39995.9
CPU system time (s): 5.17921
CPU usage (%): 798.273
Max. virtual memory (cumulated for all children) (KiB): 2178256

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39995.9
system time used= 5.17921
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 477256
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 181564
involuntary context switches= 208635

runsolver used 19.442 second user time and 53.4999 second system time

The end

Launcher Data 

Begin job on node103 at 2011-05-16 02:43:10
IDJOB=3381166
IDBENCH=83907
IDSOLVER=1865
FILE ID=node103/3381166-1305506590
RUNJOBID= node103-1305506590-20798
PBS_JOBID= 13325503
Free space on /tmp= 72960 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT03/handmade/anton/UNSAT/SGI_30_80_28_70_8-dir.shuffled-as.sat03-215.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3381166-1305506590/watcher-3381166-1305506590 -o /tmp/evaluation-result-3381166-1305506590/solver-3381166-1305506590 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3381166-1305506590.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 731331ace9dcb7404987f98927f5c914
RANDOM SEED=1383140967

node103.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.71
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      24976916 kB
Buffers:        339552 kB
Cached:        4161608 kB
SwapCached:     340184 kB
Active:        2464088 kB
Inactive:      5211480 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      24976916 kB
SwapTotal:    67111528 kB
SwapFree:     53501244 kB
Dirty:            1700 kB
Writeback:           0 kB
AnonPages:     2844696 kB
Mapped:          20172 kB
Slab:           163588 kB
PageTables:      66548 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 137669176 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73124 MiB
End job on node103 at 2011-05-16 04:06:43