Trace number 3380969

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 40000.8 5011.44

General information on the benchmark

Nameapplication/fuhs/slp-synthesis-AES/
slp-synthesis-aes-bottom16.cnf
MD5SUM73a2c2d9e5c062e561bfdb7c82071a99
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables29254
Number of clauses97456
Sum of the clauses size247986
Maximum clause length977
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 250701
Number of clauses of size 346235
Number of clauses of size 4135
Number of clauses of size 514
Number of clauses of size over 5370

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...497733/instance-3380969-1305497733.cnf
0.19/0.25	c Solving...
40000.45/5011.14	
40000.45/5011.14	*** INTERRUPTED! ***
40000.45/5011.14	*** Shutting down threads...
40000.79/5011.43	s UNKNOWN
40000.79/5011.43	
40000.79/5011.43	c INTERRUPTED   : 1
40000.79/5011.43	c Models        : 0+    
40000.79/5011.43	c Time          : 5011.142s (Solving: 5010.89s 1st Model: 0.00s Unsat: 0.00s)
40000.79/5011.43	c CPU Time      : 40000.530s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3380969-1305497733/watcher-3380969-1305497733 -o /tmp/evaluation-result-3380969-1305497733/solver-3380969-1305497733 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3380969-1305497733.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.94 7.81 7.92 2/481 28956
/proc/meminfo: memFree=16247592/32951124 swapFree=62248496/67111528
[pid=28956] ppid=28954 vsize=27800 CPUtime=0.01 cores=0-7
/proc/28956/stat : 28956 (claspmt) R 28954 28956 28930 0 -1 4202496 2487 0 0 0 1 0 0 0 25 0 1 0 661371002 28467200 2153 33554432000 4194304 4964800 140735219295536 18446744073709551615 218905391813 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/28956/statm: 6950 2155 389 189 0 1824 0

[startup+0.0361281 s]
/proc/loadavg: 6.94 7.81 7.92 2/481 28956
/proc/meminfo: memFree=16247592/32951124 swapFree=62248496/67111528
[pid=28956] ppid=28954 vsize=32248 CPUtime=0.03 cores=0-7
/proc/28956/stat : 28956 (claspmt) R 28954 28956 28930 0 -1 4202496 3854 0 0 0 3 0 0 0 25 0 1 0 661371002 33021952 3116 33554432000 4194304 4964800 140735219295536 18446744073709551615 4624884 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/28956/statm: 8062 3134 398 189 0 2936 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 32248

[startup+0.101127 s]
/proc/loadavg: 6.94 7.81 7.92 2/481 28956
/proc/meminfo: memFree=16247592/32951124 swapFree=62248496/67111528
[pid=28956] ppid=28954 vsize=36836 CPUtime=0.09 cores=0-7
/proc/28956/stat : 28956 (claspmt) R 28954 28956 28930 0 -1 4202496 5264 0 0 0 9 0 0 0 25 0 1 0 661371002 37720064 4389 33554432000 4194304 4964800 140735219295536 18446744073709551615 4615094 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/28956/statm: 9209 4389 404 189 0 4083 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 36836

[startup+0.30718 s]
/proc/loadavg: 6.94 7.81 7.92 2/481 28956
/proc/meminfo: memFree=16247592/32951124 swapFree=62248496/67111528
[pid=28956] ppid=28954 vsize=410764 CPUtime=0.33 cores=0-7
/proc/28956/stat : 28956 (claspmt) S 28954 28956 28930 0 -1 4202496 14228 0 0 0 29 4 0 0 18 0 8 0 661371002 420622336 11291 33554432000 4194304 4964800 140735219295536 18446744073709551615 218917546905 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/28956/statm: 102691 11291 443 189 0 97040 0
Current children cumulated CPU time (s) 0.33
Current children cumulated vsize (KiB) 410764

[startup+1.16888 s]
/proc/loadavg: 6.94 7.81 7.92 2/481 28956
/proc/meminfo: memFree=16247592/32951124 swapFree=62248496/67111528
[pid=28956] ppid=28954 vsize=412480 CPUtime=6.9 cores=0-7
/proc/28956/stat : 28956 (claspmt) R 28954 28956 28930 0 -1 4202496 17313 0 0 0 684 6 0 0 20 0 8 0 661371002 422379520 14376 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670264 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/28956/statm: 103120 14376 464 189 0 97469 0
Current children cumulated CPU time (s) 6.9
Current children cumulated vsize (KiB) 412480

[startup+1.50082 s]
/proc/loadavg: 6.94 7.81 7.92 2/481 28956
/proc/meminfo: memFree=16247592/32951124 swapFree=62248496/67111528
[pid=28956] ppid=28954 vsize=412876 CPUtime=9.55 cores=0-7
/proc/28956/stat : 28956 (claspmt) R 28954 28956 28930 0 -1 4202496 18363 0 0 0 949 6 0 0 20 0 8 0 661371002 422785024 15426 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670194 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/28956/statm: 103219 15426 464 189 0 97568 0
Current children cumulated CPU time (s) 9.55
Current children cumulated vsize (KiB) 412876

[startup+3.10053 s]
/proc/loadavg: 7.10 7.83 7.93 10/489 28964
/proc/meminfo: memFree=16170784/32951124 swapFree=62248496/67111528
[pid=28956] ppid=28954 vsize=414948 CPUtime=22.3 cores=0-7
/proc/28956/stat : 28956 (claspmt) R 28954 28956 28930 0 -1 4202496 22910 0 0 0 2222 8 0 0 25 0 8 0 661371002 424906752 19973 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670321 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/28956/statm: 103737 19973 464 189 0 98086 0
[pid=28956/tid=28958] ppid=28954 vsize=414948 CPUtime=2.75 cores=0-7
/proc/28956/task/28958/stat : 28958 (claspmt) R 28954 28956 28930 0 -1 4202560 1621 0 0 0 275 0 0 0 21 0 8 0 661371028 424906752 19973 33554432000 4194304 4964800 140735219295536 18446744073709551615 4668698 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=28956/tid=28959] ppid=28954 vsize=414948 CPUtime=2.73 cores=0-7
/proc/28956/task/28959/stat : 28959 (claspmt) R 28954 28956 28930 0 -1 4202560 2934 0 0 0 272 1 0 0 21 0 8 0 661371028 424906752 19973 33554432000 4194304 4964800 140735219295536 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=28956/tid=28960] ppid=28954 vsize=414948 CPUtime=2.75 cores=0-7
/proc/28956/task/28960/stat : 28960 (claspmt) R 28954 28956 28930 0 -1 4202560 2341 0 0 0 274 1 0 0 25 0 8 0 661371028 424906752 19973 33554432000 4194304 4964800 140735219295536 18446744073709551615 4669509 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=28956/tid=28961] ppid=28954 vsize=414948 CPUtime=2.75 cores=0-7
/proc/28956/task/28961/stat : 28961 (claspmt) R 28954 28956 28930 0 -1 4202560 2212 0 0 0 274 1 0 0 21 0 8 0 661371028 424906752 19973 33554432000 4194304 4964800 140735219295536 18446744073709551615 4342162 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=28956/tid=28962] ppid=28954 vsize=414948 CPUtime=2.78 cores=0-7
/proc/28956/task/28962/stat : 28962 (claspmt) R 28954 28956 28930 0 -1 4202560 1713 0 0 0 278 0 0 0 23 0 8 0 661371028 424906752 19973 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=28956/tid=28963] ppid=28954 vsize=414948 CPUtime=2.75 cores=0-7
/proc/28956/task/28963/stat : 28963 (claspmt) R 28954 28956 28930 0 -1 4202560 2310 0 0 0 274 1 0 0 21 0 8 0 661371028 424906752 19973 33554432000 4194304 4964800 140735219295536 18446744073709551615 4320849 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=28956/tid=28964] ppid=28954 vsize=414948 CPUtime=2.74 cores=0-7
/proc/28956/task/28964/stat : 28964 (claspmt) R 28954 28956 28930 0 -1 4202560 2789 0 0 0 273 1 0 0 22 0 8 0 661371028 424906752 19973 33554432000 4194304 4964800 140735219295536 18446744073709551615 4669276 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 22.3
Current children cumulated vsize (KiB) 414948

[startup+6.30096 s]
/proc/loadavg: 7.10 7.83 7.93 9/489 28964
/proc/meminfo: memFree=16154328/32951124 swapFree=62248496/67111528
[pid=28956] ppid=28954 vsize=418296 CPUtime=47.85 cores=0-7
/proc/28956/stat : 28956 (claspmt) R 28954 28956 28930 0 -1 4202496 26784 0 0 0 4774 11 0 0 25 0 8 0 661371002 428335104 23847 33554432000 4194304 4964800 140735219295536 18446744073709551615 4668568 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/28956/statm: 104574 23847 466 189 0 98923 0
[pid=28956/tid=28958] ppid=28954 vsize=418296 CPUtime=5.95 cores=0-7
/proc/28956/task/28958/stat : 28958 (claspmt) R 28954 28956 28930 0 -1 4202560 1863 0 0 0 595 0 0 0 25 0 8 0 661371028 428335104 23847 33554432000 4194304 4964800 140735219295536 18446744073709551615 4668987 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=28956/tid=28959] ppid=28954 vsize=418296 CPUtime=5.93 cores=0-7
/proc/28956/task/28959/stat : 28959 (claspmt) R 28954 28956 28930 0 -1 4202560 3834 0 0 0 591 2 0 0 25 0 8 0 661371028 428335104 23847 33554432000 4194304 4964800 140735219295536 18446744073709551615 4668794 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=28956/tid=28960] ppid=28954 vsize=418296 CPUtime=5.95 cores=0-7
/proc/28956/task/28960/stat : 28960 (claspmt) R 28954 28956 28930 0 -1 4202560 2750 0 0 0 594 1 0 0 25 0 8 0 661371028 428335104 23847 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=28956/tid=28961] ppid=28954 vsize=418296 CPUtime=5.95 cores=0-7
/proc/28956/task/28961/stat : 28961 (claspmt) R 28954 28956 28930 0 -1 4202560 2684 0 0 0 594 1 0 0 25 0 8 0 661371028 428335104 23847 33554432000 4194304 4964800 140735219295536 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=28956/tid=28962] ppid=28954 vsize=418296 CPUtime=5.94 cores=0-7
/proc/28956/task/28962/stat : 28962 (claspmt) R 28954 28956 28930 0 -1 4202560 2382 0 0 0 593 1 0 0 25 0 8 0 661371028 428335104 23847 33554432000 4194304 4964800 140735219295536 18446744073709551615 4668753 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=28956/tid=28963] ppid=28954 vsize=418296 CPUtime=5.94 cores=0-7
/proc/28956/task/28963/stat : 28963 (claspmt) R 28954 28956 28930 0 -1 4202560 2677 0 0 0 593 1 0 0 25 0 8 0 661371028 428335104 23847 33554432000 4194304 4964800 140735219295536 18446744073709551615 4668732 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=28956/tid=28964] ppid=28954 vsize=418296 CPUtime=5.94 cores=0-7
/proc/28956/task/28964/stat : 28964 (claspmt) R 28954 28956 28930 0 -1 4202560 3316 0 0 0 593 1 0 0 25 0 8 0 661371028 428335104 23847 33554432000 4194304 4964800 140735219295536 18446744073709551615 4668734 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 47.85
Current children cumulated vsize (KiB) 418296

[startup+12.7008 s]
/proc/loadavg: 7.24 7.83 7.93 9/489 28964
/proc/meminfo: memFree=16131032/32951124 swapFree=62248496/67111528
[pid=28956] ppid=28954 vsize=421960 CPUtime=99.05 cores=0-7
/proc/28956/stat : 28956 (claspmt) R 28954 28956 28930 0 -1 4202496 32429 0 0 0 9890 15 0 0 25 0 8 0 661371002 432087040 29492 33554432000 4194304 4964800 140735219295536 18446744073709551615 4668715 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/28956/statm: 105490 29492 466 189 0 99839 0
[pid=28956/tid=28958] ppid=28954 vsize=421960 CPUtime=12.36 cores=0-7
/proc/28956/task/28958/stat : 28958 (claspmt) R 28954 28956 28930 0 -1 4202560 2248 0 0 0 1236 0 0 0 25 0 8 0 661371028 432087040 29495 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=28956/tid=28959] ppid=28954 vsize=421960 CPUtime=12.35 cores=0-7
/proc/28956/task/28959/stat : 28959 (claspmt) R 28954 28956 28930 0 -1 4202560 5227 0 0 0 1232 3 0 0 25 0 8 0 661371028 432087040 29495 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670282 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=28956/tid=28960] ppid=28954 vsize=421960 CPUtime=12.36 cores=0-7
/proc/28956/task/28960/stat : 28960 (claspmt) R 28954 28956 28930 0 -1 4202560 3185 0 0 0 1235 1 0 0 25 0 8 0 661371028 432087040 29495 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670310 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=28956/tid=28961] ppid=28954 vsize=421960 CPUtime=12.36 cores=0-7
/proc/28956/task/28961/stat : 28961 (claspmt) R 28954 28956 28930 0 -1 4202560 3351 0 0 0 1235 1 0 0 25 0 8 0 661371028 432087040 29495 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=28956/tid=28962] ppid=28954 vsize=421960 CPUtime=12.24 cores=0-7
/proc/28956/task/28962/stat : 28962 (claspmt) R 28954 28956 28930 0 -1 4202560 3168 0 0 0 1223 1 0 0 25 0 8 0 661371028 432087040 29495 33554432000 4194304 4964800 140735219295536 18446744073709551615 4668712 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=28956/tid=28963] ppid=28954 vsize=421960 CPUtime=12.37 cores=0-7
/proc/28956/task/28963/stat : 28963 (claspmt) R 28954 28956 28930 0 -1 4202560 3217 0 0 0 1235 2 0 0 25 0 8 0 661371028 432087040 29495 33554432000 4194304 4964800 140735219295536 18446744073709551615 4669153 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=28956/tid=28964] ppid=28954 vsize=421960 CPUtime=12.36 cores=0-7
/proc/28956/task/28964/stat : 28964 (claspmt) R 28954 28956 28930 0 -1 4202560 4358 0 0 0 1234 2 0 0 25 0 8 0 661371028 432087040 29495 33554432000 4194304 4964800 140735219295536 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 99.05
Current children cumulated vsize (KiB) 421960

[startup+25.5005 s]
/proc/loadavg: 7.36 7.84 7.93 9/489 28964
/proc/meminfo: memFree=16087500/32951124 swapFree=62248496/67111528
[pid=28956] ppid=28954 vsize=429944 CPUtime=201.13 cores=0-7
/proc/28956/stat : 28956 (claspmt) R 28954 28956 28930 0 -1 4202496 43765 0 0 0 20090 23 0 0 25 0 8 0 661371002 440262656 40828 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670299 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/28956/statm: 107486 40828 466 189 0 101835 0
[pid=28956/tid=28958] ppid=28954 vsize=429944 CPUtime=25.14 cores=0-7
/proc/28956/task/28958/stat : 28958 (claspmt) R 28954 28956 28930 0 -1 4202560 3236 0 0 0 2513 1 0 0 25 0 8 0 661371028 440262656 40828 33554432000 4194304 4964800 140735219295536 18446744073709551615 4668734 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=28956/tid=28959] ppid=28954 vsize=429944 CPUtime=25.13 cores=0-7
/proc/28956/task/28959/stat : 28959 (claspmt) R 28954 28956 28930 0 -1 4202560 8327 0 0 0 2507 6 0 0 25 0 8 0 661371028 440262656 40828 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=28956/tid=28960] ppid=28954 vsize=429944 CPUtime=25.14 cores=0-7
/proc/28956/task/28960/stat : 28960 (claspmt) R 28954 28956 28930 0 -1 4202560 4014 0 0 0 2513 1 0 0 25 0 8 0 661371028 440262656 40828 33554432000 4194304 4964800 140735219295536 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=28956/tid=28961] ppid=28954 vsize=429944 CPUtime=25.15 cores=0-7
/proc/28956/task/28961/stat : 28961 (claspmt) R 28954 28956 28930 0 -1 4202560 4749 0 0 0 2513 2 0 0 25 0 8 0 661371028 440262656 40828 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=28956/tid=28962] ppid=28954 vsize=429944 CPUtime=24.83 cores=0-7
/proc/28956/task/28962/stat : 28962 (claspmt) R 28954 28956 28930 0 -1 4202560 4295 0 0 0 2481 2 0 0 25 0 8 0 661371028 440262656 40828 33554432000 4194304 4964800 140735219295536 18446744073709551615 4660444 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=28956/tid=28963] ppid=28954 vsize=429944 CPUtime=25.14 cores=0-7
/proc/28956/task/28963/stat : 28963 (claspmt) R 28954 28956 28930 0 -1 4202560 4131 0 0 0 2512 2 0 0 25 0 8 0 661371028 440262656 40828 33554432000 4194304 4964800 140735219295536 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=28956/tid=28964] ppid=28954 vsize=429944 CPUtime=25.14 cores=0-7
/proc/28956/task/28964/stat : 28964 (claspmt) R 28954 28956 28930 0 -1 4202560 6401 0 0 0 2511 3 0 0 25 0 8 0 661371028 440262656 40828 33554432000 4194304 4964800 140735219295536 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 201.13
Current children cumulated vsize (KiB) 429944

[startup+51.1009 s]
/proc/loadavg: 7.58 7.85 7.93 9/489 28965
/proc/meminfo: memFree=16056492/32951124 swapFree=62248496/67111528
[pid=28956] ppid=28954 vsize=438028 CPUtime=405.5 cores=0-7
/proc/28956/stat : 28956 (claspmt) R 28954 28956 28930 0 -1 4202496 51570 0 0 0 40519 31 0 0 25 0 8 0 661371002 448540672 48633 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/28956/statm: 109507 48633 466 189 0 103856 0
[pid=28956/tid=28958] ppid=28954 vsize=438028 CPUtime=50.73 cores=0-7
/proc/28956/task/28958/stat : 28958 (claspmt) R 28954 28956 28930 0 -1 4202560 4078 0 0 0 5071 2 0 0 25 0 8 0 661371028 448540672 48633 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670321 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=28956/tid=28959] ppid=28954 vsize=438028 CPUtime=50.73 cores=0-7
/proc/28956/task/28959/stat : 28959 (claspmt) R 28954 28956 28930 0 -1 4202560 9901 0 0 0 5066 7 0 0 25 0 8 0 661371028 448540672 48633 33554432000 4194304 4964800 140735219295536 18446744073709551615 4361072 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=28956/tid=28960] ppid=28954 vsize=438028 CPUtime=50.74 cores=0-7
/proc/28956/task/28960/stat : 28960 (claspmt) R 28954 28956 28930 0 -1 4202560 4426 0 0 0 5073 1 0 0 25 0 8 0 661371028 448540672 48633 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670499 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=28956/tid=28961] ppid=28954 vsize=438028 CPUtime=50.75 cores=0-7
/proc/28956/task/28961/stat : 28961 (claspmt) R 28954 28956 28930 0 -1 4202560 6077 0 0 0 5072 3 0 0 25 0 8 0 661371028 448540672 48633 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=28956/tid=28962] ppid=28954 vsize=438028 CPUtime=50.01 cores=0-7
/proc/28956/task/28962/stat : 28962 (claspmt) R 28954 28956 28930 0 -1 4202560 5493 0 0 0 4998 3 0 0 25 0 8 0 661371028 448540672 48633 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=28956/tid=28963] ppid=28954 vsize=438028 CPUtime=50.73 cores=0-7
/proc/28956/task/28963/stat : 28963 (claspmt) R 28954 28956 28930 0 -1 4202560 4567 0 0 0 5070 3 0 0 25 0 8 0 661371028 448540672 48633 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=28956/tid=28964] ppid=28954 vsize=438028 CPUtime=50.74 cores=0-7
/proc/28956/task/28964/stat : 28964 (claspmt) R 28954 28956 28930 0 -1 4202560 7956 0 0 0 5069 5 0 0 25 0 8 0 661371028 448540672 48633 33554432000 4194304 4964800 140735219295536 18446744073709551615 4669095 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 405.5
Current children cumulated vsize (KiB) 438028

[startup+102.301 s]
/proc/loadavg: 7.89 7.89 7.93 9/489 28966
/proc/meminfo: memFree=15953056/32951124 swapFree=62248496/67111528
[pid=28956] ppid=28954 vsize=457660 CPUtime=814.21 cores=0-7
/proc/28956/stat : 28956 (claspmt) R 28954 28956 28930 0 -1 4202496 77225 0 0 0 81369 52 0 0 25 0 8 0 661371002 468643840 74275 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/28956/statm: 114415 74275 466 189 0 108764 0
[pid=28956/tid=28958] ppid=28954 vsize=457660 CPUtime=101.93 cores=0-7
/proc/28956/task/28958/stat : 28958 (claspmt) R 28954 28956 28930 0 -1 4202560 5879 0 0 0 10189 4 0 0 25 0 8 0 661371028 468643840 74275 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=28956/tid=28959] ppid=28954 vsize=457660 CPUtime=101.92 cores=0-7
/proc/28956/task/28959/stat : 28959 (claspmt) R 28954 28956 28930 0 -1 4202560 14222 0 0 0 10182 10 0 0 25 0 8 0 661371028 468643840 74275 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=28956/tid=28960] ppid=28954 vsize=457660 CPUtime=101.93 cores=0-7
/proc/28956/task/28960/stat : 28960 (claspmt) R 28954 28956 28930 0 -1 4202560 6777 0 0 0 10190 3 0 0 25 0 8 0 661371028 468643840 74275 33554432000 4194304 4964800 140735219295536 18446744073709551615 4378499 0 0 4096 18946 0 0 0 -1 0 0 0 0

################
# More data... #
################

/proc/28956/task/28963/stat : 28963 (claspmt) R 28954 28956 28930 0 -1 4202560 25118 0 0 0 465713 31 0 0 25 0 8 0 661371028 1272614912 263864 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=28956/tid=28964] ppid=28954 vsize=1242788 CPUtime=4661.22 cores=0-7
/proc/28956/task/28964/stat : 28964 (claspmt) R 28954 28956 28930 0 -1 4202560 65799 0 0 0 466045 77 0 0 25 0 8 0 661371028 1272614912 263864 33554432000 4194304 4964800 140735219295536 18446744073709551615 4321020 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 37215.8
Current children cumulated vsize (KiB) 1242788

[startup+4722.3 s]
/proc/loadavg: 8.06 8.07 8.01 9/489 29126
/proc/meminfo: memFree=15190804/32951124 swapFree=62248496/67111528
[pid=28956] ppid=28954 vsize=1243156 CPUtime=37694.7 cores=0-7
/proc/28956/stat : 28956 (claspmt) R 28954 28956 28930 0 -1 4202496 273953 0 0 0 3769118 356 0 0 25 0 8 0 661371002 1272991744 263985 33554432000 4194304 4964800 140735219295536 18446744073709551615 4321076 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/28956/statm: 310789 263985 466 189 0 305138 0
[pid=28956/tid=28958] ppid=28954 vsize=1243156 CPUtime=4693.92 cores=0-7
/proc/28956/task/28958/stat : 28958 (claspmt) R 28954 28956 28930 0 -1 4202560 26106 0 0 0 469356 36 0 0 25 0 8 0 661371028 1272991744 263985 33554432000 4194304 4964800 140735219295536 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=28956/tid=28959] ppid=28954 vsize=1243156 CPUtime=4711.18 cores=0-7
/proc/28956/task/28959/stat : 28959 (claspmt) R 28954 28956 28930 0 -1 4202560 45917 0 0 0 471060 58 0 0 25 0 8 0 661371028 1272991744 263985 33554432000 4194304 4964800 140735219295536 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=28956/tid=28960] ppid=28954 vsize=1243156 CPUtime=4714.15 cores=0-7
/proc/28956/task/28960/stat : 28960 (claspmt) R 28954 28956 28930 0 -1 4202560 22737 0 0 0 471386 29 0 0 25 0 8 0 661371028 1272991744 263985 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670295 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=28956/tid=28961] ppid=28954 vsize=1243156 CPUtime=4712.03 cores=0-7
/proc/28956/task/28961/stat : 28961 (claspmt) R 28954 28956 28930 0 -1 4202560 32035 0 0 0 471157 46 0 0 25 0 8 0 661371028 1272991744 263985 33554432000 4194304 4964800 140735219295536 18446744073709551615 4668987 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=28956/tid=28962] ppid=28954 vsize=1243156 CPUtime=4702.9 cores=0-7
/proc/28956/task/28962/stat : 28962 (claspmt) R 28954 28956 28930 0 -1 4202560 29161 0 0 0 470250 40 0 0 25 0 8 0 661371028 1272991744 263985 33554432000 4194304 4964800 140735219295536 18446744073709551615 4668712 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=28956/tid=28963] ppid=28954 vsize=1243156 CPUtime=4717.44 cores=0-7
/proc/28956/task/28963/stat : 28963 (claspmt) R 28954 28956 28930 0 -1 4202560 25134 0 0 0 471713 31 0 0 25 0 8 0 661371028 1272991744 263985 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=28956/tid=28964] ppid=28954 vsize=1243156 CPUtime=4721.22 cores=0-7
/proc/28956/task/28964/stat : 28964 (claspmt) R 28954 28956 28930 0 -1 4202560 65799 0 0 0 472045 77 0 0 25 0 8 0 661371028 1272991744 263985 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 37694.7
Current children cumulated vsize (KiB) 1243156

[startup+4782.3 s]
/proc/loadavg: 8.06 8.07 8.01 9/489 29127
/proc/meminfo: memFree=15190532/32951124 swapFree=62248496/67111528
[pid=28956] ppid=28954 vsize=1243584 CPUtime=38173.8 cores=0-7
/proc/28956/stat : 28956 (claspmt) R 28954 28956 28930 0 -1 4202496 274062 0 0 0 3817026 357 0 0 25 0 8 0 661371002 1273430016 264094 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/28956/statm: 310896 264094 466 189 0 305245 0
[pid=28956/tid=28958] ppid=28954 vsize=1243584 CPUtime=4753.92 cores=0-7
/proc/28956/task/28958/stat : 28958 (claspmt) R 28954 28956 28930 0 -1 4202560 26115 0 0 0 475356 36 0 0 25 0 8 0 661371028 1273430016 264094 33554432000 4194304 4964800 140735219295536 18446744073709551615 4660384 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=28956/tid=28959] ppid=28954 vsize=1243584 CPUtime=4770.21 cores=0-7
/proc/28956/task/28959/stat : 28959 (claspmt) R 28954 28956 28930 0 -1 4202560 45927 0 0 0 476963 58 0 0 25 0 8 0 661371028 1273430016 264094 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=28956/tid=28960] ppid=28954 vsize=1243584 CPUtime=4774.16 cores=0-7
/proc/28956/task/28960/stat : 28960 (claspmt) R 28954 28956 28930 0 -1 4202560 22738 0 0 0 477387 29 0 0 25 0 8 0 661371028 1273430016 264094 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=28956/tid=28961] ppid=28954 vsize=1243584 CPUtime=4772.05 cores=0-7
/proc/28956/task/28961/stat : 28961 (claspmt) R 28954 28956 28930 0 -1 4202560 32117 0 0 0 477159 46 0 0 25 0 8 0 661371028 1273430016 264094 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=28956/tid=28962] ppid=28954 vsize=1243584 CPUtime=4762.89 cores=0-7
/proc/28956/task/28962/stat : 28962 (claspmt) R 28954 28956 28930 0 -1 4202560 29161 0 0 0 476249 40 0 0 25 0 8 0 661371028 1273430016 264094 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=28956/tid=28963] ppid=28954 vsize=1243584 CPUtime=4777.45 cores=0-7
/proc/28956/task/28963/stat : 28963 (claspmt) R 28954 28956 28930 0 -1 4202560 25136 0 0 0 477714 31 0 0 25 0 8 0 661371028 1273430016 264094 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=28956/tid=28964] ppid=28954 vsize=1243584 CPUtime=4781.24 cores=0-7
/proc/28956/task/28964/stat : 28964 (claspmt) R 28954 28956 28930 0 -1 4202560 65799 0 0 0 478047 77 0 0 25 0 8 0 661371028 1273430016 264094 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 38173.8
Current children cumulated vsize (KiB) 1243584

[startup+4842.3 s]
/proc/loadavg: 8.13 8.09 8.01 9/489 29129
/proc/meminfo: memFree=15190388/32951124 swapFree=62248496/67111528
[pid=28956] ppid=28954 vsize=1243584 CPUtime=38652.7 cores=0-7
/proc/28956/stat : 28956 (claspmt) R 28954 28956 28930 0 -1 4202496 274203 0 0 0 3864911 359 0 0 25 0 8 0 661371002 1273430016 264228 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/28956/statm: 310896 264228 466 189 0 305245 0
[pid=28956/tid=28958] ppid=28954 vsize=1243584 CPUtime=4813.9 cores=0-7
/proc/28956/task/28958/stat : 28958 (claspmt) R 28954 28956 28930 0 -1 4202560 26115 0 0 0 481354 36 0 0 25 0 8 0 661371028 1273430016 264228 33554432000 4194304 4964800 140735219295536 18446744073709551615 4320939 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=28956/tid=28959] ppid=28954 vsize=1243584 CPUtime=4829.21 cores=0-7
/proc/28956/task/28959/stat : 28959 (claspmt) R 28954 28956 28930 0 -1 4202560 46068 0 0 0 482862 59 0 0 25 0 8 0 661371028 1273430016 264228 33554432000 4194304 4964800 140735219295536 18446744073709551615 4361356 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=28956/tid=28960] ppid=28954 vsize=1243584 CPUtime=4834.15 cores=0-7
/proc/28956/task/28960/stat : 28960 (claspmt) R 28954 28956 28930 0 -1 4202560 22738 0 0 0 483385 30 0 0 25 0 8 0 661371028 1273430016 264228 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=28956/tid=28961] ppid=28954 vsize=1243584 CPUtime=4832.02 cores=0-7
/proc/28956/task/28961/stat : 28961 (claspmt) R 28954 28956 28930 0 -1 4202560 32117 0 0 0 483156 46 0 0 25 0 8 0 661371028 1273430016 264228 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=28956/tid=28962] ppid=28954 vsize=1243584 CPUtime=4822.87 cores=0-7
/proc/28956/task/28962/stat : 28962 (claspmt) R 28954 28956 28930 0 -1 4202560 29161 0 0 0 482246 41 0 0 25 0 8 0 661371028 1273430016 264228 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=28956/tid=28963] ppid=28954 vsize=1243584 CPUtime=4837.43 cores=0-7
/proc/28956/task/28963/stat : 28963 (claspmt) R 28954 28956 28930 0 -1 4202560 25136 0 0 0 483712 31 0 0 25 0 8 0 661371028 1273430016 264228 33554432000 4194304 4964800 140735219295536 18446744073709551615 4668797 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=28956/tid=28964] ppid=28954 vsize=1243584 CPUtime=4841.22 cores=0-7
/proc/28956/task/28964/stat : 28964 (claspmt) R 28954 28956 28930 0 -1 4202560 65799 0 0 0 484045 77 0 0 25 0 8 0 661371028 1273430016 264228 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 38652.7
Current children cumulated vsize (KiB) 1243584

[startup+4902.32 s]
/proc/loadavg: 8.08 8.08 8.01 9/489 29131
/proc/meminfo: memFree=15190232/32951124 swapFree=62248496/67111528
[pid=28956] ppid=28954 vsize=1243584 CPUtime=39131.8 cores=0-7
/proc/28956/stat : 28956 (claspmt) R 28954 28956 28930 0 -1 4202496 274229 0 0 0 3912823 360 0 0 25 0 8 0 661371002 1273430016 264254 33554432000 4194304 4964800 140735219295536 18446744073709551615 4668568 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/28956/statm: 310896 264254 466 189 0 305245 0
[pid=28956/tid=28958] ppid=28954 vsize=1243584 CPUtime=4873.9 cores=0-7
/proc/28956/task/28958/stat : 28958 (claspmt) R 28954 28956 28930 0 -1 4202560 26115 0 0 0 487354 36 0 0 25 0 8 0 661371028 1273430016 264254 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=28956/tid=28959] ppid=28954 vsize=1243584 CPUtime=4888.24 cores=0-7
/proc/28956/task/28959/stat : 28959 (claspmt) R 28954 28956 28930 0 -1 4202560 46068 0 0 0 488765 59 0 0 25 0 8 0 661371028 1273430016 264254 33554432000 4194304 4964800 140735219295536 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=28956/tid=28960] ppid=28954 vsize=1243584 CPUtime=4894.17 cores=0-7
/proc/28956/task/28960/stat : 28960 (claspmt) R 28954 28956 28930 0 -1 4202560 22738 0 0 0 489387 30 0 0 25 0 8 0 661371028 1273430016 264254 33554432000 4194304 4964800 140735219295536 18446744073709551615 4378499 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=28956/tid=28961] ppid=28954 vsize=1243584 CPUtime=4892.04 cores=0-7
/proc/28956/task/28961/stat : 28961 (claspmt) R 28954 28956 28930 0 -1 4202560 32117 0 0 0 489158 46 0 0 25 0 8 0 661371028 1273430016 264254 33554432000 4194304 4964800 140735219295536 18446744073709551615 4321029 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=28956/tid=28962] ppid=28954 vsize=1243584 CPUtime=4882.87 cores=0-7
/proc/28956/task/28962/stat : 28962 (claspmt) R 28954 28956 28930 0 -1 4202560 29167 0 0 0 488246 41 0 0 25 0 8 0 661371028 1273430016 264254 33554432000 4194304 4964800 140735219295536 18446744073709551615 4668734 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=28956/tid=28963] ppid=28954 vsize=1243584 CPUtime=4897.45 cores=0-7
/proc/28956/task/28963/stat : 28963 (claspmt) R 28954 28956 28930 0 -1 4202560 25137 0 0 0 489714 31 0 0 25 0 8 0 661371028 1273430016 264254 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=28956/tid=28964] ppid=28954 vsize=1243584 CPUtime=4901.25 cores=0-7
/proc/28956/task/28964/stat : 28964 (claspmt) R 28954 28956 28930 0 -1 4202560 65818 0 0 0 490047 78 0 0 25 0 8 0 661371028 1273430016 264254 33554432000 4194304 4964800 140735219295536 18446744073709551615 4320934 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 39131.8
Current children cumulated vsize (KiB) 1243584

[startup+4962.3 s]
/proc/loadavg: 8.14 8.10 8.02 9/489 29133
/proc/meminfo: memFree=15190092/32951124 swapFree=62248496/67111528
[pid=28956] ppid=28954 vsize=1243584 CPUtime=39610.7 cores=0-7
/proc/28956/stat : 28956 (claspmt) R 28954 28956 28930 0 -1 4202496 274232 0 0 0 3960705 361 0 0 25 0 8 0 661371002 1273430016 264255 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/28956/statm: 310896 264255 466 189 0 305245 0
[pid=28956/tid=28958] ppid=28954 vsize=1243584 CPUtime=4933.86 cores=0-7
/proc/28956/task/28958/stat : 28958 (claspmt) R 28954 28956 28930 0 -1 4202560 26115 0 0 0 493350 36 0 0 25 0 8 0 661371028 1273430016 264255 33554432000 4194304 4964800 140735219295536 18446744073709551615 4644916 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=28956/tid=28959] ppid=28954 vsize=1243584 CPUtime=4947.24 cores=0-7
/proc/28956/task/28959/stat : 28959 (claspmt) R 28954 28956 28930 0 -1 4202560 46068 0 0 0 494665 59 0 0 25 0 8 0 661371028 1273430016 264255 33554432000 4194304 4964800 140735219295536 18446744073709551615 4660526 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=28956/tid=28960] ppid=28954 vsize=1243584 CPUtime=4954.14 cores=0-7
/proc/28956/task/28960/stat : 28960 (claspmt) R 28954 28956 28930 0 -1 4202560 22741 0 0 0 495384 30 0 0 25 0 8 0 661371028 1273430016 264255 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=28956/tid=28961] ppid=28954 vsize=1243584 CPUtime=4952.02 cores=0-7
/proc/28956/task/28961/stat : 28961 (claspmt) R 28954 28956 28930 0 -1 4202560 32117 0 0 0 495156 46 0 0 25 0 8 0 661371028 1273430016 264255 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670437 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=28956/tid=28962] ppid=28954 vsize=1243584 CPUtime=4942.82 cores=0-7
/proc/28956/task/28962/stat : 28962 (claspmt) R 28954 28956 28930 0 -1 4202560 29167 0 0 0 494241 41 0 0 25 0 8 0 661371028 1273430016 264255 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=28956/tid=28963] ppid=28954 vsize=1243584 CPUtime=4957.43 cores=0-7
/proc/28956/task/28963/stat : 28963 (claspmt) R 28954 28956 28930 0 -1 4202560 25137 0 0 0 495712 31 0 0 25 0 8 0 661371028 1273430016 264255 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670273 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=28956/tid=28964] ppid=28954 vsize=1243584 CPUtime=4961.23 cores=0-7
/proc/28956/task/28964/stat : 28964 (claspmt) R 28954 28956 28930 0 -1 4202560 65818 0 0 0 496045 78 0 0 25 0 8 0 661371028 1273430016 264255 33554432000 4194304 4964800 140735219295536 18446744073709551615 4669479 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 39610.7
Current children cumulated vsize (KiB) 1243584



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5011.1 s]
/proc/loadavg: 8.06 8.08 8.01 10/489 29134
/proc/meminfo: memFree=15188828/32951124 swapFree=62248496/67111528
[pid=28956] ppid=28954 vsize=1244664 CPUtime=40000.4 cores=0-7
/proc/28956/stat : 28956 (claspmt) R 28954 28956 28930 0 -1 4202496 274555 0 0 0 3999684 361 0 0 25 0 8 0 661371002 1274535936 264578 33554432000 4194304 4964800 140735219295536 18446744073709551615 4321079 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/28956/statm: 311166 264578 466 189 0 305515 0
[pid=28956/tid=28958] ppid=28954 vsize=1244664 CPUtime=4982.68 cores=0-7
/proc/28956/task/28958/stat : 28958 (claspmt) R 28954 28956 28930 0 -1 4202560 26121 0 0 0 498232 36 0 0 25 0 8 0 661371028 1274535936 264578 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=28956/tid=28959] ppid=28954 vsize=1244664 CPUtime=4995.25 cores=0-7
/proc/28956/task/28959/stat : 28959 (claspmt) R 28954 28956 28930 0 -1 4202560 46068 0 0 0 499466 59 0 0 25 0 8 0 661371028 1274535936 264578 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=28956/tid=28960] ppid=28954 vsize=1244664 CPUtime=5002.97 cores=0-7
/proc/28956/task/28960/stat : 28960 (claspmt) R 28954 28956 28930 0 -1 4202560 22776 0 0 0 500267 30 0 0 25 0 8 0 661371028 1274535936 264578 33554432000 4194304 4964800 140735219295536 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=28956/tid=28961] ppid=28954 vsize=1244664 CPUtime=5000.86 cores=0-7
/proc/28956/task/28961/stat : 28961 (claspmt) R 28954 28956 28930 0 -1 4202560 32117 0 0 0 500040 46 0 0 25 0 8 0 661371028 1274535936 264578 33554432000 4194304 4964800 140735219295536 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=28956/tid=28962] ppid=28954 vsize=1244664 CPUtime=4991.65 cores=0-7
/proc/28956/task/28962/stat : 28962 (claspmt) R 28954 28956 28930 0 -1 4202560 29200 0 0 0 499124 41 0 0 25 0 8 0 661371028 1274535936 264578 33554432000 4194304 4964800 140735219295536 18446744073709551615 4669376 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=28956/tid=28963] ppid=28954 vsize=1244664 CPUtime=5006.26 cores=0-7
/proc/28956/task/28963/stat : 28963 (claspmt) R 28954 28956 28930 0 -1 4202560 25382 0 0 0 500595 31 0 0 25 0 8 0 661371028 1274535936 264578 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=28956/tid=28964] ppid=28954 vsize=1244664 CPUtime=5010.06 cores=0-7
/proc/28956/task/28964/stat : 28964 (claspmt) R 28954 28956 28930 0 -1 4202560 65818 0 0 0 500928 78 0 0 25 0 8 0 661371028 1274535936 264578 33554432000 4194304 4964800 140735219295536 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 1244664

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5011.3 s]
/proc/loadavg: 8.06 8.08 8.01 10/489 29134
/proc/meminfo: memFree=15188828/32951124 swapFree=62248496/67111528
[pid=28956] ppid=28954 vsize=1232268 CPUtime=40000.7 cores=0-7
/proc/28956/stat : 28956 (claspmt) R 28954 28956 28930 0 -1 4202496 274566 0 0 0 3999707 361 0 0 25 0 1 0 661371002 1261842432 261608 33554432000 4194304 4964800 140735219295536 18446744073709551615 218905379937 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/28956/statm: 308067 261608 476 189 0 302416 0
Current children cumulated CPU time (s) 40000.7
Current children cumulated vsize (KiB) 1232268

[startup+5011.4 s]
/proc/loadavg: 8.06 8.08 8.01 10/489 29134
/proc/meminfo: memFree=15188828/32951124 swapFree=62248496/67111528
[pid=28956] ppid=28954 vsize=964776 CPUtime=40000.8 cores=0-7
/proc/28956/stat : 28956 (claspmt) R 28954 28956 28930 0 -1 4202496 274566 0 0 0 3999716 363 0 0 25 0 1 0 661371002 987930624 178140 33554432000 4194304 4964800 140735219295536 18446744073709551615 218905774215 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/28956/statm: 241194 177661 476 189 0 235543 0
Current children cumulated CPU time (s) 40000.8
Current children cumulated vsize (KiB) 964776

Child status: 0
Real time (s): 5011.44
CPU time (s): 40000.8
CPU user time (s): 39997.2
CPU system time (s): 3.66744
CPU usage (%): 798.191
Max. virtual memory (cumulated for all children) (KiB): 1244664

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39997.2
system time used= 3.66744
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 274567
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 120377
involuntary context switches= 190768

runsolver used 23.0585 second user time and 58.959 second system time

The end

Launcher Data 

Begin job on node137 at 2011-05-16 00:15:33
IDJOB=3380969
IDBENCH=83130
IDSOLVER=1865
FILE ID=node137/3380969-1305497733
RUNJOBID= node137-1305497733-28944
PBS_JOBID= 13325445
Free space on /tmp= 73700 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT11/application/fuhs/slp-synthesis-AES/slp-synthesis-aes-bottom16.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3380969-1305497733/watcher-3380969-1305497733 -o /tmp/evaluation-result-3380969-1305497733/solver-3380969-1305497733 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3380969-1305497733.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 73a2c2d9e5c062e561bfdb7c82071a99
RANDOM SEED=1024695186

node137.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      16247872 kB
Buffers:        321040 kB
Cached:        4199244 kB
SwapCached:      25028 kB
Active:        6683316 kB
Inactive:      9743948 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      16247872 kB
SwapTotal:    67111528 kB
SwapFree:     62248496 kB
Dirty:            2864 kB
Writeback:           0 kB
AnonPages:    11891904 kB
Mapped:          30836 kB
Slab:           136272 kB
PageTables:      69212 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 146921660 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73700 MiB
End job on node137 at 2011-05-16 01:39:06