Trace number 3373163

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 4800.13 1203.33

General information on the benchmark

NameSAT05/crafted/jarvisalo05/mod2-rand3bip-sat/
mod2-rand3bip-sat-270-1.shuffled-as.sat05-2248.cnf
MD5SUM51d9aefd9da7f94abc32dd024090dc46
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark0.005998
Satisfiable
(Un)Satisfiability was proved
Number of variables270
Number of clauses1080
Sum of the clauses size3240
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 31080
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...430877/instance-3373163-1305430877.cnf
0.00/0.00	c Solving...
4800.09/1203.33	
4800.09/1203.33	*** INTERRUPTED! ***
4800.09/1203.33	*** Shutting down threads...
4800.09/1203.33	s UNKNOWN
4800.09/1203.33	
4800.09/1203.33	c INTERRUPTED   : 1
4800.09/1203.33	c Models        : 0+    
4800.09/1203.33	c Time          : 1203.327s (Solving: 1203.33s 1st Model: 0.00s Unsat: 0.00s)
4800.09/1203.33	c CPU Time      : 4800.110s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3373163-1305430877/watcher-3373163-1305430877 -o /tmp/evaluation-result-3373163-1305430877/solver-3373163-1305430877 -C 4800 -W 1300 -M 15500 HOME/claspmt -t4 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3373163-1305430877.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.06 7.78 6.12 2/349 26776
/proc/meminfo: memFree=24645676/32950928 swapFree=67056568/67111528
[pid=26776] ppid=26774 vsize=167948 CPUtime=0.01 cores=0,2,4,6
/proc/26776/stat : 26776 (claspmt) R 26774 26776 19302 0 -1 4202496 761 0 0 0 1 0 0 0 18 0 4 0 654684748 171978752 688 33554432000 4194304 4964800 140736228858192 18446744073709551615 4683730 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/26776/statm: 41987 688 454 189 0 36336 0
[pid=26776/tid=26777] ppid=26774 vsize=167948 CPUtime=0 cores=0,2,4,6
/proc/26776/task/26777/stat : 26777 (claspmt) R 26774 26776 19302 0 -1 4202560 33 0 0 0 0 0 0 0 25 0 4 0 654684748 171978752 689 33554432000 4194304 4964800 140736228858192 18446744073709551615 239476355781 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=26776/tid=26778] ppid=26774 vsize=167948 CPUtime=0 cores=0,2,4,6
/proc/26776/task/26778/stat : 26778 (claspmt) R 26774 26776 19302 0 -1 4202560 8 0 0 0 0 0 0 0 25 0 4 0 654684748 171978752 689 33554432000 4194304 4964800 140736228858192 18446744073709551615 4685574 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=26776/tid=26779] ppid=26774 vsize=167948 CPUtime=0 cores=0,2,4,6
/proc/26776/task/26779/stat : 26779 (claspmt) R 26774 26776 19302 0 -1 4202560 58 0 0 0 0 0 0 0 18 0 4 0 654684748 171978752 689 33554432000 4194304 4964800 140736228858192 18446744073709551615 239476355781 0 0 4096 18946 0 0 0 -1 4 0 0 0

[startup+0.0762041 s]
/proc/loadavg: 7.06 7.78 6.12 2/349 26776
/proc/meminfo: memFree=24645676/32950928 swapFree=67056568/67111528
[pid=26776] ppid=26774 vsize=168080 CPUtime=0.26 cores=0,2,4,6
/proc/26776/stat : 26776 (claspmt) R 26774 26776 19302 0 -1 4202496 936 0 0 0 26 0 0 0 18 0 4 0 654684748 172113920 863 33554432000 4194304 4964800 140736228858192 18446744073709551615 239476800031 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/26776/statm: 42020 863 458 189 0 36369 0
[pid=26776/tid=26777] ppid=26774 vsize=168080 CPUtime=0.06 cores=0,2,4,6
/proc/26776/task/26777/stat : 26777 (claspmt) R 26774 26776 19302 0 -1 4202560 85 0 0 0 6 0 0 0 25 0 4 0 654684748 172113920 863 33554432000 4194304 4964800 140736228858192 18446744073709551615 239476356129 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=26776/tid=26778] ppid=26774 vsize=168080 CPUtime=0.05 cores=0,2,4,6
/proc/26776/task/26778/stat : 26778 (claspmt) R 26774 26776 19302 0 -1 4202560 61 0 0 0 5 0 0 0 25 0 4 0 654684748 172113920 863 33554432000 4194304 4964800 140736228858192 18446744073709551615 239476799982 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=26776/tid=26779] ppid=26774 vsize=168080 CPUtime=0.06 cores=0,2,4,6
/proc/26776/task/26779/stat : 26779 (claspmt) R 26774 26776 19302 0 -1 4202560 83 0 0 0 6 0 0 0 18 0 4 0 654684748 172113920 863 33554432000 4194304 4964800 140736228858192 18446744073709551615 4684013 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 0.26
Current children cumulated vsize (KiB) 168080

[startup+0.113743 s]
/proc/loadavg: 7.06 7.78 6.12 2/349 26776
/proc/meminfo: memFree=24645676/32950928 swapFree=67056568/67111528
[pid=26776] ppid=26774 vsize=168212 CPUtime=0.41 cores=0,2,4,6
/proc/26776/stat : 26776 (claspmt) R 26774 26776 19302 0 -1 4202496 982 0 0 0 41 0 0 0 18 0 4 0 654684748 172249088 909 33554432000 4194304 4964800 140736228858192 18446744073709551615 4364604 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/26776/statm: 42053 909 458 189 0 36402 0
[pid=26776/tid=26777] ppid=26774 vsize=168212 CPUtime=0.1 cores=0,2,4,6
/proc/26776/task/26777/stat : 26777 (claspmt) R 26774 26776 19302 0 -1 4202560 96 0 0 0 10 0 0 0 25 0 4 0 654684748 172249088 909 33554432000 4194304 4964800 140736228858192 18446744073709551615 239476800031 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=26776/tid=26778] ppid=26774 vsize=168212 CPUtime=0.09 cores=0,2,4,6
/proc/26776/task/26778/stat : 26778 (claspmt) R 26774 26776 19302 0 -1 4202560 78 0 0 0 9 0 0 0 25 0 4 0 654684748 172249088 909 33554432000 4194304 4964800 140736228858192 18446744073709551615 239476799982 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=26776/tid=26779] ppid=26774 vsize=168212 CPUtime=0.1 cores=0,2,4,6
/proc/26776/task/26779/stat : 26779 (claspmt) R 26774 26776 19302 0 -1 4202560 91 0 0 0 10 0 0 0 18 0 4 0 654684748 172249088 909 33554432000 4194304 4964800 140736228858192 18446744073709551615 4669221 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 0.41
Current children cumulated vsize (KiB) 168212

[startup+0.300885 s]
/proc/loadavg: 7.06 7.78 6.12 2/349 26776
/proc/meminfo: memFree=24645676/32950928 swapFree=67056568/67111528
[pid=26776] ppid=26774 vsize=168608 CPUtime=1.16 cores=0,2,4,6
/proc/26776/stat : 26776 (claspmt) R 26774 26776 19302 0 -1 4202496 1259 0 0 0 115 1 0 0 18 0 4 0 654684748 172654592 1186 33554432000 4194304 4964800 140736228858192 18446744073709551615 4668814 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/26776/statm: 42152 1186 458 189 0 36501 0
[pid=26776/tid=26777] ppid=26774 vsize=168608 CPUtime=0.29 cores=0,2,4,6
/proc/26776/task/26777/stat : 26777 (claspmt) R 26774 26776 19302 0 -1 4202560 159 0 0 0 29 0 0 0 25 0 4 0 654684748 172654592 1186 33554432000 4194304 4964800 140736228858192 18446744073709551615 239476356129 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=26776/tid=26778] ppid=26774 vsize=168608 CPUtime=0.27 cores=0,2,4,6
/proc/26776/task/26778/stat : 26778 (claspmt) R 26774 26776 19302 0 -1 4202560 193 0 0 0 27 0 0 0 25 0 4 0 654684748 172654592 1186 33554432000 4194304 4964800 140736228858192 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=26776/tid=26779] ppid=26774 vsize=168608 CPUtime=0.29 cores=0,2,4,6
/proc/26776/task/26779/stat : 26779 (claspmt) R 26774 26776 19302 0 -1 4202560 148 0 0 0 29 0 0 0 18 0 4 0 654684748 172654592 1186 33554432000 4194304 4964800 140736228858192 18446744073709551615 4668725 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 1.16
Current children cumulated vsize (KiB) 168608

[startup+0.700809 s]
/proc/loadavg: 7.06 7.78 6.12 2/349 26776
/proc/meminfo: memFree=24645676/32950928 swapFree=67056568/67111528
[pid=26776] ppid=26774 vsize=168736 CPUtime=2.76 cores=0,2,4,6
/proc/26776/stat : 26776 (claspmt) R 26774 26776 19302 0 -1 4202496 1457 0 0 0 275 1 0 0 18 0 4 0 654684748 172785664 1384 33554432000 4194304 4964800 140736228858192 18446744073709551615 4670299 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/26776/statm: 42184 1384 458 189 0 36533 0
[pid=26776/tid=26777] ppid=26774 vsize=168736 CPUtime=0.68 cores=0,2,4,6
/proc/26776/task/26777/stat : 26777 (claspmt) R 26774 26776 19302 0 -1 4202560 209 0 0 0 68 0 0 0 25 0 4 0 654684748 172785664 1384 33554432000 4194304 4964800 140736228858192 18446744073709551615 4647582 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=26776/tid=26778] ppid=26774 vsize=168736 CPUtime=0.67 cores=0,2,4,6
/proc/26776/task/26778/stat : 26778 (claspmt) R 26774 26776 19302 0 -1 4202560 271 0 0 0 67 0 0 0 25 0 4 0 654684748 172785664 1384 33554432000 4194304 4964800 140736228858192 18446744073709551615 4660390 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=26776/tid=26779] ppid=26774 vsize=168736 CPUtime=0.68 cores=0,2,4,6
/proc/26776/task/26779/stat : 26779 (claspmt) R 26774 26776 19302 0 -1 4202560 193 0 0 0 68 0 0 0 18 0 4 0 654684748 172785664 1384 33554432000 4194304 4964800 140736228858192 18446744073709551615 4669210 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 2.76
Current children cumulated vsize (KiB) 168736

[startup+1.50065 s]
/proc/loadavg: 6.89 7.74 6.11 6/353 26780
/proc/meminfo: memFree=24640824/32950928 swapFree=67056568/67111528
[pid=26776] ppid=26774 vsize=169264 CPUtime=5.95 cores=0,2,4,6
/proc/26776/stat : 26776 (claspmt) R 26774 26776 19302 0 -1 4202496 1760 0 0 0 593 2 0 0 20 0 4 0 654684748 173326336 1687 33554432000 4194304 4964800 140736228858192 18446744073709551615 4370381 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/26776/statm: 42316 1687 458 189 0 36665 0
[pid=26776/tid=26777] ppid=26774 vsize=169264 CPUtime=1.48 cores=0,2,4,6
/proc/26776/task/26777/stat : 26777 (claspmt) R 26774 26776 19302 0 -1 4202560 281 0 0 0 148 0 0 0 25 0 4 0 654684748 173326336 1687 33554432000 4194304 4964800 140736228858192 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=26776/tid=26778] ppid=26774 vsize=169264 CPUtime=1.46 cores=0,2,4,6
/proc/26776/task/26778/stat : 26778 (claspmt) R 26774 26776 19302 0 -1 4202560 360 0 0 0 146 0 0 0 25 0 4 0 654684748 173326336 1687 33554432000 4194304 4964800 140736228858192 18446744073709551615 4670321 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=26776/tid=26779] ppid=26774 vsize=169264 CPUtime=1.48 cores=0,2,4,6
/proc/26776/task/26779/stat : 26779 (claspmt) R 26774 26776 19302 0 -1 4202560 233 0 0 0 148 0 0 0 20 0 4 0 654684748 173326336 1687 33554432000 4194304 4964800 140736228858192 18446744073709551615 4361126 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 5.95
Current children cumulated vsize (KiB) 169264

[startup+3.10036 s]
/proc/loadavg: 6.89 7.74 6.11 6/353 26780
/proc/meminfo: memFree=24638592/32950928 swapFree=67056568/67111528
[pid=26776] ppid=26774 vsize=169796 CPUtime=12.33 cores=0,2,4,6
/proc/26776/stat : 26776 (claspmt) R 26774 26776 19302 0 -1 4202496 2121 0 0 0 1230 3 0 0 25 0 4 0 654684748 173871104 2048 33554432000 4194304 4964800 140736228858192 18446744073709551615 4324951 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/26776/statm: 42449 2048 458 189 0 36798 0
[pid=26776/tid=26777] ppid=26774 vsize=169796 CPUtime=3.08 cores=0,2,4,6
/proc/26776/task/26777/stat : 26777 (claspmt) R 26774 26776 19302 0 -1 4202560 405 0 0 0 308 0 0 0 18 0 4 0 654684748 173871104 2048 33554432000 4194304 4964800 140736228858192 18446744073709551615 4465641 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=26776/tid=26778] ppid=26774 vsize=169796 CPUtime=3.05 cores=0,2,4,6
/proc/26776/task/26778/stat : 26778 (claspmt) R 26774 26776 19302 0 -1 4202560 473 0 0 0 304 1 0 0 18 0 4 0 654684748 173871104 2048 33554432000 4194304 4964800 140736228858192 18446744073709551615 4342208 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=26776/tid=26779] ppid=26774 vsize=169796 CPUtime=3.09 cores=0,2,4,6
/proc/26776/task/26779/stat : 26779 (claspmt) R 26774 26776 19302 0 -1 4202560 278 0 0 0 308 1 0 0 25 0 4 0 654684748 173871104 2048 33554432000 4194304 4964800 140736228858192 18446744073709551615 4668564 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 12.33
Current children cumulated vsize (KiB) 169796

[startup+6.30068 s]
/proc/loadavg: 6.89 7.74 6.11 9/359 26807
/proc/meminfo: memFree=24733116/32950928 swapFree=67056568/67111528
[pid=26776] ppid=26774 vsize=170196 CPUtime=25.09 cores=0,2,4,6
/proc/26776/stat : 26776 (claspmt) R 26774 26776 19302 0 -1 4202496 2595 0 0 0 2505 4 0 0 25 0 4 0 654684748 174280704 2522 33554432000 4194304 4964800 140736228858192 18446744073709551615 4370236 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/26776/statm: 42549 2522 458 189 0 36898 0
[pid=26776/tid=26777] ppid=26774 vsize=170196 CPUtime=6.28 cores=0,2,4,6
/proc/26776/task/26777/stat : 26777 (claspmt) R 26774 26776 19302 0 -1 4202560 570 0 0 0 627 1 0 0 25 0 4 0 654684748 174280704 2522 33554432000 4194304 4964800 140736228858192 18446744073709551615 4668702 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=26776/tid=26778] ppid=26774 vsize=170196 CPUtime=6.21 cores=0,2,4,6
/proc/26776/task/26778/stat : 26778 (claspmt) R 26774 26776 19302 0 -1 4202560 590 0 0 0 620 1 0 0 25 0 4 0 654684748 174280704 2522 33554432000 4194304 4964800 140736228858192 18446744073709551615 4362905 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=26776/tid=26779] ppid=26774 vsize=170196 CPUtime=6.28 cores=0,2,4,6
/proc/26776/task/26779/stat : 26779 (claspmt) R 26774 26776 19302 0 -1 4202560 351 0 0 0 627 1 0 0 24 0 4 0 654684748 174280704 2522 33554432000 4194304 4964800 140736228858192 18446744073709551615 4668698 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 25.09
Current children cumulated vsize (KiB) 170196

[startup+12.7005 s]
/proc/loadavg: 7.06 7.75 6.13 9/359 26807
/proc/meminfo: memFree=24726716/32950928 swapFree=67056568/67111528
[pid=26776] ppid=26774 vsize=171000 CPUtime=50.61 cores=0,2,4,6
/proc/26776/stat : 26776 (claspmt) R 26774 26776 19302 0 -1 4202496 3216 0 0 0 5056 5 0 0 25 0 4 0 654684748 175104000 3143 33554432000 4194304 4964800 140736228858192 18446744073709551615 4684800 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/26776/statm: 42750 3143 458 189 0 37099 0
[pid=26776/tid=26777] ppid=26774 vsize=171000 CPUtime=12.68 cores=0,2,4,6
/proc/26776/task/26777/stat : 26777 (claspmt) R 26774 26776 19302 0 -1 4202560 807 0 0 0 1267 1 0 0 25 0 4 0 654684748 175104000 3143 33554432000 4194304 4964800 140736228858192 18446744073709551615 4668753 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=26776/tid=26778] ppid=26774 vsize=171000 CPUtime=12.54 cores=0,2,4,6
/proc/26776/task/26778/stat : 26778 (claspmt) R 26774 26776 19302 0 -1 4202560 807 0 0 0 1253 1 0 0 25 0 4 0 654684748 175104000 3143 33554432000 4194304 4964800 140736228858192 18446744073709551615 4654518 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=26776/tid=26779] ppid=26774 vsize=171000 CPUtime=12.68 cores=0,2,4,6
/proc/26776/task/26779/stat : 26779 (claspmt) R 26774 26776 19302 0 -1 4202560 385 0 0 0 1267 1 0 0 25 0 4 0 654684748 175104000 3143 33554432000 4194304 4964800 140736228858192 18446744073709551615 239476391450 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 50.61
Current children cumulated vsize (KiB) 171000

[startup+25.501 s]
/proc/loadavg: 7.21 7.75 6.15 10/359 26807
/proc/meminfo: memFree=24720720/32950928 swapFree=67056568/67111528
[pid=26776] ppid=26774 vsize=237204 CPUtime=101.66 cores=0,2,4,6
/proc/26776/stat : 26776 (claspmt) R 26774 26776 19302 0 -1 4202496 4098 0 0 0 10160 6 0 0 25 0 4 0 654684748 242896896 4025 33554432000 4194304 4964800 140736228858192 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/26776/statm: 59301 4025 458 189 0 53650 0
[pid=26776/tid=26777] ppid=26774 vsize=237204 CPUtime=25.47 cores=0,2,4,6
/proc/26776/task/26777/stat : 26777 (claspmt) R 26774 26776 19302 0 -1 4202560 1104 0 0 0 2546 1 0 0 25 0 4 0 654684748 242896896 4025 33554432000 4194304 4964800 140736228858192 18446744073709551615 4670437 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=26776/tid=26778] ppid=26774 vsize=237204 CPUtime=25.2 cores=0,2,4,6
/proc/26776/task/26778/stat : 26778 (claspmt) R 26774 26776 19302 0 -1 4202560 1024 0 0 0 2518 2 0 0 25 0 4 0 654684748 242896896 4025 33554432000 4194304 4964800 140736228858192 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=26776/tid=26779] ppid=26774 vsize=237204 CPUtime=25.47 cores=0,2,4,6
/proc/26776/task/26779/stat : 26779 (claspmt) R 26774 26776 19302 0 -1 4202560 531 0 0 0 2546 1 0 0 25 0 4 0 654684748 242896896 4025 33554432000 4194304 4964800 140736228858192 18446744073709551615 4668592 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 101.66
Current children cumulated vsize (KiB) 237204

[startup+51.1011 s]
/proc/loadavg: 7.52 7.78 6.21 9/359 26808
/proc/meminfo: memFree=24713276/32950928 swapFree=67056568/67111528
[pid=26776] ppid=26774 vsize=237868 CPUtime=203.78 cores=0,2,4,6
/proc/26776/stat : 26776 (claspmt) R 26774 26776 19302 0 -1 4202496 4965 0 0 0 20370 8 0 0 25 0 4 0 654684748 243576832 4892 33554432000 4194304 4964800 140736228858192 18446744073709551615 4668712 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/26776/statm: 59467 4892 458 189 0 53816 0
[pid=26776/tid=26777] ppid=26774 vsize=237868 CPUtime=51.07 cores=0,2,4,6
/proc/26776/task/26777/stat : 26777 (claspmt) R 26774 26776 19302 0 -1 4202560 1293 0 0 0 5105 2 0 0 25 0 4 0 654684748 243576832 4892 33554432000 4194304 4964800 140736228858192 18446744073709551615 4670273 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=26776/tid=26778] ppid=26774 vsize=237868 CPUtime=50.52 cores=0,2,4,6
/proc/26776/task/26778/stat : 26778 (claspmt) R 26774 26776 19302 0 -1 4202560 1384 0 0 0 5050 2 0 0 25 0 4 0 654684748 243576832 4892 33554432000 4194304 4964800 140736228858192 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=26776/tid=26779] ppid=26774 vsize=237868 CPUtime=51.08 cores=0,2,4,6
/proc/26776/task/26779/stat : 26779 (claspmt) R 26774 26776 19302 0 -1 4202560 675 0 0 0 5106 2 0 0 25 0 4 0 654684748 243576832 4892 33554432000 4194304 4964800 140736228858192 18446744073709551615 4668794 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 203.78
Current children cumulated vsize (KiB) 237868

[startup+102.312 s]
/proc/loadavg: 7.79 7.81 6.30 10/359 26810
/proc/meminfo: memFree=24704188/32950928 swapFree=67056568/67111528
[pid=26776] ppid=26774 vsize=238712 CPUtime=408.06 cores=0,2,4,6
/proc/26776/stat : 26776 (claspmt) R 26774 26776 19302 0 -1 4202496 6116 0 0 0 40797 9 0 0 25 0 4 0 654684748 244441088 6043 33554432000 4194304 4964800 140736228858192 18446744073709551615 4325281 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/26776/statm: 59678 6043 458 189 0 54027 0
[pid=26776/tid=26777] ppid=26774 vsize=238712 CPUtime=102.27 cores=0,2,4,6
/proc/26776/task/26777/stat : 26777 (claspmt) R 26774 26776 19302 0 -1 4202560 1510 0 0 0 10225 2 0 0 25 0 4 0 654684748 244441088 6043 33554432000 4194304 4964800 140736228858192 18446744073709551615 4660157 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=26776/tid=26778] ppid=26774 vsize=238712 CPUtime=101.17 cores=0,2,4,6
/proc/26776/task/26778/stat : 26778 (claspmt) R 26774 26776 19302 0 -1 4202560 1688 0 0 0 10114 3 0 0 25 0 4 0 654684748 244441088 6043 33554432000 4194304 4964800 140736228858192 18446744073709551615 4361326 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=26776/tid=26779] ppid=26774 vsize=238712 CPUtime=102.29 cores=0,2,4,6
/proc/26776/task/26779/stat : 26779 (claspmt) R 26774 26776 19302 0 -1 4202560 973 0 0 0 10227 2 0 0 25 0 4 0 654684748 244441088 6043 33554432000 4194304 4964800 140736228858192 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 408.06
Current children cumulated vsize (KiB) 238712

[startup+162.301 s]

################
# More data... #
################

[pid=26776] ppid=26774 vsize=241168 CPUtime=2801.48 cores=0,2,4,6
/proc/26776/stat : 26776 (claspmt) R 26774 26776 19302 0 -1 4202496 8805 0 0 0 280137 11 0 0 25 0 4 0 654684748 246956032 8732 33554432000 4194304 4964800 140736228858192 18446744073709551615 4658589 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/26776/statm: 60292 8732 458 189 0 54641 0
[pid=26776/tid=26777] ppid=26774 vsize=241168 CPUtime=702.18 cores=0,2,4,6
/proc/26776/task/26777/stat : 26777 (claspmt) R 26774 26776 19302 0 -1 4202560 1876 0 0 0 70216 2 0 0 25 0 4 0 654684748 246956032 8732 33554432000 4194304 4964800 140736228858192 18446744073709551615 4654559 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=26776/tid=26778] ppid=26774 vsize=241168 CPUtime=694.68 cores=0,2,4,6
/proc/26776/task/26778/stat : 26778 (claspmt) R 26774 26776 19302 0 -1 4202560 2399 0 0 0 69465 3 0 0 25 0 4 0 654684748 246956032 8732 33554432000 4194304 4964800 140736228858192 18446744073709551615 4668712 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=26776/tid=26779] ppid=26774 vsize=241168 CPUtime=702.3 cores=0,2,4,6
/proc/26776/task/26779/stat : 26779 (claspmt) R 26774 26776 19302 0 -1 4202560 2064 0 0 0 70228 2 0 0 25 0 4 0 654684748 246956032 8732 33554432000 4194304 4964800 140736228858192 18446744073709551615 4670264 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 2801.48
Current children cumulated vsize (KiB) 241168

[startup+762.313 s]
/proc/loadavg: 8.04 8.02 7.18 10/359 26827
/proc/meminfo: memFree=24680696/32950928 swapFree=67056568/67111528
[pid=26776] ppid=26774 vsize=241436 CPUtime=3040.86 cores=0,2,4,6
/proc/26776/stat : 26776 (claspmt) R 26774 26776 19302 0 -1 4202496 8969 0 0 0 304075 11 0 0 25 0 4 0 654684748 247230464 8896 33554432000 4194304 4964800 140736228858192 18446744073709551615 4627536 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/26776/statm: 60359 8896 458 189 0 54708 0
[pid=26776/tid=26777] ppid=26774 vsize=241436 CPUtime=762.18 cores=0,2,4,6
/proc/26776/task/26777/stat : 26777 (claspmt) R 26774 26776 19302 0 -1 4202560 1899 0 0 0 76216 2 0 0 25 0 4 0 654684748 247230464 8896 33554432000 4194304 4964800 140736228858192 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=26776/tid=26778] ppid=26774 vsize=241436 CPUtime=754.04 cores=0,2,4,6
/proc/26776/task/26778/stat : 26778 (claspmt) R 26774 26776 19302 0 -1 4202560 2457 0 0 0 75401 3 0 0 25 0 4 0 654684748 247230464 8896 33554432000 4194304 4964800 140736228858192 18446744073709551615 4670507 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=26776/tid=26779] ppid=26774 vsize=241436 CPUtime=762.31 cores=0,2,4,6
/proc/26776/task/26779/stat : 26779 (claspmt) R 26774 26776 19302 0 -1 4202560 2116 0 0 0 76229 2 0 0 25 0 4 0 654684748 247230464 8896 33554432000 4194304 4964800 140736228858192 18446744073709551615 4659371 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3040.86
Current children cumulated vsize (KiB) 241436

[startup+822.3 s]
/proc/loadavg: 8.10 8.04 7.24 10/359 26829
/proc/meminfo: memFree=24679564/32950928 swapFree=67056568/67111528
[pid=26776] ppid=26774 vsize=241436 CPUtime=3280.16 cores=0,2,4,6
/proc/26776/stat : 26776 (claspmt) R 26774 26776 19302 0 -1 4202496 9105 0 0 0 328005 11 0 0 25 0 4 0 654684748 247230464 9032 33554432000 4194304 4964800 140736228858192 18446744073709551615 4668689 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/26776/statm: 60359 9032 458 189 0 54708 0
[pid=26776/tid=26777] ppid=26774 vsize=241436 CPUtime=822.16 cores=0,2,4,6
/proc/26776/task/26777/stat : 26777 (claspmt) R 26774 26776 19302 0 -1 4202560 1934 0 0 0 82214 2 0 0 25 0 4 0 654684748 247230464 9032 33554432000 4194304 4964800 140736228858192 18446744073709551615 4668564 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=26776/tid=26778] ppid=26774 vsize=241436 CPUtime=813.38 cores=0,2,4,6
/proc/26776/task/26778/stat : 26778 (claspmt) R 26774 26776 19302 0 -1 4202560 2548 0 0 0 81335 3 0 0 25 0 4 0 654684748 247230464 9032 33554432000 4194304 4964800 140736228858192 18446744073709551615 4361330 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=26776/tid=26779] ppid=26774 vsize=241436 CPUtime=822.31 cores=0,2,4,6
/proc/26776/task/26779/stat : 26779 (claspmt) R 26774 26776 19302 0 -1 4202560 2126 0 0 0 82228 3 0 0 25 0 4 0 654684748 247230464 9032 33554432000 4194304 4964800 140736228858192 18446744073709551615 4342206 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3280.16
Current children cumulated vsize (KiB) 241436

[startup+882.301 s]
/proc/loadavg: 8.11 8.06 7.29 9/359 26830
/proc/meminfo: memFree=24678684/32950928 swapFree=67056568/67111528
[pid=26776] ppid=26774 vsize=241576 CPUtime=3519.51 cores=0,2,4,6
/proc/26776/stat : 26776 (claspmt) R 26774 26776 19302 0 -1 4202496 9202 0 0 0 351940 11 0 0 25 0 4 0 654684748 247373824 9129 33554432000 4194304 4964800 140736228858192 18446744073709551615 4669216 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/26776/statm: 60394 9129 458 189 0 54743 0
[pid=26776/tid=26777] ppid=26774 vsize=241576 CPUtime=882.16 cores=0,2,4,6
/proc/26776/task/26777/stat : 26777 (claspmt) R 26774 26776 19302 0 -1 4202560 1971 0 0 0 88214 2 0 0 25 0 4 0 654684748 247373824 9129 33554432000 4194304 4964800 140736228858192 18446744073709551615 4320864 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=26776/tid=26778] ppid=26774 vsize=241576 CPUtime=872.73 cores=0,2,4,6
/proc/26776/task/26778/stat : 26778 (claspmt) R 26774 26776 19302 0 -1 4202560 2573 0 0 0 87270 3 0 0 25 0 4 0 654684748 247373824 9129 33554432000 4194304 4964800 140736228858192 18446744073709551615 4660870 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=26776/tid=26779] ppid=26774 vsize=241576 CPUtime=882.31 cores=0,2,4,6
/proc/26776/task/26779/stat : 26779 (claspmt) R 26774 26776 19302 0 -1 4202560 2127 0 0 0 88228 3 0 0 25 0 4 0 654684748 247373824 9129 33554432000 4194304 4964800 140736228858192 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3519.51
Current children cumulated vsize (KiB) 241576

[startup+942.3 s]
/proc/loadavg: 8.12 8.07 7.35 10/359 26832
/proc/meminfo: memFree=24677676/32950928 swapFree=67056568/67111528
[pid=26776] ppid=26774 vsize=241720 CPUtime=3758.86 cores=0,2,4,6
/proc/26776/stat : 26776 (claspmt) R 26774 26776 19302 0 -1 4202496 9312 0 0 0 375875 11 0 0 25 0 4 0 654684748 247521280 9239 33554432000 4194304 4964800 140736228858192 18446744073709551615 4654532 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/26776/statm: 60430 9239 458 189 0 54779 0
[pid=26776/tid=26777] ppid=26774 vsize=241720 CPUtime=942.14 cores=0,2,4,6
/proc/26776/task/26777/stat : 26777 (claspmt) R 26774 26776 19302 0 -1 4202560 2015 0 0 0 94212 2 0 0 25 0 4 0 654684748 247521280 9239 33554432000 4194304 4964800 140736228858192 18446744073709551615 4320848 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=26776/tid=26778] ppid=26774 vsize=241720 CPUtime=932.09 cores=0,2,4,6
/proc/26776/task/26778/stat : 26778 (claspmt) R 26774 26776 19302 0 -1 4202560 2620 0 0 0 93206 3 0 0 25 0 4 0 654684748 247521280 9239 33554432000 4194304 4964800 140736228858192 18446744073709551615 4361295 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=26776/tid=26779] ppid=26774 vsize=241720 CPUtime=942.32 cores=0,2,4,6
/proc/26776/task/26779/stat : 26779 (claspmt) R 26774 26776 19302 0 -1 4202560 2127 0 0 0 94229 3 0 0 25 0 4 0 654684748 247521280 9239 33554432000 4194304 4964800 140736228858192 18446744073709551615 4325305 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3758.86
Current children cumulated vsize (KiB) 241720

[startup+1002.3 s]
/proc/loadavg: 8.11 8.08 7.39 9/359 26834
/proc/meminfo: memFree=24676792/32950928 swapFree=67056568/67111528
[pid=26776] ppid=26774 vsize=241852 CPUtime=3998.21 cores=0,2,4,6
/proc/26776/stat : 26776 (claspmt) R 26774 26776 19302 0 -1 4202496 9440 0 0 0 399809 12 0 0 25 0 4 0 654684748 247656448 9351 33554432000 4194304 4964800 140736228858192 18446744073709551615 4683696 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/26776/statm: 60463 9351 458 189 0 54812 0
[pid=26776/tid=26777] ppid=26774 vsize=241852 CPUtime=1002.14 cores=0,2,4,6
/proc/26776/task/26777/stat : 26777 (claspmt) R 26774 26776 19302 0 -1 4202560 2039 0 0 0 100212 2 0 0 25 0 4 0 654684748 247656448 9351 33554432000 4194304 4964800 140736228858192 18446744073709551615 4321099 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=26776/tid=26778] ppid=26774 vsize=241852 CPUtime=991.44 cores=0,2,4,6
/proc/26776/task/26778/stat : 26778 (claspmt) R 26774 26776 19302 0 -1 4202560 2673 0 0 0 99141 3 0 0 25 0 4 0 654684748 247656448 9351 33554432000 4194304 4964800 140736228858192 18446744073709551615 4668773 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=26776/tid=26779] ppid=26774 vsize=241852 CPUtime=1002.32 cores=0,2,4,6
/proc/26776/task/26779/stat : 26779 (claspmt) R 26774 26776 19302 0 -1 4202560 2127 0 0 0 100229 3 0 0 25 0 4 0 654684748 247656448 9351 33554432000 4194304 4964800 140736228858192 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3998.21
Current children cumulated vsize (KiB) 241852

[startup+1062.3 s]
/proc/loadavg: 8.18 8.11 7.44 10/359 26835
/proc/meminfo: memFree=24676768/32950928 swapFree=67056568/67111528
[pid=26776] ppid=26774 vsize=241996 CPUtime=4237.57 cores=0,2,4,6
/proc/26776/stat : 26776 (claspmt) R 26774 26776 19302 0 -1 4202496 9468 0 0 0 423745 12 0 0 25 0 4 0 654684748 247803904 9379 33554432000 4194304 4964800 140736228858192 18446744073709551615 4684805 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/26776/statm: 60499 9379 458 189 0 54848 0
[pid=26776/tid=26777] ppid=26774 vsize=241996 CPUtime=1062.13 cores=0,2,4,6
/proc/26776/task/26777/stat : 26777 (claspmt) R 26774 26776 19302 0 -1 4202560 2045 0 0 0 106211 2 0 0 25 0 4 0 654684748 247803904 9379 33554432000 4194304 4964800 140736228858192 18446744073709551615 4670371 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=26776/tid=26778] ppid=26774 vsize=241996 CPUtime=1050.8 cores=0,2,4,6
/proc/26776/task/26778/stat : 26778 (claspmt) R 26774 26776 19302 0 -1 4202560 2680 0 0 0 105077 3 0 0 25 0 4 0 654684748 247803904 9379 33554432000 4194304 4964800 140736228858192 18446744073709551615 4342196 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=26776/tid=26779] ppid=26774 vsize=241996 CPUtime=1062.32 cores=0,2,4,6
/proc/26776/task/26779/stat : 26779 (claspmt) R 26774 26776 19302 0 -1 4202560 2142 0 0 0 106229 3 0 0 25 0 4 0 654684748 247803904 9379 33554432000 4194304 4964800 140736228858192 18446744073709551615 4670403 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4237.57
Current children cumulated vsize (KiB) 241996

[startup+1122.3 s]
/proc/loadavg: 8.14 8.10 7.48 9/359 26837
/proc/meminfo: memFree=24676132/32950928 swapFree=67056568/67111528
[pid=26776] ppid=26774 vsize=241996 CPUtime=4476.92 cores=0,2,4,6
/proc/26776/stat : 26776 (claspmt) R 26774 26776 19302 0 -1 4202496 9532 0 0 0 447679 13 0 0 25 0 4 0 654684748 247803904 9443 33554432000 4194304 4964800 140736228858192 18446744073709551615 4670295 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/26776/statm: 60499 9443 458 189 0 54848 0
[pid=26776/tid=26777] ppid=26774 vsize=241996 CPUtime=1122.12 cores=0,2,4,6
/proc/26776/task/26777/stat : 26777 (claspmt) R 26774 26776 19302 0 -1 4202560 2046 0 0 0 112210 2 0 0 25 0 4 0 654684748 247803904 9443 33554432000 4194304 4964800 140736228858192 18446744073709551615 4321099 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=26776/tid=26778] ppid=26774 vsize=241996 CPUtime=1110.15 cores=0,2,4,6
/proc/26776/task/26778/stat : 26778 (claspmt) R 26774 26776 19302 0 -1 4202560 2722 0 0 0 111012 3 0 0 25 0 4 0 654684748 247803904 9443 33554432000 4194304 4964800 140736228858192 18446744073709551615 4321116 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=26776/tid=26779] ppid=26774 vsize=241996 CPUtime=1122.32 cores=0,2,4,6
/proc/26776/task/26779/stat : 26779 (claspmt) R 26774 26776 19302 0 -1 4202560 2147 0 0 0 112229 3 0 0 25 0 4 0 654684748 247803904 9443 33554432000 4194304 4964800 140736228858192 18446744073709551615 4668712 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4476.92
Current children cumulated vsize (KiB) 241996

[startup+1182.3 s]
/proc/loadavg: 8.05 8.08 7.51 9/359 26838
/proc/meminfo: memFree=24675252/32950928 swapFree=67056568/67111528
[pid=26776] ppid=26774 vsize=242136 CPUtime=4716.27 cores=0,2,4,6
/proc/26776/stat : 26776 (claspmt) R 26774 26776 19302 0 -1 4202496 9573 0 0 0 471614 13 0 0 25 0 4 0 654684748 247947264 9484 33554432000 4194304 4964800 140736228858192 18446744073709551615 4669280 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/26776/statm: 60534 9484 458 189 0 54883 0
[pid=26776/tid=26777] ppid=26774 vsize=242136 CPUtime=1182.11 cores=0,2,4,6
/proc/26776/task/26777/stat : 26777 (claspmt) R 26774 26776 19302 0 -1 4202560 2046 0 0 0 118209 2 0 0 25 0 4 0 654684748 247947264 9484 33554432000 4194304 4964800 140736228858192 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=26776/tid=26778] ppid=26774 vsize=242136 CPUtime=1169.51 cores=0,2,4,6
/proc/26776/task/26778/stat : 26778 (claspmt) R 26774 26776 19302 0 -1 4202560 2722 0 0 0 116948 3 0 0 25 0 4 0 654684748 247947264 9484 33554432000 4194304 4964800 140736228858192 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=26776/tid=26779] ppid=26774 vsize=242136 CPUtime=1182.32 cores=0,2,4,6
/proc/26776/task/26779/stat : 26779 (claspmt) R 26774 26776 19302 0 -1 4202560 2168 0 0 0 118229 3 0 0 25 0 4 0 654684748 247947264 9484 33554432000 4194304 4964800 140736228858192 18446744073709551615 4670160 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4716.27
Current children cumulated vsize (KiB) 242136



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.3 s]
/proc/loadavg: 8.03 8.08 7.52 11/359 26848
/proc/meminfo: memFree=24673904/32950928 swapFree=67056568/67111528
[pid=26776] ppid=26774 vsize=242136 CPUtime=4800.09 cores=0,2,4,6
/proc/26776/stat : 26776 (claspmt) R 26774 26776 19302 0 -1 4202496 9581 0 0 0 479996 13 0 0 25 0 4 0 654684748 247947264 9492 33554432000 4194304 4964800 140736228858192 18446744073709551615 4670496 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/26776/statm: 60534 9492 458 189 0 54883 0
[pid=26776/tid=26777] ppid=26774 vsize=242136 CPUtime=1203.14 cores=0,2,4,6
/proc/26776/task/26777/stat : 26777 (claspmt) R 26774 26776 19302 0 -1 4202560 2046 0 0 0 120311 3 0 0 25 0 4 0 654684748 247947264 9492 33554432000 4194304 4964800 140736228858192 18446744073709551615 4670323 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=26776/tid=26778] ppid=26774 vsize=242136 CPUtime=1190.27 cores=0,2,4,6
/proc/26776/task/26778/stat : 26778 (claspmt) R 26774 26776 19302 0 -1 4202560 2722 0 0 0 119024 3 0 0 25 0 4 0 654684748 247947264 9492 33554432000 4194304 4964800 140736228858192 18446744073709551615 4660157 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=26776/tid=26779] ppid=26774 vsize=242136 CPUtime=1203.35 cores=0,2,4,6
/proc/26776/task/26779/stat : 26779 (claspmt) R 26774 26776 19302 0 -1 4202560 2168 0 0 0 120332 3 0 0 25 0 4 0 654684748 247947264 9492 33554432000 4194304 4964800 140736228858192 18446744073709551615 4670361 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4800.09
Current children cumulated vsize (KiB) 242136

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.3 s]
/proc/loadavg: 8.03 8.08 7.52 11/359 26848
/proc/meminfo: memFree=24673904/32950928 swapFree=67056568/67111528
[pid=26776] ppid=26774 vsize=242136 CPUtime=4800.09 cores=0,2,4,6
/proc/26776/stat : 26776 (claspmt) R 26774 26776 19302 0 -1 4202496 9581 0 0 0 479996 13 0 0 25 0 4 0 654684748 247947264 9492 33554432000 4194304 4964800 140736228858192 18446744073709551615 4670496 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/26776/statm: 60534 9492 458 189 0 54883 0
[pid=26776/tid=26777] ppid=26774 vsize=242136 CPUtime=1203.14 cores=0,2,4,6
/proc/26776/task/26777/stat : 26777 (claspmt) R 26774 26776 19302 0 -1 4202560 2046 0 0 0 120311 3 0 0 25 0 4 0 654684748 247947264 9492 33554432000 4194304 4964800 140736228858192 18446744073709551615 4670323 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=26776/tid=26778] ppid=26774 vsize=242136 CPUtime=1190.27 cores=0,2,4,6
/proc/26776/task/26778/stat : 26778 (claspmt) R 26774 26776 19302 0 -1 4202560 2722 0 0 0 119024 3 0 0 25 0 4 0 654684748 247947264 9492 33554432000 4194304 4964800 140736228858192 18446744073709551615 4660157 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=26776/tid=26779] ppid=26774 vsize=242136 CPUtime=1203.35 cores=0,2,4,6
/proc/26776/task/26779/stat : 26779 (claspmt) R 26774 26776 19302 0 -1 4202560 2168 0 0 0 120332 3 0 0 25 0 4 0 654684748 247947264 9492 33554432000 4194304 4964800 140736228858192 18446744073709551615 4670361 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4800.09
Current children cumulated vsize (KiB) 242136

Child status: 0
Real time (s): 1203.33
CPU time (s): 4800.13
CPU user time (s): 4799.99
CPU system time (s): 0.140978
CPU usage (%): 398.902
Max. virtual memory (cumulated for all children) (KiB): 242136

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4799.99
system time used= 0.140978
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 9593
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 10795
involuntary context switches= 22948

runsolver used 4.07438 second user time and 8.99463 second system time

The end

Launcher Data 

Begin job on node148 at 2011-05-15 05:41:17
IDJOB=3373163
IDBENCH=85304
IDSOLVER=1853
FILE ID=node148/3373163-1305430877
RUNJOBID= node148-1305430866-26719
PBS_JOBID= 13324551
Free space on /tmp= 72800 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT05/crafted/jarvisalo05/mod2-rand3bip-sat/mod2-rand3bip-sat-270-1.shuffled-as.sat05-2248.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3373163-1305430877/watcher-3373163-1305430877 -o /tmp/evaluation-result-3373163-1305430877/solver-3373163-1305430877 -C 4800 -W 1300 -M 15500  HOME/claspmt -t4 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3373163-1305430877.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 51d9aefd9da7f94abc32dd024090dc46
RANDOM SEED=1711404428

node148.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.854
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      24645816 kB
Buffers:        323156 kB
Cached:        4695880 kB
SwapCached:       8488 kB
Active:        4725756 kB
Inactive:      3232004 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      24645816 kB
SwapTotal:    67111528 kB
SwapFree:     67056568 kB
Dirty:          101184 kB
Writeback:           0 kB
AnonPages:     2936040 kB
Mapped:          27540 kB
Slab:           254408 kB
PageTables:      28968 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 84646672 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72896 MiB
End job on node148 at 2011-05-15 06:01:22