Trace number 3380980

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 40000.5 5021.82

General information on the benchmark

Namecrafted/spence/sgen/
sgen3-n180-s33098896-unsat.cnf
MD5SUM1221fb51c407dee7559369d7c6eedb4e
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables181
Number of clauses372
Sum of the clauses size1116
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3372
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...498725/instance-3380980-1305498725.cnf
0.00/0.00	c Solving...
40000.42/5021.81	
40000.42/5021.81	*** INTERRUPTED! ***
40000.42/5021.81	*** Shutting down threads...
40000.42/5021.82	s UNKNOWN
40000.42/5021.82	
40000.42/5021.82	c INTERRUPTED   : 1
40000.42/5021.82	c Models        : 0+    
40000.42/5021.82	c Time          : 5021.816s (Solving: 5021.82s 1st Model: 0.00s Unsat: 0.00s)
40000.42/5021.82	c CPU Time      : 40000.470s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3380980-1305498725/watcher-3380980-1305498725 -o /tmp/evaluation-result-3380980-1305498725/solver-3380980-1305498725 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3380980-1305498725.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.01 7.99 7.98 2/358 20117
/proc/meminfo: memFree=21440444/32950928 swapFree=62959384/67111528
[pid=20117] ppid=20115 vsize=184220 CPUtime=0.01 cores=0-7
/proc/20117/stat : 20117 (claspmt) S 20115 20117 20053 0 -1 4202496 808 0 0 0 1 0 0 0 25 0 8 0 117481879 188641280 731 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880670190 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/20117/statm: 46055 732 460 189 0 40404 0
[pid=20117/tid=20118] ppid=20115 vsize=184220 CPUtime=0 cores=0-7
/proc/20117/task/20118/stat : 20118 (claspmt) R 20115 20117 20053 0 -1 4202560 39 0 0 0 0 0 0 0 25 0 8 0 117481879 188641280 733 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880226337 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=20117/tid=20119] ppid=20115 vsize=184220 CPUtime=0 cores=0-7
/proc/20117/task/20119/stat : 20119 (claspmt) R 20115 20117 20053 0 -1 4202560 11 0 0 0 0 0 0 0 25 0 8 0 117481879 188641280 733 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880670190 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=20117/tid=20120] ppid=20115 vsize=184220 CPUtime=0 cores=0-7
/proc/20117/task/20120/stat : 20120 (claspmt) R 20115 20117 20053 0 -1 4202560 19 0 0 0 0 0 0 0 25 0 8 0 117481879 188641280 734 33554432000 4194304 4964800 140734025808224 18446744073709551615 4361308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20117/tid=20121] ppid=20115 vsize=184220 CPUtime=0 cores=0-7
/proc/20117/task/20121/stat : 20121 (claspmt) R 20115 20117 20053 0 -1 4202560 13 0 0 0 0 0 0 0 25 0 8 0 117481879 188641280 734 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880670190 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20117/tid=20122] ppid=20115 vsize=184220 CPUtime=0 cores=0-7
/proc/20117/task/20122/stat : 20122 (claspmt) R 20115 20117 20053 0 -1 4202560 16 0 0 0 0 0 0 0 25 0 8 0 117481879 188641280 734 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880232410 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20117/tid=20123] ppid=20115 vsize=184220 CPUtime=0 cores=0-7
/proc/20117/task/20123/stat : 20123 (claspmt) R 20115 20117 20053 0 -1 4202560 26 0 0 0 0 0 0 0 25 0 8 0 117481879 188641280 734 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880225989 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=20117/tid=20124] ppid=20115 vsize=184220 CPUtime=0 cores=0-7
/proc/20117/task/20124/stat : 20124 (claspmt) R 20115 20117 20053 0 -1 4202560 16 0 0 0 0 0 0 0 25 0 8 0 117481879 188641280 734 33554432000 4194304 4964800 140734025808224 18446744073709551615 4668592 0 0 4096 18946 0 0 0 -1 0 0 0 0

[startup+0.033899 s]
/proc/loadavg: 8.01 7.99 7.98 2/358 20117
/proc/meminfo: memFree=21440444/32950928 swapFree=62959384/67111528
[pid=20117] ppid=20115 vsize=250020 CPUtime=0.14 cores=0-7
/proc/20117/stat : 20117 (claspmt) R 20115 20117 20053 0 -1 4202496 991 0 0 0 14 0 0 0 25 0 8 0 117481879 256020480 914 33554432000 4194304 4964800 140734025808224 18446744073709551615 4644832 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/20117/statm: 62505 914 464 189 0 56854 0
[pid=20117/tid=20118] ppid=20115 vsize=250020 CPUtime=0.02 cores=0-7
/proc/20117/task/20118/stat : 20118 (claspmt) R 20115 20117 20053 0 -1 4202560 70 0 0 0 2 0 0 0 18 0 8 0 117481879 256020480 914 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880670239 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20117/tid=20119] ppid=20115 vsize=250020 CPUtime=0 cores=0-7
/proc/20117/task/20119/stat : 20119 (claspmt) R 20115 20117 20053 0 -1 4202560 26 0 0 0 0 0 0 0 18 0 8 0 117481879 256020480 914 33554432000 4194304 4964800 140734025808224 18446744073709551615 4654532 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=20117/tid=20120] ppid=20115 vsize=250020 CPUtime=0.01 cores=0-7
/proc/20117/task/20120/stat : 20120 (claspmt) R 20115 20117 20053 0 -1 4202560 40 0 0 0 1 0 0 0 18 0 8 0 117481879 256020480 914 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880670239 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20117/tid=20121] ppid=20115 vsize=250020 CPUtime=0.02 cores=0-7
/proc/20117/task/20121/stat : 20121 (claspmt) R 20115 20117 20053 0 -1 4202560 34 0 0 0 2 0 0 0 25 0 8 0 117481879 256020480 914 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880670239 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20117/tid=20122] ppid=20115 vsize=250020 CPUtime=0.01 cores=0-7
/proc/20117/task/20122/stat : 20122 (claspmt) R 20115 20117 20053 0 -1 4202560 36 0 0 0 1 0 0 0 18 0 8 0 117481879 256020480 914 33554432000 4194304 4964800 140734025808224 18446744073709551615 4362354 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20117/tid=20123] ppid=20115 vsize=250020 CPUtime=0.02 cores=0-7
/proc/20117/task/20123/stat : 20123 (claspmt) R 20115 20117 20053 0 -1 4202560 39 0 0 0 2 0 0 0 18 0 8 0 117481879 256020480 914 33554432000 4194304 4964800 140734025808224 18446744073709551615 4660099 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20117/tid=20124] ppid=20115 vsize=250020 CPUtime=0.02 cores=0-7
/proc/20117/task/20124/stat : 20124 (claspmt) R 20115 20117 20053 0 -1 4202560 64 0 0 0 2 0 0 0 25 0 8 0 117481879 256020480 914 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880222976 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 0.14
Current children cumulated vsize (KiB) 250020

[startup+0.115115 s]
/proc/loadavg: 8.01 7.99 7.98 2/358 20117
/proc/meminfo: memFree=21440444/32950928 swapFree=62959384/67111528
[pid=20117] ppid=20115 vsize=250284 CPUtime=0.73 cores=0-7
/proc/20117/stat : 20117 (claspmt) D 20115 20117 20053 0 -1 4202496 1242 0 0 0 72 1 0 0 18 0 8 0 117481879 256290816 1165 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880608439 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/20117/statm: 62571 1165 464 189 0 56920 0
[pid=20117/tid=20118] ppid=20115 vsize=250284 CPUtime=0.09 cores=0-7
/proc/20117/task/20118/stat : 20118 (claspmt) D 20115 20117 20053 0 -1 4202560 124 0 0 0 9 0 0 0 18 0 8 0 117481879 256290816 1165 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880225989 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20117/tid=20119] ppid=20115 vsize=250284 CPUtime=0.08 cores=0-7
/proc/20117/task/20119/stat : 20119 (claspmt) R 20115 20117 20053 0 -1 4202560 70 0 0 0 8 0 0 0 18 0 8 0 117481879 256290816 1165 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880226337 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=20117/tid=20120] ppid=20115 vsize=250284 CPUtime=0.08 cores=0-7
/proc/20117/task/20120/stat : 20120 (claspmt) S 20115 20117 20053 0 -1 4202560 56 0 0 0 8 0 0 0 18 0 8 0 117481879 256290816 1165 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880670190 0 0 4096 18946 18446604452199687232 0 0 -1 1 0 0 0
[pid=20117/tid=20121] ppid=20115 vsize=250284 CPUtime=0.09 cores=0-7
/proc/20117/task/20121/stat : 20121 (claspmt) S 20115 20117 20053 0 -1 4202560 53 0 0 0 9 0 0 0 18 0 8 0 117481879 256290816 1165 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880670190 0 0 4096 18946 18446604452199687232 0 0 -1 4 0 0 0
[pid=20117/tid=20122] ppid=20115 vsize=250284 CPUtime=0.09 cores=0-7
/proc/20117/task/20122/stat : 20122 (claspmt) S 20115 20117 20053 0 -1 4202560 54 0 0 0 9 0 0 0 18 0 8 0 117481879 256290816 1165 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880670190 0 0 4096 18946 18446604452199687232 0 0 -1 3 0 0 0
[pid=20117/tid=20123] ppid=20115 vsize=250284 CPUtime=0.09 cores=0-7
/proc/20117/task/20123/stat : 20123 (claspmt) D 20115 20117 20053 0 -1 4202560 53 0 0 0 9 0 0 0 18 0 8 0 117481879 256290816 1165 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880608346 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20117/tid=20124] ppid=20115 vsize=250284 CPUtime=0.09 cores=0-7
/proc/20117/task/20124/stat : 20124 (claspmt) D 20115 20117 20053 0 -1 4202560 120 0 0 0 9 0 0 0 18 0 8 0 117481879 256290816 1165 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880608439 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 0.73
Current children cumulated vsize (KiB) 250284

[startup+0.573475 s]
/proc/loadavg: 8.01 7.99 7.98 2/358 20117
/proc/meminfo: memFree=21440444/32950928 swapFree=62959384/67111528
[pid=20117] ppid=20115 vsize=316220 CPUtime=4.6 cores=0-7
/proc/20117/stat : 20117 (claspmt) S 20115 20117 20053 0 -1 4202496 1972 0 0 0 457 3 0 0 18 0 8 0 117481879 323809280 1895 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880670190 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/20117/statm: 79055 1896 464 189 0 73404 0
[pid=20117/tid=20118] ppid=20115 vsize=316220 CPUtime=0.6 cores=0-7
/proc/20117/task/20118/stat : 20118 (claspmt) R 20115 20117 20053 0 -1 4202560 247 0 0 0 60 0 0 0 18 0 8 0 117481879 323809280 1919 33554432000 4194304 4964800 140734025808224 18446744073709551615 4653452 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20117/tid=20119] ppid=20115 vsize=316220 CPUtime=0.61 cores=0-7
/proc/20117/task/20119/stat : 20119 (claspmt) R 20115 20117 20053 0 -1 4202560 176 0 0 0 61 0 0 0 18 0 8 0 117481879 323809280 1941 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880670239 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=20117/tid=20120] ppid=20115 vsize=316220 CPUtime=0.65 cores=0-7
/proc/20117/task/20120/stat : 20120 (claspmt) R 20115 20117 20053 0 -1 4202560 94 0 0 0 65 0 0 0 18 0 8 0 117481879 323809280 1958 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880670239 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20117/tid=20121] ppid=20115 vsize=316220 CPUtime=0.66 cores=0-7
/proc/20117/task/20121/stat : 20121 (claspmt) R 20115 20117 20053 0 -1 4202560 161 0 0 0 66 0 0 0 18 0 8 0 117481879 323809280 1958 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880670239 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20117/tid=20122] ppid=20115 vsize=316220 CPUtime=0.68 cores=0-7
/proc/20117/task/20122/stat : 20122 (claspmt) R 20115 20117 20053 0 -1 4202560 96 0 0 0 68 0 0 0 18 0 8 0 117481879 323809280 1986 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880670239 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20117/tid=20123] ppid=20115 vsize=316220 CPUtime=0.72 cores=0-7
/proc/20117/task/20123/stat : 20123 (claspmt) R 20115 20117 20053 0 -1 4202560 120 0 0 0 72 0 0 0 18 0 8 0 117481879 323809280 1996 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880670239 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20117/tid=20124] ppid=20115 vsize=316352 CPUtime=0.75 cores=0-7
/proc/20117/task/20124/stat : 20124 (claspmt) R 20115 20117 20053 0 -1 4202560 318 0 0 0 75 0 0 0 18 0 8 0 117481879 323944448 2031 33554432000 4194304 4964800 140734025808224 18446744073709551615 4327198 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4.6
Current children cumulated vsize (KiB) 316220

[startup+0.874207 s]
/proc/loadavg: 8.01 7.99 7.98 2/358 20117
/proc/meminfo: memFree=21440444/32950928 swapFree=62959384/67111528
[pid=20117] ppid=20115 vsize=316352 CPUtime=7.23 cores=0-7
/proc/20117/stat : 20117 (claspmt) S 20115 20117 20053 0 -1 4202496 2201 0 0 0 719 4 0 0 19 0 8 0 117481879 323944448 2124 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880670190 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/20117/statm: 79088 2129 464 189 0 73437 0
[pid=20117/tid=20118] ppid=20115 vsize=316352 CPUtime=0.92 cores=0-7
/proc/20117/task/20118/stat : 20118 (claspmt) R 20115 20117 20053 0 -1 4202560 292 0 0 0 92 0 0 0 19 0 8 0 117481879 323944448 2129 33554432000 4194304 4964800 140734025808224 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20117/tid=20119] ppid=20115 vsize=316352 CPUtime=0.9 cores=0-7
/proc/20117/task/20119/stat : 20119 (claspmt) R 20115 20117 20053 0 -1 4202560 192 0 0 0 90 0 0 0 19 0 8 0 117481879 323944448 2129 33554432000 4194304 4964800 140734025808224 18446744073709551615 4660155 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=20117/tid=20120] ppid=20115 vsize=316352 CPUtime=0.91 cores=0-7
/proc/20117/task/20120/stat : 20120 (claspmt) R 20115 20117 20053 0 -1 4202560 101 0 0 0 91 0 0 0 19 0 8 0 117481879 323944448 2129 33554432000 4194304 4964800 140734025808224 18446744073709551615 4320928 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20117/tid=20121] ppid=20115 vsize=316352 CPUtime=0.91 cores=0-7
/proc/20117/task/20121/stat : 20121 (claspmt) R 20115 20117 20053 0 -1 4202560 187 0 0 0 91 0 0 0 19 0 8 0 117481879 323944448 2129 33554432000 4194304 4964800 140734025808224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20117/tid=20122] ppid=20115 vsize=316352 CPUtime=0.91 cores=0-7
/proc/20117/task/20122/stat : 20122 (claspmt) R 20115 20117 20053 0 -1 4202560 105 0 0 0 91 0 0 0 18 0 8 0 117481879 323944448 2129 33554432000 4194304 4964800 140734025808224 18446744073709551615 4653222 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20117/tid=20123] ppid=20115 vsize=316352 CPUtime=0.91 cores=0-7
/proc/20117/task/20123/stat : 20123 (claspmt) R 20115 20117 20053 0 -1 4202560 123 0 0 0 91 0 0 0 18 0 8 0 117481879 323944448 2129 33554432000 4194304 4964800 140734025808224 18446744073709551615 4321116 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20117/tid=20124] ppid=20115 vsize=316352 CPUtime=0.92 cores=0-7
/proc/20117/task/20124/stat : 20124 (claspmt) R 20115 20117 20053 0 -1 4202560 348 0 0 0 92 0 0 0 19 0 8 0 117481879 323944448 2129 33554432000 4194304 4964800 140734025808224 18446744073709551615 4367136 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 7.23
Current children cumulated vsize (KiB) 316352

[startup+1.52456 s]
/proc/loadavg: 8.01 7.99 7.98 2/358 20117
/proc/meminfo: memFree=21440444/32950928 swapFree=62959384/67111528
[pid=20117] ppid=20115 vsize=316748 CPUtime=11.96 cores=0-7
/proc/20117/stat : 20117 (claspmt) S 20115 20117 20053 0 -1 4202496 2547 0 0 0 1189 7 0 0 18 0 8 0 117481879 324349952 2470 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880670190 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/20117/statm: 79187 2487 464 189 0 73536 0
[pid=20117/tid=20118] ppid=20115 vsize=316748 CPUtime=1.54 cores=0-7
/proc/20117/task/20118/stat : 20118 (claspmt) R 20115 20117 20053 0 -1 4202560 371 0 0 0 153 1 0 0 20 0 8 0 117481879 324349952 2487 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880670239 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20117/tid=20119] ppid=20115 vsize=316748 CPUtime=1.58 cores=0-7
/proc/20117/task/20119/stat : 20119 (claspmt) R 20115 20117 20053 0 -1 4202560 248 0 0 0 158 0 0 0 20 0 8 0 117481879 324349952 2513 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880670239 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=20117/tid=20120] ppid=20115 vsize=316748 CPUtime=1.6 cores=0-7
/proc/20117/task/20120/stat : 20120 (claspmt) R 20115 20117 20053 0 -1 4202560 112 0 0 0 159 1 0 0 20 0 8 0 117481879 324349952 2513 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880670239 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20117/tid=20121] ppid=20115 vsize=316748 CPUtime=1.62 cores=0-7
/proc/20117/task/20121/stat : 20121 (claspmt) R 20115 20117 20053 0 -1 4202560 226 0 0 0 162 0 0 0 20 0 8 0 117481879 324349952 2519 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880670239 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20117/tid=20122] ppid=20115 vsize=316748 CPUtime=1.66 cores=0-7
/proc/20117/task/20122/stat : 20122 (claspmt) R 20115 20117 20053 0 -1 4202560 122 0 0 0 165 1 0 0 18 0 8 0 117481879 324349952 2537 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880670239 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20117/tid=20123] ppid=20115 vsize=316748 CPUtime=1.67 cores=0-7
/proc/20117/task/20123/stat : 20123 (claspmt) R 20115 20117 20053 0 -1 4202560 148 0 0 0 166 1 0 0 18 0 8 0 117481879 324349952 2541 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880670239 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20117/tid=20124] ppid=20115 vsize=316748 CPUtime=1.71 cores=0-7
/proc/20117/task/20124/stat : 20124 (claspmt) R 20115 20117 20053 0 -1 4202560 425 0 0 0 171 0 0 0 20 0 8 0 117481879 324349952 2552 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880670239 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 11.96
Current children cumulated vsize (KiB) 316748

[startup+3.101 s]
/proc/loadavg: 8.01 7.99 7.98 9/366 20125
/proc/meminfo: memFree=21431468/32950928 swapFree=62959384/67111528
[pid=20117] ppid=20115 vsize=382548 CPUtime=24.5 cores=0-7
/proc/20117/stat : 20117 (claspmt) R 20115 20117 20053 0 -1 4202496 3086 0 0 0 2437 13 0 0 18 0 8 0 117481879 391729152 3009 33554432000 4194304 4964800 140734025808224 18446744073709551615 4320966 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/20117/statm: 95637 3009 464 189 0 89986 0
[pid=20117/tid=20118] ppid=20115 vsize=382548 CPUtime=3.07 cores=0-7
/proc/20117/task/20118/stat : 20118 (claspmt) R 20115 20117 20053 0 -1 4202560 503 0 0 0 305 2 0 0 20 0 8 0 117481879 391729152 3009 33554432000 4194304 4964800 140734025808224 18446744073709551615 4670371 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20117/tid=20119] ppid=20115 vsize=382548 CPUtime=3.04 cores=0-7
/proc/20117/task/20119/stat : 20119 (claspmt) R 20115 20117 20053 0 -1 4202560 326 0 0 0 303 1 0 0 18 0 8 0 117481879 391729152 3009 33554432000 4194304 4964800 140734025808224 18446744073709551615 4670273 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=20117/tid=20120] ppid=20115 vsize=382548 CPUtime=3.06 cores=0-7
/proc/20117/task/20120/stat : 20120 (claspmt) R 20115 20117 20053 0 -1 4202560 134 0 0 0 305 1 0 0 18 0 8 0 117481879 391729152 3009 33554432000 4194304 4964800 140734025808224 18446744073709551615 4342162 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20117/tid=20121] ppid=20115 vsize=382548 CPUtime=3.05 cores=0-7
/proc/20117/task/20121/stat : 20121 (claspmt) R 20115 20117 20053 0 -1 4202560 269 0 0 0 304 1 0 0 19 0 8 0 117481879 391729152 3009 33554432000 4194304 4964800 140734025808224 18446744073709551615 4359908 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20117/tid=20122] ppid=20115 vsize=382548 CPUtime=3.06 cores=0-7
/proc/20117/task/20122/stat : 20122 (claspmt) R 20115 20117 20053 0 -1 4202560 150 0 0 0 304 2 0 0 19 0 8 0 117481879 391729152 3009 33554432000 4194304 4964800 140734025808224 18446744073709551615 4669210 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20117/tid=20123] ppid=20115 vsize=382548 CPUtime=3.05 cores=0-7
/proc/20117/task/20123/stat : 20123 (claspmt) R 20115 20117 20053 0 -1 4202560 192 0 0 0 304 1 0 0 18 0 8 0 117481879 391729152 3009 33554432000 4194304 4964800 140734025808224 18446744073709551615 4361326 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20117/tid=20124] ppid=20115 vsize=382548 CPUtime=3.06 cores=0-7
/proc/20117/task/20124/stat : 20124 (claspmt) R 20115 20117 20053 0 -1 4202560 515 0 0 0 305 1 0 0 18 0 8 0 117481879 391729152 3009 33554432000 4194304 4964800 140734025808224 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 24.5
Current children cumulated vsize (KiB) 382548

[startup+6.30044 s]
/proc/loadavg: 8.09 8.01 7.98 9/366 20125
/proc/meminfo: memFree=21428416/32950928 swapFree=62959384/67111528
[pid=20117] ppid=20115 vsize=382940 CPUtime=49.93 cores=0-7
/proc/20117/stat : 20117 (claspmt) R 20115 20117 20053 0 -1 4202496 3611 0 0 0 4972 21 0 0 19 0 8 0 117481879 392130560 3534 33554432000 4194304 4964800 140734025808224 18446744073709551615 4670330 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/20117/statm: 95735 3534 464 189 0 90084 0
[pid=20117/tid=20118] ppid=20115 vsize=382940 CPUtime=6.25 cores=0-7
/proc/20117/task/20118/stat : 20118 (claspmt) R 20115 20117 20053 0 -1 4202560 659 0 0 0 623 2 0 0 18 0 8 0 117481879 392130560 3534 33554432000 4194304 4964800 140734025808224 18446744073709551615 4670335 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20117/tid=20119] ppid=20115 vsize=382940 CPUtime=6.21 cores=0-7
/proc/20117/task/20119/stat : 20119 (claspmt) R 20115 20117 20053 0 -1 4202560 374 0 0 0 619 2 0 0 18 0 8 0 117481879 392130560 3534 33554432000 4194304 4964800 140734025808224 18446744073709551615 4320955 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=20117/tid=20120] ppid=20115 vsize=382940 CPUtime=6.25 cores=0-7
/proc/20117/task/20120/stat : 20120 (claspmt) R 20115 20117 20053 0 -1 4202560 166 0 0 0 623 2 0 0 18 0 8 0 117481879 392130560 3534 33554432000 4194304 4964800 140734025808224 18446744073709551615 4660835 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20117/tid=20121] ppid=20115 vsize=382940 CPUtime=6.22 cores=0-7
/proc/20117/task/20121/stat : 20121 (claspmt) R 20115 20117 20053 0 -1 4202560 295 0 0 0 620 2 0 0 19 0 8 0 117481879 392130560 3534 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880225191 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20117/tid=20122] ppid=20115 vsize=382940 CPUtime=6.25 cores=0-7
/proc/20117/task/20122/stat : 20122 (claspmt) R 20115 20117 20053 0 -1 4202560 176 0 0 0 622 3 0 0 18 0 8 0 117481879 392130560 3534 33554432000 4194304 4964800 140734025808224 18446744073709551615 4374881 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20117/tid=20123] ppid=20115 vsize=382940 CPUtime=6.23 cores=0-7
/proc/20117/task/20123/stat : 20123 (claspmt) R 20115 20117 20053 0 -1 4202560 203 0 0 0 621 2 0 0 19 0 8 0 117481879 392130560 3534 33554432000 4194304 4964800 140734025808224 18446744073709551615 4362767 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20117/tid=20124] ppid=20115 vsize=382940 CPUtime=6.26 cores=0-7
/proc/20117/task/20124/stat : 20124 (claspmt) R 20115 20117 20053 0 -1 4202560 645 0 0 0 624 2 0 0 22 0 8 0 117481879 392130560 3534 33554432000 4194304 4964800 140734025808224 18446744073709551615 4366104 0 0 4096 18946 0 0 0 -1 0 0 0 0

################
# More data... #
################


[startup+4722.31 s]
/proc/loadavg: 7.99 8.00 7.99 9/366 20262
/proc/meminfo: memFree=21403476/32950928 swapFree=62959384/67111528
[pid=20117] ppid=20115 vsize=451648 CPUtime=37614.4 cores=0-7
/proc/20117/stat : 20117 (claspmt) R 20115 20117 20053 0 -1 4202496 8524 0 0 0 3752229 9213 0 0 18 0 8 0 117481879 462487552 8447 33554432000 4194304 4964800 140734025808224 18446744073709551615 4342174 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/20117/statm: 112912 8447 468 189 0 107261 0
[pid=20117/tid=20118] ppid=20115 vsize=451648 CPUtime=4703.72 cores=0-7
/proc/20117/task/20118/stat : 20118 (claspmt) R 20115 20117 20053 0 -1 4202560 2704 0 0 0 469287 1085 0 0 21 0 8 0 117481879 462487552 8447 33554432000 4194304 4964800 140734025808224 18446744073709551615 4321016 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=20117/tid=20119] ppid=20115 vsize=451648 CPUtime=4702.82 cores=0-7
/proc/20117/task/20119/stat : 20119 (claspmt) R 20115 20117 20053 0 -1 4202560 593 0 0 0 469043 1239 0 0 18 0 8 0 117481879 462487552 8447 33554432000 4194304 4964800 140734025808224 18446744073709551615 4361088 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20117/tid=20120] ppid=20115 vsize=451648 CPUtime=4701.44 cores=0-7
/proc/20117/task/20120/stat : 20120 (claspmt) R 20115 20117 20053 0 -1 4202560 302 0 0 0 468973 1171 0 0 22 0 8 0 117481879 462487552 8447 33554432000 4194304 4964800 140734025808224 18446744073709551615 4342208 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=20117/tid=20121] ppid=20115 vsize=451648 CPUtime=4701.42 cores=0-7
/proc/20117/task/20121/stat : 20121 (claspmt) R 20115 20117 20053 0 -1 4202560 453 0 0 0 468970 1172 0 0 19 0 8 0 117481879 462487552 8447 33554432000 4194304 4964800 140734025808224 18446744073709551615 4338784 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20117/tid=20122] ppid=20115 vsize=451648 CPUtime=4699.21 cores=0-7
/proc/20117/task/20122/stat : 20122 (claspmt) R 20115 20117 20053 0 -1 4202560 371 0 0 0 468690 1231 0 0 19 0 8 0 117481879 462487552 8447 33554432000 4194304 4964800 140734025808224 18446744073709551615 4683989 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20117/tid=20123] ppid=20115 vsize=451648 CPUtime=4699.29 cores=0-7
/proc/20117/task/20123/stat : 20123 (claspmt) R 20115 20117 20053 0 -1 4202560 367 0 0 0 468754 1175 0 0 22 0 8 0 117481879 462487552 8447 33554432000 4194304 4964800 140734025808224 18446744073709551615 4661130 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20117/tid=20124] ppid=20115 vsize=451648 CPUtime=4700.56 cores=0-7
/proc/20117/task/20124/stat : 20124 (claspmt) R 20115 20117 20053 0 -1 4202560 1838 0 0 0 468960 1096 0 0 19 0 8 0 117481879 462487552 8447 33554432000 4194304 4964800 140734025808224 18446744073709551615 4660236 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 37614.4
Current children cumulated vsize (KiB) 451648

[startup+4782.3 s]
/proc/loadavg: 8.07 8.02 8.00 9/366 20264
/proc/meminfo: memFree=21403332/32950928 swapFree=62959384/67111528
[pid=20117] ppid=20115 vsize=451648 CPUtime=38092.4 cores=0-7
/proc/20117/stat : 20117 (claspmt) R 20115 20117 20053 0 -1 4202496 8532 0 0 0 3799914 9323 0 0 19 0 8 0 117481879 462487552 8455 33554432000 4194304 4964800 140734025808224 18446744073709551615 4312198 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/20117/statm: 112912 8455 468 189 0 107261 0
[pid=20117/tid=20118] ppid=20115 vsize=451648 CPUtime=4763.61 cores=0-7
/proc/20117/task/20118/stat : 20118 (claspmt) R 20115 20117 20053 0 -1 4202560 2706 0 0 0 475265 1096 0 0 25 0 8 0 117481879 462487552 8455 33554432000 4194304 4964800 140734025808224 18446744073709551615 4627338 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=20117/tid=20119] ppid=20115 vsize=451648 CPUtime=4762.64 cores=0-7
/proc/20117/task/20119/stat : 20119 (claspmt) R 20115 20117 20053 0 -1 4202560 594 0 0 0 475010 1254 0 0 19 0 8 0 117481879 462487552 8455 33554432000 4194304 4964800 140734025808224 18446744073709551615 4670499 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20117/tid=20120] ppid=20115 vsize=451648 CPUtime=4760.63 cores=0-7
/proc/20117/task/20120/stat : 20120 (claspmt) R 20115 20117 20053 0 -1 4202560 302 0 0 0 474878 1185 0 0 18 0 8 0 117481879 462487552 8455 33554432000 4194304 4964800 140734025808224 18446744073709551615 4660144 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=20117/tid=20121] ppid=20115 vsize=451648 CPUtime=4761.16 cores=0-7
/proc/20117/task/20121/stat : 20121 (claspmt) R 20115 20117 20053 0 -1 4202560 453 0 0 0 474930 1186 0 0 18 0 8 0 117481879 462487552 8455 33554432000 4194304 4964800 140734025808224 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20117/tid=20122] ppid=20115 vsize=451648 CPUtime=4759.06 cores=0-7
/proc/20117/task/20122/stat : 20122 (claspmt) R 20115 20117 20053 0 -1 4202560 371 0 0 0 474661 1245 0 0 20 0 8 0 117481879 462487552 8455 33554432000 4194304 4964800 140734025808224 18446744073709551615 4670310 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20117/tid=20123] ppid=20115 vsize=451648 CPUtime=4759.15 cores=0-7
/proc/20117/task/20123/stat : 20123 (claspmt) R 20115 20117 20053 0 -1 4202560 367 0 0 0 474726 1189 0 0 21 0 8 0 117481879 462487552 8455 33554432000 4194304 4964800 140734025808224 18446744073709551615 4653136 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20117/tid=20124] ppid=20115 vsize=451648 CPUtime=4760.34 cores=0-7
/proc/20117/task/20124/stat : 20124 (claspmt) R 20115 20117 20053 0 -1 4202560 1843 0 0 0 474924 1110 0 0 20 0 8 0 117481879 462487552 8455 33554432000 4194304 4964800 140734025808224 18446744073709551615 4668712 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 38092.4
Current children cumulated vsize (KiB) 451648

[startup+4842.3 s]
/proc/loadavg: 8.07 8.03 8.00 9/366 20265
/proc/meminfo: memFree=21403180/32950928 swapFree=62959384/67111528
[pid=20117] ppid=20115 vsize=451648 CPUtime=38570.3 cores=0-7
/proc/20117/stat : 20117 (claspmt) R 20115 20117 20053 0 -1 4202496 8547 0 0 0 3847603 9426 0 0 25 0 8 0 117481879 462487552 8470 33554432000 4194304 4964800 140734025808224 18446744073709551615 4642509 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/20117/statm: 112912 8470 468 189 0 107261 0
[pid=20117/tid=20118] ppid=20115 vsize=451648 CPUtime=4823.5 cores=0-7
/proc/20117/task/20118/stat : 20118 (claspmt) R 20115 20117 20053 0 -1 4202560 2709 0 0 0 481243 1107 0 0 25 0 8 0 117481879 462487552 8470 33554432000 4194304 4964800 140734025808224 18446744073709551615 4669199 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=20117/tid=20119] ppid=20115 vsize=451648 CPUtime=4822.45 cores=0-7
/proc/20117/task/20119/stat : 20119 (claspmt) R 20115 20117 20053 0 -1 4202560 594 0 0 0 480976 1269 0 0 23 0 8 0 117481879 462487552 8470 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880670239 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20117/tid=20120] ppid=20115 vsize=451648 CPUtime=4820 cores=0-7
/proc/20117/task/20120/stat : 20120 (claspmt) R 20115 20117 20053 0 -1 4202560 303 0 0 0 480802 1198 0 0 25 0 8 0 117481879 462487552 8470 33554432000 4194304 4964800 140734025808224 18446744073709551615 4319742 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20117/tid=20121] ppid=20115 vsize=451648 CPUtime=4820.89 cores=0-7
/proc/20117/task/20121/stat : 20121 (claspmt) R 20115 20117 20053 0 -1 4202560 454 0 0 0 480889 1200 0 0 18 0 8 0 117481879 462487552 8470 33554432000 4194304 4964800 140734025808224 18446744073709551615 4654540 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20117/tid=20122] ppid=20115 vsize=451648 CPUtime=4818.76 cores=0-7
/proc/20117/task/20122/stat : 20122 (claspmt) R 20115 20117 20053 0 -1 4202560 371 0 0 0 480618 1258 0 0 25 0 8 0 117481879 462487552 8470 33554432000 4194304 4964800 140734025808224 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=20117/tid=20123] ppid=20115 vsize=451648 CPUtime=4819.02 cores=0-7
/proc/20117/task/20123/stat : 20123 (claspmt) R 20115 20117 20053 0 -1 4202560 367 0 0 0 480699 1203 0 0 18 0 8 0 117481879 462487552 8470 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880670239 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20117/tid=20124] ppid=20115 vsize=451648 CPUtime=4820.09 cores=0-7
/proc/20117/task/20124/stat : 20124 (claspmt) R 20115 20117 20053 0 -1 4202560 1853 0 0 0 480888 1121 0 0 25 0 8 0 117481879 462487552 8470 33554432000 4194304 4964800 140734025808224 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 38570.3
Current children cumulated vsize (KiB) 451648

[startup+4902.3 s]
/proc/loadavg: 8.07 8.03 8.00 9/366 20267
/proc/meminfo: memFree=21403280/32950928 swapFree=62959384/67111528
[pid=20117] ppid=20115 vsize=451648 CPUtime=39048.4 cores=0-7
/proc/20117/stat : 20117 (claspmt) R 20115 20117 20053 0 -1 4202496 8553 0 0 0 3895302 9535 0 0 21 0 8 0 117481879 462487552 8476 33554432000 4194304 4964800 140734025808224 18446744073709551615 4668568 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/20117/statm: 112912 8476 468 189 0 107261 0
[pid=20117/tid=20118] ppid=20115 vsize=451648 CPUtime=4883.41 cores=0-7
/proc/20117/task/20118/stat : 20118 (claspmt) R 20115 20117 20053 0 -1 4202560 2712 0 0 0 487222 1119 0 0 25 0 8 0 117481879 462487552 8476 33554432000 4194304 4964800 140734025808224 18446744073709551615 4668734 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=20117/tid=20119] ppid=20115 vsize=451648 CPUtime=4882.22 cores=0-7
/proc/20117/task/20119/stat : 20119 (claspmt) R 20115 20117 20053 0 -1 4202560 594 0 0 0 486938 1284 0 0 18 0 8 0 117481879 462487552 8476 33554432000 4194304 4964800 140734025808224 18446744073709551615 218880232349 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20117/tid=20120] ppid=20115 vsize=451648 CPUtime=4879.88 cores=0-7
/proc/20117/task/20120/stat : 20120 (claspmt) R 20115 20117 20053 0 -1 4202560 303 0 0 0 486776 1212 0 0 18 0 8 0 117481879 462487552 8476 33554432000 4194304 4964800 140734025808224 18446744073709551615 4342199 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20117/tid=20121] ppid=20115 vsize=451648 CPUtime=4880.68 cores=0-7
/proc/20117/task/20121/stat : 20121 (claspmt) R 20115 20117 20053 0 -1 4202560 454 0 0 0 486851 1217 0 0 22 0 8 0 117481879 462487552 8476 33554432000 4194304 4964800 140734025808224 18446744073709551615 4670407 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20117/tid=20122] ppid=20115 vsize=451648 CPUtime=4878.04 cores=0-7
/proc/20117/task/20122/stat : 20122 (claspmt) R 20115 20117 20053 0 -1 4202560 371 0 0 0 486533 1271 0 0 18 0 8 0 117481879 462487552 8476 33554432000 4194304 4964800 140734025808224 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=20117/tid=20123] ppid=20115 vsize=451648 CPUtime=4878.89 cores=0-7
/proc/20117/task/20123/stat : 20123 (claspmt) R 20115 20117 20053 0 -1 4202560 367 0 0 0 486673 1216 0 0 21 0 8 0 117481879 462487552 8476 33554432000 4194304 4964800 140734025808224 18446744073709551615 4670349 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20117/tid=20124] ppid=20115 vsize=451648 CPUtime=4879.88 cores=0-7
/proc/20117/task/20124/stat : 20124 (claspmt) R 20115 20117 20053 0 -1 4202560 1853 0 0 0 486854 1134 0 0 19 0 8 0 117481879 462487552 8476 33554432000 4194304 4964800 140734025808224 18446744073709551615 4669973 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 39048.4
Current children cumulated vsize (KiB) 451648

[startup+4962.3 s]
/proc/loadavg: 7.95 8.01 7.99 9/366 20269
/proc/meminfo: memFree=21403376/32950928 swapFree=62959384/67111528
[pid=20117] ppid=20115 vsize=451648 CPUtime=39526.3 cores=0-7
/proc/20117/stat : 20117 (claspmt) R 20115 20117 20053 0 -1 4202496 8553 0 0 0 3942983 9650 0 0 23 0 8 0 117481879 462487552 8476 33554432000 4194304 4964800 140734025808224 18446744073709551615 4670134 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/20117/statm: 112912 8476 468 189 0 107261 0
[pid=20117/tid=20118] ppid=20115 vsize=451648 CPUtime=4943.3 cores=0-7
/proc/20117/task/20118/stat : 20118 (claspmt) R 20115 20117 20053 0 -1 4202560 2712 0 0 0 493200 1130 0 0 25 0 8 0 117481879 462487552 8476 33554432000 4194304 4964800 140734025808224 18446744073709551615 4670534 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=20117/tid=20119] ppid=20115 vsize=451648 CPUtime=4941.88 cores=0-7
/proc/20117/task/20119/stat : 20119 (claspmt) R 20115 20117 20053 0 -1 4202560 594 0 0 0 492885 1303 0 0 23 0 8 0 117481879 462487552 8476 33554432000 4194304 4964800 140734025808224 18446744073709551615 4670313 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=20117/tid=20120] ppid=20115 vsize=451648 CPUtime=4939.75 cores=0-7
/proc/20117/task/20120/stat : 20120 (claspmt) R 20115 20117 20053 0 -1 4202560 303 0 0 0 492747 1228 0 0 19 0 8 0 117481879 462487552 8476 33554432000 4194304 4964800 140734025808224 18446744073709551615 4320961 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20117/tid=20121] ppid=20115 vsize=451648 CPUtime=4940.38 cores=0-7
/proc/20117/task/20121/stat : 20121 (claspmt) R 20115 20117 20053 0 -1 4202560 454 0 0 0 492806 1232 0 0 25 0 8 0 117481879 462487552 8476 33554432000 4194304 4964800 140734025808224 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20117/tid=20122] ppid=20115 vsize=451648 CPUtime=4937.46 cores=0-7
/proc/20117/task/20122/stat : 20122 (claspmt) R 20115 20117 20053 0 -1 4202560 371 0 0 0 492461 1285 0 0 23 0 8 0 117481879 462487552 8476 33554432000 4194304 4964800 140734025808224 18446744073709551615 4684107 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20117/tid=20123] ppid=20115 vsize=451648 CPUtime=4938.74 cores=0-7
/proc/20117/task/20123/stat : 20123 (claspmt) R 20115 20117 20053 0 -1 4202560 367 0 0 0 492644 1230 0 0 18 0 8 0 117481879 462487552 8476 33554432000 4194304 4964800 140734025808224 18446744073709551615 4669184 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20117/tid=20124] ppid=20115 vsize=451648 CPUtime=4939.65 cores=0-7
/proc/20117/task/20124/stat : 20124 (claspmt) R 20115 20117 20053 0 -1 4202560 1853 0 0 0 492817 1148 0 0 18 0 8 0 117481879 462487552 8476 33554432000 4194304 4964800 140734025808224 18446744073709551615 4321016 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 39526.3
Current children cumulated vsize (KiB) 451648



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5021.8 s]
/proc/loadavg: 7.98 8.00 7.99 9/366 20270
/proc/meminfo: memFree=21403472/32950928 swapFree=62959384/67111528
[pid=20117] ppid=20115 vsize=451648 CPUtime=40000.4 cores=0-7
/proc/20117/stat : 20117 (claspmt) R 20115 20117 20053 0 -1 4202496 8555 0 0 0 3990284 9758 0 0 19 0 8 0 117481879 462487552 8478 33554432000 4194304 4964800 140734025808224 18446744073709551615 4684120 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/20117/statm: 112912 8478 468 189 0 107261 0
[pid=20117/tid=20118] ppid=20115 vsize=451648 CPUtime=5002.72 cores=0-7
/proc/20117/task/20118/stat : 20118 (claspmt) R 20115 20117 20053 0 -1 4202560 2712 0 0 0 499133 1139 0 0 18 0 8 0 117481879 462487552 8478 33554432000 4194304 4964800 140734025808224 18446744073709551615 4320872 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=20117/tid=20119] ppid=20115 vsize=451648 CPUtime=5000.6 cores=0-7
/proc/20117/task/20119/stat : 20119 (claspmt) R 20115 20117 20053 0 -1 4202560 594 0 0 0 498740 1320 0 0 23 0 8 0 117481879 462487552 8478 33554432000 4194304 4964800 140734025808224 18446744073709551615 4660551 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=20117/tid=20120] ppid=20115 vsize=451648 CPUtime=4999.12 cores=0-7
/proc/20117/task/20120/stat : 20120 (claspmt) R 20115 20117 20053 0 -1 4202560 304 0 0 0 498669 1243 0 0 25 0 8 0 117481879 462487552 8478 33554432000 4194304 4964800 140734025808224 18446744073709551615 4321069 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20117/tid=20121] ppid=20115 vsize=451648 CPUtime=4999.61 cores=0-7
/proc/20117/task/20121/stat : 20121 (claspmt) R 20115 20117 20053 0 -1 4202560 454 0 0 0 498715 1246 0 0 23 0 8 0 117481879 462487552 8478 33554432000 4194304 4964800 140734025808224 18446744073709551615 4362375 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20117/tid=20122] ppid=20115 vsize=451648 CPUtime=4996.82 cores=0-7
/proc/20117/task/20122/stat : 20122 (claspmt) R 20115 20117 20053 0 -1 4202560 371 0 0 0 498382 1300 0 0 23 0 8 0 117481879 462487552 8478 33554432000 4194304 4964800 140734025808224 18446744073709551615 4670330 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20117/tid=20123] ppid=20115 vsize=451648 CPUtime=4998.12 cores=0-7
/proc/20117/task/20123/stat : 20123 (claspmt) R 20115 20117 20053 0 -1 4202560 367 0 0 0 498568 1244 0 0 25 0 8 0 117481879 462487552 8478 33554432000 4194304 4964800 140734025808224 18446744073709551615 4654532 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20117/tid=20124] ppid=20115 vsize=451648 CPUtime=4998.92 cores=0-7
/proc/20117/task/20124/stat : 20124 (claspmt) R 20115 20117 20053 0 -1 4202560 1854 0 0 0 498731 1161 0 0 19 0 8 0 117481879 462487552 8478 33554432000 4194304 4964800 140734025808224 18446744073709551615 4654552 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 451648

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5021.8 s]
/proc/loadavg: 7.98 8.00 7.99 9/366 20270
/proc/meminfo: memFree=21403472/32950928 swapFree=62959384/67111528
[pid=20117] ppid=20115 vsize=451648 CPUtime=40000.4 cores=0-7
/proc/20117/stat : 20117 (claspmt) R 20115 20117 20053 0 -1 4202496 8555 0 0 0 3990284 9758 0 0 19 0 8 0 117481879 462487552 8478 33554432000 4194304 4964800 140734025808224 18446744073709551615 4684120 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/20117/statm: 112912 8478 468 189 0 107261 0
[pid=20117/tid=20118] ppid=20115 vsize=451648 CPUtime=5002.72 cores=0-7
/proc/20117/task/20118/stat : 20118 (claspmt) R 20115 20117 20053 0 -1 4202560 2712 0 0 0 499133 1139 0 0 18 0 8 0 117481879 462487552 8478 33554432000 4194304 4964800 140734025808224 18446744073709551615 4320872 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=20117/tid=20119] ppid=20115 vsize=451648 CPUtime=5000.6 cores=0-7
/proc/20117/task/20119/stat : 20119 (claspmt) R 20115 20117 20053 0 -1 4202560 594 0 0 0 498740 1320 0 0 23 0 8 0 117481879 462487552 8478 33554432000 4194304 4964800 140734025808224 18446744073709551615 4660551 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=20117/tid=20120] ppid=20115 vsize=451648 CPUtime=4999.12 cores=0-7
/proc/20117/task/20120/stat : 20120 (claspmt) R 20115 20117 20053 0 -1 4202560 304 0 0 0 498669 1243 0 0 25 0 8 0 117481879 462487552 8478 33554432000 4194304 4964800 140734025808224 18446744073709551615 4321069 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20117/tid=20121] ppid=20115 vsize=451648 CPUtime=4999.61 cores=0-7
/proc/20117/task/20121/stat : 20121 (claspmt) R 20115 20117 20053 0 -1 4202560 454 0 0 0 498715 1246 0 0 23 0 8 0 117481879 462487552 8478 33554432000 4194304 4964800 140734025808224 18446744073709551615 4362375 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20117/tid=20122] ppid=20115 vsize=451648 CPUtime=4996.82 cores=0-7
/proc/20117/task/20122/stat : 20122 (claspmt) R 20115 20117 20053 0 -1 4202560 371 0 0 0 498382 1300 0 0 23 0 8 0 117481879 462487552 8478 33554432000 4194304 4964800 140734025808224 18446744073709551615 4670330 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20117/tid=20123] ppid=20115 vsize=451648 CPUtime=4998.12 cores=0-7
/proc/20117/task/20123/stat : 20123 (claspmt) R 20115 20117 20053 0 -1 4202560 367 0 0 0 498568 1244 0 0 25 0 8 0 117481879 462487552 8478 33554432000 4194304 4964800 140734025808224 18446744073709551615 4654532 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20117/tid=20124] ppid=20115 vsize=451648 CPUtime=4998.92 cores=0-7
/proc/20117/task/20124/stat : 20124 (claspmt) R 20115 20117 20053 0 -1 4202560 1854 0 0 0 498731 1161 0 0 19 0 8 0 117481879 462487552 8478 33554432000 4194304 4964800 140734025808224 18446744073709551615 4654552 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 451648

Child status: 0
Real time (s): 5021.82
CPU time (s): 40000.5
CPU user time (s): 39902.9
CPU system time (s): 97.5882
CPU usage (%): 796.533
Max. virtual memory (cumulated for all children) (KiB): 451648

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39902.9
system time used= 97.5882
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 8566
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 17171963
involuntary context switches= 185138

runsolver used 16.6895 second user time and 38.9611 second system time

The end

Launcher Data 

Begin job on node112 at 2011-05-16 00:32:05
IDJOB=3380980
IDBENCH=83140
IDSOLVER=1865
FILE ID=node112/3380980-1305498725
RUNJOBID= node112-1305498614-20067
PBS_JOBID= 13325447
Free space on /tmp= 73528 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT11/crafted/spence/sgen/sgen3-n180-s33098896-unsat.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3380980-1305498725/watcher-3380980-1305498725 -o /tmp/evaluation-result-3380980-1305498725/solver-3380980-1305498725 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3380980-1305498725.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 1221fb51c407dee7559369d7c6eedb4e
RANDOM SEED=751747746

node112.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5180.45
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      21440724 kB
Buffers:          7368 kB
Cached:         145008 kB
SwapCached:      73280 kB
Active:         118996 kB
Inactive:     11208064 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      21440724 kB
SwapTotal:    67111528 kB
SwapFree:     62959384 kB
Dirty:             824 kB
Writeback:           0 kB
AnonPages:    11110740 kB
Mapped:          20508 kB
Slab:            64448 kB
PageTables:      52456 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 74555688 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73524 MiB
End job on node112 at 2011-05-16 01:55:49