Trace number 3322580

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095? (TO) 40000.2 5011.92

General information on the benchmark

NameSAT03/handmade/stanion/hwb/
hwb-n32-02-S1478512323.shuffled-as.sat03-1638.cnf
MD5SUM1d86c7f9cd05c5f2ab070520d802c8e1
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables226
Number of clauses1078
Sum of the clauses size3624
Maximum clause length4
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 226
Number of clauses of size 3636
Number of clauses of size 4416
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...390585/instance-3322580-1304390585.cnf
0.00/0.00	c Solving...
40000.14/5011.91	
40000.14/5011.91	*** INTERRUPTED! ***
40000.14/5011.91	*** Shutting down threads...
40000.14/5011.92	s UNKNOWN
40000.14/5011.92	
40000.14/5011.92	c INTERRUPTED   : 1
40000.14/5011.92	c Models        : 0+    
40000.14/5011.92	c Time          : 5011.912s (Solving: 5011.91s 1st Model: 0.00s Unsat: 0.00s)
40000.14/5011.92	c CPU Time      : 40000.180s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3322580-1304390585/watcher-3322580-1304390585 -o /tmp/evaluation-result-3322580-1304390585/solver-3322580-1304390585 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3322580-1304390585.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.79 7.78 8.05 2/288 12332
/proc/meminfo: memFree=17423708/32950928 swapFree=67111528/67111528
[pid=12332] ppid=12330 vsize=119248 CPUtime=0 cores=0-7
/proc/12332/stat : 12332 (claspmt) S 12330 12332 12306 0 -1 4202496 851 0 0 0 0 0 0 0 25 0 8 0 6667902 122109952 774 33554432000 4194304 4964704 140736319562624 18446744073709551615 218892381081 0 0 4096 18946 18446744073709551615 0 0 17 6 0 0 0
/proc/12332/statm: 29812 774 454 189 0 24161 0
[pid=12332/tid=12333] ppid=12330 vsize=119248 CPUtime=0 cores=0-7
/proc/12332/task/12333/stat : 12333 (claspmt) R 12330 12332 12306 0 -1 4202560 23 0 0 0 0 0 0 0 25 0 8 0 6667903 122109952 774 33554432000 4194304 4964704 140736319562624 18446744073709551615 218880225989 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=12332/tid=12334] ppid=12330 vsize=119248 CPUtime=0 cores=0-7
/proc/12332/task/12334/stat : 12334 (claspmt) R 12330 12332 12306 0 -1 4202560 16 0 0 0 0 0 0 0 25 0 8 0 6667903 122109952 774 33554432000 4194304 4964704 140736319562624 18446744073709551615 218892381081 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=12332/tid=12335] ppid=12330 vsize=119248 CPUtime=0 cores=0-7
/proc/12332/task/12335/stat : 12335 (claspmt) S 12330 12332 12306 0 -1 4202560 23 0 0 0 0 0 0 0 25 0 8 0 6667903 122109952 774 33554432000 4194304 4964704 140736319562624 18446744073709551615 218892381081 0 0 4096 18946 18446604443877319808 0 0 -1 6 0 0 0
[pid=12332/tid=12336] ppid=12330 vsize=119248 CPUtime=0 cores=0-7
/proc/12332/task/12336/stat : 12336 (claspmt) S 12330 12332 12306 0 -1 4202560 20 0 0 0 0 0 0 0 25 0 8 0 6667903 122109952 774 33554432000 4194304 4964704 140736319562624 18446744073709551615 218892381081 0 0 4096 18946 18446604443877319808 0 0 -1 6 0 0 0
[pid=12332/tid=12337] ppid=12330 vsize=119248 CPUtime=0 cores=0-7
/proc/12332/task/12337/stat : 12337 (claspmt) S 12330 12332 12306 0 -1 4202560 26 0 0 0 0 0 0 0 25 0 8 0 6667903 122109952 775 33554432000 4194304 4964704 140736319562624 18446744073709551615 218892381081 0 0 4096 18946 18446604443877319808 0 0 -1 6 0 0 0
[pid=12332/tid=12338] ppid=12330 vsize=119248 CPUtime=0 cores=0-7
/proc/12332/task/12338/stat : 12338 (claspmt) S 12330 12332 12306 0 -1 4202560 24 0 0 0 0 0 0 0 25 0 8 0 6667903 122109952 775 33554432000 4194304 4964704 140736319562624 18446744073709551615 218892381081 0 0 4096 18946 18446604443877319808 0 0 -1 6 0 0 0
[pid=12332/tid=12339] ppid=12330 vsize=119248 CPUtime=0 cores=0-7
/proc/12332/task/12339/stat : 12339 (claspmt) R 12330 12332 12306 0 -1 4202560 55 0 0 0 0 0 0 0 25 0 8 0 6667903 122109952 775 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670450 0 0 4096 18946 0 0 0 -1 6 0 0 0

[startup+0.0739669 s]
/proc/loadavg: 6.79 7.78 8.05 2/288 12332
/proc/meminfo: memFree=17423708/32950928 swapFree=67111528/67111528
[pid=12332] ppid=12330 vsize=315988 CPUtime=0.42 cores=0-7
/proc/12332/stat : 12332 (claspmt) R 12330 12332 12306 0 -1 4202496 1180 0 0 0 42 0 0 0 18 0 8 0 6667902 323571712 1103 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670266 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/12332/statm: 78997 1103 464 189 0 73346 0
[pid=12332/tid=12333] ppid=12330 vsize=315988 CPUtime=0.05 cores=0-7
/proc/12332/task/12333/stat : 12333 (claspmt) R 12330 12332 12306 0 -1 4202560 88 0 0 0 5 0 0 0 18 0 8 0 6667903 323571712 1103 33554432000 4194304 4964704 140736319562624 18446744073709551615 218880670239 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=12332/tid=12334] ppid=12330 vsize=315988 CPUtime=0.05 cores=0-7
/proc/12332/task/12334/stat : 12334 (claspmt) R 12330 12332 12306 0 -1 4202560 81 0 0 0 5 0 0 0 18 0 8 0 6667903 323571712 1103 33554432000 4194304 4964704 140736319562624 18446744073709551615 218880670239 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=12332/tid=12335] ppid=12330 vsize=315988 CPUtime=0.04 cores=0-7
/proc/12332/task/12335/stat : 12335 (claspmt) R 12330 12332 12306 0 -1 4202560 54 0 0 0 4 0 0 0 25 0 8 0 6667903 323571712 1103 33554432000 4194304 4964704 140736319562624 18446744073709551615 218880670239 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=12332/tid=12336] ppid=12330 vsize=315988 CPUtime=0.05 cores=0-7
/proc/12332/task/12336/stat : 12336 (claspmt) R 12330 12332 12306 0 -1 4202560 50 0 0 0 5 0 0 0 18 0 8 0 6667903 323571712 1103 33554432000 4194304 4964704 140736319562624 18446744073709551615 4642509 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=12332/tid=12337] ppid=12330 vsize=315988 CPUtime=0.04 cores=0-7
/proc/12332/task/12337/stat : 12337 (claspmt) R 12330 12332 12306 0 -1 4202560 51 0 0 0 4 0 0 0 25 0 8 0 6667903 323571712 1103 33554432000 4194304 4964704 140736319562624 18446744073709551615 218880670239 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=12332/tid=12338] ppid=12330 vsize=315988 CPUtime=0.04 cores=0-7
/proc/12332/task/12338/stat : 12338 (claspmt) R 12330 12332 12306 0 -1 4202560 38 0 0 0 4 0 0 0 25 0 8 0 6667903 323571712 1103 33554432000 4194304 4964704 140736319562624 18446744073709551615 218880670239 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=12332/tid=12339] ppid=12330 vsize=315988 CPUtime=0.05 cores=0-7
/proc/12332/task/12339/stat : 12339 (claspmt) R 12330 12332 12306 0 -1 4202560 125 0 0 0 5 0 0 0 25 0 8 0 6667903 323571712 1103 33554432000 4194304 4964704 140736319562624 18446744073709551615 218880670190 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 0.42
Current children cumulated vsize (KiB) 315988

[startup+0.115484 s]
/proc/loadavg: 6.79 7.78 8.05 2/288 12332
/proc/meminfo: memFree=17423708/32950928 swapFree=67111528/67111528
[pid=12332] ppid=12330 vsize=315988 CPUtime=0.75 cores=0-7
/proc/12332/stat : 12332 (claspmt) R 12330 12332 12306 0 -1 4202496 1270 0 0 0 75 0 0 0 18 0 8 0 6667902 323571712 1193 33554432000 4194304 4964704 140736319562624 18446744073709551615 218880670239 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/12332/statm: 78997 1193 464 189 0 73346 0
[pid=12332/tid=12333] ppid=12330 vsize=315988 CPUtime=0.09 cores=0-7
/proc/12332/task/12333/stat : 12333 (claspmt) R 12330 12332 12306 0 -1 4202560 96 0 0 0 9 0 0 0 18 0 8 0 6667903 323571712 1193 33554432000 4194304 4964704 140736319562624 18446744073709551615 4325321 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=12332/tid=12334] ppid=12330 vsize=315988 CPUtime=0.1 cores=0-7
/proc/12332/task/12334/stat : 12334 (claspmt) R 12330 12332 12306 0 -1 4202560 88 0 0 0 10 0 0 0 18 0 8 0 6667903 323571712 1193 33554432000 4194304 4964704 140736319562624 18446744073709551615 218880670239 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=12332/tid=12335] ppid=12330 vsize=315988 CPUtime=0.09 cores=0-7
/proc/12332/task/12335/stat : 12335 (claspmt) R 12330 12332 12306 0 -1 4202560 61 0 0 0 9 0 0 0 25 0 8 0 6667903 323571712 1193 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=12332/tid=12336] ppid=12330 vsize=315988 CPUtime=0.09 cores=0-7
/proc/12332/task/12336/stat : 12336 (claspmt) R 12330 12332 12306 0 -1 4202560 62 0 0 0 9 0 0 0 18 0 8 0 6667903 323571712 1193 33554432000 4194304 4964704 140736319562624 18446744073709551615 4668609 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=12332/tid=12337] ppid=12330 vsize=315988 CPUtime=0.09 cores=0-7
/proc/12332/task/12337/stat : 12337 (claspmt) R 12330 12332 12306 0 -1 4202560 65 0 0 0 9 0 0 0 25 0 8 0 6667903 323571712 1193 33554432000 4194304 4964704 140736319562624 18446744073709551615 218880670239 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=12332/tid=12338] ppid=12330 vsize=315988 CPUtime=0.08 cores=0-7
/proc/12332/task/12338/stat : 12338 (claspmt) R 12330 12332 12306 0 -1 4202560 54 0 0 0 8 0 0 0 25 0 8 0 6667903 323571712 1193 33554432000 4194304 4964704 140736319562624 18446744073709551615 218880670239 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=12332/tid=12339] ppid=12330 vsize=315988 CPUtime=0.09 cores=0-7
/proc/12332/task/12339/stat : 12339 (claspmt) R 12330 12332 12306 0 -1 4202560 134 0 0 0 9 0 0 0 25 0 8 0 6667903 323571712 1193 33554432000 4194304 4964704 140736319562624 18446744073709551615 218880670190 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 0.75
Current children cumulated vsize (KiB) 315988

[startup+0.301231 s]
/proc/loadavg: 6.79 7.78 8.05 2/288 12332
/proc/meminfo: memFree=17423708/32950928 swapFree=67111528/67111528
[pid=12332] ppid=12330 vsize=316120 CPUtime=2.24 cores=0-7
/proc/12332/stat : 12332 (claspmt) R 12330 12332 12306 0 -1 4202496 1594 0 0 0 223 1 0 0 18 0 8 0 6667902 323706880 1517 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670219 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/12332/statm: 79030 1517 464 189 0 73379 0
[pid=12332/tid=12333] ppid=12330 vsize=316120 CPUtime=0.28 cores=0-7
/proc/12332/task/12333/stat : 12333 (claspmt) R 12330 12332 12306 0 -1 4202560 147 0 0 0 28 0 0 0 18 0 8 0 6667903 323706880 1517 33554432000 4194304 4964704 140736319562624 18446744073709551615 4660755 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=12332/tid=12334] ppid=12330 vsize=316120 CPUtime=0.28 cores=0-7
/proc/12332/task/12334/stat : 12334 (claspmt) R 12330 12332 12306 0 -1 4202560 155 0 0 0 28 0 0 0 18 0 8 0 6667903 323706880 1517 33554432000 4194304 4964704 140736319562624 18446744073709551615 4668645 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=12332/tid=12335] ppid=12330 vsize=316120 CPUtime=0.27 cores=0-7
/proc/12332/task/12335/stat : 12335 (claspmt) R 12330 12332 12306 0 -1 4202560 85 0 0 0 27 0 0 0 25 0 8 0 6667903 323706880 1517 33554432000 4194304 4964704 140736319562624 18446744073709551615 4660034 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=12332/tid=12336] ppid=12330 vsize=316120 CPUtime=0.27 cores=0-7
/proc/12332/task/12336/stat : 12336 (claspmt) R 12330 12332 12306 0 -1 4202560 93 0 0 0 27 0 0 0 18 0 8 0 6667903 323706880 1517 33554432000 4194304 4964704 140736319562624 18446744073709551615 4359943 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=12332/tid=12337] ppid=12330 vsize=316120 CPUtime=0.27 cores=0-7
/proc/12332/task/12337/stat : 12337 (claspmt) R 12330 12332 12306 0 -1 4202560 81 0 0 0 27 0 0 0 25 0 8 0 6667903 323706880 1517 33554432000 4194304 4964704 140736319562624 18446744073709551615 4342174 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=12332/tid=12338] ppid=12330 vsize=316120 CPUtime=0.27 cores=0-7
/proc/12332/task/12338/stat : 12338 (claspmt) R 12330 12332 12306 0 -1 4202560 74 0 0 0 27 0 0 0 25 0 8 0 6667903 323706880 1517 33554432000 4194304 4964704 140736319562624 18446744073709551615 218880670239 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=12332/tid=12339] ppid=12330 vsize=316120 CPUtime=0.28 cores=0-7
/proc/12332/task/12339/stat : 12339 (claspmt) R 12330 12332 12306 0 -1 4202560 206 0 0 0 28 0 0 0 25 0 8 0 6667903 323706880 1517 33554432000 4194304 4964704 140736319562624 18446744073709551615 4473200 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2.24
Current children cumulated vsize (KiB) 316120

[startup+0.701161 s]
/proc/loadavg: 6.79 7.78 8.05 2/288 12332
/proc/meminfo: memFree=17423708/32950928 swapFree=67111528/67111528
[pid=12332] ppid=12330 vsize=316516 CPUtime=5.43 cores=0-7
/proc/12332/stat : 12332 (claspmt) R 12330 12332 12306 0 -1 4202496 1995 0 0 0 541 2 0 0 18 0 8 0 6667902 324112384 1918 33554432000 4194304 4964704 140736319562624 18446744073709551615 4342144 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/12332/statm: 79129 1918 464 189 0 73478 0
[pid=12332/tid=12333] ppid=12330 vsize=316516 CPUtime=0.68 cores=0-7
/proc/12332/task/12333/stat : 12333 (claspmt) R 12330 12332 12306 0 -1 4202560 206 0 0 0 68 0 0 0 18 0 8 0 6667903 324112384 1918 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670255 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=12332/tid=12334] ppid=12330 vsize=316516 CPUtime=0.68 cores=0-7
/proc/12332/task/12334/stat : 12334 (claspmt) R 12330 12332 12306 0 -1 4202560 241 0 0 0 68 0 0 0 18 0 8 0 6667903 324112384 1918 33554432000 4194304 4964704 140736319562624 18446744073709551615 4660077 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=12332/tid=12335] ppid=12330 vsize=316516 CPUtime=0.67 cores=0-7
/proc/12332/task/12335/stat : 12335 (claspmt) R 12330 12332 12306 0 -1 4202560 113 0 0 0 67 0 0 0 18 0 8 0 6667903 324112384 1918 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=12332/tid=12336] ppid=12330 vsize=316516 CPUtime=0.67 cores=0-7
/proc/12332/task/12336/stat : 12336 (claspmt) R 12330 12332 12306 0 -1 4202560 145 0 0 0 67 0 0 0 18 0 8 0 6667903 324112384 1918 33554432000 4194304 4964704 140736319562624 18446744073709551615 4669349 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=12332/tid=12337] ppid=12330 vsize=316516 CPUtime=0.67 cores=0-7
/proc/12332/task/12337/stat : 12337 (claspmt) R 12330 12332 12306 0 -1 4202560 97 0 0 0 67 0 0 0 25 0 8 0 6667903 324112384 1918 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=12332/tid=12338] ppid=12330 vsize=316516 CPUtime=0.67 cores=0-7
/proc/12332/task/12338/stat : 12338 (claspmt) R 12330 12332 12306 0 -1 4202560 117 0 0 0 67 0 0 0 25 0 8 0 6667903 324112384 1918 33554432000 4194304 4964704 140736319562624 18446744073709551615 4669270 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=12332/tid=12339] ppid=12330 vsize=316516 CPUtime=0.68 cores=0-7
/proc/12332/task/12339/stat : 12339 (claspmt) R 12330 12332 12306 0 -1 4202560 275 0 0 0 68 0 0 0 25 0 8 0 6667903 324112384 1918 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670281 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 5.43
Current children cumulated vsize (KiB) 316516

[startup+1.50102 s]
/proc/loadavg: 6.79 7.78 8.05 9/296 12340
/proc/meminfo: memFree=17417196/32950928 swapFree=67111528/67111528
[pid=12332] ppid=12330 vsize=316908 CPUtime=11.81 cores=0-7
/proc/12332/stat : 12332 (claspmt) R 12330 12332 12306 0 -1 4202496 2387 0 0 0 1178 3 0 0 18 0 8 0 6667902 324513792 2310 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670281 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/12332/statm: 79227 2310 464 189 0 73576 0
[pid=12332/tid=12333] ppid=12330 vsize=316908 CPUtime=1.47 cores=0-7
/proc/12332/task/12333/stat : 12333 (claspmt) R 12330 12332 12306 0 -1 4202560 262 0 0 0 147 0 0 0 18 0 8 0 6667903 324513792 2310 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=12332/tid=12334] ppid=12330 vsize=316908 CPUtime=1.48 cores=0-7
/proc/12332/task/12334/stat : 12334 (claspmt) R 12330 12332 12306 0 -1 4202560 307 0 0 0 148 0 0 0 20 0 8 0 6667903 324513792 2310 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670184 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=12332/tid=12335] ppid=12330 vsize=316908 CPUtime=1.46 cores=0-7
/proc/12332/task/12335/stat : 12335 (claspmt) R 12330 12332 12306 0 -1 4202560 145 0 0 0 146 0 0 0 19 0 8 0 6667903 324513792 2310 33554432000 4194304 4964704 140736319562624 18446744073709551615 4669200 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=12332/tid=12336] ppid=12330 vsize=316908 CPUtime=1.47 cores=0-7
/proc/12332/task/12336/stat : 12336 (claspmt) R 12330 12332 12306 0 -1 4202560 207 0 0 0 147 0 0 0 18 0 8 0 6667903 324513792 2310 33554432000 4194304 4964704 140736319562624 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=12332/tid=12337] ppid=12330 vsize=316908 CPUtime=1.46 cores=0-7
/proc/12332/task/12337/stat : 12337 (claspmt) R 12330 12332 12306 0 -1 4202560 117 0 0 0 146 0 0 0 25 0 8 0 6667903 324513792 2310 33554432000 4194304 4964704 140736319562624 18446744073709551615 4321023 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=12332/tid=12338] ppid=12330 vsize=316908 CPUtime=1.46 cores=0-7
/proc/12332/task/12338/stat : 12338 (claspmt) R 12330 12332 12306 0 -1 4202560 143 0 0 0 146 0 0 0 18 0 8 0 6667903 324513792 2310 33554432000 4194304 4964704 140736319562624 18446744073709551615 4660043 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=12332/tid=12339] ppid=12330 vsize=316908 CPUtime=1.47 cores=0-7
/proc/12332/task/12339/stat : 12339 (claspmt) R 12330 12332 12306 0 -1 4202560 365 0 0 0 147 0 0 0 25 0 8 0 6667903 324513792 2310 33554432000 4194304 4964704 140736319562624 18446744073709551615 4668618 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 11.81
Current children cumulated vsize (KiB) 316908

[startup+3.10074 s]
/proc/loadavg: 6.79 7.78 8.05 9/296 12340
/proc/meminfo: memFree=17415336/32950928 swapFree=67111528/67111528
[pid=12332] ppid=12330 vsize=317296 CPUtime=24.57 cores=0-7
/proc/12332/stat : 12332 (claspmt) R 12330 12332 12306 0 -1 4202496 2852 0 0 0 2450 7 0 0 19 0 8 0 6667902 324911104 2775 33554432000 4194304 4964704 140736319562624 18446744073709551615 4342208 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/12332/statm: 79324 2777 464 189 0 73673 0
[pid=12332/tid=12333] ppid=12330 vsize=317296 CPUtime=3.06 cores=0-7
/proc/12332/task/12333/stat : 12333 (claspmt) R 12330 12332 12306 0 -1 4202560 358 0 0 0 306 0 0 0 21 0 8 0 6667903 324911104 2777 33554432000 4194304 4964704 140736319562624 18446744073709551615 4668632 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=12332/tid=12334] ppid=12330 vsize=317296 CPUtime=3.07 cores=0-7
/proc/12332/task/12334/stat : 12334 (claspmt) R 12330 12332 12306 0 -1 4202560 400 0 0 0 307 0 0 0 19 0 8 0 6667903 324911104 2777 33554432000 4194304 4964704 140736319562624 18446744073709551615 218880261899 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=12332/tid=12335] ppid=12330 vsize=317296 CPUtime=3.06 cores=0-7
/proc/12332/task/12335/stat : 12335 (claspmt) R 12330 12332 12306 0 -1 4202560 156 0 0 0 306 0 0 0 24 0 8 0 6667903 324911104 2777 33554432000 4194304 4964704 140736319562624 18446744073709551615 4669480 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=12332/tid=12336] ppid=12330 vsize=317296 CPUtime=3.06 cores=0-7
/proc/12332/task/12336/stat : 12336 (claspmt) R 12330 12332 12306 0 -1 4202560 268 0 0 0 306 0 0 0 21 0 8 0 6667903 324911104 2777 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670263 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=12332/tid=12337] ppid=12330 vsize=317296 CPUtime=3.07 cores=0-7
/proc/12332/task/12337/stat : 12337 (claspmt) R 12330 12332 12306 0 -1 4202560 142 0 0 0 306 1 0 0 25 0 8 0 6667903 324911104 2777 33554432000 4194304 4964704 140736319562624 18446744073709551615 4642509 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=12332/tid=12338] ppid=12330 vsize=317296 CPUtime=3.06 cores=0-7
/proc/12332/task/12338/stat : 12338 (claspmt) R 12330 12332 12306 0 -1 4202560 165 0 0 0 306 0 0 0 19 0 8 0 6667903 324911104 2777 33554432000 4194304 4964704 140736319562624 18446744073709551615 218880670239 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=12332/tid=12339] ppid=12330 vsize=317296 CPUtime=3.05 cores=0-7
/proc/12332/task/12339/stat : 12339 (claspmt) R 12330 12332 12306 0 -1 4202560 450 0 0 0 305 0 0 0 19 0 8 0 6667903 324911104 2777 33554432000 4194304 4964704 140736319562624 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 24.57
Current children cumulated vsize (KiB) 317296

[startup+6.30118 s]
/proc/loadavg: 6.89 7.78 8.05 9/296 12340
/proc/meminfo: memFree=17412804/32950928 swapFree=67111528/67111528
[pid=12332] ppid=12330 vsize=317692 CPUtime=50.09 cores=0-7
/proc/12332/stat : 12332 (claspmt) R 12330 12332 12306 0 -1 4202496 3472 0 0 0 4996 13 0 0 18 0 8 0 6667902 325316608 3395 33554432000 4194304 4964704 140736319562624 18446744073709551615 4342199 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/12332/statm: 79423 3395 464 189 0 73772 0
[pid=12332/tid=12333] ppid=12330 vsize=317692 CPUtime=6.25 cores=0-7
/proc/12332/task/12333/stat : 12333 (claspmt) R 12330 12332 12306 0 -1 4202560 485 0 0 0 624 1 0 0 18 0 8 0 6667903 325316608 3395 33554432000 4194304 4964704 140736319562624 18446744073709551615 4668725 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=12332/tid=12334] ppid=12330 vsize=317692 CPUtime=6.23 cores=0-7
/proc/12332/task/12334/stat : 12334 (claspmt) R 12330 12332 12306 0 -1 4202560 493 0 0 0 622 1 0 0 25 0 8 0 6667903 325316608 3395 33554432000 4194304 4964704 140736319562624 18446744073709551615 4660755 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=12332/tid=12335] ppid=12330 vsize=317692 CPUtime=6.25 cores=0-7
/proc/12332/task/12335/stat : 12335 (claspmt) R 12330 12332 12306 0 -1 4202560 178 0 0 0 624 1 0 0 25 0 8 0 6667903 325316608 3395 33554432000 4194304 4964704 140736319562624 18446744073709551615 4669296 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=12332/tid=12336] ppid=12330 vsize=317692 CPUtime=6.26 cores=0-7
/proc/12332/task/12336/stat : 12336 (claspmt) R 12330 12332 12306 0 -1 4202560 373 0 0 0 624 2 0 0 25 0 8 0 6667903 325316608 3395 33554432000 4194304 4964704 140736319562624 18446744073709551615 4658308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=12332/tid=12337] ppid=12330 vsize=317692 CPUtime=6.27 cores=0-7
/proc/12332/task/12337/stat : 12337 (claspmt) R 12330 12332 12306 0 -1 4202560 171 0 0 0 625 2 0 0 18 0 8 0 6667903 325316608 3395 33554432000 4194304 4964704 140736319562624 18446744073709551615 4660364 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=12332/tid=12338] ppid=12330 vsize=317692 CPUtime=6.25 cores=0-7
/proc/12332/task/12338/stat : 12338 (claspmt) R 12330 12332 12306 0 -1 4202560 204 0 0 0 624 1 0 0 25 0 8 0 6667903 325316608 3395 33554432000 4194304 4964704 140736319562624 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=12332/tid=12339] ppid=12330 vsize=317692 CPUtime=6.25 cores=0-7
/proc/12332/task/12339/stat : 12339 (claspmt) R 12330 12332 12306 0 -1 4202560 558 0 0 0 624 1 0 0 19 0 8 0 6667903 325316608 3395 33554432000 4194304 4964704 140736319562624 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 0 0 0 0

################
# More data... #
################


[startup+4722.31 s]
/proc/loadavg: 8.00 8.01 8.00 9/296 12487
/proc/meminfo: memFree=17382448/32950928 swapFree=67111528/67111528
[pid=12332] ppid=12330 vsize=388280 CPUtime=37688.7 cores=0-7
/proc/12332/stat : 12332 (claspmt) R 12330 12332 12306 0 -1 4202496 10069 0 0 0 3765164 3709 0 0 21 0 8 0 6667902 397598720 9992 33554432000 4194304 4964704 140736319562624 18446744073709551615 4688896 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/12332/statm: 97070 9992 464 189 0 91419 0
[pid=12332/tid=12333] ppid=12330 vsize=388280 CPUtime=4709.94 cores=0-7
/proc/12332/task/12333/stat : 12333 (claspmt) R 12330 12332 12306 0 -1 4202560 1499 0 0 0 470492 502 0 0 25 0 8 0 6667903 397598720 9992 33554432000 4194304 4964704 140736319562624 18446744073709551615 4668654 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=12332/tid=12334] ppid=12330 vsize=388280 CPUtime=4709.95 cores=0-7
/proc/12332/task/12334/stat : 12334 (claspmt) R 12330 12332 12306 0 -1 4202560 1384 0 0 0 470520 475 0 0 25 0 8 0 6667903 397598720 9992 33554432000 4194304 4964704 140736319562624 18446744073709551615 4361118 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=12332/tid=12335] ppid=12330 vsize=388280 CPUtime=4712.83 cores=0-7
/proc/12332/task/12335/stat : 12335 (claspmt) R 12330 12332 12306 0 -1 4202560 938 0 0 0 470825 458 0 0 21 0 8 0 6667903 397598720 9992 33554432000 4194304 4964704 140736319562624 18446744073709551615 4321116 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=12332/tid=12336] ppid=12330 vsize=388280 CPUtime=4713.27 cores=0-7
/proc/12332/task/12336/stat : 12336 (claspmt) R 12330 12332 12306 0 -1 4202560 1128 0 0 0 470871 456 0 0 21 0 8 0 6667903 397598720 9992 33554432000 4194304 4964704 140736319562624 18446744073709551615 4669472 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=12332/tid=12337] ppid=12330 vsize=388280 CPUtime=4705.76 cores=0-7
/proc/12332/task/12337/stat : 12337 (claspmt) R 12330 12332 12306 0 -1 4202560 926 0 0 0 470111 465 0 0 18 0 8 0 6667903 397598720 9992 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670312 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=12332/tid=12338] ppid=12330 vsize=388280 CPUtime=4712.84 cores=0-7
/proc/12332/task/12338/stat : 12338 (claspmt) R 12330 12332 12306 0 -1 4202560 1103 0 0 0 470810 474 0 0 21 0 8 0 6667903 397598720 9992 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=12332/tid=12339] ppid=12330 vsize=388280 CPUtime=4716.05 cores=0-7
/proc/12332/task/12339/stat : 12339 (claspmt) R 12330 12332 12306 0 -1 4202560 1372 0 0 0 471181 424 0 0 25 0 8 0 6667903 397598720 9992 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 37688.7
Current children cumulated vsize (KiB) 388280

[startup+4782.3 s]
/proc/loadavg: 8.00 8.00 8.00 9/296 12488
/proc/meminfo: memFree=17382428/32950928 swapFree=67111528/67111528
[pid=12332] ppid=12330 vsize=388280 CPUtime=38167.6 cores=0-7
/proc/12332/stat : 12332 (claspmt) R 12330 12332 12306 0 -1 4202496 10080 0 0 0 3813006 3752 0 0 25 0 8 0 6667902 397598720 10003 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/12332/statm: 97070 10003 464 189 0 91419 0
[pid=12332/tid=12333] ppid=12330 vsize=388280 CPUtime=4769.81 cores=0-7
/proc/12332/task/12333/stat : 12333 (claspmt) R 12330 12332 12306 0 -1 4202560 1501 0 0 0 476473 508 0 0 18 0 8 0 6667903 397598720 10003 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670057 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=12332/tid=12334] ppid=12330 vsize=388280 CPUtime=4769.9 cores=0-7
/proc/12332/task/12334/stat : 12334 (claspmt) R 12330 12332 12306 0 -1 4202560 1390 0 0 0 476511 479 0 0 25 0 8 0 6667903 397598720 10003 33554432000 4194304 4964704 140736319562624 18446744073709551615 4631256 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=12332/tid=12335] ppid=12330 vsize=388280 CPUtime=4772.61 cores=0-7
/proc/12332/task/12335/stat : 12335 (claspmt) R 12330 12332 12306 0 -1 4202560 938 0 0 0 476799 462 0 0 19 0 8 0 6667903 397598720 10003 33554432000 4194304 4964704 140736319562624 18446744073709551615 4321121 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=12332/tid=12336] ppid=12330 vsize=388280 CPUtime=4773.18 cores=0-7
/proc/12332/task/12336/stat : 12336 (claspmt) R 12330 12332 12306 0 -1 4202560 1128 0 0 0 476856 462 0 0 25 0 8 0 6667903 397598720 10003 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=12332/tid=12337] ppid=12330 vsize=388280 CPUtime=4765.35 cores=0-7
/proc/12332/task/12337/stat : 12337 (claspmt) R 12330 12332 12306 0 -1 4202560 926 0 0 0 476063 472 0 0 23 0 8 0 6667903 397598720 10003 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670263 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=12332/tid=12338] ppid=12330 vsize=388280 CPUtime=4772.77 cores=0-7
/proc/12332/task/12338/stat : 12338 (claspmt) R 12330 12332 12306 0 -1 4202560 1104 0 0 0 476798 479 0 0 25 0 8 0 6667903 397598720 10003 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=12332/tid=12339] ppid=12330 vsize=388280 CPUtime=4775.94 cores=0-7
/proc/12332/task/12339/stat : 12339 (claspmt) R 12330 12332 12306 0 -1 4202560 1374 0 0 0 477164 430 0 0 25 0 8 0 6667903 397598720 10003 33554432000 4194304 4964704 140736319562624 18446744073709551615 4669472 0 0 4096 18946 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38167.6
Current children cumulated vsize (KiB) 388280

[startup+4842.3 s]
/proc/loadavg: 8.00 8.00 8.00 9/296 12490
/proc/meminfo: memFree=17382404/32950928 swapFree=67111528/67111528
[pid=12332] ppid=12330 vsize=388280 CPUtime=38646.4 cores=0-7
/proc/12332/stat : 12332 (claspmt) R 12330 12332 12306 0 -1 4202496 10088 0 0 0 3860836 3806 0 0 25 0 8 0 6667902 397598720 10011 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670250 0 0 4096 18946 18446744073709551615 0 0 17 3 0 0 0
/proc/12332/statm: 97070 10011 464 189 0 91419 0
[pid=12332/tid=12333] ppid=12330 vsize=388280 CPUtime=4829.69 cores=0-7
/proc/12332/task/12333/stat : 12333 (claspmt) R 12330 12332 12306 0 -1 4202560 1501 0 0 0 482452 517 0 0 25 0 8 0 6667903 397598720 10011 33554432000 4194304 4964704 140736319562624 18446744073709551615 4321002 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=12332/tid=12334] ppid=12330 vsize=388280 CPUtime=4829.86 cores=0-7
/proc/12332/task/12334/stat : 12334 (claspmt) R 12330 12332 12306 0 -1 4202560 1390 0 0 0 482502 484 0 0 21 0 8 0 6667903 397598720 10011 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670100 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=12332/tid=12335] ppid=12330 vsize=388280 CPUtime=4832 cores=0-7
/proc/12332/task/12335/stat : 12335 (claspmt) R 12330 12332 12306 0 -1 4202560 941 0 0 0 482730 470 0 0 25 0 8 0 6667903 397598720 10011 33554432000 4194304 4964704 140736319562624 18446744073709551615 4324838 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=12332/tid=12336] ppid=12330 vsize=388280 CPUtime=4833.09 cores=0-7
/proc/12332/task/12336/stat : 12336 (claspmt) R 12330 12332 12306 0 -1 4202560 1132 0 0 0 482840 469 0 0 25 0 8 0 6667903 397598720 10011 33554432000 4194304 4964704 140736319562624 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=12332/tid=12337] ppid=12330 vsize=388280 CPUtime=4825.29 cores=0-7
/proc/12332/task/12337/stat : 12337 (claspmt) R 12330 12332 12306 0 -1 4202560 927 0 0 0 482050 479 0 0 25 0 8 0 6667903 397598720 10011 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670100 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=12332/tid=12338] ppid=12330 vsize=388280 CPUtime=4832.72 cores=0-7
/proc/12332/task/12338/stat : 12338 (claspmt) R 12330 12332 12306 0 -1 4202560 1104 0 0 0 482786 486 0 0 25 0 8 0 6667903 397598720 10011 33554432000 4194304 4964704 140736319562624 18446744073709551615 4669844 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=12332/tid=12339] ppid=12330 vsize=388280 CPUtime=4835.83 cores=0-7
/proc/12332/task/12339/stat : 12339 (claspmt) R 12330 12332 12306 0 -1 4202560 1374 0 0 0 483146 437 0 0 25 0 8 0 6667903 397598720 10011 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670255 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 38646.4
Current children cumulated vsize (KiB) 388280

[startup+4902.3 s]
/proc/loadavg: 8.06 8.01 8.00 9/296 12492
/proc/meminfo: memFree=17382508/32950928 swapFree=67111528/67111528
[pid=12332] ppid=12330 vsize=388280 CPUtime=39125.3 cores=0-7
/proc/12332/stat : 12332 (claspmt) R 12330 12332 12306 0 -1 4202496 10092 0 0 0 3908674 3855 0 0 25 0 8 0 6667902 397598720 10015 33554432000 4194304 4964704 140736319562624 18446744073709551615 4668632 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/12332/statm: 97070 10015 464 189 0 91419 0
[pid=12332/tid=12333] ppid=12330 vsize=388280 CPUtime=4889.58 cores=0-7
/proc/12332/task/12333/stat : 12333 (claspmt) R 12330 12332 12306 0 -1 4202560 1503 0 0 0 488434 524 0 0 25 0 8 0 6667903 397598720 10015 33554432000 4194304 4964704 140736319562624 18446744073709551615 4669429 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=12332/tid=12334] ppid=12330 vsize=388280 CPUtime=4889.82 cores=0-7
/proc/12332/task/12334/stat : 12334 (claspmt) R 12330 12332 12306 0 -1 4202560 1390 0 0 0 488493 489 0 0 25 0 8 0 6667903 397598720 10015 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670269 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=12332/tid=12335] ppid=12330 vsize=388280 CPUtime=4891.78 cores=0-7
/proc/12332/task/12335/stat : 12335 (claspmt) R 12330 12332 12306 0 -1 4202560 941 0 0 0 488700 478 0 0 25 0 8 0 6667903 397598720 10015 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670179 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=12332/tid=12336] ppid=12330 vsize=388280 CPUtime=4892.99 cores=0-7
/proc/12332/task/12336/stat : 12336 (claspmt) R 12330 12332 12306 0 -1 4202560 1132 0 0 0 488825 474 0 0 22 0 8 0 6667903 397598720 10015 33554432000 4194304 4964704 140736319562624 18446744073709551615 4320890 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=12332/tid=12337] ppid=12330 vsize=388280 CPUtime=4885.22 cores=0-7
/proc/12332/task/12337/stat : 12337 (claspmt) R 12330 12332 12306 0 -1 4202560 928 0 0 0 488037 485 0 0 25 0 8 0 6667903 397598720 10015 33554432000 4194304 4964704 140736319562624 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=12332/tid=12338] ppid=12330 vsize=388280 CPUtime=4892.65 cores=0-7
/proc/12332/task/12338/stat : 12338 (claspmt) R 12330 12332 12306 0 -1 4202560 1104 0 0 0 488773 492 0 0 18 0 8 0 6667903 397598720 10015 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=12332/tid=12339] ppid=12330 vsize=388280 CPUtime=4895.73 cores=0-7
/proc/12332/task/12339/stat : 12339 (claspmt) R 12330 12332 12306 0 -1 4202560 1374 0 0 0 489130 443 0 0 18 0 8 0 6667903 397598720 10015 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 39125.3
Current children cumulated vsize (KiB) 388280

[startup+4962.3 s]
/proc/loadavg: 8.02 8.01 8.00 9/296 12494
/proc/meminfo: memFree=17382356/32950928 swapFree=67111528/67111528
[pid=12332] ppid=12330 vsize=388280 CPUtime=39604.2 cores=0-7
/proc/12332/stat : 12332 (claspmt) R 12330 12332 12306 0 -1 4202496 10098 0 0 0 3956523 3897 0 0 24 0 8 0 6667902 397598720 10021 33554432000 4194304 4964704 140736319562624 18446744073709551615 4659143 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/12332/statm: 97070 10021 464 189 0 91419 0
[pid=12332/tid=12333] ppid=12330 vsize=388280 CPUtime=4949.46 cores=0-7
/proc/12332/task/12333/stat : 12333 (claspmt) R 12330 12332 12306 0 -1 4202560 1503 0 0 0 494416 530 0 0 18 0 8 0 6667903 397598720 10021 33554432000 4194304 4964704 140736319562624 18446744073709551615 4669427 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=12332/tid=12334] ppid=12330 vsize=388280 CPUtime=4949.8 cores=0-7
/proc/12332/task/12334/stat : 12334 (claspmt) R 12330 12332 12306 0 -1 4202560 1390 0 0 0 494487 493 0 0 25 0 8 0 6667903 397598720 10021 33554432000 4194304 4964704 140736319562624 18446744073709551615 4668709 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=12332/tid=12335] ppid=12330 vsize=388280 CPUtime=4951.73 cores=0-7
/proc/12332/task/12335/stat : 12335 (claspmt) R 12330 12332 12306 0 -1 4202560 942 0 0 0 494691 482 0 0 25 0 8 0 6667903 397598720 10021 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=12332/tid=12336] ppid=12330 vsize=388280 CPUtime=4952.91 cores=0-7
/proc/12332/task/12336/stat : 12336 (claspmt) R 12330 12332 12306 0 -1 4202560 1132 0 0 0 494812 479 0 0 25 0 8 0 6667903 397598720 10021 33554432000 4194304 4964704 140736319562624 18446744073709551615 4669296 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=12332/tid=12337] ppid=12330 vsize=388280 CPUtime=4945.16 cores=0-7
/proc/12332/task/12337/stat : 12337 (claspmt) R 12330 12332 12306 0 -1 4202560 933 0 0 0 494027 489 0 0 25 0 8 0 6667903 397598720 10021 33554432000 4194304 4964704 140736319562624 18446744073709551615 4668673 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=12332/tid=12338] ppid=12330 vsize=388280 CPUtime=4952.59 cores=0-7
/proc/12332/task/12338/stat : 12338 (claspmt) R 12330 12332 12306 0 -1 4202560 1104 0 0 0 494761 498 0 0 25 0 8 0 6667903 397598720 10021 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670255 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=12332/tid=12339] ppid=12330 vsize=388280 CPUtime=4955.63 cores=0-7
/proc/12332/task/12339/stat : 12339 (claspmt) R 12330 12332 12306 0 -1 4202560 1374 0 0 0 495114 449 0 0 25 0 8 0 6667903 397598720 10021 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670215 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 39604.2
Current children cumulated vsize (KiB) 388280



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5011.9 s]
/proc/loadavg: 8.01 8.00 8.00 9/296 12495
/proc/meminfo: memFree=17382456/32950928 swapFree=67111528/67111528
[pid=12332] ppid=12330 vsize=388280 CPUtime=40000.1 cores=0-7
/proc/12332/stat : 12332 (claspmt) R 12330 12332 12306 0 -1 4202496 10098 0 0 0 3996078 3936 0 0 25 0 8 0 6667902 397598720 10021 33554432000 4194304 4964704 140736319562624 18446744073709551615 4660367 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/12332/statm: 97070 10021 464 189 0 91419 0
[pid=12332/tid=12333] ppid=12330 vsize=388280 CPUtime=4998.98 cores=0-7
/proc/12332/task/12333/stat : 12333 (claspmt) R 12330 12332 12306 0 -1 4202560 1503 0 0 0 499363 535 0 0 25 0 8 0 6667903 397598720 10021 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670263 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=12332/tid=12334] ppid=12330 vsize=388280 CPUtime=4999.38 cores=0-7
/proc/12332/task/12334/stat : 12334 (claspmt) R 12330 12332 12306 0 -1 4202560 1390 0 0 0 499440 498 0 0 25 0 8 0 6667903 397598720 10021 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670100 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=12332/tid=12335] ppid=12330 vsize=388280 CPUtime=5001.3 cores=0-7
/proc/12332/task/12335/stat : 12335 (claspmt) R 12330 12332 12306 0 -1 4202560 942 0 0 0 499643 487 0 0 25 0 8 0 6667903 397598720 10021 33554432000 4194304 4964704 140736319562624 18446744073709551615 4653161 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=12332/tid=12336] ppid=12330 vsize=388280 CPUtime=5002.43 cores=0-7
/proc/12332/task/12336/stat : 12336 (claspmt) R 12330 12332 12306 0 -1 4202560 1132 0 0 0 499760 483 0 0 18 0 8 0 6667903 397598720 10021 33554432000 4194304 4964704 140736319562624 18446744073709551615 4653154 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=12332/tid=12337] ppid=12330 vsize=388280 CPUtime=4994.73 cores=0-7
/proc/12332/task/12337/stat : 12337 (claspmt) R 12330 12332 12306 0 -1 4202560 933 0 0 0 498977 496 0 0 21 0 8 0 6667903 397598720 10021 33554432000 4194304 4964704 140736319562624 18446744073709551615 4364587 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=12332/tid=12338] ppid=12330 vsize=388280 CPUtime=5002.15 cores=0-7
/proc/12332/task/12338/stat : 12338 (claspmt) R 12330 12332 12306 0 -1 4202560 1104 0 0 0 499713 502 0 0 25 0 8 0 6667903 397598720 10021 33554432000 4194304 4964704 140736319562624 18446744073709551615 4668598 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=12332/tid=12339] ppid=12330 vsize=388280 CPUtime=5005.17 cores=0-7
/proc/12332/task/12339/stat : 12339 (claspmt) R 12330 12332 12306 0 -1 4202560 1374 0 0 0 500062 455 0 0 18 0 8 0 6667903 397598720 10021 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 40000.1
Current children cumulated vsize (KiB) 388280

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5011.9 s]
/proc/loadavg: 8.01 8.00 8.00 9/296 12495
/proc/meminfo: memFree=17382456/32950928 swapFree=67111528/67111528
[pid=12332] ppid=12330 vsize=388280 CPUtime=40000.1 cores=0-7
/proc/12332/stat : 12332 (claspmt) R 12330 12332 12306 0 -1 4202496 10098 0 0 0 3996078 3936 0 0 25 0 8 0 6667902 397598720 10021 33554432000 4194304 4964704 140736319562624 18446744073709551615 4660367 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/12332/statm: 97070 10021 464 189 0 91419 0
[pid=12332/tid=12333] ppid=12330 vsize=388280 CPUtime=4998.98 cores=0-7
/proc/12332/task/12333/stat : 12333 (claspmt) R 12330 12332 12306 0 -1 4202560 1503 0 0 0 499363 535 0 0 25 0 8 0 6667903 397598720 10021 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670263 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=12332/tid=12334] ppid=12330 vsize=388280 CPUtime=4999.38 cores=0-7
/proc/12332/task/12334/stat : 12334 (claspmt) R 12330 12332 12306 0 -1 4202560 1390 0 0 0 499440 498 0 0 25 0 8 0 6667903 397598720 10021 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670100 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=12332/tid=12335] ppid=12330 vsize=388280 CPUtime=5001.3 cores=0-7
/proc/12332/task/12335/stat : 12335 (claspmt) R 12330 12332 12306 0 -1 4202560 942 0 0 0 499643 487 0 0 25 0 8 0 6667903 397598720 10021 33554432000 4194304 4964704 140736319562624 18446744073709551615 4653161 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=12332/tid=12336] ppid=12330 vsize=388280 CPUtime=5002.43 cores=0-7
/proc/12332/task/12336/stat : 12336 (claspmt) R 12330 12332 12306 0 -1 4202560 1132 0 0 0 499760 483 0 0 18 0 8 0 6667903 397598720 10021 33554432000 4194304 4964704 140736319562624 18446744073709551615 4653154 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=12332/tid=12337] ppid=12330 vsize=388280 CPUtime=4994.73 cores=0-7
/proc/12332/task/12337/stat : 12337 (claspmt) R 12330 12332 12306 0 -1 4202560 933 0 0 0 498977 496 0 0 21 0 8 0 6667903 397598720 10021 33554432000 4194304 4964704 140736319562624 18446744073709551615 4364587 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=12332/tid=12338] ppid=12330 vsize=388280 CPUtime=5002.15 cores=0-7
/proc/12332/task/12338/stat : 12338 (claspmt) R 12330 12332 12306 0 -1 4202560 1104 0 0 0 499713 502 0 0 25 0 8 0 6667903 397598720 10021 33554432000 4194304 4964704 140736319562624 18446744073709551615 4668598 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=12332/tid=12339] ppid=12330 vsize=388280 CPUtime=5005.17 cores=0-7
/proc/12332/task/12339/stat : 12339 (claspmt) R 12330 12332 12306 0 -1 4202560 1374 0 0 0 500062 455 0 0 18 0 8 0 6667903 397598720 10021 33554432000 4194304 4964704 140736319562624 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 40000.1
Current children cumulated vsize (KiB) 388280

Child status: 0
Real time (s): 5011.92
CPU time (s): 40000.2
CPU user time (s): 39960.8
CPU system time (s): 39.363
CPU usage (%): 798.101
Max. virtual memory (cumulated for all children) (KiB): 388280

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39960.8
system time used= 39.363
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 10119
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 6592560
involuntary context switches= 188446

runsolver used 14.3438 second user time and 31.2343 second system time

The end

Launcher Data 

Begin job on node112 at 2011-05-03 04:43:05
IDJOB=3322580
IDBENCH=84021
IDSOLVER=1711
FILE ID=node112/3322580-1304390585
RUNJOBID= node112-1304390585-12320
PBS_JOBID= 13181835
Free space on /tmp= 73592 MiB

SOLVER NAME= claspmt 2.0-R4095
BENCH NAME= SAT03/handmade/stanion/hwb/hwb-n32-02-S1478512323.shuffled-as.sat03-1638.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3322580-1304390585/watcher-3322580-1304390585 -o /tmp/evaluation-result-3322580-1304390585/solver-3322580-1304390585 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3322580-1304390585.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 1d86c7f9cd05c5f2ab070520d802c8e1
RANDOM SEED=210262347

node112.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5180.45
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.847
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      17423988 kB
Buffers:        128292 kB
Cached:        1449680 kB
SwapCached:          0 kB
Active:       14110492 kB
Inactive:      1221756 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      17423988 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:             940 kB
Writeback:           0 kB
AnonPages:    13753692 kB
Mapped:          20500 kB
Slab:            88580 kB
PageTables:      41984 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 34983416 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73592 MiB
End job on node112 at 2011-05-03 06:06:39