Trace number 3322402

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095? (TO) 35695.5 5100.3

General information on the benchmark

Namecrafted/mosoi/289/
289-unsat-21x14.cnf
MD5SUMf76302f2442dfc7cabc2315e61e55fea
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables1176
Number of clauses78498
Sum of the clauses size310464
Maximum clause length4
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 21764
Number of clauses of size 30
Number of clauses of size 476734
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...383492/instance-3322402-1304383492.cnf
1.69/1.70	c Solving...
35695.19/5100.01	
35695.19/5100.01	*** INTERRUPTED! ***
35695.19/5100.01	*** Shutting down threads...
35695.42/5100.29	s UNKNOWN
35695.42/5100.29	
35695.42/5100.29	c INTERRUPTED   : 1
35695.42/5100.29	c Models        : 0+    
35695.42/5100.29	c Time          : 5100.012s (Solving: 5098.31s 1st Model: 0.00s Unsat: 0.00s)
35695.42/5100.29	c CPU Time      : 35695.240s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3322402-1304383492/watcher-3322402-1304383492 -o /tmp/evaluation-result-3322402-1304383492/solver-3322402-1304383492 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3322402-1304383492.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.36 8.91 8.96 2/232 30378
/proc/meminfo: memFree=15412800/32951124 swapFree=67110188/67111528
[pid=30378] ppid=30376 vsize=21952 CPUtime=0 cores=0-7
/proc/30378/stat : 30378 (claspmt) R 30376 30378 29040 0 -1 4202496 801 0 0 0 0 0 0 0 22 0 1 0 549949913 22478848 721 33554432000 4194304 4964704 140736331414656 18446744073709551615 222921437893 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/30378/statm: 5488 721 386 189 0 362 0

[startup+0.082475 s]
/proc/loadavg: 8.36 8.91 8.96 2/232 30378
/proc/meminfo: memFree=15412800/32951124 swapFree=67110188/67111528
[pid=30378] ppid=30376 vsize=27856 CPUtime=0.07 cores=0-7
/proc/30378/stat : 30378 (claspmt) R 30376 30378 29040 0 -1 4202496 2538 0 0 0 7 0 0 0 22 0 1 0 549949913 28524544 2156 33554432000 4194304 4964704 140736331414656 18446744073709551615 4607641 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/30378/statm: 6964 2156 399 189 0 1838 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 27856

[startup+0.100473 s]
/proc/loadavg: 8.36 8.91 8.96 2/232 30378
/proc/meminfo: memFree=15412800/32951124 swapFree=67110188/67111528
[pid=30378] ppid=30376 vsize=27856 CPUtime=0.09 cores=0-7
/proc/30378/stat : 30378 (claspmt) R 30376 30378 29040 0 -1 4202496 2538 0 0 0 9 0 0 0 22 0 1 0 549949913 28524544 2156 33554432000 4194304 4964704 140736331414656 18446744073709551615 4607585 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/30378/statm: 6964 2156 399 189 0 1838 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 27856

[startup+0.300433 s]
/proc/loadavg: 8.36 8.91 8.96 2/232 30378
/proc/meminfo: memFree=15412800/32951124 swapFree=67110188/67111528
[pid=30378] ppid=30376 vsize=28252 CPUtime=0.29 cores=0-7
/proc/30378/stat : 30378 (claspmt) R 30376 30378 29040 0 -1 4202496 2629 0 0 0 29 0 0 0 23 0 1 0 549949913 28930048 2247 33554432000 4194304 4964704 140736331414656 18446744073709551615 4600434 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/30378/statm: 7063 2247 400 189 0 1937 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 28252

[startup+0.700361 s]
/proc/loadavg: 8.36 8.91 8.96 2/232 30378
/proc/meminfo: memFree=15412800/32951124 swapFree=67110188/67111528
[pid=30378] ppid=30376 vsize=31156 CPUtime=0.69 cores=0-7
/proc/30378/stat : 30378 (claspmt) R 30376 30378 29040 0 -1 4202496 3384 0 0 0 69 0 0 0 25 0 1 0 549949913 31903744 3002 33554432000 4194304 4964704 140736331414656 18446744073709551615 4616033 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/30378/statm: 7789 3002 400 189 0 2663 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 31156

[startup+1.50022 s]
/proc/loadavg: 8.36 8.91 8.96 3/233 30379
/proc/meminfo: memFree=15400640/32951124 swapFree=67110188/67111528
[pid=30378] ppid=30376 vsize=33928 CPUtime=1.49 cores=0-7
/proc/30378/stat : 30378 (claspmt) R 30376 30378 29040 0 -1 4202496 4068 0 0 0 149 0 0 0 25 0 1 0 549949913 34742272 3686 33554432000 4194304 4964704 140736331414656 18446744073709551615 4607590 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/30378/statm: 8482 3686 400 189 0 3356 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 33928

[startup+3.10089 s]
/proc/loadavg: 8.36 8.91 8.96 10/240 30386
/proc/meminfo: memFree=15343420/32951124 swapFree=67110188/67111528
[pid=30378] ppid=30376 vsize=536936 CPUtime=10.93 cores=0-7
/proc/30378/stat : 30378 (claspmt) R 30376 30378 29040 0 -1 4202496 27288 0 0 0 1080 13 0 0 18 0 8 0 549949913 549822464 26731 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670258 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/30378/statm: 134234 26731 464 189 0 128583 0
[pid=30378/tid=30380] ppid=30376 vsize=536936 CPUtime=1.2 cores=0-7
/proc/30378/task/30380/stat : 30380 (claspmt) R 30376 30378 29040 0 -1 4202560 2705 0 0 0 119 1 0 0 18 0 8 0 549950083 549822464 26731 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=30378/tid=30381] ppid=30376 vsize=536936 CPUtime=1.19 cores=0-7
/proc/30378/task/30381/stat : 30381 (claspmt) R 30376 30378 29040 0 -1 4202560 3069 0 0 0 117 2 0 0 18 0 8 0 549950083 549822464 26731 33554432000 4194304 4964704 140736331414656 18446744073709551615 4320984 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=30378/tid=30382] ppid=30376 vsize=536936 CPUtime=0.95 cores=0-7
/proc/30378/task/30382/stat : 30382 (claspmt) R 30376 30378 29040 0 -1 4202560 2852 0 0 0 93 2 0 0 18 0 8 0 549950083 549822464 26731 33554432000 4194304 4964704 140736331414656 18446744073709551615 4627951 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=30378/tid=30383] ppid=30376 vsize=536936 CPUtime=1.26 cores=0-7
/proc/30378/task/30383/stat : 30383 (claspmt) R 30376 30378 29040 0 -1 4202560 2717 0 0 0 125 1 0 0 18 0 8 0 549950083 549822464 26731 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670574 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30378/tid=30384] ppid=30376 vsize=536936 CPUtime=1.08 cores=0-7
/proc/30378/task/30384/stat : 30384 (claspmt) R 30376 30378 29040 0 -1 4202560 2396 0 0 0 107 1 0 0 18 0 8 0 549950084 549822464 26731 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670193 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30378/tid=30385] ppid=30376 vsize=536936 CPUtime=1.06 cores=0-7
/proc/30378/task/30385/stat : 30385 (claspmt) R 30376 30378 29040 0 -1 4202560 2728 0 0 0 105 1 0 0 19 0 8 0 549950084 549822464 26737 33554432000 4194304 4964704 140736331414656 18446744073709551615 4361097 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30378/tid=30386] ppid=30376 vsize=536936 CPUtime=1.3 cores=0-7
/proc/30378/task/30386/stat : 30386 (claspmt) R 30376 30378 29040 0 -1 4202560 3456 0 0 0 129 1 0 0 18 0 8 0 549950084 549822464 26751 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 10.93
Current children cumulated vsize (KiB) 536936

[startup+6.30131 s]
/proc/loadavg: 8.41 8.92 8.96 10/240 30386
/proc/meminfo: memFree=15270252/32951124 swapFree=67110188/67111528
[pid=30378] ppid=30376 vsize=543612 CPUtime=33.28 cores=0-7
/proc/30378/stat : 30378 (claspmt) R 30376 30378 29040 0 -1 4202496 39342 0 0 0 3310 18 0 0 25 0 8 0 549949913 556658688 38763 33554432000 4194304 4964704 140736331414656 18446744073709551615 4654527 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/30378/statm: 135903 38763 466 189 0 130252 0
[pid=30378/tid=30380] ppid=30376 vsize=543612 CPUtime=4.41 cores=0-7
/proc/30378/task/30380/stat : 30380 (claspmt) R 30376 30378 29040 0 -1 4202560 3626 0 0 0 439 2 0 0 25 0 8 0 549950083 556658688 38763 33554432000 4194304 4964704 140736331414656 18446744073709551615 4668488 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=30378/tid=30381] ppid=30376 vsize=543612 CPUtime=3.43 cores=0-7
/proc/30378/task/30381/stat : 30381 (claspmt) R 30376 30378 29040 0 -1 4202560 4874 0 0 0 341 2 0 0 22 0 8 0 549950083 556658688 38763 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670357 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=30378/tid=30382] ppid=30376 vsize=543612 CPUtime=3.65 cores=0-7
/proc/30378/task/30382/stat : 30382 (claspmt) R 30376 30378 29040 0 -1 4202560 4547 0 0 0 363 2 0 0 24 0 8 0 549950083 556658688 38763 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670243 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30378/tid=30383] ppid=30376 vsize=543612 CPUtime=4.22 cores=0-7
/proc/30378/task/30383/stat : 30383 (claspmt) R 30376 30378 29040 0 -1 4202560 4632 0 0 0 421 1 0 0 25 0 8 0 549950083 556658688 38763 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30378/tid=30384] ppid=30376 vsize=543612 CPUtime=3.59 cores=0-7
/proc/30378/task/30384/stat : 30384 (claspmt) R 30376 30378 29040 0 -1 4202560 3257 0 0 0 358 1 0 0 23 0 8 0 549950084 556658688 38763 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30378/tid=30385] ppid=30376 vsize=543612 CPUtime=3.4 cores=0-7
/proc/30378/task/30385/stat : 30385 (claspmt) R 30376 30378 29040 0 -1 4202560 3802 0 0 0 338 2 0 0 21 0 8 0 549950084 556658688 38763 33554432000 4194304 4964704 140736331414656 18446744073709551615 4668488 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=30378/tid=30386] ppid=30376 vsize=543612 CPUtime=4.5 cores=0-7
/proc/30378/task/30386/stat : 30386 (claspmt) R 30376 30378 29040 0 -1 4202560 5968 0 0 0 448 2 0 0 25 0 8 0 549950084 556658688 38763 33554432000 4194304 4964704 140736331414656 18446744073709551615 4366624 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 33.28
Current children cumulated vsize (KiB) 543612

[startup+12.7011 s]
/proc/loadavg: 8.46 8.92 8.96 10/240 30386
/proc/meminfo: memFree=15219080/32951124 swapFree=67110188/67111528
[pid=30378] ppid=30376 vsize=548180 CPUtime=77.97 cores=0-7
/proc/30378/stat : 30378 (claspmt) R 30376 30378 29040 0 -1 4202496 50161 0 0 0 7772 25 0 0 25 0 8 0 549949913 561336320 49582 33554432000 4194304 4964704 140736331414656 18446744073709551615 4321079 0 0 4096 18946 18446744073709551615 0 0 17 4 0 0 0
/proc/30378/statm: 137045 49582 466 189 0 131394 0
[pid=30378/tid=30380] ppid=30376 vsize=548180 CPUtime=9.23 cores=0-7
/proc/30378/task/30380/stat : 30380 (claspmt) R 30376 30378 29040 0 -1 4202560 4537 0 0 0 920 3 0 0 23 0 8 0 549950083 561336320 49582 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30378/tid=30381] ppid=30376 vsize=548180 CPUtime=9.83 cores=0-7
/proc/30378/task/30381/stat : 30381 (claspmt) R 30376 30378 29040 0 -1 4202560 6608 0 0 0 980 3 0 0 25 0 8 0 549950083 561336320 49582 33554432000 4194304 4964704 140736331414656 18446744073709551615 4627293 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=30378/tid=30382] ppid=30376 vsize=548180 CPUtime=10.01 cores=0-7
/proc/30378/task/30382/stat : 30382 (claspmt) R 30376 30378 29040 0 -1 4202560 6557 0 0 0 998 3 0 0 25 0 8 0 549950083 561336320 49582 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=30378/tid=30383] ppid=30376 vsize=548180 CPUtime=10.51 cores=0-7
/proc/30378/task/30383/stat : 30383 (claspmt) R 30376 30378 29040 0 -1 4202560 5667 0 0 0 1049 2 0 0 25 0 8 0 549950083 561336320 49582 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670243 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=30378/tid=30384] ppid=30376 vsize=548180 CPUtime=8.39 cores=0-7
/proc/30378/task/30384/stat : 30384 (claspmt) R 30376 30378 29040 0 -1 4202560 4124 0 0 0 837 2 0 0 25 0 8 0 549950084 561336320 49582 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30378/tid=30385] ppid=30376 vsize=548180 CPUtime=7.77 cores=0-7
/proc/30378/task/30385/stat : 30385 (claspmt) R 30376 30378 29040 0 -1 4202560 5653 0 0 0 774 3 0 0 25 0 8 0 549950084 561336320 49582 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30378/tid=30386] ppid=30376 vsize=548180 CPUtime=10.89 cores=0-7
/proc/30378/task/30386/stat : 30386 (claspmt) R 30376 30378 29040 0 -1 4202560 6815 0 0 0 1087 2 0 0 25 0 8 0 549950084 561336320 49582 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670419 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 77.97
Current children cumulated vsize (KiB) 548180

[startup+25.5008 s]
/proc/loadavg: 8.58 8.92 8.96 10/240 30386
/proc/meminfo: memFree=15155260/32951124 swapFree=67110188/67111528
[pid=30378] ppid=30376 vsize=556560 CPUtime=167.35 cores=0-7
/proc/30378/stat : 30378 (claspmt) R 30376 30378 29040 0 -1 4202496 66435 0 0 0 16700 35 0 0 25 0 8 0 549949913 569917440 65585 33554432000 4194304 4964704 140736331414656 18446744073709551615 4321121 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/30378/statm: 139140 65585 466 189 0 133489 0
[pid=30378/tid=30380] ppid=30376 vsize=556560 CPUtime=18.67 cores=0-7
/proc/30378/task/30380/stat : 30380 (claspmt) R 30376 30378 29040 0 -1 4202560 6173 0 0 0 1864 3 0 0 18 0 8 0 549950083 569917440 65586 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=30378/tid=30381] ppid=30376 vsize=556560 CPUtime=20.66 cores=0-7
/proc/30378/task/30381/stat : 30381 (claspmt) R 30376 30378 29040 0 -1 4202560 9766 0 0 0 2060 6 0 0 25 0 8 0 549950083 569917440 65586 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30378/tid=30382] ppid=30376 vsize=556560 CPUtime=22.8 cores=0-7
/proc/30378/task/30382/stat : 30382 (claspmt) R 30376 30378 29040 0 -1 4202560 7811 0 0 0 2276 4 0 0 25 0 8 0 549950083 569917440 65586 33554432000 4194304 4964704 140736331414656 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30378/tid=30383] ppid=30376 vsize=556560 CPUtime=23.21 cores=0-7
/proc/30378/task/30383/stat : 30383 (claspmt) R 30376 30378 29040 0 -1 4202560 8080 0 0 0 2317 4 0 0 25 0 8 0 549950083 569917440 65586 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670419 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=30378/tid=30384] ppid=30376 vsize=556560 CPUtime=20.65 cores=0-7
/proc/30378/task/30384/stat : 30384 (claspmt) R 30376 30378 29040 0 -1 4202560 5762 0 0 0 2062 3 0 0 18 0 8 0 549950084 569917440 65586 33554432000 4194304 4964704 140736331414656 18446744073709551615 222921438241 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=30378/tid=30385] ppid=30376 vsize=556560 CPUtime=16.33 cores=0-7
/proc/30378/task/30385/stat : 30385 (claspmt) R 30376 30378 29040 0 -1 4202560 7940 0 0 0 1629 4 0 0 25 0 8 0 549950084 569917440 65586 33554432000 4194304 4964704 140736331414656 18446744073709551615 222921820343 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=30378/tid=30386] ppid=30376 vsize=556560 CPUtime=21.04 cores=0-7
/proc/30378/task/30386/stat : 30386 (claspmt) R 30376 30378 29040 0 -1 4202560 8996 0 0 0 2101 3 0 0 25 0 8 0 549950084 569917440 65586 33554432000 4194304 4964704 140736331414656 18446744073709551615 4653154 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 167.35
Current children cumulated vsize (KiB) 556560

[startup+51.1011 s]
/proc/loadavg: 8.72 8.92 8.96 10/240 30387
/proc/meminfo: memFree=15074956/32951124 swapFree=67110188/67111528
[pid=30378] ppid=30376 vsize=563368 CPUtime=346.35 cores=0-7
/proc/30378/stat : 30378 (claspmt) R 30376 30378 29040 0 -1 4202496 84789 0 0 0 34586 49 0 0 25 0 8 0 549949913 576888832 83935 33554432000 4194304 4964704 140736331414656 18446744073709551615 4325338 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/30378/statm: 140842 83935 466 189 0 135191 0
[pid=30378/tid=30380] ppid=30376 vsize=563368 CPUtime=35.49 cores=0-7
/proc/30378/task/30380/stat : 30380 (claspmt) R 30376 30378 29040 0 -1 4202560 7686 0 0 0 3545 4 0 0 25 0 8 0 549950083 576888832 83935 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670357 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30378/tid=30381] ppid=30376 vsize=563368 CPUtime=44.1 cores=0-7
/proc/30378/task/30381/stat : 30381 (claspmt) R 30376 30378 29040 0 -1 4202560 13454 0 0 0 4401 9 0 0 25 0 8 0 549950083 576888832 83935 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670233 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=30378/tid=30382] ppid=30376 vsize=563368 CPUtime=36.87 cores=0-7
/proc/30378/task/30382/stat : 30382 (claspmt) R 30376 30378 29040 0 -1 4202560 8365 0 0 0 3682 5 0 0 25 0 8 0 549950083 576888832 83935 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670466 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=30378/tid=30383] ppid=30376 vsize=563368 CPUtime=48.81 cores=0-7
/proc/30378/task/30383/stat : 30383 (claspmt) R 30376 30378 29040 0 -1 4202560 9497 0 0 0 4877 4 0 0 25 0 8 0 549950083 576888832 83935 33554432000 4194304 4964704 140736331414656 18446744073709551615 4660034 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=30378/tid=30384] ppid=30376 vsize=563368 CPUtime=44.79 cores=0-7
/proc/30378/task/30384/stat : 30384 (claspmt) R 30376 30378 29040 0 -1 4202560 8278 0 0 0 4472 7 0 0 25 0 8 0 549950084 576888832 83935 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=30378/tid=30385] ppid=30376 vsize=563368 CPUtime=40.1 cores=0-7
/proc/30378/task/30385/stat : 30385 (claspmt) R 30376 30378 29040 0 -1 4202560 10095 0 0 0 4004 6 0 0 25 0 8 0 549950084 576888832 83935 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670255 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30378/tid=30386] ppid=30376 vsize=563368 CPUtime=46.64 cores=0-7
/proc/30378/task/30386/stat : 30386 (claspmt) R 30376 30378 29040 0 -1 4202560 13479 0 0 0 4658 6 0 0 25 0 8 0 549950084 576888832 83935 33554432000 4194304 4964704 140736331414656 18446744073709551615 4320958 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 346.35
Current children cumulated vsize (KiB) 563368

[startup+102.309 s]
/proc/loadavg: 8.88 8.93 8.96 10/240 30388
/proc/meminfo: memFree=14926372/32951124 swapFree=67110188/67111528
[pid=30378] ppid=30376 vsize=710592 CPUtime=704.2 cores=0-7
/proc/30378/stat : 30378 (claspmt) R 30376 30378 29040 0 -1 4202496 120398 0 0 0 70339 81 0 0 25 0 8 0 549949913 727646208 119138 33554432000 4194304 4964704 140736331414656 18446744073709551615 4362211 0 0 4096 18946 18446744073709551615 0 0 17 7 0 0 0
/proc/30378/statm: 177648 119138 466 189 0 171997 0
[pid=30378/tid=30380] ppid=30376 vsize=710592 CPUtime=70.26 cores=0-7
/proc/30378/task/30380/stat : 30380 (claspmt) R 30376 30378 29040 0 -1 4202560 11028 0 0 0 7019 7 0 0 25 0 8 0 549950083 727646208 119138 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30378/tid=30381] ppid=30376 vsize=710592 CPUtime=94.47 cores=0-7
/proc/30378/task/30381/stat : 30381 (claspmt) R 30376 30378 29040 0 -1 4202560 18243 0 0 0 9433 14 0 0 25 0 8 0 549950083 727646208 119138 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=30378/tid=30382] ppid=30376 vsize=710592 CPUtime=76.37 cores=0-7
/proc/30378/task/30382/stat : 30382 (claspmt) R 30376 30378 29040 0 -1 4202560 11015 0 0 0 7628 9 0 0 25 0 8 0 549950083 727646208 119138 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0

################
# More data... #
################

[pid=30378/tid=30381] ppid=30376 vsize=2073340 CPUtime=4442.55 cores=0-7
/proc/30378/task/30381/stat : 30381 (claspmt) R 30376 30378 29040 0 -1 4202560 59699 0 0 0 444183 72 0 0 25 0 8 0 549950083 2123100160 427220 33554432000 4194304 4964704 140736331414656 18446744073709551615 222921882094 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=30378/tid=30382] ppid=30376 vsize=2073340 CPUtime=3898.97 cores=0-7
/proc/30378/task/30382/stat : 30382 (claspmt) R 30376 30378 29040 0 -1 4202560 53863 0 0 0 389818 79 0 0 25 0 8 0 549950083 2123100160 427220 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670243 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30378/tid=30383] ppid=30376 vsize=2073340 CPUtime=4530.17 cores=0-7
/proc/30378/task/30383/stat : 30383 (claspmt) R 30376 30378 29040 0 -1 4202560 37109 0 0 0 452979 38 0 0 25 0 8 0 549950083 2123100160 427220 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30378/tid=30384] ppid=30376 vsize=2073340 CPUtime=4580.46 cores=0-7
/proc/30378/task/30384/stat : 30384 (claspmt) R 30376 30378 29040 0 -1 4202560 47160 0 0 0 457990 56 0 0 25 0 8 0 549950084 2123100160 427220 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670440 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30378/tid=30385] ppid=30376 vsize=2073340 CPUtime=3919.13 cores=0-7
/proc/30378/task/30385/stat : 30385 (claspmt) R 30376 30378 29040 0 -1 4202560 51208 0 0 0 391834 79 0 0 25 0 8 0 549950084 2123100160 427220 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=30378/tid=30386] ppid=30376 vsize=2073340 CPUtime=4091.79 cores=0-7
/proc/30378/task/30386/stat : 30386 (claspmt) R 30376 30378 29040 0 -1 4202560 81225 0 0 0 409079 100 0 0 25 0 8 0 549950084 2123100160 427220 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 33456.4
Current children cumulated vsize (KiB) 2073340

[startup+4842.3 s]
/proc/loadavg: 9.15 9.16 9.05 10/240 31037
/proc/meminfo: memFree=12936820/32951124 swapFree=67110188/67111528
[pid=30378] ppid=30376 vsize=2073340 CPUtime=33875.9 cores=0-7
/proc/30378/stat : 30378 (claspmt) R 30376 30378 29040 0 -1 4202496 437228 0 0 0 3387024 568 0 0 25 0 8 0 549949913 2123100160 427496 33554432000 4194304 4964704 140736331414656 18446744073709551615 4320951 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/30378/statm: 518335 427496 466 189 0 512684 0
[pid=30378/tid=30380] ppid=30376 vsize=2073340 CPUtime=4169.46 cores=0-7
/proc/30378/task/30380/stat : 30380 (claspmt) R 30376 30378 29040 0 -1 4202560 43253 0 0 0 416884 62 0 0 25 0 8 0 549950083 2123100160 427496 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=30378/tid=30381] ppid=30376 vsize=2073340 CPUtime=4492.07 cores=0-7
/proc/30378/task/30381/stat : 30381 (claspmt) R 30376 30378 29040 0 -1 4202560 59940 0 0 0 449135 72 0 0 25 0 8 0 549950083 2123100160 427496 33554432000 4194304 4964704 140736331414656 18446744073709551615 4320897 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30378/tid=30382] ppid=30376 vsize=2073340 CPUtime=3957.35 cores=0-7
/proc/30378/task/30382/stat : 30382 (claspmt) R 30376 30378 29040 0 -1 4202560 53865 0 0 0 395656 79 0 0 24 0 8 0 549950083 2123100160 427496 33554432000 4194304 4964704 140736331414656 18446744073709551615 4642509 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30378/tid=30383] ppid=30376 vsize=2073340 CPUtime=4581.59 cores=0-7
/proc/30378/task/30383/stat : 30383 (claspmt) R 30376 30378 29040 0 -1 4202560 37109 0 0 0 458121 38 0 0 25 0 8 0 549950083 2123100160 427496 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670215 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30378/tid=30384] ppid=30376 vsize=2073340 CPUtime=4634.55 cores=0-7
/proc/30378/task/30384/stat : 30384 (claspmt) R 30376 30378 29040 0 -1 4202560 47160 0 0 0 463399 56 0 0 25 0 8 0 549950084 2123100160 427496 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30378/tid=30385] ppid=30376 vsize=2073340 CPUtime=3973.96 cores=0-7
/proc/30378/task/30385/stat : 30385 (claspmt) R 30376 30378 29040 0 -1 4202560 51208 0 0 0 397316 80 0 0 25 0 8 0 549950084 2123100160 427496 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=30378/tid=30386] ppid=30376 vsize=2073340 CPUtime=4151.79 cores=0-7
/proc/30378/task/30386/stat : 30386 (claspmt) R 30376 30378 29040 0 -1 4202560 81226 0 0 0 415079 100 0 0 25 0 8 0 549950084 2123100160 427496 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 33875.9
Current children cumulated vsize (KiB) 2073340

[startup+4902.3 s]
/proc/loadavg: 9.05 9.12 9.05 10/240 31038
/proc/meminfo: memFree=12926376/32951124 swapFree=67110188/67111528
[pid=30378] ppid=30376 vsize=2073340 CPUtime=34295.4 cores=0-7
/proc/30378/stat : 30378 (claspmt) R 30376 30378 29040 0 -1 4202496 437524 0 0 0 3428973 571 0 0 25 0 8 0 549949913 2123100160 427654 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670323 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/30378/statm: 518335 427654 466 189 0 512684 0
[pid=30378/tid=30380] ppid=30376 vsize=2073340 CPUtime=4228.82 cores=0-7
/proc/30378/task/30380/stat : 30380 (claspmt) R 30376 30378 29040 0 -1 4202560 43423 0 0 0 422819 63 0 0 25 0 8 0 549950083 2123100160 427654 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30378/tid=30381] ppid=30376 vsize=2073340 CPUtime=4552.03 cores=0-7
/proc/30378/task/30381/stat : 30381 (claspmt) R 30376 30378 29040 0 -1 4202560 60060 0 0 0 455131 72 0 0 25 0 8 0 549950083 2123100160 427654 33554432000 4194304 4964704 140736331414656 18446744073709551615 4660050 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=30378/tid=30382] ppid=30376 vsize=2073340 CPUtime=3988.06 cores=0-7
/proc/30378/task/30382/stat : 30382 (claspmt) R 30376 30378 29040 0 -1 4202560 53866 0 0 0 398726 80 0 0 25 0 8 0 549950083 2123100160 427654 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30378/tid=30383] ppid=30376 vsize=2073340 CPUtime=4629.4 cores=0-7
/proc/30378/task/30383/stat : 30383 (claspmt) R 30376 30378 29040 0 -1 4202560 37109 0 0 0 462901 39 0 0 25 0 8 0 549950083 2123100160 427654 33554432000 4194304 4964704 140736331414656 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30378/tid=30384] ppid=30376 vsize=2073340 CPUtime=4694.55 cores=0-7
/proc/30378/task/30384/stat : 30384 (claspmt) R 30376 30378 29040 0 -1 4202560 47161 0 0 0 469399 56 0 0 25 0 8 0 549950084 2123100160 427654 33554432000 4194304 4964704 140736331414656 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30378/tid=30385] ppid=30376 vsize=2073340 CPUtime=4033.95 cores=0-7
/proc/30378/task/30385/stat : 30385 (claspmt) R 30376 30378 29040 0 -1 4202560 51208 0 0 0 403315 80 0 0 25 0 8 0 549950084 2123100160 427654 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670233 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=30378/tid=30386] ppid=30376 vsize=2073340 CPUtime=4193.49 cores=0-7
/proc/30378/task/30386/stat : 30386 (claspmt) R 30376 30378 29040 0 -1 4202560 81226 0 0 0 419249 100 0 0 25 0 8 0 549950084 2123100160 427654 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 34295.4
Current children cumulated vsize (KiB) 2073340

[startup+4962.3 s]
/proc/loadavg: 9.02 9.10 9.04 10/240 31040
/proc/meminfo: memFree=12916172/32951124 swapFree=67110188/67111528
[pid=30378] ppid=30376 vsize=2073340 CPUtime=34715 cores=0-7
/proc/30378/stat : 30378 (claspmt) R 30376 30378 29040 0 -1 4202496 437839 0 0 0 3470924 573 0 0 25 0 8 0 549949913 2123100160 427792 33554432000 4194304 4964704 140736331414656 18446744073709551615 4320909 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/30378/statm: 518335 427792 466 189 0 512684 0
[pid=30378/tid=30380] ppid=30376 vsize=2073340 CPUtime=4273.18 cores=0-7
/proc/30378/task/30380/stat : 30380 (claspmt) R 30376 30378 29040 0 -1 4202560 43594 0 0 0 427255 63 0 0 25 0 8 0 549950083 2123100160 427792 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30378/tid=30381] ppid=30376 vsize=2073340 CPUtime=4605.33 cores=0-7
/proc/30378/task/30381/stat : 30381 (claspmt) R 30376 30378 29040 0 -1 4202560 60130 0 0 0 460461 72 0 0 25 0 8 0 549950083 2123100160 427792 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670215 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=30378/tid=30382] ppid=30376 vsize=2073340 CPUtime=4045.12 cores=0-7
/proc/30378/task/30382/stat : 30382 (claspmt) R 30376 30378 29040 0 -1 4202560 53866 0 0 0 404432 80 0 0 25 0 8 0 549950083 2123100160 427792 33554432000 4194304 4964704 140736331414656 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=30378/tid=30383] ppid=30376 vsize=2073340 CPUtime=4685.26 cores=0-7
/proc/30378/task/30383/stat : 30383 (claspmt) R 30376 30378 29040 0 -1 4202560 37110 0 0 0 468487 39 0 0 25 0 8 0 549950083 2123100160 427792 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30378/tid=30384] ppid=30376 vsize=2073340 CPUtime=4751.36 cores=0-7
/proc/30378/task/30384/stat : 30384 (claspmt) R 30376 30378 29040 0 -1 4202560 47162 0 0 0 475079 57 0 0 25 0 8 0 549950084 2123100160 427792 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30378/tid=30385] ppid=30376 vsize=2073340 CPUtime=4091.98 cores=0-7
/proc/30378/task/30385/stat : 30385 (claspmt) R 30376 30378 29040 0 -1 4202560 51209 0 0 0 409118 80 0 0 25 0 8 0 549950084 2123100160 427792 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=30378/tid=30386] ppid=30376 vsize=2073340 CPUtime=4244.87 cores=0-7
/proc/30378/task/30386/stat : 30386 (claspmt) R 30376 30378 29040 0 -1 4202560 81297 0 0 0 424386 101 0 0 25 0 8 0 549950084 2123100160 427792 33554432000 4194304 4964704 140736331414656 18446744073709551615 222921882094 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 34715
Current children cumulated vsize (KiB) 2073340

[startup+5022.3 s]
/proc/loadavg: 9.00 9.08 9.03 10/240 31042
/proc/meminfo: memFree=12907584/32951124 swapFree=67110188/67111528
[pid=30378] ppid=30376 vsize=2073340 CPUtime=35144.9 cores=0-7
/proc/30378/stat : 30378 (claspmt) R 30376 30378 29040 0 -1 4202496 438112 0 0 0 3513920 574 0 0 25 0 8 0 549949913 2123100160 427952 33554432000 4194304 4964704 140736331414656 18446744073709551615 4653154 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/30378/statm: 518335 427952 466 189 0 512684 0
[pid=30378/tid=30380] ppid=30376 vsize=2073340 CPUtime=4322.81 cores=0-7
/proc/30378/task/30380/stat : 30380 (claspmt) R 30376 30378 29040 0 -1 4202560 43594 0 0 0 432218 63 0 0 25 0 8 0 549950083 2123100160 427952 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=30378/tid=30381] ppid=30376 vsize=2073340 CPUtime=4657.8 cores=0-7
/proc/30378/task/30381/stat : 30381 (claspmt) R 30376 30378 29040 0 -1 4202560 60130 0 0 0 465707 73 0 0 25 0 8 0 549950083 2123100160 427952 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=30378/tid=30382] ppid=30376 vsize=2073340 CPUtime=4092.4 cores=0-7
/proc/30378/task/30382/stat : 30382 (claspmt) R 30376 30378 29040 0 -1 4202560 53868 0 0 0 409160 80 0 0 25 0 8 0 549950083 2123100160 427952 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=30378/tid=30383] ppid=30376 vsize=2073340 CPUtime=4745.25 cores=0-7
/proc/30378/task/30383/stat : 30383 (claspmt) R 30376 30378 29040 0 -1 4202560 37120 0 0 0 474486 39 0 0 25 0 8 0 549950083 2123100160 427952 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30378/tid=30384] ppid=30376 vsize=2073340 CPUtime=4811.36 cores=0-7
/proc/30378/task/30384/stat : 30384 (claspmt) R 30376 30378 29040 0 -1 4202560 47336 0 0 0 481079 57 0 0 25 0 8 0 549950084 2123100160 427952 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=30378/tid=30385] ppid=30376 vsize=2073340 CPUtime=4151.95 cores=0-7
/proc/30378/task/30385/stat : 30385 (claspmt) R 30376 30378 29040 0 -1 4202560 51296 0 0 0 415114 81 0 0 25 0 8 0 549950084 2123100160 427952 33554432000 4194304 4964704 140736331414656 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30378/tid=30386] ppid=30376 vsize=2073340 CPUtime=4299.17 cores=0-7
/proc/30378/task/30386/stat : 30386 (claspmt) R 30376 30378 29040 0 -1 4202560 81297 0 0 0 429816 101 0 0 25 0 8 0 549950084 2123100160 427952 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 35144.9
Current children cumulated vsize (KiB) 2073340

[startup+5082.3 s]
/proc/loadavg: 9.06 9.08 9.03 10/240 31043
/proc/meminfo: memFree=12898744/32951124 swapFree=67110188/67111528
[pid=30378] ppid=30376 vsize=2073340 CPUtime=35571.4 cores=0-7
/proc/30378/stat : 30378 (claspmt) R 30376 30378 29040 0 -1 4202496 438584 0 0 0 3556561 576 0 0 25 0 8 0 549949913 2123100160 428286 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670255 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/30378/statm: 518335 428286 466 189 0 512684 0
[pid=30378/tid=30380] ppid=30376 vsize=2073340 CPUtime=4382.8 cores=0-7
/proc/30378/task/30380/stat : 30380 (claspmt) R 30376 30378 29040 0 -1 4202560 43764 0 0 0 438216 64 0 0 25 0 8 0 549950083 2123100160 428286 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=30378/tid=30381] ppid=30376 vsize=2073340 CPUtime=4717.8 cores=0-7
/proc/30378/task/30381/stat : 30381 (claspmt) R 30376 30378 29040 0 -1 4202560 60130 0 0 0 471707 73 0 0 25 0 8 0 549950083 2123100160 428286 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=30378/tid=30382] ppid=30376 vsize=2073340 CPUtime=4127.97 cores=0-7
/proc/30378/task/30382/stat : 30382 (claspmt) R 30376 30378 29040 0 -1 4202560 53869 0 0 0 412717 80 0 0 25 0 8 0 549950083 2123100160 428286 33554432000 4194304 4964704 140736331414656 18446744073709551615 4332351 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30378/tid=30383] ppid=30376 vsize=2073340 CPUtime=4802.54 cores=0-7
/proc/30378/task/30383/stat : 30383 (claspmt) R 30376 30378 29040 0 -1 4202560 37143 0 0 0 480215 39 0 0 25 0 8 0 549950083 2123100160 428286 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30378/tid=30384] ppid=30376 vsize=2073340 CPUtime=4856.79 cores=0-7
/proc/30378/task/30384/stat : 30384 (claspmt) R 30376 30378 29040 0 -1 4202560 47338 0 0 0 485622 57 0 0 25 0 8 0 549950084 2123100160 428286 33554432000 4194304 4964704 140736331414656 18446744073709551615 4362383 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=30378/tid=30385] ppid=30376 vsize=2073340 CPUtime=4211.51 cores=0-7
/proc/30378/task/30385/stat : 30385 (claspmt) R 30376 30378 29040 0 -1 4202560 51472 0 0 0 421070 81 0 0 25 0 8 0 549950084 2123100160 428286 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30378/tid=30386] ppid=30376 vsize=2073340 CPUtime=4347.77 cores=0-7
/proc/30378/task/30386/stat : 30386 (claspmt) R 30376 30378 29040 0 -1 4202560 81297 0 0 0 434676 101 0 0 25 0 8 0 549950084 2123100160 428286 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 35571.4
Current children cumulated vsize (KiB) 2073340



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5100 s]
/proc/loadavg: 9.04 9.07 9.03 10/240 31044
/proc/meminfo: memFree=12896140/32951124 swapFree=67110188/67111528
[pid=30378] ppid=30376 vsize=2073340 CPUtime=35695.2 cores=0-7
/proc/30378/stat : 30378 (claspmt) R 30376 30378 29040 0 -1 4202496 438584 0 0 0 3568941 578 0 0 25 0 8 0 549949913 2123100160 428286 33554432000 4194304 4964704 140736331414656 18446744073709551615 4659143 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/30378/statm: 518335 428286 466 189 0 512684 0
[pid=30378/tid=30380] ppid=30376 vsize=2073340 CPUtime=4400.5 cores=0-7
/proc/30378/task/30380/stat : 30380 (claspmt) R 30376 30378 29040 0 -1 4202560 43764 0 0 0 439986 64 0 0 25 0 8 0 549950083 2123100160 428286 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=30378/tid=30381] ppid=30376 vsize=2073340 CPUtime=4735.5 cores=0-7
/proc/30378/task/30381/stat : 30381 (claspmt) R 30376 30378 29040 0 -1 4202560 60130 0 0 0 473477 73 0 0 25 0 8 0 549950083 2123100160 428286 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670193 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=30378/tid=30382] ppid=30376 vsize=2073340 CPUtime=4136.15 cores=0-7
/proc/30378/task/30382/stat : 30382 (claspmt) R 30376 30378 29040 0 -1 4202560 53869 0 0 0 413535 80 0 0 25 0 8 0 549950083 2123100160 428286 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30378/tid=30383] ppid=30376 vsize=2073340 CPUtime=4812.06 cores=0-7
/proc/30378/task/30383/stat : 30383 (claspmt) R 30376 30378 29040 0 -1 4202560 37143 0 0 0 481167 39 0 0 25 0 8 0 549950083 2123100160 428286 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=30378/tid=30384] ppid=30376 vsize=2073340 CPUtime=4874.5 cores=0-7
/proc/30378/task/30384/stat : 30384 (claspmt) R 30376 30378 29040 0 -1 4202560 47338 0 0 0 487392 58 0 0 25 0 8 0 549950084 2123100160 428286 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=30378/tid=30385] ppid=30376 vsize=2073340 CPUtime=4229.08 cores=0-7
/proc/30378/task/30385/stat : 30385 (claspmt) R 30376 30378 29040 0 -1 4202560 51472 0 0 0 422827 81 0 0 25 0 8 0 549950084 2123100160 428286 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=30378/tid=30386] ppid=30376 vsize=2073340 CPUtime=4365.48 cores=0-7
/proc/30378/task/30386/stat : 30386 (claspmt) R 30376 30378 29040 0 -1 4202560 81297 0 0 0 436447 101 0 0 25 0 8 0 549950084 2123100160 428286 33554432000 4194304 4964704 140736331414656 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 35695.2
Current children cumulated vsize (KiB) 2073340

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5100.2 s]
/proc/loadavg: 9.04 9.07 9.03 10/240 31044
/proc/meminfo: memFree=12896140/32951124 swapFree=67110188/67111528
[pid=30378] ppid=30376 vsize=2065896 CPUtime=35695.4 cores=0-7
/proc/30378/stat : 30378 (claspmt) R 30376 30378 29040 0 -1 4202496 438594 0 0 0 3568964 578 0 0 25 0 1 0 549949913 2115477504 425475 33554432000 4194304 4964704 140736331414656 18446744073709551615 222921433649 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/30378/statm: 516474 425475 476 189 0 510823 0
Current children cumulated CPU time (s) 35695.4
Current children cumulated vsize (KiB) 2065896

Child status: 0
Real time (s): 5100.3
CPU time (s): 35695.5
CPU user time (s): 35689.7
CPU system time (s): 5.84711
CPU usage (%): 699.871
Max. virtual memory (cumulated for all children) (KiB): 2073340

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 35689.7
system time used= 5.84711
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 438595
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 197686
involuntary context switches= 262931

runsolver used 10.5334 second user time and 26.582 second system time

The end

Launcher Data 

Begin job on node109 at 2011-05-03 02:44:52
IDJOB=3322402
IDBENCH=83167
IDSOLVER=1711
FILE ID=node109/3322402-1304383492
RUNJOBID= node109-1304371481-29054
PBS_JOBID= 13181774
Free space on /tmp= 73236 MiB

SOLVER NAME= claspmt 2.0-R4095
BENCH NAME= SAT11/crafted/mosoi/289/289-unsat-21x14.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3322402-1304383492/watcher-3322402-1304383492 -o /tmp/evaluation-result-3322402-1304383492/solver-3322402-1304383492 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3322402-1304383492.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= f76302f2442dfc7cabc2315e61e55fea
RANDOM SEED=68423898

node109.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      15413196 kB
Buffers:        870620 kB
Cached:        2805840 kB
SwapCached:        712 kB
Active:       13823568 kB
Inactive:      3282048 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      15413196 kB
SwapTotal:    67111528 kB
SwapFree:     67110188 kB
Dirty:            2808 kB
Writeback:           0 kB
AnonPages:    13428700 kB
Mapped:          21512 kB
Slab:           333852 kB
PageTables:      33820 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 20254208 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73324 MiB
End job on node109 at 2011-05-03 04:09:54