Trace number 3322398

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095? (TO) 40000.7 5008.55

General information on the benchmark

Namecrafted/mosoi/289/
289-unsat-5x41.cnf
MD5SUM420773b7b438dd18c8a88a15fb5bcae1
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables820
Number of clauses34235
Sum of the clauses size134480
Maximum clause length4
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 21230
Number of clauses of size 30
Number of clauses of size 433005
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...382792/instance-3322398-1304382792.cnf
0.39/0.43	c Solving...
40000.42/5008.31	
40000.42/5008.31	*** INTERRUPTED! ***
40000.42/5008.31	*** Shutting down threads...
40000.66/5008.54	s UNKNOWN
40000.66/5008.54	
40000.66/5008.54	c INTERRUPTED   : 1
40000.66/5008.54	c Models        : 0+    
40000.66/5008.54	c Time          : 5008.315s (Solving: 5007.88s 1st Model: 0.00s Unsat: 0.00s)
40000.66/5008.54	c CPU Time      : 40000.460s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3322398-1304382792/watcher-3322398-1304382792 -o /tmp/evaluation-result-3322398-1304382792/solver-3322398-1304382792 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3322398-1304382792.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.02 8.43 8.49 1/329 20983
/proc/meminfo: memFree=19486624/32951124 swapFree=67111516/67111528
[pid=20983] ppid=20981 vsize=22600 CPUtime=0 cores=0-7
/proc/20983/stat : 20983 (claspmt) R 20981 20983 19640 0 -1 4202496 1030 0 0 0 0 0 0 0 25 0 1 0 549880199 23142400 864 33554432000 4194304 4964704 140737416811776 18446744073709551615 270613110528 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/20983/statm: 5650 865 386 189 0 524 0

[startup+0.0259421 s]
/proc/loadavg: 8.02 8.43 8.49 1/329 20983
/proc/meminfo: memFree=19486624/32951124 swapFree=67111516/67111528
[pid=20983] ppid=20981 vsize=24140 CPUtime=0.02 cores=0-7
/proc/20983/stat : 20983 (claspmt) R 20981 20983 19640 0 -1 4202496 1418 0 0 0 2 0 0 0 25 0 1 0 549880199 24719360 1245 33554432000 4194304 4964704 140737416811776 18446744073709551615 4607585 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/20983/statm: 6035 1245 399 189 0 909 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 24140

[startup+0.10093 s]
/proc/loadavg: 8.02 8.43 8.49 1/329 20983
/proc/meminfo: memFree=19486624/32951124 swapFree=67111516/67111528
[pid=20983] ppid=20981 vsize=24404 CPUtime=0.09 cores=0-7
/proc/20983/stat : 20983 (claspmt) R 20981 20983 19640 0 -1 4202496 1507 0 0 0 9 0 0 0 25 0 1 0 549880199 24989696 1334 33554432000 4194304 4964704 140737416811776 18446744073709551615 4600695 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/20983/statm: 6101 1334 400 189 0 975 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 24404

[startup+0.300895 s]
/proc/loadavg: 8.02 8.43 8.49 1/329 20983
/proc/meminfo: memFree=19486624/32951124 swapFree=67111516/67111528
[pid=20983] ppid=20981 vsize=26648 CPUtime=0.29 cores=0-7
/proc/20983/stat : 20983 (claspmt) R 20981 20983 19640 0 -1 4202496 2039 0 0 0 29 0 0 0 25 0 1 0 549880199 27287552 1866 33554432000 4194304 4964704 140737416811776 18446744073709551615 270612771525 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/20983/statm: 6662 1866 400 189 0 1536 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 26648

[startup+0.83091 s]
/proc/loadavg: 8.02 8.43 8.49 1/329 20983
/proc/meminfo: memFree=19486624/32951124 swapFree=67111516/67111528
[pid=20983] ppid=20981 vsize=458888 CPUtime=4.84 cores=0-7
/proc/20983/stat : 20983 (claspmt) R 20981 20983 19640 0 -1 4202496 14385 0 0 0 477 7 0 0 20 0 8 0 549880199 469901312 14122 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/20983/statm: 114755 14363 464 189 0 109104 0
Current children cumulated CPU time (s) 4.84
Current children cumulated vsize (KiB) 458888

[startup+1.50065 s]
/proc/loadavg: 8.02 8.43 8.49 9/337 20992
/proc/meminfo: memFree=19425560/32951124 swapFree=67111516/67111528
[pid=20983] ppid=20981 vsize=460652 CPUtime=8.74 cores=0-7
/proc/20983/stat : 20983 (claspmt) R 20981 20983 19640 0 -1 4202496 17164 0 0 0 866 8 0 0 21 0 8 0 549880199 471707648 16901 33554432000 4194304 4964704 140737416811776 18446744073709551615 270612777986 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/20983/statm: 115163 16901 464 189 0 109512 0
[pid=20983/tid=20985] ppid=20981 vsize=460652 CPUtime=1.03 cores=0-7
/proc/20983/task/20985/stat : 20985 (claspmt) R 20981 20983 19640 0 -1 4202560 1632 0 0 0 102 1 0 0 19 0 8 0 549880243 471707648 16901 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=20983/tid=20986] ppid=20981 vsize=460652 CPUtime=1.03 cores=0-7
/proc/20983/task/20986/stat : 20986 (claspmt) R 20981 20983 19640 0 -1 4202560 1979 0 0 0 103 0 0 0 19 0 8 0 549880243 471707648 16901 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670419 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20983/tid=20987] ppid=20981 vsize=460652 CPUtime=1.03 cores=0-7
/proc/20983/task/20987/stat : 20987 (claspmt) R 20981 20983 19640 0 -1 4202560 1824 0 0 0 103 0 0 0 19 0 8 0 549880243 471707648 16901 33554432000 4194304 4964704 140737416811776 18446744073709551615 4654527 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20983/tid=20988] ppid=20981 vsize=460652 CPUtime=1.02 cores=0-7
/proc/20983/task/20988/stat : 20988 (claspmt) R 20981 20983 19640 0 -1 4202560 2208 0 0 0 102 0 0 0 19 0 8 0 549880243 471707648 16901 33554432000 4194304 4964704 140737416811776 18446744073709551615 4342199 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20983/tid=20989] ppid=20981 vsize=460652 CPUtime=1.04 cores=0-7
/proc/20983/task/20989/stat : 20989 (claspmt) R 20981 20983 19640 0 -1 4202560 1553 0 0 0 104 0 0 0 18 0 8 0 549880243 471707648 16901 33554432000 4194304 4964704 140737416811776 18446744073709551615 4325336 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20983/tid=20990] ppid=20981 vsize=460652 CPUtime=1.03 cores=0-7
/proc/20983/task/20990/stat : 20990 (claspmt) R 20981 20983 19640 0 -1 4202560 2042 0 0 0 102 1 0 0 19 0 8 0 549880243 471707648 16901 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670215 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20983/tid=20991] ppid=20981 vsize=460652 CPUtime=1.03 cores=0-7
/proc/20983/task/20991/stat : 20991 (claspmt) R 20981 20983 19640 0 -1 4202560 2036 0 0 0 102 1 0 0 19 0 8 0 549880243 471707648 16901 33554432000 4194304 4964704 140737416811776 18446744073709551615 4327260 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 8.74
Current children cumulated vsize (KiB) 460652

[startup+3.10037 s]
/proc/loadavg: 8.02 8.43 8.49 9/337 20992
/proc/meminfo: memFree=19407952/32951124 swapFree=67111516/67111528
[pid=20983] ppid=20981 vsize=464416 CPUtime=21.52 cores=0-7
/proc/20983/stat : 20983 (claspmt) R 20981 20983 19640 0 -1 4202496 22248 0 0 0 2141 11 0 0 25 0 8 0 549880199 475561984 21985 33554432000 4194304 4964704 140737416811776 18446744073709551615 4312130 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/20983/statm: 116104 21985 466 189 0 110453 0
[pid=20983/tid=20985] ppid=20981 vsize=464416 CPUtime=2.63 cores=0-7
/proc/20983/task/20985/stat : 20985 (claspmt) R 20981 20983 19640 0 -1 4202560 2301 0 0 0 262 1 0 0 22 0 8 0 549880243 475561984 21985 33554432000 4194304 4964704 140737416811776 18446744073709551615 4660043 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=20983/tid=20986] ppid=20981 vsize=464416 CPUtime=2.63 cores=0-7
/proc/20983/task/20986/stat : 20986 (claspmt) R 20981 20983 19640 0 -1 4202560 2703 0 0 0 262 1 0 0 22 0 8 0 549880243 475561984 21985 33554432000 4194304 4964704 140737416811776 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20983/tid=20987] ppid=20981 vsize=464416 CPUtime=2.63 cores=0-7
/proc/20983/task/20987/stat : 20987 (claspmt) R 20981 20983 19640 0 -1 4202560 2328 0 0 0 262 1 0 0 22 0 8 0 549880243 475561984 21985 33554432000 4194304 4964704 140737416811776 18446744073709551615 4320872 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20983/tid=20988] ppid=20981 vsize=464416 CPUtime=2.63 cores=0-7
/proc/20983/task/20988/stat : 20988 (claspmt) R 20981 20983 19640 0 -1 4202560 2832 0 0 0 262 1 0 0 22 0 8 0 549880243 475561984 21985 33554432000 4194304 4964704 140737416811776 18446744073709551615 4658372 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20983/tid=20989] ppid=20981 vsize=464416 CPUtime=2.64 cores=0-7
/proc/20983/task/20989/stat : 20989 (claspmt) R 20981 20983 19640 0 -1 4202560 1991 0 0 0 263 1 0 0 21 0 8 0 549880243 475561984 21985 33554432000 4194304 4964704 140737416811776 18446744073709551615 4627536 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20983/tid=20990] ppid=20981 vsize=464416 CPUtime=2.63 cores=0-7
/proc/20983/task/20990/stat : 20990 (claspmt) R 20981 20983 19640 0 -1 4202560 3077 0 0 0 262 1 0 0 22 0 8 0 549880243 475561984 21985 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20983/tid=20991] ppid=20981 vsize=464416 CPUtime=2.61 cores=0-7
/proc/20983/task/20991/stat : 20991 (claspmt) R 20981 20983 19640 0 -1 4202560 2804 0 0 0 260 1 0 0 22 0 8 0 549880243 475561984 21985 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 21.52
Current children cumulated vsize (KiB) 464416

[startup+6.3008 s]
/proc/loadavg: 8.02 8.42 8.48 9/337 20992
/proc/meminfo: memFree=19377456/32951124 swapFree=67111516/67111528
[pid=20983] ppid=20981 vsize=467984 CPUtime=47.08 cores=0-7
/proc/20983/stat : 20983 (claspmt) R 20981 20983 19640 0 -1 4202496 28574 0 0 0 4693 15 0 0 18 0 8 0 549880199 479215616 28299 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670574 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/20983/statm: 116996 28299 466 189 0 111345 0
[pid=20983/tid=20985] ppid=20981 vsize=467984 CPUtime=5.82 cores=0-7
/proc/20983/task/20985/stat : 20985 (claspmt) R 20981 20983 19640 0 -1 4202560 3061 0 0 0 581 1 0 0 18 0 8 0 549880243 479215616 28299 33554432000 4194304 4964704 140737416811776 18446744073709551615 4369927 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=20983/tid=20986] ppid=20981 vsize=467984 CPUtime=5.83 cores=0-7
/proc/20983/task/20986/stat : 20986 (claspmt) R 20981 20983 19640 0 -1 4202560 3456 0 0 0 582 1 0 0 25 0 8 0 549880243 479215616 28299 33554432000 4194304 4964704 140737416811776 18446744073709551615 4320905 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20983/tid=20987] ppid=20981 vsize=467984 CPUtime=5.83 cores=0-7
/proc/20983/task/20987/stat : 20987 (claspmt) R 20981 20983 19640 0 -1 4202560 3249 0 0 0 581 2 0 0 25 0 8 0 549880243 479215616 28299 33554432000 4194304 4964704 140737416811776 18446744073709551615 4320976 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20983/tid=20988] ppid=20981 vsize=467984 CPUtime=5.82 cores=0-7
/proc/20983/task/20988/stat : 20988 (claspmt) R 20981 20983 19640 0 -1 4202560 3678 0 0 0 581 1 0 0 25 0 8 0 549880243 479215616 28299 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670419 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20983/tid=20989] ppid=20981 vsize=467984 CPUtime=5.84 cores=0-7
/proc/20983/task/20989/stat : 20989 (claspmt) R 20981 20983 19640 0 -1 4202560 2562 0 0 0 583 1 0 0 18 0 8 0 549880243 479215616 28299 33554432000 4194304 4964704 140737416811776 18446744073709551615 4325305 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20983/tid=20990] ppid=20981 vsize=467984 CPUtime=5.83 cores=0-7
/proc/20983/task/20990/stat : 20990 (claspmt) R 20981 20983 19640 0 -1 4202560 4112 0 0 0 581 2 0 0 25 0 8 0 549880243 479215616 28299 33554432000 4194304 4964704 140737416811776 18446744073709551615 4627181 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20983/tid=20991] ppid=20981 vsize=467984 CPUtime=5.78 cores=0-7
/proc/20983/task/20991/stat : 20991 (claspmt) R 20981 20983 19640 0 -1 4202560 3627 0 0 0 576 2 0 0 25 0 8 0 549880243 479215616 28299 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 47.08
Current children cumulated vsize (KiB) 467984

[startup+12.7007 s]
/proc/loadavg: 8.02 8.41 8.48 9/337 20992
/proc/meminfo: memFree=19329880/32951124 swapFree=67111516/67111528
[pid=20983] ppid=20981 vsize=476420 CPUtime=98.18 cores=0-7
/proc/20983/stat : 20983 (claspmt) R 20981 20983 19640 0 -1 4202496 40770 0 0 0 9795 23 0 0 25 0 8 0 549880199 487854080 40374 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/20983/statm: 119105 40374 466 189 0 113454 0
[pid=20983/tid=20985] ppid=20981 vsize=476420 CPUtime=12.22 cores=0-7
/proc/20983/task/20985/stat : 20985 (claspmt) R 20981 20983 19640 0 -1 4202560 4835 0 0 0 1220 2 0 0 18 0 8 0 549880243 487854080 40374 33554432000 4194304 4964704 140737416811776 18446744073709551615 4320905 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=20983/tid=20986] ppid=20981 vsize=476420 CPUtime=12.22 cores=0-7
/proc/20983/task/20986/stat : 20986 (claspmt) R 20981 20983 19640 0 -1 4202560 5261 0 0 0 1220 2 0 0 25 0 8 0 549880243 487854080 40374 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20983/tid=20987] ppid=20981 vsize=476420 CPUtime=12.22 cores=0-7
/proc/20983/task/20987/stat : 20987 (claspmt) R 20981 20983 19640 0 -1 4202560 4384 0 0 0 1220 2 0 0 25 0 8 0 549880243 487854080 40374 33554432000 4194304 4964704 140737416811776 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20983/tid=20988] ppid=20981 vsize=476420 CPUtime=12.22 cores=0-7
/proc/20983/task/20988/stat : 20988 (claspmt) R 20981 20983 19640 0 -1 4202560 5391 0 0 0 1220 2 0 0 21 0 8 0 549880243 487854080 40374 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670184 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20983/tid=20989] ppid=20981 vsize=476420 CPUtime=12.24 cores=0-7
/proc/20983/task/20989/stat : 20989 (claspmt) R 20981 20983 19640 0 -1 4202560 3999 0 0 0 1221 3 0 0 18 0 8 0 549880243 487854080 40374 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20983/tid=20990] ppid=20981 vsize=476420 CPUtime=12.22 cores=0-7
/proc/20983/task/20990/stat : 20990 (claspmt) R 20981 20983 19640 0 -1 4202560 5655 0 0 0 1220 2 0 0 25 0 8 0 549880243 487854080 40374 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20983/tid=20991] ppid=20981 vsize=476420 CPUtime=12.11 cores=0-7
/proc/20983/task/20991/stat : 20991 (claspmt) R 20981 20983 19640 0 -1 4202560 4978 0 0 0 1208 3 0 0 25 0 8 0 549880243 487854080 40374 33554432000 4194304 4964704 140737416811776 18446744073709551615 4321057 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 98.18
Current children cumulated vsize (KiB) 476420

[startup+25.5004 s]
/proc/loadavg: 8.01 8.39 8.47 9/337 20992
/proc/meminfo: memFree=19273460/32951124 swapFree=67111516/67111528
[pid=20983] ppid=20981 vsize=482472 CPUtime=200.47 cores=0-7
/proc/20983/stat : 20983 (claspmt) R 20981 20983 19640 0 -1 4202496 55582 0 0 0 20015 32 0 0 25 0 8 0 549880199 494051328 54968 33554432000 4194304 4964704 140737416811776 18446744073709551615 4362211 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/20983/statm: 120618 54968 466 189 0 114967 0
[pid=20983/tid=20985] ppid=20981 vsize=482472 CPUtime=25.01 cores=0-7
/proc/20983/task/20985/stat : 20985 (claspmt) R 20981 20983 19640 0 -1 4202560 7355 0 0 0 2498 3 0 0 25 0 8 0 549880243 494051328 54968 33554432000 4194304 4964704 140737416811776 18446744073709551615 4321150 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=20983/tid=20986] ppid=20981 vsize=482472 CPUtime=25.04 cores=0-7
/proc/20983/task/20986/stat : 20986 (claspmt) R 20981 20983 19640 0 -1 4202560 7721 0 0 0 2500 4 0 0 25 0 8 0 549880243 494051328 54968 33554432000 4194304 4964704 140737416811776 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20983/tid=20987] ppid=20981 vsize=482472 CPUtime=25.03 cores=0-7
/proc/20983/task/20987/stat : 20987 (claspmt) R 20981 20983 19640 0 -1 4202560 5690 0 0 0 2499 4 0 0 25 0 8 0 549880243 494051328 54968 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670427 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20983/tid=20988] ppid=20981 vsize=482472 CPUtime=25.02 cores=0-7
/proc/20983/task/20988/stat : 20988 (claspmt) R 20981 20983 19640 0 -1 4202560 7615 0 0 0 2498 4 0 0 25 0 8 0 549880243 494051328 54968 33554432000 4194304 4964704 140737416811776 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20983/tid=20989] ppid=20981 vsize=482472 CPUtime=25.05 cores=0-7
/proc/20983/task/20989/stat : 20989 (claspmt) R 20981 20983 19640 0 -1 4202560 5442 0 0 0 2501 4 0 0 25 0 8 0 549880243 494051328 54968 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20983/tid=20990] ppid=20981 vsize=482472 CPUtime=25.04 cores=0-7
/proc/20983/task/20990/stat : 20990 (claspmt) R 20981 20983 19640 0 -1 4202560 7418 0 0 0 2500 4 0 0 25 0 8 0 549880243 494051328 54968 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20983/tid=20991] ppid=20981 vsize=482472 CPUtime=24.77 cores=0-7
/proc/20983/task/20991/stat : 20991 (claspmt) R 20981 20983 19640 0 -1 4202560 6290 0 0 0 2474 3 0 0 25 0 8 0 549880243 494051328 54968 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670266 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 200.47
Current children cumulated vsize (KiB) 482472

[startup+51.1009 s]
/proc/loadavg: 7.95 8.34 8.45 9/337 20993
/proc/meminfo: memFree=19187712/32951124 swapFree=67111516/67111528
[pid=20983] ppid=20981 vsize=496468 CPUtime=404.84 cores=0-7
/proc/20983/stat : 20983 (claspmt) R 20981 20983 19640 0 -1 4202496 77124 0 0 0 40437 47 0 0 25 0 8 0 549880199 508383232 76366 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/20983/statm: 124117 76366 466 189 0 118466 0
[pid=20983/tid=20985] ppid=20981 vsize=496468 CPUtime=50.58 cores=0-7
/proc/20983/task/20985/stat : 20985 (claspmt) R 20981 20983 19640 0 -1 4202560 9833 0 0 0 5053 5 0 0 25 0 8 0 549880243 508383232 76366 33554432000 4194304 4964704 140737416811776 18446744073709551615 4682039 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=20983/tid=20986] ppid=20981 vsize=496468 CPUtime=50.61 cores=0-7
/proc/20983/task/20986/stat : 20986 (claspmt) R 20981 20983 19640 0 -1 4202560 11810 0 0 0 5056 5 0 0 25 0 8 0 549880243 508383232 76366 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=20983/tid=20987] ppid=20981 vsize=496468 CPUtime=50.61 cores=0-7
/proc/20983/task/20987/stat : 20987 (claspmt) R 20981 20983 19640 0 -1 4202560 7957 0 0 0 5055 6 0 0 25 0 8 0 549880243 508383232 76366 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670283 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20983/tid=20988] ppid=20981 vsize=496468 CPUtime=50.61 cores=0-7
/proc/20983/task/20988/stat : 20988 (claspmt) R 20981 20983 19640 0 -1 4202560 9935 0 0 0 5055 6 0 0 25 0 8 0 549880243 508383232 76366 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20983/tid=20989] ppid=20981 vsize=496468 CPUtime=50.62 cores=0-7
/proc/20983/task/20989/stat : 20989 (claspmt) R 20981 20983 19640 0 -1 4202560 7247 0 0 0 5057 5 0 0 25 0 8 0 549880243 508383232 76366 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20983/tid=20990] ppid=20981 vsize=496468 CPUtime=50.61 cores=0-7
/proc/20983/task/20990/stat : 20990 (claspmt) R 20981 20983 19640 0 -1 4202560 10066 0 0 0 5056 5 0 0 25 0 8 0 549880243 508383232 76366 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20983/tid=20991] ppid=20981 vsize=496468 CPUtime=50.07 cores=0-7
/proc/20983/task/20991/stat : 20991 (claspmt) R 20981 20983 19640 0 -1 4202560 10065 0 0 0 5001 6 0 0 25 0 8 0 549880243 508383232 76366 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 404.84

################
# More data... #
################

[pid=20983/tid=20986] ppid=20981 vsize=1836084 CPUtime=4647.52 cores=0-7
/proc/20983/task/20986/stat : 20986 (claspmt) R 20981 20983 19640 0 -1 4202560 82051 0 0 0 464656 96 0 0 25 0 8 0 549880243 1880150016 347434 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20983/tid=20987] ppid=20981 vsize=1836084 CPUtime=4661.08 cores=0-7
/proc/20983/task/20987/stat : 20987 (claspmt) R 20981 20983 19640 0 -1 4202560 28784 0 0 0 466052 56 0 0 25 0 8 0 549880243 1880150016 347434 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20983/tid=20988] ppid=20981 vsize=1836084 CPUtime=4655.08 cores=0-7
/proc/20983/task/20988/stat : 20988 (claspmt) R 20981 20983 19640 0 -1 4202560 46877 0 0 0 465435 73 0 0 25 0 8 0 549880243 1880150016 347434 33554432000 4194304 4964704 140737416811776 18446744073709551615 4338788 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20983/tid=20989] ppid=20981 vsize=1836084 CPUtime=4637.17 cores=0-7
/proc/20983/task/20989/stat : 20989 (claspmt) R 20981 20983 19640 0 -1 4202560 45048 0 0 0 463645 72 0 0 25 0 8 0 549880243 1880150016 347434 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670241 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=20983/tid=20990] ppid=20981 vsize=1836084 CPUtime=4653.06 cores=0-7
/proc/20983/task/20990/stat : 20990 (claspmt) R 20981 20983 19640 0 -1 4202560 35778 0 0 0 465223 83 0 0 25 0 8 0 549880243 1880150016 347434 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20983/tid=20991] ppid=20981 vsize=1836084 CPUtime=4660.55 cores=0-7
/proc/20983/task/20991/stat : 20991 (claspmt) R 20981 20983 19640 0 -1 4202560 40141 0 0 0 465977 78 0 0 25 0 8 0 549880243 1880150016 347434 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 37236.6
Current children cumulated vsize (KiB) 1836084

[startup+4722.3 s]
/proc/loadavg: 8.13 8.07 8.04 9/337 21129
/proc/meminfo: memFree=18093232/32951124 swapFree=67111516/67111528
[pid=20983] ppid=20981 vsize=1837172 CPUtime=37715.8 cores=0-7
/proc/20983/stat : 20983 (claspmt) R 20981 20983 19640 0 -1 4202496 372728 0 0 0 3770967 617 0 0 25 0 8 0 549880199 1881264128 348917 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/20983/statm: 459293 348917 466 189 0 453642 0
[pid=20983/tid=20985] ppid=20981 vsize=1837172 CPUtime=4720.28 cores=0-7
/proc/20983/task/20985/stat : 20985 (claspmt) R 20981 20983 19640 0 -1 4202560 59574 0 0 0 471947 81 0 0 25 0 8 0 549880243 1881264128 348917 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670263 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=20983/tid=20986] ppid=20981 vsize=1837172 CPUtime=4707.51 cores=0-7
/proc/20983/task/20986/stat : 20986 (claspmt) R 20981 20983 19640 0 -1 4202560 82482 0 0 0 470654 97 0 0 25 0 8 0 549880243 1881264128 348917 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670233 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20983/tid=20987] ppid=20981 vsize=1837172 CPUtime=4721.08 cores=0-7
/proc/20983/task/20987/stat : 20987 (claspmt) R 20981 20983 19640 0 -1 4202560 28813 0 0 0 472052 56 0 0 25 0 8 0 549880243 1881264128 348917 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20983/tid=20988] ppid=20981 vsize=1837172 CPUtime=4715.06 cores=0-7
/proc/20983/task/20988/stat : 20988 (claspmt) R 20981 20983 19640 0 -1 4202560 47499 0 0 0 471432 74 0 0 25 0 8 0 549880243 1881264128 348917 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20983/tid=20989] ppid=20981 vsize=1837172 CPUtime=4696.52 cores=0-7
/proc/20983/task/20989/stat : 20989 (claspmt) R 20981 20983 19640 0 -1 4202560 45136 0 0 0 469580 72 0 0 25 0 8 0 549880243 1881264128 348917 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670233 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=20983/tid=20990] ppid=20981 vsize=1837172 CPUtime=4713.06 cores=0-7
/proc/20983/task/20990/stat : 20990 (claspmt) R 20981 20983 19640 0 -1 4202560 35850 0 0 0 471222 84 0 0 25 0 8 0 549880243 1881264128 348917 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670215 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20983/tid=20991] ppid=20981 vsize=1837172 CPUtime=4720.54 cores=0-7
/proc/20983/task/20991/stat : 20991 (claspmt) R 20981 20983 19640 0 -1 4202560 40148 0 0 0 471975 79 0 0 25 0 8 0 549880243 1881264128 348917 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 37715.8
Current children cumulated vsize (KiB) 1837172

[startup+4782.3 s]
/proc/loadavg: 8.12 8.07 8.04 9/337 21131
/proc/meminfo: memFree=18088124/32951124 swapFree=67111516/67111528
[pid=20983] ppid=20981 vsize=1837172 CPUtime=38195.1 cores=0-7
/proc/20983/stat : 20983 (claspmt) R 20981 20983 19640 0 -1 4202496 374211 0 0 0 3818891 621 0 0 25 0 8 0 549880199 1881264128 350218 33554432000 4194304 4964704 140737416811776 18446744073709551615 4660415 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/20983/statm: 459293 350218 466 189 0 453642 0
[pid=20983/tid=20985] ppid=20981 vsize=1837172 CPUtime=4780.26 cores=0-7
/proc/20983/task/20985/stat : 20985 (claspmt) R 20981 20983 19640 0 -1 4202560 59836 0 0 0 477945 81 0 0 25 0 8 0 549880243 1881264128 350218 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670454 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=20983/tid=20986] ppid=20981 vsize=1837172 CPUtime=4767.5 cores=0-7
/proc/20983/task/20986/stat : 20986 (claspmt) R 20981 20983 19640 0 -1 4202560 83359 0 0 0 476652 98 0 0 25 0 8 0 549880243 1881264128 350218 33554432000 4194304 4964704 140737416811776 18446744073709551615 270612808322 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20983/tid=20987] ppid=20981 vsize=1837172 CPUtime=4781.08 cores=0-7
/proc/20983/task/20987/stat : 20987 (claspmt) R 20981 20983 19640 0 -1 4202560 28833 0 0 0 478051 57 0 0 25 0 8 0 549880243 1881264128 350218 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20983/tid=20988] ppid=20981 vsize=1837172 CPUtime=4775.05 cores=0-7
/proc/20983/task/20988/stat : 20988 (claspmt) R 20981 20983 19640 0 -1 4202560 47660 0 0 0 477431 74 0 0 25 0 8 0 549880243 1881264128 350218 33554432000 4194304 4964704 140737416811776 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20983/tid=20989] ppid=20981 vsize=1837172 CPUtime=4755.86 cores=0-7
/proc/20983/task/20989/stat : 20989 (claspmt) R 20981 20983 19640 0 -1 4202560 45142 0 0 0 475513 73 0 0 25 0 8 0 549880243 1881264128 350218 33554432000 4194304 4964704 140737416811776 18446744073709551615 270612801488 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=20983/tid=20990] ppid=20981 vsize=1837172 CPUtime=4773.05 cores=0-7
/proc/20983/task/20990/stat : 20990 (claspmt) R 20981 20983 19640 0 -1 4202560 35975 0 0 0 477221 84 0 0 25 0 8 0 549880243 1881264128 350218 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20983/tid=20991] ppid=20981 vsize=1837172 CPUtime=4780.53 cores=0-7
/proc/20983/task/20991/stat : 20991 (claspmt) R 20981 20983 19640 0 -1 4202560 40176 0 0 0 477974 79 0 0 25 0 8 0 549880243 1881264128 350218 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38195.1
Current children cumulated vsize (KiB) 1837172

[startup+4842.3 s]
/proc/loadavg: 8.11 8.07 8.03 9/337 21133
/proc/meminfo: memFree=18084016/32951124 swapFree=67111516/67111528
[pid=20983] ppid=20981 vsize=1837456 CPUtime=38674.4 cores=0-7
/proc/20983/stat : 20983 (claspmt) R 20981 20983 19640 0 -1 4202496 375464 0 0 0 3866811 626 0 0 25 0 8 0 549880199 1881554944 351177 33554432000 4194304 4964704 140737416811776 18446744073709551615 4684659 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/20983/statm: 459364 351177 466 189 0 453713 0
[pid=20983/tid=20985] ppid=20981 vsize=1837456 CPUtime=4840.24 cores=0-7
/proc/20983/task/20985/stat : 20985 (claspmt) R 20981 20983 19640 0 -1 4202560 60249 0 0 0 483942 82 0 0 25 0 8 0 549880243 1881554944 351177 33554432000 4194304 4964704 140737416811776 18446744073709551615 4653129 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=20983/tid=20986] ppid=20981 vsize=1837456 CPUtime=4827.5 cores=0-7
/proc/20983/task/20986/stat : 20986 (claspmt) R 20981 20983 19640 0 -1 4202560 83610 0 0 0 482651 99 0 0 25 0 8 0 549880243 1881554944 351177 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20983/tid=20987] ppid=20981 vsize=1837456 CPUtime=4841.06 cores=0-7
/proc/20983/task/20987/stat : 20987 (claspmt) R 20981 20983 19640 0 -1 4202560 28858 0 0 0 484049 57 0 0 25 0 8 0 549880243 1881554944 351177 33554432000 4194304 4964704 140737416811776 18446744073709551615 4320966 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20983/tid=20988] ppid=20981 vsize=1837456 CPUtime=4835.02 cores=0-7
/proc/20983/task/20988/stat : 20988 (claspmt) R 20981 20983 19640 0 -1 4202560 47780 0 0 0 483427 75 0 0 25 0 8 0 549880243 1881554944 351177 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670454 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20983/tid=20989] ppid=20981 vsize=1837456 CPUtime=4815.21 cores=0-7
/proc/20983/task/20989/stat : 20989 (claspmt) R 20981 20983 19640 0 -1 4202560 45339 0 0 0 481447 74 0 0 25 0 8 0 549880243 1881554944 351177 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670215 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=20983/tid=20990] ppid=20981 vsize=1837456 CPUtime=4833.03 cores=0-7
/proc/20983/task/20990/stat : 20990 (claspmt) R 20981 20983 19640 0 -1 4202560 36151 0 0 0 483219 84 0 0 25 0 8 0 549880243 1881554944 351177 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20983/tid=20991] ppid=20981 vsize=1837456 CPUtime=4840.53 cores=0-7
/proc/20983/task/20991/stat : 20991 (claspmt) R 20981 20983 19640 0 -1 4202560 40176 0 0 0 483973 80 0 0 25 0 8 0 549880243 1881554944 351177 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38674.4
Current children cumulated vsize (KiB) 1837456

[startup+4902.3 s]
/proc/loadavg: 8.10 8.07 8.03 9/337 21134
/proc/meminfo: memFree=18077044/32951124 swapFree=67111516/67111528
[pid=20983] ppid=20981 vsize=1837456 CPUtime=39153.6 cores=0-7
/proc/20983/stat : 20983 (claspmt) R 20981 20983 19640 0 -1 4202496 377348 0 0 0 3914735 630 0 0 25 0 8 0 549880199 1881554944 352918 33554432000 4194304 4964704 140737416811776 18446744073709551615 4334283 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/20983/statm: 459364 352918 466 189 0 453713 0
[pid=20983/tid=20985] ppid=20981 vsize=1837456 CPUtime=4900.23 cores=0-7
/proc/20983/task/20985/stat : 20985 (claspmt) R 20981 20983 19640 0 -1 4202560 60596 0 0 0 489940 83 0 0 25 0 8 0 549880243 1881554944 352918 33554432000 4194304 4964704 140737416811776 18446744073709551615 4321020 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=20983/tid=20986] ppid=20981 vsize=1837456 CPUtime=4887.49 cores=0-7
/proc/20983/task/20986/stat : 20986 (claspmt) R 20981 20983 19640 0 -1 4202560 83908 0 0 0 488650 99 0 0 25 0 8 0 549880243 1881554944 352918 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670255 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20983/tid=20987] ppid=20981 vsize=1837456 CPUtime=4901.06 cores=0-7
/proc/20983/task/20987/stat : 20987 (claspmt) R 20981 20983 19640 0 -1 4202560 28914 0 0 0 490048 58 0 0 25 0 8 0 549880243 1881554944 352918 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20983/tid=20988] ppid=20981 vsize=1837456 CPUtime=4895.01 cores=0-7
/proc/20983/task/20988/stat : 20988 (claspmt) R 20981 20983 19640 0 -1 4202560 48069 0 0 0 489426 75 0 0 25 0 8 0 549880243 1881554944 352918 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20983/tid=20989] ppid=20981 vsize=1837456 CPUtime=4874.54 cores=0-7
/proc/20983/task/20989/stat : 20989 (claspmt) R 20981 20983 19640 0 -1 4202560 45469 0 0 0 487380 74 0 0 25 0 8 0 549880243 1881554944 352918 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670281 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=20983/tid=20990] ppid=20981 vsize=1837456 CPUtime=4893.02 cores=0-7
/proc/20983/task/20990/stat : 20990 (claspmt) R 20981 20983 19640 0 -1 4202560 36632 0 0 0 489216 86 0 0 25 0 8 0 549880243 1881554944 352918 33554432000 4194304 4964704 140737416811776 18446744073709551615 4325293 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20983/tid=20991] ppid=20981 vsize=1837456 CPUtime=4900.52 cores=0-7
/proc/20983/task/20991/stat : 20991 (claspmt) R 20981 20983 19640 0 -1 4202560 40459 0 0 0 489971 81 0 0 25 0 8 0 549880243 1881554944 352918 33554432000 4194304 4964704 140737416811776 18446744073709551615 4644864 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39153.7
Current children cumulated vsize (KiB) 1837456

[startup+4962.3 s]
/proc/loadavg: 8.04 8.05 8.03 9/337 21136
/proc/meminfo: memFree=18073296/32951124 swapFree=67111516/67111528
[pid=20983] ppid=20981 vsize=1837992 CPUtime=39633 cores=0-7
/proc/20983/stat : 20983 (claspmt) R 20981 20983 19640 0 -1 4202496 378490 0 0 0 3962668 634 0 0 25 0 8 0 549880199 1882103808 353881 33554432000 4194304 4964704 140737416811776 18446744073709551615 4321016 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/20983/statm: 459498 353881 466 189 0 453847 0
[pid=20983/tid=20985] ppid=20981 vsize=1837992 CPUtime=4960.24 cores=0-7
/proc/20983/task/20985/stat : 20985 (claspmt) R 20981 20983 19640 0 -1 4202560 61049 0 0 0 495940 84 0 0 25 0 8 0 549880243 1882103808 353881 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670419 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=20983/tid=20986] ppid=20981 vsize=1837992 CPUtime=4947.51 cores=0-7
/proc/20983/task/20986/stat : 20986 (claspmt) R 20981 20983 19640 0 -1 4202560 83962 0 0 0 494651 100 0 0 25 0 8 0 549880243 1882103808 353881 33554432000 4194304 4964704 140737416811776 18446744073709551615 270612808330 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20983/tid=20987] ppid=20981 vsize=1837992 CPUtime=4961.07 cores=0-7
/proc/20983/task/20987/stat : 20987 (claspmt) R 20981 20983 19640 0 -1 4202560 29001 0 0 0 496049 58 0 0 25 0 8 0 549880243 1882103808 353881 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20983/tid=20988] ppid=20981 vsize=1837992 CPUtime=4955.01 cores=0-7
/proc/20983/task/20988/stat : 20988 (claspmt) R 20981 20983 19640 0 -1 4202560 48256 0 0 0 495425 76 0 0 25 0 8 0 549880243 1882103808 353881 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20983/tid=20989] ppid=20981 vsize=1837992 CPUtime=4933.89 cores=0-7
/proc/20983/task/20989/stat : 20989 (claspmt) R 20981 20983 19640 0 -1 4202560 45720 0 0 0 493314 75 0 0 25 0 8 0 549880243 1882103808 353881 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=20983/tid=20990] ppid=20981 vsize=1837992 CPUtime=4953.02 cores=0-7
/proc/20983/task/20990/stat : 20990 (claspmt) R 20981 20983 19640 0 -1 4202560 36658 0 0 0 495216 86 0 0 25 0 8 0 549880243 1882103808 353881 33554432000 4194304 4964704 140737416811776 18446744073709551615 4660075 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20983/tid=20991] ppid=20981 vsize=1837992 CPUtime=4960.52 cores=0-7
/proc/20983/task/20991/stat : 20991 (claspmt) R 20981 20983 19640 0 -1 4202560 40543 0 0 0 495971 81 0 0 25 0 8 0 549880243 1882103808 353881 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39633
Current children cumulated vsize (KiB) 1837992



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5008.3 s]
/proc/loadavg: 8.01 8.04 8.02 9/337 21137
/proc/meminfo: memFree=18070800/32951124 swapFree=67111516/67111528
[pid=20983] ppid=20981 vsize=1838312 CPUtime=40000.4 cores=0-7
/proc/20983/stat : 20983 (claspmt) R 20981 20983 19640 0 -1 4202496 379278 0 0 0 3999404 638 0 0 25 0 8 0 549880199 1882431488 354429 33554432000 4194304 4964704 140737416811776 18446744073709551615 4320981 0 0 4096 18946 18446744073709551615 0 0 17 0 0 0 0
/proc/20983/statm: 459578 354429 466 189 0 453927 0
[pid=20983/tid=20985] ppid=20981 vsize=1838312 CPUtime=5006.21 cores=0-7
/proc/20983/task/20985/stat : 20985 (claspmt) R 20981 20983 19640 0 -1 4202560 61510 0 0 0 500536 85 0 0 25 0 8 0 549880243 1882431488 354429 33554432000 4194304 4964704 140737416811776 18446744073709551615 4362279 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=20983/tid=20986] ppid=20981 vsize=1838312 CPUtime=4993.5 cores=0-7
/proc/20983/task/20986/stat : 20986 (claspmt) R 20981 20983 19640 0 -1 4202560 83962 0 0 0 499250 100 0 0 25 0 8 0 549880243 1882431488 354429 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=20983/tid=20987] ppid=20981 vsize=1838312 CPUtime=5007.06 cores=0-7
/proc/20983/task/20987/stat : 20987 (claspmt) R 20981 20983 19640 0 -1 4202560 29096 0 0 0 500647 59 0 0 25 0 8 0 549880243 1882431488 354429 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=20983/tid=20988] ppid=20981 vsize=1838312 CPUtime=5000.99 cores=0-7
/proc/20983/task/20988/stat : 20988 (claspmt) R 20981 20983 19640 0 -1 4202560 48415 0 0 0 500022 77 0 0 25 0 8 0 549880243 1882431488 354429 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=20983/tid=20989] ppid=20981 vsize=1838312 CPUtime=4979.38 cores=0-7
/proc/20983/task/20989/stat : 20989 (claspmt) R 20981 20983 19640 0 -1 4202560 45720 0 0 0 497862 76 0 0 25 0 8 0 549880243 1882431488 354429 33554432000 4194304 4964704 140737416811776 18446744073709551615 4660170 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=20983/tid=20990] ppid=20981 vsize=1838312 CPUtime=4999.01 cores=0-7
/proc/20983/task/20990/stat : 20990 (claspmt) R 20981 20983 19640 0 -1 4202560 36712 0 0 0 499815 86 0 0 21 0 8 0 549880243 1882431488 354429 33554432000 4194304 4964704 140737416811776 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=20983/tid=20991] ppid=20981 vsize=1838312 CPUtime=5006.52 cores=0-7
/proc/20983/task/20991/stat : 20991 (claspmt) R 20981 20983 19640 0 -1 4202560 40562 0 0 0 500570 82 0 0 25 0 8 0 549880243 1882431488 354429 33554432000 4194304 4964704 140737416811776 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 1838312

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5008.5 s]
/proc/loadavg: 8.01 8.04 8.02 2/331 21138
/proc/meminfo: memFree=18504352/32951124 swapFree=67111516/67111528
[pid=20983] ppid=20981 vsize=967216 CPUtime=40000.7 cores=0-7
/proc/20983/stat : 20983 (claspmt) R 20981 20983 19640 0 -1 4202496 379288 0 0 0 3999425 641 0 0 25 0 1 0 549880199 990429184 182462 33554432000 4194304 4964704 140737416811776 18446744073709551615 270613153927 0 0 4096 18946 0 0 0 17 0 0 0 0
/proc/20983/statm: 241804 182462 476 189 0 236153 0
Current children cumulated CPU time (s) 40000.7
Current children cumulated vsize (KiB) 967216

Child status: 0
Real time (s): 5008.55
CPU time (s): 40000.7
CPU user time (s): 39994.3
CPU system time (s): 6.45402
CPU usage (%): 798.648
Max. virtual memory (cumulated for all children) (KiB): 1838312

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39994.3
system time used= 6.45402
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 379289
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 573660
involuntary context switches= 189273

runsolver used 15.6476 second user time and 38.9981 second system time

The end

Launcher Data 

Begin job on node108 at 2011-05-03 02:33:13
IDJOB=3322398
IDBENCH=83163
IDSOLVER=1711
FILE ID=node108/3322398-1304382792
RUNJOBID= node108-1304371481-19654
PBS_JOBID= 13181775
Free space on /tmp= 73316 MiB

SOLVER NAME= claspmt 2.0-R4095
BENCH NAME= SAT11/crafted/mosoi/289/289-unsat-5x41.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3322398-1304382792/watcher-3322398-1304382792 -o /tmp/evaluation-result-3322398-1304382792/solver-3322398-1304382792 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3322398-1304382792.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 420773b7b438dd18c8a88a15fb5bcae1
RANDOM SEED=919912394

node108.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.820
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      19486780 kB
Buffers:        684972 kB
Cached:        2186484 kB
SwapCached:         12 kB
Active:       10526208 kB
Inactive:      2494156 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      19486780 kB
SwapTotal:    67111528 kB
SwapFree:     67111516 kB
Dirty:            1752 kB
Writeback:           0 kB
AnonPages:    10149832 kB
Mapped:          28388 kB
Slab:           337544 kB
PageTables:      38164 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 52245704 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73312 MiB
End job on node108 at 2011-05-03 03:56:43