Trace number 3380724

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 40001.4 5011.21

General information on the benchmark

NameCRAFTED/rbsat/crafted/
forced/rbsat-v2400c266431gyes2.cnf
MD5SUMaac2bd287b8cd6b96cbf7ecc463acf19
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2400
Number of clauses266431
Sum of the clauses size535112
Maximum clause length32
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2266356
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 575

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...490248/instance-3380724-1305490248.cnf
1.49/1.50	c Solving...
40000.66/5010.53	
40000.66/5010.53	*** INTERRUPTED! ***
40000.66/5010.53	*** Shutting down threads...
40001.37/5011.20	s UNKNOWN
40001.37/5011.20	
40001.37/5011.20	c INTERRUPTED   : 1
40001.37/5011.20	c Models        : 0+    
40001.37/5011.20	c Time          : 5010.535s (Solving: 5009.03s 1st Model: 0.00s Unsat: 0.00s)
40001.37/5011.20	c CPU Time      : 40000.710s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3380724-1305490248/watcher-3380724-1305490248 -o /tmp/evaluation-result-3380724-1305490248/solver-3380724-1305490248 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3380724-1305490248.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.03 8.04 8.00 1/412 25039
/proc/meminfo: memFree=20132024/32950940 swapFree=67111356/67111528
[pid=25039] ppid=25037 vsize=23496 CPUtime=0.01 cores=0-7
/proc/25039/stat : 25039 (claspmt) R 25037 25039 24923 0 -1 4202496 1335 0 0 0 1 0 0 0 25 0 1 0 739255582 24059904 1076 33554432000 4194304 4964800 140734079197072 18446744073709551615 215988253381 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/25039/statm: 5874 1078 386 189 0 748 0

[startup+0.0250539 s]
/proc/loadavg: 8.03 8.04 8.00 1/412 25039
/proc/meminfo: memFree=20132024/32950940 swapFree=67111356/67111528
[pid=25039] ppid=25037 vsize=25548 CPUtime=0.02 cores=0-7
/proc/25039/stat : 25039 (claspmt) R 25037 25039 24923 0 -1 4202496 2090 0 0 0 2 0 0 0 25 0 1 0 739255582 26161152 1529 33554432000 4194304 4964800 140734079197072 18446744073709551615 215988253381 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/25039/statm: 6387 1531 386 189 0 1261 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 25548

[startup+0.10111 s]
/proc/loadavg: 8.03 8.04 8.00 1/412 25039
/proc/meminfo: memFree=20132024/32950940 swapFree=67111356/67111528
[pid=25039] ppid=25037 vsize=38000 CPUtime=0.09 cores=0-7
/proc/25039/stat : 25039 (claspmt) R 25037 25039 24923 0 -1 4202496 5853 0 0 0 8 1 0 0 25 0 1 0 739255582 38912000 4592 33554432000 4194304 4964800 140734079197072 18446744073709551615 4607590 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/25039/statm: 9500 4592 399 189 0 4374 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 38000

[startup+0.301004 s]
/proc/loadavg: 8.03 8.04 8.00 1/412 25039
/proc/meminfo: memFree=20132024/32950940 swapFree=67111356/67111528
[pid=25039] ppid=25037 vsize=38000 CPUtime=0.29 cores=0-7
/proc/25039/stat : 25039 (claspmt) R 25037 25039 24923 0 -1 4202496 5853 0 0 0 28 1 0 0 25 0 1 0 739255582 38912000 4592 33554432000 4194304 4964800 140734079197072 18446744073709551615 4607513 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/25039/statm: 9500 4592 399 189 0 4374 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 38000

[startup+0.700934 s]
/proc/loadavg: 8.03 8.04 8.00 1/412 25039
/proc/meminfo: memFree=20132024/32950940 swapFree=67111356/67111528
[pid=25039] ppid=25037 vsize=38000 CPUtime=0.69 cores=0-7
/proc/25039/stat : 25039 (claspmt) R 25037 25039 24923 0 -1 4202496 5855 0 0 0 68 1 0 0 25 0 1 0 739255582 38912000 4594 33554432000 4194304 4964800 140734079197072 18446744073709551615 4600360 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/25039/statm: 9500 4594 400 189 0 4374 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 38000

[startup+1.50079 s]
/proc/loadavg: 8.03 8.04 8.00 2/413 25040
/proc/meminfo: memFree=20112804/32950940 swapFree=67111356/67111528
[pid=25039] ppid=25037 vsize=42096 CPUtime=1.49 cores=0-7
/proc/25039/stat : 25039 (claspmt) R 25037 25039 24923 0 -1 4202496 6876 0 0 0 148 1 0 0 25 0 1 0 739255582 43106304 5615 33554432000 4194304 4964800 140734079197072 18446744073709551615 215988247545 0 0 4096 18946 0 0 0 17 6 0 0 0
/proc/25039/statm: 10524 5615 402 189 0 5398 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 42096

[startup+3.10052 s]
/proc/loadavg: 8.03 8.04 8.00 9/420 25047
/proc/meminfo: memFree=20085344/32950940 swapFree=67111356/67111528
[pid=25039] ppid=25037 vsize=464884 CPUtime=12.67 cores=0-7
/proc/25039/stat : 25039 (claspmt) R 25037 25039 24923 0 -1 4202496 20981 0 0 0 1258 9 0 0 25 0 8 0 739255582 476041216 19161 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670264 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/25039/statm: 116221 19162 464 189 0 110570 0
[pid=25039/tid=25041] ppid=25037 vsize=464884 CPUtime=1.37 cores=0-7
/proc/25039/task/25041/stat : 25041 (claspmt) R 25037 25039 24923 0 -1 4202560 1609 0 0 0 136 1 0 0 17 0 8 0 739255733 476041216 19162 33554432000 4194304 4964800 140734079197072 18446744073709551615 4627401 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25039/tid=25042] ppid=25037 vsize=464884 CPUtime=1.36 cores=0-7
/proc/25039/task/25042/stat : 25042 (claspmt) R 25037 25039 24923 0 -1 4202560 1634 0 0 0 136 0 0 0 17 0 8 0 739255733 476041216 19162 33554432000 4194304 4964800 140734079197072 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=25039/tid=25043] ppid=25037 vsize=464884 CPUtime=1.37 cores=0-7
/proc/25039/task/25043/stat : 25043 (claspmt) R 25037 25039 24923 0 -1 4202560 1695 0 0 0 136 1 0 0 17 0 8 0 739255733 476041216 19162 33554432000 4194304 4964800 140734079197072 18446744073709551615 4325293 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=25039/tid=25044] ppid=25037 vsize=464884 CPUtime=1.36 cores=0-7
/proc/25039/task/25044/stat : 25044 (claspmt) R 25037 25039 24923 0 -1 4202560 1728 0 0 0 136 0 0 0 17 0 8 0 739255733 476041216 19162 33554432000 4194304 4964800 140734079197072 18446744073709551615 4321079 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=25039/tid=25045] ppid=25037 vsize=464884 CPUtime=1.57 cores=0-7
/proc/25039/task/25045/stat : 25045 (claspmt) R 25037 25039 24923 0 -1 4202560 1019 0 0 0 157 0 0 0 20 0 8 0 739255733 476041216 19162 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670264 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25039/tid=25046] ppid=25037 vsize=464884 CPUtime=1.35 cores=0-7
/proc/25039/task/25046/stat : 25046 (claspmt) R 25037 25039 24923 0 -1 4202560 2198 0 0 0 134 1 0 0 17 0 8 0 739255733 476041216 19162 33554432000 4194304 4964800 140734079197072 18446744073709551615 4628014 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25039/tid=25047] ppid=25037 vsize=464884 CPUtime=1.37 cores=0-7
/proc/25039/task/25047/stat : 25047 (claspmt) R 25037 25039 24923 0 -1 4202560 2837 0 0 0 137 0 0 0 17 0 8 0 739255733 476041216 19162 33554432000 4194304 4964800 140734079197072 18446744073709551615 215988253729 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 12.67
Current children cumulated vsize (KiB) 464884

[startup+6.30096 s]
/proc/loadavg: 8.10 8.05 8.01 9/420 25047
/proc/meminfo: memFree=20006296/32950940 swapFree=67111356/67111528
[pid=25039] ppid=25037 vsize=472884 CPUtime=38.22 cores=0-7
/proc/25039/stat : 25039 (claspmt) R 25037 25039 24923 0 -1 4202496 38611 0 0 0 3804 18 0 0 25 0 8 0 739255582 484233216 36791 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/25039/statm: 118221 36791 466 189 0 112570 0
[pid=25039/tid=25041] ppid=25037 vsize=472884 CPUtime=4.56 cores=0-7
/proc/25039/task/25041/stat : 25041 (claspmt) R 25037 25039 24923 0 -1 4202560 3131 0 0 0 455 1 0 0 23 0 8 0 739255733 484233216 36791 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670351 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25039/tid=25042] ppid=25037 vsize=472884 CPUtime=4.56 cores=0-7
/proc/25039/task/25042/stat : 25042 (claspmt) R 25037 25039 24923 0 -1 4202560 3914 0 0 0 454 2 0 0 24 0 8 0 739255733 484233216 36791 33554432000 4194304 4964800 140734079197072 18446744073709551615 4627536 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=25039/tid=25043] ppid=25037 vsize=472884 CPUtime=4.57 cores=0-7
/proc/25039/task/25043/stat : 25043 (claspmt) R 25037 25039 24923 0 -1 4202560 4083 0 0 0 454 3 0 0 24 0 8 0 739255733 484233216 36792 33554432000 4194304 4964800 140734079197072 18446744073709551615 215988253729 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=25039/tid=25044] ppid=25037 vsize=472884 CPUtime=4.56 cores=0-7
/proc/25039/task/25044/stat : 25044 (claspmt) R 25037 25039 24923 0 -1 4202560 3642 0 0 0 455 1 0 0 23 0 8 0 739255733 484233216 36792 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670310 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=25039/tid=25045] ppid=25037 vsize=472884 CPUtime=4.76 cores=0-7
/proc/25039/task/25045/stat : 25045 (claspmt) R 25037 25039 24923 0 -1 4202560 2135 0 0 0 476 0 0 0 25 0 8 0 739255733 484233216 36792 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25039/tid=25046] ppid=25037 vsize=472884 CPUtime=4.51 cores=0-7
/proc/25039/task/25046/stat : 25046 (claspmt) R 25037 25039 24923 0 -1 4202560 4026 0 0 0 449 2 0 0 24 0 8 0 739255733 484233216 36792 33554432000 4194304 4964800 140734079197072 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25039/tid=25047] ppid=25037 vsize=472884 CPUtime=4.57 cores=0-7
/proc/25039/task/25047/stat : 25047 (claspmt) R 25037 25039 24923 0 -1 4202560 7702 0 0 0 454 3 0 0 24 0 8 0 739255733 484233216 36792 33554432000 4194304 4964800 140734079197072 18446744073709551615 215988253381 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 38.22
Current children cumulated vsize (KiB) 472884

[startup+12.7008 s]
/proc/loadavg: 8.10 8.05 8.00 9/420 25047
/proc/meminfo: memFree=19890048/32950940 swapFree=67111356/67111528
[pid=25039] ppid=25037 vsize=492176 CPUtime=89.32 cores=0-7
/proc/25039/stat : 25039 (claspmt) R 25037 25039 24923 0 -1 4202496 66621 0 0 0 8899 33 0 0 25 0 8 0 739255582 503988224 64801 33554432000 4194304 4964800 140734079197072 18446744073709551615 4332387 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/25039/statm: 123044 64802 466 189 0 117393 0
[pid=25039/tid=25041] ppid=25037 vsize=492176 CPUtime=10.96 cores=0-7
/proc/25039/task/25041/stat : 25041 (claspmt) R 25037 25039 24923 0 -1 4202560 6342 0 0 0 1092 4 0 0 25 0 8 0 739255733 503988224 64802 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670321 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25039/tid=25042] ppid=25037 vsize=492176 CPUtime=10.96 cores=0-7
/proc/25039/task/25042/stat : 25042 (claspmt) R 25037 25039 24923 0 -1 4202560 7979 0 0 0 1092 4 0 0 25 0 8 0 739255733 503988224 64802 33554432000 4194304 4964800 140734079197072 18446744073709551615 4361072 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=25039/tid=25043] ppid=25037 vsize=492176 CPUtime=10.97 cores=0-7
/proc/25039/task/25043/stat : 25043 (claspmt) R 25037 25039 24923 0 -1 4202560 6759 0 0 0 1093 4 0 0 25 0 8 0 739255733 503988224 64802 33554432000 4194304 4964800 140734079197072 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=25039/tid=25044] ppid=25037 vsize=492176 CPUtime=10.96 cores=0-7
/proc/25039/task/25044/stat : 25044 (claspmt) R 25037 25039 24923 0 -1 4202560 6270 0 0 0 1093 3 0 0 25 0 8 0 739255733 503988224 64802 33554432000 4194304 4964800 140734079197072 18446744073709551615 4321016 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=25039/tid=25045] ppid=25037 vsize=492176 CPUtime=11.16 cores=0-7
/proc/25039/task/25045/stat : 25045 (claspmt) R 25037 25039 24923 0 -1 4202560 4855 0 0 0 1115 1 0 0 25 0 8 0 739255733 503988224 64802 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670264 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25039/tid=25046] ppid=25037 vsize=492176 CPUtime=10.82 cores=0-7
/proc/25039/task/25046/stat : 25046 (claspmt) R 25037 25039 24923 0 -1 4202560 5984 0 0 0 1079 3 0 0 25 0 8 0 739255733 503988224 64802 33554432000 4194304 4964800 140734079197072 18446744073709551615 4314304 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25039/tid=25047] ppid=25037 vsize=492176 CPUtime=10.97 cores=0-7
/proc/25039/task/25047/stat : 25047 (claspmt) R 25037 25039 24923 0 -1 4202560 15720 0 0 0 1091 6 0 0 18 0 8 0 739255733 503988224 64802 33554432000 4194304 4964800 140734079197072 18446744073709551615 215988253729 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 89.32
Current children cumulated vsize (KiB) 492176

[startup+25.5006 s]
/proc/loadavg: 8.07 8.05 8.00 9/420 25048
/proc/meminfo: memFree=19804612/32950940 swapFree=67111356/67111528
[pid=25039] ppid=25037 vsize=499196 CPUtime=191.52 cores=0-7
/proc/25039/stat : 25039 (claspmt) R 25037 25039 24923 0 -1 4202496 84394 0 0 0 19106 46 0 0 25 0 8 0 739255582 511176704 82574 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/25039/statm: 124799 82574 466 189 0 119148 0
[pid=25039/tid=25041] ppid=25037 vsize=499196 CPUtime=23.75 cores=0-7
/proc/25039/task/25041/stat : 25041 (claspmt) R 25037 25039 24923 0 -1 4202560 8725 0 0 0 2369 6 0 0 25 0 8 0 739255733 511176704 82574 33554432000 4194304 4964800 140734079197072 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25039/tid=25042] ppid=25037 vsize=499196 CPUtime=23.76 cores=0-7
/proc/25039/task/25042/stat : 25042 (claspmt) R 25037 25039 24923 0 -1 4202560 10448 0 0 0 2369 7 0 0 25 0 8 0 739255733 511176704 82574 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=25039/tid=25043] ppid=25037 vsize=499196 CPUtime=23.77 cores=0-7
/proc/25039/task/25043/stat : 25043 (claspmt) R 25037 25039 24923 0 -1 4202560 7977 0 0 0 2372 5 0 0 25 0 8 0 739255733 511176704 82574 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=25039/tid=25044] ppid=25037 vsize=499196 CPUtime=23.76 cores=0-7
/proc/25039/task/25044/stat : 25044 (claspmt) R 25037 25039 24923 0 -1 4202560 7690 0 0 0 2372 4 0 0 25 0 8 0 739255733 511176704 82574 33554432000 4194304 4964800 140734079197072 18446744073709551615 4627951 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=25039/tid=25045] ppid=25037 vsize=499196 CPUtime=23.96 cores=0-7
/proc/25039/task/25045/stat : 25045 (claspmt) R 25037 25039 24923 0 -1 4202560 6512 0 0 0 2394 2 0 0 25 0 8 0 739255733 511176704 82574 33554432000 4194304 4964800 140734079197072 18446744073709551615 4668998 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25039/tid=25046] ppid=25037 vsize=499196 CPUtime=23.44 cores=0-7
/proc/25039/task/25046/stat : 25046 (claspmt) R 25037 25039 24923 0 -1 4202560 7287 0 0 0 2340 4 0 0 25 0 8 0 739255733 511176704 82574 33554432000 4194304 4964800 140734079197072 18446744073709551615 4320944 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25039/tid=25047] ppid=25037 vsize=499196 CPUtime=23.76 cores=0-7
/proc/25039/task/25047/stat : 25047 (claspmt) R 25037 25039 24923 0 -1 4202560 22010 0 0 0 2366 10 0 0 25 0 8 0 739255733 511176704 82574 33554432000 4194304 4964800 140734079197072 18446744073709551615 4668998 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 191.52
Current children cumulated vsize (KiB) 499196

[startup+51.1011 s]
/proc/loadavg: 8.05 8.04 8.00 9/420 25048
/proc/meminfo: memFree=19752876/32950940 swapFree=67111356/67111528
[pid=25039] ppid=25037 vsize=503652 CPUtime=395.96 cores=0-7
/proc/25039/stat : 25039 (claspmt) R 25037 25039 24923 0 -1 4202496 97155 0 0 0 39540 56 0 0 25 0 8 0 739255582 515739648 95328 33554432000 4194304 4964800 140734079197072 18446744073709551615 4627338 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/25039/statm: 125913 95328 466 189 0 120262 0
[pid=25039/tid=25041] ppid=25037 vsize=503652 CPUtime=49.34 cores=0-7
/proc/25039/task/25041/stat : 25041 (claspmt) R 25037 25039 24923 0 -1 4202560 10169 0 0 0 4927 7 0 0 25 0 8 0 739255733 515739648 95328 33554432000 4194304 4964800 140734079197072 18446744073709551615 4687112 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25039/tid=25042] ppid=25037 vsize=503652 CPUtime=49.36 cores=0-7
/proc/25039/task/25042/stat : 25042 (claspmt) R 25037 25039 24923 0 -1 4202560 12398 0 0 0 4928 8 0 0 25 0 8 0 739255733 515739648 95328 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=25039/tid=25043] ppid=25037 vsize=503652 CPUtime=49.36 cores=0-7
/proc/25039/task/25043/stat : 25043 (claspmt) R 25037 25039 24923 0 -1 4202560 8744 0 0 0 4931 5 0 0 25 0 8 0 739255733 515739648 95328 33554432000 4194304 4964800 140734079197072 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=25039/tid=25044] ppid=25037 vsize=503652 CPUtime=49.36 cores=0-7
/proc/25039/task/25044/stat : 25044 (claspmt) R 25037 25039 24923 0 -1 4202560 9224 0 0 0 4931 5 0 0 25 0 8 0 739255733 515739648 95328 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=25039/tid=25045] ppid=25037 vsize=503652 CPUtime=49.57 cores=0-7
/proc/25039/task/25045/stat : 25045 (claspmt) R 25037 25039 24923 0 -1 4202560 7073 0 0 0 4954 3 0 0 25 0 8 0 739255733 515739648 95328 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25039/tid=25046] ppid=25037 vsize=503652 CPUtime=48.69 cores=0-7
/proc/25039/task/25046/stat : 25046 (claspmt) R 25037 25039 24923 0 -1 4202560 7863 0 0 0 4865 4 0 0 25 0 8 0 739255733 515739648 95328 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25039/tid=25047] ppid=25037 vsize=503652 CPUtime=49.36 cores=0-7
/proc/25039/task/25047/stat : 25047 (claspmt) R 25037 25039 24923 0 -1 4202560 25370 0 0 0 4924 12 0 0 25 0 8 0 739255733 515739648 95328 33554432000 4194304 4964800 140734079197072 18446744073709551615 4365931 0 0 4096 18946 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 395.96
Current children cumulated vsize (KiB) 503652

[startup+102.316 s]
/proc/loadavg: 8.10 8.05 8.01 9/420 25050
/proc/meminfo: memFree=19654764/32950940 swapFree=67111356/67111528
[pid=25039] ppid=25037 vsize=653080 CPUtime=804.93 cores=0-7
/proc/25039/stat : 25039 (claspmt) R 25037 25039 24923 0 -1 4202496 121986 0 0 0 80415 78 0 0 25 0 8 0 739255582 668753920 120093 33554432000 4194304 4964800 140734079197072 18446744073709551615 4320909 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/25039/statm: 163270 120094 466 189 0 157619 0
[pid=25039/tid=25041] ppid=25037 vsize=653080 CPUtime=100.55 cores=0-7
/proc/25039/task/25041/stat : 25041 (claspmt) R 25037 25039 24923 0 -1 4202560 14440 0 0 0 10044 11 0 0 25 0 8 0 739255733 668753920 120094 33554432000 4194304 4964800 140734079197072 18446744073709551615 215988253729 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25039/tid=25042] ppid=25037 vsize=653080 CPUtime=100.56 cores=0-7
/proc/25039/task/25042/stat : 25042 (claspmt) R 25037 25039 24923 0 -1 4202560 14066 0 0 0 10047 9 0 0 25 0 8 0 739255733 668753920 120094 33554432000 4194304 4964800 140734079197072 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=25039/tid=25043] ppid=25037 vsize=653080 CPUtime=100.58 cores=0-7
/proc/25039/task/25043/stat : 25043 (claspmt) R 25037 25039 24923 0 -1 4202560 10781 0 0 0 10051 7 0 0 25 0 8 0 739255733 668753920 120094 33554432000 4194304 4964800 140734079197072 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 6 0 0 0

################
# More data... #
################

/proc/25039/task/25046/stat : 25046 (claspmt) R 25037 25039 24923 0 -1 4202560 33994 0 0 0 465595 43 0 0 25 0 8 0 739255733 2861850624 653489 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25039/tid=25047] ppid=25037 vsize=2794776 CPUtime=4627.41 cores=0-7
/proc/25039/task/25047/stat : 25047 (claspmt) R 25037 25039 24923 0 -1 4202560 361661 0 0 0 462350 391 0 0 25 0 8 0 739255733 2861850624 653489 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 37219.8
Current children cumulated vsize (KiB) 2794776

[startup+4722.32 s]
/proc/loadavg: 8.06 8.04 8.00 9/420 25185
/proc/meminfo: memFree=17513616/32950940 swapFree=67111356/67111528
[pid=25039] ppid=25037 vsize=2794776 CPUtime=37699.1 cores=0-7
/proc/25039/stat : 25039 (claspmt) R 25037 25039 24923 0 -1 4202496 660649 0 0 0 3769159 746 0 0 25 0 8 0 739255582 2861850624 653535 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670295 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/25039/statm: 698694 653535 466 189 0 693043 0
[pid=25039/tid=25041] ppid=25037 vsize=2794776 CPUtime=4706.11 cores=0-7
/proc/25039/task/25041/stat : 25041 (claspmt) R 25037 25039 24923 0 -1 4202560 51055 0 0 0 470552 59 0 0 25 0 8 0 739255733 2861850624 653535 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=25039/tid=25042] ppid=25037 vsize=2794776 CPUtime=4719.3 cores=0-7
/proc/25039/task/25042/stat : 25042 (claspmt) R 25037 25039 24923 0 -1 4202560 47708 0 0 0 471883 47 0 0 25 0 8 0 739255733 2861850624 653535 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=25039/tid=25043] ppid=25037 vsize=2794776 CPUtime=4708.16 cores=0-7
/proc/25039/task/25043/stat : 25043 (claspmt) R 25037 25039 24923 0 -1 4202560 30510 0 0 0 470777 39 0 0 25 0 8 0 739255733 2861850624 653535 33554432000 4194304 4964800 140734079197072 18446744073709551615 4321038 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=25039/tid=25044] ppid=25037 vsize=2794776 CPUtime=4720.45 cores=0-7
/proc/25039/task/25044/stat : 25044 (claspmt) R 25037 25039 24923 0 -1 4202560 26301 0 0 0 472015 30 0 0 25 0 8 0 739255733 2861850624 653535 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=25039/tid=25045] ppid=25037 vsize=2794776 CPUtime=4719.95 cores=0-7
/proc/25039/task/25045/stat : 25045 (claspmt) R 25037 25039 24923 0 -1 4202560 32549 0 0 0 471962 33 0 0 25 0 8 0 739255733 2861850624 653535 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25039/tid=25046] ppid=25037 vsize=2794776 CPUtime=4716.27 cores=0-7
/proc/25039/task/25046/stat : 25046 (claspmt) R 25037 25039 24923 0 -1 4202560 33994 0 0 0 471584 43 0 0 25 0 8 0 739255733 2861850624 653535 33554432000 4194304 4964800 140734079197072 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25039/tid=25047] ppid=25037 vsize=2794776 CPUtime=4686.7 cores=0-7
/proc/25039/task/25047/stat : 25047 (claspmt) R 25037 25039 24923 0 -1 4202560 361663 0 0 0 468279 391 0 0 25 0 8 0 739255733 2861850624 653535 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 37699.1
Current children cumulated vsize (KiB) 2794776

[startup+4782.3 s]
/proc/loadavg: 8.06 8.05 8.00 9/420 25187
/proc/meminfo: memFree=17513456/32950940 swapFree=67111356/67111528
[pid=25039] ppid=25037 vsize=2794776 CPUtime=38178.1 cores=0-7
/proc/25039/stat : 25039 (claspmt) R 25037 25039 24923 0 -1 4202496 660732 0 0 0 3817065 746 0 0 25 0 8 0 739255582 2861850624 653618 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/25039/statm: 698694 653618 466 189 0 693043 0
[pid=25039/tid=25041] ppid=25037 vsize=2794776 CPUtime=4766.1 cores=0-7
/proc/25039/task/25041/stat : 25041 (claspmt) R 25037 25039 24923 0 -1 4202560 51055 0 0 0 476551 59 0 0 25 0 8 0 739255733 2861850624 653618 33554432000 4194304 4964800 140734079197072 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=25039/tid=25042] ppid=25037 vsize=2794776 CPUtime=4779.28 cores=0-7
/proc/25039/task/25042/stat : 25042 (claspmt) R 25037 25039 24923 0 -1 4202560 47708 0 0 0 477880 48 0 0 25 0 8 0 739255733 2861850624 653618 33554432000 4194304 4964800 140734079197072 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=25039/tid=25043] ppid=25037 vsize=2794776 CPUtime=4768.15 cores=0-7
/proc/25039/task/25043/stat : 25043 (claspmt) R 25037 25039 24923 0 -1 4202560 30511 0 0 0 476776 39 0 0 25 0 8 0 739255733 2861850624 653618 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=25039/tid=25044] ppid=25037 vsize=2794776 CPUtime=4780.43 cores=0-7
/proc/25039/task/25044/stat : 25044 (claspmt) R 25037 25039 24923 0 -1 4202560 26301 0 0 0 478013 30 0 0 25 0 8 0 739255733 2861850624 653618 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=25039/tid=25045] ppid=25037 vsize=2794776 CPUtime=4779.94 cores=0-7
/proc/25039/task/25045/stat : 25045 (claspmt) R 25037 25039 24923 0 -1 4202560 32629 0 0 0 477961 33 0 0 25 0 8 0 739255733 2861850624 653618 33554432000 4194304 4964800 140734079197072 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25039/tid=25046] ppid=25037 vsize=2794776 CPUtime=4775.45 cores=0-7
/proc/25039/task/25046/stat : 25046 (claspmt) R 25037 25039 24923 0 -1 4202560 33994 0 0 0 477502 43 0 0 25 0 8 0 739255733 2861850624 653618 33554432000 4194304 4964800 140734079197072 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25039/tid=25047] ppid=25037 vsize=2794776 CPUtime=4746.69 cores=0-7
/proc/25039/task/25047/stat : 25047 (claspmt) R 25037 25039 24923 0 -1 4202560 361664 0 0 0 474278 391 0 0 25 0 8 0 739255733 2861850624 653618 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38178.1
Current children cumulated vsize (KiB) 2794776

[startup+4842.3 s]
/proc/loadavg: 8.06 8.05 8.00 9/420 25188
/proc/meminfo: memFree=17511440/32950940 swapFree=67111356/67111528
[pid=25039] ppid=25037 vsize=2794776 CPUtime=38657.3 cores=0-7
/proc/25039/stat : 25039 (claspmt) R 25037 25039 24923 0 -1 4202496 661362 0 0 0 3864980 746 0 0 25 0 8 0 739255582 2861850624 654173 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670346 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/25039/statm: 698694 654173 466 189 0 693043 0
[pid=25039/tid=25041] ppid=25037 vsize=2794776 CPUtime=4826.1 cores=0-7
/proc/25039/task/25041/stat : 25041 (claspmt) R 25037 25039 24923 0 -1 4202560 51055 0 0 0 482551 59 0 0 25 0 8 0 739255733 2861850624 654173 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=25039/tid=25042] ppid=25037 vsize=2794776 CPUtime=4839.26 cores=0-7
/proc/25039/task/25042/stat : 25042 (claspmt) R 25037 25039 24923 0 -1 4202560 48154 0 0 0 483878 48 0 0 25 0 8 0 739255733 2861850624 654173 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=25039/tid=25043] ppid=25037 vsize=2794776 CPUtime=4828.15 cores=0-7
/proc/25039/task/25043/stat : 25043 (claspmt) R 25037 25039 24923 0 -1 4202560 30511 0 0 0 482776 39 0 0 25 0 8 0 739255733 2861850624 654173 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=25039/tid=25044] ppid=25037 vsize=2794776 CPUtime=4840.43 cores=0-7
/proc/25039/task/25044/stat : 25044 (claspmt) R 25037 25039 24923 0 -1 4202560 26419 0 0 0 484013 30 0 0 25 0 8 0 739255733 2861850624 654173 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=25039/tid=25045] ppid=25037 vsize=2794776 CPUtime=4839.93 cores=0-7
/proc/25039/task/25045/stat : 25045 (claspmt) R 25037 25039 24923 0 -1 4202560 32694 0 0 0 483960 33 0 0 25 0 8 0 739255733 2861850624 654173 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25039/tid=25046] ppid=25037 vsize=2794776 CPUtime=4834.62 cores=0-7
/proc/25039/task/25046/stat : 25046 (claspmt) R 25037 25039 24923 0 -1 4202560 33994 0 0 0 483419 43 0 0 25 0 8 0 739255733 2861850624 654173 33554432000 4194304 4964800 140734079197072 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25039/tid=25047] ppid=25037 vsize=2794776 CPUtime=4806.68 cores=0-7
/proc/25039/task/25047/stat : 25047 (claspmt) R 25037 25039 24923 0 -1 4202560 361665 0 0 0 480277 391 0 0 25 0 8 0 739255733 2861850624 654173 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38657.3
Current children cumulated vsize (KiB) 2794776

[startup+4902.3 s]
/proc/loadavg: 7.99 8.04 8.00 9/420 25190
/proc/meminfo: memFree=17508920/32950940 swapFree=67111356/67111528
[pid=25039] ppid=25037 vsize=2794776 CPUtime=39136.4 cores=0-7
/proc/25039/stat : 25039 (claspmt) R 25037 25039 24923 0 -1 4202496 662245 0 0 0 3912894 748 0 0 25 0 8 0 739255582 2861850624 654750 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/25039/statm: 698694 654750 466 189 0 693043 0
[pid=25039/tid=25041] ppid=25037 vsize=2794776 CPUtime=4886.1 cores=0-7
/proc/25039/task/25041/stat : 25041 (claspmt) R 25037 25039 24923 0 -1 4202560 51069 0 0 0 488551 59 0 0 25 0 8 0 739255733 2861850624 654750 33554432000 4194304 4964800 140734079197072 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=25039/tid=25042] ppid=25037 vsize=2794776 CPUtime=4899.25 cores=0-7
/proc/25039/task/25042/stat : 25042 (claspmt) R 25037 25039 24923 0 -1 4202560 48207 0 0 0 489877 48 0 0 25 0 8 0 739255733 2861850624 654750 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=25039/tid=25043] ppid=25037 vsize=2794776 CPUtime=4888.15 cores=0-7
/proc/25039/task/25043/stat : 25043 (claspmt) R 25037 25039 24923 0 -1 4202560 30512 0 0 0 488776 39 0 0 25 0 8 0 739255733 2861850624 654750 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=25039/tid=25044] ppid=25037 vsize=2794776 CPUtime=4900.43 cores=0-7
/proc/25039/task/25044/stat : 25044 (claspmt) R 25037 25039 24923 0 -1 4202560 26569 0 0 0 490013 30 0 0 25 0 8 0 739255733 2861850624 654750 33554432000 4194304 4964800 140734079197072 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=25039/tid=25045] ppid=25037 vsize=2794776 CPUtime=4899.9 cores=0-7
/proc/25039/task/25045/stat : 25045 (claspmt) R 25037 25039 24923 0 -1 4202560 32704 0 0 0 489956 34 0 0 25 0 8 0 739255733 2861850624 654750 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25039/tid=25046] ppid=25037 vsize=2794776 CPUtime=4893.82 cores=0-7
/proc/25039/task/25046/stat : 25046 (claspmt) R 25037 25039 24923 0 -1 4202560 34622 0 0 0 489338 44 0 0 25 0 8 0 739255733 2861850624 654750 33554432000 4194304 4964800 140734079197072 18446744073709551615 4684107 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25039/tid=25047] ppid=25037 vsize=2794776 CPUtime=4866.69 cores=0-7
/proc/25039/task/25047/stat : 25047 (claspmt) R 25037 25039 24923 0 -1 4202560 361692 0 0 0 486277 392 0 0 25 0 8 0 739255733 2861850624 654750 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39136.4
Current children cumulated vsize (KiB) 2794776

[startup+4962.3 s]
/proc/loadavg: 8.06 8.06 8.00 9/420 25191
/proc/meminfo: memFree=17508640/32950940 swapFree=67111356/67111528
[pid=25039] ppid=25037 vsize=2794776 CPUtime=39615.5 cores=0-7
/proc/25039/stat : 25039 (claspmt) R 25037 25039 24923 0 -1 4202496 662344 0 0 0 3960800 749 0 0 25 0 8 0 739255582 2861850624 654820 33554432000 4194304 4964800 140734079197072 18446744073709551615 4668900 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/25039/statm: 698694 654820 466 189 0 693043 0
[pid=25039/tid=25041] ppid=25037 vsize=2794776 CPUtime=4946.1 cores=0-7
/proc/25039/task/25041/stat : 25041 (claspmt) R 25037 25039 24923 0 -1 4202560 51070 0 0 0 494551 59 0 0 25 0 8 0 739255733 2861850624 654820 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=25039/tid=25042] ppid=25037 vsize=2794776 CPUtime=4959.19 cores=0-7
/proc/25039/task/25042/stat : 25042 (claspmt) R 25037 25039 24923 0 -1 4202560 48221 0 0 0 495871 48 0 0 25 0 8 0 739255733 2861850624 654820 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=25039/tid=25043] ppid=25037 vsize=2794776 CPUtime=4948.13 cores=0-7
/proc/25039/task/25043/stat : 25043 (claspmt) R 25037 25039 24923 0 -1 4202560 30512 0 0 0 494774 39 0 0 25 0 8 0 739255733 2861850624 654820 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=25039/tid=25044] ppid=25037 vsize=2794776 CPUtime=4960.43 cores=0-7
/proc/25039/task/25044/stat : 25044 (claspmt) R 25037 25039 24923 0 -1 4202560 26594 0 0 0 496013 30 0 0 25 0 8 0 739255733 2861850624 654820 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=25039/tid=25045] ppid=25037 vsize=2794776 CPUtime=4959.9 cores=0-7
/proc/25039/task/25045/stat : 25045 (claspmt) R 25037 25039 24923 0 -1 4202560 32729 0 0 0 495956 34 0 0 25 0 8 0 739255733 2861850624 654820 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25039/tid=25046] ppid=25037 vsize=2794776 CPUtime=4952.98 cores=0-7
/proc/25039/task/25046/stat : 25046 (claspmt) R 25037 25039 24923 0 -1 4202560 34628 0 0 0 495254 44 0 0 25 0 8 0 739255733 2861850624 654820 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670361 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25039/tid=25047] ppid=25037 vsize=2794776 CPUtime=4926.68 cores=0-7
/proc/25039/task/25047/stat : 25047 (claspmt) R 25037 25039 24923 0 -1 4202560 361692 0 0 0 492276 392 0 0 25 0 8 0 739255733 2861850624 654820 33554432000 4194304 4964800 140734079197072 18446744073709551615 4320981 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39615.5
Current children cumulated vsize (KiB) 2794776



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5010.52 s]
/proc/loadavg: 8.11 8.08 8.01 9/420 25193
/proc/meminfo: memFree=17508364/32950940 swapFree=67111356/67111528
[pid=25039] ppid=25037 vsize=2794776 CPUtime=40000.7 cores=0-7
/proc/25039/stat : 25039 (claspmt) R 25037 25039 24923 0 -1 4202496 662385 0 0 0 3999317 749 0 0 25 0 8 0 739255582 2861850624 654861 33554432000 4194304 4964800 140734079197072 18446744073709551615 4627299 0 0 4096 18946 18446744073709551615 0 0 17 5 0 0 0
/proc/25039/statm: 698694 654861 466 189 0 693043 0
[pid=25039/tid=25041] ppid=25037 vsize=2794776 CPUtime=4994.33 cores=0-7
/proc/25039/task/25041/stat : 25041 (claspmt) R 25037 25039 24923 0 -1 4202560 51073 0 0 0 499374 59 0 0 25 0 8 0 739255733 2861850624 654861 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=25039/tid=25042] ppid=25037 vsize=2794776 CPUtime=5007.41 cores=0-7
/proc/25039/task/25042/stat : 25042 (claspmt) R 25037 25039 24923 0 -1 4202560 48221 0 0 0 500693 48 0 0 25 0 8 0 739255733 2861850624 654861 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670310 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=25039/tid=25043] ppid=25037 vsize=2794776 CPUtime=4996.37 cores=0-7
/proc/25039/task/25043/stat : 25043 (claspmt) R 25037 25039 24923 0 -1 4202560 30513 0 0 0 499598 39 0 0 25 0 8 0 739255733 2861850624 654861 33554432000 4194304 4964800 140734079197072 18446744073709551615 4628046 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=25039/tid=25044] ppid=25037 vsize=2794776 CPUtime=5008.67 cores=0-7
/proc/25039/task/25044/stat : 25044 (claspmt) R 25037 25039 24923 0 -1 4202560 26601 0 0 0 500837 30 0 0 25 0 8 0 739255733 2861850624 654861 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670330 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=25039/tid=25045] ppid=25037 vsize=2794776 CPUtime=5008.13 cores=0-7
/proc/25039/task/25045/stat : 25045 (claspmt) R 25037 25039 24923 0 -1 4202560 32730 0 0 0 500779 34 0 0 25 0 8 0 739255733 2861850624 654861 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25039/tid=25046] ppid=25037 vsize=2794776 CPUtime=5000.54 cores=0-7
/proc/25039/task/25046/stat : 25046 (claspmt) R 25037 25039 24923 0 -1 4202560 34629 0 0 0 500010 44 0 0 25 0 8 0 739255733 2861850624 654861 33554432000 4194304 4964800 140734079197072 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25039/tid=25047] ppid=25037 vsize=2794776 CPUtime=4974.91 cores=0-7
/proc/25039/task/25047/stat : 25047 (claspmt) R 25037 25039 24923 0 -1 4202560 361695 0 0 0 497099 392 0 0 25 0 8 0 739255733 2861850624 654861 33554432000 4194304 4964800 140734079197072 18446744073709551615 4670323 0 0 4096 18946 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40000.7
Current children cumulated vsize (KiB) 2794776

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5011 s]
/proc/loadavg: 8.11 8.08 8.01 9/420 25193
/proc/meminfo: memFree=17508364/32950940 swapFree=67111356/67111528
[pid=25039] ppid=25037 vsize=2778064 CPUtime=40001.2 cores=0-7
/proc/25039/stat : 25039 (claspmt) R 25037 25039 24923 0 -1 4202496 662395 0 0 0 3999368 749 0 0 25 0 1 0 739255582 2844737536 650993 33554432000 4194304 4964800 140734079197072 18446744073709551615 215988249147 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/25039/statm: 694516 650993 476 189 0 688865 0
Current children cumulated CPU time (s) 40001.2
Current children cumulated vsize (KiB) 2778064

[startup+5011.2 s]
/proc/loadavg: 8.11 8.08 8.01 9/420 25193
/proc/meminfo: memFree=17508364/32950940 swapFree=67111356/67111528
[pid=25039] ppid=25037 vsize=595652 CPUtime=40001.4 cores=0-7
/proc/25039/stat : 25039 (claspmt) R 25037 25039 24923 0 -1 4202496 662395 0 0 0 3999377 760 0 0 25 0 1 0 739255582 609947648 42620 33554432000 4194304 4964800 140734079197072 18446744073709551615 215988635783 0 0 4096 18946 0 0 0 17 5 0 0 0
/proc/25039/statm: 148913 42620 476 189 0 143262 0
Current children cumulated CPU time (s) 40001.4
Current children cumulated vsize (KiB) 595652

Child status: 0
Real time (s): 5011.21
CPU time (s): 40001.4
CPU user time (s): 39993.8
CPU system time (s): 7.61584
CPU usage (%): 798.238
Max. virtual memory (cumulated for all children) (KiB): 2794776

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39993.8
system time used= 7.61584
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 662396
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 72758
involuntary context switches= 190521

runsolver used 18.9091 second user time and 49.4705 second system time

The end

Launcher Data 

Begin job on node117 at 2011-05-15 22:10:48
IDJOB=3380724
IDBENCH=71612
IDSOLVER=1865
FILE ID=node117/3380724-1305490248
RUNJOBID= node117-1305488481-24937
PBS_JOBID= 13325399
Free space on /tmp= 73508 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT09/CRAFTED/rbsat/crafted/forced/rbsat-v2400c266431gyes2.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3380724-1305490248/watcher-3380724-1305490248 -o /tmp/evaluation-result-3380724-1305490248/solver-3380724-1305490248 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3380724-1305490248.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= aac2bd287b8cd6b96cbf7ecc463acf19
RANDOM SEED=334629717

node117.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5267.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950940 kB
MemFree:      20132552 kB
Buffers:        334636 kB
Cached:        3743952 kB
SwapCached:          0 kB
Active:       10137428 kB
Inactive:      2321860 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950940 kB
LowFree:      20132552 kB
SwapTotal:    67111528 kB
SwapFree:     67111356 kB
Dirty:            4312 kB
Writeback:           0 kB
AnonPages:     8380624 kB
Mapped:          32536 kB
Slab:           245180 kB
PageTables:      45248 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586996 kB
Committed_AS: 102962248 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73508 MiB
End job on node117 at 2011-05-15 23:34:21