Trace number 3322400

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095? (TO) 40001.9 5040.82

General information on the benchmark

Namecrafted/mosoi/289/
289-unsat-21x33.cnf
MD5SUMf7c29bbc3bef00d1e7cc799553ba4813
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2772
Number of clauses448371
Sum of the clauses size1785168
Maximum clause length4
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 24158
Number of clauses of size 30
Number of clauses of size 4444213
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...382835/instance-3322400-1304382835.cnf
35.49/35.59	c Solving...
40000.44/5039.41	
40000.44/5039.41	*** INTERRUPTED! ***
40000.44/5039.41	*** Shutting down threads...
40001.86/5040.81	s UNKNOWN
40001.86/5040.81	
40001.86/5040.81	c INTERRUPTED   : 1
40001.86/5040.81	c Models        : 0+    
40001.86/5040.81	c Time          : 5039.419s (Solving: 5003.83s 1st Model: 0.00s Unsat: 0.00s)
40001.86/5040.81	c CPU Time      : 40000.490s

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3322400-1304382835/watcher-3322400-1304382835 -o /tmp/evaluation-result-3322400-1304382835/solver-3322400-1304382835 -C 40000 -W 5100 -M 31000 HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3322400-1304382835.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.00 8.43 8.45 1/350 25532
/proc/meminfo: memFree=19488944/32951124 swapFree=67111524/67111528
[pid=25532] ppid=25530 vsize=23204 CPUtime=0 cores=0-7
/proc/25532/stat : 25532 (claspmt) R 25530 25532 23451 0 -1 4202496 1190 0 0 0 0 0 0 0 21 0 1 0 530699352 23760896 1008 33554432000 4194304 4964704 140737244064960 18446744073709551615 233510297344 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/25532/statm: 5801 1009 386 189 0 675 0

[startup+0.09247 s]
/proc/loadavg: 8.00 8.43 8.45 1/350 25532
/proc/meminfo: memFree=19488944/32951124 swapFree=67111524/67111528
[pid=25532] ppid=25530 vsize=37192 CPUtime=0.09 cores=0-7
/proc/25532/stat : 25532 (claspmt) R 25530 25532 23451 0 -1 4202496 5710 0 0 0 8 1 0 0 21 0 1 0 530699352 38084608 4468 33554432000 4194304 4964704 140737244064960 18446744073709551615 4599599 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/25532/statm: 9298 4470 386 189 0 4172 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 37192

[startup+0.100467 s]
/proc/loadavg: 8.00 8.43 8.45 1/350 25532
/proc/meminfo: memFree=19488944/32951124 swapFree=67111524/67111528
[pid=25532] ppid=25530 vsize=42028 CPUtime=0.09 cores=0-7
/proc/25532/stat : 25532 (claspmt) R 25530 25532 23451 0 -1 4202496 6160 0 0 0 8 1 0 0 21 0 1 0 530699352 43036672 4919 33554432000 4194304 4964704 140737244064960 18446744073709551615 233509994965 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/25532/statm: 10507 4939 386 189 0 5381 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 42028

[startup+0.300433 s]
/proc/loadavg: 8.00 8.43 8.45 1/350 25532
/proc/meminfo: memFree=19488944/32951124 swapFree=67111524/67111528
[pid=25532] ppid=25530 vsize=58060 CPUtime=0.29 cores=0-7
/proc/25532/stat : 25532 (claspmt) R 25530 25532 23451 0 -1 4202496 11579 0 0 0 28 1 0 0 22 0 1 0 530699352 59453440 9703 33554432000 4194304 4964704 140737244064960 18446744073709551615 4607590 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/25532/statm: 14515 9703 399 189 0 9389 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 58060

[startup+0.700359 s]
/proc/loadavg: 8.00 8.43 8.45 1/350 25532
/proc/meminfo: memFree=19488944/32951124 swapFree=67111524/67111528
[pid=25532] ppid=25530 vsize=58060 CPUtime=0.69 cores=0-7
/proc/25532/stat : 25532 (claspmt) R 25530 25532 23451 0 -1 4202496 11579 0 0 0 68 1 0 0 23 0 1 0 530699352 59453440 9703 33554432000 4194304 4964704 140737244064960 18446744073709551615 4607536 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/25532/statm: 14515 9703 399 189 0 9389 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 58060

[startup+1.50022 s]
/proc/loadavg: 8.00 8.43 8.45 2/351 25533
/proc/meminfo: memFree=19452240/32951124 swapFree=67111524/67111528
[pid=25532] ppid=25530 vsize=58060 CPUtime=1.49 cores=0-7
/proc/25532/stat : 25532 (claspmt) R 25530 25532 23451 0 -1 4202496 11579 0 0 0 148 1 0 0 25 0 1 0 530699352 59453440 9703 33554432000 4194304 4964704 140737244064960 18446744073709551615 4607590 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/25532/statm: 14515 9703 399 189 0 9389 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 58060

[startup+3.10095 s]
/proc/loadavg: 8.00 8.43 8.45 2/351 25533
/proc/meminfo: memFree=19451000/32951124 swapFree=67111524/67111528
[pid=25532] ppid=25530 vsize=58984 CPUtime=3.09 cores=0-7
/proc/25532/stat : 25532 (claspmt) R 25530 25532 23451 0 -1 4202496 11802 0 0 0 308 1 0 0 25 0 1 0 530699352 60399616 9926 33554432000 4194304 4964704 140737244064960 18446744073709551615 4600360 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/25532/statm: 14746 9926 400 189 0 9620 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 58984

[startup+6.30036 s]
/proc/loadavg: 7.44 8.31 8.41 2/351 25533
/proc/meminfo: memFree=19446420/32951124 swapFree=67111524/67111528
[pid=25532] ppid=25530 vsize=65472 CPUtime=6.29 cores=0-7
/proc/25532/stat : 25532 (claspmt) R 25530 25532 23451 0 -1 4202496 13429 0 0 0 627 2 0 0 25 0 1 0 530699352 67043328 11553 33554432000 4194304 4964704 140737244064960 18446744073709551615 4600360 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/25532/statm: 16368 11553 400 189 0 11242 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 65472

[startup+12.7002 s]
/proc/loadavg: 6.92 8.19 8.37 2/351 25534
/proc/meminfo: memFree=19434300/32951124 swapFree=67111524/67111528
[pid=25532] ppid=25530 vsize=78312 CPUtime=12.69 cores=0-7
/proc/25532/stat : 25532 (claspmt) R 25530 25532 23451 0 -1 4202496 16632 0 0 0 1267 2 0 0 25 0 1 0 530699352 80191488 14756 33554432000 4194304 4964704 140737244064960 18446744073709551615 4600360 0 0 4096 18946 0 0 0 17 4 0 0 0
/proc/25532/statm: 19578 14756 400 189 0 14452 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 78312

[startup+25.5009 s]
/proc/loadavg: 5.61 7.83 8.25 2/351 25534
/proc/meminfo: memFree=19420356/32951124 swapFree=67111524/67111528
[pid=25532] ppid=25530 vsize=93148 CPUtime=25.49 cores=0-7
/proc/25532/stat : 25532 (claspmt) R 25530 25532 23451 0 -1 4202496 20348 0 0 0 2546 3 0 0 25 0 1 0 530699352 95383552 18472 33554432000 4194304 4964704 140737244064960 18446744073709551615 4607590 0 0 4096 18946 0 0 0 17 0 0 0 0
/proc/25532/statm: 23287 18472 400 189 0 18161 0
Current children cumulated CPU time (s) 25.49
Current children cumulated vsize (KiB) 93148

[startup+51.1013 s]
/proc/loadavg: 5.58 7.63 8.17 9/358 25542
/proc/meminfo: memFree=18887696/32951124 swapFree=67111524/67111528
[pid=25532] ppid=25530 vsize=1020240 CPUtime=157.18 cores=0-7
/proc/25532/stat : 25532 (claspmt) R 25530 25532 23451 0 -1 4202496 157176 0 0 0 15641 77 0 0 25 0 8 0 530699352 1044725760 153446 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/25532/statm: 255060 153446 466 189 0 249409 0
[pid=25532/tid=25535] ppid=25530 vsize=1020240 CPUtime=15.2 cores=0-7
/proc/25532/task/25535/stat : 25535 (claspmt) R 25530 25532 23451 0 -1 4202560 18568 0 0 0 1510 10 0 0 25 0 8 0 530702912 1044725760 153446 33554432000 4194304 4964704 140737244064960 18446744073709551615 4364587 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=25532/tid=25536] ppid=25530 vsize=1020240 CPUtime=15.05 cores=0-7
/proc/25532/task/25536/stat : 25536 (claspmt) R 25530 25532 23451 0 -1 4202560 16993 0 0 0 1497 8 0 0 25 0 8 0 530702912 1044725760 153446 33554432000 4194304 4964704 140737244064960 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25532/tid=25537] ppid=25530 vsize=1020240 CPUtime=15.22 cores=0-7
/proc/25532/task/25537/stat : 25537 (claspmt) R 25530 25532 23451 0 -1 4202560 13879 0 0 0 1515 7 0 0 25 0 8 0 530702912 1044725760 153446 33554432000 4194304 4964704 140737244064960 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=25532/tid=25538] ppid=25530 vsize=1020240 CPUtime=15.21 cores=0-7
/proc/25532/task/25538/stat : 25538 (claspmt) R 25530 25532 23451 0 -1 4202560 17270 0 0 0 1511 10 0 0 25 0 8 0 530702912 1044725760 153446 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=25532/tid=25539] ppid=25530 vsize=1020240 CPUtime=15.28 cores=0-7
/proc/25532/task/25539/stat : 25539 (claspmt) R 25530 25532 23451 0 -1 4202560 15053 0 0 0 1521 7 0 0 25 0 8 0 530702912 1044725760 153446 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25532/tid=25540] ppid=25530 vsize=1020240 CPUtime=15.2 cores=0-7
/proc/25532/task/25540/stat : 25540 (claspmt) R 25530 25532 23451 0 -1 4202560 16593 0 0 0 1512 8 0 0 25 0 8 0 530702912 1044725760 153446 33554432000 4194304 4964704 140737244064960 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25532/tid=25541] ppid=25530 vsize=1020240 CPUtime=15.22 cores=0-7
/proc/25532/task/25541/stat : 25541 (claspmt) R 25530 25532 23451 0 -1 4202560 18574 0 0 0 1512 10 0 0 25 0 8 0 530702912 1044725760 153446 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 157.18
Current children cumulated vsize (KiB) 1020240

[startup+102.313 s]
/proc/loadavg: 6.96 7.68 8.16 9/358 25543
/proc/meminfo: memFree=18695180/32951124 swapFree=67111524/67111528
[pid=25532] ppid=25530 vsize=1054996 CPUtime=566.14 cores=0-7
/proc/25532/stat : 25532 (claspmt) R 25530 25532 23451 0 -1 4202496 203295 0 0 0 56502 112 0 0 18 0 8 0 530699352 1080315904 199521 33554432000 4194304 4964704 140737244064960 18446744073709551615 4684027 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/25532/statm: 263749 199521 466 189 0 258098 0
[pid=25532/tid=25535] ppid=25530 vsize=1054996 CPUtime=66.38 cores=0-7
/proc/25532/task/25535/stat : 25535 (claspmt) R 25530 25532 23451 0 -1 4202560 24644 0 0 0 6622 16 0 0 25 0 8 0 530702912 1080315904 199521 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670219 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=25532/tid=25536] ppid=25530 vsize=1054996 CPUtime=65.66 cores=0-7
/proc/25532/task/25536/stat : 25536 (claspmt) R 25530 25532 23451 0 -1 4202560 22998 0 0 0 6553 13 0 0 25 0 8 0 530702912 1080315904 199521 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25532/tid=25537] ppid=25530 vsize=1054996 CPUtime=66.42 cores=0-7
/proc/25532/task/25537/stat : 25537 (claspmt) R 25530 25532 23451 0 -1 4202560 17439 0 0 0 6632 10 0 0 25 0 8 0 530702912 1080315904 199521 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=25532/tid=25538] ppid=25530 vsize=1054996 CPUtime=66.42 cores=0-7
/proc/25532/task/25538/stat : 25538 (claspmt) R 25530 25532 23451 0 -1 4202560 20786 0 0 0 6629 13 0 0 25 0 8 0 530702912 1080315904 199521 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=25532/tid=25539] ppid=25530 vsize=1054996 CPUtime=66.49 cores=0-7
/proc/25532/task/25539/stat : 25539 (claspmt) R 25530 25532 23451 0 -1 4202560 19921 0 0 0 6637 12 0 0 25 0 8 0 530702912 1080315904 199521 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25532/tid=25540] ppid=25530 vsize=1054996 CPUtime=66.39 cores=0-7
/proc/25532/task/25540/stat : 25540 (claspmt) R 25530 25532 23451 0 -1 4202560 20202 0 0 0 6629 10 0 0 25 0 8 0 530702912 1080315904 199521 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670230 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25532/tid=25541] ppid=25530 vsize=1054996 CPUtime=66.41 cores=0-7
/proc/25532/task/25541/stat : 25541 (claspmt) R 25530 25532 23451 0 -1 4202560 29895 0 0 0 6623 18 0 0 25 0 8 0 530702912 1080315904 199521 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 566.14
Current children cumulated vsize (KiB) 1054996

[startup+162.3 s]
/proc/loadavg: 7.67 7.75 8.16 9/358 25545
/proc/meminfo: memFree=18494012/32951124 swapFree=67111524/67111528
[pid=25532] ppid=25530 vsize=1216568 CPUtime=1045.26 cores=0-7
/proc/25532/stat : 25532 (claspmt) R 25530 25532 23451 0 -1 4202496 253398 0 0 0 104370 156 0 0 25 0 8 0 530699352 1245765632 249482 33554432000 4194304 4964704 140737244064960 18446744073709551615 4332343 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/25532/statm: 304142 249482 466 189 0 298491 0
[pid=25532/tid=25535] ppid=25530 vsize=1216568 CPUtime=126.35 cores=0-7
/proc/25532/task/25535/stat : 25535 (claspmt) R 25530 25532 23451 0 -1 4202560 31457 0 0 0 12613 22 0 0 25 0 8 0 530702912 1245765632 249482 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=25532/tid=25536] ppid=25530 vsize=1216568 CPUtime=124.95 cores=0-7
/proc/25532/task/25536/stat : 25536 (claspmt) R 25530 25532 23451 0 -1 4202560 29750 0 0 0 12477 18 0 0 25 0 8 0 530702912 1245765632 249482 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25532/tid=25537] ppid=25530 vsize=1216568 CPUtime=126.39 cores=0-7
/proc/25532/task/25537/stat : 25537 (claspmt) R 25530 25532 23451 0 -1 4202560 20971 0 0 0 12627 12 0 0 25 0 8 0 530702912 1245765632 249482 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=25532/tid=25538] ppid=25530 vsize=1216568 CPUtime=126.4 cores=0-7
/proc/25532/task/25538/stat : 25538 (claspmt) R 25530 25532 23451 0 -1 4202560 24505 0 0 0 12624 16 0 0 25 0 8 0 530702912 1245765632 249482 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=25532/tid=25539] ppid=25530 vsize=1216568 CPUtime=126.48 cores=0-7
/proc/25532/task/25539/stat : 25539 (claspmt) R 25530 25532 23451 0 -1 4202560 23264 0 0 0 12632 16 0 0 25 0 8 0 530702912 1245765632 249482 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25532/tid=25540] ppid=25530 vsize=1216568 CPUtime=126.34 cores=0-7
/proc/25532/task/25540/stat : 25540 (claspmt) R 25530 25532 23451 0 -1 4202560 24675 0 0 0 12621 13 0 0 25 0 8 0 530702912 1245765632 249482 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670250 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25532/tid=25541] ppid=25530 vsize=1216568 CPUtime=126.4 cores=0-7
/proc/25532/task/25541/stat : 25541 (claspmt) R 25530 25532 23451 0 -1 4202560 42520 0 0 0 12610 30 0 0 25 0 8 0 530702912 1245765632 249482 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1045.26
Current children cumulated vsize (KiB) 1216568

[startup+222.301 s]
/proc/loadavg: 7.92 7.81 8.15 9/358 25546
/proc/meminfo: memFree=18306244/32951124 swapFree=67111524/67111528
[pid=25532] ppid=25530 vsize=1492676 CPUtime=1524.51 cores=0-7
/proc/25532/stat : 25532 (claspmt) R 25530 25532 23451 0 -1 4202496 300748 0 0 0 152242 209 0 0 25 0 8 0 530699352 1528500224 296727 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/25532/statm: 373169 296727 466 189 0 367518 0
[pid=25532/tid=25535] ppid=25530 vsize=1492676 CPUtime=186.34 cores=0-7
/proc/25532/task/25535/stat : 25535 (claspmt) R 25530 25532 23451 0 -1 4202560 37834 0 0 0 18603 31 0 0 25 0 8 0 530702912 1528500224 296727 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=25532/tid=25536] ppid=25530 vsize=1492676 CPUtime=184.24 cores=0-7
/proc/25532/task/25536/stat : 25536 (claspmt) R 25530 25532 23451 0 -1 4202560 35586 0 0 0 18397 27 0 0 25 0 8 0 530702912 1528500224 296727 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25532/tid=25537] ppid=25530 vsize=1492676 CPUtime=186.4 cores=0-7
/proc/25532/task/25537/stat : 25537 (claspmt) R 25530 25532 23451 0 -1 4202560 25432 0 0 0 18622 18 0 0 25 0 8 0 530702912 1528500224 296727 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=25532/tid=25538] ppid=25530 vsize=1492676 CPUtime=186.4 cores=0-7
/proc/25532/task/25538/stat : 25538 (claspmt) R 25530 25532 23451 0 -1 4202560 28364 0 0 0 18620 20 0 0 25 0 8 0 530702912 1528500224 296727 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=25532/tid=25539] ppid=25530 vsize=1492676 CPUtime=186.47 cores=0-7
/proc/25532/task/25539/stat : 25539 (claspmt) R 25530 25532 23451 0 -1 4202560 26314 0 0 0 18626 21 0 0 25 0 8 0 530702912 1528500224 296727 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25532/tid=25540] ppid=25530 vsize=1492676 CPUtime=186.33 cores=0-7
/proc/25532/task/25540/stat : 25540 (claspmt) R 25530 25532 23451 0 -1 4202560 28089 0 0 0 18616 17 0 0 25 0 8 0 530702912 1528500224 296727 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25532/tid=25541] ppid=25530 vsize=1492676 CPUtime=186.39 cores=0-7
/proc/25532/task/25541/stat : 25541 (claspmt) R 25530 25532 23451 0 -1 4202560 53373 0 0 0 18599 40 0 0 25 0 8 0 530702912 1528500224 296727 33554432000 4194304 4964704 140737244064960 18446744073709551615 4321102 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1524.51
Current children cumulated vsize (KiB) 1492676

################
# More data... #
################

[pid=25532] ppid=25530 vsize=3828856 CPUtime=37946.7 cores=0-7
/proc/25532/stat : 25532 (claspmt) R 25530 25532 23451 0 -1 4202496 878711 0 0 0 3793741 926 0 0 25 0 8 0 530699352 3920748544 861770 33554432000 4194304 4964704 140737244064960 18446744073709551615 4342206 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/25532/statm: 957214 861770 466 189 0 951563 0
[pid=25532/tid=25535] ppid=25530 vsize=3828856 CPUtime=4735.6 cores=0-7
/proc/25532/task/25535/stat : 25535 (claspmt) R 25530 25532 23451 0 -1 4202560 82843 0 0 0 473453 107 0 0 25 0 8 0 530702912 3920748544 861770 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25532/tid=25536] ppid=25530 vsize=3828856 CPUtime=4739.73 cores=0-7
/proc/25532/task/25536/stat : 25536 (claspmt) R 25530 25532 23451 0 -1 4202560 124081 0 0 0 473861 112 0 0 25 0 8 0 530702912 3920748544 861770 33554432000 4194304 4964704 140737244064960 18446744073709551615 4320944 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=25532/tid=25537] ppid=25530 vsize=3828856 CPUtime=4720.14 cores=0-7
/proc/25532/task/25537/stat : 25537 (claspmt) R 25530 25532 23451 0 -1 4202560 59446 0 0 0 471944 70 0 0 25 0 8 0 530702912 3920748544 861770 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=25532/tid=25538] ppid=25530 vsize=3828856 CPUtime=4738.28 cores=0-7
/proc/25532/task/25538/stat : 25538 (claspmt) R 25530 25532 23451 0 -1 4202560 56868 0 0 0 473777 51 0 0 25 0 8 0 530702912 3920748544 861770 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=25532/tid=25539] ppid=25530 vsize=3828856 CPUtime=4740.13 cores=0-7
/proc/25532/task/25539/stat : 25539 (claspmt) R 25530 25532 23451 0 -1 4202560 58712 0 0 0 473949 64 0 0 25 0 8 0 530702912 3920748544 861770 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25532/tid=25540] ppid=25530 vsize=3828856 CPUtime=4745.18 cores=0-7
/proc/25532/task/25540/stat : 25540 (claspmt) R 25530 25532 23451 0 -1 4202560 69696 0 0 0 474451 67 0 0 25 0 8 0 530702912 3920748544 861770 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670260 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25532/tid=25541] ppid=25530 vsize=3828856 CPUtime=4746.02 cores=0-7
/proc/25532/task/25541/stat : 25541 (claspmt) R 25530 25532 23451 0 -1 4202560 197795 0 0 0 474386 216 0 0 25 0 8 0 530702912 3920748544 861770 33554432000 4194304 4964704 140737244064960 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 37946.7
Current children cumulated vsize (KiB) 3828856

[startup+4842.3 s]
/proc/loadavg: 8.03 8.04 8.00 9/358 25681
/proc/meminfo: memFree=15989244/32951124 swapFree=67111524/67111528
[pid=25532] ppid=25530 vsize=3828856 CPUtime=38425.9 cores=0-7
/proc/25532/stat : 25532 (claspmt) R 25530 25532 23451 0 -1 4202496 891307 0 0 0 3841652 942 0 0 25 0 8 0 530699352 3920748544 874366 33554432000 4194304 4964704 140737244064960 18446744073709551615 4325293 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/25532/statm: 957214 874366 466 189 0 951563 0
[pid=25532/tid=25535] ppid=25530 vsize=3828856 CPUtime=4794.89 cores=0-7
/proc/25532/task/25535/stat : 25535 (claspmt) R 25530 25532 23451 0 -1 4202560 82845 0 0 0 479382 107 0 0 25 0 8 0 530702912 3920748544 874366 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25532/tid=25536] ppid=25530 vsize=3828856 CPUtime=4799.74 cores=0-7
/proc/25532/task/25536/stat : 25536 (claspmt) R 25530 25532 23451 0 -1 4202560 125803 0 0 0 479860 114 0 0 25 0 8 0 530702912 3920748544 874366 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=25532/tid=25537] ppid=25530 vsize=3828856 CPUtime=4780.15 cores=0-7
/proc/25532/task/25537/stat : 25537 (claspmt) R 25530 25532 23451 0 -1 4202560 59448 0 0 0 477945 70 0 0 25 0 8 0 530702912 3920748544 874366 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=25532/tid=25538] ppid=25530 vsize=3828856 CPUtime=4798.29 cores=0-7
/proc/25532/task/25538/stat : 25538 (claspmt) R 25530 25532 23451 0 -1 4202560 57025 0 0 0 479777 52 0 0 25 0 8 0 530702912 3920748544 874366 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=25532/tid=25539] ppid=25530 vsize=3828856 CPUtime=4800.13 cores=0-7
/proc/25532/task/25539/stat : 25539 (claspmt) R 25530 25532 23451 0 -1 4202560 58889 0 0 0 479949 64 0 0 25 0 8 0 530702912 3920748544 874366 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670248 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25532/tid=25540] ppid=25530 vsize=3828856 CPUtime=4805.16 cores=0-7
/proc/25532/task/25540/stat : 25540 (claspmt) R 25530 25532 23451 0 -1 4202560 69998 0 0 0 480449 67 0 0 25 0 8 0 530702912 3920748544 874366 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25532/tid=25541] ppid=25530 vsize=3828856 CPUtime=4806.01 cores=0-7
/proc/25532/task/25541/stat : 25541 (claspmt) R 25530 25532 23451 0 -1 4202560 198703 0 0 0 480384 217 0 0 25 0 8 0 530702912 3920748544 874366 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 38425.9
Current children cumulated vsize (KiB) 3828856

[startup+4902.3 s]
/proc/loadavg: 8.04 8.05 8.00 9/358 25683
/proc/meminfo: memFree=15976056/32951124 swapFree=67111524/67111528
[pid=25532] ppid=25530 vsize=3841160 CPUtime=38905.2 cores=0-7
/proc/25532/stat : 25532 (claspmt) R 25530 25532 23451 0 -1 4202496 894663 0 0 0 3889575 946 0 0 25 0 8 0 530699352 3933347840 877722 33554432000 4194304 4964704 140737244064960 18446744073709551615 4659143 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/25532/statm: 960290 877722 466 189 0 954639 0
[pid=25532/tid=25535] ppid=25530 vsize=3841160 CPUtime=4854.19 cores=0-7
/proc/25532/task/25535/stat : 25535 (claspmt) R 25530 25532 23451 0 -1 4202560 82987 0 0 0 485312 107 0 0 25 0 8 0 530702912 3933347840 877722 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25532/tid=25536] ppid=25530 vsize=3841160 CPUtime=4859.74 cores=0-7
/proc/25532/task/25536/stat : 25536 (claspmt) R 25530 25532 23451 0 -1 4202560 126772 0 0 0 485858 116 0 0 25 0 8 0 530702912 3933347840 877722 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=25532/tid=25537] ppid=25530 vsize=3841160 CPUtime=4840.13 cores=0-7
/proc/25532/task/25537/stat : 25537 (claspmt) R 25530 25532 23451 0 -1 4202560 59450 0 0 0 483942 71 0 0 25 0 8 0 530702912 3933347840 877722 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=25532/tid=25538] ppid=25530 vsize=3841160 CPUtime=4858.29 cores=0-7
/proc/25532/task/25538/stat : 25538 (claspmt) R 25530 25532 23451 0 -1 4202560 57104 0 0 0 485777 52 0 0 25 0 8 0 530702912 3933347840 877722 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=25532/tid=25539] ppid=25530 vsize=3841160 CPUtime=4860.12 cores=0-7
/proc/25532/task/25539/stat : 25539 (claspmt) R 25530 25532 23451 0 -1 4202560 61020 0 0 0 485948 64 0 0 25 0 8 0 530702912 3933347840 877722 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25532/tid=25540] ppid=25530 vsize=3841160 CPUtime=4865.16 cores=0-7
/proc/25532/task/25540/stat : 25540 (claspmt) R 25530 25532 23451 0 -1 4202560 69998 0 0 0 486448 68 0 0 25 0 8 0 530702912 3933347840 877722 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25532/tid=25541] ppid=25530 vsize=3841160 CPUtime=4866.01 cores=0-7
/proc/25532/task/25541/stat : 25541 (claspmt) R 25530 25532 23451 0 -1 4202560 198703 0 0 0 486384 217 0 0 25 0 8 0 530702912 3933347840 877722 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 38905.2
Current children cumulated vsize (KiB) 3841160

[startup+4962.3 s]
/proc/loadavg: 8.09 8.05 8.00 9/358 25685
/proc/meminfo: memFree=15950464/32951124 swapFree=67111524/67111528
[pid=25532] ppid=25530 vsize=3841160 CPUtime=39384.5 cores=0-7
/proc/25532/stat : 25532 (claspmt) R 25530 25532 23451 0 -1 4202496 901087 0 0 0 3937491 955 0 0 25 0 8 0 530699352 3933347840 884146 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/25532/statm: 960290 884146 466 189 0 954639 0
[pid=25532/tid=25535] ppid=25530 vsize=3841160 CPUtime=4913.47 cores=0-7
/proc/25532/task/25535/stat : 25535 (claspmt) R 25530 25532 23451 0 -1 4202560 83980 0 0 0 491239 108 0 0 25 0 8 0 530702912 3933347840 884146 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25532/tid=25536] ppid=25530 vsize=3841160 CPUtime=4919.7 cores=0-7
/proc/25532/task/25536/stat : 25536 (claspmt) R 25530 25532 23451 0 -1 4202560 127255 0 0 0 491854 116 0 0 25 0 8 0 530702912 3933347840 884146 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=25532/tid=25537] ppid=25530 vsize=3841160 CPUtime=4900.13 cores=0-7
/proc/25532/task/25537/stat : 25537 (claspmt) R 25530 25532 23451 0 -1 4202560 59473 0 0 0 489942 71 0 0 25 0 8 0 530702912 3933347840 884146 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=25532/tid=25538] ppid=25530 vsize=3841160 CPUtime=4918.29 cores=0-7
/proc/25532/task/25538/stat : 25538 (claspmt) R 25530 25532 23451 0 -1 4202560 57443 0 0 0 491776 53 0 0 25 0 8 0 530702912 3933347840 884146 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=25532/tid=25539] ppid=25530 vsize=3841160 CPUtime=4920.13 cores=0-7
/proc/25532/task/25539/stat : 25539 (claspmt) R 25530 25532 23451 0 -1 4202560 61590 0 0 0 491948 65 0 0 25 0 8 0 530702912 3933347840 884146 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25532/tid=25540] ppid=25530 vsize=3841160 CPUtime=4925.15 cores=0-7
/proc/25532/task/25540/stat : 25540 (claspmt) R 25530 25532 23451 0 -1 4202560 70164 0 0 0 492447 68 0 0 25 0 8 0 530702912 3933347840 884146 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25532/tid=25541] ppid=25530 vsize=3841160 CPUtime=4926.01 cores=0-7
/proc/25532/task/25541/stat : 25541 (claspmt) R 25530 25532 23451 0 -1 4202560 199711 0 0 0 492382 219 0 0 25 0 8 0 530702912 3933347840 884146 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 39384.5
Current children cumulated vsize (KiB) 3841160

[startup+5022.3 s]
/proc/loadavg: 8.10 8.06 8.00 9/358 25686
/proc/meminfo: memFree=15948568/32951124 swapFree=67111524/67111528
[pid=25532] ppid=25530 vsize=3841584 CPUtime=39863.8 cores=0-7
/proc/25532/stat : 25532 (claspmt) R 25530 25532 23451 0 -1 4202496 901528 0 0 0 3985419 956 0 0 25 0 8 0 530699352 3933782016 884587 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670419 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/25532/statm: 960396 884587 466 189 0 954745 0
[pid=25532/tid=25535] ppid=25530 vsize=3841584 CPUtime=4972.77 cores=0-7
/proc/25532/task/25535/stat : 25535 (claspmt) R 25530 25532 23451 0 -1 4202560 84073 0 0 0 497168 109 0 0 25 0 8 0 530702912 3933782016 884587 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25532/tid=25536] ppid=25530 vsize=3841584 CPUtime=4979.7 cores=0-7
/proc/25532/task/25536/stat : 25536 (claspmt) R 25530 25532 23451 0 -1 4202560 127258 0 0 0 497854 116 0 0 25 0 8 0 530702912 3933782016 884587 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=25532/tid=25537] ppid=25530 vsize=3841584 CPUtime=4960.13 cores=0-7
/proc/25532/task/25537/stat : 25537 (claspmt) R 25530 25532 23451 0 -1 4202560 59543 0 0 0 495942 71 0 0 25 0 8 0 530702912 3933782016 884587 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=25532/tid=25538] ppid=25530 vsize=3841584 CPUtime=4978.29 cores=0-7
/proc/25532/task/25538/stat : 25538 (claspmt) R 25530 25532 23451 0 -1 4202560 57468 0 0 0 497776 53 0 0 25 0 8 0 530702912 3933782016 884587 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=25532/tid=25539] ppid=25530 vsize=3841584 CPUtime=4980.13 cores=0-7
/proc/25532/task/25539/stat : 25539 (claspmt) R 25530 25532 23451 0 -1 4202560 61653 0 0 0 497948 65 0 0 25 0 8 0 530702912 3933782016 884587 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25532/tid=25540] ppid=25530 vsize=3841584 CPUtime=4985.14 cores=0-7
/proc/25532/task/25540/stat : 25540 (claspmt) R 25530 25532 23451 0 -1 4202560 70346 0 0 0 498446 68 0 0 25 0 8 0 530702912 3933782016 884587 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25532/tid=25541] ppid=25530 vsize=3841584 CPUtime=4986.01 cores=0-7
/proc/25532/task/25541/stat : 25541 (claspmt) R 25530 25532 23451 0 -1 4202560 199713 0 0 0 498382 219 0 0 25 0 8 0 530702912 3933782016 884587 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670184 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 39863.8
Current children cumulated vsize (KiB) 3841584



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5039.4 s]
/proc/loadavg: 8.07 8.05 8.00 9/358 25687
/proc/meminfo: memFree=15946444/32951124 swapFree=67111524/67111528
[pid=25532] ppid=25530 vsize=3841880 CPUtime=40000.4 cores=0-7
/proc/25532/stat : 25532 (claspmt) R 25530 25532 23451 0 -1 4202496 902341 0 0 0 3999087 957 0 0 25 0 8 0 530699352 3934085120 885207 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 18446744073709551615 0 0 17 2 0 0 0
/proc/25532/statm: 960470 885207 466 189 0 954819 0
[pid=25532/tid=25535] ppid=25530 vsize=3841880 CPUtime=4989.66 cores=0-7
/proc/25532/task/25535/stat : 25535 (claspmt) R 25530 25532 23451 0 -1 4202560 84095 0 0 0 498857 109 0 0 25 0 8 0 530702912 3934085120 885207 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=25532/tid=25536] ppid=25530 vsize=3841880 CPUtime=4996.82 cores=0-7
/proc/25532/task/25536/stat : 25536 (claspmt) R 25530 25532 23451 0 -1 4202560 127259 0 0 0 499566 116 0 0 25 0 8 0 530702912 3934085120 885207 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=25532/tid=25537] ppid=25530 vsize=3841880 CPUtime=4977.25 cores=0-7
/proc/25532/task/25537/stat : 25537 (claspmt) R 25530 25532 23451 0 -1 4202560 59555 0 0 0 497654 71 0 0 25 0 8 0 530702912 3934085120 885207 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670269 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=25532/tid=25538] ppid=25530 vsize=3841880 CPUtime=4995.4 cores=0-7
/proc/25532/task/25538/stat : 25538 (claspmt) R 25530 25532 23451 0 -1 4202560 57468 0 0 0 499487 53 0 0 25 0 8 0 530702912 3934085120 885207 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670454 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=25532/tid=25539] ppid=25530 vsize=3841880 CPUtime=4997.24 cores=0-7
/proc/25532/task/25539/stat : 25539 (claspmt) R 25530 25532 23451 0 -1 4202560 61667 0 0 0 499659 65 0 0 25 0 8 0 530702912 3934085120 885207 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=25532/tid=25540] ppid=25530 vsize=3841880 CPUtime=5002.24 cores=0-7
/proc/25532/task/25540/stat : 25540 (claspmt) R 25530 25532 23451 0 -1 4202560 71045 0 0 0 500156 68 0 0 25 0 8 0 530702912 3934085120 885207 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=25532/tid=25541] ppid=25530 vsize=3841880 CPUtime=5003.12 cores=0-7
/proc/25532/task/25541/stat : 25541 (claspmt) R 25530 25532 23451 0 -1 4202560 199753 0 0 0 500093 219 0 0 25 0 8 0 530702912 3934085120 885207 33554432000 4194304 4964704 140737244064960 18446744073709551615 4670228 0 0 4096 18946 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 3841880

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5039.8 s]
/proc/loadavg: 8.07 8.05 8.00 9/358 25687
/proc/meminfo: memFree=15946444/32951124 swapFree=67111524/67111528
[pid=25532] ppid=25530 vsize=3833732 CPUtime=40000.9 cores=0-7
/proc/25532/stat : 25532 (claspmt) R 25530 25532 23451 0 -1 4202496 902351 0 0 0 3999129 957 0 0 25 0 1 0 530699352 3925741568 882778 33554432000 4194304 4964704 140737244064960 18446744073709551615 233509952715 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/25532/statm: 958433 882778 476 189 0 952782 0
Current children cumulated CPU time (s) 40000.9
Current children cumulated vsize (KiB) 3833732

[startup+5040.6 s]
/proc/loadavg: 8.07 8.05 8.00 2/352 25688
/proc/meminfo: memFree=15974392/32951124 swapFree=67111524/67111528
[pid=25532] ppid=25530 vsize=3793524 CPUtime=40001.7 cores=0-7
/proc/25532/stat : 25532 (claspmt) R 25530 25532 23451 0 -1 4202496 902351 0 0 0 3999209 957 0 0 25 0 1 0 530699352 3884568576 872472 33554432000 4194304 4964704 140737244064960 18446744073709551615 233509954097 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/25532/statm: 948381 872472 476 189 0 942730 0
Current children cumulated CPU time (s) 40001.7
Current children cumulated vsize (KiB) 3793524

[startup+5040.8 s]
/proc/loadavg: 8.07 8.05 8.00 2/352 25688
/proc/meminfo: memFree=15974392/32951124 swapFree=67111524/67111528
[pid=25532] ppid=25530 vsize=1036980 CPUtime=40001.9 cores=0-7
/proc/25532/stat : 25532 (claspmt) R 25530 25532 23451 0 -1 4202496 902351 0 0 0 3999217 969 0 0 25 0 1 0 530699352 1061867520 130378 33554432000 4194304 4964704 140737244064960 18446744073709551615 233510323306 0 0 4096 18946 0 0 0 17 2 0 0 0
/proc/25532/statm: 259245 130378 476 189 0 253594 0
Current children cumulated CPU time (s) 40001.9
Current children cumulated vsize (KiB) 1036980

Child status: 0
Real time (s): 5040.82
CPU time (s): 40001.9
CPU user time (s): 39992.2
CPU system time (s): 9.71852
CPU usage (%): 793.558
Max. virtual memory (cumulated for all children) (KiB): 3841880

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39992.2
system time used= 9.71852
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 902352
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 196436
involuntary context switches= 186606

runsolver used 16.8214 second user time and 42.6605 second system time

The end

Launcher Data 

Begin job on node145 at 2011-05-03 02:33:55
IDJOB=3322400
IDBENCH=83165
IDSOLVER=1711
FILE ID=node145/3322400-1304382835
RUNJOBID= node145-1304366464-23465
PBS_JOBID= 13181729
Free space on /tmp= 73584 MiB

SOLVER NAME= claspmt 2.0-R4095
BENCH NAME= SAT11/crafted/mosoi/289/289-unsat-21x33.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3322400-1304382835/watcher-3322400-1304382835 -o /tmp/evaluation-result-3322400-1304382835/solver-3322400-1304382835 -C 40000 -W 5100 -M 31000  HOME/claspmt -t8 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3322400-1304382835.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= f7c29bbc3bef00d1e7cc799553ba4813
RANDOM SEED=304279325

node145.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.57
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.786
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      19489224 kB
Buffers:       1332456 kB
Cached:        6072580 kB
SwapCached:          4 kB
Active:        7909968 kB
Inactive:      4974220 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      19489224 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:           11720 kB
Writeback:           0 kB
AnonPages:     5478996 kB
Mapped:          27068 kB
Slab:           477124 kB
PageTables:      33816 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 79341964 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73584 MiB
End job on node145 at 2011-05-03 03:57:56