Trace number 3371436

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.39 1203.77

General information on the benchmark

NameAPPLICATIONS/bitverif/countbitsarray/
countbitsarray04_32.cnf
MD5SUM2fe3650d4a602f63086e937176360ebb
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables8750
Number of clauses25865
Sum of the clauses size60349
Maximum clause length3
Minimum clause length1
Number of clauses of size 12
Number of clauses of size 217242
Number of clauses of size 38621
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3371436-1305416999/watcher-3371436-1305416999 -o /tmp/evaluation-result-3371436-1305416999/solver-3371436-1305416999 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3371436-1305416999.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.09 6.27 7.07 8/370 6660
/proc/meminfo: memFree=17779816/32951124 swapFree=45193364/67111528
[pid=6660] ppid=6658 vsize=9456 CPUtime=0 cores=0,2,4,6
/proc/6660/stat : 6660 (plingeling) R 6658 6660 5935 0 -1 4202496 441 1252 0 0 0 0 0 0 25 0 1 0 731930594 9682944 372 33554432000 4194304 4417452 140736235573088 18446744073709551615 224284925696 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/6660/statm: 2364 372 148 55 0 250 0

[startup+0.033411 s]
/proc/loadavg: 7.09 6.27 7.07 8/370 6660
/proc/meminfo: memFree=17779816/32951124 swapFree=45193364/67111528
[pid=6660] ppid=6658 vsize=56828 CPUtime=0.02 cores=0,2,4,6
/proc/6660/stat : 6660 (plingeling) S 6658 6660 5935 0 -1 4202496 2171 1252 0 0 2 0 0 0 25 0 5 0 731930594 58191872 2002 33554432000 4194304 4417452 140736235573088 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6660/statm: 14207 2002 158 55 0 12093 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 56828

[startup+0.139345 s]
/proc/loadavg: 7.09 6.27 7.07 8/370 6660
/proc/meminfo: memFree=17779816/32951124 swapFree=45193364/67111528
[pid=6660] ppid=6658 vsize=122612 CPUtime=0.44 cores=0,2,4,6
/proc/6660/stat : 6660 (plingeling) S 6658 6660 5935 0 -1 4202496 3724 1252 0 0 44 0 0 0 25 0 5 0 731930594 125554688 2084 33554432000 4194304 4417452 140736235573088 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6660/statm: 30653 2084 174 55 0 28539 0
Current children cumulated CPU time (s) 0.44
Current children cumulated vsize (KiB) 122612

[startup+0.300319 s]
/proc/loadavg: 7.09 6.27 7.07 8/370 6660
/proc/meminfo: memFree=17779816/32951124 swapFree=45193364/67111528
[pid=6660] ppid=6658 vsize=124148 CPUtime=1.09 cores=0,2,4,6
/proc/6660/stat : 6660 (plingeling) S 6658 6660 5935 0 -1 4202496 4110 1252 0 0 109 0 0 0 25 0 5 0 731930594 127127552 2470 33554432000 4194304 4417452 140736235573088 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6660/statm: 31037 2470 176 55 0 28923 0
Current children cumulated CPU time (s) 1.09
Current children cumulated vsize (KiB) 124148

[startup+0.700251 s]
/proc/loadavg: 7.09 6.27 7.07 8/370 6660
/proc/meminfo: memFree=17779816/32951124 swapFree=45193364/67111528
[pid=6660] ppid=6658 vsize=125152 CPUtime=2.69 cores=0,2,4,6
/proc/6660/stat : 6660 (plingeling) S 6658 6660 5935 0 -1 4202496 4376 1252 0 0 269 0 0 0 25 0 5 0 731930594 128155648 2736 33554432000 4194304 4417452 140736235573088 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6660/statm: 31288 2736 179 55 0 29174 0
Current children cumulated CPU time (s) 2.69
Current children cumulated vsize (KiB) 125152

[startup+1.50112 s]
/proc/loadavg: 7.09 6.27 7.07 9/375 6670
/proc/meminfo: memFree=17766872/32951124 swapFree=45193364/67111528
[pid=6660] ppid=6658 vsize=130908 CPUtime=5.88 cores=0,2,4,6
/proc/6660/stat : 6660 (plingeling) S 6658 6660 5935 0 -1 4202496 6410 1252 0 0 588 0 0 0 25 0 5 0 731930594 134049792 4176 33554432000 4194304 4417452 140736235573088 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6660/statm: 32727 4176 180 55 0 30613 0
[pid=6660/tid=6667] ppid=6658 vsize=130908 CPUtime=1.46 cores=0,2,4,6
/proc/6660/task/6667/stat : 6667 (plingeling) R 6658 6660 5935 0 -1 4202560 626 1252 0 0 146 0 0 0 25 0 5 0 731930597 134049792 4176 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364807 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6660/tid=6668] ppid=6658 vsize=130908 CPUtime=1.46 cores=0,2,4,6
/proc/6660/task/6668/stat : 6668 (plingeling) R 6658 6660 5935 0 -1 4202560 1218 1252 0 0 146 0 0 0 20 0 5 0 731930597 134049792 4176 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6660/tid=6669] ppid=6658 vsize=130908 CPUtime=1.46 cores=0,2,4,6
/proc/6660/task/6669/stat : 6669 (plingeling) R 6658 6660 5935 0 -1 4202560 984 1252 0 0 146 0 0 0 20 0 5 0 731930597 134049792 4176 33554432000 4194304 4417452 140736235573088 18446744073709551615 4232550 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6660/tid=6670] ppid=6658 vsize=130908 CPUtime=1.45 cores=0,2,4,6
/proc/6660/task/6670/stat : 6670 (plingeling) R 6658 6660 5935 0 -1 4202560 1416 1252 0 0 145 0 0 0 25 0 5 0 731930597 134049792 4176 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 5.88
Current children cumulated vsize (KiB) 130908

[startup+3.10085 s]
/proc/loadavg: 7.09 6.27 7.07 9/375 6670
/proc/meminfo: memFree=17760176/32951124 swapFree=45193364/67111528
[pid=6660] ppid=6658 vsize=141912 CPUtime=12.26 cores=0,2,4,6
/proc/6660/stat : 6660 (plingeling) S 6658 6660 5935 0 -1 4202496 13396 1252 0 0 1225 1 0 0 25 0 5 0 731930594 145317888 6946 33554432000 4194304 4417452 140736235573088 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6660/statm: 35478 6946 180 55 0 33364 0
[pid=6660/tid=6667] ppid=6658 vsize=141912 CPUtime=3.06 cores=0,2,4,6
/proc/6660/task/6667/stat : 6667 (plingeling) R 6658 6660 5935 0 -1 4202560 2018 1252 0 0 306 0 0 0 25 0 5 0 731930597 145317888 6946 33554432000 4194304 4417452 140736235573088 18446744073709551615 4276028 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6660/tid=6668] ppid=6658 vsize=141912 CPUtime=3.06 cores=0,2,4,6
/proc/6660/task/6668/stat : 6668 (plingeling) R 6658 6660 5935 0 -1 4202560 3356 1252 0 0 306 0 0 0 25 0 5 0 731930597 145317888 6946 33554432000 4194304 4417452 140736235573088 18446744073709551615 4366065 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6660/tid=6669] ppid=6658 vsize=141912 CPUtime=3.06 cores=0,2,4,6
/proc/6660/task/6669/stat : 6669 (plingeling) R 6658 6660 5935 0 -1 4202560 2239 1252 0 0 306 0 0 0 25 0 5 0 731930597 145317888 6946 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364653 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6660/tid=6670] ppid=6658 vsize=141912 CPUtime=3.03 cores=0,2,4,6
/proc/6660/task/6670/stat : 6670 (plingeling) R 6658 6660 5935 0 -1 4202560 3617 1252 0 0 303 0 0 0 25 0 5 0 731930597 145317888 6946 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 12.26
Current children cumulated vsize (KiB) 141912

[startup+6.30032 s]
/proc/loadavg: 7.16 6.30 7.07 9/375 6670
/proc/meminfo: memFree=17750120/32951124 swapFree=45193364/67111528
[pid=6660] ppid=6658 vsize=141452 CPUtime=25.02 cores=0,2,4,6
/proc/6660/stat : 6660 (plingeling) S 6658 6660 5935 0 -1 4202496 28310 1252 0 0 2498 4 0 0 25 0 5 0 731930594 144846848 6844 33554432000 4194304 4417452 140736235573088 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6660/statm: 35363 6844 180 55 0 33249 0
[pid=6660/tid=6667] ppid=6658 vsize=141452 CPUtime=6.26 cores=0,2,4,6
/proc/6660/task/6667/stat : 6667 (plingeling) R 6658 6660 5935 0 -1 4202560 6906 1252 0 0 625 1 0 0 25 0 5 0 731930597 144846848 6844 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364537 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6660/tid=6668] ppid=6658 vsize=141452 CPUtime=6.25 cores=0,2,4,6
/proc/6660/task/6668/stat : 6668 (plingeling) R 6658 6660 5935 0 -1 4202560 7347 1252 0 0 625 0 0 0 25 0 5 0 731930597 144846848 6844 33554432000 4194304 4417452 140736235573088 18446744073709551615 4257471 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6660/tid=6669] ppid=6658 vsize=141452 CPUtime=6.26 cores=0,2,4,6
/proc/6660/task/6669/stat : 6669 (plingeling) R 6658 6660 5935 0 -1 4202560 5638 1252 0 0 625 1 0 0 25 0 5 0 731930597 144846848 6844 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6660/tid=6670] ppid=6658 vsize=141452 CPUtime=6.19 cores=0,2,4,6
/proc/6660/task/6670/stat : 6670 (plingeling) R 6658 6660 5935 0 -1 4202560 6253 1252 0 0 619 0 0 0 25 0 5 0 731930597 144846848 6844 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364572 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 25.02
Current children cumulated vsize (KiB) 141452

[startup+12.7003 s]
/proc/loadavg: 7.31 6.34 7.08 9/375 6670
/proc/meminfo: memFree=17736636/32951124 swapFree=45193364/67111528
[pid=6660] ppid=6658 vsize=156564 CPUtime=50.53 cores=0,2,4,6
/proc/6660/stat : 6660 (plingeling) S 6658 6660 5935 0 -1 4202496 58111 1252 0 0 5044 9 0 0 25 0 5 0 731930594 160321536 10596 33554432000 4194304 4417452 140736235573088 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6660/statm: 39141 10596 180 55 0 37027 0
[pid=6660/tid=6667] ppid=6658 vsize=156564 CPUtime=12.65 cores=0,2,4,6
/proc/6660/task/6667/stat : 6667 (plingeling) R 6658 6660 5935 0 -1 4202560 14003 1252 0 0 1263 2 0 0 25 0 5 0 731930597 160321536 10596 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6660/tid=6668] ppid=6658 vsize=156564 CPUtime=12.66 cores=0,2,4,6
/proc/6660/task/6668/stat : 6668 (plingeling) R 6658 6660 5935 0 -1 4202560 15892 1252 0 0 1264 2 0 0 25 0 5 0 731930597 160321536 10596 33554432000 4194304 4417452 140736235573088 18446744073709551615 4250986 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6660/tid=6669] ppid=6658 vsize=156564 CPUtime=12.66 cores=0,2,4,6
/proc/6660/task/6669/stat : 6669 (plingeling) R 6658 6660 5935 0 -1 4202560 12850 1252 0 0 1264 2 0 0 25 0 5 0 731930597 160321536 10596 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6660/tid=6670] ppid=6658 vsize=156564 CPUtime=12.52 cores=0,2,4,6
/proc/6660/task/6670/stat : 6670 (plingeling) R 6658 6660 5935 0 -1 4202560 13200 1252 0 0 1250 2 0 0 25 0 5 0 731930597 160321536 10596 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364896 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 50.53
Current children cumulated vsize (KiB) 156564

[startup+25.5011 s]
/proc/loadavg: 7.46 6.42 7.10 9/375 6670
/proc/meminfo: memFree=17721136/32951124 swapFree=45193364/67111528
[pid=6660] ppid=6658 vsize=175572 CPUtime=101.57 cores=0,2,4,6
/proc/6660/stat : 6660 (plingeling) S 6658 6660 5935 0 -1 4202496 109631 1252 0 0 10139 18 0 0 25 0 5 0 731930594 179785728 15348 33554432000 4194304 4417452 140736235573088 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6660/statm: 43893 15348 180 55 0 41779 0
[pid=6660/tid=6667] ppid=6658 vsize=175572 CPUtime=25.44 cores=0,2,4,6
/proc/6660/task/6667/stat : 6667 (plingeling) R 6658 6660 5935 0 -1 4202560 27986 1252 0 0 2540 4 0 0 25 0 5 0 731930597 179785728 15348 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364558 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6660/tid=6668] ppid=6658 vsize=175572 CPUtime=25.46 cores=0,2,4,6
/proc/6660/task/6668/stat : 6668 (plingeling) R 6658 6660 5935 0 -1 4202560 30443 1252 0 0 2541 5 0 0 25 0 5 0 731930597 179785728 15348 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6660/tid=6669] ppid=6658 vsize=175572 CPUtime=25.46 cores=0,2,4,6
/proc/6660/task/6669/stat : 6669 (plingeling) R 6658 6660 5935 0 -1 4202560 25940 1252 0 0 2542 4 0 0 25 0 5 0 731930597 179785728 15348 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6660/tid=6670] ppid=6658 vsize=175572 CPUtime=25.16 cores=0,2,4,6
/proc/6660/task/6670/stat : 6670 (plingeling) R 6658 6660 5935 0 -1 4202560 23096 1252 0 0 2512 4 0 0 25 0 5 0 731930597 179785728 15348 33554432000 4194304 4417452 140736235573088 18446744073709551615 4250941 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 101.57
Current children cumulated vsize (KiB) 175572

[startup+51.1009 s]
/proc/loadavg: 7.64 6.55 7.12 9/375 6672
/proc/meminfo: memFree=17674992/32951124 swapFree=45193364/67111528
[pid=6660] ppid=6658 vsize=192080 CPUtime=203.68 cores=0,2,4,6
/proc/6660/stat : 6660 (plingeling) S 6658 6660 5935 0 -1 4202496 194572 1252 0 0 20336 32 0 0 25 0 5 0 731930594 196689920 19507 33554432000 4194304 4417452 140736235573088 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6660/statm: 48020 19507 180 55 0 45906 0
[pid=6660/tid=6667] ppid=6658 vsize=192080 CPUtime=51.05 cores=0,2,4,6
/proc/6660/task/6667/stat : 6667 (plingeling) R 6658 6660 5935 0 -1 4202560 49156 1252 0 0 5097 8 0 0 25 0 5 0 731930597 196689920 19507 33554432000 4194304 4417452 140736235573088 18446744073709551615 4365088 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6660/tid=6668] ppid=6658 vsize=192080 CPUtime=51.05 cores=0,2,4,6
/proc/6660/task/6668/stat : 6668 (plingeling) R 6658 6660 5935 0 -1 4202560 50925 1252 0 0 5097 8 0 0 25 0 5 0 731930597 196689920 19507 33554432000 4194304 4417452 140736235573088 18446744073709551615 4366391 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6660/tid=6669] ppid=6658 vsize=192080 CPUtime=51.06 cores=0,2,4,6
/proc/6660/task/6669/stat : 6669 (plingeling) R 6658 6660 5935 0 -1 4202560 51136 1252 0 0 5098 8 0 0 25 0 5 0 731930597 196689920 19507 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6660/tid=6670] ppid=6658 vsize=192080 CPUtime=50.46 cores=0,2,4,6
/proc/6660/task/6670/stat : 6670 (plingeling) R 6658 6660 5935 0 -1 4202560 41189 1252 0 0 5039 7 0 0 25 0 5 0 731930597 196689920 19507 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364693 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 203.68
Current children cumulated vsize (KiB) 192080

[startup+102.314 s]
/proc/loadavg: 7.88 6.78 7.17 9/375 6672
/proc/meminfo: memFree=17572412/32951124 swapFree=45193364/67111528
[pid=6660] ppid=6658 vsize=288128 CPUtime=407.91 cores=0,2,4,6
/proc/6660/stat : 6660 (plingeling) S 6658 6660 5935 0 -1 4202496 313833 1252 0 0 40739 52 0 0 25 0 5 0 731930594 295043072 43487 33554432000 4194304 4417452 140736235573088 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6660/statm: 72032 43487 180 55 0 69918 0
[pid=6660/tid=6667] ppid=6658 vsize=288128 CPUtime=102.25 cores=0,2,4,6
/proc/6660/task/6667/stat : 6667 (plingeling) R 6658 6660 5935 0 -1 4202560 75092 1252 0 0 10212 13 0 0 25 0 5 0 731930597 295043072 43487 33554432000 4194304 4417452 140736235573088 18446744073709551615 4357260 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6660/tid=6668] ppid=6658 vsize=288128 CPUtime=102.27 cores=0,2,4,6
/proc/6660/task/6668/stat : 6668 (plingeling) R 6658 6660 5935 0 -1 4202560 82580 1252 0 0 10214 13 0 0 25 0 5 0 731930597 295043072 43487 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6660/tid=6669] ppid=6658 vsize=288128 CPUtime=102.27 cores=0,2,4,6
/proc/6660/task/6669/stat : 6669 (plingeling) R 6658 6660 5935 0 -1 4202560 86697 1252 0 0 10213 14 0 0 25 0 5 0 731930597 295043072 43487 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6660/tid=6670] ppid=6658 vsize=288128 CPUtime=101.07 cores=0,2,4,6
/proc/6660/task/6670/stat : 6670 (plingeling) R 6658 6660 5935 0 -1 4202560 67298 1252 0 0 10096 11 0 0 25 0 5 0 731930597 295043072 43487 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 407.91
Current children cumulated vsize (KiB) 288128

[startup+162.3 s]
/proc/loadavg: 7.99 7.02 7.22 9/375 6674
/proc/meminfo: memFree=17459420/32951124 swapFree=45193364/67111528
[pid=6660] ppid=6658 vsize=400244 CPUtime=647.14 cores=0,2,4,6
/proc/6660/stat : 6660 (plingeling) S 6658 6660 5935 0 -1 4202496 403279 1252 0 0 64646 68 0 0 25 0 5 0 731930594 409849856 71516 33554432000 4194304 4417452 140736235573088 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6660/statm: 100061 71516 180 55 0 97947 0
[pid=6660/tid=6667] ppid=6658 vsize=400244 CPUtime=162.22 cores=0,2,4,6
/proc/6660/task/6667/stat : 6667 (plingeling) R 6658 6660 5935 0 -1 4202560 98681 1252 0 0 16205 17 0 0 25 0 5 0 731930597 409849856 71516 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6660/tid=6668] ppid=6658 vsize=400244 CPUtime=162.26 cores=0,2,4,6
/proc/6660/task/6668/stat : 6668 (plingeling) R 6658 6660 5935 0 -1 4202560 95888 1252 0 0 16210 16 0 0 25 0 5 0 731930597 409849856 71516 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6660/tid=6669] ppid=6658 vsize=400244 CPUtime=162.26 cores=0,2,4,6
/proc/6660/task/6669/stat : 6669 (plingeling) R 6658 6660 5935 0 -1 4202560 114057 1252 0 0 16207 19 0 0 25 0 5 0 731930597 409849856 71516 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6660/tid=6670] ppid=6658 vsize=400244 CPUtime=160.35 cores=0,2,4,6
/proc/6660/task/6670/stat : 6670 (plingeling) R 6658 6660 5935 0 -1 4202560 92487 1252 0 0 16020 15 0 0 25 0 5 0 731930597 409849856 71516 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 647.14
Current children cumulated vsize (KiB) 400244


################
# More data... #
################

[pid=6660/tid=6668] ppid=6658 vsize=909828 CPUtime=762.26 cores=0,2,4,6
/proc/6660/task/6668/stat : 6668 (plingeling) R 6658 6660 5935 0 -1 4202560 348185 1252 0 0 76168 58 0 0 25 0 5 0 731930597 931663872 198944 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6660/tid=6669] ppid=6658 vsize=909828 CPUtime=762.28 cores=0,2,4,6
/proc/6660/task/6669/stat : 6669 (plingeling) R 6658 6660 5935 0 -1 4202560 309469 1252 0 0 76177 51 0 0 25 0 5 0 731930597 931663872 198944 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6660/tid=6670] ppid=6658 vsize=909828 CPUtime=753.27 cores=0,2,4,6
/proc/6660/task/6670/stat : 6670 (plingeling) R 6658 6660 5935 0 -1 4202560 245962 1252 0 0 75290 37 0 0 25 0 5 0 731930597 931663872 198944 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3039.99
Current children cumulated vsize (KiB) 909828

[startup+822.3 s]
/proc/loadavg: 8.14 8.01 7.67 9/375 6773
/proc/meminfo: memFree=16976816/32951124 swapFree=45193364/67111528
[pid=6660] ppid=6658 vsize=965100 CPUtime=3279.22 cores=0,2,4,6
/proc/6660/stat : 6660 (plingeling) S 6658 6660 5935 0 -1 4202496 1282149 1252 0 0 327715 207 0 0 25 0 5 0 731930594 988262400 212762 33554432000 4194304 4417452 140736235573088 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6660/statm: 241275 212762 180 55 0 239161 0
[pid=6660/tid=6667] ppid=6658 vsize=965100 CPUtime=822.09 cores=0,2,4,6
/proc/6660/task/6667/stat : 6667 (plingeling) R 6658 6660 5935 0 -1 4202560 349724 1252 0 0 82154 55 0 0 25 0 5 0 731930597 988262400 212762 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6660/tid=6668] ppid=6658 vsize=965100 CPUtime=822.25 cores=0,2,4,6
/proc/6660/task/6668/stat : 6668 (plingeling) R 6658 6660 5935 0 -1 4202560 361892 1252 0 0 82164 61 0 0 25 0 5 0 731930597 988262400 212762 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6660/tid=6669] ppid=6658 vsize=965100 CPUtime=822.27 cores=0,2,4,6
/proc/6660/task/6669/stat : 6669 (plingeling) R 6658 6660 5935 0 -1 4202560 309469 1252 0 0 82176 51 0 0 25 0 5 0 731930597 988262400 212762 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6660/tid=6670] ppid=6658 vsize=965100 CPUtime=812.55 cores=0,2,4,6
/proc/6660/task/6670/stat : 6670 (plingeling) R 6658 6660 5935 0 -1 4202560 258898 1252 0 0 81217 38 0 0 25 0 5 0 731930597 988262400 212762 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3279.22
Current children cumulated vsize (KiB) 965100

[startup+882.3 s]
/proc/loadavg: 8.13 8.03 7.70 9/375 6775
/proc/meminfo: memFree=16970848/32951124 swapFree=45193364/67111528
[pid=6660] ppid=6658 vsize=924616 CPUtime=3518.5 cores=0,2,4,6
/proc/6660/stat : 6660 (plingeling) S 6658 6660 5935 0 -1 4202496 1301495 1252 0 0 351639 211 0 0 25 0 5 0 731930594 946806784 202641 33554432000 4194304 4417452 140736235573088 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6660/statm: 231154 202641 180 55 0 229040 0
[pid=6660/tid=6667] ppid=6658 vsize=924616 CPUtime=882.08 cores=0,2,4,6
/proc/6660/task/6667/stat : 6667 (plingeling) R 6658 6660 5935 0 -1 4202560 357916 1252 0 0 88152 56 0 0 25 0 5 0 731930597 946806784 202641 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6660/tid=6668] ppid=6658 vsize=924616 CPUtime=882.25 cores=0,2,4,6
/proc/6660/task/6668/stat : 6668 (plingeling) R 6658 6660 5935 0 -1 4202560 361892 1252 0 0 88164 61 0 0 25 0 5 0 731930597 946806784 202641 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6660/tid=6669] ppid=6658 vsize=924616 CPUtime=882.27 cores=0,2,4,6
/proc/6660/task/6669/stat : 6669 (plingeling) R 6658 6660 5935 0 -1 4202560 309469 1252 0 0 88176 51 0 0 25 0 5 0 731930597 946806784 202641 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6660/tid=6670] ppid=6658 vsize=924616 CPUtime=871.84 cores=0,2,4,6
/proc/6660/task/6670/stat : 6670 (plingeling) R 6658 6660 5935 0 -1 4202560 270052 1252 0 0 87143 41 0 0 25 0 5 0 731930597 946806784 202641 33554432000 4194304 4417452 140736235573088 18446744073709551615 4353557 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3518.5
Current children cumulated vsize (KiB) 924616

[startup+942.3 s]
/proc/loadavg: 8.11 8.05 7.73 9/375 6777
/proc/meminfo: memFree=17001680/32951124 swapFree=45193364/67111528
[pid=6660] ppid=6658 vsize=836956 CPUtime=3757.8 cores=0,2,4,6
/proc/6660/stat : 6660 (plingeling) S 6658 6660 5935 0 -1 4202496 1336277 1252 0 0 375564 216 0 0 25 0 5 0 731930594 857042944 180694 33554432000 4194304 4417452 140736235573088 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6660/statm: 209239 180694 180 55 0 207125 0
[pid=6660/tid=6667] ppid=6658 vsize=836956 CPUtime=942.07 cores=0,2,4,6
/proc/6660/task/6667/stat : 6667 (plingeling) R 6658 6660 5935 0 -1 4202560 357916 1252 0 0 94151 56 0 0 25 0 5 0 731930597 857042944 180694 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6660/tid=6668] ppid=6658 vsize=836956 CPUtime=942.25 cores=0,2,4,6
/proc/6660/task/6668/stat : 6668 (plingeling) R 6658 6660 5935 0 -1 4202560 361892 1252 0 0 94164 61 0 0 25 0 5 0 731930597 857042944 180694 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6660/tid=6669] ppid=6658 vsize=836956 CPUtime=942.27 cores=0,2,4,6
/proc/6660/task/6669/stat : 6669 (plingeling) R 6658 6660 5935 0 -1 4202560 335354 1252 0 0 94172 55 0 0 25 0 5 0 731930597 857042944 180694 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364675 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6660/tid=6670] ppid=6658 vsize=836956 CPUtime=931.14 cores=0,2,4,6
/proc/6660/task/6670/stat : 6670 (plingeling) R 6658 6660 5935 0 -1 4202560 278949 1252 0 0 93072 42 0 0 25 0 5 0 731930597 857042944 180694 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3757.8
Current children cumulated vsize (KiB) 836956

[startup+1002.3 s]
/proc/loadavg: 8.04 8.04 7.74 9/375 6777
/proc/meminfo: memFree=16780312/32951124 swapFree=45193364/67111528
[pid=6660] ppid=6658 vsize=1004296 CPUtime=3997.08 cores=0,2,4,6
/proc/6660/stat : 6660 (plingeling) S 6658 6660 5935 0 -1 4202496 1412340 1252 0 0 399481 227 0 0 25 0 5 0 731930594 1028399104 222529 33554432000 4194304 4417452 140736235573088 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6660/statm: 251074 222529 180 55 0 248960 0
[pid=6660/tid=6667] ppid=6658 vsize=1004296 CPUtime=1002.07 cores=0,2,4,6
/proc/6660/task/6667/stat : 6667 (plingeling) R 6658 6660 5935 0 -1 4202560 363691 1252 0 0 100150 57 0 0 25 0 5 0 731930597 1028399104 222529 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6660/tid=6668] ppid=6658 vsize=1004296 CPUtime=1002.25 cores=0,2,4,6
/proc/6660/task/6668/stat : 6668 (plingeling) R 6658 6660 5935 0 -1 4202560 391229 1252 0 0 100159 66 0 0 25 0 5 0 731930597 1028399104 222529 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6660/tid=6669] ppid=6658 vsize=1004296 CPUtime=1002.27 cores=0,2,4,6
/proc/6660/task/6669/stat : 6669 (plingeling) R 6658 6660 5935 0 -1 4202560 364749 1252 0 0 100168 59 0 0 25 0 5 0 731930597 1028399104 222529 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6660/tid=6670] ppid=6658 vsize=1004296 CPUtime=990.43 cores=0,2,4,6
/proc/6660/task/6670/stat : 6670 (plingeling) R 6658 6660 5935 0 -1 4202560 290505 1252 0 0 99000 43 0 0 25 0 5 0 731930597 1028399104 222529 33554432000 4194304 4417452 140736235573088 18446744073709551615 4365936 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3997.08
Current children cumulated vsize (KiB) 1004296

[startup+1062.3 s]
/proc/loadavg: 8.08 8.06 7.76 9/375 6779
/proc/meminfo: memFree=16612988/32951124 swapFree=45193364/67111528
[pid=6660] ppid=6658 vsize=1120616 CPUtime=4236.35 cores=0,2,4,6
/proc/6660/stat : 6660 (plingeling) S 6658 6660 5935 0 -1 4202496 1472640 1252 0 0 423397 238 0 0 25 0 5 0 731930594 1147510784 251609 33554432000 4194304 4417452 140736235573088 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6660/statm: 280154 251609 180 55 0 278040 0
[pid=6660/tid=6667] ppid=6658 vsize=1120616 CPUtime=1062.05 cores=0,2,4,6
/proc/6660/task/6667/stat : 6667 (plingeling) R 6658 6660 5935 0 -1 4202560 376521 1252 0 0 106145 60 0 0 25 0 5 0 731930597 1147510784 251609 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6660/tid=6668] ppid=6658 vsize=1120616 CPUtime=1062.24 cores=0,2,4,6
/proc/6660/task/6668/stat : 6668 (plingeling) R 6658 6660 5935 0 -1 4202560 429324 1252 0 0 106151 73 0 0 25 0 5 0 731930597 1147510784 251609 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6660/tid=6669] ppid=6658 vsize=1120616 CPUtime=1062.27 cores=0,2,4,6
/proc/6660/task/6669/stat : 6669 (plingeling) R 6658 6660 5935 0 -1 4202560 364749 1252 0 0 106168 59 0 0 25 0 5 0 731930597 1147510784 251609 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6660/tid=6670] ppid=6658 vsize=1120616 CPUtime=1049.73 cores=0,2,4,6
/proc/6660/task/6670/stat : 6670 (plingeling) R 6658 6660 5935 0 -1 4202560 299880 1252 0 0 104929 44 0 0 25 0 5 0 731930597 1147510784 251609 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4236.35
Current children cumulated vsize (KiB) 1120616

[startup+1122.3 s]
/proc/loadavg: 8.11 8.06 7.78 9/375 6781
/proc/meminfo: memFree=16768824/32951124 swapFree=45193364/67111528
[pid=6660] ppid=6658 vsize=1136300 CPUtime=4475.63 cores=0,2,4,6
/proc/6660/stat : 6660 (plingeling) S 6658 6660 5935 0 -1 4202496 1483686 1252 0 0 447323 240 0 0 25 0 5 0 731930594 1163571200 255530 33554432000 4194304 4417452 140736235573088 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6660/statm: 284075 255530 180 55 0 281961 0
[pid=6660/tid=6667] ppid=6658 vsize=1136300 CPUtime=1122.04 cores=0,2,4,6
/proc/6660/task/6667/stat : 6667 (plingeling) R 6658 6660 5935 0 -1 4202560 376521 1252 0 0 112144 60 0 0 25 0 5 0 731930597 1163571200 255530 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6660/tid=6668] ppid=6658 vsize=1136300 CPUtime=1122.24 cores=0,2,4,6
/proc/6660/task/6668/stat : 6668 (plingeling) R 6658 6660 5935 0 -1 4202560 429324 1252 0 0 112151 73 0 0 25 0 5 0 731930597 1163571200 255530 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6660/tid=6669] ppid=6658 vsize=1136300 CPUtime=1122.28 cores=0,2,4,6
/proc/6660/task/6669/stat : 6669 (plingeling) R 6658 6660 5935 0 -1 4202560 368476 1252 0 0 112168 60 0 0 25 0 5 0 731930597 1163571200 255530 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364958 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6660/tid=6670] ppid=6658 vsize=1136300 CPUtime=1109.01 cores=0,2,4,6
/proc/6660/task/6670/stat : 6670 (plingeling) R 6658 6660 5935 0 -1 4202560 307199 1252 0 0 110856 45 0 0 25 0 5 0 731930597 1163571200 255530 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4475.63
Current children cumulated vsize (KiB) 1136300

[startup+1182.3 s]
/proc/loadavg: 8.07 8.06 7.79 9/375 6783
/proc/meminfo: memFree=16485452/32951124 swapFree=45193364/67111528
[pid=6660] ppid=6658 vsize=1298652 CPUtime=4714.92 cores=0,2,4,6
/proc/6660/stat : 6660 (plingeling) S 6658 6660 5935 0 -1 4202496 1541154 1252 0 0 471241 251 0 0 25 0 5 0 731930594 1329819648 296118 33554432000 4194304 4417452 140736235573088 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6660/statm: 324663 296118 180 55 0 322549 0
[pid=6660/tid=6667] ppid=6658 vsize=1298652 CPUtime=1182.04 cores=0,2,4,6
/proc/6660/task/6667/stat : 6667 (plingeling) R 6658 6660 5935 0 -1 4202560 386176 1252 0 0 118142 62 0 0 25 0 5 0 731930597 1329819648 296118 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6660/tid=6668] ppid=6658 vsize=1298652 CPUtime=1182.24 cores=0,2,4,6
/proc/6660/task/6668/stat : 6668 (plingeling) R 6658 6660 5935 0 -1 4202560 429795 1252 0 0 118151 73 0 0 25 0 5 0 731930597 1329819648 296118 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6660/tid=6669] ppid=6658 vsize=1298652 CPUtime=1182.27 cores=0,2,4,6
/proc/6660/task/6669/stat : 6669 (plingeling) R 6658 6660 5935 0 -1 4202560 405176 1252 0 0 118160 67 0 0 25 0 5 0 731930597 1329819648 296118 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6660/tid=6670] ppid=6658 vsize=1298652 CPUtime=1168.32 cores=0,2,4,6
/proc/6660/task/6670/stat : 6670 (plingeling) R 6658 6660 5935 0 -1 4202560 317841 1252 0 0 116785 47 0 0 25 0 5 0 731930597 1329819648 296118 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4714.92
Current children cumulated vsize (KiB) 1298652



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.7 s]
/proc/loadavg: 8.05 8.06 7.80 9/375 6783
/proc/meminfo: memFree=16405220/32951124 swapFree=45193364/67111528
[pid=6660] ppid=6658 vsize=1282760 CPUtime=4800.28 cores=0,2,4,6
/proc/6660/stat : 6660 (plingeling) S 6658 6660 5935 0 -1 4202496 1544769 1252 0 0 479776 252 0 0 25 0 5 0 731930594 1313546240 292145 33554432000 4194304 4417452 140736235573088 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6660/statm: 320690 292145 180 55 0 318576 0
[pid=6660/tid=6667] ppid=6658 vsize=1282760 CPUtime=1203.44 cores=0,2,4,6
/proc/6660/task/6667/stat : 6667 (plingeling) R 6658 6660 5935 0 -1 4202560 386176 1252 0 0 120282 62 0 0 25 0 5 0 731930597 1313546240 292145 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6660/tid=6668] ppid=6658 vsize=1282760 CPUtime=1203.65 cores=0,2,4,6
/proc/6660/task/6668/stat : 6668 (plingeling) R 6658 6660 5935 0 -1 4202560 429795 1252 0 0 120292 73 0 0 25 0 5 0 731930597 1313546240 292145 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6660/tid=6669] ppid=6658 vsize=1282760 CPUtime=1203.68 cores=0,2,4,6
/proc/6660/task/6669/stat : 6669 (plingeling) R 6658 6660 5935 0 -1 4202560 405176 1252 0 0 120301 67 0 0 25 0 5 0 731930597 1313546240 292145 33554432000 4194304 4417452 140736235573088 18446744073709551615 4366079 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6660/tid=6670] ppid=6658 vsize=1282760 CPUtime=1189.45 cores=0,2,4,6
/proc/6660/task/6670/stat : 6670 (plingeling) R 6658 6660 5935 0 -1 4202560 321456 1252 0 0 118898 47 0 0 25 0 5 0 731930597 1313546240 292145 33554432000 4194304 4417452 140736235573088 18446744073709551615 4365946 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4800.28
Current children cumulated vsize (KiB) 1282760

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.7 s]
/proc/loadavg: 8.05 8.06 7.80 9/375 6783
/proc/meminfo: memFree=16405220/32951124 swapFree=45193364/67111528
[pid=6660] ppid=6658 vsize=1282760 CPUtime=4800.28 cores=0,2,4,6
/proc/6660/stat : 6660 (plingeling) S 6658 6660 5935 0 -1 4202496 1544769 1252 0 0 479776 252 0 0 25 0 5 0 731930594 1313546240 292145 33554432000 4194304 4417452 140736235573088 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/6660/statm: 320690 292145 180 55 0 318576 0
[pid=6660/tid=6667] ppid=6658 vsize=1282760 CPUtime=1203.44 cores=0,2,4,6
/proc/6660/task/6667/stat : 6667 (plingeling) R 6658 6660 5935 0 -1 4202560 386176 1252 0 0 120282 62 0 0 25 0 5 0 731930597 1313546240 292145 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=6660/tid=6668] ppid=6658 vsize=1282760 CPUtime=1203.65 cores=0,2,4,6
/proc/6660/task/6668/stat : 6668 (plingeling) R 6658 6660 5935 0 -1 4202560 429795 1252 0 0 120292 73 0 0 25 0 5 0 731930597 1313546240 292145 33554432000 4194304 4417452 140736235573088 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=6660/tid=6669] ppid=6658 vsize=1282760 CPUtime=1203.68 cores=0,2,4,6
/proc/6660/task/6669/stat : 6669 (plingeling) R 6658 6660 5935 0 -1 4202560 405176 1252 0 0 120301 67 0 0 25 0 5 0 731930597 1313546240 292145 33554432000 4194304 4417452 140736235573088 18446744073709551615 4366079 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=6660/tid=6670] ppid=6658 vsize=1282760 CPUtime=1189.45 cores=0,2,4,6
/proc/6660/task/6670/stat : 6670 (plingeling) R 6658 6660 5935 0 -1 4202560 321456 1252 0 0 118898 47 0 0 25 0 5 0 731930597 1313546240 292145 33554432000 4194304 4417452 140736235573088 18446744073709551615 4365946 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4800.28
Current children cumulated vsize (KiB) 1282760

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1203.77
CPU time (s): 4800.39
CPU user time (s): 4797.81
CPU system time (s): 2.57261
CPU usage (%): 398.779
Max. virtual memory (cumulated for all children) (KiB): 1298652

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4797.81
system time used= 2.57261
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1546021
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 111
involuntary context switches= 22219

runsolver used 4.02839 second user time and 10.2194 second system time

The end

Launcher Data 

Begin job on node118 at 2011-05-15 01:49:59
IDJOB=3371436
IDBENCH=70810
IDSOLVER=1848
FILE ID=node118/3371436-1305416999
RUNJOBID= node118-1305416895-6593
PBS_JOBID= 13324527
Free space on /tmp= 73028 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT09/APPLICATIONS/bitverif/countbitsarray/countbitsarray04_32.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3371436-1305416999/watcher-3371436-1305416999 -o /tmp/evaluation-result-3371436-1305416999/solver-3371436-1305416999 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3371436-1305416999.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 2fe3650d4a602f63086e937176360ebb
RANDOM SEED=449324097

node118.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5374.59
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      17780096 kB
Buffers:        279220 kB
Cached:        4409944 kB
SwapCached:      22084 kB
Active:       12276916 kB
Inactive:      2587992 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      17780096 kB
SwapTotal:    67111528 kB
SwapFree:     45193364 kB
Dirty:             396 kB
Writeback:           0 kB
AnonPages:    10166440 kB
Mapped:          15596 kB
Slab:           148520 kB
PageTables:      91136 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 89230476 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73048 MiB
End job on node118 at 2011-05-15 02:10:05