Trace number 3371475

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.25 1202.63

General information on the benchmark

Nameapplication/kullmann/AES/Benchmarks/
128/aes_128_3_keyfind_1.cnf
MD5SUMff59ad0e88befc51bfa041e45143dc53
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2808
Number of clauses30432
Sum of the clauses size167020
Maximum clause length8
Minimum clause length1
Number of clauses of size 1280
Number of clauses of size 2384
Number of clauses of size 35024
Number of clauses of size 4960
Number of clauses of size 56144
Number of clauses of size over 517640

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3371475-1305428833/watcher-3371475-1305428833 -o /tmp/evaluation-result-3371475-1305428833/solver-3371475-1305428833 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3371475-1305428833.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.28 1.79 1.92 2/259 25613
/proc/meminfo: memFree=23081616/32951124 swapFree=61565116/67111528
[pid=25613] ppid=25611 vsize=9712 CPUtime=0 cores=1,3,5,7
/proc/25613/stat : 25613 (plingeling) R 25611 25613 18974 0 -1 4202496 490 1254 0 0 0 0 0 0 21 0 1 0 654481535 9945088 419 33554432000 4194304 4417452 140734559798816 18446744073709551615 246516834048 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/25613/statm: 2428 419 148 55 0 314 0

[startup+0.0564769 s]
/proc/loadavg: 1.28 1.79 1.92 2/259 25613
/proc/meminfo: memFree=23081616/32951124 swapFree=61565116/67111528
[pid=25613] ppid=25611 vsize=58648 CPUtime=0.05 cores=1,3,5,7
/proc/25613/stat : 25613 (plingeling) S 25611 25613 18974 0 -1 4202496 2681 1254 0 0 5 0 0 0 21 0 5 0 654481535 60055552 2434 33554432000 4194304 4417452 140734559798816 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/25613/statm: 14662 2434 167 55 0 12548 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 58648

[startup+0.156458 s]
/proc/loadavg: 1.28 1.79 1.92 2/259 25613
/proc/meminfo: memFree=23081616/32951124 swapFree=61565116/67111528
[pid=25613] ppid=25611 vsize=60540 CPUtime=0.25 cores=1,3,5,7
/proc/25613/stat : 25613 (plingeling) S 25611 25613 18974 0 -1 4202496 3386 1254 0 0 25 0 0 0 21 0 5 0 654481535 61992960 2934 33554432000 4194304 4417452 140734559798816 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/25613/statm: 15135 2934 175 55 0 13021 0
Current children cumulated CPU time (s) 0.25
Current children cumulated vsize (KiB) 60540

[startup+0.300434 s]
/proc/loadavg: 1.28 1.79 1.92 2/259 25613
/proc/meminfo: memFree=23081616/32951124 swapFree=61565116/67111528
[pid=25613] ppid=25611 vsize=63108 CPUtime=0.7 cores=1,3,5,7
/proc/25613/stat : 25613 (plingeling) S 25611 25613 18974 0 -1 4202496 4036 1254 0 0 70 0 0 0 21 0 5 0 654481535 64622592 3584 33554432000 4194304 4417452 140734559798816 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/25613/statm: 15777 3584 179 55 0 13663 0
Current children cumulated CPU time (s) 0.7
Current children cumulated vsize (KiB) 63108

[startup+0.700359 s]
/proc/loadavg: 1.28 1.79 1.92 2/259 25613
/proc/meminfo: memFree=23081616/32951124 swapFree=61565116/67111528
[pid=25613] ppid=25611 vsize=129536 CPUtime=2.31 cores=1,3,5,7
/proc/25613/stat : 25613 (plingeling) S 25611 25613 18974 0 -1 4202496 4658 1254 0 0 230 1 0 0 21 0 5 0 654481535 132644864 3823 33554432000 4194304 4417452 140734559798816 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/25613/statm: 32384 3823 179 55 0 30270 0
Current children cumulated CPU time (s) 2.31
Current children cumulated vsize (KiB) 129536

[startup+1.50021 s]
/proc/loadavg: 1.28 1.79 1.92 9/271 25638
/proc/meminfo: memFree=23051004/32951124 swapFree=61565116/67111528
[pid=25613] ppid=25611 vsize=132868 CPUtime=5.5 cores=1,3,5,7
/proc/25613/stat : 25613 (plingeling) S 25611 25613 18974 0 -1 4202496 7824 1254 0 0 549 1 0 0 21 0 5 0 654481535 136056832 4656 33554432000 4194304 4417452 140734559798816 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/25613/statm: 33217 4656 179 55 0 31103 0
[pid=25613/tid=25631] ppid=25611 vsize=132868 CPUtime=1.36 cores=1,3,5,7
/proc/25613/task/25631/stat : 25631 (plingeling) R 25611 25613 18974 0 -1 4202560 1394 1254 0 0 136 0 0 0 25 0 5 0 654481541 136056832 4656 33554432000 4194304 4417452 140734559798816 18446744073709551615 4353533 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25613/tid=25632] ppid=25611 vsize=132868 CPUtime=1.31 cores=1,3,5,7
/proc/25613/task/25632/stat : 25632 (plingeling) R 25611 25613 18974 0 -1 4202560 575 1254 0 0 131 0 0 0 25 0 5 0 654481541 136056832 4656 33554432000 4194304 4417452 140734559798816 18446744073709551615 4366447 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25613/tid=25633] ppid=25611 vsize=132868 CPUtime=1.34 cores=1,3,5,7
/proc/25613/task/25633/stat : 25633 (plingeling) R 25611 25613 18974 0 -1 4202560 1964 1254 0 0 134 0 0 0 25 0 5 0 654481541 136056832 4656 33554432000 4194304 4417452 140734559798816 18446744073709551615 4358082 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25613/tid=25634] ppid=25611 vsize=132868 CPUtime=1.43 cores=1,3,5,7
/proc/25613/task/25634/stat : 25634 (plingeling) R 25611 25613 18974 0 -1 4202560 1235 1254 0 0 143 0 0 0 25 0 5 0 654481541 136056832 4656 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 5.5
Current children cumulated vsize (KiB) 132868

[startup+3.10092 s]
/proc/loadavg: 1.28 1.79 1.92 9/271 25638
/proc/meminfo: memFree=23040712/32951124 swapFree=61565116/67111528
[pid=25613] ppid=25611 vsize=140756 CPUtime=11.89 cores=1,3,5,7
/proc/25613/stat : 25613 (plingeling) S 25611 25613 18974 0 -1 4202496 16613 1254 0 0 1187 2 0 0 21 0 5 0 654481535 144134144 6660 33554432000 4194304 4417452 140734559798816 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/25613/statm: 35189 6660 179 55 0 33075 0
[pid=25613/tid=25631] ppid=25611 vsize=140756 CPUtime=2.95 cores=1,3,5,7
/proc/25613/task/25631/stat : 25631 (plingeling) R 25611 25613 18974 0 -1 4202560 3416 1254 0 0 295 0 0 0 25 0 5 0 654481541 144134144 6660 33554432000 4194304 4417452 140734559798816 18446744073709551615 4252507 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25613/tid=25632] ppid=25611 vsize=140756 CPUtime=2.91 cores=1,3,5,7
/proc/25613/task/25632/stat : 25632 (plingeling) R 25611 25613 18974 0 -1 4202560 2469 1254 0 0 291 0 0 0 25 0 5 0 654481541 144134144 6660 33554432000 4194304 4417452 140734559798816 18446744073709551615 4248252 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25613/tid=25633] ppid=25611 vsize=140756 CPUtime=2.93 cores=1,3,5,7
/proc/25613/task/25633/stat : 25633 (plingeling) R 25611 25613 18974 0 -1 4202560 5080 1254 0 0 293 0 0 0 25 0 5 0 654481541 144134144 6660 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364616 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25613/tid=25634] ppid=25611 vsize=140756 CPUtime=3.02 cores=1,3,5,7
/proc/25613/task/25634/stat : 25634 (plingeling) R 25611 25613 18974 0 -1 4202560 2992 1254 0 0 302 0 0 0 25 0 5 0 654481541 144134144 6660 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 11.89
Current children cumulated vsize (KiB) 140756

[startup+6.30034 s]
/proc/loadavg: 1.82 1.90 1.96 9/271 25638
/proc/meminfo: memFree=23021808/32951124 swapFree=61565116/67111528
[pid=25613] ppid=25611 vsize=150668 CPUtime=24.66 cores=1,3,5,7
/proc/25613/stat : 25613 (plingeling) S 25611 25613 18974 0 -1 4202496 33572 1254 0 0 2461 5 0 0 21 0 5 0 654481535 154284032 9110 33554432000 4194304 4417452 140734559798816 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/25613/statm: 37667 9110 179 55 0 35553 0
[pid=25613/tid=25631] ppid=25611 vsize=150668 CPUtime=6.15 cores=1,3,5,7
/proc/25613/task/25631/stat : 25631 (plingeling) R 25611 25613 18974 0 -1 4202560 8464 1254 0 0 614 1 0 0 25 0 5 0 654481541 154284032 9110 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25613/tid=25632] ppid=25611 vsize=150668 CPUtime=6.1 cores=1,3,5,7
/proc/25613/task/25632/stat : 25632 (plingeling) R 25611 25613 18974 0 -1 4202560 7276 1254 0 0 610 0 0 0 25 0 5 0 654481541 154284032 9110 33554432000 4194304 4417452 140734559798816 18446744073709551615 18446744073699065939 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25613/tid=25633] ppid=25611 vsize=150668 CPUtime=6.13 cores=1,3,5,7
/proc/25613/task/25633/stat : 25633 (plingeling) R 25611 25613 18974 0 -1 4202560 10227 1254 0 0 612 1 0 0 25 0 5 0 654481541 154284032 9110 33554432000 4194304 4417452 140734559798816 18446744073709551615 4251100 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25613/tid=25634] ppid=25611 vsize=150668 CPUtime=6.19 cores=1,3,5,7
/proc/25613/task/25634/stat : 25634 (plingeling) R 25611 25613 18974 0 -1 4202560 4949 1254 0 0 619 0 0 0 25 0 5 0 654481541 154284032 9110 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364765 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 24.66
Current children cumulated vsize (KiB) 150668

[startup+12.7012 s]
/proc/loadavg: 2.31 2.00 1.99 9/271 25638
/proc/meminfo: memFree=22999592/32951124 swapFree=61565116/67111528
[pid=25613] ppid=25611 vsize=162800 CPUtime=50.21 cores=1,3,5,7
/proc/25613/stat : 25613 (plingeling) S 25611 25613 18974 0 -1 4202496 59643 1254 0 0 5010 11 0 0 21 0 5 0 654481535 166707200 12144 33554432000 4194304 4417452 140734559798816 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/25613/statm: 40700 12144 179 55 0 38586 0
[pid=25613/tid=25631] ppid=25611 vsize=162800 CPUtime=12.55 cores=1,3,5,7
/proc/25613/task/25631/stat : 25631 (plingeling) R 25611 25613 18974 0 -1 4202560 16837 1254 0 0 1252 3 0 0 25 0 5 0 654481541 166707200 12144 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364583 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25613/tid=25632] ppid=25611 vsize=162800 CPUtime=12.5 cores=1,3,5,7
/proc/25613/task/25632/stat : 25632 (plingeling) R 25611 25613 18974 0 -1 4202560 13812 1254 0 0 1248 2 0 0 25 0 5 0 654481541 166707200 12144 33554432000 4194304 4417452 140734559798816 18446744073709551615 4250488 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25613/tid=25633] ppid=25611 vsize=162800 CPUtime=12.53 cores=1,3,5,7
/proc/25613/task/25633/stat : 25633 (plingeling) R 25611 25613 18974 0 -1 4202560 17737 1254 0 0 1251 2 0 0 25 0 5 0 654481541 166707200 12144 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25613/tid=25634] ppid=25611 vsize=162800 CPUtime=12.54 cores=1,3,5,7
/proc/25613/task/25634/stat : 25634 (plingeling) R 25611 25613 18974 0 -1 4202560 8601 1254 0 0 1253 1 0 0 25 0 5 0 654481541 166707200 12144 33554432000 4194304 4417452 140734559798816 18446744073709551615 4236649 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.21
Current children cumulated vsize (KiB) 162800

[startup+25.5008 s]
/proc/loadavg: 3.87 2.36 2.11 9/271 25639
/proc/meminfo: memFree=22980012/32951124 swapFree=61565116/67111528
[pid=25613] ppid=25611 vsize=166352 CPUtime=101.3 cores=1,3,5,7
/proc/25613/stat : 25613 (plingeling) S 25611 25613 18974 0 -1 4202496 103703 1254 0 0 10112 18 0 0 21 0 5 0 654481535 170344448 13032 33554432000 4194304 4417452 140734559798816 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/25613/statm: 41588 13032 179 55 0 39474 0
[pid=25613/tid=25631] ppid=25611 vsize=166352 CPUtime=25.35 cores=1,3,5,7
/proc/25613/task/25631/stat : 25631 (plingeling) R 25611 25613 18974 0 -1 4202560 31816 1254 0 0 2530 5 0 0 25 0 5 0 654481541 170344448 13032 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364737 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25613/tid=25632] ppid=25611 vsize=166352 CPUtime=25.3 cores=1,3,5,7
/proc/25613/task/25632/stat : 25632 (plingeling) R 25611 25613 18974 0 -1 4202560 24799 1254 0 0 2526 4 0 0 25 0 5 0 654481541 170344448 13032 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25613/tid=25633] ppid=25611 vsize=166352 CPUtime=25.34 cores=1,3,5,7
/proc/25613/task/25633/stat : 25633 (plingeling) R 25611 25613 18974 0 -1 4202560 30259 1254 0 0 2529 5 0 0 25 0 5 0 654481541 170344448 13032 33554432000 4194304 4417452 140734559798816 18446744073709551615 4236649 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25613/tid=25634] ppid=25611 vsize=166352 CPUtime=25.23 cores=1,3,5,7
/proc/25613/task/25634/stat : 25634 (plingeling) R 25611 25613 18974 0 -1 4202560 14173 1254 0 0 2521 2 0 0 25 0 5 0 654481541 170344448 13032 33554432000 4194304 4417452 140734559798816 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.3
Current children cumulated vsize (KiB) 166352

[startup+51.1012 s]
/proc/loadavg: 5.28 2.81 2.26 9/271 25639
/proc/meminfo: memFree=22914128/32951124 swapFree=61565116/67111528
[pid=25613] ppid=25611 vsize=187820 CPUtime=203.5 cores=1,3,5,7
/proc/25613/stat : 25613 (plingeling) S 25611 25613 18974 0 -1 4202496 159671 1254 0 0 20323 27 0 0 21 0 5 0 654481535 192327680 18399 33554432000 4194304 4417452 140734559798816 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/25613/statm: 46955 18399 179 55 0 44841 0
[pid=25613/tid=25631] ppid=25611 vsize=187820 CPUtime=50.95 cores=1,3,5,7
/proc/25613/task/25631/stat : 25631 (plingeling) R 25611 25613 18974 0 -1 4202560 48453 1254 0 0 5087 8 0 0 25 0 5 0 654481541 192327680 18399 33554432000 4194304 4417452 140734559798816 18446744073709551615 4251172 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25613/tid=25632] ppid=25611 vsize=187820 CPUtime=50.91 cores=1,3,5,7
/proc/25613/task/25632/stat : 25632 (plingeling) R 25611 25613 18974 0 -1 4202560 38000 1254 0 0 5084 7 0 0 25 0 5 0 654481541 192327680 18399 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25613/tid=25633] ppid=25611 vsize=187820 CPUtime=50.93 cores=1,3,5,7
/proc/25613/task/25633/stat : 25633 (plingeling) R 25611 25613 18974 0 -1 4202560 48979 1254 0 0 5086 7 0 0 25 0 5 0 654481541 192327680 18399 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364380 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25613/tid=25634] ppid=25611 vsize=187820 CPUtime=50.63 cores=1,3,5,7
/proc/25613/task/25634/stat : 25634 (plingeling) R 25611 25613 18974 0 -1 4202560 21583 1254 0 0 5060 3 0 0 25 0 5 0 654481541 192327680 18399 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 203.5
Current children cumulated vsize (KiB) 187820

[startup+102.31 s]
/proc/loadavg: 6.98 3.65 2.58 9/271 25641
/proc/meminfo: memFree=22866252/32951124 swapFree=61565116/67111528
[pid=25613] ppid=25611 vsize=225532 CPUtime=407.92 cores=1,3,5,7
/proc/25613/stat : 25613 (plingeling) S 25611 25613 18974 0 -1 4202496 263322 1254 0 0 40748 44 0 0 21 0 5 0 654481535 230944768 27827 33554432000 4194304 4417452 140734559798816 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/25613/statm: 56383 27827 179 55 0 54269 0
[pid=25613/tid=25631] ppid=25611 vsize=225532 CPUtime=102.16 cores=1,3,5,7
/proc/25613/task/25631/stat : 25631 (plingeling) R 25611 25613 18974 0 -1 4202560 83766 1254 0 0 10203 13 0 0 25 0 5 0 654481541 230944768 27827 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25613/tid=25632] ppid=25611 vsize=225532 CPUtime=102.11 cores=1,3,5,7
/proc/25613/task/25632/stat : 25632 (plingeling) R 25611 25613 18974 0 -1 4202560 63232 1254 0 0 10200 11 0 0 25 0 5 0 654481541 230944768 27827 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25613/tid=25633] ppid=25611 vsize=225532 CPUtime=102.15 cores=1,3,5,7
/proc/25613/task/25633/stat : 25633 (plingeling) R 25611 25613 18974 0 -1 4202560 73154 1254 0 0 10204 11 0 0 25 0 5 0 654481541 230944768 27827 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25613/tid=25634] ppid=25611 vsize=225532 CPUtime=101.42 cores=1,3,5,7
/proc/25613/task/25634/stat : 25634 (plingeling) R 25611 25613 18974 0 -1 4202560 40514 1254 0 0 10135 7 0 0 25 0 5 0 654481541 230944768 27827 33554432000 4194304 4417452 140734559798816 18446744073709551615 4366126 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 407.92
Current children cumulated vsize (KiB) 225532

[startup+162.301 s]
/proc/loadavg: 7.67 4.46 2.92 9/271 25642
/proc/meminfo: memFree=22787356/32951124 swapFree=61565116/67111528
[pid=25613] ppid=25611 vsize=251764 CPUtime=647.4 cores=1,3,5,7
/proc/25613/stat : 25613 (plingeling) S 25611 25613 18974 0 -1 4202496 343165 1254 0 0 64686 54 0 0 21 0 5 0 654481535 257806336 34385 33554432000 4194304 4417452 140734559798816 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/25613/statm: 62941 34385 179 55 0 60827 0
[pid=25613/tid=25631] ppid=25611 vsize=251764 CPUtime=162.16 cores=1,3,5,7
/proc/25613/task/25631/stat : 25631 (plingeling) R 25611 25613 18974 0 -1 4202560 112918 1254 0 0 16199 17 0 0 25 0 5 0 654481541 257806336 34385 33554432000 4194304 4417452 140734559798816 18446744073709551615 4264959 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25613/tid=25632] ppid=25611 vsize=251764 CPUtime=162.11 cores=1,3,5,7
/proc/25613/task/25632/stat : 25632 (plingeling) R 25611 25613 18974 0 -1 4202560 86743 1254 0 0 16197 14 0 0 25 0 5 0 654481541 257806336 34385 33554432000 4194304 4417452 140734559798816 18446744073709551615 4365962 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25613/tid=25633] ppid=25611 vsize=251764 CPUtime=162.13 cores=1,3,5,7
/proc/25613/task/25633/stat : 25633 (plingeling) R 25611 25613 18974 0 -1 4202560 82296 1254 0 0 16201 12 0 0 25 0 5 0 654481541 257806336 34385 33554432000 4194304 4417452 140734559798816 18446744073709551615 4357902 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25613/tid=25634] ppid=25611 vsize=251764 CPUtime=160.92 cores=1,3,5,7
/proc/25613/task/25634/stat : 25634 (plingeling) R 25611 25613 18974 0 -1 4202560 58552 1254 0 0 16083 9 0 0 25 0 5 0 654481541 257806336 34385 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 647.4
Current children cumulated vsize (KiB) 251764


################
# More data... #
################

[pid=25613/tid=25632] ppid=25611 vsize=352976 CPUtime=702.1 cores=1,3,5,7
/proc/25613/task/25632/stat : 25632 (plingeling) R 25611 25613 18974 0 -1 4202560 176164 1254 0 0 70184 26 0 0 25 0 5 0 654481541 361447424 59720 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25613/tid=25633] ppid=25611 vsize=352976 CPUtime=702.15 cores=1,3,5,7
/proc/25613/task/25633/stat : 25633 (plingeling) R 25611 25613 18974 0 -1 4202560 207422 1254 0 0 70185 30 0 0 25 0 5 0 654481541 361447424 59720 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364711 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25613/tid=25634] ppid=25611 vsize=352976 CPUtime=696.52 cores=1,3,5,7
/proc/25613/task/25634/stat : 25634 (plingeling) R 25611 25613 18974 0 -1 4202560 146547 1254 0 0 69631 21 0 0 25 0 5 0 654481541 361447424 59720 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364972 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2803.03
Current children cumulated vsize (KiB) 352976

[startup+762.31 s]
/proc/loadavg: 8.13 7.69 5.42 9/271 25658
/proc/meminfo: memFree=22545824/32951124 swapFree=61565116/67111528
[pid=25613] ppid=25611 vsize=407804 CPUtime=3042.57 cores=1,3,5,7
/proc/25613/stat : 25613 (plingeling) S 25611 25613 18974 0 -1 4202496 844350 1254 0 0 304130 127 0 0 21 0 5 0 654481535 417591296 73396 33554432000 4194304 4417452 140734559798816 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/25613/statm: 101951 73396 180 55 0 99837 0
[pid=25613/tid=25631] ppid=25611 vsize=407804 CPUtime=762.18 cores=1,3,5,7
/proc/25613/task/25631/stat : 25631 (plingeling) R 25611 25613 18974 0 -1 4202560 284556 1254 0 0 76175 43 0 0 25 0 5 0 654481541 417591296 73396 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364583 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25613/tid=25632] ppid=25611 vsize=407804 CPUtime=762.12 cores=1,3,5,7
/proc/25613/task/25632/stat : 25632 (plingeling) R 25611 25613 18974 0 -1 4202560 186852 1254 0 0 76183 29 0 0 25 0 5 0 654481541 417591296 73396 33554432000 4194304 4417452 140734559798816 18446744073709551615 4321595 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25613/tid=25633] ppid=25611 vsize=407804 CPUtime=762.16 cores=1,3,5,7
/proc/25613/task/25633/stat : 25633 (plingeling) R 25611 25613 18974 0 -1 4202560 219509 1254 0 0 76185 31 0 0 25 0 5 0 654481541 417591296 73396 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364653 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25613/tid=25634] ppid=25611 vsize=407804 CPUtime=756.04 cores=1,3,5,7
/proc/25613/task/25634/stat : 25634 (plingeling) R 25611 25613 18974 0 -1 4202560 150777 1254 0 0 75582 22 0 0 25 0 5 0 654481541 417591296 73396 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364693 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3042.57
Current children cumulated vsize (KiB) 407804

[startup+822.301 s]
/proc/loadavg: 8.05 7.75 5.58 9/271 25660
/proc/meminfo: memFree=22505140/32951124 swapFree=61565116/67111528
[pid=25613] ppid=25611 vsize=446204 CPUtime=3282.05 cores=1,3,5,7
/proc/25613/stat : 25613 (plingeling) S 25611 25613 18974 0 -1 4202496 874362 1254 0 0 328073 132 0 0 21 0 5 0 654481535 456912896 82996 33554432000 4194304 4417452 140734559798816 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/25613/statm: 111551 82996 180 55 0 109437 0
[pid=25613/tid=25631] ppid=25611 vsize=446204 CPUtime=822.18 cores=1,3,5,7
/proc/25613/task/25631/stat : 25631 (plingeling) R 25611 25613 18974 0 -1 4202560 288898 1254 0 0 82174 44 0 0 25 0 5 0 654481541 456912896 82996 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25613/tid=25632] ppid=25611 vsize=446204 CPUtime=822.11 cores=1,3,5,7
/proc/25613/task/25632/stat : 25632 (plingeling) R 25611 25613 18974 0 -1 4202560 192700 1254 0 0 82181 30 0 0 25 0 5 0 654481541 456912896 82996 33554432000 4194304 4417452 140734559798816 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25613/tid=25633] ppid=25611 vsize=446204 CPUtime=822.16 cores=1,3,5,7
/proc/25613/task/25633/stat : 25633 (plingeling) R 25611 25613 18974 0 -1 4202560 231110 1254 0 0 82182 34 0 0 25 0 5 0 654481541 456912896 82996 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25613/tid=25634] ppid=25611 vsize=446204 CPUtime=815.54 cores=1,3,5,7
/proc/25613/task/25634/stat : 25634 (plingeling) R 25611 25613 18974 0 -1 4202560 158998 1254 0 0 81531 23 0 0 25 0 5 0 654481541 456912896 82996 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3282.05
Current children cumulated vsize (KiB) 446204

[startup+882.301 s]
/proc/loadavg: 8.02 7.79 5.73 9/271 25661
/proc/meminfo: memFree=22500888/32951124 swapFree=61565116/67111528
[pid=25613] ppid=25611 vsize=454828 CPUtime=3521.57 cores=1,3,5,7
/proc/25613/stat : 25613 (plingeling) S 25611 25613 18974 0 -1 4202496 880479 1254 0 0 352023 134 0 0 21 0 5 0 654481535 465743872 85152 33554432000 4194304 4417452 140734559798816 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/25613/statm: 113707 85152 180 55 0 111593 0
[pid=25613/tid=25631] ppid=25611 vsize=454828 CPUtime=882.18 cores=1,3,5,7
/proc/25613/task/25631/stat : 25631 (plingeling) R 25611 25613 18974 0 -1 4202560 289185 1254 0 0 88174 44 0 0 25 0 5 0 654481541 465743872 85152 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25613/tid=25632] ppid=25611 vsize=454828 CPUtime=882.11 cores=1,3,5,7
/proc/25613/task/25632/stat : 25632 (plingeling) R 25611 25613 18974 0 -1 4202560 192982 1254 0 0 88181 30 0 0 25 0 5 0 654481541 465743872 85152 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25613/tid=25633] ppid=25611 vsize=454828 CPUtime=882.16 cores=1,3,5,7
/proc/25613/task/25633/stat : 25633 (plingeling) R 25611 25613 18974 0 -1 4202560 233369 1254 0 0 88181 35 0 0 25 0 5 0 654481541 465743872 85152 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25613/tid=25634] ppid=25611 vsize=454828 CPUtime=875.05 cores=1,3,5,7
/proc/25613/task/25634/stat : 25634 (plingeling) R 25611 25613 18974 0 -1 4202560 162287 1254 0 0 87481 24 0 0 25 0 5 0 654481541 465743872 85152 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364542 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3521.57
Current children cumulated vsize (KiB) 454828

[startup+942.301 s]
/proc/loadavg: 8.21 7.89 5.89 9/271 25663
/proc/meminfo: memFree=22518732/32951124 swapFree=61565116/67111528
[pid=25613] ppid=25611 vsize=454004 CPUtime=3761.08 cores=1,3,5,7
/proc/25613/stat : 25613 (plingeling) S 25611 25613 18974 0 -1 4202496 883600 1254 0 0 375974 134 0 0 21 0 5 0 654481535 464900096 84946 33554432000 4194304 4417452 140734559798816 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/25613/statm: 113501 84946 180 55 0 111387 0
[pid=25613/tid=25631] ppid=25611 vsize=454004 CPUtime=942.18 cores=1,3,5,7
/proc/25613/task/25631/stat : 25631 (plingeling) R 25611 25613 18974 0 -1 4202560 289346 1254 0 0 94174 44 0 0 25 0 5 0 654481541 464900096 84946 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25613/tid=25632] ppid=25611 vsize=454004 CPUtime=942.11 cores=1,3,5,7
/proc/25613/task/25632/stat : 25632 (plingeling) R 25611 25613 18974 0 -1 4202560 193309 1254 0 0 94181 30 0 0 25 0 5 0 654481541 464900096 84946 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364765 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25613/tid=25633] ppid=25611 vsize=454004 CPUtime=942.16 cores=1,3,5,7
/proc/25613/task/25633/stat : 25633 (plingeling) R 25611 25613 18974 0 -1 4202560 234579 1254 0 0 94181 35 0 0 25 0 5 0 654481541 464900096 84946 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364741 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25613/tid=25634] ppid=25611 vsize=454004 CPUtime=934.57 cores=1,3,5,7
/proc/25613/task/25634/stat : 25634 (plingeling) R 25611 25613 18974 0 -1 4202560 163710 1254 0 0 93433 24 0 0 25 0 5 0 654481541 464900096 84946 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364812 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3761.08
Current children cumulated vsize (KiB) 454004

[startup+1002.3 s]
/proc/loadavg: 8.08 7.90 6.02 9/271 25665
/proc/meminfo: memFree=22493904/32951124 swapFree=61565116/67111528
[pid=25613] ppid=25611 vsize=454856 CPUtime=4000.6 cores=1,3,5,7
/proc/25613/stat : 25613 (plingeling) S 25611 25613 18974 0 -1 4202496 902840 1254 0 0 399923 137 0 0 21 0 5 0 654481535 465772544 85159 33554432000 4194304 4417452 140734559798816 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/25613/statm: 113714 85159 180 55 0 111600 0
[pid=25613/tid=25631] ppid=25611 vsize=454856 CPUtime=1002.18 cores=1,3,5,7
/proc/25613/task/25631/stat : 25631 (plingeling) R 25611 25613 18974 0 -1 4202560 295765 1254 0 0 100173 45 0 0 25 0 5 0 654481541 465772544 85159 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25613/tid=25632] ppid=25611 vsize=454856 CPUtime=1002.1 cores=1,3,5,7
/proc/25613/task/25632/stat : 25632 (plingeling) R 25611 25613 18974 0 -1 4202560 197832 1254 0 0 100180 30 0 0 25 0 5 0 654481541 465772544 85159 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25613/tid=25633] ppid=25611 vsize=454856 CPUtime=1002.17 cores=1,3,5,7
/proc/25613/task/25633/stat : 25633 (plingeling) R 25611 25613 18974 0 -1 4202560 241937 1254 0 0 100181 36 0 0 25 0 5 0 654481541 465772544 85159 33554432000 4194304 4417452 140734559798816 18446744073709551615 4366400 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25613/tid=25634] ppid=25611 vsize=454856 CPUtime=994.07 cores=1,3,5,7
/proc/25613/task/25634/stat : 25634 (plingeling) R 25611 25613 18974 0 -1 4202560 164650 1254 0 0 99383 24 0 0 25 0 5 0 654481541 465772544 85159 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4000.6
Current children cumulated vsize (KiB) 454856

[startup+1062.3 s]
/proc/loadavg: 8.22 7.98 6.16 9/271 25666
/proc/meminfo: memFree=22517204/32951124 swapFree=61565116/67111528
[pid=25613] ppid=25611 vsize=455624 CPUtime=4240.12 cores=1,3,5,7
/proc/25613/stat : 25613 (plingeling) S 25611 25613 18974 0 -1 4202496 906428 1254 0 0 423875 137 0 0 21 0 5 0 654481535 466558976 85351 33554432000 4194304 4417452 140734559798816 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/25613/statm: 113906 85351 180 55 0 111792 0
[pid=25613/tid=25631] ppid=25611 vsize=455624 CPUtime=1062.19 cores=1,3,5,7
/proc/25613/task/25631/stat : 25631 (plingeling) R 25611 25613 18974 0 -1 4202560 295951 1254 0 0 106174 45 0 0 25 0 5 0 654481541 466558976 85351 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25613/tid=25632] ppid=25611 vsize=455624 CPUtime=1062.1 cores=1,3,5,7
/proc/25613/task/25632/stat : 25632 (plingeling) R 25611 25613 18974 0 -1 4202560 198261 1254 0 0 106180 30 0 0 25 0 5 0 654481541 466558976 85351 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25613/tid=25633] ppid=25611 vsize=455624 CPUtime=1062.17 cores=1,3,5,7
/proc/25613/task/25633/stat : 25633 (plingeling) R 25611 25613 18974 0 -1 4202560 244107 1254 0 0 106181 36 0 0 25 0 5 0 654481541 466558976 85351 33554432000 4194304 4417452 140734559798816 18446744073709551615 4353962 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25613/tid=25634] ppid=25611 vsize=455624 CPUtime=1053.59 cores=1,3,5,7
/proc/25613/task/25634/stat : 25634 (plingeling) R 25611 25613 18974 0 -1 4202560 165453 1254 0 0 105335 24 0 0 25 0 5 0 654481541 466558976 85351 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364558 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4240.12
Current children cumulated vsize (KiB) 455624

[startup+1122.3 s]
/proc/loadavg: 8.12 7.99 6.28 9/271 25668
/proc/meminfo: memFree=22499688/32951124 swapFree=61565116/67111528
[pid=25613] ppid=25611 vsize=456096 CPUtime=4479.63 cores=1,3,5,7
/proc/25613/stat : 25613 (plingeling) S 25611 25613 18974 0 -1 4202496 907746 1254 0 0 447826 137 0 0 21 0 5 0 654481535 467042304 85469 33554432000 4194304 4417452 140734559798816 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/25613/statm: 114024 85469 180 55 0 111910 0
[pid=25613/tid=25631] ppid=25611 vsize=456096 CPUtime=1122.19 cores=1,3,5,7
/proc/25613/task/25631/stat : 25631 (plingeling) R 25611 25613 18974 0 -1 4202560 296028 1254 0 0 112174 45 0 0 25 0 5 0 654481541 467042304 85469 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25613/tid=25632] ppid=25611 vsize=456096 CPUtime=1122.11 cores=1,3,5,7
/proc/25613/task/25632/stat : 25632 (plingeling) R 25611 25613 18974 0 -1 4202560 198463 1254 0 0 112180 31 0 0 25 0 5 0 654481541 467042304 85469 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364466 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25613/tid=25633] ppid=25611 vsize=456096 CPUtime=1122.17 cores=1,3,5,7
/proc/25613/task/25633/stat : 25633 (plingeling) R 25611 25613 18974 0 -1 4202560 244236 1254 0 0 112181 36 0 0 25 0 5 0 654481541 467042304 85469 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25613/tid=25634] ppid=25611 vsize=456096 CPUtime=1113.09 cores=1,3,5,7
/proc/25613/task/25634/stat : 25634 (plingeling) R 25611 25613 18974 0 -1 4202560 166363 1254 0 0 111285 24 0 0 25 0 5 0 654481541 467042304 85469 33554432000 4194304 4417452 140734559798816 18446744073709551615 4366099 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4479.63
Current children cumulated vsize (KiB) 456096

[startup+1182.3 s]
/proc/loadavg: 8.13 8.02 6.39 9/271 25669
/proc/meminfo: memFree=22426384/32951124 swapFree=61565116/67111528
[pid=25613] ppid=25611 vsize=493964 CPUtime=4719.15 cores=1,3,5,7
/proc/25613/stat : 25613 (plingeling) S 25611 25613 18974 0 -1 4202496 931900 1254 0 0 471775 140 0 0 21 0 5 0 654481535 505819136 94936 33554432000 4194304 4417452 140734559798816 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/25613/statm: 123491 94936 180 55 0 121377 0
[pid=25613/tid=25631] ppid=25611 vsize=493964 CPUtime=1182.18 cores=1,3,5,7
/proc/25613/task/25631/stat : 25631 (plingeling) R 25611 25613 18974 0 -1 4202560 300357 1254 0 0 118173 45 0 0 25 0 5 0 654481541 505819136 94936 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25613/tid=25632] ppid=25611 vsize=493964 CPUtime=1182.11 cores=1,3,5,7
/proc/25613/task/25632/stat : 25632 (plingeling) R 25611 25613 18974 0 -1 4202560 207021 1254 0 0 118179 32 0 0 25 0 5 0 654481541 505819136 94936 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25613/tid=25633] ppid=25611 vsize=493964 CPUtime=1182.16 cores=1,3,5,7
/proc/25613/task/25633/stat : 25633 (plingeling) R 25611 25613 18974 0 -1 4202560 250546 1254 0 0 118180 36 0 0 25 0 5 0 654481541 505819136 94936 33554432000 4194304 4417452 140734559798816 18446744073709551615 4357207 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25613/tid=25634] ppid=25611 vsize=493964 CPUtime=1172.61 cores=1,3,5,7
/proc/25613/task/25634/stat : 25634 (plingeling) R 25611 25613 18974 0 -1 4202560 171320 1254 0 0 117236 25 0 0 25 0 5 0 654481541 505819136 94936 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4719.15
Current children cumulated vsize (KiB) 493964



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1202.6 s]
/proc/loadavg: 8.24 8.05 6.44 9/271 25670
/proc/meminfo: memFree=22445836/32951124 swapFree=61565116/67111528
[pid=25613] ppid=25611 vsize=501716 CPUtime=4800.19 cores=1,3,5,7
/proc/25613/stat : 25613 (plingeling) S 25611 25613 18974 0 -1 4202496 933913 1254 0 0 479879 140 0 0 21 0 5 0 654481535 513757184 96874 33554432000 4194304 4417452 140734559798816 18446744073709551615 246528636901 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/25613/statm: 125429 96874 180 55 0 123315 0
[pid=25613/tid=25631] ppid=25611 vsize=501716 CPUtime=1202.49 cores=1,3,5,7
/proc/25613/task/25631/stat : 25631 (plingeling) R 25611 25613 18974 0 -1 4202560 300456 1254 0 0 120204 45 0 0 25 0 5 0 654481541 513757184 96874 33554432000 4194304 4417452 140734559798816 18446744073709551615 4366316 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=25613/tid=25632] ppid=25611 vsize=501716 CPUtime=1202.42 cores=1,3,5,7
/proc/25613/task/25632/stat : 25632 (plingeling) R 25611 25613 18974 0 -1 4202560 207021 1254 0 0 120210 32 0 0 25 0 5 0 654481541 513757184 96874 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=25613/tid=25633] ppid=25611 vsize=501716 CPUtime=1202.48 cores=1,3,5,7
/proc/25613/task/25633/stat : 25633 (plingeling) R 25611 25613 18974 0 -1 4202560 252145 1254 0 0 120211 37 0 0 25 0 5 0 654481541 513757184 96874 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25613/tid=25634] ppid=25611 vsize=501716 CPUtime=1192.74 cores=1,3,5,7
/proc/25613/task/25634/stat : 25634 (plingeling) R 25611 25613 18974 0 -1 4202560 171635 1254 0 0 119249 25 0 0 25 0 5 0 654481541 513757184 96874 33554432000 4194304 4417452 140734559798816 18446744073709551615 4364972 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4800.19
Current children cumulated vsize (KiB) 501716

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1202.63
CPU time (s): 4800.25
CPU user time (s): 4798.83
CPU system time (s): 1.42478
CPU usage (%): 399.146
Max. virtual memory (cumulated for all children) (KiB): 541080

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4798.83
system time used= 1.42478
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 935167
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 119
involuntary context switches= 22771

runsolver used 2.6146 second user time and 7.2409 second system time

The end

Launcher Data 

Begin job on node134 at 2011-05-15 05:07:13
IDJOB=3371475
IDBENCH=83408
IDSOLVER=1848
FILE ID=node134/3371475-1305428833
RUNJOBID= node134-1305428833-25600
PBS_JOBID= 13324593
Free space on /tmp= 73508 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/application/kullmann/AES/Benchmarks/128/aes_128_3_keyfind_1.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3371475-1305428833/watcher-3371475-1305428833 -o /tmp/evaluation-result-3371475-1305428833/solver-3371475-1305428833 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3371475-1305428833.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= ff59ad0e88befc51bfa041e45143dc53
RANDOM SEED=249366464

node134.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 7186.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      23082268 kB
Buffers:        266048 kB
Cached:        6054988 kB
SwapCached:      13768 kB
Active:         943068 kB
Inactive:      8711544 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      23082268 kB
SwapTotal:    67111528 kB
SwapFree:     61565116 kB
Dirty:             788 kB
Writeback:           0 kB
AnonPages:     3327548 kB
Mapped:          11800 kB
Slab:           122824 kB
PageTables:      30980 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 38876156 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73508 MiB
End job on node134 at 2011-05-15 05:27:18