Trace number 3291415

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 4800.26 1201.82

General information on the benchmark

NameSAT03/industrial/goldberg/hard_eq_check/
k2mul.miter.shuffled-as.sat03-355.cnf
MD5SUM77e0700cbaeeb3d63a4820608fc84761
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark45.3381
Satisfiable
(Un)Satisfiability was proved
Number of variables11680
Number of clauses74581
Sum of the clauses size252050
Maximum clause length86
Minimum clause length1
Number of clauses of size 14
Number of clauses of size 217176
Number of clauses of size 334696
Number of clauses of size 40
Number of clauses of size 522704
Number of clauses of size over 51

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3291415-1304101452/watcher-3291415-1304101452 -o /tmp/evaluation-result-3291415-1304101452/solver-3291415-1304101452 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3291415-1304101452.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.82 7.81 7.95 2/174 5698
/proc/meminfo: memFree=23618900/32950928 swapFree=67071984/67111528
[pid=5698] ppid=5696 vsize=10620 CPUtime=0 cores=0,2,4,6
/proc/5698/stat : 5698 (plingeling) R 5696 5698 5016 0 -1 4202496 758 1256 0 0 0 0 0 0 23 0 1 0 600375847 10874880 655 33554432000 4194304 4417332 140737359425312 18446744073709551615 260905087956 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/5698/statm: 2655 655 148 55 0 541 0

[startup+0.028745 s]
/proc/loadavg: 6.82 7.81 7.95 2/174 5698
/proc/meminfo: memFree=23618900/32950928 swapFree=67071984/67111528
[pid=5698] ppid=5696 vsize=16504 CPUtime=0.02 cores=0,2,4,6
/proc/5698/stat : 5698 (plingeling) R 5696 5698 5016 0 -1 4202496 2245 1256 0 0 2 0 0 0 23 0 1 0 600375847 16900096 2134 33554432000 4194304 4417332 140737359425312 18446744073709551615 4264911 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/5698/statm: 4126 2134 148 55 0 2012 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 16504

[startup+0.100849 s]
/proc/loadavg: 6.82 7.81 7.95 2/174 5698
/proc/meminfo: memFree=23618900/32950928 swapFree=67071984/67111528
[pid=5698] ppid=5696 vsize=24316 CPUtime=0.09 cores=0,2,4,6
/proc/5698/stat : 5698 (plingeling) R 5696 5698 5016 0 -1 4202496 4198 1256 0 0 9 0 0 0 23 0 1 0 600375847 24899584 4087 33554432000 4194304 4417332 140737359425312 18446744073709551615 4231321 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/5698/statm: 6079 4087 148 55 0 3965 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 24316

[startup+0.300747 s]
/proc/loadavg: 6.82 7.81 7.95 2/174 5698
/proc/meminfo: memFree=23618900/32950928 swapFree=67071984/67111528
[pid=5698] ppid=5696 vsize=74504 CPUtime=0.74 cores=0,2,4,6
/proc/5698/stat : 5698 (plingeling) S 5696 5698 5016 0 -1 4202496 8054 1256 0 0 74 0 0 0 23 0 5 0 600375847 76292096 6406 33554432000 4194304 4417332 140737359425312 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/5698/statm: 18626 6406 172 55 0 16512 0
Current children cumulated CPU time (s) 0.74
Current children cumulated vsize (KiB) 74504

[startup+0.700623 s]
/proc/loadavg: 6.82 7.81 7.95 2/174 5698
/proc/meminfo: memFree=23618900/32950928 swapFree=67071984/67111528
[pid=5698] ppid=5696 vsize=76052 CPUtime=2.34 cores=0,2,4,6
/proc/5698/stat : 5698 (plingeling) S 5696 5698 5016 0 -1 4202496 9516 1256 0 0 234 0 0 0 23 0 5 0 600375847 77877248 6797 33554432000 4194304 4417332 140737359425312 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/5698/statm: 19013 6797 176 55 0 16899 0
Current children cumulated CPU time (s) 2.34
Current children cumulated vsize (KiB) 76052

[startup+1.50047 s]
/proc/loadavg: 6.82 7.81 7.95 9/186 5723
/proc/meminfo: memFree=23555076/32950928 swapFree=67071984/67111528
[pid=5698] ppid=5696 vsize=144928 CPUtime=5.54 cores=0,2,4,6
/proc/5698/stat : 5698 (plingeling) S 5696 5698 5016 0 -1 4202496 10394 1256 0 0 554 0 0 0 23 0 5 0 600375847 148406272 7675 33554432000 4194304 4417332 140737359425312 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/5698/statm: 36232 7675 180 55 0 34118 0
[pid=5698/tid=5716] ppid=5696 vsize=144928 CPUtime=1.35 cores=0,2,4,6
/proc/5698/task/5716/stat : 5716 (plingeling) R 5696 5698 5016 0 -1 4202560 1205 1256 0 0 135 0 0 0 25 0 5 0 600375862 148406272 7675 33554432000 4194304 4417332 140737359425312 18446744073709551615 4253228 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5698/tid=5717] ppid=5696 vsize=144928 CPUtime=1.34 cores=0,2,4,6
/proc/5698/task/5717/stat : 5717 (plingeling) R 5696 5698 5016 0 -1 4202560 1120 1256 0 0 134 0 0 0 25 0 5 0 600375862 148406272 7675 33554432000 4194304 4417332 140737359425312 18446744073709551615 4273113 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5698/tid=5718] ppid=5696 vsize=144928 CPUtime=1.35 cores=0,2,4,6
/proc/5698/task/5718/stat : 5718 (plingeling) R 5696 5698 5016 0 -1 4202560 1151 1256 0 0 135 0 0 0 25 0 5 0 600375862 148406272 7675 33554432000 4194304 4417332 140737359425312 18446744073709551615 4232656 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5698/tid=5719] ppid=5696 vsize=144928 CPUtime=1.35 cores=0,2,4,6
/proc/5698/task/5719/stat : 5719 (plingeling) R 5696 5698 5016 0 -1 4202560 148 1256 0 0 135 0 0 0 25 0 5 0 600375862 148406272 7675 33554432000 4194304 4417332 140737359425312 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 5.54
Current children cumulated vsize (KiB) 144928

[startup+3.10118 s]
/proc/loadavg: 6.82 7.81 7.95 9/186 5723
/proc/meminfo: memFree=23542608/32950928 swapFree=67071984/67111528
[pid=5698] ppid=5696 vsize=144928 CPUtime=11.93 cores=0,2,4,6
/proc/5698/stat : 5698 (plingeling) S 5696 5698 5016 0 -1 4202496 10396 1256 0 0 1193 0 0 0 23 0 5 0 600375847 148406272 7677 33554432000 4194304 4417332 140737359425312 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/5698/statm: 36232 7677 180 55 0 34118 0
[pid=5698/tid=5716] ppid=5696 vsize=144928 CPUtime=2.95 cores=0,2,4,6
/proc/5698/task/5716/stat : 5716 (plingeling) R 5696 5698 5016 0 -1 4202560 1207 1256 0 0 295 0 0 0 25 0 5 0 600375862 148406272 7677 33554432000 4194304 4417332 140737359425312 18446744073709551615 4236766 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5698/tid=5717] ppid=5696 vsize=144928 CPUtime=2.93 cores=0,2,4,6
/proc/5698/task/5717/stat : 5717 (plingeling) R 5696 5698 5016 0 -1 4202560 1120 1256 0 0 293 0 0 0 25 0 5 0 600375862 148406272 7677 33554432000 4194304 4417332 140737359425312 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5698/tid=5718] ppid=5696 vsize=144928 CPUtime=2.95 cores=0,2,4,6
/proc/5698/task/5718/stat : 5718 (plingeling) R 5696 5698 5016 0 -1 4202560 1151 1256 0 0 295 0 0 0 25 0 5 0 600375862 148406272 7677 33554432000 4194304 4417332 140737359425312 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5698/tid=5719] ppid=5696 vsize=144928 CPUtime=2.95 cores=0,2,4,6
/proc/5698/task/5719/stat : 5719 (plingeling) R 5696 5698 5016 0 -1 4202560 148 1256 0 0 295 0 0 0 25 0 5 0 600375862 148406272 7677 33554432000 4194304 4417332 140737359425312 18446744073709551615 4364498 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 11.93
Current children cumulated vsize (KiB) 144928

[startup+6.3006 s]
/proc/loadavg: 7.00 7.83 7.96 9/186 5723
/proc/meminfo: memFree=23536208/32950928 swapFree=67071984/67111528
[pid=5698] ppid=5696 vsize=149176 CPUtime=24.71 cores=0,2,4,6
/proc/5698/stat : 5698 (plingeling) S 5696 5698 5016 0 -1 4202496 11458 1256 0 0 2471 0 0 0 23 0 5 0 600375847 152756224 8739 33554432000 4194304 4417332 140737359425312 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/5698/statm: 37294 8739 180 55 0 35180 0
[pid=5698/tid=5716] ppid=5696 vsize=149176 CPUtime=6.15 cores=0,2,4,6
/proc/5698/task/5716/stat : 5716 (plingeling) R 5696 5698 5016 0 -1 4202560 1207 1256 0 0 615 0 0 0 25 0 5 0 600375862 152756224 8739 33554432000 4194304 4417332 140737359425312 18446744073709551615 4364544 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5698/tid=5717] ppid=5696 vsize=149176 CPUtime=6.12 cores=0,2,4,6
/proc/5698/task/5717/stat : 5717 (plingeling) R 5696 5698 5016 0 -1 4202560 1120 1256 0 0 612 0 0 0 25 0 5 0 600375862 152756224 8739 33554432000 4194304 4417332 140737359425312 18446744073709551615 4236923 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5698/tid=5718] ppid=5696 vsize=149176 CPUtime=6.15 cores=0,2,4,6
/proc/5698/task/5718/stat : 5718 (plingeling) R 5696 5698 5016 0 -1 4202560 1295 1256 0 0 615 0 0 0 25 0 5 0 600375862 152756224 8739 33554432000 4194304 4417332 140737359425312 18446744073709551615 4364742 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5698/tid=5719] ppid=5696 vsize=149176 CPUtime=6.15 cores=0,2,4,6
/proc/5698/task/5719/stat : 5719 (plingeling) R 5696 5698 5016 0 -1 4202560 1066 1256 0 0 615 0 0 0 25 0 5 0 600375862 152756224 8739 33554432000 4194304 4417332 140737359425312 18446744073709551615 4358027 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 24.71
Current children cumulated vsize (KiB) 149176

[startup+12.7004 s]
/proc/loadavg: 7.08 7.83 7.96 9/186 5724
/proc/meminfo: memFree=23522372/32950928 swapFree=67071984/67111528
[pid=5698] ppid=5696 vsize=153416 CPUtime=50.28 cores=0,2,4,6
/proc/5698/stat : 5698 (plingeling) S 5696 5698 5016 0 -1 4202496 16044 1256 0 0 5027 1 0 0 23 0 5 0 600375847 157097984 9820 33554432000 4194304 4417332 140737359425312 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/5698/statm: 38354 9820 181 55 0 36240 0
[pid=5698/tid=5716] ppid=5696 vsize=153416 CPUtime=12.55 cores=0,2,4,6
/proc/5698/task/5716/stat : 5716 (plingeling) R 5696 5698 5016 0 -1 4202560 1208 1256 0 0 1255 0 0 0 25 0 5 0 600375862 157097984 9820 33554432000 4194304 4417332 140737359425312 18446744073709551615 4357806 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5698/tid=5717] ppid=5696 vsize=153416 CPUtime=12.48 cores=0,2,4,6
/proc/5698/task/5717/stat : 5717 (plingeling) R 5696 5698 5016 0 -1 4202560 1120 1256 0 0 1248 0 0 0 25 0 5 0 600375862 157097984 9820 33554432000 4194304 4417332 140737359425312 18446744073709551615 4237139 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5698/tid=5718] ppid=5696 vsize=153416 CPUtime=12.55 cores=0,2,4,6
/proc/5698/task/5718/stat : 5718 (plingeling) R 5696 5698 5016 0 -1 4202560 1606 1256 0 0 1255 0 0 0 25 0 5 0 600375862 157097984 9820 33554432000 4194304 4417332 140737359425312 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5698/tid=5719] ppid=5696 vsize=153416 CPUtime=12.54 cores=0,2,4,6
/proc/5698/task/5719/stat : 5719 (plingeling) R 5696 5698 5016 0 -1 4202560 5340 1256 0 0 1254 0 0 0 25 0 5 0 600375862 157097984 9820 33554432000 4194304 4417332 140737359425312 18446744073709551615 4364519 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 50.28
Current children cumulated vsize (KiB) 153416

[startup+25.5011 s]
/proc/loadavg: 7.35 7.86 7.96 9/186 5724
/proc/meminfo: memFree=23502296/32950928 swapFree=67071984/67111528
[pid=5698] ppid=5696 vsize=167076 CPUtime=101.41 cores=0,2,4,6
/proc/5698/stat : 5698 (plingeling) S 5696 5698 5016 0 -1 4202496 27345 1256 0 0 10139 2 0 0 23 0 5 0 600375847 171085824 13233 33554432000 4194304 4417332 140737359425312 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/5698/statm: 41769 13233 181 55 0 39655 0
[pid=5698/tid=5716] ppid=5696 vsize=167076 CPUtime=25.35 cores=0,2,4,6
/proc/5698/task/5716/stat : 5716 (plingeling) R 5696 5698 5016 0 -1 4202560 1604 1256 0 0 2535 0 0 0 25 0 5 0 600375862 171085824 13233 33554432000 4194304 4417332 140737359425312 18446744073709551615 4365097 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5698/tid=5717] ppid=5696 vsize=167076 CPUtime=25.21 cores=0,2,4,6
/proc/5698/task/5717/stat : 5717 (plingeling) R 5696 5698 5016 0 -1 4202560 3812 1256 0 0 2521 0 0 0 25 0 5 0 600375862 171085824 13233 33554432000 4194304 4417332 140737359425312 18446744073709551615 4364362 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5698/tid=5718] ppid=5696 vsize=167076 CPUtime=25.35 cores=0,2,4,6
/proc/5698/task/5718/stat : 5718 (plingeling) R 5696 5698 5016 0 -1 4202560 1745 1256 0 0 2535 0 0 0 25 0 5 0 600375862 171085824 13233 33554432000 4194304 4417332 140737359425312 18446744073709551615 4248999 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5698/tid=5719] ppid=5696 vsize=167076 CPUtime=25.35 cores=0,2,4,6
/proc/5698/task/5719/stat : 5719 (plingeling) R 5696 5698 5016 0 -1 4202560 13414 1256 0 0 2534 1 0 0 25 0 5 0 600375862 171085824 13233 33554432000 4194304 4417332 140737359425312 18446744073709551615 4364722 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 101.41
Current children cumulated vsize (KiB) 167076

[startup+51.1004 s]
/proc/loadavg: 7.57 7.87 7.96 9/186 5725
/proc/meminfo: memFree=23447368/32950928 swapFree=67071984/67111528
[pid=5698] ppid=5696 vsize=198900 CPUtime=203.67 cores=0,2,4,6
/proc/5698/stat : 5698 (plingeling) S 5696 5698 5016 0 -1 4202496 59629 1256 0 0 20359 8 0 0 23 0 5 0 600375847 203673600 21177 33554432000 4194304 4417332 140737359425312 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/5698/statm: 49725 21177 181 55 0 47611 0
[pid=5698/tid=5716] ppid=5696 vsize=198900 CPUtime=50.95 cores=0,2,4,6
/proc/5698/task/5716/stat : 5716 (plingeling) R 5696 5698 5016 0 -1 4202560 7582 1256 0 0 5095 0 0 0 25 0 5 0 600375862 203673600 21177 33554432000 4194304 4417332 140737359425312 18446744073709551615 4232664 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5698/tid=5717] ppid=5696 vsize=198900 CPUtime=50.67 cores=0,2,4,6
/proc/5698/task/5717/stat : 5717 (plingeling) R 5696 5698 5016 0 -1 4202560 9594 1256 0 0 5066 1 0 0 25 0 5 0 600375862 203673600 21177 33554432000 4194304 4417332 140737359425312 18446744073709551615 4232787 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5698/tid=5718] ppid=5696 vsize=198900 CPUtime=50.94 cores=0,2,4,6
/proc/5698/task/5718/stat : 5718 (plingeling) R 5696 5698 5016 0 -1 4202560 7224 1256 0 0 5094 0 0 0 25 0 5 0 600375862 203673600 21177 33554432000 4194304 4417332 140737359425312 18446744073709551615 18446744073699066094 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5698/tid=5719] ppid=5696 vsize=198900 CPUtime=50.94 cores=0,2,4,6
/proc/5698/task/5719/stat : 5719 (plingeling) R 5696 5698 5016 0 -1 4202560 28459 1256 0 0 5090 4 0 0 25 0 5 0 600375862 203673600 21177 33554432000 4194304 4417332 140737359425312 18446744073709551615 4250502 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 203.67
Current children cumulated vsize (KiB) 198900

[startup+102.307 s]
/proc/loadavg: 7.81 7.89 7.96 9/186 5726
/proc/meminfo: memFree=23378896/32950928 swapFree=67071984/67111528
[pid=5698] ppid=5696 vsize=201840 CPUtime=408.22 cores=0,2,4,6
/proc/5698/stat : 5698 (plingeling) S 5696 5698 5016 0 -1 4202496 133415 1256 0 0 40801 21 0 0 23 0 5 0 600375847 206684160 21914 33554432000 4194304 4417332 140737359425312 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/5698/statm: 50460 21914 181 55 0 48346 0
[pid=5698/tid=5716] ppid=5696 vsize=201840 CPUtime=102.15 cores=0,2,4,6
/proc/5698/task/5716/stat : 5716 (plingeling) R 5696 5698 5016 0 -1 4202560 21356 1256 0 0 10213 2 0 0 25 0 5 0 600375862 206684160 21914 33554432000 4194304 4417332 140737359425312 18446744073709551615 4353552 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5698/tid=5717] ppid=5696 vsize=201840 CPUtime=101.59 cores=0,2,4,6
/proc/5698/task/5717/stat : 5717 (plingeling) R 5696 5698 5016 0 -1 4202560 26135 1256 0 0 10155 4 0 0 25 0 5 0 600375862 206684160 21914 33554432000 4194304 4417332 140737359425312 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5698/tid=5718] ppid=5696 vsize=201840 CPUtime=102.15 cores=0,2,4,6
/proc/5698/task/5718/stat : 5718 (plingeling) R 5696 5698 5016 0 -1 4202560 18765 1256 0 0 10212 3 0 0 25 0 5 0 600375862 206684160 21914 33554432000 4194304 4417332 140737359425312 18446744073709551615 4364732 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5698/tid=5719] ppid=5696 vsize=201840 CPUtime=102.15 cores=0,2,4,6
/proc/5698/task/5719/stat : 5719 (plingeling) R 5696 5698 5016 0 -1 4202560 60389 1256 0 0 10205 10 0 0 25 0 5 0 600375862 206684160 21914 33554432000 4194304 4417332 140737359425312 18446744073709551615 4364329 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 408.22
Current children cumulated vsize (KiB) 201840

[startup+162.301 s]
/proc/loadavg: 7.93 7.90 7.96 9/186 5728
/proc/meminfo: memFree=23279048/32950928 swapFree=67071984/67111528
[pid=5698] ppid=5696 vsize=217776 CPUtime=647.88 cores=0,2,4,6
/proc/5698/stat : 5698 (plingeling) S 5696 5698 5016 0 -1 4202496 220414 1256 0 0 64752 36 0 0 23 0 5 0 600375847 223002624 25898 33554432000 4194304 4417332 140737359425312 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/5698/statm: 54444 25898 181 55 0 52330 0
[pid=5698/tid=5716] ppid=5696 vsize=217776 CPUtime=162.15 cores=0,2,4,6
/proc/5698/task/5716/stat : 5716 (plingeling) R 5696 5698 5016 0 -1 4202560 35112 1256 0 0 16210 5 0 0 25 0 5 0 600375862 223002624 25898 33554432000 4194304 4417332 140737359425312 18446744073709551615 4263109 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5698/tid=5717] ppid=5696 vsize=217776 CPUtime=161.26 cores=0,2,4,6
/proc/5698/task/5717/stat : 5717 (plingeling) R 5696 5698 5016 0 -1 4202560 43479 1256 0 0 16119 7 0 0 25 0 5 0 600375862 223002624 25898 33554432000 4194304 4417332 140737359425312 18446744073709551615 4366061 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5698/tid=5718] ppid=5696 vsize=217776 CPUtime=162.14 cores=0,2,4,6
/proc/5698/task/5718/stat : 5718 (plingeling) R 5696 5698 5016 0 -1 4202560 34563 1256 0 0 16208 6 0 0 25 0 5 0 600375862 223002624 25898 33554432000 4194304 4417332 140737359425312 18446744073709551615 4236800 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5698/tid=5719] ppid=5696 vsize=217776 CPUtime=162.15 cores=0,2,4,6
/proc/5698/task/5719/stat : 5719 (plingeling) R 5696 5698 5016 0 -1 4202560 100490 1256 0 0 16199 16 0 0 25 0 5 0 600375862 223002624 25898 33554432000 4194304 4417332 140737359425312 18446744073709551615 4321669 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 647.88
Current children cumulated vsize (KiB) 217776


################
# More data... #
################

[pid=5698/tid=5717] ppid=5696 vsize=332912 CPUtime=698.29 cores=0,2,4,6
/proc/5698/task/5717/stat : 5717 (plingeling) R 5696 5698 5016 0 -1 4202560 227056 1256 0 0 69797 32 0 0 25 0 5 0 600375862 340901888 54701 33554432000 4194304 4417332 140737359425312 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5698/tid=5718] ppid=5696 vsize=332912 CPUtime=702.15 cores=0,2,4,6
/proc/5698/task/5718/stat : 5718 (plingeling) R 5696 5698 5016 0 -1 4202560 228481 1256 0 0 70179 36 0 0 25 0 5 0 600375862 340901888 54701 33554432000 4194304 4417332 140737359425312 18446744073709551615 4346399 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5698/tid=5719] ppid=5696 vsize=332912 CPUtime=702.16 cores=0,2,4,6
/proc/5698/task/5719/stat : 5719 (plingeling) R 5696 5698 5016 0 -1 4202560 434616 1256 0 0 70152 64 0 0 25 0 5 0 600375862 340901888 54701 33554432000 4194304 4417332 140737359425312 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 2804.93
Current children cumulated vsize (KiB) 332912

[startup+762.307 s]
/proc/loadavg: 8.06 8.02 8.00 9/186 5744
/proc/meminfo: memFree=21641360/32950928 swapFree=67071984/67111528
[pid=5698] ppid=5696 vsize=356324 CPUtime=3044.63 cores=0,2,4,6
/proc/5698/stat : 5698 (plingeling) S 5696 5698 5016 0 -1 4202496 1201876 1256 0 0 304281 182 0 0 23 0 5 0 600375847 364875776 60554 33554432000 4194304 4417332 140737359425312 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/5698/statm: 89081 60554 181 55 0 86967 0
[pid=5698/tid=5716] ppid=5696 vsize=356324 CPUtime=762.17 cores=0,2,4,6
/proc/5698/task/5716/stat : 5716 (plingeling) R 5696 5698 5016 0 -1 4202560 238657 1256 0 0 76179 38 0 0 25 0 5 0 600375862 364875776 60554 33554432000 4194304 4417332 140737359425312 18446744073709551615 4365897 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5698/tid=5717] ppid=5696 vsize=356324 CPUtime=757.95 cores=0,2,4,6
/proc/5698/task/5717/stat : 5717 (plingeling) R 5696 5698 5016 0 -1 4202560 248389 1256 0 0 75761 34 0 0 25 0 5 0 600375862 364875776 60554 33554432000 4194304 4417332 140737359425312 18446744073709551615 4356917 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5698/tid=5718] ppid=5696 vsize=356324 CPUtime=762.15 cores=0,2,4,6
/proc/5698/task/5718/stat : 5718 (plingeling) R 5696 5698 5016 0 -1 4202560 250243 1256 0 0 76176 39 0 0 25 0 5 0 600375862 364875776 60554 33554432000 4194304 4417332 140737359425312 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5698/tid=5719] ppid=5696 vsize=356324 CPUtime=762.17 cores=0,2,4,6
/proc/5698/task/5719/stat : 5719 (plingeling) R 5696 5698 5016 0 -1 4202560 457817 1256 0 0 76149 68 0 0 25 0 5 0 600375862 364875776 60554 33554432000 4194304 4417332 140737359425312 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3044.63
Current children cumulated vsize (KiB) 356324

[startup+822.301 s]
/proc/loadavg: 8.14 8.04 8.00 9/186 5745
/proc/meminfo: memFree=21355240/32950928 swapFree=67071984/67111528
[pid=5698] ppid=5696 vsize=343596 CPUtime=3284.27 cores=0,2,4,6
/proc/5698/stat : 5698 (plingeling) S 5696 5698 5016 0 -1 4202496 1296038 1256 0 0 328233 194 0 0 23 0 5 0 600375847 351842304 57372 33554432000 4194304 4417332 140737359425312 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/5698/statm: 85899 57372 181 55 0 83785 0
[pid=5698/tid=5716] ppid=5696 vsize=343596 CPUtime=822.17 cores=0,2,4,6
/proc/5698/task/5716/stat : 5716 (plingeling) R 5696 5698 5016 0 -1 4202560 258104 1256 0 0 82176 41 0 0 25 0 5 0 600375862 351842304 57372 33554432000 4194304 4417332 140737359425312 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5698/tid=5717] ppid=5696 vsize=343596 CPUtime=817.62 cores=0,2,4,6
/proc/5698/task/5717/stat : 5717 (plingeling) R 5696 5698 5016 0 -1 4202560 269127 1256 0 0 81725 37 0 0 25 0 5 0 600375862 351842304 57372 33554432000 4194304 4417332 140737359425312 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5698/tid=5718] ppid=5696 vsize=343596 CPUtime=822.14 cores=0,2,4,6
/proc/5698/task/5718/stat : 5718 (plingeling) R 5696 5698 5016 0 -1 4202560 271821 1256 0 0 82172 42 0 0 25 0 5 0 600375862 351842304 57372 33554432000 4194304 4417332 140737359425312 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5698/tid=5719] ppid=5696 vsize=343596 CPUtime=822.17 cores=0,2,4,6
/proc/5698/task/5719/stat : 5719 (plingeling) R 5696 5698 5016 0 -1 4202560 490216 1256 0 0 82145 72 0 0 25 0 5 0 600375862 351842304 57372 33554432000 4194304 4417332 140737359425312 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3284.27
Current children cumulated vsize (KiB) 343596

[startup+882.301 s]
/proc/loadavg: 8.15 8.06 8.01 9/186 5749
/proc/meminfo: memFree=20927652/32950928 swapFree=67071984/67111528
[pid=5698] ppid=5696 vsize=352692 CPUtime=3523.94 cores=0,2,4,6
/proc/5698/stat : 5698 (plingeling) S 5696 5698 5016 0 -1 4202496 1398943 1256 0 0 352184 210 0 0 23 0 5 0 600375847 361156608 59646 33554432000 4194304 4417332 140737359425312 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/5698/statm: 88173 59646 181 55 0 86059 0
[pid=5698/tid=5716] ppid=5696 vsize=352692 CPUtime=882.17 cores=0,2,4,6
/proc/5698/task/5716/stat : 5716 (plingeling) R 5696 5698 5016 0 -1 4202560 288134 1256 0 0 88172 45 0 0 25 0 5 0 600375862 361156608 59646 33554432000 4194304 4417332 140737359425312 18446744073709551615 4365960 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5698/tid=5717] ppid=5696 vsize=352692 CPUtime=877.29 cores=0,2,4,6
/proc/5698/task/5717/stat : 5717 (plingeling) R 5696 5698 5016 0 -1 4202560 290212 1256 0 0 87688 41 0 0 25 0 5 0 600375862 361156608 59646 33554432000 4194304 4417332 140737359425312 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5698/tid=5718] ppid=5696 vsize=352692 CPUtime=882.14 cores=0,2,4,6
/proc/5698/task/5718/stat : 5718 (plingeling) R 5696 5698 5016 0 -1 4202560 297497 1256 0 0 88167 47 0 0 25 0 5 0 600375862 361156608 59646 33554432000 4194304 4417332 140737359425312 18446744073709551615 4366359 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5698/tid=5719] ppid=5696 vsize=352692 CPUtime=882.17 cores=0,2,4,6
/proc/5698/task/5719/stat : 5719 (plingeling) R 5696 5698 5016 0 -1 4202560 516330 1256 0 0 88142 75 0 0 25 0 5 0 600375862 361156608 59646 33554432000 4194304 4417332 140737359425312 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3523.94
Current children cumulated vsize (KiB) 352692

[startup+942.301 s]
/proc/loadavg: 8.34 8.15 8.04 9/186 5751
/proc/meminfo: memFree=20740260/32950928 swapFree=67071984/67111528
[pid=5698] ppid=5696 vsize=376140 CPUtime=3763.62 cores=0,2,4,6
/proc/5698/stat : 5698 (plingeling) S 5696 5698 5016 0 -1 4202496 1494183 1256 0 0 376138 224 0 0 23 0 5 0 600375847 385167360 65508 33554432000 4194304 4417332 140737359425312 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/5698/statm: 94035 65508 181 55 0 91921 0
[pid=5698/tid=5716] ppid=5696 vsize=376140 CPUtime=942.17 cores=0,2,4,6
/proc/5698/task/5716/stat : 5716 (plingeling) R 5696 5698 5016 0 -1 4202560 308817 1256 0 0 94169 48 0 0 25 0 5 0 600375862 385167360 65508 33554432000 4194304 4417332 140737359425312 18446744073709551615 4357525 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5698/tid=5717] ppid=5696 vsize=376140 CPUtime=936.96 cores=0,2,4,6
/proc/5698/task/5717/stat : 5717 (plingeling) R 5696 5698 5016 0 -1 4202560 314659 1256 0 0 93652 44 0 0 25 0 5 0 600375862 385167360 65508 33554432000 4194304 4417332 140737359425312 18446744073709551615 4325672 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5698/tid=5718] ppid=5696 vsize=376140 CPUtime=942.14 cores=0,2,4,6
/proc/5698/task/5718/stat : 5718 (plingeling) R 5696 5698 5016 0 -1 4202560 320295 1256 0 0 94164 50 0 0 25 0 5 0 600375862 385167360 65508 33554432000 4194304 4417332 140737359425312 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5698/tid=5719] ppid=5696 vsize=376140 CPUtime=942.17 cores=0,2,4,6
/proc/5698/task/5719/stat : 5719 (plingeling) R 5696 5698 5016 0 -1 4202560 543642 1256 0 0 94138 79 0 0 25 0 5 0 600375862 385167360 65508 33554432000 4194304 4417332 140737359425312 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3763.62
Current children cumulated vsize (KiB) 376140

[startup+1002.3 s]
/proc/loadavg: 8.17 8.13 8.04 9/186 5753
/proc/meminfo: memFree=21968696/32950928 swapFree=67071984/67111528
[pid=5698] ppid=5696 vsize=365620 CPUtime=4003.29 cores=0,2,4,6
/proc/5698/stat : 5698 (plingeling) S 5696 5698 5016 0 -1 4202496 1600149 1256 0 0 400087 242 0 0 23 0 5 0 600375847 374394880 62878 33554432000 4194304 4417332 140737359425312 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/5698/statm: 91405 62878 181 55 0 89291 0
[pid=5698/tid=5716] ppid=5696 vsize=365620 CPUtime=1002.18 cores=0,2,4,6
/proc/5698/task/5716/stat : 5716 (plingeling) R 5696 5698 5016 0 -1 4202560 330997 1256 0 0 100166 52 0 0 25 0 5 0 600375862 374394880 62878 33554432000 4194304 4417332 140737359425312 18446744073709551615 4357111 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5698/tid=5717] ppid=5696 vsize=365620 CPUtime=996.63 cores=0,2,4,6
/proc/5698/task/5717/stat : 5717 (plingeling) R 5696 5698 5016 0 -1 4202560 332909 1256 0 0 99616 47 0 0 25 0 5 0 600375862 374394880 62878 33554432000 4194304 4417332 140737359425312 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5698/tid=5718] ppid=5696 vsize=365620 CPUtime=1002.14 cores=0,2,4,6
/proc/5698/task/5718/stat : 5718 (plingeling) R 5696 5698 5016 0 -1 4202560 340605 1256 0 0 100160 54 0 0 25 0 5 0 600375862 374394880 62878 33554432000 4194304 4417332 140737359425312 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5698/tid=5719] ppid=5696 vsize=365620 CPUtime=1002.18 cores=0,2,4,6
/proc/5698/task/5719/stat : 5719 (plingeling) R 5696 5698 5016 0 -1 4202560 588868 1256 0 0 100131 87 0 0 25 0 5 0 600375862 374394880 62878 33554432000 4194304 4417332 140737359425312 18446744073709551615 4236603 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4003.29
Current children cumulated vsize (KiB) 365620

[startup+1062.3 s]
/proc/loadavg: 8.11 8.12 8.04 9/186 5755
/proc/meminfo: memFree=21844520/32950928 swapFree=67071984/67111528
[pid=5698] ppid=5696 vsize=342408 CPUtime=4242.97 cores=0,2,4,6
/proc/5698/stat : 5698 (plingeling) S 5696 5698 5016 0 -1 4202496 1680235 1256 0 0 424045 252 0 0 23 0 5 0 600375847 350625792 57075 33554432000 4194304 4417332 140737359425312 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/5698/statm: 85602 57075 181 55 0 83488 0
[pid=5698/tid=5716] ppid=5696 vsize=342408 CPUtime=1062.18 cores=0,2,4,6
/proc/5698/task/5716/stat : 5716 (plingeling) R 5696 5698 5016 0 -1 4202560 344861 1256 0 0 106165 53 0 0 25 0 5 0 600375862 350625792 57075 33554432000 4194304 4417332 140737359425312 18446744073709551615 4310661 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5698/tid=5717] ppid=5696 vsize=342408 CPUtime=1056.31 cores=0,2,4,6
/proc/5698/task/5717/stat : 5717 (plingeling) R 5696 5698 5016 0 -1 4202560 355329 1256 0 0 105580 51 0 0 25 0 5 0 600375862 350625792 57075 33554432000 4194304 4417332 140737359425312 18446744073709551615 4364796 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5698/tid=5718] ppid=5696 vsize=342408 CPUtime=1062.15 cores=0,2,4,6
/proc/5698/task/5718/stat : 5718 (plingeling) R 5696 5698 5016 0 -1 4202560 360333 1256 0 0 106158 57 0 0 25 0 5 0 600375862 350625792 57075 33554432000 4194304 4417332 140737359425312 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5698/tid=5719] ppid=5696 vsize=342408 CPUtime=1062.17 cores=0,2,4,6
/proc/5698/task/5719/stat : 5719 (plingeling) R 5696 5698 5016 0 -1 4202560 612942 1256 0 0 106128 89 0 0 25 0 5 0 600375862 350625792 57075 33554432000 4194304 4417332 140737359425312 18446744073709551615 18446744073699066098 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4242.97
Current children cumulated vsize (KiB) 342408

[startup+1122.3 s]
/proc/loadavg: 8.13 8.13 8.04 9/186 5756
/proc/meminfo: memFree=21671736/32950928 swapFree=67071984/67111528
[pid=5698] ppid=5696 vsize=353396 CPUtime=4482.63 cores=0,2,4,6
/proc/5698/stat : 5698 (plingeling) S 5696 5698 5016 0 -1 4202496 1768645 1256 0 0 447998 265 0 0 23 0 5 0 600375847 361877504 59822 33554432000 4194304 4417332 140737359425312 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/5698/statm: 88349 59822 181 55 0 86235 0
[pid=5698/tid=5716] ppid=5696 vsize=353396 CPUtime=1122.18 cores=0,2,4,6
/proc/5698/task/5716/stat : 5716 (plingeling) R 5696 5698 5016 0 -1 4202560 371588 1256 0 0 112161 57 0 0 25 0 5 0 600375862 361877504 59822 33554432000 4194304 4417332 140737359425312 18446744073709551615 4356960 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5698/tid=5717] ppid=5696 vsize=353396 CPUtime=1115.97 cores=0,2,4,6
/proc/5698/task/5717/stat : 5717 (plingeling) R 5696 5698 5016 0 -1 4202560 367990 1256 0 0 111544 53 0 0 25 0 5 0 600375862 361877504 59822 33554432000 4194304 4417332 140737359425312 18446744073709551615 4364796 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5698/tid=5718] ppid=5696 vsize=353396 CPUtime=1122.14 cores=0,2,4,6
/proc/5698/task/5718/stat : 5718 (plingeling) R 5696 5698 5016 0 -1 4202560 381563 1256 0 0 112154 60 0 0 25 0 5 0 600375862 361877504 59822 33554432000 4194304 4417332 140737359425312 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5698/tid=5719] ppid=5696 vsize=353396 CPUtime=1122.17 cores=0,2,4,6
/proc/5698/task/5719/stat : 5719 (plingeling) R 5696 5698 5016 0 -1 4202560 640734 1256 0 0 112124 93 0 0 25 0 5 0 600375862 361877504 59822 33554432000 4194304 4417332 140737359425312 18446744073709551615 4355764 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4482.63
Current children cumulated vsize (KiB) 353396

[startup+1182.3 s]
/proc/loadavg: 8.09 8.11 8.04 9/186 5758
/proc/meminfo: memFree=21442168/32950928 swapFree=67071984/67111528
[pid=5698] ppid=5696 vsize=353192 CPUtime=4722.32 cores=0,2,4,6
/proc/5698/stat : 5698 (plingeling) S 5696 5698 5016 0 -1 4202496 1879620 1256 0 0 471948 284 0 0 23 0 5 0 600375847 361668608 59771 33554432000 4194304 4417332 140737359425312 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/5698/statm: 88298 59771 181 55 0 86184 0
[pid=5698/tid=5716] ppid=5696 vsize=353192 CPUtime=1182.18 cores=0,2,4,6
/proc/5698/task/5716/stat : 5716 (plingeling) R 5696 5698 5016 0 -1 4202560 395750 1256 0 0 118157 61 0 0 25 0 5 0 600375862 361668608 59771 33554432000 4194304 4417332 140737359425312 18446744073709551615 4365859 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5698/tid=5717] ppid=5696 vsize=353192 CPUtime=1175.64 cores=0,2,4,6
/proc/5698/task/5717/stat : 5717 (plingeling) R 5696 5698 5016 0 -1 4202560 390986 1256 0 0 117508 56 0 0 25 0 5 0 600375862 361668608 59771 33554432000 4194304 4417332 140737359425312 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5698/tid=5718] ppid=5696 vsize=353192 CPUtime=1182.14 cores=0,2,4,6
/proc/5698/task/5718/stat : 5718 (plingeling) R 5696 5698 5016 0 -1 4202560 405510 1256 0 0 118150 64 0 0 25 0 5 0 600375862 361668608 59771 33554432000 4194304 4417332 140737359425312 18446744073709551615 4357283 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5698/tid=5719] ppid=5696 vsize=353192 CPUtime=1182.18 cores=0,2,4,6
/proc/5698/task/5719/stat : 5719 (plingeling) R 5696 5698 5016 0 -1 4202560 680604 1256 0 0 118118 100 0 0 25 0 5 0 600375862 361668608 59771 33554432000 4194304 4417332 140737359425312 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4722.32
Current children cumulated vsize (KiB) 353192



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.8 s]
/proc/loadavg: 8.14 8.12 8.05 9/186 5758
/proc/meminfo: memFree=21305012/32950928 swapFree=67071984/67111528
[pid=5698] ppid=5696 vsize=373672 CPUtime=4800.22 cores=0,2,4,6
/proc/5698/stat : 5698 (plingeling) S 5696 5698 5016 0 -1 4202496 1897232 1256 0 0 479736 286 0 0 23 0 5 0 600375847 382640128 64891 33554432000 4194304 4417332 140737359425312 18446744073709551615 260917196773 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/5698/statm: 93418 64891 181 55 0 91304 0
[pid=5698/tid=5716] ppid=5696 vsize=373672 CPUtime=1201.68 cores=0,2,4,6
/proc/5698/task/5716/stat : 5716 (plingeling) R 5696 5698 5016 0 -1 4202560 400173 1256 0 0 120107 61 0 0 25 0 5 0 600375862 382640128 64891 33554432000 4194304 4417332 140737359425312 18446744073709551615 4365864 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=5698/tid=5717] ppid=5696 vsize=373672 CPUtime=1195.03 cores=0,2,4,6
/proc/5698/task/5717/stat : 5717 (plingeling) R 5696 5698 5016 0 -1 4202560 398979 1256 0 0 119446 57 0 0 25 0 5 0 600375862 382640128 64891 33554432000 4194304 4417332 140737359425312 18446744073709551615 4364742 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=5698/tid=5718] ppid=5696 vsize=373672 CPUtime=1201.65 cores=0,2,4,6
/proc/5698/task/5718/stat : 5718 (plingeling) R 5696 5698 5016 0 -1 4202560 410275 1256 0 0 120100 65 0 0 25 0 5 0 600375862 382640128 64891 33554432000 4194304 4417332 140737359425312 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=5698/tid=5719] ppid=5696 vsize=373672 CPUtime=1201.68 cores=0,2,4,6
/proc/5698/task/5719/stat : 5719 (plingeling) R 5696 5698 5016 0 -1 4202560 681035 1256 0 0 120068 100 0 0 25 0 5 0 600375862 382640128 64891 33554432000 4194304 4417332 140737359425312 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.22
Current children cumulated vsize (KiB) 373672

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1201.82
CPU time (s): 4800.26
CPU user time (s): 4797.39
CPU system time (s): 2.87856
CPU usage (%): 399.415
Max. virtual memory (cumulated for all children) (KiB): 438068

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4797.39
system time used= 2.87856
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1898488
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 89
involuntary context switches= 33835

runsolver used 2.02269 second user time and 4.52331 second system time

The end

Launcher Data 

Begin job on node116 at 2011-04-29 20:24:12
IDJOB=3291415
IDBENCH=84215
IDSOLVER=1605
FILE ID=node116/3291415-1304101452
RUNJOBID= node116-1304101452-5684
PBS_JOBID= 13170597
Free space on /tmp= 72512 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT03/industrial/goldberg/hard_eq_check/k2mul.miter.shuffled-as.sat03-355.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3291415-1304101452/watcher-3291415-1304101452 -o /tmp/evaluation-result-3291415-1304101452/solver-3291415-1304101452 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3291415-1304101452.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 77e0700cbaeeb3d63a4820608fc84761
RANDOM SEED=419047906

node116.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      23619180 kB
Buffers:       1905212 kB
Cached:        6955120 kB
SwapCached:       9732 kB
Active:        4404092 kB
Inactive:      4499036 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      23619180 kB
SwapTotal:    67111528 kB
SwapFree:     67071984 kB
Dirty:            2892 kB
Writeback:           0 kB
AnonPages:       36120 kB
Mapped:          13448 kB
Slab:           365724 kB
PageTables:       4508 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   194420 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72512 MiB
End job on node116 at 2011-04-29 20:44:16