Trace number 3262166

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 4800.26 1201.92

General information on the benchmark

Namecrafted/kullmann/VanDerWaerden/VanderWaerden_pd_3k/
VanDerWaerden_pd_2-3-23_506.cnf
MD5SUM3c65ff80757b12edd41f8ee6da874877
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark89.0965
Satisfiable
(Un)Satisfiability was proved
Number of variables253
Number of clauses34386
Sum of the clauses size155873
Maximum clause length23
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 284
Number of clauses of size 331656
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 52646

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3262166-1304092886/watcher-3262166-1304092886 -o /tmp/evaluation-result-3262166-1304092886/solver-3262166-1304092886 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3262166-1304092886.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.02 7.83 7.93 2/177 16226
/proc/meminfo: memFree=30926832/32950928 swapFree=67111448/67111528
[pid=16226] ppid=16224 vsize=9432 CPUtime=0 cores=1,3,5,7
/proc/16226/stat : 16226 (plingeling) R 16224 16226 15338 0 -1 4202496 451 1251 0 0 0 0 0 0 23 0 1 0 520887343 9658368 380 33554432000 4194304 4417332 140735718968256 18446744073709551615 251814240000 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/16226/statm: 2358 380 147 55 0 244 0

[startup+0.078985 s]
/proc/loadavg: 7.02 7.83 7.93 2/177 16226
/proc/meminfo: memFree=30926832/32950928 swapFree=67111448/67111528
[pid=16226] ppid=16224 vsize=59264 CPUtime=0.16 cores=1,3,5,7
/proc/16226/stat : 16226 (plingeling) S 16224 16226 15338 0 -1 4202496 2761 1251 0 0 16 0 0 0 23 0 5 0 520887343 60686336 2604 33554432000 4194304 4417332 140735718968256 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/16226/statm: 14816 2604 173 55 0 12702 0
Current children cumulated CPU time (s) 0.16
Current children cumulated vsize (KiB) 59264

[startup+0.10098 s]
/proc/loadavg: 7.02 7.83 7.93 2/177 16226
/proc/meminfo: memFree=30926832/32950928 swapFree=67111448/67111528
[pid=16226] ppid=16224 vsize=62976 CPUtime=0.25 cores=1,3,5,7
/proc/16226/stat : 16226 (plingeling) S 16224 16226 15338 0 -1 4202496 3828 1251 0 0 25 0 0 0 23 0 5 0 520887343 64487424 3542 33554432000 4194304 4417332 140735718968256 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/16226/statm: 15744 3542 175 55 0 13630 0
Current children cumulated CPU time (s) 0.25
Current children cumulated vsize (KiB) 62976

[startup+0.300948 s]
/proc/loadavg: 7.02 7.83 7.93 2/177 16226
/proc/meminfo: memFree=30926832/32950928 swapFree=67111448/67111528
[pid=16226] ppid=16224 vsize=61324 CPUtime=1.05 cores=1,3,5,7
/proc/16226/stat : 16226 (plingeling) S 16224 16226 15338 0 -1 4202496 4084 1251 0 0 105 0 0 0 23 0 5 0 520887343 62795776 3056 33554432000 4194304 4417332 140735718968256 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/16226/statm: 15331 3056 181 55 0 13217 0
Current children cumulated CPU time (s) 1.05
Current children cumulated vsize (KiB) 61324

[startup+0.700878 s]
/proc/loadavg: 7.02 7.83 7.93 2/177 16226
/proc/meminfo: memFree=30926832/32950928 swapFree=67111448/67111528
[pid=16226] ppid=16224 vsize=62252 CPUtime=2.65 cores=1,3,5,7
/proc/16226/stat : 16226 (plingeling) S 16224 16226 15338 0 -1 4202496 4379 1251 0 0 265 0 0 0 23 0 5 0 520887343 63746048 3351 33554432000 4194304 4417332 140735718968256 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/16226/statm: 15563 3351 181 55 0 13449 0
Current children cumulated CPU time (s) 2.65
Current children cumulated vsize (KiB) 62252

[startup+1.50075 s]
/proc/loadavg: 7.02 7.83 7.93 10/186 16240
/proc/meminfo: memFree=30903440/32950928 swapFree=67111448/67111528
[pid=16226] ppid=16224 vsize=65044 CPUtime=5.84 cores=1,3,5,7
/proc/16226/stat : 16226 (plingeling) S 16224 16226 15338 0 -1 4202496 5077 1251 0 0 584 0 0 0 23 0 5 0 520887343 66605056 4049 33554432000 4194304 4417332 140735718968256 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/16226/statm: 16261 4049 181 55 0 14147 0
[pid=16226/tid=16237] ppid=16224 vsize=65044 CPUtime=1.45 cores=1,3,5,7
/proc/16226/task/16237/stat : 16237 (plingeling) R 16224 16226 15338 0 -1 4202560 598 1251 0 0 145 0 0 0 25 0 5 0 520887348 66605056 4049 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16226/tid=16238] ppid=16224 vsize=65044 CPUtime=1.45 cores=1,3,5,7
/proc/16226/task/16238/stat : 16238 (plingeling) R 16224 16226 15338 0 -1 4202560 556 1251 0 0 145 0 0 0 25 0 5 0 520887348 66605056 4049 33554432000 4194304 4417332 140735718968256 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16226/tid=16239] ppid=16224 vsize=65044 CPUtime=1.45 cores=1,3,5,7
/proc/16226/task/16239/stat : 16239 (plingeling) R 16224 16226 15338 0 -1 4202560 354 1251 0 0 145 0 0 0 25 0 5 0 520887348 66605056 4049 33554432000 4194304 4417332 140735718968256 18446744073709551615 4366029 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16226/tid=16240] ppid=16224 vsize=65044 CPUtime=1.44 cores=1,3,5,7
/proc/16226/task/16240/stat : 16240 (plingeling) R 16224 16226 15338 0 -1 4202560 844 1251 0 0 144 0 0 0 25 0 5 0 520887348 66605056 4049 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364732 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 5.84
Current children cumulated vsize (KiB) 65044

[startup+3.10048 s]
/proc/loadavg: 7.02 7.83 7.93 10/186 16240
/proc/meminfo: memFree=30888312/32950928 swapFree=67111448/67111528
[pid=16226] ppid=16224 vsize=142548 CPUtime=12.23 cores=1,3,5,7
/proc/16226/stat : 16226 (plingeling) S 16224 16226 15338 0 -1 4202496 8201 1251 0 0 1223 0 0 0 23 0 5 0 520887343 145969152 7074 33554432000 4194304 4417332 140735718968256 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/16226/statm: 35637 7074 181 55 0 33523 0
[pid=16226/tid=16237] ppid=16224 vsize=142548 CPUtime=3.05 cores=1,3,5,7
/proc/16226/task/16237/stat : 16237 (plingeling) R 16224 16226 15338 0 -1 4202560 1451 1251 0 0 305 0 0 0 25 0 5 0 520887348 145969152 7074 33554432000 4194304 4417332 140735718968256 18446744073709551615 4312663 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16226/tid=16238] ppid=16224 vsize=142548 CPUtime=3.04 cores=1,3,5,7
/proc/16226/task/16238/stat : 16238 (plingeling) R 16224 16226 15338 0 -1 4202560 1244 1251 0 0 304 0 0 0 25 0 5 0 520887348 145969152 7074 33554432000 4194304 4417332 140735718968256 18446744073709551615 4254990 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16226/tid=16239] ppid=16224 vsize=142548 CPUtime=3.04 cores=1,3,5,7
/proc/16226/task/16239/stat : 16239 (plingeling) R 16224 16226 15338 0 -1 4202560 1203 1251 0 0 304 0 0 0 25 0 5 0 520887348 145969152 7074 33554432000 4194304 4417332 140735718968256 18446744073709551615 4258418 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16226/tid=16240] ppid=16224 vsize=142548 CPUtime=3.03 cores=1,3,5,7
/proc/16226/task/16240/stat : 16240 (plingeling) R 16224 16226 15338 0 -1 4202560 1578 1251 0 0 303 0 0 0 25 0 5 0 520887348 145969152 7074 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364463 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 12.23
Current children cumulated vsize (KiB) 142548

[startup+6.30094 s]
/proc/loadavg: 7.10 7.84 7.93 10/186 16240
/proc/meminfo: memFree=30863644/32950928 swapFree=67111448/67111528
[pid=16226] ppid=16224 vsize=151412 CPUtime=25.01 cores=1,3,5,7
/proc/16226/stat : 16226 (plingeling) S 16224 16226 15338 0 -1 4202496 11127 1251 0 0 2501 0 0 0 23 0 5 0 520887343 155045888 9290 33554432000 4194304 4417332 140735718968256 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/16226/statm: 37853 9290 181 55 0 35739 0
[pid=16226/tid=16237] ppid=16224 vsize=151412 CPUtime=6.25 cores=1,3,5,7
/proc/16226/task/16237/stat : 16237 (plingeling) R 16224 16226 15338 0 -1 4202560 2155 1251 0 0 625 0 0 0 25 0 5 0 520887348 155045888 9290 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364463 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16226/tid=16238] ppid=16224 vsize=151412 CPUtime=6.24 cores=1,3,5,7
/proc/16226/task/16238/stat : 16238 (plingeling) R 16224 16226 15338 0 -1 4202560 1930 1251 0 0 624 0 0 0 25 0 5 0 520887348 155045888 9290 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16226/tid=16239] ppid=16224 vsize=151412 CPUtime=6.25 cores=1,3,5,7
/proc/16226/task/16239/stat : 16239 (plingeling) R 16224 16226 15338 0 -1 4202560 1744 1251 0 0 625 0 0 0 25 0 5 0 520887348 155045888 9290 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364750 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16226/tid=16240] ppid=16224 vsize=151412 CPUtime=6.22 cores=1,3,5,7
/proc/16226/task/16240/stat : 16240 (plingeling) R 16224 16226 15338 0 -1 4202560 2573 1251 0 0 622 0 0 0 25 0 5 0 520887348 155045888 9290 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 25.01
Current children cumulated vsize (KiB) 151412

[startup+12.7009 s]
/proc/loadavg: 7.17 7.84 7.93 9/186 16240
/proc/meminfo: memFree=30845772/32950928 swapFree=67111448/67111528
[pid=16226] ppid=16224 vsize=157940 CPUtime=50.58 cores=1,3,5,7
/proc/16226/stat : 16226 (plingeling) S 16224 16226 15338 0 -1 4202496 14682 1251 0 0 5057 1 0 0 23 0 5 0 520887343 161730560 10942 33554432000 4194304 4417332 140735718968256 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/16226/statm: 39485 10942 181 55 0 37371 0
[pid=16226/tid=16237] ppid=16224 vsize=157940 CPUtime=12.65 cores=1,3,5,7
/proc/16226/task/16237/stat : 16237 (plingeling) R 16224 16226 15338 0 -1 4202560 2671 1251 0 0 1265 0 0 0 25 0 5 0 520887348 161730560 10942 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364742 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16226/tid=16238] ppid=16224 vsize=157940 CPUtime=12.64 cores=1,3,5,7
/proc/16226/task/16238/stat : 16238 (plingeling) R 16224 16226 15338 0 -1 4202560 3637 1251 0 0 1264 0 0 0 25 0 5 0 520887348 161730560 10942 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364526 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16226/tid=16239] ppid=16224 vsize=157940 CPUtime=12.65 cores=1,3,5,7
/proc/16226/task/16239/stat : 16239 (plingeling) R 16224 16226 15338 0 -1 4202560 1744 1251 0 0 1265 0 0 0 25 0 5 0 520887348 161730560 10942 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364734 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16226/tid=16240] ppid=16224 vsize=157940 CPUtime=12.58 cores=1,3,5,7
/proc/16226/task/16240/stat : 16240 (plingeling) R 16224 16226 15338 0 -1 4202560 3905 1251 0 0 1258 0 0 0 25 0 5 0 520887348 161730560 10942 33554432000 4194304 4417332 140735718968256 18446744073709551615 4353974 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.58
Current children cumulated vsize (KiB) 157940

[startup+25.5007 s]
/proc/loadavg: 7.35 7.85 7.93 10/186 16240
/proc/meminfo: memFree=30832868/32950928 swapFree=67111448/67111528
[pid=16226] ppid=16224 vsize=169100 CPUtime=101.71 cores=1,3,5,7
/proc/16226/stat : 16226 (plingeling) S 16224 16226 15338 0 -1 4202496 30613 1251 0 0 10168 3 0 0 23 0 5 0 520887343 173158400 13732 33554432000 4194304 4417332 140735718968256 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/16226/statm: 42275 13732 181 55 0 40161 0
[pid=16226/tid=16237] ppid=16224 vsize=169100 CPUtime=25.45 cores=1,3,5,7
/proc/16226/task/16237/stat : 16237 (plingeling) R 16224 16226 15338 0 -1 4202560 7205 1251 0 0 2545 0 0 0 25 0 5 0 520887348 173158400 13732 33554432000 4194304 4417332 140735718968256 18446744073709551615 4366610 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16226/tid=16238] ppid=16224 vsize=169100 CPUtime=25.44 cores=1,3,5,7
/proc/16226/task/16238/stat : 16238 (plingeling) R 16224 16226 15338 0 -1 4202560 9654 1251 0 0 2543 1 0 0 25 0 5 0 520887348 173158400 13732 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16226/tid=16239] ppid=16224 vsize=169100 CPUtime=25.45 cores=1,3,5,7
/proc/16226/task/16239/stat : 16239 (plingeling) R 16224 16226 15338 0 -1 4202560 1744 1251 0 0 2545 0 0 0 25 0 5 0 520887348 173158400 13732 33554432000 4194304 4417332 140735718968256 18446744073709551615 4250532 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16226/tid=16240] ppid=16224 vsize=169100 CPUtime=25.31 cores=1,3,5,7
/proc/16226/task/16240/stat : 16240 (plingeling) R 16224 16226 15338 0 -1 4202560 9285 1251 0 0 2530 1 0 0 25 0 5 0 520887348 173158400 13732 33554432000 4194304 4417332 140735718968256 18446744073709551615 4366716 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.71
Current children cumulated vsize (KiB) 169100

[startup+51.1004 s]
/proc/loadavg: 7.69 7.89 7.94 10/186 16241
/proc/meminfo: memFree=30815592/32950928 swapFree=67111448/67111528
[pid=16226] ppid=16224 vsize=175688 CPUtime=203.99 cores=1,3,5,7
/proc/16226/stat : 16226 (plingeling) S 16224 16226 15338 0 -1 4202496 74433 1251 0 0 20389 10 0 0 23 0 5 0 520887343 179904512 15359 33554432000 4194304 4417332 140735718968256 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/16226/statm: 43922 15359 181 55 0 41808 0
[pid=16226/tid=16237] ppid=16224 vsize=175688 CPUtime=51.05 cores=1,3,5,7
/proc/16226/task/16237/stat : 16237 (plingeling) R 16224 16226 15338 0 -1 4202560 20175 1251 0 0 5103 2 0 0 25 0 5 0 520887348 179904512 15359 33554432000 4194304 4417332 140735718968256 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16226/tid=16238] ppid=16224 vsize=175688 CPUtime=51.04 cores=1,3,5,7
/proc/16226/task/16238/stat : 16238 (plingeling) R 16224 16226 15338 0 -1 4202560 19091 1251 0 0 5102 2 0 0 25 0 5 0 520887348 179904512 15359 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16226/tid=16239] ppid=16224 vsize=175688 CPUtime=51.04 cores=1,3,5,7
/proc/16226/task/16239/stat : 16239 (plingeling) R 16224 16226 15338 0 -1 4202560 8359 1251 0 0 5104 0 0 0 25 0 5 0 520887348 179904512 15359 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16226/tid=16240] ppid=16224 vsize=175688 CPUtime=50.78 cores=1,3,5,7
/proc/16226/task/16240/stat : 16240 (plingeling) R 16224 16226 15338 0 -1 4202560 24083 1251 0 0 5074 4 0 0 25 0 5 0 520887348 179904512 15359 33554432000 4194304 4417332 140735718968256 18446744073709551615 4366381 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 203.99
Current children cumulated vsize (KiB) 175688

[startup+102.307 s]
/proc/loadavg: 7.99 7.93 7.95 10/186 16242
/proc/meminfo: memFree=30782092/32950928 swapFree=67111448/67111528
[pid=16226] ppid=16224 vsize=190988 CPUtime=408.54 cores=1,3,5,7
/proc/16226/stat : 16226 (plingeling) S 16224 16226 15338 0 -1 4202496 174177 1251 0 0 40829 25 0 0 23 0 5 0 520887343 195571712 19184 33554432000 4194304 4417332 140735718968256 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/16226/statm: 47747 19184 181 55 0 45633 0
[pid=16226/tid=16237] ppid=16224 vsize=190988 CPUtime=102.26 cores=1,3,5,7
/proc/16226/task/16237/stat : 16237 (plingeling) R 16224 16226 15338 0 -1 4202560 45052 1251 0 0 10220 6 0 0 25 0 5 0 520887348 195571712 19184 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16226/tid=16238] ppid=16224 vsize=190988 CPUtime=102.25 cores=1,3,5,7
/proc/16226/task/16238/stat : 16238 (plingeling) R 16224 16226 15338 0 -1 4202560 39725 1251 0 0 10220 5 0 0 25 0 5 0 520887348 195571712 19184 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16226/tid=16239] ppid=16224 vsize=190988 CPUtime=102.24 cores=1,3,5,7
/proc/16226/task/16239/stat : 16239 (plingeling) R 16224 16226 15338 0 -1 4202560 32218 1251 0 0 10221 3 0 0 25 0 5 0 520887348 195571712 19184 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364581 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16226/tid=16240] ppid=16224 vsize=190988 CPUtime=101.72 cores=1,3,5,7
/proc/16226/task/16240/stat : 16240 (plingeling) R 16224 16226 15338 0 -1 4202560 54457 1251 0 0 10163 9 0 0 25 0 5 0 520887348 195571712 19184 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364938 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 408.54
Current children cumulated vsize (KiB) 190988

[startup+162.301 s]
/proc/loadavg: 8.11 7.97 7.96 10/186 16244
/proc/meminfo: memFree=30740536/32950928 swapFree=67111448/67111528
[pid=16226] ppid=16224 vsize=211628 CPUtime=648.2 cores=1,3,5,7
/proc/16226/stat : 16226 (plingeling) S 16224 16226 15338 0 -1 4202496 283487 1251 0 0 64778 42 0 0 23 0 5 0 520887343 216707072 24344 33554432000 4194304 4417332 140735718968256 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/16226/statm: 52907 24344 181 55 0 50793 0
[pid=16226/tid=16237] ppid=16224 vsize=211628 CPUtime=162.24 cores=1,3,5,7
/proc/16226/task/16237/stat : 16237 (plingeling) R 16224 16226 15338 0 -1 4202560 75180 1251 0 0 16213 11 0 0 25 0 5 0 520887348 216707072 24344 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16226/tid=16238] ppid=16224 vsize=211628 CPUtime=162.24 cores=1,3,5,7
/proc/16226/task/16238/stat : 16238 (plingeling) R 16224 16226 15338 0 -1 4202560 60759 1251 0 0 16216 8 0 0 25 0 5 0 520887348 216707072 24344 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364487 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16226/tid=16239] ppid=16224 vsize=211628 CPUtime=162.24 cores=1,3,5,7
/proc/16226/task/16239/stat : 16239 (plingeling) R 16224 16226 15338 0 -1 4202560 61501 1251 0 0 16217 7 0 0 25 0 5 0 520887348 216707072 24344 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364470 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16226/tid=16240] ppid=16224 vsize=211628 CPUtime=161.4 cores=1,3,5,7
/proc/16226/task/16240/stat : 16240 (plingeling) R 16224 16226 15338 0 -1 4202560 83322 1251 0 0 16126 14 0 0 25 0 5 0 520887348 216707072 24344 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 648.2
Current children cumulated vsize (KiB) 211628


################
# More data... #
################

[pid=16226/tid=16238] ppid=16224 vsize=276808 CPUtime=762.26 cores=1,3,5,7
/proc/16226/task/16238/stat : 16238 (plingeling) R 16224 16226 15338 0 -1 4202560 290618 1251 0 0 76182 44 0 0 25 0 5 0 520887348 283451392 40639 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16226/tid=16239] ppid=16224 vsize=276808 CPUtime=762.26 cores=1,3,5,7
/proc/16226/task/16239/stat : 16239 (plingeling) R 16224 16226 15338 0 -1 4202560 273171 1251 0 0 76186 40 0 0 25 0 5 0 520887348 283451392 40639 33554432000 4194304 4417332 140735718968256 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16226/tid=16240] ppid=16224 vsize=276808 CPUtime=758.14 cores=1,3,5,7
/proc/16226/task/16240/stat : 16240 (plingeling) R 16224 16226 15338 0 -1 4202560 338717 1251 0 0 75760 54 0 0 25 0 5 0 520887348 283451392 40639 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3044.19
Current children cumulated vsize (KiB) 276808

[startup+822.301 s]
/proc/loadavg: 8.20 8.09 8.02 9/186 16265
/proc/meminfo: memFree=30606236/32950928 swapFree=67111448/67111528
[pid=16226] ppid=16224 vsize=267164 CPUtime=3283.85 cores=1,3,5,7
/proc/16226/stat : 16226 (plingeling) S 16224 16226 15338 0 -1 4202496 1297892 1251 0 0 328183 202 0 0 23 0 5 0 520887343 273575936 38228 33554432000 4194304 4417332 140735718968256 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/16226/statm: 66791 38228 181 55 0 64677 0
[pid=16226/tid=16237] ppid=16224 vsize=267164 CPUtime=821.45 cores=1,3,5,7
/proc/16226/task/16237/stat : 16237 (plingeling) R 16224 16226 15338 0 -1 4202560 325422 1251 0 0 82093 52 0 0 25 0 5 0 520887348 273575936 38228 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16226/tid=16238] ppid=16224 vsize=267164 CPUtime=822.25 cores=1,3,5,7
/proc/16226/task/16238/stat : 16238 (plingeling) R 16224 16226 15338 0 -1 4202560 311444 1251 0 0 82178 47 0 0 25 0 5 0 520887348 273575936 38228 33554432000 4194304 4417332 140735718968256 18446744073709551615 4250651 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16226/tid=16239] ppid=16224 vsize=267164 CPUtime=822.26 cores=1,3,5,7
/proc/16226/task/16239/stat : 16239 (plingeling) R 16224 16226 15338 0 -1 4202560 295528 1251 0 0 82183 43 0 0 25 0 5 0 520887348 273575936 38228 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16226/tid=16240] ppid=16224 vsize=267164 CPUtime=817.81 cores=1,3,5,7
/proc/16226/task/16240/stat : 16240 (plingeling) R 16224 16226 15338 0 -1 4202560 362773 1251 0 0 81723 58 0 0 25 0 5 0 520887348 273575936 38228 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3283.85
Current children cumulated vsize (KiB) 267164

[startup+882.301 s]
/proc/loadavg: 8.13 8.08 8.01 10/186 16266
/proc/meminfo: memFree=30542112/32950928 swapFree=67111448/67111528
[pid=16226] ppid=16224 vsize=308908 CPUtime=3523.52 cores=1,3,5,7
/proc/16226/stat : 16226 (plingeling) S 16224 16226 15338 0 -1 4202496 1406940 1251 0 0 352134 218 0 0 23 0 5 0 520887343 316321792 48664 33554432000 4194304 4417332 140735718968256 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/16226/statm: 77227 48664 181 55 0 75113 0
[pid=16226/tid=16237] ppid=16224 vsize=308908 CPUtime=881.45 cores=1,3,5,7
/proc/16226/task/16237/stat : 16237 (plingeling) R 16224 16226 15338 0 -1 4202560 352210 1251 0 0 88089 56 0 0 25 0 5 0 520887348 316321792 48664 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16226/tid=16238] ppid=16224 vsize=308908 CPUtime=882.26 cores=1,3,5,7
/proc/16226/task/16238/stat : 16238 (plingeling) R 16224 16226 15338 0 -1 4202560 337442 1251 0 0 88175 51 0 0 25 0 5 0 520887348 316321792 48664 33554432000 4194304 4417332 140735718968256 18446744073709551615 4352237 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16226/tid=16239] ppid=16224 vsize=308908 CPUtime=882.25 cores=1,3,5,7
/proc/16226/task/16239/stat : 16239 (plingeling) R 16224 16226 15338 0 -1 4202560 327744 1251 0 0 88178 47 0 0 25 0 5 0 520887348 316321792 48664 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364713 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16226/tid=16240] ppid=16224 vsize=308908 CPUtime=877.49 cores=1,3,5,7
/proc/16226/task/16240/stat : 16240 (plingeling) R 16224 16226 15338 0 -1 4202560 386819 1251 0 0 87687 62 0 0 25 0 5 0 520887348 316321792 48664 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3523.52
Current children cumulated vsize (KiB) 308908

[startup+942.301 s]
/proc/loadavg: 8.10 8.08 8.01 10/186 16268
/proc/meminfo: memFree=30584504/32950928 swapFree=67111448/67111528
[pid=16226] ppid=16224 vsize=297460 CPUtime=3763.2 cores=1,3,5,7
/proc/16226/stat : 16226 (plingeling) S 16224 16226 15338 0 -1 4202496 1467857 1251 0 0 376092 228 0 0 23 0 5 0 520887343 304599040 45802 33554432000 4194304 4417332 140735718968256 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/16226/statm: 74365 45802 181 55 0 72251 0
[pid=16226/tid=16237] ppid=16224 vsize=297460 CPUtime=941.45 cores=1,3,5,7
/proc/16226/task/16237/stat : 16237 (plingeling) R 16224 16226 15338 0 -1 4202560 363108 1251 0 0 94088 57 0 0 25 0 5 0 520887348 304599040 45802 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364919 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16226/tid=16238] ppid=16224 vsize=297460 CPUtime=942.25 cores=1,3,5,7
/proc/16226/task/16238/stat : 16238 (plingeling) R 16224 16226 15338 0 -1 4202560 351112 1251 0 0 94172 53 0 0 25 0 5 0 520887348 304599040 45802 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364667 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16226/tid=16239] ppid=16224 vsize=297460 CPUtime=942.25 cores=1,3,5,7
/proc/16226/task/16239/stat : 16239 (plingeling) R 16224 16226 15338 0 -1 4202560 342761 1251 0 0 94176 49 0 0 25 0 5 0 520887348 304599040 45802 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16226/tid=16240] ppid=16224 vsize=297460 CPUtime=937.16 cores=1,3,5,7
/proc/16226/task/16240/stat : 16240 (plingeling) R 16224 16226 15338 0 -1 4202560 408151 1251 0 0 93650 66 0 0 25 0 5 0 520887348 304599040 45802 33554432000 4194304 4417332 140735718968256 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3763.2
Current children cumulated vsize (KiB) 297460

[startup+1002.3 s]
/proc/loadavg: 8.04 8.07 8.01 10/186 16269
/proc/meminfo: memFree=30576924/32950928 swapFree=67111448/67111528
[pid=16226] ppid=16224 vsize=275416 CPUtime=4002.87 cores=1,3,5,7
/proc/16226/stat : 16226 (plingeling) S 16224 16226 15338 0 -1 4202496 1553785 1251 0 0 400047 240 0 0 23 0 5 0 520887343 282025984 40291 33554432000 4194304 4417332 140735718968256 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/16226/statm: 68854 40291 181 55 0 66740 0
[pid=16226/tid=16237] ppid=16224 vsize=275416 CPUtime=1001.46 cores=1,3,5,7
/proc/16226/task/16237/stat : 16237 (plingeling) R 16224 16226 15338 0 -1 4202560 379804 1251 0 0 100086 60 0 0 25 0 5 0 520887348 282025984 40291 33554432000 4194304 4417332 140735718968256 18446744073709551615 4366297 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16226/tid=16238] ppid=16224 vsize=275416 CPUtime=1002.26 cores=1,3,5,7
/proc/16226/task/16238/stat : 16238 (plingeling) R 16224 16226 15338 0 -1 4202560 380473 1251 0 0 100168 58 0 0 25 0 5 0 520887348 282025984 40291 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16226/tid=16239] ppid=16224 vsize=275416 CPUtime=1002.26 cores=1,3,5,7
/proc/16226/task/16239/stat : 16239 (plingeling) R 16224 16226 15338 0 -1 4202560 360546 1251 0 0 100174 52 0 0 25 0 5 0 520887348 282025984 40291 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364470 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16226/tid=16240] ppid=16224 vsize=275416 CPUtime=996.83 cores=1,3,5,7
/proc/16226/task/16240/stat : 16240 (plingeling) R 16224 16226 15338 0 -1 4202560 430237 1251 0 0 99614 69 0 0 25 0 5 0 520887348 282025984 40291 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4002.87
Current children cumulated vsize (KiB) 275416

[startup+1062.3 s]
/proc/loadavg: 8.07 8.07 8.01 10/186 16271
/proc/meminfo: memFree=30611500/32950928 swapFree=67111448/67111528
[pid=16226] ppid=16224 vsize=283156 CPUtime=4242.55 cores=1,3,5,7
/proc/16226/stat : 16226 (plingeling) S 16224 16226 15338 0 -1 4202496 1617459 1251 0 0 424006 249 0 0 23 0 5 0 520887343 289951744 42226 33554432000 4194304 4417332 140735718968256 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/16226/statm: 70789 42226 181 55 0 68675 0
[pid=16226/tid=16237] ppid=16224 vsize=283156 CPUtime=1061.46 cores=1,3,5,7
/proc/16226/task/16237/stat : 16237 (plingeling) R 16224 16226 15338 0 -1 4202560 397330 1251 0 0 106084 62 0 0 25 0 5 0 520887348 289951744 42226 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364680 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16226/tid=16238] ppid=16224 vsize=283156 CPUtime=1062.26 cores=1,3,5,7
/proc/16226/task/16238/stat : 16238 (plingeling) R 16224 16226 15338 0 -1 4202560 392630 1251 0 0 106166 60 0 0 25 0 5 0 520887348 289951744 42226 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16226/tid=16239] ppid=16224 vsize=283156 CPUtime=1062.25 cores=1,3,5,7
/proc/16226/task/16239/stat : 16239 (plingeling) R 16224 16226 15338 0 -1 4202560 376736 1251 0 0 106172 53 0 0 25 0 5 0 520887348 289951744 42226 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364899 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16226/tid=16240] ppid=16224 vsize=283156 CPUtime=1056.51 cores=1,3,5,7
/proc/16226/task/16240/stat : 16240 (plingeling) R 16224 16226 15338 0 -1 4202560 448038 1251 0 0 105578 73 0 0 25 0 5 0 520887348 289951744 42226 33554432000 4194304 4417332 140735718968256 18446744073709551615 4366303 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4242.55
Current children cumulated vsize (KiB) 283156

[startup+1122.3 s]
/proc/loadavg: 8.08 8.07 8.01 10/186 16273
/proc/meminfo: memFree=30585568/32950928 swapFree=67111448/67111528
[pid=16226] ppid=16224 vsize=293916 CPUtime=4482.23 cores=1,3,5,7
/proc/16226/stat : 16226 (plingeling) S 16224 16226 15338 0 -1 4202496 1691445 1251 0 0 447961 262 0 0 23 0 5 0 520887343 300969984 44916 33554432000 4194304 4417332 140735718968256 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/16226/statm: 73479 44916 181 55 0 71365 0
[pid=16226/tid=16237] ppid=16224 vsize=293916 CPUtime=1121.45 cores=1,3,5,7
/proc/16226/task/16237/stat : 16237 (plingeling) R 16224 16226 15338 0 -1 4202560 416855 1251 0 0 112080 65 0 0 25 0 5 0 520887348 300969984 44916 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16226/tid=16238] ppid=16224 vsize=293916 CPUtime=1122.26 cores=1,3,5,7
/proc/16226/task/16238/stat : 16238 (plingeling) R 16224 16226 15338 0 -1 4202560 412825 1251 0 0 112163 63 0 0 25 0 5 0 520887348 300969984 44916 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16226/tid=16239] ppid=16224 vsize=293916 CPUtime=1122.25 cores=1,3,5,7
/proc/16226/task/16239/stat : 16239 (plingeling) R 16224 16226 15338 0 -1 4202560 396208 1251 0 0 112168 57 0 0 25 0 5 0 520887348 300969984 44916 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364722 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16226/tid=16240] ppid=16224 vsize=293916 CPUtime=1116.18 cores=1,3,5,7
/proc/16226/task/16240/stat : 16240 (plingeling) R 16224 16226 15338 0 -1 4202560 462832 1251 0 0 111543 75 0 0 25 0 5 0 520887348 300969984 44916 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364683 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4482.23
Current children cumulated vsize (KiB) 293916

[startup+1182.3 s]
/proc/loadavg: 8.07 8.07 8.01 10/186 16274
/proc/meminfo: memFree=30527272/32950928 swapFree=67111448/67111528
[pid=16226] ppid=16224 vsize=325036 CPUtime=4721.9 cores=1,3,5,7
/proc/16226/stat : 16226 (plingeling) S 16224 16226 15338 0 -1 4202496 1783359 1251 0 0 471916 274 0 0 23 0 5 0 520887343 332836864 52696 33554432000 4194304 4417332 140735718968256 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/16226/statm: 81259 52696 181 55 0 79145 0
[pid=16226/tid=16237] ppid=16224 vsize=325036 CPUtime=1181.46 cores=1,3,5,7
/proc/16226/task/16237/stat : 16237 (plingeling) R 16224 16226 15338 0 -1 4202560 438071 1251 0 0 118078 68 0 0 25 0 5 0 520887348 332836864 52696 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364732 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16226/tid=16238] ppid=16224 vsize=325036 CPUtime=1182.26 cores=1,3,5,7
/proc/16226/task/16238/stat : 16238 (plingeling) R 16224 16226 15338 0 -1 4202560 440646 1251 0 0 118159 67 0 0 25 0 5 0 520887348 332836864 52696 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16226/tid=16239] ppid=16224 vsize=325036 CPUtime=1182.26 cores=1,3,5,7
/proc/16226/task/16239/stat : 16239 (plingeling) R 16224 16226 15338 0 -1 4202560 418303 1251 0 0 118166 60 0 0 25 0 5 0 520887348 332836864 52696 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364732 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16226/tid=16240] ppid=16224 vsize=325036 CPUtime=1175.86 cores=1,3,5,7
/proc/16226/task/16240/stat : 16240 (plingeling) R 16224 16226 15338 0 -1 4202560 483614 1251 0 0 117508 78 0 0 25 0 5 0 520887348 332836864 52696 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4721.9
Current children cumulated vsize (KiB) 325036



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.9 s]
/proc/loadavg: 8.05 8.06 8.00 5/180 16276
/proc/meminfo: memFree=30729932/32950928 swapFree=67111448/67111528
[pid=16226] ppid=16224 vsize=314288 CPUtime=4800.21 cores=1,3,5,7
/proc/16226/stat : 16226 (plingeling) S 16224 16226 15338 0 -1 4202496 1798933 1251 0 0 479745 276 0 0 23 0 5 0 520887343 321830912 50009 33554432000 4194304 4417332 140735718968256 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/16226/statm: 78572 50009 181 55 0 76458 0
[pid=16226/tid=16237] ppid=16224 vsize=314288 CPUtime=1201.06 cores=1,3,5,7
/proc/16226/task/16237/stat : 16237 (plingeling) R 16224 16226 15338 0 -1 4202560 440153 1251 0 0 120038 68 0 0 25 0 5 0 520887348 321830912 50009 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16226/tid=16238] ppid=16224 vsize=314288 CPUtime=1201.87 cores=1,3,5,7
/proc/16226/task/16238/stat : 16238 (plingeling) R 16224 16226 15338 0 -1 4202560 445995 1251 0 0 120119 68 0 0 25 0 5 0 520887348 321830912 50009 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16226/tid=16239] ppid=16224 vsize=314288 CPUtime=1201.87 cores=1,3,5,7
/proc/16226/task/16239/stat : 16239 (plingeling) R 16224 16226 15338 0 -1 4202560 422880 1251 0 0 120126 61 0 0 25 0 5 0 520887348 321830912 50009 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16226/tid=16240] ppid=16224 vsize=314288 CPUtime=1195.35 cores=1,3,5,7
/proc/16226/task/16240/stat : 16240 (plingeling) R 16224 16226 15338 0 -1 4202560 487180 1251 0 0 119457 78 0 0 25 0 5 0 520887348 321830912 50009 33554432000 4194304 4417332 140735718968256 18446744073709551615 4365007 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4800.21
Current children cumulated vsize (KiB) 314288

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.9 s]
/proc/loadavg: 8.05 8.06 8.00 5/180 16276
/proc/meminfo: memFree=30729932/32950928 swapFree=67111448/67111528
[pid=16226] ppid=16224 vsize=314288 CPUtime=4800.21 cores=1,3,5,7
/proc/16226/stat : 16226 (plingeling) S 16224 16226 15338 0 -1 4202496 1798933 1251 0 0 479745 276 0 0 23 0 5 0 520887343 321830912 50009 33554432000 4194304 4417332 140735718968256 18446744073709551615 251826042853 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/16226/statm: 78572 50009 181 55 0 76458 0
[pid=16226/tid=16237] ppid=16224 vsize=314288 CPUtime=1201.06 cores=1,3,5,7
/proc/16226/task/16237/stat : 16237 (plingeling) R 16224 16226 15338 0 -1 4202560 440153 1251 0 0 120038 68 0 0 25 0 5 0 520887348 321830912 50009 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16226/tid=16238] ppid=16224 vsize=314288 CPUtime=1201.87 cores=1,3,5,7
/proc/16226/task/16238/stat : 16238 (plingeling) R 16224 16226 15338 0 -1 4202560 445995 1251 0 0 120119 68 0 0 25 0 5 0 520887348 321830912 50009 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16226/tid=16239] ppid=16224 vsize=314288 CPUtime=1201.87 cores=1,3,5,7
/proc/16226/task/16239/stat : 16239 (plingeling) R 16224 16226 15338 0 -1 4202560 422880 1251 0 0 120126 61 0 0 25 0 5 0 520887348 321830912 50009 33554432000 4194304 4417332 140735718968256 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16226/tid=16240] ppid=16224 vsize=314288 CPUtime=1195.35 cores=1,3,5,7
/proc/16226/task/16240/stat : 16240 (plingeling) R 16224 16226 15338 0 -1 4202560 487180 1251 0 0 119457 78 0 0 25 0 5 0 520887348 321830912 50009 33554432000 4194304 4417332 140735718968256 18446744073709551615 4365007 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4800.21
Current children cumulated vsize (KiB) 314288

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1201.92
CPU time (s): 4800.26
CPU user time (s): 4797.47
CPU system time (s): 2.78058
CPU usage (%): 399.382
Max. virtual memory (cumulated for all children) (KiB): 337288

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4797.47
system time used= 2.78058
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1800184
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 51
involuntary context switches= 21229

runsolver used 1.9837 second user time and 4.53731 second system time

The end

Launcher Data 

Begin job on node126 at 2011-04-29 18:01:26
IDJOB=3262166
IDBENCH=83280
IDSOLVER=1605
FILE ID=node126/3262166-1304092886
RUNJOBID= node126-1304092886-16202
PBS_JOBID= 13170545
Free space on /tmp= 71596 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT11/crafted/kullmann/VanDerWaerden/VanderWaerden_pd_3k/VanDerWaerden_pd_2-3-23_506.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3262166-1304092886/watcher-3262166-1304092886 -o /tmp/evaluation-result-3262166-1304092886/solver-3262166-1304092886 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3262166-1304092886.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 3c65ff80757b12edd41f8ee6da874877
RANDOM SEED=679651078

node126.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      30926988 kB
Buffers:        244972 kB
Cached:        1397740 kB
SwapCached:         76 kB
Active:         119016 kB
Inactive:      1592624 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      30926988 kB
SwapTotal:    67111528 kB
SwapFree:     67111448 kB
Dirty:            1828 kB
Writeback:           0 kB
AnonPages:       69352 kB
Mapped:          15272 kB
Slab:           246952 kB
PageTables:       4836 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   211476 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71596 MiB
End job on node126 at 2011-04-29 18:21:30