Trace number 3280375

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 4800.13 1202.54

General information on the benchmark

Nameindustrial/anbulagan/
hard-unsat/partial-5-11-u.cnf
MD5SUM8e22d84b1ae625d44fbf300a32374a2d
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables164158
Number of clauses730719
Sum of the clauses size1660140
Maximum clause length10
Minimum clause length1
Number of clauses of size 112474
Number of clauses of size 2600009
Number of clauses of size 387256
Number of clauses of size 43440
Number of clauses of size 513760
Number of clauses of size over 513780

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3280375-1304037280/watcher-3280375-1304037280 -o /tmp/evaluation-result-3280375-1304037280/solver-3280375-1304037280 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3280375-1304037280.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.81 7.20 7.71 3/177 28294
/proc/meminfo: memFree=20477000/32950928 swapFree=67111524/67111528
[pid=28294] ppid=28292 vsize=11616 CPUtime=0 cores=1,3,5,7
/proc/28294/stat : 28294 (plingeling) R 28292 28294 28251 0 -1 4202496 997 1254 0 0 0 0 0 0 20 0 1 0 593958740 11894784 926 33554432000 4194304 4417332 140735603715520 18446744073709551615 231895184365 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/28294/statm: 2937 932 146 55 0 823 0

[startup+0.0368431 s]
/proc/loadavg: 4.81 7.20 7.71 3/177 28294
/proc/meminfo: memFree=20477000/32950928 swapFree=67111524/67111528
[pid=28294] ppid=28292 vsize=23124 CPUtime=0.03 cores=1,3,5,7
/proc/28294/stat : 28294 (plingeling) R 28292 28294 28251 0 -1 4202496 3875 1254 0 0 3 0 0 0 20 0 1 0 593958740 23678976 3804 33554432000 4194304 4417332 140735603715520 18446744073709551615 4235968 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/28294/statm: 5781 3804 147 55 0 3667 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 23124

[startup+0.100826 s]
/proc/loadavg: 4.81 7.20 7.71 3/177 28294
/proc/meminfo: memFree=20477000/32950928 swapFree=67111524/67111528
[pid=28294] ppid=28292 vsize=34524 CPUtime=0.09 cores=1,3,5,7
/proc/28294/stat : 28294 (plingeling) R 28292 28294 28251 0 -1 4202496 6725 1254 0 0 9 0 0 0 20 0 1 0 593958740 35352576 6654 33554432000 4194304 4417332 140735603715520 18446744073709551615 4262065 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/28294/statm: 8631 6654 147 55 0 6517 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 34524

[startup+0.300793 s]
/proc/loadavg: 4.81 7.20 7.71 3/177 28294
/proc/meminfo: memFree=20477000/32950928 swapFree=67111524/67111528
[pid=28294] ppid=28292 vsize=102628 CPUtime=0.29 cores=1,3,5,7
/proc/28294/stat : 28294 (plingeling) R 28292 28294 28251 0 -1 4202496 23751 1254 0 0 27 2 0 0 21 0 1 0 593958740 105091072 23680 33554432000 4194304 4417332 140735603715520 18446744073709551615 231895490304 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/28294/statm: 25657 23680 147 55 0 23543 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 102628

[startup+0.700719 s]
/proc/loadavg: 4.81 7.20 7.71 3/177 28294
/proc/meminfo: memFree=20477000/32950928 swapFree=67111524/67111528
[pid=28294] ppid=28292 vsize=122580 CPUtime=0.69 cores=1,3,5,7
/proc/28294/stat : 28294 (plingeling) R 28292 28294 28251 0 -1 4202496 28843 1254 0 0 66 3 0 0 22 0 1 0 593958740 125521920 28669 33554432000 4194304 4417332 140735603715520 18446744073709551615 231895490304 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/28294/statm: 30645 28669 148 55 0 28531 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 122580

[startup+1.5005 s]
/proc/loadavg: 4.81 7.20 7.71 10/186 28308
/proc/meminfo: memFree=20250280/32950928 swapFree=67111524/67111528
[pid=28294] ppid=28292 vsize=245260 CPUtime=3.3 cores=1,3,5,7
/proc/28294/stat : 28294 (plingeling) S 28292 28294 28251 0 -1 4202496 60292 1254 0 0 323 7 0 0 23 0 5 0 593958740 251146240 36760 33554432000 4194304 4417332 140735603715520 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28294/statm: 61315 36760 174 55 0 59201 0
[pid=28294/tid=28305] ppid=28292 vsize=245260 CPUtime=0.6 cores=1,3,5,7
/proc/28294/task/28305/stat : 28305 (plingeling) R 28292 28294 28251 0 -1 4202560 7163 1254 0 0 59 1 0 0 18 0 5 0 593958828 251146240 36760 33554432000 4194304 4417332 140735603715520 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28294/tid=28306] ppid=28292 vsize=245260 CPUtime=0.59 cores=1,3,5,7
/proc/28294/task/28306/stat : 28306 (plingeling) R 28292 28294 28251 0 -1 4202560 6192 1254 0 0 59 0 0 0 18 0 5 0 593958828 251146240 36760 33554432000 4194304 4417332 140735603715520 18446744073709551615 4284626 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28294/tid=28307] ppid=28292 vsize=245260 CPUtime=0.59 cores=1,3,5,7
/proc/28294/task/28307/stat : 28307 (plingeling) R 28292 28294 28251 0 -1 4202560 6876 1254 0 0 59 0 0 0 18 0 5 0 593958828 251146240 36760 33554432000 4194304 4417332 140735603715520 18446744073709551615 4288172 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28294/tid=28308] ppid=28292 vsize=245260 CPUtime=0.6 cores=1,3,5,7
/proc/28294/task/28308/stat : 28308 (plingeling) R 28292 28294 28251 0 -1 4202560 2867 1254 0 0 60 0 0 0 25 0 5 0 593958828 251146240 36760 33554432000 4194304 4417332 140735603715520 18446744073709551615 4264858 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3.3
Current children cumulated vsize (KiB) 245260

[startup+3.1012 s]
/proc/loadavg: 4.81 7.20 7.71 10/186 28308
/proc/meminfo: memFree=20229324/32950928 swapFree=67111524/67111528
[pid=28294] ppid=28292 vsize=249480 CPUtime=9.7 cores=1,3,5,7
/proc/28294/stat : 28294 (plingeling) S 28292 28294 28251 0 -1 4202496 72638 1254 0 0 961 9 0 0 23 0 5 0 593958740 255467520 35536 33554432000 4194304 4417332 140735603715520 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28294/statm: 62370 35536 177 55 0 60256 0
[pid=28294/tid=28305] ppid=28292 vsize=249480 CPUtime=2.2 cores=1,3,5,7
/proc/28294/task/28305/stat : 28305 (plingeling) R 28292 28294 28251 0 -1 4202560 12165 1254 0 0 219 1 0 0 21 0 5 0 593958828 255467520 35536 33554432000 4194304 4417332 140735603715520 18446744073709551615 4265450 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28294/tid=28306] ppid=28292 vsize=249480 CPUtime=2.2 cores=1,3,5,7
/proc/28294/task/28306/stat : 28306 (plingeling) R 28292 28294 28251 0 -1 4202560 10360 1254 0 0 219 1 0 0 21 0 5 0 593958828 255467520 35536 33554432000 4194304 4417332 140735603715520 18446744073709551615 4264232 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28294/tid=28307] ppid=28292 vsize=249480 CPUtime=2.19 cores=1,3,5,7
/proc/28294/task/28307/stat : 28307 (plingeling) R 28292 28294 28251 0 -1 4202560 10051 1254 0 0 218 1 0 0 21 0 5 0 593958828 255467520 35536 33554432000 4194304 4417332 140735603715520 18446744073709551615 4235981 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28294/tid=28308] ppid=28292 vsize=249480 CPUtime=2.2 cores=1,3,5,7
/proc/28294/task/28308/stat : 28308 (plingeling) R 28292 28294 28251 0 -1 4202560 2868 1254 0 0 220 0 0 0 25 0 5 0 593958828 255467520 35536 33554432000 4194304 4417332 140735603715520 18446744073709551615 4259068 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 9.7
Current children cumulated vsize (KiB) 249480

[startup+6.3006 s]
/proc/loadavg: 5.06 7.21 7.72 10/186 28308
/proc/meminfo: memFree=20283264/32950928 swapFree=67111524/67111528
[pid=28294] ppid=28292 vsize=243744 CPUtime=22.48 cores=1,3,5,7
/proc/28294/stat : 28294 (plingeling) S 28292 28294 28251 0 -1 4202496 76612 1254 0 0 2239 9 0 0 23 0 5 0 593958740 249593856 34172 33554432000 4194304 4417332 140735603715520 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28294/statm: 60936 34172 181 55 0 58822 0
[pid=28294/tid=28305] ppid=28292 vsize=243744 CPUtime=5.41 cores=1,3,5,7
/proc/28294/task/28305/stat : 28305 (plingeling) R 28292 28294 28251 0 -1 4202560 14258 1254 0 0 539 2 0 0 25 0 5 0 593958828 249593856 34172 33554432000 4194304 4417332 140735603715520 18446744073709551615 4253228 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28294/tid=28306] ppid=28292 vsize=243744 CPUtime=5.4 cores=1,3,5,7
/proc/28294/task/28306/stat : 28306 (plingeling) R 28292 28294 28251 0 -1 4202560 12237 1254 0 0 539 1 0 0 25 0 5 0 593958828 249593856 34172 33554432000 4194304 4417332 140735603715520 18446744073709551615 4253214 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28294/tid=28307] ppid=28292 vsize=243744 CPUtime=5.37 cores=1,3,5,7
/proc/28294/task/28307/stat : 28307 (plingeling) R 28292 28294 28251 0 -1 4202560 10055 1254 0 0 536 1 0 0 25 0 5 0 593958828 249593856 34172 33554432000 4194304 4417332 140735603715520 18446744073709551615 4265443 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28294/tid=28308] ppid=28292 vsize=243744 CPUtime=5.4 cores=1,3,5,7
/proc/28294/task/28308/stat : 28308 (plingeling) R 28292 28294 28251 0 -1 4202560 2868 1254 0 0 540 0 0 0 25 0 5 0 593958828 249593856 34172 33554432000 4194304 4417332 140735603715520 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 22.48
Current children cumulated vsize (KiB) 243744

[startup+12.7004 s]
/proc/loadavg: 5.30 7.23 7.72 9/186 28308
/proc/meminfo: memFree=20284572/32950928 swapFree=67111524/67111528
[pid=28294] ppid=28292 vsize=240228 CPUtime=48.05 cores=1,3,5,7
/proc/28294/stat : 28294 (plingeling) S 28292 28294 28251 0 -1 4202496 78838 1254 0 0 4795 10 0 0 23 0 5 0 593958740 245993472 34126 33554432000 4194304 4417332 140735603715520 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28294/statm: 60057 34126 181 55 0 57943 0
[pid=28294/tid=28305] ppid=28292 vsize=240228 CPUtime=11.81 cores=1,3,5,7
/proc/28294/task/28305/stat : 28305 (plingeling) R 28292 28294 28251 0 -1 4202560 14258 1254 0 0 1179 2 0 0 25 0 5 0 593958828 245993472 34126 33554432000 4194304 4417332 140735603715520 18446744073709551615 4364383 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28294/tid=28306] ppid=28292 vsize=240228 CPUtime=11.8 cores=1,3,5,7
/proc/28294/task/28306/stat : 28306 (plingeling) R 28292 28294 28251 0 -1 4202560 13206 1254 0 0 1179 1 0 0 25 0 5 0 593958828 245993472 34126 33554432000 4194304 4417332 140735603715520 18446744073709551615 4364519 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28294/tid=28307] ppid=28292 vsize=240228 CPUtime=11.74 cores=1,3,5,7
/proc/28294/task/28307/stat : 28307 (plingeling) R 28292 28294 28251 0 -1 4202560 10482 1254 0 0 1173 1 0 0 25 0 5 0 593958828 245993472 34126 33554432000 4194304 4417332 140735603715520 18446744073709551615 4364683 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28294/tid=28308] ppid=28292 vsize=240228 CPUtime=11.8 cores=1,3,5,7
/proc/28294/task/28308/stat : 28308 (plingeling) R 28292 28294 28251 0 -1 4202560 3698 1254 0 0 1180 0 0 0 25 0 5 0 593958828 245993472 34126 33554432000 4194304 4417332 140735603715520 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 48.05
Current children cumulated vsize (KiB) 240228

[startup+25.501 s]
/proc/loadavg: 5.90 7.26 7.72 11/186 28309
/proc/meminfo: memFree=20270432/32950928 swapFree=67111524/67111528
[pid=28294] ppid=28292 vsize=237436 CPUtime=99.18 cores=1,3,5,7
/proc/28294/stat : 28294 (plingeling) S 28292 28294 28251 0 -1 4202496 88882 1254 0 0 9907 11 0 0 23 0 5 0 593958740 243134464 33713 33554432000 4194304 4417332 140735603715520 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28294/statm: 59359 33713 181 55 0 57245 0
[pid=28294/tid=28305] ppid=28292 vsize=237436 CPUtime=24.61 cores=1,3,5,7
/proc/28294/task/28305/stat : 28305 (plingeling) R 28292 28294 28251 0 -1 4202560 14893 1254 0 0 2459 2 0 0 25 0 5 0 593958828 243134464 33713 33554432000 4194304 4417332 140735603715520 18446744073709551615 4364502 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28294/tid=28306] ppid=28292 vsize=237436 CPUtime=24.6 cores=1,3,5,7
/proc/28294/task/28306/stat : 28306 (plingeling) R 28292 28294 28251 0 -1 4202560 19649 1254 0 0 2458 2 0 0 25 0 5 0 593958828 243134464 33713 33554432000 4194304 4417332 140735603715520 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28294/tid=28307] ppid=28292 vsize=237436 CPUtime=24.46 cores=1,3,5,7
/proc/28294/task/28307/stat : 28307 (plingeling) R 28292 28294 28251 0 -1 4202560 12924 1254 0 0 2445 1 0 0 25 0 5 0 593958828 243134464 33713 33554432000 4194304 4417332 140735603715520 18446744073709551615 4265443 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28294/tid=28308] ppid=28292 vsize=237436 CPUtime=24.6 cores=1,3,5,7
/proc/28294/task/28308/stat : 28308 (plingeling) R 28292 28294 28251 0 -1 4202560 4222 1254 0 0 2460 0 0 0 25 0 5 0 593958828 243134464 33713 33554432000 4194304 4417332 140735603715520 18446744073709551615 4357458 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 99.18
Current children cumulated vsize (KiB) 237436

[startup+51.1012 s]
/proc/loadavg: 6.61 7.32 7.73 9/186 28309
/proc/meminfo: memFree=20191580/32950928 swapFree=67111524/67111528
[pid=28294] ppid=28292 vsize=274948 CPUtime=201.44 cores=1,3,5,7
/proc/28294/stat : 28294 (plingeling) S 28292 28294 28251 0 -1 4202496 115110 1254 0 0 20130 14 0 0 23 0 5 0 593958740 281546752 47055 33554432000 4194304 4417332 140735603715520 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28294/statm: 68737 47055 181 55 0 66623 0
[pid=28294/tid=28305] ppid=28292 vsize=274948 CPUtime=50.19 cores=1,3,5,7
/proc/28294/task/28305/stat : 28305 (plingeling) R 28292 28294 28251 0 -1 4202560 20267 1254 0 0 5017 2 0 0 25 0 5 0 593958828 281546752 47055 33554432000 4194304 4417332 140735603715520 18446744073709551615 4250977 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28294/tid=28306] ppid=28292 vsize=274948 CPUtime=50.2 cores=1,3,5,7
/proc/28294/task/28306/stat : 28306 (plingeling) R 28292 28294 28251 0 -1 4202560 35632 1254 0 0 5016 4 0 0 25 0 5 0 593958828 281546752 47055 33554432000 4194304 4417332 140735603715520 18446744073709551615 4364504 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28294/tid=28307] ppid=28292 vsize=274948 CPUtime=49.93 cores=1,3,5,7
/proc/28294/task/28307/stat : 28307 (plingeling) R 28292 28294 28251 0 -1 4202560 16496 1254 0 0 4991 2 0 0 25 0 5 0 593958828 281546752 47055 33554432000 4194304 4417332 140735603715520 18446744073709551615 4353465 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28294/tid=28308] ppid=28292 vsize=274948 CPUtime=50.2 cores=1,3,5,7
/proc/28294/task/28308/stat : 28308 (plingeling) R 28292 28294 28251 0 -1 4202560 5521 1254 0 0 5020 0 0 0 25 0 5 0 593958828 281546752 47055 33554432000 4194304 4417332 140735603715520 18446744073709551615 4237169 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 201.44
Current children cumulated vsize (KiB) 274948

[startup+102.307 s]
/proc/loadavg: 7.45 7.44 7.74 9/186 28311
/proc/meminfo: memFree=20131904/32950928 swapFree=67111524/67111528
[pid=28294] ppid=28292 vsize=311652 CPUtime=405.99 cores=1,3,5,7
/proc/28294/stat : 28294 (plingeling) S 28292 28294 28251 0 -1 4202496 155792 1254 0 0 40579 20 0 0 23 0 5 0 593958740 319131648 57511 33554432000 4194304 4417332 140735603715520 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28294/statm: 77913 57511 181 55 0 75799 0
[pid=28294/tid=28305] ppid=28292 vsize=311652 CPUtime=101.41 cores=1,3,5,7
/proc/28294/task/28305/stat : 28305 (plingeling) R 28292 28294 28251 0 -1 4202560 24273 1254 0 0 10138 3 0 0 25 0 5 0 593958828 319131648 57511 33554432000 4194304 4417332 140735603715520 18446744073709551615 4309982 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28294/tid=28306] ppid=28292 vsize=311652 CPUtime=101.41 cores=1,3,5,7
/proc/28294/task/28306/stat : 28306 (plingeling) R 28292 28294 28251 0 -1 4202560 59767 1254 0 0 10134 7 0 0 25 0 5 0 593958828 319131648 57511 33554432000 4194304 4417332 140735603715520 18446744073709551615 4270235 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28294/tid=28307] ppid=28292 vsize=311652 CPUtime=100.85 cores=1,3,5,7
/proc/28294/task/28307/stat : 28307 (plingeling) R 28292 28294 28251 0 -1 4202560 23106 1254 0 0 10082 3 0 0 25 0 5 0 593958828 319131648 57511 33554432000 4194304 4417332 140735603715520 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28294/tid=28308] ppid=28292 vsize=311652 CPUtime=101.41 cores=1,3,5,7
/proc/28294/task/28308/stat : 28308 (plingeling) R 28292 28294 28251 0 -1 4202560 11452 1254 0 0 10140 1 0 0 25 0 5 0 593958828 319131648 57511 33554432000 4194304 4417332 140735603715520 18446744073709551615 4353461 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 405.99
Current children cumulated vsize (KiB) 311652

[startup+162.3 s]
/proc/loadavg: 7.85 7.56 7.76 9/186 28312
/proc/meminfo: memFree=20099000/32950928 swapFree=67111524/67111528
[pid=28294] ppid=28292 vsize=335152 CPUtime=645.64 cores=1,3,5,7
/proc/28294/stat : 28294 (plingeling) S 28292 28294 28251 0 -1 4202496 195767 1254 0 0 64539 25 0 0 23 0 5 0 593958740 343195648 65068 33554432000 4194304 4417332 140735603715520 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28294/statm: 83788 65068 181 55 0 81674 0
[pid=28294/tid=28305] ppid=28292 vsize=335152 CPUtime=161.4 cores=1,3,5,7
/proc/28294/task/28305/stat : 28305 (plingeling) R 28292 28294 28251 0 -1 4202560 29093 1254 0 0 16136 4 0 0 25 0 5 0 593958828 343195648 65068 33554432000 4194304 4417332 140735603715520 18446744073709551615 4287573 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28294/tid=28306] ppid=28292 vsize=335152 CPUtime=161.41 cores=1,3,5,7
/proc/28294/task/28306/stat : 28306 (plingeling) R 28292 28294 28251 0 -1 4202560 87608 1254 0 0 16130 11 0 0 25 0 5 0 593958828 343195648 65068 33554432000 4194304 4417332 140735603715520 18446744073709551615 4309899 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28294/tid=28307] ppid=28292 vsize=335152 CPUtime=160.52 cores=1,3,5,7
/proc/28294/task/28307/stat : 28307 (plingeling) R 28292 28294 28251 0 -1 4202560 28095 1254 0 0 16049 3 0 0 25 0 5 0 593958828 343195648 65068 33554432000 4194304 4417332 140735603715520 18446744073709551615 4364722 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28294/tid=28308] ppid=28292 vsize=335152 CPUtime=161.41 cores=1,3,5,7
/proc/28294/task/28308/stat : 28308 (plingeling) R 28292 28294 28251 0 -1 4202560 13777 1254 0 0 16139 2 0 0 25 0 5 0 593958828 343195648 65068 33554432000 4194304 4417332 140735603715520 18446744073709551615 4366359 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 645.64
Current children cumulated vsize (KiB) 335152


################
# More data... #
################

[pid=28294/tid=28306] ppid=28292 vsize=527204 CPUtime=761.43 cores=1,3,5,7
/proc/28294/task/28306/stat : 28306 (plingeling) R 28292 28294 28251 0 -1 4202560 372383 1254 0 0 76095 48 0 0 25 0 5 0 593958828 539856896 104136 33554432000 4194304 4417332 140735603715520 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28294/tid=28307] ppid=28292 vsize=527204 CPUtime=757.26 cores=1,3,5,7
/proc/28294/task/28307/stat : 28307 (plingeling) R 28292 28294 28251 0 -1 4202560 100224 1254 0 0 75713 13 0 0 25 0 5 0 593958828 539856896 104136 33554432000 4194304 4417332 140735603715520 18446744073709551615 4346375 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28294/tid=28308] ppid=28292 vsize=527204 CPUtime=761.43 cores=1,3,5,7
/proc/28294/task/28308/stat : 28308 (plingeling) R 28292 28294 28251 0 -1 4202560 187735 1254 0 0 76116 27 0 0 25 0 5 0 593958828 539856896 104136 33554432000 4194304 4417332 140735603715520 18446744073709551615 4364473 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3041.64
Current children cumulated vsize (KiB) 527204

[startup+822.301 s]
/proc/loadavg: 8.00 8.03 7.93 9/186 28333
/proc/meminfo: memFree=19797144/32950928 swapFree=67111524/67111528
[pid=28294] ppid=28292 vsize=582376 CPUtime=3281.29 cores=1,3,5,7
/proc/28294/stat : 28294 (plingeling) S 28292 28294 28251 0 -1 4202496 855780 1254 0 0 328016 113 0 0 23 0 5 0 593958740 596353024 119777 33554432000 4194304 4417332 140735603715520 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28294/statm: 145594 119777 181 55 0 143480 0
[pid=28294/tid=28305] ppid=28292 vsize=582376 CPUtime=820.6 cores=1,3,5,7
/proc/28294/task/28305/stat : 28305 (plingeling) R 28292 28294 28251 0 -1 4202560 113127 1254 0 0 82046 14 0 0 25 0 5 0 593958828 596353024 119777 33554432000 4194304 4417332 140735603715520 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28294/tid=28306] ppid=28292 vsize=582376 CPUtime=821.43 cores=1,3,5,7
/proc/28294/task/28306/stat : 28306 (plingeling) R 28292 28294 28251 0 -1 4202560 401142 1254 0 0 82091 52 0 0 25 0 5 0 593958828 596353024 119777 33554432000 4194304 4417332 140735603715520 18446744073709551615 4353519 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28294/tid=28307] ppid=28292 vsize=582376 CPUtime=816.93 cores=1,3,5,7
/proc/28294/task/28307/stat : 28307 (plingeling) R 28292 28294 28251 0 -1 4202560 106127 1254 0 0 81679 14 0 0 25 0 5 0 593958828 596353024 119777 33554432000 4194304 4417332 140735603715520 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28294/tid=28308] ppid=28292 vsize=582376 CPUtime=821.42 cores=1,3,5,7
/proc/28294/task/28308/stat : 28308 (plingeling) R 28292 28294 28251 0 -1 4202560 198190 1254 0 0 82115 27 0 0 25 0 5 0 593958828 596353024 119777 33554432000 4194304 4417332 140735603715520 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3281.29
Current children cumulated vsize (KiB) 582376

[startup+882.301 s]
/proc/loadavg: 8.05 8.03 7.93 10/186 28334
/proc/meminfo: memFree=19816692/32950928 swapFree=67111524/67111528
[pid=28294] ppid=28292 vsize=511020 CPUtime=3520.97 cores=1,3,5,7
/proc/28294/stat : 28294 (plingeling) S 28292 28294 28251 0 -1 4202496 893765 1254 0 0 351978 119 0 0 23 0 5 0 593958740 523284480 110949 33554432000 4194304 4417332 140735603715520 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28294/statm: 127755 110949 181 55 0 125641 0
[pid=28294/tid=28305] ppid=28292 vsize=511020 CPUtime=880.61 cores=1,3,5,7
/proc/28294/task/28305/stat : 28305 (plingeling) R 28292 28294 28251 0 -1 4202560 118889 1254 0 0 88046 15 0 0 25 0 5 0 593958828 523284480 110949 33554432000 4194304 4417332 140735603715520 18446744073709551615 4236574 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28294/tid=28306] ppid=28292 vsize=511020 CPUtime=881.43 cores=1,3,5,7
/proc/28294/task/28306/stat : 28306 (plingeling) R 28292 28294 28251 0 -1 4202560 414910 1254 0 0 88089 54 0 0 25 0 5 0 593958828 523284480 110949 33554432000 4194304 4417332 140735603715520 18446744073709551615 4346401 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28294/tid=28307] ppid=28292 vsize=511020 CPUtime=876.6 cores=1,3,5,7
/proc/28294/task/28307/stat : 28307 (plingeling) R 28292 28294 28251 0 -1 4202560 107997 1254 0 0 87646 14 0 0 25 0 5 0 593958828 523284480 110949 33554432000 4194304 4417332 140735603715520 18446744073709551615 4354410 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28294/tid=28308] ppid=28292 vsize=511020 CPUtime=881.43 cores=1,3,5,7
/proc/28294/task/28308/stat : 28308 (plingeling) R 28292 28294 28251 0 -1 4202560 214775 1254 0 0 88113 30 0 0 25 0 5 0 593958828 523284480 110949 33554432000 4194304 4417332 140735603715520 18446744073709551615 4357458 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3520.97
Current children cumulated vsize (KiB) 511020

[startup+942.3 s]
/proc/loadavg: 8.06 8.04 7.94 10/186 28336
/proc/meminfo: memFree=19753272/32950928 swapFree=67111524/67111528
[pid=28294] ppid=28292 vsize=562816 CPUtime=3760.65 cores=1,3,5,7
/proc/28294/stat : 28294 (plingeling) S 28292 28294 28251 0 -1 4202496 950849 1254 0 0 375939 126 0 0 23 0 5 0 593958740 576323584 117216 33554432000 4194304 4417332 140735603715520 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28294/statm: 140704 117216 181 55 0 138590 0
[pid=28294/tid=28305] ppid=28292 vsize=562816 CPUtime=940.61 cores=1,3,5,7
/proc/28294/task/28305/stat : 28305 (plingeling) R 28292 28294 28251 0 -1 4202560 123748 1254 0 0 94045 16 0 0 25 0 5 0 593958828 576323584 117216 33554432000 4194304 4417332 140735603715520 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28294/tid=28306] ppid=28292 vsize=562816 CPUtime=941.42 cores=1,3,5,7
/proc/28294/task/28306/stat : 28306 (plingeling) R 28292 28294 28251 0 -1 4202560 439481 1254 0 0 94086 56 0 0 25 0 5 0 593958828 576323584 117216 33554432000 4194304 4417332 140735603715520 18446744073709551615 4364742 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28294/tid=28307] ppid=28292 vsize=562816 CPUtime=936.27 cores=1,3,5,7
/proc/28294/task/28307/stat : 28307 (plingeling) R 28292 28294 28251 0 -1 4202560 111622 1254 0 0 93613 14 0 0 25 0 5 0 593958828 576323584 117216 33554432000 4194304 4417332 140735603715520 18446744073709551615 4354410 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28294/tid=28308] ppid=28292 vsize=562816 CPUtime=941.43 cores=1,3,5,7
/proc/28294/task/28308/stat : 28308 (plingeling) R 28292 28294 28251 0 -1 4202560 238804 1254 0 0 94109 34 0 0 25 0 5 0 593958828 576323584 117216 33554432000 4194304 4417332 140735603715520 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3760.65
Current children cumulated vsize (KiB) 562816

[startup+1002.3 s]
/proc/loadavg: 8.11 8.06 7.95 9/186 28338
/proc/meminfo: memFree=19769096/32950928 swapFree=67111524/67111528
[pid=28294] ppid=28292 vsize=555828 CPUtime=4000.33 cores=1,3,5,7
/proc/28294/stat : 28294 (plingeling) S 28292 28294 28251 0 -1 4202496 1016259 1254 0 0 399898 135 0 0 23 0 5 0 593958740 569167872 124850 33554432000 4194304 4417332 140735603715520 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28294/statm: 138957 124850 181 55 0 136843 0
[pid=28294/tid=28305] ppid=28292 vsize=555828 CPUtime=1000.61 cores=1,3,5,7
/proc/28294/task/28305/stat : 28305 (plingeling) R 28292 28294 28251 0 -1 4202560 141001 1254 0 0 100043 18 0 0 25 0 5 0 593958828 569167872 124850 33554432000 4194304 4417332 140735603715520 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28294/tid=28306] ppid=28292 vsize=555828 CPUtime=1001.43 cores=1,3,5,7
/proc/28294/task/28306/stat : 28306 (plingeling) R 28292 28294 28251 0 -1 4202560 473770 1254 0 0 100082 61 0 0 25 0 5 0 593958828 569167872 124850 33554432000 4194304 4417332 140735603715520 18446744073709551615 4364470 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28294/tid=28307] ppid=28292 vsize=555828 CPUtime=995.94 cores=1,3,5,7
/proc/28294/task/28307/stat : 28307 (plingeling) R 28292 28294 28251 0 -1 4202560 117346 1254 0 0 99579 15 0 0 25 0 5 0 593958828 569167872 124850 33554432000 4194304 4417332 140735603715520 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28294/tid=28308] ppid=28292 vsize=555828 CPUtime=1001.43 cores=1,3,5,7
/proc/28294/task/28308/stat : 28308 (plingeling) R 28292 28294 28251 0 -1 4202560 246948 1254 0 0 100108 35 0 0 25 0 5 0 593958828 569167872 124850 33554432000 4194304 4417332 140735603715520 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4000.33
Current children cumulated vsize (KiB) 555828

[startup+1062.3 s]
/proc/loadavg: 8.08 8.06 7.95 9/186 28339
/proc/meminfo: memFree=19662516/32950928 swapFree=67111524/67111528
[pid=28294] ppid=28292 vsize=649580 CPUtime=4240.02 cores=1,3,5,7
/proc/28294/stat : 28294 (plingeling) S 28292 28294 28251 0 -1 4202496 1089283 1254 0 0 423858 144 0 0 23 0 5 0 593958740 665169920 141522 33554432000 4194304 4417332 140735603715520 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28294/statm: 162395 141522 181 55 0 160281 0
[pid=28294/tid=28305] ppid=28292 vsize=649580 CPUtime=1060.61 cores=1,3,5,7
/proc/28294/task/28305/stat : 28305 (plingeling) R 28292 28294 28251 0 -1 4202560 149524 1254 0 0 106042 19 0 0 25 0 5 0 593958828 665169920 141522 33554432000 4194304 4417332 140735603715520 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28294/tid=28306] ppid=28292 vsize=649580 CPUtime=1061.43 cores=1,3,5,7
/proc/28294/task/28306/stat : 28306 (plingeling) R 28292 28294 28251 0 -1 4202560 506721 1254 0 0 106079 64 0 0 25 0 5 0 593958828 665169920 141522 33554432000 4194304 4417332 140735603715520 18446744073709551615 4312162 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28294/tid=28307] ppid=28292 vsize=649580 CPUtime=1055.62 cores=1,3,5,7
/proc/28294/task/28307/stat : 28307 (plingeling) R 28292 28294 28251 0 -1 4202560 130638 1254 0 0 105545 17 0 0 25 0 5 0 593958828 665169920 141522 33554432000 4194304 4417332 140735603715520 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28294/tid=28308] ppid=28292 vsize=649580 CPUtime=1061.43 cores=1,3,5,7
/proc/28294/task/28308/stat : 28308 (plingeling) R 28292 28294 28251 0 -1 4202560 265206 1254 0 0 106106 37 0 0 25 0 5 0 593958828 665169920 141522 33554432000 4194304 4417332 140735603715520 18446744073709551615 4236852 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4240.02
Current children cumulated vsize (KiB) 649580

[startup+1122.3 s]
/proc/loadavg: 8.03 8.05 7.95 9/186 28341
/proc/meminfo: memFree=19637672/32950928 swapFree=67111524/67111528
[pid=28294] ppid=28292 vsize=671068 CPUtime=4479.69 cores=1,3,5,7
/proc/28294/stat : 28294 (plingeling) S 28292 28294 28251 0 -1 4202496 1134797 1254 0 0 447819 150 0 0 23 0 5 0 593958740 687173632 146230 33554432000 4194304 4417332 140735603715520 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28294/statm: 167767 146230 181 55 0 165653 0
[pid=28294/tid=28305] ppid=28292 vsize=671068 CPUtime=1120.61 cores=1,3,5,7
/proc/28294/task/28305/stat : 28305 (plingeling) R 28292 28294 28251 0 -1 4202560 152011 1254 0 0 112042 19 0 0 25 0 5 0 593958828 687173632 146230 33554432000 4194304 4417332 140735603715520 18446744073709551615 4236649 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28294/tid=28306] ppid=28292 vsize=671068 CPUtime=1121.44 cores=1,3,5,7
/proc/28294/task/28306/stat : 28306 (plingeling) R 28292 28294 28251 0 -1 4202560 523719 1254 0 0 112077 67 0 0 25 0 5 0 593958828 687173632 146230 33554432000 4194304 4417332 140735603715520 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28294/tid=28307] ppid=28292 vsize=671068 CPUtime=1115.28 cores=1,3,5,7
/proc/28294/task/28307/stat : 28307 (plingeling) R 28292 28294 28251 0 -1 4202560 132752 1254 0 0 111511 17 0 0 25 0 5 0 593958828 687173632 146230 33554432000 4194304 4417332 140735603715520 18446744073709551615 4364724 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28294/tid=28308] ppid=28292 vsize=671068 CPUtime=1121.43 cores=1,3,5,7
/proc/28294/task/28308/stat : 28308 (plingeling) R 28292 28294 28251 0 -1 4202560 289121 1254 0 0 112103 40 0 0 25 0 5 0 593958828 687173632 146230 33554432000 4194304 4417332 140735603715520 18446744073709551615 4357458 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4479.69
Current children cumulated vsize (KiB) 671068

[startup+1182.3 s]
/proc/loadavg: 8.01 8.04 7.95 10/186 28342
/proc/meminfo: memFree=19679408/32950928 swapFree=67111524/67111528
[pid=28294] ppid=28292 vsize=624132 CPUtime=4719.36 cores=1,3,5,7
/proc/28294/stat : 28294 (plingeling) S 28292 28294 28251 0 -1 4202496 1231959 1254 0 0 471771 165 0 0 23 0 5 0 593958740 639111168 139808 33554432000 4194304 4417332 140735603715520 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28294/statm: 156033 139808 181 55 0 153919 0
[pid=28294/tid=28305] ppid=28292 vsize=624132 CPUtime=1180.61 cores=1,3,5,7
/proc/28294/task/28305/stat : 28305 (plingeling) R 28292 28294 28251 0 -1 4202560 173197 1254 0 0 118039 22 0 0 25 0 5 0 593958828 639111168 139808 33554432000 4194304 4417332 140735603715520 18446744073709551615 4250524 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28294/tid=28306] ppid=28292 vsize=624132 CPUtime=1181.44 cores=1,3,5,7
/proc/28294/task/28306/stat : 28306 (plingeling) R 28292 28294 28251 0 -1 4202560 545286 1254 0 0 118074 70 0 0 25 0 5 0 593958828 639111168 139808 33554432000 4194304 4417332 140735603715520 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28294/tid=28307] ppid=28292 vsize=624132 CPUtime=1174.97 cores=1,3,5,7
/proc/28294/task/28307/stat : 28307 (plingeling) R 28292 28294 28251 0 -1 4202560 140957 1254 0 0 117478 19 0 0 25 0 5 0 593958828 639111168 139808 33554432000 4194304 4417332 140735603715520 18446744073709551615 4275936 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28294/tid=28308] ppid=28292 vsize=624132 CPUtime=1181.44 cores=1,3,5,7
/proc/28294/task/28308/stat : 28308 (plingeling) R 28292 28294 28251 0 -1 4202560 335325 1254 0 0 118096 48 0 0 25 0 5 0 593958828 639111168 139808 33554432000 4194304 4417332 140735603715520 18446744073709551615 4237180 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4719.36
Current children cumulated vsize (KiB) 624132



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1202.5 s]
/proc/loadavg: 8.00 8.03 7.95 5/180 28344
/proc/meminfo: memFree=19905484/32950928 swapFree=67111524/67111528
[pid=28294] ppid=28292 vsize=661548 CPUtime=4800.08 cores=1,3,5,7
/proc/28294/stat : 28294 (plingeling) S 28292 28294 28251 0 -1 4202496 1259682 1254 0 0 479839 169 0 0 23 0 5 0 593958740 677425152 149162 33554432000 4194304 4417332 140735603715520 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28294/statm: 165387 149162 181 55 0 163273 0
[pid=28294/tid=28305] ppid=28292 vsize=661548 CPUtime=1200.81 cores=1,3,5,7
/proc/28294/task/28305/stat : 28305 (plingeling) R 28292 28294 28251 0 -1 4202560 180983 1254 0 0 120058 23 0 0 25 0 5 0 593958828 677425152 149162 33554432000 4194304 4417332 140735603715520 18446744073709551615 4353461 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28294/tid=28306] ppid=28292 vsize=661548 CPUtime=1201.64 cores=1,3,5,7
/proc/28294/task/28306/stat : 28306 (plingeling) R 28292 28294 28251 0 -1 4202560 551366 1254 0 0 120093 71 0 0 25 0 5 0 593958828 677425152 149162 33554432000 4194304 4417332 140735603715520 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28294/tid=28307] ppid=28292 vsize=661548 CPUtime=1195.05 cores=1,3,5,7
/proc/28294/task/28307/stat : 28307 (plingeling) R 28292 28294 28251 0 -1 4202560 151899 1254 0 0 119485 20 0 0 25 0 5 0 593958828 677425152 149162 33554432000 4194304 4417332 140735603715520 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28294/tid=28308] ppid=28292 vsize=661548 CPUtime=1201.64 cores=1,3,5,7
/proc/28294/task/28308/stat : 28308 (plingeling) R 28292 28294 28251 0 -1 4202560 338240 1254 0 0 120116 48 0 0 25 0 5 0 593958828 677425152 149162 33554432000 4194304 4417332 140735603715520 18446744073709551615 4237221 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4800.08
Current children cumulated vsize (KiB) 661548

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1202.5 s]
/proc/loadavg: 8.00 8.03 7.95 5/180 28344
/proc/meminfo: memFree=19905484/32950928 swapFree=67111524/67111528
[pid=28294] ppid=28292 vsize=661548 CPUtime=4800.08 cores=1,3,5,7
/proc/28294/stat : 28294 (plingeling) S 28292 28294 28251 0 -1 4202496 1259682 1254 0 0 479839 169 0 0 23 0 5 0 593958740 677425152 149162 33554432000 4194304 4417332 140735603715520 18446744073709551615 231907293157 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/28294/statm: 165387 149162 181 55 0 163273 0
[pid=28294/tid=28305] ppid=28292 vsize=661548 CPUtime=1200.81 cores=1,3,5,7
/proc/28294/task/28305/stat : 28305 (plingeling) R 28292 28294 28251 0 -1 4202560 180983 1254 0 0 120058 23 0 0 25 0 5 0 593958828 677425152 149162 33554432000 4194304 4417332 140735603715520 18446744073709551615 4353461 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=28294/tid=28306] ppid=28292 vsize=661548 CPUtime=1201.64 cores=1,3,5,7
/proc/28294/task/28306/stat : 28306 (plingeling) R 28292 28294 28251 0 -1 4202560 551366 1254 0 0 120093 71 0 0 25 0 5 0 593958828 677425152 149162 33554432000 4194304 4417332 140735603715520 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28294/tid=28307] ppid=28292 vsize=661548 CPUtime=1195.05 cores=1,3,5,7
/proc/28294/task/28307/stat : 28307 (plingeling) R 28292 28294 28251 0 -1 4202560 151899 1254 0 0 119485 20 0 0 25 0 5 0 593958828 677425152 149162 33554432000 4194304 4417332 140735603715520 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28294/tid=28308] ppid=28292 vsize=661548 CPUtime=1201.64 cores=1,3,5,7
/proc/28294/task/28308/stat : 28308 (plingeling) R 28292 28294 28251 0 -1 4202560 338240 1254 0 0 120116 48 0 0 25 0 5 0 593958828 677425152 149162 33554432000 4194304 4417332 140735603715520 18446744073709551615 4237221 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4800.08
Current children cumulated vsize (KiB) 661548

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1202.54
CPU time (s): 4800.13
CPU user time (s): 4798.41
CPU system time (s): 1.71974
CPU usage (%): 399.167
Max. virtual memory (cumulated for all children) (KiB): 735968

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4798.41
system time used= 1.71974
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1260936
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 218
involuntary context switches= 36678

runsolver used 1.89671 second user time and 4.67229 second system time

The end

Launcher Data 

Begin job on node114 at 2011-04-29 02:34:41
IDJOB=3280375
IDBENCH=20386
IDSOLVER=1605
FILE ID=node114/3280375-1304037280
RUNJOBID= node114-1304037280-28270
PBS_JOBID= 13170369
Free space on /tmp= 71780 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT07/industrial/anbulagan/hard-unsat/partial-5-11-u.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3280375-1304037280/watcher-3280375-1304037280 -o /tmp/evaluation-result-3280375-1304037280/solver-3280375-1304037280 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3280375-1304037280.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 8e22d84b1ae625d44fbf300a32374a2d
RANDOM SEED=1614623673

node114.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.73
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      20477288 kB
Buffers:       2248932 kB
Cached:        9552196 kB
SwapCached:          4 kB
Active:        3378904 kB
Inactive:      8513504 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      20477288 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:           23396 kB
Writeback:           0 kB
AnonPages:       90104 kB
Mapped:          14716 kB
Slab:           515852 kB
PageTables:       5032 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   231496 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71784 MiB
End job on node114 at 2011-04-29 02:54:45