Trace number 3288839

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 4800.3 1201.93

General information on the benchmark

Nameapplication/fuhs/slp-synthesis-AES/
slp-synthesis-aes-bottom26.cnf
MD5SUM98fee1752ddf8cd7cd77925ecf4bfe6e
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables73074
Number of clauses245746
Sum of the clauses size628346
Maximum clause length2232
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 2124096
Number of clauses of size 3120415
Number of clauses of size 4350
Number of clauses of size 524
Number of clauses of size over 5860

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3288839-1304071823/watcher-3288839-1304071823 -o /tmp/evaluation-result-3288839-1304071823/solver-3288839-1304071823 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3288839-1304071823.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.14 7.84 7.53 2/177 14691
/proc/meminfo: memFree=26292380/32950928 swapFree=67111364/67111528
[pid=14691] ppid=14689 vsize=15748 CPUtime=0 cores=0,2,4,6
/proc/14691/stat : 14691 (plingeling) R 14689 14691 12570 0 -1 4202496 1842 1251 0 0 0 0 0 0 21 0 1 0 518781948 16125952 1771 33554432000 4194304 4417332 140734751174480 18446744073709551615 208079926365 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/14691/statm: 3937 1784 140 55 0 1823 0

[startup+0.049025 s]
/proc/loadavg: 7.14 7.84 7.53 2/177 14691
/proc/meminfo: memFree=26292380/32950928 swapFree=67111364/67111528
[pid=14691] ppid=14689 vsize=26172 CPUtime=0.03 cores=0,2,4,6
/proc/14691/stat : 14691 (plingeling) R 14689 14691 12570 0 -1 4202496 4685 1251 0 0 3 0 0 0 21 0 1 0 518781948 26800128 4566 33554432000 4194304 4417332 140734751174480 18446744073709551615 208080232192 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/14691/statm: 6543 4566 148 55 0 4429 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 26172

[startup+0.101015 s]
/proc/loadavg: 7.14 7.84 7.53 2/177 14691
/proc/meminfo: memFree=26292380/32950928 swapFree=67111364/67111528
[pid=14691] ppid=14689 vsize=34000 CPUtime=0.09 cores=0,2,4,6
/proc/14691/stat : 14691 (plingeling) R 14689 14691 12570 0 -1 4202496 6642 1251 0 0 8 1 0 0 21 0 1 0 518781948 34816000 6523 33554432000 4194304 4417332 140734751174480 18446744073709551615 4231321 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/14691/statm: 8500 6523 148 55 0 6386 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 34000

[startup+0.300977 s]
/proc/loadavg: 7.14 7.84 7.53 2/177 14691
/proc/meminfo: memFree=26292380/32950928 swapFree=67111364/67111528
[pid=14691] ppid=14689 vsize=80872 CPUtime=0.28 cores=0,2,4,6
/proc/14691/stat : 14691 (plingeling) R 14689 14691 12570 0 -1 4202496 18360 1251 0 0 26 2 0 0 22 0 1 0 518781948 82812928 18241 33554432000 4194304 4417332 140734751174480 18446744073709551615 4264900 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/14691/statm: 20218 18241 148 55 0 18104 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 80872

[startup+0.700904 s]
/proc/loadavg: 7.14 7.84 7.53 2/177 14691
/proc/meminfo: memFree=26292380/32950928 swapFree=67111364/67111528
[pid=14691] ppid=14689 vsize=179816 CPUtime=1.83 cores=0,2,4,6
/proc/14691/stat : 14691 (plingeling) S 14689 14691 12570 0 -1 4202496 31785 1251 0 0 179 4 0 0 22 0 5 0 518781948 184131584 18563 33554432000 4194304 4417332 140734751174480 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/14691/statm: 44954 18563 174 55 0 42840 0
Current children cumulated CPU time (s) 1.83
Current children cumulated vsize (KiB) 179816

[startup+1.50075 s]
/proc/loadavg: 7.14 7.84 7.53 9/186 14705
/proc/meminfo: memFree=26199192/32950928 swapFree=67111364/67111528
[pid=14691] ppid=14689 vsize=173744 CPUtime=5.03 cores=0,2,4,6
/proc/14691/stat : 14691 (plingeling) S 14689 14691 12570 0 -1 4202496 45575 1251 0 0 497 6 0 0 22 0 5 0 518781948 177913856 19047 33554432000 4194304 4417332 140734751174480 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/14691/statm: 43436 19047 175 55 0 41322 0
[pid=14691/tid=14702] ppid=14689 vsize=173744 CPUtime=1.17 cores=0,2,4,6
/proc/14691/task/14702/stat : 14702 (plingeling) R 14689 14691 12570 0 -1 4202560 8908 1251 0 0 117 0 0 0 25 0 5 0 518781980 177913856 19047 33554432000 4194304 4417332 140734751174480 18446744073709551615 4288064 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14691/tid=14703] ppid=14689 vsize=173744 CPUtime=1.17 cores=0,2,4,6
/proc/14691/task/14703/stat : 14703 (plingeling) R 14689 14691 12570 0 -1 4202560 10819 1251 0 0 116 1 0 0 25 0 5 0 518781980 177913856 19047 33554432000 4194304 4417332 140734751174480 18446744073709551615 4283136 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14691/tid=14704] ppid=14689 vsize=173744 CPUtime=1.16 cores=0,2,4,6
/proc/14691/task/14704/stat : 14704 (plingeling) R 14689 14691 12570 0 -1 4202560 6016 1251 0 0 116 0 0 0 25 0 5 0 518781980 177913856 19047 33554432000 4194304 4417332 140734751174480 18446744073709551615 4284301 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14691/tid=14705] ppid=14689 vsize=173744 CPUtime=1.17 cores=0,2,4,6
/proc/14691/task/14705/stat : 14705 (plingeling) R 14689 14691 12570 0 -1 4202560 1460 1251 0 0 117 0 0 0 25 0 5 0 518781980 177913856 19047 33554432000 4194304 4417332 140734751174480 18446744073709551615 4236649 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 5.03
Current children cumulated vsize (KiB) 173744

[startup+3.10045 s]
/proc/loadavg: 7.29 7.86 7.54 9/186 14705
/proc/meminfo: memFree=26187040/32950928 swapFree=67111364/67111528
[pid=14691] ppid=14689 vsize=192764 CPUtime=11.42 cores=0,2,4,6
/proc/14691/stat : 14691 (plingeling) S 14689 14691 12570 0 -1 4202496 56494 1251 0 0 1135 7 0 0 22 0 5 0 518781948 197390336 20949 33554432000 4194304 4417332 140734751174480 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/14691/statm: 48191 20949 179 55 0 46077 0
[pid=14691/tid=14702] ppid=14689 vsize=192764 CPUtime=2.78 cores=0,2,4,6
/proc/14691/task/14702/stat : 14702 (plingeling) R 14689 14691 12570 0 -1 4202560 11212 1251 0 0 277 1 0 0 25 0 5 0 518781980 197390336 20949 33554432000 4194304 4417332 140734751174480 18446744073709551615 4320605 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14691/tid=14703] ppid=14689 vsize=192764 CPUtime=2.76 cores=0,2,4,6
/proc/14691/task/14703/stat : 14703 (plingeling) R 14689 14691 12570 0 -1 4202560 16190 1251 0 0 275 1 0 0 25 0 5 0 518781980 197390336 20949 33554432000 4194304 4417332 140734751174480 18446744073709551615 4320383 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14691/tid=14704] ppid=14689 vsize=192764 CPUtime=2.77 cores=0,2,4,6
/proc/14691/task/14704/stat : 14704 (plingeling) R 14689 14691 12570 0 -1 4202560 9260 1251 0 0 276 1 0 0 25 0 5 0 518781980 197390336 20949 33554432000 4194304 4417332 140734751174480 18446744073709551615 4289668 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14691/tid=14705] ppid=14689 vsize=192764 CPUtime=2.76 cores=0,2,4,6
/proc/14691/task/14705/stat : 14705 (plingeling) R 14689 14691 12570 0 -1 4202560 1460 1251 0 0 276 0 0 0 25 0 5 0 518781980 197390336 20949 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 11.42
Current children cumulated vsize (KiB) 192764

[startup+6.30084 s]
/proc/loadavg: 7.29 7.86 7.54 9/186 14705
/proc/meminfo: memFree=26179604/32950928 swapFree=67111364/67111528
[pid=14691] ppid=14689 vsize=202224 CPUtime=24.2 cores=0,2,4,6
/proc/14691/stat : 14691 (plingeling) S 14689 14691 12570 0 -1 4202496 58936 1251 0 0 2413 7 0 0 22 0 5 0 518781948 207077376 23328 33554432000 4194304 4417332 140734751174480 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/14691/statm: 50556 23328 181 55 0 48442 0
[pid=14691/tid=14702] ppid=14689 vsize=202224 CPUtime=5.97 cores=0,2,4,6
/proc/14691/task/14702/stat : 14702 (plingeling) R 14689 14691 12570 0 -1 4202560 12589 1251 0 0 596 1 0 0 25 0 5 0 518781980 207077376 23328 33554432000 4194304 4417332 140734751174480 18446744073709551615 4250017 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14691/tid=14703] ppid=14689 vsize=202224 CPUtime=5.96 cores=0,2,4,6
/proc/14691/task/14703/stat : 14703 (plingeling) R 14689 14691 12570 0 -1 4202560 17220 1251 0 0 595 1 0 0 25 0 5 0 518781980 207077376 23328 33554432000 4194304 4417332 140734751174480 18446744073709551615 4258134 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14691/tid=14704] ppid=14689 vsize=202224 CPUtime=5.97 cores=0,2,4,6
/proc/14691/task/14704/stat : 14704 (plingeling) R 14689 14691 12570 0 -1 4202560 9295 1251 0 0 596 1 0 0 25 0 5 0 518781980 207077376 23328 33554432000 4194304 4417332 140734751174480 18446744073709551615 4275894 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14691/tid=14705] ppid=14689 vsize=202224 CPUtime=5.95 cores=0,2,4,6
/proc/14691/task/14705/stat : 14705 (plingeling) R 14689 14691 12570 0 -1 4202560 1460 1251 0 0 595 0 0 0 25 0 5 0 518781980 207077376 23328 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 24.2
Current children cumulated vsize (KiB) 202224

[startup+12.7006 s]
/proc/loadavg: 7.40 7.86 7.55 9/186 14706
/proc/meminfo: memFree=26163296/32950928 swapFree=67111364/67111528
[pid=14691] ppid=14689 vsize=201884 CPUtime=49.76 cores=0,2,4,6
/proc/14691/stat : 14691 (plingeling) S 14689 14691 12570 0 -1 4202496 58979 1251 0 0 4969 7 0 0 22 0 5 0 518781948 206729216 23026 33554432000 4194304 4417332 140734751174480 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/14691/statm: 50471 23026 181 55 0 48357 0
[pid=14691/tid=14702] ppid=14689 vsize=201884 CPUtime=12.37 cores=0,2,4,6
/proc/14691/task/14702/stat : 14702 (plingeling) R 14689 14691 12570 0 -1 4202560 12610 1251 0 0 1236 1 0 0 25 0 5 0 518781980 206729216 23026 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14691/tid=14703] ppid=14689 vsize=201884 CPUtime=12.36 cores=0,2,4,6
/proc/14691/task/14703/stat : 14703 (plingeling) R 14689 14691 12570 0 -1 4202560 17242 1251 0 0 1235 1 0 0 25 0 5 0 518781980 206729216 23026 33554432000 4194304 4417332 140734751174480 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14691/tid=14704] ppid=14689 vsize=201884 CPUtime=12.37 cores=0,2,4,6
/proc/14691/task/14704/stat : 14704 (plingeling) R 14689 14691 12570 0 -1 4202560 9295 1251 0 0 1236 1 0 0 25 0 5 0 518781980 206729216 23026 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14691/tid=14705] ppid=14689 vsize=201884 CPUtime=12.31 cores=0,2,4,6
/proc/14691/task/14705/stat : 14705 (plingeling) R 14689 14691 12570 0 -1 4202560 1460 1251 0 0 1231 0 0 0 25 0 5 0 518781980 206729216 23026 33554432000 4194304 4417332 140734751174480 18446744073709551615 4365900 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 49.76
Current children cumulated vsize (KiB) 201884

[startup+25.5012 s]
/proc/loadavg: 7.49 7.86 7.55 9/186 14706
/proc/meminfo: memFree=26128944/32950928 swapFree=67111364/67111528
[pid=14691] ppid=14689 vsize=228728 CPUtime=100.9 cores=0,2,4,6
/proc/14691/stat : 14691 (plingeling) S 14689 14691 12570 0 -1 4202496 66336 1251 0 0 10081 9 0 0 22 0 5 0 518781948 234217472 29743 33554432000 4194304 4417332 140734751174480 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/14691/statm: 57182 29743 181 55 0 55068 0
[pid=14691/tid=14702] ppid=14689 vsize=228728 CPUtime=25.17 cores=0,2,4,6
/proc/14691/task/14702/stat : 14702 (plingeling) R 14689 14691 12570 0 -1 4202560 13163 1251 0 0 2516 1 0 0 25 0 5 0 518781980 234217472 29743 33554432000 4194304 4417332 140734751174480 18446744073709551615 4277864 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14691/tid=14703] ppid=14689 vsize=228728 CPUtime=25.17 cores=0,2,4,6
/proc/14691/task/14703/stat : 14703 (plingeling) R 14689 14691 12570 0 -1 4202560 17529 1251 0 0 2515 2 0 0 25 0 5 0 518781980 234217472 29743 33554432000 4194304 4417332 140734751174480 18446744073709551615 4357838 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14691/tid=14704] ppid=14689 vsize=228728 CPUtime=25.17 cores=0,2,4,6
/proc/14691/task/14704/stat : 14704 (plingeling) R 14689 14691 12570 0 -1 4202560 9930 1251 0 0 2516 1 0 0 25 0 5 0 518781980 234217472 29743 33554432000 4194304 4417332 140734751174480 18446744073709551615 4275978 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14691/tid=14705] ppid=14689 vsize=228728 CPUtime=25.04 cores=0,2,4,6
/proc/14691/task/14705/stat : 14705 (plingeling) R 14689 14691 12570 0 -1 4202560 7342 1251 0 0 2503 1 0 0 25 0 5 0 518781980 234217472 29743 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364526 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 100.9
Current children cumulated vsize (KiB) 228728

[startup+51.1003 s]
/proc/loadavg: 7.74 7.89 7.56 9/186 14707
/proc/meminfo: memFree=26083896/32950928 swapFree=67111364/67111528
[pid=14691] ppid=14689 vsize=247324 CPUtime=203.16 cores=0,2,4,6
/proc/14691/stat : 14691 (plingeling) S 14689 14691 12570 0 -1 4202496 73154 1251 0 0 20306 10 0 0 22 0 5 0 518781948 253259776 34392 33554432000 4194304 4417332 140734751174480 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/14691/statm: 61831 34392 181 55 0 59717 0
[pid=14691/tid=14702] ppid=14689 vsize=247324 CPUtime=50.76 cores=0,2,4,6
/proc/14691/task/14702/stat : 14702 (plingeling) R 14689 14691 12570 0 -1 4202560 15508 1251 0 0 5075 1 0 0 25 0 5 0 518781980 253259776 34392 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364796 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14691/tid=14703] ppid=14689 vsize=247324 CPUtime=50.77 cores=0,2,4,6
/proc/14691/task/14703/stat : 14703 (plingeling) R 14689 14691 12570 0 -1 4202560 17824 1251 0 0 5075 2 0 0 25 0 5 0 518781980 253259776 34392 33554432000 4194304 4417332 140734751174480 18446744073709551615 4352306 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14691/tid=14704] ppid=14689 vsize=247324 CPUtime=50.77 cores=0,2,4,6
/proc/14691/task/14704/stat : 14704 (plingeling) R 14689 14691 12570 0 -1 4202560 11726 1251 0 0 5076 1 0 0 25 0 5 0 518781980 253259776 34392 33554432000 4194304 4417332 140734751174480 18446744073709551615 4357518 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14691/tid=14705] ppid=14689 vsize=247324 CPUtime=50.51 cores=0,2,4,6
/proc/14691/task/14705/stat : 14705 (plingeling) R 14689 14691 12570 0 -1 4202560 9724 1251 0 0 5050 1 0 0 25 0 5 0 518781980 253259776 34392 33554432000 4194304 4417332 140734751174480 18446744073709551615 4237290 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 203.16
Current children cumulated vsize (KiB) 247324

[startup+102.307 s]
/proc/loadavg: 7.93 7.92 7.59 9/186 14708
/proc/meminfo: memFree=26046792/32950928 swapFree=67111364/67111528
[pid=14691] ppid=14689 vsize=255060 CPUtime=407.72 cores=0,2,4,6
/proc/14691/stat : 14691 (plingeling) S 14689 14691 12570 0 -1 4202496 99437 1251 0 0 40757 15 0 0 22 0 5 0 518781948 261181440 36326 33554432000 4194304 4417332 140734751174480 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/14691/statm: 63765 36326 181 55 0 61651 0
[pid=14691/tid=14702] ppid=14689 vsize=255060 CPUtime=101.97 cores=0,2,4,6
/proc/14691/task/14702/stat : 14702 (plingeling) R 14689 14691 12570 0 -1 4202560 19661 1251 0 0 10195 2 0 0 25 0 5 0 518781980 261181440 36326 33554432000 4194304 4417332 140734751174480 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14691/tid=14703] ppid=14689 vsize=255060 CPUtime=101.98 cores=0,2,4,6
/proc/14691/task/14703/stat : 14703 (plingeling) R 14689 14691 12570 0 -1 4202560 20684 1251 0 0 10195 3 0 0 25 0 5 0 518781980 261181440 36326 33554432000 4194304 4417332 140734751174480 18446744073709551615 4353381 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14691/tid=14704] ppid=14689 vsize=255060 CPUtime=101.97 cores=0,2,4,6
/proc/14691/task/14704/stat : 14704 (plingeling) R 14689 14691 12570 0 -1 4202560 12725 1251 0 0 10196 1 0 0 25 0 5 0 518781980 261181440 36326 33554432000 4194304 4417332 140734751174480 18446744073709551615 4237290 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14691/tid=14705] ppid=14689 vsize=255060 CPUtime=101.44 cores=0,2,4,6
/proc/14691/task/14705/stat : 14705 (plingeling) R 14689 14691 12570 0 -1 4202560 27995 1251 0 0 10140 4 0 0 25 0 5 0 518781980 261181440 36326 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364375 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 407.72
Current children cumulated vsize (KiB) 255060

[startup+162.301 s]
/proc/loadavg: 8.01 7.94 7.62 9/186 14710
/proc/meminfo: memFree=25980044/32950928 swapFree=67111364/67111528
[pid=14691] ppid=14689 vsize=286964 CPUtime=647.38 cores=0,2,4,6
/proc/14691/stat : 14691 (plingeling) S 14689 14691 12570 0 -1 4202496 151650 1251 0 0 64716 22 0 0 22 0 5 0 518781948 293851136 44302 33554432000 4194304 4417332 140734751174480 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/14691/statm: 71741 44302 181 55 0 69627 0
[pid=14691/tid=14702] ppid=14689 vsize=286964 CPUtime=161.97 cores=0,2,4,6
/proc/14691/task/14702/stat : 14702 (plingeling) R 14689 14691 12570 0 -1 4202560 37674 1251 0 0 16192 5 0 0 25 0 5 0 518781980 293851136 44302 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14691/tid=14703] ppid=14689 vsize=286964 CPUtime=161.97 cores=0,2,4,6
/proc/14691/task/14703/stat : 14703 (plingeling) R 14689 14691 12570 0 -1 4202560 31748 1251 0 0 16193 4 0 0 25 0 5 0 518781980 293851136 44302 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14691/tid=14704] ppid=14689 vsize=286964 CPUtime=161.97 cores=0,2,4,6
/proc/14691/task/14704/stat : 14704 (plingeling) R 14689 14691 12570 0 -1 4202560 25019 1251 0 0 16194 3 0 0 25 0 5 0 518781980 293851136 44302 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364452 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14691/tid=14705] ppid=14689 vsize=286964 CPUtime=161.12 cores=0,2,4,6
/proc/14691/task/14705/stat : 14705 (plingeling) R 14689 14691 12570 0 -1 4202560 38837 1251 0 0 16107 5 0 0 25 0 5 0 518781980 293851136 44302 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 647.38
Current children cumulated vsize (KiB) 286964


################
# More data... #
################

[pid=14691/tid=14703] ppid=14689 vsize=510800 CPUtime=762 cores=0,2,4,6
/proc/14691/task/14703/stat : 14703 (plingeling) R 14689 14691 12570 0 -1 4202560 74545 1251 0 0 76189 11 0 0 25 0 5 0 518781980 523059200 100261 33554432000 4194304 4417332 140734751174480 18446744073709551615 4366011 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14691/tid=14704] ppid=14689 vsize=510800 CPUtime=761.99 cores=0,2,4,6
/proc/14691/task/14704/stat : 14704 (plingeling) R 14689 14691 12570 0 -1 4202560 60667 1251 0 0 76192 7 0 0 25 0 5 0 518781980 523059200 100261 33554432000 4194304 4417332 140734751174480 18446744073709551615 4309988 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14691/tid=14705] ppid=14689 vsize=510800 CPUtime=757.94 cores=0,2,4,6
/proc/14691/task/14705/stat : 14705 (plingeling) R 14689 14691 12570 0 -1 4202560 154777 1251 0 0 75773 21 0 0 25 0 5 0 518781980 523059200 100261 33554432000 4194304 4417332 140734751174480 18446744073709551615 4356917 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3044.22
Current children cumulated vsize (KiB) 510800

[startup+822.3 s]
/proc/loadavg: 8.13 8.10 7.86 9/186 14727
/proc/meminfo: memFree=25670860/32950928 swapFree=67111364/67111528
[pid=14691] ppid=14689 vsize=452776 CPUtime=3283.88 cores=0,2,4,6
/proc/14691/stat : 14691 (plingeling) S 14689 14691 12570 0 -1 4202496 438506 1251 0 0 328327 61 0 0 22 0 5 0 518781948 463642624 85787 33554432000 4194304 4417332 140734751174480 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/14691/statm: 113194 85787 181 55 0 111080 0
[pid=14691/tid=14702] ppid=14689 vsize=452776 CPUtime=821.93 cores=0,2,4,6
/proc/14691/task/14702/stat : 14702 (plingeling) R 14689 14691 12570 0 -1 4202560 118481 1251 0 0 82177 16 0 0 25 0 5 0 518781980 463642624 85787 33554432000 4194304 4417332 140734751174480 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14691/tid=14703] ppid=14689 vsize=452776 CPUtime=821.99 cores=0,2,4,6
/proc/14691/task/14703/stat : 14703 (plingeling) R 14689 14691 12570 0 -1 4202560 78657 1251 0 0 82188 11 0 0 25 0 5 0 518781980 463642624 85787 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364664 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14691/tid=14704] ppid=14689 vsize=452776 CPUtime=821.99 cores=0,2,4,6
/proc/14691/task/14704/stat : 14704 (plingeling) R 14689 14691 12570 0 -1 4202560 60667 1251 0 0 82192 7 0 0 25 0 5 0 518781980 463642624 85787 33554432000 4194304 4417332 140734751174480 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14691/tid=14705] ppid=14689 vsize=452776 CPUtime=817.61 cores=0,2,4,6
/proc/14691/task/14705/stat : 14705 (plingeling) R 14689 14691 12570 0 -1 4202560 162329 1251 0 0 81739 22 0 0 25 0 5 0 518781980 463642624 85787 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364531 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3283.88
Current children cumulated vsize (KiB) 452776

[startup+882.301 s]
/proc/loadavg: 8.09 8.09 7.87 9/186 14729
/proc/meminfo: memFree=25768048/32950928 swapFree=67111364/67111528
[pid=14691] ppid=14689 vsize=465336 CPUtime=3523.55 cores=0,2,4,6
/proc/14691/stat : 14691 (plingeling) S 14689 14691 12570 0 -1 4202496 446186 1251 0 0 352293 62 0 0 22 0 5 0 518781948 476504064 88895 33554432000 4194304 4417332 140734751174480 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/14691/statm: 116334 88895 181 55 0 114220 0
[pid=14691/tid=14702] ppid=14689 vsize=465336 CPUtime=881.92 cores=0,2,4,6
/proc/14691/task/14702/stat : 14702 (plingeling) R 14689 14691 12570 0 -1 4202560 119817 1251 0 0 88176 16 0 0 25 0 5 0 518781980 476504064 88895 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14691/tid=14703] ppid=14689 vsize=465336 CPUtime=881.99 cores=0,2,4,6
/proc/14691/task/14703/stat : 14703 (plingeling) R 14689 14691 12570 0 -1 4202560 80449 1251 0 0 88188 11 0 0 25 0 5 0 518781980 476504064 88895 33554432000 4194304 4417332 140734751174480 18446744073709551615 4250447 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14691/tid=14704] ppid=14689 vsize=465336 CPUtime=882 cores=0,2,4,6
/proc/14691/task/14704/stat : 14704 (plingeling) R 14689 14691 12570 0 -1 4202560 64780 1251 0 0 88192 8 0 0 25 0 5 0 518781980 476504064 88895 33554432000 4194304 4417332 140734751174480 18446744073709551615 4236634 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14691/tid=14705] ppid=14689 vsize=465336 CPUtime=877.29 cores=0,2,4,6
/proc/14691/task/14705/stat : 14705 (plingeling) R 14689 14691 12570 0 -1 4202560 162768 1251 0 0 87707 22 0 0 25 0 5 0 518781980 476504064 88895 33554432000 4194304 4417332 140734751174480 18446744073709551615 4365915 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3523.55
Current children cumulated vsize (KiB) 465336

[startup+942.3 s]
/proc/loadavg: 8.11 8.10 7.88 9/186 14730
/proc/meminfo: memFree=25707864/32950928 swapFree=67111364/67111528
[pid=14691] ppid=14689 vsize=461764 CPUtime=3763.23 cores=0,2,4,6
/proc/14691/stat : 14691 (plingeling) S 14689 14691 12570 0 -1 4202496 466140 1251 0 0 376258 65 0 0 22 0 5 0 518781948 472846336 88002 33554432000 4194304 4417332 140734751174480 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/14691/statm: 115441 88002 181 55 0 113327 0
[pid=14691/tid=14702] ppid=14689 vsize=461764 CPUtime=941.92 cores=0,2,4,6
/proc/14691/task/14702/stat : 14702 (plingeling) R 14689 14691 12570 0 -1 4202560 119835 1251 0 0 94176 16 0 0 25 0 5 0 518781980 472846336 88002 33554432000 4194304 4417332 140734751174480 18446744073709551615 4365960 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14691/tid=14703] ppid=14689 vsize=461764 CPUtime=942 cores=0,2,4,6
/proc/14691/task/14703/stat : 14703 (plingeling) R 14689 14691 12570 0 -1 4202560 87920 1251 0 0 94188 12 0 0 25 0 5 0 518781980 472846336 88002 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14691/tid=14704] ppid=14689 vsize=461764 CPUtime=942 cores=0,2,4,6
/proc/14691/task/14704/stat : 14704 (plingeling) R 14689 14691 12570 0 -1 4202560 73149 1251 0 0 94190 10 0 0 25 0 5 0 518781980 472846336 88002 33554432000 4194304 4417332 140734751174480 18446744073709551615 4365991 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14691/tid=14705] ppid=14689 vsize=461764 CPUtime=936.96 cores=0,2,4,6
/proc/14691/task/14705/stat : 14705 (plingeling) R 14689 14691 12570 0 -1 4202560 166864 1251 0 0 93674 22 0 0 25 0 5 0 518781980 472846336 88002 33554432000 4194304 4417332 140734751174480 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3763.23
Current children cumulated vsize (KiB) 461764

[startup+1002.3 s]
/proc/loadavg: 8.19 8.12 7.91 9/186 14732
/proc/meminfo: memFree=25650300/32950928 swapFree=67111364/67111528
[pid=14691] ppid=14689 vsize=523836 CPUtime=4002.91 cores=0,2,4,6
/proc/14691/stat : 14691 (plingeling) S 14689 14691 12570 0 -1 4202496 481659 1251 0 0 400224 67 0 0 22 0 5 0 518781948 536408064 103520 33554432000 4194304 4417332 140734751174480 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/14691/statm: 130959 103520 181 55 0 128845 0
[pid=14691/tid=14702] ppid=14689 vsize=523836 CPUtime=1001.92 cores=0,2,4,6
/proc/14691/task/14702/stat : 14702 (plingeling) R 14689 14691 12570 0 -1 4202560 123931 1251 0 0 100175 17 0 0 25 0 5 0 518781980 536408064 103520 33554432000 4194304 4417332 140734751174480 18446744073709551615 4357108 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14691/tid=14703] ppid=14689 vsize=523836 CPUtime=1002 cores=0,2,4,6
/proc/14691/task/14703/stat : 14703 (plingeling) R 14689 14691 12570 0 -1 4202560 89136 1251 0 0 100188 12 0 0 25 0 5 0 518781980 536408064 103520 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14691/tid=14704] ppid=14689 vsize=523836 CPUtime=1002 cores=0,2,4,6
/proc/14691/task/14704/stat : 14704 (plingeling) R 14689 14691 12570 0 -1 4202560 79120 1251 0 0 100190 10 0 0 25 0 5 0 518781980 536408064 103520 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14691/tid=14705] ppid=14689 vsize=523836 CPUtime=996.64 cores=0,2,4,6
/proc/14691/task/14705/stat : 14705 (plingeling) R 14689 14691 12570 0 -1 4202560 171100 1251 0 0 99641 23 0 0 25 0 5 0 518781980 536408064 103520 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364463 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4002.91
Current children cumulated vsize (KiB) 523836

[startup+1062.3 s]
/proc/loadavg: 8.18 8.13 7.92 10/186 14734
/proc/meminfo: memFree=25630048/32950928 swapFree=67111364/67111528
[pid=14691] ppid=14689 vsize=544956 CPUtime=4242.58 cores=0,2,4,6
/proc/14691/stat : 14691 (plingeling) S 14689 14691 12570 0 -1 4202496 495170 1251 0 0 424189 69 0 0 22 0 5 0 518781948 558034944 108800 33554432000 4194304 4417332 140734751174480 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/14691/statm: 136239 108800 181 55 0 134125 0
[pid=14691/tid=14702] ppid=14689 vsize=544956 CPUtime=1061.91 cores=0,2,4,6
/proc/14691/task/14702/stat : 14702 (plingeling) R 14689 14691 12570 0 -1 4202560 123947 1251 0 0 106174 17 0 0 25 0 5 0 518781980 558034944 108800 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364463 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14691/tid=14703] ppid=14689 vsize=544956 CPUtime=1062 cores=0,2,4,6
/proc/14691/task/14703/stat : 14703 (plingeling) R 14689 14691 12570 0 -1 4202560 89154 1251 0 0 106188 12 0 0 25 0 5 0 518781980 558034944 108800 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14691/tid=14704] ppid=14689 vsize=544956 CPUtime=1062 cores=0,2,4,6
/proc/14691/task/14704/stat : 14704 (plingeling) R 14689 14691 12570 0 -1 4202560 79120 1251 0 0 106190 10 0 0 25 0 5 0 518781980 558034944 108800 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14691/tid=14705] ppid=14689 vsize=544956 CPUtime=1056.31 cores=0,2,4,6
/proc/14691/task/14705/stat : 14705 (plingeling) R 14689 14691 12570 0 -1 4202560 184577 1251 0 0 105606 25 0 0 25 0 5 0 518781980 558034944 108800 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364487 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4242.58
Current children cumulated vsize (KiB) 544956

[startup+1122.3 s]
/proc/loadavg: 8.24 8.15 7.93 9/186 14735
/proc/meminfo: memFree=25546440/32950928 swapFree=67111364/67111528
[pid=14691] ppid=14689 vsize=595152 CPUtime=4482.25 cores=0,2,4,6
/proc/14691/stat : 14691 (plingeling) S 14689 14691 12570 0 -1 4202496 540180 1251 0 0 448149 76 0 0 22 0 5 0 518781948 609435648 121349 33554432000 4194304 4417332 140734751174480 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/14691/statm: 148788 121349 181 55 0 146674 0
[pid=14691/tid=14702] ppid=14689 vsize=595152 CPUtime=1121.9 cores=0,2,4,6
/proc/14691/task/14702/stat : 14702 (plingeling) R 14689 14691 12570 0 -1 4202560 132139 1251 0 0 112172 18 0 0 25 0 5 0 518781980 609435648 121349 33554432000 4194304 4417332 140734751174480 18446744073709551615 4236888 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14691/tid=14703] ppid=14689 vsize=595152 CPUtime=1122.01 cores=0,2,4,6
/proc/14691/task/14703/stat : 14703 (plingeling) R 14689 14691 12570 0 -1 4202560 108532 1251 0 0 112186 15 0 0 25 0 5 0 518781980 609435648 121349 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364487 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14691/tid=14704] ppid=14689 vsize=595152 CPUtime=1122 cores=0,2,4,6
/proc/14691/task/14704/stat : 14704 (plingeling) R 14689 14691 12570 0 -1 4202560 92672 1251 0 0 112188 12 0 0 25 0 5 0 518781980 609435648 121349 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364732 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14691/tid=14705] ppid=14689 vsize=595152 CPUtime=1115.98 cores=0,2,4,6
/proc/14691/task/14705/stat : 14705 (plingeling) R 14689 14691 12570 0 -1 4202560 188465 1251 0 0 111572 26 0 0 25 0 5 0 518781980 609435648 121349 33554432000 4194304 4417332 140734751174480 18446744073709551615 4353461 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4482.25
Current children cumulated vsize (KiB) 595152

[startup+1182.3 s]
/proc/loadavg: 8.16 8.14 7.94 9/186 14737
/proc/meminfo: memFree=25624516/32950928 swapFree=67111364/67111528
[pid=14691] ppid=14689 vsize=556232 CPUtime=4721.93 cores=0,2,4,6
/proc/14691/stat : 14691 (plingeling) S 14689 14691 12570 0 -1 4202496 570571 1251 0 0 472112 81 0 0 22 0 5 0 518781948 569581568 109572 33554432000 4194304 4417332 140734751174480 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/14691/statm: 139058 109572 181 55 0 136944 0
[pid=14691/tid=14702] ppid=14689 vsize=556232 CPUtime=1181.9 cores=0,2,4,6
/proc/14691/task/14702/stat : 14702 (plingeling) R 14689 14691 12570 0 -1 4202560 132139 1251 0 0 118172 18 0 0 25 0 5 0 518781980 569581568 109572 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364788 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14691/tid=14703] ppid=14689 vsize=556232 CPUtime=1182.01 cores=0,2,4,6
/proc/14691/task/14703/stat : 14703 (plingeling) R 14689 14691 12570 0 -1 4202560 108549 1251 0 0 118186 15 0 0 25 0 5 0 518781980 569581568 109572 33554432000 4194304 4417332 140734751174480 18446744073709551615 4259793 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14691/tid=14704] ppid=14689 vsize=556232 CPUtime=1182.01 cores=0,2,4,6
/proc/14691/task/14704/stat : 14704 (plingeling) R 14689 14691 12570 0 -1 4202560 110505 1251 0 0 118186 15 0 0 25 0 5 0 518781980 569581568 109572 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364526 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14691/tid=14705] ppid=14689 vsize=556232 CPUtime=1175.66 cores=0,2,4,6
/proc/14691/task/14705/stat : 14705 (plingeling) R 14689 14691 12570 0 -1 4202560 201006 1251 0 0 117538 28 0 0 25 0 5 0 518781980 569581568 109572 33554432000 4194304 4417332 140734751174480 18446744073709551615 4357340 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4721.93
Current children cumulated vsize (KiB) 556232



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.9 s]
/proc/loadavg: 8.11 8.13 7.94 5/180 14738
/proc/meminfo: memFree=25828284/32950928 swapFree=67111364/67111528
[pid=14691] ppid=14689 vsize=597500 CPUtime=4800.24 cores=0,2,4,6
/proc/14691/stat : 14691 (plingeling) S 14689 14691 12570 0 -1 4202496 584234 1251 0 0 479941 83 0 0 22 0 5 0 518781948 611840000 119889 33554432000 4194304 4417332 140734751174480 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/14691/statm: 149375 119889 181 55 0 147261 0
[pid=14691/tid=14702] ppid=14689 vsize=597500 CPUtime=1201.51 cores=0,2,4,6
/proc/14691/task/14702/stat : 14702 (plingeling) R 14689 14691 12570 0 -1 4202560 132157 1251 0 0 120133 18 0 0 25 0 5 0 518781980 611840000 119889 33554432000 4194304 4417332 140734751174480 18446744073709551615 4365065 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14691/tid=14703] ppid=14689 vsize=597500 CPUtime=1201.61 cores=0,2,4,6
/proc/14691/task/14703/stat : 14703 (plingeling) R 14689 14691 12570 0 -1 4202560 112645 1251 0 0 120146 15 0 0 25 0 5 0 518781980 611840000 119889 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14691/tid=14704] ppid=14689 vsize=597500 CPUtime=1201.61 cores=0,2,4,6
/proc/14691/task/14704/stat : 14704 (plingeling) R 14689 14691 12570 0 -1 4202560 110505 1251 0 0 120146 15 0 0 25 0 5 0 518781980 611840000 119889 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14691/tid=14705] ppid=14689 vsize=597500 CPUtime=1195.16 cores=0,2,4,6
/proc/14691/task/14705/stat : 14705 (plingeling) R 14689 14691 12570 0 -1 4202560 210555 1251 0 0 119486 30 0 0 25 0 5 0 518781980 611840000 119889 33554432000 4194304 4417332 140734751174480 18446744073709551615 4236549 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4800.24
Current children cumulated vsize (KiB) 597500

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.9 s]
/proc/loadavg: 8.11 8.13 7.94 5/180 14738
/proc/meminfo: memFree=25828284/32950928 swapFree=67111364/67111528
[pid=14691] ppid=14689 vsize=597500 CPUtime=4800.24 cores=0,2,4,6
/proc/14691/stat : 14691 (plingeling) S 14689 14691 12570 0 -1 4202496 584234 1251 0 0 479941 83 0 0 22 0 5 0 518781948 611840000 119889 33554432000 4194304 4417332 140734751174480 18446744073709551615 208092035045 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/14691/statm: 149375 119889 181 55 0 147261 0
[pid=14691/tid=14702] ppid=14689 vsize=597500 CPUtime=1201.51 cores=0,2,4,6
/proc/14691/task/14702/stat : 14702 (plingeling) R 14689 14691 12570 0 -1 4202560 132157 1251 0 0 120133 18 0 0 25 0 5 0 518781980 611840000 119889 33554432000 4194304 4417332 140734751174480 18446744073709551615 4365065 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=14691/tid=14703] ppid=14689 vsize=597500 CPUtime=1201.61 cores=0,2,4,6
/proc/14691/task/14703/stat : 14703 (plingeling) R 14689 14691 12570 0 -1 4202560 112645 1251 0 0 120146 15 0 0 25 0 5 0 518781980 611840000 119889 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=14691/tid=14704] ppid=14689 vsize=597500 CPUtime=1201.61 cores=0,2,4,6
/proc/14691/task/14704/stat : 14704 (plingeling) R 14689 14691 12570 0 -1 4202560 110505 1251 0 0 120146 15 0 0 25 0 5 0 518781980 611840000 119889 33554432000 4194304 4417332 140734751174480 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=14691/tid=14705] ppid=14689 vsize=597500 CPUtime=1195.16 cores=0,2,4,6
/proc/14691/task/14705/stat : 14705 (plingeling) R 14689 14691 12570 0 -1 4202560 210555 1251 0 0 119486 30 0 0 25 0 5 0 518781980 611840000 119889 33554432000 4194304 4417332 140734751174480 18446744073709551615 4236549 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4800.24
Current children cumulated vsize (KiB) 597500

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1201.93
CPU time (s): 4800.3
CPU user time (s): 4799.44
CPU system time (s): 0.858869
CPU usage (%): 399.381
Max. virtual memory (cumulated for all children) (KiB): 632876

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4799.44
system time used= 0.858869
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 585485
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 195
involuntary context switches= 25500

runsolver used 1.92371 second user time and 4.54431 second system time

The end

Launcher Data 

Begin job on node127 at 2011-04-29 12:10:23
IDJOB=3288839
IDBENCH=83107
IDSOLVER=1605
FILE ID=node127/3288839-1304071823
RUNJOBID= node127-1304071822-14666
PBS_JOBID= 13170417
Free space on /tmp= 71816 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT11/application/fuhs/slp-synthesis-AES/slp-synthesis-aes-bottom26.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3288839-1304071823/watcher-3288839-1304071823 -o /tmp/evaluation-result-3288839-1304071823/solver-3288839-1304071823 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3288839-1304071823.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 98fee1752ddf8cd7cd77925ecf4bfe6e
RANDOM SEED=1947543403

node127.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.60
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      26292412 kB
Buffers:       1390692 kB
Cached:        4709388 kB
SwapCached:          0 kB
Active:        1776040 kB
Inactive:      4406756 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      26292412 kB
SwapTotal:    67111528 kB
SwapFree:     67111364 kB
Dirty:            5948 kB
Writeback:           0 kB
AnonPages:       82480 kB
Mapped:          14852 kB
Slab:           410608 kB
PageTables:       4964 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   227180 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71816 MiB
End job on node127 at 2011-04-29 12:30:27