Trace number 3372725

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40000.9 5007.49

General information on the benchmark

Nameapplication/fuhs/slp-synthesis-AES/
slp-synthesis-aes-bottom21.cnf
MD5SUMe6407843d1fbd00b68d4ad600a25dfb2
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables48639
Number of clauses162901
Sum of the clauses size415741
Maximum clause length1542
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 283161
Number of clauses of size 378900
Number of clauses of size 4230
Number of clauses of size 519
Number of clauses of size over 5590

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3372725-1305441272/watcher-3372725-1305441272 -o /tmp/evaluation-result-3372725-1305441272/solver-3372725-1305441272 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3372725-1305441272.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.07 8.01 7.89 2/343 31822
/proc/meminfo: memFree=23759060/32950928 swapFree=45937192/67111528
[pid=31822] ppid=31820 vsize=18968 CPUtime=0 cores=0-7
/proc/31822/stat : 31822 (plingeling) R 31820 31822 31604 0 -1 4202496 2833 1251 0 0 0 0 0 0 21 0 1 0 655723868 19423232 2763 33554432000 4194304 4417452 140734655339520 18446744073709551615 254500691712 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/31822/statm: 4742 2763 148 55 0 2628 0

[startup+0.064368 s]
/proc/loadavg: 8.07 8.01 7.89 2/343 31822
/proc/meminfo: memFree=23759060/32950928 swapFree=45937192/67111528
[pid=31822] ppid=31820 vsize=29704 CPUtime=0.05 cores=0-7
/proc/31822/stat : 31822 (plingeling) R 31820 31822 31604 0 -1 4202496 5504 1251 0 0 5 0 0 0 21 0 1 0 655723868 30416896 5434 33554432000 4194304 4417452 140734655339520 18446744073709551615 254500691712 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/31822/statm: 7426 5434 149 55 0 5312 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 29704

[startup+0.100368 s]
/proc/loadavg: 8.07 8.01 7.89 2/343 31822
/proc/meminfo: memFree=23759060/32950928 swapFree=45937192/67111528
[pid=31822] ppid=31820 vsize=33800 CPUtime=0.08 cores=0-7
/proc/31822/stat : 31822 (plingeling) R 31820 31822 31604 0 -1 4202496 6528 1251 0 0 8 0 0 0 21 0 1 0 655723868 34611200 6458 33554432000 4194304 4417452 140734655339520 18446744073709551615 254500310572 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/31822/statm: 8450 6458 149 55 0 6336 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 33800

[startup+0.300325 s]
/proc/loadavg: 8.07 8.01 7.89 2/343 31822
/proc/meminfo: memFree=23759060/32950928 swapFree=45937192/67111528
[pid=31822] ppid=31820 vsize=81352 CPUtime=0.28 cores=0-7
/proc/31822/stat : 31822 (plingeling) R 31820 31822 31604 0 -1 4202496 18400 1251 0 0 27 1 0 0 22 0 1 0 655723868 83304448 18330 33554432000 4194304 4417452 140734655339520 18446744073709551615 4274358 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/31822/statm: 20338 18330 149 55 0 18224 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 81352

[startup+0.70017 s]
/proc/loadavg: 8.07 8.01 7.89 2/343 31822
/proc/meminfo: memFree=23759060/32950928 swapFree=45937192/67111528
[pid=31822] ppid=31820 vsize=230208 CPUtime=2.72 cores=0-7
/proc/31822/stat : 31822 (plingeling) S 31820 31822 31604 0 -1 4202496 31820 1251 0 0 268 4 0 0 18 0 9 0 655723868 235732992 22694 33554432000 4194304 4417452 140734655339520 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/31822/statm: 57552 22694 175 55 0 55438 0
Current children cumulated CPU time (s) 2.72
Current children cumulated vsize (KiB) 230208

[startup+1.50101 s]
/proc/loadavg: 8.07 8.01 7.89 9/352 31836
/proc/meminfo: memFree=23668112/32950928 swapFree=45937192/67111528
[pid=31822] ppid=31820 vsize=230208 CPUtime=9.11 cores=0-7
/proc/31822/stat : 31822 (plingeling) S 31820 31822 31604 0 -1 4202496 36579 1251 0 0 907 4 0 0 18 0 9 0 655723868 235732992 22783 33554432000 4194304 4417452 140734655339520 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/31822/statm: 57552 22783 175 55 0 55438 0
[pid=31822/tid=31829] ppid=31820 vsize=230208 CPUtime=1.09 cores=0-7
/proc/31822/task/31829/stat : 31829 (plingeling) R 31820 31822 31604 0 -1 4202560 1297 1251 0 0 109 0 0 0 19 0 9 0 655723907 235732992 22783 33554432000 4194304 4417452 140734655339520 18446744073709551615 4284514 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31822/tid=31830] ppid=31820 vsize=230208 CPUtime=1.09 cores=0-7
/proc/31822/task/31830/stat : 31830 (plingeling) R 31820 31822 31604 0 -1 4202560 1250 1251 0 0 109 0 0 0 25 0 9 0 655723907 235732992 22783 33554432000 4194304 4417452 140734655339520 18446744073709551615 4288259 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=31822/tid=31831] ppid=31820 vsize=230208 CPUtime=1.06 cores=0-7
/proc/31822/task/31831/stat : 31831 (plingeling) R 31820 31822 31604 0 -1 4202560 5953 1251 0 0 105 1 0 0 25 0 9 0 655723907 235732992 22783 33554432000 4194304 4417452 140734655339520 18446744073709551615 4284595 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31822/tid=31832] ppid=31820 vsize=230208 CPUtime=1.1 cores=0-7
/proc/31822/task/31832/stat : 31832 (plingeling) R 31820 31822 31604 0 -1 4202560 1031 1251 0 0 110 0 0 0 19 0 9 0 655723907 235732992 22783 33554432000 4194304 4417452 140734655339520 18446744073709551615 4358297 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=31822/tid=31833] ppid=31820 vsize=230208 CPUtime=1.09 cores=0-7
/proc/31822/task/31833/stat : 31833 (plingeling) R 31820 31822 31604 0 -1 4202560 1289 1251 0 0 109 0 0 0 19 0 9 0 655723907 235732992 22783 33554432000 4194304 4417452 140734655339520 18446744073709551615 4282352 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31822/tid=31834] ppid=31820 vsize=230208 CPUtime=1.08 cores=0-7
/proc/31822/task/31834/stat : 31834 (plingeling) R 31820 31822 31604 0 -1 4202560 4127 1251 0 0 108 0 0 0 21 0 9 0 655723908 235732992 22783 33554432000 4194304 4417452 140734655339520 18446744073709551615 4234242 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31822/tid=31835] ppid=31820 vsize=230208 CPUtime=1.07 cores=0-7
/proc/31822/task/31835/stat : 31835 (plingeling) R 31820 31822 31604 0 -1 4202560 2204 1251 0 0 107 0 0 0 22 0 9 0 655723908 235732992 22783 33554432000 4194304 4417452 140734655339520 18446744073709551615 4284341 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=31822/tid=31836] ppid=31820 vsize=230208 CPUtime=1.08 cores=0-7
/proc/31822/task/31836/stat : 31836 (plingeling) R 31820 31822 31604 0 -1 4202560 1012 1251 0 0 108 0 0 0 21 0 9 0 655723908 235732992 22783 33554432000 4194304 4417452 140734655339520 18446744073709551615 4287201 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 9.11
Current children cumulated vsize (KiB) 230208

[startup+3.10069 s]
/proc/loadavg: 8.07 8.01 7.89 9/352 31837
/proc/meminfo: memFree=23665384/32950928 swapFree=45937192/67111528
[pid=31822] ppid=31820 vsize=264576 CPUtime=21.9 cores=0-7
/proc/31822/stat : 31822 (plingeling) S 31820 31822 31604 0 -1 4202496 51567 1251 0 0 2183 7 0 0 18 0 9 0 655723868 270925824 28085 33554432000 4194304 4417452 140734655339520 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/31822/statm: 66144 28085 180 55 0 64030 0
[pid=31822/tid=31829] ppid=31820 vsize=264576 CPUtime=2.69 cores=0-7
/proc/31822/task/31829/stat : 31829 (plingeling) R 31820 31822 31604 0 -1 4202560 3190 1251 0 0 269 0 0 0 22 0 9 0 655723907 270925824 28085 33554432000 4194304 4417452 140734655339520 18446744073709551615 4346468 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31822/tid=31830] ppid=31820 vsize=264576 CPUtime=2.69 cores=0-7
/proc/31822/task/31830/stat : 31830 (plingeling) R 31820 31822 31604 0 -1 4202560 3165 1251 0 0 269 0 0 0 25 0 9 0 655723907 270925824 28085 33554432000 4194304 4417452 140734655339520 18446744073709551615 4330274 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=31822/tid=31831] ppid=31820 vsize=264576 CPUtime=2.65 cores=0-7
/proc/31822/task/31831/stat : 31831 (plingeling) R 31820 31822 31604 0 -1 4202560 8146 1251 0 0 264 1 0 0 25 0 9 0 655723907 270925824 28085 33554432000 4194304 4417452 140734655339520 18446744073709551615 4233563 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31822/tid=31832] ppid=31820 vsize=264576 CPUtime=2.7 cores=0-7
/proc/31822/task/31832/stat : 31832 (plingeling) R 31820 31822 31604 0 -1 4202560 1031 1251 0 0 270 0 0 0 23 0 9 0 655723907 270925824 28085 33554432000 4194304 4417452 140734655339520 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=31822/tid=31833] ppid=31820 vsize=264576 CPUtime=2.69 cores=0-7
/proc/31822/task/31833/stat : 31833 (plingeling) R 31820 31822 31604 0 -1 4202560 3177 1251 0 0 269 0 0 0 23 0 9 0 655723907 270925824 28085 33554432000 4194304 4417452 140734655339520 18446744073709551615 4346458 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31822/tid=31834] ppid=31820 vsize=264576 CPUtime=2.68 cores=0-7
/proc/31822/task/31834/stat : 31834 (plingeling) R 31820 31822 31604 0 -1 4202560 6388 1251 0 0 268 0 0 0 25 0 9 0 655723908 270925824 28085 33554432000 4194304 4417452 140734655339520 18446744073709551615 4270932 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31822/tid=31835] ppid=31820 vsize=264576 CPUtime=2.67 cores=0-7
/proc/31822/task/31835/stat : 31835 (plingeling) R 31820 31822 31604 0 -1 4202560 6011 1251 0 0 267 0 0 0 25 0 9 0 655723908 270925824 28085 33554432000 4194304 4417452 140734655339520 18446744073709551615 4258261 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=31822/tid=31836] ppid=31820 vsize=264576 CPUtime=2.68 cores=0-7
/proc/31822/task/31836/stat : 31836 (plingeling) R 31820 31822 31604 0 -1 4202560 2043 1251 0 0 268 0 0 0 25 0 9 0 655723908 270925824 28085 33554432000 4194304 4417452 140734655339520 18446744073709551615 4274358 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 21.9
Current children cumulated vsize (KiB) 264576

[startup+6.30105 s]
/proc/loadavg: 8.07 8.01 7.89 9/352 31837
/proc/meminfo: memFree=23647652/32950928 swapFree=45937192/67111528
[pid=31822] ppid=31820 vsize=255704 CPUtime=47.47 cores=0-7
/proc/31822/stat : 31822 (plingeling) S 31820 31822 31604 0 -1 4202496 57359 1251 0 0 4739 8 0 0 18 0 9 0 655723868 261840896 26526 33554432000 4194304 4417452 140734655339520 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/31822/statm: 63926 26526 180 55 0 61812 0
[pid=31822/tid=31829] ppid=31820 vsize=255704 CPUtime=5.89 cores=0-7
/proc/31822/task/31829/stat : 31829 (plingeling) R 31820 31822 31604 0 -1 4202560 4034 1251 0 0 589 0 0 0 25 0 9 0 655723907 261840896 26526 33554432000 4194304 4417452 140734655339520 18446744073709551615 4232623 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31822/tid=31830] ppid=31820 vsize=255704 CPUtime=5.89 cores=0-7
/proc/31822/task/31830/stat : 31830 (plingeling) R 31820 31822 31604 0 -1 4202560 4010 1251 0 0 589 0 0 0 25 0 9 0 655723907 261840896 26526 33554432000 4194304 4417452 140734655339520 18446744073709551615 4236649 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=31822/tid=31831] ppid=31820 vsize=255704 CPUtime=5.81 cores=0-7
/proc/31822/task/31831/stat : 31831 (plingeling) R 31820 31822 31604 0 -1 4202560 9463 1251 0 0 580 1 0 0 25 0 9 0 655723907 261840896 26526 33554432000 4194304 4417452 140734655339520 18446744073709551615 4250795 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31822/tid=31832] ppid=31820 vsize=255704 CPUtime=5.9 cores=0-7
/proc/31822/task/31832/stat : 31832 (plingeling) R 31820 31822 31604 0 -1 4202560 2055 1251 0 0 590 0 0 0 25 0 9 0 655723907 261840896 26526 33554432000 4194304 4417452 140734655339520 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=31822/tid=31833] ppid=31820 vsize=255704 CPUtime=5.89 cores=0-7
/proc/31822/task/31833/stat : 31833 (plingeling) R 31820 31822 31604 0 -1 4202560 4276 1251 0 0 589 0 0 0 25 0 9 0 655723907 261840896 26526 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31822/tid=31834] ppid=31820 vsize=255704 CPUtime=5.89 cores=0-7
/proc/31822/task/31834/stat : 31834 (plingeling) R 31820 31822 31604 0 -1 4202560 7032 1251 0 0 588 1 0 0 25 0 9 0 655723908 261840896 26526 33554432000 4194304 4417452 140734655339520 18446744073709551615 4232715 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31822/tid=31835] ppid=31820 vsize=255704 CPUtime=5.87 cores=0-7
/proc/31822/task/31835/stat : 31835 (plingeling) R 31820 31822 31604 0 -1 4202560 6030 1251 0 0 587 0 0 0 25 0 9 0 655723908 261840896 26526 33554432000 4194304 4417452 140734655339520 18446744073709551615 4366407 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=31822/tid=31836] ppid=31820 vsize=255704 CPUtime=5.88 cores=0-7
/proc/31822/task/31836/stat : 31836 (plingeling) R 31820 31822 31604 0 -1 4202560 2043 1251 0 0 588 0 0 0 25 0 9 0 655723908 261840896 26526 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 47.47
Current children cumulated vsize (KiB) 255704

[startup+12.7008 s]
/proc/loadavg: 8.06 8.01 7.89 9/352 31837
/proc/meminfo: memFree=23634876/32950928 swapFree=45937192/67111528
[pid=31822] ppid=31820 vsize=271276 CPUtime=98.6 cores=0-7
/proc/31822/stat : 31822 (plingeling) S 31820 31822 31604 0 -1 4202496 61389 1251 0 0 9851 9 0 0 18 0 9 0 655723868 277786624 30419 33554432000 4194304 4417452 140734655339520 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/31822/statm: 67819 30419 180 55 0 65705 0
[pid=31822/tid=31829] ppid=31820 vsize=271276 CPUtime=12.29 cores=0-7
/proc/31822/task/31829/stat : 31829 (plingeling) R 31820 31822 31604 0 -1 4202560 4057 1251 0 0 1229 0 0 0 25 0 9 0 655723907 277786624 30419 33554432000 4194304 4417452 140734655339520 18446744073709551615 4365150 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31822/tid=31830] ppid=31820 vsize=271276 CPUtime=12.29 cores=0-7
/proc/31822/task/31830/stat : 31830 (plingeling) R 31820 31822 31604 0 -1 4202560 4035 1251 0 0 1229 0 0 0 25 0 9 0 655723907 277786624 30419 33554432000 4194304 4417452 140734655339520 18446744073709551615 4358082 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=31822/tid=31831] ppid=31820 vsize=271276 CPUtime=12.14 cores=0-7
/proc/31822/task/31831/stat : 31831 (plingeling) R 31820 31822 31604 0 -1 4202560 10172 1251 0 0 1213 1 0 0 25 0 9 0 655723907 277786624 30419 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364542 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31822/tid=31832] ppid=31820 vsize=271276 CPUtime=12.3 cores=0-7
/proc/31822/task/31832/stat : 31832 (plingeling) R 31820 31822 31604 0 -1 4202560 3244 1251 0 0 1230 0 0 0 25 0 9 0 655723907 277786624 30419 33554432000 4194304 4417452 140734655339520 18446744073709551615 4354070 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=31822/tid=31833] ppid=31820 vsize=271276 CPUtime=12.29 cores=0-7
/proc/31822/task/31833/stat : 31833 (plingeling) R 31820 31822 31604 0 -1 4202560 4412 1251 0 0 1229 0 0 0 25 0 9 0 655723907 277786624 30419 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31822/tid=31834] ppid=31820 vsize=271276 CPUtime=12.29 cores=0-7
/proc/31822/task/31834/stat : 31834 (plingeling) R 31820 31822 31604 0 -1 4202560 7056 1251 0 0 1228 1 0 0 25 0 9 0 655723908 277786624 30419 33554432000 4194304 4417452 140734655339520 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31822/tid=31835] ppid=31820 vsize=271276 CPUtime=12.27 cores=0-7
/proc/31822/task/31835/stat : 31835 (plingeling) R 31820 31822 31604 0 -1 4202560 7800 1251 0 0 1227 0 0 0 25 0 9 0 655723908 277786624 30419 33554432000 4194304 4417452 140734655339520 18446744073709551615 4237789 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=31822/tid=31836] ppid=31820 vsize=271276 CPUtime=12.28 cores=0-7
/proc/31822/task/31836/stat : 31836 (plingeling) R 31820 31822 31604 0 -1 4202560 2197 1251 0 0 1228 0 0 0 25 0 9 0 655723908 277786624 30419 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364576 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 98.6
Current children cumulated vsize (KiB) 271276

[startup+25.5012 s]
/proc/loadavg: 8.05 8.01 7.89 9/352 31837
/proc/meminfo: memFree=23598788/32950928 swapFree=45937192/67111528
[pid=31822] ppid=31820 vsize=310392 CPUtime=200.86 cores=0-7
/proc/31822/stat : 31822 (plingeling) S 31820 31822 31604 0 -1 4202496 73309 1251 0 0 20076 10 0 0 18 0 9 0 655723868 317841408 40210 33554432000 4194304 4417452 140734655339520 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/31822/statm: 77598 40210 180 55 0 75484 0
[pid=31822/tid=31829] ppid=31820 vsize=310392 CPUtime=25.09 cores=0-7
/proc/31822/task/31829/stat : 31829 (plingeling) R 31820 31822 31604 0 -1 4202560 5624 1251 0 0 2509 0 0 0 25 0 9 0 655723907 317841408 40210 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364446 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31822/tid=31830] ppid=31820 vsize=310392 CPUtime=25.09 cores=0-7
/proc/31822/task/31830/stat : 31830 (plingeling) R 31820 31822 31604 0 -1 4202560 5419 1251 0 0 2509 0 0 0 25 0 9 0 655723907 317841408 40210 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=31822/tid=31831] ppid=31820 vsize=310392 CPUtime=24.8 cores=0-7
/proc/31822/task/31831/stat : 31831 (plingeling) R 31820 31822 31604 0 -1 4202560 12071 1251 0 0 2478 2 0 0 25 0 9 0 655723907 317841408 40210 33554432000 4194304 4417452 140734655339520 18446744073709551615 4366507 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31822/tid=31832] ppid=31820 vsize=310392 CPUtime=25.09 cores=0-7
/proc/31822/task/31832/stat : 31832 (plingeling) R 31820 31822 31604 0 -1 4202560 5027 1251 0 0 2509 0 0 0 25 0 9 0 655723907 317841408 40210 33554432000 4194304 4417452 140734655339520 18446744073709551615 4365973 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=31822/tid=31833] ppid=31820 vsize=310392 CPUtime=25.08 cores=0-7
/proc/31822/task/31833/stat : 31833 (plingeling) R 31820 31822 31604 0 -1 4202560 6738 1251 0 0 2508 0 0 0 25 0 9 0 655723907 317841408 40210 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31822/tid=31834] ppid=31820 vsize=310392 CPUtime=25.09 cores=0-7
/proc/31822/task/31834/stat : 31834 (plingeling) R 31820 31822 31604 0 -1 4202560 7068 1251 0 0 2508 1 0 0 25 0 9 0 655723908 317841408 40210 33554432000 4194304 4417452 140734655339520 18446744073709551615 4232801 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31822/tid=31835] ppid=31820 vsize=310392 CPUtime=25.07 cores=0-7
/proc/31822/task/31835/stat : 31835 (plingeling) R 31820 31822 31604 0 -1 4202560 8842 1251 0 0 2507 0 0 0 25 0 9 0 655723908 317841408 40210 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364653 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=31822/tid=31836] ppid=31820 vsize=310392 CPUtime=25.07 cores=0-7
/proc/31822/task/31836/stat : 31836 (plingeling) R 31820 31822 31604 0 -1 4202560 4104 1251 0 0 2507 0 0 0 25 0 9 0 655723908 317841408 40210 33554432000 4194304 4417452 140734655339520 18446744073709551615 4236706 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 200.86
Current children cumulated vsize (KiB) 310392

[startup+51.1011 s]
/proc/loadavg: 8.09 8.02 7.90 9/352 31838
/proc/meminfo: memFree=23574740/32950928 swapFree=45937192/67111528
[pid=31822] ppid=31820 vsize=329136 CPUtime=405.37 cores=0-7
/proc/31822/stat : 31822 (plingeling) S 31820 31822 31604 0 -1 4202496 100810 1251 0 0 40523 14 0 0 18 0 9 0 655723868 337035264 44960 33554432000 4194304 4417452 140734655339520 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/31822/statm: 82284 44960 180 55 0 80170 0
[pid=31822/tid=31829] ppid=31820 vsize=329136 CPUtime=50.69 cores=0-7
/proc/31822/task/31829/stat : 31829 (plingeling) R 31820 31822 31604 0 -1 4202560 8606 1251 0 0 5068 1 0 0 25 0 9 0 655723907 337035264 44960 33554432000 4194304 4417452 140734655339520 18446744073709551615 4250080 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31822/tid=31830] ppid=31820 vsize=329136 CPUtime=50.7 cores=0-7
/proc/31822/task/31830/stat : 31830 (plingeling) R 31820 31822 31604 0 -1 4202560 7144 1251 0 0 5069 1 0 0 25 0 9 0 655723907 337035264 44960 33554432000 4194304 4417452 140734655339520 18446744073709551615 4236750 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=31822/tid=31831] ppid=31820 vsize=329136 CPUtime=50.12 cores=0-7

################
# More data... #
################


[startup+4722.31 s]
/proc/loadavg: 8.14 8.08 8.01 9/352 31974
/proc/meminfo: memFree=19710388/32950928 swapFree=45937192/67111528
[pid=31822] ppid=31820 vsize=4257120 CPUtime=37723.7 cores=0-7
/proc/31822/stat : 31822 (plingeling) S 31820 31822 31604 0 -1 4202496 4899904 1251 0 0 3771492 881 0 0 18 0 9 0 655723868 4359290880 1026887 33554432000 4194304 4417452 140734655339520 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/31822/statm: 1064280 1026887 180 55 0 1062166 0
[pid=31822/tid=31829] ppid=31820 vsize=4257120 CPUtime=4722.09 cores=0-7
/proc/31822/task/31829/stat : 31829 (plingeling) R 31820 31822 31604 0 -1 4202560 598293 1251 0 0 472099 110 0 0 25 0 9 0 655723907 4359290880 1026887 33554432000 4194304 4417452 140734655339520 18446744073709551615 4353557 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31822/tid=31830] ppid=31820 vsize=4257120 CPUtime=4722.08 cores=0-7
/proc/31822/task/31830/stat : 31830 (plingeling) R 31820 31822 31604 0 -1 4202560 631943 1251 0 0 472103 105 0 0 25 0 9 0 655723907 4359290880 1026887 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=31822/tid=31831] ppid=31820 vsize=4257120 CPUtime=4670.31 cores=0-7
/proc/31822/task/31831/stat : 31831 (plingeling) R 31820 31822 31604 0 -1 4202560 637696 1251 0 0 466909 122 0 0 25 0 9 0 655723907 4359290880 1026887 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31822/tid=31832] ppid=31820 vsize=4257120 CPUtime=4721.22 cores=0-7
/proc/31822/task/31832/stat : 31832 (plingeling) R 31820 31822 31604 0 -1 4202560 256618 1251 0 0 472084 38 0 0 25 0 9 0 655723907 4359290880 1026887 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=31822/tid=31833] ppid=31820 vsize=4257120 CPUtime=4722.1 cores=0-7
/proc/31822/task/31833/stat : 31833 (plingeling) R 31820 31822 31604 0 -1 4202560 777749 1251 0 0 472064 146 0 0 25 0 9 0 655723907 4359290880 1026887 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31822/tid=31834] ppid=31820 vsize=4257120 CPUtime=4722.1 cores=0-7
/proc/31822/task/31834/stat : 31834 (plingeling) R 31820 31822 31604 0 -1 4202560 535989 1251 0 0 472121 89 0 0 25 0 9 0 655723908 4359290880 1026887 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31822/tid=31835] ppid=31820 vsize=4257120 CPUtime=4721.27 cores=0-7
/proc/31822/task/31835/stat : 31835 (plingeling) R 31820 31822 31604 0 -1 4202560 788452 1251 0 0 471966 161 0 0 25 0 9 0 655723908 4359290880 1026887 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=31822/tid=31836] ppid=31820 vsize=4257120 CPUtime=4722.09 cores=0-7
/proc/31822/task/31836/stat : 31836 (plingeling) R 31820 31822 31604 0 -1 4202560 654748 1251 0 0 472105 104 0 0 25 0 9 0 655723908 4359290880 1026887 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 37723.7
Current children cumulated vsize (KiB) 4257120

[startup+4782.3 s]
/proc/loadavg: 8.10 8.08 8.01 9/352 31976
/proc/meminfo: memFree=19850992/32950928 swapFree=45937192/67111528
[pid=31822] ppid=31820 vsize=4051576 CPUtime=38203 cores=0-7
/proc/31822/stat : 31822 (plingeling) S 31820 31822 31604 0 -1 4202496 4961507 1251 0 0 3819410 889 0 0 18 0 9 0 655723868 4148813824 975501 33554432000 4194304 4417452 140734655339520 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/31822/statm: 1012894 975501 180 55 0 1010780 0
[pid=31822/tid=31829] ppid=31820 vsize=4051576 CPUtime=4782.08 cores=0-7
/proc/31822/task/31829/stat : 31829 (plingeling) R 31820 31822 31604 0 -1 4202560 598293 1251 0 0 478098 110 0 0 25 0 9 0 655723907 4148813824 975501 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364741 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31822/tid=31830] ppid=31820 vsize=4051576 CPUtime=4782.07 cores=0-7
/proc/31822/task/31830/stat : 31830 (plingeling) R 31820 31822 31604 0 -1 4202560 631943 1251 0 0 478102 105 0 0 25 0 9 0 655723907 4148813824 975501 33554432000 4194304 4417452 140734655339520 18446744073709551615 4259140 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=31822/tid=31831] ppid=31820 vsize=4051576 CPUtime=4729.65 cores=0-7
/proc/31822/task/31831/stat : 31831 (plingeling) R 31820 31822 31604 0 -1 4202560 637696 1251 0 0 472843 122 0 0 25 0 9 0 655723907 4148813824 975501 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31822/tid=31832] ppid=31820 vsize=4051576 CPUtime=4781.21 cores=0-7
/proc/31822/task/31832/stat : 31832 (plingeling) R 31820 31822 31604 0 -1 4202560 256618 1251 0 0 478083 38 0 0 25 0 9 0 655723907 4148813824 975501 33554432000 4194304 4417452 140734655339520 18446744073709551615 4321680 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=31822/tid=31833] ppid=31820 vsize=4051576 CPUtime=4782.1 cores=0-7
/proc/31822/task/31833/stat : 31833 (plingeling) R 31820 31822 31604 0 -1 4202560 834491 1251 0 0 478057 153 0 0 25 0 9 0 655723907 4148813824 975501 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31822/tid=31834] ppid=31820 vsize=4051576 CPUtime=4782.09 cores=0-7
/proc/31822/task/31834/stat : 31834 (plingeling) R 31820 31822 31604 0 -1 4202560 535989 1251 0 0 478120 89 0 0 25 0 9 0 655723908 4148813824 975501 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31822/tid=31835] ppid=31820 vsize=4051576 CPUtime=4781.26 cores=0-7
/proc/31822/task/31835/stat : 31835 (plingeling) R 31820 31822 31604 0 -1 4202560 788452 1251 0 0 477965 161 0 0 25 0 9 0 655723908 4148813824 975501 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=31822/tid=31836] ppid=31820 vsize=4051576 CPUtime=4782.09 cores=0-7
/proc/31822/task/31836/stat : 31836 (plingeling) R 31820 31822 31604 0 -1 4202560 659609 1251 0 0 478103 106 0 0 25 0 9 0 655723908 4148813824 975501 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38203
Current children cumulated vsize (KiB) 4051576

[startup+4842.3 s]
/proc/loadavg: 8.08 8.07 8.01 9/352 31978
/proc/meminfo: memFree=20182548/32950928 swapFree=45937192/67111528
[pid=31822] ppid=31820 vsize=3720484 CPUtime=38682.4 cores=0-7
/proc/31822/stat : 31822 (plingeling) S 31820 31822 31604 0 -1 4202496 5000388 1251 0 0 3867340 895 0 0 18 0 9 0 655723868 3809775616 892728 33554432000 4194304 4417452 140734655339520 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/31822/statm: 930121 892728 180 55 0 928007 0
[pid=31822/tid=31829] ppid=31820 vsize=3720484 CPUtime=4842.08 cores=0-7
/proc/31822/task/31829/stat : 31829 (plingeling) R 31820 31822 31604 0 -1 4202560 598293 1251 0 0 484098 110 0 0 25 0 9 0 655723907 3809775616 892728 33554432000 4194304 4417452 140734655339520 18446744073709551615 4263033 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31822/tid=31830] ppid=31820 vsize=3720484 CPUtime=4842.07 cores=0-7
/proc/31822/task/31830/stat : 31830 (plingeling) R 31820 31822 31604 0 -1 4202560 631943 1251 0 0 484102 105 0 0 25 0 9 0 655723907 3809775616 892728 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=31822/tid=31831] ppid=31820 vsize=3720484 CPUtime=4788.99 cores=0-7
/proc/31822/task/31831/stat : 31831 (plingeling) R 31820 31822 31604 0 -1 4202560 643156 1251 0 0 478775 124 0 0 25 0 9 0 655723907 3809775616 892728 33554432000 4194304 4417452 140734655339520 18446744073709551615 4366065 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31822/tid=31832] ppid=31820 vsize=3720484 CPUtime=4841.2 cores=0-7
/proc/31822/task/31832/stat : 31832 (plingeling) R 31820 31822 31604 0 -1 4202560 256618 1251 0 0 484082 38 0 0 25 0 9 0 655723907 3809775616 892728 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=31822/tid=31833] ppid=31820 vsize=3720484 CPUtime=4842.09 cores=0-7
/proc/31822/task/31833/stat : 31833 (plingeling) R 31820 31822 31604 0 -1 4202560 850875 1251 0 0 484055 154 0 0 25 0 9 0 655723907 3809775616 892728 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31822/tid=31834] ppid=31820 vsize=3720484 CPUtime=4842.09 cores=0-7
/proc/31822/task/31834/stat : 31834 (plingeling) R 31820 31822 31604 0 -1 4202560 535989 1251 0 0 484120 89 0 0 25 0 9 0 655723908 3809775616 892728 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31822/tid=31835] ppid=31820 vsize=3720484 CPUtime=4841.26 cores=0-7
/proc/31822/task/31835/stat : 31835 (plingeling) R 31820 31822 31604 0 -1 4202560 788452 1251 0 0 483965 161 0 0 25 0 9 0 655723908 3809775616 892728 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=31822/tid=31836] ppid=31820 vsize=3720484 CPUtime=4842.09 cores=0-7
/proc/31822/task/31836/stat : 31836 (plingeling) R 31820 31822 31604 0 -1 4202560 676646 1251 0 0 484101 108 0 0 25 0 9 0 655723908 3809775616 892728 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38682.3
Current children cumulated vsize (KiB) 3720484

[startup+4902.3 s]
/proc/loadavg: 8.07 8.07 8.01 9/352 31979
/proc/meminfo: memFree=20138384/32950928 swapFree=45937192/67111528
[pid=31822] ppid=31820 vsize=3764604 CPUtime=39161.7 cores=0-7
/proc/31822/stat : 31822 (plingeling) S 31820 31822 31604 0 -1 4202496 5011418 1251 0 0 3915274 896 0 0 18 0 9 0 655723868 3854954496 903758 33554432000 4194304 4417452 140734655339520 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/31822/statm: 941151 903758 180 55 0 939037 0
[pid=31822/tid=31829] ppid=31820 vsize=3764604 CPUtime=4902.09 cores=0-7
/proc/31822/task/31829/stat : 31829 (plingeling) R 31820 31822 31604 0 -1 4202560 598293 1251 0 0 490099 110 0 0 25 0 9 0 655723907 3854954496 903758 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31822/tid=31830] ppid=31820 vsize=3764604 CPUtime=4902.07 cores=0-7
/proc/31822/task/31830/stat : 31830 (plingeling) R 31820 31822 31604 0 -1 4202560 641014 1251 0 0 490101 106 0 0 25 0 9 0 655723907 3854954496 903758 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=31822/tid=31831] ppid=31820 vsize=3764604 CPUtime=4848.34 cores=0-7
/proc/31822/task/31831/stat : 31831 (plingeling) R 31820 31822 31604 0 -1 4202560 644189 1251 0 0 484710 124 0 0 25 0 9 0 655723907 3854954496 903758 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364653 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31822/tid=31832] ppid=31820 vsize=3764604 CPUtime=4901.19 cores=0-7
/proc/31822/task/31832/stat : 31832 (plingeling) R 31820 31822 31604 0 -1 4202560 256618 1251 0 0 490081 38 0 0 25 0 9 0 655723907 3854954496 903758 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=31822/tid=31833] ppid=31820 vsize=3764604 CPUtime=4902.1 cores=0-7
/proc/31822/task/31833/stat : 31833 (plingeling) R 31820 31822 31604 0 -1 4202560 850875 1251 0 0 490056 154 0 0 25 0 9 0 655723907 3854954496 903758 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31822/tid=31834] ppid=31820 vsize=3764604 CPUtime=4902.1 cores=0-7
/proc/31822/task/31834/stat : 31834 (plingeling) R 31820 31822 31604 0 -1 4202560 535989 1251 0 0 490121 89 0 0 25 0 9 0 655723908 3854954496 903758 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31822/tid=31835] ppid=31820 vsize=3764604 CPUtime=4901.27 cores=0-7
/proc/31822/task/31835/stat : 31835 (plingeling) R 31820 31822 31604 0 -1 4202560 788452 1251 0 0 489966 161 0 0 25 0 9 0 655723908 3854954496 903758 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364687 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=31822/tid=31836] ppid=31820 vsize=3764604 CPUtime=4902.09 cores=0-7
/proc/31822/task/31836/stat : 31836 (plingeling) R 31820 31822 31604 0 -1 4202560 677572 1251 0 0 490101 108 0 0 25 0 9 0 655723908 3854954496 903758 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364809 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39161.7
Current children cumulated vsize (KiB) 3764604

[startup+4962.3 s]
/proc/loadavg: 8.06 8.07 8.01 9/352 31981
/proc/meminfo: memFree=19811272/32950928 swapFree=45937192/67111528
[pid=31822] ppid=31820 vsize=4090900 CPUtime=39641 cores=0-7
/proc/31822/stat : 31822 (plingeling) S 31820 31822 31604 0 -1 4202496 5148026 1251 0 0 3963178 924 0 0 18 0 9 0 655723868 4189081600 985344 33554432000 4194304 4417452 140734655339520 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/31822/statm: 1022725 985344 180 55 0 1020611 0
[pid=31822/tid=31829] ppid=31820 vsize=4090900 CPUtime=4962.09 cores=0-7
/proc/31822/task/31829/stat : 31829 (plingeling) R 31820 31822 31604 0 -1 4202560 647970 1251 0 0 496085 124 0 0 25 0 9 0 655723907 4189081600 985344 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31822/tid=31830] ppid=31820 vsize=4090900 CPUtime=4962.08 cores=0-7
/proc/31822/task/31830/stat : 31830 (plingeling) R 31820 31822 31604 0 -1 4202560 683380 1251 0 0 496092 116 0 0 25 0 9 0 655723907 4189081600 985344 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=31822/tid=31831] ppid=31820 vsize=4090900 CPUtime=4907.69 cores=0-7
/proc/31822/task/31831/stat : 31831 (plingeling) R 31820 31822 31604 0 -1 4202560 645369 1251 0 0 490644 125 0 0 25 0 9 0 655723907 4189081600 985344 33554432000 4194304 4417452 140734655339520 18446744073709551615 4365106 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31822/tid=31832] ppid=31820 vsize=4090900 CPUtime=4961.18 cores=0-7
/proc/31822/task/31832/stat : 31832 (plingeling) R 31820 31822 31604 0 -1 4202560 256618 1251 0 0 496080 38 0 0 25 0 9 0 655723907 4189081600 985344 33554432000 4194304 4417452 140734655339520 18446744073709551615 4236649 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=31822/tid=31833] ppid=31820 vsize=4090900 CPUtime=4962.1 cores=0-7
/proc/31822/task/31833/stat : 31833 (plingeling) R 31820 31822 31604 0 -1 4202560 850875 1251 0 0 496056 154 0 0 25 0 9 0 655723907 4189081600 985344 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31822/tid=31834] ppid=31820 vsize=4090900 CPUtime=4962.1 cores=0-7
/proc/31822/task/31834/stat : 31834 (plingeling) R 31820 31822 31604 0 -1 4202560 535989 1251 0 0 496121 89 0 0 25 0 9 0 655723908 4189081600 985344 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31822/tid=31835] ppid=31820 vsize=4090900 CPUtime=4961.27 cores=0-7
/proc/31822/task/31835/stat : 31835 (plingeling) R 31820 31822 31604 0 -1 4202560 788452 1251 0 0 495966 161 0 0 25 0 9 0 655723908 4189081600 985344 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=31822/tid=31836] ppid=31820 vsize=4090900 CPUtime=4962.07 cores=0-7
/proc/31822/task/31836/stat : 31836 (plingeling) R 31820 31822 31604 0 -1 4202560 720957 1251 0 0 496094 113 0 0 25 0 9 0 655723908 4189081600 985344 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39641
Current children cumulated vsize (KiB) 4090900



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5007.3 s]
/proc/loadavg: 8.07 8.08 8.01 9/352 31982
/proc/meminfo: memFree=19811128/32950928 swapFree=45937192/67111528
[pid=31822] ppid=31820 vsize=4090900 CPUtime=40000.6 cores=0-7
/proc/31822/stat : 31822 (plingeling) S 31820 31822 31604 0 -1 4202496 5148026 1251 0 0 3999138 924 0 0 18 0 9 0 655723868 4189081600 985344 33554432000 4194304 4417452 140734655339520 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/31822/statm: 1022725 985344 180 55 0 1020611 0
[pid=31822/tid=31829] ppid=31820 vsize=4090900 CPUtime=5007.11 cores=0-7
/proc/31822/task/31829/stat : 31829 (plingeling) R 31820 31822 31604 0 -1 4202560 647970 1251 0 0 500587 124 0 0 25 0 9 0 655723907 4189081600 985344 33554432000 4194304 4417452 140734655339520 18446744073709551615 4308347 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=31822/tid=31830] ppid=31820 vsize=4090900 CPUtime=5007.1 cores=0-7
/proc/31822/task/31830/stat : 31830 (plingeling) R 31820 31822 31604 0 -1 4202560 683380 1251 0 0 500594 116 0 0 25 0 9 0 655723907 4189081600 985344 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=31822/tid=31831] ppid=31820 vsize=4090900 CPUtime=4952.19 cores=0-7
/proc/31822/task/31831/stat : 31831 (plingeling) R 31820 31822 31604 0 -1 4202560 645369 1251 0 0 495094 125 0 0 25 0 9 0 655723907 4189081600 985344 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=31822/tid=31832] ppid=31820 vsize=4090900 CPUtime=5006.19 cores=0-7
/proc/31822/task/31832/stat : 31832 (plingeling) R 31820 31822 31604 0 -1 4202560 256618 1251 0 0 500581 38 0 0 25 0 9 0 655723907 4189081600 985344 33554432000 4194304 4417452 140734655339520 18446744073709551615 4237290 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=31822/tid=31833] ppid=31820 vsize=4090900 CPUtime=5007.11 cores=0-7
/proc/31822/task/31833/stat : 31833 (plingeling) R 31820 31822 31604 0 -1 4202560 850875 1251 0 0 500557 154 0 0 25 0 9 0 655723907 4189081600 985344 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=31822/tid=31834] ppid=31820 vsize=4090900 CPUtime=5007.11 cores=0-7
/proc/31822/task/31834/stat : 31834 (plingeling) R 31820 31822 31604 0 -1 4202560 535989 1251 0 0 500622 89 0 0 25 0 9 0 655723908 4189081600 985344 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=31822/tid=31835] ppid=31820 vsize=4090900 CPUtime=5006.28 cores=0-7
/proc/31822/task/31835/stat : 31835 (plingeling) R 31820 31822 31604 0 -1 4202560 788452 1251 0 0 500467 161 0 0 25 0 9 0 655723908 4189081600 985344 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=31822/tid=31836] ppid=31820 vsize=4090900 CPUtime=5007.08 cores=0-7
/proc/31822/task/31836/stat : 31836 (plingeling) R 31820 31822 31604 0 -1 4202560 720957 1251 0 0 500595 113 0 0 25 0 9 0 655723908 4189081600 985344 33554432000 4194304 4417452 140734655339520 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 4090900

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5007.42 s]
/proc/loadavg: 8.07 8.08 8.01 9/352 31982
/proc/meminfo: memFree=19811128/32950928 swapFree=45937192/67111528
[pid=31822] ppid=31820 vsize=0 CPUtime=40000.8 cores=0-7
/proc/31822/stat : 31822 (plingeling) Z 31820 31822 31604 0 -1 4203548 5148026 1251 0 0 3999143 934 0 0 18 0 2 0 655723868 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 6 0 0 0
/proc/31822/statm: 0 0 0 0 0 0 0
[pid=31822/tid=31831] ppid=31820 vsize=0 CPUtime=4952.3 cores=0-7
/proc/31822/task/31831/stat : 31831 (plingeling) R 31820 31822 31604 0 -1 4203588 645369 1251 0 0 495095 135 0 0 25 0 2 0 655723907 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 40000.8
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5007.49
CPU time (s): 40000.9
CPU user time (s): 39991.4
CPU system time (s): 9.41757
CPU usage (%): 798.821
Max. virtual memory (cumulated for all children) (KiB): 4336612

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39991.4
system time used= 9.41757
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 5149277
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 578
involuntary context switches= 194242

runsolver used 15.3377 second user time and 39.417 second system time

The end

Launcher Data 

Begin job on node149 at 2011-05-15 08:34:32
IDJOB=3372725
IDBENCH=83135
IDSOLVER=1851
FILE ID=node149/3372725-1305441272
RUNJOBID= node149-1305437501-31618
PBS_JOBID= 13324778
Free space on /tmp= 73616 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/application/fuhs/slp-synthesis-AES/slp-synthesis-aes-bottom21.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3372725-1305441272/watcher-3372725-1305441272 -o /tmp/evaluation-result-3372725-1305441272/solver-3372725-1305441272 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3372725-1305441272.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= e6407843d1fbd00b68d4ad600a25dfb2
RANDOM SEED=804254721

node149.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.73
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      23759464 kB
Buffers:        318620 kB
Cached:        4237736 kB
SwapCached:      19984 kB
Active:        6357776 kB
Inactive:      2538260 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      23759464 kB
SwapTotal:    67111528 kB
SwapFree:     45937192 kB
Dirty:            3108 kB
Writeback:           0 kB
AnonPages:     4332512 kB
Mapped:          13636 kB
Slab:           156776 kB
PageTables:      71936 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 80188784 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73616 MiB
End job on node149 at 2011-05-15 09:58:01