Trace number 3309801

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 40001 5004.3

General information on the benchmark

Nameapplication/fuhs/slp-synthesis-AES/
slp-synthesis-aes-top23.cnf
MD5SUM6ee28191d649bfeef8c7d8c569a3cd59
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark640.829
Satisfiable
(Un)Satisfiability was proved
Number of variables60823
Number of clauses193450
Sum of the clauses size472824
Maximum clause length2414
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 2117689
Number of clauses of size 374473
Number of clauses of size 4275
Number of clauses of size 521
Number of clauses of size over 5991

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3309801-1304009020/watcher-3309801-1304009020 -o /tmp/evaluation-result-3309801-1304009020/solver-3309801-1304009020 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3309801-1304009020.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.45 7.94 7.96 2/170 16325
/proc/meminfo: memFree=25527436/32951124 swapFree=67051468/67111528
[pid=16325] ppid=16323 vsize=16644 CPUtime=0 cores=0-7
/proc/16325/stat : 16325 (plingeling) R 16323 16325 15797 0 -1 4202496 2176 1260 0 0 0 0 0 0 25 0 1 0 512504792 17043456 2105 33554432000 4194304 4417332 140734624308992 18446744073709551615 218370651229 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/16325/statm: 4161 2117 140 55 0 2047 0

[startup+0.0210841 s]
/proc/loadavg: 7.45 7.94 7.96 2/170 16325
/proc/meminfo: memFree=25527436/32951124 swapFree=67051468/67111528
[pid=16325] ppid=16323 vsize=20876 CPUtime=0.01 cores=0-7
/proc/16325/stat : 16325 (plingeling) R 16323 16325 15797 0 -1 4202496 3313 1260 0 0 1 0 0 0 25 0 1 0 512504792 21377024 3242 33554432000 4194304 4417332 140734624308992 18446744073709551615 4264942 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/16325/statm: 5219 3242 147 55 0 3105 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 20876

[startup+0.101069 s]
/proc/loadavg: 7.45 7.94 7.96 2/170 16325
/proc/meminfo: memFree=25527436/32951124 swapFree=67051468/67111528
[pid=16325] ppid=16323 vsize=34800 CPUtime=0.08 cores=0-7
/proc/16325/stat : 16325 (plingeling) R 16323 16325 15797 0 -1 4202496 6779 1260 0 0 8 0 0 0 25 0 1 0 512504792 35635200 6708 33554432000 4194304 4417332 140734624308992 18446744073709551615 218371015082 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/16325/statm: 8828 6726 148 55 0 6714 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 34800

[startup+0.301032 s]
/proc/loadavg: 7.45 7.94 7.96 2/170 16325
/proc/meminfo: memFree=25527436/32951124 swapFree=67051468/67111528
[pid=16325] ppid=16323 vsize=65068 CPUtime=0.29 cores=0-7
/proc/16325/stat : 16325 (plingeling) R 16323 16325 15797 0 -1 4202496 14362 1260 0 0 28 1 0 0 25 0 1 0 512504792 66629632 14291 33554432000 4194304 4417332 140734624308992 18446744073709551615 4209584 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/16325/statm: 16267 14291 148 55 0 14153 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 65068

[startup+0.70093 s]
/proc/loadavg: 7.45 7.94 7.96 2/170 16325
/proc/meminfo: memFree=25527436/32951124 swapFree=67051468/67111528
[pid=16325] ppid=16323 vsize=295784 CPUtime=2.14 cores=0-7
/proc/16325/stat : 16325 (plingeling) S 16323 16325 15797 0 -1 4202496 33340 1260 0 0 211 3 0 0 25 0 9 0 512504792 302882816 24053 33554432000 4194304 4417332 140734624308992 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16325/statm: 73946 24053 173 55 0 71832 0
Current children cumulated CPU time (s) 2.14
Current children cumulated vsize (KiB) 295784

[startup+1.50079 s]
/proc/loadavg: 7.45 7.94 7.96 9/179 16339
/proc/meminfo: memFree=25430776/32951124 swapFree=67051468/67111528
[pid=16325] ppid=16323 vsize=292508 CPUtime=8.53 cores=0-7
/proc/16325/stat : 16325 (plingeling) S 16323 16325 15797 0 -1 4202496 48595 1260 0 0 848 5 0 0 25 0 9 0 512504792 299528192 25543 33554432000 4194304 4417332 140734624308992 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16325/statm: 73127 25543 175 55 0 71013 0
[pid=16325/tid=16332] ppid=16323 vsize=292508 CPUtime=1.01 cores=0-7
/proc/16325/task/16332/stat : 16332 (plingeling) R 16323 16325 15797 0 -1 4202560 3165 1260 0 0 101 0 0 0 19 0 9 0 512504840 299528192 25543 33554432000 4194304 4417332 140734624308992 18446744073709551615 4291462 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16325/tid=16333] ppid=16323 vsize=292508 CPUtime=1 cores=0-7
/proc/16325/task/16333/stat : 16333 (plingeling) R 16323 16325 15797 0 -1 4202560 4822 1260 0 0 100 0 0 0 25 0 9 0 512504840 299528192 25543 33554432000 4194304 4417332 140734624308992 18446744073709551615 4267685 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16325/tid=16334] ppid=16323 vsize=292508 CPUtime=1.02 cores=0-7
/proc/16325/task/16334/stat : 16334 (plingeling) R 16323 16325 15797 0 -1 4202560 1610 1260 0 0 102 0 0 0 25 0 9 0 512504840 299528192 25543 33554432000 4194304 4417332 140734624308992 18446744073709551615 4286414 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16325/tid=16335] ppid=16323 vsize=292508 CPUtime=1.01 cores=0-7
/proc/16325/task/16335/stat : 16335 (plingeling) R 16323 16325 15797 0 -1 4202560 1179 1260 0 0 101 0 0 0 19 0 9 0 512504840 299528192 25543 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16325/tid=16336] ppid=16323 vsize=292508 CPUtime=1.01 cores=0-7
/proc/16325/task/16336/stat : 16336 (plingeling) R 16323 16325 15797 0 -1 4202560 1600 1260 0 0 101 0 0 0 25 0 9 0 512504840 299528192 25543 33554432000 4194304 4417332 140734624308992 18446744073709551615 4285006 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16325/tid=16337] ppid=16323 vsize=292508 CPUtime=1 cores=0-7
/proc/16325/task/16337/stat : 16337 (plingeling) R 16323 16325 15797 0 -1 4202560 4491 1260 0 0 100 0 0 0 19 0 9 0 512504840 299528192 25543 33554432000 4194304 4417332 140734624308992 18446744073709551615 4306569 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16325/tid=16338] ppid=16323 vsize=292508 CPUtime=0.96 cores=0-7
/proc/16325/task/16338/stat : 16338 (plingeling) R 16323 16325 15797 0 -1 4202560 5341 1260 0 0 95 1 0 0 25 0 9 0 512504840 299528192 25543 33554432000 4194304 4417332 140734624308992 18446744073709551615 4284341 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16325/tid=16339] ppid=16323 vsize=292508 CPUtime=1 cores=0-7
/proc/16325/task/16339/stat : 16339 (plingeling) R 16323 16325 15797 0 -1 4202560 7912 1260 0 0 100 0 0 0 25 0 9 0 512504840 299528192 25543 33554432000 4194304 4417332 140734624308992 18446744073709551615 4284341 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 8.53
Current children cumulated vsize (KiB) 292508

[startup+3.1005 s]
/proc/loadavg: 7.45 7.94 7.96 9/179 16339
/proc/meminfo: memFree=25425692/32951124 swapFree=67051468/67111528
[pid=16325] ppid=16323 vsize=300832 CPUtime=21.32 cores=0-7
/proc/16325/stat : 16325 (plingeling) S 16323 16325 15797 0 -1 4202496 60232 1260 0 0 2125 7 0 0 25 0 9 0 512504792 308051968 23543 33554432000 4194304 4417332 140734624308992 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16325/statm: 75208 23543 181 55 0 73094 0
[pid=16325/tid=16332] ppid=16323 vsize=300832 CPUtime=2.6 cores=0-7
/proc/16325/task/16332/stat : 16332 (plingeling) R 16323 16325 15797 0 -1 4202560 3821 1260 0 0 260 0 0 0 22 0 9 0 512504840 308051968 23543 33554432000 4194304 4417332 140734624308992 18446744073709551615 4346425 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16325/tid=16333] ppid=16323 vsize=300832 CPUtime=2.6 cores=0-7
/proc/16325/task/16333/stat : 16333 (plingeling) R 16323 16325 15797 0 -1 4202560 6724 1260 0 0 260 0 0 0 25 0 9 0 512504840 308051968 23543 33554432000 4194304 4417332 140734624308992 18446744073709551615 4330404 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16325/tid=16334] ppid=16323 vsize=300832 CPUtime=2.61 cores=0-7
/proc/16325/task/16334/stat : 16334 (plingeling) R 16323 16325 15797 0 -1 4202560 2767 1260 0 0 261 0 0 0 25 0 9 0 512504840 308051968 23543 33554432000 4194304 4417332 140734624308992 18446744073709551615 4258451 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16325/tid=16335] ppid=16323 vsize=300832 CPUtime=2.61 cores=0-7
/proc/16325/task/16335/stat : 16335 (plingeling) R 16323 16325 15797 0 -1 4202560 1179 1260 0 0 261 0 0 0 22 0 9 0 512504840 308051968 23543 33554432000 4194304 4417332 140734624308992 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16325/tid=16336] ppid=16323 vsize=300832 CPUtime=2.61 cores=0-7
/proc/16325/task/16336/stat : 16336 (plingeling) R 16323 16325 15797 0 -1 4202560 3642 1260 0 0 261 0 0 0 25 0 9 0 512504840 308051968 23543 33554432000 4194304 4417332 140734624308992 18446744073709551615 4339873 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16325/tid=16337] ppid=16323 vsize=300832 CPUtime=2.6 cores=0-7
/proc/16325/task/16337/stat : 16337 (plingeling) R 16323 16325 15797 0 -1 4202560 6821 1260 0 0 260 0 0 0 22 0 9 0 512504840 308051968 23543 33554432000 4194304 4417332 140734624308992 18446744073709551615 4346399 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16325/tid=16338] ppid=16323 vsize=300832 CPUtime=2.55 cores=0-7
/proc/16325/task/16338/stat : 16338 (plingeling) R 16323 16325 15797 0 -1 4202560 5650 1260 0 0 254 1 0 0 25 0 9 0 512504840 308051968 23543 33554432000 4194304 4417332 140734624308992 18446744073709551615 4346401 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16325/tid=16339] ppid=16323 vsize=300832 CPUtime=2.6 cores=0-7
/proc/16325/task/16339/stat : 16339 (plingeling) R 16323 16325 15797 0 -1 4202560 11153 1260 0 0 259 1 0 0 25 0 9 0 512504840 308051968 23543 33554432000 4194304 4417332 140734624308992 18446744073709551615 4330365 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 21.32
Current children cumulated vsize (KiB) 300832

[startup+6.30093 s]
/proc/loadavg: 7.50 7.94 7.96 9/179 16339
/proc/meminfo: memFree=25382168/32951124 swapFree=67051468/67111528
[pid=16325] ppid=16323 vsize=348864 CPUtime=46.91 cores=0-7
/proc/16325/stat : 16325 (plingeling) S 16323 16325 15797 0 -1 4202496 74067 1260 0 0 4682 9 0 0 25 0 9 0 512504792 357236736 36898 33554432000 4194304 4417332 140734624308992 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16325/statm: 87216 36898 181 55 0 85102 0
[pid=16325/tid=16332] ppid=16323 vsize=348864 CPUtime=5.8 cores=0-7
/proc/16325/task/16332/stat : 16332 (plingeling) R 16323 16325 15797 0 -1 4202560 5998 1260 0 0 580 0 0 0 25 0 9 0 512504840 357236736 36898 33554432000 4194304 4417332 140734624308992 18446744073709551615 4250072 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16325/tid=16333] ppid=16323 vsize=348864 CPUtime=5.8 cores=0-7
/proc/16325/task/16333/stat : 16333 (plingeling) R 16323 16325 15797 0 -1 4202560 8604 1260 0 0 579 1 0 0 25 0 9 0 512504840 357236736 36898 33554432000 4194304 4417332 140734624308992 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16325/tid=16334] ppid=16323 vsize=348864 CPUtime=5.81 cores=0-7
/proc/16325/task/16334/stat : 16334 (plingeling) R 16323 16325 15797 0 -1 4202560 3791 1260 0 0 581 0 0 0 25 0 9 0 512504840 357236736 36898 33554432000 4194304 4417332 140734624308992 18446744073709551615 4336643 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16325/tid=16335] ppid=16323 vsize=348864 CPUtime=5.82 cores=0-7
/proc/16325/task/16335/stat : 16335 (plingeling) R 16323 16325 15797 0 -1 4202560 1179 1260 0 0 582 0 0 0 25 0 9 0 512504840 357236736 36898 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364475 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16325/tid=16336] ppid=16323 vsize=348864 CPUtime=5.8 cores=0-7
/proc/16325/task/16336/stat : 16336 (plingeling) R 16323 16325 15797 0 -1 4202560 5874 1260 0 0 580 0 0 0 25 0 9 0 512504840 357236736 36898 33554432000 4194304 4417332 140734624308992 18446744073709551615 4253179 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16325/tid=16337] ppid=16323 vsize=348864 CPUtime=5.8 cores=0-7
/proc/16325/task/16337/stat : 16337 (plingeling) R 16323 16325 15797 0 -1 4202560 8961 1260 0 0 579 1 0 0 25 0 9 0 512504840 357236736 36898 33554432000 4194304 4417332 140734624308992 18446744073709551615 4250080 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16325/tid=16338] ppid=16323 vsize=348864 CPUtime=5.74 cores=0-7
/proc/16325/task/16338/stat : 16338 (plingeling) R 16323 16325 15797 0 -1 4202560 6804 1260 0 0 573 1 0 0 25 0 9 0 512504840 357236736 36898 33554432000 4194304 4417332 140734624308992 18446744073709551615 4378059 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16325/tid=16339] ppid=16323 vsize=348864 CPUtime=5.8 cores=0-7
/proc/16325/task/16339/stat : 16339 (plingeling) R 16323 16325 15797 0 -1 4202560 14381 1260 0 0 579 1 0 0 25 0 9 0 512504840 357236736 36898 33554432000 4194304 4417332 140734624308992 18446744073709551615 4253179 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 46.91
Current children cumulated vsize (KiB) 348864

[startup+12.7008 s]
/proc/loadavg: 7.54 7.94 7.96 9/179 16339
/proc/meminfo: memFree=25390368/32951124 swapFree=67051468/67111528
[pid=16325] ppid=16323 vsize=337348 CPUtime=98.07 cores=0-7
/proc/16325/stat : 16325 (plingeling) S 16323 16325 15797 0 -1 4202496 76697 1260 0 0 9798 9 0 0 25 0 9 0 512504792 345444352 33731 33554432000 4194304 4417332 140734624308992 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16325/statm: 84337 33731 181 55 0 82223 0
[pid=16325/tid=16332] ppid=16323 vsize=337348 CPUtime=12.2 cores=0-7
/proc/16325/task/16332/stat : 16332 (plingeling) R 16323 16325 15797 0 -1 4202560 5998 1260 0 0 1220 0 0 0 25 0 9 0 512504840 345444352 33731 33554432000 4194304 4417332 140734624308992 18446744073709551615 4232721 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16325/tid=16333] ppid=16323 vsize=337348 CPUtime=12.2 cores=0-7
/proc/16325/task/16333/stat : 16333 (plingeling) R 16323 16325 15797 0 -1 4202560 9639 1260 0 0 1219 1 0 0 25 0 9 0 512504840 345444352 33731 33554432000 4194304 4417332 140734624308992 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16325/tid=16334] ppid=16323 vsize=337348 CPUtime=12.21 cores=0-7
/proc/16325/task/16334/stat : 16334 (plingeling) R 16323 16325 15797 0 -1 4202560 3791 1260 0 0 1221 0 0 0 25 0 9 0 512504840 345444352 33731 33554432000 4194304 4417332 140734624308992 18446744073709551615 4377305 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16325/tid=16335] ppid=16323 vsize=337348 CPUtime=12.22 cores=0-7
/proc/16325/task/16335/stat : 16335 (plingeling) R 16323 16325 15797 0 -1 4202560 1494 1260 0 0 1222 0 0 0 25 0 9 0 512504840 345444352 33731 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364404 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16325/tid=16336] ppid=16323 vsize=337348 CPUtime=12.2 cores=0-7
/proc/16325/task/16336/stat : 16336 (plingeling) R 16323 16325 15797 0 -1 4202560 6899 1260 0 0 1220 0 0 0 25 0 9 0 512504840 345444352 33731 33554432000 4194304 4417332 140734624308992 18446744073709551615 4232654 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16325/tid=16337] ppid=16323 vsize=337348 CPUtime=12.2 cores=0-7
/proc/16325/task/16337/stat : 16337 (plingeling) R 16323 16325 15797 0 -1 4202560 8971 1260 0 0 1219 1 0 0 25 0 9 0 512504840 345444352 33731 33554432000 4194304 4417332 140734624308992 18446744073709551615 4254843 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16325/tid=16338] ppid=16323 vsize=337348 CPUtime=12.1 cores=0-7
/proc/16325/task/16338/stat : 16338 (plingeling) R 16323 16325 15797 0 -1 4202560 6825 1260 0 0 1209 1 0 0 25 0 9 0 512504840 345444352 33731 33554432000 4194304 4417332 140734624308992 18446744073709551615 4250017 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16325/tid=16339] ppid=16323 vsize=337348 CPUtime=12.2 cores=0-7
/proc/16325/task/16339/stat : 16339 (plingeling) R 16323 16325 15797 0 -1 4202560 14605 1260 0 0 1219 1 0 0 25 0 9 0 512504840 345444352 33731 33554432000 4194304 4417332 140734624308992 18446744073709551615 4258418 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 98.07
Current children cumulated vsize (KiB) 337348

[startup+25.5005 s]
/proc/loadavg: 7.64 7.94 7.96 9/179 16339
/proc/meminfo: memFree=25386316/32951124 swapFree=67051468/67111528
[pid=16325] ppid=16323 vsize=347168 CPUtime=200.4 cores=0-7
/proc/16325/stat : 16325 (plingeling) S 16323 16325 15797 0 -1 4202496 80328 1260 0 0 20030 10 0 0 25 0 9 0 512504792 355500032 35974 33554432000 4194304 4417332 140734624308992 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16325/statm: 86792 35974 181 55 0 84678 0
[pid=16325/tid=16332] ppid=16323 vsize=347168 CPUtime=25.01 cores=0-7
/proc/16325/task/16332/stat : 16332 (plingeling) R 16323 16325 15797 0 -1 4202560 7042 1260 0 0 2500 1 0 0 25 0 9 0 512504840 355500032 35974 33554432000 4194304 4417332 140734624308992 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16325/tid=16333] ppid=16323 vsize=347168 CPUtime=25 cores=0-7
/proc/16325/task/16333/stat : 16333 (plingeling) R 16323 16325 15797 0 -1 4202560 9648 1260 0 0 2499 1 0 0 25 0 9 0 512504840 355500032 35974 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364565 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16325/tid=16334] ppid=16323 vsize=347168 CPUtime=25.01 cores=0-7
/proc/16325/task/16334/stat : 16334 (plingeling) R 16323 16325 15797 0 -1 4202560 3797 1260 0 0 2501 0 0 0 25 0 9 0 512504840 355500032 35974 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364639 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16325/tid=16335] ppid=16323 vsize=347168 CPUtime=25.01 cores=0-7
/proc/16325/task/16335/stat : 16335 (plingeling) R 16323 16325 15797 0 -1 4202560 2806 1260 0 0 2501 0 0 0 25 0 9 0 512504840 355500032 35974 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364375 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16325/tid=16336] ppid=16323 vsize=347168 CPUtime=25 cores=0-7
/proc/16325/task/16336/stat : 16336 (plingeling) R 16323 16325 15797 0 -1 4202560 7078 1260 0 0 2500 0 0 0 25 0 9 0 512504840 355500032 35974 33554432000 4194304 4417332 140734624308992 18446744073709551615 4237002 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16325/tid=16337] ppid=16323 vsize=347168 CPUtime=25 cores=0-7
/proc/16325/task/16337/stat : 16337 (plingeling) R 16323 16325 15797 0 -1 4202560 10012 1260 0 0 2499 1 0 0 25 0 9 0 512504840 355500032 35974 33554432000 4194304 4417332 140734624308992 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16325/tid=16338] ppid=16323 vsize=347168 CPUtime=24.83 cores=0-7
/proc/16325/task/16338/stat : 16338 (plingeling) R 16323 16325 15797 0 -1 4202560 6846 1260 0 0 2482 1 0 0 25 0 9 0 512504840 355500032 35974 33554432000 4194304 4417332 140734624308992 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16325/tid=16339] ppid=16323 vsize=347168 CPUtime=25 cores=0-7
/proc/16325/task/16339/stat : 16339 (plingeling) R 16323 16325 15797 0 -1 4202560 14624 1260 0 0 2499 1 0 0 25 0 9 0 512504840 355500032 35974 33554432000 4194304 4417332 140734624308992 18446744073709551615 4239395 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 200.4
Current children cumulated vsize (KiB) 347168

[startup+51.101 s]
/proc/loadavg: 7.76 7.94 7.96 9/179 16340
/proc/meminfo: memFree=25333000/32951124 swapFree=67051468/67111528
[pid=16325] ppid=16323 vsize=399668 CPUtime=405.08 cores=0-7
/proc/16325/stat : 16325 (plingeling) S 16323 16325 15797 0 -1 4202496 116584 1260 0 0 40493 15 0 0 25 0 9 0 512504792 409260032 49099 33554432000 4194304 4417332 140734624308992 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16325/statm: 99917 49099 181 55 0 97803 0
[pid=16325/tid=16332] ppid=16323 vsize=399668 CPUtime=50.61 cores=0-7
/proc/16325/task/16332/stat : 16332 (plingeling) R 16323 16325 15797 0 -1 4202560 12683 1260 0 0 5059 2 0 0 25 0 9 0 512504840 409260032 49099 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364919 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16325/tid=16333] ppid=16323 vsize=399668 CPUtime=50.6 cores=0-7
/proc/16325/task/16333/stat : 16333 (plingeling) R 16323 16325 15797 0 -1 4202560 16514 1260 0 0 5058 2 0 0 25 0 9 0 512504840 409260032 49099 33554432000 4194304 4417332 140734624308992 18446744073709551615 4310199 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16325/tid=16334] ppid=16323 vsize=399668 CPUtime=50.61 cores=0-7

################
# More data... #
################


[startup+4722.31 s]
/proc/loadavg: 8.00 8.00 8.00 9/179 16486
/proc/meminfo: memFree=22065000/32951124 swapFree=67051468/67111528
[pid=16325] ppid=16323 vsize=3652208 CPUtime=37748.7 cores=0-7
/proc/16325/stat : 16325 (plingeling) S 16323 16325 15797 0 -1 4202496 6396815 1260 0 0 3773652 1219 0 0 25 0 9 0 512504792 3739860992 864264 33554432000 4194304 4417332 140734624308992 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16325/statm: 913052 864264 181 55 0 910938 0
[pid=16325/tid=16332] ppid=16323 vsize=3652208 CPUtime=4721.9 cores=0-7
/proc/16325/task/16332/stat : 16332 (plingeling) R 16323 16325 15797 0 -1 4202560 709380 1260 0 0 472059 131 0 0 25 0 9 0 512504840 3739860992 864264 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364463 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16325/tid=16333] ppid=16323 vsize=3652208 CPUtime=4721.43 cores=0-7
/proc/16325/task/16333/stat : 16333 (plingeling) R 16323 16325 15797 0 -1 4202560 902486 1260 0 0 471958 185 0 0 25 0 9 0 512504840 3739860992 864264 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364736 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16325/tid=16334] ppid=16323 vsize=3652208 CPUtime=4721.93 cores=0-7
/proc/16325/task/16334/stat : 16334 (plingeling) R 16323 16325 15797 0 -1 4202560 841983 1260 0 0 472017 176 0 0 25 0 9 0 512504840 3739860992 864264 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16325/tid=16335] ppid=16323 vsize=3652208 CPUtime=4721.1 cores=0-7
/proc/16325/task/16335/stat : 16335 (plingeling) R 16323 16325 15797 0 -1 4202560 587165 1260 0 0 472018 92 0 0 25 0 9 0 512504840 3739860992 864264 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16325/tid=16336] ppid=16323 vsize=3652208 CPUtime=4721.93 cores=0-7
/proc/16325/task/16336/stat : 16336 (plingeling) R 16323 16325 15797 0 -1 4202560 878913 1260 0 0 472019 174 0 0 25 0 9 0 512504840 3739860992 864264 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16325/tid=16337] ppid=16323 vsize=3652208 CPUtime=4721.89 cores=0-7
/proc/16325/task/16337/stat : 16337 (plingeling) R 16323 16325 15797 0 -1 4202560 760080 1260 0 0 472036 153 0 0 25 0 9 0 512504840 3739860992 864264 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364942 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16325/tid=16338] ppid=16323 vsize=3652208 CPUtime=4696.2 cores=0-7
/proc/16325/task/16338/stat : 16338 (plingeling) R 16323 16325 15797 0 -1 4202560 826943 1260 0 0 469459 161 0 0 25 0 9 0 512504840 3739860992 864264 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364482 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16325/tid=16339] ppid=16323 vsize=3652208 CPUtime=4721.77 cores=0-7
/proc/16325/task/16339/stat : 16339 (plingeling) R 16323 16325 15797 0 -1 4202560 871390 1260 0 0 472037 140 0 0 25 0 9 0 512504840 3739860992 864264 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 37748.7
Current children cumulated vsize (KiB) 3652208

[startup+4782.3 s]
/proc/loadavg: 8.08 8.02 8.01 9/179 16487
/proc/meminfo: memFree=22028624/32951124 swapFree=67051468/67111528
[pid=16325] ppid=16323 vsize=3689160 CPUtime=38228.4 cores=0-7
/proc/16325/stat : 16325 (plingeling) S 16323 16325 15797 0 -1 4202496 6406054 1260 0 0 3821615 1220 0 0 25 0 9 0 512504792 3777699840 873502 33554432000 4194304 4417332 140734624308992 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16325/statm: 922290 873502 181 55 0 920176 0
[pid=16325/tid=16332] ppid=16323 vsize=3689160 CPUtime=4781.89 cores=0-7
/proc/16325/task/16332/stat : 16332 (plingeling) R 16323 16325 15797 0 -1 4202560 711437 1260 0 0 478058 131 0 0 25 0 9 0 512504840 3777699840 873502 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364683 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16325/tid=16333] ppid=16323 vsize=3689160 CPUtime=4781.42 cores=0-7
/proc/16325/task/16333/stat : 16333 (plingeling) R 16323 16325 15797 0 -1 4202560 902486 1260 0 0 477957 185 0 0 25 0 9 0 512504840 3777699840 873502 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16325/tid=16334] ppid=16323 vsize=3689160 CPUtime=4781.93 cores=0-7
/proc/16325/task/16334/stat : 16334 (plingeling) R 16323 16325 15797 0 -1 4202560 841983 1260 0 0 478017 176 0 0 25 0 9 0 512504840 3777699840 873502 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16325/tid=16335] ppid=16323 vsize=3689160 CPUtime=4781.1 cores=0-7
/proc/16325/task/16335/stat : 16335 (plingeling) R 16323 16325 15797 0 -1 4202560 594340 1260 0 0 478017 93 0 0 25 0 9 0 512504840 3777699840 873502 33554432000 4194304 4417332 140734624308992 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16325/tid=16336] ppid=16323 vsize=3689160 CPUtime=4781.93 cores=0-7
/proc/16325/task/16336/stat : 16336 (plingeling) R 16323 16325 15797 0 -1 4202560 878913 1260 0 0 478019 174 0 0 25 0 9 0 512504840 3777699840 873502 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364544 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16325/tid=16337] ppid=16323 vsize=3689160 CPUtime=4781.89 cores=0-7
/proc/16325/task/16337/stat : 16337 (plingeling) R 16323 16325 15797 0 -1 4202560 760080 1260 0 0 478036 153 0 0 25 0 9 0 512504840 3777699840 873502 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16325/tid=16338] ppid=16323 vsize=3689160 CPUtime=4755.87 cores=0-7
/proc/16325/task/16338/stat : 16338 (plingeling) R 16323 16325 15797 0 -1 4202560 826950 1260 0 0 475426 161 0 0 25 0 9 0 512504840 3777699840 873502 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364973 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16325/tid=16339] ppid=16323 vsize=3689160 CPUtime=4781.77 cores=0-7
/proc/16325/task/16339/stat : 16339 (plingeling) R 16323 16325 15797 0 -1 4202560 871390 1260 0 0 478037 140 0 0 25 0 9 0 512504840 3777699840 873502 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 38228.3
Current children cumulated vsize (KiB) 3689160

[startup+4842.3 s]
/proc/loadavg: 8.27 8.09 8.03 9/179 16492
/proc/meminfo: memFree=21919968/32951124 swapFree=67051468/67111528
[pid=16325] ppid=16323 vsize=3797264 CPUtime=38707.2 cores=0-7
/proc/16325/stat : 16325 (plingeling) S 16323 16325 15797 0 -1 4202496 6446248 1260 0 0 3869488 1231 0 0 25 0 9 0 512504792 3888398336 900528 33554432000 4194304 4417332 140734624308992 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16325/statm: 949316 900528 181 55 0 947202 0
[pid=16325/tid=16332] ppid=16323 vsize=3797264 CPUtime=4841.89 cores=0-7
/proc/16325/task/16332/stat : 16332 (plingeling) R 16323 16325 15797 0 -1 4202560 711437 1260 0 0 484058 131 0 0 25 0 9 0 512504840 3888398336 900528 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364680 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16325/tid=16333] ppid=16323 vsize=3797264 CPUtime=4841.42 cores=0-7
/proc/16325/task/16333/stat : 16333 (plingeling) R 16323 16325 15797 0 -1 4202560 935254 1260 0 0 483946 196 0 0 25 0 9 0 512504840 3888398336 900528 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364808 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16325/tid=16334] ppid=16323 vsize=3797264 CPUtime=4841.93 cores=0-7
/proc/16325/task/16334/stat : 16334 (plingeling) R 16323 16325 15797 0 -1 4202560 841983 1260 0 0 484017 176 0 0 25 0 9 0 512504840 3888398336 900528 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364957 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16325/tid=16335] ppid=16323 vsize=3797264 CPUtime=4840.27 cores=0-7
/proc/16325/task/16335/stat : 16335 (plingeling) R 16323 16325 15797 0 -1 4202560 601766 1260 0 0 483933 94 0 0 25 0 9 0 512504840 3888398336 900528 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16325/tid=16336] ppid=16323 vsize=3797264 CPUtime=4841.93 cores=0-7
/proc/16325/task/16336/stat : 16336 (plingeling) R 16323 16325 15797 0 -1 4202560 878913 1260 0 0 484019 174 0 0 25 0 9 0 512504840 3888398336 900528 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16325/tid=16337] ppid=16323 vsize=3797264 CPUtime=4841.89 cores=0-7
/proc/16325/task/16337/stat : 16337 (plingeling) R 16323 16325 15797 0 -1 4202560 760080 1260 0 0 484036 153 0 0 25 0 9 0 512504840 3888398336 900528 33554432000 4194304 4417332 140734624308992 18446744073709551615 4237189 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16325/tid=16338] ppid=16323 vsize=3797264 CPUtime=4815.54 cores=0-7
/proc/16325/task/16338/stat : 16338 (plingeling) R 16323 16325 15797 0 -1 4202560 826950 1260 0 0 481393 161 0 0 25 0 9 0 512504840 3888398336 900528 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16325/tid=16339] ppid=16323 vsize=3797264 CPUtime=4841.77 cores=0-7
/proc/16325/task/16339/stat : 16339 (plingeling) R 16323 16325 15797 0 -1 4202560 871390 1260 0 0 484037 140 0 0 25 0 9 0 512504840 3888398336 900528 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364742 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 38707.2
Current children cumulated vsize (KiB) 3797264

[startup+4902.3 s]
/proc/loadavg: 8.10 8.07 8.02 9/179 16493
/proc/meminfo: memFree=21606092/32951124 swapFree=67051468/67111528
[pid=16325] ppid=16323 vsize=4111576 CPUtime=39186.9 cores=0-7
/proc/16325/stat : 16325 (plingeling) S 16323 16325 15797 0 -1 4202496 6614170 1260 0 0 3917423 1265 0 0 25 0 9 0 512504792 4210253824 979106 33554432000 4194304 4417332 140734624308992 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16325/statm: 1027894 979106 181 55 0 1025780 0
[pid=16325/tid=16332] ppid=16323 vsize=4111576 CPUtime=4901.89 cores=0-7
/proc/16325/task/16332/stat : 16332 (plingeling) R 16323 16325 15797 0 -1 4202560 718385 1260 0 0 490057 132 0 0 25 0 9 0 512504840 4210253824 979106 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16325/tid=16333] ppid=16323 vsize=4111576 CPUtime=4901.42 cores=0-7
/proc/16325/task/16333/stat : 16333 (plingeling) R 16323 16325 15797 0 -1 4202560 935254 1260 0 0 489946 196 0 0 25 0 9 0 512504840 4210253824 979106 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16325/tid=16334] ppid=16323 vsize=4111576 CPUtime=4901.94 cores=0-7
/proc/16325/task/16334/stat : 16334 (plingeling) R 16323 16325 15797 0 -1 4202560 902568 1260 0 0 490001 193 0 0 25 0 9 0 512504840 4210253824 979106 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16325/tid=16335] ppid=16323 vsize=4111576 CPUtime=4900.28 cores=0-7
/proc/16325/task/16335/stat : 16335 (plingeling) R 16323 16325 15797 0 -1 4202560 607671 1260 0 0 489932 96 0 0 25 0 9 0 512504840 4210253824 979106 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16325/tid=16336] ppid=16323 vsize=4111576 CPUtime=4901.94 cores=0-7
/proc/16325/task/16336/stat : 16336 (plingeling) R 16323 16325 15797 0 -1 4202560 911681 1260 0 0 490016 178 0 0 25 0 9 0 512504840 4210253824 979106 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16325/tid=16337] ppid=16323 vsize=4111576 CPUtime=4901.89 cores=0-7
/proc/16325/task/16337/stat : 16337 (plingeling) R 16323 16325 15797 0 -1 4202560 760080 1260 0 0 490036 153 0 0 25 0 9 0 512504840 4210253824 979106 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16325/tid=16338] ppid=16323 vsize=4111576 CPUtime=4875.22 cores=0-7
/proc/16325/task/16338/stat : 16338 (plingeling) R 16323 16325 15797 0 -1 4202560 832838 1260 0 0 487359 163 0 0 25 0 9 0 512504840 4210253824 979106 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16325/tid=16339] ppid=16323 vsize=4111576 CPUtime=4901.77 cores=0-7
/proc/16325/task/16339/stat : 16339 (plingeling) R 16323 16325 15797 0 -1 4202560 927218 1260 0 0 490029 148 0 0 25 0 9 0 512504840 4210253824 979106 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 39186.9
Current children cumulated vsize (KiB) 4111576

[startup+4962.3 s]
/proc/loadavg: 8.03 8.05 8.01 9/179 16495
/proc/meminfo: memFree=21494936/32951124 swapFree=67051468/67111528
[pid=16325] ppid=16323 vsize=4222412 CPUtime=39666.6 cores=0-7
/proc/16325/stat : 16325 (plingeling) S 16323 16325 15797 0 -1 4202496 6674448 1260 0 0 3965376 1280 0 0 25 0 9 0 512504792 4323749888 1006815 33554432000 4194304 4417332 140734624308992 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16325/statm: 1055603 1006815 181 55 0 1053489 0
[pid=16325/tid=16332] ppid=16323 vsize=4222412 CPUtime=4961.89 cores=0-7
/proc/16325/task/16332/stat : 16332 (plingeling) R 16323 16325 15797 0 -1 4202560 718385 1260 0 0 496057 132 0 0 25 0 9 0 512504840 4323749888 1006815 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16325/tid=16333] ppid=16323 vsize=4222412 CPUtime=4961.42 cores=0-7
/proc/16325/task/16333/stat : 16333 (plingeling) R 16323 16325 15797 0 -1 4202560 935254 1260 0 0 495946 196 0 0 25 0 9 0 512504840 4323749888 1006815 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16325/tid=16334] ppid=16323 vsize=4222412 CPUtime=4961.94 cores=0-7
/proc/16325/task/16334/stat : 16334 (plingeling) R 16323 16325 15797 0 -1 4202560 902568 1260 0 0 496001 193 0 0 25 0 9 0 512504840 4323749888 1006815 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364742 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16325/tid=16335] ppid=16323 vsize=4222412 CPUtime=4960.28 cores=0-7
/proc/16325/task/16335/stat : 16335 (plingeling) R 16323 16325 15797 0 -1 4202560 615510 1260 0 0 495931 97 0 0 25 0 9 0 512504840 4323749888 1006815 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16325/tid=16336] ppid=16323 vsize=4222412 CPUtime=4961.94 cores=0-7
/proc/16325/task/16336/stat : 16336 (plingeling) R 16323 16325 15797 0 -1 4202560 911681 1260 0 0 496016 178 0 0 25 0 9 0 512504840 4323749888 1006815 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16325/tid=16337] ppid=16323 vsize=4222412 CPUtime=4961.89 cores=0-7
/proc/16325/task/16337/stat : 16337 (plingeling) R 16323 16325 15797 0 -1 4202560 812519 1260 0 0 496022 167 0 0 25 0 9 0 512504840 4323749888 1006815 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16325/tid=16338] ppid=16323 vsize=4222412 CPUtime=4934.9 cores=0-7
/proc/16325/task/16338/stat : 16338 (plingeling) R 16323 16325 15797 0 -1 4202560 832838 1260 0 0 493327 163 0 0 25 0 9 0 512504840 4323749888 1006815 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16325/tid=16339] ppid=16323 vsize=4222412 CPUtime=4961.77 cores=0-7
/proc/16325/task/16339/stat : 16339 (plingeling) R 16323 16325 15797 0 -1 4202560 927218 1260 0 0 496029 148 0 0 25 0 9 0 512504840 4323749888 1006815 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364509 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 39666.6
Current children cumulated vsize (KiB) 4222412



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5004.1 s]
/proc/loadavg: 8.02 8.04 8.01 9/179 16496
/proc/meminfo: memFree=21353784/32951124 swapFree=67051468/67111528
[pid=16325] ppid=16323 vsize=4363296 CPUtime=40000.8 cores=0-7
/proc/16325/stat : 16325 (plingeling) S 16323 16325 15797 0 -1 4202496 6709669 1260 0 0 3998794 1283 0 0 25 0 9 0 512504792 4468015104 1042036 33554432000 4194304 4417332 140734624308992 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/16325/statm: 1090824 1042036 181 55 0 1088710 0
[pid=16325/tid=16332] ppid=16323 vsize=4363296 CPUtime=5003.69 cores=0-7
/proc/16325/task/16332/stat : 16332 (plingeling) R 16323 16325 15797 0 -1 4202560 718385 1260 0 0 500237 132 0 0 25 0 9 0 512504840 4468015104 1042036 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=16325/tid=16333] ppid=16323 vsize=4363296 CPUtime=5003.22 cores=0-7
/proc/16325/task/16333/stat : 16333 (plingeling) R 16323 16325 15797 0 -1 4202560 935254 1260 0 0 500126 196 0 0 25 0 9 0 512504840 4468015104 1042036 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16325/tid=16334] ppid=16323 vsize=4363296 CPUtime=5003.74 cores=0-7
/proc/16325/task/16334/stat : 16334 (plingeling) R 16323 16325 15797 0 -1 4202560 902568 1260 0 0 500181 193 0 0 25 0 9 0 512504840 4468015104 1042036 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16325/tid=16335] ppid=16323 vsize=4363296 CPUtime=5002.09 cores=0-7
/proc/16325/task/16335/stat : 16335 (plingeling) R 16323 16325 15797 0 -1 4202560 617963 1260 0 0 500112 97 0 0 25 0 9 0 512504840 4468015104 1042036 33554432000 4194304 4417332 140734624308992 18446744073709551615 4365894 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=16325/tid=16336] ppid=16323 vsize=4363296 CPUtime=5003.74 cores=0-7
/proc/16325/task/16336/stat : 16336 (plingeling) R 16323 16325 15797 0 -1 4202560 911681 1260 0 0 500196 178 0 0 25 0 9 0 512504840 4468015104 1042036 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=16325/tid=16337] ppid=16323 vsize=4363296 CPUtime=5003.7 cores=0-7
/proc/16325/task/16337/stat : 16337 (plingeling) R 16323 16325 15797 0 -1 4202560 812519 1260 0 0 500203 167 0 0 25 0 9 0 512504840 4468015104 1042036 33554432000 4194304 4417332 140734624308992 18446744073709551615 4364750 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16325/tid=16338] ppid=16323 vsize=4363296 CPUtime=4976.47 cores=0-7
/proc/16325/task/16338/stat : 16338 (plingeling) R 16323 16325 15797 0 -1 4202560 832838 1260 0 0 497484 163 0 0 25 0 9 0 512504840 4468015104 1042036 33554432000 4194304 4417332 140734624308992 18446744073709551615 4263067 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=16325/tid=16339] ppid=16323 vsize=4363296 CPUtime=5003.58 cores=0-7
/proc/16325/task/16339/stat : 16339 (plingeling) R 16323 16325 15797 0 -1 4202560 959986 1260 0 0 500206 152 0 0 25 0 9 0 512504840 4468015104 1042036 33554432000 4194304 4417332 140734624308992 18446744073709551615 4366025 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 40000.8
Current children cumulated vsize (KiB) 4363296

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5004.21 s]
/proc/loadavg: 8.02 8.04 8.01 9/179 16496
/proc/meminfo: memFree=21353784/32951124 swapFree=67051468/67111528
[pid=16325] ppid=16323 vsize=0 CPUtime=40000.9 cores=0-7
/proc/16325/stat : 16325 (plingeling) Z 16323 16325 15797 0 -1 4203548 6709669 1260 0 0 3998798 1293 0 0 18 0 2 0 512504792 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/16325/statm: 0 0 0 0 0 0 0
[pid=16325/tid=16338] ppid=16323 vsize=0 CPUtime=4976.57 cores=0-7
/proc/16325/task/16338/stat : 16338 (plingeling) R 16323 16325 15797 0 -1 4203588 832838 1260 0 0 497484 173 0 0 25 0 2 0 512504840 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40000.9
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5004.3
CPU time (s): 40001
CPU user time (s): 39988
CPU system time (s): 13.03
CPU usage (%): 799.333
Max. virtual memory (cumulated for all children) (KiB): 4363296

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39988
system time used= 13.03
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 6710929
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 495
involuntary context switches= 193248

runsolver used 7.51386 second user time and 19.716 second system time

The end

Launcher Data 

Begin job on node101 at 2011-04-28 18:43:40
IDJOB=3309801
IDBENCH=83114
IDSOLVER=1695
FILE ID=node101/3309801-1304009020
RUNJOBID= node101-1303997715-15811
PBS_JOBID= 13168787
Free space on /tmp= 72020 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT11/application/fuhs/slp-synthesis-AES/slp-synthesis-aes-top23.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3309801-1304009020/watcher-3309801-1304009020 -o /tmp/evaluation-result-3309801-1304009020/solver-3309801-1304009020 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3309801-1304009020.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 6ee28191d649bfeef8c7d8c569a3cd59
RANDOM SEED=368490675

node101.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      25527840 kB
Buffers:       1997380 kB
Cached:        4947120 kB
SwapCached:      30452 kB
Active:        2757020 kB
Inactive:      4220964 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      25527840 kB
SwapTotal:    67111528 kB
SwapFree:     67051468 kB
Dirty:            3488 kB
Writeback:           0 kB
AnonPages:       25044 kB
Mapped:           8816 kB
Slab:           378268 kB
PageTables:       4308 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   185956 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72020 MiB
End job on node101 at 2011-04-28 20:07:06