Trace number 3289161

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 4800.29 1201.94

General information on the benchmark

Nameapplication/fuhs/slp-synthesis-AES/
slp-synthesis-aes-top24.cnf
MD5SUM1f485d737caaaca49e2c467c83cbdcec
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark520.047
Satisfiable
(Un)Satisfiability was proved
Number of variables65983
Number of clauses209942
Sum of the clauses size513194
Maximum clause length2578
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 2127592
Number of clauses of size 380970
Number of clauses of size 4299
Number of clauses of size 522
Number of clauses of size over 51058

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3289161-1304080536/watcher-3289161-1304080536 -o /tmp/evaluation-result-3289161-1304080536/solver-3289161-1304080536 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3289161-1304080536.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.42 7.93 7.98 2/177 19636
/proc/meminfo: memFree=18405396/32951124 swapFree=67110396/67111528
[pid=19636] ppid=19634 vsize=17716 CPUtime=0 cores=0,2,4,6
/proc/19636/stat : 19636 (plingeling) R 19634 19636 19297 0 -1 4202496 2061 1250 0 0 0 0 0 0 22 0 1 0 598284290 18141184 1992 33554432000 4194304 4417332 140736279404656 18446744073709551615 224284619869 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/19636/statm: 4429 2003 140 55 0 2315 0

[startup+0.0846019 s]
/proc/loadavg: 7.42 7.93 7.98 2/177 19636
/proc/meminfo: memFree=18405396/32951124 swapFree=67110396/67111528
[pid=19636] ppid=19634 vsize=30840 CPUtime=0.07 cores=0,2,4,6
/proc/19636/stat : 19636 (plingeling) R 19634 19636 19297 0 -1 4202496 5817 1250 0 0 7 0 0 0 22 0 1 0 598284290 31580160 5731 33554432000 4194304 4417332 140736279404656 18446744073709551615 224284619869 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/19636/statm: 7966 5733 148 55 0 5852 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 30840

[startup+0.100602 s]
/proc/loadavg: 7.42 7.93 7.98 2/177 19636
/proc/meminfo: memFree=18405396/32951124 swapFree=67110396/67111528
[pid=19636] ppid=19634 vsize=33912 CPUtime=0.09 cores=0,2,4,6
/proc/19636/stat : 19636 (plingeling) R 19634 19636 19297 0 -1 4202496 6587 1250 0 0 9 0 0 0 22 0 1 0 598284290 34725888 6501 33554432000 4194304 4417332 140736279404656 18446744073709551615 224284925696 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/19636/statm: 8478 6501 148 55 0 6364 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 33912

[startup+0.300477 s]
/proc/loadavg: 7.42 7.93 7.98 2/177 19636
/proc/meminfo: memFree=18405396/32951124 swapFree=67110396/67111528
[pid=19636] ppid=19634 vsize=161092 CPUtime=0.41 cores=0,2,4,6
/proc/19636/stat : 19636 (plingeling) S 19634 19636 19297 0 -1 4202496 20560 1250 0 0 40 1 0 0 23 0 5 0 598284290 164958208 13414 33554432000 4194304 4417332 140736279404656 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/19636/statm: 40273 13414 167 55 0 38159 0
Current children cumulated CPU time (s) 0.41
Current children cumulated vsize (KiB) 161092

[startup+0.700413 s]
/proc/loadavg: 7.42 7.93 7.98 2/177 19636
/proc/meminfo: memFree=18405396/32951124 swapFree=67110396/67111528
[pid=19636] ppid=19634 vsize=162864 CPUtime=2.01 cores=0,2,4,6
/proc/19636/stat : 19636 (plingeling) S 19634 19636 19297 0 -1 4202496 26039 1250 0 0 199 2 0 0 23 0 5 0 598284290 166772736 13813 33554432000 4194304 4417332 140736279404656 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/19636/statm: 40716 13813 174 55 0 38602 0
Current children cumulated CPU time (s) 2.01
Current children cumulated vsize (KiB) 162864

[startup+1.50028 s]
/proc/loadavg: 7.42 7.93 7.98 10/186 19652
/proc/meminfo: memFree=18310464/32951124 swapFree=67110396/67111528
[pid=19636] ppid=19634 vsize=164344 CPUtime=5.21 cores=0,2,4,6
/proc/19636/stat : 19636 (plingeling) S 19634 19636 19297 0 -1 4202496 31257 1250 0 0 518 3 0 0 23 0 5 0 598284290 168288256 13393 33554432000 4194304 4417332 140736279404656 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/19636/statm: 41086 13393 174 55 0 38972 0
[pid=19636/tid=19649] ppid=19634 vsize=164344 CPUtime=1.23 cores=0,2,4,6
/proc/19636/task/19649/stat : 19649 (plingeling) R 19634 19636 19297 0 -1 4202560 3139 1250 0 0 123 0 0 0 25 0 5 0 598284315 168288256 13393 33554432000 4194304 4417332 140736279404656 18446744073709551615 4286414 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=19636/tid=19650] ppid=19634 vsize=164344 CPUtime=1.22 cores=0,2,4,6
/proc/19636/task/19650/stat : 19650 (plingeling) R 19634 19636 19297 0 -1 4202560 9005 1250 0 0 122 0 0 0 25 0 5 0 598284315 168288256 13393 33554432000 4194304 4417332 140736279404656 18446744073709551615 4282247 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=19636/tid=19651] ppid=19634 vsize=164344 CPUtime=1.23 cores=0,2,4,6
/proc/19636/task/19651/stat : 19651 (plingeling) R 19634 19636 19297 0 -1 4202560 3466 1250 0 0 123 0 0 0 19 0 5 0 598284315 168288256 13393 33554432000 4194304 4417332 140736279404656 18446744073709551615 4287304 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=19636/tid=19652] ppid=19634 vsize=164344 CPUtime=1.24 cores=0,2,4,6
/proc/19636/task/19652/stat : 19652 (plingeling) R 19634 19636 19297 0 -1 4202560 1345 1250 0 0 124 0 0 0 25 0 5 0 598284315 168288256 13393 33554432000 4194304 4417332 140736279404656 18446744073709551615 4366440 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 5.21
Current children cumulated vsize (KiB) 164344

[startup+3.10101 s]
/proc/loadavg: 7.47 7.93 7.98 9/186 19652
/proc/meminfo: memFree=18306500/32951124 swapFree=67110396/67111528
[pid=19636] ppid=19634 vsize=187460 CPUtime=11.61 cores=0,2,4,6
/proc/19636/stat : 19636 (plingeling) S 19634 19636 19297 0 -1 4202496 41106 1250 0 0 1157 4 0 0 23 0 5 0 598284290 191959040 19695 33554432000 4194304 4417332 140736279404656 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/19636/statm: 46865 19695 180 55 0 44751 0
[pid=19636/tid=19649] ppid=19634 vsize=187460 CPUtime=2.83 cores=0,2,4,6
/proc/19636/task/19649/stat : 19649 (plingeling) R 19634 19636 19297 0 -1 4202560 5870 1250 0 0 283 0 0 0 25 0 5 0 598284315 191959040 19695 33554432000 4194304 4417332 140736279404656 18446744073709551615 4257072 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=19636/tid=19650] ppid=19634 vsize=187460 CPUtime=2.82 cores=0,2,4,6
/proc/19636/task/19650/stat : 19650 (plingeling) R 19634 19636 19297 0 -1 4202560 13160 1250 0 0 281 1 0 0 25 0 5 0 598284315 191959040 19695 33554432000 4194304 4417332 140736279404656 18446744073709551615 4253152 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=19636/tid=19651] ppid=19634 vsize=187460 CPUtime=2.83 cores=0,2,4,6
/proc/19636/task/19651/stat : 19651 (plingeling) R 19634 19636 19297 0 -1 4202560 5405 1250 0 0 283 0 0 0 23 0 5 0 598284315 191959040 19695 33554432000 4194304 4417332 140736279404656 18446744073709551615 4337073 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=19636/tid=19652] ppid=19634 vsize=187460 CPUtime=2.84 cores=0,2,4,6
/proc/19636/task/19652/stat : 19652 (plingeling) R 19634 19636 19297 0 -1 4202560 2369 1250 0 0 284 0 0 0 25 0 5 0 598284315 191959040 19695 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364565 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 11.61
Current children cumulated vsize (KiB) 187460

[startup+6.30048 s]
/proc/loadavg: 7.47 7.93 7.98 9/186 19652
/proc/meminfo: memFree=18270224/32951124 swapFree=67110396/67111528
[pid=19636] ppid=19634 vsize=199504 CPUtime=24.38 cores=0,2,4,6
/proc/19636/stat : 19636 (plingeling) S 19634 19636 19297 0 -1 4202496 44413 1250 0 0 2434 4 0 0 23 0 5 0 598284290 204292096 22522 33554432000 4194304 4417332 140736279404656 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/19636/statm: 49876 22522 180 55 0 47762 0
[pid=19636/tid=19649] ppid=19634 vsize=199504 CPUtime=6.03 cores=0,2,4,6
/proc/19636/task/19649/stat : 19649 (plingeling) R 19634 19636 19297 0 -1 4202560 7119 1250 0 0 603 0 0 0 25 0 5 0 598284315 204292096 22522 33554432000 4194304 4417332 140736279404656 18446744073709551615 4272133 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=19636/tid=19650] ppid=19634 vsize=199504 CPUtime=6 cores=0,2,4,6
/proc/19636/task/19650/stat : 19650 (plingeling) R 19634 19636 19297 0 -1 4202560 15218 1250 0 0 599 1 0 0 25 0 5 0 598284315 204292096 22522 33554432000 4194304 4417332 140736279404656 18446744073709551615 4252252 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=19636/tid=19651] ppid=19634 vsize=199504 CPUtime=6.03 cores=0,2,4,6
/proc/19636/task/19651/stat : 19651 (plingeling) R 19634 19636 19297 0 -1 4202560 5405 1250 0 0 603 0 0 0 25 0 5 0 598284315 204292096 22522 33554432000 4194304 4417332 140736279404656 18446744073709551615 4258491 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=19636/tid=19652] ppid=19634 vsize=199504 CPUtime=6.03 cores=0,2,4,6
/proc/19636/task/19652/stat : 19652 (plingeling) R 19634 19636 19297 0 -1 4202560 2369 1250 0 0 603 0 0 0 25 0 5 0 598284315 204292096 22522 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 24.38
Current children cumulated vsize (KiB) 199504

[startup+12.7004 s]
/proc/loadavg: 7.55 7.93 7.98 10/186 19652
/proc/meminfo: memFree=18252612/32951124 swapFree=67110396/67111528
[pid=19636] ppid=19634 vsize=201844 CPUtime=49.95 cores=0,2,4,6
/proc/19636/stat : 19636 (plingeling) S 19634 19636 19297 0 -1 4202496 45411 1250 0 0 4990 5 0 0 23 0 5 0 598284290 206688256 23491 33554432000 4194304 4417332 140736279404656 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/19636/statm: 50461 23491 180 55 0 48347 0
[pid=19636/tid=19649] ppid=19634 vsize=201844 CPUtime=12.43 cores=0,2,4,6
/proc/19636/task/19649/stat : 19649 (plingeling) R 19634 19636 19297 0 -1 4202560 7119 1250 0 0 1243 0 0 0 25 0 5 0 598284315 206688256 23491 33554432000 4194304 4417332 140736279404656 18446744073709551615 4250017 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=19636/tid=19650] ppid=19634 vsize=201844 CPUtime=12.37 cores=0,2,4,6
/proc/19636/task/19650/stat : 19650 (plingeling) R 19634 19636 19297 0 -1 4202560 15602 1250 0 0 1236 1 0 0 25 0 5 0 598284315 206688256 23491 33554432000 4194304 4417332 140736279404656 18446744073709551615 4250017 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=19636/tid=19651] ppid=19634 vsize=201844 CPUtime=12.43 cores=0,2,4,6
/proc/19636/task/19651/stat : 19651 (plingeling) R 19634 19636 19297 0 -1 4202560 5405 1250 0 0 1243 0 0 0 25 0 5 0 598284315 206688256 23491 33554432000 4194304 4417332 140736279404656 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=19636/tid=19652] ppid=19634 vsize=201844 CPUtime=12.43 cores=0,2,4,6
/proc/19636/task/19652/stat : 19652 (plingeling) R 19634 19636 19297 0 -1 4202560 2983 1250 0 0 1243 0 0 0 25 0 5 0 598284315 206688256 23491 33554432000 4194304 4417332 140736279404656 18446744073709551615 4236761 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 49.95
Current children cumulated vsize (KiB) 201844

[startup+25.5003 s]
/proc/loadavg: 7.69 7.95 7.99 10/186 19652
/proc/meminfo: memFree=18277920/32951124 swapFree=67110396/67111528
[pid=19636] ppid=19634 vsize=193316 CPUtime=101.07 cores=0,2,4,6
/proc/19636/stat : 19636 (plingeling) S 19634 19636 19297 0 -1 4202496 47503 1250 0 0 10102 5 0 0 23 0 5 0 598284290 197955584 20455 33554432000 4194304 4417332 140736279404656 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/19636/statm: 48329 20455 180 55 0 46215 0
[pid=19636/tid=19649] ppid=19634 vsize=193316 CPUtime=25.23 cores=0,2,4,6
/proc/19636/task/19649/stat : 19649 (plingeling) R 19634 19636 19297 0 -1 4202560 7132 1250 0 0 2523 0 0 0 25 0 5 0 598284315 197955584 20455 33554432000 4194304 4417332 140736279404656 18446744073709551615 4356899 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=19636/tid=19650] ppid=19634 vsize=193316 CPUtime=25.1 cores=0,2,4,6
/proc/19636/task/19650/stat : 19650 (plingeling) R 19634 19636 19297 0 -1 4202560 17633 1250 0 0 2508 2 0 0 25 0 5 0 598284315 197955584 20455 33554432000 4194304 4417332 140736279404656 18446744073709551615 4237262 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=19636/tid=19651] ppid=19634 vsize=193316 CPUtime=25.23 cores=0,2,4,6
/proc/19636/task/19651/stat : 19651 (plingeling) R 19634 19636 19297 0 -1 4202560 5453 1250 0 0 2523 0 0 0 25 0 5 0 598284315 197955584 20455 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=19636/tid=19652] ppid=19634 vsize=193316 CPUtime=25.23 cores=0,2,4,6
/proc/19636/task/19652/stat : 19652 (plingeling) R 19634 19636 19297 0 -1 4202560 2983 1250 0 0 2523 0 0 0 25 0 5 0 598284315 197955584 20455 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 101.07
Current children cumulated vsize (KiB) 193316

[startup+51.101 s]
/proc/loadavg: 7.85 7.97 7.99 10/186 19652
/proc/meminfo: memFree=18242136/32951124 swapFree=67110396/67111528
[pid=19636] ppid=19634 vsize=199312 CPUtime=203.34 cores=0,2,4,6
/proc/19636/stat : 19636 (plingeling) S 19634 19636 19297 0 -1 4202496 59373 1250 0 0 20327 7 0 0 23 0 5 0 598284290 204095488 23277 33554432000 4194304 4417332 140736279404656 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/19636/statm: 49828 23277 180 55 0 47714 0
[pid=19636/tid=19649] ppid=19634 vsize=199312 CPUtime=50.84 cores=0,2,4,6
/proc/19636/task/19649/stat : 19649 (plingeling) R 19634 19636 19297 0 -1 4202560 9629 1250 0 0 5083 1 0 0 25 0 5 0 598284315 204095488 23277 33554432000 4194304 4417332 140736279404656 18446744073709551615 4236790 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=19636/tid=19650] ppid=19634 vsize=199312 CPUtime=50.55 cores=0,2,4,6
/proc/19636/task/19650/stat : 19650 (plingeling) R 19634 19636 19297 0 -1 4202560 24852 1250 0 0 5053 2 0 0 25 0 5 0 598284315 204095488 23277 33554432000 4194304 4417332 140736279404656 18446744073709551615 4250476 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=19636/tid=19651] ppid=19634 vsize=199312 CPUtime=50.83 cores=0,2,4,6
/proc/19636/task/19651/stat : 19651 (plingeling) R 19634 19636 19297 0 -1 4202560 6641 1250 0 0 5082 1 0 0 25 0 5 0 598284315 204095488 23277 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=19636/tid=19652] ppid=19634 vsize=199312 CPUtime=50.83 cores=0,2,4,6
/proc/19636/task/19652/stat : 19652 (plingeling) R 19634 19636 19297 0 -1 4202560 3949 1250 0 0 5083 0 0 0 25 0 5 0 598284315 204095488 23277 33554432000 4194304 4417332 140736279404656 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 203.34
Current children cumulated vsize (KiB) 199312

[startup+102.307 s]
/proc/loadavg: 8.00 7.99 8.00 10/186 19654
/proc/meminfo: memFree=18172304/32951124 swapFree=67110396/67111528
[pid=19636] ppid=19634 vsize=233816 CPUtime=407.88 cores=0,2,4,6
/proc/19636/stat : 19636 (plingeling) S 19634 19636 19297 0 -1 4202496 100668 1250 0 0 40775 13 0 0 23 0 5 0 598284290 239427584 33941 33554432000 4194304 4417332 140736279404656 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/19636/statm: 58454 33941 180 55 0 56340 0
[pid=19636/tid=19649] ppid=19634 vsize=233816 CPUtime=102.04 cores=0,2,4,6
/proc/19636/task/19649/stat : 19649 (plingeling) R 19634 19636 19297 0 -1 4202560 14597 1250 0 0 10203 1 0 0 25 0 5 0 598284315 239427584 33941 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=19636/tid=19650] ppid=19634 vsize=233816 CPUtime=101.49 cores=0,2,4,6
/proc/19636/task/19650/stat : 19650 (plingeling) R 19634 19636 19297 0 -1 4202560 44096 1250 0 0 10144 5 0 0 25 0 5 0 598284315 239427584 33941 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=19636/tid=19651] ppid=19634 vsize=233816 CPUtime=102.04 cores=0,2,4,6
/proc/19636/task/19651/stat : 19651 (plingeling) R 19634 19636 19297 0 -1 4202560 15499 1250 0 0 10202 2 0 0 25 0 5 0 598284315 239427584 33941 33554432000 4194304 4417332 140736279404656 18446744073709551615 4365960 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=19636/tid=19652] ppid=19634 vsize=233816 CPUtime=102.03 cores=0,2,4,6
/proc/19636/task/19652/stat : 19652 (plingeling) R 19634 19636 19297 0 -1 4202560 12174 1250 0 0 10201 2 0 0 25 0 5 0 598284315 239427584 33941 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364502 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 407.88
Current children cumulated vsize (KiB) 233816

[startup+162.301 s]
/proc/loadavg: 8.05 8.00 8.00 10/186 19656
/proc/meminfo: memFree=18141772/32951124 swapFree=67110396/67111528
[pid=19636] ppid=19634 vsize=254012 CPUtime=647.52 cores=0,2,4,6
/proc/19636/stat : 19636 (plingeling) S 19634 19636 19297 0 -1 4202496 179075 1250 0 0 64725 27 0 0 23 0 5 0 598284290 260108288 37617 33554432000 4194304 4417332 140736279404656 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/19636/statm: 63503 37617 180 55 0 61389 0
[pid=19636/tid=19649] ppid=19634 vsize=254012 CPUtime=162.04 cores=0,2,4,6
/proc/19636/task/19649/stat : 19649 (plingeling) R 19634 19636 19297 0 -1 4202560 36049 1250 0 0 16198 6 0 0 25 0 5 0 598284315 260108288 37617 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=19636/tid=19650] ppid=19634 vsize=254012 CPUtime=161.16 cores=0,2,4,6
/proc/19636/task/19650/stat : 19650 (plingeling) R 19634 19636 19297 0 -1 4202560 68362 1250 0 0 16107 9 0 0 25 0 5 0 598284315 260108288 37617 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=19636/tid=19651] ppid=19634 vsize=254012 CPUtime=162.03 cores=0,2,4,6
/proc/19636/task/19651/stat : 19651 (plingeling) R 19634 19636 19297 0 -1 4202560 32486 1250 0 0 16198 5 0 0 25 0 5 0 598284315 260108288 37617 33554432000 4194304 4417332 140736279404656 18446744073709551615 4253179 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=19636/tid=19652] ppid=19634 vsize=254012 CPUtime=162.02 cores=0,2,4,6
/proc/19636/task/19652/stat : 19652 (plingeling) R 19634 19636 19297 0 -1 4202560 27876 1250 0 0 16197 5 0 0 25 0 5 0 598284315 260108288 37617 33554432000 4194304 4417332 140736279404656 18446744073709551615 4281860 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 647.52
Current children cumulated vsize (KiB) 254012


################
# More data... #
################

[pid=19636/tid=19650] ppid=19634 vsize=517464 CPUtime=757.89 cores=0,2,4,6
/proc/19636/task/19650/stat : 19650 (plingeling) R 19634 19636 19297 0 -1 4202560 228585 1250 0 0 75751 38 0 0 25 0 5 0 598284315 529883136 101235 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=19636/tid=19651] ppid=19634 vsize=517464 CPUtime=762.05 cores=0,2,4,6
/proc/19636/task/19651/stat : 19651 (plingeling) R 19634 19636 19297 0 -1 4202560 203717 1250 0 0 76173 32 0 0 25 0 5 0 598284315 529883136 101235 33554432000 4194304 4417332 140736279404656 18446744073709551615 4365869 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=19636/tid=19652] ppid=19634 vsize=517464 CPUtime=761.97 cores=0,2,4,6
/proc/19636/task/19652/stat : 19652 (plingeling) R 19634 19636 19297 0 -1 4202560 145676 1250 0 0 76176 21 0 0 25 0 5 0 598284315 529883136 101235 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364671 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3044.25
Current children cumulated vsize (KiB) 517464

[startup+822.301 s]
/proc/loadavg: 8.10 8.09 8.04 10/186 19675
/proc/meminfo: memFree=17547912/32951124 swapFree=67110396/67111528
[pid=19636] ppid=19634 vsize=571716 CPUtime=3283.9 cores=0,2,4,6
/proc/19636/stat : 19636 (plingeling) S 19634 19636 19297 0 -1 4202496 843673 1250 0 0 328258 132 0 0 23 0 5 0 598284290 585437184 115041 33554432000 4194304 4417332 140736279404656 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/19636/statm: 142929 115041 180 55 0 140815 0
[pid=19636/tid=19649] ppid=19634 vsize=571716 CPUtime=822.04 cores=0,2,4,6
/proc/19636/task/19649/stat : 19649 (plingeling) R 19634 19636 19297 0 -1 4202560 222856 1250 0 0 82170 34 0 0 25 0 5 0 598284315 585437184 115041 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=19636/tid=19650] ppid=19634 vsize=571716 CPUtime=817.56 cores=0,2,4,6
/proc/19636/task/19650/stat : 19650 (plingeling) R 19634 19636 19297 0 -1 4202560 237524 1250 0 0 81717 39 0 0 25 0 5 0 598284315 585437184 115041 33554432000 4194304 4417332 140736279404656 18446744073709551615 4277552 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=19636/tid=19651] ppid=19634 vsize=571716 CPUtime=822.04 cores=0,2,4,6
/proc/19636/task/19651/stat : 19651 (plingeling) R 19634 19636 19297 0 -1 4202560 211717 1250 0 0 82171 33 0 0 25 0 5 0 598284315 585437184 115041 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364652 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=19636/tid=19652] ppid=19634 vsize=571716 CPUtime=821.97 cores=0,2,4,6
/proc/19636/task/19652/stat : 19652 (plingeling) R 19634 19636 19297 0 -1 4202560 157274 1250 0 0 82174 23 0 0 25 0 5 0 598284315 585437184 115041 33554432000 4194304 4417332 140736279404656 18446744073709551615 4365918 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3283.9
Current children cumulated vsize (KiB) 571716

[startup+882.301 s]
/proc/loadavg: 8.09 8.09 8.04 10/186 19677
/proc/meminfo: memFree=17438876/32951124 swapFree=67110396/67111528
[pid=19636] ppid=19634 vsize=545668 CPUtime=3523.57 cores=0,2,4,6
/proc/19636/stat : 19636 (plingeling) S 19634 19636 19297 0 -1 4202496 897334 1250 0 0 352216 141 0 0 23 0 5 0 598284290 558764032 108502 33554432000 4194304 4417332 140736279404656 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/19636/statm: 136417 108502 180 55 0 134303 0
[pid=19636/tid=19649] ppid=19634 vsize=545668 CPUtime=882.04 cores=0,2,4,6
/proc/19636/task/19649/stat : 19649 (plingeling) R 19634 19636 19297 0 -1 4202560 230534 1250 0 0 88169 35 0 0 25 0 5 0 598284315 558764032 108502 33554432000 4194304 4417332 140736279404656 18446744073709551615 4236649 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=19636/tid=19650] ppid=19634 vsize=545668 CPUtime=877.23 cores=0,2,4,6
/proc/19636/task/19650/stat : 19650 (plingeling) R 19634 19636 19297 0 -1 4202560 251525 1250 0 0 87682 41 0 0 25 0 5 0 598284315 558764032 108502 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=19636/tid=19651] ppid=19634 vsize=545668 CPUtime=882.04 cores=0,2,4,6
/proc/19636/task/19651/stat : 19651 (plingeling) R 19634 19636 19297 0 -1 4202560 233334 1250 0 0 88167 37 0 0 25 0 5 0 598284315 558764032 108502 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364565 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=19636/tid=19652] ppid=19634 vsize=545668 CPUtime=881.95 cores=0,2,4,6
/proc/19636/task/19652/stat : 19652 (plingeling) R 19634 19636 19297 0 -1 4202560 167639 1250 0 0 88171 24 0 0 25 0 5 0 598284315 558764032 108502 33554432000 4194304 4417332 140736279404656 18446744073709551615 4265465 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3523.57
Current children cumulated vsize (KiB) 545668

[startup+942.301 s]
/proc/loadavg: 8.08 8.08 8.04 10/186 19679
/proc/meminfo: memFree=17380444/32951124 swapFree=67110396/67111528
[pid=19636] ppid=19634 vsize=602404 CPUtime=3763.24 cores=0,2,4,6
/proc/19636/stat : 19636 (plingeling) S 19634 19636 19297 0 -1 4202496 940449 1250 0 0 376177 147 0 0 23 0 5 0 598284290 616861696 122686 33554432000 4194304 4417332 140736279404656 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/19636/statm: 150601 122686 180 55 0 148487 0
[pid=19636/tid=19649] ppid=19634 vsize=602404 CPUtime=942.05 cores=0,2,4,6
/proc/19636/task/19649/stat : 19649 (plingeling) R 19634 19636 19297 0 -1 4202560 238733 1250 0 0 94168 37 0 0 25 0 5 0 598284315 616861696 122686 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=19636/tid=19650] ppid=19634 vsize=602404 CPUtime=936.91 cores=0,2,4,6
/proc/19636/task/19650/stat : 19650 (plingeling) R 19634 19636 19297 0 -1 4202560 256907 1250 0 0 93648 43 0 0 25 0 5 0 598284315 616861696 122686 33554432000 4194304 4417332 140736279404656 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=19636/tid=19651] ppid=19634 vsize=602404 CPUtime=942.04 cores=0,2,4,6
/proc/19636/task/19651/stat : 19651 (plingeling) R 19634 19636 19297 0 -1 4202560 253926 1250 0 0 94165 39 0 0 25 0 5 0 598284315 616861696 122686 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=19636/tid=19652] ppid=19634 vsize=602404 CPUtime=941.96 cores=0,2,4,6
/proc/19636/task/19652/stat : 19652 (plingeling) R 19634 19636 19297 0 -1 4202560 176581 1250 0 0 94170 26 0 0 25 0 5 0 598284315 616861696 122686 33554432000 4194304 4417332 140736279404656 18446744073709551615 4365065 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3763.24
Current children cumulated vsize (KiB) 602404

[startup+1002.3 s]
/proc/loadavg: 8.13 8.10 8.04 9/186 19681
/proc/meminfo: memFree=17259260/32951124 swapFree=67110396/67111528
[pid=19636] ppid=19634 vsize=666024 CPUtime=4002.92 cores=0,2,4,6
/proc/19636/stat : 19636 (plingeling) S 19634 19636 19297 0 -1 4202496 996957 1250 0 0 400133 159 0 0 23 0 5 0 598284290 682008576 138519 33554432000 4194304 4417332 140736279404656 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/19636/statm: 166506 138519 180 55 0 164392 0
[pid=19636/tid=19649] ppid=19634 vsize=666024 CPUtime=1002.05 cores=0,2,4,6
/proc/19636/task/19649/stat : 19649 (plingeling) R 19634 19636 19297 0 -1 4202560 255612 1250 0 0 100165 40 0 0 25 0 5 0 598284315 682008576 138519 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=19636/tid=19650] ppid=19634 vsize=666024 CPUtime=996.58 cores=0,2,4,6
/proc/19636/task/19650/stat : 19650 (plingeling) R 19634 19636 19297 0 -1 4202560 275513 1250 0 0 99612 46 0 0 25 0 5 0 598284315 682008576 138519 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=19636/tid=19651] ppid=19634 vsize=666024 CPUtime=1002.05 cores=0,2,4,6
/proc/19636/task/19651/stat : 19651 (plingeling) R 19634 19636 19297 0 -1 4202560 266741 1250 0 0 100163 42 0 0 25 0 5 0 598284315 682008576 138519 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=19636/tid=19652] ppid=19634 vsize=666024 CPUtime=1001.95 cores=0,2,4,6
/proc/19636/task/19652/stat : 19652 (plingeling) R 19634 19636 19297 0 -1 4202560 184789 1250 0 0 100167 28 0 0 25 0 5 0 598284315 682008576 138519 33554432000 4194304 4417332 140736279404656 18446744073709551615 4357929 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4002.92
Current children cumulated vsize (KiB) 666024

[startup+1062.3 s]
/proc/loadavg: 8.13 8.11 8.05 10/186 19683
/proc/meminfo: memFree=17219428/32951124 swapFree=67110396/67111528
[pid=19636] ppid=19634 vsize=658716 CPUtime=4242.58 cores=0,2,4,6
/proc/19636/stat : 19636 (plingeling) S 19634 19636 19297 0 -1 4202496 1084828 1250 0 0 424084 174 0 0 23 0 5 0 598284290 674525184 136753 33554432000 4194304 4417332 140736279404656 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/19636/statm: 164679 136753 180 55 0 162565 0
[pid=19636/tid=19649] ppid=19634 vsize=658716 CPUtime=1062.05 cores=0,2,4,6
/proc/19636/task/19649/stat : 19649 (plingeling) R 19634 19636 19297 0 -1 4202560 278719 1250 0 0 106160 45 0 0 25 0 5 0 598284315 674525184 136753 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364547 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=19636/tid=19650] ppid=19634 vsize=658716 CPUtime=1056.26 cores=0,2,4,6
/proc/19636/task/19650/stat : 19650 (plingeling) R 19634 19636 19297 0 -1 4202560 323088 1250 0 0 105571 55 0 0 25 0 5 0 598284315 674525184 136753 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=19636/tid=19651] ppid=19634 vsize=658716 CPUtime=1062.05 cores=0,2,4,6
/proc/19636/task/19651/stat : 19651 (plingeling) R 19634 19636 19297 0 -1 4202560 274280 1250 0 0 106162 43 0 0 25 0 5 0 598284315 674525184 136753 33554432000 4194304 4417332 140736279404656 18446744073709551615 4366109 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=19636/tid=19652] ppid=19634 vsize=658716 CPUtime=1061.94 cores=0,2,4,6
/proc/19636/task/19652/stat : 19652 (plingeling) R 19634 19636 19297 0 -1 4202560 194439 1250 0 0 106165 29 0 0 25 0 5 0 598284315 674525184 136753 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4242.58
Current children cumulated vsize (KiB) 658716

[startup+1122.3 s]
/proc/loadavg: 8.09 8.10 8.05 10/186 19683
/proc/meminfo: memFree=17298008/32951124 swapFree=67110396/67111528
[pid=19636] ppid=19634 vsize=631228 CPUtime=4482.25 cores=0,2,4,6
/proc/19636/stat : 19636 (plingeling) S 19634 19636 19297 0 -1 4202496 1131057 1250 0 0 448043 182 0 0 23 0 5 0 598284290 646377472 129867 33554432000 4194304 4417332 140736279404656 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/19636/statm: 157807 129867 180 55 0 155693 0
[pid=19636/tid=19649] ppid=19634 vsize=631228 CPUtime=1122.05 cores=0,2,4,6
/proc/19636/task/19649/stat : 19649 (plingeling) R 19634 19636 19297 0 -1 4202560 294270 1250 0 0 112157 48 0 0 25 0 5 0 598284315 646377472 129867 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364680 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=19636/tid=19650] ppid=19634 vsize=631228 CPUtime=1115.93 cores=0,2,4,6
/proc/19636/task/19650/stat : 19650 (plingeling) R 19634 19636 19297 0 -1 4202560 324777 1250 0 0 111538 55 0 0 25 0 5 0 598284315 646377472 129867 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=19636/tid=19651] ppid=19634 vsize=631228 CPUtime=1122.05 cores=0,2,4,6
/proc/19636/task/19651/stat : 19651 (plingeling) R 19634 19636 19297 0 -1 4202560 295563 1250 0 0 112158 47 0 0 25 0 5 0 598284315 646377472 129867 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=19636/tid=19652] ppid=19634 vsize=631228 CPUtime=1121.94 cores=0,2,4,6
/proc/19636/task/19652/stat : 19652 (plingeling) R 19634 19636 19297 0 -1 4202560 202145 1250 0 0 112164 30 0 0 25 0 5 0 598284315 646377472 129867 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4482.25
Current children cumulated vsize (KiB) 631228

[startup+1182.3 s]
/proc/loadavg: 8.07 8.09 8.05 10/186 19685
/proc/meminfo: memFree=17149048/32951124 swapFree=67110396/67111528
[pid=19636] ppid=19634 vsize=703480 CPUtime=4721.93 cores=0,2,4,6
/proc/19636/stat : 19636 (plingeling) S 19634 19636 19297 0 -1 4202496 1164541 1250 0 0 472007 186 0 0 23 0 5 0 598284290 720363520 147930 33554432000 4194304 4417332 140736279404656 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/19636/statm: 175870 147930 180 55 0 173756 0
[pid=19636/tid=19649] ppid=19634 vsize=703480 CPUtime=1182.05 cores=0,2,4,6
/proc/19636/task/19649/stat : 19649 (plingeling) R 19634 19636 19297 0 -1 4202560 302462 1250 0 0 118157 48 0 0 25 0 5 0 598284315 720363520 147930 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364683 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=19636/tid=19650] ppid=19634 vsize=703480 CPUtime=1175.6 cores=0,2,4,6
/proc/19636/task/19650/stat : 19650 (plingeling) R 19634 19636 19297 0 -1 4202560 330500 1250 0 0 117505 55 0 0 25 0 5 0 598284315 720363520 147930 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=19636/tid=19651] ppid=19634 vsize=703480 CPUtime=1182.05 cores=0,2,4,6
/proc/19636/task/19651/stat : 19651 (plingeling) R 19634 19636 19297 0 -1 4202560 303755 1250 0 0 118157 48 0 0 25 0 5 0 598284315 720363520 147930 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=19636/tid=19652] ppid=19634 vsize=703480 CPUtime=1181.94 cores=0,2,4,6
/proc/19636/task/19652/stat : 19652 (plingeling) R 19634 19636 19297 0 -1 4202560 213522 1250 0 0 118162 32 0 0 25 0 5 0 598284315 720363520 147930 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364463 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4721.93
Current children cumulated vsize (KiB) 703480



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.9 s]
/proc/loadavg: 8.12 8.10 8.05 9/186 19687
/proc/meminfo: memFree=17112460/32951124 swapFree=67110396/67111528
[pid=19636] ppid=19634 vsize=679904 CPUtime=4800.23 cores=0,2,4,6
/proc/19636/stat : 19636 (plingeling) S 19634 19636 19297 0 -1 4202496 1166967 1250 0 0 479837 186 0 0 23 0 5 0 598284290 696221696 142164 33554432000 4194304 4417332 140736279404656 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/19636/statm: 169976 142164 180 55 0 167862 0
[pid=19636/tid=19649] ppid=19634 vsize=679904 CPUtime=1201.65 cores=0,2,4,6
/proc/19636/task/19649/stat : 19649 (plingeling) R 19634 19636 19297 0 -1 4202560 302462 1250 0 0 120117 48 0 0 25 0 5 0 598284315 696221696 142164 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=19636/tid=19650] ppid=19634 vsize=679904 CPUtime=1195.1 cores=0,2,4,6
/proc/19636/task/19650/stat : 19650 (plingeling) R 19634 19636 19297 0 -1 4202560 330628 1250 0 0 119455 55 0 0 25 0 5 0 598284315 696221696 142164 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=19636/tid=19651] ppid=19634 vsize=679904 CPUtime=1201.66 cores=0,2,4,6
/proc/19636/task/19651/stat : 19651 (plingeling) R 19634 19636 19297 0 -1 4202560 303755 1250 0 0 120118 48 0 0 25 0 5 0 598284315 696221696 142164 33554432000 4194304 4417332 140736279404656 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=19636/tid=19652] ppid=19634 vsize=679904 CPUtime=1201.54 cores=0,2,4,6
/proc/19636/task/19652/stat : 19652 (plingeling) R 19634 19636 19297 0 -1 4202560 215820 1250 0 0 120122 32 0 0 25 0 5 0 598284315 696221696 142164 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.23
Current children cumulated vsize (KiB) 679904

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.9 s]
/proc/loadavg: 8.12 8.10 8.05 9/186 19687
/proc/meminfo: memFree=17112460/32951124 swapFree=67110396/67111528
[pid=19636] ppid=19634 vsize=679904 CPUtime=4800.23 cores=0,2,4,6
/proc/19636/stat : 19636 (plingeling) S 19634 19636 19297 0 -1 4202496 1166967 1250 0 0 479837 186 0 0 23 0 5 0 598284290 696221696 142164 33554432000 4194304 4417332 140736279404656 18446744073709551615 224296728549 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/19636/statm: 169976 142164 180 55 0 167862 0
[pid=19636/tid=19649] ppid=19634 vsize=679904 CPUtime=1201.65 cores=0,2,4,6
/proc/19636/task/19649/stat : 19649 (plingeling) R 19634 19636 19297 0 -1 4202560 302462 1250 0 0 120117 48 0 0 25 0 5 0 598284315 696221696 142164 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=19636/tid=19650] ppid=19634 vsize=679904 CPUtime=1195.1 cores=0,2,4,6
/proc/19636/task/19650/stat : 19650 (plingeling) R 19634 19636 19297 0 -1 4202560 330628 1250 0 0 119455 55 0 0 25 0 5 0 598284315 696221696 142164 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=19636/tid=19651] ppid=19634 vsize=679904 CPUtime=1201.66 cores=0,2,4,6
/proc/19636/task/19651/stat : 19651 (plingeling) R 19634 19636 19297 0 -1 4202560 303755 1250 0 0 120118 48 0 0 25 0 5 0 598284315 696221696 142164 33554432000 4194304 4417332 140736279404656 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=19636/tid=19652] ppid=19634 vsize=679904 CPUtime=1201.54 cores=0,2,4,6
/proc/19636/task/19652/stat : 19652 (plingeling) R 19634 19636 19297 0 -1 4202560 215820 1250 0 0 120122 32 0 0 25 0 5 0 598284315 696221696 142164 33554432000 4194304 4417332 140736279404656 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.23
Current children cumulated vsize (KiB) 679904

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1201.94
CPU time (s): 4800.29
CPU user time (s): 4798.4
CPU system time (s): 1.89371
CPU usage (%): 399.379
Max. virtual memory (cumulated for all children) (KiB): 721508

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4798.4
system time used= 1.89371
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1168217
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 101
involuntary context switches= 21066

runsolver used 2.0027 second user time and 4.55931 second system time

The end

Launcher Data 

Begin job on node118 at 2011-04-29 14:35:36
IDJOB=3289161
IDBENCH=83116
IDSOLVER=1605
FILE ID=node118/3289161-1304080536
RUNJOBID= node118-1304080536-19611
PBS_JOBID= 13170505
Free space on /tmp= 71612 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT11/application/fuhs/slp-synthesis-AES/slp-synthesis-aes-top24.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3289161-1304080536/watcher-3289161-1304080536 -o /tmp/evaluation-result-3289161-1304080536/solver-3289161-1304080536 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3289161-1304080536.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 1f485d737caaaca49e2c467c83cbdcec
RANDOM SEED=1771892462

node118.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.842
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5374.59
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      18405676 kB
Buffers:       1901680 kB
Cached:       12031196 kB
SwapCached:        600 kB
Active:        9497688 kB
Inactive:      4512068 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      18405676 kB
SwapTotal:    67111528 kB
SwapFree:     67110396 kB
Dirty:            6300 kB
Writeback:           0 kB
AnonPages:       76164 kB
Mapped:          14964 kB
Slab:           472376 kB
PageTables:       5280 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   218664 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71612 MiB
End job on node118 at 2011-04-29 14:55:40