Trace number 3370995

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.44 1203.76

General information on the benchmark

NameSAT04/handmade/kexu/frb-0.6/
frb75-13-2.used-as.sat04-878.cnf
MD5SUM00e6f1cbbaaa8f43dc6451d49bfa1cef
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark0.241962
Satisfiable
(Un)Satisfiability was proved
Number of variables975
Number of clauses34275
Sum of the clauses size69375
Maximum clause length13
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 234200
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 575

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3370995-1305430863/watcher-3370995-1305430863 -o /tmp/evaluation-result-3370995-1305430863/solver-3370995-1305430863 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3370995-1305430863.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.77 8.01 7.72 6/374 26363
/proc/meminfo: memFree=22524952/32951124 swapFree=67111348/67111528
[pid=26363] ppid=26361 vsize=9628 CPUtime=0 cores=0,2,4,6
/proc/26363/stat : 26363 (plingeling) R 26361 26363 26007 0 -1 4202496 499 1252 0 0 0 0 0 0 25 0 1 0 654684042 9859072 429 33554432000 4194304 4417452 140734949742640 18446744073709551615 212687675136 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/26363/statm: 2407 429 147 55 0 293 0

[startup+0.022208 s]
/proc/loadavg: 7.77 8.01 7.72 6/374 26363
/proc/meminfo: memFree=22524952/32951124 swapFree=67111348/67111528
[pid=26363] ppid=26361 vsize=13248 CPUtime=0.01 cores=0,2,4,6
/proc/26363/stat : 26363 (plingeling) R 26361 26363 26007 0 -1 4202496 1495 1252 0 0 1 0 0 0 25 0 1 0 654684042 13565952 1335 33554432000 4194304 4417452 140734949742640 18446744073709551615 212687675136 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/26363/statm: 3312 1335 148 55 0 1198 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 13248

[startup+0.149119 s]
/proc/loadavg: 7.77 8.01 7.72 6/374 26363
/proc/meminfo: memFree=22524952/32951124 swapFree=67111348/67111528
[pid=26363] ppid=26361 vsize=57800 CPUtime=0.5 cores=0,2,4,6
/proc/26363/stat : 26363 (plingeling) S 26361 26363 26007 0 -1 4202496 2848 1252 0 0 50 0 0 0 25 0 5 0 654684042 59187200 2237 33554432000 4194304 4417452 140734949742640 18446744073709551615 212699477989 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/26363/statm: 14450 2237 180 55 0 12336 0
Current children cumulated CPU time (s) 0.5
Current children cumulated vsize (KiB) 57800

[startup+0.300091 s]
/proc/loadavg: 7.77 8.01 7.72 6/374 26363
/proc/meminfo: memFree=22524952/32951124 swapFree=67111348/67111528
[pid=26363] ppid=26361 vsize=59592 CPUtime=1.1 cores=0,2,4,6
/proc/26363/stat : 26363 (plingeling) S 26361 26363 26007 0 -1 4202496 3485 1252 0 0 110 0 0 0 25 0 5 0 654684042 61022208 2626 33554432000 4194304 4417452 140734949742640 18446744073709551615 212699477989 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/26363/statm: 14898 2626 180 55 0 12784 0
Current children cumulated CPU time (s) 1.1
Current children cumulated vsize (KiB) 59592

[startup+0.701019 s]
/proc/loadavg: 7.77 8.01 7.72 6/374 26363
/proc/meminfo: memFree=22524952/32951124 swapFree=67111348/67111528
[pid=26363] ppid=26361 vsize=59444 CPUtime=2.7 cores=0,2,4,6
/proc/26363/stat : 26363 (plingeling) S 26361 26363 26007 0 -1 4202496 4630 1252 0 0 270 0 0 0 25 0 5 0 654684042 60870656 2647 33554432000 4194304 4417452 140734949742640 18446744073709551615 212699477989 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/26363/statm: 14861 2647 180 55 0 12747 0
Current children cumulated CPU time (s) 2.7
Current children cumulated vsize (KiB) 59444

[startup+1.50087 s]
/proc/loadavg: 7.77 8.01 7.72 9/379 26373
/proc/meminfo: memFree=22514256/32951124 swapFree=67111348/67111528
[pid=26363] ppid=26361 vsize=66420 CPUtime=5.89 cores=0,2,4,6
/proc/26363/stat : 26363 (plingeling) S 26361 26363 26007 0 -1 4202496 8051 1252 0 0 589 0 0 0 25 0 5 0 654684042 68014080 4392 33554432000 4194304 4417452 140734949742640 18446744073709551615 212699477989 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/26363/statm: 16605 4392 180 55 0 14491 0
[pid=26363/tid=26370] ppid=26361 vsize=66420 CPUtime=1.47 cores=0,2,4,6
/proc/26363/task/26370/stat : 26370 (plingeling) R 26361 26363 26007 0 -1 4202560 628 1252 0 0 147 0 0 0 25 0 5 0 654684045 68014080 4392 33554432000 4194304 4417452 140734949742640 18446744073709551615 4365075 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26363/tid=26371] ppid=26361 vsize=66420 CPUtime=1.45 cores=0,2,4,6
/proc/26363/task/26371/stat : 26371 (plingeling) R 26361 26363 26007 0 -1 4202560 528 1252 0 0 145 0 0 0 25 0 5 0 654684045 68014080 4392 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26363/tid=26372] ppid=26361 vsize=66420 CPUtime=1.47 cores=0,2,4,6
/proc/26363/task/26372/stat : 26372 (plingeling) R 26361 26363 26007 0 -1 4202560 1399 1252 0 0 147 0 0 0 25 0 5 0 654684045 68014080 4392 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364604 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26363/tid=26373] ppid=26361 vsize=66420 CPUtime=1.46 cores=0,2,4,6
/proc/26363/task/26373/stat : 26373 (plingeling) R 26361 26363 26007 0 -1 4202560 3989 1252 0 0 146 0 0 0 25 0 5 0 654684045 68014080 4392 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 5.89
Current children cumulated vsize (KiB) 66420

[startup+3.10058 s]
/proc/loadavg: 7.77 8.01 7.72 9/379 26373
/proc/meminfo: memFree=22506444/32951124 swapFree=67111348/67111528
[pid=26363] ppid=26361 vsize=74116 CPUtime=12.28 cores=0,2,4,6
/proc/26363/stat : 26363 (plingeling) S 26361 26363 26007 0 -1 4202496 15323 1252 0 0 1227 1 0 0 25 0 5 0 654684042 75894784 6333 33554432000 4194304 4417452 140734949742640 18446744073709551615 212699477989 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/26363/statm: 18529 6333 180 55 0 16415 0
[pid=26363/tid=26370] ppid=26361 vsize=74116 CPUtime=3.07 cores=0,2,4,6
/proc/26363/task/26370/stat : 26370 (plingeling) R 26361 26363 26007 0 -1 4202560 1709 1252 0 0 307 0 0 0 25 0 5 0 654684045 75894784 6333 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26363/tid=26371] ppid=26361 vsize=74116 CPUtime=3.04 cores=0,2,4,6
/proc/26363/task/26371/stat : 26371 (plingeling) R 26361 26363 26007 0 -1 4202560 1459 1252 0 0 304 0 0 0 25 0 5 0 654684045 75894784 6333 33554432000 4194304 4417452 140734949742640 18446744073709551615 4302006 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26363/tid=26372] ppid=26361 vsize=74116 CPUtime=3.07 cores=0,2,4,6
/proc/26363/task/26372/stat : 26372 (plingeling) R 26361 26363 26007 0 -1 4202560 2692 1252 0 0 307 0 0 0 25 0 5 0 654684045 75894784 6333 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26363/tid=26373] ppid=26361 vsize=74116 CPUtime=3.06 cores=0,2,4,6
/proc/26363/task/26373/stat : 26373 (plingeling) R 26361 26363 26007 0 -1 4202560 7956 1252 0 0 306 0 0 0 25 0 5 0 654684045 75894784 6333 33554432000 4194304 4417452 140734949742640 18446744073709551615 4251160 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 12.28
Current children cumulated vsize (KiB) 74116

[startup+6.301 s]
/proc/loadavg: 7.62 7.98 7.71 9/379 26410
/proc/meminfo: memFree=22588472/32951124 swapFree=67111348/67111528
[pid=26363] ppid=26361 vsize=86992 CPUtime=25.04 cores=0,2,4,6
/proc/26363/stat : 26363 (plingeling) S 26361 26363 26007 0 -1 4202496 27225 1252 0 0 2501 3 0 0 25 0 5 0 654684042 89079808 9567 33554432000 4194304 4417452 140734949742640 18446744073709551615 212699477989 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/26363/statm: 21748 9567 180 55 0 19634 0
[pid=26363/tid=26370] ppid=26361 vsize=86992 CPUtime=6.27 cores=0,2,4,6
/proc/26363/task/26370/stat : 26370 (plingeling) R 26361 26363 26007 0 -1 4202560 4860 1252 0 0 627 0 0 0 25 0 5 0 654684045 89079808 9567 33554432000 4194304 4417452 140734949742640 18446744073709551615 4354150 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26363/tid=26371] ppid=26361 vsize=86992 CPUtime=6.2 cores=0,2,4,6
/proc/26363/task/26371/stat : 26371 (plingeling) R 26361 26363 26007 0 -1 4202560 4076 1252 0 0 620 0 0 0 25 0 5 0 654684045 89079808 9567 33554432000 4194304 4417452 140734949742640 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26363/tid=26372] ppid=26361 vsize=86992 CPUtime=6.27 cores=0,2,4,6
/proc/26363/task/26372/stat : 26372 (plingeling) R 26361 26363 26007 0 -1 4202560 4063 1252 0 0 627 0 0 0 25 0 5 0 654684045 89079808 9567 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26363/tid=26373] ppid=26361 vsize=86992 CPUtime=6.25 cores=0,2,4,6
/proc/26363/task/26373/stat : 26373 (plingeling) R 26361 26363 26007 0 -1 4202560 12719 1252 0 0 624 1 0 0 25 0 5 0 654684045 89079808 9567 33554432000 4194304 4417452 140734949742640 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 25.04
Current children cumulated vsize (KiB) 86992

[startup+12.7008 s]
/proc/loadavg: 7.65 7.98 7.71 9/379 26410
/proc/meminfo: memFree=22541232/32951124 swapFree=67111348/67111528
[pid=26363] ppid=26361 vsize=89568 CPUtime=50.55 cores=0,2,4,6
/proc/26363/stat : 26363 (plingeling) S 26361 26363 26007 0 -1 4202496 50114 1252 0 0 5049 6 0 0 25 0 5 0 654684042 91717632 10179 33554432000 4194304 4417452 140734949742640 18446744073709551615 212699477989 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/26363/statm: 22392 10179 180 55 0 20278 0
[pid=26363/tid=26370] ppid=26361 vsize=89568 CPUtime=12.66 cores=0,2,4,6
/proc/26363/task/26370/stat : 26370 (plingeling) R 26361 26363 26007 0 -1 4202560 9061 1252 0 0 1266 0 0 0 25 0 5 0 654684045 91717632 10179 33554432000 4194304 4417452 140734949742640 18446744073709551615 4232680 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26363/tid=26371] ppid=26361 vsize=89568 CPUtime=12.52 cores=0,2,4,6
/proc/26363/task/26371/stat : 26371 (plingeling) R 26361 26363 26007 0 -1 4202560 9606 1252 0 0 1251 1 0 0 25 0 5 0 654684045 91717632 10179 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364807 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26363/tid=26372] ppid=26361 vsize=89568 CPUtime=12.66 cores=0,2,4,6
/proc/26363/task/26372/stat : 26372 (plingeling) R 26361 26363 26007 0 -1 4202560 8138 1252 0 0 1266 0 0 0 25 0 5 0 654684045 91717632 10179 33554432000 4194304 4417452 140734949742640 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26363/tid=26373] ppid=26361 vsize=89568 CPUtime=12.66 cores=0,2,4,6
/proc/26363/task/26373/stat : 26373 (plingeling) R 26361 26363 26007 0 -1 4202560 21802 1252 0 0 1263 3 0 0 25 0 5 0 654684045 91717632 10179 33554432000 4194304 4417452 140734949742640 18446744073709551615 4264774 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 50.55
Current children cumulated vsize (KiB) 89568

[startup+25.5005 s]
/proc/loadavg: 7.73 7.98 7.71 9/379 26410
/proc/meminfo: memFree=22521336/32951124 swapFree=67111348/67111528
[pid=26363] ppid=26361 vsize=102368 CPUtime=101.6 cores=0,2,4,6
/proc/26363/stat : 26363 (plingeling) S 26361 26363 26007 0 -1 4202496 89663 1252 0 0 10148 12 0 0 25 0 5 0 654684042 104824832 13379 33554432000 4194304 4417452 140734949742640 18446744073709551615 212699477989 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/26363/statm: 25592 13379 180 55 0 23478 0
[pid=26363/tid=26370] ppid=26361 vsize=102368 CPUtime=25.47 cores=0,2,4,6
/proc/26363/task/26370/stat : 26370 (plingeling) R 26361 26363 26007 0 -1 4202560 17919 1252 0 0 2545 2 0 0 25 0 5 0 654684045 104824832 13379 33554432000 4194304 4417452 140734949742640 18446744073709551615 4263155 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26363/tid=26371] ppid=26361 vsize=102368 CPUtime=25.17 cores=0,2,4,6
/proc/26363/task/26371/stat : 26371 (plingeling) R 26361 26363 26007 0 -1 4202560 18351 1252 0 0 2515 2 0 0 25 0 5 0 654684045 104824832 13379 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26363/tid=26372] ppid=26361 vsize=102368 CPUtime=25.46 cores=0,2,4,6
/proc/26363/task/26372/stat : 26372 (plingeling) R 26361 26363 26007 0 -1 4202560 15914 1252 0 0 2545 1 0 0 25 0 5 0 654684045 104824832 13379 33554432000 4194304 4417452 140734949742640 18446744073709551615 4354506 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26363/tid=26373] ppid=26361 vsize=102368 CPUtime=25.45 cores=0,2,4,6
/proc/26363/task/26373/stat : 26373 (plingeling) R 26361 26363 26007 0 -1 4202560 35972 1252 0 0 2540 5 0 0 25 0 5 0 654684045 104824832 13379 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 101.6
Current children cumulated vsize (KiB) 102368

[startup+51.1008 s]
/proc/loadavg: 7.89 8.00 7.73 9/379 26411
/proc/meminfo: memFree=22447668/32951124 swapFree=67111348/67111528
[pid=26363] ppid=26361 vsize=131216 CPUtime=203.69 cores=0,2,4,6
/proc/26363/stat : 26363 (plingeling) S 26361 26363 26007 0 -1 4202496 157828 1252 0 0 20347 22 0 0 25 0 5 0 654684042 134365184 20591 33554432000 4194304 4417452 140734949742640 18446744073709551615 212699477989 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/26363/statm: 32804 20591 180 55 0 30690 0
[pid=26363/tid=26370] ppid=26361 vsize=131216 CPUtime=51.06 cores=0,2,4,6
/proc/26363/task/26370/stat : 26370 (plingeling) R 26361 26363 26007 0 -1 4202560 34542 1252 0 0 5102 4 0 0 25 0 5 0 654684045 134365184 20591 33554432000 4194304 4417452 140734949742640 18446744073709551615 4311243 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26363/tid=26371] ppid=26361 vsize=131216 CPUtime=50.46 cores=0,2,4,6
/proc/26363/task/26371/stat : 26371 (plingeling) R 26361 26363 26007 0 -1 4202560 37178 1252 0 0 5041 5 0 0 25 0 5 0 654684045 134365184 20591 33554432000 4194304 4417452 140734949742640 18446744073709551615 4353782 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26363/tid=26372] ppid=26361 vsize=131216 CPUtime=51.06 cores=0,2,4,6
/proc/26363/task/26372/stat : 26372 (plingeling) R 26361 26363 26007 0 -1 4202560 31281 1252 0 0 5103 3 0 0 25 0 5 0 654684045 134365184 20591 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26363/tid=26373] ppid=26361 vsize=131216 CPUtime=51.04 cores=0,2,4,6
/proc/26363/task/26373/stat : 26373 (plingeling) R 26361 26363 26007 0 -1 4202560 53320 1252 0 0 5097 7 0 0 25 0 5 0 654684045 134365184 20591 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 203.69
Current children cumulated vsize (KiB) 131216

[startup+102.313 s]
/proc/loadavg: 7.95 8.00 7.74 9/379 26412
/proc/meminfo: memFree=22404008/32951124 swapFree=67111348/67111528
[pid=26363] ppid=26361 vsize=156828 CPUtime=407.93 cores=0,2,4,6
/proc/26363/stat : 26363 (plingeling) S 26361 26363 26007 0 -1 4202496 259428 1252 0 0 40755 38 0 0 25 0 5 0 654684042 160591872 27026 33554432000 4194304 4417452 140734949742640 18446744073709551615 212699477989 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/26363/statm: 39207 27026 180 55 0 37093 0
[pid=26363/tid=26370] ppid=26361 vsize=156828 CPUtime=102.28 cores=0,2,4,6
/proc/26363/task/26370/stat : 26370 (plingeling) R 26361 26363 26007 0 -1 4202560 56615 1252 0 0 10221 7 0 0 25 0 5 0 654684045 160591872 27026 33554432000 4194304 4417452 140734949742640 18446744073709551615 4277871 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26363/tid=26371] ppid=26361 vsize=156828 CPUtime=101.06 cores=0,2,4,6
/proc/26363/task/26371/stat : 26371 (plingeling) R 26361 26363 26007 0 -1 4202560 61069 1252 0 0 10097 9 0 0 25 0 5 0 654684045 160591872 27026 33554432000 4194304 4417452 140734949742640 18446744073709551615 4365004 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26363/tid=26372] ppid=26361 vsize=156828 CPUtime=102.28 cores=0,2,4,6
/proc/26363/task/26372/stat : 26372 (plingeling) R 26361 26363 26007 0 -1 4202560 52466 1252 0 0 10221 7 0 0 25 0 5 0 654684045 160591872 27026 33554432000 4194304 4417452 140734949742640 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26363/tid=26373] ppid=26361 vsize=156828 CPUtime=102.26 cores=0,2,4,6
/proc/26363/task/26373/stat : 26373 (plingeling) R 26361 26363 26007 0 -1 4202560 87771 1252 0 0 10213 13 0 0 25 0 5 0 654684045 160591872 27026 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 407.93
Current children cumulated vsize (KiB) 156828

[startup+162.301 s]
/proc/loadavg: 8.17 8.04 7.77 9/379 26414
/proc/meminfo: memFree=22253716/32951124 swapFree=67111348/67111528
[pid=26363] ppid=26361 vsize=223304 CPUtime=647.17 cores=0,2,4,6
/proc/26363/stat : 26363 (plingeling) S 26361 26363 26007 0 -1 4202496 335196 1252 0 0 64666 51 0 0 25 0 5 0 654684042 228663296 43613 33554432000 4194304 4417452 140734949742640 18446744073709551615 212699477989 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/26363/statm: 55826 43613 180 55 0 53712 0
[pid=26363/tid=26370] ppid=26361 vsize=223304 CPUtime=162.27 cores=0,2,4,6
/proc/26363/task/26370/stat : 26370 (plingeling) R 26361 26363 26007 0 -1 4202560 79521 1252 0 0 16216 11 0 0 25 0 5 0 654684045 228663296 43613 33554432000 4194304 4417452 140734949742640 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26363/tid=26371] ppid=26361 vsize=223304 CPUtime=160.36 cores=0,2,4,6
/proc/26363/task/26371/stat : 26371 (plingeling) R 26361 26363 26007 0 -1 4202560 83349 1252 0 0 16022 14 0 0 25 0 5 0 654684045 228663296 43613 33554432000 4194304 4417452 140734949742640 18446744073709551615 4344176 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26363/tid=26372] ppid=26361 vsize=223304 CPUtime=162.26 cores=0,2,4,6
/proc/26363/task/26372/stat : 26372 (plingeling) R 26361 26363 26007 0 -1 4202560 71075 1252 0 0 16216 10 0 0 25 0 5 0 654684045 228663296 43613 33554432000 4194304 4417452 140734949742640 18446744073709551615 4365075 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26363/tid=26373] ppid=26361 vsize=223304 CPUtime=162.24 cores=0,2,4,6
/proc/26363/task/26373/stat : 26373 (plingeling) R 26361 26363 26007 0 -1 4202560 99744 1252 0 0 16209 15 0 0 25 0 5 0 654684045 228663296 43613 33554432000 4194304 4417452 140734949742640 18446744073709551615 4366126 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 647.17
Current children cumulated vsize (KiB) 223304


################
# More data... #
################

[pid=26363/tid=26371] ppid=26361 vsize=801036 CPUtime=753.27 cores=0,2,4,6
/proc/26363/task/26371/stat : 26371 (plingeling) R 26361 26363 26007 0 -1 4202560 212565 1252 0 0 75290 37 0 0 25 0 5 0 654684045 820260864 188046 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26363/tid=26372] ppid=26361 vsize=801036 CPUtime=762.3 cores=0,2,4,6
/proc/26363/task/26372/stat : 26372 (plingeling) R 26361 26363 26007 0 -1 4202560 184277 1252 0 0 76199 31 0 0 25 0 5 0 654684045 820260864 188046 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26363/tid=26373] ppid=26361 vsize=801036 CPUtime=762.15 cores=0,2,4,6
/proc/26363/task/26373/stat : 26373 (plingeling) R 26361 26363 26007 0 -1 4202560 222170 1252 0 0 76178 37 0 0 25 0 5 0 654684045 820260864 188046 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3040.07
Current children cumulated vsize (KiB) 801036

[startup+822.301 s]
/proc/loadavg: 8.08 8.10 7.92 9/379 26431
/proc/meminfo: memFree=21114148/32951124 swapFree=67111348/67111528
[pid=26363] ppid=26361 vsize=797532 CPUtime=3279.3 cores=0,2,4,6
/proc/26363/stat : 26363 (plingeling) S 26361 26363 26007 0 -1 4202496 886073 1252 0 0 327782 148 0 0 25 0 5 0 654684042 816672768 187170 33554432000 4194304 4417452 140734949742640 18446744073709551615 212699477989 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/26363/statm: 199383 187170 180 55 0 197269 0
[pid=26363/tid=26370] ppid=26361 vsize=797532 CPUtime=822.29 cores=0,2,4,6
/proc/26363/task/26370/stat : 26370 (plingeling) R 26361 26363 26007 0 -1 4202560 256227 1252 0 0 82189 40 0 0 25 0 5 0 654684045 816672768 187170 33554432000 4194304 4417452 140734949742640 18446744073709551615 4343986 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26363/tid=26371] ppid=26361 vsize=797532 CPUtime=812.56 cores=0,2,4,6
/proc/26363/task/26371/stat : 26371 (plingeling) R 26361 26363 26007 0 -1 4202560 221892 1252 0 0 81217 39 0 0 25 0 5 0 654684045 816672768 187170 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26363/tid=26372] ppid=26361 vsize=797532 CPUtime=822.29 cores=0,2,4,6
/proc/26363/task/26372/stat : 26372 (plingeling) R 26361 26363 26007 0 -1 4202560 184277 1252 0 0 82198 31 0 0 25 0 5 0 654684045 816672768 187170 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26363/tid=26373] ppid=26361 vsize=797532 CPUtime=822.12 cores=0,2,4,6
/proc/26363/task/26373/stat : 26373 (plingeling) R 26361 26363 26007 0 -1 4202560 222170 1252 0 0 82175 37 0 0 25 0 5 0 654684045 816672768 187170 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3279.3
Current children cumulated vsize (KiB) 797532

[startup+882.3 s]
/proc/loadavg: 8.14 8.11 7.93 9/379 26433
/proc/meminfo: memFree=21039976/32951124 swapFree=67111348/67111528
[pid=26363] ppid=26361 vsize=873448 CPUtime=3518.59 cores=0,2,4,6
/proc/26363/stat : 26363 (plingeling) S 26361 26363 26007 0 -1 4202496 905052 1252 0 0 351708 151 0 0 25 0 5 0 654684042 894410752 206149 33554432000 4194304 4417452 140734949742640 18446744073709551615 212699477989 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/26363/statm: 218362 206149 180 55 0 216248 0
[pid=26363/tid=26370] ppid=26361 vsize=873448 CPUtime=882.29 cores=0,2,4,6
/proc/26363/task/26370/stat : 26370 (plingeling) R 26361 26363 26007 0 -1 4202560 256227 1252 0 0 88189 40 0 0 25 0 5 0 654684045 894410752 206149 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26363/tid=26371] ppid=26361 vsize=873448 CPUtime=871.84 cores=0,2,4,6
/proc/26363/task/26371/stat : 26371 (plingeling) R 26361 26363 26007 0 -1 4202560 234490 1252 0 0 87144 40 0 0 25 0 5 0 654684045 894410752 206149 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364724 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26363/tid=26372] ppid=26361 vsize=873448 CPUtime=882.29 cores=0,2,4,6
/proc/26363/task/26372/stat : 26372 (plingeling) R 26361 26363 26007 0 -1 4202560 184277 1252 0 0 88198 31 0 0 25 0 5 0 654684045 894410752 206149 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26363/tid=26373] ppid=26361 vsize=873448 CPUtime=882.11 cores=0,2,4,6
/proc/26363/task/26373/stat : 26373 (plingeling) R 26361 26363 26007 0 -1 4202560 228551 1252 0 0 88173 38 0 0 25 0 5 0 654684045 894410752 206149 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3518.59
Current children cumulated vsize (KiB) 873448

[startup+942.3 s]
/proc/loadavg: 8.16 8.12 7.94 9/379 26435
/proc/meminfo: memFree=20942384/32951124 swapFree=67111348/67111528
[pid=26363] ppid=26361 vsize=852112 CPUtime=3757.88 cores=0,2,4,6
/proc/26363/stat : 26363 (plingeling) S 26361 26363 26007 0 -1 4202496 968203 1252 0 0 375628 160 0 0 25 0 5 0 654684042 872562688 200815 33554432000 4194304 4417452 140734949742640 18446744073709551615 212699477989 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/26363/statm: 213028 200815 180 55 0 210914 0
[pid=26363/tid=26370] ppid=26361 vsize=852112 CPUtime=942.29 cores=0,2,4,6
/proc/26363/task/26370/stat : 26370 (plingeling) R 26361 26363 26007 0 -1 4202560 256227 1252 0 0 94189 40 0 0 25 0 5 0 654684045 872562688 200815 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26363/tid=26371] ppid=26361 vsize=852112 CPUtime=931.14 cores=0,2,4,6
/proc/26363/task/26371/stat : 26371 (plingeling) R 26361 26363 26007 0 -1 4202560 239319 1252 0 0 93073 41 0 0 25 0 5 0 654684045 872562688 200815 33554432000 4194304 4417452 140734949742640 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26363/tid=26372] ppid=26361 vsize=852112 CPUtime=942.29 cores=0,2,4,6
/proc/26363/task/26372/stat : 26372 (plingeling) R 26361 26363 26007 0 -1 4202560 208100 1252 0 0 94194 35 0 0 25 0 5 0 654684045 872562688 200815 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26363/tid=26373] ppid=26361 vsize=852112 CPUtime=942.11 cores=0,2,4,6
/proc/26363/task/26373/stat : 26373 (plingeling) R 26361 26363 26007 0 -1 4202560 263050 1252 0 0 94168 43 0 0 25 0 5 0 654684045 872562688 200815 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3757.88
Current children cumulated vsize (KiB) 852112

[startup+1002.3 s]
/proc/loadavg: 8.14 8.11 7.95 9/379 26436
/proc/meminfo: memFree=21150816/32951124 swapFree=67111348/67111528
[pid=26363] ppid=26361 vsize=662992 CPUtime=3997.15 cores=0,2,4,6
/proc/26363/stat : 26363 (plingeling) S 26361 26363 26007 0 -1 4202496 1015965 1252 0 0 399548 167 0 0 25 0 5 0 654684042 678903808 153567 33554432000 4194304 4417452 140734949742640 18446744073709551615 212699477989 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/26363/statm: 165748 153567 180 55 0 163634 0
[pid=26363/tid=26370] ppid=26361 vsize=662992 CPUtime=1002.3 cores=0,2,4,6
/proc/26363/task/26370/stat : 26370 (plingeling) R 26361 26363 26007 0 -1 4202560 271195 1252 0 0 100187 43 0 0 25 0 5 0 654684045 678903808 153567 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26363/tid=26371] ppid=26361 vsize=662992 CPUtime=990.43 cores=0,2,4,6
/proc/26363/task/26371/stat : 26371 (plingeling) R 26361 26363 26007 0 -1 4202560 240657 1252 0 0 99002 41 0 0 25 0 5 0 654684045 678903808 153567 33554432000 4194304 4417452 140734949742640 18446744073709551615 4309897 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26363/tid=26372] ppid=26361 vsize=662992 CPUtime=1002.29 cores=0,2,4,6
/proc/26363/task/26372/stat : 26372 (plingeling) R 26361 26363 26007 0 -1 4202560 235460 1252 0 0 100190 39 0 0 25 0 5 0 654684045 678903808 153567 33554432000 4194304 4417452 140734949742640 18446744073709551615 4366400 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26363/tid=26373] ppid=26361 vsize=662992 CPUtime=1002.09 cores=0,2,4,6
/proc/26363/task/26373/stat : 26373 (plingeling) R 26361 26363 26007 0 -1 4202560 267146 1252 0 0 100166 43 0 0 25 0 5 0 654684045 678903808 153567 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364735 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3997.15
Current children cumulated vsize (KiB) 662992

[startup+1062.3 s]
/proc/loadavg: 8.16 8.12 7.96 9/379 26438
/proc/meminfo: memFree=21006724/32951124 swapFree=67111348/67111528
[pid=26363] ppid=26361 vsize=737268 CPUtime=4236.44 cores=0,2,4,6
/proc/26363/stat : 26363 (plingeling) S 26361 26363 26007 0 -1 4202496 1048529 1252 0 0 423472 172 0 0 25 0 5 0 654684042 754962432 171302 33554432000 4194304 4417452 140734949742640 18446744073709551615 212699477989 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/26363/statm: 184317 171302 180 55 0 182203 0
[pid=26363/tid=26370] ppid=26361 vsize=737268 CPUtime=1062.3 cores=0,2,4,6
/proc/26363/task/26370/stat : 26370 (plingeling) R 26361 26363 26007 0 -1 4202560 284273 1252 0 0 106185 45 0 0 25 0 5 0 654684045 754962432 171302 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26363/tid=26371] ppid=26361 vsize=737268 CPUtime=1049.73 cores=0,2,4,6
/proc/26363/task/26371/stat : 26371 (plingeling) R 26361 26363 26007 0 -1 4202560 260143 1252 0 0 104929 44 0 0 25 0 5 0 654684045 754962432 171302 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26363/tid=26372] ppid=26361 vsize=737268 CPUtime=1062.29 cores=0,2,4,6
/proc/26363/task/26372/stat : 26372 (plingeling) R 26361 26363 26007 0 -1 4202560 235460 1252 0 0 106190 39 0 0 25 0 5 0 654684045 754962432 171302 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26363/tid=26373] ppid=26361 vsize=737268 CPUtime=1062.08 cores=0,2,4,6
/proc/26363/task/26373/stat : 26373 (plingeling) R 26361 26363 26007 0 -1 4202560 267146 1252 0 0 106165 43 0 0 25 0 5 0 654684045 754962432 171302 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364724 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4236.44
Current children cumulated vsize (KiB) 737268

[startup+1122.3 s]
/proc/loadavg: 8.13 8.11 7.96 9/379 26439
/proc/meminfo: memFree=21105540/32951124 swapFree=67111348/67111528
[pid=26363] ppid=26361 vsize=810348 CPUtime=4475.73 cores=0,2,4,6
/proc/26363/stat : 26363 (plingeling) S 26361 26363 26007 0 -1 4202496 1076158 1252 0 0 447396 177 0 0 25 0 5 0 654684042 829796352 189643 33554432000 4194304 4417452 140734949742640 18446744073709551615 212699477989 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/26363/statm: 202587 189643 180 55 0 200473 0
[pid=26363/tid=26370] ppid=26361 vsize=810348 CPUtime=1122.3 cores=0,2,4,6
/proc/26363/task/26370/stat : 26370 (plingeling) R 26361 26363 26007 0 -1 4202560 301069 1252 0 0 112183 47 0 0 25 0 5 0 654684045 829796352 189643 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26363/tid=26371] ppid=26361 vsize=810348 CPUtime=1109.01 cores=0,2,4,6
/proc/26363/task/26371/stat : 26371 (plingeling) R 26361 26363 26007 0 -1 4202560 260143 1252 0 0 110857 44 0 0 25 0 5 0 654684045 829796352 189643 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364765 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26363/tid=26372] ppid=26361 vsize=810348 CPUtime=1122.29 cores=0,2,4,6
/proc/26363/task/26372/stat : 26372 (plingeling) R 26361 26363 26007 0 -1 4202560 235460 1252 0 0 112190 39 0 0 25 0 5 0 654684045 829796352 189643 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364807 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26363/tid=26373] ppid=26361 vsize=810348 CPUtime=1122.08 cores=0,2,4,6
/proc/26363/task/26373/stat : 26373 (plingeling) R 26361 26363 26007 0 -1 4202560 277979 1252 0 0 112162 46 0 0 25 0 5 0 654684045 829796352 189643 33554432000 4194304 4417452 140734949742640 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4475.73
Current children cumulated vsize (KiB) 810348

[startup+1182.3 s]
/proc/loadavg: 8.18 8.12 7.97 9/379 26441
/proc/meminfo: memFree=20988968/32951124 swapFree=67111348/67111528
[pid=26363] ppid=26361 vsize=881264 CPUtime=4715.01 cores=0,2,4,6
/proc/26363/stat : 26363 (plingeling) S 26361 26363 26007 0 -1 4202496 1093887 1252 0 0 471320 181 0 0 25 0 5 0 654684042 902414336 207372 33554432000 4194304 4417452 140734949742640 18446744073709551615 212699477989 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/26363/statm: 220316 207372 180 55 0 218202 0
[pid=26363/tid=26370] ppid=26361 vsize=881264 CPUtime=1182.3 cores=0,2,4,6
/proc/26363/task/26370/stat : 26370 (plingeling) R 26361 26363 26007 0 -1 4202560 301069 1252 0 0 118183 47 0 0 25 0 5 0 654684045 902414336 207372 33554432000 4194304 4417452 140734949742640 18446744073709551615 4353615 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26363/tid=26371] ppid=26361 vsize=881264 CPUtime=1168.31 cores=0,2,4,6
/proc/26363/task/26371/stat : 26371 (plingeling) R 26361 26363 26007 0 -1 4202560 260143 1252 0 0 116787 44 0 0 25 0 5 0 654684045 902414336 207372 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26363/tid=26372] ppid=26361 vsize=881264 CPUtime=1182.29 cores=0,2,4,6
/proc/26363/task/26372/stat : 26372 (plingeling) R 26361 26363 26007 0 -1 4202560 240611 1252 0 0 118189 40 0 0 25 0 5 0 654684045 902414336 207372 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364699 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26363/tid=26373] ppid=26361 vsize=881264 CPUtime=1182.06 cores=0,2,4,6
/proc/26363/task/26373/stat : 26373 (plingeling) R 26361 26363 26007 0 -1 4202560 290557 1252 0 0 118157 49 0 0 25 0 5 0 654684045 902414336 207372 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4715.01
Current children cumulated vsize (KiB) 881264



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.7 s]
/proc/loadavg: 8.13 8.11 7.97 9/379 26451
/proc/meminfo: memFree=20986696/32951124 swapFree=67111348/67111528
[pid=26363] ppid=26361 vsize=881852 CPUtime=4800.35 cores=0,2,4,6
/proc/26363/stat : 26363 (plingeling) S 26361 26363 26007 0 -1 4202496 1094034 1252 0 0 479853 182 0 0 25 0 5 0 654684042 903016448 207519 33554432000 4194304 4417452 140734949742640 18446744073709551615 212699477989 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/26363/statm: 220463 207519 180 55 0 218349 0
[pid=26363/tid=26370] ppid=26361 vsize=881852 CPUtime=1203.72 cores=0,2,4,6
/proc/26363/task/26370/stat : 26370 (plingeling) R 26361 26363 26007 0 -1 4202560 301069 1252 0 0 120325 47 0 0 25 0 5 0 654684045 903016448 207519 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26363/tid=26371] ppid=26361 vsize=881852 CPUtime=1189.45 cores=0,2,4,6
/proc/26363/task/26371/stat : 26371 (plingeling) R 26361 26363 26007 0 -1 4202560 260290 1252 0 0 118901 44 0 0 25 0 5 0 654684045 903016448 207519 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26363/tid=26372] ppid=26361 vsize=881852 CPUtime=1203.69 cores=0,2,4,6
/proc/26363/task/26372/stat : 26372 (plingeling) R 26361 26363 26007 0 -1 4202560 240611 1252 0 0 120328 41 0 0 25 0 5 0 654684045 903016448 207519 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26363/tid=26373] ppid=26361 vsize=881852 CPUtime=1203.47 cores=0,2,4,6
/proc/26363/task/26373/stat : 26373 (plingeling) R 26361 26363 26007 0 -1 4202560 290557 1252 0 0 120298 49 0 0 25 0 5 0 654684045 903016448 207519 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.35
Current children cumulated vsize (KiB) 881852

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.7 s]
/proc/loadavg: 8.13 8.11 7.97 9/379 26451
/proc/meminfo: memFree=20986696/32951124 swapFree=67111348/67111528
[pid=26363] ppid=26361 vsize=881852 CPUtime=4800.35 cores=0,2,4,6
/proc/26363/stat : 26363 (plingeling) S 26361 26363 26007 0 -1 4202496 1094034 1252 0 0 479853 182 0 0 25 0 5 0 654684042 903016448 207519 33554432000 4194304 4417452 140734949742640 18446744073709551615 212699477989 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/26363/statm: 220463 207519 180 55 0 218349 0
[pid=26363/tid=26370] ppid=26361 vsize=881852 CPUtime=1203.72 cores=0,2,4,6
/proc/26363/task/26370/stat : 26370 (plingeling) R 26361 26363 26007 0 -1 4202560 301069 1252 0 0 120325 47 0 0 25 0 5 0 654684045 903016448 207519 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26363/tid=26371] ppid=26361 vsize=881852 CPUtime=1189.45 cores=0,2,4,6
/proc/26363/task/26371/stat : 26371 (plingeling) R 26361 26363 26007 0 -1 4202560 260290 1252 0 0 118901 44 0 0 25 0 5 0 654684045 903016448 207519 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26363/tid=26372] ppid=26361 vsize=881852 CPUtime=1203.69 cores=0,2,4,6
/proc/26363/task/26372/stat : 26372 (plingeling) R 26361 26363 26007 0 -1 4202560 240611 1252 0 0 120328 41 0 0 25 0 5 0 654684045 903016448 207519 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26363/tid=26373] ppid=26361 vsize=881852 CPUtime=1203.47 cores=0,2,4,6
/proc/26363/task/26373/stat : 26373 (plingeling) R 26361 26363 26007 0 -1 4202560 290557 1252 0 0 120298 49 0 0 25 0 5 0 654684045 903016448 207519 33554432000 4194304 4417452 140734949742640 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.35
Current children cumulated vsize (KiB) 881852

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1203.76
CPU time (s): 4800.44
CPU user time (s): 4798.58
CPU system time (s): 1.86472
CPU usage (%): 398.788
Max. virtual memory (cumulated for all children) (KiB): 984144

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4798.58
system time used= 1.86472
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1095287
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 244
involuntary context switches= 25982

runsolver used 4.11537 second user time and 10.1345 second system time

The end

Launcher Data 

Begin job on node139 at 2011-05-15 05:41:03
IDJOB=3370995
IDBENCH=84551
IDSOLVER=1848
FILE ID=node139/3370995-1305430863
RUNJOBID= node139-1305427968-26025
PBS_JOBID= 13324610
Free space on /tmp= 73408 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT04/handmade/kexu/frb-0.6/frb75-13-2.used-as.sat04-878.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3370995-1305430863/watcher-3370995-1305430863 -o /tmp/evaluation-result-3370995-1305430863/solver-3370995-1305430863 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3370995-1305430863.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 00e6f1cbbaaa8f43dc6451d49bfa1cef
RANDOM SEED=1503364358

node139.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5321.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.852
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      22525116 kB
Buffers:        304388 kB
Cached:        2860900 kB
SwapCached:          0 kB
Active:        8101588 kB
Inactive:      2020368 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      22525116 kB
SwapTotal:    67111528 kB
SwapFree:     67111348 kB
Dirty:             440 kB
Writeback:           0 kB
AnonPages:     6956468 kB
Mapped:          29212 kB
Slab:           202152 kB
PageTables:      38488 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 76946964 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73408 MiB
End job on node139 at 2011-05-15 06:01:09