Trace number 3371186

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.48 1204.03

General information on the benchmark

Namecrafted/spence/sgen/
sgen3-n280-s93756482-sat.cnf
MD5SUMbadb3b4ac12b29ada7dd8ed96ed75c80
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark3.66244
Satisfiable
(Un)Satisfiability was proved
Number of variables280
Number of clauses672
Sum of the clauses size1680
Maximum clause length5
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2560
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 5112
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3371186-1305424586/watcher-3371186-1305424586 -o /tmp/evaluation-result-3371186-1305424586/solver-3371186-1305424586 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3371186-1305424586.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.85 7.76 7.25 3/349 23782
/proc/meminfo: memFree=24066944/32950928 swapFree=45937192/67111528
[pid=23780] ppid=23777 vsize=115632 CPUtime=0.01 cores=0,2,4,6
/proc/23780/stat : 23780 (plingeling) S 23777 23780 23744 0 -1 4202496 390 1252 0 0 1 0 0 0 22 0 5 0 654055308 118407168 315 33554432000 4194304 4417452 140733276160528 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23780/statm: 28908 315 176 55 0 26794 0
[pid=23780/tid=23799] ppid=23777 vsize=115632 CPUtime=0 cores=0,2,4,6
/proc/23780/task/23799/stat : 23799 (plingeling) R 23777 23780 23744 0 -1 4202560 37 1252 0 0 0 0 0 0 24 0 5 0 654055309 118407168 315 33554432000 4194304 4417452 140733276160528 18446744073709551615 4357283 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23780/tid=23800] ppid=23777 vsize=115632 CPUtime=0 cores=0,2,4,6
/proc/23780/task/23800/stat : 23800 (plingeling) R 23777 23780 23744 0 -1 4202560 0 1252 0 0 0 0 0 0 24 0 5 0 654055309 118407168 315 33554432000 4194304 4417452 140733276160528 18446744073709551615 254500748273 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23780/tid=23801] ppid=23777 vsize=115632 CPUtime=0 cores=0,2,4,6
/proc/23780/task/23801/stat : 23801 (plingeling) R 23777 23780 23744 0 -1 4202560 1 1252 0 0 0 0 0 0 24 0 5 0 654055309 118407168 315 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364437 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23780/tid=23802] ppid=23777 vsize=115632 CPUtime=0 cores=0,2,4,6
/proc/23780/task/23802/stat : 23802 (plingeling) R 23777 23780 23744 0 -1 4202560 33 1252 0 0 0 0 0 0 24 0 5 0 654055309 118407168 315 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364807 0 0 4096 17442 0 0 0 -1 0 0 0 0

[startup+0.066471 s]
/proc/loadavg: 6.85 7.76 7.25 3/349 23782
/proc/meminfo: memFree=24066944/32950928 swapFree=45937192/67111528
[pid=23780] ppid=23777 vsize=117400 CPUtime=0.22 cores=0,2,4,6
/proc/23780/stat : 23780 (plingeling) S 23777 23780 23744 0 -1 4202496 899 1252 0 0 22 0 0 0 22 0 5 0 654055308 120217600 794 33554432000 4194304 4417452 140733276160528 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23780/statm: 29350 794 180 55 0 27236 0
[pid=23780/tid=23799] ppid=23777 vsize=117400 CPUtime=0.05 cores=0,2,4,6
/proc/23780/task/23799/stat : 23799 (plingeling) R 23777 23780 23744 0 -1 4202560 108 1252 0 0 5 0 0 0 24 0 5 0 654055309 120217600 794 33554432000 4194304 4417452 140733276160528 18446744073709551615 4252176 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23780/tid=23800] ppid=23777 vsize=117400 CPUtime=0.05 cores=0,2,4,6
/proc/23780/task/23800/stat : 23800 (plingeling) R 23777 23780 23744 0 -1 4202560 70 1252 0 0 5 0 0 0 25 0 5 0 654055309 120217600 794 33554432000 4194304 4417452 140733276160528 18446744073709551615 4310483 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23780/tid=23801] ppid=23777 vsize=117400 CPUtime=0.05 cores=0,2,4,6
/proc/23780/task/23801/stat : 23801 (plingeling) R 23777 23780 23744 0 -1 4202560 70 1252 0 0 5 0 0 0 25 0 5 0 654055309 120217600 794 33554432000 4194304 4417452 140733276160528 18446744073709551615 4251272 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23780/tid=23802] ppid=23777 vsize=117400 CPUtime=0.05 cores=0,2,4,6
/proc/23780/task/23802/stat : 23802 (plingeling) R 23777 23780 23744 0 -1 4202560 332 1252 0 0 5 0 0 0 24 0 5 0 654055309 120217600 794 33554432000 4194304 4417452 140733276160528 18446744073709551615 4356889 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 0.22
Current children cumulated vsize (KiB) 117400

[startup+0.100463 s]
/proc/loadavg: 6.85 7.76 7.25 3/349 23782
/proc/meminfo: memFree=24066944/32950928 swapFree=45937192/67111528
[pid=23780] ppid=23777 vsize=117900 CPUtime=0.35 cores=0,2,4,6
/proc/23780/stat : 23780 (plingeling) S 23777 23780 23744 0 -1 4202496 1095 1252 0 0 35 0 0 0 22 0 5 0 654055308 120729600 897 33554432000 4194304 4417452 140733276160528 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23780/statm: 29475 897 180 55 0 27361 0
[pid=23780/tid=23799] ppid=23777 vsize=117900 CPUtime=0.09 cores=0,2,4,6
/proc/23780/task/23799/stat : 23799 (plingeling) R 23777 23780 23744 0 -1 4202560 163 1252 0 0 9 0 0 0 25 0 5 0 654055309 120729600 897 33554432000 4194304 4417452 140733276160528 18446744073709551615 4309925 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23780/tid=23800] ppid=23777 vsize=117900 CPUtime=0.08 cores=0,2,4,6
/proc/23780/task/23800/stat : 23800 (plingeling) R 23777 23780 23744 0 -1 4202560 70 1252 0 0 8 0 0 0 25 0 5 0 654055309 120729600 897 33554432000 4194304 4417452 140733276160528 18446744073709551615 4366099 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23780/tid=23801] ppid=23777 vsize=117900 CPUtime=0.08 cores=0,2,4,6
/proc/23780/task/23801/stat : 23801 (plingeling) R 23777 23780 23744 0 -1 4202560 145 1252 0 0 8 0 0 0 25 0 5 0 654055309 120729600 897 33554432000 4194304 4417452 140733276160528 18446744073709551615 4308810 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23780/tid=23802] ppid=23777 vsize=117900 CPUtime=0.09 cores=0,2,4,6
/proc/23780/task/23802/stat : 23802 (plingeling) R 23777 23780 23744 0 -1 4202560 398 1252 0 0 9 0 0 0 24 0 5 0 654055309 120729600 897 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 0.35
Current children cumulated vsize (KiB) 117900

[startup+0.300425 s]
/proc/loadavg: 6.85 7.76 7.25 3/349 23782
/proc/meminfo: memFree=24066944/32950928 swapFree=45937192/67111528
[pid=23780] ppid=23777 vsize=121804 CPUtime=1.15 cores=0,2,4,6
/proc/23780/stat : 23780 (plingeling) S 23777 23780 23744 0 -1 4202496 2692 1252 0 0 115 0 0 0 22 0 5 0 654055308 124727296 1888 33554432000 4194304 4417452 140733276160528 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23780/statm: 30451 1888 180 55 0 28337 0
[pid=23780/tid=23799] ppid=23777 vsize=121804 CPUtime=0.29 cores=0,2,4,6
/proc/23780/task/23799/stat : 23799 (plingeling) R 23777 23780 23744 0 -1 4202560 441 1252 0 0 29 0 0 0 25 0 5 0 654055309 124727296 1888 33554432000 4194304 4417452 140733276160528 18446744073709551615 4308025 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23780/tid=23800] ppid=23777 vsize=121804 CPUtime=0.28 cores=0,2,4,6
/proc/23780/task/23800/stat : 23800 (plingeling) R 23777 23780 23744 0 -1 4202560 401 1252 0 0 28 0 0 0 25 0 5 0 654055309 124727296 1888 33554432000 4194304 4417452 140733276160528 18446744073709551615 4309183 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23780/tid=23801] ppid=23777 vsize=121804 CPUtime=0.28 cores=0,2,4,6
/proc/23780/task/23801/stat : 23801 (plingeling) R 23777 23780 23744 0 -1 4202560 429 1252 0 0 28 0 0 0 25 0 5 0 654055309 124727296 1888 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23780/tid=23802] ppid=23777 vsize=121804 CPUtime=0.28 cores=0,2,4,6
/proc/23780/task/23802/stat : 23802 (plingeling) R 23777 23780 23744 0 -1 4202560 1102 1252 0 0 28 0 0 0 25 0 5 0 654055309 124727296 1888 33554432000 4194304 4417452 140733276160528 18446744073709551615 4250502 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1.15
Current children cumulated vsize (KiB) 121804

[startup+0.700347 s]
/proc/loadavg: 6.85 7.76 7.25 3/349 23782
/proc/meminfo: memFree=24066944/32950928 swapFree=45937192/67111528
[pid=23780] ppid=23777 vsize=123148 CPUtime=2.75 cores=0,2,4,6
/proc/23780/stat : 23780 (plingeling) S 23777 23780 23744 0 -1 4202496 6274 1252 0 0 274 1 0 0 22 0 5 0 654055308 126103552 2246 33554432000 4194304 4417452 140733276160528 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23780/statm: 30787 2246 180 55 0 28673 0
[pid=23780/tid=23799] ppid=23777 vsize=123148 CPUtime=0.68 cores=0,2,4,6
/proc/23780/task/23799/stat : 23799 (plingeling) R 23777 23780 23744 0 -1 4202560 1258 1252 0 0 68 0 0 0 25 0 5 0 654055309 126103552 2246 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364896 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23780/tid=23800] ppid=23777 vsize=123148 CPUtime=0.68 cores=0,2,4,6
/proc/23780/task/23800/stat : 23800 (plingeling) R 23777 23780 23744 0 -1 4202560 1206 1252 0 0 68 0 0 0 25 0 5 0 654055309 126103552 2246 33554432000 4194304 4417452 140733276160528 18446744073709551615 4309907 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23780/tid=23801] ppid=23777 vsize=123148 CPUtime=0.68 cores=0,2,4,6
/proc/23780/task/23801/stat : 23801 (plingeling) R 23777 23780 23744 0 -1 4202560 1334 1252 0 0 68 0 0 0 25 0 5 0 654055309 126103552 2246 33554432000 4194304 4417452 140733276160528 18446744073709551615 4353512 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23780/tid=23802] ppid=23777 vsize=123148 CPUtime=0.68 cores=0,2,4,6
/proc/23780/task/23802/stat : 23802 (plingeling) R 23777 23780 23744 0 -1 4202560 2157 1252 0 0 68 0 0 0 25 0 5 0 654055309 126103552 2246 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2.75
Current children cumulated vsize (KiB) 123148

[startup+1.50019 s]
/proc/loadavg: 6.94 7.77 7.25 10/359 23805
/proc/meminfo: memFree=24046536/32950928 swapFree=45937192/67111528
[pid=23780] ppid=23777 vsize=126324 CPUtime=5.94 cores=0,2,4,6
/proc/23780/stat : 23780 (plingeling) S 23777 23780 23744 0 -1 4202496 13810 1252 0 0 592 2 0 0 22 0 5 0 654055308 129355776 3040 33554432000 4194304 4417452 140733276160528 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23780/statm: 31581 3040 180 55 0 29467 0
[pid=23780/tid=23799] ppid=23777 vsize=126324 CPUtime=1.48 cores=0,2,4,6
/proc/23780/task/23799/stat : 23799 (plingeling) R 23777 23780 23744 0 -1 4202560 3015 1252 0 0 148 0 0 0 25 0 5 0 654055309 129355776 3040 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23780/tid=23800] ppid=23777 vsize=126324 CPUtime=1.47 cores=0,2,4,6
/proc/23780/task/23800/stat : 23800 (plingeling) R 23777 23780 23744 0 -1 4202560 3458 1252 0 0 147 0 0 0 25 0 5 0 654055309 129355776 3040 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23780/tid=23801] ppid=23777 vsize=126324 CPUtime=1.47 cores=0,2,4,6
/proc/23780/task/23801/stat : 23801 (plingeling) R 23777 23780 23744 0 -1 4202560 3151 1252 0 0 147 0 0 0 25 0 5 0 654055309 129355776 3040 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23780/tid=23802] ppid=23777 vsize=126324 CPUtime=1.48 cores=0,2,4,6
/proc/23780/task/23802/stat : 23802 (plingeling) R 23777 23780 23744 0 -1 4202560 3867 1252 0 0 148 0 0 0 25 0 5 0 654055309 129355776 3040 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 5.94
Current children cumulated vsize (KiB) 126324

[startup+3.10087 s]
/proc/loadavg: 6.94 7.77 7.25 10/359 23805
/proc/meminfo: memFree=24037232/32950928 swapFree=45937192/67111528
[pid=23780] ppid=23777 vsize=129940 CPUtime=12.32 cores=0,2,4,6
/proc/23780/stat : 23780 (plingeling) S 23777 23780 23744 0 -1 4202496 28587 1252 0 0 1228 4 0 0 22 0 5 0 654055308 133058560 3934 33554432000 4194304 4417452 140733276160528 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23780/statm: 32485 3934 180 55 0 30371 0
[pid=23780/tid=23799] ppid=23777 vsize=129940 CPUtime=3.08 cores=0,2,4,6
/proc/23780/task/23799/stat : 23799 (plingeling) R 23777 23780 23744 0 -1 4202560 7230 1252 0 0 307 1 0 0 25 0 5 0 654055309 133058560 3934 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364741 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23780/tid=23800] ppid=23777 vsize=129940 CPUtime=3.08 cores=0,2,4,6
/proc/23780/task/23800/stat : 23800 (plingeling) R 23777 23780 23744 0 -1 4202560 7702 1252 0 0 307 1 0 0 25 0 5 0 654055309 133058560 3934 33554432000 4194304 4417452 140733276160528 18446744073709551615 4365027 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23780/tid=23801] ppid=23777 vsize=129940 CPUtime=3.05 cores=0,2,4,6
/proc/23780/task/23801/stat : 23801 (plingeling) R 23777 23780 23744 0 -1 4202560 7312 1252 0 0 305 0 0 0 25 0 5 0 654055309 133058560 3934 33554432000 4194304 4417452 140733276160528 18446744073709551615 4356856 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23780/tid=23802] ppid=23777 vsize=129940 CPUtime=3.08 cores=0,2,4,6
/proc/23780/task/23802/stat : 23802 (plingeling) R 23777 23780 23744 0 -1 4202560 6024 1252 0 0 307 1 0 0 25 0 5 0 654055309 133058560 3934 33554432000 4194304 4417452 140733276160528 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 12.32
Current children cumulated vsize (KiB) 129940

[startup+6.30023 s]
/proc/loadavg: 6.94 7.77 7.25 10/359 23805
/proc/meminfo: memFree=24025460/32950928 swapFree=45937192/67111528
[pid=23780] ppid=23777 vsize=135300 CPUtime=25.08 cores=0,2,4,6
/proc/23780/stat : 23780 (plingeling) S 23777 23780 23744 0 -1 4202496 56452 1252 0 0 2500 8 0 0 22 0 5 0 654055308 138547200 5283 33554432000 4194304 4417452 140733276160528 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23780/statm: 33825 5283 180 55 0 31711 0
[pid=23780/tid=23799] ppid=23777 vsize=135300 CPUtime=6.28 cores=0,2,4,6
/proc/23780/task/23799/stat : 23799 (plingeling) R 23777 23780 23744 0 -1 4202560 15223 1252 0 0 626 2 0 0 25 0 5 0 654055309 138547200 5283 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23780/tid=23800] ppid=23777 vsize=135300 CPUtime=6.28 cores=0,2,4,6
/proc/23780/task/23800/stat : 23800 (plingeling) R 23777 23780 23744 0 -1 4202560 14597 1252 0 0 626 2 0 0 25 0 5 0 654055309 138547200 5283 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364650 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23780/tid=23801] ppid=23777 vsize=135300 CPUtime=6.22 cores=0,2,4,6
/proc/23780/task/23801/stat : 23801 (plingeling) R 23777 23780 23744 0 -1 4202560 14818 1252 0 0 621 1 0 0 25 0 5 0 654055309 138547200 5283 33554432000 4194304 4417452 140733276160528 18446744073709551615 4365027 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23780/tid=23802] ppid=23777 vsize=135300 CPUtime=6.28 cores=0,2,4,6
/proc/23780/task/23802/stat : 23802 (plingeling) R 23777 23780 23744 0 -1 4202560 11495 1252 0 0 626 2 0 0 25 0 5 0 654055309 138547200 5283 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 25.08
Current children cumulated vsize (KiB) 135300

[startup+12.701 s]
/proc/loadavg: 7.11 7.77 7.26 10/359 23805
/proc/meminfo: memFree=24003596/32950928 swapFree=45937192/67111528
[pid=23780] ppid=23777 vsize=142628 CPUtime=50.61 cores=0,2,4,6
/proc/23780/stat : 23780 (plingeling) S 23777 23780 23744 0 -1 4202496 98070 1252 0 0 5047 14 0 0 22 0 5 0 654055308 146051072 7093 33554432000 4194304 4417452 140733276160528 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23780/statm: 35657 7093 180 55 0 33543 0
[pid=23780/tid=23799] ppid=23777 vsize=142628 CPUtime=12.68 cores=0,2,4,6
/proc/23780/task/23799/stat : 23799 (plingeling) R 23777 23780 23744 0 -1 4202560 26696 1252 0 0 1265 3 0 0 25 0 5 0 654055309 146051072 7093 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23780/tid=23800] ppid=23777 vsize=142628 CPUtime=12.68 cores=0,2,4,6
/proc/23780/task/23800/stat : 23800 (plingeling) R 23777 23780 23744 0 -1 4202560 27951 1252 0 0 1264 4 0 0 25 0 5 0 654055309 146051072 7093 33554432000 4194304 4417452 140733276160528 18446744073709551615 4237276 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23780/tid=23801] ppid=23777 vsize=142628 CPUtime=12.53 cores=0,2,4,6
/proc/23780/task/23801/stat : 23801 (plingeling) R 23777 23780 23744 0 -1 4202560 26015 1252 0 0 1251 2 0 0 25 0 5 0 654055309 146051072 7093 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23780/tid=23802] ppid=23777 vsize=142628 CPUtime=12.69 cores=0,2,4,6
/proc/23780/task/23802/stat : 23802 (plingeling) R 23777 23780 23744 0 -1 4202560 17089 1252 0 0 1266 3 0 0 25 0 5 0 654055309 146051072 7093 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 50.61
Current children cumulated vsize (KiB) 142628

[startup+25.5004 s]
/proc/loadavg: 7.24 7.78 7.27 10/359 23805
/proc/meminfo: memFree=23995360/32950928 swapFree=45937192/67111528
[pid=23780] ppid=23777 vsize=151740 CPUtime=101.65 cores=0,2,4,6
/proc/23780/stat : 23780 (plingeling) S 23777 23780 23744 0 -1 4202496 175404 1252 0 0 10139 26 0 0 22 0 5 0 654055308 155381760 9417 33554432000 4194304 4417452 140733276160528 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23780/statm: 37935 9417 180 55 0 35821 0
[pid=23780/tid=23799] ppid=23777 vsize=151740 CPUtime=25.49 cores=0,2,4,6
/proc/23780/task/23799/stat : 23799 (plingeling) R 23777 23780 23744 0 -1 4202560 51971 1252 0 0 2542 7 0 0 25 0 5 0 654055309 155381760 9417 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23780/tid=23800] ppid=23777 vsize=151740 CPUtime=25.48 cores=0,2,4,6
/proc/23780/task/23800/stat : 23800 (plingeling) R 23777 23780 23744 0 -1 4202560 48699 1252 0 0 2540 8 0 0 25 0 5 0 654055309 155381760 9417 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364780 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23780/tid=23801] ppid=23777 vsize=151740 CPUtime=25.19 cores=0,2,4,6
/proc/23780/task/23801/stat : 23801 (plingeling) R 23777 23780 23744 0 -1 4202560 46846 1252 0 0 2513 6 0 0 25 0 5 0 654055309 155381760 9417 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23780/tid=23802] ppid=23777 vsize=151740 CPUtime=25.48 cores=0,2,4,6
/proc/23780/task/23802/stat : 23802 (plingeling) R 23777 23780 23744 0 -1 4202560 27569 1252 0 0 2544 4 0 0 25 0 5 0 654055309 155381760 9417 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 101.65
Current children cumulated vsize (KiB) 151740

[startup+51.1003 s]
/proc/loadavg: 7.62 7.83 7.29 10/359 23806
/proc/meminfo: memFree=23957264/32950928 swapFree=45937192/67111528
[pid=23780] ppid=23777 vsize=168252 CPUtime=203.75 cores=0,2,4,6
/proc/23780/stat : 23780 (plingeling) S 23777 23780 23744 0 -1 4202496 393219 1252 0 0 20317 58 0 0 22 0 5 0 654055308 172290048 13516 33554432000 4194304 4417452 140733276160528 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23780/statm: 42063 13516 180 55 0 39949 0
[pid=23780/tid=23799] ppid=23777 vsize=168252 CPUtime=51.08 cores=0,2,4,6
/proc/23780/task/23799/stat : 23799 (plingeling) R 23777 23780 23744 0 -1 4202560 115703 1252 0 0 5092 16 0 0 25 0 5 0 654055309 172290048 13516 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23780/tid=23800] ppid=23777 vsize=168252 CPUtime=51.06 cores=0,2,4,6
/proc/23780/task/23800/stat : 23800 (plingeling) R 23777 23780 23744 0 -1 4202560 113442 1252 0 0 5090 16 0 0 25 0 5 0 654055309 172290048 13516 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23780/tid=23801] ppid=23777 vsize=168252 CPUtime=50.48 cores=0,2,4,6

################
# More data... #
################

[pid=23780/tid=23800] ppid=23777 vsize=212628 CPUtime=702.16 cores=0,2,4,6
/proc/23780/task/23800/stat : 23800 (plingeling) R 23777 23780 23744 0 -1 4202560 428943 1252 0 0 70150 66 0 0 25 0 5 0 654055309 217731072 24639 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23780/tid=23801] ppid=23777 vsize=212628 CPUtime=693.97 cores=0,2,4,6
/proc/23780/task/23801/stat : 23801 (plingeling) R 23777 23780 23744 0 -1 4202560 358484 1252 0 0 69341 56 0 0 25 0 5 0 654055309 217731072 24639 33554432000 4194304 4417452 140733276160528 18446744073709551615 4259603 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23780/tid=23802] ppid=23777 vsize=212628 CPUtime=701.22 cores=0,2,4,6
/proc/23780/task/23802/stat : 23802 (plingeling) R 23777 23780 23744 0 -1 4202560 288445 1252 0 0 70076 46 0 0 25 0 5 0 654055309 217731072 24639 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2799.53
Current children cumulated vsize (KiB) 212628

[startup+762.314 s]
/proc/loadavg: 8.21 8.16 7.76 10/359 24310
/proc/meminfo: memFree=23821696/32950928 swapFree=45937192/67111528
[pid=23780] ppid=23777 vsize=257808 CPUtime=3038.86 cores=0,2,4,6
/proc/23780/stat : 23780 (plingeling) S 23777 23780 23744 0 -1 4202496 1508066 1252 0 0 303653 233 0 0 22 0 5 0 654055308 263995392 35907 33554432000 4194304 4417452 140733276160528 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23780/statm: 64452 35907 180 55 0 62338 0
[pid=23780/tid=23799] ppid=23777 vsize=257808 CPUtime=762.17 cores=0,2,4,6
/proc/23780/task/23799/stat : 23799 (plingeling) R 23777 23780 23744 0 -1 4202560 382492 1252 0 0 76160 57 0 0 25 0 5 0 654055309 263995392 35907 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23780/tid=23800] ppid=23777 vsize=257808 CPUtime=762.15 cores=0,2,4,6
/proc/23780/task/23800/stat : 23800 (plingeling) R 23777 23780 23744 0 -1 4202560 445587 1252 0 0 76147 68 0 0 25 0 5 0 654055309 263995392 35907 33554432000 4194304 4417452 140733276160528 18446744073709551615 4366105 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23780/tid=23801] ppid=23777 vsize=257808 CPUtime=753.28 cores=0,2,4,6
/proc/23780/task/23801/stat : 23801 (plingeling) R 23777 23780 23744 0 -1 4202560 376112 1252 0 0 75270 58 0 0 25 0 5 0 654055309 263995392 35907 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23780/tid=23802] ppid=23777 vsize=257808 CPUtime=761.24 cores=0,2,4,6
/proc/23780/task/23802/stat : 23802 (plingeling) R 23777 23780 23744 0 -1 4202560 303556 1252 0 0 76075 49 0 0 25 0 5 0 654055309 263995392 35907 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3038.86
Current children cumulated vsize (KiB) 257808

[startup+822.301 s]
/proc/loadavg: 8.16 8.14 7.78 10/359 24311
/proc/meminfo: memFree=23843240/32950928 swapFree=45937192/67111528
[pid=23780] ppid=23777 vsize=238284 CPUtime=3278.11 cores=0,2,4,6
/proc/23780/stat : 23780 (plingeling) S 23777 23780 23744 0 -1 4202496 1560073 1252 0 0 327570 241 0 0 22 0 5 0 654055308 244002816 31024 33554432000 4194304 4417452 140733276160528 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23780/statm: 59571 31024 180 55 0 57457 0
[pid=23780/tid=23799] ppid=23777 vsize=238284 CPUtime=822.15 cores=0,2,4,6
/proc/23780/task/23799/stat : 23799 (plingeling) R 23777 23780 23744 0 -1 4202560 391742 1252 0 0 82157 58 0 0 25 0 5 0 654055309 244002816 31024 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364741 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23780/tid=23800] ppid=23777 vsize=238284 CPUtime=822.13 cores=0,2,4,6
/proc/23780/task/23800/stat : 23800 (plingeling) R 23777 23780 23744 0 -1 4202560 459188 1252 0 0 82143 70 0 0 25 0 5 0 654055309 244002816 31024 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23780/tid=23801] ppid=23777 vsize=238284 CPUtime=812.56 cores=0,2,4,6
/proc/23780/task/23801/stat : 23801 (plingeling) R 23777 23780 23744 0 -1 4202560 387118 1252 0 0 81197 59 0 0 25 0 5 0 654055309 244002816 31024 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23780/tid=23802] ppid=23777 vsize=238284 CPUtime=821.23 cores=0,2,4,6
/proc/23780/task/23802/stat : 23802 (plingeling) R 23777 23780 23744 0 -1 4202560 321706 1252 0 0 82072 51 0 0 25 0 5 0 654055309 244002816 31024 33554432000 4194304 4417452 140733276160528 18446744073709551615 4250502 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3278.11
Current children cumulated vsize (KiB) 238284

[startup+882.301 s]
/proc/loadavg: 8.13 8.13 7.79 10/359 24313
/proc/meminfo: memFree=23837392/32950928 swapFree=45937192/67111528
[pid=23780] ppid=23777 vsize=237356 CPUtime=3517.39 cores=0,2,4,6
/proc/23780/stat : 23780 (plingeling) S 23777 23780 23744 0 -1 4202496 1612336 1252 0 0 351491 248 0 0 22 0 5 0 654055308 243052544 30792 33554432000 4194304 4417452 140733276160528 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23780/statm: 59339 30792 180 55 0 57225 0
[pid=23780/tid=23799] ppid=23777 vsize=237356 CPUtime=882.15 cores=0,2,4,6
/proc/23780/task/23799/stat : 23799 (plingeling) R 23777 23780 23744 0 -1 4202560 405193 1252 0 0 88155 60 0 0 25 0 5 0 654055309 243052544 30792 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23780/tid=23800] ppid=23777 vsize=237356 CPUtime=882.13 cores=0,2,4,6
/proc/23780/task/23800/stat : 23800 (plingeling) R 23777 23780 23744 0 -1 4202560 474640 1252 0 0 88141 72 0 0 25 0 5 0 654055309 243052544 30792 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23780/tid=23801] ppid=23777 vsize=237356 CPUtime=871.86 cores=0,2,4,6
/proc/23780/task/23801/stat : 23801 (plingeling) R 23777 23780 23744 0 -1 4202560 394399 1252 0 0 87125 61 0 0 25 0 5 0 654055309 243052544 30792 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23780/tid=23802] ppid=23777 vsize=237356 CPUtime=881.23 cores=0,2,4,6
/proc/23780/task/23802/stat : 23802 (plingeling) R 23777 23780 23744 0 -1 4202560 337785 1252 0 0 88069 54 0 0 25 0 5 0 654055309 243052544 30792 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364716 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3517.39
Current children cumulated vsize (KiB) 237356

[startup+942.301 s]
/proc/loadavg: 8.11 8.12 7.81 10/359 24315
/proc/meminfo: memFree=23825960/32950928 swapFree=45937192/67111528
[pid=23780] ppid=23777 vsize=243896 CPUtime=3756.68 cores=0,2,4,6
/proc/23780/stat : 23780 (plingeling) S 23777 23780 23744 0 -1 4202496 1657870 1252 0 0 375413 255 0 0 22 0 5 0 654055308 249749504 32440 33554432000 4194304 4417452 140733276160528 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23780/statm: 60974 32440 180 55 0 58860 0
[pid=23780/tid=23799] ppid=23777 vsize=243896 CPUtime=942.15 cores=0,2,4,6
/proc/23780/task/23799/stat : 23799 (plingeling) R 23777 23780 23744 0 -1 4202560 414565 1252 0 0 94154 61 0 0 25 0 5 0 654055309 249749504 32440 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23780/tid=23800] ppid=23777 vsize=243896 CPUtime=942.11 cores=0,2,4,6
/proc/23780/task/23800/stat : 23800 (plingeling) R 23777 23780 23744 0 -1 4202560 483577 1252 0 0 94138 73 0 0 25 0 5 0 654055309 249749504 32440 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23780/tid=23801] ppid=23777 vsize=243896 CPUtime=931.16 cores=0,2,4,6
/proc/23780/task/23801/stat : 23801 (plingeling) R 23777 23780 23744 0 -1 4202560 406195 1252 0 0 93053 63 0 0 25 0 5 0 654055309 249749504 32440 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23780/tid=23802] ppid=23777 vsize=243896 CPUtime=941.23 cores=0,2,4,6
/proc/23780/task/23802/stat : 23802 (plingeling) R 23777 23780 23744 0 -1 4202560 353214 1252 0 0 94067 56 0 0 25 0 5 0 654055309 249749504 32440 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3756.68
Current children cumulated vsize (KiB) 243896

[startup+1002.3 s]
/proc/loadavg: 8.16 8.13 7.83 10/359 24316
/proc/meminfo: memFree=23787744/32950928 swapFree=45937192/67111528
[pid=23780] ppid=23777 vsize=279912 CPUtime=3995.98 cores=0,2,4,6
/proc/23780/stat : 23780 (plingeling) S 23777 23780 23744 0 -1 4202496 1718185 1252 0 0 399332 266 0 0 22 0 5 0 654055308 286629888 41431 33554432000 4194304 4417452 140733276160528 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23780/statm: 69978 41431 180 55 0 67864 0
[pid=23780/tid=23799] ppid=23777 vsize=279912 CPUtime=1002.16 cores=0,2,4,6
/proc/23780/task/23799/stat : 23799 (plingeling) R 23777 23780 23744 0 -1 4202560 426849 1252 0 0 100152 64 0 0 25 0 5 0 654055309 286629888 41431 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23780/tid=23800] ppid=23777 vsize=279912 CPUtime=1002.1 cores=0,2,4,6
/proc/23780/task/23800/stat : 23800 (plingeling) R 23777 23780 23744 0 -1 4202560 498333 1252 0 0 100135 75 0 0 25 0 5 0 654055309 286629888 41431 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23780/tid=23801] ppid=23777 vsize=279912 CPUtime=990.45 cores=0,2,4,6
/proc/23780/task/23801/stat : 23801 (plingeling) R 23777 23780 23744 0 -1 4202560 421481 1252 0 0 98979 66 0 0 25 0 5 0 654055309 286629888 41431 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23780/tid=23802] ppid=23777 vsize=279912 CPUtime=1001.23 cores=0,2,4,6
/proc/23780/task/23802/stat : 23802 (plingeling) R 23777 23780 23744 0 -1 4202560 371203 1252 0 0 100064 59 0 0 25 0 5 0 654055309 286629888 41431 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3995.98
Current children cumulated vsize (KiB) 279912

[startup+1062.3 s]
/proc/loadavg: 8.17 8.13 7.85 10/359 24318
/proc/meminfo: memFree=23816996/32950928 swapFree=45937192/67111528
[pid=23780] ppid=23777 vsize=283088 CPUtime=4235.28 cores=0,2,4,6
/proc/23780/stat : 23780 (plingeling) S 23777 23780 23744 0 -1 4202496 1775145 1252 0 0 423255 273 0 0 22 0 5 0 654055308 289882112 42224 33554432000 4194304 4417452 140733276160528 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23780/statm: 70772 42224 180 55 0 68658 0
[pid=23780/tid=23799] ppid=23777 vsize=283088 CPUtime=1062.16 cores=0,2,4,6
/proc/23780/task/23799/stat : 23799 (plingeling) R 23777 23780 23744 0 -1 4202560 439439 1252 0 0 106151 65 0 0 25 0 5 0 654055309 289882112 42224 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23780/tid=23800] ppid=23777 vsize=283088 CPUtime=1062.1 cores=0,2,4,6
/proc/23780/task/23800/stat : 23800 (plingeling) R 23777 23780 23744 0 -1 4202560 509197 1252 0 0 106133 77 0 0 25 0 5 0 654055309 289882112 42224 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23780/tid=23801] ppid=23777 vsize=283088 CPUtime=1049.75 cores=0,2,4,6
/proc/23780/task/23801/stat : 23801 (plingeling) R 23777 23780 23744 0 -1 4202560 431707 1252 0 0 104908 67 0 0 25 0 5 0 654055309 289882112 42224 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23780/tid=23802] ppid=23777 vsize=283088 CPUtime=1061.24 cores=0,2,4,6
/proc/23780/task/23802/stat : 23802 (plingeling) R 23777 23780 23744 0 -1 4202560 394483 1252 0 0 106062 62 0 0 25 0 5 0 654055309 289882112 42224 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4235.28
Current children cumulated vsize (KiB) 283088

[startup+1122.3 s]
/proc/loadavg: 8.17 8.14 7.86 10/359 24319
/proc/meminfo: memFree=23780260/32950928 swapFree=45937192/67111528
[pid=23780] ppid=23777 vsize=249368 CPUtime=4474.56 cores=0,2,4,6
/proc/23780/stat : 23780 (plingeling) S 23777 23780 23744 0 -1 4202496 1832571 1252 0 0 447173 283 0 0 22 0 5 0 654055308 255352832 33815 33554432000 4194304 4417452 140733276160528 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23780/statm: 62342 33815 180 55 0 60228 0
[pid=23780/tid=23799] ppid=23777 vsize=249368 CPUtime=1122.16 cores=0,2,4,6
/proc/23780/task/23799/stat : 23799 (plingeling) R 23777 23780 23744 0 -1 4202560 462039 1252 0 0 112147 69 0 0 25 0 5 0 654055309 255352832 33815 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23780/tid=23800] ppid=23777 vsize=249368 CPUtime=1122.09 cores=0,2,4,6
/proc/23780/task/23800/stat : 23800 (plingeling) R 23777 23780 23744 0 -1 4202560 516434 1252 0 0 112130 79 0 0 25 0 5 0 654055309 255352832 33815 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23780/tid=23801] ppid=23777 vsize=249368 CPUtime=1109.04 cores=0,2,4,6
/proc/23780/task/23801/stat : 23801 (plingeling) R 23777 23780 23744 0 -1 4202560 441788 1252 0 0 110835 69 0 0 25 0 5 0 654055309 255352832 33815 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23780/tid=23802] ppid=23777 vsize=249368 CPUtime=1121.25 cores=0,2,4,6
/proc/23780/task/23802/stat : 23802 (plingeling) R 23777 23780 23744 0 -1 4202560 411991 1252 0 0 112060 65 0 0 25 0 5 0 654055309 255352832 33815 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4474.56
Current children cumulated vsize (KiB) 249368

[startup+1182.3 s]
/proc/loadavg: 8.11 8.12 7.88 9/359 24321
/proc/meminfo: memFree=23796480/32950928 swapFree=45937192/67111528
[pid=23780] ppid=23777 vsize=256952 CPUtime=4713.86 cores=0,2,4,6
/proc/23780/stat : 23780 (plingeling) S 23777 23780 23744 0 -1 4202496 1874786 1252 0 0 471095 291 0 0 22 0 5 0 654055308 263118848 35691 33554432000 4194304 4417452 140733276160528 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23780/statm: 64238 35691 180 55 0 62124 0
[pid=23780/tid=23799] ppid=23777 vsize=256952 CPUtime=1182.16 cores=0,2,4,6
/proc/23780/task/23799/stat : 23799 (plingeling) R 23777 23780 23744 0 -1 4202560 470505 1252 0 0 118146 70 0 0 25 0 5 0 654055309 263118848 35691 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23780/tid=23800] ppid=23777 vsize=256952 CPUtime=1182.08 cores=0,2,4,6
/proc/23780/task/23800/stat : 23800 (plingeling) R 23777 23780 23744 0 -1 4202560 527025 1252 0 0 118128 80 0 0 25 0 5 0 654055309 263118848 35691 33554432000 4194304 4417452 140733276160528 18446744073709551615 4365027 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23780/tid=23801] ppid=23777 vsize=256952 CPUtime=1168.34 cores=0,2,4,6
/proc/23780/task/23801/stat : 23801 (plingeling) R 23777 23780 23744 0 -1 4202560 449571 1252 0 0 116763 71 0 0 25 0 5 0 654055309 263118848 35691 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23780/tid=23802] ppid=23777 vsize=256952 CPUtime=1181.24 cores=0,2,4,6
/proc/23780/task/23802/stat : 23802 (plingeling) R 23777 23780 23744 0 -1 4202560 427366 1252 0 0 118056 68 0 0 25 0 5 0 654055309 263118848 35691 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4713.86
Current children cumulated vsize (KiB) 256952



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1204 s]
/proc/loadavg: 8.08 8.11 7.88 5/354 24323
/proc/meminfo: memFree=23972704/32950928 swapFree=45937192/67111528
[pid=23780] ppid=23777 vsize=255668 CPUtime=4800.42 cores=0,2,4,6
/proc/23780/stat : 23780 (plingeling) S 23777 23780 23744 0 -1 4202496 1887898 1252 0 0 479749 293 0 0 22 0 5 0 654055308 261804032 35401 33554432000 4194304 4417452 140733276160528 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 2 0 0 0
/proc/23780/statm: 63917 35401 180 55 0 61803 0
[pid=23780/tid=23799] ppid=23777 vsize=255668 CPUtime=1203.87 cores=0,2,4,6
/proc/23780/task/23799/stat : 23799 (plingeling) R 23777 23780 23744 0 -1 4202560 471519 1252 0 0 120317 70 0 0 25 0 5 0 654055309 261804032 35401 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=23780/tid=23800] ppid=23777 vsize=255668 CPUtime=1203.79 cores=0,2,4,6
/proc/23780/task/23800/stat : 23800 (plingeling) R 23777 23780 23744 0 -1 4202560 535122 1252 0 0 120297 82 0 0 25 0 5 0 654055309 261804032 35401 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=23780/tid=23801] ppid=23777 vsize=255668 CPUtime=1189.78 cores=0,2,4,6
/proc/23780/task/23801/stat : 23801 (plingeling) R 23777 23780 23744 0 -1 4202560 451730 1252 0 0 118907 71 0 0 25 0 5 0 654055309 261804032 35401 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=23780/tid=23802] ppid=23777 vsize=255668 CPUtime=1202.96 cores=0,2,4,6
/proc/23780/task/23802/stat : 23802 (plingeling) R 23777 23780 23744 0 -1 4202560 429208 1252 0 0 120228 68 0 0 25 0 5 0 654055309 261804032 35401 33554432000 4194304 4417452 140733276160528 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.42
Current children cumulated vsize (KiB) 255668

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1204.03
CPU time (s): 4800.48
CPU user time (s): 4797.52
CPU system time (s): 2.95255
CPU usage (%): 398.702
Max. virtual memory (cumulated for all children) (KiB): 298652

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4797.52
system time used= 2.95255
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1889150
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 827
involuntary context switches= 45041

runsolver used 3.9224 second user time and 10.2964 second system time

The end

Launcher Data 

Begin job on node149 at 2011-05-15 03:56:26
IDJOB=3371186
IDBENCH=83149
IDSOLVER=1848
FILE ID=node149/3371186-1305424586
RUNJOBID= node149-1305424586-23762
PBS_JOBID= 13324600
Free space on /tmp= 73572 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/crafted/spence/sgen/sgen3-n280-s93756482-sat.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3371186-1305424586/watcher-3371186-1305424586 -o /tmp/evaluation-result-3371186-1305424586/solver-3371186-1305424586 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3371186-1305424586.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= badb3b4ac12b29ada7dd8ed96ed75c80
RANDOM SEED=1230116099

node149.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.73
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      24067612 kB
Buffers:        314792 kB
Cached:        3935412 kB
SwapCached:      19984 kB
Active:        6338976 kB
Inactive:      2248040 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      24067612 kB
SwapTotal:    67111528 kB
SwapFree:     45937192 kB
Dirty:             144 kB
Writeback:           0 kB
AnonPages:     4329560 kB
Mapped:          12972 kB
Slab:           157932 kB
PageTables:      72224 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 80187364 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73620 MiB
End job on node149 at 2011-05-15 04:16:32