Trace number 3393972

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40001.1 5010.66

General information on the benchmark

NameCRAFTED/edgematching/
compact/em_12_2_4_cmp.cnf
MD5SUM7ea97a36d5ab6c8a5aa99830d5253591
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark61.2837
Satisfiable
(Un)Satisfiability was proved
Number of variables12584
Number of clauses155712
Sum of the clauses size523920
Maximum clause length100
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 217404
Number of clauses of size 3103200
Number of clauses of size 417828
Number of clauses of size 517000
Number of clauses of size over 5280

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3393972-1306077232/watcher-3393972-1306077232 -o /tmp/evaluation-result-3393972-1306077232/solver-3393972-1306077232 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3393972-1306077232.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.04 8.05 8.00 2/462 4389
/proc/meminfo: memFree=12616524/32951124 swapFree=57414376/67111528
[pid=4389] ppid=4387 vsize=11048 CPUtime=0 cores=0-7
/proc/4389/stat : 4389 (plingeling) R 4387 4389 4363 0 -1 4202496 853 1255 0 0 0 0 0 0 23 0 1 0 797953721 11313152 782 33554432000 4194304 4417452 140734197376720 18446744073709551615 252457762571 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/4389/statm: 2762 782 146 55 0 648 0

[startup+0.112733 s]
/proc/loadavg: 8.04 8.05 8.00 2/462 4389
/proc/meminfo: memFree=12616524/32951124 swapFree=57414376/67111528
[pid=4389] ppid=4387 vsize=39776 CPUtime=0.1 cores=0-7
/proc/4389/stat : 4389 (plingeling) R 4387 4389 4363 0 -1 4202496 8135 1255 0 0 10 0 0 0 23 0 1 0 797953721 40730624 7968 33554432000 4194304 4417452 140734197376720 18446744073709551615 4323119 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/4389/statm: 9944 7968 149 55 0 7830 0
Current children cumulated CPU time (s) 0.1
Current children cumulated vsize (KiB) 39776

[startup+0.200715 s]
/proc/loadavg: 8.04 8.05 8.00 2/462 4389
/proc/meminfo: memFree=12616524/32951124 swapFree=57414376/67111528
[pid=4389] ppid=4387 vsize=56160 CPUtime=0.19 cores=0-7
/proc/4389/stat : 4389 (plingeling) R 4387 4389 4363 0 -1 4202496 12231 1255 0 0 18 1 0 0 23 0 1 0 797953721 57507840 12064 33554432000 4194304 4417452 140734197376720 18446744073709551615 4227769 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/4389/statm: 14040 12064 149 55 0 11926 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 56160

[startup+0.300715 s]
/proc/loadavg: 8.04 8.05 8.00 2/462 4389
/proc/meminfo: memFree=12616524/32951124 swapFree=57414376/67111528
[pid=4389] ppid=4387 vsize=93024 CPUtime=0.28 cores=0-7
/proc/4389/stat : 4389 (plingeling) R 4387 4389 4363 0 -1 4202496 21447 1255 0 0 27 1 0 0 24 0 1 0 797953721 95256576 21280 33554432000 4194304 4417452 140734197376720 18446744073709551615 4231552 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/4389/statm: 23256 21280 149 55 0 21142 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 93024

[startup+1.26148 s]
/proc/loadavg: 8.04 8.05 8.00 2/462 4389
/proc/meminfo: memFree=12616524/32951124 swapFree=57414376/67111528
[pid=4389] ppid=4387 vsize=321952 CPUtime=7.19 cores=0-7
/proc/4389/stat : 4389 (plingeling) S 4387 4389 4363 0 -1 4202496 42435 1255 0 0 714 5 0 0 25 0 9 0 797953721 329678848 25436 33554432000 4194304 4417452 140734197376720 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4389/statm: 80488 25436 180 55 0 78374 0
Current children cumulated CPU time (s) 7.19
Current children cumulated vsize (KiB) 321952

[startup+1.50044 s]
/proc/loadavg: 8.04 8.05 8.00 2/462 4389
/proc/meminfo: memFree=12616524/32951124 swapFree=57414376/67111528
[pid=4389] ppid=4387 vsize=326856 CPUtime=9.1 cores=0-7
/proc/4389/stat : 4389 (plingeling) S 4387 4389 4363 0 -1 4202496 43691 1255 0 0 905 5 0 0 25 0 9 0 797953721 334700544 26692 33554432000 4194304 4417452 140734197376720 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4389/statm: 81714 26692 180 55 0 79600 0
Current children cumulated CPU time (s) 9.1
Current children cumulated vsize (KiB) 326856

[startup+3.10114 s]
/proc/loadavg: 8.03 8.05 8.00 9/471 4403
/proc/meminfo: memFree=12504472/32951124 swapFree=57414376/67111528
[pid=4389] ppid=4387 vsize=332852 CPUtime=21.87 cores=0-7
/proc/4389/stat : 4389 (plingeling) S 4387 4389 4363 0 -1 4202496 50807 1255 0 0 2181 6 0 0 25 0 9 0 797953721 340840448 28191 33554432000 4194304 4417452 140734197376720 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4389/statm: 83213 28191 180 55 0 81099 0
[pid=4389/tid=4396] ppid=4387 vsize=332852 CPUtime=2.65 cores=0-7
/proc/4389/task/4396/stat : 4396 (plingeling) R 4387 4389 4363 0 -1 4202560 1992 1255 0 0 265 0 0 0 23 0 9 0 797953762 340840448 28191 33554432000 4194304 4417452 140734197376720 18446744073709551615 4265450 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4389/tid=4397] ppid=4387 vsize=332852 CPUtime=2.68 cores=0-7
/proc/4389/task/4397/stat : 4397 (plingeling) R 4387 4389 4363 0 -1 4202560 2584 1255 0 0 268 0 0 0 25 0 9 0 797953762 340840448 28191 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4389/tid=4398] ppid=4387 vsize=332852 CPUtime=2.68 cores=0-7
/proc/4389/task/4398/stat : 4398 (plingeling) R 4387 4389 4363 0 -1 4202560 2035 1255 0 0 268 0 0 0 23 0 9 0 797953762 340840448 28191 33554432000 4194304 4417452 140734197376720 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4389/tid=4399] ppid=4387 vsize=332852 CPUtime=2.68 cores=0-7
/proc/4389/task/4399/stat : 4399 (plingeling) R 4387 4389 4363 0 -1 4202560 12930 1255 0 0 267 1 0 0 25 0 9 0 797953762 340840448 28191 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364838 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4389/tid=4400] ppid=4387 vsize=332852 CPUtime=2.68 cores=0-7
/proc/4389/task/4400/stat : 4400 (plingeling) R 4387 4389 4363 0 -1 4202560 2648 1255 0 0 268 0 0 0 25 0 9 0 797953762 340840448 28191 33554432000 4194304 4417452 140734197376720 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4389/tid=4401] ppid=4387 vsize=332852 CPUtime=2.68 cores=0-7
/proc/4389/task/4401/stat : 4401 (plingeling) R 4387 4389 4363 0 -1 4202560 2940 1255 0 0 268 0 0 0 22 0 9 0 797953762 340840448 28191 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364925 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4389/tid=4402] ppid=4387 vsize=332852 CPUtime=2.68 cores=0-7
/proc/4389/task/4402/stat : 4402 (plingeling) R 4387 4389 4363 0 -1 4202560 2079 1255 0 0 268 0 0 0 22 0 9 0 797953763 340840448 28191 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4389/tid=4403] ppid=4387 vsize=332852 CPUtime=2.68 cores=0-7
/proc/4389/task/4403/stat : 4403 (plingeling) R 4387 4389 4363 0 -1 4202560 2136 1255 0 0 268 0 0 0 23 0 9 0 797953763 340840448 28191 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 21.87
Current children cumulated vsize (KiB) 332852

[startup+6.31755 s]
/proc/loadavg: 8.03 8.05 8.00 9/471 4403
/proc/meminfo: memFree=12472976/32951124 swapFree=57414376/67111528
[pid=4389] ppid=4387 vsize=362944 CPUtime=47.55 cores=0-7
/proc/4389/stat : 4389 (plingeling) S 4387 4389 4363 0 -1 4202496 68233 1255 0 0 4746 9 0 0 25 0 9 0 797953721 371654656 35714 33554432000 4194304 4417452 140734197376720 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4389/statm: 90736 35714 180 55 0 88622 0
[pid=4389/tid=4396] ppid=4387 vsize=362944 CPUtime=5.81 cores=0-7
/proc/4389/task/4396/stat : 4396 (plingeling) R 4387 4389 4363 0 -1 4202560 1992 1255 0 0 581 0 0 0 25 0 9 0 797953762 371654656 35714 33554432000 4194304 4417452 140734197376720 18446744073709551615 4232126 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4389/tid=4397] ppid=4387 vsize=362944 CPUtime=5.9 cores=0-7
/proc/4389/task/4397/stat : 4397 (plingeling) R 4387 4389 4363 0 -1 4202560 2934 1255 0 0 590 0 0 0 25 0 9 0 797953762 371654656 35714 33554432000 4194304 4417452 140734197376720 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4389/tid=4398] ppid=4387 vsize=362944 CPUtime=5.89 cores=0-7
/proc/4389/task/4398/stat : 4398 (plingeling) R 4387 4389 4363 0 -1 4202560 2035 1255 0 0 589 0 0 0 25 0 9 0 797953762 371654656 35714 33554432000 4194304 4417452 140734197376720 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4389/tid=4399] ppid=4387 vsize=362944 CPUtime=5.9 cores=0-7
/proc/4389/task/4399/stat : 4399 (plingeling) R 4387 4389 4363 0 -1 4202560 28278 1255 0 0 586 4 0 0 25 0 9 0 797953762 371654656 35714 33554432000 4194304 4417452 140734197376720 18446744073709551615 4357554 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4389/tid=4400] ppid=4387 vsize=362944 CPUtime=5.89 cores=0-7
/proc/4389/task/4400/stat : 4400 (plingeling) R 4387 4389 4363 0 -1 4202560 3032 1255 0 0 589 0 0 0 25 0 9 0 797953762 371654656 35714 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4389/tid=4401] ppid=4387 vsize=362944 CPUtime=5.89 cores=0-7
/proc/4389/task/4401/stat : 4401 (plingeling) R 4387 4389 4363 0 -1 4202560 4284 1255 0 0 589 0 0 0 25 0 9 0 797953762 371654656 35714 33554432000 4194304 4417452 140734197376720 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4389/tid=4402] ppid=4387 vsize=362944 CPUtime=5.9 cores=0-7
/proc/4389/task/4402/stat : 4402 (plingeling) R 4387 4389 4363 0 -1 4202560 2079 1255 0 0 590 0 0 0 25 0 9 0 797953763 371654656 35714 33554432000 4194304 4417452 140734197376720 18446744073709551615 4236928 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4389/tid=4403] ppid=4387 vsize=362944 CPUtime=5.89 cores=0-7
/proc/4389/task/4403/stat : 4403 (plingeling) R 4387 4389 4363 0 -1 4202560 2136 1255 0 0 589 0 0 0 25 0 9 0 797953763 371654656 35714 33554432000 4194304 4417452 140734197376720 18446744073709551615 4245661 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 47.55
Current children cumulated vsize (KiB) 362944

[startup+12.7004 s]
/proc/loadavg: 8.03 8.05 8.00 9/471 4403
/proc/meminfo: memFree=12450092/32951124 swapFree=57414376/67111528
[pid=4389] ppid=4387 vsize=371680 CPUtime=98.53 cores=0-7
/proc/4389/stat : 4389 (plingeling) S 4387 4389 4363 0 -1 4202496 92066 1255 0 0 9840 13 0 0 25 0 9 0 797953721 380600320 37898 33554432000 4194304 4417452 140734197376720 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4389/statm: 92920 37898 180 55 0 90806 0
[pid=4389/tid=4396] ppid=4387 vsize=371680 CPUtime=12.11 cores=0-7
/proc/4389/task/4396/stat : 4396 (plingeling) R 4387 4389 4363 0 -1 4202560 1992 1255 0 0 1211 0 0 0 25 0 9 0 797953762 380600320 37898 33554432000 4194304 4417452 140734197376720 18446744073709551615 4236800 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4389/tid=4397] ppid=4387 vsize=371680 CPUtime=12.28 cores=0-7
/proc/4389/task/4397/stat : 4397 (plingeling) R 4387 4389 4363 0 -1 4202560 3069 1255 0 0 1228 0 0 0 25 0 9 0 797953762 380600320 37898 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4389/tid=4398] ppid=4387 vsize=371680 CPUtime=12.28 cores=0-7
/proc/4389/task/4398/stat : 4398 (plingeling) R 4387 4389 4363 0 -1 4202560 2035 1255 0 0 1228 0 0 0 25 0 9 0 797953762 380600320 37898 33554432000 4194304 4417452 140734197376720 18446744073709551615 4299767 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4389/tid=4399] ppid=4387 vsize=371680 CPUtime=12.28 cores=0-7
/proc/4389/task/4399/stat : 4399 (plingeling) R 4387 4389 4363 0 -1 4202560 50737 1255 0 0 1221 7 0 0 25 0 9 0 797953762 380600320 37898 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4389/tid=4400] ppid=4387 vsize=371680 CPUtime=12.28 cores=0-7
/proc/4389/task/4400/stat : 4400 (plingeling) R 4387 4389 4363 0 -1 4202560 3533 1255 0 0 1228 0 0 0 25 0 9 0 797953762 380600320 37898 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4389/tid=4401] ppid=4387 vsize=371680 CPUtime=12.28 cores=0-7
/proc/4389/task/4401/stat : 4401 (plingeling) R 4387 4389 4363 0 -1 4202560 4796 1255 0 0 1228 0 0 0 25 0 9 0 797953762 380600320 37898 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364537 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4389/tid=4402] ppid=4387 vsize=371680 CPUtime=12.28 cores=0-7
/proc/4389/task/4402/stat : 4402 (plingeling) R 4387 4389 4363 0 -1 4202560 2079 1255 0 0 1228 0 0 0 25 0 9 0 797953763 380600320 37898 33554432000 4194304 4417452 140734197376720 18446744073709551615 4232787 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4389/tid=4403] ppid=4387 vsize=371680 CPUtime=12.28 cores=0-7
/proc/4389/task/4403/stat : 4403 (plingeling) R 4387 4389 4363 0 -1 4202560 2362 1255 0 0 1228 0 0 0 25 0 9 0 797953763 380600320 37898 33554432000 4194304 4417452 140734197376720 18446744073709551615 4232683 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 98.53
Current children cumulated vsize (KiB) 371680

[startup+25.501 s]
/proc/loadavg: 8.02 8.05 8.00 9/471 4404
/proc/meminfo: memFree=12454888/32951124 swapFree=57414376/67111528
[pid=4389] ppid=4387 vsize=404588 CPUtime=200.72 cores=0-7
/proc/4389/stat : 4389 (plingeling) S 4387 4389 4363 0 -1 4202496 124601 1255 0 0 20054 18 0 0 25 0 9 0 797953721 414298112 46125 33554432000 4194304 4417452 140734197376720 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4389/statm: 101147 46125 180 55 0 99033 0
[pid=4389/tid=4396] ppid=4387 vsize=404588 CPUtime=24.71 cores=0-7
/proc/4389/task/4396/stat : 4396 (plingeling) R 4387 4389 4363 0 -1 4202560 1992 1255 0 0 2471 0 0 0 25 0 9 0 797953762 414298112 46125 33554432000 4194304 4417452 140734197376720 18446744073709551615 4365973 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4389/tid=4397] ppid=4387 vsize=404588 CPUtime=25.08 cores=0-7
/proc/4389/task/4397/stat : 4397 (plingeling) R 4387 4389 4363 0 -1 4202560 3069 1255 0 0 2508 0 0 0 25 0 9 0 797953762 414298112 46125 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4389/tid=4398] ppid=4387 vsize=404588 CPUtime=25.08 cores=0-7
/proc/4389/task/4398/stat : 4398 (plingeling) R 4387 4389 4363 0 -1 4202560 2035 1255 0 0 2508 0 0 0 25 0 9 0 797953762 414298112 46125 33554432000 4194304 4417452 140734197376720 18446744073709551615 4232464 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4389/tid=4399] ppid=4387 vsize=404588 CPUtime=25.08 cores=0-7
/proc/4389/task/4399/stat : 4399 (plingeling) R 4387 4389 4363 0 -1 4202560 82415 1255 0 0 2496 12 0 0 25 0 9 0 797953762 414298112 46125 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364992 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4389/tid=4400] ppid=4387 vsize=404588 CPUtime=25.07 cores=0-7
/proc/4389/task/4400/stat : 4400 (plingeling) R 4387 4389 4363 0 -1 4202560 3533 1255 0 0 2507 0 0 0 25 0 9 0 797953762 414298112 46125 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364476 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4389/tid=4401] ppid=4387 vsize=404588 CPUtime=25.08 cores=0-7
/proc/4389/task/4401/stat : 4401 (plingeling) R 4387 4389 4363 0 -1 4202560 5653 1255 0 0 2508 0 0 0 25 0 9 0 797953762 414298112 46125 33554432000 4194304 4417452 140734197376720 18446744073709551615 4365538 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4389/tid=4402] ppid=4387 vsize=404588 CPUtime=25.08 cores=0-7
/proc/4389/task/4402/stat : 4402 (plingeling) R 4387 4389 4363 0 -1 4202560 2079 1255 0 0 2508 0 0 0 25 0 9 0 797953763 414298112 46125 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4389/tid=4403] ppid=4387 vsize=404588 CPUtime=25.08 cores=0-7
/proc/4389/task/4403/stat : 4403 (plingeling) R 4387 4389 4363 0 -1 4202560 2362 1255 0 0 2508 0 0 0 25 0 9 0 797953763 414298112 46125 33554432000 4194304 4417452 140734197376720 18446744073709551615 4232654 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 200.72
Current children cumulated vsize (KiB) 404588

[startup+51.1012 s]
/proc/loadavg: 8.08 8.06 8.00 9/471 4404
/proc/meminfo: memFree=12430244/32951124 swapFree=57414376/67111528
[pid=4389] ppid=4387 vsize=407992 CPUtime=405.14 cores=0-7
/proc/4389/stat : 4389 (plingeling) S 4387 4389 4363 0 -1 4202496 195934 1255 0 0 40485 29 0 0 25 0 9 0 797953721 417783808 46976 33554432000 4194304 4417452 140734197376720 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4389/statm: 101998 46976 180 55 0 99884 0
[pid=4389/tid=4396] ppid=4387 vsize=407992 CPUtime=49.92 cores=0-7
/proc/4389/task/4396/stat : 4396 (plingeling) R 4387 4389 4363 0 -1 4202560 2368 1255 0 0 4992 0 0 0 25 0 9 0 797953762 417783808 46976 33554432000 4194304 4417452 140734197376720 18446744073709551615 4366559 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4389/tid=4397] ppid=4387 vsize=407992 CPUtime=50.68 cores=0-7
/proc/4389/task/4397/stat : 4397 (plingeling) R 4387 4389 4363 0 -1 4202560 3069 1255 0 0 5068 0 0 0 25 0 9 0 797953762 417783808 46976 33554432000 4194304 4417452 140734197376720 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4389/tid=4398] ppid=4387 vsize=407992 CPUtime=50.68 cores=0-7
/proc/4389/task/4398/stat : 4398 (plingeling) R 4387 4389 4363 0 -1 4202560 3124 1255 0 0 5068 0 0 0 25 0 9 0 797953762 417783808 46976 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4389/tid=4399] ppid=4387 vsize=407992 CPUtime=50.68 cores=0-7
/proc/4389/task/4399/stat : 4399 (plingeling) R 4387 4389 4363 0 -1 4202560 152283 1255 0 0 5045 23 0 0 25 0 9 0 797953762 417783808 46976 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364653 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4389/tid=4400] ppid=4387 vsize=407992 CPUtime=50.67 cores=0-7
/proc/4389/task/4400/stat : 4400 (plingeling) R 4387 4389 4363 0 -1 4202560 3533 1255 0 0 5067 0 0 0 25 0 9 0 797953762 417783808 46976 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364542 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4389/tid=4401] ppid=4387 vsize=407992 CPUtime=50.68 cores=0-7
/proc/4389/task/4401/stat : 4401 (plingeling) R 4387 4389 4363 0 -1 4202560 5653 1255 0 0 5068 0 0 0 25 0 9 0 797953762 417783808 46976 33554432000 4194304 4417452 140734197376720 18446744073709551615 4365943 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4389/tid=4402] ppid=4387 vsize=407992 CPUtime=50.68 cores=0-7
/proc/4389/task/4402/stat : 4402 (plingeling) R 4387 4389 4363 0 -1 4202560 2079 1255 0 0 5068 0 0 0 25 0 9 0 797953763 417783808 46976 33554432000 4194304 4417452 140734197376720 18446744073709551615 4365996 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4389/tid=4403] ppid=4387 vsize=407992 CPUtime=50.68 cores=0-7
/proc/4389/task/4403/stat : 4403 (plingeling) R 4387 4389 4363 0 -1 4202560 2362 1255 0 0 5068 0 0 0 25 0 9 0 797953763 417783808 46976 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364537 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 405.14
Current children cumulated vsize (KiB) 407992

[startup+102.301 s]
/proc/loadavg: 8.09 8.06 8.00 9/471 4406

################
# More data... #
################


[startup+4722.3 s]
/proc/loadavg: 8.02 8.04 8.00 9/471 4541
/proc/meminfo: memFree=1576444/32951124 swapFree=57414376/67111528
[pid=4389] ppid=4387 vsize=11238728 CPUtime=37701.1 cores=0-7
/proc/4389/stat : 4389 (plingeling) S 4387 4389 4363 0 -1 4202496 11064100 1255 0 0 3767877 2234 0 0 25 0 9 0 797953721 11508457472 2754669 33554432000 4194304 4417452 140734197376720 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4389/statm: 2809682 2754669 180 55 0 2807568 0
[pid=4389/tid=4396] ppid=4387 vsize=11238728 CPUtime=4696.87 cores=0-7
/proc/4389/task/4396/stat : 4396 (plingeling) R 4387 4389 4363 0 -1 4202560 959746 1255 0 0 469508 179 0 0 25 0 9 0 797953762 11508457472 2754669 33554432000 4194304 4417452 140734197376720 18446744073709551615 4306216 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4389/tid=4397] ppid=4387 vsize=11238728 CPUtime=4721.33 cores=0-7
/proc/4389/task/4397/stat : 4397 (plingeling) R 4387 4389 4363 0 -1 4202560 945459 1255 0 0 471926 207 0 0 25 0 9 0 797953762 11508457472 2754669 33554432000 4194304 4417452 140734197376720 18446744073709551615 4232783 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4389/tid=4398] ppid=4387 vsize=11238728 CPUtime=4721.8 cores=0-7
/proc/4389/task/4398/stat : 4398 (plingeling) R 4387 4389 4363 0 -1 4202560 1279979 1255 0 0 471938 242 0 0 25 0 9 0 797953762 11508457472 2754669 33554432000 4194304 4417452 140734197376720 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4389/tid=4399] ppid=4387 vsize=11238728 CPUtime=4721.97 cores=0-7
/proc/4389/task/4399/stat : 4399 (plingeling) R 4387 4389 4363 0 -1 4202560 3994585 1255 0 0 471307 890 0 0 25 0 9 0 797953762 11508457472 2754669 33554432000 4194304 4417452 140734197376720 18446744073709551615 4357260 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4389/tid=4400] ppid=4387 vsize=11238728 CPUtime=4720.8 cores=0-7
/proc/4389/task/4400/stat : 4400 (plingeling) R 4387 4389 4363 0 -1 4202560 903656 1255 0 0 471901 179 0 0 25 0 9 0 797953762 11508457472 2754669 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4389/tid=4401] ppid=4387 vsize=11238728 CPUtime=4722 cores=0-7
/proc/4389/task/4401/stat : 4401 (plingeling) R 4387 4389 4363 0 -1 4202560 1307355 1255 0 0 471966 234 0 0 25 0 9 0 797953762 11508457472 2754669 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4389/tid=4402] ppid=4387 vsize=11238728 CPUtime=4673.87 cores=0-7
/proc/4389/task/4402/stat : 4402 (plingeling) R 4387 4389 4363 0 -1 4202560 837922 1255 0 0 467247 140 0 0 25 0 9 0 797953763 11508457472 2754669 33554432000 4194304 4417452 140734197376720 18446744073709551615 4251164 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4389/tid=4403] ppid=4387 vsize=11238728 CPUtime=4721.99 cores=0-7
/proc/4389/task/4403/stat : 4403 (plingeling) R 4387 4389 4363 0 -1 4202560 813935 1255 0 0 472043 156 0 0 25 0 9 0 797953763 11508457472 2754669 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 37701.1
Current children cumulated vsize (KiB) 11238728

[startup+4782.3 s]
/proc/loadavg: 8.05 8.05 8.00 9/471 4543
/proc/meminfo: memFree=3251656/32951124 swapFree=57414376/67111528
[pid=4389] ppid=4387 vsize=9567056 CPUtime=38180.1 cores=0-7
/proc/4389/stat : 4389 (plingeling) S 4387 4389 4363 0 -1 4202496 11298083 1255 0 0 3815729 2286 0 0 25 0 9 0 797953721 9796665344 2336752 33554432000 4194304 4417452 140734197376720 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4389/statm: 2391764 2336752 180 55 0 2389650 0
[pid=4389/tid=4396] ppid=4387 vsize=9567056 CPUtime=4756.88 cores=0-7
/proc/4389/task/4396/stat : 4396 (plingeling) R 4387 4389 4363 0 -1 4202560 1138104 1255 0 0 475472 216 0 0 25 0 9 0 797953762 9796665344 2336752 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4389/tid=4397] ppid=4387 vsize=9567056 CPUtime=4781.33 cores=0-7
/proc/4389/task/4397/stat : 4397 (plingeling) R 4387 4389 4363 0 -1 4202560 945459 1255 0 0 477926 207 0 0 25 0 9 0 797953762 9796665344 2336752 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4389/tid=4398] ppid=4387 vsize=9567056 CPUtime=4781.81 cores=0-7
/proc/4389/task/4398/stat : 4398 (plingeling) R 4387 4389 4363 0 -1 4202560 1285551 1255 0 0 477937 244 0 0 25 0 9 0 797953762 9796665344 2336752 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4389/tid=4399] ppid=4387 vsize=9567056 CPUtime=4781.97 cores=0-7
/proc/4389/task/4399/stat : 4399 (plingeling) R 4387 4389 4363 0 -1 4202560 3994585 1255 0 0 477307 890 0 0 25 0 9 0 797953762 9796665344 2336752 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4389/tid=4400] ppid=4387 vsize=9567056 CPUtime=4780.79 cores=0-7
/proc/4389/task/4400/stat : 4400 (plingeling) R 4387 4389 4363 0 -1 4202560 903656 1255 0 0 477900 179 0 0 25 0 9 0 797953762 9796665344 2336752 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4389/tid=4401] ppid=4387 vsize=9567056 CPUtime=4782.01 cores=0-7
/proc/4389/task/4401/stat : 4401 (plingeling) R 4387 4389 4363 0 -1 4202560 1339406 1255 0 0 477959 242 0 0 25 0 9 0 797953762 9796665344 2336752 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4389/tid=4402] ppid=4387 vsize=9567056 CPUtime=4732.95 cores=0-7
/proc/4389/task/4402/stat : 4402 (plingeling) R 4387 4389 4363 0 -1 4202560 837922 1255 0 0 473155 140 0 0 25 0 9 0 797953763 9796665344 2336752 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364807 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4389/tid=4403] ppid=4387 vsize=9567056 CPUtime=4781.94 cores=0-7
/proc/4389/task/4403/stat : 4403 (plingeling) R 4387 4389 4363 0 -1 4202560 831937 1255 0 0 478032 162 0 0 25 0 9 0 797953763 9796665344 2336752 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 38180.1
Current children cumulated vsize (KiB) 9567056

[startup+4842.3 s]
/proc/loadavg: 8.02 8.04 8.00 9/471 4544
/proc/meminfo: memFree=2972788/32951124 swapFree=57414376/67111528
[pid=4389] ppid=4387 vsize=9845296 CPUtime=38659.2 cores=0-7
/proc/4389/stat : 4389 (plingeling) S 4387 4389 4363 0 -1 4202496 11370104 1255 0 0 3863625 2296 0 0 25 0 9 0 797953721 10081583104 2406312 33554432000 4194304 4417452 140734197376720 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4389/statm: 2461324 2406312 180 55 0 2459210 0
[pid=4389/tid=4396] ppid=4387 vsize=9845296 CPUtime=4816.88 cores=0-7
/proc/4389/task/4396/stat : 4396 (plingeling) R 4387 4389 4363 0 -1 4202560 1138104 1255 0 0 481472 216 0 0 25 0 9 0 797953762 10081583104 2406312 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4389/tid=4397] ppid=4387 vsize=9845296 CPUtime=4841.33 cores=0-7
/proc/4389/task/4397/stat : 4397 (plingeling) R 4387 4389 4363 0 -1 4202560 945459 1255 0 0 483926 207 0 0 25 0 9 0 797953762 10081583104 2406312 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4389/tid=4398] ppid=4387 vsize=9845296 CPUtime=4841.81 cores=0-7
/proc/4389/task/4398/stat : 4398 (plingeling) R 4387 4389 4363 0 -1 4202560 1320776 1255 0 0 483931 250 0 0 25 0 9 0 797953762 10081583104 2406312 33554432000 4194304 4417452 140734197376720 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4389/tid=4399] ppid=4387 vsize=9845296 CPUtime=4841.97 cores=0-7
/proc/4389/task/4399/stat : 4399 (plingeling) R 4387 4389 4363 0 -1 4202560 3994585 1255 0 0 483307 890 0 0 25 0 9 0 797953762 10081583104 2406312 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4389/tid=4400] ppid=4387 vsize=9845296 CPUtime=4840.77 cores=0-7
/proc/4389/task/4400/stat : 4400 (plingeling) R 4387 4389 4363 0 -1 4202560 903656 1255 0 0 483898 179 0 0 25 0 9 0 797953762 10081583104 2406312 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4389/tid=4401] ppid=4387 vsize=9845296 CPUtime=4842 cores=0-7
/proc/4389/task/4401/stat : 4401 (plingeling) R 4387 4389 4363 0 -1 4202560 1349478 1255 0 0 483957 243 0 0 25 0 9 0 797953762 10081583104 2406312 33554432000 4194304 4417452 140734197376720 18446744073709551615 4354497 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4389/tid=4402] ppid=4387 vsize=9845296 CPUtime=4792.02 cores=0-7
/proc/4389/task/4402/stat : 4402 (plingeling) R 4387 4389 4363 0 -1 4202560 837922 1255 0 0 479062 140 0 0 25 0 9 0 797953763 10081583104 2406312 33554432000 4194304 4417452 140734197376720 18446744073709551615 4357554 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4389/tid=4403] ppid=4387 vsize=9845296 CPUtime=4841.94 cores=0-7
/proc/4389/task/4403/stat : 4403 (plingeling) R 4387 4389 4363 0 -1 4202560 858661 1255 0 0 484030 164 0 0 25 0 9 0 797953763 10081583104 2406312 33554432000 4194304 4417452 140734197376720 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 38659.2
Current children cumulated vsize (KiB) 9845296

[startup+4902.3 s]
/proc/loadavg: 8.04 8.04 8.00 9/471 4546
/proc/meminfo: memFree=3265684/32951124 swapFree=57414376/67111528
[pid=4389] ppid=4387 vsize=9553120 CPUtime=39138.3 cores=0-7
/proc/4389/stat : 4389 (plingeling) S 4387 4389 4363 0 -1 4202496 11537779 1255 0 0 3911487 2345 0 0 25 0 9 0 797953721 9782394880 2333268 33554432000 4194304 4417452 140734197376720 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4389/statm: 2388280 2333268 180 55 0 2386166 0
[pid=4389/tid=4396] ppid=4387 vsize=9553120 CPUtime=4876.9 cores=0-7
/proc/4389/task/4396/stat : 4396 (plingeling) R 4387 4389 4363 0 -1 4202560 1138104 1255 0 0 487474 216 0 0 25 0 9 0 797953762 9782394880 2333268 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4389/tid=4397] ppid=4387 vsize=9553120 CPUtime=4901.35 cores=0-7
/proc/4389/task/4397/stat : 4397 (plingeling) R 4387 4389 4363 0 -1 4202560 956741 1255 0 0 489923 212 0 0 25 0 9 0 797953762 9782394880 2333268 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4389/tid=4398] ppid=4387 vsize=9553120 CPUtime=4901.83 cores=0-7
/proc/4389/task/4398/stat : 4398 (plingeling) R 4387 4389 4363 0 -1 4202560 1343636 1255 0 0 489930 253 0 0 25 0 9 0 797953762 9782394880 2333268 33554432000 4194304 4417452 140734197376720 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4389/tid=4399] ppid=4387 vsize=9553120 CPUtime=4902 cores=0-7
/proc/4389/task/4399/stat : 4399 (plingeling) R 4387 4389 4363 0 -1 4202560 4099059 1255 0 0 489272 928 0 0 25 0 9 0 797953762 9782394880 2333268 33554432000 4194304 4417452 140734197376720 18446744073709551615 4365959 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4389/tid=4400] ppid=4387 vsize=9553120 CPUtime=4900.77 cores=0-7
/proc/4389/task/4400/stat : 4400 (plingeling) R 4387 4389 4363 0 -1 4202560 903656 1255 0 0 489898 179 0 0 25 0 9 0 797953762 9782394880 2333268 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364829 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4389/tid=4401] ppid=4387 vsize=9553120 CPUtime=4902.02 cores=0-7
/proc/4389/task/4401/stat : 4401 (plingeling) R 4387 4389 4363 0 -1 4202560 1361753 1255 0 0 489957 245 0 0 25 0 9 0 797953762 9782394880 2333268 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4389/tid=4402] ppid=4387 vsize=9553120 CPUtime=4851.1 cores=0-7
/proc/4389/task/4402/stat : 4402 (plingeling) R 4387 4389 4363 0 -1 4202560 837922 1255 0 0 484970 140 0 0 25 0 9 0 797953763 9782394880 2333268 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4389/tid=4403] ppid=4387 vsize=9553120 CPUtime=4901.96 cores=0-7
/proc/4389/task/4403/stat : 4403 (plingeling) R 4387 4389 4363 0 -1 4202560 875445 1255 0 0 490030 166 0 0 25 0 9 0 797953763 9782394880 2333268 33554432000 4194304 4417452 140734197376720 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 39138.3
Current children cumulated vsize (KiB) 9553120

[startup+4962.3 s]
/proc/loadavg: 8.17 8.06 8.01 9/471 4547
/proc/meminfo: memFree=2732620/32951124 swapFree=57414376/67111528
[pid=4389] ppid=4387 vsize=10085652 CPUtime=39617.4 cores=0-7
/proc/4389/stat : 4389 (plingeling) S 4387 4389 4363 0 -1 4202496 11670912 1255 0 0 3959357 2379 0 0 25 0 9 0 797953721 10327707648 2466401 33554432000 4194304 4417452 140734197376720 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4389/statm: 2521413 2466401 180 55 0 2519299 0
[pid=4389/tid=4396] ppid=4387 vsize=10085652 CPUtime=4936.89 cores=0-7
/proc/4389/task/4396/stat : 4396 (plingeling) R 4387 4389 4363 0 -1 4202560 1138104 1255 0 0 493473 216 0 0 25 0 9 0 797953762 10327707648 2466401 33554432000 4194304 4417452 140734197376720 18446744073709551615 4365042 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4389/tid=4397] ppid=4387 vsize=10085652 CPUtime=4961.34 cores=0-7
/proc/4389/task/4397/stat : 4397 (plingeling) R 4387 4389 4363 0 -1 4202560 967209 1255 0 0 495920 214 0 0 25 0 9 0 797953762 10327707648 2466401 33554432000 4194304 4417452 140734197376720 18446744073709551615 4365984 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4389/tid=4398] ppid=4387 vsize=10085652 CPUtime=4961.81 cores=0-7
/proc/4389/task/4398/stat : 4398 (plingeling) R 4387 4389 4363 0 -1 4202560 1387667 1255 0 0 495923 258 0 0 25 0 9 0 797953762 10327707648 2466401 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4389/tid=4399] ppid=4387 vsize=10085652 CPUtime=4961.98 cores=0-7
/proc/4389/task/4399/stat : 4399 (plingeling) R 4387 4389 4363 0 -1 4202560 4154377 1255 0 0 495250 948 0 0 25 0 9 0 797953762 10327707648 2466401 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4389/tid=4400] ppid=4387 vsize=10085652 CPUtime=4960.75 cores=0-7
/proc/4389/task/4400/stat : 4400 (plingeling) R 4387 4389 4363 0 -1 4202560 903656 1255 0 0 495896 179 0 0 25 0 9 0 797953762 10327707648 2466401 33554432000 4194304 4417452 140734197376720 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4389/tid=4401] ppid=4387 vsize=10085652 CPUtime=4962.01 cores=0-7
/proc/4389/task/4401/stat : 4401 (plingeling) R 4387 4389 4363 0 -1 4202560 1370081 1255 0 0 495953 248 0 0 25 0 9 0 797953762 10327707648 2466401 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4389/tid=4402] ppid=4387 vsize=10085652 CPUtime=4910.16 cores=0-7
/proc/4389/task/4402/stat : 4402 (plingeling) R 4387 4389 4363 0 -1 4202560 837922 1255 0 0 490876 140 0 0 25 0 9 0 797953763 10327707648 2466401 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4389/tid=4403] ppid=4387 vsize=10085652 CPUtime=4961.94 cores=0-7
/proc/4389/task/4403/stat : 4403 (plingeling) R 4387 4389 4363 0 -1 4202560 890433 1255 0 0 496024 170 0 0 25 0 9 0 797953763 10327707648 2466401 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 39617.4
Current children cumulated vsize (KiB) 10085652



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5010.3 s]
/proc/loadavg: 8.12 8.07 8.01 9/471 4549
/proc/meminfo: memFree=2153260/32951124 swapFree=57414376/67111528
[pid=4389] ppid=4387 vsize=10663868 CPUtime=40000.6 cores=0-7
/proc/4389/stat : 4389 (plingeling) S 4387 4389 4363 0 -1 4202496 11848837 1255 0 0 3997656 2409 0 0 25 0 9 0 797953721 10919800832 2610954 33554432000 4194304 4417452 140734197376720 18446744073709551615 252469868517 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/4389/statm: 2665967 2610954 180 55 0 2663853 0
[pid=4389/tid=4396] ppid=4387 vsize=10663868 CPUtime=4984.9 cores=0-7
/proc/4389/task/4396/stat : 4396 (plingeling) R 4387 4389 4363 0 -1 4202560 1138104 1255 0 0 498274 216 0 0 25 0 9 0 797953762 10919800832 2610954 33554432000 4194304 4417452 140734197376720 18446744073709551615 4259140 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4389/tid=4397] ppid=4387 vsize=10663868 CPUtime=5009.36 cores=0-7
/proc/4389/task/4397/stat : 4397 (plingeling) R 4387 4389 4363 0 -1 4202560 973681 1255 0 0 500721 215 0 0 25 0 9 0 797953762 10919800832 2610954 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4389/tid=4398] ppid=4387 vsize=10663868 CPUtime=5009.83 cores=0-7
/proc/4389/task/4398/stat : 4398 (plingeling) R 4387 4389 4363 0 -1 4202560 1436181 1255 0 0 500715 268 0 0 25 0 9 0 797953762 10919800832 2610954 33554432000 4194304 4417452 140734197376720 18446744073709551615 4264934 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4389/tid=4399] ppid=4387 vsize=10663868 CPUtime=5010 cores=0-7
/proc/4389/task/4399/stat : 4399 (plingeling) R 4387 4389 4363 0 -1 4202560 4154377 1255 0 0 500052 948 0 0 25 0 9 0 797953762 10919800832 2610954 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4389/tid=4400] ppid=4387 vsize=10663868 CPUtime=5008.75 cores=0-7
/proc/4389/task/4400/stat : 4400 (plingeling) R 4387 4389 4363 0 -1 4202560 903656 1255 0 0 500696 179 0 0 25 0 9 0 797953762 10919800832 2610954 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4389/tid=4401] ppid=4387 vsize=10663868 CPUtime=5010.03 cores=0-7
/proc/4389/task/4401/stat : 4401 (plingeling) R 4387 4389 4363 0 -1 4202560 1401468 1255 0 0 500751 252 0 0 25 0 9 0 797953762 10919800832 2610954 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4389/tid=4402] ppid=4387 vsize=10663868 CPUtime=4957.42 cores=0-7
/proc/4389/task/4402/stat : 4402 (plingeling) R 4387 4389 4363 0 -1 4202560 837939 1255 0 0 495602 140 0 0 25 0 9 0 797953763 10919800832 2610954 33554432000 4194304 4417452 140734197376720 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4389/tid=4403] ppid=4387 vsize=10663868 CPUtime=5009.96 cores=0-7
/proc/4389/task/4403/stat : 4403 (plingeling) R 4387 4389 4363 0 -1 4202560 981968 1255 0 0 500811 185 0 0 25 0 9 0 797953763 10919800832 2610954 33554432000 4194304 4417452 140734197376720 18446744073709551615 4357392 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 10663868

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5010.6 s]
/proc/loadavg: 8.12 8.07 8.01 9/471 4549
/proc/meminfo: memFree=2153260/32951124 swapFree=57414376/67111528
[pid=4389] ppid=4387 vsize=0 CPUtime=40001.1 cores=0-7
/proc/4389/stat : 4389 (plingeling) Z 4387 4389 4363 0 -1 4203548 11848837 1255 0 0 3997669 2437 0 0 18 0 2 0 797953721 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/4389/statm: 0 0 0 0 0 0 0
[pid=4389/tid=4402] ppid=4387 vsize=0 CPUtime=4957.7 cores=0-7
/proc/4389/task/4402/stat : 4402 (plingeling) R 4387 4389 4363 0 -1 4203588 837939 1255 0 0 495602 168 0 0 25 0 2 0 797953763 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 40001.1
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5010.66
CPU time (s): 40001.1
CPU user time (s): 39976.7
CPU system time (s): 24.4323
CPU usage (%): 798.321
Max. virtual memory (cumulated for all children) (KiB): 11267268

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39976.7
system time used= 24.4323
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 11850092
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 683
involuntary context switches= 143248

runsolver used 22.4376 second user time and 54.9616 second system time

The end

Launcher Data 

Begin job on node111 at 2011-05-22 17:13:52
IDJOB=3393972
IDBENCH=70908
IDSOLVER=1851
FILE ID=node111/3393972-1306077232
RUNJOBID= node111-1306077232-4377
PBS_JOBID= 13387905
Free space on /tmp= 73340 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT09/CRAFTED/edgematching/compact/em_12_2_4_cmp.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3393972-1306077232/watcher-3393972-1306077232 -o /tmp/evaluation-result-3393972-1306077232/solver-3393972-1306077232 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3393972-1306077232.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 7ea97a36d5ab6c8a5aa99830d5253591
RANDOM SEED=348506644

node111.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5428.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      12616556 kB
Buffers:        334552 kB
Cached:       11190108 kB
SwapCached:      19180 kB
Active:       10281088 kB
Inactive:      9761112 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      12616556 kB
SwapTotal:    67111528 kB
SwapFree:     57414376 kB
Dirty:            3524 kB
Writeback:           0 kB
AnonPages:     8507732 kB
Mapped:          26396 kB
Slab:           155620 kB
PageTables:      66196 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 149059688 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73340 MiB
End job on node111 at 2011-05-22 18:37:25