Trace number 3371143

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.3 1203.44

General information on the benchmark

NameSAT05/crafted/sabharwal05/counting/php/unsat/
easier/php-045-035.shuffled-as.sat05-1181.cnf
MD5SUM057b78ff24cc0773c9c67e770a7b9de2
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark44.4072
Satisfiable
(Un)Satisfiability was proved
Number of variables1575
Number of clauses34695
Sum of the clauses size70875
Maximum clause length35
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 234650
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 545

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3371143-1305431576/watcher-3371143-1305431576 -o /tmp/evaluation-result-3371143-1305431576/solver-3371143-1305431576 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3371143-1305431576.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.76 7.90 6.35 5/311 29728
/proc/meminfo: memFree=17992668/32950928 swapFree=67111348/67111528
[pid=29728] ppid=29726 vsize=9664 CPUtime=0 cores=0,2,4,6
/proc/29728/stat : 29728 (plingeling) R 29726 29728 20898 0 -1 4202496 509 1259 0 0 0 0 0 0 25 0 1 0 654757285 9895936 438 33554432000 4194304 4417452 140733983650384 18446744073709551615 274473967360 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/29728/statm: 2416 438 147 55 0 302 0

[startup+0.072902 s]
/proc/loadavg: 7.76 7.90 6.35 5/311 29728
/proc/meminfo: memFree=17992668/32950928 swapFree=67111348/67111528
[pid=29728] ppid=29726 vsize=56868 CPUtime=0.18 cores=0,2,4,6
/proc/29728/stat : 29728 (plingeling) S 29726 29728 20898 0 -1 4202496 2405 1259 0 0 18 0 0 0 25 0 5 0 654757285 58232832 1983 33554432000 4194304 4417452 140733983650384 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29728/statm: 14217 1998 174 55 0 12103 0
Current children cumulated CPU time (s) 0.18
Current children cumulated vsize (KiB) 56868

[startup+0.153388 s]
/proc/loadavg: 7.76 7.90 6.35 5/311 29728
/proc/meminfo: memFree=17992668/32950928 swapFree=67111348/67111528
[pid=29728] ppid=29726 vsize=59284 CPUtime=0.5 cores=0,2,4,6
/proc/29728/stat : 29728 (plingeling) S 29726 29728 20898 0 -1 4202496 3713 1259 0 0 50 0 0 0 25 0 5 0 654757285 60706816 2608 33554432000 4194304 4417452 140733983650384 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29728/statm: 14821 2608 180 55 0 12707 0
Current children cumulated CPU time (s) 0.5
Current children cumulated vsize (KiB) 59284

[startup+0.300362 s]
/proc/loadavg: 7.76 7.90 6.35 5/311 29728
/proc/meminfo: memFree=17992668/32950928 swapFree=67111348/67111528
[pid=29728] ppid=29726 vsize=59596 CPUtime=1.09 cores=0,2,4,6
/proc/29728/stat : 29728 (plingeling) S 29726 29728 20898 0 -1 4202496 4430 1259 0 0 109 0 0 0 25 0 5 0 654757285 61026304 2686 33554432000 4194304 4417452 140733983650384 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29728/statm: 14899 2686 180 55 0 12785 0
Current children cumulated CPU time (s) 1.09
Current children cumulated vsize (KiB) 59596

[startup+0.700294 s]
/proc/loadavg: 7.76 7.90 6.35 5/311 29728
/proc/meminfo: memFree=17992668/32950928 swapFree=67111348/67111528
[pid=29728] ppid=29726 vsize=135204 CPUtime=2.69 cores=0,2,4,6
/proc/29728/stat : 29728 (plingeling) S 29726 29728 20898 0 -1 4202496 9433 1259 0 0 269 0 0 0 25 0 5 0 654757285 138448896 5254 33554432000 4194304 4417452 140733983650384 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29728/statm: 33801 5254 180 55 0 31687 0
Current children cumulated CPU time (s) 2.69
Current children cumulated vsize (KiB) 135204

[startup+1.50115 s]
/proc/loadavg: 7.76 7.90 6.35 9/316 29738
/proc/meminfo: memFree=17959516/32950928 swapFree=67111348/67111528
[pid=29728] ppid=29726 vsize=149396 CPUtime=5.88 cores=0,2,4,6
/proc/29728/stat : 29728 (plingeling) S 29726 29728 20898 0 -1 4202496 24187 1259 0 0 585 3 0 0 25 0 5 0 654757285 152981504 8770 33554432000 4194304 4417452 140733983650384 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29728/statm: 37349 8770 180 55 0 35235 0
[pid=29728/tid=29735] ppid=29726 vsize=149396 CPUtime=1.46 cores=0,2,4,6
/proc/29728/task/29735/stat : 29735 (plingeling) R 29726 29728 20898 0 -1 4202560 4378 1259 0 0 146 0 0 0 25 0 5 0 654757288 152981504 8770 33554432000 4194304 4417452 140733983650384 18446744073709551615 4232584 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29728/tid=29736] ppid=29726 vsize=149396 CPUtime=1.44 cores=0,2,4,6
/proc/29728/task/29736/stat : 29736 (plingeling) R 29726 29728 20898 0 -1 4202560 4622 1259 0 0 144 0 0 0 25 0 5 0 654757288 152981504 8770 33554432000 4194304 4417452 140733983650384 18446744073709551615 4365130 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29728/tid=29737] ppid=29726 vsize=149396 CPUtime=1.46 cores=0,2,4,6
/proc/29728/task/29737/stat : 29737 (plingeling) R 29726 29728 20898 0 -1 4202560 4205 1259 0 0 146 0 0 0 25 0 5 0 654757288 152981504 8770 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364807 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29728/tid=29738] ppid=29726 vsize=149396 CPUtime=1.46 cores=0,2,4,6
/proc/29728/task/29738/stat : 29738 (plingeling) R 29726 29728 20898 0 -1 4202560 9432 1259 0 0 145 1 0 0 25 0 5 0 654757288 152981504 8770 33554432000 4194304 4417452 140733983650384 18446744073709551615 4354506 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 5.88
Current children cumulated vsize (KiB) 149396

[startup+3.10087 s]
/proc/loadavg: 7.76 7.90 6.35 9/316 29738
/proc/meminfo: memFree=17935956/32950928 swapFree=67111348/67111528
[pid=29728] ppid=29726 vsize=163580 CPUtime=12.27 cores=0,2,4,6
/proc/29728/stat : 29728 (plingeling) S 29726 29728 20898 0 -1 4202496 51470 1259 0 0 1219 8 0 0 25 0 5 0 654757285 167505920 12316 33554432000 4194304 4417452 140733983650384 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29728/statm: 40895 12316 180 55 0 38781 0
[pid=29728/tid=29735] ppid=29726 vsize=163580 CPUtime=3.06 cores=0,2,4,6
/proc/29728/task/29735/stat : 29735 (plingeling) R 29726 29728 20898 0 -1 4202560 9393 1259 0 0 305 1 0 0 25 0 5 0 654757288 167505920 12316 33554432000 4194304 4417452 140733983650384 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29728/tid=29736] ppid=29726 vsize=163580 CPUtime=3.03 cores=0,2,4,6
/proc/29728/task/29736/stat : 29736 (plingeling) R 29726 29728 20898 0 -1 4202560 10104 1259 0 0 302 1 0 0 25 0 5 0 654757288 167505920 12316 33554432000 4194304 4417452 140733983650384 18446744073709551615 4265473 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29728/tid=29737] ppid=29726 vsize=163580 CPUtime=3.06 cores=0,2,4,6
/proc/29728/task/29737/stat : 29737 (plingeling) R 29726 29728 20898 0 -1 4202560 9604 1259 0 0 305 1 0 0 25 0 5 0 654757288 167505920 12316 33554432000 4194304 4417452 140733983650384 18446744073709551615 4250017 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29728/tid=29738] ppid=29726 vsize=163580 CPUtime=3.06 cores=0,2,4,6
/proc/29728/task/29738/stat : 29738 (plingeling) R 29726 29728 20898 0 -1 4202560 20819 1259 0 0 302 4 0 0 25 0 5 0 654757288 167505920 12316 33554432000 4194304 4417452 140733983650384 18446744073709551615 4357714 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 12.27
Current children cumulated vsize (KiB) 163580

[startup+6.30031 s]
/proc/loadavg: 7.86 7.92 6.36 9/316 29739
/proc/meminfo: memFree=17886140/32950928 swapFree=67111348/67111528
[pid=29728] ppid=29726 vsize=209580 CPUtime=25.03 cores=0,2,4,6
/proc/29728/stat : 29728 (plingeling) S 29726 29728 20898 0 -1 4202496 103425 1259 0 0 2486 17 0 0 25 0 5 0 654757285 214609920 23816 33554432000 4194304 4417452 140733983650384 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29728/statm: 52395 23816 180 55 0 50281 0
[pid=29728/tid=29735] ppid=29726 vsize=209580 CPUtime=6.26 cores=0,2,4,6
/proc/29728/task/29735/stat : 29735 (plingeling) R 29726 29728 20898 0 -1 4202560 24540 1259 0 0 623 3 0 0 25 0 5 0 654757288 214609920 23816 33554432000 4194304 4417452 140733983650384 18446744073709551615 4255606 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29728/tid=29736] ppid=29726 vsize=209580 CPUtime=6.2 cores=0,2,4,6
/proc/29728/task/29736/stat : 29736 (plingeling) R 29726 29728 20898 0 -1 4202560 22279 1259 0 0 617 3 0 0 25 0 5 0 654757288 214609920 23816 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29728/tid=29737] ppid=29726 vsize=209580 CPUtime=6.25 cores=0,2,4,6
/proc/29728/task/29737/stat : 29737 (plingeling) R 29726 29728 20898 0 -1 4202560 23585 1259 0 0 622 3 0 0 25 0 5 0 654757288 214609920 23816 33554432000 4194304 4417452 140733983650384 18446744073709551615 4232629 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29728/tid=29738] ppid=29726 vsize=209580 CPUtime=6.26 cores=0,2,4,6
/proc/29728/task/29738/stat : 29738 (plingeling) R 29726 29728 20898 0 -1 4202560 31471 1259 0 0 620 6 0 0 25 0 5 0 654757288 214609920 23816 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 25.03
Current children cumulated vsize (KiB) 209580

[startup+12.7012 s]
/proc/loadavg: 7.87 7.92 6.37 9/316 29739
/proc/meminfo: memFree=17823684/32950928 swapFree=67111348/67111528
[pid=29728] ppid=29726 vsize=239848 CPUtime=50.57 cores=0,2,4,6
/proc/29728/stat : 29728 (plingeling) S 29726 29728 20898 0 -1 4202496 190404 1259 0 0 5025 32 0 0 25 0 5 0 654757285 245604352 31430 33554432000 4194304 4417452 140733983650384 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29728/statm: 59962 31430 180 55 0 57848 0
[pid=29728/tid=29735] ppid=29726 vsize=239848 CPUtime=12.66 cores=0,2,4,6
/proc/29728/task/29735/stat : 29735 (plingeling) R 29726 29728 20898 0 -1 4202560 47206 1259 0 0 1259 7 0 0 25 0 5 0 654757288 245604352 31430 33554432000 4194304 4417452 140733983650384 18446744073709551615 4232693 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29728/tid=29736] ppid=29726 vsize=239848 CPUtime=12.53 cores=0,2,4,6
/proc/29728/task/29736/stat : 29736 (plingeling) R 29726 29728 20898 0 -1 4202560 44056 1259 0 0 1246 7 0 0 25 0 5 0 654757288 245604352 31430 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364807 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29728/tid=29737] ppid=29726 vsize=239848 CPUtime=12.65 cores=0,2,4,6
/proc/29728/task/29737/stat : 29737 (plingeling) R 29726 29728 20898 0 -1 4202560 48260 1259 0 0 1258 7 0 0 25 0 5 0 654757288 245604352 31430 33554432000 4194304 4417452 140733983650384 18446744073709551615 4343760 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29728/tid=29738] ppid=29726 vsize=239848 CPUtime=12.66 cores=0,2,4,6
/proc/29728/task/29738/stat : 29738 (plingeling) R 29726 29728 20898 0 -1 4202560 49332 1259 0 0 1257 9 0 0 25 0 5 0 654757288 245604352 31430 33554432000 4194304 4417452 140733983650384 18446744073709551615 4353512 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 50.57
Current children cumulated vsize (KiB) 239848

[startup+25.5009 s]
/proc/loadavg: 7.90 7.92 6.40 9/316 29739
/proc/meminfo: memFree=17647996/32950928 swapFree=67111348/67111528
[pid=29728] ppid=29726 vsize=384656 CPUtime=101.64 cores=0,2,4,6
/proc/29728/stat : 29728 (plingeling) S 29726 29728 20898 0 -1 4202496 344760 1259 0 0 10109 55 0 0 25 0 5 0 654757285 393887744 67600 33554432000 4194304 4417452 140733983650384 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29728/statm: 96164 67600 180 55 0 94050 0
[pid=29728/tid=29735] ppid=29726 vsize=384656 CPUtime=25.46 cores=0,2,4,6
/proc/29728/task/29735/stat : 29735 (plingeling) R 29726 29728 20898 0 -1 4202560 85416 1259 0 0 2534 12 0 0 25 0 5 0 654757288 393887744 67600 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29728/tid=29736] ppid=29726 vsize=384656 CPUtime=25.2 cores=0,2,4,6
/proc/29728/task/29736/stat : 29736 (plingeling) R 29726 29728 20898 0 -1 4202560 84336 1259 0 0 2508 12 0 0 25 0 5 0 654757288 393887744 67600 33554432000 4194304 4417452 140733983650384 18446744073709551615 4366099 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29728/tid=29737] ppid=29726 vsize=384656 CPUtime=25.45 cores=0,2,4,6
/proc/29728/task/29737/stat : 29737 (plingeling) R 29726 29728 20898 0 -1 4202560 84651 1259 0 0 2533 12 0 0 25 0 5 0 654757288 393887744 67600 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29728/tid=29738] ppid=29726 vsize=384656 CPUtime=25.46 cores=0,2,4,6
/proc/29728/task/29738/stat : 29738 (plingeling) R 29726 29728 20898 0 -1 4202560 88807 1259 0 0 2530 16 0 0 25 0 5 0 654757288 393887744 67600 33554432000 4194304 4417452 140733983650384 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 101.64
Current children cumulated vsize (KiB) 384656

[startup+51.1004 s]
/proc/loadavg: 8.01 7.94 6.44 10/316 29740
/proc/meminfo: memFree=17491620/32950928 swapFree=67111348/67111528
[pid=29728] ppid=29726 vsize=496160 CPUtime=203.77 cores=0,2,4,6
/proc/29728/stat : 29728 (plingeling) S 29726 29728 20898 0 -1 4202496 590919 1259 0 0 20286 91 0 0 25 0 5 0 654757285 508067840 100723 33554432000 4194304 4417452 140733983650384 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29728/statm: 124040 100723 180 55 0 121926 0
[pid=29728/tid=29735] ppid=29726 vsize=496160 CPUtime=51.06 cores=0,2,4,6
/proc/29728/task/29735/stat : 29735 (plingeling) R 29726 29728 20898 0 -1 4202560 144271 1259 0 0 5084 22 0 0 25 0 5 0 654757288 508067840 100723 33554432000 4194304 4417452 140733983650384 18446744073709551615 4305920 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29728/tid=29736] ppid=29726 vsize=496160 CPUtime=50.56 cores=0,2,4,6
/proc/29728/task/29736/stat : 29736 (plingeling) R 29726 29728 20898 0 -1 4202560 158532 1259 0 0 5033 23 0 0 25 0 5 0 654757288 508067840 100723 33554432000 4194304 4417452 140733983650384 18446744073709551615 4310251 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29728/tid=29737] ppid=29726 vsize=496160 CPUtime=51.04 cores=0,2,4,6
/proc/29728/task/29737/stat : 29737 (plingeling) R 29726 29728 20898 0 -1 4202560 153883 1259 0 0 5082 22 0 0 25 0 5 0 654757288 508067840 100723 33554432000 4194304 4417452 140733983650384 18446744073709551615 4199406 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29728/tid=29738] ppid=29726 vsize=496160 CPUtime=51.06 cores=0,2,4,6
/proc/29728/task/29738/stat : 29738 (plingeling) R 29726 29728 20898 0 -1 4202560 132683 1259 0 0 5083 23 0 0 25 0 5 0 654757288 508067840 100723 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 203.77
Current children cumulated vsize (KiB) 496160

[startup+102.311 s]
/proc/loadavg: 8.04 7.96 6.53 9/316 29741
/proc/meminfo: memFree=17297776/32950928 swapFree=67111348/67111528
[pid=29728] ppid=29726 vsize=644144 CPUtime=408.09 cores=0,2,4,6
/proc/29728/stat : 29728 (plingeling) S 29726 29728 20898 0 -1 4202496 1054964 1259 0 0 40647 162 0 0 25 0 5 0 654757285 659603456 133479 33554432000 4194304 4417452 140733983650384 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29728/statm: 161036 133479 180 55 0 158922 0
[pid=29728/tid=29735] ppid=29726 vsize=644144 CPUtime=102.27 cores=0,2,4,6
/proc/29728/task/29735/stat : 29735 (plingeling) R 29726 29728 20898 0 -1 4202560 288227 1259 0 0 10184 43 0 0 25 0 5 0 654757288 659603456 133479 33554432000 4194304 4417452 140733983650384 18446744073709551615 4310569 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29728/tid=29736] ppid=29726 vsize=644144 CPUtime=101.25 cores=0,2,4,6
/proc/29728/task/29736/stat : 29736 (plingeling) R 29726 29728 20898 0 -1 4202560 279064 1259 0 0 10086 39 0 0 25 0 5 0 654757288 659603456 133479 33554432000 4194304 4417452 140733983650384 18446744073709551615 4306322 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29728/tid=29737] ppid=29726 vsize=644144 CPUtime=102.24 cores=0,2,4,6
/proc/29728/task/29737/stat : 29737 (plingeling) R 29726 29728 20898 0 -1 4202560 274379 1259 0 0 10183 41 0 0 25 0 5 0 654757288 659603456 133479 33554432000 4194304 4417452 140733983650384 18446744073709551615 4259922 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29728/tid=29738] ppid=29726 vsize=644144 CPUtime=102.28 cores=0,2,4,6
/proc/29728/task/29738/stat : 29738 (plingeling) R 29726 29728 20898 0 -1 4202560 211744 1259 0 0 10190 38 0 0 25 0 5 0 654757288 659603456 133479 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 408.09
Current children cumulated vsize (KiB) 644144

[startup+162.301 s]
/proc/loadavg: 8.09 7.99 6.62 9/316 29743
/proc/meminfo: memFree=16996792/32950928 swapFree=67111348/67111528
[pid=29728] ppid=29726 vsize=812736 CPUtime=647.46 cores=0,2,4,6
/proc/29728/stat : 29728 (plingeling) S 29726 29728 20898 0 -1 4202496 1596247 1259 0 0 64503 243 0 0 25 0 5 0 654757285 832241664 179145 33554432000 4194304 4417452 140733983650384 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29728/statm: 203184 179145 180 55 0 201070 0
[pid=29728/tid=29735] ppid=29726 vsize=812736 CPUtime=162.26 cores=0,2,4,6
/proc/29728/task/29735/stat : 29735 (plingeling) R 29726 29728 20898 0 -1 4202560 463801 1259 0 0 16158 68 0 0 25 0 5 0 654757288 832241664 179145 33554432000 4194304 4417452 140733983650384 18446744073709551615 4357554 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29728/tid=29736] ppid=29726 vsize=812736 CPUtime=160.65 cores=0,2,4,6
/proc/29728/task/29736/stat : 29736 (plingeling) R 29726 29728 20898 0 -1 4202560 437666 1259 0 0 16002 63 0 0 25 0 5 0 654757288 832241664 179145 33554432000 4194304 4417452 140733983650384 18446744073709551615 4366657 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29728/tid=29737] ppid=29726 vsize=812736 CPUtime=162.23 cores=0,2,4,6
/proc/29728/task/29737/stat : 29737 (plingeling) R 29726 29728 20898 0 -1 4202560 429649 1259 0 0 16159 64 0 0 25 0 5 0 654757288 832241664 179145 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364650 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29728/tid=29738] ppid=29726 vsize=812736 CPUtime=162.26 cores=0,2,4,6
/proc/29728/task/29738/stat : 29738 (plingeling) R 29726 29728 20898 0 -1 4202560 263581 1259 0 0 16180 46 0 0 25 0 5 0 654757288 832241664 179145 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 647.46
Current children cumulated vsize (KiB) 812736


################
# More data... #
################

/proc/meminfo: memFree=15441844/32950928 swapFree=67111348/67111528
[pid=29728] ppid=29726 vsize=2159008 CPUtime=3040.58 cores=0,2,4,6
/proc/29728/stat : 29728 (plingeling) S 29726 29728 20898 0 -1 4202496 4127952 1259 0 0 303444 614 0 0 25 0 5 0 654757285 2210824192 511188 33554432000 4194304 4417452 140733983650384 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29728/statm: 539752 511188 180 55 0 537638 0
[pid=29728/tid=29735] ppid=29726 vsize=2159008 CPUtime=762.2 cores=0,2,4,6
/proc/29728/task/29735/stat : 29735 (plingeling) R 29726 29728 20898 0 -1 4202560 1222342 1259 0 0 76043 177 0 0 25 0 5 0 654757288 2210824192 511188 33554432000 4194304 4417452 140733983650384 18446744073709551615 4263109 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29728/tid=29736] ppid=29726 vsize=2159008 CPUtime=753.91 cores=0,2,4,6
/proc/29728/task/29736/stat : 29736 (plingeling) R 29726 29728 20898 0 -1 4202560 1201648 1259 0 0 75210 181 0 0 25 0 5 0 654757288 2210824192 511188 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29728/tid=29737] ppid=29726 vsize=2159008 CPUtime=762.14 cores=0,2,4,6
/proc/29728/task/29737/stat : 29737 (plingeling) R 29726 29728 20898 0 -1 4202560 1072976 1259 0 0 76060 154 0 0 25 0 5 0 654757288 2210824192 511188 33554432000 4194304 4417452 140733983650384 18446744073709551615 4265687 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29728/tid=29738] ppid=29726 vsize=2159008 CPUtime=762.27 cores=0,2,4,6
/proc/29728/task/29738/stat : 29738 (plingeling) R 29726 29728 20898 0 -1 4202560 629436 1259 0 0 76127 100 0 0 25 0 5 0 654757288 2210824192 511188 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3040.58
Current children cumulated vsize (KiB) 2159008

[startup+822.301 s]
/proc/loadavg: 8.17 8.13 7.38 9/316 29808
/proc/meminfo: memFree=15314840/32950928 swapFree=67111348/67111528
[pid=29728] ppid=29726 vsize=2370880 CPUtime=3279.92 cores=0,2,4,6
/proc/29728/stat : 29728 (plingeling) S 29726 29728 20898 0 -1 4202496 4306166 1259 0 0 327350 642 0 0 25 0 5 0 654757285 2427781120 564156 33554432000 4194304 4417452 140733983650384 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29728/statm: 592720 564156 180 55 0 590606 0
[pid=29728/tid=29735] ppid=29726 vsize=2370880 CPUtime=822.2 cores=0,2,4,6
/proc/29728/task/29735/stat : 29735 (plingeling) R 29726 29728 20898 0 -1 4202560 1254773 1259 0 0 82038 182 0 0 25 0 5 0 654757288 2427781120 564156 33554432000 4194304 4417452 140733983650384 18446744073709551615 4309899 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29728/tid=29736] ppid=29726 vsize=2370880 CPUtime=813.31 cores=0,2,4,6
/proc/29728/task/29736/stat : 29736 (plingeling) R 29726 29728 20898 0 -1 4202560 1277122 1259 0 0 81138 193 0 0 25 0 5 0 654757288 2427781120 564156 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29728/tid=29737] ppid=29726 vsize=2370880 CPUtime=822.11 cores=0,2,4,6
/proc/29728/task/29737/stat : 29737 (plingeling) R 29726 29728 20898 0 -1 4202560 1126815 1259 0 0 82048 163 0 0 25 0 5 0 654757288 2427781120 564156 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29728/tid=29738] ppid=29726 vsize=2370880 CPUtime=822.26 cores=0,2,4,6
/proc/29728/task/29738/stat : 29738 (plingeling) R 29726 29728 20898 0 -1 4202560 645906 1259 0 0 82123 103 0 0 25 0 5 0 654757288 2427781120 564156 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3279.92
Current children cumulated vsize (KiB) 2370880

[startup+882.301 s]
/proc/loadavg: 8.06 8.11 7.42 9/316 29810
/proc/meminfo: memFree=15225280/32950928 swapFree=67111348/67111528
[pid=29728] ppid=29726 vsize=2406336 CPUtime=3519.32 cores=0,2,4,6
/proc/29728/stat : 29728 (plingeling) S 29726 29728 20898 0 -1 4202496 4394993 1259 0 0 351276 656 0 0 25 0 5 0 654757285 2464088064 573020 33554432000 4194304 4417452 140733983650384 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29728/statm: 601584 573020 180 55 0 599470 0
[pid=29728/tid=29735] ppid=29726 vsize=2406336 CPUtime=882.2 cores=0,2,4,6
/proc/29728/task/29735/stat : 29735 (plingeling) R 29726 29728 20898 0 -1 4202560 1270854 1259 0 0 88035 185 0 0 25 0 5 0 654757288 2464088064 573020 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29728/tid=29736] ppid=29726 vsize=2406336 CPUtime=872.7 cores=0,2,4,6
/proc/29728/task/29736/stat : 29736 (plingeling) R 29726 29728 20898 0 -1 4202560 1318928 1259 0 0 87071 199 0 0 25 0 5 0 654757288 2464088064 573020 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29728/tid=29737] ppid=29726 vsize=2406336 CPUtime=882.1 cores=0,2,4,6
/proc/29728/task/29737/stat : 29737 (plingeling) R 29726 29728 20898 0 -1 4202560 1157755 1259 0 0 88043 167 0 0 25 0 5 0 654757288 2464088064 573020 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29728/tid=29738] ppid=29726 vsize=2406336 CPUtime=882.26 cores=0,2,4,6
/proc/29728/task/29738/stat : 29738 (plingeling) R 29726 29728 20898 0 -1 4202560 645906 1259 0 0 88123 103 0 0 25 0 5 0 654757288 2464088064 573020 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3519.32
Current children cumulated vsize (KiB) 2406336

[startup+942.301 s]
/proc/loadavg: 8.13 8.11 7.46 9/316 29811
/proc/meminfo: memFree=15123812/32950928 swapFree=67111348/67111528
[pid=29728] ppid=29726 vsize=2404700 CPUtime=3758.72 cores=0,2,4,6
/proc/29728/stat : 29728 (plingeling) S 29726 29728 20898 0 -1 4202496 4596705 1259 0 0 375188 684 0 0 25 0 5 0 654757285 2462412800 572611 33554432000 4194304 4417452 140733983650384 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29728/statm: 601175 572611 180 55 0 599061 0
[pid=29728/tid=29735] ppid=29726 vsize=2404700 CPUtime=942.19 cores=0,2,4,6
/proc/29728/task/29735/stat : 29735 (plingeling) R 29726 29728 20898 0 -1 4202560 1318175 1259 0 0 94028 191 0 0 25 0 5 0 654757288 2462412800 572611 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29728/tid=29736] ppid=29726 vsize=2404700 CPUtime=932.11 cores=0,2,4,6
/proc/29728/task/29736/stat : 29736 (plingeling) R 29726 29728 20898 0 -1 4202560 1318928 1259 0 0 93012 199 0 0 25 0 5 0 654757288 2462412800 572611 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29728/tid=29737] ppid=29726 vsize=2404700 CPUtime=942.09 cores=0,2,4,6
/proc/29728/task/29737/stat : 29737 (plingeling) R 29726 29728 20898 0 -1 4202560 1278772 1259 0 0 94024 185 0 0 25 0 5 0 654757288 2462412800 572611 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29728/tid=29738] ppid=29726 vsize=2404700 CPUtime=942.26 cores=0,2,4,6
/proc/29728/task/29738/stat : 29738 (plingeling) R 29726 29728 20898 0 -1 4202560 679280 1259 0 0 94119 107 0 0 25 0 5 0 654757288 2462412800 572611 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364724 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3758.72
Current children cumulated vsize (KiB) 2404700

[startup+1002.3 s]
/proc/loadavg: 8.05 8.09 7.49 9/316 29813
/proc/meminfo: memFree=15158616/32950928 swapFree=67111348/67111528
[pid=29728] ppid=29726 vsize=2431376 CPUtime=3998.11 cores=0,2,4,6
/proc/29728/stat : 29728 (plingeling) S 29726 29728 20898 0 -1 4202496 4708688 1259 0 0 399109 702 0 0 25 0 5 0 654757285 2489729024 579280 33554432000 4194304 4417452 140733983650384 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29728/statm: 607844 579280 180 55 0 605730 0
[pid=29728/tid=29735] ppid=29726 vsize=2431376 CPUtime=1002.2 cores=0,2,4,6
/proc/29728/task/29735/stat : 29735 (plingeling) R 29726 29728 20898 0 -1 4202560 1340079 1259 0 0 100026 194 0 0 25 0 5 0 654757288 2489729024 579280 33554432000 4194304 4417452 140733983650384 18446744073709551615 4234650 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29728/tid=29736] ppid=29726 vsize=2431376 CPUtime=991.5 cores=0,2,4,6
/proc/29728/task/29736/stat : 29736 (plingeling) R 29726 29728 20898 0 -1 4202560 1347814 1259 0 0 98947 203 0 0 25 0 5 0 654757288 2489729024 579280 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29728/tid=29737] ppid=29726 vsize=2431376 CPUtime=1002.07 cores=0,2,4,6
/proc/29728/task/29737/stat : 29737 (plingeling) R 29726 29728 20898 0 -1 4202560 1307025 1259 0 0 100019 188 0 0 25 0 5 0 654757288 2489729024 579280 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29728/tid=29738] ppid=29726 vsize=2431376 CPUtime=1002.27 cores=0,2,4,6
/proc/29728/task/29738/stat : 29738 (plingeling) R 29726 29728 20898 0 -1 4202560 712220 1259 0 0 100112 115 0 0 25 0 5 0 654757288 2489729024 579280 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3998.11
Current children cumulated vsize (KiB) 2431376

[startup+1062.3 s]
/proc/loadavg: 8.05 8.09 7.53 9/316 29815
/proc/meminfo: memFree=15219592/32950928 swapFree=67111348/67111528
[pid=29728] ppid=29726 vsize=2506988 CPUtime=4237.51 cores=0,2,4,6
/proc/29728/stat : 29728 (plingeling) S 29726 29728 20898 0 -1 4202496 4922364 1259 0 0 423011 740 0 0 25 0 5 0 654757285 2567155712 598183 33554432000 4194304 4417452 140733983650384 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29728/statm: 626747 598183 180 55 0 624633 0
[pid=29728/tid=29735] ppid=29726 vsize=2506988 CPUtime=1062.2 cores=0,2,4,6
/proc/29728/task/29735/stat : 29735 (plingeling) R 29726 29728 20898 0 -1 4202560 1476475 1259 0 0 106001 219 0 0 25 0 5 0 654757288 2567155712 598183 33554432000 4194304 4417452 140733983650384 18446744073709551615 4344147 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29728/tid=29736] ppid=29726 vsize=2506988 CPUtime=1050.92 cores=0,2,4,6
/proc/29728/task/29736/stat : 29736 (plingeling) R 29726 29728 20898 0 -1 4202560 1393548 1259 0 0 104881 211 0 0 25 0 5 0 654757288 2567155712 598183 33554432000 4194304 4417452 140733983650384 18446744073709551615 4366065 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29728/tid=29737] ppid=29726 vsize=2506988 CPUtime=1062.07 cores=0,2,4,6
/proc/29728/task/29737/stat : 29737 (plingeling) R 29726 29728 20898 0 -1 4202560 1338571 1259 0 0 106013 194 0 0 25 0 5 0 654757288 2567155712 598183 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29728/tid=29738] ppid=29726 vsize=2506988 CPUtime=1062.27 cores=0,2,4,6
/proc/29728/task/29738/stat : 29738 (plingeling) R 29726 29728 20898 0 -1 4202560 712220 1259 0 0 106112 115 0 0 25 0 5 0 654757288 2567155712 598183 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4237.51
Current children cumulated vsize (KiB) 2506988

[startup+1122.3 s]
/proc/loadavg: 8.10 8.10 7.56 9/316 29816
/proc/meminfo: memFree=14592000/32950928 swapFree=67111348/67111528
[pid=29728] ppid=29726 vsize=2983628 CPUtime=4476.9 cores=0,2,4,6
/proc/29728/stat : 29728 (plingeling) S 29726 29728 20898 0 -1 4202496 5135473 1259 0 0 446920 770 0 0 25 0 5 0 654757285 3055235072 717343 33554432000 4194304 4417452 140733983650384 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29728/statm: 745907 717343 180 55 0 743793 0
[pid=29728/tid=29735] ppid=29726 vsize=2983628 CPUtime=1122.2 cores=0,2,4,6
/proc/29728/task/29735/stat : 29735 (plingeling) R 29726 29728 20898 0 -1 4202560 1476475 1259 0 0 112001 219 0 0 25 0 5 0 654757288 3055235072 717343 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29728/tid=29736] ppid=29726 vsize=2983628 CPUtime=1110.31 cores=0,2,4,6
/proc/29728/task/29736/stat : 29736 (plingeling) R 29726 29728 20898 0 -1 4202560 1483280 1259 0 0 110810 221 0 0 25 0 5 0 654757288 3055235072 717343 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29728/tid=29737] ppid=29726 vsize=2983628 CPUtime=1122.06 cores=0,2,4,6
/proc/29728/task/29737/stat : 29737 (plingeling) R 29726 29728 20898 0 -1 4202560 1371112 1259 0 0 112008 198 0 0 25 0 5 0 654757288 3055235072 717343 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29728/tid=29738] ppid=29726 vsize=2983628 CPUtime=1122.27 cores=0,2,4,6
/proc/29728/task/29738/stat : 29738 (plingeling) R 29726 29728 20898 0 -1 4202560 803056 1259 0 0 112096 131 0 0 25 0 5 0 654757288 3055235072 717343 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4476.9
Current children cumulated vsize (KiB) 2983628

[startup+1182.3 s]
/proc/loadavg: 8.15 8.11 7.60 9/316 29818
/proc/meminfo: memFree=14907048/32950928 swapFree=67111348/67111528
[pid=29728] ppid=29726 vsize=2643748 CPUtime=4716.29 cores=0,2,4,6
/proc/29728/stat : 29728 (plingeling) S 29726 29728 20898 0 -1 4202496 5522913 1259 0 0 470806 823 0 0 25 0 5 0 654757285 2707197952 632373 33554432000 4194304 4417452 140733983650384 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29728/statm: 660937 632373 180 55 0 658823 0
[pid=29728/tid=29735] ppid=29726 vsize=2643748 CPUtime=1182.2 cores=0,2,4,6
/proc/29728/task/29735/stat : 29735 (plingeling) R 29726 29728 20898 0 -1 4202560 1660474 1259 0 0 117980 240 0 0 25 0 5 0 654757288 2707197952 632373 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364821 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29728/tid=29736] ppid=29726 vsize=2643748 CPUtime=1169.72 cores=0,2,4,6
/proc/29728/task/29736/stat : 29736 (plingeling) R 29726 29728 20898 0 -1 4202560 1550999 1259 0 0 116740 232 0 0 25 0 5 0 654757288 2707197952 632373 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29728/tid=29737] ppid=29726 vsize=2643748 CPUtime=1182.05 cores=0,2,4,6
/proc/29728/task/29737/stat : 29737 (plingeling) R 29726 29728 20898 0 -1 4202560 1379577 1259 0 0 118006 199 0 0 25 0 5 0 654757288 2707197952 632373 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29728/tid=29738] ppid=29726 vsize=2643748 CPUtime=1182.27 cores=0,2,4,6
/proc/29728/task/29738/stat : 29738 (plingeling) R 29726 29728 20898 0 -1 4202560 930313 1259 0 0 118076 151 0 0 25 0 5 0 654757288 2707197952 632373 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4716.29
Current children cumulated vsize (KiB) 2643748



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.3 s]
/proc/loadavg: 8.11 8.10 7.61 9/316 29818
/proc/meminfo: memFree=14792960/32950928 swapFree=67111348/67111528
[pid=29728] ppid=29726 vsize=2675784 CPUtime=4800.14 cores=0,2,4,6
/proc/29728/stat : 29728 (plingeling) S 29726 29728 20898 0 -1 4202496 5539645 1259 0 0 479189 825 0 0 25 0 5 0 654757285 2740002816 640382 33554432000 4194304 4417452 140733983650384 18446744073709551615 274485770213 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/29728/statm: 668946 640382 180 55 0 666832 0
[pid=29728/tid=29735] ppid=29726 vsize=2675784 CPUtime=1203.23 cores=0,2,4,6
/proc/29728/task/29735/stat : 29735 (plingeling) R 29726 29728 20898 0 -1 4202560 1660474 1259 0 0 120083 240 0 0 25 0 5 0 654757288 2740002816 640382 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=29728/tid=29736] ppid=29726 vsize=2675784 CPUtime=1190.51 cores=0,2,4,6
/proc/29728/task/29736/stat : 29736 (plingeling) R 29726 29728 20898 0 -1 4202560 1554471 1259 0 0 118818 233 0 0 25 0 5 0 654757288 2740002816 640382 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=29728/tid=29737] ppid=29726 vsize=2675784 CPUtime=1203.08 cores=0,2,4,6
/proc/29728/task/29737/stat : 29737 (plingeling) R 29726 29728 20898 0 -1 4202560 1385735 1259 0 0 120108 200 0 0 25 0 5 0 654757288 2740002816 640382 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=29728/tid=29738] ppid=29726 vsize=2675784 CPUtime=1203.29 cores=0,2,4,6
/proc/29728/task/29738/stat : 29738 (plingeling) R 29726 29728 20898 0 -1 4202560 937415 1259 0 0 120177 152 0 0 25 0 5 0 654757288 2740002816 640382 33554432000 4194304 4417452 140733983650384 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.14
Current children cumulated vsize (KiB) 2675784

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.43 s]
/proc/loadavg: 8.11 8.10 7.61 9/316 29818
/proc/meminfo: memFree=14792960/32950928 swapFree=67111348/67111528
[pid=29728] ppid=29726 vsize=0 CPUtime=4800.27 cores=0,2,4,6
/proc/29728/stat : 29728 (plingeling) Z 29726 29728 20898 0 -1 4203548 5539645 1259 0 0 479192 835 0 0 18 0 2 0 654757285 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/29728/statm: 0 0 0 0 0 0 0
[pid=29728/tid=29736] ppid=29726 vsize=0 CPUtime=1190.62 cores=0,2,4,6
/proc/29728/task/29736/stat : 29736 (plingeling) R 29726 29728 20898 0 -1 4203588 1554471 1259 0 0 118819 243 0 0 25 0 2 0 654757288 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.27
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1203.44
CPU time (s): 4800.3
CPU user time (s): 4791.93
CPU system time (s): 8.37373
CPU usage (%): 398.881
Max. virtual memory (cumulated for all children) (KiB): 3120220

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4791.93
system time used= 8.37373
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 5540904
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 147
involuntary context switches= 26879

runsolver used 3.53846 second user time and 8.48071 second system time

The end

Launcher Data 

Begin job on node125 at 2011-05-15 05:52:56
IDJOB=3371143
IDBENCH=85438
IDSOLVER=1848
FILE ID=node125/3371143-1305431576
RUNJOBID= node125-1305430370-29557
PBS_JOBID= 13324587
Free space on /tmp= 73232 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT05/crafted/sabharwal05/counting/php/unsat/easier/php-045-035.shuffled-as.sat05-1181.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3371143-1305431576/watcher-3371143-1305431576 -o /tmp/evaluation-result-3371143-1305431576/solver-3371143-1305431576 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3371143-1305431576.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 057b78ff24cc0773c9c67e770a7b9de2
RANDOM SEED=407923095

node125.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5266.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      17992700 kB
Buffers:        267084 kB
Cached:        4253940 kB
SwapCached:          0 kB
Active:       11179040 kB
Inactive:      3406064 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      17992700 kB
SwapTotal:    67111528 kB
SwapFree:     67111348 kB
Dirty:             440 kB
Writeback:           0 kB
AnonPages:    10063936 kB
Mapped:          29208 kB
Slab:           274100 kB
PageTables:      35592 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 51053596 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73228 MiB
End job on node125 at 2011-05-15 06:13:02