Trace number 3370915

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.17 1204.54

General information on the benchmark

Namecrafted/kullmann/GreenTao/
GreenTaok1k2/GreenTao_2-4-4_512.cnf
MD5SUMc1f047ec179094963b21d7b90509d2b9
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables510
Number of clauses4492
Sum of the clauses size17968
Maximum clause length4
Minimum clause length4
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 44492
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3370915-1305427003/watcher-3370915-1305427003 -o /tmp/evaluation-result-3370915-1305427003/solver-3370915-1305427003 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3370915-1305427003.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.72 9.12 8.71 6/360 21991
/proc/meminfo: memFree=24052036/32950928 swapFree=45937192/67111528
[pid=21991] ppid=21989 vsize=51080 CPUtime=0 cores=0,2,4,6
/proc/21991/stat : 21991 (plingeling) S 21989 21991 23744 0 -1 4202496 657 1251 0 0 0 0 0 0 25 0 5 0 654296958 52305920 572 33554432000 4194304 4417452 140736334100784 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21991/statm: 12770 572 166 55 0 10656 0
[pid=21991/tid=22000] ppid=21989 vsize=51164 CPUtime=0 cores=0,2,4,6
/proc/21991/task/22000/stat : 22000 (plingeling) R 21989 21991 23744 0 -1 4202560 49 1251 0 0 0 0 0 0 25 0 5 0 654296959 52391936 601 33554432000 4194304 4417452 140736334100784 18446744073709551615 4278378 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21991/tid=22001] ppid=21989 vsize=51164 CPUtime=0 cores=0,2,4,6
/proc/21991/task/22001/stat : 22001 (plingeling) R 21989 21991 23744 0 -1 4202560 0 1251 0 0 0 0 0 0 25 0 5 0 654296959 52391936 601 33554432000 4194304 4417452 140736334100784 18446744073709551615 254500748273 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21991/tid=22002] ppid=21989 vsize=51164 CPUtime=0 cores=0,2,4,6
/proc/21991/task/22002/stat : 22002 (plingeling) R 21989 21991 23744 0 -1 4202560 2 1251 0 0 0 0 0 0 25 0 5 0 654296959 52391936 601 33554432000 4194304 4417452 140736334100784 18446744073709551615 254500796959 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21991/tid=22003] ppid=21989 vsize=51164 CPUtime=0 cores=0,2,4,6
/proc/21991/task/22003/stat : 22003 (plingeling) R 21989 21991 23744 0 -1 4202560 4 1251 0 0 0 0 0 0 25 0 5 0 654296959 52391936 601 33554432000 4194304 4417452 140736334100784 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 2 0 0 0

[startup+0.138033 s]
/proc/loadavg: 8.72 9.12 8.71 6/360 21991
/proc/meminfo: memFree=24052036/32950928 swapFree=45937192/67111528
[pid=21991] ppid=21989 vsize=120732 CPUtime=0.51 cores=0,2,4,6
/proc/21991/stat : 21991 (plingeling) S 21989 21991 23744 0 -1 4202496 1715 1251 0 0 51 0 0 0 25 0 5 0 654296958 123629568 1593 33554432000 4194304 4417452 140736334100784 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21991/statm: 30183 1593 178 55 0 28069 0
[pid=21991/tid=22000] ppid=21989 vsize=120732 CPUtime=0.12 cores=0,2,4,6
/proc/21991/task/22000/stat : 22000 (plingeling) R 21989 21991 23744 0 -1 4202560 263 1251 0 0 12 0 0 0 25 0 5 0 654296959 123629568 1593 33554432000 4194304 4417452 140736334100784 18446744073709551615 4365209 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21991/tid=22001] ppid=21989 vsize=120732 CPUtime=0.12 cores=0,2,4,6
/proc/21991/task/22001/stat : 22001 (plingeling) R 21989 21991 23744 0 -1 4202560 198 1251 0 0 12 0 0 0 25 0 5 0 654296959 123629568 1593 33554432000 4194304 4417452 140736334100784 18446744073709551615 4250513 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21991/tid=22002] ppid=21989 vsize=120732 CPUtime=0.12 cores=0,2,4,6
/proc/21991/task/22002/stat : 22002 (plingeling) R 21989 21991 23744 0 -1 4202560 245 1251 0 0 12 0 0 0 25 0 5 0 654296959 123629568 1593 33554432000 4194304 4417452 140736334100784 18446744073709551615 4283536 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21991/tid=22003] ppid=21989 vsize=120732 CPUtime=0.12 cores=0,2,4,6
/proc/21991/task/22003/stat : 22003 (plingeling) R 21989 21991 23744 0 -1 4202560 367 1251 0 0 12 0 0 0 25 0 5 0 654296959 123629568 1593 33554432000 4194304 4417452 140736334100784 18446744073709551615 4357207 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 0.51
Current children cumulated vsize (KiB) 120732

[startup+0.201021 s]
/proc/loadavg: 8.72 9.12 8.71 6/360 21991
/proc/meminfo: memFree=24052036/32950928 swapFree=45937192/67111528
[pid=21991] ppid=21989 vsize=120752 CPUtime=0.76 cores=0,2,4,6
/proc/21991/stat : 21991 (plingeling) S 21989 21991 23744 0 -1 4202496 2020 1251 0 0 76 0 0 0 25 0 5 0 654296958 123650048 1627 33554432000 4194304 4417452 140736334100784 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21991/statm: 30188 1627 178 55 0 28074 0
[pid=21991/tid=22000] ppid=21989 vsize=120752 CPUtime=0.19 cores=0,2,4,6
/proc/21991/task/22000/stat : 22000 (plingeling) R 21989 21991 23744 0 -1 4202560 311 1251 0 0 19 0 0 0 25 0 5 0 654296959 123650048 1627 33554432000 4194304 4417452 140736334100784 18446744073709551615 4236564 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21991/tid=22001] ppid=21989 vsize=120752 CPUtime=0.18 cores=0,2,4,6
/proc/21991/task/22001/stat : 22001 (plingeling) R 21989 21991 23744 0 -1 4202560 274 1251 0 0 18 0 0 0 25 0 5 0 654296959 123650048 1627 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364542 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21991/tid=22002] ppid=21989 vsize=120752 CPUtime=0.18 cores=0,2,4,6
/proc/21991/task/22002/stat : 22002 (plingeling) R 21989 21991 23744 0 -1 4202560 310 1251 0 0 18 0 0 0 25 0 5 0 654296959 123650048 1627 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21991/tid=22003] ppid=21989 vsize=120752 CPUtime=0.18 cores=0,2,4,6
/proc/21991/task/22003/stat : 22003 (plingeling) R 21989 21991 23744 0 -1 4202560 483 1251 0 0 18 0 0 0 25 0 5 0 654296959 123650048 1627 33554432000 4194304 4417452 140736334100784 18446744073709551615 4251258 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 0.76
Current children cumulated vsize (KiB) 120752

[startup+0.301002 s]
/proc/loadavg: 8.72 9.12 8.71 6/360 21991
/proc/meminfo: memFree=24052036/32950928 swapFree=45937192/67111528
[pid=21991] ppid=21989 vsize=121136 CPUtime=1.16 cores=0,2,4,6
/proc/21991/stat : 21991 (plingeling) S 21989 21991 23744 0 -1 4202496 2731 1251 0 0 116 0 0 0 25 0 5 0 654296958 124043264 1733 33554432000 4194304 4417452 140736334100784 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21991/statm: 30284 1733 178 55 0 28170 0
[pid=21991/tid=22000] ppid=21989 vsize=121136 CPUtime=0.29 cores=0,2,4,6
/proc/21991/task/22000/stat : 22000 (plingeling) R 21989 21991 23744 0 -1 4202560 492 1251 0 0 29 0 0 0 25 0 5 0 654296959 124043264 1733 33554432000 4194304 4417452 140736334100784 18446744073709551615 4199406 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21991/tid=22001] ppid=21989 vsize=121136 CPUtime=0.28 cores=0,2,4,6
/proc/21991/task/22001/stat : 22001 (plingeling) R 21989 21991 23744 0 -1 4202560 466 1251 0 0 28 0 0 0 25 0 5 0 654296959 124043264 1733 33554432000 4194304 4417452 140736334100784 18446744073709551615 4250873 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21991/tid=22002] ppid=21989 vsize=121136 CPUtime=0.28 cores=0,2,4,6
/proc/21991/task/22002/stat : 22002 (plingeling) R 21989 21991 23744 0 -1 4202560 480 1251 0 0 28 0 0 0 25 0 5 0 654296959 124043264 1733 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21991/tid=22003] ppid=21989 vsize=121136 CPUtime=0.28 cores=0,2,4,6
/proc/21991/task/22003/stat : 22003 (plingeling) R 21989 21991 23744 0 -1 4202560 651 1251 0 0 28 0 0 0 25 0 5 0 654296959 124043264 1733 33554432000 4194304 4417452 140736334100784 18446744073709551615 4365075 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 1.16
Current children cumulated vsize (KiB) 121136

[startup+0.700923 s]
/proc/loadavg: 8.72 9.12 8.71 6/360 21991
/proc/meminfo: memFree=24052036/32950928 swapFree=45937192/67111528
[pid=21991] ppid=21989 vsize=123324 CPUtime=2.75 cores=0,2,4,6
/proc/21991/stat : 21991 (plingeling) S 21989 21991 23744 0 -1 4202496 5602 1251 0 0 275 0 0 0 25 0 5 0 654296958 126283776 2280 33554432000 4194304 4417452 140736334100784 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21991/statm: 30831 2280 178 55 0 28717 0
[pid=21991/tid=22000] ppid=21989 vsize=123324 CPUtime=0.68 cores=0,2,4,6
/proc/21991/task/22000/stat : 22000 (plingeling) R 21989 21991 23744 0 -1 4202560 1220 1251 0 0 68 0 0 0 25 0 5 0 654296959 126283776 2280 33554432000 4194304 4417452 140736334100784 18446744073709551615 4366543 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21991/tid=22001] ppid=21989 vsize=123324 CPUtime=0.68 cores=0,2,4,6
/proc/21991/task/22001/stat : 22001 (plingeling) R 21989 21991 23744 0 -1 4202560 1238 1251 0 0 68 0 0 0 25 0 5 0 654296959 126283776 2280 33554432000 4194304 4417452 140736334100784 18446744073709551615 4352536 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21991/tid=22002] ppid=21989 vsize=123324 CPUtime=0.68 cores=0,2,4,6
/proc/21991/task/22002/stat : 22002 (plingeling) R 21989 21991 23744 0 -1 4202560 1092 1251 0 0 68 0 0 0 25 0 5 0 654296959 126283776 2280 33554432000 4194304 4417452 140736334100784 18446744073709551615 4353585 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21991/tid=22003] ppid=21989 vsize=123324 CPUtime=0.68 cores=0,2,4,6
/proc/21991/task/22003/stat : 22003 (plingeling) R 21989 21991 23744 0 -1 4202560 1410 1251 0 0 68 0 0 0 25 0 5 0 654296959 126283776 2280 33554432000 4194304 4417452 140736334100784 18446744073709551615 4366126 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 2.75
Current children cumulated vsize (KiB) 123324

[startup+1.50076 s]
/proc/loadavg: 8.72 9.12 8.71 10/366 22024
/proc/meminfo: memFree=24027976/32950928 swapFree=45937192/67111528
[pid=21991] ppid=21989 vsize=126312 CPUtime=5.93 cores=0,2,4,6
/proc/21991/stat : 21991 (plingeling) S 21989 21991 23744 0 -1 4202496 10879 1251 0 0 592 1 0 0 25 0 5 0 654296958 129343488 3027 33554432000 4194304 4417452 140736334100784 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21991/statm: 31578 3027 178 55 0 29464 0
[pid=21991/tid=22000] ppid=21989 vsize=126312 CPUtime=1.48 cores=0,2,4,6
/proc/21991/task/22000/stat : 22000 (plingeling) R 21989 21991 23744 0 -1 4202560 2922 1251 0 0 148 0 0 0 25 0 5 0 654296959 129343488 3027 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364537 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21991/tid=22001] ppid=21989 vsize=126312 CPUtime=1.46 cores=0,2,4,6
/proc/21991/task/22001/stat : 22001 (plingeling) R 21989 21991 23744 0 -1 4202560 2398 1251 0 0 146 0 0 0 25 0 5 0 654296959 129343488 3027 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364632 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21991/tid=22002] ppid=21989 vsize=126312 CPUtime=1.48 cores=0,2,4,6
/proc/21991/task/22002/stat : 22002 (plingeling) R 21989 21991 23744 0 -1 4202560 2406 1251 0 0 148 0 0 0 25 0 5 0 654296959 129343488 3027 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21991/tid=22003] ppid=21989 vsize=126312 CPUtime=1.48 cores=0,2,4,6
/proc/21991/task/22003/stat : 22003 (plingeling) R 21989 21991 23744 0 -1 4202560 2511 1251 0 0 148 0 0 0 25 0 5 0 654296959 129343488 3027 33554432000 4194304 4417452 140736334100784 18446744073709551615 4353830 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 5.93
Current children cumulated vsize (KiB) 126312

[startup+3.10045 s]
/proc/loadavg: 8.72 9.12 8.71 10/367 22052
/proc/meminfo: memFree=24024248/32950928 swapFree=45937192/67111528
[pid=21991] ppid=21989 vsize=132672 CPUtime=12.3 cores=0,2,4,6
/proc/21991/stat : 21991 (plingeling) S 21989 21991 23744 0 -1 4202496 21151 1251 0 0 1226 4 0 0 25 0 5 0 654296958 135856128 4602 33554432000 4194304 4417452 140736334100784 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21991/statm: 33168 4602 178 55 0 31054 0
[pid=21991/tid=22000] ppid=21989 vsize=132672 CPUtime=3.08 cores=0,2,4,6
/proc/21991/task/22000/stat : 22000 (plingeling) R 21989 21991 23744 0 -1 4202560 5452 1251 0 0 307 1 0 0 25 0 5 0 654296959 135856128 4602 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364537 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21991/tid=22001] ppid=21989 vsize=132672 CPUtime=3.04 cores=0,2,4,6
/proc/21991/task/22001/stat : 22001 (plingeling) R 21989 21991 23744 0 -1 4202560 4631 1251 0 0 303 1 0 0 25 0 5 0 654296959 135856128 4602 33554432000 4194304 4417452 140736334100784 18446744073709551615 4358357 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21991/tid=22002] ppid=21989 vsize=132672 CPUtime=3.06 cores=0,2,4,6
/proc/21991/task/22002/stat : 22002 (plingeling) R 21989 21991 23744 0 -1 4202560 5256 1251 0 0 306 0 0 0 25 0 5 0 654296959 135856128 4602 33554432000 4194304 4417452 140736334100784 18446744073709551615 4355870 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21991/tid=22003] ppid=21989 vsize=132672 CPUtime=3.08 cores=0,2,4,6
/proc/21991/task/22003/stat : 22003 (plingeling) R 21989 21991 23744 0 -1 4202560 5170 1251 0 0 307 1 0 0 25 0 5 0 654296959 135856128 4602 33554432000 4194304 4417452 140736334100784 18446744073709551615 4251086 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 12.3
Current children cumulated vsize (KiB) 132672

[startup+6.30081 s]
/proc/loadavg: 8.82 9.13 8.71 10/367 22162
/proc/meminfo: memFree=24016080/32950928 swapFree=45937192/67111528
[pid=21991] ppid=21989 vsize=134636 CPUtime=25.01 cores=0,2,4,6
/proc/21991/stat : 21991 (plingeling) S 21989 21991 23744 0 -1 4202496 35469 1251 0 0 2494 7 0 0 25 0 5 0 654296958 137867264 5108 33554432000 4194304 4417452 140736334100784 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21991/statm: 33659 5108 178 55 0 31545 0
[pid=21991/tid=22000] ppid=21989 vsize=134636 CPUtime=6.29 cores=0,2,4,6
/proc/21991/task/22000/stat : 22000 (plingeling) R 21989 21991 23744 0 -1 4202560 9741 1251 0 0 627 2 0 0 25 0 5 0 654296959 137867264 5108 33554432000 4194304 4417452 140736334100784 18446744073709551615 4357207 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21991/tid=22001] ppid=21989 vsize=134636 CPUtime=6.21 cores=0,2,4,6
/proc/21991/task/22001/stat : 22001 (plingeling) R 21989 21991 23744 0 -1 4202560 7984 1251 0 0 619 2 0 0 25 0 5 0 654296959 137867264 5108 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21991/tid=22002] ppid=21989 vsize=134636 CPUtime=6.22 cores=0,2,4,6
/proc/21991/task/22002/stat : 22002 (plingeling) R 21989 21991 23744 0 -1 4202560 8419 1251 0 0 621 1 0 0 25 0 5 0 654296959 137867264 5108 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364653 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21991/tid=22003] ppid=21989 vsize=134636 CPUtime=6.26 cores=0,2,4,6
/proc/21991/task/22003/stat : 22003 (plingeling) R 21989 21991 23744 0 -1 4202560 8683 1251 0 0 625 1 0 0 25 0 5 0 654296959 137867264 5108 33554432000 4194304 4417452 140736334100784 18446744073709551615 4354506 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 25.01
Current children cumulated vsize (KiB) 134636

[startup+12.7005 s]
/proc/loadavg: 8.84 9.13 8.71 10/366 22377
/proc/meminfo: memFree=23990252/32950928 swapFree=45937192/67111528
[pid=21991] ppid=21989 vsize=149732 CPUtime=50.43 cores=0,2,4,6
/proc/21991/stat : 21991 (plingeling) S 21989 21991 23744 0 -1 4202496 60101 1251 0 0 5031 12 0 0 25 0 5 0 654296958 153325568 8859 33554432000 4194304 4417452 140736334100784 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21991/statm: 37433 8859 178 55 0 35319 0
[pid=21991/tid=22000] ppid=21989 vsize=149732 CPUtime=12.68 cores=0,2,4,6
/proc/21991/task/22000/stat : 22000 (plingeling) R 21989 21991 23744 0 -1 4202560 15219 1251 0 0 1265 3 0 0 25 0 5 0 654296959 153325568 8859 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364621 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21991/tid=22001] ppid=21989 vsize=149732 CPUtime=12.52 cores=0,2,4,6
/proc/21991/task/22001/stat : 22001 (plingeling) R 21989 21991 23744 0 -1 4202560 14363 1251 0 0 1249 3 0 0 25 0 5 0 654296959 153325568 8859 33554432000 4194304 4417452 140736334100784 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21991/tid=22002] ppid=21989 vsize=149732 CPUtime=12.56 cores=0,2,4,6
/proc/21991/task/22002/stat : 22002 (plingeling) R 21989 21991 23744 0 -1 4202560 14993 1251 0 0 1254 2 0 0 25 0 5 0 654296959 153325568 8859 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21991/tid=22003] ppid=21989 vsize=149732 CPUtime=12.63 cores=0,2,4,6
/proc/21991/task/22003/stat : 22003 (plingeling) R 21989 21991 23744 0 -1 4202560 14884 1251 0 0 1260 3 0 0 25 0 5 0 654296959 153325568 8859 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364572 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 50.43
Current children cumulated vsize (KiB) 149732

[startup+25.501 s]
/proc/loadavg: 8.95 9.14 8.72 10/367 22796
/proc/meminfo: memFree=23961152/32950928 swapFree=45937192/67111528
[pid=21991] ppid=21989 vsize=158548 CPUtime=101.3 cores=0,2,4,6
/proc/21991/stat : 21991 (plingeling) S 21989 21991 23744 0 -1 4202496 107344 1251 0 0 10110 20 0 0 25 0 5 0 654296958 162353152 11054 33554432000 4194304 4417452 140736334100784 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21991/statm: 39637 11054 178 55 0 37523 0
[pid=21991/tid=22000] ppid=21989 vsize=158548 CPUtime=25.41 cores=0,2,4,6
/proc/21991/task/22000/stat : 22000 (plingeling) R 21989 21991 23744 0 -1 4202560 25784 1251 0 0 2537 4 0 0 25 0 5 0 654296959 162353152 11054 33554432000 4194304 4417452 140736334100784 18446744073709551615 4365027 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21991/tid=22001] ppid=21989 vsize=158548 CPUtime=25.14 cores=0,2,4,6
/proc/21991/task/22001/stat : 22001 (plingeling) R 21989 21991 23744 0 -1 4202560 27976 1251 0 0 2509 5 0 0 25 0 5 0 654296959 162353152 11054 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21991/tid=22002] ppid=21989 vsize=158548 CPUtime=25.33 cores=0,2,4,6
/proc/21991/task/22002/stat : 22002 (plingeling) R 21989 21991 23744 0 -1 4202560 28996 1251 0 0 2528 5 0 0 25 0 5 0 654296959 162353152 11054 33554432000 4194304 4417452 140736334100784 18446744073709551615 4236544 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21991/tid=22003] ppid=21989 vsize=158548 CPUtime=25.39 cores=0,2,4,6
/proc/21991/task/22003/stat : 22003 (plingeling) R 21989 21991 23744 0 -1 4202560 23946 1251 0 0 2535 4 0 0 25 0 5 0 654296959 162353152 11054 33554432000 4194304 4417452 140736334100784 18446744073709551615 254500749738 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 101.3
Current children cumulated vsize (KiB) 158548

[startup+51.1009 s]
/proc/loadavg: 8.96 9.13 8.73 10/367 23559
/proc/meminfo: memFree=23935368/32950928 swapFree=45937192/67111528
[pid=21991] ppid=21989 vsize=169960 CPUtime=203 cores=0,2,4,6
/proc/21991/stat : 21991 (plingeling) S 21989 21991 23744 0 -1 4202496 181683 1251 0 0 20268 32 0 0 25 0 5 0 654296958 174039040 13939 33554432000 4194304 4417452 140736334100784 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21991/statm: 42490 13939 178 55 0 40376 0
[pid=21991/tid=22000] ppid=21989 vsize=169960 CPUtime=50.94 cores=0,2,4,6
/proc/21991/task/22000/stat : 22000 (plingeling) R 21989 21991 23744 0 -1 4202560 42196 1251 0 0 5088 6 0 0 25 0 5 0 654296959 174039040 13939 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364626 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21991/tid=22001] ppid=21989 vsize=169960 CPUtime=50.39 cores=0,2,4,6
/proc/21991/task/22001/stat : 22001 (plingeling) R 21989 21991 23744 0 -1 4202560 46562 1251 0 0 5031 8 0 0 25 0 5 0 654296959 174039040 13939 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21991/tid=22002] ppid=21989 vsize=169960 CPUtime=50.64 cores=0,2,4,6

################
# More data... #
################

[pid=21991/tid=22001] ppid=21989 vsize=282348 CPUtime=693.7 cores=0,2,4,6
/proc/21991/task/22001/stat : 22001 (plingeling) R 21989 21991 23744 0 -1 4202560 232992 1251 0 0 69333 37 0 0 25 0 5 0 654296959 289124352 42004 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21991/tid=22002] ppid=21989 vsize=282348 CPUtime=700.77 cores=0,2,4,6
/proc/21991/task/22002/stat : 22002 (plingeling) R 21989 21991 23744 0 -1 4202560 270036 1251 0 0 70033 44 0 0 25 0 5 0 654296959 289124352 42004 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21991/tid=22003] ppid=21989 vsize=282348 CPUtime=701.55 cores=0,2,4,6
/proc/21991/task/22003/stat : 22003 (plingeling) R 21989 21991 23744 0 -1 4202560 239736 1251 0 0 70116 39 0 0 25 0 5 0 654296959 289124352 42004 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 2796.77
Current children cumulated vsize (KiB) 282348

[startup+762.3 s]
/proc/loadavg: 8.16 8.35 8.54 9/359 30463
/proc/meminfo: memFree=23681476/32950928 swapFree=45937192/67111528
[pid=21991] ppid=21989 vsize=290656 CPUtime=3036.13 cores=0,2,4,6
/proc/21991/stat : 21991 (plingeling) S 21989 21991 23744 0 -1 4202496 1052006 1251 0 0 303445 168 0 0 25 0 5 0 654296958 297631744 44081 33554432000 4194304 4417452 140736334100784 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21991/statm: 72664 44081 178 55 0 70550 0
[pid=21991/tid=22000] ppid=21989 vsize=290656 CPUtime=760.72 cores=0,2,4,6
/proc/21991/task/22000/stat : 22000 (plingeling) R 21989 21991 23744 0 -1 4202560 269019 1251 0 0 76032 40 0 0 25 0 5 0 654296959 297631744 44081 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21991/tid=22001] ppid=21989 vsize=290656 CPUtime=753.06 cores=0,2,4,6
/proc/21991/task/22001/stat : 22001 (plingeling) R 21989 21991 23744 0 -1 4202560 245943 1251 0 0 75267 39 0 0 25 0 5 0 654296959 297631744 44081 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21991/tid=22002] ppid=21989 vsize=290656 CPUtime=760.79 cores=0,2,4,6
/proc/21991/task/22002/stat : 22002 (plingeling) R 21989 21991 23744 0 -1 4202560 282468 1251 0 0 76032 47 0 0 25 0 5 0 654296959 297631744 44081 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21991/tid=22003] ppid=21989 vsize=290656 CPUtime=761.55 cores=0,2,4,6
/proc/21991/task/22003/stat : 22003 (plingeling) R 21989 21991 23744 0 -1 4202560 253934 1251 0 0 76113 42 0 0 25 0 5 0 654296959 297631744 44081 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3036.13
Current children cumulated vsize (KiB) 290656

[startup+822.3 s]
/proc/loadavg: 8.16 8.31 8.52 9/359 30465
/proc/meminfo: memFree=23687992/32950928 swapFree=45937192/67111528
[pid=21991] ppid=21989 vsize=286996 CPUtime=3275.44 cores=0,2,4,6
/proc/21991/stat : 21991 (plingeling) S 21989 21991 23744 0 -1 4202496 1098336 1251 0 0 327369 175 0 0 25 0 5 0 654296958 293883904 43194 33554432000 4194304 4417452 140736334100784 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21991/statm: 71749 43194 178 55 0 69635 0
[pid=21991/tid=22000] ppid=21989 vsize=286996 CPUtime=820.71 cores=0,2,4,6
/proc/21991/task/22000/stat : 22000 (plingeling) R 21989 21991 23744 0 -1 4202560 281881 1251 0 0 82030 41 0 0 25 0 5 0 654296959 293883904 43194 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21991/tid=22001] ppid=21989 vsize=286996 CPUtime=812.38 cores=0,2,4,6
/proc/21991/task/22001/stat : 22001 (plingeling) R 21989 21991 23744 0 -1 4202560 257373 1251 0 0 81198 40 0 0 25 0 5 0 654296959 293883904 43194 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21991/tid=22002] ppid=21989 vsize=286996 CPUtime=820.77 cores=0,2,4,6
/proc/21991/task/22002/stat : 22002 (plingeling) R 21989 21991 23744 0 -1 4202560 293766 1251 0 0 82029 48 0 0 25 0 5 0 654296959 293883904 43194 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21991/tid=22003] ppid=21989 vsize=286996 CPUtime=821.53 cores=0,2,4,6
/proc/21991/task/22003/stat : 22003 (plingeling) R 21989 21991 23744 0 -1 4202560 264674 1251 0 0 82109 44 0 0 25 0 5 0 654296959 293883904 43194 33554432000 4194304 4417452 140736334100784 18446744073709551615 4366105 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3275.44
Current children cumulated vsize (KiB) 286996

[startup+882.313 s]
/proc/loadavg: 8.15 8.28 8.49 9/359 30466
/proc/meminfo: memFree=23677272/32950928 swapFree=45937192/67111528
[pid=21991] ppid=21989 vsize=293180 CPUtime=3514.83 cores=0,2,4,6
/proc/21991/stat : 21991 (plingeling) S 21989 21991 23744 0 -1 4202496 1154316 1251 0 0 351298 185 0 0 25 0 5 0 654296958 300216320 44740 33554432000 4194304 4417452 140736334100784 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21991/statm: 73295 44740 178 55 0 71181 0
[pid=21991/tid=22000] ppid=21989 vsize=293180 CPUtime=880.72 cores=0,2,4,6
/proc/21991/task/22000/stat : 22000 (plingeling) R 21989 21991 23744 0 -1 4202560 299228 1251 0 0 88027 45 0 0 25 0 5 0 654296959 300216320 44740 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21991/tid=22001] ppid=21989 vsize=293180 CPUtime=871.74 cores=0,2,4,6
/proc/21991/task/22001/stat : 22001 (plingeling) R 21989 21991 23744 0 -1 4202560 268190 1251 0 0 87132 42 0 0 25 0 5 0 654296959 300216320 44740 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21991/tid=22002] ppid=21989 vsize=293180 CPUtime=880.79 cores=0,2,4,6
/proc/21991/task/22002/stat : 22002 (plingeling) R 21989 21991 23744 0 -1 4202560 308003 1251 0 0 88029 50 0 0 25 0 5 0 654296959 300216320 44740 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21991/tid=22003] ppid=21989 vsize=293180 CPUtime=881.54 cores=0,2,4,6
/proc/21991/task/22003/stat : 22003 (plingeling) R 21989 21991 23744 0 -1 4202560 278253 1251 0 0 88108 46 0 0 25 0 5 0 654296959 300216320 44740 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364972 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3514.83
Current children cumulated vsize (KiB) 293180

[startup+942.301 s]
/proc/loadavg: 8.05 8.23 8.46 9/359 30468
/proc/meminfo: memFree=23658996/32950928 swapFree=45937192/67111528
[pid=21991] ppid=21989 vsize=302724 CPUtime=3754.12 cores=0,2,4,6
/proc/21991/stat : 21991 (plingeling) S 21989 21991 23744 0 -1 4202496 1202664 1251 0 0 375219 193 0 0 25 0 5 0 654296958 309989376 47126 33554432000 4194304 4417452 140736334100784 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21991/statm: 75681 47126 178 55 0 73567 0
[pid=21991/tid=22000] ppid=21989 vsize=302724 CPUtime=940.71 cores=0,2,4,6
/proc/21991/task/22000/stat : 22000 (plingeling) R 21989 21991 23744 0 -1 4202560 311975 1251 0 0 94024 47 0 0 25 0 5 0 654296959 309989376 47126 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364724 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21991/tid=22001] ppid=21989 vsize=302724 CPUtime=931.08 cores=0,2,4,6
/proc/21991/task/22001/stat : 22001 (plingeling) R 21989 21991 23744 0 -1 4202560 279528 1251 0 0 93064 44 0 0 25 0 5 0 654296959 309989376 47126 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21991/tid=22002] ppid=21989 vsize=302724 CPUtime=940.79 cores=0,2,4,6
/proc/21991/task/22002/stat : 22002 (plingeling) R 21989 21991 23744 0 -1 4202560 321037 1251 0 0 94026 53 0 0 25 0 5 0 654296959 309989376 47126 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21991/tid=22003] ppid=21989 vsize=302724 CPUtime=941.5 cores=0,2,4,6
/proc/21991/task/22003/stat : 22003 (plingeling) R 21989 21991 23744 0 -1 4202560 289482 1251 0 0 94103 47 0 0 25 0 5 0 654296959 309989376 47126 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364807 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3754.12
Current children cumulated vsize (KiB) 302724

[startup+1002.3 s]
/proc/loadavg: 8.05 8.20 8.43 9/359 30470
/proc/meminfo: memFree=23639596/32950928 swapFree=45937192/67111528
[pid=21991] ppid=21989 vsize=329352 CPUtime=3993.46 cores=0,2,4,6
/proc/21991/stat : 21991 (plingeling) S 21989 21991 23744 0 -1 4202496 1254747 1251 0 0 399143 203 0 0 25 0 5 0 654296958 337256448 53783 33554432000 4194304 4417452 140736334100784 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21991/statm: 82338 53783 178 55 0 80224 0
[pid=21991/tid=22000] ppid=21989 vsize=329352 CPUtime=1000.72 cores=0,2,4,6
/proc/21991/task/22000/stat : 22000 (plingeling) R 21989 21991 23744 0 -1 4202560 320570 1251 0 0 100023 49 0 0 25 0 5 0 654296959 337256448 53783 33554432000 4194304 4417452 140736334100784 18446744073709551615 4365027 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21991/tid=22001] ppid=21989 vsize=329352 CPUtime=990.42 cores=0,2,4,6
/proc/21991/task/22001/stat : 22001 (plingeling) R 21989 21991 23744 0 -1 4202560 293258 1251 0 0 98995 47 0 0 25 0 5 0 654296959 337256448 53783 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21991/tid=22002] ppid=21989 vsize=329352 CPUtime=1000.78 cores=0,2,4,6
/proc/21991/task/22002/stat : 22002 (plingeling) R 21989 21991 23744 0 -1 4202560 337238 1251 0 0 100023 55 0 0 25 0 5 0 654296959 337256448 53783 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364579 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21991/tid=22003] ppid=21989 vsize=329352 CPUtime=1001.5 cores=0,2,4,6
/proc/21991/task/22003/stat : 22003 (plingeling) R 21989 21991 23744 0 -1 4202560 303039 1251 0 0 100100 50 0 0 25 0 5 0 654296959 337256448 53783 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3993.46
Current children cumulated vsize (KiB) 329352

[startup+1062.3 s]
/proc/loadavg: 8.02 8.16 8.40 9/359 30471
/proc/meminfo: memFree=23667320/32950928 swapFree=45937192/67111528
[pid=21991] ppid=21989 vsize=328132 CPUtime=4232.81 cores=0,2,4,6
/proc/21991/stat : 21991 (plingeling) S 21989 21991 23744 0 -1 4202496 1302461 1251 0 0 423071 210 0 0 25 0 5 0 654296958 336007168 53478 33554432000 4194304 4417452 140736334100784 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21991/statm: 82033 53478 178 55 0 79919 0
[pid=21991/tid=22000] ppid=21989 vsize=328132 CPUtime=1060.72 cores=0,2,4,6
/proc/21991/task/22000/stat : 22000 (plingeling) R 21989 21991 23744 0 -1 4202560 333759 1251 0 0 106021 51 0 0 25 0 5 0 654296959 336007168 53478 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21991/tid=22001] ppid=21989 vsize=328132 CPUtime=1049.77 cores=0,2,4,6
/proc/21991/task/22001/stat : 22001 (plingeling) R 21989 21991 23744 0 -1 4202560 302846 1251 0 0 104929 48 0 0 25 0 5 0 654296959 336007168 53478 33554432000 4194304 4417452 140736334100784 18446744073709551615 4365027 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21991/tid=22002] ppid=21989 vsize=328132 CPUtime=1060.79 cores=0,2,4,6
/proc/21991/task/22002/stat : 22002 (plingeling) R 21989 21991 23744 0 -1 4202560 348739 1251 0 0 106022 57 0 0 25 0 5 0 654296959 336007168 53478 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21991/tid=22003] ppid=21989 vsize=328132 CPUtime=1061.49 cores=0,2,4,6
/proc/21991/task/22003/stat : 22003 (plingeling) R 21989 21991 23744 0 -1 4202560 316475 1251 0 0 106097 52 0 0 25 0 5 0 654296959 336007168 53478 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364572 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4232.81
Current children cumulated vsize (KiB) 328132

[startup+1122.3 s]
/proc/loadavg: 8.00 8.13 8.37 9/359 30473
/proc/meminfo: memFree=23673724/32950928 swapFree=45937192/67111528
[pid=21991] ppid=21989 vsize=283160 CPUtime=4472.13 cores=0,2,4,6
/proc/21991/stat : 21991 (plingeling) S 21989 21991 23744 0 -1 4202496 1346006 1251 0 0 446997 216 0 0 25 0 5 0 654296958 289955840 42207 33554432000 4194304 4417452 140736334100784 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21991/statm: 70790 42207 178 55 0 68676 0
[pid=21991/tid=22000] ppid=21989 vsize=283160 CPUtime=1120.72 cores=0,2,4,6
/proc/21991/task/22000/stat : 22000 (plingeling) R 21989 21991 23744 0 -1 4202560 344113 1251 0 0 112020 52 0 0 25 0 5 0 654296959 289955840 42207 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364741 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21991/tid=22001] ppid=21989 vsize=283160 CPUtime=1109.11 cores=0,2,4,6
/proc/21991/task/22001/stat : 22001 (plingeling) R 21989 21991 23744 0 -1 4202560 312397 1251 0 0 110861 50 0 0 25 0 5 0 654296959 289955840 42207 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21991/tid=22002] ppid=21989 vsize=283160 CPUtime=1120.79 cores=0,2,4,6
/proc/21991/task/22002/stat : 22002 (plingeling) R 21989 21991 23744 0 -1 4202560 362356 1251 0 0 112020 59 0 0 25 0 5 0 654296959 289955840 42207 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21991/tid=22003] ppid=21989 vsize=283160 CPUtime=1121.47 cores=0,2,4,6
/proc/21991/task/22003/stat : 22003 (plingeling) R 21989 21991 23744 0 -1 4202560 326498 1251 0 0 112093 54 0 0 25 0 5 0 654296959 289955840 42207 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4472.13
Current children cumulated vsize (KiB) 283160

[startup+1182.3 s]
/proc/loadavg: 8.07 8.12 8.35 9/359 30474
/proc/meminfo: memFree=23685212/32950928 swapFree=45937192/67111528
[pid=21991] ppid=21989 vsize=285224 CPUtime=4711.48 cores=0,2,4,6
/proc/21991/stat : 21991 (plingeling) S 21989 21991 23744 0 -1 4202496 1389071 1251 0 0 470927 221 0 0 25 0 5 0 654296958 292069376 42731 33554432000 4194304 4417452 140736334100784 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21991/statm: 71306 42731 178 55 0 69192 0
[pid=21991/tid=22000] ppid=21989 vsize=285224 CPUtime=1180.73 cores=0,2,4,6
/proc/21991/task/22000/stat : 22000 (plingeling) R 21989 21991 23744 0 -1 4202560 353555 1251 0 0 118020 53 0 0 25 0 5 0 654296959 292069376 42731 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21991/tid=22001] ppid=21989 vsize=285224 CPUtime=1168.46 cores=0,2,4,6
/proc/21991/task/22001/stat : 22001 (plingeling) R 21989 21991 23744 0 -1 4202560 320559 1251 0 0 116795 51 0 0 25 0 5 0 654296959 292069376 42731 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21991/tid=22002] ppid=21989 vsize=285224 CPUtime=1180.79 cores=0,2,4,6
/proc/21991/task/22002/stat : 22002 (plingeling) R 21989 21991 23744 0 -1 4202560 376412 1251 0 0 118019 60 0 0 25 0 5 0 654296959 292069376 42731 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21991/tid=22003] ppid=21989 vsize=285224 CPUtime=1181.47 cores=0,2,4,6
/proc/21991/task/22003/stat : 22003 (plingeling) R 21989 21991 23744 0 -1 4202560 337903 1251 0 0 118091 56 0 0 25 0 5 0 654296959 292069376 42731 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4711.48
Current children cumulated vsize (KiB) 285224



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1204.5 s]
/proc/loadavg: 8.11 8.12 8.35 6/353 30476
/proc/meminfo: memFree=23926416/32950928 swapFree=45937192/67111528
[pid=21991] ppid=21989 vsize=297240 CPUtime=4800.11 cores=0,2,4,6
/proc/21991/stat : 21991 (plingeling) S 21989 21991 23744 0 -1 4202496 1408208 1251 0 0 479787 224 0 0 25 0 5 0 654296958 304373760 45727 33554432000 4194304 4417452 140736334100784 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/21991/statm: 74310 45727 178 55 0 72196 0
[pid=21991/tid=22000] ppid=21989 vsize=297240 CPUtime=1202.95 cores=0,2,4,6
/proc/21991/task/22000/stat : 22000 (plingeling) R 21989 21991 23744 0 -1 4202560 360657 1251 0 0 120241 54 0 0 25 0 5 0 654296959 304373760 45727 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=21991/tid=22001] ppid=21989 vsize=297240 CPUtime=1190.42 cores=0,2,4,6
/proc/21991/task/22001/stat : 22001 (plingeling) R 21989 21991 23744 0 -1 4202560 322167 1251 0 0 118991 51 0 0 25 0 5 0 654296959 304373760 45727 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=21991/tid=22002] ppid=21989 vsize=297240 CPUtime=1203.02 cores=0,2,4,6
/proc/21991/task/22002/stat : 22002 (plingeling) R 21989 21991 23744 0 -1 4202560 384495 1251 0 0 120241 61 0 0 25 0 5 0 654296959 304373760 45727 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=21991/tid=22003] ppid=21989 vsize=297240 CPUtime=1203.7 cores=0,2,4,6
/proc/21991/task/22003/stat : 22003 (plingeling) R 21989 21991 23744 0 -1 4202560 340247 1251 0 0 120314 56 0 0 25 0 5 0 654296959 304373760 45727 33554432000 4194304 4417452 140736334100784 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.11
Current children cumulated vsize (KiB) 297240

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1204.54
CPU time (s): 4800.17
CPU user time (s): 4797.91
CPU system time (s): 2.25666
CPU usage (%): 398.506
Max. virtual memory (cumulated for all children) (KiB): 370532

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4797.91
system time used= 2.25666
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1409459
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 127
involuntary context switches= 33326

runsolver used 3.85141 second user time and 9.59154 second system time

The end

Launcher Data 

Begin job on node149 at 2011-05-15 04:36:43
IDJOB=3370915
IDBENCH=83247
IDSOLVER=1848
FILE ID=node149/3370915-1305427003
RUNJOBID= node149-1305424586-23762
PBS_JOBID= 13324600
Free space on /tmp= 73616 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/crafted/kullmann/GreenTao/GreenTaok1k2/GreenTao_2-4-4_512.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3370915-1305427003/watcher-3370915-1305427003 -o /tmp/evaluation-result-3370915-1305427003/solver-3370915-1305427003 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3370915-1305427003.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= c1f047ec179094963b21d7b90509d2b9
RANDOM SEED=2135669071

node149.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.73
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      24051936 kB
Buffers:        315804 kB
Cached:        3890176 kB
SwapCached:      19984 kB
Active:        6375480 kB
Inactive:      2223600 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      24051936 kB
SwapTotal:    67111528 kB
SwapFree:     45937192 kB
Dirty:            3576 kB
Writeback:           0 kB
AnonPages:     4385708 kB
Mapped:          15016 kB
Slab:           157980 kB
PageTables:      72528 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 80309184 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73620 MiB
End job on node149 at 2011-05-15 04:56:49