Trace number 3371158

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.12 1203.97

General information on the benchmark

NameCRAFTED/rbsat/random/
forced/rbsat-v1820c171155gyes3.cnf
MD5SUMcd19d9eb9be97e183fa1a1d47d679d01
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables1820
Number of clauses171155
Sum of the clauses size344000
Maximum clause length28
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2171090
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 565

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3371158-1305419374/watcher-3371158-1305419374 -o /tmp/evaluation-result-3371158-1305419374/solver-3371158-1305419374 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3371158-1305419374.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.72 5.06 6.65 2/380 13222
/proc/meminfo: memFree=14869192/32951124 swapFree=65586204/67111528
[pid=13222] ppid=13220 vsize=9536 CPUtime=0 cores=1,3,5,7
/proc/13222/stat : 13222 (plingeling) R 13220 13222 12655 0 -1 4202496 473 1250 0 0 0 0 0 0 23 0 1 0 732168040 9764864 404 33554432000 4194304 4417452 140737227992848 18446744073709551615 216735178496 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/13222/statm: 2384 404 147 55 0 270 0

[startup+0.0665701 s]
/proc/loadavg: 3.72 5.06 6.65 2/380 13222
/proc/meminfo: memFree=14869192/32951124 swapFree=65586204/67111528
[pid=13222] ppid=13220 vsize=17640 CPUtime=0.05 cores=1,3,5,7
/proc/13222/stat : 13222 (plingeling) R 13220 13222 12655 0 -1 4202496 2479 1250 0 0 5 0 0 0 24 0 1 0 732168040 18063360 2410 33554432000 4194304 4417452 140737227992848 18446744073709551615 216735178496 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/13222/statm: 4410 2410 148 55 0 2296 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 17640

[startup+0.100565 s]
/proc/loadavg: 3.72 5.06 6.65 2/380 13222
/proc/meminfo: memFree=14869192/32951124 swapFree=65586204/67111528
[pid=13222] ppid=13220 vsize=25832 CPUtime=0.09 cores=1,3,5,7
/proc/13222/stat : 13222 (plingeling) R 13220 13222 12655 0 -1 4202496 4527 1250 0 0 9 0 0 0 24 0 1 0 732168040 26451968 4458 33554432000 4194304 4417452 140737227992848 18446744073709551615 216735178496 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/13222/statm: 6458 4458 148 55 0 4344 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 25832

[startup+0.300481 s]
/proc/loadavg: 3.72 5.06 6.65 2/380 13222
/proc/meminfo: memFree=14869192/32951124 swapFree=65586204/67111528
[pid=13222] ppid=13220 vsize=136424 CPUtime=0.85 cores=1,3,5,7
/proc/13222/stat : 13222 (plingeling) S 13220 13222 12655 0 -1 4202496 9275 1250 0 0 84 1 0 0 24 0 5 0 732168040 139698176 5565 33554432000 4194304 4417452 140737227992848 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/13222/statm: 34106 5565 176 55 0 31992 0
Current children cumulated CPU time (s) 0.85
Current children cumulated vsize (KiB) 136424

[startup+0.700409 s]
/proc/loadavg: 3.72 5.06 6.65 2/380 13222
/proc/meminfo: memFree=14869192/32951124 swapFree=65586204/67111528
[pid=13222] ppid=13220 vsize=152316 CPUtime=2.45 cores=1,3,5,7
/proc/13222/stat : 13222 (plingeling) S 13220 13222 12655 0 -1 4202496 14178 1250 0 0 243 2 0 0 24 0 5 0 732168040 155971584 9509 33554432000 4194304 4417452 140737227992848 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/13222/statm: 38079 9509 179 55 0 35965 0
Current children cumulated CPU time (s) 2.45
Current children cumulated vsize (KiB) 152316

[startup+1.50027 s]
/proc/loadavg: 3.82 5.06 6.64 9/392 13247
/proc/meminfo: memFree=14776716/32951124 swapFree=65586204/67111528
[pid=13222] ppid=13220 vsize=152720 CPUtime=5.64 cores=1,3,5,7
/proc/13222/stat : 13222 (plingeling) S 13220 13222 12655 0 -1 4202496 19428 1250 0 0 562 2 0 0 24 0 5 0 732168040 156385280 9610 33554432000 4194304 4417452 140737227992848 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/13222/statm: 38180 9610 179 55 0 36066 0
[pid=13222/tid=13240] ppid=13220 vsize=152720 CPUtime=1.38 cores=1,3,5,7
/proc/13222/task/13240/stat : 13240 (plingeling) R 13220 13222 12655 0 -1 4202560 2182 1250 0 0 138 0 0 0 25 0 5 0 732168051 156385280 9610 33554432000 4194304 4417452 140737227992848 18446744073709551615 4249130 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=13222/tid=13241] ppid=13220 vsize=152720 CPUtime=1.38 cores=1,3,5,7
/proc/13222/task/13241/stat : 13241 (plingeling) R 13220 13222 12655 0 -1 4202560 1438 1250 0 0 138 0 0 0 25 0 5 0 732168051 156385280 9610 33554432000 4194304 4417452 140737227992848 18446744073709551615 4236255 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=13222/tid=13242] ppid=13220 vsize=152720 CPUtime=1.37 cores=1,3,5,7
/proc/13222/task/13242/stat : 13242 (plingeling) R 13220 13222 12655 0 -1 4202560 1995 1250 0 0 137 0 0 0 25 0 5 0 732168051 156385280 9610 33554432000 4194304 4417452 140737227992848 18446744073709551615 4377419 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=13222/tid=13243] ppid=13220 vsize=152720 CPUtime=1.37 cores=1,3,5,7
/proc/13222/task/13243/stat : 13243 (plingeling) R 13220 13222 12655 0 -1 4202560 9274 1250 0 0 137 0 0 0 25 0 5 0 732168051 156385280 9610 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 5.64
Current children cumulated vsize (KiB) 152720

[startup+3.10099 s]
/proc/loadavg: 3.82 5.06 6.64 9/392 13247
/proc/meminfo: memFree=14748312/32951124 swapFree=65586204/67111528
[pid=13222] ppid=13220 vsize=171680 CPUtime=12.02 cores=1,3,5,7
/proc/13222/stat : 13222 (plingeling) S 13220 13222 12655 0 -1 4202496 28516 1250 0 0 1199 3 0 0 24 0 5 0 732168040 175800320 14350 33554432000 4194304 4417452 140737227992848 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/13222/statm: 42920 14350 179 55 0 40806 0
[pid=13222/tid=13240] ppid=13220 vsize=171680 CPUtime=2.98 cores=1,3,5,7
/proc/13222/task/13240/stat : 13240 (plingeling) R 13220 13222 12655 0 -1 4202560 3125 1250 0 0 298 0 0 0 25 0 5 0 732168051 175800320 14350 33554432000 4194304 4417452 140737227992848 18446744073709551615 4249584 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=13222/tid=13241] ppid=13220 vsize=171680 CPUtime=2.98 cores=1,3,5,7
/proc/13222/task/13241/stat : 13241 (plingeling) R 13220 13222 12655 0 -1 4202560 2381 1250 0 0 298 0 0 0 25 0 5 0 732168051 175800320 14350 33554432000 4194304 4417452 140737227992848 18446744073709551615 4249556 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=13222/tid=13242] ppid=13220 vsize=171680 CPUtime=2.95 cores=1,3,5,7
/proc/13222/task/13242/stat : 13242 (plingeling) R 13220 13222 12655 0 -1 4202560 1995 1250 0 0 295 0 0 0 25 0 5 0 732168051 175800320 14350 33554432000 4194304 4417452 140737227992848 18446744073709551615 216734872537 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=13222/tid=13243] ppid=13220 vsize=171680 CPUtime=2.97 cores=1,3,5,7
/proc/13222/task/13243/stat : 13243 (plingeling) R 13220 13222 12655 0 -1 4202560 16476 1250 0 0 296 1 0 0 25 0 5 0 732168051 175800320 14350 33554432000 4194304 4417452 140737227992848 18446744073709551615 4366117 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 12.02
Current children cumulated vsize (KiB) 171680

[startup+6.30044 s]
/proc/loadavg: 4.15 5.11 6.65 9/392 13247
/proc/meminfo: memFree=14725496/32951124 swapFree=65586204/67111528
[pid=13222] ppid=13220 vsize=182208 CPUtime=24.79 cores=1,3,5,7
/proc/13222/stat : 13222 (plingeling) S 13220 13222 12655 0 -1 4202496 40904 1250 0 0 2474 5 0 0 24 0 5 0 732168040 186580992 16982 33554432000 4194304 4417452 140737227992848 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/13222/statm: 45552 16982 179 55 0 43438 0
[pid=13222/tid=13240] ppid=13220 vsize=182208 CPUtime=6.18 cores=1,3,5,7
/proc/13222/task/13240/stat : 13240 (plingeling) R 13220 13222 12655 0 -1 4202560 3125 1250 0 0 618 0 0 0 25 0 5 0 732168051 186580992 16982 33554432000 4194304 4417452 140737227992848 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=13222/tid=13241] ppid=13220 vsize=182208 CPUtime=6.18 cores=1,3,5,7
/proc/13222/task/13241/stat : 13241 (plingeling) R 13220 13222 12655 0 -1 4202560 2381 1250 0 0 618 0 0 0 25 0 5 0 732168051 186580992 16982 33554432000 4194304 4417452 140737227992848 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=13222/tid=13242] ppid=13220 vsize=182208 CPUtime=6.12 cores=1,3,5,7
/proc/13222/task/13242/stat : 13242 (plingeling) R 13220 13222 12655 0 -1 4202560 2069 1250 0 0 612 0 0 0 25 0 5 0 732168051 186580992 16982 33554432000 4194304 4417452 140737227992848 18446744073709551615 4357219 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=13222/tid=13243] ppid=13220 vsize=182208 CPUtime=6.18 cores=1,3,5,7
/proc/13222/task/13243/stat : 13243 (plingeling) R 13220 13222 12655 0 -1 4202560 28790 1250 0 0 615 3 0 0 25 0 5 0 732168051 186580992 16982 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 24.79
Current children cumulated vsize (KiB) 182208

[startup+12.7003 s]
/proc/loadavg: 4.46 5.16 6.66 9/392 13247
/proc/meminfo: memFree=14651892/32951124 swapFree=65586204/67111528
[pid=13222] ppid=13220 vsize=215696 CPUtime=50.3 cores=1,3,5,7
/proc/13222/stat : 13222 (plingeling) S 13220 13222 12655 0 -1 4202496 69778 1250 0 0 5021 9 0 0 24 0 5 0 732168040 220872704 25354 33554432000 4194304 4417452 140737227992848 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/13222/statm: 53924 25354 179 55 0 51810 0
[pid=13222/tid=13240] ppid=13220 vsize=215696 CPUtime=12.57 cores=1,3,5,7
/proc/13222/task/13240/stat : 13240 (plingeling) R 13220 13222 12655 0 -1 4202560 4068 1250 0 0 1257 0 0 0 25 0 5 0 732168051 220872704 25354 33554432000 4194304 4417452 140737227992848 18446744073709551615 4250074 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=13222/tid=13241] ppid=13220 vsize=215696 CPUtime=12.58 cores=1,3,5,7
/proc/13222/task/13241/stat : 13241 (plingeling) R 13220 13222 12655 0 -1 4202560 3324 1250 0 0 1258 0 0 0 25 0 5 0 732168051 220872704 25354 33554432000 4194304 4417452 140737227992848 18446744073709551615 4250291 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=13222/tid=13242] ppid=13220 vsize=215696 CPUtime=12.44 cores=1,3,5,7
/proc/13222/task/13242/stat : 13242 (plingeling) R 13220 13222 12655 0 -1 4202560 2637 1250 0 0 1244 0 0 0 25 0 5 0 732168051 220872704 25354 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364805 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=13222/tid=13243] ppid=13220 vsize=215696 CPUtime=12.58 cores=1,3,5,7
/proc/13222/task/13243/stat : 13243 (plingeling) R 13220 13222 12655 0 -1 4202560 55210 1250 0 0 1251 7 0 0 25 0 5 0 732168051 220872704 25354 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364693 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 50.3
Current children cumulated vsize (KiB) 215696

[startup+25.5011 s]
/proc/loadavg: 5.00 5.25 6.67 9/392 13248
/proc/meminfo: memFree=14520216/32951124 swapFree=65586204/67111528
[pid=13222] ppid=13220 vsize=274744 CPUtime=101.35 cores=1,3,5,7
/proc/13222/stat : 13222 (plingeling) S 13220 13222 12655 0 -1 4202496 103415 1250 0 0 10121 14 0 0 24 0 5 0 732168040 281337856 40116 33554432000 4194304 4417452 140737227992848 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/13222/statm: 68686 40116 179 55 0 66572 0
[pid=13222/tid=13240] ppid=13220 vsize=274744 CPUtime=25.37 cores=1,3,5,7
/proc/13222/task/13240/stat : 13240 (plingeling) R 13220 13222 12655 0 -1 4202560 4925 1250 0 0 2536 1 0 0 25 0 5 0 732168051 281337856 40116 33554432000 4194304 4417452 140737227992848 18446744073709551615 4357566 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=13222/tid=13241] ppid=13220 vsize=274744 CPUtime=25.38 cores=1,3,5,7
/proc/13222/task/13241/stat : 13241 (plingeling) R 13220 13222 12655 0 -1 4202560 3682 1250 0 0 2538 0 0 0 25 0 5 0 732168051 281337856 40116 33554432000 4194304 4417452 140737227992848 18446744073709551615 4366407 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=13222/tid=13242] ppid=13220 vsize=274744 CPUtime=25.07 cores=1,3,5,7
/proc/13222/task/13242/stat : 13242 (plingeling) R 13220 13222 12655 0 -1 4202560 6112 1250 0 0 2507 0 0 0 25 0 5 0 732168051 281337856 40116 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=13222/tid=13243] ppid=13220 vsize=274744 CPUtime=25.37 cores=1,3,5,7
/proc/13222/task/13243/stat : 13243 (plingeling) R 13220 13222 12655 0 -1 4202560 84157 1250 0 0 2527 10 0 0 25 0 5 0 732168051 281337856 40116 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 101.35
Current children cumulated vsize (KiB) 274744

[startup+51.1006 s]
/proc/loadavg: 6.24 5.53 6.72 9/392 13248
/proc/meminfo: memFree=14436208/32951124 swapFree=65586204/67111528
[pid=13222] ppid=13220 vsize=239248 CPUtime=203.43 cores=1,3,5,7
/proc/13222/stat : 13222 (plingeling) S 13220 13222 12655 0 -1 4202496 144875 1250 0 0 20323 20 0 0 24 0 5 0 732168040 244989952 31242 33554432000 4194304 4417452 140737227992848 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/13222/statm: 59812 31242 179 55 0 57698 0
[pid=13222/tid=13240] ppid=13220 vsize=239248 CPUtime=50.97 cores=1,3,5,7
/proc/13222/task/13240/stat : 13240 (plingeling) R 13220 13222 12655 0 -1 4202560 5688 1250 0 0 5096 1 0 0 25 0 5 0 732168051 244989952 31242 33554432000 4194304 4417452 140737227992848 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=13222/tid=13241] ppid=13220 vsize=239248 CPUtime=50.97 cores=1,3,5,7
/proc/13222/task/13241/stat : 13241 (plingeling) R 13220 13222 12655 0 -1 4202560 5470 1250 0 0 5097 0 0 0 25 0 5 0 732168051 244989952 31242 33554432000 4194304 4417452 140737227992848 18446744073709551615 4250381 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=13222/tid=13242] ppid=13220 vsize=239248 CPUtime=50.37 cores=1,3,5,7
/proc/13222/task/13242/stat : 13242 (plingeling) R 13220 13222 12655 0 -1 4202560 7559 1250 0 0 5036 1 0 0 25 0 5 0 732168051 244989952 31242 33554432000 4194304 4417452 140737227992848 18446744073709551615 4249130 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=13222/tid=13243] ppid=13220 vsize=239248 CPUtime=50.98 cores=1,3,5,7
/proc/13222/task/13243/stat : 13243 (plingeling) R 13220 13222 12655 0 -1 4202560 121619 1250 0 0 5082 16 0 0 25 0 5 0 732168051 244989952 31242 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364838 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 203.43
Current children cumulated vsize (KiB) 239248

[startup+102.315 s]
/proc/loadavg: 7.50 5.97 6.80 9/392 13250
/proc/meminfo: memFree=14224412/32951124 swapFree=65586204/67111528
[pid=13222] ppid=13220 vsize=477476 CPUtime=407.63 cores=1,3,5,7
/proc/13222/stat : 13222 (plingeling) S 13220 13222 12655 0 -1 4202496 212993 1250 0 0 40733 30 0 0 24 0 5 0 732168040 488935424 90800 33554432000 4194304 4417452 140737227992848 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/13222/statm: 119369 90800 180 55 0 117255 0
[pid=13222/tid=13240] ppid=13220 vsize=477476 CPUtime=102.17 cores=1,3,5,7
/proc/13222/task/13240/stat : 13240 (plingeling) R 13220 13222 12655 0 -1 4202560 10738 1250 0 0 10216 1 0 0 25 0 5 0 732168051 488935424 90800 33554432000 4194304 4417452 140737227992848 18446744073709551615 4236816 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=13222/tid=13241] ppid=13220 vsize=477476 CPUtime=102.2 cores=1,3,5,7
/proc/13222/task/13241/stat : 13241 (plingeling) R 13220 13222 12655 0 -1 4202560 7657 1250 0 0 10219 1 0 0 25 0 5 0 732168051 488935424 90800 33554432000 4194304 4417452 140737227992848 18446744073709551615 4237189 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=13222/tid=13242] ppid=13220 vsize=477476 CPUtime=100.94 cores=1,3,5,7
/proc/13222/task/13242/stat : 13242 (plingeling) R 13220 13222 12655 0 -1 4202560 18323 1250 0 0 10092 2 0 0 25 0 5 0 732168051 488935424 90800 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=13222/tid=13243] ppid=13220 vsize=477476 CPUtime=102.19 cores=1,3,5,7
/proc/13222/task/13243/stat : 13243 (plingeling) R 13220 13222 12655 0 -1 4202560 171736 1250 0 0 10195 24 0 0 25 0 5 0 732168051 488935424 90800 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 407.63
Current children cumulated vsize (KiB) 477476

[startup+162.3 s]
/proc/loadavg: 7.89 6.35 6.88 9/392 13251
/proc/meminfo: memFree=13842472/32951124 swapFree=65586204/67111528
[pid=13222] ppid=13220 vsize=521984 CPUtime=646.82 cores=1,3,5,7
/proc/13222/stat : 13222 (plingeling) S 13220 13222 12655 0 -1 4202496 270023 1250 0 0 64644 38 0 0 24 0 5 0 732168040 534511616 101966 33554432000 4194304 4417452 140737227992848 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/13222/statm: 130496 101966 180 55 0 128382 0
[pid=13222/tid=13240] ppid=13220 vsize=521984 CPUtime=162.14 cores=1,3,5,7
/proc/13222/task/13240/stat : 13240 (plingeling) R 13220 13222 12655 0 -1 4202560 14406 1250 0 0 16213 1 0 0 25 0 5 0 732168051 534511616 101966 33554432000 4194304 4417452 140737227992848 18446744073709551615 4249143 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=13222/tid=13241] ppid=13220 vsize=521984 CPUtime=162.18 cores=1,3,5,7
/proc/13222/task/13241/stat : 13241 (plingeling) R 13220 13222 12655 0 -1 4202560 8371 1250 0 0 16217 1 0 0 25 0 5 0 732168051 534511616 101966 33554432000 4194304 4417452 140737227992848 18446744073709551615 4366458 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=13222/tid=13242] ppid=13220 vsize=521984 CPUtime=160.17 cores=1,3,5,7
/proc/13222/task/13242/stat : 13242 (plingeling) R 13220 13222 12655 0 -1 4202560 20404 1250 0 0 16015 2 0 0 25 0 5 0 732168051 534511616 101966 33554432000 4194304 4417452 140737227992848 18446744073709551615 4365004 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=13222/tid=13243] ppid=13220 vsize=521984 CPUtime=162.18 cores=1,3,5,7
/proc/13222/task/13243/stat : 13243 (plingeling) R 13220 13222 12655 0 -1 4202560 222303 1250 0 0 16187 31 0 0 25 0 5 0 732168051 534511616 101966 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 646.82
Current children cumulated vsize (KiB) 521984


################
# More data... #
################

[pid=13222/tid=13241] ppid=13220 vsize=933568 CPUtime=762.21 cores=1,3,5,7
/proc/13222/task/13241/stat : 13241 (plingeling) R 13220 13222 12655 0 -1 4202560 24104 1250 0 0 76218 3 0 0 25 0 5 0 732168051 955973632 204854 33554432000 4194304 4417452 140737227992848 18446744073709551615 4307807 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=13222/tid=13242] ppid=13220 vsize=933568 CPUtime=752.65 cores=1,3,5,7
/proc/13222/task/13242/stat : 13242 (plingeling) R 13220 13222 12655 0 -1 4202560 33180 1250 0 0 75261 4 0 0 25 0 5 0 732168051 955973632 204854 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364607 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=13222/tid=13243] ppid=13220 vsize=933568 CPUtime=762.21 cores=1,3,5,7
/proc/13222/task/13243/stat : 13243 (plingeling) R 13220 13222 12655 0 -1 4202560 489349 1250 0 0 76147 74 0 0 25 0 5 0 732168051 955973632 204854 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3039.25
Current children cumulated vsize (KiB) 933568

[startup+822.301 s]
/proc/loadavg: 8.14 7.94 7.50 9/392 13272
/proc/meminfo: memFree=12984500/32951124 swapFree=65586204/67111528
[pid=13222] ppid=13220 vsize=950472 CPUtime=3278.43 cores=1,3,5,7
/proc/13222/stat : 13222 (plingeling) S 13220 13222 12655 0 -1 4202496 594768 1250 0 0 327754 89 0 0 24 0 5 0 732168040 973283328 209080 33554432000 4194304 4417452 140737227992848 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/13222/statm: 237618 209080 180 55 0 235504 0
[pid=13222/tid=13240] ppid=13220 vsize=950472 CPUtime=822.01 cores=1,3,5,7
/proc/13222/task/13240/stat : 13240 (plingeling) R 13220 13222 12655 0 -1 4202560 27082 1250 0 0 82198 3 0 0 25 0 5 0 732168051 973283328 209080 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=13222/tid=13241] ppid=13220 vsize=950472 CPUtime=822.19 cores=1,3,5,7
/proc/13222/task/13241/stat : 13241 (plingeling) R 13220 13222 12655 0 -1 4202560 26152 1250 0 0 82216 3 0 0 25 0 5 0 732168051 973283328 209080 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=13222/tid=13242] ppid=13220 vsize=950472 CPUtime=811.89 cores=1,3,5,7
/proc/13222/task/13242/stat : 13242 (plingeling) R 13220 13222 12655 0 -1 4202560 35228 1250 0 0 81185 4 0 0 25 0 5 0 732168051 973283328 209080 33554432000 4194304 4417452 140737227992848 18446744073709551615 4352064 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=13222/tid=13243] ppid=13220 vsize=950472 CPUtime=822.2 cores=1,3,5,7
/proc/13222/task/13243/stat : 13243 (plingeling) R 13220 13222 12655 0 -1 4202560 501767 1250 0 0 82144 76 0 0 25 0 5 0 732168051 973283328 209080 33554432000 4194304 4417452 140737227992848 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3278.43
Current children cumulated vsize (KiB) 950472

[startup+882.301 s]
/proc/loadavg: 8.16 7.98 7.54 9/392 13274
/proc/meminfo: memFree=12682548/32951124 swapFree=65586204/67111528
[pid=13222] ppid=13220 vsize=962008 CPUtime=3517.68 cores=1,3,5,7
/proc/13222/stat : 13222 (plingeling) S 13220 13222 12655 0 -1 4202496 599700 1250 0 0 351679 89 0 0 24 0 5 0 732168040 985096192 211964 33554432000 4194304 4417452 140737227992848 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/13222/statm: 240502 211964 180 55 0 238388 0
[pid=13222/tid=13240] ppid=13220 vsize=962008 CPUtime=882 cores=1,3,5,7
/proc/13222/task/13240/stat : 13240 (plingeling) R 13220 13222 12655 0 -1 4202560 29966 1250 0 0 88197 3 0 0 25 0 5 0 732168051 985096192 211964 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=13222/tid=13241] ppid=13220 vsize=962008 CPUtime=882.19 cores=1,3,5,7
/proc/13222/task/13241/stat : 13241 (plingeling) R 13220 13222 12655 0 -1 4202560 28200 1250 0 0 88216 3 0 0 25 0 5 0 732168051 985096192 211964 33554432000 4194304 4417452 140737227992848 18446744073709551615 4358116 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=13222/tid=13242] ppid=13220 vsize=962008 CPUtime=871.14 cores=1,3,5,7
/proc/13222/task/13242/stat : 13242 (plingeling) R 13220 13222 12655 0 -1 4202560 35228 1250 0 0 87110 4 0 0 25 0 5 0 732168051 985096192 211964 33554432000 4194304 4417452 140737227992848 18446744073709551615 4366234 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=13222/tid=13243] ppid=13220 vsize=962008 CPUtime=882.2 cores=1,3,5,7
/proc/13222/task/13243/stat : 13243 (plingeling) R 13220 13222 12655 0 -1 4202560 501767 1250 0 0 88144 76 0 0 25 0 5 0 732168051 985096192 211964 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364594 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3517.68
Current children cumulated vsize (KiB) 962008

[startup+942.301 s]
/proc/loadavg: 8.13 7.99 7.57 9/392 13275
/proc/meminfo: memFree=12557900/32951124 swapFree=65586204/67111528
[pid=13222] ppid=13220 vsize=1087640 CPUtime=3756.94 cores=1,3,5,7
/proc/13222/stat : 13222 (plingeling) S 13220 13222 12655 0 -1 4202496 633156 1250 0 0 375601 93 0 0 24 0 5 0 732168040 1113743360 243372 33554432000 4194304 4417452 140737227992848 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/13222/statm: 271910 243372 180 55 0 269796 0
[pid=13222/tid=13240] ppid=13220 vsize=1087640 CPUtime=942 cores=1,3,5,7
/proc/13222/task/13240/stat : 13240 (plingeling) R 13220 13222 12655 0 -1 4202560 32014 1250 0 0 94196 4 0 0 25 0 5 0 732168051 1113743360 243372 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=13222/tid=13241] ppid=13220 vsize=1087640 CPUtime=942.19 cores=1,3,5,7
/proc/13222/task/13241/stat : 13241 (plingeling) R 13220 13222 12655 0 -1 4202560 30248 1250 0 0 94216 3 0 0 25 0 5 0 732168051 1113743360 243372 33554432000 4194304 4417452 140737227992848 18446744073709551615 4354461 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=13222/tid=13242] ppid=13220 vsize=1087640 CPUtime=930.41 cores=1,3,5,7
/proc/13222/task/13242/stat : 13242 (plingeling) R 13220 13222 12655 0 -1 4202560 37276 1250 0 0 93036 5 0 0 25 0 5 0 732168051 1113743360 243372 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=13222/tid=13243] ppid=13220 vsize=1087640 CPUtime=942.2 cores=1,3,5,7
/proc/13222/task/13243/stat : 13243 (plingeling) R 13220 13222 12655 0 -1 4202560 529079 1250 0 0 94141 79 0 0 25 0 5 0 732168051 1113743360 243372 33554432000 4194304 4417452 140737227992848 18446744073709551615 4365058 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3756.94
Current children cumulated vsize (KiB) 1087640

[startup+1002.3 s]
/proc/loadavg: 8.19 8.03 7.60 9/392 13277
/proc/meminfo: memFree=12121892/32951124 swapFree=65586204/67111528
[pid=13222] ppid=13220 vsize=1271284 CPUtime=3996.15 cores=1,3,5,7
/proc/13222/stat : 13222 (plingeling) S 13220 13222 12655 0 -1 4202496 679067 1250 0 0 399514 101 0 0 24 0 5 0 732168040 1301794816 289283 33554432000 4194304 4417452 140737227992848 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/13222/statm: 317821 289283 180 55 0 315707 0
[pid=13222/tid=13240] ppid=13220 vsize=1271284 CPUtime=1001.98 cores=1,3,5,7
/proc/13222/task/13240/stat : 13240 (plingeling) R 13220 13222 12655 0 -1 4202560 34917 1250 0 0 100194 4 0 0 25 0 5 0 732168051 1301794816 289283 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=13222/tid=13241] ppid=13220 vsize=1271284 CPUtime=1002.19 cores=1,3,5,7
/proc/13222/task/13241/stat : 13241 (plingeling) R 13220 13222 12655 0 -1 4202560 30248 1250 0 0 100216 3 0 0 25 0 5 0 732168051 1301794816 289283 33554432000 4194304 4417452 140737227992848 18446744073709551615 4336723 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=13222/tid=13242] ppid=13220 vsize=1271284 CPUtime=989.64 cores=1,3,5,7
/proc/13222/task/13242/stat : 13242 (plingeling) R 13220 13222 12655 0 -1 4202560 39324 1250 0 0 98959 5 0 0 25 0 5 0 732168051 1301794816 289283 33554432000 4194304 4417452 140737227992848 18446744073709551615 4356783 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=13222/tid=13243] ppid=13220 vsize=1271284 CPUtime=1002.2 cores=1,3,5,7
/proc/13222/task/13243/stat : 13243 (plingeling) R 13220 13222 12655 0 -1 4202560 570039 1250 0 0 100133 87 0 0 25 0 5 0 732168051 1301794816 289283 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3996.15
Current children cumulated vsize (KiB) 1271284

[startup+1062.3 s]
/proc/loadavg: 8.12 8.03 7.63 9/392 13278
/proc/meminfo: memFree=12088152/32951124 swapFree=65586204/67111528
[pid=13222] ppid=13220 vsize=1276596 CPUtime=4235.4 cores=1,3,5,7
/proc/13222/stat : 13222 (plingeling) S 13220 13222 12655 0 -1 4202496 680395 1250 0 0 423439 101 0 0 24 0 5 0 732168040 1307234304 290611 33554432000 4194304 4417452 140737227992848 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/13222/statm: 319149 290611 180 55 0 317035 0
[pid=13222/tid=13240] ppid=13220 vsize=1276596 CPUtime=1061.97 cores=1,3,5,7
/proc/13222/task/13240/stat : 13240 (plingeling) R 13220 13222 12655 0 -1 4202560 34917 1250 0 0 106193 4 0 0 25 0 5 0 732168051 1307234304 290611 33554432000 4194304 4417452 140737227992848 18446744073709551615 4236670 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=13222/tid=13241] ppid=13220 vsize=1276596 CPUtime=1062.2 cores=1,3,5,7
/proc/13222/task/13241/stat : 13241 (plingeling) R 13220 13222 12655 0 -1 4202560 31576 1250 0 0 106216 4 0 0 25 0 5 0 732168051 1307234304 290611 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=13222/tid=13242] ppid=13220 vsize=1276596 CPUtime=1048.9 cores=1,3,5,7
/proc/13222/task/13242/stat : 13242 (plingeling) R 13220 13222 12655 0 -1 4202560 39324 1250 0 0 104885 5 0 0 25 0 5 0 732168051 1307234304 290611 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=13222/tid=13243] ppid=13220 vsize=1276596 CPUtime=1062.2 cores=1,3,5,7
/proc/13222/task/13243/stat : 13243 (plingeling) R 13220 13222 12655 0 -1 4202560 570039 1250 0 0 106133 87 0 0 25 0 5 0 732168051 1307234304 290611 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4235.4
Current children cumulated vsize (KiB) 1276596

[startup+1122.3 s]
/proc/loadavg: 8.09 8.04 7.66 9/392 13280
/proc/meminfo: memFree=12036424/32951124 swapFree=65586204/67111528
[pid=13222] ppid=13220 vsize=1328108 CPUtime=4474.64 cores=1,3,5,7
/proc/13222/stat : 13222 (plingeling) S 13220 13222 12655 0 -1 4202496 714340 1250 0 0 447359 105 0 0 24 0 5 0 732168040 1359982592 303489 33554432000 4194304 4417452 140737227992848 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/13222/statm: 332027 303489 180 55 0 329913 0
[pid=13222/tid=13240] ppid=13220 vsize=1328108 CPUtime=1121.96 cores=1,3,5,7
/proc/13222/task/13240/stat : 13240 (plingeling) R 13220 13222 12655 0 -1 4202560 38981 1250 0 0 112192 4 0 0 25 0 5 0 732168051 1359982592 303489 33554432000 4194304 4417452 140737227992848 18446744073709551615 4356884 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=13222/tid=13241] ppid=13220 vsize=1328108 CPUtime=1122.2 cores=1,3,5,7
/proc/13222/task/13241/stat : 13241 (plingeling) R 13220 13222 12655 0 -1 4202560 43921 1250 0 0 112215 5 0 0 25 0 5 0 732168051 1359982592 303489 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=13222/tid=13242] ppid=13220 vsize=1328108 CPUtime=1108.13 cores=1,3,5,7
/proc/13222/task/13242/stat : 13242 (plingeling) R 13220 13222 12655 0 -1 4202560 39324 1250 0 0 110808 5 0 0 25 0 5 0 732168051 1359982592 303489 33554432000 4194304 4417452 140737227992848 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=13222/tid=13243] ppid=13220 vsize=1328108 CPUtime=1122.2 cores=1,3,5,7
/proc/13222/task/13243/stat : 13243 (plingeling) R 13220 13222 12655 0 -1 4202560 587575 1250 0 0 112132 88 0 0 25 0 5 0 732168051 1359982592 303489 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364854 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4474.64
Current children cumulated vsize (KiB) 1328108

[startup+1182.3 s]
/proc/loadavg: 8.08 8.04 7.68 9/392 13281
/proc/meminfo: memFree=11962620/32951124 swapFree=65586204/67111528
[pid=13222] ppid=13220 vsize=1328108 CPUtime=4713.87 cores=1,3,5,7
/proc/13222/stat : 13222 (plingeling) S 13220 13222 12655 0 -1 4202496 714340 1250 0 0 471282 105 0 0 24 0 5 0 732168040 1359982592 303489 33554432000 4194304 4417452 140737227992848 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/13222/statm: 332027 303489 180 55 0 329913 0
[pid=13222/tid=13240] ppid=13220 vsize=1328108 CPUtime=1181.95 cores=1,3,5,7
/proc/13222/task/13240/stat : 13240 (plingeling) R 13220 13222 12655 0 -1 4202560 38981 1250 0 0 118191 4 0 0 25 0 5 0 732168051 1359982592 303489 33554432000 4194304 4417452 140737227992848 18446744073709551615 4366377 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=13222/tid=13241] ppid=13220 vsize=1328108 CPUtime=1182.2 cores=1,3,5,7
/proc/13222/task/13241/stat : 13241 (plingeling) R 13220 13222 12655 0 -1 4202560 43921 1250 0 0 118215 5 0 0 25 0 5 0 732168051 1359982592 303489 33554432000 4194304 4417452 140737227992848 18446744073709551615 4366450 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=13222/tid=13242] ppid=13220 vsize=1328108 CPUtime=1167.38 cores=1,3,5,7
/proc/13222/task/13242/stat : 13242 (plingeling) R 13220 13222 12655 0 -1 4202560 39324 1250 0 0 116733 5 0 0 25 0 5 0 732168051 1359982592 303489 33554432000 4194304 4417452 140737227992848 18446744073709551615 4236601 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=13222/tid=13243] ppid=13220 vsize=1328108 CPUtime=1182.2 cores=1,3,5,7
/proc/13222/task/13243/stat : 13243 (plingeling) R 13220 13222 12655 0 -1 4202560 587575 1250 0 0 118132 88 0 0 25 0 5 0 732168051 1359982592 303489 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4713.87
Current children cumulated vsize (KiB) 1328108



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.9 s]
/proc/loadavg: 8.13 8.06 7.69 9/392 13282
/proc/meminfo: memFree=11962488/32951124 swapFree=65586204/67111528
[pid=13222] ppid=13220 vsize=1328108 CPUtime=4800.02 cores=1,3,5,7
/proc/13222/stat : 13222 (plingeling) S 13220 13222 12655 0 -1 4202496 714340 1250 0 0 479897 105 0 0 24 0 5 0 732168040 1359982592 303489 33554432000 4194304 4417452 140737227992848 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/13222/statm: 332027 303489 180 55 0 329913 0
[pid=13222/tid=13240] ppid=13220 vsize=1328108 CPUtime=1203.56 cores=1,3,5,7
/proc/13222/task/13240/stat : 13240 (plingeling) R 13220 13222 12655 0 -1 4202560 38981 1250 0 0 120352 4 0 0 25 0 5 0 732168051 1359982592 303489 33554432000 4194304 4417452 140737227992848 18446744073709551615 4250765 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=13222/tid=13241] ppid=13220 vsize=1328108 CPUtime=1203.81 cores=1,3,5,7
/proc/13222/task/13241/stat : 13241 (plingeling) R 13220 13222 12655 0 -1 4202560 43921 1250 0 0 120376 5 0 0 25 0 5 0 732168051 1359982592 303489 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=13222/tid=13242] ppid=13220 vsize=1328108 CPUtime=1188.7 cores=1,3,5,7
/proc/13222/task/13242/stat : 13242 (plingeling) R 13220 13222 12655 0 -1 4202560 39324 1250 0 0 118865 5 0 0 25 0 5 0 732168051 1359982592 303489 33554432000 4194304 4417452 140737227992848 18446744073709551615 4357207 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=13222/tid=13243] ppid=13220 vsize=1328108 CPUtime=1203.82 cores=1,3,5,7
/proc/13222/task/13243/stat : 13243 (plingeling) R 13220 13222 12655 0 -1 4202560 587575 1250 0 0 120294 88 0 0 25 0 5 0 732168051 1359982592 303489 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.02
Current children cumulated vsize (KiB) 1328108

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.9 s]
/proc/loadavg: 8.13 8.06 7.69 9/392 13282
/proc/meminfo: memFree=11962488/32951124 swapFree=65586204/67111528
[pid=13222] ppid=13220 vsize=1328108 CPUtime=4800.02 cores=1,3,5,7
/proc/13222/stat : 13222 (plingeling) S 13220 13222 12655 0 -1 4202496 714340 1250 0 0 479897 105 0 0 24 0 5 0 732168040 1359982592 303489 33554432000 4194304 4417452 140737227992848 18446744073709551615 216746981349 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/13222/statm: 332027 303489 180 55 0 329913 0
[pid=13222/tid=13240] ppid=13220 vsize=1328108 CPUtime=1203.56 cores=1,3,5,7
/proc/13222/task/13240/stat : 13240 (plingeling) R 13220 13222 12655 0 -1 4202560 38981 1250 0 0 120352 4 0 0 25 0 5 0 732168051 1359982592 303489 33554432000 4194304 4417452 140737227992848 18446744073709551615 4250765 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=13222/tid=13241] ppid=13220 vsize=1328108 CPUtime=1203.81 cores=1,3,5,7
/proc/13222/task/13241/stat : 13241 (plingeling) R 13220 13222 12655 0 -1 4202560 43921 1250 0 0 120376 5 0 0 25 0 5 0 732168051 1359982592 303489 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=13222/tid=13242] ppid=13220 vsize=1328108 CPUtime=1188.7 cores=1,3,5,7
/proc/13222/task/13242/stat : 13242 (plingeling) R 13220 13222 12655 0 -1 4202560 39324 1250 0 0 118865 5 0 0 25 0 5 0 732168051 1359982592 303489 33554432000 4194304 4417452 140737227992848 18446744073709551615 4357207 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=13222/tid=13243] ppid=13220 vsize=1328108 CPUtime=1203.82 cores=1,3,5,7
/proc/13222/task/13243/stat : 13243 (plingeling) R 13220 13222 12655 0 -1 4202560 587575 1250 0 0 120294 88 0 0 25 0 5 0 732168051 1359982592 303489 33554432000 4194304 4417452 140737227992848 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.02
Current children cumulated vsize (KiB) 1328108

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1203.97
CPU time (s): 4800.12
CPU user time (s): 4799.01
CPU system time (s): 1.10783
CPU usage (%): 398.69
Max. virtual memory (cumulated for all children) (KiB): 1328108

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4799.01
system time used= 1.10783
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 715590
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 129
involuntary context switches= 37602

runsolver used 4.46132 second user time and 10.6614 second system time

The end

Launcher Data 

Begin job on node119 at 2011-05-15 02:29:34
IDJOB=3371158
IDBENCH=71409
IDSOLVER=1848
FILE ID=node119/3371158-1305419374
RUNJOBID= node119-1305419374-13209
PBS_JOBID= 13324563
Free space on /tmp= 73192 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT09/CRAFTED/rbsat/random/forced/rbsat-v1820c171155gyes3.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3371158-1305419374/watcher-3371158-1305419374 -o /tmp/evaluation-result-3371158-1305419374/solver-3371158-1305419374 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3371158-1305419374.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= cd19d9eb9be97e183fa1a1d47d679d01
RANDOM SEED=1501331976

node119.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.63
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      14869472 kB
Buffers:        287712 kB
Cached:        4578684 kB
SwapCached:       4756 kB
Active:       14705424 kB
Inactive:      3044912 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      14869472 kB
SwapTotal:    67111528 kB
SwapFree:     65586204 kB
Dirty:            4772 kB
Writeback:          16 kB
AnonPages:    12882176 kB
Mapped:          29564 kB
Slab:           210604 kB
PageTables:      56392 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 102464616 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73188 MiB
End job on node119 at 2011-05-15 02:49:40