Trace number 3394732

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 40001 5010.35

General information on the benchmark

NameSAT03/handmade/anton/UNSAT/
SGI_30_60_29_30_6-dir.shuffled-as.sat03-130.cnf
MD5SUMe826117d87949a5b285e72732bfa395b
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark2182.13
Satisfiable
(Un)Satisfiability was proved
Number of variables870
Number of clauses69964
Sum of the clauses size140740
Maximum clause length30
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 269935
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 529

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3394732-1306104710/watcher-3394732-1306104710 -o /tmp/evaluation-result-3394732-1306104710/solver-3394732-1306104710 -C 40000 -W 5100 -M 31000 plingeling -t 8 HOME/instance-3394732-1306104710.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.49 7.99 8.05 2/451 4304
/proc/meminfo: memFree=29614072/32951124 swapFree=60688720/67111528
[pid=4304] ppid=4302 vsize=9584 CPUtime=0 cores=0-7
/proc/4304/stat : 4304 (plingeling) R 4302 4304 4091 0 -1 4202496 457 1254 0 0 0 0 0 0 24 0 1 0 722069203 9814016 387 33554432000 4194304 4417452 140733237395248 18446744073709551615 232335892224 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/4304/statm: 2396 387 147 55 0 282 0

[startup+0.0605349 s]
/proc/loadavg: 7.49 7.99 8.05 2/451 4304
/proc/meminfo: memFree=29614072/32951124 swapFree=60688720/67111528
[pid=4304] ppid=4302 vsize=17728 CPUtime=0.05 cores=0-7
/proc/4304/stat : 4304 (plingeling) R 4302 4304 4091 0 -1 4202496 2674 1254 0 0 5 0 0 0 24 0 1 0 722069203 18153472 2456 33554432000 4194304 4417452 140733237395248 18446744073709551615 232335511129 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/4304/statm: 4432 2456 148 55 0 2318 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 17728

[startup+0.100528 s]
/proc/loadavg: 7.49 7.99 8.05 2/451 4304
/proc/meminfo: memFree=29614072/32951124 swapFree=60688720/67111528
[pid=4304] ppid=4302 vsize=25920 CPUtime=0.08 cores=0-7
/proc/4304/stat : 4304 (plingeling) R 4302 4304 4091 0 -1 4202496 4722 1254 0 0 8 0 0 0 24 0 1 0 722069203 26542080 4504 33554432000 4194304 4417452 140733237395248 18446744073709551615 4229149 0 0 4096 17442 0 0 0 17 2 0 0 0
/proc/4304/statm: 6480 4504 148 55 0 4366 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 25920

[startup+0.300452 s]
/proc/loadavg: 7.49 7.99 8.05 2/451 4304
/proc/meminfo: memFree=29614072/32951124 swapFree=60688720/67111528
[pid=4304] ppid=4302 vsize=181952 CPUtime=1.66 cores=0-7
/proc/4304/stat : 4304 (plingeling) S 4302 4304 4091 0 -1 4202496 8860 1254 0 0 165 1 0 0 18 0 9 0 722069203 186318848 6577 33554432000 4194304 4417452 140733237395248 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4304/statm: 45488 6577 179 55 0 43374 0
Current children cumulated CPU time (s) 1.66
Current children cumulated vsize (KiB) 181952

[startup+0.70038 s]
/proc/loadavg: 7.49 7.99 8.05 2/451 4304
/proc/meminfo: memFree=29614072/32951124 swapFree=60688720/67111528
[pid=4304] ppid=4302 vsize=185028 CPUtime=4.86 cores=0-7
/proc/4304/stat : 4304 (plingeling) S 4302 4304 4091 0 -1 4202496 10704 1254 0 0 485 1 0 0 18 0 9 0 722069203 189468672 7484 33554432000 4194304 4417452 140733237395248 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4304/statm: 46257 7484 180 55 0 44143 0
Current children cumulated CPU time (s) 4.86
Current children cumulated vsize (KiB) 185028

[startup+1.50124 s]
/proc/loadavg: 7.49 7.99 8.05 9/460 4318
/proc/meminfo: memFree=29575924/32951124 swapFree=60688720/67111528
[pid=4304] ppid=4302 vsize=197976 CPUtime=11.25 cores=0-7
/proc/4304/stat : 4304 (plingeling) S 4302 4304 4091 0 -1 4202496 15784 1254 0 0 1123 2 0 0 18 0 9 0 722069203 202727424 10711 33554432000 4194304 4417452 140733237395248 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4304/statm: 49494 10711 180 55 0 47380 0
[pid=4304/tid=4311] ppid=4302 vsize=197976 CPUtime=1.39 cores=0-7
/proc/4304/task/4311/stat : 4311 (plingeling) R 4302 4304 4091 0 -1 4202560 547 1254 0 0 139 0 0 0 25 0 9 0 722069213 202727424 10711 33554432000 4194304 4417452 140733237395248 18446744073709551615 4250000 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4304/tid=4312] ppid=4302 vsize=197976 CPUtime=1.39 cores=0-7
/proc/4304/task/4312/stat : 4312 (plingeling) R 4302 4304 4091 0 -1 4202560 757 1254 0 0 139 0 0 0 25 0 9 0 722069213 202727424 10711 33554432000 4194304 4417452 140733237395248 18446744073709551615 4249935 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4304/tid=4313] ppid=4302 vsize=197976 CPUtime=1.39 cores=0-7
/proc/4304/task/4313/stat : 4313 (plingeling) R 4302 4304 4091 0 -1 4202560 1097 1254 0 0 139 0 0 0 25 0 9 0 722069213 202727424 10711 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364735 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4304/tid=4314] ppid=4302 vsize=197976 CPUtime=1.39 cores=0-7
/proc/4304/task/4314/stat : 4314 (plingeling) R 4302 4304 4091 0 -1 4202560 6255 1254 0 0 138 1 0 0 25 0 9 0 722069213 202727424 10711 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4304/tid=4315] ppid=4302 vsize=197976 CPUtime=1.37 cores=0-7
/proc/4304/task/4315/stat : 4315 (plingeling) R 4302 4304 4091 0 -1 4202560 425 1254 0 0 137 0 0 0 25 0 9 0 722069213 202727424 10711 33554432000 4194304 4417452 140733237395248 18446744073709551615 4248926 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4304/tid=4316] ppid=4302 vsize=197976 CPUtime=1.39 cores=0-7
/proc/4304/task/4316/stat : 4316 (plingeling) R 4302 4304 4091 0 -1 4202560 468 1254 0 0 139 0 0 0 22 0 9 0 722069214 202727424 10711 33554432000 4194304 4417452 140733237395248 18446744073709551615 4337303 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4304/tid=4317] ppid=4302 vsize=197976 CPUtime=1.39 cores=0-7
/proc/4304/task/4317/stat : 4317 (plingeling) R 4302 4304 4091 0 -1 4202560 1015 1254 0 0 139 0 0 0 22 0 9 0 722069214 202727424 10711 33554432000 4194304 4417452 140733237395248 18446744073709551615 4365092 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4304/tid=4318] ppid=4302 vsize=197976 CPUtime=1.39 cores=0-7
/proc/4304/task/4318/stat : 4318 (plingeling) R 4302 4304 4091 0 -1 4202560 482 1254 0 0 139 0 0 0 22 0 9 0 722069214 202727424 10711 33554432000 4194304 4417452 140733237395248 18446744073709551615 4249186 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 11.25
Current children cumulated vsize (KiB) 197976

[startup+3.10096 s]
/proc/loadavg: 7.49 7.99 8.05 9/460 4318
/proc/meminfo: memFree=29571360/32951124 swapFree=60688720/67111528
[pid=4304] ppid=4302 vsize=214216 CPUtime=24.04 cores=0-7
/proc/4304/stat : 4304 (plingeling) S 4302 4304 4091 0 -1 4202496 26546 1254 0 0 2399 5 0 0 18 0 9 0 722069203 219357184 14771 33554432000 4194304 4417452 140733237395248 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4304/statm: 53554 14771 180 55 0 51440 0
[pid=4304/tid=4311] ppid=4302 vsize=214216 CPUtime=2.99 cores=0-7
/proc/4304/task/4311/stat : 4311 (plingeling) R 4302 4304 4091 0 -1 4202560 997 1254 0 0 299 0 0 0 25 0 9 0 722069213 219357184 14771 33554432000 4194304 4417452 140733237395248 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4304/tid=4312] ppid=4302 vsize=214216 CPUtime=2.99 cores=0-7
/proc/4304/task/4312/stat : 4312 (plingeling) R 4302 4304 4091 0 -1 4202560 757 1254 0 0 299 0 0 0 25 0 9 0 722069213 219357184 14771 33554432000 4194304 4417452 140733237395248 18446744073709551615 4258436 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4304/tid=4313] ppid=4302 vsize=214216 CPUtime=2.99 cores=0-7
/proc/4304/task/4313/stat : 4313 (plingeling) R 4302 4304 4091 0 -1 4202560 2947 1254 0 0 299 0 0 0 25 0 9 0 722069213 219357184 14771 33554432000 4194304 4417452 140733237395248 18446744073709551615 4265443 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4304/tid=4314] ppid=4302 vsize=214216 CPUtime=2.99 cores=0-7
/proc/4304/task/4314/stat : 4314 (plingeling) R 4302 4304 4091 0 -1 4202560 11529 1254 0 0 297 2 0 0 25 0 9 0 722069213 219357184 14771 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4304/tid=4315] ppid=4302 vsize=214216 CPUtime=2.95 cores=0-7
/proc/4304/task/4315/stat : 4315 (plingeling) R 4302 4304 4091 0 -1 4202560 875 1254 0 0 295 0 0 0 25 0 9 0 722069213 219357184 14771 33554432000 4194304 4417452 140733237395248 18446744073709551615 4258632 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4304/tid=4316] ppid=4302 vsize=214216 CPUtime=2.99 cores=0-7
/proc/4304/task/4316/stat : 4316 (plingeling) R 4302 4304 4091 0 -1 4202560 918 1254 0 0 299 0 0 0 25 0 9 0 722069214 219357184 14771 33554432000 4194304 4417452 140733237395248 18446744073709551615 4250058 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4304/tid=4317] ppid=4302 vsize=214216 CPUtime=2.99 cores=0-7
/proc/4304/task/4317/stat : 4317 (plingeling) R 4302 4304 4091 0 -1 4202560 2853 1254 0 0 299 0 0 0 25 0 9 0 722069214 219357184 14771 33554432000 4194304 4417452 140733237395248 18446744073709551615 4357283 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4304/tid=4318] ppid=4302 vsize=214216 CPUtime=2.99 cores=0-7
/proc/4304/task/4318/stat : 4318 (plingeling) R 4302 4304 4091 0 -1 4202560 932 1254 0 0 299 0 0 0 25 0 9 0 722069214 219357184 14771 33554432000 4194304 4417452 140733237395248 18446744073709551615 4250017 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 24.04
Current children cumulated vsize (KiB) 214216

[startup+6.30039 s]
/proc/loadavg: 7.53 7.99 8.05 9/460 4318
/proc/meminfo: memFree=29547436/32951124 swapFree=60688720/67111528
[pid=4304] ppid=4302 vsize=240432 CPUtime=49.58 cores=0-7
/proc/4304/stat : 4304 (plingeling) S 4302 4304 4091 0 -1 4202496 41265 1254 0 0 4950 8 0 0 18 0 9 0 722069203 246202368 21325 33554432000 4194304 4417452 140733237395248 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4304/statm: 60108 21325 180 55 0 57994 0
[pid=4304/tid=4311] ppid=4302 vsize=240432 CPUtime=6.19 cores=0-7
/proc/4304/task/4311/stat : 4311 (plingeling) R 4302 4304 4091 0 -1 4202560 1673 1254 0 0 619 0 0 0 25 0 9 0 722069213 246202368 21325 33554432000 4194304 4417452 140733237395248 18446744073709551615 4236603 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4304/tid=4312] ppid=4302 vsize=240432 CPUtime=6.19 cores=0-7
/proc/4304/task/4312/stat : 4312 (plingeling) R 4302 4304 4091 0 -1 4202560 1504 1254 0 0 619 0 0 0 25 0 9 0 722069213 246202368 21325 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364821 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4304/tid=4313] ppid=4302 vsize=240432 CPUtime=6.19 cores=0-7
/proc/4304/task/4313/stat : 4313 (plingeling) R 4302 4304 4091 0 -1 4202560 5828 1254 0 0 619 0 0 0 25 0 9 0 722069213 246202368 21325 33554432000 4194304 4417452 140733237395248 18446744073709551615 4366048 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4304/tid=4314] ppid=4302 vsize=240432 CPUtime=6.18 cores=0-7
/proc/4304/task/4314/stat : 4314 (plingeling) R 4302 4304 4091 0 -1 4202560 17444 1254 0 0 615 3 0 0 25 0 9 0 722069213 246202368 21325 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4304/tid=4315] ppid=4302 vsize=240432 CPUtime=6.11 cores=0-7
/proc/4304/task/4315/stat : 4315 (plingeling) R 4302 4304 4091 0 -1 4202560 1266 1254 0 0 611 0 0 0 25 0 9 0 722069213 246202368 21325 33554432000 4194304 4417452 140733237395248 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4304/tid=4316] ppid=4302 vsize=240432 CPUtime=6.19 cores=0-7
/proc/4304/task/4316/stat : 4316 (plingeling) R 4302 4304 4091 0 -1 4202560 1606 1254 0 0 619 0 0 0 25 0 9 0 722069214 246202368 21325 33554432000 4194304 4417452 140733237395248 18446744073709551615 4365075 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4304/tid=4317] ppid=4302 vsize=240432 CPUtime=6.19 cores=0-7
/proc/4304/task/4317/stat : 4317 (plingeling) R 4302 4304 4091 0 -1 4202560 5982 1254 0 0 618 1 0 0 25 0 9 0 722069214 246202368 21325 33554432000 4194304 4417452 140733237395248 18446744073709551615 4357976 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4304/tid=4318] ppid=4302 vsize=240432 CPUtime=6.19 cores=0-7
/proc/4304/task/4318/stat : 4318 (plingeling) R 4302 4304 4091 0 -1 4202560 1224 1254 0 0 619 0 0 0 25 0 9 0 722069214 246202368 21325 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 49.58
Current children cumulated vsize (KiB) 240432

[startup+12.7013 s]
/proc/loadavg: 7.57 7.99 8.05 9/460 4318
/proc/meminfo: memFree=29494764/32951124 swapFree=60688720/67111528
[pid=4304] ppid=4302 vsize=278544 CPUtime=100.69 cores=0-7
/proc/4304/stat : 4304 (plingeling) S 4302 4304 4091 0 -1 4202496 71419 1254 0 0 10056 13 0 0 18 0 9 0 722069203 285229056 30853 33554432000 4194304 4417452 140733237395248 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4304/statm: 69636 30853 180 55 0 67522 0
[pid=4304/tid=4311] ppid=4302 vsize=278544 CPUtime=12.59 cores=0-7
/proc/4304/task/4311/stat : 4311 (plingeling) R 4302 4304 4091 0 -1 4202560 4685 1254 0 0 1259 0 0 0 25 0 9 0 722069213 285229056 30853 33554432000 4194304 4417452 140733237395248 18446744073709551615 4251045 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4304/tid=4312] ppid=4302 vsize=278544 CPUtime=12.59 cores=0-7
/proc/4304/task/4312/stat : 4312 (plingeling) R 4302 4304 4091 0 -1 4202560 3643 1254 0 0 1259 0 0 0 25 0 9 0 722069213 285229056 30853 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4304/tid=4313] ppid=4302 vsize=278544 CPUtime=12.59 cores=0-7
/proc/4304/task/4313/stat : 4313 (plingeling) R 4302 4304 4091 0 -1 4202560 10306 1254 0 0 1258 1 0 0 25 0 9 0 722069213 285229056 30853 33554432000 4194304 4417452 140733237395248 18446744073709551615 4248225 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4304/tid=4314] ppid=4302 vsize=278544 CPUtime=12.59 cores=0-7
/proc/4304/task/4314/stat : 4314 (plingeling) R 4302 4304 4091 0 -1 4202560 23505 1254 0 0 1254 5 0 0 25 0 9 0 722069213 285229056 30853 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364564 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4304/tid=4315] ppid=4302 vsize=278544 CPUtime=12.4 cores=0-7
/proc/4304/task/4315/stat : 4315 (plingeling) R 4302 4304 4091 0 -1 4202560 5042 1254 0 0 1240 0 0 0 25 0 9 0 722069213 285229056 30853 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4304/tid=4316] ppid=4302 vsize=278544 CPUtime=12.59 cores=0-7
/proc/4304/task/4316/stat : 4316 (plingeling) R 4302 4304 4091 0 -1 4202560 6385 1254 0 0 1258 1 0 0 25 0 9 0 722069214 285229056 30853 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4304/tid=4317] ppid=4302 vsize=278544 CPUtime=12.58 cores=0-7
/proc/4304/task/4317/stat : 4317 (plingeling) R 4302 4304 4091 0 -1 4202560 10475 1254 0 0 1257 1 0 0 25 0 9 0 722069214 285229056 30853 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4304/tid=4318] ppid=4302 vsize=278544 CPUtime=12.59 cores=0-7
/proc/4304/task/4318/stat : 4318 (plingeling) R 4302 4304 4091 0 -1 4202560 2640 1254 0 0 1259 0 0 0 25 0 9 0 722069214 285229056 30853 33554432000 4194304 4417452 140733237395248 18446744073709551615 4365036 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 100.69
Current children cumulated vsize (KiB) 278544

[startup+25.501 s]
/proc/loadavg: 7.66 7.99 8.05 9/460 4319
/proc/meminfo: memFree=29383396/32951124 swapFree=60688720/67111528
[pid=4304] ppid=4302 vsize=398244 CPUtime=202.89 cores=0-7
/proc/4304/stat : 4304 (plingeling) S 4302 4304 4091 0 -1 4202496 136731 1254 0 0 20265 24 0 0 18 0 9 0 722069203 407801856 61138 33554432000 4194304 4417452 140733237395248 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4304/statm: 99561 61138 180 55 0 97447 0
[pid=4304/tid=4311] ppid=4302 vsize=398244 CPUtime=25.39 cores=0-7
/proc/4304/task/4311/stat : 4311 (plingeling) R 4302 4304 4091 0 -1 4202560 10797 1254 0 0 2538 1 0 0 25 0 9 0 722069213 407801856 61138 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4304/tid=4312] ppid=4302 vsize=398244 CPUtime=25.39 cores=0-7
/proc/4304/task/4312/stat : 4312 (plingeling) R 4302 4304 4091 0 -1 4202560 13006 1254 0 0 2538 1 0 0 25 0 9 0 722069213 407801856 61138 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4304/tid=4313] ppid=4302 vsize=398244 CPUtime=25.39 cores=0-7
/proc/4304/task/4313/stat : 4313 (plingeling) R 4302 4304 4091 0 -1 4202560 18959 1254 0 0 2537 2 0 0 25 0 9 0 722069213 407801856 61138 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=4304/tid=4314] ppid=4302 vsize=398244 CPUtime=25.39 cores=0-7
/proc/4304/task/4314/stat : 4314 (plingeling) R 4302 4304 4091 0 -1 4202560 35870 1254 0 0 2531 8 0 0 25 0 9 0 722069213 407801856 61138 33554432000 4194304 4417452 140733237395248 18446744073709551615 4366126 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4304/tid=4315] ppid=4302 vsize=398244 CPUtime=25 cores=0-7
/proc/4304/task/4315/stat : 4315 (plingeling) R 4302 4304 4091 0 -1 4202560 14632 1254 0 0 2499 1 0 0 25 0 9 0 722069213 407801856 61138 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4304/tid=4316] ppid=4302 vsize=398244 CPUtime=25.38 cores=0-7
/proc/4304/task/4316/stat : 4316 (plingeling) R 4302 4304 4091 0 -1 4202560 15971 1254 0 0 2536 2 0 0 25 0 9 0 722069214 407801856 61138 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364835 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4304/tid=4317] ppid=4302 vsize=398244 CPUtime=25.38 cores=0-7
/proc/4304/task/4317/stat : 4317 (plingeling) R 4302 4304 4091 0 -1 4202560 15943 1254 0 0 2536 2 0 0 25 0 9 0 722069214 407801856 61138 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4304/tid=4318] ppid=4302 vsize=398244 CPUtime=25.39 cores=0-7
/proc/4304/task/4318/stat : 4318 (plingeling) R 4302 4304 4091 0 -1 4202560 6815 1254 0 0 2538 1 0 0 25 0 9 0 722069214 407801856 61138 33554432000 4194304 4417452 140733237395248 18446744073709551615 4236574 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 202.89
Current children cumulated vsize (KiB) 398244

[startup+51.1004 s]
/proc/loadavg: 7.84 8.00 8.05 9/460 4319
/proc/meminfo: memFree=29224936/32951124 swapFree=60688720/67111528
[pid=4304] ppid=4302 vsize=544324 CPUtime=407.29 cores=0-7
/proc/4304/stat : 4304 (plingeling) S 4302 4304 4091 0 -1 4202496 246989 1254 0 0 40686 43 0 0 18 0 9 0 722069203 557387776 101108 33554432000 4194304 4417452 140733237395248 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4304/statm: 136081 101108 180 55 0 133967 0
[pid=4304/tid=4311] ppid=4302 vsize=544324 CPUtime=50.99 cores=0-7
/proc/4304/task/4311/stat : 4311 (plingeling) R 4302 4304 4091 0 -1 4202560 19715 1254 0 0 5096 3 0 0 25 0 9 0 722069213 557387776 101108 33554432000 4194304 4417452 140733237395248 18446744073709551615 4366065 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4304/tid=4312] ppid=4302 vsize=544324 CPUtime=50.99 cores=0-7
/proc/4304/task/4312/stat : 4312 (plingeling) R 4302 4304 4091 0 -1 4202560 25015 1254 0 0 5096 3 0 0 25 0 9 0 722069213 557387776 101108 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4304/tid=4313] ppid=4302 vsize=544324 CPUtime=50.99 cores=0-7

################
# More data... #
################

[pid=4304/tid=4313] ppid=4302 vsize=4700008 CPUtime=4658.51 cores=0-7
/proc/4304/task/4313/stat : 4313 (plingeling) R 4302 4304 4091 0 -1 4202560 3792781 1254 0 0 465333 518 0 0 25 0 9 0 722069213 4812808192 1148588 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4304/tid=4314] ppid=4302 vsize=4700008 CPUtime=4722.28 cores=0-7
/proc/4304/task/4314/stat : 4314 (plingeling) R 4302 4304 4091 0 -1 4202560 806278 1254 0 0 472052 176 0 0 25 0 9 0 722069213 4812808192 1148588 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4304/tid=4315] ppid=4302 vsize=4700008 CPUtime=4713.11 cores=0-7
/proc/4304/task/4315/stat : 4315 (plingeling) R 4302 4304 4091 0 -1 4202560 4221509 1254 0 0 470741 570 0 0 25 0 9 0 722069213 4812808192 1148588 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4304/tid=4316] ppid=4302 vsize=4700008 CPUtime=4720.8 cores=0-7
/proc/4304/task/4316/stat : 4316 (plingeling) R 4302 4304 4091 0 -1 4202560 3336015 1254 0 0 471616 464 0 0 25 0 9 0 722069214 4812808192 1148588 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4304/tid=4317] ppid=4302 vsize=4700008 CPUtime=4721.67 cores=0-7
/proc/4304/task/4317/stat : 4317 (plingeling) R 4302 4304 4091 0 -1 4202560 2597131 1254 0 0 471813 354 0 0 25 0 9 0 722069214 4812808192 1148588 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4304/tid=4318] ppid=4302 vsize=4700008 CPUtime=4722.29 cores=0-7
/proc/4304/task/4318/stat : 4318 (plingeling) R 4302 4304 4091 0 -1 4202560 3034354 1254 0 0 471799 430 0 0 25 0 9 0 722069214 4812808192 1148588 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 37702.8
Current children cumulated vsize (KiB) 4700008

[startup+4782.3 s]
/proc/loadavg: 8.16 8.19 8.16 9/460 4469
/proc/meminfo: memFree=25002308/32951124 swapFree=60688720/67111528
[pid=4304] ppid=4302 vsize=4704168 CPUtime=38181.7 cores=0-7
/proc/4304/stat : 4304 (plingeling) S 4302 4304 4091 0 -1 4202496 23157306 1254 0 0 3814905 3267 0 0 18 0 9 0 722069203 4817068032 1149628 33554432000 4194304 4417452 140733237395248 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4304/statm: 1176042 1149628 180 55 0 1173928 0
[pid=4304/tid=4311] ppid=4302 vsize=4704168 CPUtime=4782.29 cores=0-7
/proc/4304/task/4311/stat : 4311 (plingeling) R 4302 4304 4091 0 -1 4202560 2370194 1254 0 0 477888 341 0 0 25 0 9 0 722069213 4817068032 1149628 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4304/tid=4312] ppid=4302 vsize=4704168 CPUtime=4781.62 cores=0-7
/proc/4304/task/4312/stat : 4312 (plingeling) R 4302 4304 4091 0 -1 4202560 2911346 1254 0 0 477762 400 0 0 25 0 9 0 722069213 4817068032 1149628 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4304/tid=4313] ppid=4302 vsize=4704168 CPUtime=4717.58 cores=0-7
/proc/4304/task/4313/stat : 4313 (plingeling) R 4302 4304 4091 0 -1 4202560 3792781 1254 0 0 471240 518 0 0 25 0 9 0 722069213 4817068032 1149628 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4304/tid=4314] ppid=4302 vsize=4704168 CPUtime=4782.27 cores=0-7
/proc/4304/task/4314/stat : 4314 (plingeling) R 4302 4304 4091 0 -1 4202560 806278 1254 0 0 478051 176 0 0 25 0 9 0 722069213 4817068032 1149628 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4304/tid=4315] ppid=4302 vsize=4704168 CPUtime=4773.09 cores=0-7
/proc/4304/task/4315/stat : 4315 (plingeling) R 4302 4304 4091 0 -1 4202560 4287045 1254 0 0 476732 577 0 0 25 0 9 0 722069213 4817068032 1149628 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4304/tid=4316] ppid=4302 vsize=4704168 CPUtime=4780.77 cores=0-7
/proc/4304/task/4316/stat : 4316 (plingeling) R 4302 4304 4091 0 -1 4202560 3336015 1254 0 0 477613 464 0 0 25 0 9 0 722069214 4817068032 1149628 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4304/tid=4317] ppid=4302 vsize=4704168 CPUtime=4781.65 cores=0-7
/proc/4304/task/4317/stat : 4317 (plingeling) R 4302 4304 4091 0 -1 4202560 2597131 1254 0 0 477811 354 0 0 25 0 9 0 722069214 4817068032 1149628 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4304/tid=4318] ppid=4302 vsize=4704168 CPUtime=4782.27 cores=0-7
/proc/4304/task/4318/stat : 4318 (plingeling) R 4302 4304 4091 0 -1 4202560 3051778 1254 0 0 477796 431 0 0 25 0 9 0 722069214 4817068032 1149628 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 38181.7
Current children cumulated vsize (KiB) 4704168

[startup+4842.3 s]
/proc/loadavg: 8.09 8.17 8.16 9/460 4471
/proc/meminfo: memFree=24848512/32951124 swapFree=60688720/67111528
[pid=4304] ppid=4302 vsize=4896936 CPUtime=38660.8 cores=0-7
/proc/4304/stat : 4304 (plingeling) S 4302 4304 4091 0 -1 4202496 23367801 1254 0 0 3862788 3290 0 0 18 0 9 0 722069203 5014462464 1188091 33554432000 4194304 4417452 140733237395248 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4304/statm: 1224234 1188091 180 55 0 1222120 0
[pid=4304/tid=4311] ppid=4302 vsize=4896936 CPUtime=4842.3 cores=0-7
/proc/4304/task/4311/stat : 4311 (plingeling) R 4302 4304 4091 0 -1 4202560 2442385 1254 0 0 483880 350 0 0 25 0 9 0 722069213 5014462464 1188091 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4304/tid=4312] ppid=4302 vsize=4896936 CPUtime=4841.62 cores=0-7
/proc/4304/task/4312/stat : 4312 (plingeling) R 4302 4304 4091 0 -1 4202560 2911346 1254 0 0 483762 400 0 0 25 0 9 0 722069213 5014462464 1188091 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4304/tid=4313] ppid=4302 vsize=4896936 CPUtime=4776.64 cores=0-7
/proc/4304/task/4313/stat : 4313 (plingeling) R 4302 4304 4091 0 -1 4202560 3792781 1254 0 0 477146 518 0 0 25 0 9 0 722069213 5014462464 1188091 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364768 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4304/tid=4314] ppid=4302 vsize=4896936 CPUtime=4842.27 cores=0-7
/proc/4304/task/4314/stat : 4314 (plingeling) R 4302 4304 4091 0 -1 4202560 838086 1254 0 0 484048 179 0 0 25 0 9 0 722069213 5014462464 1188091 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4304/tid=4315] ppid=4302 vsize=4896936 CPUtime=4833.08 cores=0-7
/proc/4304/task/4315/stat : 4315 (plingeling) R 4302 4304 4091 0 -1 4202560 4385349 1254 0 0 482720 588 0 0 25 0 9 0 722069213 5014462464 1188091 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4304/tid=4316] ppid=4302 vsize=4896936 CPUtime=4840.77 cores=0-7
/proc/4304/task/4316/stat : 4316 (plingeling) R 4302 4304 4091 0 -1 4202560 3336015 1254 0 0 483613 464 0 0 25 0 9 0 722069214 5014462464 1188091 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4304/tid=4317] ppid=4302 vsize=4896936 CPUtime=4841.65 cores=0-7
/proc/4304/task/4317/stat : 4317 (plingeling) R 4302 4304 4091 0 -1 4202560 2597131 1254 0 0 483811 354 0 0 25 0 9 0 722069214 5014462464 1188091 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4304/tid=4318] ppid=4302 vsize=4896936 CPUtime=4842.27 cores=0-7
/proc/4304/task/4318/stat : 4318 (plingeling) R 4302 4304 4091 0 -1 4202560 3059970 1254 0 0 483795 432 0 0 25 0 9 0 722069214 5014462464 1188091 33554432000 4194304 4417452 140733237395248 18446744073709551615 4366385 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 38660.8
Current children cumulated vsize (KiB) 4896936

[startup+4902.3 s]
/proc/loadavg: 8.31 8.22 8.18 9/460 4473
/proc/meminfo: memFree=24848604/32951124 swapFree=60688720/67111528
[pid=4304] ppid=4302 vsize=4896936 CPUtime=39139.9 cores=0-7
/proc/4304/stat : 4304 (plingeling) S 4302 4304 4091 0 -1 4202496 23597177 1254 0 0 3910668 3318 0 0 18 0 9 0 722069203 5014462464 1188091 33554432000 4194304 4417452 140733237395248 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4304/statm: 1224234 1188091 180 55 0 1222120 0
[pid=4304/tid=4311] ppid=4302 vsize=4896936 CPUtime=4902.3 cores=0-7
/proc/4304/task/4311/stat : 4311 (plingeling) R 4302 4304 4091 0 -1 4202560 2548881 1254 0 0 489868 362 0 0 25 0 9 0 722069213 5014462464 1188091 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364829 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4304/tid=4312] ppid=4302 vsize=4896936 CPUtime=4901.63 cores=0-7
/proc/4304/task/4312/stat : 4312 (plingeling) R 4302 4304 4091 0 -1 4202560 2911346 1254 0 0 489763 400 0 0 25 0 9 0 722069213 5014462464 1188091 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4304/tid=4313] ppid=4302 vsize=4896936 CPUtime=4835.73 cores=0-7
/proc/4304/task/4313/stat : 4313 (plingeling) R 4302 4304 4091 0 -1 4202560 3792781 1254 0 0 483055 518 0 0 25 0 9 0 722069213 5014462464 1188091 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4304/tid=4314] ppid=4302 vsize=4896936 CPUtime=4902.27 cores=0-7
/proc/4304/task/4314/stat : 4314 (plingeling) R 4302 4304 4091 0 -1 4202560 838086 1254 0 0 490048 179 0 0 25 0 9 0 722069213 5014462464 1188091 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4304/tid=4315] ppid=4302 vsize=4896936 CPUtime=4893.09 cores=0-7
/proc/4304/task/4315/stat : 4315 (plingeling) R 4302 4304 4091 0 -1 4202560 4467269 1254 0 0 488710 599 0 0 25 0 9 0 722069213 5014462464 1188091 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4304/tid=4316] ppid=4302 vsize=4896936 CPUtime=4900.75 cores=0-7
/proc/4304/task/4316/stat : 4316 (plingeling) R 4302 4304 4091 0 -1 4202560 3336015 1254 0 0 489611 464 0 0 25 0 9 0 722069214 5014462464 1188091 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4304/tid=4317] ppid=4302 vsize=4896936 CPUtime=4901.65 cores=0-7
/proc/4304/task/4317/stat : 4317 (plingeling) R 4302 4304 4091 0 -1 4202560 2597131 1254 0 0 489811 354 0 0 25 0 9 0 722069214 5014462464 1188091 33554432000 4194304 4417452 140733237395248 18446744073709551615 4366385 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4304/tid=4318] ppid=4302 vsize=4896936 CPUtime=4902.28 cores=0-7
/proc/4304/task/4318/stat : 4318 (plingeling) R 4302 4304 4091 0 -1 4202560 3100930 1254 0 0 489791 437 0 0 25 0 9 0 722069214 5014462464 1188091 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 39139.9
Current children cumulated vsize (KiB) 4896936

[startup+4962.3 s]
/proc/loadavg: 8.22 8.21 8.18 9/460 4474
/proc/meminfo: memFree=24867544/32951124 swapFree=60688720/67111528
[pid=4304] ppid=4302 vsize=4878392 CPUtime=39618.9 cores=0-7
/proc/4304/stat : 4304 (plingeling) S 4302 4304 4091 0 -1 4202496 23846493 1254 0 0 3958544 3347 0 0 18 0 9 0 722069203 4995473408 1183455 33554432000 4194304 4417452 140733237395248 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4304/statm: 1219598 1183455 180 55 0 1217484 0
[pid=4304/tid=4311] ppid=4302 vsize=4878392 CPUtime=4962.3 cores=0-7
/proc/4304/task/4311/stat : 4311 (plingeling) R 4302 4304 4091 0 -1 4202560 2658933 1254 0 0 495855 375 0 0 25 0 9 0 722069213 4995473408 1183455 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4304/tid=4312] ppid=4302 vsize=4878392 CPUtime=4961.63 cores=0-7
/proc/4304/task/4312/stat : 4312 (plingeling) R 4302 4304 4091 0 -1 4202560 2911346 1254 0 0 495763 400 0 0 25 0 9 0 722069213 4995473408 1183455 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4304/tid=4313] ppid=4302 vsize=4878392 CPUtime=4894.79 cores=0-7
/proc/4304/task/4313/stat : 4313 (plingeling) R 4302 4304 4091 0 -1 4202560 3792781 1254 0 0 488961 518 0 0 25 0 9 0 722069213 4995473408 1183455 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4304/tid=4314] ppid=4302 vsize=4878392 CPUtime=4962.27 cores=0-7
/proc/4304/task/4314/stat : 4314 (plingeling) R 4302 4304 4091 0 -1 4202560 838086 1254 0 0 496048 179 0 0 25 0 9 0 722069213 4995473408 1183455 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4304/tid=4315] ppid=4302 vsize=4878392 CPUtime=4953.09 cores=0-7
/proc/4304/task/4315/stat : 4315 (plingeling) R 4302 4304 4091 0 -1 4202560 4524613 1254 0 0 494703 606 0 0 25 0 9 0 722069213 4995473408 1183455 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4304/tid=4316] ppid=4302 vsize=4878392 CPUtime=4960.74 cores=0-7
/proc/4304/task/4316/stat : 4316 (plingeling) R 4302 4304 4091 0 -1 4202560 3336015 1254 0 0 495610 464 0 0 25 0 9 0 722069214 4995473408 1183455 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4304/tid=4317] ppid=4302 vsize=4878392 CPUtime=4961.65 cores=0-7
/proc/4304/task/4317/stat : 4317 (plingeling) R 4302 4304 4091 0 -1 4202560 2597131 1254 0 0 495811 354 0 0 25 0 9 0 722069214 4995473408 1183455 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4304/tid=4318] ppid=4302 vsize=4878392 CPUtime=4962.28 cores=0-7
/proc/4304/task/4318/stat : 4318 (plingeling) R 4302 4304 4091 0 -1 4202560 3182850 1254 0 0 495781 447 0 0 25 0 9 0 722069214 4995473408 1183455 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364821 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 39618.9
Current children cumulated vsize (KiB) 4878392



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5010.1 s]
/proc/loadavg: 8.19 8.20 8.18 9/460 4475
/proc/meminfo: memFree=24736200/32951124 swapFree=60688720/67111528
[pid=4304] ppid=4302 vsize=4977268 CPUtime=40000.6 cores=0-7
/proc/4304/stat : 4304 (plingeling) S 4302 4304 4091 0 -1 4202496 24045362 1254 0 0 3996684 3378 0 0 18 0 9 0 722069203 5096722432 1216219 33554432000 4194304 4417452 140733237395248 18446744073709551615 232347695077 0 0 4096 17442 18446744073709551615 0 0 17 7 0 0 0
/proc/4304/statm: 1244317 1216219 180 55 0 1242203 0
[pid=4304/tid=4311] ppid=4302 vsize=4977268 CPUtime=5010.12 cores=0-7
/proc/4304/task/4311/stat : 4311 (plingeling) R 4302 4304 4091 0 -1 4202560 2765610 1254 0 0 500618 394 0 0 25 0 9 0 722069213 5096722432 1216219 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=4304/tid=4312] ppid=4302 vsize=4977268 CPUtime=5009.45 cores=0-7
/proc/4304/task/4312/stat : 4312 (plingeling) R 4302 4304 4091 0 -1 4202560 2927730 1254 0 0 500542 403 0 0 25 0 9 0 722069213 5096722432 1216219 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=4304/tid=4313] ppid=4302 vsize=4977268 CPUtime=4941.87 cores=0-7
/proc/4304/task/4313/stat : 4313 (plingeling) R 4302 4304 4091 0 -1 4202560 3809165 1254 0 0 493667 520 0 0 25 0 9 0 722069213 5096722432 1216219 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=4304/tid=4314] ppid=4302 vsize=4977268 CPUtime=5010.09 cores=0-7
/proc/4304/task/4314/stat : 4314 (plingeling) R 4302 4304 4091 0 -1 4202560 854470 1254 0 0 500828 181 0 0 25 0 9 0 722069213 5096722432 1216219 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=4304/tid=4315] ppid=4302 vsize=4977268 CPUtime=5000.9 cores=0-7
/proc/4304/task/4315/stat : 4315 (plingeling) R 4302 4304 4091 0 -1 4202560 4540997 1254 0 0 499482 608 0 0 25 0 9 0 722069213 5096722432 1216219 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=4304/tid=4316] ppid=4302 vsize=4977268 CPUtime=5008.54 cores=0-7
/proc/4304/task/4316/stat : 4316 (plingeling) R 4302 4304 4091 0 -1 4202560 3336015 1254 0 0 500390 464 0 0 25 0 9 0 722069214 5096722432 1216219 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=4304/tid=4317] ppid=4302 vsize=4977268 CPUtime=5009.47 cores=0-7
/proc/4304/task/4317/stat : 4317 (plingeling) R 4302 4304 4091 0 -1 4202560 2597131 1254 0 0 500593 354 0 0 25 0 9 0 722069214 5096722432 1216219 33554432000 4194304 4417452 140733237395248 18446744073709551615 4263155 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=4304/tid=4318] ppid=4302 vsize=4977268 CPUtime=5010.09 cores=0-7
/proc/4304/task/4318/stat : 4318 (plingeling) R 4302 4304 4091 0 -1 4202560 3209506 1254 0 0 500559 450 0 0 25 0 9 0 722069214 5096722432 1216219 33554432000 4194304 4417452 140733237395248 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 4977268

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5010.22 s]
/proc/loadavg: 8.19 8.20 8.18 9/460 4475
/proc/meminfo: memFree=24736200/32951124 swapFree=60688720/67111528
[pid=4304] ppid=4302 vsize=0 CPUtime=40000.8 cores=0-7
/proc/4304/stat : 4304 (plingeling) Z 4302 4304 4091 0 -1 4203548 24045362 1254 0 0 3996696 3388 0 0 18 0 2 0 722069203 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 7 0 0 0
/proc/4304/statm: 0 0 0 0 0 0 0
[pid=4304/tid=4313] ppid=4302 vsize=0 CPUtime=4941.97 cores=0-7
/proc/4304/task/4313/stat : 4313 (plingeling) R 4302 4304 4091 0 -1 4203588 3809165 1254 0 0 493667 530 0 0 25 0 2 0 722069213 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40000.8
Current children cumulated vsize (KiB) 0

[startup+5010.3 s]
/proc/loadavg: 8.19 8.20 8.18 9/460 4475
/proc/meminfo: memFree=24736200/32951124 swapFree=60688720/67111528
[pid=4304] ppid=4302 vsize=0 CPUtime=40000.9 cores=0-7
/proc/4304/stat : 4304 (plingeling) Z 4302 4304 4091 0 -1 4203548 24045362 1254 0 0 3996696 3396 0 0 18 0 2 0 722069203 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 7 0 0 0
/proc/4304/statm: 0 0 0 0 0 0 0
[pid=4304/tid=4313] ppid=4302 vsize=0 CPUtime=4942.05 cores=0-7
/proc/4304/task/4313/stat : 4313 (plingeling) R 4302 4304 4091 0 -1 4203588 3809165 1254 0 0 493667 538 0 0 25 0 2 0 722069213 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40000.9
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5010.35
CPU time (s): 40001
CPU user time (s): 39967
CPU system time (s): 34.0068
CPU usage (%): 798.367
Max. virtual memory (cumulated for all children) (KiB): 5165984

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39967
system time used= 34.0068
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 24046616
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 248
involuntary context switches= 348894

runsolver used 22.2066 second user time and 55.1486 second system time

The end

Launcher Data 

Begin job on node132 at 2011-05-23 00:51:50
IDJOB=3394732
IDBENCH=83986
IDSOLVER=1851
FILE ID=node132/3394732-1306104710
RUNJOBID= node132-1306099697-4105
PBS_JOBID= 13387978
Free space on /tmp= 73268 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT03/handmade/anton/UNSAT/SGI_30_60_29_30_6-dir.shuffled-as.sat03-130.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3394732-1306104710/watcher-3394732-1306104710 -o /tmp/evaluation-result-3394732-1306104710/solver-3394732-1306104710 -C 40000 -W 5100 -M 31000  plingeling -t 8 HOME/instance-3394732-1306104710.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= e826117d87949a5b285e72732bfa395b
RANDOM SEED=1389110246

node132.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.833
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      29614352 kB
Buffers:         12332 kB
Cached:          92680 kB
SwapCached:      14568 kB
Active:          64412 kB
Inactive:      3097152 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      29614352 kB
SwapTotal:    67111528 kB
SwapFree:     60688720 kB
Dirty:             792 kB
Writeback:          16 kB
AnonPages:     3049908 kB
Mapped:          18560 kB
Slab:            56132 kB
PageTables:      49752 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 109198248 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73268 MiB
End job on node132 at 2011-05-23 02:15:23