Trace number 3371109

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.17 1203.35

General information on the benchmark

NameSAT05/crafted/sorge05/QG7/
gensys-icl007.shuffled-as.sat05-3133.cnf
MD5SUM88361752c7b0c05e849791aafdd7bdba
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark220.275
Satisfiable
(Un)Satisfiability was proved
Number of variables762
Number of clauses17262
Sum of the clauses size76511
Maximum clause length13
Minimum clause length1
Number of clauses of size 142
Number of clauses of size 21135
Number of clauses of size 38742
Number of clauses of size 4236
Number of clauses of size 5256
Number of clauses of size over 56851

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3371109-1305431944/watcher-3371109-1305431944 -o /tmp/evaluation-result-3371109-1305431944/solver-3371109-1305431944 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3371109-1305431944.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.38 4.52 6.18 5/353 12314
/proc/meminfo: memFree=26158168/32951124 swapFree=53501236/67111528
[pid=12314] ppid=12312 vsize=10704 CPUtime=0 cores=1,3,5,7
/proc/12314/stat : 12314 (plingeling) R 12312 12314 11874 0 -1 4202496 737 1252 0 0 0 0 0 0 25 0 1 0 654797017 10960896 668 33554432000 4194304 4417452 140736082868912 18446744073709551615 216634515200 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/12314/statm: 2676 668 149 55 0 562 0

[startup+0.059084 s]
/proc/loadavg: 3.38 4.52 6.18 5/353 12314
/proc/meminfo: memFree=26158168/32951124 swapFree=53501236/67111528
[pid=12314] ppid=12312 vsize=251908 CPUtime=0.14 cores=1,3,5,7
/proc/12314/stat : 12314 (plingeling) S 12312 12314 11874 0 -1 4202496 1705 1252 0 0 14 0 0 0 25 0 5 0 654797017 257953792 1610 33554432000 4194304 4417452 140736082868912 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/12314/statm: 62977 1610 174 55 0 60863 0
Current children cumulated CPU time (s) 0.14
Current children cumulated vsize (KiB) 251908

[startup+0.134572 s]
/proc/loadavg: 3.38 4.52 6.18 5/353 12314
/proc/meminfo: memFree=26158168/32951124 swapFree=53501236/67111528
[pid=12314] ppid=12312 vsize=186904 CPUtime=0.44 cores=1,3,5,7
/proc/12314/stat : 12314 (plingeling) S 12312 12314 11874 0 -1 4202496 1883 1252 0 0 44 0 0 0 25 0 5 0 654797017 191389696 1788 33554432000 4194304 4417452 140736082868912 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/12314/statm: 46726 1788 180 55 0 44612 0
Current children cumulated CPU time (s) 0.44
Current children cumulated vsize (KiB) 186904

[startup+0.300545 s]
/proc/loadavg: 3.38 4.52 6.18 5/353 12314
/proc/meminfo: memFree=26158168/32951124 swapFree=53501236/67111528
[pid=12314] ppid=12312 vsize=187288 CPUtime=1.1 cores=1,3,5,7
/proc/12314/stat : 12314 (plingeling) S 12312 12314 11874 0 -1 4202496 2184 1252 0 0 110 0 0 0 25 0 5 0 654797017 191782912 1888 33554432000 4194304 4417452 140736082868912 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/12314/statm: 46822 1888 180 55 0 44708 0
Current children cumulated CPU time (s) 1.1
Current children cumulated vsize (KiB) 187288

[startup+0.700475 s]
/proc/loadavg: 3.38 4.52 6.18 5/353 12314
/proc/meminfo: memFree=26158168/32951124 swapFree=53501236/67111528
[pid=12314] ppid=12312 vsize=189696 CPUtime=2.7 cores=1,3,5,7
/proc/12314/stat : 12314 (plingeling) S 12312 12314 11874 0 -1 4202496 3662 1252 0 0 270 0 0 0 25 0 5 0 654797017 194248704 2479 33554432000 4194304 4417452 140736082868912 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/12314/statm: 47424 2479 180 55 0 45310 0
Current children cumulated CPU time (s) 2.7
Current children cumulated vsize (KiB) 189696

[startup+1.50034 s]
/proc/loadavg: 3.38 4.52 6.18 9/358 12324
/proc/meminfo: memFree=26139032/32951124 swapFree=53501236/67111528
[pid=12314] ppid=12312 vsize=192232 CPUtime=5.89 cores=1,3,5,7
/proc/12314/stat : 12314 (plingeling) S 12312 12314 11874 0 -1 4202496 7136 1252 0 0 588 1 0 0 25 0 5 0 654797017 196845568 3145 33554432000 4194304 4417452 140736082868912 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/12314/statm: 48058 3145 180 55 0 45944 0
[pid=12314/tid=12321] ppid=12312 vsize=192232 CPUtime=1.47 cores=1,3,5,7
/proc/12314/task/12321/stat : 12321 (plingeling) R 12312 12314 11874 0 -1 4202560 1505 1252 0 0 147 0 0 0 25 0 5 0 654797020 196845568 3145 33554432000 4194304 4417452 140736082868912 18446744073709551615 4321404 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12314/tid=12322] ppid=12312 vsize=192232 CPUtime=1.46 cores=1,3,5,7
/proc/12314/task/12322/stat : 12322 (plingeling) R 12312 12314 11874 0 -1 4202560 1474 1252 0 0 146 0 0 0 25 0 5 0 654797020 196845568 3145 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364819 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12314/tid=12323] ppid=12312 vsize=192232 CPUtime=1.46 cores=1,3,5,7
/proc/12314/task/12323/stat : 12323 (plingeling) R 12312 12314 11874 0 -1 4202560 797 1252 0 0 146 0 0 0 25 0 5 0 654797020 196845568 3145 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12314/tid=12324] ppid=12312 vsize=192232 CPUtime=1.45 cores=1,3,5,7
/proc/12314/task/12324/stat : 12324 (plingeling) R 12312 12314 11874 0 -1 4202560 1697 1252 0 0 145 0 0 0 25 0 5 0 654797020 196845568 3145 33554432000 4194304 4417452 140736082868912 18446744073709551615 4366130 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 5.89
Current children cumulated vsize (KiB) 192232

[startup+3.10105 s]
/proc/loadavg: 3.38 4.52 6.18 9/358 12324
/proc/meminfo: memFree=26124896/32951124 swapFree=53501236/67111528
[pid=12314] ppid=12312 vsize=196900 CPUtime=12.28 cores=1,3,5,7
/proc/12314/stat : 12314 (plingeling) S 12312 12314 11874 0 -1 4202496 20749 1252 0 0 1225 3 0 0 25 0 5 0 654797017 201625600 4294 33554432000 4194304 4417452 140736082868912 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/12314/statm: 49225 4294 180 55 0 47111 0
[pid=12314/tid=12321] ppid=12312 vsize=196900 CPUtime=3.06 cores=1,3,5,7
/proc/12314/task/12321/stat : 12321 (plingeling) R 12312 12314 11874 0 -1 4202560 5151 1252 0 0 306 0 0 0 25 0 5 0 654797020 201625600 4294 33554432000 4194304 4417452 140736082868912 18446744073709551615 4365042 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12314/tid=12322] ppid=12312 vsize=196900 CPUtime=3.06 cores=1,3,5,7
/proc/12314/task/12322/stat : 12322 (plingeling) R 12312 12314 11874 0 -1 4202560 6702 1252 0 0 305 1 0 0 25 0 5 0 654797020 201625600 4294 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364669 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12314/tid=12323] ppid=12312 vsize=196900 CPUtime=3.06 cores=1,3,5,7
/proc/12314/task/12323/stat : 12323 (plingeling) R 12312 12314 11874 0 -1 4202560 4117 1252 0 0 306 0 0 0 25 0 5 0 654797020 201625600 4294 33554432000 4194304 4417452 140736082868912 18446744073709551615 4199406 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12314/tid=12324] ppid=12312 vsize=196900 CPUtime=3.04 cores=1,3,5,7
/proc/12314/task/12324/stat : 12324 (plingeling) R 12312 12314 11874 0 -1 4202560 3116 1252 0 0 304 0 0 0 25 0 5 0 654797020 201625600 4294 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 12.28
Current children cumulated vsize (KiB) 196900

[startup+6.30049 s]
/proc/loadavg: 3.75 4.58 6.19 9/358 12324
/proc/meminfo: memFree=26106308/32951124 swapFree=53501236/67111528
[pid=12314] ppid=12312 vsize=209088 CPUtime=25.04 cores=1,3,5,7
/proc/12314/stat : 12314 (plingeling) S 12312 12314 11874 0 -1 4202496 36186 1252 0 0 2499 5 0 0 25 0 5 0 654797017 214106112 7365 33554432000 4194304 4417452 140736082868912 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/12314/statm: 52272 7365 180 55 0 50158 0
[pid=12314/tid=12321] ppid=12312 vsize=209088 CPUtime=6.26 cores=1,3,5,7
/proc/12314/task/12321/stat : 12321 (plingeling) R 12312 12314 11874 0 -1 4202560 9259 1252 0 0 625 1 0 0 25 0 5 0 654797020 214106112 7365 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364650 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12314/tid=12322] ppid=12312 vsize=209088 CPUtime=6.26 cores=1,3,5,7
/proc/12314/task/12322/stat : 12322 (plingeling) R 12312 12314 11874 0 -1 4202560 11289 1252 0 0 625 1 0 0 25 0 5 0 654797020 214106112 7365 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12314/tid=12323] ppid=12312 vsize=209088 CPUtime=6.26 cores=1,3,5,7
/proc/12314/task/12323/stat : 12323 (plingeling) R 12312 12314 11874 0 -1 4202560 8164 1252 0 0 626 0 0 0 25 0 5 0 654797020 214106112 7365 33554432000 4194304 4417452 140736082868912 18446744073709551615 4199406 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12314/tid=12324] ppid=12312 vsize=209088 CPUtime=6.2 cores=1,3,5,7
/proc/12314/task/12324/stat : 12324 (plingeling) R 12312 12314 11874 0 -1 4202560 5811 1252 0 0 620 0 0 0 25 0 5 0 654797020 214106112 7365 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 25.04
Current children cumulated vsize (KiB) 209088

[startup+12.7004 s]
/proc/loadavg: 4.09 4.63 6.20 9/358 12324
/proc/meminfo: memFree=26068188/32951124 swapFree=53501236/67111528
[pid=12314] ppid=12312 vsize=220588 CPUtime=50.56 cores=1,3,5,7
/proc/12314/stat : 12314 (plingeling) S 12312 12314 11874 0 -1 4202496 65100 1252 0 0 5048 8 0 0 25 0 5 0 654797017 225882112 10208 33554432000 4194304 4417452 140736082868912 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/12314/statm: 55147 10208 180 55 0 53033 0
[pid=12314/tid=12321] ppid=12312 vsize=220588 CPUtime=12.66 cores=1,3,5,7
/proc/12314/task/12321/stat : 12321 (plingeling) R 12312 12314 11874 0 -1 4202560 18732 1252 0 0 1264 2 0 0 25 0 5 0 654797020 225882112 10208 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364838 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12314/tid=12322] ppid=12312 vsize=220588 CPUtime=12.66 cores=1,3,5,7
/proc/12314/task/12322/stat : 12322 (plingeling) R 12312 12314 11874 0 -1 4202560 24019 1252 0 0 1263 3 0 0 25 0 5 0 654797020 225882112 10208 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12314/tid=12323] ppid=12312 vsize=220588 CPUtime=12.66 cores=1,3,5,7
/proc/12314/task/12323/stat : 12323 (plingeling) R 12312 12314 11874 0 -1 4202560 10831 1252 0 0 1266 0 0 0 25 0 5 0 654797020 225882112 10208 33554432000 4194304 4417452 140736082868912 18446744073709551615 4366400 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12314/tid=12324] ppid=12312 vsize=220588 CPUtime=12.53 cores=1,3,5,7
/proc/12314/task/12324/stat : 12324 (plingeling) R 12312 12314 11874 0 -1 4202560 9855 1252 0 0 1252 1 0 0 25 0 5 0 654797020 225882112 10208 33554432000 4194304 4417452 140736082868912 18446744073709551615 4250936 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.56
Current children cumulated vsize (KiB) 220588

[startup+25.5011 s]
/proc/loadavg: 4.96 4.80 6.23 9/358 12324
/proc/meminfo: memFree=26012408/32951124 swapFree=53501236/67111528
[pid=12314] ppid=12312 vsize=236372 CPUtime=101.63 cores=1,3,5,7
/proc/12314/stat : 12314 (plingeling) S 12312 12314 11874 0 -1 4202496 133452 1252 0 0 10144 19 0 0 25 0 5 0 654797017 242044928 14165 33554432000 4194304 4417452 140736082868912 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/12314/statm: 59093 14165 180 55 0 56979 0
[pid=12314/tid=12321] ppid=12312 vsize=236372 CPUtime=25.47 cores=1,3,5,7
/proc/12314/task/12321/stat : 12321 (plingeling) R 12312 12314 11874 0 -1 4202560 38605 1252 0 0 2541 6 0 0 25 0 5 0 654797020 242044928 14165 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364558 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12314/tid=12322] ppid=12312 vsize=236372 CPUtime=25.46 cores=1,3,5,7
/proc/12314/task/12322/stat : 12322 (plingeling) R 12312 12314 11874 0 -1 4202560 46003 1252 0 0 2540 6 0 0 25 0 5 0 654797020 242044928 14165 33554432000 4194304 4417452 140736082868912 18446744073709551615 4366065 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12314/tid=12323] ppid=12312 vsize=236372 CPUtime=25.46 cores=1,3,5,7
/proc/12314/task/12323/stat : 12323 (plingeling) R 12312 12314 11874 0 -1 4202560 31837 1252 0 0 2542 4 0 0 25 0 5 0 654797020 242044928 14165 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364829 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12314/tid=12324] ppid=12312 vsize=236372 CPUtime=25.18 cores=1,3,5,7
/proc/12314/task/12324/stat : 12324 (plingeling) R 12312 12314 11874 0 -1 4202560 15344 1252 0 0 2516 2 0 0 25 0 5 0 654797020 242044928 14165 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.63
Current children cumulated vsize (KiB) 236372

[startup+51.1006 s]
/proc/loadavg: 6.00 5.05 6.27 9/358 12325
/proc/meminfo: memFree=25956692/32951124 swapFree=53501236/67111528
[pid=12314] ppid=12312 vsize=252384 CPUtime=203.75 cores=1,3,5,7
/proc/12314/stat : 12314 (plingeling) S 12312 12314 11874 0 -1 4202496 299951 1252 0 0 20334 41 0 0 25 0 5 0 654797017 258441216 18189 33554432000 4194304 4417452 140736082868912 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/12314/statm: 63096 18189 180 55 0 60982 0
[pid=12314/tid=12321] ppid=12312 vsize=252384 CPUtime=51.07 cores=1,3,5,7
/proc/12314/task/12321/stat : 12321 (plingeling) R 12312 12314 11874 0 -1 4202560 84604 1252 0 0 5095 12 0 0 25 0 5 0 654797020 258441216 18189 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12314/tid=12322] ppid=12312 vsize=252384 CPUtime=51.06 cores=1,3,5,7
/proc/12314/task/12322/stat : 12322 (plingeling) R 12312 12314 11874 0 -1 4202560 109378 1252 0 0 5092 14 0 0 25 0 5 0 654797020 258441216 18189 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12314/tid=12323] ppid=12312 vsize=252384 CPUtime=51.06 cores=1,3,5,7
/proc/12314/task/12323/stat : 12323 (plingeling) R 12312 12314 11874 0 -1 4202560 78446 1252 0 0 5096 10 0 0 25 0 5 0 654797020 258441216 18189 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364817 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12314/tid=12324] ppid=12312 vsize=252384 CPUtime=50.51 cores=1,3,5,7
/proc/12314/task/12324/stat : 12324 (plingeling) R 12312 12314 11874 0 -1 4202560 25860 1252 0 0 5047 4 0 0 25 0 5 0 654797020 258441216 18189 33554432000 4194304 4417452 140736082868912 18446744073709551615 4365042 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 203.75
Current children cumulated vsize (KiB) 252384

[startup+102.314 s]
/proc/loadavg: 7.13 5.51 6.36 9/358 12327
/proc/meminfo: memFree=25871480/32951124 swapFree=53501236/67111528
[pid=12314] ppid=12312 vsize=278484 CPUtime=408.04 cores=1,3,5,7
/proc/12314/stat : 12314 (plingeling) S 12312 12314 11874 0 -1 4202496 464424 1252 0 0 40741 63 0 0 25 0 5 0 654797017 285167616 24682 33554432000 4194304 4417452 140736082868912 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/12314/statm: 69621 24682 180 55 0 67507 0
[pid=12314/tid=12321] ppid=12312 vsize=278484 CPUtime=102.27 cores=1,3,5,7
/proc/12314/task/12321/stat : 12321 (plingeling) R 12312 12314 11874 0 -1 4202560 138131 1252 0 0 10209 18 0 0 25 0 5 0 654797020 285167616 24682 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12314/tid=12322] ppid=12312 vsize=278484 CPUtime=102.27 cores=1,3,5,7
/proc/12314/task/12322/stat : 12322 (plingeling) R 12312 12314 11874 0 -1 4202560 152004 1252 0 0 10208 19 0 0 25 0 5 0 654797020 285167616 24682 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12314/tid=12323] ppid=12312 vsize=278484 CPUtime=102.27 cores=1,3,5,7
/proc/12314/task/12323/stat : 12323 (plingeling) R 12312 12314 11874 0 -1 4202560 128735 1252 0 0 10210 17 0 0 25 0 5 0 654797020 285167616 24682 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12314/tid=12324] ppid=12312 vsize=278484 CPUtime=101.15 cores=1,3,5,7
/proc/12314/task/12324/stat : 12324 (plingeling) R 12312 12314 11874 0 -1 4202560 43891 1252 0 0 10109 6 0 0 25 0 5 0 654797020 285167616 24682 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 408.04
Current children cumulated vsize (KiB) 278484

[startup+162.301 s]
/proc/loadavg: 7.68 5.96 6.46 9/358 12337
/proc/meminfo: memFree=25739792/32951124 swapFree=53501236/67111528
[pid=12314] ppid=12312 vsize=305388 CPUtime=647.33 cores=1,3,5,7
/proc/12314/stat : 12314 (plingeling) S 12312 12314 11874 0 -1 4202496 645823 1252 0 0 64645 88 0 0 25 0 5 0 654797017 312717312 31408 33554432000 4194304 4417452 140736082868912 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/12314/statm: 76347 31408 180 55 0 74233 0
[pid=12314/tid=12321] ppid=12312 vsize=305388 CPUtime=162.26 cores=1,3,5,7
/proc/12314/task/12321/stat : 12321 (plingeling) R 12312 12314 11874 0 -1 4202560 216041 1252 0 0 16197 29 0 0 25 0 5 0 654797020 312717312 31408 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12314/tid=12322] ppid=12312 vsize=305388 CPUtime=162.26 cores=1,3,5,7
/proc/12314/task/12322/stat : 12322 (plingeling) R 12312 12314 11874 0 -1 4202560 190350 1252 0 0 16201 25 0 0 25 0 5 0 654797020 312717312 31408 33554432000 4194304 4417452 140736082868912 18446744073709551615 216634207952 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12314/tid=12323] ppid=12312 vsize=305388 CPUtime=162.26 cores=1,3,5,7
/proc/12314/task/12323/stat : 12323 (plingeling) R 12312 12314 11874 0 -1 4202560 174375 1252 0 0 16203 23 0 0 25 0 5 0 654797020 312717312 31408 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12314/tid=12324] ppid=12312 vsize=305388 CPUtime=160.49 cores=1,3,5,7
/proc/12314/task/12324/stat : 12324 (plingeling) R 12312 12314 11874 0 -1 4202560 63394 1252 0 0 16040 9 0 0 25 0 5 0 654797020 312717312 31408 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364881 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 647.33
Current children cumulated vsize (KiB) 305388


################
# More data... #
################

[pid=12314/tid=12322] ppid=12312 vsize=487640 CPUtime=762.29 cores=1,3,5,7
/proc/12314/task/12322/stat : 12322 (plingeling) R 12312 12314 11874 0 -1 4202560 637206 1252 0 0 76142 87 0 0 25 0 5 0 654797020 499343360 76971 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12314/tid=12323] ppid=12312 vsize=487640 CPUtime=762.28 cores=1,3,5,7
/proc/12314/task/12323/stat : 12323 (plingeling) R 12312 12314 11874 0 -1 4202560 484746 1252 0 0 76165 63 0 0 25 0 5 0 654797020 499343360 76971 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12314/tid=12324] ppid=12312 vsize=487640 CPUtime=753.95 cores=1,3,5,7
/proc/12314/task/12324/stat : 12324 (plingeling) R 12312 12314 11874 0 -1 4202560 221820 1252 0 0 75361 34 0 0 25 0 5 0 654797020 499343360 76971 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3040.86
Current children cumulated vsize (KiB) 487640

[startup+822.301 s]
/proc/loadavg: 8.05 7.86 7.29 9/358 12358
/proc/meminfo: memFree=25280012/32951124 swapFree=53501236/67111528
[pid=12314] ppid=12312 vsize=503400 CPUtime=3280.17 cores=1,3,5,7
/proc/12314/stat : 12314 (plingeling) S 12312 12314 11874 0 -1 4202496 2169377 1252 0 0 327721 296 0 0 25 0 5 0 654797017 515481600 80911 33554432000 4194304 4417452 140736082868912 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/12314/statm: 125850 80911 180 55 0 123736 0
[pid=12314/tid=12321] ppid=12312 vsize=503400 CPUtime=822.28 cores=1,3,5,7
/proc/12314/task/12321/stat : 12321 (plingeling) R 12312 12314 11874 0 -1 4202560 662987 1252 0 0 82138 90 0 0 25 0 5 0 654797020 515481600 80911 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364741 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12314/tid=12322] ppid=12312 vsize=503400 CPUtime=822.28 cores=1,3,5,7
/proc/12314/task/12322/stat : 12322 (plingeling) R 12312 12314 11874 0 -1 4202560 768278 1252 0 0 82124 104 0 0 25 0 5 0 654797020 515481600 80911 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12314/tid=12323] ppid=12312 vsize=503400 CPUtime=822.27 cores=1,3,5,7
/proc/12314/task/12323/stat : 12323 (plingeling) R 12312 12314 11874 0 -1 4202560 501576 1252 0 0 82162 65 0 0 25 0 5 0 654797020 515481600 80911 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12314/tid=12324] ppid=12312 vsize=503400 CPUtime=813.29 cores=1,3,5,7
/proc/12314/task/12324/stat : 12324 (plingeling) R 12312 12314 11874 0 -1 4202560 234873 1252 0 0 81293 36 0 0 25 0 5 0 654797020 515481600 80911 33554432000 4194304 4417452 140736082868912 18446744073709551615 4366391 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3280.17
Current children cumulated vsize (KiB) 503400

[startup+882.3 s]
/proc/loadavg: 8.13 7.91 7.34 9/358 12359
/proc/meminfo: memFree=25360704/32951124 swapFree=53501236/67111528
[pid=12314] ppid=12312 vsize=456132 CPUtime=3519.51 cores=1,3,5,7
/proc/12314/stat : 12314 (plingeling) S 12312 12314 11874 0 -1 4202496 2293266 1252 0 0 351638 313 0 0 25 0 5 0 654797017 467079168 69094 33554432000 4194304 4417452 140736082868912 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/12314/statm: 114033 69094 180 55 0 111919 0
[pid=12314/tid=12321] ppid=12312 vsize=456132 CPUtime=882.28 cores=1,3,5,7
/proc/12314/task/12321/stat : 12321 (plingeling) R 12312 12314 11874 0 -1 4202560 716235 1252 0 0 88132 96 0 0 25 0 5 0 654797020 467079168 69094 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364537 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12314/tid=12322] ppid=12312 vsize=456132 CPUtime=882.28 cores=1,3,5,7
/proc/12314/task/12322/stat : 12322 (plingeling) R 12312 12314 11874 0 -1 4202560 814195 1252 0 0 88117 111 0 0 25 0 5 0 654797020 467079168 69094 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12314/tid=12323] ppid=12312 vsize=456132 CPUtime=882.27 cores=1,3,5,7
/proc/12314/task/12323/stat : 12323 (plingeling) R 12312 12314 11874 0 -1 4202560 515935 1252 0 0 88160 67 0 0 25 0 5 0 654797020 467079168 69094 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12314/tid=12324] ppid=12312 vsize=456132 CPUtime=872.62 cores=1,3,5,7
/proc/12314/task/12324/stat : 12324 (plingeling) R 12312 12314 11874 0 -1 4202560 245238 1252 0 0 87225 37 0 0 25 0 5 0 654797020 467079168 69094 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364555 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3519.51
Current children cumulated vsize (KiB) 456132

[startup+942.301 s]
/proc/loadavg: 8.12 7.94 7.38 9/358 12361
/proc/meminfo: memFree=25297060/32951124 swapFree=53501236/67111528
[pid=12314] ppid=12312 vsize=450400 CPUtime=3758.87 cores=1,3,5,7
/proc/12314/stat : 12314 (plingeling) S 12312 12314 11874 0 -1 4202496 2423986 1252 0 0 375557 330 0 0 25 0 5 0 654797017 461209600 67661 33554432000 4194304 4417452 140736082868912 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/12314/statm: 112600 67661 180 55 0 110486 0
[pid=12314/tid=12321] ppid=12312 vsize=450400 CPUtime=942.27 cores=1,3,5,7
/proc/12314/task/12321/stat : 12321 (plingeling) R 12312 12314 11874 0 -1 4202560 826827 1252 0 0 94117 110 0 0 25 0 5 0 654797020 461209600 67661 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12314/tid=12322] ppid=12312 vsize=450400 CPUtime=942.28 cores=1,3,5,7
/proc/12314/task/12322/stat : 12322 (plingeling) R 12312 12314 11874 0 -1 4202560 816524 1252 0 0 94117 111 0 0 25 0 5 0 654797020 461209600 67661 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12314/tid=12323] ppid=12312 vsize=450400 CPUtime=942.27 cores=1,3,5,7
/proc/12314/task/12323/stat : 12323 (plingeling) R 12312 12314 11874 0 -1 4202560 524127 1252 0 0 94159 68 0 0 25 0 5 0 654797020 461209600 67661 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12314/tid=12324] ppid=12312 vsize=450400 CPUtime=931.98 cores=1,3,5,7
/proc/12314/task/12324/stat : 12324 (plingeling) R 12312 12314 11874 0 -1 4202560 254845 1252 0 0 93159 39 0 0 25 0 5 0 654797020 461209600 67661 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3758.87
Current children cumulated vsize (KiB) 450400

[startup+1002.3 s]
/proc/loadavg: 8.23 8.00 7.43 9/358 12363
/proc/meminfo: memFree=25395588/32951124 swapFree=53501236/67111528
[pid=12314] ppid=12312 vsize=421056 CPUtime=3998.22 cores=1,3,5,7
/proc/12314/stat : 12314 (plingeling) S 12312 12314 11874 0 -1 4202496 2496318 1252 0 0 399482 340 0 0 25 0 5 0 654797017 431161344 60325 33554432000 4194304 4417452 140736082868912 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/12314/statm: 105264 60325 180 55 0 103150 0
[pid=12314/tid=12321] ppid=12312 vsize=421056 CPUtime=1002.28 cores=1,3,5,7
/proc/12314/task/12321/stat : 12321 (plingeling) R 12312 12314 11874 0 -1 4202560 856746 1252 0 0 100113 115 0 0 25 0 5 0 654797020 431161344 60325 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12314/tid=12322] ppid=12312 vsize=421056 CPUtime=1002.28 cores=1,3,5,7
/proc/12314/task/12322/stat : 12322 (plingeling) R 12312 12314 11874 0 -1 4202560 818866 1252 0 0 100117 111 0 0 25 0 5 0 654797020 431161344 60325 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12314/tid=12323] ppid=12312 vsize=421056 CPUtime=1002.28 cores=1,3,5,7
/proc/12314/task/12323/stat : 12323 (plingeling) R 12312 12314 11874 0 -1 4202560 556895 1252 0 0 100155 73 0 0 25 0 5 0 654797020 431161344 60325 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12314/tid=12324] ppid=12312 vsize=421056 CPUtime=991.32 cores=1,3,5,7
/proc/12314/task/12324/stat : 12324 (plingeling) R 12312 12314 11874 0 -1 4202560 262148 1252 0 0 99092 40 0 0 25 0 5 0 654797020 431161344 60325 33554432000 4194304 4417452 140736082868912 18446744073709551615 4365075 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3998.22
Current children cumulated vsize (KiB) 421056

[startup+1062.3 s]
/proc/loadavg: 8.12 8.01 7.47 9/358 12364
/proc/meminfo: memFree=25256068/32951124 swapFree=53501236/67111528
[pid=12314] ppid=12312 vsize=478400 CPUtime=4237.57 cores=1,3,5,7
/proc/12314/stat : 12314 (plingeling) S 12312 12314 11874 0 -1 4202496 2610261 1252 0 0 423403 354 0 0 25 0 5 0 654797017 489881600 74661 33554432000 4194304 4417452 140736082868912 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/12314/statm: 119600 74661 180 55 0 117486 0
[pid=12314/tid=12321] ppid=12312 vsize=478400 CPUtime=1062.28 cores=1,3,5,7
/proc/12314/task/12321/stat : 12321 (plingeling) R 12312 12314 11874 0 -1 4202560 864098 1252 0 0 106112 116 0 0 25 0 5 0 654797020 489881600 74661 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12314/tid=12322] ppid=12312 vsize=478400 CPUtime=1062.28 cores=1,3,5,7
/proc/12314/task/12322/stat : 12322 (plingeling) R 12312 12314 11874 0 -1 4202560 835250 1252 0 0 106115 113 0 0 25 0 5 0 654797020 489881600 74661 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12314/tid=12323] ppid=12312 vsize=478400 CPUtime=1062.27 cores=1,3,5,7
/proc/12314/task/12323/stat : 12323 (plingeling) R 12312 12314 11874 0 -1 4202560 636275 1252 0 0 106145 82 0 0 25 0 5 0 654797020 489881600 74661 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12314/tid=12324] ppid=12312 vsize=478400 CPUtime=1050.67 cores=1,3,5,7
/proc/12314/task/12324/stat : 12324 (plingeling) R 12312 12314 11874 0 -1 4202560 272975 1252 0 0 105026 41 0 0 25 0 5 0 654797020 489881600 74661 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4237.57
Current children cumulated vsize (KiB) 478400

[startup+1122.3 s]
/proc/loadavg: 8.18 8.05 7.52 9/358 12366
/proc/meminfo: memFree=25168100/32951124 swapFree=53501236/67111528
[pid=12314] ppid=12312 vsize=538828 CPUtime=4476.91 cores=1,3,5,7
/proc/12314/stat : 12314 (plingeling) S 12312 12314 11874 0 -1 4202496 2749085 1252 0 0 447318 373 0 0 25 0 5 0 654797017 551759872 89800 33554432000 4194304 4417452 140736082868912 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/12314/statm: 134707 89800 180 55 0 132593 0
[pid=12314/tid=12321] ppid=12312 vsize=538828 CPUtime=1122.28 cores=1,3,5,7
/proc/12314/task/12321/stat : 12321 (plingeling) R 12312 12314 11874 0 -1 4202560 868194 1252 0 0 112111 117 0 0 25 0 5 0 654797020 551759872 89800 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364653 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12314/tid=12322] ppid=12312 vsize=538828 CPUtime=1122.29 cores=1,3,5,7
/proc/12314/task/12322/stat : 12322 (plingeling) R 12312 12314 11874 0 -1 4202560 858284 1252 0 0 112113 116 0 0 25 0 5 0 654797020 551759872 89800 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364597 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12314/tid=12323] ppid=12312 vsize=538828 CPUtime=1122.28 cores=1,3,5,7
/proc/12314/task/12323/stat : 12323 (plingeling) R 12312 12314 11874 0 -1 4202560 729758 1252 0 0 112134 94 0 0 25 0 5 0 654797020 551759872 89800 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12314/tid=12324] ppid=12312 vsize=538828 CPUtime=1110.01 cores=1,3,5,7
/proc/12314/task/12324/stat : 12324 (plingeling) R 12312 12314 11874 0 -1 4202560 291186 1252 0 0 110957 44 0 0 25 0 5 0 654797020 551759872 89800 33554432000 4194304 4417452 140736082868912 18446744073709551615 4366391 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4476.91
Current children cumulated vsize (KiB) 538828

[startup+1182.3 s]
/proc/loadavg: 8.07 8.04 7.55 9/358 12367
/proc/meminfo: memFree=25148964/32951124 swapFree=53501236/67111528
[pid=12314] ppid=12312 vsize=554992 CPUtime=4716.27 cores=1,3,5,7
/proc/12314/stat : 12314 (plingeling) S 12312 12314 11874 0 -1 4202496 2902338 1252 0 0 471235 392 0 0 25 0 5 0 654797017 568311808 93841 33554432000 4194304 4417452 140736082868912 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/12314/statm: 138748 93841 180 55 0 136634 0
[pid=12314/tid=12321] ppid=12312 vsize=554992 CPUtime=1182.29 cores=1,3,5,7
/proc/12314/task/12321/stat : 12321 (plingeling) R 12312 12314 11874 0 -1 4202560 884578 1252 0 0 118109 120 0 0 25 0 5 0 654797020 568311808 93841 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364572 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12314/tid=12322] ppid=12312 vsize=554992 CPUtime=1182.29 cores=1,3,5,7
/proc/12314/task/12322/stat : 12322 (plingeling) R 12312 12314 11874 0 -1 4202560 895326 1252 0 0 118108 121 0 0 25 0 5 0 654797020 568311808 93841 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12314/tid=12323] ppid=12312 vsize=554992 CPUtime=1182.28 cores=1,3,5,7
/proc/12314/task/12323/stat : 12323 (plingeling) R 12312 12314 11874 0 -1 4202560 823966 1252 0 0 118123 105 0 0 25 0 5 0 654797020 568311808 93841 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12314/tid=12324] ppid=12312 vsize=554992 CPUtime=1169.36 cores=1,3,5,7
/proc/12314/task/12324/stat : 12324 (plingeling) R 12312 12314 11874 0 -1 4202560 296805 1252 0 0 116891 45 0 0 25 0 5 0 654797020 568311808 93841 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4716.27
Current children cumulated vsize (KiB) 554992



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.3 s]
/proc/loadavg: 8.12 8.05 7.56 9/358 12368
/proc/meminfo: memFree=25080144/32951124 swapFree=53501236/67111528
[pid=12314] ppid=12312 vsize=554992 CPUtime=4800.11 cores=1,3,5,7
/proc/12314/stat : 12314 (plingeling) S 12312 12314 11874 0 -1 4202496 2959682 1252 0 0 479613 398 0 0 25 0 5 0 654797017 568311808 93841 33554432000 4194304 4417452 140736082868912 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/12314/statm: 138748 93841 180 55 0 136634 0
[pid=12314/tid=12321] ppid=12312 vsize=554992 CPUtime=1203.32 cores=1,3,5,7
/proc/12314/task/12321/stat : 12321 (plingeling) R 12312 12314 11874 0 -1 4202560 884578 1252 0 0 120212 120 0 0 25 0 5 0 654797020 568311808 93841 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12314/tid=12322] ppid=12312 vsize=554992 CPUtime=1203.31 cores=1,3,5,7
/proc/12314/task/12322/stat : 12322 (plingeling) R 12312 12314 11874 0 -1 4202560 919902 1252 0 0 120208 123 0 0 25 0 5 0 654797020 568311808 93841 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364684 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12314/tid=12323] ppid=12312 vsize=554992 CPUtime=1203.31 cores=1,3,5,7
/proc/12314/task/12323/stat : 12323 (plingeling) R 12312 12314 11874 0 -1 4202560 856734 1252 0 0 120222 109 0 0 25 0 5 0 654797020 568311808 93841 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12314/tid=12324] ppid=12312 vsize=554992 CPUtime=1190.13 cores=1,3,5,7
/proc/12314/task/12324/stat : 12324 (plingeling) R 12312 12314 11874 0 -1 4202560 296805 1252 0 0 118968 45 0 0 25 0 5 0 654797020 568311808 93841 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4800.11
Current children cumulated vsize (KiB) 554992

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.3 s]
/proc/loadavg: 8.12 8.05 7.56 9/358 12368
/proc/meminfo: memFree=25080144/32951124 swapFree=53501236/67111528
[pid=12314] ppid=12312 vsize=554992 CPUtime=4800.11 cores=1,3,5,7
/proc/12314/stat : 12314 (plingeling) S 12312 12314 11874 0 -1 4202496 2959682 1252 0 0 479613 398 0 0 25 0 5 0 654797017 568311808 93841 33554432000 4194304 4417452 140736082868912 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/12314/statm: 138748 93841 180 55 0 136634 0
[pid=12314/tid=12321] ppid=12312 vsize=554992 CPUtime=1203.32 cores=1,3,5,7
/proc/12314/task/12321/stat : 12321 (plingeling) R 12312 12314 11874 0 -1 4202560 884578 1252 0 0 120212 120 0 0 25 0 5 0 654797020 568311808 93841 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12314/tid=12322] ppid=12312 vsize=554992 CPUtime=1203.31 cores=1,3,5,7
/proc/12314/task/12322/stat : 12322 (plingeling) R 12312 12314 11874 0 -1 4202560 919902 1252 0 0 120208 123 0 0 25 0 5 0 654797020 568311808 93841 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364684 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12314/tid=12323] ppid=12312 vsize=554992 CPUtime=1203.31 cores=1,3,5,7
/proc/12314/task/12323/stat : 12323 (plingeling) R 12312 12314 11874 0 -1 4202560 856734 1252 0 0 120222 109 0 0 25 0 5 0 654797020 568311808 93841 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12314/tid=12324] ppid=12312 vsize=554992 CPUtime=1190.13 cores=1,3,5,7
/proc/12314/task/12324/stat : 12324 (plingeling) R 12312 12314 11874 0 -1 4202560 296805 1252 0 0 118968 45 0 0 25 0 5 0 654797020 568311808 93841 33554432000 4194304 4417452 140736082868912 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4800.11
Current children cumulated vsize (KiB) 554992

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1203.35
CPU time (s): 4800.17
CPU user time (s): 4796.16
CPU system time (s): 4.00439
CPU usage (%): 398.901
Max. virtual memory (cumulated for all children) (KiB): 587488

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4796.16
system time used= 4.00439
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2960934
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 148
involuntary context switches= 36394

runsolver used 3.66944 second user time and 9.48856 second system time

The end

Launcher Data 

Begin job on node103 at 2011-05-15 05:59:04
IDJOB=3371109
IDBENCH=85826
IDSOLVER=1848
FILE ID=node103/3371109-1305431944
RUNJOBID= node103-1305431944-12249
PBS_JOBID= 13324609
Free space on /tmp= 73048 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT05/crafted/sorge05/QG7/gensys-icl007.shuffled-as.sat05-3133.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3371109-1305431944/watcher-3371109-1305431944 -o /tmp/evaluation-result-3371109-1305431944/solver-3371109-1305431944 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3371109-1305431944.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 88361752c7b0c05e849791aafdd7bdba
RANDOM SEED=199499721

node103.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.71
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      26158556 kB
Buffers:        308988 kB
Cached:        3937884 kB
SwapCached:     340192 kB
Active:        1346720 kB
Inactive:      5176176 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      26158556 kB
SwapTotal:    67111528 kB
SwapFree:     53501236 kB
Dirty:            1320 kB
Writeback:           0 kB
AnonPages:     1947048 kB
Mapped:          10588 kB
Slab:           152052 kB
PageTables:      52552 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 78359736 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73048 MiB
End job on node103 at 2011-05-15 06:19:09