Trace number 3265642

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 4800.35 1201.94

General information on the benchmark

NameSAT05/crafted/sorge05/QG7a/
gensys-ukn009.shuffled-as.sat05-3849.cnf
MD5SUMf2a1eb7937a7cda114ce344a30ca1145
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark824.93
Satisfiable
(Un)Satisfiability was proved
Number of variables2969
Number of clauses18718
Sum of the clauses size134151
Maximum clause length13
Minimum clause length1
Number of clauses of size 142
Number of clauses of size 2435
Number of clauses of size 32341
Number of clauses of size 4498
Number of clauses of size 5677
Number of clauses of size over 514725

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3265642-1304106819/watcher-3265642-1304106819 -o /tmp/evaluation-result-3265642-1304106819/solver-3265642-1304106819 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3265642-1304106819.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.01 7.97 7.98 6/181 20875
/proc/meminfo: memFree=30259328/32951124 swapFree=67111416/67111528
[pid=20875] ppid=20873 vsize=9672 CPUtime=0 cores=1,3,5,7
/proc/20875/stat : 20875 (plingeling) R 20873 20875 19850 0 -1 4202496 487 1253 0 0 0 0 0 0 25 0 1 0 522284294 9904128 416 33554432000 4194304 4417332 140735299422096 18446744073709551615 212435710932 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/20875/statm: 2418 416 147 55 0 304 0

[startup+0.0468359 s]
/proc/loadavg: 8.01 7.97 7.98 6/181 20875
/proc/meminfo: memFree=30259328/32951124 swapFree=67111416/67111528
[pid=20875] ppid=20873 vsize=57032 CPUtime=0.03 cores=1,3,5,7
/proc/20875/stat : 20875 (plingeling) S 20873 20875 19850 0 -1 4202496 2317 1253 0 0 3 0 0 0 25 0 5 0 522284294 58400768 2051 33554432000 4194304 4417332 140735299422096 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/20875/statm: 14258 2051 157 55 0 12144 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 57032

[startup+0.15078 s]
/proc/loadavg: 8.01 7.97 7.98 6/181 20875
/proc/meminfo: memFree=30259328/32951124 swapFree=67111416/67111528
[pid=20875] ppid=20873 vsize=124148 CPUtime=0.44 cores=1,3,5,7
/proc/20875/stat : 20875 (plingeling) S 20873 20875 19850 0 -1 4202496 3369 1253 0 0 44 0 0 0 25 0 5 0 522284294 127127552 2490 33554432000 4194304 4417332 140735299422096 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/20875/statm: 31037 2490 180 55 0 28923 0
Current children cumulated CPU time (s) 0.44
Current children cumulated vsize (KiB) 124148

[startup+0.300752 s]
/proc/loadavg: 8.01 7.97 7.98 6/181 20875
/proc/meminfo: memFree=30259328/32951124 swapFree=67111416/67111528
[pid=20875] ppid=20873 vsize=126056 CPUtime=1.04 cores=1,3,5,7
/proc/20875/stat : 20875 (plingeling) S 20873 20875 19850 0 -1 4202496 4015 1253 0 0 104 0 0 0 25 0 5 0 522284294 129081344 2970 33554432000 4194304 4417332 140735299422096 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/20875/statm: 31514 2970 181 55 0 29400 0
Current children cumulated CPU time (s) 1.04
Current children cumulated vsize (KiB) 126056

[startup+0.700676 s]
/proc/loadavg: 8.01 7.97 7.98 6/181 20875
/proc/meminfo: memFree=30259328/32951124 swapFree=67111416/67111528
[pid=20875] ppid=20873 vsize=128980 CPUtime=2.64 cores=1,3,5,7
/proc/20875/stat : 20875 (plingeling) S 20873 20875 19850 0 -1 4202496 6008 1253 0 0 264 0 0 0 25 0 5 0 522284294 132075520 3710 33554432000 4194304 4417332 140735299422096 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/20875/statm: 32245 3710 181 55 0 30131 0
Current children cumulated CPU time (s) 2.64
Current children cumulated vsize (KiB) 128980

[startup+1.50053 s]
/proc/loadavg: 8.01 7.97 7.98 9/186 20885
/proc/meminfo: memFree=30239828/32951124 swapFree=67111416/67111528
[pid=20875] ppid=20873 vsize=135920 CPUtime=5.83 cores=1,3,5,7
/proc/20875/stat : 20875 (plingeling) S 20873 20875 19850 0 -1 4202496 12209 1253 0 0 582 1 0 0 25 0 5 0 522284294 139182080 5416 33554432000 4194304 4417332 140735299422096 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/20875/statm: 33980 5416 181 55 0 31866 0
[pid=20875/tid=20882] ppid=20873 vsize=135920 CPUtime=1.45 cores=1,3,5,7
/proc/20875/task/20882/stat : 20882 (plingeling) R 20873 20875 19850 0 -1 4202560 1809 1253 0 0 145 0 0 0 20 0 5 0 522284299 139182080 5416 33554432000 4194304 4417332 140735299422096 18446744073709551615 4356687 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20875/tid=20883] ppid=20873 vsize=135920 CPUtime=1.44 cores=1,3,5,7
/proc/20875/task/20883/stat : 20883 (plingeling) R 20873 20875 19850 0 -1 4202560 2356 1253 0 0 144 0 0 0 25 0 5 0 522284299 139182080 5416 33554432000 4194304 4417332 140735299422096 18446744073709551615 4365960 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20875/tid=20884] ppid=20873 vsize=135920 CPUtime=1.44 cores=1,3,5,7
/proc/20875/task/20884/stat : 20884 (plingeling) R 20873 20875 19850 0 -1 4202560 2275 1253 0 0 144 0 0 0 25 0 5 0 522284299 139182080 5416 33554432000 4194304 4417332 140735299422096 18446744073709551615 4236816 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20875/tid=20885] ppid=20873 vsize=135920 CPUtime=1.44 cores=1,3,5,7
/proc/20875/task/20885/stat : 20885 (plingeling) R 20873 20875 19850 0 -1 4202560 3457 1253 0 0 144 0 0 0 25 0 5 0 522284299 139182080 5416 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364919 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 5.83
Current children cumulated vsize (KiB) 135920

[startup+3.10122 s]
/proc/loadavg: 8.01 7.97 7.98 9/186 20885
/proc/meminfo: memFree=30223012/32951124 swapFree=67111416/67111528
[pid=20875] ppid=20873 vsize=143276 CPUtime=12.24 cores=1,3,5,7
/proc/20875/stat : 20875 (plingeling) S 20873 20875 19850 0 -1 4202496 22621 1253 0 0 1220 4 0 0 25 0 5 0 522284294 146714624 7288 33554432000 4194304 4417332 140735299422096 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/20875/statm: 35819 7288 181 55 0 33705 0
[pid=20875/tid=20882] ppid=20873 vsize=143276 CPUtime=3.04 cores=1,3,5,7
/proc/20875/task/20882/stat : 20882 (plingeling) R 20873 20875 19850 0 -1 4202560 4607 1253 0 0 304 0 0 0 25 0 5 0 522284299 146714624 7288 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364547 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20875/tid=20883] ppid=20873 vsize=143276 CPUtime=3.04 cores=1,3,5,7
/proc/20875/task/20883/stat : 20883 (plingeling) R 20873 20875 19850 0 -1 4202560 4785 1253 0 0 304 0 0 0 25 0 5 0 522284299 146714624 7288 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20875/tid=20884] ppid=20873 vsize=143276 CPUtime=3.04 cores=1,3,5,7
/proc/20875/task/20884/stat : 20884 (plingeling) R 20873 20875 19850 0 -1 4202560 5082 1253 0 0 304 0 0 0 25 0 5 0 522284299 146714624 7288 33554432000 4194304 4417332 140735299422096 18446744073709551615 4258359 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20875/tid=20885] ppid=20873 vsize=143276 CPUtime=3.03 cores=1,3,5,7
/proc/20875/task/20885/stat : 20885 (plingeling) R 20873 20875 19850 0 -1 4202560 5835 1253 0 0 302 1 0 0 25 0 5 0 522284299 146714624 7288 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 12.24
Current children cumulated vsize (KiB) 143276

[startup+6.30061 s]
/proc/loadavg: 8.01 7.97 7.98 9/186 20885
/proc/meminfo: memFree=30169068/32951124 swapFree=67111416/67111528
[pid=20875] ppid=20873 vsize=147380 CPUtime=25.01 cores=1,3,5,7
/proc/20875/stat : 20875 (plingeling) S 20873 20875 19850 0 -1 4202496 36546 1253 0 0 2494 7 0 0 25 0 5 0 522284294 150917120 8300 33554432000 4194304 4417332 140735299422096 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/20875/statm: 36845 8300 181 55 0 34731 0
[pid=20875/tid=20882] ppid=20873 vsize=147380 CPUtime=6.24 cores=1,3,5,7
/proc/20875/task/20882/stat : 20882 (plingeling) R 20873 20875 19850 0 -1 4202560 7759 1253 0 0 623 1 0 0 25 0 5 0 522284299 150917120 8300 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364750 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20875/tid=20883] ppid=20873 vsize=147380 CPUtime=6.24 cores=1,3,5,7
/proc/20875/task/20883/stat : 20883 (plingeling) R 20873 20875 19850 0 -1 4202560 7671 1253 0 0 623 1 0 0 25 0 5 0 522284299 150917120 8300 33554432000 4194304 4417332 140735299422096 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20875/tid=20884] ppid=20873 vsize=147380 CPUtime=6.24 cores=1,3,5,7
/proc/20875/task/20884/stat : 20884 (plingeling) R 20873 20875 19850 0 -1 4202560 9174 1253 0 0 623 1 0 0 25 0 5 0 522284299 150917120 8300 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364732 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20875/tid=20885] ppid=20873 vsize=147380 CPUtime=6.22 cores=1,3,5,7
/proc/20875/task/20885/stat : 20885 (plingeling) R 20873 20875 19850 0 -1 4202560 9630 1253 0 0 620 2 0 0 25 0 5 0 522284299 150917120 8300 33554432000 4194304 4417332 140735299422096 18446744073709551615 4365994 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 25.01
Current children cumulated vsize (KiB) 147380

[startup+12.7004 s]
/proc/loadavg: 8.00 7.97 7.98 9/186 20885
/proc/meminfo: memFree=30158784/32951124 swapFree=67111416/67111528
[pid=20875] ppid=20873 vsize=165796 CPUtime=50.57 cores=1,3,5,7
/proc/20875/stat : 20875 (plingeling) S 20873 20875 19850 0 -1 4202496 61868 1253 0 0 5045 12 0 0 25 0 5 0 522284294 169775104 12936 33554432000 4194304 4417332 140735299422096 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/20875/statm: 41449 12936 181 55 0 39335 0
[pid=20875/tid=20882] ppid=20873 vsize=165796 CPUtime=12.65 cores=1,3,5,7
/proc/20875/task/20882/stat : 20882 (plingeling) R 20873 20875 19850 0 -1 4202560 14374 1253 0 0 1262 3 0 0 25 0 5 0 522284299 169775104 12936 33554432000 4194304 4417332 140735299422096 18446744073709551615 4236986 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20875/tid=20883] ppid=20873 vsize=165796 CPUtime=12.64 cores=1,3,5,7
/proc/20875/task/20883/stat : 20883 (plingeling) R 20873 20875 19850 0 -1 4202560 14463 1253 0 0 1261 3 0 0 25 0 5 0 522284299 169775104 12936 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20875/tid=20884] ppid=20873 vsize=165796 CPUtime=12.64 cores=1,3,5,7
/proc/20875/task/20884/stat : 20884 (plingeling) R 20873 20875 19850 0 -1 4202560 15193 1253 0 0 1262 2 0 0 25 0 5 0 522284299 169775104 12936 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20875/tid=20885] ppid=20873 vsize=165796 CPUtime=12.58 cores=1,3,5,7
/proc/20875/task/20885/stat : 20885 (plingeling) R 20873 20875 19850 0 -1 4202560 15526 1253 0 0 1255 3 0 0 25 0 5 0 522284299 169775104 12936 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.57
Current children cumulated vsize (KiB) 165796

[startup+25.501 s]
/proc/loadavg: 8.00 7.97 7.98 9/186 20885
/proc/meminfo: memFree=30187408/32951124 swapFree=67111416/67111528
[pid=20875] ppid=20873 vsize=203252 CPUtime=101.71 cores=1,3,5,7
/proc/20875/stat : 20875 (plingeling) S 20873 20875 19850 0 -1 4202496 98537 1253 0 0 10152 19 0 0 25 0 5 0 522284294 208130048 22268 33554432000 4194304 4417332 140735299422096 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/20875/statm: 50813 22268 181 55 0 48699 0
[pid=20875/tid=20882] ppid=20873 vsize=203252 CPUtime=25.45 cores=1,3,5,7
/proc/20875/task/20882/stat : 20882 (plingeling) R 20873 20875 19850 0 -1 4202560 22950 1253 0 0 2541 4 0 0 25 0 5 0 522284299 208130048 22268 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20875/tid=20883] ppid=20873 vsize=203252 CPUtime=25.44 cores=1,3,5,7
/proc/20875/task/20883/stat : 20883 (plingeling) R 20873 20875 19850 0 -1 4202560 23195 1253 0 0 2540 4 0 0 25 0 5 0 522284299 208130048 22268 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20875/tid=20884] ppid=20873 vsize=203252 CPUtime=25.44 cores=1,3,5,7
/proc/20875/task/20884/stat : 20884 (plingeling) R 20873 20875 19850 0 -1 4202560 24139 1253 0 0 2540 4 0 0 25 0 5 0 522284299 208130048 22268 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20875/tid=20885] ppid=20873 vsize=203252 CPUtime=25.31 cores=1,3,5,7
/proc/20875/task/20885/stat : 20885 (plingeling) R 20873 20875 19850 0 -1 4202560 25941 1253 0 0 2526 5 0 0 25 0 5 0 522284299 208130048 22268 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364990 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.71
Current children cumulated vsize (KiB) 203252

[startup+51.1011 s]
/proc/loadavg: 8.08 7.99 7.98 9/186 20886
/proc/meminfo: memFree=30121520/32951124 swapFree=67111416/67111528
[pid=20875] ppid=20873 vsize=250368 CPUtime=203.97 cores=1,3,5,7
/proc/20875/stat : 20875 (plingeling) S 20873 20875 19850 0 -1 4202496 141080 1253 0 0 20371 26 0 0 25 0 5 0 522284294 256376832 34047 33554432000 4194304 4417332 140735299422096 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/20875/statm: 62592 34047 181 55 0 60478 0
[pid=20875/tid=20882] ppid=20873 vsize=250368 CPUtime=51.05 cores=1,3,5,7
/proc/20875/task/20882/stat : 20882 (plingeling) R 20873 20875 19850 0 -1 4202560 34030 1253 0 0 5099 6 0 0 25 0 5 0 522284299 256376832 34047 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20875/tid=20883] ppid=20873 vsize=250368 CPUtime=51.04 cores=1,3,5,7
/proc/20875/task/20883/stat : 20883 (plingeling) R 20873 20875 19850 0 -1 4202560 34657 1253 0 0 5098 6 0 0 25 0 5 0 522284299 256376832 34047 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20875/tid=20884] ppid=20873 vsize=250368 CPUtime=51.04 cores=1,3,5,7
/proc/20875/task/20884/stat : 20884 (plingeling) R 20873 20875 19850 0 -1 4202560 34359 1253 0 0 5098 6 0 0 25 0 5 0 522284299 256376832 34047 33554432000 4194304 4417332 140735299422096 18446744073709551615 4365986 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20875/tid=20885] ppid=20873 vsize=250368 CPUtime=50.77 cores=1,3,5,7
/proc/20875/task/20885/stat : 20885 (plingeling) R 20873 20875 19850 0 -1 4202560 35722 1253 0 0 5071 6 0 0 25 0 5 0 522284299 256376832 34047 33554432000 4194304 4417332 140735299422096 18446744073709551615 4366303 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 203.97
Current children cumulated vsize (KiB) 250368

[startup+102.306 s]
/proc/loadavg: 8.10 8.00 7.99 9/186 20887
/proc/meminfo: memFree=30019312/32951124 swapFree=67111416/67111528
[pid=20875] ppid=20873 vsize=296972 CPUtime=408.52 cores=1,3,5,7
/proc/20875/stat : 20875 (plingeling) S 20873 20875 19850 0 -1 4202496 201209 1253 0 0 40815 37 0 0 25 0 5 0 522284294 304099328 45698 33554432000 4194304 4417332 140735299422096 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/20875/statm: 74243 45698 181 55 0 72129 0
[pid=20875/tid=20882] ppid=20873 vsize=296972 CPUtime=102.25 cores=1,3,5,7
/proc/20875/task/20882/stat : 20882 (plingeling) R 20873 20875 19850 0 -1 4202560 48615 1253 0 0 10217 8 0 0 25 0 5 0 522284299 304099328 45698 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20875/tid=20883] ppid=20873 vsize=296972 CPUtime=102.24 cores=1,3,5,7
/proc/20875/task/20883/stat : 20883 (plingeling) R 20873 20875 19850 0 -1 4202560 52107 1253 0 0 10216 8 0 0 25 0 5 0 522284299 304099328 45698 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20875/tid=20884] ppid=20873 vsize=296972 CPUtime=102.25 cores=1,3,5,7
/proc/20875/task/20884/stat : 20884 (plingeling) R 20873 20875 19850 0 -1 4202560 53238 1253 0 0 10215 10 0 0 25 0 5 0 522284299 304099328 45698 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20875/tid=20885] ppid=20873 vsize=296972 CPUtime=101.7 cores=1,3,5,7
/proc/20875/task/20885/stat : 20885 (plingeling) R 20873 20875 19850 0 -1 4202560 44937 1253 0 0 10162 8 0 0 25 0 5 0 522284299 304099328 45698 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364656 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 408.52
Current children cumulated vsize (KiB) 296972

[startup+162.301 s]
/proc/loadavg: 8.16 8.03 8.00 9/186 20889
/proc/meminfo: memFree=29733332/32951124 swapFree=67111416/67111528
[pid=20875] ppid=20873 vsize=379136 CPUtime=648.18 cores=1,3,5,7
/proc/20875/stat : 20875 (plingeling) S 20873 20875 19850 0 -1 4202496 252323 1253 0 0 64772 46 0 0 25 0 5 0 522284294 388235264 66239 33554432000 4194304 4417332 140735299422096 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/20875/statm: 94784 66239 181 55 0 92670 0
[pid=20875/tid=20882] ppid=20873 vsize=379136 CPUtime=162.25 cores=1,3,5,7
/proc/20875/task/20882/stat : 20882 (plingeling) R 20873 20875 19850 0 -1 4202560 57596 1253 0 0 16215 10 0 0 25 0 5 0 522284299 388235264 66239 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364547 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20875/tid=20883] ppid=20873 vsize=379136 CPUtime=162.24 cores=1,3,5,7
/proc/20875/task/20883/stat : 20883 (plingeling) R 20873 20875 19850 0 -1 4202560 72197 1253 0 0 16212 12 0 0 25 0 5 0 522284299 388235264 66239 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364463 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20875/tid=20884] ppid=20873 vsize=379136 CPUtime=162.25 cores=1,3,5,7
/proc/20875/task/20884/stat : 20884 (plingeling) R 20873 20875 19850 0 -1 4202560 65179 1253 0 0 16212 13 0 0 25 0 5 0 522284299 388235264 66239 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20875/tid=20885] ppid=20873 vsize=379136 CPUtime=161.36 cores=1,3,5,7
/proc/20875/task/20885/stat : 20885 (plingeling) R 20873 20875 19850 0 -1 4202560 55039 1253 0 0 16127 9 0 0 25 0 5 0 522284299 388235264 66239 33554432000 4194304 4417332 140735299422096 18446744073709551615 4232508 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 648.18
Current children cumulated vsize (KiB) 379136


################
# More data... #
################

[pid=20875/tid=20883] ppid=20873 vsize=705940 CPUtime=762.25 cores=1,3,5,7
/proc/20875/task/20883/stat : 20883 (plingeling) R 20873 20875 19850 0 -1 4202560 161258 1253 0 0 76198 27 0 0 25 0 5 0 522284299 722882560 147940 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364736 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20875/tid=20884] ppid=20873 vsize=705940 CPUtime=761.44 cores=1,3,5,7
/proc/20875/task/20884/stat : 20884 (plingeling) R 20873 20875 19850 0 -1 4202560 174758 1253 0 0 76114 30 0 0 25 0 5 0 522284299 722882560 147940 33554432000 4194304 4417332 140735299422096 18446744073709551615 4250941 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20875/tid=20885] ppid=20873 vsize=705940 CPUtime=758.12 cores=1,3,5,7
/proc/20875/task/20885/stat : 20885 (plingeling) R 20873 20875 19850 0 -1 4202560 116178 1253 0 0 75791 21 0 0 25 0 5 0 522284299 722882560 147940 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3044.15
Current children cumulated vsize (KiB) 705940

[startup+822.3 s]
/proc/loadavg: 8.23 8.12 8.03 9/186 20919
/proc/meminfo: memFree=28886888/32951124 swapFree=67111416/67111528
[pid=20875] ppid=20873 vsize=739092 CPUtime=3283.81 cores=1,3,5,7
/proc/20875/stat : 20875 (plingeling) S 20873 20875 19850 0 -1 4202496 615883 1253 0 0 328276 105 0 0 25 0 5 0 522284294 756830208 156228 33554432000 4194304 4417332 140735299422096 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/20875/statm: 184773 156228 181 55 0 182659 0
[pid=20875/tid=20882] ppid=20873 vsize=739092 CPUtime=822.27 cores=1,3,5,7
/proc/20875/task/20882/stat : 20882 (plingeling) R 20873 20875 19850 0 -1 4202560 161281 1253 0 0 82201 26 0 0 25 0 5 0 522284299 756830208 156228 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20875/tid=20883] ppid=20873 vsize=739092 CPUtime=822.25 cores=1,3,5,7
/proc/20875/task/20883/stat : 20883 (plingeling) R 20873 20875 19850 0 -1 4202560 161258 1253 0 0 82198 27 0 0 25 0 5 0 522284299 756830208 156228 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20875/tid=20884] ppid=20873 vsize=739092 CPUtime=821.44 cores=1,3,5,7
/proc/20875/task/20884/stat : 20884 (plingeling) R 20873 20875 19850 0 -1 4202560 174758 1253 0 0 82114 30 0 0 25 0 5 0 522284299 756830208 156228 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20875/tid=20885] ppid=20873 vsize=739092 CPUtime=817.79 cores=1,3,5,7
/proc/20875/task/20885/stat : 20885 (plingeling) R 20873 20875 19850 0 -1 4202560 116274 1253 0 0 81758 21 0 0 25 0 5 0 522284299 756830208 156228 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3283.81
Current children cumulated vsize (KiB) 739092

[startup+882.301 s]
/proc/loadavg: 8.22 8.13 8.03 9/186 20920
/proc/meminfo: memFree=29059080/32951124 swapFree=67111416/67111528
[pid=20875] ppid=20873 vsize=667324 CPUtime=3523.5 cores=1,3,5,7
/proc/20875/stat : 20875 (plingeling) S 20873 20875 19850 0 -1 4202496 644538 1253 0 0 352240 110 0 0 25 0 5 0 522284294 683339776 138286 33554432000 4194304 4417332 140735299422096 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/20875/statm: 166831 138286 181 55 0 164717 0
[pid=20875/tid=20882] ppid=20873 vsize=667324 CPUtime=882.27 cores=1,3,5,7
/proc/20875/task/20882/stat : 20882 (plingeling) R 20873 20875 19850 0 -1 4202560 161281 1253 0 0 88201 26 0 0 25 0 5 0 522284299 683339776 138286 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20875/tid=20883] ppid=20873 vsize=667324 CPUtime=882.24 cores=1,3,5,7
/proc/20875/task/20883/stat : 20883 (plingeling) R 20873 20875 19850 0 -1 4202560 185513 1253 0 0 88194 30 0 0 25 0 5 0 522284299 683339776 138286 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20875/tid=20884] ppid=20873 vsize=667324 CPUtime=881.44 cores=1,3,5,7
/proc/20875/task/20884/stat : 20884 (plingeling) R 20873 20875 19850 0 -1 4202560 174758 1253 0 0 88114 30 0 0 25 0 5 0 522284299 683339776 138286 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364656 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20875/tid=20885] ppid=20873 vsize=667324 CPUtime=877.46 cores=1,3,5,7
/proc/20875/task/20885/stat : 20885 (plingeling) R 20873 20875 19850 0 -1 4202560 120674 1253 0 0 87725 21 0 0 25 0 5 0 522284299 683339776 138286 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3523.5
Current children cumulated vsize (KiB) 667324

[startup+942.301 s]
/proc/loadavg: 8.14 8.12 8.03 9/186 20922
/proc/meminfo: memFree=28915464/32951124 swapFree=67111416/67111528
[pid=20875] ppid=20873 vsize=710288 CPUtime=3763.18 cores=1,3,5,7
/proc/20875/stat : 20875 (plingeling) S 20873 20875 19850 0 -1 4202496 669468 1253 0 0 376203 115 0 0 25 0 5 0 522284294 727334912 149027 33554432000 4194304 4417332 140735299422096 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/20875/statm: 177572 149027 181 55 0 175458 0
[pid=20875/tid=20882] ppid=20873 vsize=710288 CPUtime=942.28 cores=1,3,5,7
/proc/20875/task/20882/stat : 20882 (plingeling) R 20873 20875 19850 0 -1 4202560 166742 1253 0 0 94201 27 0 0 25 0 5 0 522284299 727334912 149027 33554432000 4194304 4417332 140735299422096 18446744073709551615 4366716 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20875/tid=20883] ppid=20873 vsize=710288 CPUtime=942.25 cores=1,3,5,7
/proc/20875/task/20883/stat : 20883 (plingeling) R 20873 20875 19850 0 -1 4202560 187442 1253 0 0 94194 31 0 0 25 0 5 0 522284299 727334912 149027 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20875/tid=20884] ppid=20873 vsize=710288 CPUtime=941.44 cores=1,3,5,7
/proc/20875/task/20884/stat : 20884 (plingeling) R 20873 20875 19850 0 -1 4202560 189488 1253 0 0 94111 33 0 0 25 0 5 0 522284299 727334912 149027 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20875/tid=20885] ppid=20873 vsize=710288 CPUtime=937.13 cores=1,3,5,7
/proc/20875/task/20885/stat : 20885 (plingeling) R 20873 20875 19850 0 -1 4202560 123484 1253 0 0 93692 21 0 0 25 0 5 0 522284299 727334912 149027 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3763.18
Current children cumulated vsize (KiB) 710288

[startup+1002.3 s]
/proc/loadavg: 8.12 8.11 8.03 9/186 20923
/proc/meminfo: memFree=29077612/32951124 swapFree=67111416/67111528
[pid=20875] ppid=20873 vsize=664340 CPUtime=4002.85 cores=1,3,5,7
/proc/20875/stat : 20875 (plingeling) S 20873 20875 19850 0 -1 4202496 683247 1253 0 0 400168 117 0 0 25 0 5 0 522284294 680284160 137540 33554432000 4194304 4417332 140735299422096 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/20875/statm: 166085 137540 181 55 0 163971 0
[pid=20875/tid=20882] ppid=20873 vsize=664340 CPUtime=1002.28 cores=1,3,5,7
/proc/20875/task/20882/stat : 20882 (plingeling) R 20873 20875 19850 0 -1 4202560 170028 1253 0 0 100200 28 0 0 25 0 5 0 522284299 680284160 137540 33554432000 4194304 4417332 140735299422096 18446744073709551615 4308818 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20875/tid=20883] ppid=20873 vsize=664340 CPUtime=1002.25 cores=1,3,5,7
/proc/20875/task/20883/stat : 20883 (plingeling) R 20873 20875 19850 0 -1 4202560 190078 1253 0 0 100194 31 0 0 25 0 5 0 522284299 680284160 137540 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20875/tid=20884] ppid=20873 vsize=664340 CPUtime=1001.44 cores=1,3,5,7
/proc/20875/task/20884/stat : 20884 (plingeling) R 20873 20875 19850 0 -1 4202560 197345 1253 0 0 100110 34 0 0 25 0 5 0 522284299 680284160 137540 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20875/tid=20885] ppid=20873 vsize=664340 CPUtime=996.81 cores=1,3,5,7
/proc/20875/task/20885/stat : 20885 (plingeling) R 20873 20875 19850 0 -1 4202560 123484 1253 0 0 99660 21 0 0 25 0 5 0 522284299 680284160 137540 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4002.85
Current children cumulated vsize (KiB) 664340

[startup+1062.3 s]
/proc/loadavg: 6.34 7.66 7.88 5/176 20943
/proc/meminfo: memFree=29930448/32951124 swapFree=67111416/67111528
[pid=20875] ppid=20873 vsize=797952 CPUtime=4242.55 cores=1,3,5,7
/proc/20875/stat : 20875 (plingeling) S 20873 20875 19850 0 -1 4202496 734901 1253 0 0 424131 124 0 0 25 0 5 0 522284294 817102848 170943 33554432000 4194304 4417332 140735299422096 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/20875/statm: 199488 170943 181 55 0 197374 0
[pid=20875/tid=20882] ppid=20873 vsize=797952 CPUtime=1062.29 cores=1,3,5,7
/proc/20875/task/20882/stat : 20882 (plingeling) R 20873 20875 19850 0 -1 4202560 193000 1253 0 0 106197 32 0 0 25 0 5 0 522284299 817102848 170943 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20875/tid=20883] ppid=20873 vsize=797952 CPUtime=1062.25 cores=1,3,5,7
/proc/20875/task/20883/stat : 20883 (plingeling) R 20873 20875 19850 0 -1 4202560 198270 1253 0 0 106192 33 0 0 25 0 5 0 522284299 817102848 170943 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20875/tid=20884] ppid=20873 vsize=797952 CPUtime=1061.44 cores=1,3,5,7
/proc/20875/task/20884/stat : 20884 (plingeling) R 20873 20875 19850 0 -1 4202560 206938 1253 0 0 106109 35 0 0 25 0 5 0 522284299 817102848 170943 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20875/tid=20885] ppid=20873 vsize=797952 CPUtime=1056.5 cores=1,3,5,7
/proc/20875/task/20885/stat : 20885 (plingeling) R 20873 20875 19850 0 -1 4202560 134381 1253 0 0 105628 22 0 0 25 0 5 0 522284299 817102848 170943 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4242.55
Current children cumulated vsize (KiB) 797952

[startup+1122.3 s]
/proc/loadavg: 4.86 6.99 7.64 5/176 20945
/proc/meminfo: memFree=29915396/32951124 swapFree=67111416/67111528
[pid=20875] ppid=20873 vsize=813100 CPUtime=4482.24 cores=1,3,5,7
/proc/20875/stat : 20875 (plingeling) S 20873 20875 19850 0 -1 4202496 738688 1253 0 0 448099 125 0 0 25 0 5 0 522284294 832614400 174730 33554432000 4194304 4417332 140735299422096 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/20875/statm: 203275 174730 181 55 0 201161 0
[pid=20875/tid=20882] ppid=20873 vsize=813100 CPUtime=1122.29 cores=1,3,5,7
/proc/20875/task/20882/stat : 20882 (plingeling) R 20873 20875 19850 0 -1 4202560 193000 1253 0 0 112197 32 0 0 25 0 5 0 522284299 832614400 174730 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20875/tid=20883] ppid=20873 vsize=813100 CPUtime=1122.24 cores=1,3,5,7
/proc/20875/task/20883/stat : 20883 (plingeling) R 20873 20875 19850 0 -1 4202560 199797 1253 0 0 112191 33 0 0 25 0 5 0 522284299 832614400 174730 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364683 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20875/tid=20884] ppid=20873 vsize=813100 CPUtime=1121.44 cores=1,3,5,7
/proc/20875/task/20884/stat : 20884 (plingeling) R 20873 20875 19850 0 -1 4202560 209198 1253 0 0 112109 35 0 0 25 0 5 0 522284299 832614400 174730 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20875/tid=20885] ppid=20873 vsize=813100 CPUtime=1116.18 cores=1,3,5,7
/proc/20875/task/20885/stat : 20885 (plingeling) R 20873 20875 19850 0 -1 4202560 134381 1253 0 0 111596 22 0 0 25 0 5 0 522284299 832614400 174730 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4482.24
Current children cumulated vsize (KiB) 813100

[startup+1182.3 s]
/proc/loadavg: 4.31 6.45 7.41 5/176 20946
/proc/meminfo: memFree=30009208/32951124 swapFree=67111416/67111528
[pid=20875] ppid=20873 vsize=719412 CPUtime=4721.97 cores=1,3,5,7
/proc/20875/stat : 20875 (plingeling) S 20873 20875 19850 0 -1 4202496 803127 1253 0 0 472061 136 0 0 25 0 5 0 522284294 736677888 151308 33554432000 4194304 4417332 140735299422096 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/20875/statm: 179853 151308 181 55 0 177739 0
[pid=20875/tid=20882] ppid=20873 vsize=719412 CPUtime=1182.29 cores=1,3,5,7
/proc/20875/task/20882/stat : 20882 (plingeling) R 20873 20875 19850 0 -1 4202560 209689 1253 0 0 118195 34 0 0 25 0 5 0 522284299 736677888 151308 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20875/tid=20883] ppid=20873 vsize=719412 CPUtime=1182.25 cores=1,3,5,7
/proc/20875/task/20883/stat : 20883 (plingeling) R 20873 20875 19850 0 -1 4202560 216626 1253 0 0 118188 37 0 0 25 0 5 0 522284299 736677888 151308 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364753 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20875/tid=20884] ppid=20873 vsize=719412 CPUtime=1181.45 cores=1,3,5,7
/proc/20875/task/20884/stat : 20884 (plingeling) R 20873 20875 19850 0 -1 4202560 226762 1253 0 0 118107 38 0 0 25 0 5 0 522284299 736677888 151308 33554432000 4194304 4417332 140735299422096 18446744073709551615 4356802 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20875/tid=20885] ppid=20873 vsize=719412 CPUtime=1175.9 cores=1,3,5,7
/proc/20875/task/20885/stat : 20885 (plingeling) R 20873 20875 19850 0 -1 4202560 147738 1253 0 0 117565 25 0 0 25 0 5 0 522284299 736677888 151308 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364887 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4721.97
Current children cumulated vsize (KiB) 719412



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.9 s]
/proc/loadavg: 4.22 6.29 7.33 5/176 20947
/proc/meminfo: memFree=29973368/32951124 swapFree=67111416/67111528
[pid=20875] ppid=20873 vsize=755112 CPUtime=4800.29 cores=1,3,5,7
/proc/20875/stat : 20875 (plingeling) S 20873 20875 19850 0 -1 4202496 824066 1253 0 0 479891 138 0 0 25 0 5 0 522284294 773234688 160233 33554432000 4194304 4417332 140735299422096 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/20875/statm: 188778 160233 181 55 0 186664 0
[pid=20875/tid=20882] ppid=20873 vsize=755112 CPUtime=1201.89 cores=1,3,5,7
/proc/20875/task/20882/stat : 20882 (plingeling) R 20873 20875 19850 0 -1 4202560 217881 1253 0 0 120155 34 0 0 25 0 5 0 522284299 773234688 160233 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20875/tid=20883] ppid=20873 vsize=755112 CPUtime=1201.85 cores=1,3,5,7
/proc/20875/task/20883/stat : 20883 (plingeling) R 20873 20875 19850 0 -1 4202560 229323 1253 0 0 120147 38 0 0 25 0 5 0 522284299 773234688 160233 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20875/tid=20884] ppid=20873 vsize=755112 CPUtime=1201.06 cores=1,3,5,7
/proc/20875/task/20884/stat : 20884 (plingeling) R 20873 20875 19850 0 -1 4202560 226762 1253 0 0 120068 38 0 0 25 0 5 0 522284299 773234688 160233 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20875/tid=20885] ppid=20873 vsize=755112 CPUtime=1195.41 cores=1,3,5,7
/proc/20875/task/20885/stat : 20885 (plingeling) R 20873 20875 19850 0 -1 4202560 147788 1253 0 0 119516 25 0 0 25 0 5 0 522284299 773234688 160233 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4800.29
Current children cumulated vsize (KiB) 755112

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.9 s]
/proc/loadavg: 4.22 6.29 7.33 5/176 20947
/proc/meminfo: memFree=29973368/32951124 swapFree=67111416/67111528
[pid=20875] ppid=20873 vsize=755112 CPUtime=4800.29 cores=1,3,5,7
/proc/20875/stat : 20875 (plingeling) S 20873 20875 19850 0 -1 4202496 824066 1253 0 0 479891 138 0 0 25 0 5 0 522284294 773234688 160233 33554432000 4194304 4417332 140735299422096 18446744073709551615 212447819749 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/20875/statm: 188778 160233 181 55 0 186664 0
[pid=20875/tid=20882] ppid=20873 vsize=755112 CPUtime=1201.89 cores=1,3,5,7
/proc/20875/task/20882/stat : 20882 (plingeling) R 20873 20875 19850 0 -1 4202560 217881 1253 0 0 120155 34 0 0 25 0 5 0 522284299 773234688 160233 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=20875/tid=20883] ppid=20873 vsize=755112 CPUtime=1201.85 cores=1,3,5,7
/proc/20875/task/20883/stat : 20883 (plingeling) R 20873 20875 19850 0 -1 4202560 229323 1253 0 0 120147 38 0 0 25 0 5 0 522284299 773234688 160233 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=20875/tid=20884] ppid=20873 vsize=755112 CPUtime=1201.06 cores=1,3,5,7
/proc/20875/task/20884/stat : 20884 (plingeling) R 20873 20875 19850 0 -1 4202560 226762 1253 0 0 120068 38 0 0 25 0 5 0 522284299 773234688 160233 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=20875/tid=20885] ppid=20873 vsize=755112 CPUtime=1195.41 cores=1,3,5,7
/proc/20875/task/20885/stat : 20885 (plingeling) R 20873 20875 19850 0 -1 4202560 147788 1253 0 0 119516 25 0 0 25 0 5 0 522284299 773234688 160233 33554432000 4194304 4417332 140735299422096 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4800.29
Current children cumulated vsize (KiB) 755112

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1201.94
CPU time (s): 4800.35
CPU user time (s): 4798.94
CPU system time (s): 1.40979
CPU usage (%): 399.384
Max. virtual memory (cumulated for all children) (KiB): 815248

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4798.94
system time used= 1.40978
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 825320
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 215
involuntary context switches= 34616

runsolver used 1.90771 second user time and 4.5763 second system time

The end

Launcher Data 

Begin job on node102 at 2011-04-29 21:53:39
IDJOB=3265642
IDBENCH=85793
IDSOLVER=1605
FILE ID=node102/3265642-1304106819
RUNJOBID= node102-1304105437-20556
PBS_JOBID= 13170630
Free space on /tmp= 71548 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT05/crafted/sorge05/QG7a/gensys-ukn009.shuffled-as.sat05-3849.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3265642-1304106819/watcher-3265642-1304106819 -o /tmp/evaluation-result-3265642-1304106819/solver-3265642-1304106819 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3265642-1304106819.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= f2a1eb7937a7cda114ce344a30ca1145
RANDOM SEED=257134759

node102.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      30259608 kB
Buffers:        597396 kB
Cached:        1269484 kB
SwapCached:          0 kB
Active:         476040 kB
Inactive:      1798024 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      30259608 kB
SwapTotal:    67111528 kB
SwapFree:     67111416 kB
Dirty:             880 kB
Writeback:           0 kB
AnonPages:      406404 kB
Mapped:          15492 kB
Slab:           351808 kB
PageTables:       5164 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   591520 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71560 MiB
End job on node102 at 2011-04-29 22:13:43