Trace number 3264234

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 4800.13 1201.95

General information on the benchmark

NameSAT04/handmade/kexu/frb-0.8/
frb59-26-1.used-as.sat04-891.cnf
MD5SUM5877850d9f604b6973676132cbba5c90
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables1534
Number of clauses132295
Sum of the clauses size266006
Maximum clause length26
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2132236
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 559

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3264234-1304103087/watcher-3264234-1304103087 -o /tmp/evaluation-result-3264234-1304103087/solver-3264234-1304103087 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3264234-1304103087.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.26 7.99 7.94 3/181 7936
/proc/meminfo: memFree=29720888/32951124 swapFree=67111352/67111528
[pid=7936] ppid=7933 vsize=9508 CPUtime=0 cores=1,3,5,7
/proc/7936/stat : 7936 (plingeling) R 7933 7936 6968 0 -1 4202496 464 1250 0 0 0 0 0 0 20 0 1 0 521906551 9736192 394 33554432000 4194304 4417332 140734188564176 18446744073709551615 255377301248 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/7936/statm: 2377 394 146 55 0 263 0

[startup+0.0802539 s]
/proc/loadavg: 7.26 7.99 7.94 3/181 7936
/proc/meminfo: memFree=29720888/32951124 swapFree=67111352/67111528
[pid=7936] ppid=7933 vsize=25892 CPUtime=0.07 cores=1,3,5,7
/proc/7936/stat : 7936 (plingeling) R 7933 7936 6968 0 -1 4202496 4613 1250 0 0 7 0 0 0 20 0 1 0 521906551 26513408 4495 33554432000 4194304 4417332 140734188564176 18446744073709551615 255377301248 0 0 4096 17442 0 0 0 17 1 0 0 0
/proc/7936/statm: 6473 4495 147 55 0 4359 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 25892

[startup+0.100213 s]
/proc/loadavg: 7.26 7.99 7.94 3/181 7936
/proc/meminfo: memFree=29720888/32951124 swapFree=67111352/67111528
[pid=7936] ppid=7933 vsize=132532 CPUtime=0.11 cores=1,3,5,7
/proc/7936/stat : 7936 (plingeling) S 7933 7936 6968 0 -1 4202496 4668 1250 0 0 11 0 0 0 20 0 5 0 521906551 135712768 4546 33554432000 4194304 4417332 140734188564176 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7936/statm: 33133 4546 167 55 0 31019 0
Current children cumulated CPU time (s) 0.11
Current children cumulated vsize (KiB) 132532

[startup+0.300176 s]
/proc/loadavg: 7.26 7.99 7.94 3/181 7936
/proc/meminfo: memFree=29720888/32951124 swapFree=67111352/67111528
[pid=7936] ppid=7933 vsize=134952 CPUtime=0.91 cores=1,3,5,7
/proc/7936/stat : 7936 (plingeling) S 7933 7936 6968 0 -1 4202496 9509 1250 0 0 90 1 0 0 20 0 5 0 521906551 138190848 5161 33554432000 4194304 4417332 140734188564176 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7936/statm: 33738 5161 177 55 0 31624 0
Current children cumulated CPU time (s) 0.91
Current children cumulated vsize (KiB) 134952

[startup+0.700101 s]
/proc/loadavg: 7.26 7.99 7.94 3/181 7936
/proc/meminfo: memFree=29720888/32951124 swapFree=67111352/67111528
[pid=7936] ppid=7933 vsize=144644 CPUtime=2.5 cores=1,3,5,7
/proc/7936/stat : 7936 (plingeling) S 7933 7936 6968 0 -1 4202496 13095 1250 0 0 249 1 0 0 20 0 5 0 521906551 148115456 7588 33554432000 4194304 4417332 140734188564176 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7936/statm: 36161 7588 181 55 0 34047 0
Current children cumulated CPU time (s) 2.5
Current children cumulated vsize (KiB) 144644

[startup+1.50095 s]
/proc/loadavg: 7.26 7.99 7.94 10/190 7955
/proc/meminfo: memFree=29650772/32951124 swapFree=67111352/67111528
[pid=7936] ppid=7933 vsize=148156 CPUtime=5.71 cores=1,3,5,7
/proc/7936/stat : 7936 (plingeling) S 7933 7936 6968 0 -1 4202496 16133 1250 0 0 569 2 0 0 20 0 5 0 521906551 151711744 8247 33554432000 4194304 4417332 140734188564176 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7936/statm: 37039 8247 181 55 0 34925 0
[pid=7936/tid=7952] ppid=7933 vsize=148156 CPUtime=1.4 cores=1,3,5,7
/proc/7936/task/7952/stat : 7952 (plingeling) R 7933 7936 6968 0 -1 4202560 2285 1250 0 0 140 0 0 0 25 0 5 0 521906560 151711744 8247 33554432000 4194304 4417332 140734188564176 18446744073709551615 4249601 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7936/tid=7953] ppid=7933 vsize=148156 CPUtime=1.4 cores=1,3,5,7
/proc/7936/task/7953/stat : 7953 (plingeling) R 7933 7936 6968 0 -1 4202560 1727 1250 0 0 140 0 0 0 25 0 5 0 521906560 151711744 8247 33554432000 4194304 4417332 140734188564176 18446744073709551615 4335906 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7936/tid=7954] ppid=7933 vsize=148156 CPUtime=1.4 cores=1,3,5,7
/proc/7936/task/7954/stat : 7954 (plingeling) R 7933 7936 6968 0 -1 4202560 1318 1250 0 0 140 0 0 0 25 0 5 0 521906560 151711744 8247 33554432000 4194304 4417332 140734188564176 18446744073709551615 4356888 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7936/tid=7955] ppid=7933 vsize=148156 CPUtime=1.39 cores=1,3,5,7
/proc/7936/task/7955/stat : 7955 (plingeling) R 7933 7936 6968 0 -1 4202560 6178 1250 0 0 139 0 0 0 25 0 5 0 521906560 151711744 8247 33554432000 4194304 4417332 140734188564176 18446744073709551615 4250432 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 5.71
Current children cumulated vsize (KiB) 148156

[startup+3.10066 s]
/proc/loadavg: 7.26 7.99 7.94 10/190 7955
/proc/meminfo: memFree=29630932/32951124 swapFree=67111352/67111528
[pid=7936] ppid=7933 vsize=164176 CPUtime=12.1 cores=1,3,5,7
/proc/7936/stat : 7936 (plingeling) S 7933 7936 6968 0 -1 4202496 23285 1250 0 0 1207 3 0 0 20 0 5 0 521906551 168116224 12495 33554432000 4194304 4417332 140734188564176 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7936/statm: 41044 12495 181 55 0 38930 0
[pid=7936/tid=7952] ppid=7933 vsize=164176 CPUtime=3 cores=1,3,5,7
/proc/7936/task/7952/stat : 7952 (plingeling) R 7933 7936 6968 0 -1 4202560 3082 1250 0 0 300 0 0 0 25 0 5 0 521906560 168116224 12495 33554432000 4194304 4417332 140734188564176 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7936/tid=7953] ppid=7933 vsize=164176 CPUtime=3 cores=1,3,5,7
/proc/7936/task/7953/stat : 7953 (plingeling) R 7933 7936 6968 0 -1 4202560 2524 1250 0 0 300 0 0 0 25 0 5 0 521906560 168116224 12495 33554432000 4194304 4417332 140734188564176 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7936/tid=7954] ppid=7933 vsize=164176 CPUtime=3 cores=1,3,5,7
/proc/7936/task/7954/stat : 7954 (plingeling) R 7933 7936 6968 0 -1 4202560 1318 1250 0 0 300 0 0 0 25 0 5 0 521906560 168116224 12495 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364713 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7936/tid=7955] ppid=7933 vsize=164176 CPUtime=2.98 cores=1,3,5,7
/proc/7936/task/7955/stat : 7955 (plingeling) R 7933 7936 6968 0 -1 4202560 11736 1250 0 0 297 1 0 0 25 0 5 0 521906560 168116224 12495 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364470 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 12.1
Current children cumulated vsize (KiB) 164176

[startup+6.30106 s]
/proc/loadavg: 7.32 7.99 7.94 10/190 7955
/proc/meminfo: memFree=29621820/32951124 swapFree=67111352/67111528
[pid=7936] ppid=7933 vsize=175428 CPUtime=24.88 cores=1,3,5,7
/proc/7936/stat : 7936 (plingeling) S 7933 7936 6968 0 -1 4202496 35697 1250 0 0 2484 4 0 0 20 0 5 0 521906551 179638272 15310 33554432000 4194304 4417332 140734188564176 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7936/statm: 43857 15310 181 55 0 41743 0
[pid=7936/tid=7952] ppid=7933 vsize=175428 CPUtime=6.2 cores=1,3,5,7
/proc/7936/task/7952/stat : 7952 (plingeling) R 7933 7936 6968 0 -1 4202560 3082 1250 0 0 620 0 0 0 25 0 5 0 521906560 179638272 15310 33554432000 4194304 4417332 140734188564176 18446744073709551615 4249186 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7936/tid=7953] ppid=7933 vsize=175428 CPUtime=6.2 cores=1,3,5,7
/proc/7936/task/7953/stat : 7953 (plingeling) R 7933 7936 6968 0 -1 4202560 2524 1250 0 0 620 0 0 0 25 0 5 0 521906560 179638272 15310 33554432000 4194304 4417332 140734188564176 18446744073709551615 4249909 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7936/tid=7954] ppid=7933 vsize=175428 CPUtime=6.2 cores=1,3,5,7
/proc/7936/task/7954/stat : 7954 (plingeling) R 7933 7936 6968 0 -1 4202560 1318 1250 0 0 620 0 0 0 25 0 5 0 521906560 179638272 15310 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364750 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7936/tid=7955] ppid=7933 vsize=175428 CPUtime=6.17 cores=1,3,5,7
/proc/7936/task/7955/stat : 7955 (plingeling) R 7933 7936 6968 0 -1 4202560 24148 1250 0 0 614 3 0 0 25 0 5 0 521906560 179638272 15310 33554432000 4194304 4417332 140734188564176 18446744073709551615 4321690 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 24.88
Current children cumulated vsize (KiB) 175428

[startup+12.7009 s]
/proc/loadavg: 7.37 7.99 7.94 10/190 7955
/proc/meminfo: memFree=29547164/32951124 swapFree=67111352/67111528
[pid=7936] ppid=7933 vsize=220996 CPUtime=50.45 cores=1,3,5,7
/proc/7936/stat : 7936 (plingeling) S 7933 7936 6968 0 -1 4202496 54902 1250 0 0 5037 8 0 0 20 0 5 0 521906551 226299904 26676 33554432000 4194304 4417332 140734188564176 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7936/statm: 55249 26676 181 55 0 53135 0
[pid=7936/tid=7952] ppid=7933 vsize=220996 CPUtime=12.6 cores=1,3,5,7
/proc/7936/task/7952/stat : 7952 (plingeling) R 7933 7936 6968 0 -1 4202560 3879 1250 0 0 1260 0 0 0 25 0 5 0 521906560 226299904 26676 33554432000 4194304 4417332 140734188564176 18446744073709551615 4249130 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7936/tid=7953] ppid=7933 vsize=220996 CPUtime=12.6 cores=1,3,5,7
/proc/7936/task/7953/stat : 7953 (plingeling) R 7933 7936 6968 0 -1 4202560 3321 1250 0 0 1260 0 0 0 25 0 5 0 521906560 226299904 26676 33554432000 4194304 4417332 140734188564176 18446744073709551615 4249124 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7936/tid=7954] ppid=7933 vsize=220996 CPUtime=12.6 cores=1,3,5,7
/proc/7936/task/7954/stat : 7954 (plingeling) R 7933 7936 6968 0 -1 4202560 1633 1250 0 0 1260 0 0 0 25 0 5 0 521906560 226299904 26676 33554432000 4194304 4417332 140734188564176 18446744073709551615 4258267 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7936/tid=7955] ppid=7933 vsize=220996 CPUtime=12.53 cores=1,3,5,7
/proc/7936/task/7955/stat : 7955 (plingeling) R 7933 7936 6968 0 -1 4202560 41444 1250 0 0 1247 6 0 0 25 0 5 0 521906560 226299904 26676 33554432000 4194304 4417332 140734188564176 18446744073709551615 4365963 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.45
Current children cumulated vsize (KiB) 220996

[startup+25.5005 s]
/proc/loadavg: 7.51 7.99 7.94 10/190 7955
/proc/meminfo: memFree=29455680/32951124 swapFree=67111352/67111528
[pid=7936] ppid=7933 vsize=225592 CPUtime=101.57 cores=1,3,5,7
/proc/7936/stat : 7936 (plingeling) S 7933 7936 6968 0 -1 4202496 95593 1250 0 0 10144 13 0 0 20 0 5 0 521906551 231006208 27880 33554432000 4194304 4417332 140734188564176 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7936/statm: 56398 27880 181 55 0 54284 0
[pid=7936/tid=7952] ppid=7933 vsize=225592 CPUtime=25.4 cores=1,3,5,7
/proc/7936/task/7952/stat : 7952 (plingeling) R 7933 7936 6968 0 -1 4202560 4933 1250 0 0 2540 0 0 0 25 0 5 0 521906560 231006208 27880 33554432000 4194304 4417332 140734188564176 18446744073709551615 4306272 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7936/tid=7953] ppid=7933 vsize=225592 CPUtime=25.4 cores=1,3,5,7
/proc/7936/task/7953/stat : 7953 (plingeling) R 7933 7936 6968 0 -1 4202560 4233 1250 0 0 2540 0 0 0 25 0 5 0 521906560 231006208 27880 33554432000 4194304 4417332 140734188564176 18446744073709551615 4311304 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7936/tid=7954] ppid=7933 vsize=225592 CPUtime=25.4 cores=1,3,5,7
/proc/7936/task/7954/stat : 7954 (plingeling) R 7933 7936 6968 0 -1 4202560 5482 1250 0 0 2540 0 0 0 25 0 5 0 521906560 231006208 27880 33554432000 4194304 4417332 140734188564176 18446744073709551615 4365986 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7936/tid=7955] ppid=7933 vsize=225592 CPUtime=25.26 cores=1,3,5,7
/proc/7936/task/7955/stat : 7955 (plingeling) R 7933 7936 6968 0 -1 4202560 76320 1250 0 0 2515 11 0 0 25 0 5 0 521906560 231006208 27880 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364498 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.57
Current children cumulated vsize (KiB) 225592

[startup+51.1008 s]
/proc/loadavg: 7.86 8.03 7.95 10/190 7956
/proc/meminfo: memFree=29277156/32951124 swapFree=67111352/67111528
[pid=7936] ppid=7933 vsize=334920 CPUtime=203.83 cores=1,3,5,7
/proc/7936/stat : 7936 (plingeling) S 7933 7936 6968 0 -1 4202496 157337 1250 0 0 20361 22 0 0 20 0 5 0 521906551 342958080 55188 33554432000 4194304 4417332 140734188564176 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7936/statm: 83730 55188 181 55 0 81616 0
[pid=7936/tid=7952] ppid=7933 vsize=334920 CPUtime=51 cores=1,3,5,7
/proc/7936/task/7952/stat : 7952 (plingeling) R 7933 7936 6968 0 -1 4202560 9053 1250 0 0 5099 1 0 0 25 0 5 0 521906560 342958080 55188 33554432000 4194304 4417332 140734188564176 18446744073709551615 4275646 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7936/tid=7953] ppid=7933 vsize=334920 CPUtime=50.99 cores=1,3,5,7
/proc/7936/task/7953/stat : 7953 (plingeling) R 7933 7936 6968 0 -1 4202560 6067 1250 0 0 5099 0 0 0 25 0 5 0 521906560 342958080 55188 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7936/tid=7954] ppid=7933 vsize=334920 CPUtime=50.99 cores=1,3,5,7
/proc/7936/task/7954/stat : 7954 (plingeling) R 7933 7936 6968 0 -1 4202560 10482 1250 0 0 5099 0 0 0 25 0 5 0 521906560 342958080 55188 33554432000 4194304 4417332 140734188564176 18446744073709551615 4236941 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7936/tid=7955] ppid=7933 vsize=334920 CPUtime=50.73 cores=1,3,5,7
/proc/7936/task/7955/stat : 7955 (plingeling) R 7933 7936 6968 0 -1 4202560 127110 1250 0 0 5054 19 0 0 25 0 5 0 521906560 342958080 55188 33554432000 4194304 4417332 140734188564176 18446744073709551615 4250593 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 203.83
Current children cumulated vsize (KiB) 334920

[startup+102.307 s]
/proc/loadavg: 8.14 8.07 7.97 10/190 7958
/proc/meminfo: memFree=29088424/32951124 swapFree=67111352/67111528
[pid=7936] ppid=7933 vsize=383212 CPUtime=408.38 cores=1,3,5,7
/proc/7936/stat : 7936 (plingeling) S 7933 7936 6968 0 -1 4202496 250816 1250 0 0 40800 38 0 0 20 0 5 0 521906551 392409088 67261 33554432000 4194304 4417332 140734188564176 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7936/statm: 95803 67261 181 55 0 93689 0
[pid=7936/tid=7952] ppid=7933 vsize=383212 CPUtime=102.21 cores=1,3,5,7
/proc/7936/task/7952/stat : 7952 (plingeling) R 7933 7936 6968 0 -1 4202560 21962 1250 0 0 10218 3 0 0 25 0 5 0 521906560 392409088 67261 33554432000 4194304 4417332 140734188564176 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7936/tid=7953] ppid=7933 vsize=383212 CPUtime=102.21 cores=1,3,5,7
/proc/7936/task/7953/stat : 7953 (plingeling) R 7933 7936 6968 0 -1 4202560 9581 1250 0 0 10220 1 0 0 25 0 5 0 521906560 392409088 67261 33554432000 4194304 4417332 140734188564176 18446744073709551615 4278260 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7936/tid=7954] ppid=7933 vsize=383212 CPUtime=102.21 cores=1,3,5,7
/proc/7936/task/7954/stat : 7954 (plingeling) R 7933 7936 6968 0 -1 4202560 26806 1250 0 0 10218 3 0 0 25 0 5 0 521906560 392409088 67261 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364473 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7936/tid=7955] ppid=7933 vsize=383212 CPUtime=101.65 cores=1,3,5,7
/proc/7936/task/7955/stat : 7955 (plingeling) R 7933 7936 6968 0 -1 4202560 187842 1250 0 0 10136 29 0 0 25 0 5 0 521906560 392409088 67261 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364722 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 408.38
Current children cumulated vsize (KiB) 383212

[startup+162.301 s]
/proc/loadavg: 8.12 8.07 7.97 10/190 7959
/proc/meminfo: memFree=28897804/32951124 swapFree=67111352/67111528
[pid=7936] ppid=7933 vsize=560692 CPUtime=648.04 cores=1,3,5,7
/proc/7936/stat : 7936 (plingeling) S 7933 7936 6968 0 -1 4202496 321568 1250 0 0 64755 49 0 0 20 0 5 0 521906551 574148608 111663 33554432000 4194304 4417332 140734188564176 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7936/statm: 140173 111663 181 55 0 138059 0
[pid=7936/tid=7952] ppid=7933 vsize=560692 CPUtime=162.21 cores=1,3,5,7
/proc/7936/task/7952/stat : 7952 (plingeling) R 7933 7936 6968 0 -1 4202560 33783 1250 0 0 16216 5 0 0 25 0 5 0 521906560 574148608 111663 33554432000 4194304 4417332 140734188564176 18446744073709551615 4357164 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7936/tid=7953] ppid=7933 vsize=560692 CPUtime=162.19 cores=1,3,5,7
/proc/7936/task/7953/stat : 7953 (plingeling) R 7933 7936 6968 0 -1 4202560 14020 1250 0 0 16218 1 0 0 25 0 5 0 521906560 574148608 111663 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7936/tid=7954] ppid=7933 vsize=560692 CPUtime=162.21 cores=1,3,5,7
/proc/7936/task/7954/stat : 7954 (plingeling) R 7933 7936 6968 0 -1 4202560 41570 1250 0 0 16216 5 0 0 25 0 5 0 521906560 574148608 111663 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364473 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7936/tid=7955] ppid=7933 vsize=560692 CPUtime=161.32 cores=1,3,5,7
/proc/7936/task/7955/stat : 7955 (plingeling) R 7933 7936 6968 0 -1 4202560 227570 1250 0 0 16096 36 0 0 25 0 5 0 521906560 574148608 111663 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 648.04
Current children cumulated vsize (KiB) 560692


################
# More data... #
################

[pid=7936/tid=7953] ppid=7933 vsize=1046964 CPUtime=762.2 cores=1,3,5,7
/proc/7936/task/7953/stat : 7953 (plingeling) R 7933 7936 6968 0 -1 4202560 30138 1250 0 0 76217 3 0 0 25 0 5 0 521906560 1072091136 233199 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364375 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7936/tid=7954] ppid=7933 vsize=1046964 CPUtime=762.22 cores=1,3,5,7
/proc/7936/task/7954/stat : 7954 (plingeling) R 7933 7936 6968 0 -1 4202560 155723 1250 0 0 76203 19 0 0 25 0 5 0 521906560 1072091136 233199 33554432000 4194304 4417332 140734188564176 18446744073709551615 4344081 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7936/tid=7955] ppid=7933 vsize=1046964 CPUtime=758.08 cores=1,3,5,7
/proc/7936/task/7955/stat : 7955 (plingeling) R 7933 7936 6968 0 -1 4202560 487665 1250 0 0 75729 79 0 0 25 0 5 0 521906560 1072091136 233199 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3044.84
Current children cumulated vsize (KiB) 1046964

[startup+822.301 s]
/proc/loadavg: 8.45 8.18 8.04 10/190 7989
/proc/meminfo: memFree=27949976/32951124 swapFree=67111352/67111528
[pid=7936] ppid=7933 vsize=1103692 CPUtime=3283.67 cores=1,3,5,7
/proc/7936/stat : 7936 (plingeling) S 7933 7936 6968 0 -1 4202496 845841 1250 0 0 328240 127 0 0 20 0 5 0 521906551 1130180608 247381 33554432000 4194304 4417332 140734188564176 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7936/statm: 275923 247381 181 55 0 273809 0
[pid=7936/tid=7952] ppid=7933 vsize=1103692 CPUtime=822.21 cores=1,3,5,7
/proc/7936/task/7952/stat : 7952 (plingeling) R 7933 7936 6968 0 -1 4202560 127673 1250 0 0 82204 17 0 0 25 0 5 0 521906560 1130180608 247381 33554432000 4194304 4417332 140734188564176 18446744073709551615 4365960 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7936/tid=7953] ppid=7933 vsize=1103692 CPUtime=822.2 cores=1,3,5,7
/proc/7936/task/7953/stat : 7953 (plingeling) R 7933 7936 6968 0 -1 4202560 32186 1250 0 0 82216 4 0 0 25 0 5 0 521906560 1130180608 247381 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364796 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7936/tid=7954] ppid=7933 vsize=1103692 CPUtime=821.4 cores=1,3,5,7
/proc/7936/task/7954/stat : 7954 (plingeling) R 7933 7936 6968 0 -1 4202560 169115 1250 0 0 82118 22 0 0 25 0 5 0 521906560 1130180608 247381 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364753 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7936/tid=7955] ppid=7933 vsize=1103692 CPUtime=817.75 cores=1,3,5,7
/proc/7936/task/7955/stat : 7955 (plingeling) R 7933 7936 6968 0 -1 4202560 512242 1250 0 0 81693 82 0 0 25 0 5 0 521906560 1130180608 247381 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3283.67
Current children cumulated vsize (KiB) 1103692

[startup+882.301 s]
/proc/loadavg: 8.31 8.17 8.05 10/190 7990
/proc/meminfo: memFree=27875764/32951124 swapFree=67111352/67111528
[pid=7936] ppid=7933 vsize=1127880 CPUtime=3523.35 cores=1,3,5,7
/proc/7936/stat : 7936 (plingeling) S 7933 7936 6968 0 -1 4202496 880465 1250 0 0 352201 134 0 0 20 0 5 0 521906551 1154949120 253428 33554432000 4194304 4417332 140734188564176 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7936/statm: 281970 253428 181 55 0 279856 0
[pid=7936/tid=7952] ppid=7933 vsize=1127880 CPUtime=882.22 cores=1,3,5,7
/proc/7936/task/7952/stat : 7952 (plingeling) R 7933 7936 6968 0 -1 4202560 138301 1250 0 0 88202 20 0 0 25 0 5 0 521906560 1154949120 253428 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7936/tid=7953] ppid=7933 vsize=1127880 CPUtime=882.2 cores=1,3,5,7
/proc/7936/task/7953/stat : 7953 (plingeling) R 7933 7936 6968 0 -1 4202560 32186 1250 0 0 88216 4 0 0 25 0 5 0 521906560 1154949120 253428 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364942 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7936/tid=7954] ppid=7933 vsize=1127880 CPUtime=881.39 cores=1,3,5,7
/proc/7936/task/7954/stat : 7954 (plingeling) R 7933 7936 6968 0 -1 4202560 185635 1250 0 0 88115 24 0 0 25 0 5 0 521906560 1154949120 253428 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364519 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7936/tid=7955] ppid=7933 vsize=1127880 CPUtime=877.42 cores=1,3,5,7
/proc/7936/task/7955/stat : 7955 (plingeling) R 7933 7936 6968 0 -1 4202560 519718 1250 0 0 87658 84 0 0 25 0 5 0 521906560 1154949120 253428 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364732 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3523.35
Current children cumulated vsize (KiB) 1127880

[startup+942.301 s]
/proc/loadavg: 8.25 8.17 8.05 10/190 7992
/proc/meminfo: memFree=27766132/32951124 swapFree=67111352/67111528
[pid=7936] ppid=7933 vsize=1145096 CPUtime=3763.03 cores=1,3,5,7
/proc/7936/stat : 7936 (plingeling) S 7933 7936 6968 0 -1 4202496 893749 1250 0 0 376168 135 0 0 20 0 5 0 521906551 1172578304 257732 33554432000 4194304 4417332 140734188564176 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7936/statm: 286274 257732 181 55 0 284160 0
[pid=7936/tid=7952] ppid=7933 vsize=1145096 CPUtime=942.21 cores=1,3,5,7
/proc/7936/task/7952/stat : 7952 (plingeling) R 7933 7936 6968 0 -1 4202560 143265 1250 0 0 94201 20 0 0 25 0 5 0 521906560 1172578304 257732 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7936/tid=7953] ppid=7933 vsize=1145096 CPUtime=942.19 cores=1,3,5,7
/proc/7936/task/7953/stat : 7953 (plingeling) R 7933 7936 6968 0 -1 4202560 36282 1250 0 0 94215 4 0 0 25 0 5 0 521906560 1172578304 257732 33554432000 4194304 4417332 140734188564176 18446744073709551615 4365963 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7936/tid=7954] ppid=7933 vsize=1145096 CPUtime=941.4 cores=1,3,5,7
/proc/7936/task/7954/stat : 7954 (plingeling) R 7933 7936 6968 0 -1 4202560 185635 1250 0 0 94116 24 0 0 25 0 5 0 521906560 1172578304 257732 33554432000 4194304 4417332 140734188564176 18446744073709551615 4311347 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7936/tid=7955] ppid=7933 vsize=1145096 CPUtime=937.1 cores=1,3,5,7
/proc/7936/task/7955/stat : 7955 (plingeling) R 7933 7936 6968 0 -1 4202560 523942 1250 0 0 93626 84 0 0 25 0 5 0 521906560 1172578304 257732 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3763.03
Current children cumulated vsize (KiB) 1145096

[startup+1002.3 s]
/proc/loadavg: 8.30 8.20 8.07 11/190 7994
/proc/meminfo: memFree=27996004/32951124 swapFree=67111352/67111528
[pid=7936] ppid=7933 vsize=809148 CPUtime=4002.71 cores=1,3,5,7
/proc/7936/stat : 7936 (plingeling) S 7933 7936 6968 0 -1 4202496 969604 1250 0 0 400124 147 0 0 20 0 5 0 521906551 828567552 173745 33554432000 4194304 4417332 140734188564176 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7936/statm: 202287 173745 181 55 0 200173 0
[pid=7936/tid=7952] ppid=7933 vsize=809148 CPUtime=1002.23 cores=1,3,5,7
/proc/7936/task/7952/stat : 7952 (plingeling) R 7933 7936 6968 0 -1 4202560 153986 1250 0 0 100201 22 0 0 25 0 5 0 521906560 828567552 173745 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364671 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7936/tid=7953] ppid=7933 vsize=809148 CPUtime=1002.2 cores=1,3,5,7
/proc/7936/task/7953/stat : 7953 (plingeling) R 7933 7936 6968 0 -1 4202560 38330 1250 0 0 100215 5 0 0 25 0 5 0 521906560 828567552 173745 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364734 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7936/tid=7954] ppid=7933 vsize=809148 CPUtime=1001.4 cores=1,3,5,7
/proc/7936/task/7954/stat : 7954 (plingeling) R 7933 7936 6968 0 -1 4202560 199466 1250 0 0 100114 26 0 0 25 0 5 0 521906560 828567552 173745 33554432000 4194304 4417332 140734188564176 18446744073709551615 4343691 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7936/tid=7955] ppid=7933 vsize=809148 CPUtime=996.77 cores=1,3,5,7
/proc/7936/task/7955/stat : 7955 (plingeling) R 7933 7936 6968 0 -1 4202560 573197 1250 0 0 99584 93 0 0 25 0 5 0 521906560 828567552 173745 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4002.71
Current children cumulated vsize (KiB) 809148

[startup+1062.3 s]
/proc/loadavg: 8.22 8.19 8.07 10/190 7995
/proc/meminfo: memFree=27838724/32951124 swapFree=67111352/67111528
[pid=7936] ppid=7933 vsize=826880 CPUtime=4242.39 cores=1,3,5,7
/proc/7936/stat : 7936 (plingeling) S 7933 7936 6968 0 -1 4202496 987730 1250 0 0 424090 149 0 0 20 0 5 0 521906551 846725120 178178 33554432000 4194304 4417332 140734188564176 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7936/statm: 206720 178178 181 55 0 204606 0
[pid=7936/tid=7952] ppid=7933 vsize=826880 CPUtime=1062.22 cores=1,3,5,7
/proc/7936/task/7952/stat : 7952 (plingeling) R 7933 7936 6968 0 -1 4202560 157542 1250 0 0 106200 22 0 0 25 0 5 0 521906560 846725120 178178 33554432000 4194304 4417332 140734188564176 18446744073709551615 4365198 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7936/tid=7953] ppid=7933 vsize=826880 CPUtime=1062.2 cores=1,3,5,7
/proc/7936/task/7953/stat : 7953 (plingeling) R 7933 7936 6968 0 -1 4202560 38330 1250 0 0 106215 5 0 0 25 0 5 0 521906560 846725120 178178 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7936/tid=7954] ppid=7933 vsize=826880 CPUtime=1061.4 cores=1,3,5,7
/proc/7936/task/7954/stat : 7954 (plingeling) R 7933 7936 6968 0 -1 4202560 201950 1250 0 0 106114 26 0 0 25 0 5 0 521906560 846725120 178178 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7936/tid=7955] ppid=7933 vsize=826880 CPUtime=1056.45 cores=1,3,5,7
/proc/7936/task/7955/stat : 7955 (plingeling) R 7933 7936 6968 0 -1 4202560 585283 1250 0 0 105551 94 0 0 25 0 5 0 521906560 846725120 178178 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4242.39
Current children cumulated vsize (KiB) 826880

[startup+1122.3 s]
/proc/loadavg: 8.27 8.21 8.09 10/190 7997
/proc/meminfo: memFree=27762332/32951124 swapFree=67111352/67111528
[pid=7936] ppid=7933 vsize=896104 CPUtime=4482.07 cores=1,3,5,7
/proc/7936/stat : 7936 (plingeling) S 7933 7936 6968 0 -1 4202496 1022584 1250 0 0 448054 153 0 0 20 0 5 0 521906551 917610496 195484 33554432000 4194304 4417332 140734188564176 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7936/statm: 224026 195484 181 55 0 221912 0
[pid=7936/tid=7952] ppid=7933 vsize=896104 CPUtime=1122.23 cores=1,3,5,7
/proc/7936/task/7952/stat : 7952 (plingeling) R 7933 7936 6968 0 -1 4202560 166576 1250 0 0 112200 23 0 0 25 0 5 0 521906560 917610496 195484 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7936/tid=7953] ppid=7933 vsize=896104 CPUtime=1122.19 cores=1,3,5,7
/proc/7936/task/7953/stat : 7953 (plingeling) R 7933 7936 6968 0 -1 4202560 44474 1250 0 0 112214 5 0 0 25 0 5 0 521906560 917610496 195484 33554432000 4194304 4417332 140734188564176 18446744073709551615 4346244 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7936/tid=7954] ppid=7933 vsize=896104 CPUtime=1121.41 cores=1,3,5,7
/proc/7936/task/7954/stat : 7954 (plingeling) R 7933 7936 6968 0 -1 4202560 207038 1250 0 0 112114 27 0 0 25 0 5 0 521906560 917610496 195484 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7936/tid=7955] ppid=7933 vsize=896104 CPUtime=1116.12 cores=1,3,5,7
/proc/7936/task/7955/stat : 7955 (plingeling) R 7933 7936 6968 0 -1 4202560 599871 1250 0 0 111516 96 0 0 25 0 5 0 521906560 917610496 195484 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4482.07
Current children cumulated vsize (KiB) 896104

[startup+1182.3 s]
/proc/loadavg: 8.10 8.17 8.08 10/190 7998
/proc/meminfo: memFree=28177456/32951124 swapFree=67111352/67111528
[pid=7936] ppid=7933 vsize=966868 CPUtime=4721.75 cores=1,3,5,7
/proc/7936/stat : 7936 (plingeling) S 7933 7936 6968 0 -1 4202496 1057866 1250 0 0 472018 157 0 0 20 0 5 0 521906551 990072832 213175 33554432000 4194304 4417332 140734188564176 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7936/statm: 241717 213175 181 55 0 239603 0
[pid=7936/tid=7952] ppid=7933 vsize=966868 CPUtime=1182.23 cores=1,3,5,7
/proc/7936/task/7952/stat : 7952 (plingeling) R 7933 7936 6968 0 -1 4202560 174085 1250 0 0 118199 24 0 0 25 0 5 0 521906560 990072832 213175 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364732 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7936/tid=7953] ppid=7933 vsize=966868 CPUtime=1182.2 cores=1,3,5,7
/proc/7936/task/7953/stat : 7953 (plingeling) R 7933 7936 6968 0 -1 4202560 44474 1250 0 0 118215 5 0 0 25 0 5 0 521906560 990072832 213175 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7936/tid=7954] ppid=7933 vsize=966868 CPUtime=1181.41 cores=1,3,5,7
/proc/7936/task/7954/stat : 7954 (plingeling) R 7933 7936 6968 0 -1 4202560 216495 1250 0 0 118113 28 0 0 25 0 5 0 521906560 990072832 213175 33554432000 4194304 4417332 140734188564176 18446744073709551615 4236601 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7936/tid=7955] ppid=7933 vsize=966868 CPUtime=1175.8 cores=1,3,5,7
/proc/7936/task/7955/stat : 7955 (plingeling) R 7933 7936 6968 0 -1 4202560 618187 1250 0 0 117483 97 0 0 25 0 5 0 521906560 990072832 213175 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4721.75
Current children cumulated vsize (KiB) 966868



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.9 s]
/proc/loadavg: 8.07 8.16 8.08 5/185 8000
/proc/meminfo: memFree=28847340/32951124 swapFree=67111352/67111528
[pid=7936] ppid=7933 vsize=983040 CPUtime=4800.06 cores=1,3,5,7
/proc/7936/stat : 7936 (plingeling) S 7933 7936 6968 0 -1 4202496 1061909 1250 0 0 479849 157 0 0 20 0 5 0 521906551 1006632960 217218 33554432000 4194304 4417332 140734188564176 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7936/statm: 245760 217218 181 55 0 243646 0
[pid=7936/tid=7952] ppid=7933 vsize=983040 CPUtime=1201.83 cores=1,3,5,7
/proc/7936/task/7952/stat : 7952 (plingeling) R 7933 7936 6968 0 -1 4202560 174085 1250 0 0 120159 24 0 0 25 0 5 0 521906560 1006632960 217218 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7936/tid=7953] ppid=7933 vsize=983040 CPUtime=1201.8 cores=1,3,5,7
/proc/7936/task/7953/stat : 7953 (plingeling) R 7933 7936 6968 0 -1 4202560 44474 1250 0 0 120175 5 0 0 25 0 5 0 521906560 1006632960 217218 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364818 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7936/tid=7954] ppid=7933 vsize=983040 CPUtime=1201.01 cores=1,3,5,7
/proc/7936/task/7954/stat : 7954 (plingeling) R 7933 7936 6968 0 -1 4202560 216495 1250 0 0 120073 28 0 0 25 0 5 0 521906560 1006632960 217218 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7936/tid=7955] ppid=7933 vsize=983040 CPUtime=1195.29 cores=1,3,5,7
/proc/7936/task/7955/stat : 7955 (plingeling) R 7933 7936 6968 0 -1 4202560 622230 1250 0 0 119431 98 0 0 25 0 5 0 521906560 1006632960 217218 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4800.06
Current children cumulated vsize (KiB) 983040

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.9 s]
/proc/loadavg: 8.07 8.16 8.08 5/185 8000
/proc/meminfo: memFree=28847340/32951124 swapFree=67111352/67111528
[pid=7936] ppid=7933 vsize=983040 CPUtime=4800.06 cores=1,3,5,7
/proc/7936/stat : 7936 (plingeling) S 7933 7936 6968 0 -1 4202496 1061909 1250 0 0 479849 157 0 0 20 0 5 0 521906551 1006632960 217218 33554432000 4194304 4417332 140734188564176 18446744073709551615 255389104101 0 0 4096 17442 18446744073709551615 0 0 17 1 0 0 0
/proc/7936/statm: 245760 217218 181 55 0 243646 0
[pid=7936/tid=7952] ppid=7933 vsize=983040 CPUtime=1201.83 cores=1,3,5,7
/proc/7936/task/7952/stat : 7952 (plingeling) R 7933 7936 6968 0 -1 4202560 174085 1250 0 0 120159 24 0 0 25 0 5 0 521906560 1006632960 217218 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=7936/tid=7953] ppid=7933 vsize=983040 CPUtime=1201.8 cores=1,3,5,7
/proc/7936/task/7953/stat : 7953 (plingeling) R 7933 7936 6968 0 -1 4202560 44474 1250 0 0 120175 5 0 0 25 0 5 0 521906560 1006632960 217218 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364818 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=7936/tid=7954] ppid=7933 vsize=983040 CPUtime=1201.01 cores=1,3,5,7
/proc/7936/task/7954/stat : 7954 (plingeling) R 7933 7936 6968 0 -1 4202560 216495 1250 0 0 120073 28 0 0 25 0 5 0 521906560 1006632960 217218 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7936/tid=7955] ppid=7933 vsize=983040 CPUtime=1195.29 cores=1,3,5,7
/proc/7936/task/7955/stat : 7955 (plingeling) R 7933 7936 6968 0 -1 4202560 622230 1250 0 0 119431 98 0 0 25 0 5 0 521906560 1006632960 217218 33554432000 4194304 4417332 140734188564176 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4800.06
Current children cumulated vsize (KiB) 983040

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1201.95
CPU time (s): 4800.13
CPU user time (s): 4798.51
CPU system time (s): 1.61875
CPU usage (%): 399.362
Max. virtual memory (cumulated for all children) (KiB): 1372628

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4798.51
system time used= 1.61875
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1063159
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 110
involuntary context switches= 40185

runsolver used 1.89171 second user time and 4.66429 second system time

The end

Launcher Data 

Begin job on node135 at 2011-04-29 20:51:27
IDJOB=3264234
IDBENCH=84568
IDSOLVER=1605
FILE ID=node135/3264234-1304103087
RUNJOBID= node135-1304103087-7917
PBS_JOBID= 13170608
Free space on /tmp= 71116 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT04/handmade/kexu/frb-0.8/frb59-26-1.used-as.sat04-891.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3264234-1304103087/watcher-3264234-1304103087 -o /tmp/evaluation-result-3264234-1304103087/solver-3264234-1304103087 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3264234-1304103087.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 5877850d9f604b6973676132cbba5c90
RANDOM SEED=314271229

node135.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 6640.14
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.78
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      29721316 kB
Buffers:        914468 kB
Cached:        1732176 kB
SwapCached:        172 kB
Active:         121476 kB
Inactive:      2591060 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      29721316 kB
SwapTotal:    67111528 kB
SwapFree:     67111352 kB
Dirty:            3592 kB
Writeback:           0 kB
AnonPages:       65800 kB
Mapped:          14500 kB
Slab:           453884 kB
PageTables:       4852 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   189692 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71120 MiB
End job on node135 at 2011-04-29 21:11:31