Trace number 3370912

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.34 1203.69

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m62-q1327.5-n60-p15-s1351253.cnf
MD5SUMb23c52475e3d098cb9d5a92c025d6982
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark7.18991
Satisfiable
(Un)Satisfiability was proved
Number of variables3420
Number of clauses1120760
Sum of the clauses size2244820
Maximum clause length62
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 21120700
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 560

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3370912-1305433026/watcher-3370912-1305433026 -o /tmp/evaluation-result-3370912-1305433026/solver-3370912-1305433026 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3370912-1305433026.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.71 5.75 6.13 5/328 1150
/proc/meminfo: memFree=24470056/32951124 swapFree=44485764/67111528
[pid=1150] ppid=1148 vsize=9144 CPUtime=0 cores=1,3,5,7
/proc/1150/stat : 1150 (plingeling) R 1148 1150 390 0 -1 4202496 348 1256 0 0 0 0 0 0 21 0 1 0 654903121 9363456 278 33554432000 4194304 4417452 140737144465344 18446744073709551615 244446641931 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/1150/statm: 2286 293 147 55 0 172 0

[startup+0.0473101 s]
/proc/loadavg: 7.71 5.75 6.13 5/328 1150
/proc/meminfo: memFree=24470056/32951124 swapFree=44485764/67111528
[pid=1150] ppid=1148 vsize=14336 CPUtime=0.04 cores=1,3,5,7
/proc/1150/stat : 1150 (plingeling) R 1148 1150 390 0 -1 4202496 1709 1256 0 0 4 0 0 0 21 0 1 0 654903121 14680064 1607 33554432000 4194304 4417452 140737144465344 18446744073709551615 4322385 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/1150/statm: 3584 1607 148 55 0 1470 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 14336

[startup+0.100299 s]
/proc/loadavg: 7.71 5.75 6.13 5/328 1150
/proc/meminfo: memFree=24470056/32951124 swapFree=44485764/67111528
[pid=1150] ppid=1148 vsize=22528 CPUtime=0.09 cores=1,3,5,7
/proc/1150/stat : 1150 (plingeling) R 1148 1150 390 0 -1 4202496 3430 1256 0 0 9 0 0 0 21 0 1 0 654903121 23068672 3328 33554432000 4194304 4417452 140737144465344 18446744073709551615 244446639197 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/1150/statm: 5632 3352 148 55 0 3518 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 22528

[startup+0.300265 s]
/proc/loadavg: 7.71 5.75 6.13 5/328 1150
/proc/meminfo: memFree=24470056/32951124 swapFree=44485764/67111528
[pid=1150] ppid=1148 vsize=43008 CPUtime=0.29 cores=1,3,5,7
/proc/1150/stat : 1150 (plingeling) R 1148 1150 390 0 -1 4202496 8877 1256 0 0 28 1 0 0 22 0 1 0 654903121 44040192 8775 33554432000 4194304 4417452 140737144465344 18446744073709551615 4227882 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/1150/statm: 10752 8775 148 55 0 8638 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 43008

[startup+0.700188 s]
/proc/loadavg: 7.71 5.75 6.13 5/328 1150
/proc/meminfo: memFree=24470056/32951124 swapFree=44485764/67111528
[pid=1150] ppid=1148 vsize=141312 CPUtime=0.68 cores=1,3,5,7
/proc/1150/stat : 1150 (plingeling) R 1148 1150 390 0 -1 4202496 33453 1256 0 0 64 4 0 0 23 0 1 0 654903121 144703488 33351 33554432000 4194304 4417452 140737144465344 18446744073709551615 244446945024 0 0 4096 17442 0 0 0 17 5 0 0 0
/proc/1150/statm: 35328 33351 148 55 0 33214 0
Current children cumulated CPU time (s) 0.68
Current children cumulated vsize (KiB) 141312

[startup+1.501 s]
/proc/loadavg: 7.49 5.73 6.12 9/333 1160
/proc/meminfo: memFree=24331640/32951124 swapFree=44485764/67111528
[pid=1150] ppid=1148 vsize=193448 CPUtime=3.78 cores=1,3,5,7
/proc/1150/stat : 1150 (plingeling) S 1148 1150 390 0 -1 4202496 38756 1256 0 0 372 6 0 0 24 0 5 0 654903121 198090752 36144 33554432000 4194304 4417452 140737144465344 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/1150/statm: 48362 36144 175 55 0 46248 0
[pid=1150/tid=1157] ppid=1148 vsize=193448 CPUtime=0.76 cores=1,3,5,7
/proc/1150/task/1157/stat : 1157 (plingeling) R 1148 1150 390 0 -1 4202560 77 1256 0 0 76 0 0 0 25 0 5 0 654903195 198090752 36144 33554432000 4194304 4417452 140737144465344 18446744073709551615 4326427 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=1150/tid=1158] ppid=1148 vsize=196060 CPUtime=0.76 cores=1,3,5,7
/proc/1150/task/1158/stat : 1158 (plingeling) R 1148 1150 390 0 -1 4202560 189 1256 0 0 76 0 0 0 25 0 5 0 654903195 200765440 36144 33554432000 4194304 4417452 140737144465344 18446744073709551615 4265443 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=1150/tid=1159] ppid=1148 vsize=194752 CPUtime=0.75 cores=1,3,5,7
/proc/1150/task/1159/stat : 1159 (plingeling) R 1148 1150 390 0 -1 4202560 188 1256 0 0 75 0 0 0 25 0 5 0 654903195 199426048 36157 33554432000 4194304 4417452 140737144465344 18446744073709551615 4277491 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=1150/tid=1160] ppid=1148 vsize=194752 CPUtime=0.76 cores=1,3,5,7
/proc/1150/task/1160/stat : 1160 (plingeling) R 1148 1150 390 0 -1 4202560 4865 1256 0 0 75 1 0 0 25 0 5 0 654903195 199426048 36172 33554432000 4194304 4417452 140737144465344 18446744073709551615 244446639197 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3.78
Current children cumulated vsize (KiB) 193448

[startup+3.10071 s]
/proc/loadavg: 7.49 5.73 6.12 9/333 1160
/proc/meminfo: memFree=24294820/32951124 swapFree=44485764/67111528
[pid=1150] ppid=1148 vsize=298072 CPUtime=10.16 cores=1,3,5,7
/proc/1150/stat : 1150 (plingeling) S 1148 1150 390 0 -1 4202496 82594 1256 0 0 1004 12 0 0 24 0 5 0 654903121 305225728 45954 33554432000 4194304 4417452 140737144465344 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/1150/statm: 74518 45954 180 55 0 72404 0
[pid=1150/tid=1157] ppid=1148 vsize=298072 CPUtime=2.36 cores=1,3,5,7
/proc/1150/task/1157/stat : 1157 (plingeling) R 1148 1150 390 0 -1 4202560 13365 1256 0 0 235 1 0 0 25 0 5 0 654903195 305225728 45954 33554432000 4194304 4417452 140737144465344 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=1150/tid=1158] ppid=1148 vsize=298072 CPUtime=2.35 cores=1,3,5,7
/proc/1150/task/1158/stat : 1158 (plingeling) R 1148 1150 390 0 -1 4202560 12962 1256 0 0 234 1 0 0 20 0 5 0 654903195 305225728 45954 33554432000 4194304 4417452 140737144465344 18446744073709551615 4249584 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=1150/tid=1159] ppid=1148 vsize=298072 CPUtime=2.33 cores=1,3,5,7
/proc/1150/task/1159/stat : 1159 (plingeling) R 1148 1150 390 0 -1 4202560 9287 1256 0 0 232 1 0 0 25 0 5 0 654903195 305225728 45954 33554432000 4194304 4417452 140737144465344 18446744073709551615 4235183 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=1150/tid=1160] ppid=1148 vsize=298072 CPUtime=2.36 cores=1,3,5,7
/proc/1150/task/1160/stat : 1160 (plingeling) R 1148 1150 390 0 -1 4202560 13515 1256 0 0 234 2 0 0 25 0 5 0 654903195 305225728 45954 33554432000 4194304 4417452 140737144465344 18446744073709551615 4364466 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 10.16
Current children cumulated vsize (KiB) 298072

[startup+6.30111 s]
/proc/loadavg: 7.53 5.77 6.13 9/333 1160
/proc/meminfo: memFree=24216452/32951124 swapFree=44485764/67111528
[pid=1150] ppid=1148 vsize=349672 CPUtime=22.93 cores=1,3,5,7
/proc/1150/stat : 1150 (plingeling) S 1148 1150 390 0 -1 4202496 98647 1256 0 0 2280 13 0 0 24 0 5 0 654903121 358064128 57765 33554432000 4194304 4417452 140737144465344 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/1150/statm: 87418 57765 180 55 0 85304 0
[pid=1150/tid=1157] ppid=1148 vsize=349672 CPUtime=5.56 cores=1,3,5,7
/proc/1150/task/1157/stat : 1157 (plingeling) R 1148 1150 390 0 -1 4202560 13365 1256 0 0 555 1 0 0 25 0 5 0 654903195 358064128 57765 33554432000 4194304 4417452 140737144465344 18446744073709551615 4249181 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=1150/tid=1158] ppid=1148 vsize=349672 CPUtime=5.55 cores=1,3,5,7
/proc/1150/task/1158/stat : 1158 (plingeling) R 1148 1150 390 0 -1 4202560 12962 1256 0 0 554 1 0 0 25 0 5 0 654903195 358064128 57765 33554432000 4194304 4417452 140737144465344 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=1150/tid=1159] ppid=1148 vsize=349672 CPUtime=5.5 cores=1,3,5,7
/proc/1150/task/1159/stat : 1159 (plingeling) R 1148 1150 390 0 -1 4202560 9287 1256 0 0 549 1 0 0 25 0 5 0 654903195 358064128 57765 33554432000 4194304 4417452 140737144465344 18446744073709551615 4235378 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=1150/tid=1160] ppid=1148 vsize=349672 CPUtime=5.56 cores=1,3,5,7
/proc/1150/task/1160/stat : 1160 (plingeling) R 1148 1150 390 0 -1 4202560 29568 1256 0 0 553 3 0 0 25 0 5 0 654903195 358064128 57765 33554432000 4194304 4417452 140737144465344 18446744073709551615 4320983 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 22.93
Current children cumulated vsize (KiB) 349672

[startup+12.7009 s]
/proc/loadavg: 7.57 5.81 6.14 9/333 1160
/proc/meminfo: memFree=24139976/32951124 swapFree=44485764/67111528
[pid=1150] ppid=1148 vsize=380792 CPUtime=48.46 cores=1,3,5,7
/proc/1150/stat : 1150 (plingeling) S 1148 1150 390 0 -1 4202496 131088 1256 0 0 4829 17 0 0 24 0 5 0 654903121 389931008 66634 33554432000 4194304 4417452 140737144465344 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/1150/statm: 95198 66634 180 55 0 93084 0
[pid=1150/tid=1157] ppid=1148 vsize=380792 CPUtime=11.96 cores=1,3,5,7
/proc/1150/task/1157/stat : 1157 (plingeling) R 1148 1150 390 0 -1 4202560 16845 1256 0 0 1194 2 0 0 25 0 5 0 654903195 389931008 66634 33554432000 4194304 4417452 140737144465344 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=1150/tid=1158] ppid=1148 vsize=380792 CPUtime=11.95 cores=1,3,5,7
/proc/1150/task/1158/stat : 1158 (plingeling) R 1148 1150 390 0 -1 4202560 12962 1256 0 0 1194 1 0 0 25 0 5 0 654903195 389931008 66634 33554432000 4194304 4417452 140737144465344 18446744073709551615 4248999 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=1150/tid=1159] ppid=1148 vsize=380792 CPUtime=11.84 cores=1,3,5,7
/proc/1150/task/1159/stat : 1159 (plingeling) R 1148 1150 390 0 -1 4202560 9287 1256 0 0 1183 1 0 0 25 0 5 0 654903195 389931008 66634 33554432000 4194304 4417452 140737144465344 18446744073709551615 4250072 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=1150/tid=1160] ppid=1148 vsize=380792 CPUtime=11.96 cores=1,3,5,7
/proc/1150/task/1160/stat : 1160 (plingeling) R 1148 1150 390 0 -1 4202560 58529 1256 0 0 1189 7 0 0 25 0 5 0 654903195 389931008 66634 33554432000 4194304 4417452 140737144465344 18446744073709551615 4366051 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 48.46
Current children cumulated vsize (KiB) 380792

[startup+25.5005 s]
/proc/loadavg: 7.66 5.92 6.17 9/333 1160
/proc/meminfo: memFree=23972632/32951124 swapFree=44485764/67111528
[pid=1150] ppid=1148 vsize=423028 CPUtime=99.52 cores=1,3,5,7
/proc/1150/stat : 1150 (plingeling) S 1148 1150 390 0 -1 4202496 195821 1256 0 0 9924 28 0 0 24 0 5 0 654903121 433180672 77225 33554432000 4194304 4417452 140737144465344 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/1150/statm: 105757 77225 180 55 0 103643 0
[pid=1150/tid=1157] ppid=1148 vsize=423028 CPUtime=24.76 cores=1,3,5,7
/proc/1150/task/1157/stat : 1157 (plingeling) R 1148 1150 390 0 -1 4202560 16845 1256 0 0 2474 2 0 0 25 0 5 0 654903195 433180672 77225 33554432000 4194304 4417452 140737144465344 18446744073709551615 4307540 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=1150/tid=1158] ppid=1148 vsize=423028 CPUtime=24.74 cores=1,3,5,7
/proc/1150/task/1158/stat : 1158 (plingeling) R 1148 1150 390 0 -1 4202560 12962 1256 0 0 2473 1 0 0 25 0 5 0 654903195 433180672 77225 33554432000 4194304 4417452 140737144465344 18446744073709551615 4284677 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=1150/tid=1159] ppid=1148 vsize=423028 CPUtime=24.5 cores=1,3,5,7
/proc/1150/task/1159/stat : 1159 (plingeling) R 1148 1150 390 0 -1 4202560 13383 1256 0 0 2448 2 0 0 25 0 5 0 654903195 433180672 77225 33554432000 4194304 4417452 140737144465344 18446744073709551615 4258491 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=1150/tid=1160] ppid=1148 vsize=423028 CPUtime=24.76 cores=1,3,5,7
/proc/1150/task/1160/stat : 1160 (plingeling) R 1148 1150 390 0 -1 4202560 119166 1256 0 0 2459 17 0 0 25 0 5 0 654903195 433180672 77225 33554432000 4194304 4417452 140737144465344 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 99.52
Current children cumulated vsize (KiB) 423028

[startup+51.1008 s]
/proc/loadavg: 7.92 6.11 6.23 9/333 1161
/proc/meminfo: memFree=24140532/32951124 swapFree=44485764/67111528
[pid=1150] ppid=1148 vsize=594792 CPUtime=201.66 cores=1,3,5,7
/proc/1150/stat : 1150 (plingeling) S 1148 1150 390 0 -1 4202496 262505 1256 0 0 20129 37 0 0 24 0 5 0 654903121 609067008 120134 33554432000 4194304 4417452 140737144465344 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/1150/statm: 148698 120134 180 55 0 146584 0
[pid=1150/tid=1157] ppid=1148 vsize=594792 CPUtime=50.36 cores=1,3,5,7
/proc/1150/task/1157/stat : 1157 (plingeling) R 1148 1150 390 0 -1 4202560 16845 1256 0 0 5034 2 0 0 25 0 5 0 654903195 609067008 120134 33554432000 4194304 4417452 140737144465344 18446744073709551615 4254893 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=1150/tid=1158] ppid=1148 vsize=594792 CPUtime=50.35 cores=1,3,5,7
/proc/1150/task/1158/stat : 1158 (plingeling) R 1148 1150 390 0 -1 4202560 12962 1256 0 0 5034 1 0 0 25 0 5 0 654903195 609067008 120134 33554432000 4194304 4417452 140737144465344 18446744073709551615 4254810 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=1150/tid=1159] ppid=1148 vsize=594792 CPUtime=49.84 cores=1,3,5,7
/proc/1150/task/1159/stat : 1159 (plingeling) R 1148 1150 390 0 -1 4202560 17479 1256 0 0 4982 2 0 0 25 0 5 0 654903195 609067008 120134 33554432000 4194304 4417452 140737144465344 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=1150/tid=1160] ppid=1148 vsize=594792 CPUtime=50.36 cores=1,3,5,7
/proc/1150/task/1160/stat : 1160 (plingeling) R 1148 1150 390 0 -1 4202560 181754 1256 0 0 5011 25 0 0 25 0 5 0 654903195 609067008 120134 33554432000 4194304 4417452 140737144465344 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 201.66
Current children cumulated vsize (KiB) 594792

[startup+102.312 s]
/proc/loadavg: 7.96 6.40 6.32 9/333 1162
/proc/meminfo: memFree=23625140/32951124 swapFree=44485764/67111528
[pid=1150] ppid=1148 vsize=795628 CPUtime=405.99 cores=1,3,5,7
/proc/1150/stat : 1150 (plingeling) S 1148 1150 390 0 -1 4202496 341543 1256 0 0 40555 44 0 0 24 0 5 0 654903121 814723072 170343 33554432000 4194304 4417452 140737144465344 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/1150/statm: 198907 170343 180 55 0 196793 0
[pid=1150/tid=1157] ppid=1148 vsize=795628 CPUtime=101.57 cores=1,3,5,7
/proc/1150/task/1157/stat : 1157 (plingeling) R 1148 1150 390 0 -1 4202560 16845 1256 0 0 10155 2 0 0 25 0 5 0 654903195 814723072 170343 33554432000 4194304 4417452 140737144465344 18446744073709551615 4258206 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=1150/tid=1158] ppid=1148 vsize=795628 CPUtime=101.55 cores=1,3,5,7
/proc/1150/task/1158/stat : 1158 (plingeling) R 1148 1150 390 0 -1 4202560 12962 1256 0 0 10154 1 0 0 25 0 5 0 654903195 814723072 170343 33554432000 4194304 4417452 140737144465344 18446744073709551615 4236255 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=1150/tid=1159] ppid=1148 vsize=795628 CPUtime=100.54 cores=1,3,5,7
/proc/1150/task/1159/stat : 1159 (plingeling) R 1148 1150 390 0 -1 4202560 17479 1256 0 0 10052 2 0 0 25 0 5 0 654903195 814723072 170343 33554432000 4194304 4417452 140737144465344 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=1150/tid=1160] ppid=1148 vsize=795628 CPUtime=101.57 cores=1,3,5,7
/proc/1150/task/1160/stat : 1160 (plingeling) R 1148 1150 390 0 -1 4202560 260792 1256 0 0 10124 33 0 0 25 0 5 0 654903195 814723072 170343 33554432000 4194304 4417452 140737144465344 18446744073709551615 4365004 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 405.99
Current children cumulated vsize (KiB) 795628

[startup+162.301 s]
/proc/loadavg: 8.13 6.72 6.43 9/333 1164
/proc/meminfo: memFree=23391112/32951124 swapFree=44485764/67111528
[pid=1150] ppid=1148 vsize=767912 CPUtime=645.35 cores=1,3,5,7
/proc/1150/stat : 1150 (plingeling) S 1148 1150 390 0 -1 4202496 419206 1256 0 0 64482 53 0 0 24 0 5 0 654903121 786341888 163414 33554432000 4194304 4417452 140737144465344 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/1150/statm: 191978 163414 180 55 0 189864 0
[pid=1150/tid=1157] ppid=1148 vsize=767912 CPUtime=161.56 cores=1,3,5,7
/proc/1150/task/1157/stat : 1157 (plingeling) R 1148 1150 390 0 -1 4202560 16845 1256 0 0 16154 2 0 0 25 0 5 0 654903195 786341888 163414 33554432000 4194304 4417452 140737144465344 18446744073709551615 4364861 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=1150/tid=1158] ppid=1148 vsize=767912 CPUtime=161.54 cores=1,3,5,7
/proc/1150/task/1158/stat : 1158 (plingeling) R 1148 1150 390 0 -1 4202560 12962 1256 0 0 16153 1 0 0 25 0 5 0 654903195 786341888 163414 33554432000 4194304 4417452 140737144465344 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=1150/tid=1159] ppid=1148 vsize=767912 CPUtime=159.93 cores=1,3,5,7
/proc/1150/task/1159/stat : 1159 (plingeling) R 1148 1150 390 0 -1 4202560 17479 1256 0 0 15991 2 0 0 25 0 5 0 654903195 786341888 163414 33554432000 4194304 4417452 140737144465344 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=1150/tid=1160] ppid=1148 vsize=767912 CPUtime=161.56 cores=1,3,5,7
/proc/1150/task/1160/stat : 1160 (plingeling) R 1148 1150 390 0 -1 4202560 338455 1256 0 0 16114 42 0 0 25 0 5 0 654903195 786341888 163414 33554432000 4194304 4417452 140737144465344 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 645.35
Current children cumulated vsize (KiB) 767912


################
# More data... #
################

[pid=1150/tid=1158] ppid=1148 vsize=1716076 CPUtime=701.56 cores=1,3,5,7
/proc/1150/task/1158/stat : 1158 (plingeling) R 1148 1150 390 0 -1 4202560 13696 1256 0 0 70155 1 0 0 25 0 5 0 654903195 1757261824 400455 33554432000 4194304 4417452 140737144465344 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=1150/tid=1159] ppid=1148 vsize=1716076 CPUtime=698.84 cores=1,3,5,7
/proc/1150/task/1159/stat : 1159 (plingeling) R 1148 1150 390 0 -1 4202560 21541 1256 0 0 69881 3 0 0 25 0 5 0 654903195 1757261824 400455 33554432000 4194304 4417452 140737144465344 18446744073709551615 4336304 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=1150/tid=1160] ppid=1148 vsize=1716076 CPUtime=697.26 cores=1,3,5,7
/proc/1150/task/1160/stat : 1160 (plingeling) R 1148 1150 390 0 -1 4202560 945004 1256 0 0 69598 128 0 0 25 0 5 0 654903195 1757261824 400455 33554432000 4194304 4417452 140737144465344 18446744073709551615 4364650 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2799.99
Current children cumulated vsize (KiB) 1716076

[startup+762.312 s]
/proc/loadavg: 8.09 7.91 7.21 9/333 1180
/proc/meminfo: memFree=21922824/32951124 swapFree=44485764/67111528
[pid=1150] ppid=1148 vsize=1847152 CPUtime=3039.45 cores=1,3,5,7
/proc/1150/stat : 1150 (plingeling) S 1148 1150 390 0 -1 4202496 1070109 1256 0 0 303799 146 0 0 24 0 5 0 654903121 1891483648 433224 33554432000 4194304 4417452 140737144465344 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/1150/statm: 461788 433224 180 55 0 459674 0
[pid=1150/tid=1157] ppid=1148 vsize=1847152 CPUtime=761.59 cores=1,3,5,7
/proc/1150/task/1157/stat : 1157 (plingeling) R 1148 1150 390 0 -1 4202560 23634 1256 0 0 76157 2 0 0 25 0 5 0 654903195 1891483648 433224 33554432000 4194304 4417452 140737144465344 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=1150/tid=1158] ppid=1148 vsize=1847152 CPUtime=761.57 cores=1,3,5,7
/proc/1150/task/1158/stat : 1158 (plingeling) R 1148 1150 390 0 -1 4202560 13696 1256 0 0 76156 1 0 0 25 0 5 0 654903195 1891483648 433224 33554432000 4194304 4417452 140737144465344 18446744073709551615 4258263 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=1150/tid=1159] ppid=1148 vsize=1847152 CPUtime=758.85 cores=1,3,5,7
/proc/1150/task/1159/stat : 1159 (plingeling) R 1148 1150 390 0 -1 4202560 21541 1256 0 0 75882 3 0 0 25 0 5 0 654903195 1891483648 433224 33554432000 4194304 4417452 140737144465344 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=1150/tid=1160] ppid=1148 vsize=1847152 CPUtime=756.67 cores=1,3,5,7
/proc/1150/task/1160/stat : 1160 (plingeling) R 1148 1150 390 0 -1 4202560 977773 1256 0 0 75534 133 0 0 25 0 5 0 654903195 1891483648 433224 33554432000 4194304 4417452 140737144465344 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3039.45
Current children cumulated vsize (KiB) 1847152

[startup+822.301 s]
/proc/loadavg: 8.18 7.97 7.27 9/333 1181
/proc/meminfo: memFree=21327324/32951124 swapFree=44485764/67111528
[pid=1150] ppid=1148 vsize=2174832 CPUtime=3278.81 cores=1,3,5,7
/proc/1150/stat : 1150 (plingeling) S 1148 1150 390 0 -1 4202496 1152029 1256 0 0 327724 157 0 0 24 0 5 0 654903121 2227027968 515144 33554432000 4194304 4417452 140737144465344 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/1150/statm: 543708 515144 180 55 0 541594 0
[pid=1150/tid=1157] ppid=1148 vsize=2174832 CPUtime=821.58 cores=1,3,5,7
/proc/1150/task/1157/stat : 1157 (plingeling) R 1148 1150 390 0 -1 4202560 23634 1256 0 0 82156 2 0 0 25 0 5 0 654903195 2227027968 515144 33554432000 4194304 4417452 140737144465344 18446744073709551615 4249935 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=1150/tid=1158] ppid=1148 vsize=2174832 CPUtime=821.56 cores=1,3,5,7
/proc/1150/task/1158/stat : 1158 (plingeling) R 1148 1150 390 0 -1 4202560 13696 1256 0 0 82155 1 0 0 25 0 5 0 654903195 2227027968 515144 33554432000 4194304 4417452 140737144465344 18446744073709551615 4274913 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=1150/tid=1159] ppid=1148 vsize=2174832 CPUtime=818.84 cores=1,3,5,7
/proc/1150/task/1159/stat : 1159 (plingeling) R 1148 1150 390 0 -1 4202560 21541 1256 0 0 81881 3 0 0 25 0 5 0 654903195 2227027968 515144 33554432000 4194304 4417452 140737144465344 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=1150/tid=1160] ppid=1148 vsize=2174832 CPUtime=816.07 cores=1,3,5,7
/proc/1150/task/1160/stat : 1160 (plingeling) R 1148 1150 390 0 -1 4202560 1059693 1256 0 0 81462 145 0 0 25 0 5 0 654903195 2227027968 515144 33554432000 4194304 4417452 140737144465344 18446744073709551615 4357260 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3278.81
Current children cumulated vsize (KiB) 2174832

[startup+882.301 s]
/proc/loadavg: 8.33 8.03 7.33 9/333 1183
/proc/meminfo: memFree=21183832/32951124 swapFree=44485764/67111528
[pid=1150] ppid=1148 vsize=2174832 CPUtime=3518.2 cores=1,3,5,7
/proc/1150/stat : 1150 (plingeling) S 1148 1150 390 0 -1 4202496 1152029 1256 0 0 351663 157 0 0 24 0 5 0 654903121 2227027968 515144 33554432000 4194304 4417452 140737144465344 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/1150/statm: 543708 515144 180 55 0 541594 0
[pid=1150/tid=1157] ppid=1148 vsize=2174832 CPUtime=881.58 cores=1,3,5,7
/proc/1150/task/1157/stat : 1157 (plingeling) R 1148 1150 390 0 -1 4202560 23634 1256 0 0 88156 2 0 0 25 0 5 0 654903195 2227027968 515144 33554432000 4194304 4417452 140737144465344 18446744073709551615 4366472 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=1150/tid=1158] ppid=1148 vsize=2174832 CPUtime=881.56 cores=1,3,5,7
/proc/1150/task/1158/stat : 1158 (plingeling) R 1148 1150 390 0 -1 4202560 13696 1256 0 0 88155 1 0 0 25 0 5 0 654903195 2227027968 515144 33554432000 4194304 4417452 140737144465344 18446744073709551615 4364684 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=1150/tid=1159] ppid=1148 vsize=2174832 CPUtime=878.84 cores=1,3,5,7
/proc/1150/task/1159/stat : 1159 (plingeling) R 1148 1150 390 0 -1 4202560 21541 1256 0 0 87881 3 0 0 25 0 5 0 654903195 2227027968 515144 33554432000 4194304 4417452 140737144465344 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=1150/tid=1160] ppid=1148 vsize=2174832 CPUtime=875.46 cores=1,3,5,7
/proc/1150/task/1160/stat : 1160 (plingeling) R 1148 1150 390 0 -1 4202560 1059693 1256 0 0 87401 145 0 0 25 0 5 0 654903195 2227027968 515144 33554432000 4194304 4417452 140737144465344 18446744073709551615 4364641 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3518.2
Current children cumulated vsize (KiB) 2174832

[startup+942.301 s]
/proc/loadavg: 8.19 8.04 7.38 9/333 1185
/proc/meminfo: memFree=20743592/32951124 swapFree=44485764/67111528
[pid=1150] ppid=1148 vsize=2305904 CPUtime=3757.61 cores=1,3,5,7
/proc/1150/stat : 1150 (plingeling) S 1148 1150 390 0 -1 4202496 1184797 1256 0 0 375601 160 0 0 24 0 5 0 654903121 2361245696 547912 33554432000 4194304 4417452 140737144465344 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/1150/statm: 576476 547912 180 55 0 574362 0
[pid=1150/tid=1157] ppid=1148 vsize=2305904 CPUtime=941.58 cores=1,3,5,7
/proc/1150/task/1157/stat : 1157 (plingeling) R 1148 1150 390 0 -1 4202560 23634 1256 0 0 94156 2 0 0 25 0 5 0 654903195 2361245696 547912 33554432000 4194304 4417452 140737144465344 18446744073709551615 4277864 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=1150/tid=1158] ppid=1148 vsize=2305904 CPUtime=941.57 cores=1,3,5,7
/proc/1150/task/1158/stat : 1158 (plingeling) R 1148 1150 390 0 -1 4202560 13696 1256 0 0 94156 1 0 0 25 0 5 0 654903195 2361245696 547912 33554432000 4194304 4417452 140737144465344 18446744073709551615 4258436 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=1150/tid=1159] ppid=1148 vsize=2305904 CPUtime=938.84 cores=1,3,5,7
/proc/1150/task/1159/stat : 1159 (plingeling) R 1148 1150 390 0 -1 4202560 21541 1256 0 0 93881 3 0 0 25 0 5 0 654903195 2361245696 547912 33554432000 4194304 4417452 140737144465344 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=1150/tid=1160] ppid=1148 vsize=2305904 CPUtime=934.86 cores=1,3,5,7
/proc/1150/task/1160/stat : 1160 (plingeling) R 1148 1150 390 0 -1 4202560 1092461 1256 0 0 93339 147 0 0 25 0 5 0 654903195 2361245696 547912 33554432000 4194304 4417452 140737144465344 18446744073709551615 4357329 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3757.61
Current children cumulated vsize (KiB) 2305904

[startup+1002.3 s]
/proc/loadavg: 8.19 8.06 7.42 9/333 1186
/proc/meminfo: memFree=21271428/32951124 swapFree=44485764/67111528
[pid=1150] ppid=1148 vsize=1620456 CPUtime=3997.01 cores=1,3,5,7
/proc/1150/stat : 1150 (plingeling) S 1148 1150 390 0 -1 4202496 1237857 1256 0 0 399532 169 0 0 24 0 5 0 654903121 1659346944 376550 33554432000 4194304 4417452 140737144465344 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/1150/statm: 405114 376550 180 55 0 403000 0
[pid=1150/tid=1157] ppid=1148 vsize=1620456 CPUtime=1001.58 cores=1,3,5,7
/proc/1150/task/1157/stat : 1157 (plingeling) R 1148 1150 390 0 -1 4202560 23634 1256 0 0 100156 2 0 0 25 0 5 0 654903195 1659346944 376550 33554432000 4194304 4417452 140737144465344 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=1150/tid=1158] ppid=1148 vsize=1620456 CPUtime=1001.57 cores=1,3,5,7
/proc/1150/task/1158/stat : 1158 (plingeling) R 1148 1150 390 0 -1 4202560 13696 1256 0 0 100156 1 0 0 25 0 5 0 654903195 1659346944 376550 33554432000 4194304 4417452 140737144465344 18446744073709551615 4249158 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=1150/tid=1159] ppid=1148 vsize=1620456 CPUtime=998.84 cores=1,3,5,7
/proc/1150/task/1159/stat : 1159 (plingeling) R 1148 1150 390 0 -1 4202560 21541 1256 0 0 99881 3 0 0 25 0 5 0 654903195 1659346944 376550 33554432000 4194304 4417452 140737144465344 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=1150/tid=1160] ppid=1148 vsize=1620456 CPUtime=994.25 cores=1,3,5,7
/proc/1150/task/1160/stat : 1160 (plingeling) R 1148 1150 390 0 -1 4202560 1145521 1256 0 0 99269 156 0 0 25 0 5 0 654903195 1659346944 376550 33554432000 4194304 4417452 140737144465344 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3997.01
Current children cumulated vsize (KiB) 1620456

[startup+1062.3 s]
/proc/loadavg: 8.07 8.05 7.46 9/333 1188
/proc/meminfo: memFree=20939316/32951124 swapFree=44485764/67111528
[pid=1150] ppid=1148 vsize=1865836 CPUtime=4236.4 cores=1,3,5,7
/proc/1150/stat : 1150 (plingeling) S 1148 1150 390 0 -1 4202496 1318179 1256 0 0 423463 177 0 0 24 0 5 0 654903121 1910616064 437895 33554432000 4194304 4417452 140737144465344 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/1150/statm: 466459 437895 180 55 0 464345 0
[pid=1150/tid=1157] ppid=1148 vsize=1865836 CPUtime=1061.59 cores=1,3,5,7
/proc/1150/task/1157/stat : 1157 (plingeling) R 1148 1150 390 0 -1 4202560 23634 1256 0 0 106157 2 0 0 25 0 5 0 654903195 1910616064 437895 33554432000 4194304 4417452 140737144465344 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=1150/tid=1158] ppid=1148 vsize=1865836 CPUtime=1061.57 cores=1,3,5,7
/proc/1150/task/1158/stat : 1158 (plingeling) R 1148 1150 390 0 -1 4202560 13696 1256 0 0 106156 1 0 0 25 0 5 0 654903195 1910616064 437895 33554432000 4194304 4417452 140737144465344 18446744073709551615 4364548 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=1150/tid=1159] ppid=1148 vsize=1865836 CPUtime=1058.84 cores=1,3,5,7
/proc/1150/task/1159/stat : 1159 (plingeling) R 1148 1150 390 0 -1 4202560 69036 1256 0 0 105875 9 0 0 25 0 5 0 654903195 1910616064 437895 33554432000 4194304 4417452 140737144465344 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=1150/tid=1160] ppid=1148 vsize=1865836 CPUtime=1053.65 cores=1,3,5,7
/proc/1150/task/1160/stat : 1160 (plingeling) R 1148 1150 390 0 -1 4202560 1178348 1256 0 0 105206 159 0 0 25 0 5 0 654903195 1910616064 437895 33554432000 4194304 4417452 140737144465344 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4236.4
Current children cumulated vsize (KiB) 1865836

[startup+1122.3 s]
/proc/loadavg: 5.91 7.48 7.30 5/323 1207
/proc/meminfo: memFree=23283984/32951124 swapFree=44485764/67111528
[pid=1150] ppid=1148 vsize=2040252 CPUtime=4475.8 cores=1,3,5,7
/proc/1150/stat : 1150 (plingeling) S 1148 1150 390 0 -1 4202496 1403793 1256 0 0 447394 186 0 0 24 0 5 0 654903121 2089218048 481499 33554432000 4194304 4417452 140737144465344 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/1150/statm: 510063 481499 180 55 0 507949 0
[pid=1150/tid=1157] ppid=1148 vsize=2040252 CPUtime=1121.59 cores=1,3,5,7
/proc/1150/task/1157/stat : 1157 (plingeling) R 1148 1150 390 0 -1 4202560 30123 1256 0 0 112156 3 0 0 25 0 5 0 654903195 2089218048 481499 33554432000 4194304 4417452 140737144465344 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=1150/tid=1158] ppid=1148 vsize=2040252 CPUtime=1121.57 cores=1,3,5,7
/proc/1150/task/1158/stat : 1158 (plingeling) R 1148 1150 390 0 -1 4202560 16775 1256 0 0 112156 1 0 0 25 0 5 0 654903195 2089218048 481499 33554432000 4194304 4417452 140737144465344 18446744073709551615 4253473 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=1150/tid=1159] ppid=1148 vsize=2040252 CPUtime=1118.83 cores=1,3,5,7
/proc/1150/task/1159/stat : 1159 (plingeling) R 1148 1150 390 0 -1 4202560 95814 1256 0 0 111871 12 0 0 25 0 5 0 654903195 2089218048 481499 33554432000 4194304 4417452 140737144465344 18446744073709551615 4249209 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=1150/tid=1160] ppid=1148 vsize=2040252 CPUtime=1113.04 cores=1,3,5,7
/proc/1150/task/1160/stat : 1160 (plingeling) R 1148 1150 390 0 -1 4202560 1227616 1256 0 0 111142 162 0 0 25 0 5 0 654903195 2089218048 481499 33554432000 4194304 4417452 140737144465344 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4475.8
Current children cumulated vsize (KiB) 2040252

[startup+1182.3 s]
/proc/loadavg: 4.76 6.86 7.10 5/323 1209
/proc/meminfo: memFree=23141604/32951124 swapFree=44485764/67111528
[pid=1150] ppid=1148 vsize=2182524 CPUtime=4715.22 cores=1,3,5,7
/proc/1150/stat : 1150 (plingeling) S 1148 1150 390 0 -1 4202496 1439361 1256 0 0 471331 191 0 0 24 0 5 0 654903121 2234904576 517067 33554432000 4194304 4417452 140737144465344 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/1150/statm: 545631 517067 180 55 0 543517 0
[pid=1150/tid=1157] ppid=1148 vsize=2182524 CPUtime=1181.59 cores=1,3,5,7
/proc/1150/task/1157/stat : 1157 (plingeling) R 1148 1150 390 0 -1 4202560 30123 1256 0 0 118156 3 0 0 25 0 5 0 654903195 2234904576 517067 33554432000 4194304 4417452 140737144465344 18446744073709551615 4249597 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=1150/tid=1158] ppid=1148 vsize=2182524 CPUtime=1181.57 cores=1,3,5,7
/proc/1150/task/1158/stat : 1158 (plingeling) R 1148 1150 390 0 -1 4202560 16775 1256 0 0 118156 1 0 0 25 0 5 0 654903195 2234904576 517067 33554432000 4194304 4417452 140737144465344 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=1150/tid=1159] ppid=1148 vsize=2182524 CPUtime=1178.83 cores=1,3,5,7
/proc/1150/task/1159/stat : 1159 (plingeling) R 1148 1150 390 0 -1 4202560 95814 1256 0 0 117871 12 0 0 25 0 5 0 654903195 2234904576 517067 33554432000 4194304 4417452 140737144465344 18446744073709551615 4249186 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=1150/tid=1160] ppid=1148 vsize=2182524 CPUtime=1172.45 cores=1,3,5,7
/proc/1150/task/1160/stat : 1160 (plingeling) R 1148 1150 390 0 -1 4202560 1263184 1256 0 0 117078 167 0 0 25 0 5 0 654903195 2234904576 517067 33554432000 4194304 4417452 140737144465344 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4715.22
Current children cumulated vsize (KiB) 2182524



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.6 s]
/proc/loadavg: 4.54 6.67 7.03 5/323 1210
/proc/meminfo: memFree=23141472/32951124 swapFree=44485764/67111528
[pid=1150] ppid=1148 vsize=2182524 CPUtime=4800.21 cores=1,3,5,7
/proc/1150/stat : 1150 (plingeling) S 1148 1150 390 0 -1 4202496 1439361 1256 0 0 479830 191 0 0 24 0 5 0 654903121 2234904576 517067 33554432000 4194304 4417452 140737144465344 18446744073709551615 244458747877 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/1150/statm: 545631 517067 180 55 0 543517 0
[pid=1150/tid=1157] ppid=1148 vsize=2182524 CPUtime=1202.9 cores=1,3,5,7
/proc/1150/task/1157/stat : 1157 (plingeling) R 1148 1150 390 0 -1 4202560 30123 1256 0 0 120287 3 0 0 25 0 5 0 654903195 2234904576 517067 33554432000 4194304 4417452 140737144465344 18446744073709551615 4258604 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=1150/tid=1158] ppid=1148 vsize=2182524 CPUtime=1202.88 cores=1,3,5,7
/proc/1150/task/1158/stat : 1158 (plingeling) R 1148 1150 390 0 -1 4202560 16775 1256 0 0 120287 1 0 0 25 0 5 0 654903195 2234904576 517067 33554432000 4194304 4417452 140737144465344 18446744073709551615 4249584 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=1150/tid=1159] ppid=1148 vsize=2182524 CPUtime=1200.14 cores=1,3,5,7
/proc/1150/task/1159/stat : 1159 (plingeling) R 1148 1150 390 0 -1 4202560 95814 1256 0 0 120002 12 0 0 25 0 5 0 654903195 2234904576 517067 33554432000 4194304 4417452 140737144465344 18446744073709551615 4249584 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=1150/tid=1160] ppid=1148 vsize=2182524 CPUtime=1193.53 cores=1,3,5,7
/proc/1150/task/1160/stat : 1160 (plingeling) R 1148 1150 390 0 -1 4202560 1263184 1256 0 0 119186 167 0 0 25 0 5 0 654903195 2234904576 517067 33554432000 4194304 4417452 140737144465344 18446744073709551615 4366065 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4800.21
Current children cumulated vsize (KiB) 2182524

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1203.69
CPU time (s): 4800.34
CPU user time (s): 4798.35
CPU system time (s): 1.9887
CPU usage (%): 398.801
Max. virtual memory (cumulated for all children) (KiB): 2305904

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4798.35
system time used= 1.9887
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1440618
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 108
involuntary context switches= 22375

runsolver used 3.43148 second user time and 8.67868 second system time

The end

Launcher Data 

Begin job on node105 at 2011-05-15 06:17:07
IDJOB=3370912
IDBENCH=88217
IDSOLVER=1848
FILE ID=node105/3370912-1305433026
RUNJOBID= node105-1305432876-971
PBS_JOBID= 13324613
Free space on /tmp= 73592 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m62-q1327.5-n60-p15-s1351253.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3370912-1305433026/watcher-3370912-1305433026 -o /tmp/evaluation-result-3370912-1305433026/solver-3370912-1305433026 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3370912-1305433026.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= b23c52475e3d098cb9d5a92c025d6982
RANDOM SEED=1480436693

node105.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5410.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      24470212 kB
Buffers:        287684 kB
Cached:        5994152 kB
SwapCached:      39656 kB
Active:        1515024 kB
Inactive:      6703536 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      24470212 kB
SwapTotal:    67111528 kB
SwapFree:     44485764 kB
Dirty:           14792 kB
Writeback:           0 kB
AnonPages:     1904528 kB
Mapped:          10832 kB
Slab:           129804 kB
PageTables:      67544 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 66111588 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73608 MiB
End job on node105 at 2011-05-15 06:37:12