Trace number 3370901

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 4800.15 1203.49

General information on the benchmark

Namecrafted/anton/SRHD-SGI/
srhd-sgi-m42-q585-n40-p15-s54275047.cnf
MD5SUM67948a6aed9ea09317abe3c8a9a27a03
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark2.77358
Satisfiable
(Un)Satisfiability was proved
Number of variables1590
Number of clauses272291
Sum of the clauses size546092
Maximum clause length42
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2272251
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 540

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3370901-1305433149/watcher-3370901-1305433149 -o /tmp/evaluation-result-3370901-1305433149/solver-3370901-1305433149 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3370901-1305433149.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.47 7.92 7.52 6/353 12432
/proc/meminfo: memFree=26153460/32951124 swapFree=53501236/67111528
[pid=12432] ppid=12430 vsize=10060 CPUtime=0 cores=1,3,5,7
/proc/12432/stat : 12432 (plingeling) R 12430 12432 11874 0 -1 4202496 607 1251 0 0 0 0 0 0 25 0 1 0 654917567 10301440 537 33554432000 4194304 4417452 140736726228192 18446744073709551615 216634515200 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/12432/statm: 2515 537 147 55 0 401 0

[startup+0.035796 s]
/proc/loadavg: 7.47 7.92 7.52 6/353 12432
/proc/meminfo: memFree=26153460/32951124 swapFree=53501236/67111528
[pid=12432] ppid=12430 vsize=13584 CPUtime=0.03 cores=1,3,5,7
/proc/12432/stat : 12432 (plingeling) R 12430 12432 11874 0 -1 4202496 1537 1251 0 0 3 0 0 0 25 0 1 0 654917567 13910016 1419 33554432000 4194304 4417452 140736726228192 18446744073709551615 4231377 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/12432/statm: 3396 1419 148 55 0 1282 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 13584

[startup+0.100787 s]
/proc/loadavg: 7.47 7.92 7.52 6/353 12432
/proc/meminfo: memFree=26153460/32951124 swapFree=53501236/67111528
[pid=12432] ppid=12430 vsize=25872 CPUtime=0.09 cores=1,3,5,7
/proc/12432/stat : 12432 (plingeling) R 12430 12432 11874 0 -1 4202496 4609 1251 0 0 9 0 0 0 25 0 1 0 654917567 26492928 4491 33554432000 4194304 4417452 140736726228192 18446744073709551615 216634134115 0 0 4096 17442 0 0 0 17 3 0 0 0
/proc/12432/statm: 6468 4491 148 55 0 4354 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 25872

[startup+0.300717 s]
/proc/loadavg: 7.47 7.92 7.52 6/353 12432
/proc/meminfo: memFree=26153460/32951124 swapFree=53501236/67111528
[pid=12432] ppid=12430 vsize=150560 CPUtime=0.66 cores=1,3,5,7
/proc/12432/stat : 12432 (plingeling) S 12430 12432 11874 0 -1 4202496 9496 1251 0 0 65 1 0 0 25 0 5 0 654917567 154173440 9093 33554432000 4194304 4417452 140736726228192 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/12432/statm: 37640 9093 174 55 0 35526 0
Current children cumulated CPU time (s) 0.66
Current children cumulated vsize (KiB) 150560

[startup+0.700649 s]
/proc/loadavg: 7.47 7.92 7.52 6/353 12432
/proc/meminfo: memFree=26153460/32951124 swapFree=53501236/67111528
[pid=12432] ppid=12430 vsize=161452 CPUtime=2.26 cores=1,3,5,7
/proc/12432/stat : 12432 (plingeling) S 12430 12432 11874 0 -1 4202496 20237 1251 0 0 223 3 0 0 25 0 5 0 654917567 165326848 11789 33554432000 4194304 4417452 140736726228192 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/12432/statm: 40363 11789 179 55 0 38249 0
Current children cumulated CPU time (s) 2.26
Current children cumulated vsize (KiB) 161452

[startup+1.5005 s]
/proc/loadavg: 7.47 7.92 7.52 9/358 12442
/proc/meminfo: memFree=26090304/32951124 swapFree=53501236/67111528
[pid=12432] ppid=12430 vsize=166140 CPUtime=5.44 cores=1,3,5,7
/proc/12432/stat : 12432 (plingeling) S 12430 12432 11874 0 -1 4202496 23161 1251 0 0 541 3 0 0 25 0 5 0 654917567 170127360 12961 33554432000 4194304 4417452 140736726228192 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/12432/statm: 41535 12961 179 55 0 39421 0
[pid=12432/tid=12439] ppid=12430 vsize=166140 CPUtime=1.32 cores=1,3,5,7
/proc/12432/task/12439/stat : 12439 (plingeling) R 12430 12432 11874 0 -1 4202560 2908 1251 0 0 132 0 0 0 25 0 5 0 654917585 170127360 12961 33554432000 4194304 4417452 140736726228192 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12432/tid=12440] ppid=12430 vsize=166140 CPUtime=1.31 cores=1,3,5,7
/proc/12432/task/12440/stat : 12440 (plingeling) R 12430 12432 11874 0 -1 4202560 2770 1251 0 0 131 0 0 0 25 0 5 0 654917585 170127360 12961 33554432000 4194304 4417452 140736726228192 18446744073709551615 4249222 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12432/tid=12441] ppid=12430 vsize=166140 CPUtime=1.31 cores=1,3,5,7
/proc/12432/task/12441/stat : 12441 (plingeling) R 12430 12432 11874 0 -1 4202560 2452 1251 0 0 131 0 0 0 25 0 5 0 654917585 170127360 12961 33554432000 4194304 4417452 140736726228192 18446744073709551615 4335976 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12432/tid=12442] ppid=12430 vsize=166140 CPUtime=1.3 cores=1,3,5,7
/proc/12432/task/12442/stat : 12442 (plingeling) R 12430 12432 11874 0 -1 4202560 6314 1251 0 0 129 1 0 0 25 0 5 0 654917585 170127360 12961 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364634 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 5.44
Current children cumulated vsize (KiB) 166140

[startup+3.10022 s]
/proc/loadavg: 7.47 7.92 7.52 9/358 12442
/proc/meminfo: memFree=26077656/32951124 swapFree=53501236/67111528
[pid=12432] ppid=12430 vsize=187280 CPUtime=11.83 cores=1,3,5,7
/proc/12432/stat : 12432 (plingeling) S 12430 12432 11874 0 -1 4202496 32725 1251 0 0 1179 4 0 0 25 0 5 0 654917567 191774720 18246 33554432000 4194304 4417452 140736726228192 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/12432/statm: 46820 18246 179 55 0 44706 0
[pid=12432/tid=12439] ppid=12430 vsize=187280 CPUtime=2.92 cores=1,3,5,7
/proc/12432/task/12439/stat : 12439 (plingeling) R 12430 12432 11874 0 -1 4202560 3506 1251 0 0 292 0 0 0 25 0 5 0 654917585 191774720 18246 33554432000 4194304 4417452 140736726228192 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12432/tid=12440] ppid=12430 vsize=187280 CPUtime=2.91 cores=1,3,5,7
/proc/12432/task/12440/stat : 12440 (plingeling) R 12430 12432 11874 0 -1 4202560 3585 1251 0 0 291 0 0 0 25 0 5 0 654917585 191774720 18246 33554432000 4194304 4417452 140736726228192 18446744073709551615 4258491 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12432/tid=12441] ppid=12430 vsize=187280 CPUtime=2.91 cores=1,3,5,7
/proc/12432/task/12441/stat : 12441 (plingeling) R 12430 12432 11874 0 -1 4202560 2452 1251 0 0 291 0 0 0 25 0 5 0 654917585 191774720 18246 33554432000 4194304 4417452 140736726228192 18446744073709551615 4250003 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12432/tid=12442] ppid=12430 vsize=187280 CPUtime=2.89 cores=1,3,5,7
/proc/12432/task/12442/stat : 12442 (plingeling) R 12430 12432 11874 0 -1 4202560 14465 1251 0 0 287 2 0 0 25 0 5 0 654917585 191774720 18246 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 11.83
Current children cumulated vsize (KiB) 187280

[startup+6.30066 s]
/proc/loadavg: 7.51 7.92 7.52 9/358 12442
/proc/meminfo: memFree=26026084/32951124 swapFree=53501236/67111528
[pid=12432] ppid=12430 vsize=190216 CPUtime=24.59 cores=1,3,5,7
/proc/12432/stat : 12432 (plingeling) S 12430 12432 11874 0 -1 4202496 43922 1251 0 0 2453 6 0 0 25 0 5 0 654917567 194781184 18980 33554432000 4194304 4417452 140736726228192 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/12432/statm: 47554 18980 179 55 0 45440 0
[pid=12432/tid=12439] ppid=12430 vsize=190216 CPUtime=6.12 cores=1,3,5,7
/proc/12432/task/12439/stat : 12439 (plingeling) R 12430 12432 11874 0 -1 4202560 3506 1251 0 0 612 0 0 0 25 0 5 0 654917585 194781184 18980 33554432000 4194304 4417452 140736726228192 18446744073709551615 4249597 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12432/tid=12440] ppid=12430 vsize=190216 CPUtime=6.11 cores=1,3,5,7
/proc/12432/task/12440/stat : 12440 (plingeling) R 12430 12432 11874 0 -1 4202560 3585 1251 0 0 611 0 0 0 25 0 5 0 654917585 194781184 18980 33554432000 4194304 4417452 140736726228192 18446744073709551615 4258627 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12432/tid=12441] ppid=12430 vsize=190216 CPUtime=6.11 cores=1,3,5,7
/proc/12432/task/12441/stat : 12441 (plingeling) R 12430 12432 11874 0 -1 4202560 2643 1251 0 0 611 0 0 0 21 0 5 0 654917585 194781184 18980 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12432/tid=12442] ppid=12430 vsize=190216 CPUtime=6.06 cores=1,3,5,7
/proc/12432/task/12442/stat : 12442 (plingeling) R 12430 12432 11874 0 -1 4202560 25471 1251 0 0 602 4 0 0 25 0 5 0 654917585 194781184 18980 33554432000 4194304 4417452 140736726228192 18446744073709551615 4365050 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 24.59
Current children cumulated vsize (KiB) 190216

[startup+12.7005 s]
/proc/loadavg: 7.55 7.92 7.52 9/358 12478
/proc/meminfo: memFree=25947888/32951124 swapFree=53501236/67111528
[pid=12432] ppid=12430 vsize=226356 CPUtime=50.12 cores=1,3,5,7
/proc/12432/stat : 12432 (plingeling) S 12430 12432 11874 0 -1 4202496 63907 1251 0 0 5003 9 0 0 25 0 5 0 654917567 231788544 28007 33554432000 4194304 4417452 140736726228192 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/12432/statm: 56589 28007 179 55 0 54475 0
[pid=12432/tid=12439] ppid=12430 vsize=226356 CPUtime=12.52 cores=1,3,5,7
/proc/12432/task/12439/stat : 12439 (plingeling) R 12430 12432 11874 0 -1 4202560 3506 1251 0 0 1252 0 0 0 25 0 5 0 654917585 231788544 28007 33554432000 4194304 4417452 140736726228192 18446744073709551615 4249130 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12432/tid=12440] ppid=12430 vsize=226356 CPUtime=12.51 cores=1,3,5,7
/proc/12432/task/12440/stat : 12440 (plingeling) R 12430 12432 11874 0 -1 4202560 3585 1251 0 0 1251 0 0 0 25 0 5 0 654917585 231788544 28007 33554432000 4194304 4417452 140736726228192 18446744073709551615 4249165 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12432/tid=12441] ppid=12430 vsize=226356 CPUtime=12.51 cores=1,3,5,7
/proc/12432/task/12441/stat : 12441 (plingeling) R 12430 12432 11874 0 -1 4202560 3668 1251 0 0 1251 0 0 0 25 0 5 0 654917585 231788544 28007 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364827 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12432/tid=12442] ppid=12430 vsize=226356 CPUtime=12.38 cores=1,3,5,7
/proc/12432/task/12442/stat : 12442 (plingeling) R 12430 12432 11874 0 -1 4202560 44431 1251 0 0 1231 7 0 0 25 0 5 0 654917585 231788544 28007 33554432000 4194304 4417452 140736726228192 18446744073709551615 4365058 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.12
Current children cumulated vsize (KiB) 226356

[startup+25.5003 s]
/proc/loadavg: 7.65 7.92 7.53 9/358 12479
/proc/meminfo: memFree=25796308/32951124 swapFree=53501236/67111528
[pid=12432] ppid=12430 vsize=287172 CPUtime=101.18 cores=1,3,5,7
/proc/12432/stat : 12432 (plingeling) S 12430 12432 11874 0 -1 4202496 96865 1251 0 0 10104 14 0 0 25 0 5 0 654917567 294064128 43220 33554432000 4194304 4417452 140736726228192 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/12432/statm: 71793 43220 180 55 0 69679 0
[pid=12432/tid=12439] ppid=12430 vsize=287172 CPUtime=25.32 cores=1,3,5,7
/proc/12432/task/12439/stat : 12439 (plingeling) R 12430 12432 11874 0 -1 4202560 5798 1251 0 0 2532 0 0 0 25 0 5 0 654917585 294064128 43220 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12432/tid=12440] ppid=12430 vsize=287172 CPUtime=25.31 cores=1,3,5,7
/proc/12432/task/12440/stat : 12440 (plingeling) R 12430 12432 11874 0 -1 4202560 3586 1251 0 0 2531 0 0 0 25 0 5 0 654917585 294064128 43220 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364807 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12432/tid=12441] ppid=12430 vsize=287172 CPUtime=25.31 cores=1,3,5,7
/proc/12432/task/12441/stat : 12441 (plingeling) R 12430 12432 11874 0 -1 4202560 8813 1251 0 0 2531 0 0 0 25 0 5 0 654917585 294064128 43220 33554432000 4194304 4417452 140736726228192 18446744073709551615 4258394 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12432/tid=12442] ppid=12430 vsize=287172 CPUtime=25.04 cores=1,3,5,7
/proc/12432/task/12442/stat : 12442 (plingeling) R 12430 12432 11874 0 -1 4202560 69951 1251 0 0 2493 11 0 0 25 0 5 0 654917585 294064128 43220 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364629 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.18
Current children cumulated vsize (KiB) 287172

[startup+51.1008 s]
/proc/loadavg: 7.77 7.93 7.54 9/358 12479
/proc/meminfo: memFree=25620872/32951124 swapFree=53501236/67111528
[pid=12432] ppid=12430 vsize=387724 CPUtime=203.3 cores=1,3,5,7
/proc/12432/stat : 12432 (plingeling) S 12430 12432 11874 0 -1 4202496 158561 1251 0 0 20308 22 0 0 25 0 5 0 654917567 397029376 68358 33554432000 4194304 4417452 140736726228192 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/12432/statm: 96931 68358 180 55 0 94817 0
[pid=12432/tid=12439] ppid=12430 vsize=387724 CPUtime=50.92 cores=1,3,5,7
/proc/12432/task/12439/stat : 12439 (plingeling) R 12430 12432 11874 0 -1 4202560 6207 1251 0 0 5092 0 0 0 25 0 5 0 654917585 397029376 68358 33554432000 4194304 4417452 140736726228192 18446744073709551615 4258392 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12432/tid=12440] ppid=12430 vsize=387724 CPUtime=50.91 cores=1,3,5,7
/proc/12432/task/12440/stat : 12440 (plingeling) R 12430 12432 11874 0 -1 4202560 4404 1251 0 0 5091 0 0 0 25 0 5 0 654917585 397029376 68358 33554432000 4194304 4417452 140736726228192 18446744073709551615 4248888 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12432/tid=12441] ppid=12430 vsize=387724 CPUtime=50.91 cores=1,3,5,7
/proc/12432/task/12441/stat : 12441 (plingeling) R 12430 12432 11874 0 -1 4202560 21151 1251 0 0 5090 1 0 0 25 0 5 0 654917585 397029376 68358 33554432000 4194304 4417452 140736726228192 18446744073709551615 4366400 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12432/tid=12442] ppid=12430 vsize=387724 CPUtime=50.36 cores=1,3,5,7
/proc/12432/task/12442/stat : 12442 (plingeling) R 12430 12432 11874 0 -1 4202560 118082 1251 0 0 5018 18 0 0 25 0 5 0 654917585 397029376 68358 33554432000 4194304 4417452 140736726228192 18446744073709551615 4366385 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 203.3
Current children cumulated vsize (KiB) 387724

[startup+102.313 s]
/proc/loadavg: 7.90 7.94 7.56 9/358 12481
/proc/meminfo: memFree=25375812/32951124 swapFree=53501236/67111528
[pid=12432] ppid=12430 vsize=433144 CPUtime=407.59 cores=1,3,5,7
/proc/12432/stat : 12432 (plingeling) S 12430 12432 11874 0 -1 4202496 214944 1251 0 0 40729 30 0 0 25 0 5 0 654917567 443539456 79713 33554432000 4194304 4417452 140736726228192 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/12432/statm: 108286 79713 180 55 0 106172 0
[pid=12432/tid=12439] ppid=12430 vsize=433144 CPUtime=102.13 cores=1,3,5,7
/proc/12432/task/12439/stat : 12439 (plingeling) R 12430 12432 11874 0 -1 4202560 11550 1251 0 0 10212 1 0 0 25 0 5 0 654917585 443539456 79713 33554432000 4194304 4417452 140736726228192 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12432/tid=12440] ppid=12430 vsize=433144 CPUtime=102.13 cores=1,3,5,7
/proc/12432/task/12440/stat : 12440 (plingeling) R 12430 12432 11874 0 -1 4202560 13348 1251 0 0 10212 1 0 0 25 0 5 0 654917585 443539456 79713 33554432000 4194304 4417452 140736726228192 18446744073709551615 4258176 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12432/tid=12441] ppid=12430 vsize=433144 CPUtime=102.13 cores=1,3,5,7
/proc/12432/task/12441/stat : 12441 (plingeling) R 12430 12432 11874 0 -1 4202560 24790 1251 0 0 10211 2 0 0 25 0 5 0 654917585 443539456 79713 33554432000 4194304 4417452 140736726228192 18446744073709551615 4249186 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12432/tid=12442] ppid=12430 vsize=433144 CPUtime=101 cores=1,3,5,7
/proc/12432/task/12442/stat : 12442 (plingeling) R 12430 12432 11874 0 -1 4202560 156539 1251 0 0 10076 24 0 0 25 0 5 0 654917585 443539456 79713 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 407.59
Current children cumulated vsize (KiB) 433144

[startup+162.301 s]
/proc/loadavg: 7.96 7.94 7.59 9/358 12482
/proc/meminfo: memFree=25071588/32951124 swapFree=53501236/67111528
[pid=12432] ppid=12430 vsize=632724 CPUtime=646.9 cores=1,3,5,7
/proc/12432/stat : 12432 (plingeling) S 12430 12432 11874 0 -1 4202496 285187 1251 0 0 64650 40 0 0 25 0 5 0 654917567 647909376 129608 33554432000 4194304 4417452 140736726228192 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/12432/statm: 158181 129608 180 55 0 156067 0
[pid=12432/tid=12439] ppid=12430 vsize=632724 CPUtime=162.12 cores=1,3,5,7
/proc/12432/task/12439/stat : 12439 (plingeling) R 12430 12432 11874 0 -1 4202560 23432 1251 0 0 16209 3 0 0 25 0 5 0 654917585 647909376 129608 33554432000 4194304 4417452 140736726228192 18446744073709551615 4366126 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12432/tid=12440] ppid=12430 vsize=632724 CPUtime=162.12 cores=1,3,5,7
/proc/12432/task/12440/stat : 12440 (plingeling) R 12430 12432 11874 0 -1 4202560 19233 1251 0 0 16210 2 0 0 25 0 5 0 654917585 647909376 129608 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12432/tid=12441] ppid=12430 vsize=632724 CPUtime=162.11 cores=1,3,5,7
/proc/12432/task/12441/stat : 12441 (plingeling) R 12430 12432 11874 0 -1 4202560 45858 1251 0 0 16207 4 0 0 25 0 5 0 654917585 647909376 129608 33554432000 4194304 4417452 140736726228192 18446744073709551615 4307843 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12432/tid=12442] ppid=12430 vsize=632724 CPUtime=160.34 cores=1,3,5,7
/proc/12432/task/12442/stat : 12442 (plingeling) R 12430 12432 11874 0 -1 4202560 187947 1251 0 0 16006 28 0 0 25 0 5 0 654917585 647909376 129608 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 646.9
Current children cumulated vsize (KiB) 632724


################
# More data... #
################

[pid=12432/tid=12440] ppid=12430 vsize=1312988 CPUtime=702.11 cores=1,3,5,7
/proc/12432/task/12440/stat : 12440 (plingeling) R 12430 12432 11874 0 -1 4202560 186315 1251 0 0 70183 28 0 0 25 0 5 0 654917585 1344499712 299676 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364696 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12432/tid=12441] ppid=12430 vsize=1312988 CPUtime=702.12 cores=1,3,5,7
/proc/12432/task/12441/stat : 12441 (plingeling) R 12430 12432 11874 0 -1 4202560 243220 1251 0 0 70184 28 0 0 25 0 5 0 654917585 1344499712 299676 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12432/tid=12442] ppid=12430 vsize=1312988 CPUtime=694.46 cores=1,3,5,7
/proc/12432/task/12442/stat : 12442 (plingeling) R 12430 12432 11874 0 -1 4202560 324013 1251 0 0 69395 51 0 0 25 0 5 0 654917585 1344499712 299676 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2801.03
Current children cumulated vsize (KiB) 1312988

[startup+762.313 s]
/proc/loadavg: 8.00 8.00 7.78 9/358 12498
/proc/meminfo: memFree=22836564/32951124 swapFree=53501236/67111528
[pid=12432] ppid=12430 vsize=1320036 CPUtime=3040.41 cores=1,3,5,7
/proc/12432/stat : 12432 (plingeling) S 12430 12432 11874 0 -1 4202496 985072 1251 0 0 303896 145 0 0 25 0 5 0 654917567 1351716864 301438 33554432000 4194304 4417452 140736726228192 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/12432/statm: 330009 301438 180 55 0 327895 0
[pid=12432/tid=12439] ppid=12430 vsize=1320036 CPUtime=762.14 cores=1,3,5,7
/proc/12432/task/12439/stat : 12439 (plingeling) R 12430 12432 11874 0 -1 4202560 112645 1251 0 0 76195 19 0 0 25 0 5 0 654917585 1351716864 301438 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12432/tid=12440] ppid=12430 vsize=1320036 CPUtime=762.13 cores=1,3,5,7
/proc/12432/task/12440/stat : 12440 (plingeling) R 12430 12432 11874 0 -1 4202560 255515 1251 0 0 76173 40 0 0 25 0 5 0 654917585 1351716864 301438 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364550 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12432/tid=12441] ppid=12430 vsize=1320036 CPUtime=762.13 cores=1,3,5,7
/proc/12432/task/12441/stat : 12441 (plingeling) R 12430 12432 11874 0 -1 4202560 267796 1251 0 0 76182 31 0 0 25 0 5 0 654917585 1351716864 301438 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12432/tid=12442] ppid=12430 vsize=1320036 CPUtime=753.81 cores=1,3,5,7
/proc/12432/task/12442/stat : 12442 (plingeling) R 12430 12432 11874 0 -1 4202560 340399 1251 0 0 75329 52 0 0 25 0 5 0 654917585 1351716864 301438 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3040.41
Current children cumulated vsize (KiB) 1320036

[startup+822.301 s]
/proc/loadavg: 8.07 8.02 7.80 9/358 12500
/proc/meminfo: memFree=23191408/32951124 swapFree=53501236/67111528
[pid=12432] ppid=12430 vsize=1497868 CPUtime=3279.73 cores=1,3,5,7
/proc/12432/stat : 12432 (plingeling) S 12430 12432 11874 0 -1 4202496 1054106 1251 0 0 327817 156 0 0 25 0 5 0 654917567 1533816832 345896 33554432000 4194304 4417452 140736726228192 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/12432/statm: 374467 345896 180 55 0 372353 0
[pid=12432/tid=12439] ppid=12430 vsize=1497868 CPUtime=822.13 cores=1,3,5,7
/proc/12432/task/12439/stat : 12439 (plingeling) R 12430 12432 11874 0 -1 4202560 120837 1251 0 0 82192 21 0 0 25 0 5 0 654917585 1533816832 345896 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364829 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12432/tid=12440] ppid=12430 vsize=1497868 CPUtime=822.12 cores=1,3,5,7
/proc/12432/task/12440/stat : 12440 (plingeling) R 12430 12432 11874 0 -1 4202560 269668 1251 0 0 82169 43 0 0 25 0 5 0 654917585 1533816832 345896 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364821 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12432/tid=12441] ppid=12430 vsize=1497868 CPUtime=822.13 cores=1,3,5,7
/proc/12432/task/12441/stat : 12441 (plingeling) R 12430 12432 11874 0 -1 4202560 292372 1251 0 0 82178 35 0 0 25 0 5 0 654917585 1533816832 345896 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12432/tid=12442] ppid=12430 vsize=1497868 CPUtime=813.15 cores=1,3,5,7
/proc/12432/task/12442/stat : 12442 (plingeling) R 12430 12432 11874 0 -1 4202560 362512 1251 0 0 81260 55 0 0 25 0 5 0 654917585 1533816832 345896 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3279.73
Current children cumulated vsize (KiB) 1497868

[startup+882.3 s]
/proc/loadavg: 8.20 8.06 7.83 9/358 12501
/proc/meminfo: memFree=22774220/32951124 swapFree=53501236/67111528
[pid=12432] ppid=12430 vsize=1785708 CPUtime=3519.06 cores=1,3,5,7
/proc/12432/stat : 12432 (plingeling) S 12430 12432 11874 0 -1 4202496 1156441 1251 0 0 351736 170 0 0 25 0 5 0 654917567 1828564992 417856 33554432000 4194304 4417452 140736726228192 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/12432/statm: 446427 417856 180 55 0 444313 0
[pid=12432/tid=12439] ppid=12430 vsize=1785708 CPUtime=882.12 cores=1,3,5,7
/proc/12432/task/12439/stat : 12439 (plingeling) R 12430 12432 11874 0 -1 4202560 129634 1251 0 0 88191 21 0 0 25 0 5 0 654917585 1828564992 417856 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12432/tid=12440] ppid=12430 vsize=1785708 CPUtime=882.12 cores=1,3,5,7
/proc/12432/task/12440/stat : 12440 (plingeling) R 12430 12432 11874 0 -1 4202560 290812 1251 0 0 88166 46 0 0 25 0 5 0 654917585 1828564992 417856 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12432/tid=12441] ppid=12430 vsize=1785708 CPUtime=882.13 cores=1,3,5,7
/proc/12432/task/12441/stat : 12441 (plingeling) R 12430 12432 11874 0 -1 4202560 325755 1251 0 0 88174 39 0 0 25 0 5 0 654917585 1828564992 417856 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364737 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12432/tid=12442] ppid=12430 vsize=1785708 CPUtime=872.49 cores=1,3,5,7
/proc/12432/task/12442/stat : 12442 (plingeling) R 12430 12432 11874 0 -1 4202560 401523 1251 0 0 87187 62 0 0 25 0 5 0 654917585 1828564992 417856 33554432000 4194304 4417452 140736726228192 18446744073709551615 4366055 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3519.06
Current children cumulated vsize (KiB) 1785708

[startup+942.301 s]
/proc/loadavg: 8.21 8.08 7.84 9/358 12503
/proc/meminfo: memFree=22538844/32951124 swapFree=53501236/67111528
[pid=12432] ppid=12430 vsize=1869508 CPUtime=3758.41 cores=1,3,5,7
/proc/12432/stat : 12432 (plingeling) S 12430 12432 11874 0 -1 4202496 1201967 1251 0 0 375663 178 0 0 25 0 5 0 654917567 1914376192 438806 33554432000 4194304 4417452 140736726228192 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/12432/statm: 467377 438806 180 55 0 465263 0
[pid=12432/tid=12439] ppid=12430 vsize=1869508 CPUtime=942.13 cores=1,3,5,7
/proc/12432/task/12439/stat : 12439 (plingeling) R 12430 12432 11874 0 -1 4202560 129634 1251 0 0 94192 21 0 0 25 0 5 0 654917585 1914376192 438806 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364572 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12432/tid=12440] ppid=12430 vsize=1869508 CPUtime=942.12 cores=1,3,5,7
/proc/12432/task/12440/stat : 12440 (plingeling) R 12430 12432 11874 0 -1 4202560 295378 1251 0 0 94165 47 0 0 25 0 5 0 654917585 1914376192 438806 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12432/tid=12441] ppid=12430 vsize=1869508 CPUtime=942.12 cores=1,3,5,7
/proc/12432/task/12441/stat : 12441 (plingeling) R 12430 12432 11874 0 -1 4202560 350331 1251 0 0 94170 42 0 0 25 0 5 0 654917585 1914376192 438806 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364896 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12432/tid=12442] ppid=12430 vsize=1869508 CPUtime=931.84 cores=1,3,5,7
/proc/12432/task/12442/stat : 12442 (plingeling) R 12430 12432 11874 0 -1 4202560 417907 1251 0 0 93119 65 0 0 25 0 5 0 654917585 1914376192 438806 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3758.41
Current children cumulated vsize (KiB) 1869508

[startup+1002.3 s]
/proc/loadavg: 8.33 8.12 7.87 9/358 12505
/proc/meminfo: memFree=21998528/32951124 swapFree=53501236/67111528
[pid=12432] ppid=12430 vsize=1880148 CPUtime=3997.76 cores=1,3,5,7
/proc/12432/stat : 12432 (plingeling) S 12430 12432 11874 0 -1 4202496 1226543 1251 0 0 399593 183 0 0 25 0 5 0 654917567 1925271552 441466 33554432000 4194304 4417452 140736726228192 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/12432/statm: 470037 441466 180 55 0 467923 0
[pid=12432/tid=12439] ppid=12430 vsize=1880148 CPUtime=1002.13 cores=1,3,5,7
/proc/12432/task/12439/stat : 12439 (plingeling) R 12430 12432 11874 0 -1 4202560 129634 1251 0 0 100192 21 0 0 25 0 5 0 654917585 1925271552 441466 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364798 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12432/tid=12440] ppid=12430 vsize=1880148 CPUtime=1002.12 cores=1,3,5,7
/proc/12432/task/12440/stat : 12440 (plingeling) R 12430 12432 11874 0 -1 4202560 295378 1251 0 0 100165 47 0 0 25 0 5 0 654917585 1925271552 441466 33554432000 4194304 4417452 140736726228192 18446744073709551615 4346482 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12432/tid=12441] ppid=12430 vsize=1880148 CPUtime=1002.12 cores=1,3,5,7
/proc/12432/task/12441/stat : 12441 (plingeling) R 12430 12432 11874 0 -1 4202560 358523 1251 0 0 100169 43 0 0 25 0 5 0 654917585 1925271552 441466 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12432/tid=12442] ppid=12430 vsize=1880148 CPUtime=991.18 cores=1,3,5,7
/proc/12432/task/12442/stat : 12442 (plingeling) R 12430 12432 11874 0 -1 4202560 434291 1251 0 0 99049 69 0 0 25 0 5 0 654917585 1925271552 441466 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3997.76
Current children cumulated vsize (KiB) 1880148

[startup+1062.3 s]
/proc/loadavg: 8.24 8.13 7.89 9/358 12506
/proc/meminfo: memFree=21784452/32951124 swapFree=53501236/67111528
[pid=12432] ppid=12430 vsize=1940312 CPUtime=4237.11 cores=1,3,5,7
/proc/12432/stat : 12432 (plingeling) S 12430 12432 11874 0 -1 4202496 1276652 1251 0 0 423521 190 0 0 25 0 5 0 654917567 1986879488 456507 33554432000 4194304 4417452 140736726228192 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/12432/statm: 485078 456507 180 55 0 482964 0
[pid=12432/tid=12439] ppid=12430 vsize=1940312 CPUtime=1062.13 cores=1,3,5,7
/proc/12432/task/12439/stat : 12439 (plingeling) R 12430 12432 11874 0 -1 4202560 129794 1251 0 0 106191 22 0 0 25 0 5 0 654917585 1986879488 456507 33554432000 4194304 4417452 140736726228192 18446744073709551615 4336280 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12432/tid=12440] ppid=12430 vsize=1940312 CPUtime=1062.12 cores=1,3,5,7
/proc/12432/task/12440/stat : 12440 (plingeling) R 12430 12432 11874 0 -1 4202560 337135 1251 0 0 106160 52 0 0 25 0 5 0 654917585 1986879488 456507 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12432/tid=12441] ppid=12430 vsize=1940312 CPUtime=1062.12 cores=1,3,5,7
/proc/12432/task/12441/stat : 12441 (plingeling) R 12430 12432 11874 0 -1 4202560 366715 1251 0 0 106168 44 0 0 25 0 5 0 654917585 1986879488 456507 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12432/tid=12442] ppid=12430 vsize=1940312 CPUtime=1050.53 cores=1,3,5,7
/proc/12432/task/12442/stat : 12442 (plingeling) R 12430 12432 11874 0 -1 4202560 434291 1251 0 0 104984 69 0 0 25 0 5 0 654917585 1986879488 456507 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4237.11
Current children cumulated vsize (KiB) 1940312

[startup+1122.3 s]
/proc/loadavg: 8.20 8.14 7.90 9/358 12508
/proc/meminfo: memFree=21559348/32951124 swapFree=53501236/67111528
[pid=12432] ppid=12430 vsize=2006680 CPUtime=4476.46 cores=1,3,5,7
/proc/12432/stat : 12432 (plingeling) S 12430 12432 11874 0 -1 4202496 1326100 1251 0 0 447445 201 0 0 25 0 5 0 654917567 2054840320 473099 33554432000 4194304 4417452 140736726228192 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/12432/statm: 501670 473099 180 55 0 499556 0
[pid=12432/tid=12439] ppid=12430 vsize=2006680 CPUtime=1122.13 cores=1,3,5,7
/proc/12432/task/12439/stat : 12439 (plingeling) R 12430 12432 11874 0 -1 4202560 145905 1251 0 0 112188 25 0 0 25 0 5 0 654917585 2054840320 473099 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12432/tid=12440] ppid=12430 vsize=2006680 CPUtime=1122.13 cores=1,3,5,7
/proc/12432/task/12440/stat : 12440 (plingeling) R 12430 12432 11874 0 -1 4202560 353519 1251 0 0 112157 56 0 0 25 0 5 0 654917585 2054840320 473099 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12432/tid=12441] ppid=12430 vsize=2006680 CPUtime=1122.12 cores=1,3,5,7
/proc/12432/task/12441/stat : 12441 (plingeling) R 12430 12432 11874 0 -1 4202560 367284 1251 0 0 112168 44 0 0 25 0 5 0 654917585 2054840320 473099 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12432/tid=12442] ppid=12430 vsize=2006680 CPUtime=1109.86 cores=1,3,5,7
/proc/12432/task/12442/stat : 12442 (plingeling) R 12430 12432 11874 0 -1 4202560 450675 1251 0 0 110914 72 0 0 25 0 5 0 654917585 2054840320 473099 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4476.46
Current children cumulated vsize (KiB) 2006680

[startup+1182.3 s]
/proc/loadavg: 8.18 8.14 7.92 9/358 12509
/proc/meminfo: memFree=21813492/32951124 swapFree=53501236/67111528
[pid=12432] ppid=12430 vsize=1588988 CPUtime=4715.81 cores=1,3,5,7
/proc/12432/stat : 12432 (plingeling) S 12430 12432 11874 0 -1 4202496 1374998 1251 0 0 471371 210 0 0 25 0 5 0 654917567 1627123712 368676 33554432000 4194304 4417452 140736726228192 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/12432/statm: 397247 368676 180 55 0 395133 0
[pid=12432/tid=12439] ppid=12430 vsize=1588988 CPUtime=1182.13 cores=1,3,5,7
/proc/12432/task/12439/stat : 12439 (plingeling) R 12430 12432 11874 0 -1 4202560 160358 1251 0 0 118186 27 0 0 25 0 5 0 654917585 1627123712 368676 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12432/tid=12440] ppid=12430 vsize=1588988 CPUtime=1182.13 cores=1,3,5,7
/proc/12432/task/12440/stat : 12440 (plingeling) R 12430 12432 11874 0 -1 4202560 369903 1251 0 0 118154 59 0 0 25 0 5 0 654917585 1627123712 368676 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12432/tid=12441] ppid=12430 vsize=1588988 CPUtime=1182.13 cores=1,3,5,7
/proc/12432/task/12441/stat : 12441 (plingeling) R 12430 12432 11874 0 -1 4202560 368422 1251 0 0 118168 45 0 0 25 0 5 0 654917585 1627123712 368676 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12432/tid=12442] ppid=12430 vsize=1588988 CPUtime=1169.21 cores=1,3,5,7
/proc/12432/task/12442/stat : 12442 (plingeling) R 12430 12432 11874 0 -1 4202560 467598 1251 0 0 116845 76 0 0 25 0 5 0 654917585 1627123712 368676 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364616 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4715.81
Current children cumulated vsize (KiB) 1588988



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.41 s]
/proc/loadavg: 8.12 8.13 7.92 10/358 12510
/proc/meminfo: memFree=21692456/32951124 swapFree=53501236/67111528
[pid=12432] ppid=12430 vsize=1689480 CPUtime=4800.05 cores=1,3,5,7
/proc/12432/stat : 12432 (plingeling) S 12430 12432 11874 0 -1 4202496 1408313 1251 0 0 479791 214 0 0 25 0 5 0 654917567 1730027520 393799 33554432000 4194304 4417452 140736726228192 18446744073709551615 216646318053 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/12432/statm: 422370 393799 180 55 0 420256 0
[pid=12432/tid=12439] ppid=12430 vsize=1689480 CPUtime=1203.26 cores=1,3,5,7
/proc/12432/task/12439/stat : 12439 (plingeling) R 12430 12432 11874 0 -1 4202560 160358 1251 0 0 120299 27 0 0 25 0 5 0 654917585 1730027520 393799 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364756 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=12432/tid=12440] ppid=12430 vsize=1689480 CPUtime=1203.25 cores=1,3,5,7
/proc/12432/task/12440/stat : 12440 (plingeling) R 12430 12432 11874 0 -1 4202560 386287 1251 0 0 120264 61 0 0 25 0 5 0 654917585 1730027520 393799 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364533 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=12432/tid=12441] ppid=12430 vsize=1689480 CPUtime=1203.26 cores=1,3,5,7
/proc/12432/task/12441/stat : 12441 (plingeling) R 12430 12432 11874 0 -1 4202560 378889 1251 0 0 120280 46 0 0 25 0 5 0 654917585 1730027520 393799 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12432/tid=12442] ppid=12430 vsize=1689480 CPUtime=1190.09 cores=1,3,5,7
/proc/12432/task/12442/stat : 12442 (plingeling) R 12430 12432 11874 0 -1 4202560 474062 1251 0 0 118932 77 0 0 25 0 5 0 654917585 1730027520 393799 33554432000 4194304 4417452 140736726228192 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4800.05
Current children cumulated vsize (KiB) 1689480

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1203.49
CPU time (s): 4800.15
CPU user time (s): 4797.95
CPU system time (s): 2.20067
CPU usage (%): 398.852
Max. virtual memory (cumulated for all children) (KiB): 2066444

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4797.95
system time used= 2.20066
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1409564
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 75
involuntary context switches= 36634

runsolver used 3.57945 second user time and 9.57354 second system time

The end

Launcher Data 

Begin job on node103 at 2011-05-15 06:19:10
IDJOB=3370901
IDBENCH=88225
IDSOLVER=1848
FILE ID=node103/3370901-1305433149
RUNJOBID= node103-1305431944-12249
PBS_JOBID= 13324609
Free space on /tmp= 73044 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/crafted/anton/SRHD-SGI/srhd-sgi-m42-q585-n40-p15-s54275047.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3370901-1305433149/watcher-3370901-1305433149 -o /tmp/evaluation-result-3370901-1305433149/solver-3370901-1305433149 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3370901-1305433149.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 67948a6aed9ea09317abe3c8a9a27a03
RANDOM SEED=136001208

node103.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.71
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      26153616 kB
Buffers:        309116 kB
Cached:        3941020 kB
SwapCached:     340192 kB
Active:        1350196 kB
Inactive:      5176756 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      26153616 kB
SwapTotal:    67111528 kB
SwapFree:     53501236 kB
Dirty:            4336 kB
Writeback:           0 kB
AnonPages:     1947000 kB
Mapped:          10820 kB
Slab:           152048 kB
PageTables:      52496 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 78360476 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73040 MiB
End job on node103 at 2011-05-15 06:39:15