Trace number 3292841

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 4800.32 1201.75

General information on the benchmark

NameSAT02/industrial/aloul/
Homer/homer17.shuffled.cnf
MD5SUMca62f6128dc3cad7229c7b9ee33a7492
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark0.484925
Satisfiable
(Un)Satisfiability was proved
Number of variables286
Number of clauses1742
Sum of the clauses size3718
Maximum clause length11
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 21716
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 526

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3292841-1304110265/watcher-3292841-1304110265 -o /tmp/evaluation-result-3292841-1304110265/solver-3292841-1304110265 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3292841-1304110265.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.03 7.92 7.91 5/181 3699
/proc/meminfo: memFree=30394220/32950928 swapFree=67111404/67111528
[pid=3699] ppid=3697 vsize=49976 CPUtime=0 cores=1,3,5,7
/proc/3699/stat : 3699 (plingeling) S 3697 3699 2999 0 -1 4202496 374 1257 0 0 0 0 0 0 25 0 5 0 522625279 51175424 299 33554432000 4194304 4417332 140736962089664 18446744073709551615 235606669285 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/3699/statm: 12494 299 170 55 0 10380 0
[pid=3699/tid=3705] ppid=3697 vsize=49976 CPUtime=0 cores=1,3,5,7
/proc/3699/task/3705/stat : 3705 (plingeling) R 3697 3699 2999 0 -1 4202560 17 1257 0 0 0 0 0 0 25 0 5 0 522625280 51175424 299 33554432000 4194304 4417332 140736962089664 18446744073709551615 235594971679 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3699/tid=3706] ppid=3697 vsize=49976 CPUtime=0 cores=1,3,5,7
/proc/3699/task/3706/stat : 3706 (plingeling) R 3697 3699 2999 0 -1 4202560 0 1257 0 0 0 0 0 0 25 0 5 0 522625280 51175424 299 33554432000 4194304 4417332 140736962089664 18446744073709551615 235594922993 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3699/tid=3707] ppid=3697 vsize=49976 CPUtime=0 cores=1,3,5,7
/proc/3699/task/3707/stat : 3707 (plingeling) R 3697 3699 2999 0 -1 4202560 1 1257 0 0 0 0 0 0 25 0 5 0 522625280 51175424 300 33554432000 4194304 4417332 140736962089664 18446744073709551615 4199406 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3699/tid=3708] ppid=3697 vsize=49976 CPUtime=0 cores=1,3,5,7
/proc/3699/task/3708/stat : 3708 (plingeling) R 3697 3699 2999 0 -1 4202560 4 1257 0 0 0 0 0 0 25 0 5 0 522625280 51175424 300 33554432000 4194304 4417332 140736962089664 18446744073709551615 235594971630 0 0 4096 17442 0 0 0 -1 1 0 0 0

[startup+0.0322029 s]
/proc/loadavg: 8.03 7.92 7.91 5/181 3699
/proc/meminfo: memFree=30394220/32950928 swapFree=67111404/67111528
[pid=3699] ppid=3697 vsize=51384 CPUtime=0.1 cores=1,3,5,7
/proc/3699/stat : 3699 (plingeling) S 3697 3699 2999 0 -1 4202496 801 1257 0 0 10 0 0 0 25 0 5 0 522625279 52617216 652 33554432000 4194304 4417332 140736962089664 18446744073709551615 235606669285 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/3699/statm: 12846 652 181 55 0 10732 0
[pid=3699/tid=3705] ppid=3697 vsize=51384 CPUtime=0.02 cores=1,3,5,7
/proc/3699/task/3705/stat : 3705 (plingeling) R 3697 3699 2999 0 -1 4202560 123 1257 0 0 2 0 0 0 25 0 5 0 522625280 52617216 652 33554432000 4194304 4417332 140736962089664 18446744073709551615 4320992 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3699/tid=3706] ppid=3697 vsize=51384 CPUtime=0.02 cores=1,3,5,7
/proc/3699/task/3706/stat : 3706 (plingeling) R 3697 3699 2999 0 -1 4202560 55 1257 0 0 2 0 0 0 25 0 5 0 522625280 52617216 652 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364938 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3699/tid=3707] ppid=3697 vsize=51384 CPUtime=0.02 cores=1,3,5,7
/proc/3699/task/3707/stat : 3707 (plingeling) R 3697 3699 2999 0 -1 4202560 65 1257 0 0 2 0 0 0 25 0 5 0 522625280 52617216 652 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364722 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3699/tid=3708] ppid=3697 vsize=51384 CPUtime=0.02 cores=1,3,5,7
/proc/3699/task/3708/stat : 3708 (plingeling) R 3697 3699 2999 0 -1 4202560 206 1257 0 0 2 0 0 0 25 0 5 0 522625280 52617216 652 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364722 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 0.1
Current children cumulated vsize (KiB) 51384

[startup+0.132186 s]
/proc/loadavg: 8.03 7.92 7.91 5/181 3699
/proc/meminfo: memFree=30394220/32950928 swapFree=67111404/67111528
[pid=3699] ppid=3697 vsize=119624 CPUtime=0.5 cores=1,3,5,7
/proc/3699/stat : 3699 (plingeling) S 3697 3699 2999 0 -1 4202496 1848 1257 0 0 50 0 0 0 25 0 5 0 522625279 122494976 1342 33554432000 4194304 4417332 140736962089664 18446744073709551615 235606669285 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/3699/statm: 29906 1342 181 55 0 27792 0
[pid=3699/tid=3705] ppid=3697 vsize=119624 CPUtime=0.12 cores=1,3,5,7
/proc/3699/task/3705/stat : 3705 (plingeling) R 3697 3699 2999 0 -1 4202560 320 1257 0 0 12 0 0 0 25 0 5 0 522625280 122494976 1342 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364470 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3699/tid=3706] ppid=3697 vsize=119624 CPUtime=0.12 cores=1,3,5,7
/proc/3699/task/3706/stat : 3706 (plingeling) R 3697 3699 2999 0 -1 4202560 235 1257 0 0 12 0 0 0 25 0 5 0 522625280 122494976 1342 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364722 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3699/tid=3707] ppid=3697 vsize=119624 CPUtime=0.12 cores=1,3,5,7
/proc/3699/task/3707/stat : 3707 (plingeling) R 3697 3699 2999 0 -1 4202560 314 1257 0 0 12 0 0 0 25 0 5 0 522625280 122494976 1342 33554432000 4194304 4417332 140736962089664 18446744073709551615 4309899 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3699/tid=3708] ppid=3697 vsize=119624 CPUtime=0.12 cores=1,3,5,7
/proc/3699/task/3708/stat : 3708 (plingeling) R 3697 3699 2999 0 -1 4202560 627 1257 0 0 12 0 0 0 25 0 5 0 522625280 122494976 1342 33554432000 4194304 4417332 140736962089664 18446744073709551615 4365960 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 0.5
Current children cumulated vsize (KiB) 119624

[startup+0.301156 s]
/proc/loadavg: 8.03 7.92 7.91 5/181 3699
/proc/meminfo: memFree=30394220/32950928 swapFree=67111404/67111528
[pid=3699] ppid=3697 vsize=122716 CPUtime=1.17 cores=1,3,5,7
/proc/3699/stat : 3699 (plingeling) S 3697 3699 2999 0 -1 4202496 3872 1257 0 0 117 0 0 0 25 0 5 0 522625279 125661184 2115 33554432000 4194304 4417332 140736962089664 18446744073709551615 235606669285 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/3699/statm: 30679 2115 181 55 0 28565 0
[pid=3699/tid=3705] ppid=3697 vsize=122716 CPUtime=0.29 cores=1,3,5,7
/proc/3699/task/3705/stat : 3705 (plingeling) R 3697 3699 2999 0 -1 4202560 874 1257 0 0 29 0 0 0 25 0 5 0 522625280 125661184 2115 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364671 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3699/tid=3706] ppid=3697 vsize=122716 CPUtime=0.29 cores=1,3,5,7
/proc/3699/task/3706/stat : 3706 (plingeling) R 3697 3699 2999 0 -1 4202560 733 1257 0 0 29 0 0 0 25 0 5 0 522625280 125661184 2115 33554432000 4194304 4417332 140736962089664 18446744073709551615 4246406 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3699/tid=3707] ppid=3697 vsize=122716 CPUtime=0.29 cores=1,3,5,7
/proc/3699/task/3707/stat : 3707 (plingeling) R 3697 3699 2999 0 -1 4202560 824 1257 0 0 29 0 0 0 25 0 5 0 522625280 125661184 2115 33554432000 4194304 4417332 140736962089664 18446744073709551615 4237221 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3699/tid=3708] ppid=3697 vsize=122716 CPUtime=0.29 cores=1,3,5,7
/proc/3699/task/3708/stat : 3708 (plingeling) R 3697 3699 2999 0 -1 4202560 1089 1257 0 0 29 0 0 0 25 0 5 0 522625280 125661184 2115 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364599 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1.17
Current children cumulated vsize (KiB) 122716

[startup+0.701087 s]
/proc/loadavg: 8.03 7.92 7.91 5/181 3699
/proc/meminfo: memFree=30394220/32950928 swapFree=67111404/67111528
[pid=3699] ppid=3697 vsize=126204 CPUtime=2.77 cores=1,3,5,7
/proc/3699/stat : 3699 (plingeling) S 3697 3699 2999 0 -1 4202496 7472 1257 0 0 277 0 0 0 25 0 5 0 522625279 129232896 2997 33554432000 4194304 4417332 140736962089664 18446744073709551615 235606669285 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/3699/statm: 31551 2997 181 55 0 29437 0
[pid=3699/tid=3705] ppid=3697 vsize=126204 CPUtime=0.69 cores=1,3,5,7
/proc/3699/task/3705/stat : 3705 (plingeling) R 3697 3699 2999 0 -1 4202560 1876 1257 0 0 69 0 0 0 25 0 5 0 522625280 129232896 2997 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364713 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3699/tid=3706] ppid=3697 vsize=126204 CPUtime=0.69 cores=1,3,5,7
/proc/3699/task/3706/stat : 3706 (plingeling) R 3697 3699 2999 0 -1 4202560 1426 1257 0 0 69 0 0 0 25 0 5 0 522625280 129232896 2997 33554432000 4194304 4417332 140736962089664 18446744073709551615 4365065 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3699/tid=3707] ppid=3697 vsize=126204 CPUtime=0.69 cores=1,3,5,7
/proc/3699/task/3707/stat : 3707 (plingeling) R 3697 3699 2999 0 -1 4202560 1821 1257 0 0 69 0 0 0 25 0 5 0 522625280 129232896 2997 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364519 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3699/tid=3708] ppid=3697 vsize=126204 CPUtime=0.68 cores=1,3,5,7
/proc/3699/task/3708/stat : 3708 (plingeling) R 3697 3699 2999 0 -1 4202560 1997 1257 0 0 68 0 0 0 25 0 5 0 522625280 129232896 2997 33554432000 4194304 4417332 140736962089664 18446744073709551615 4356768 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2.77
Current children cumulated vsize (KiB) 126204

[startup+1.50095 s]
/proc/loadavg: 8.03 7.92 7.91 9/186 3709
/proc/meminfo: memFree=30348432/32950928 swapFree=67111404/67111528
[pid=3699] ppid=3697 vsize=128172 CPUtime=5.97 cores=1,3,5,7
/proc/3699/stat : 3699 (plingeling) S 3697 3699 2999 0 -1 4202496 15829 1257 0 0 595 2 0 0 25 0 5 0 522625279 131248128 3492 33554432000 4194304 4417332 140736962089664 18446744073709551615 235606669285 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/3699/statm: 32043 3492 181 55 0 29929 0
[pid=3699/tid=3705] ppid=3697 vsize=128172 CPUtime=1.49 cores=1,3,5,7
/proc/3699/task/3705/stat : 3705 (plingeling) R 3697 3699 2999 0 -1 4202560 4469 1257 0 0 149 0 0 0 25 0 5 0 522625280 131248128 3492 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364565 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3699/tid=3706] ppid=3697 vsize=128172 CPUtime=1.48 cores=1,3,5,7
/proc/3699/task/3706/stat : 3706 (plingeling) R 3697 3699 2999 0 -1 4202560 3568 1257 0 0 148 0 0 0 25 0 5 0 522625280 131248128 3492 33554432000 4194304 4417332 140736962089664 18446744073709551615 4321293 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3699/tid=3707] ppid=3697 vsize=128172 CPUtime=1.48 cores=1,3,5,7
/proc/3699/task/3707/stat : 3707 (plingeling) R 3697 3699 2999 0 -1 4202560 4092 1257 0 0 148 0 0 0 25 0 5 0 522625280 131248128 3492 33554432000 4194304 4417332 140736962089664 18446744073709551615 18446744073699065939 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3699/tid=3708] ppid=3697 vsize=128172 CPUtime=1.48 cores=1,3,5,7
/proc/3699/task/3708/stat : 3708 (plingeling) R 3697 3699 2999 0 -1 4202560 3348 1257 0 0 148 0 0 0 25 0 5 0 522625280 131248128 3492 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 5.97
Current children cumulated vsize (KiB) 128172

[startup+3.10067 s]
/proc/loadavg: 8.03 7.92 7.91 9/186 3709
/proc/meminfo: memFree=30344216/32950928 swapFree=67111404/67111528
[pid=3699] ppid=3697 vsize=136684 CPUtime=12.36 cores=1,3,5,7
/proc/3699/stat : 3699 (plingeling) S 3697 3699 2999 0 -1 4202496 28577 1257 0 0 1232 4 0 0 25 0 5 0 522625279 139964416 5620 33554432000 4194304 4417332 140736962089664 18446744073709551615 235606669285 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/3699/statm: 34171 5620 181 55 0 32057 0
[pid=3699/tid=3705] ppid=3697 vsize=136684 CPUtime=3.08 cores=1,3,5,7
/proc/3699/task/3705/stat : 3705 (plingeling) R 3697 3699 2999 0 -1 4202560 7791 1257 0 0 308 0 0 0 25 0 5 0 522625280 139964416 5620 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364565 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3699/tid=3706] ppid=3697 vsize=136684 CPUtime=3.07 cores=1,3,5,7
/proc/3699/task/3706/stat : 3706 (plingeling) R 3697 3699 2999 0 -1 4202560 7215 1257 0 0 307 0 0 0 25 0 5 0 522625280 139964416 5620 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364776 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3699/tid=3707] ppid=3697 vsize=136684 CPUtime=3.08 cores=1,3,5,7
/proc/3699/task/3707/stat : 3707 (plingeling) R 3697 3699 2999 0 -1 4202560 7083 1257 0 0 307 1 0 0 25 0 5 0 522625280 139964416 5620 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3699/tid=3708] ppid=3697 vsize=136684 CPUtime=3.08 cores=1,3,5,7
/proc/3699/task/3708/stat : 3708 (plingeling) R 3697 3699 2999 0 -1 4202560 6136 1257 0 0 307 1 0 0 25 0 5 0 522625280 139964416 5620 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 12.36
Current children cumulated vsize (KiB) 136684

[startup+6.3011 s]
/proc/loadavg: 8.03 7.92 7.91 9/186 3710
/proc/meminfo: memFree=30310888/32950928 swapFree=67111404/67111528
[pid=3699] ppid=3697 vsize=151228 CPUtime=25.14 cores=1,3,5,7
/proc/3699/stat : 3699 (plingeling) S 3697 3699 2999 0 -1 4202496 51897 1257 0 0 2506 8 0 0 25 0 5 0 522625279 154857472 9224 33554432000 4194304 4417332 140736962089664 18446744073709551615 235606669285 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/3699/statm: 37807 9224 181 55 0 35693 0
[pid=3699/tid=3705] ppid=3697 vsize=151228 CPUtime=6.28 cores=1,3,5,7
/proc/3699/task/3705/stat : 3705 (plingeling) R 3697 3699 2999 0 -1 4202560 13979 1257 0 0 627 1 0 0 25 0 5 0 522625280 154857472 9224 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364568 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3699/tid=3706] ppid=3697 vsize=151228 CPUtime=6.26 cores=1,3,5,7
/proc/3699/task/3706/stat : 3706 (plingeling) R 3697 3699 2999 0 -1 4202560 14324 1257 0 0 624 2 0 0 25 0 5 0 522625280 154857472 9224 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3699/tid=3707] ppid=3697 vsize=151228 CPUtime=6.29 cores=1,3,5,7
/proc/3699/task/3707/stat : 3707 (plingeling) R 3697 3699 2999 0 -1 4202560 13343 1257 0 0 627 2 0 0 25 0 5 0 522625280 154857472 9224 33554432000 4194304 4417332 140736962089664 18446744073709551615 4365977 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3699/tid=3708] ppid=3697 vsize=151228 CPUtime=6.29 cores=1,3,5,7
/proc/3699/task/3708/stat : 3708 (plingeling) R 3697 3699 2999 0 -1 4202560 9899 1257 0 0 627 2 0 0 25 0 5 0 522625280 154857472 9224 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364509 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 25.14
Current children cumulated vsize (KiB) 151228

[startup+12.701 s]
/proc/loadavg: 8.11 7.94 7.92 9/186 3710
/proc/meminfo: memFree=30227516/32950928 swapFree=67111404/67111528
[pid=3699] ppid=3697 vsize=156460 CPUtime=50.7 cores=1,3,5,7
/proc/3699/stat : 3699 (plingeling) S 3697 3699 2999 0 -1 4202496 87986 1257 0 0 5057 13 0 0 25 0 5 0 522625279 160215040 10564 33554432000 4194304 4417332 140736962089664 18446744073709551615 235606669285 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/3699/statm: 39115 10564 181 55 0 37001 0
[pid=3699/tid=3705] ppid=3697 vsize=156460 CPUtime=12.69 cores=1,3,5,7
/proc/3699/task/3705/stat : 3705 (plingeling) R 3697 3699 2999 0 -1 4202560 23809 1257 0 0 1266 3 0 0 25 0 5 0 522625280 160215040 10564 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3699/tid=3706] ppid=3697 vsize=156460 CPUtime=12.62 cores=1,3,5,7
/proc/3699/task/3706/stat : 3706 (plingeling) R 3697 3699 2999 0 -1 4202560 24067 1257 0 0 1259 3 0 0 25 0 5 0 522625280 160215040 10564 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3699/tid=3707] ppid=3697 vsize=156460 CPUtime=12.68 cores=1,3,5,7
/proc/3699/task/3707/stat : 3707 (plingeling) R 3697 3699 2999 0 -1 4202560 23119 1257 0 0 1265 3 0 0 25 0 5 0 522625280 160215040 10564 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3699/tid=3708] ppid=3697 vsize=156460 CPUtime=12.68 cores=1,3,5,7
/proc/3699/task/3708/stat : 3708 (plingeling) R 3697 3699 2999 0 -1 4202560 16639 1257 0 0 1266 2 0 0 25 0 5 0 522625280 160215040 10564 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364742 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 50.7
Current children cumulated vsize (KiB) 156460

[startup+25.5007 s]
/proc/loadavg: 8.09 7.94 7.92 9/186 3710
/proc/meminfo: memFree=30192056/32950928 swapFree=67111404/67111528
[pid=3699] ppid=3697 vsize=179332 CPUtime=101.83 cores=1,3,5,7
/proc/3699/stat : 3699 (plingeling) S 3697 3699 2999 0 -1 4202496 147257 1257 0 0 10159 24 0 0 25 0 5 0 522625279 183635968 16250 33554432000 4194304 4417332 140736962089664 18446744073709551615 235606669285 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/3699/statm: 44833 16250 181 55 0 42719 0
[pid=3699/tid=3705] ppid=3697 vsize=179332 CPUtime=25.49 cores=1,3,5,7
/proc/3699/task/3705/stat : 3705 (plingeling) R 3697 3699 2999 0 -1 4202560 39996 1257 0 0 2543 6 0 0 25 0 5 0 522625280 183635968 16250 33554432000 4194304 4417332 140736962089664 18446744073709551615 18446744073699065897 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3699/tid=3706] ppid=3697 vsize=179332 CPUtime=25.35 cores=1,3,5,7
/proc/3699/task/3706/stat : 3706 (plingeling) R 3697 3699 2999 0 -1 4202560 38838 1257 0 0 2528 7 0 0 25 0 5 0 522625280 183635968 16250 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364519 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3699/tid=3707] ppid=3697 vsize=179332 CPUtime=25.48 cores=1,3,5,7
/proc/3699/task/3707/stat : 3707 (plingeling) R 3697 3699 2999 0 -1 4202560 42461 1257 0 0 2543 5 0 0 25 0 5 0 522625280 183635968 16250 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3699/tid=3708] ppid=3697 vsize=179332 CPUtime=25.48 cores=1,3,5,7
/proc/3699/task/3708/stat : 3708 (plingeling) R 3697 3699 2999 0 -1 4202560 25610 1257 0 0 2544 4 0 0 25 0 5 0 522625280 183635968 16250 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364541 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 101.83
Current children cumulated vsize (KiB) 179332

[startup+51.1012 s]
/proc/loadavg: 8.13 7.96 7.93 9/186 3711
/proc/meminfo: memFree=30070120/32950928 swapFree=67111404/67111528
[pid=3699] ppid=3697 vsize=217572 CPUtime=204.09 cores=1,3,5,7
/proc/3699/stat : 3699 (plingeling) S 3697 3699 2999 0 -1 4202496 254859 1257 0 0 20369 40 0 0 25 0 5 0 522625279 222793728 25831 33554432000 4194304 4417332 140736962089664 18446744073709551615 235606669285 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/3699/statm: 54393 25831 181 55 0 52279 0
[pid=3699/tid=3705] ppid=3697 vsize=217572 CPUtime=51.09 cores=1,3,5,7
/proc/3699/task/3705/stat : 3705 (plingeling) R 3697 3699 2999 0 -1 4202560 77160 1257 0 0 5098 11 0 0 25 0 5 0 522625280 222793728 25831 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364631 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3699/tid=3706] ppid=3697 vsize=217572 CPUtime=50.81 cores=1,3,5,7
/proc/3699/task/3706/stat : 3706 (plingeling) R 3697 3699 2999 0 -1 4202560 63664 1257 0 0 5071 10 0 0 25 0 5 0 522625280 222793728 25831 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364568 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3699/tid=3707] ppid=3697 vsize=217572 CPUtime=51.08 cores=1,3,5,7

################
# More data... #
################

[pid=3699/tid=3706] ppid=3697 vsize=784372 CPUtime=758.14 cores=1,3,5,7
/proc/3699/task/3706/stat : 3706 (plingeling) R 3697 3699 2999 0 -1 4202560 388564 1257 0 0 75752 62 0 0 25 0 5 0 522625280 803196928 167510 33554432000 4194304 4417332 140736962089664 18446744073709551615 4366297 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3699/tid=3707] ppid=3697 vsize=784372 CPUtime=762.3 cores=1,3,5,7
/proc/3699/task/3707/stat : 3707 (plingeling) R 3697 3699 2999 0 -1 4202560 414262 1257 0 0 76168 62 0 0 25 0 5 0 522625280 803196928 167510 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3699/tid=3708] ppid=3697 vsize=784372 CPUtime=762.26 cores=1,3,5,7
/proc/3699/task/3708/stat : 3708 (plingeling) R 3697 3699 2999 0 -1 4202560 215946 1257 0 0 76190 36 0 0 25 0 5 0 522625280 803196928 167510 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3045.03
Current children cumulated vsize (KiB) 784372

[startup+822.301 s]
/proc/loadavg: 8.24 8.10 8.02 9/186 3779
/proc/meminfo: memFree=28752504/32950928 swapFree=67111404/67111528
[pid=3699] ppid=3697 vsize=861612 CPUtime=3284.69 cores=1,3,5,7
/proc/3699/stat : 3699 (plingeling) S 3697 3699 2999 0 -1 4202496 1443144 1257 0 0 328242 227 0 0 25 0 5 0 522625279 882290688 186820 33554432000 4194304 4417332 140736962089664 18446744073709551615 235606669285 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/3699/statm: 215403 186820 181 55 0 213289 0
[pid=3699/tid=3705] ppid=3697 vsize=861612 CPUtime=822.3 cores=1,3,5,7
/proc/3699/task/3705/stat : 3705 (plingeling) R 3697 3699 2999 0 -1 4202560 387513 1257 0 0 82169 61 0 0 25 0 5 0 522625280 882290688 186820 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3699/tid=3706] ppid=3697 vsize=861612 CPUtime=817.8 cores=1,3,5,7
/proc/3699/task/3706/stat : 3706 (plingeling) R 3697 3699 2999 0 -1 4202560 388564 1257 0 0 81718 62 0 0 25 0 5 0 522625280 882290688 186820 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3699/tid=3707] ppid=3697 vsize=861612 CPUtime=822.3 cores=1,3,5,7
/proc/3699/task/3707/stat : 3707 (plingeling) R 3697 3699 2999 0 -1 4202560 433504 1257 0 0 82165 65 0 0 25 0 5 0 522625280 882290688 186820 33554432000 4194304 4417332 140736962089664 18446744073709551615 4277864 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3699/tid=3708] ppid=3697 vsize=861612 CPUtime=822.26 cores=1,3,5,7
/proc/3699/task/3708/stat : 3708 (plingeling) R 3697 3699 2999 0 -1 4202560 233211 1257 0 0 82188 38 0 0 25 0 5 0 522625280 882290688 186820 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3284.69
Current children cumulated vsize (KiB) 861612

[startup+882.301 s]
/proc/loadavg: 8.15 8.10 8.02 9/186 3781
/proc/meminfo: memFree=28573820/32950928 swapFree=67111404/67111528
[pid=3699] ppid=3697 vsize=1031264 CPUtime=3524.37 cores=1,3,5,7
/proc/3699/stat : 3699 (plingeling) S 3697 3699 2999 0 -1 4202496 1510502 1257 0 0 352198 239 0 0 25 0 5 0 522625279 1056014336 229233 33554432000 4194304 4417332 140736962089664 18446744073709551615 235606669285 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/3699/statm: 257816 229233 181 55 0 255702 0
[pid=3699/tid=3705] ppid=3697 vsize=1031264 CPUtime=882.3 cores=1,3,5,7
/proc/3699/task/3705/stat : 3705 (plingeling) R 3697 3699 2999 0 -1 4202560 387513 1257 0 0 88169 61 0 0 25 0 5 0 522625280 1056014336 229233 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364611 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3699/tid=3706] ppid=3697 vsize=1031264 CPUtime=877.47 cores=1,3,5,7
/proc/3699/task/3706/stat : 3706 (plingeling) R 3697 3699 2999 0 -1 4202560 434836 1257 0 0 87677 70 0 0 25 0 5 0 522625280 1056014336 229233 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3699/tid=3707] ppid=3697 vsize=1031264 CPUtime=882.29 cores=1,3,5,7
/proc/3699/task/3707/stat : 3707 (plingeling) R 3697 3699 2999 0 -1 4202560 446398 1257 0 0 88163 66 0 0 25 0 5 0 522625280 1056014336 229233 33554432000 4194304 4417332 140736962089664 18446744073709551615 4307997 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3699/tid=3708] ppid=3697 vsize=1031264 CPUtime=882.26 cores=1,3,5,7
/proc/3699/task/3708/stat : 3708 (plingeling) R 3697 3699 2999 0 -1 4202560 241403 1257 0 0 88187 39 0 0 25 0 5 0 522625280 1056014336 229233 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3524.37
Current children cumulated vsize (KiB) 1031264

[startup+942.301 s]
/proc/loadavg: 8.12 8.09 8.02 9/186 3783
/proc/meminfo: memFree=28789420/32950928 swapFree=67111404/67111528
[pid=3699] ppid=3697 vsize=938644 CPUtime=3764.03 cores=1,3,5,7
/proc/3699/stat : 3699 (plingeling) S 3697 3699 2999 0 -1 4202496 1599363 1257 0 0 376150 253 0 0 25 0 5 0 522625279 961171456 206078 33554432000 4194304 4417332 140736962089664 18446744073709551615 235606669285 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/3699/statm: 234661 206078 181 55 0 232547 0
[pid=3699/tid=3705] ppid=3697 vsize=938644 CPUtime=942.3 cores=1,3,5,7
/proc/3699/task/3705/stat : 3705 (plingeling) R 3697 3699 2999 0 -1 4202560 404569 1257 0 0 94166 64 0 0 25 0 5 0 522625280 961171456 206078 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3699/tid=3706] ppid=3697 vsize=938644 CPUtime=937.15 cores=1,3,5,7
/proc/3699/task/3706/stat : 3706 (plingeling) R 3697 3699 2999 0 -1 4202560 465379 1257 0 0 93640 75 0 0 25 0 5 0 522625280 961171456 206078 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3699/tid=3707] ppid=3697 vsize=938644 CPUtime=942.3 cores=1,3,5,7
/proc/3699/task/3707/stat : 3707 (plingeling) R 3697 3699 2999 0 -1 4202560 476236 1257 0 0 94159 71 0 0 25 0 5 0 522625280 961171456 206078 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364727 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3699/tid=3708] ppid=3697 vsize=938644 CPUtime=942.25 cores=1,3,5,7
/proc/3699/task/3708/stat : 3708 (plingeling) R 3697 3699 2999 0 -1 4202560 252827 1257 0 0 94184 41 0 0 25 0 5 0 522625280 961171456 206078 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3764.03
Current children cumulated vsize (KiB) 938644

[startup+1002.3 s]
/proc/loadavg: 8.04 8.07 8.02 9/186 3784
/proc/meminfo: memFree=28901216/32950928 swapFree=67111404/67111528
[pid=3699] ppid=3697 vsize=1057264 CPUtime=4003.71 cores=1,3,5,7
/proc/3699/stat : 3699 (plingeling) S 3697 3699 2999 0 -1 4202496 1633102 1257 0 0 400112 259 0 0 25 0 5 0 522625279 1082638336 235733 33554432000 4194304 4417332 140736962089664 18446744073709551615 235606669285 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/3699/statm: 264316 235733 181 55 0 262202 0
[pid=3699/tid=3705] ppid=3697 vsize=1057264 CPUtime=1002.3 cores=1,3,5,7
/proc/3699/task/3705/stat : 3705 (plingeling) R 3697 3699 2999 0 -1 4202560 404569 1257 0 0 100166 64 0 0 25 0 5 0 522625280 1082638336 235733 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364683 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3699/tid=3706] ppid=3697 vsize=1057264 CPUtime=996.82 cores=1,3,5,7
/proc/3699/task/3706/stat : 3706 (plingeling) R 3697 3699 2999 0 -1 4202560 465379 1257 0 0 99607 75 0 0 25 0 5 0 522625280 1082638336 235733 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3699/tid=3707] ppid=3697 vsize=1057264 CPUtime=1002.3 cores=1,3,5,7
/proc/3699/task/3707/stat : 3707 (plingeling) R 3697 3699 2999 0 -1 4202560 493591 1257 0 0 100156 74 0 0 25 0 5 0 522625280 1082638336 235733 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3699/tid=3708] ppid=3697 vsize=1057264 CPUtime=1002.25 cores=1,3,5,7
/proc/3699/task/3708/stat : 3708 (plingeling) R 3697 3699 2999 0 -1 4202560 269211 1257 0 0 100181 44 0 0 25 0 5 0 522625280 1082638336 235733 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364599 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4003.71
Current children cumulated vsize (KiB) 1057264

[startup+1062.3 s]
/proc/loadavg: 8.01 8.06 8.01 9/186 3786
/proc/meminfo: memFree=28957468/32950928 swapFree=67111404/67111528
[pid=3699] ppid=3697 vsize=985300 CPUtime=4243.39 cores=1,3,5,7
/proc/3699/stat : 3699 (plingeling) S 3697 3699 2999 0 -1 4202496 1717571 1257 0 0 424066 273 0 0 25 0 5 0 522625279 1008947200 217742 33554432000 4194304 4417332 140736962089664 18446744073709551615 235606669285 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/3699/statm: 246325 217742 181 55 0 244211 0
[pid=3699/tid=3705] ppid=3697 vsize=985300 CPUtime=1062.31 cores=1,3,5,7
/proc/3699/task/3705/stat : 3705 (plingeling) R 3697 3699 2999 0 -1 4202560 467690 1257 0 0 106156 75 0 0 25 0 5 0 522625280 1008947200 217742 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364713 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3699/tid=3706] ppid=3697 vsize=985300 CPUtime=1056.49 cores=1,3,5,7
/proc/3699/task/3706/stat : 3706 (plingeling) R 3697 3699 2999 0 -1 4202560 465379 1257 0 0 105574 75 0 0 25 0 5 0 522625280 1008947200 217742 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364732 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3699/tid=3707] ppid=3697 vsize=985300 CPUtime=1062.3 cores=1,3,5,7
/proc/3699/task/3707/stat : 3707 (plingeling) R 3697 3699 2999 0 -1 4202560 495319 1257 0 0 106156 74 0 0 25 0 5 0 522625280 1008947200 217742 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364724 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3699/tid=3708] ppid=3697 vsize=985300 CPUtime=1062.26 cores=1,3,5,7
/proc/3699/task/3708/stat : 3708 (plingeling) R 3697 3699 2999 0 -1 4202560 288831 1257 0 0 106178 48 0 0 25 0 5 0 522625280 1008947200 217742 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4243.39
Current children cumulated vsize (KiB) 985300

[startup+1122.3 s]
/proc/loadavg: 8.00 8.04 8.00 9/186 3787
/proc/meminfo: memFree=28866392/32950928 swapFree=67111404/67111528
[pid=3699] ppid=3697 vsize=1076560 CPUtime=4483.06 cores=1,3,5,7
/proc/3699/stat : 3699 (plingeling) S 3697 3699 2999 0 -1 4202496 1820543 1257 0 0 448019 287 0 0 25 0 5 0 522625279 1102397440 240556 33554432000 4194304 4417332 140736962089664 18446744073709551615 235606669285 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/3699/statm: 269140 240556 181 55 0 267026 0
[pid=3699/tid=3705] ppid=3697 vsize=1076560 CPUtime=1122.31 cores=1,3,5,7
/proc/3699/task/3705/stat : 3705 (plingeling) R 3697 3699 2999 0 -1 4202560 508818 1257 0 0 112152 79 0 0 25 0 5 0 522625280 1102397440 240556 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3699/tid=3706] ppid=3697 vsize=1076560 CPUtime=1116.16 cores=1,3,5,7
/proc/3699/task/3706/stat : 3706 (plingeling) R 3697 3699 2999 0 -1 4202560 465379 1257 0 0 111541 75 0 0 25 0 5 0 522625280 1102397440 240556 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3699/tid=3707] ppid=3697 vsize=1076560 CPUtime=1122.3 cores=1,3,5,7
/proc/3699/task/3707/stat : 3707 (plingeling) R 3697 3699 2999 0 -1 4202560 540779 1257 0 0 112149 81 0 0 25 0 5 0 522625280 1102397440 240556 33554432000 4194304 4417332 140736962089664 18446744073709551615 4365960 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3699/tid=3708] ppid=3697 vsize=1076560 CPUtime=1122.26 cores=1,3,5,7
/proc/3699/task/3708/stat : 3708 (plingeling) R 3697 3699 2999 0 -1 4202560 305215 1257 0 0 112176 50 0 0 25 0 5 0 522625280 1102397440 240556 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4483.06
Current children cumulated vsize (KiB) 1076560

[startup+1182.3 s]
/proc/loadavg: 8.00 8.03 8.00 9/186 3789
/proc/meminfo: memFree=28896208/32950928 swapFree=67111404/67111528
[pid=3699] ppid=3697 vsize=1064936 CPUtime=4722.74 cores=1,3,5,7
/proc/3699/stat : 3699 (plingeling) S 3697 3699 2999 0 -1 4202496 1866789 1257 0 0 471980 294 0 0 25 0 5 0 522625279 1090494464 237650 33554432000 4194304 4417332 140736962089664 18446744073709551615 235606669285 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/3699/statm: 266234 237650 181 55 0 264120 0
[pid=3699/tid=3705] ppid=3697 vsize=1064936 CPUtime=1182.31 cores=1,3,5,7
/proc/3699/task/3705/stat : 3705 (plingeling) R 3697 3699 2999 0 -1 4202560 543214 1257 0 0 118148 83 0 0 25 0 5 0 522625280 1090494464 237650 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3699/tid=3706] ppid=3697 vsize=1064936 CPUtime=1175.84 cores=1,3,5,7
/proc/3699/task/3706/stat : 3706 (plingeling) R 3697 3699 2999 0 -1 4202560 469037 1257 0 0 117508 76 0 0 25 0 5 0 522625280 1090494464 237650 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3699/tid=3707] ppid=3697 vsize=1064936 CPUtime=1182.3 cores=1,3,5,7
/proc/3699/task/3707/stat : 3707 (plingeling) R 3697 3699 2999 0 -1 4202560 540779 1257 0 0 118149 81 0 0 25 0 5 0 522625280 1090494464 237650 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364470 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3699/tid=3708] ppid=3697 vsize=1064936 CPUtime=1182.26 cores=1,3,5,7
/proc/3699/task/3708/stat : 3708 (plingeling) R 3697 3699 2999 0 -1 4202560 313407 1257 0 0 118174 52 0 0 25 0 5 0 522625280 1090494464 237650 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4722.74
Current children cumulated vsize (KiB) 1064936



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 8.00 8.03 8.00 9/186 3789
/proc/meminfo: memFree=28924700/32950928 swapFree=67111404/67111528
[pid=3699] ppid=3697 vsize=1033512 CPUtime=4800.25 cores=1,3,5,7
/proc/3699/stat : 3699 (plingeling) S 3697 3699 2999 0 -1 4202496 1875317 1257 0 0 479730 295 0 0 25 0 5 0 522625279 1058316288 229794 33554432000 4194304 4417332 140736962089664 18446744073709551615 235606669285 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/3699/statm: 258378 229794 181 55 0 256264 0
[pid=3699/tid=3705] ppid=3697 vsize=1033512 CPUtime=1201.72 cores=1,3,5,7
/proc/3699/task/3705/stat : 3705 (plingeling) R 3697 3699 2999 0 -1 4202560 551742 1257 0 0 120088 84 0 0 25 0 5 0 522625280 1058316288 229794 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364565 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3699/tid=3706] ppid=3697 vsize=1033512 CPUtime=1195.13 cores=1,3,5,7
/proc/3699/task/3706/stat : 3706 (plingeling) R 3697 3699 2999 0 -1 4202560 469037 1257 0 0 119437 76 0 0 25 0 5 0 522625280 1058316288 229794 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3699/tid=3707] ppid=3697 vsize=1033512 CPUtime=1201.71 cores=1,3,5,7
/proc/3699/task/3707/stat : 3707 (plingeling) R 3697 3699 2999 0 -1 4202560 540779 1257 0 0 120090 81 0 0 25 0 5 0 522625280 1058316288 229794 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3699/tid=3708] ppid=3697 vsize=1033512 CPUtime=1201.66 cores=1,3,5,7
/proc/3699/task/3708/stat : 3708 (plingeling) R 3697 3699 2999 0 -1 4202560 313407 1257 0 0 120114 52 0 0 25 0 5 0 522625280 1058316288 229794 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.25
Current children cumulated vsize (KiB) 1033512

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.7 s]
/proc/loadavg: 8.00 8.03 8.00 9/186 3789
/proc/meminfo: memFree=28924700/32950928 swapFree=67111404/67111528
[pid=3699] ppid=3697 vsize=1033512 CPUtime=4800.25 cores=1,3,5,7
/proc/3699/stat : 3699 (plingeling) S 3697 3699 2999 0 -1 4202496 1875317 1257 0 0 479730 295 0 0 25 0 5 0 522625279 1058316288 229794 33554432000 4194304 4417332 140736962089664 18446744073709551615 235606669285 0 0 4096 17442 18446744073709551615 0 0 17 3 0 0 0
/proc/3699/statm: 258378 229794 181 55 0 256264 0
[pid=3699/tid=3705] ppid=3697 vsize=1033512 CPUtime=1201.72 cores=1,3,5,7
/proc/3699/task/3705/stat : 3705 (plingeling) R 3697 3699 2999 0 -1 4202560 551742 1257 0 0 120088 84 0 0 25 0 5 0 522625280 1058316288 229794 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364565 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=3699/tid=3706] ppid=3697 vsize=1033512 CPUtime=1195.13 cores=1,3,5,7
/proc/3699/task/3706/stat : 3706 (plingeling) R 3697 3699 2999 0 -1 4202560 469037 1257 0 0 119437 76 0 0 25 0 5 0 522625280 1058316288 229794 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=3699/tid=3707] ppid=3697 vsize=1033512 CPUtime=1201.71 cores=1,3,5,7
/proc/3699/task/3707/stat : 3707 (plingeling) R 3697 3699 2999 0 -1 4202560 540779 1257 0 0 120090 81 0 0 25 0 5 0 522625280 1058316288 229794 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=3699/tid=3708] ppid=3697 vsize=1033512 CPUtime=1201.66 cores=1,3,5,7
/proc/3699/task/3708/stat : 3708 (plingeling) R 3697 3699 2999 0 -1 4202560 313407 1257 0 0 120114 52 0 0 25 0 5 0 522625280 1058316288 229794 33554432000 4194304 4417332 140736962089664 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.25
Current children cumulated vsize (KiB) 1033512

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1201.75
CPU time (s): 4800.32
CPU user time (s): 4797.33
CPU system time (s): 2.99254
CPU usage (%): 399.445
Max. virtual memory (cumulated for all children) (KiB): 1163192

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4797.33
system time used= 2.99254
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1876574
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 141
involuntary context switches= 32019

runsolver used 1.90771 second user time and 4.65729 second system time

The end

Launcher Data 

Begin job on node120 at 2011-04-29 22:51:05
IDJOB=3292841
IDBENCH=87946
IDSOLVER=1605
FILE ID=node120/3292841-1304110265
RUNJOBID= node120-1304109773-3464
PBS_JOBID= 13170661
Free space on /tmp= 72076 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT02/industrial/aloul/Homer/homer17.shuffled.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3292841-1304110265/watcher-3292841-1304110265 -o /tmp/evaluation-result-3292841-1304110265/solver-3292841-1304110265 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3292841-1304110265.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= ca62f6128dc3cad7229c7b9ee33a7492
RANDOM SEED=1692854780

node120.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.78
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      30394492 kB
Buffers:        490528 kB
Cached:         611816 kB
SwapCached:          8 kB
Active:        1433400 kB
Inactive:       762564 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      30394492 kB
SwapTotal:    67111528 kB
SwapFree:     67111404 kB
Dirty:              52 kB
Writeback:           0 kB
AnonPages:     1093904 kB
Mapped:          15688 kB
Slab:           295876 kB
PageTables:       6676 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:  1279408 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71888 MiB
End job on node120 at 2011-04-29 23:11:09