Trace number 3293623

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 4800.28 1205.08

General information on the benchmark

NameSAT-Race-2010-CNF/hardware-verification/
velev/11pipe_11_ooo.cnf
MD5SUMf603f11fbfe855df381282b7bce010ea
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables88289
Number of clauses4187694
Sum of the clauses size12410354
Maximum clause length341
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 23770000
Number of clauses of size 3347602
Number of clauses of size 41622
Number of clauses of size 51351
Number of clauses of size over 567118

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3293623-1304111273/watcher-3293623-1304111273 -o /tmp/evaluation-result-3293623-1304111273/solver-3293623-1304111273 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3293623-1304111273.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.42 5.05 6.61 2/173 26923
/proc/meminfo: memFree=25146748/32951124 swapFree=67051468/67111528
[pid=26923] ppid=26921 vsize=11800 CPUtime=0 cores=0,2,4,6
/proc/26923/stat : 26923 (runsolver) S 26921 26923 26140 0 -1 4202560 41 0 0 0 0 0 0 0 17 0 1 0 522730098 12083200 51 33554432000 4194304 5434984 140733911425488 18446744073709551615 4881607 0 2147483391 4096 24578 18446744071699960631 0 0 17 2 0 0 0
/proc/26923/statm: 2950 51 31 303 0 2644 0

[startup+0.081061 s]
/proc/loadavg: 3.42 5.05 6.61 2/173 26923
/proc/meminfo: memFree=25146748/32951124 swapFree=67051468/67111528
[pid=26923] ppid=26921 vsize=16580 CPUtime=0.06 cores=0,2,4,6
/proc/26923/stat : 26923 (plingeling) R 26921 26923 26140 0 -1 4202496 2239 1260 0 0 3 3 0 0 18 0 1 0 522730098 16977920 2169 33554432000 4194304 4417332 140737126547168 18446744073709551615 4209451 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/26923/statm: 4145 2169 148 55 0 2031 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 16580

[startup+0.10109 s]
/proc/loadavg: 3.42 5.05 6.61 2/173 26923
/proc/meminfo: memFree=25146748/32951124 swapFree=67051468/67111528
[pid=26923] ppid=26921 vsize=21700 CPUtime=0.09 cores=0,2,4,6
/proc/26923/stat : 26923 (plingeling) R 26921 26923 26140 0 -1 4202496 3437 1260 0 0 5 4 0 0 18 0 1 0 522730098 22220800 3368 33554432000 4194304 4417332 140737126547168 18446744073709551615 218370651229 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/26923/statm: 5425 3389 148 55 0 3311 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 21700

[startup+0.301026 s]
/proc/loadavg: 3.42 5.05 6.61 2/173 26923
/proc/meminfo: memFree=25146748/32951124 swapFree=67051468/67111528
[pid=26923] ppid=26921 vsize=39520 CPUtime=0.28 cores=0,2,4,6
/proc/26923/stat : 26923 (plingeling) R 26921 26923 26140 0 -1 4202496 8007 1260 0 0 23 5 0 0 19 0 1 0 522730098 40468480 7904 33554432000 4194304 4417332 140737126547168 18446744073709551615 218370957056 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/26923/statm: 9880 7904 148 55 0 7766 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 39520

[startup+0.700956 s]
/proc/loadavg: 3.42 5.05 6.61 2/173 26923
/proc/meminfo: memFree=25146748/32951124 swapFree=67051468/67111528
[pid=26923] ppid=26921 vsize=101488 CPUtime=0.68 cores=0,2,4,6
/proc/26923/stat : 26923 (plingeling) R 26921 26923 26140 0 -1 4202496 23499 1260 0 0 61 7 0 0 19 0 1 0 522730098 103923712 23396 33554432000 4194304 4417332 140737126547168 18446744073709551615 4209621 0 0 4096 17442 0 0 0 17 6 0 0 0
/proc/26923/statm: 25372 23396 148 55 0 23258 0
Current children cumulated CPU time (s) 0.68
Current children cumulated vsize (KiB) 101488

[startup+1.50086 s]
/proc/loadavg: 3.42 5.05 6.61 3/178 26940
/proc/meminfo: memFree=24887688/32951124 swapFree=67051468/67111528
[pid=26923] ppid=26921 vsize=183408 CPUtime=1.48 cores=0,2,4,6
/proc/26923/stat : 26923 (plingeling) R 26921 26923 26140 0 -1 4202496 43979 1260 0 0 139 9 0 0 21 0 1 0 522730098 187809792 43876 33554432000 4194304 4417332 140737126547168 18446744073709551615 4231544 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/26923/statm: 45852 43876 148 55 0 43738 0
Current children cumulated CPU time (s) 1.48
Current children cumulated vsize (KiB) 183408

[startup+3.10059 s]
/proc/loadavg: 3.42 5.05 6.61 3/178 26940
/proc/meminfo: memFree=24690032/32951124 swapFree=67051468/67111528
[pid=26923] ppid=26921 vsize=356732 CPUtime=3.08 cores=0,2,4,6
/proc/26923/stat : 26923 (plingeling) R 26921 26923 26140 0 -1 4202496 87310 1260 0 0 292 16 0 0 25 0 1 0 522730098 365293568 87207 33554432000 4194304 4417332 140737126547168 18446744073709551615 218370957056 0 0 4096 17442 0 0 0 17 0 0 0 0
/proc/26923/statm: 89183 87207 148 55 0 87069 0
Current children cumulated CPU time (s) 3.08
Current children cumulated vsize (KiB) 356732

[startup+6.3009 s]
/proc/loadavg: 3.31 5.00 6.59 10/186 26948
/proc/meminfo: memFree=24259660/32951124 swapFree=67051468/67111528
[pid=26923] ppid=26921 vsize=485520 CPUtime=11.81 cores=0,2,4,6
/proc/26923/stat : 26923 (plingeling) S 26921 26923 26140 0 -1 4202496 103213 1260 0 0 1159 22 0 0 25 0 5 0 522730098 497172480 93501 33554432000 4194304 4417332 140737126547168 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/26923/statm: 121380 93501 174 55 0 119266 0
[pid=26923/tid=26941] ppid=26921 vsize=485520 CPUtime=1.84 cores=0,2,4,6
/proc/26923/task/26941/stat : 26941 (plingeling) R 26921 26923 26140 0 -1 4202560 4379 1260 0 0 184 0 0 0 20 0 5 0 522730543 497172480 93501 33554432000 4194304 4417332 140737126547168 18446744073709551615 4284732 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26923/tid=26942] ppid=26921 vsize=485520 CPUtime=1.84 cores=0,2,4,6
/proc/26923/task/26942/stat : 26942 (plingeling) R 26921 26923 26140 0 -1 4202560 2373 1260 0 0 184 0 0 0 25 0 5 0 522730543 497172480 93501 33554432000 4194304 4417332 140737126547168 18446744073709551615 4284741 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26923/tid=26943] ppid=26921 vsize=485520 CPUtime=1.82 cores=0,2,4,6
/proc/26923/task/26943/stat : 26943 (plingeling) R 26921 26923 26140 0 -1 4202560 2466 1260 0 0 182 0 0 0 20 0 5 0 522730543 497172480 93501 33554432000 4194304 4417332 140737126547168 18446744073709551615 4310288 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26923/tid=26944] ppid=26921 vsize=485520 CPUtime=1.84 cores=0,2,4,6
/proc/26923/task/26944/stat : 26944 (plingeling) R 26921 26923 26140 0 -1 4202560 2000 1260 0 0 182 2 0 0 25 0 5 0 522730543 497172480 93501 33554432000 4194304 4417332 140737126547168 18446744073709551615 4236874 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 11.81
Current children cumulated vsize (KiB) 485520

[startup+12.7008 s]
/proc/loadavg: 3.68 5.05 6.59 9/186 26948
/proc/meminfo: memFree=23881976/32951124 swapFree=67051468/67111528
[pid=26923] ppid=26921 vsize=647132 CPUtime=37.36 cores=0,2,4,6
/proc/26923/stat : 26923 (plingeling) S 26921 26923 26140 0 -1 4202496 251453 1260 0 0 3691 45 0 0 25 0 5 0 522730098 662663168 133908 33554432000 4194304 4417332 140737126547168 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/26923/statm: 161783 133908 178 55 0 159669 0
[pid=26923/tid=26941] ppid=26921 vsize=647132 CPUtime=8.24 cores=0,2,4,6
/proc/26923/task/26941/stat : 26941 (plingeling) R 26921 26923 26140 0 -1 4202560 61009 1260 0 0 815 9 0 0 25 0 5 0 522730543 662663168 133908 33554432000 4194304 4417332 140737126547168 18446744073709551615 4254088 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26923/tid=26942] ppid=26921 vsize=647132 CPUtime=8.24 cores=0,2,4,6
/proc/26923/task/26942/stat : 26942 (plingeling) R 26921 26923 26140 0 -1 4202560 39215 1260 0 0 820 4 0 0 25 0 5 0 522730543 662663168 133908 33554432000 4194304 4417332 140737126547168 18446744073709551615 4302988 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26923/tid=26943] ppid=26921 vsize=647132 CPUtime=8.17 cores=0,2,4,6
/proc/26923/task/26943/stat : 26943 (plingeling) R 26921 26923 26140 0 -1 4202560 57234 1260 0 0 808 9 0 0 25 0 5 0 522730543 662663168 133908 33554432000 4194304 4417332 140737126547168 18446744073709551615 4255010 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26923/tid=26944] ppid=26921 vsize=647132 CPUtime=8.24 cores=0,2,4,6
/proc/26923/task/26944/stat : 26944 (plingeling) R 26921 26923 26140 0 -1 4202560 2000 1260 0 0 821 3 0 0 25 0 5 0 522730543 662663168 133908 33554432000 4194304 4417332 140737126547168 18446744073709551615 4366440 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 37.36
Current children cumulated vsize (KiB) 647132

[startup+25.5005 s]
/proc/loadavg: 4.64 5.19 6.62 9/186 26948
/proc/meminfo: memFree=23687100/32951124 swapFree=67051468/67111528
[pid=26923] ppid=26921 vsize=758224 CPUtime=88.49 cores=0,2,4,6
/proc/26923/stat : 26923 (plingeling) S 26921 26923 26140 0 -1 4202496 279256 1260 0 0 8799 50 0 0 25 0 5 0 522730098 776421376 161696 33554432000 4194304 4417332 140737126547168 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/26923/statm: 189556 161696 181 55 0 187442 0
[pid=26923/tid=26941] ppid=26921 vsize=758224 CPUtime=21.04 cores=0,2,4,6
/proc/26923/task/26941/stat : 26941 (plingeling) R 26921 26923 26140 0 -1 4202560 64815 1260 0 0 2094 10 0 0 25 0 5 0 522730543 776421376 161696 33554432000 4194304 4417332 140737126547168 18446744073709551615 4232666 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26923/tid=26942] ppid=26921 vsize=758224 CPUtime=21.04 cores=0,2,4,6
/proc/26923/task/26942/stat : 26942 (plingeling) R 26921 26923 26140 0 -1 4202560 59404 1260 0 0 2096 8 0 0 25 0 5 0 522730543 776421376 161696 33554432000 4194304 4417332 140737126547168 18446744073709551615 4236803 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26923/tid=26943] ppid=26921 vsize=758224 CPUtime=20.9 cores=0,2,4,6
/proc/26923/task/26943/stat : 26943 (plingeling) R 26921 26923 26140 0 -1 4202560 61042 1260 0 0 2081 9 0 0 25 0 5 0 522730543 776421376 161696 33554432000 4194304 4417332 140737126547168 18446744073709551615 4364509 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26923/tid=26944] ppid=26921 vsize=758224 CPUtime=21.04 cores=0,2,4,6
/proc/26923/task/26944/stat : 26944 (plingeling) R 26921 26923 26140 0 -1 4202560 2000 1260 0 0 2101 3 0 0 25 0 5 0 522730543 776421376 161696 33554432000 4194304 4417332 140737126547168 18446744073709551615 4364734 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 88.49
Current children cumulated vsize (KiB) 758224

[startup+51.1009 s]
/proc/loadavg: 5.85 5.43 6.66 9/186 26949
/proc/meminfo: memFree=23573072/32951124 swapFree=67051468/67111528
[pid=26923] ppid=26921 vsize=859520 CPUtime=190.75 cores=0,2,4,6
/proc/26923/stat : 26923 (plingeling) S 26921 26923 26140 0 -1 4202496 409731 1260 0 0 19008 67 0 0 25 0 5 0 522730098 880148480 186330 33554432000 4194304 4417332 140737126547168 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/26923/statm: 214880 186330 181 55 0 212766 0
[pid=26923/tid=26941] ppid=26921 vsize=859520 CPUtime=46.64 cores=0,2,4,6
/proc/26923/task/26941/stat : 26941 (plingeling) R 26921 26923 26140 0 -1 4202560 97665 1260 0 0 4650 14 0 0 25 0 5 0 522730543 880148480 186330 33554432000 4194304 4417332 140737126547168 18446744073709551615 4305778 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26923/tid=26942] ppid=26921 vsize=859520 CPUtime=46.64 cores=0,2,4,6
/proc/26923/task/26942/stat : 26942 (plingeling) R 26921 26923 26140 0 -1 4202560 99794 1260 0 0 4651 13 0 0 25 0 5 0 522730543 880148480 186330 33554432000 4194304 4417332 140737126547168 18446744073709551615 4335896 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26923/tid=26943] ppid=26921 vsize=859520 CPUtime=46.36 cores=0,2,4,6
/proc/26923/task/26943/stat : 26943 (plingeling) R 26921 26923 26140 0 -1 4202560 99514 1260 0 0 4621 15 0 0 25 0 5 0 522730543 880148480 186330 33554432000 4194304 4417332 140737126547168 18446744073709551615 4320251 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26923/tid=26944] ppid=26921 vsize=859520 CPUtime=46.63 cores=0,2,4,6
/proc/26923/task/26944/stat : 26944 (plingeling) R 26921 26923 26140 0 -1 4202560 20763 1260 0 0 4659 4 0 0 25 0 5 0 522730543 880148480 186330 33554432000 4194304 4417332 140737126547168 18446744073709551615 4358118 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 190.75
Current children cumulated vsize (KiB) 859520

[startup+102.307 s]
/proc/loadavg: 7.06 5.82 6.73 9/186 26950
/proc/meminfo: memFree=23548236/32951124 swapFree=67051468/67111528
[pid=26923] ppid=26921 vsize=837280 CPUtime=395.28 cores=0,2,4,6
/proc/26923/stat : 26923 (plingeling) S 26921 26923 26140 0 -1 4202496 514039 1260 0 0 39449 79 0 0 25 0 5 0 522730098 857374720 180770 33554432000 4194304 4417332 140737126547168 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/26923/statm: 209320 180770 181 55 0 207206 0
[pid=26923/tid=26941] ppid=26921 vsize=837280 CPUtime=97.85 cores=0,2,4,6
/proc/26923/task/26941/stat : 26941 (plingeling) R 26921 26923 26140 0 -1 4202560 132272 1260 0 0 9766 19 0 0 25 0 5 0 522730543 857374720 180770 33554432000 4194304 4417332 140737126547168 18446744073709551615 4265429 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26923/tid=26942] ppid=26921 vsize=837280 CPUtime=97.85 cores=0,2,4,6
/proc/26923/task/26942/stat : 26942 (plingeling) R 26921 26923 26140 0 -1 4202560 132636 1260 0 0 9768 17 0 0 25 0 5 0 522730543 857374720 180770 33554432000 4194304 4417332 140737126547168 18446744073709551615 4250651 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26923/tid=26943] ppid=26921 vsize=837280 CPUtime=97.29 cores=0,2,4,6
/proc/26923/task/26943/stat : 26943 (plingeling) R 26921 26923 26140 0 -1 4202560 136373 1260 0 0 9709 20 0 0 25 0 5 0 522730543 857374720 180770 33554432000 4194304 4417332 140737126547168 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26923/tid=26944] ppid=26921 vsize=837280 CPUtime=97.83 cores=0,2,4,6
/proc/26923/task/26944/stat : 26944 (plingeling) R 26921 26923 26140 0 -1 4202560 20763 1260 0 0 9779 4 0 0 25 0 5 0 522730543 857374720 180770 33554432000 4194304 4417332 140737126547168 18446744073709551615 4365846 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 395.28
Current children cumulated vsize (KiB) 837280

[startup+162.301 s]
/proc/loadavg: 7.69 6.23 6.81 9/186 26952
/proc/meminfo: memFree=23471712/32951124 swapFree=67051468/67111528
[pid=26923] ppid=26921 vsize=881040 CPUtime=634.94 cores=0,2,4,6
/proc/26923/stat : 26923 (plingeling) S 26921 26923 26140 0 -1 4202496 630085 1260 0 0 63401 93 0 0 25 0 5 0 522730098 902184960 191716 33554432000 4194304 4417332 140737126547168 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/26923/statm: 220260 191716 181 55 0 218146 0
[pid=26923/tid=26941] ppid=26921 vsize=881040 CPUtime=157.85 cores=0,2,4,6
/proc/26923/task/26941/stat : 26941 (plingeling) R 26921 26923 26140 0 -1 4202560 167165 1260 0 0 15762 23 0 0 25 0 5 0 522730543 902184960 191716 33554432000 4194304 4417332 140737126547168 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26923/tid=26942] ppid=26921 vsize=881040 CPUtime=157.84 cores=0,2,4,6
/proc/26923/task/26942/stat : 26942 (plingeling) R 26921 26923 26140 0 -1 4202560 166584 1260 0 0 15763 21 0 0 25 0 5 0 522730543 902184960 191716 33554432000 4194304 4417332 140737126547168 18446744073709551615 4236549 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26923/tid=26943] ppid=26921 vsize=881040 CPUtime=156.96 cores=0,2,4,6
/proc/26923/task/26943/stat : 26943 (plingeling) R 26921 26923 26140 0 -1 4202560 173024 1260 0 0 15672 24 0 0 25 0 5 0 522730543 902184960 191716 33554432000 4194304 4417332 140737126547168 18446744073709551615 4364375 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26923/tid=26944] ppid=26921 vsize=881040 CPUtime=157.83 cores=0,2,4,6
/proc/26923/task/26944/stat : 26944 (plingeling) R 26921 26923 26140 0 -1 4202560 31317 1260 0 0 15778 5 0 0 25 0 5 0 522730543 902184960 191716 33554432000 4194304 4417332 140737126547168 18446744073709551615 4354414 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 634.94
Current children cumulated vsize (KiB) 881040

[startup+222.3 s]
/proc/loadavg: 7.92 6.56 6.88 9/186 26953
/proc/meminfo: memFree=23426776/32951124 swapFree=67051468/67111528
[pid=26923] ppid=26921 vsize=905716 CPUtime=874.62 cores=0,2,4,6
/proc/26923/stat : 26923 (plingeling) S 26921 26923 26140 0 -1 4202496 656503 1260 0 0 87365 97 0 0 25 0 5 0 522730098 927453184 197885 33554432000 4194304 4417332 140737126547168 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/26923/statm: 226429 197885 181 55 0 224315 0
[pid=26923/tid=26941] ppid=26921 vsize=905716 CPUtime=217.84 cores=0,2,4,6
/proc/26923/task/26941/stat : 26941 (plingeling) R 26921 26923 26140 0 -1 4202560 167165 1260 0 0 21761 23 0 0 25 0 5 0 522730543 927453184 197885 33554432000 4194304 4417332 140737126547168 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26923/tid=26942] ppid=26921 vsize=905716 CPUtime=217.84 cores=0,2,4,6
/proc/26923/task/26942/stat : 26942 (plingeling) R 26921 26923 26140 0 -1 4202560 185083 1260 0 0 21761 23 0 0 25 0 5 0 522730543 927453184 197885 33554432000 4194304 4417332 140737126547168 18446744073709551615 4302940 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26923/tid=26943] ppid=26921 vsize=905716 CPUtime=216.63 cores=0,2,4,6
/proc/26923/task/26943/stat : 26943 (plingeling) R 26921 26923 26140 0 -1 4202560 175514 1260 0 0 21639 24 0 0 25 0 5 0 522730543 927453184 197885 33554432000 4194304 4417332 140737126547168 18446744073709551615 4235491 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26923/tid=26944] ppid=26921 vsize=905716 CPUtime=217.82 cores=0,2,4,6
/proc/26923/task/26944/stat : 26944 (plingeling) R 26921 26923 26140 0 -1 4202560 36746 1260 0 0 21777 5 0 0 25 0 5 0 522730543 927453184 197885 33554432000 4194304 4417332 140737126547168 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 874.62
Current children cumulated vsize (KiB) 905716

################
# More data... #
################

[pid=26923/tid=26942] ppid=26921 vsize=1544364 CPUtime=697.85 cores=0,2,4,6
/proc/26923/task/26942/stat : 26942 (plingeling) R 26921 26923 26140 0 -1 4202560 274584 1260 0 0 69751 34 0 0 25 0 5 0 522730543 1581428736 357547 33554432000 4194304 4417332 140737126547168 18446744073709551615 4236649 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26923/tid=26943] ppid=26921 vsize=1544364 CPUtime=694.03 cores=0,2,4,6
/proc/26923/task/26943/stat : 26943 (plingeling) R 26921 26923 26140 0 -1 4202560 269308 1260 0 0 69367 36 0 0 25 0 5 0 522730543 1581428736 357547 33554432000 4194304 4417332 140737126547168 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26923/tid=26944] ppid=26921 vsize=1544364 CPUtime=697.84 cores=0,2,4,6
/proc/26923/task/26944/stat : 26944 (plingeling) R 26921 26923 26140 0 -1 4202560 256516 1260 0 0 69751 33 0 0 25 0 5 0 522730543 1581428736 357547 33554432000 4194304 4417332 140737126547168 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 2792.05
Current children cumulated vsize (KiB) 1544364

[startup+762.306 s]
/proc/loadavg: 8.08 7.84 7.40 9/186 26968
/proc/meminfo: memFree=22715220/32951124 swapFree=67051468/67111528
[pid=26923] ppid=26921 vsize=1433900 CPUtime=3031.75 cores=0,2,4,6
/proc/26923/stat : 26923 (plingeling) S 26921 26923 26140 0 -1 4202496 1215635 1260 0 0 303006 169 0 0 25 0 5 0 522730098 1468313600 329931 33554432000 4194304 4417332 140737126547168 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/26923/statm: 358475 329931 181 55 0 356361 0
[pid=26923/tid=26941] ppid=26921 vsize=1433900 CPUtime=757.85 cores=0,2,4,6
/proc/26923/task/26941/stat : 26941 (plingeling) R 26921 26923 26140 0 -1 4202560 247927 1260 0 0 75752 33 0 0 25 0 5 0 522730543 1468313600 329931 33554432000 4194304 4417332 140737126547168 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26923/tid=26942] ppid=26921 vsize=1433900 CPUtime=757.85 cores=0,2,4,6
/proc/26923/task/26942/stat : 26942 (plingeling) R 26921 26923 26140 0 -1 4202560 306552 1260 0 0 75748 37 0 0 25 0 5 0 522730543 1468313600 329931 33554432000 4194304 4417332 140737126547168 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26923/tid=26943] ppid=26921 vsize=1433900 CPUtime=753.71 cores=0,2,4,6
/proc/26923/task/26943/stat : 26943 (plingeling) R 26921 26923 26140 0 -1 4202560 284654 1260 0 0 75333 38 0 0 25 0 5 0 522730543 1468313600 329931 33554432000 4194304 4417332 140737126547168 18446744073709551615 4237169 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26923/tid=26944] ppid=26921 vsize=1433900 CPUtime=757.84 cores=0,2,4,6
/proc/26923/task/26944/stat : 26944 (plingeling) R 26921 26923 26140 0 -1 4202560 284507 1260 0 0 75745 39 0 0 25 0 5 0 522730543 1468313600 329931 33554432000 4194304 4417332 140737126547168 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3031.75
Current children cumulated vsize (KiB) 1433900

[startup+822.301 s]
/proc/loadavg: 8.11 7.89 7.44 9/186 26969
/proc/meminfo: memFree=22645748/32951124 swapFree=67051468/67111528
[pid=26923] ppid=26921 vsize=1517100 CPUtime=3271.41 cores=0,2,4,6
/proc/26923/stat : 26923 (plingeling) S 26921 26923 26140 0 -1 4202496 1236504 1260 0 0 326969 172 0 0 25 0 5 0 522730098 1553510400 350731 33554432000 4194304 4417332 140737126547168 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/26923/statm: 379275 350731 181 55 0 377161 0
[pid=26923/tid=26941] ppid=26921 vsize=1517100 CPUtime=817.85 cores=0,2,4,6
/proc/26923/task/26941/stat : 26941 (plingeling) R 26921 26923 26140 0 -1 4202560 247927 1260 0 0 81751 34 0 0 25 0 5 0 522730543 1553510400 350731 33554432000 4194304 4417332 140737126547168 18446744073709551615 4232615 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26923/tid=26942] ppid=26921 vsize=1517100 CPUtime=817.85 cores=0,2,4,6
/proc/26923/task/26942/stat : 26942 (plingeling) R 26921 26923 26140 0 -1 4202560 306552 1260 0 0 81748 37 0 0 25 0 5 0 522730543 1553510400 350731 33554432000 4194304 4417332 140737126547168 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26923/tid=26943] ppid=26921 vsize=1517100 CPUtime=813.39 cores=0,2,4,6
/proc/26923/task/26943/stat : 26943 (plingeling) R 26921 26923 26140 0 -1 4202560 287093 1260 0 0 81300 39 0 0 25 0 5 0 522730543 1553510400 350731 33554432000 4194304 4417332 140737126547168 18446744073709551615 4236971 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26923/tid=26944] ppid=26921 vsize=1517100 CPUtime=817.84 cores=0,2,4,6
/proc/26923/task/26944/stat : 26944 (plingeling) R 26921 26923 26140 0 -1 4202560 302937 1260 0 0 81743 41 0 0 25 0 5 0 522730543 1553510400 350731 33554432000 4194304 4417332 140737126547168 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3271.41
Current children cumulated vsize (KiB) 1517100

[startup+882.301 s]
/proc/loadavg: 8.24 7.97 7.50 9/186 26971
/proc/meminfo: memFree=22687388/32951124 swapFree=67051468/67111528
[pid=26923] ppid=26921 vsize=1471272 CPUtime=3511.08 cores=0,2,4,6
/proc/26923/stat : 26923 (plingeling) S 26921 26923 26140 0 -1 4202496 1288616 1260 0 0 350929 179 0 0 25 0 5 0 522730098 1506582528 339274 33554432000 4194304 4417332 140737126547168 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/26923/statm: 367818 339274 181 55 0 365704 0
[pid=26923/tid=26941] ppid=26921 vsize=1471272 CPUtime=877.85 cores=0,2,4,6
/proc/26923/task/26941/stat : 26941 (plingeling) R 26921 26923 26140 0 -1 4202560 247927 1260 0 0 87751 34 0 0 25 0 5 0 522730543 1506582528 339274 33554432000 4194304 4417332 140737126547168 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26923/tid=26942] ppid=26921 vsize=1471272 CPUtime=877.85 cores=0,2,4,6
/proc/26923/task/26942/stat : 26942 (plingeling) R 26921 26923 26140 0 -1 4202560 315131 1260 0 0 87747 38 0 0 25 0 5 0 522730543 1506582528 339274 33554432000 4194304 4417332 140737126547168 18446744073709551615 4357123 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26923/tid=26943] ppid=26921 vsize=1471272 CPUtime=873.06 cores=0,2,4,6
/proc/26923/task/26943/stat : 26943 (plingeling) R 26921 26923 26140 0 -1 4202560 289311 1260 0 0 87267 39 0 0 25 0 5 0 522730543 1506582528 339274 33554432000 4194304 4417332 140737126547168 18446744073709551615 4366384 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26923/tid=26944] ppid=26921 vsize=1471272 CPUtime=877.85 cores=0,2,4,6
/proc/26923/task/26944/stat : 26944 (plingeling) R 26921 26923 26140 0 -1 4202560 344252 1260 0 0 87737 48 0 0 25 0 5 0 522730543 1506582528 339274 33554432000 4194304 4417332 140737126547168 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3511.08
Current children cumulated vsize (KiB) 1471272

[startup+942.301 s]
/proc/loadavg: 8.20 8.00 7.54 9/186 26972
/proc/meminfo: memFree=22558036/32951124 swapFree=67051468/67111528
[pid=26923] ppid=26921 vsize=1553016 CPUtime=3750.78 cores=0,2,4,6
/proc/26923/stat : 26923 (plingeling) S 26921 26923 26140 0 -1 4202496 1346807 1260 0 0 374891 187 0 0 25 0 5 0 522730098 1590288384 359710 33554432000 4194304 4417332 140737126547168 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/26923/statm: 388254 359710 181 55 0 386140 0
[pid=26923/tid=26941] ppid=26921 vsize=1553016 CPUtime=937.86 cores=0,2,4,6
/proc/26923/task/26941/stat : 26941 (plingeling) R 26921 26923 26140 0 -1 4202560 247988 1260 0 0 93752 34 0 0 25 0 5 0 522730543 1590288384 359710 33554432000 4194304 4417332 140737126547168 18446744073709551615 4364796 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26923/tid=26942] ppid=26921 vsize=1553016 CPUtime=937.86 cores=0,2,4,6
/proc/26923/task/26942/stat : 26942 (plingeling) R 26921 26923 26140 0 -1 4202560 317703 1260 0 0 93747 39 0 0 25 0 5 0 522730543 1590288384 359710 33554432000 4194304 4417332 140737126547168 18446744073709551615 4281536 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26923/tid=26943] ppid=26921 vsize=1553016 CPUtime=932.75 cores=0,2,4,6
/proc/26923/task/26943/stat : 26943 (plingeling) R 26921 26923 26140 0 -1 4202560 318181 1260 0 0 93232 43 0 0 25 0 5 0 522730543 1590288384 359710 33554432000 4194304 4417332 140737126547168 18446744073709551615 4236961 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26923/tid=26944] ppid=26921 vsize=1553016 CPUtime=937.85 cores=0,2,4,6
/proc/26923/task/26944/stat : 26944 (plingeling) R 26921 26923 26140 0 -1 4202560 370940 1260 0 0 93734 51 0 0 25 0 5 0 522730543 1590288384 359710 33554432000 4194304 4417332 140737126547168 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3750.78
Current children cumulated vsize (KiB) 1553016

[startup+1002.3 s]
/proc/loadavg: 8.18 8.03 7.57 9/186 26974
/proc/meminfo: memFree=22648392/32951124 swapFree=67051468/67111528
[pid=26923] ppid=26921 vsize=1402424 CPUtime=3990.45 cores=0,2,4,6
/proc/26923/stat : 26923 (plingeling) S 26921 26923 26140 0 -1 4202496 1413328 1260 0 0 398846 199 0 0 25 0 5 0 522730098 1436082176 322094 33554432000 4194304 4417332 140737126547168 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/26923/statm: 350606 322094 181 55 0 348492 0
[pid=26923/tid=26941] ppid=26921 vsize=1402424 CPUtime=997.86 cores=0,2,4,6
/proc/26923/task/26941/stat : 26941 (plingeling) R 26921 26923 26140 0 -1 4202560 256890 1260 0 0 99751 35 0 0 25 0 5 0 522730543 1436082176 322094 33554432000 4194304 4417332 140737126547168 18446744073709551615 4366303 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26923/tid=26942] ppid=26921 vsize=1402424 CPUtime=997.86 cores=0,2,4,6
/proc/26923/task/26942/stat : 26942 (plingeling) R 26921 26923 26140 0 -1 4202560 336459 1260 0 0 99744 42 0 0 25 0 5 0 522730543 1436082176 322094 33554432000 4194304 4417332 140737126547168 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26923/tid=26943] ppid=26921 vsize=1402424 CPUtime=992.42 cores=0,2,4,6
/proc/26923/task/26943/stat : 26943 (plingeling) R 26921 26923 26140 0 -1 4202560 318963 1260 0 0 99199 43 0 0 25 0 5 0 522730543 1436082176 322094 33554432000 4194304 4417332 140737126547168 18446744073709551615 4346152 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26923/tid=26944] ppid=26921 vsize=1402424 CPUtime=997.85 cores=0,2,4,6
/proc/26923/task/26944/stat : 26944 (plingeling) R 26921 26923 26140 0 -1 4202560 409021 1260 0 0 99726 59 0 0 25 0 5 0 522730543 1436082176 322094 33554432000 4194304 4417332 140737126547168 18446744073709551615 4236549 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3990.45
Current children cumulated vsize (KiB) 1402424

[startup+1062.3 s]
/proc/loadavg: 8.16 8.05 7.61 9/186 26976
/proc/meminfo: memFree=22306876/32951124 swapFree=67051468/67111528
[pid=26923] ppid=26921 vsize=1752032 CPUtime=4230.13 cores=0,2,4,6
/proc/26923/stat : 26923 (plingeling) S 26921 26923 26140 0 -1 4202496 1514264 1260 0 0 422802 211 0 0 25 0 5 0 522730098 1794080768 409496 33554432000 4194304 4417332 140737126547168 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/26923/statm: 438008 409496 181 55 0 435894 0
[pid=26923/tid=26941] ppid=26921 vsize=1752032 CPUtime=1057.86 cores=0,2,4,6
/proc/26923/task/26941/stat : 26941 (plingeling) R 26921 26923 26140 0 -1 4202560 259500 1260 0 0 105751 35 0 0 25 0 5 0 522730543 1794080768 409496 33554432000 4194304 4417332 140737126547168 18446744073709551615 4364568 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26923/tid=26942] ppid=26921 vsize=1752032 CPUtime=1057.86 cores=0,2,4,6
/proc/26923/task/26942/stat : 26942 (plingeling) R 26921 26923 26140 0 -1 4202560 336459 1260 0 0 105744 42 0 0 25 0 5 0 522730543 1794080768 409496 33554432000 4194304 4417332 140737126547168 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26923/tid=26943] ppid=26921 vsize=1752032 CPUtime=1052.1 cores=0,2,4,6
/proc/26923/task/26943/stat : 26943 (plingeling) R 26921 26923 26140 0 -1 4202560 353342 1260 0 0 105162 48 0 0 25 0 5 0 522730543 1794080768 409496 33554432000 4194304 4417332 140737126547168 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26923/tid=26944] ppid=26921 vsize=1752032 CPUtime=1057.85 cores=0,2,4,6
/proc/26923/task/26944/stat : 26944 (plingeling) R 26921 26923 26140 0 -1 4202560 472968 1260 0 0 105720 65 0 0 25 0 5 0 522730543 1794080768 409496 33554432000 4194304 4417332 140737126547168 18446744073709551615 4250599 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4230.13
Current children cumulated vsize (KiB) 1752032

[startup+1122.3 s]
/proc/loadavg: 8.15 8.07 7.64 9/186 26977
/proc/meminfo: memFree=22448552/32951124 swapFree=67051468/67111528
[pid=26923] ppid=26921 vsize=1611660 CPUtime=4469.81 cores=0,2,4,6
/proc/26923/stat : 26923 (plingeling) S 26921 26923 26140 0 -1 4202496 1575755 1260 0 0 446761 220 0 0 25 0 5 0 522730098 1650339840 374403 33554432000 4194304 4417332 140737126547168 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/26923/statm: 402915 374403 181 55 0 400801 0
[pid=26923/tid=26941] ppid=26921 vsize=1611660 CPUtime=1117.86 cores=0,2,4,6
/proc/26923/task/26941/stat : 26941 (plingeling) R 26921 26923 26140 0 -1 4202560 261610 1260 0 0 111751 35 0 0 25 0 5 0 522730543 1650339840 374403 33554432000 4194304 4417332 140737126547168 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26923/tid=26942] ppid=26921 vsize=1611660 CPUtime=1117.86 cores=0,2,4,6
/proc/26923/task/26942/stat : 26942 (plingeling) R 26921 26923 26140 0 -1 4202560 357037 1260 0 0 111742 44 0 0 25 0 5 0 522730543 1650339840 374403 33554432000 4194304 4417332 140737126547168 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26923/tid=26943] ppid=26921 vsize=1611660 CPUtime=1111.77 cores=0,2,4,6
/proc/26923/task/26943/stat : 26943 (plingeling) R 26921 26923 26140 0 -1 4202560 363183 1260 0 0 111128 49 0 0 25 0 5 0 522730543 1650339840 374403 33554432000 4194304 4417332 140737126547168 18446744073709551615 4232654 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26923/tid=26944] ppid=26921 vsize=1611660 CPUtime=1117.85 cores=0,2,4,6
/proc/26923/task/26944/stat : 26944 (plingeling) R 26921 26923 26140 0 -1 4202560 501930 1260 0 0 111714 71 0 0 25 0 5 0 522730543 1650339840 374403 33554432000 4194304 4417332 140737126547168 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4469.81
Current children cumulated vsize (KiB) 1611660

[startup+1182.3 s]
/proc/loadavg: 8.22 8.10 7.67 9/186 26979
/proc/meminfo: memFree=22393228/32951124 swapFree=67051468/67111528
[pid=26923] ppid=26921 vsize=1659044 CPUtime=4709.49 cores=0,2,4,6
/proc/26923/stat : 26923 (plingeling) S 26921 26923 26140 0 -1 4202496 1665985 1260 0 0 470715 234 0 0 25 0 5 0 522730098 1698861056 386249 33554432000 4194304 4417332 140737126547168 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/26923/statm: 414761 386249 181 55 0 412647 0
[pid=26923/tid=26941] ppid=26921 vsize=1659044 CPUtime=1177.86 cores=0,2,4,6
/proc/26923/task/26941/stat : 26941 (plingeling) R 26921 26923 26140 0 -1 4202560 261668 1260 0 0 117751 35 0 0 25 0 5 0 522730543 1698861056 386249 33554432000 4194304 4417332 140737126547168 18446744073709551615 4357123 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26923/tid=26942] ppid=26921 vsize=1659044 CPUtime=1177.87 cores=0,2,4,6
/proc/26923/task/26942/stat : 26942 (plingeling) R 26921 26923 26140 0 -1 4202560 386651 1260 0 0 117737 50 0 0 25 0 5 0 522730543 1698861056 386249 33554432000 4194304 4417332 140737126547168 18446744073709551615 4346430 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26923/tid=26943] ppid=26921 vsize=1659044 CPUtime=1171.44 cores=0,2,4,6
/proc/26923/task/26943/stat : 26943 (plingeling) R 26921 26923 26140 0 -1 4202560 388274 1260 0 0 117092 52 0 0 25 0 5 0 522730543 1698861056 386249 33554432000 4194304 4417332 140737126547168 18446744073709551615 4265443 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26923/tid=26944] ppid=26921 vsize=1659044 CPUtime=1177.85 cores=0,2,4,6
/proc/26923/task/26944/stat : 26944 (plingeling) R 26921 26923 26140 0 -1 4202560 537397 1260 0 0 117709 76 0 0 25 0 5 0 522730543 1698861056 386249 33554432000 4194304 4417332 140737126547168 18446744073709551615 4237011 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4709.49
Current children cumulated vsize (KiB) 1659044



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1205 s]
/proc/loadavg: 8.14 8.09 7.68 9/186 26980
/proc/meminfo: memFree=22344972/32951124 swapFree=67051468/67111528
[pid=26923] ppid=26921 vsize=1692328 CPUtime=4800.18 cores=0,2,4,6
/proc/26923/stat : 26923 (plingeling) S 26921 26923 26140 0 -1 4202496 1683055 1260 0 0 479782 236 0 0 25 0 5 0 522730098 1732943872 394570 33554432000 4194304 4417332 140737126547168 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 0 0 0 0
/proc/26923/statm: 423082 394570 181 55 0 420968 0
[pid=26923/tid=26941] ppid=26921 vsize=1692328 CPUtime=1200.56 cores=0,2,4,6
/proc/26923/task/26941/stat : 26941 (plingeling) R 26921 26923 26140 0 -1 4202560 261668 1260 0 0 120021 35 0 0 25 0 5 0 522730543 1732943872 394570 33554432000 4194304 4417332 140737126547168 18446744073709551615 4364465 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26923/tid=26942] ppid=26921 vsize=1692328 CPUtime=1200.57 cores=0,2,4,6
/proc/26923/task/26942/stat : 26942 (plingeling) R 26921 26923 26140 0 -1 4202560 403721 1260 0 0 120006 51 0 0 25 0 5 0 522730543 1732943872 394570 33554432000 4194304 4417332 140737126547168 18446744073709551615 4364938 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26923/tid=26943] ppid=26921 vsize=1692328 CPUtime=1194.02 cores=0,2,4,6
/proc/26923/task/26943/stat : 26943 (plingeling) R 26921 26923 26140 0 -1 4202560 388274 1260 0 0 119349 53 0 0 25 0 5 0 522730543 1732943872 394570 33554432000 4194304 4417332 140737126547168 18446744073709551615 4346425 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26923/tid=26944] ppid=26921 vsize=1692328 CPUtime=1200.56 cores=0,2,4,6
/proc/26923/task/26944/stat : 26944 (plingeling) R 26921 26923 26140 0 -1 4202560 537397 1260 0 0 119980 76 0 0 25 0 5 0 522730543 1732943872 394570 33554432000 4194304 4417332 140737126547168 18446744073709551615 4364531 0 0 4096 17442 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.18
Current children cumulated vsize (KiB) 1692328

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1205.08
CPU time (s): 4800.28
CPU user time (s): 4797.85
CPU system time (s): 2.43063
CPU usage (%): 398.338
Max. virtual memory (cumulated for all children) (KiB): 1948160

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4797.85
system time used= 2.43063
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1684315
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 153
involuntary context switches= 21010

runsolver used 1.84272 second user time and 4.70328 second system time

The end

Launcher Data 

Begin job on node101 at 2011-04-29 23:07:53
IDJOB=3293623
IDBENCH=88163
IDSOLVER=1605
FILE ID=node101/3293623-1304111273
RUNJOBID= node101-1304111273-26911
PBS_JOBID= 13170653
Free space on /tmp= 71920 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT-Race-2010-CNF/hardware-verification/velev/11pipe_11_ooo.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3293623-1304111273/watcher-3293623-1304111273 -o /tmp/evaluation-result-3293623-1304111273/solver-3293623-1304111273 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3293623-1304111273.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= f603f11fbfe855df381282b7bce010ea
RANDOM SEED=740692601

node101.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      25149576 kB
Buffers:       2030708 kB
Cached:        5288804 kB
SwapCached:      30452 kB
Active:        3259668 kB
Inactive:      4093716 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      25149576 kB
SwapTotal:    67111528 kB
SwapFree:     67051468 kB
Dirty:          115552 kB
Writeback:           0 kB
AnonPages:       24616 kB
Mapped:           8552 kB
Slab:           384560 kB
PageTables:       4360 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   188448 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71852 MiB
End job on node101 at 2011-04-29 23:28:00