Trace number 3273627

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587? (TO) 4800.37 1201.72

General information on the benchmark

Namecrafted/spence/sgen/
sgen3-n150-s89334302-unsat.cnf
MD5SUMe0b064ae815c6a8bb8a9898180562bec
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables153
Number of clauses316
Sum of the clauses size948
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3316
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3273627-1304083941/watcher-3273627-1304083941 -o /tmp/evaluation-result-3273627-1304083941/solver-3273627-1304083941 -C 4800 -W 1300 -M 15500 plingeling -t 4 HOME/instance-3273627-1304083941.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.47 8.00 8.04 2/172 23725
/proc/meminfo: memFree=25325756/32951124 swapFree=67051468/67111528
[pid=23724] ppid=23722 vsize=49836 CPUtime=0 cores=1,3,5,7
/proc/23724/stat : 23724 (plingeling) S 23722 23724 23289 0 -1 4202496 340 1260 0 0 0 0 0 0 18 0 5 0 519996885 51032064 266 33554432000 4194304 4417332 140736535217904 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/23724/statm: 12459 266 173 55 0 10345 0
[pid=23724/tid=23736] ppid=23722 vsize=49836 CPUtime=0 cores=1,3,5,7
/proc/23724/task/23736/stat : 23736 (plingeling) R 23722 23724 23289 0 -1 4202560 24 1260 0 0 0 0 0 0 19 0 5 0 519996886 51032064 270 33554432000 4194304 4417332 140736535217904 18446744073709551615 4252947 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23724/tid=23737] ppid=23722 vsize=49836 CPUtime=0 cores=1,3,5,7
/proc/23724/task/23737/stat : 23737 (plingeling) R 23722 23724 23289 0 -1 4202560 0 1260 0 0 0 0 0 0 19 0 5 0 519996886 51032064 270 33554432000 4194304 4417332 140736535217904 18446744073709551615 218371013617 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23724/tid=23738] ppid=23722 vsize=49836 CPUtime=0 cores=1,3,5,7
/proc/23724/task/23738/stat : 23738 (plingeling) R 23722 23724 23289 0 -1 4202560 1 1260 0 0 0 0 0 0 19 0 5 0 519996886 51032064 270 33554432000 4194304 4417332 140736535217904 18446744073709551615 4199406 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23724/tid=23739] ppid=23722 vsize=49836 CPUtime=0 cores=1,3,5,7
/proc/23724/task/23739/stat : 23739 (plingeling) R 23722 23724 23289 0 -1 4202560 12 1260 0 0 0 0 0 0 19 0 5 0 519996886 51032064 270 33554432000 4194304 4417332 140736535217904 18446744073709551615 218370651097 0 0 4096 17442 0 0 0 -1 3 0 0 0

[startup+0.0503531 s]
/proc/loadavg: 7.47 8.00 8.04 2/172 23725
/proc/meminfo: memFree=25325756/32951124 swapFree=67051468/67111528
[pid=23724] ppid=23722 vsize=117016 CPUtime=0.17 cores=1,3,5,7
/proc/23724/stat : 23724 (plingeling) S 23722 23724 23289 0 -1 4202496 783 1260 0 0 17 0 0 0 18 0 5 0 519996885 119824384 672 33554432000 4194304 4417332 140736535217904 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/23724/statm: 29254 672 178 55 0 27140 0
[pid=23724/tid=23736] ppid=23722 vsize=117016 CPUtime=0.04 cores=1,3,5,7
/proc/23724/task/23736/stat : 23736 (plingeling) R 23722 23724 23289 0 -1 4202560 65 1260 0 0 4 0 0 0 19 0 5 0 519996886 119824384 672 33554432000 4194304 4417332 140736535217904 18446744073709551615 4353403 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23724/tid=23737] ppid=23722 vsize=117016 CPUtime=0.04 cores=1,3,5,7
/proc/23724/task/23737/stat : 23737 (plingeling) R 23722 23724 23289 0 -1 4202560 51 1260 0 0 4 0 0 0 20 0 5 0 519996886 119824384 672 33554432000 4194304 4417332 140736535217904 18446744073709551615 18446744073699065863 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23724/tid=23738] ppid=23722 vsize=117016 CPUtime=0.04 cores=1,3,5,7
/proc/23724/task/23738/stat : 23738 (plingeling) R 23722 23724 23289 0 -1 4202560 193 1260 0 0 4 0 0 0 20 0 5 0 519996886 119824384 672 33554432000 4194304 4417332 140736535217904 18446744073709551615 4250559 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23724/tid=23739] ppid=23722 vsize=117016 CPUtime=0.04 cores=1,3,5,7
/proc/23724/task/23739/stat : 23739 (plingeling) R 23722 23724 23289 0 -1 4202560 167 1260 0 0 4 0 0 0 19 0 5 0 519996886 119824384 672 33554432000 4194304 4417332 140736535217904 18446744073709551615 218370651117 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 0.17
Current children cumulated vsize (KiB) 117016

[startup+0.100344 s]
/proc/loadavg: 7.47 8.00 8.04 2/172 23725
/proc/meminfo: memFree=25325756/32951124 swapFree=67051468/67111528
[pid=23724] ppid=23722 vsize=119020 CPUtime=0.37 cores=1,3,5,7
/proc/23724/stat : 23724 (plingeling) S 23722 23724 23289 0 -1 4202496 1491 1260 0 0 37 0 0 0 18 0 5 0 519996885 121876480 1201 33554432000 4194304 4417332 140736535217904 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/23724/statm: 29755 1201 178 55 0 27641 0
[pid=23724/tid=23736] ppid=23722 vsize=119020 CPUtime=0.09 cores=1,3,5,7
/proc/23724/task/23736/stat : 23736 (plingeling) R 23722 23724 23289 0 -1 4202560 261 1260 0 0 9 0 0 0 19 0 5 0 519996886 121876480 1201 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23724/tid=23737] ppid=23722 vsize=119020 CPUtime=0.09 cores=1,3,5,7
/proc/23724/task/23737/stat : 23737 (plingeling) R 23722 23724 23289 0 -1 4202560 153 1260 0 0 9 0 0 0 20 0 5 0 519996886 121876480 1201 33554432000 4194304 4417332 140736535217904 18446744073709551615 4354645 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23724/tid=23738] ppid=23722 vsize=119020 CPUtime=0.09 cores=1,3,5,7
/proc/23724/task/23738/stat : 23738 (plingeling) R 23722 23724 23289 0 -1 4202560 465 1260 0 0 9 0 0 0 20 0 5 0 519996886 121876480 1201 33554432000 4194304 4417332 140736535217904 18446744073709551615 4358825 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23724/tid=23739] ppid=23722 vsize=119020 CPUtime=0.09 cores=1,3,5,7
/proc/23724/task/23739/stat : 23739 (plingeling) R 23722 23724 23289 0 -1 4202560 305 1260 0 0 9 0 0 0 19 0 5 0 519996886 121876480 1201 33554432000 4194304 4417332 140736535217904 18446744073709551615 4320880 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 0.37
Current children cumulated vsize (KiB) 119020

[startup+0.300309 s]
/proc/loadavg: 7.47 8.00 8.04 2/172 23725
/proc/meminfo: memFree=25325756/32951124 swapFree=67051468/67111528
[pid=23724] ppid=23722 vsize=121200 CPUtime=1.17 cores=1,3,5,7
/proc/23724/stat : 23724 (plingeling) S 23722 23724 23289 0 -1 4202496 3577 1260 0 0 117 0 0 0 18 0 5 0 519996885 124108800 1750 33554432000 4194304 4417332 140736535217904 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/23724/statm: 30300 1750 178 55 0 28186 0
[pid=23724/tid=23736] ppid=23722 vsize=121200 CPUtime=0.29 cores=1,3,5,7
/proc/23724/task/23736/stat : 23736 (plingeling) R 23722 23724 23289 0 -1 4202560 809 1260 0 0 29 0 0 0 20 0 5 0 519996886 124108800 1750 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364973 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23724/tid=23737] ppid=23722 vsize=121200 CPUtime=0.28 cores=1,3,5,7
/proc/23724/task/23737/stat : 23737 (plingeling) R 23722 23724 23289 0 -1 4202560 699 1260 0 0 28 0 0 0 20 0 5 0 519996886 124108800 1750 33554432000 4194304 4417332 140736535217904 18446744073709551615 4355764 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23724/tid=23738] ppid=23722 vsize=121200 CPUtime=0.29 cores=1,3,5,7
/proc/23724/task/23738/stat : 23738 (plingeling) R 23722 23724 23289 0 -1 4202560 1070 1260 0 0 29 0 0 0 20 0 5 0 519996886 124108800 1750 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364463 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23724/tid=23739] ppid=23722 vsize=121200 CPUtime=0.29 cores=1,3,5,7
/proc/23724/task/23739/stat : 23739 (plingeling) R 23722 23724 23289 0 -1 4202560 692 1260 0 0 29 0 0 0 20 0 5 0 519996886 124108800 1750 33554432000 4194304 4417332 140736535217904 18446744073709551615 4321585 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 1.17
Current children cumulated vsize (KiB) 121200

[startup+0.700238 s]
/proc/loadavg: 7.47 8.00 8.04 2/172 23725
/proc/meminfo: memFree=25325756/32951124 swapFree=67051468/67111528
[pid=23724] ppid=23722 vsize=123028 CPUtime=2.78 cores=1,3,5,7
/proc/23724/stat : 23724 (plingeling) S 23722 23724 23289 0 -1 4202496 6513 1260 0 0 277 1 0 0 18 0 5 0 519996885 125980672 2176 33554432000 4194304 4417332 140736535217904 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/23724/statm: 30757 2176 179 55 0 28643 0
[pid=23724/tid=23736] ppid=23722 vsize=123028 CPUtime=0.69 cores=1,3,5,7
/proc/23724/task/23736/stat : 23736 (plingeling) R 23722 23724 23289 0 -1 4202560 1775 1260 0 0 69 0 0 0 21 0 5 0 519996886 125980672 2176 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364742 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23724/tid=23737] ppid=23722 vsize=123028 CPUtime=0.68 cores=1,3,5,7
/proc/23724/task/23737/stat : 23737 (plingeling) R 23722 23724 23289 0 -1 4202560 1312 1260 0 0 68 0 0 0 21 0 5 0 519996886 125980672 2176 33554432000 4194304 4417332 140736535217904 18446744073709551615 4232867 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23724/tid=23738] ppid=23722 vsize=123028 CPUtime=0.69 cores=1,3,5,7
/proc/23724/task/23738/stat : 23738 (plingeling) R 23722 23724 23289 0 -1 4202560 1855 1260 0 0 69 0 0 0 21 0 5 0 519996886 125980672 2176 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364732 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23724/tid=23739] ppid=23722 vsize=123028 CPUtime=0.69 cores=1,3,5,7
/proc/23724/task/23739/stat : 23739 (plingeling) R 23722 23724 23289 0 -1 4202560 1264 1260 0 0 69 0 0 0 21 0 5 0 519996886 125980672 2176 33554432000 4194304 4417332 140736535217904 18446744073709551615 4250432 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 2.78
Current children cumulated vsize (KiB) 123028

[startup+1.5011 s]
/proc/loadavg: 7.47 8.00 8.04 9/186 23749
/proc/meminfo: memFree=25303644/32951124 swapFree=67051468/67111528
[pid=23724] ppid=23722 vsize=126716 CPUtime=5.97 cores=1,3,5,7
/proc/23724/stat : 23724 (plingeling) S 23722 23724 23289 0 -1 4202496 12710 1260 0 0 595 2 0 0 18 0 5 0 519996885 129757184 3130 33554432000 4194304 4417332 140736535217904 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/23724/statm: 31679 3130 179 55 0 29565 0
[pid=23724/tid=23736] ppid=23722 vsize=126716 CPUtime=1.49 cores=1,3,5,7
/proc/23724/task/23736/stat : 23736 (plingeling) R 23722 23724 23289 0 -1 4202560 3258 1260 0 0 149 0 0 0 23 0 5 0 519996886 129757184 3130 33554432000 4194304 4417332 140736535217904 18446744073709551615 18446744073699065939 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23724/tid=23737] ppid=23722 vsize=126716 CPUtime=1.47 cores=1,3,5,7
/proc/23724/task/23737/stat : 23737 (plingeling) R 23722 23724 23289 0 -1 4202560 2992 1260 0 0 147 0 0 0 23 0 5 0 519996886 129757184 3130 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364742 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23724/tid=23738] ppid=23722 vsize=126716 CPUtime=1.48 cores=1,3,5,7
/proc/23724/task/23738/stat : 23738 (plingeling) R 23722 23724 23289 0 -1 4202560 3560 1260 0 0 148 0 0 0 23 0 5 0 519996886 129757184 3130 33554432000 4194304 4417332 140736535217904 18446744073709551615 4365960 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23724/tid=23739] ppid=23722 vsize=126716 CPUtime=1.49 cores=1,3,5,7
/proc/23724/task/23739/stat : 23739 (plingeling) R 23722 23724 23289 0 -1 4202560 2593 1260 0 0 149 0 0 0 23 0 5 0 519996886 129757184 3130 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364502 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 5.97
Current children cumulated vsize (KiB) 126716

[startup+3.10081 s]
/proc/loadavg: 7.47 8.00 8.04 9/186 23749
/proc/meminfo: memFree=25294840/32951124 swapFree=67051468/67111528
[pid=23724] ppid=23722 vsize=132072 CPUtime=12.36 cores=1,3,5,7
/proc/23724/stat : 23724 (plingeling) S 23722 23724 23289 0 -1 4202496 24209 1260 0 0 1232 4 0 0 18 0 5 0 519996885 135241728 4469 33554432000 4194304 4417332 140736535217904 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/23724/statm: 33018 4469 179 55 0 30904 0
[pid=23724/tid=23736] ppid=23722 vsize=132072 CPUtime=3.08 cores=1,3,5,7
/proc/23724/task/23736/stat : 23736 (plingeling) R 23722 23724 23289 0 -1 4202560 5866 1260 0 0 308 0 0 0 25 0 5 0 519996886 135241728 4469 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364680 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23724/tid=23737] ppid=23722 vsize=132072 CPUtime=3.07 cores=1,3,5,7
/proc/23724/task/23737/stat : 23737 (plingeling) R 23722 23724 23289 0 -1 4202560 6200 1260 0 0 306 1 0 0 25 0 5 0 519996886 135241728 4469 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23724/tid=23738] ppid=23722 vsize=132072 CPUtime=3.09 cores=1,3,5,7
/proc/23724/task/23738/stat : 23738 (plingeling) R 23722 23724 23289 0 -1 4202560 6412 1260 0 0 308 1 0 0 25 0 5 0 519996886 135241728 4469 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364526 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23724/tid=23739] ppid=23722 vsize=132072 CPUtime=3.08 cores=1,3,5,7
/proc/23724/task/23739/stat : 23739 (plingeling) R 23722 23724 23289 0 -1 4202560 5424 1260 0 0 308 0 0 0 25 0 5 0 519996886 135241728 4469 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364732 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 12.36
Current children cumulated vsize (KiB) 132072

[startup+6.30024 s]
/proc/loadavg: 7.51 8.00 8.04 9/186 23749
/proc/meminfo: memFree=25281080/32951124 swapFree=67051468/67111528
[pid=23724] ppid=23722 vsize=138632 CPUtime=25.15 cores=1,3,5,7
/proc/23724/stat : 23724 (plingeling) S 23722 23724 23289 0 -1 4202496 42762 1260 0 0 2507 8 0 0 18 0 5 0 519996885 141959168 6109 33554432000 4194304 4417332 140736535217904 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/23724/statm: 34658 6109 179 55 0 32544 0
[pid=23724/tid=23736] ppid=23722 vsize=138632 CPUtime=6.28 cores=1,3,5,7
/proc/23724/task/23736/stat : 23736 (plingeling) R 23722 23724 23289 0 -1 4202560 10043 1260 0 0 627 1 0 0 25 0 5 0 519996886 141959168 6109 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364470 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23724/tid=23737] ppid=23722 vsize=138632 CPUtime=6.26 cores=1,3,5,7
/proc/23724/task/23737/stat : 23737 (plingeling) R 23722 23724 23289 0 -1 4202560 11150 1260 0 0 624 2 0 0 25 0 5 0 519996886 141959168 6109 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23724/tid=23738] ppid=23722 vsize=138632 CPUtime=6.29 cores=1,3,5,7
/proc/23724/task/23738/stat : 23738 (plingeling) R 23722 23724 23289 0 -1 4202560 11920 1260 0 0 627 2 0 0 25 0 5 0 519996886 141959168 6109 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23724/tid=23739] ppid=23722 vsize=138632 CPUtime=6.28 cores=1,3,5,7
/proc/23724/task/23739/stat : 23739 (plingeling) R 23722 23724 23289 0 -1 4202560 9342 1260 0 0 627 1 0 0 25 0 5 0 519996886 141959168 6109 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 25.15
Current children cumulated vsize (KiB) 138632

[startup+12.7011 s]
/proc/loadavg: 7.55 8.00 8.04 9/186 23749
/proc/meminfo: memFree=25259752/32951124 swapFree=67051468/67111528
[pid=23724] ppid=23722 vsize=141588 CPUtime=50.71 cores=1,3,5,7
/proc/23724/stat : 23724 (plingeling) S 23722 23724 23289 0 -1 4202496 70480 1260 0 0 5058 13 0 0 18 0 5 0 519996885 144986112 6816 33554432000 4194304 4417332 140736535217904 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/23724/statm: 35397 6816 179 55 0 33283 0
[pid=23724/tid=23736] ppid=23722 vsize=141588 CPUtime=12.69 cores=1,3,5,7
/proc/23724/task/23736/stat : 23736 (plingeling) R 23722 23724 23289 0 -1 4202560 17035 1260 0 0 1266 3 0 0 25 0 5 0 519996886 144986112 6816 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23724/tid=23737] ppid=23722 vsize=141588 CPUtime=12.61 cores=1,3,5,7
/proc/23724/task/23737/stat : 23737 (plingeling) R 23722 23724 23289 0 -1 4202560 19221 1260 0 0 1258 3 0 0 25 0 5 0 519996886 144986112 6816 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364526 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23724/tid=23738] ppid=23722 vsize=141588 CPUtime=12.69 cores=1,3,5,7
/proc/23724/task/23738/stat : 23738 (plingeling) R 23722 23724 23289 0 -1 4202560 18265 1260 0 0 1266 3 0 0 25 0 5 0 519996886 144986112 6816 33554432000 4194304 4417332 140736535217904 18446744073709551615 4352452 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23724/tid=23739] ppid=23722 vsize=141588 CPUtime=12.69 cores=1,3,5,7
/proc/23724/task/23739/stat : 23739 (plingeling) R 23722 23724 23289 0 -1 4202560 15652 1260 0 0 1266 3 0 0 25 0 5 0 519996886 144986112 6816 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 50.71
Current children cumulated vsize (KiB) 141588

[startup+25.5008 s]
/proc/loadavg: 7.65 8.00 8.03 9/186 23749
/proc/meminfo: memFree=25243752/32951124 swapFree=67051468/67111528
[pid=23724] ppid=23722 vsize=151760 CPUtime=101.84 cores=1,3,5,7
/proc/23724/stat : 23724 (plingeling) S 23722 23724 23289 0 -1 4202496 119486 1260 0 0 10162 22 0 0 18 0 5 0 519996885 155402240 9359 33554432000 4194304 4417332 140736535217904 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/23724/statm: 37940 9359 179 55 0 35826 0
[pid=23724/tid=23736] ppid=23722 vsize=151760 CPUtime=25.49 cores=1,3,5,7
/proc/23724/task/23736/stat : 23736 (plingeling) R 23722 23724 23289 0 -1 4202560 29109 1260 0 0 2544 5 0 0 25 0 5 0 519996886 155402240 9359 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364470 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23724/tid=23737] ppid=23722 vsize=151760 CPUtime=25.34 cores=1,3,5,7
/proc/23724/task/23737/stat : 23737 (plingeling) R 23722 23724 23289 0 -1 4202560 31645 1260 0 0 2529 5 0 0 25 0 5 0 519996886 155402240 9359 33554432000 4194304 4417332 140736535217904 18446744073709551615 18446744073699065904 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23724/tid=23738] ppid=23722 vsize=151760 CPUtime=25.49 cores=1,3,5,7
/proc/23724/task/23738/stat : 23738 (plingeling) R 23722 23724 23289 0 -1 4202560 31790 1260 0 0 2543 6 0 0 25 0 5 0 519996886 155402240 9359 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364973 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23724/tid=23739] ppid=23722 vsize=151760 CPUtime=25.48 cores=1,3,5,7
/proc/23724/task/23739/stat : 23739 (plingeling) R 23722 23724 23289 0 -1 4202560 26635 1260 0 0 2544 4 0 0 25 0 5 0 519996886 155402240 9359 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364568 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 101.84
Current children cumulated vsize (KiB) 151760

[startup+51.1003 s]
/proc/loadavg: 7.77 8.00 8.03 9/186 23750
/proc/meminfo: memFree=25218088/32951124 swapFree=67051468/67111528
[pid=23724] ppid=23722 vsize=156944 CPUtime=204.1 cores=1,3,5,7
/proc/23724/stat : 23724 (plingeling) S 23722 23724 23289 0 -1 4202496 196054 1260 0 0 20375 35 0 0 18 0 5 0 519996885 160710656 10666 33554432000 4194304 4417332 140736535217904 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/23724/statm: 39236 10666 179 55 0 37122 0
[pid=23724/tid=23736] ppid=23722 vsize=156944 CPUtime=51.09 cores=1,3,5,7
/proc/23724/task/23736/stat : 23736 (plingeling) R 23722 23724 23289 0 -1 4202560 50619 1260 0 0 5101 8 0 0 25 0 5 0 519996886 160710656 10666 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23724/tid=23737] ppid=23722 vsize=156944 CPUtime=50.8 cores=1,3,5,7
/proc/23724/task/23737/stat : 23737 (plingeling) R 23722 23724 23289 0 -1 4202560 54109 1260 0 0 5071 9 0 0 25 0 5 0 519996886 160710656 10666 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364736 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23724/tid=23738] ppid=23722 vsize=156944 CPUtime=51.08 cores=1,3,5,7

################
# More data... #
################

[pid=23724/tid=23737] ppid=23722 vsize=291008 CPUtime=758.13 cores=1,3,5,7
/proc/23724/task/23737/stat : 23737 (plingeling) R 23722 23724 23289 0 -1 4202560 340709 1260 0 0 75758 55 0 0 25 0 5 0 519996886 297992192 44171 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364656 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23724/tid=23738] ppid=23722 vsize=291008 CPUtime=762.31 cores=1,3,5,7
/proc/23724/task/23738/stat : 23738 (plingeling) R 23722 23724 23289 0 -1 4202560 344403 1260 0 0 76180 51 0 0 25 0 5 0 519996886 297992192 44171 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364549 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23724/tid=23739] ppid=23722 vsize=291008 CPUtime=762.31 cores=1,3,5,7
/proc/23724/task/23739/stat : 23739 (plingeling) R 23722 23724 23289 0 -1 4202560 304549 1260 0 0 76182 49 0 0 25 0 5 0 519996886 297992192 44171 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3045.1
Current children cumulated vsize (KiB) 291008

[startup+822.301 s]
/proc/loadavg: 8.06 8.04 8.01 9/186 23770
/proc/meminfo: memFree=24984632/32951124 swapFree=67051468/67111528
[pid=23724] ppid=23722 vsize=265696 CPUtime=3284.76 cores=1,3,5,7
/proc/23724/stat : 23724 (plingeling) S 23722 23724 23289 0 -1 4202496 1426811 1260 0 0 328249 227 0 0 18 0 5 0 519996885 272072704 37843 33554432000 4194304 4417332 140736535217904 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/23724/statm: 66424 37843 179 55 0 64310 0
[pid=23724/tid=23736] ppid=23722 vsize=265696 CPUtime=822.31 cores=1,3,5,7
/proc/23724/task/23736/stat : 23736 (plingeling) R 23722 23724 23289 0 -1 4202560 377118 1260 0 0 82170 61 0 0 25 0 5 0 519996886 272072704 37843 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23724/tid=23737] ppid=23722 vsize=265696 CPUtime=817.81 cores=1,3,5,7
/proc/23724/task/23737/stat : 23737 (plingeling) R 23722 23724 23289 0 -1 4202560 361079 1260 0 0 81722 59 0 0 25 0 5 0 519996886 272072704 37843 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364822 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23724/tid=23738] ppid=23722 vsize=265696 CPUtime=822.3 cores=1,3,5,7
/proc/23724/task/23738/stat : 23738 (plingeling) R 23722 23724 23289 0 -1 4202560 362822 1260 0 0 82177 53 0 0 25 0 5 0 519996886 272072704 37843 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23724/tid=23739] ppid=23722 vsize=265696 CPUtime=822.3 cores=1,3,5,7
/proc/23724/task/23739/stat : 23739 (plingeling) R 23722 23724 23289 0 -1 4202560 325485 1260 0 0 82178 52 0 0 25 0 5 0 519996886 272072704 37843 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364526 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3284.76
Current children cumulated vsize (KiB) 265696

[startup+882.301 s]
/proc/loadavg: 8.06 8.05 8.01 9/186 23772
/proc/meminfo: memFree=25041260/32951124 swapFree=67051468/67111528
[pid=23724] ppid=23722 vsize=254264 CPUtime=3524.43 cores=1,3,5,7
/proc/23724/stat : 23724 (plingeling) S 23722 23724 23289 0 -1 4202496 1510624 1260 0 0 352205 238 0 0 18 0 5 0 519996885 260366336 35010 33554432000 4194304 4417332 140736535217904 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/23724/statm: 63566 35010 179 55 0 61452 0
[pid=23724/tid=23736] ppid=23722 vsize=254264 CPUtime=882.31 cores=1,3,5,7
/proc/23724/task/23736/stat : 23736 (plingeling) R 23722 23724 23289 0 -1 4202560 395643 1260 0 0 88168 63 0 0 25 0 5 0 519996886 260366336 35010 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23724/tid=23737] ppid=23722 vsize=254264 CPUtime=877.47 cores=1,3,5,7
/proc/23724/task/23737/stat : 23737 (plingeling) R 23722 23724 23289 0 -1 4202560 385092 1260 0 0 87685 62 0 0 25 0 5 0 519996886 260366336 35010 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23724/tid=23738] ppid=23722 vsize=254264 CPUtime=882.3 cores=1,3,5,7
/proc/23724/task/23738/stat : 23738 (plingeling) R 23722 23724 23289 0 -1 4202560 385677 1260 0 0 88174 56 0 0 25 0 5 0 519996886 260366336 35010 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364479 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23724/tid=23739] ppid=23722 vsize=254264 CPUtime=882.31 cores=1,3,5,7
/proc/23724/task/23739/stat : 23739 (plingeling) R 23722 23724 23289 0 -1 4202560 343905 1260 0 0 88176 55 0 0 25 0 5 0 519996886 260366336 35010 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364526 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3524.43
Current children cumulated vsize (KiB) 254264

[startup+942.3 s]
/proc/loadavg: 8.11 8.06 8.01 9/186 23774
/proc/meminfo: memFree=24996336/32951124 swapFree=67051468/67111528
[pid=23724] ppid=23722 vsize=277764 CPUtime=3764.11 cores=1,3,5,7
/proc/23724/stat : 23724 (plingeling) S 23722 23724 23289 0 -1 4202496 1587844 1260 0 0 376162 249 0 0 18 0 5 0 519996885 284430336 40878 33554432000 4194304 4417332 140736535217904 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/23724/statm: 69441 40878 179 55 0 67327 0
[pid=23724/tid=23736] ppid=23722 vsize=277764 CPUtime=942.31 cores=1,3,5,7
/proc/23724/task/23736/stat : 23736 (plingeling) R 23722 23724 23289 0 -1 4202560 419860 1260 0 0 94164 67 0 0 25 0 5 0 519996886 284430336 40878 33554432000 4194304 4417332 140736535217904 18446744073709551615 4365977 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23724/tid=23737] ppid=23722 vsize=277764 CPUtime=937.15 cores=1,3,5,7
/proc/23724/task/23737/stat : 23737 (plingeling) R 23722 23724 23289 0 -1 4202560 404505 1260 0 0 93649 66 0 0 25 0 5 0 519996886 284430336 40878 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23724/tid=23738] ppid=23722 vsize=277764 CPUtime=942.31 cores=1,3,5,7
/proc/23724/task/23738/stat : 23738 (plingeling) R 23722 23724 23289 0 -1 4202560 402680 1260 0 0 94173 58 0 0 25 0 5 0 519996886 284430336 40878 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364560 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23724/tid=23739] ppid=23722 vsize=277764 CPUtime=942.31 cores=1,3,5,7
/proc/23724/task/23739/stat : 23739 (plingeling) R 23722 23724 23289 0 -1 4202560 360492 1260 0 0 94175 56 0 0 25 0 5 0 519996886 284430336 40878 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3764.11
Current children cumulated vsize (KiB) 277764

[startup+1002.3 s]
/proc/loadavg: 8.12 8.08 8.02 9/186 23775
/proc/meminfo: memFree=24979048/32951124 swapFree=67051468/67111528
[pid=23724] ppid=23722 vsize=299456 CPUtime=4003.79 cores=1,3,5,7
/proc/23724/stat : 23724 (plingeling) S 23722 23724 23289 0 -1 4202496 1677884 1260 0 0 400116 263 0 0 18 0 5 0 519996885 306642944 46283 33554432000 4194304 4417332 140736535217904 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/23724/statm: 74864 46283 179 55 0 72750 0
[pid=23724/tid=23736] ppid=23722 vsize=299456 CPUtime=1002.31 cores=1,3,5,7
/proc/23724/task/23736/stat : 23736 (plingeling) R 23722 23724 23289 0 -1 4202560 444557 1260 0 0 100160 71 0 0 25 0 5 0 519996886 306642944 46283 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364526 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23724/tid=23737] ppid=23722 vsize=299456 CPUtime=996.82 cores=1,3,5,7
/proc/23724/task/23737/stat : 23737 (plingeling) R 23722 23724 23289 0 -1 4202560 430679 1260 0 0 99612 70 0 0 25 0 5 0 519996886 306642944 46283 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23724/tid=23738] ppid=23722 vsize=299456 CPUtime=1002.31 cores=1,3,5,7
/proc/23724/task/23738/stat : 23738 (plingeling) R 23722 23724 23289 0 -1 4202560 428489 1260 0 0 100169 62 0 0 25 0 5 0 519996886 306642944 46283 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23724/tid=23739] ppid=23722 vsize=299456 CPUtime=1002.31 cores=1,3,5,7
/proc/23724/task/23739/stat : 23739 (plingeling) R 23722 23724 23289 0 -1 4202560 373852 1260 0 0 100173 58 0 0 25 0 5 0 519996886 306642944 46283 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364652 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4003.79
Current children cumulated vsize (KiB) 299456

[startup+1062.3 s]
/proc/loadavg: 8.08 8.08 8.02 9/186 23777
/proc/meminfo: memFree=24996856/32951124 swapFree=67051468/67111528
[pid=23724] ppid=23722 vsize=275984 CPUtime=4243.46 cores=1,3,5,7
/proc/23724/stat : 23724 (plingeling) S 23722 23724 23289 0 -1 4202496 1752318 1260 0 0 424073 273 0 0 18 0 5 0 519996885 282607616 40425 33554432000 4194304 4417332 140736535217904 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/23724/statm: 68996 40425 179 55 0 66882 0
[pid=23724/tid=23736] ppid=23722 vsize=275984 CPUtime=1062.31 cores=1,3,5,7
/proc/23724/task/23736/stat : 23736 (plingeling) R 23722 23724 23289 0 -1 4202560 465995 1260 0 0 106158 73 0 0 25 0 5 0 519996886 282607616 40425 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23724/tid=23737] ppid=23722 vsize=275984 CPUtime=1056.49 cores=1,3,5,7
/proc/23724/task/23737/stat : 23737 (plingeling) R 23722 23724 23289 0 -1 4202560 448719 1260 0 0 105576 73 0 0 25 0 5 0 519996886 282607616 40425 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23724/tid=23738] ppid=23722 vsize=275984 CPUtime=1062.31 cores=1,3,5,7
/proc/23724/task/23738/stat : 23738 (plingeling) R 23722 23724 23289 0 -1 4202560 444187 1260 0 0 106166 65 0 0 25 0 5 0 519996886 282607616 40425 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364565 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23724/tid=23739] ppid=23722 vsize=275984 CPUtime=1062.32 cores=1,3,5,7
/proc/23724/task/23739/stat : 23739 (plingeling) R 23722 23724 23289 0 -1 4202560 393110 1260 0 0 106171 61 0 0 25 0 5 0 519996886 282607616 40425 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364722 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4243.46
Current children cumulated vsize (KiB) 275984

[startup+1122.3 s]
/proc/loadavg: 8.10 8.09 8.02 9/186 23778
/proc/meminfo: memFree=24949324/32951124 swapFree=67051468/67111528
[pid=23724] ppid=23722 vsize=303444 CPUtime=4483.14 cores=1,3,5,7
/proc/23724/stat : 23724 (plingeling) S 23722 23724 23289 0 -1 4202496 1829373 1260 0 0 448029 285 0 0 18 0 5 0 519996885 310726656 47294 33554432000 4194304 4417332 140736535217904 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/23724/statm: 75861 47294 179 55 0 73747 0
[pid=23724/tid=23736] ppid=23722 vsize=303444 CPUtime=1122.31 cores=1,3,5,7
/proc/23724/task/23736/stat : 23736 (plingeling) R 23722 23724 23289 0 -1 4202560 480029 1260 0 0 112156 75 0 0 25 0 5 0 519996886 310726656 47294 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364448 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23724/tid=23737] ppid=23722 vsize=303444 CPUtime=1116.16 cores=1,3,5,7
/proc/23724/task/23737/stat : 23737 (plingeling) R 23722 23724 23289 0 -1 4202560 469425 1260 0 0 111540 76 0 0 25 0 5 0 519996886 310726656 47294 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23724/tid=23738] ppid=23722 vsize=303444 CPUtime=1122.31 cores=1,3,5,7
/proc/23724/task/23738/stat : 23738 (plingeling) R 23722 23724 23289 0 -1 4202560 469065 1260 0 0 112162 69 0 0 25 0 5 0 519996886 310726656 47294 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23724/tid=23739] ppid=23722 vsize=303444 CPUtime=1122.32 cores=1,3,5,7
/proc/23724/task/23739/stat : 23739 (plingeling) R 23722 23724 23289 0 -1 4202560 410547 1260 0 0 112169 63 0 0 25 0 5 0 519996886 310726656 47294 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4483.14
Current children cumulated vsize (KiB) 303444

[startup+1182.3 s]
/proc/loadavg: 8.11 8.09 8.02 9/186 23780
/proc/meminfo: memFree=24969248/32951124 swapFree=67051468/67111528
[pid=23724] ppid=23722 vsize=293476 CPUtime=4722.82 cores=1,3,5,7
/proc/23724/stat : 23724 (plingeling) S 23722 23724 23289 0 -1 4202496 1899362 1260 0 0 471987 295 0 0 18 0 5 0 519996885 300519424 44817 33554432000 4194304 4417332 140736535217904 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/23724/statm: 73369 44817 179 55 0 71255 0
[pid=23724/tid=23736] ppid=23722 vsize=293476 CPUtime=1182.32 cores=1,3,5,7
/proc/23724/task/23736/stat : 23736 (plingeling) R 23722 23724 23289 0 -1 4202560 496741 1260 0 0 118154 78 0 0 25 0 5 0 519996886 300519424 44817 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23724/tid=23737] ppid=23722 vsize=293476 CPUtime=1175.84 cores=1,3,5,7
/proc/23724/task/23737/stat : 23737 (plingeling) R 23722 23724 23289 0 -1 4202560 489511 1260 0 0 117505 79 0 0 25 0 5 0 519996886 300519424 44817 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364536 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23724/tid=23738] ppid=23722 vsize=293476 CPUtime=1182.31 cores=1,3,5,7
/proc/23724/task/23738/stat : 23738 (plingeling) R 23722 23724 23289 0 -1 4202560 487333 1260 0 0 118160 71 0 0 25 0 5 0 519996886 300519424 44817 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364556 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23724/tid=23739] ppid=23722 vsize=293476 CPUtime=1182.32 cores=1,3,5,7
/proc/23724/task/23739/stat : 23739 (plingeling) R 23722 23724 23289 0 -1 4202560 425470 1260 0 0 118167 65 0 0 25 0 5 0 519996886 300519424 44817 33554432000 4194304 4417332 140736535217904 18446744073709551615 4251104 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4722.82
Current children cumulated vsize (KiB) 293476



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 8.08 8.08 8.02 5/180 23782
/proc/meminfo: memFree=25124156/32951124 swapFree=67051468/67111528
[pid=23724] ppid=23722 vsize=313152 CPUtime=4800.33 cores=1,3,5,7
/proc/23724/stat : 23724 (plingeling) S 23722 23724 23289 0 -1 4202496 1922519 1260 0 0 479735 298 0 0 18 0 5 0 519996885 320667648 49706 33554432000 4194304 4417332 140736535217904 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/23724/statm: 78288 49706 179 55 0 76174 0
[pid=23724/tid=23736] ppid=23722 vsize=313152 CPUtime=1201.72 cores=1,3,5,7
/proc/23724/task/23736/stat : 23736 (plingeling) R 23722 23724 23289 0 -1 4202560 500196 1260 0 0 120094 78 0 0 25 0 5 0 519996886 320667648 49706 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23724/tid=23737] ppid=23722 vsize=313152 CPUtime=1195.13 cores=1,3,5,7
/proc/23724/task/23737/stat : 23737 (plingeling) R 23722 23724 23289 0 -1 4202560 498785 1260 0 0 119432 81 0 0 25 0 5 0 519996886 320667648 49706 33554432000 4194304 4417332 140736535217904 18446744073709551615 4251261 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23724/tid=23738] ppid=23722 vsize=313152 CPUtime=1201.72 cores=1,3,5,7
/proc/23724/task/23738/stat : 23738 (plingeling) R 23722 23724 23289 0 -1 4202560 492847 1260 0 0 120100 72 0 0 25 0 5 0 519996886 320667648 49706 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23724/tid=23739] ppid=23722 vsize=313152 CPUtime=1201.72 cores=1,3,5,7
/proc/23724/task/23739/stat : 23739 (plingeling) R 23722 23724 23289 0 -1 4202560 430384 1260 0 0 120107 65 0 0 25 0 5 0 519996886 320667648 49706 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364526 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4800.33
Current children cumulated vsize (KiB) 313152

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.7 s]
/proc/loadavg: 8.08 8.08 8.02 5/180 23782
/proc/meminfo: memFree=25124156/32951124 swapFree=67051468/67111528
[pid=23724] ppid=23722 vsize=313152 CPUtime=4800.33 cores=1,3,5,7
/proc/23724/stat : 23724 (plingeling) S 23722 23724 23289 0 -1 4202496 1922519 1260 0 0 479735 298 0 0 18 0 5 0 519996885 320667648 49706 33554432000 4194304 4417332 140736535217904 18446744073709551615 218382759909 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/23724/statm: 78288 49706 179 55 0 76174 0
[pid=23724/tid=23736] ppid=23722 vsize=313152 CPUtime=1201.72 cores=1,3,5,7
/proc/23724/task/23736/stat : 23736 (plingeling) R 23722 23724 23289 0 -1 4202560 500196 1260 0 0 120094 78 0 0 25 0 5 0 519996886 320667648 49706 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364494 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=23724/tid=23737] ppid=23722 vsize=313152 CPUtime=1195.13 cores=1,3,5,7
/proc/23724/task/23737/stat : 23737 (plingeling) R 23722 23724 23289 0 -1 4202560 498785 1260 0 0 119432 81 0 0 25 0 5 0 519996886 320667648 49706 33554432000 4194304 4417332 140736535217904 18446744073709551615 4251261 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=23724/tid=23738] ppid=23722 vsize=313152 CPUtime=1201.72 cores=1,3,5,7
/proc/23724/task/23738/stat : 23738 (plingeling) R 23722 23724 23289 0 -1 4202560 492847 1260 0 0 120100 72 0 0 25 0 5 0 519996886 320667648 49706 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364642 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=23724/tid=23739] ppid=23722 vsize=313152 CPUtime=1201.72 cores=1,3,5,7
/proc/23724/task/23739/stat : 23739 (plingeling) R 23722 23724 23289 0 -1 4202560 430384 1260 0 0 120107 65 0 0 25 0 5 0 519996886 320667648 49706 33554432000 4194304 4417332 140736535217904 18446744073709551615 4364526 0 0 4096 17442 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4800.33
Current children cumulated vsize (KiB) 313152

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1201.72
CPU time (s): 4800.37
CPU user time (s): 4797.37
CPU system time (s): 2.99954
CPU usage (%): 399.458
Max. virtual memory (cumulated for all children) (KiB): 322076

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4797.37
system time used= 2.99954
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1923780
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 164
involuntary context switches= 21427

runsolver used 1.86172 second user time and 4.70228 second system time

The end

Launcher Data 

Begin job on node101 at 2011-04-29 15:32:21
IDJOB=3273627
IDBENCH=83143
IDSOLVER=1605
FILE ID=node101/3273627-1304083941
RUNJOBID= node101-1304082737-23604
PBS_JOBID= 13170528
Free space on /tmp= 72024 MiB

SOLVER NAME= Plingeling 587
BENCH NAME= SAT11/crafted/spence/sgen/sgen3-n150-s89334302-unsat.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3273627-1304083941/watcher-3273627-1304083941 -o /tmp/evaluation-result-3273627-1304083941/solver-3273627-1304083941 -C 4800 -W 1300 -M 15500  plingeling -t 4 HOME/instance-3273627-1304083941.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= e0b064ae815c6a8bb8a9898180562bec
RANDOM SEED=928725945

node101.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      25326028 kB
Buffers:       2023104 kB
Cached:        5124944 kB
SwapCached:      30452 kB
Active:        3055040 kB
Inactive:      4126916 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      25326028 kB
SwapTotal:    67111528 kB
SwapFree:     67051468 kB
Dirty:              56 kB
Writeback:           0 kB
AnonPages:       24728 kB
Mapped:           8604 kB
Slab:           380040 kB
PageTables:       4680 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   188868 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72024 MiB
End job on node101 at 2011-04-29 15:52:25