Trace number 1781128

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth-5-0 2009-03-19 (4 threads)? (MO) 1762.69 891.853

General information on the benchmark

NameAPPLICATIONS/satComp09_BioInstances/
rbcl_xits_09_UNKNOWN.cnf
MD5SUM18ad1dee1e9d863458e75a20545ae68a
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables1430
Number of clauses79453
Sum of the clauses size236162
Maximum clause length10
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 22358
Number of clauses of size 377072
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 523

Solver Data 

0.12/0.12	c Command: HOME/ttsth-5-0 -num-threads 4 HOME/instance-1781128-1243685601.cnf 
0.12/0.12	c Using SATTIMEOUT of 10000
0.12/0.12	c Using SATRAM of 2047
0.12/0.12	c Size of subset node = 16
0.12/0.12	c Variables = 1430
0.12/0.12	c Clauses = 79453
0.12/0.12	c Literals = 236162
0.12/0.12	c Reduced size:
0.12/0.12	c Variables = 1430
0.12/0.12	c Clauses = 79453
0.12/0.12	c Literals = 236162
0.12/0.12	c a2
0.32/0.32	c a1
0.62/0.68	c Independent sub-problems = 1
0.62/0.68	c Sub-problem with 1430 variables and 79453 clauses
0.62/0.68	c using permutation method 4
0.62/0.68	c count = 1
0.62/0.68	c Ordering finished 0.660 seconds
0.62/0.68	c mapped 0.690 seconds
0.62/0.68	c permuted 0.690 seconds
0.62/0.68	c about to sort
0.72/0.71	c sorted
1.12/1.12	c clauses now 79453
1.12/1.12	c init finished 0.740 seconds
1.12/1.12	c tree built 1.130 seconds
1.12/1.12	c 
1.12/1.12	c PA progress                      | CPU progress Partial assignments | improvements      CPU time        PA/sec        nodes
1.12/1.12	c ---------------------------------|----------------------------------|------------------------------------------------------
1.12/1.12	c ................................ | ..........                       |                        1.1           0.0             
1.12/1.12	c ................................ | ..........                     1 |                        1.1           0.9             
1.12/1.12	c ................................ | ..........                     2 |                        1.1           1.8             
1.12/1.12	c ................................ | ..........                     3 |                        1.1           2.7             
1.12/1.12	c ................................ | ..........                     4 |                        1.1           3.5             
1.12/1.12	c ................................ | ..........                     5 |                        1.1           4.4             
1.12/1.13	c ................................ | ..........                  1000 | 38                     1.1         877.2          333
200.02/104.61	c ................................ | ..........               1354000 | 118593               200.0        6769.0     27413777
399.90/206.27	c ................................ | ..........               2408000 | 271617               400.1        6018.8     43361362
599.99/307.68	c ................................ | ..........               3083000 | 442774               600.1        5137.3     57679328
800.31/409.27	c ................................ | ..........               3641000 | 572613               800.5        4548.6     71385754
1000.16/508.26	c ................................ | *.........               4175000 | 695982              1000.3        4173.9     83197933
1199.86/608.88	c ................................ | *.........               4546000 | 800366              1200.0        3788.3     93453704
1400.12/709.66	c ................................ | *.........               4899000 | 900456              1400.2        3498.7    104697868
1600.27/810.18	c ................................ | *.........               5277000 | 1008136             1600.4        3297.3    115455562

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver version 3.2.9 (svn:503) (c) roussel@cril.univ-artois.fr

command line: runsolver --timestamp -w /tmp/evaluation-result-1781128-1243685601/watcher-1781128-1243685601 -o /tmp/evaluation-result-1781128-1243685601/solver-1781128-1243685601 -C 10000 -W 1000 -M 2047 HOME/ttsth-5-0 -num-threads 4 HOME/instance-1781128-1243685601.cnf 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 10000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 10030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 2096128 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 2147328 KiB
Current StackSize limit: 8192 KiB


[startup+0 s]
/proc/loadavg: 7.00 7.22 6.94 5/111 31955
/proc/meminfo: memFree=2954612/3368532 swapFree=1953788/1959888
[pid=31955] ppid=31953 vsize=1012 CPUtime=0.01
/proc/31955/stat : 31955 (ttsth-5-0) R 31953 31955 20065 0 -1 4202496 150 0 0 0 1 0 0 0 25 0 1 0 1133224211 1036288 105 4294967295 134512640 135237372 3213450208 3213449764 134738888 0 0 0 0 0 0 0 17 0 0 0 0
/proc/31955/statm: 253 105 75 177 0 75 0

[startup+0.111682 s]
/proc/loadavg: 7.00 7.22 6.94 5/111 31955
/proc/meminfo: memFree=2954612/3368532 swapFree=1953788/1959888
[pid=31955] ppid=31953 vsize=5736 CPUtime=0.12
/proc/31955/stat : 31955 (ttsth-5-0) R 31953 31955 20065 0 -1 4202496 1308 0 0 0 10 2 0 0 25 0 1 0 1133224211 5873664 1263 4294967295 134512640 135237372 3213450208 3213449932 134533782 0 0 0 0 0 0 0 17 0 0 0 0
/proc/31955/statm: 1434 1263 75 177 0 1256 0
Current children cumulated CPU time (s) 0.12
Current children cumulated vsize (KiB) 5736

[startup+0.211671 s]
/proc/loadavg: 7.00 7.22 6.94 5/111 31955
/proc/meminfo: memFree=2954612/3368532 swapFree=1953788/1959888
[pid=31955] ppid=31953 vsize=11704 CPUtime=0.22
/proc/31955/stat : 31955 (ttsth-5-0) R 31953 31955 20065 0 -1 4202496 2723 0 0 0 20 2 0 0 25 0 1 0 1133224211 11984896 2677 4294967295 134512640 135237372 3213450208 3213448624 134749353 0 0 0 0 0 0 0 17 0 0 0 0
/proc/31955/statm: 2926 2677 76 177 0 2748 0
Current children cumulated CPU time (s) 0.22
Current children cumulated vsize (KiB) 11704

[startup+0.311661 s]
/proc/loadavg: 7.00 7.22 6.94 5/111 31955
/proc/meminfo: memFree=2954612/3368532 swapFree=1953788/1959888
[pid=31955] ppid=31953 vsize=11704 CPUtime=0.32
/proc/31955/stat : 31955 (ttsth-5-0) R 31953 31955 20065 0 -1 4202496 2723 0 0 0 30 2 0 0 25 0 1 0 1133224211 11984896 2677 4294967295 134512640 135237372 3213450208 3213448624 134749353 0 0 0 0 0 0 0 17 0 0 0 0
/proc/31955/statm: 2926 2677 76 177 0 2748 0
Current children cumulated CPU time (s) 0.32
Current children cumulated vsize (KiB) 11704

[startup+0.71163 s]
/proc/loadavg: 7.00 7.22 6.94 5/111 31955
/proc/meminfo: memFree=2954612/3368532 swapFree=1953788/1959888
[pid=31955] ppid=31953 vsize=11936 CPUtime=0.72
/proc/31955/stat : 31955 (ttsth-5-0) R 31953 31955 20065 0 -1 4202496 4082 0 0 0 70 2 0 0 25 0 1 0 1133224211 12222464 2854 4294967295 134512640 135237372 3213450208 3213449520 134680285 0 0 0 0 0 0 0 17 0 0 0 0
/proc/31955/statm: 2984 2854 85 177 0 2806 0
Current children cumulated CPU time (s) 0.72
Current children cumulated vsize (KiB) 11936

[startup+1.51156 s]
/proc/loadavg: 7.00 7.22 6.94 5/112 31956
/proc/meminfo: memFree=2933892/3368532 swapFree=1953788/1959888
[pid=31955] ppid=31953 vsize=58160 CPUtime=1.47
/proc/31955/stat : 31955 (ttsth-5-0) S 31953 31955 20065 0 -1 4202496 7172 0 0 0 144 3 0 0 25 0 5 0 1133224211 59555840 5944 4294967295 134512640 135237372 3213450208 3213449804 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/31955/statm: 14540 5944 95 177 0 14362 0
Current children cumulated CPU time (s) 1.47
Current children cumulated vsize (KiB) 58160

[startup+3.11144 s]
/proc/loadavg: 7.00 7.22 6.94 8/116 31960
/proc/meminfo: memFree=2926800/3368532 swapFree=1953788/1959888
[pid=31955] ppid=31953 vsize=69700 CPUtime=4.2
/proc/31955/stat : 31955 (ttsth-5-0) S 31953 31955 20065 0 -1 4202496 10009 0 0 0 417 3 0 0 25 0 5 0 1133224211 71372800 8781 4294967295 134512640 135237372 3213450208 3213449804 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/31955/statm: 17425 8781 95 177 0 17247 0
[pid=31955/tid=31957] ppid=31953 vsize=69700 CPUtime=0.88
/proc/31955/task/31957/stat : 31957 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 772 0 0 0 88 0 0 0 25 0 5 0 1133224324 71372800 8781 4294967295 134512640 135237372 3213450208 3086367296 134546026 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=31955/tid=31958] ppid=31953 vsize=69700 CPUtime=0.72
/proc/31955/task/31958/stat : 31958 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 1056 0 0 0 72 0 0 0 25 0 5 0 1133224324 71372800 8782 4294967295 134512640 135237372 3213450208 3077995436 134680208 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=31955/tid=31959] ppid=31953 vsize=69700 CPUtime=0.81
/proc/31955/task/31959/stat : 31959 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 942 0 0 0 81 0 0 0 25 0 5 0 1133224324 71372800 8782 4294967295 134512640 135237372 3213450208 3069592624 134591065 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=31955/tid=31960] ppid=31953 vsize=69700 CPUtime=0.66
/proc/31955/task/31960/stat : 31960 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 674 0 0 0 66 0 0 0 25 0 5 0 1133224324 71372800 8782 4294967295 134512640 135237372 3213450208 3061200748 134519936 0 0 0 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4.2
Current children cumulated vsize (KiB) 69700

[startup+6.31117 s]
/proc/loadavg: 7.00 7.22 6.94 8/116 31961
/proc/meminfo: memFree=2905128/3368532 swapFree=1953788/1959888
[pid=31955] ppid=31953 vsize=89884 CPUtime=10.36
/proc/31955/stat : 31955 (ttsth-5-0) S 31953 31955 20065 0 -1 4202496 15115 0 0 0 1030 6 0 0 25 0 5 0 1133224211 92041216 13887 4294967295 134512640 135237372 3213450208 3213449804 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/31955/statm: 22471 13887 95 177 0 22293 0
[pid=31955/tid=31957] ppid=31953 vsize=89884 CPUtime=2.44
/proc/31955/task/31957/stat : 31957 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 1962 0 0 0 243 1 0 0 25 0 5 0 1133224324 92041216 13887 4294967295 134512640 135237372 3213450208 3086378912 134590180 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=31955/tid=31958] ppid=31953 vsize=89884 CPUtime=2.32
/proc/31955/task/31958/stat : 31958 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 2300 0 0 0 230 2 0 0 25 0 5 0 1133224324 92041216 13887 4294967295 134512640 135237372 3213450208 3077967408 134588779 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=31955/tid=31959] ppid=31953 vsize=89884 CPUtime=2.21
/proc/31955/task/31959/stat : 31959 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 2132 0 0 0 221 0 0 0 25 0 5 0 1133224324 92041216 13887 4294967295 134512640 135237372 3213450208 3069605008 134590126 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=31955/tid=31960] ppid=31953 vsize=89884 CPUtime=2.26
/proc/31955/task/31960/stat : 31960 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 2156 0 0 0 226 0 0 0 25 0 5 0 1133224324 92041216 13887 4294967295 134512640 135237372 3213450208 3061202844 134550031 0 0 0 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 10.36
Current children cumulated vsize (KiB) 89884

[startup+12.7106 s]
/proc/loadavg: 7.00 7.21 6.94 8/116 31961
/proc/meminfo: memFree=2865300/3368532 swapFree=1953788/1959888
[pid=31955] ppid=31953 vsize=130464 CPUtime=21.16
/proc/31955/stat : 31955 (ttsth-5-0) S 31953 31955 20065 0 -1 4202496 24573 0 0 0 2105 11 0 0 25 0 5 0 1133224211 133595136 23345 4294967295 134512640 135237372 3213450208 3213449804 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/31955/statm: 32616 23346 95 177 0 32438 0
[pid=31955/tid=31957] ppid=31953 vsize=130464 CPUtime=4.94
/proc/31955/task/31957/stat : 31957 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 4133 0 0 0 492 2 0 0 25 0 5 0 1133224324 133595136 23346 4294967295 134512640 135237372 3213450208 3086347744 134680503 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=31955/tid=31958] ppid=31953 vsize=130464 CPUtime=5.12
/proc/31955/task/31958/stat : 31958 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 5329 0 0 0 508 4 0 0 25 0 5 0 1133224324 133595136 23346 4294967295 134512640 135237372 3213450208 3077969808 134562338 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=31955/tid=31959] ppid=31953 vsize=130464 CPUtime=5.01
/proc/31955/task/31959/stat : 31959 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 4566 0 0 0 500 1 0 0 25 0 5 0 1133224324 133595136 23346 4294967295 134512640 135237372 3213450208 3069582112 134592831 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=31955/tid=31960] ppid=31953 vsize=130464 CPUtime=4.97
/proc/31955/task/31960/stat : 31960 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 3981 0 0 0 496 1 0 0 25 0 5 0 1133224324 133595136 23346 4294967295 134512640 135237372 3213450208 3061233952 134549888 0 0 0 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 21.16
Current children cumulated vsize (KiB) 130464

[startup+25.5096 s]
/proc/loadavg: 7.07 7.22 6.94 8/116 31961
/proc/meminfo: memFree=2798180/3368532 swapFree=1953788/1959888
[pid=31955] ppid=31953 vsize=196560 CPUtime=44.33
/proc/31955/stat : 31955 (ttsth-5-0) S 31953 31955 20065 0 -1 4202496 40641 0 0 0 4418 15 0 0 25 0 5 0 1133224211 201277440 39413 4294967295 134512640 135237372 3213450208 3213449804 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/31955/statm: 49140 39413 95 177 0 48962 0
[pid=31955/tid=31957] ppid=31953 vsize=196560 CPUtime=10.93
/proc/31955/task/31957/stat : 31957 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 8183 0 0 0 1091 2 0 0 25 0 5 0 1133224324 201277440 39413 4294967295 134512640 135237372 3213450208 3086342320 134749339 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=31955/tid=31958] ppid=31953 vsize=196560 CPUtime=10.82
/proc/31955/task/31958/stat : 31958 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 8959 0 0 0 1077 5 0 0 25 0 5 0 1133224324 201277440 39413 4294967295 134512640 135237372 3213450208 3077970576 134589526 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=31955/tid=31959] ppid=31953 vsize=196560 CPUtime=10.71
/proc/31955/task/31959/stat : 31959 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 8176 0 0 0 1070 1 0 0 25 0 5 0 1133224324 201277440 39413 4294967295 134512640 135237372 3213450208 3069589232 134562341 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=31955/tid=31960] ppid=31953 vsize=196560 CPUtime=10.74
/proc/31955/task/31960/stat : 31960 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 8758 0 0 0 1070 4 0 0 25 0 5 0 1133224324 201277440 39413 4294967295 134512640 135237372 3213450208 3061199620 134636382 0 0 0 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 44.33
Current children cumulated vsize (KiB) 196560

[startup+51.1075 s]
/proc/loadavg: 7.17 7.23 6.95 8/116 32148
/proc/meminfo: memFree=2693928/3368532 swapFree=1953788/1959888
[pid=31955] ppid=31953 vsize=308320 CPUtime=94.55
/proc/31955/stat : 31955 (ttsth-5-0) S 31953 31955 20065 0 -1 4202496 67121 0 0 0 9424 31 0 0 25 0 5 0 1133224211 315719680 65893 4294967295 134512640 135237372 3213450208 3213449804 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/31955/statm: 77080 65893 95 177 0 76902 0
[pid=31955/tid=31957] ppid=31953 vsize=308320 CPUtime=23.71
/proc/31955/task/31957/stat : 31957 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 15154 0 0 0 2366 5 0 0 25 0 5 0 1133224324 315719680 65893 4294967295 134512640 135237372 3213450208 3086382832 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=31955/tid=31958] ppid=31953 vsize=308320 CPUtime=23.31
/proc/31955/task/31958/stat : 31958 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 14672 0 0 0 2323 8 0 0 25 0 5 0 1133224324 315719680 65893 4294967295 134512640 135237372 3213450208 3077996304 134546026 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=31955/tid=31959] ppid=31953 vsize=308320 CPUtime=23.09
/proc/31955/task/31959/stat : 31959 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 14375 0 0 0 2303 6 0 0 25 0 5 0 1133224324 315719680 65893 4294967295 134512640 135237372 3213450208 3069590944 134546034 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=31955/tid=31960] ppid=31953 vsize=308320 CPUtime=23.31
/proc/31955/task/31960/stat : 31960 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 16355 0 0 0 2322 9 0 0 25 0 5 0 1133224324 315719680 65893 4294967295 134512640 135237372 3213450208 3061207456 134680727 0 0 0 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 94.55
Current children cumulated vsize (KiB) 308320

[startup+102.303 s]
/proc/loadavg: 7.26 7.25 6.97 8/116 32334
/proc/meminfo: memFree=2507232/3368532 swapFree=1953788/1959888
[pid=31955] ppid=31953 vsize=502820 CPUtime=195.51
/proc/31955/stat : 31955 (ttsth-5-0) S 31953 31955 20065 0 -1 4202496 112910 0 0 0 19502 49 0 0 25 0 5 0 1133224211 514887680 111682 4294967295 134512640 135237372 3213450208 3213449804 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/31955/statm: 125705 111683 95 177 0 125527 0
[pid=31955/tid=31957] ppid=31953 vsize=502820 CPUtime=49.03
/proc/31955/task/31957/stat : 31957 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 28543 0 0 0 4890 13 0 0 25 0 5 0 1133224324 514887680 111683 4294967295 134512640 135237372 3213450208 3086372144 134546034 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=31955/tid=31958] ppid=31953 vsize=502820 CPUtime=48.52
/proc/31955/task/31958/stat : 31958 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 24519 0 0 0 4841 11 0 0 25 0 5 0 1133224324 514887680 111683 4294967295 134512640 135237372 3213450208 3077973488 134562338 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=31955/tid=31959] ppid=31953 vsize=502820 CPUtime=48.02
/proc/31955/task/31959/stat : 31959 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 26052 0 0 0 4794 8 0 0 25 0 5 0 1133224324 514887680 111683 4294967295 134512640 135237372 3213450208 3069609164 134680214 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=31955/tid=31960] ppid=31953 vsize=502820 CPUtime=48.81
/proc/31955/task/31960/stat : 31960 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 27232 0 0 0 4867 14 0 0 25 0 5 0 1133224324 514887680 111683 4294967295 134512640 135237372 3213450208 3061210632 134549877 0 0 0 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 195.51
Current children cumulated vsize (KiB) 502820

[startup+162.308 s]
/proc/loadavg: 7.40 7.30 7.01 9/116 32522
/proc/meminfo: memFree=2349468/3368532 swapFree=1953788/1959888
[pid=31955] ppid=31953 vsize=670736 CPUtime=313.22
/proc/31955/stat : 31955 (ttsth-5-0) S 31953 31955 20065 0 -1 4202496 152727 0 0 0 31247 75 0 0 25 0 5 0 1133224211 686833664 151499 4294967295 134512640 135237372 3213450208 3213449804 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/31955/statm: 167684 151499 95 177 0 167506 0
[pid=31955/tid=31957] ppid=31953 vsize=670736 CPUtime=78.54
/proc/31955/task/31957/stat : 31957 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 43920 0 0 0 7835 19 0 0 25 0 5 0 1133224324 686833664 151499 4294967295 134512640 135237372 3213450208 3086401792 134549905 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=31955/tid=31958] ppid=31953 vsize=670736 CPUtime=77.62
/proc/31955/task/31958/stat : 31958 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 31685 0 0 0 7742 20 0 0 25 0 5 0 1133224324 686833664 151499 4294967295 134512640 135237372 3213450208 3077973024 134562459 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=31955/tid=31959] ppid=31953 vsize=670736 CPUtime=77.72
/proc/31955/task/31959/stat : 31959 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 35035 0 0 0 7753 19 0 0 25 0 5 0 1133224324 686833664 151499 4294967295 134512640 135237372 3213450208 3069595072 134562360 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=31955/tid=31960] ppid=31953 vsize=670736 CPUtime=78.21
/proc/31955/task/31960/stat : 31960 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 35522 0 0 0 7807 14 0 0 25 0 5 0 1133224324 686833664 151499 4294967295 134512640 135237372 3213450208 3061201824 134562338 0 0 0 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 313.22
Current children cumulated vsize (KiB) 670736

[startup+222.323 s]
/proc/loadavg: 7.31 7.30 7.02 8/116 32713
/proc/meminfo: memFree=2212256/3368532 swapFree=1953788/1959888
[pid=31955] ppid=31953 vsize=814680 CPUtime=431.84
/proc/31955/stat : 31955 (ttsth-5-0) S 31953 31955 20065 0 -1 4202496 186521 0 0 0 43085 99 0 0 25 0 5 0 1133224211 834232320 185293 4294967295 134512640 135237372 3213450208 3213449804 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/31955/statm: 203670 185293 95 177 0 203492 0
[pid=31955/tid=31957] ppid=31953 vsize=814680 CPUtime=108.44
/proc/31955/task/31957/stat : 31957 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 55068 0 0 0 10821 23 0 0 25 0 5 0 1133224324 834232320 185293 4294967295 134512640 135237372 3213450208 3086361424 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=31955/tid=31958] ppid=31953 vsize=814680 CPUtime=107.02
/proc/31955/task/31958/stat : 31958 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 40587 0 0 0 10675 27 0 0 25 0 5 0 1133224324 834232320 185293 4294967295 134512640 135237372 3213450208 3077979328 134564220 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=31955/tid=31959] ppid=31953 vsize=814680 CPUtime=107.51
/proc/31955/task/31959/stat : 31959 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 41557 0 0 0 10725 26 0 0 25 0 5 0 1133224324 834232320 185293 4294967295 134512640 135237372 3213450208 3069581648 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=31955/tid=31960] ppid=31953 vsize=814680 CPUtime=107.75
/proc/31955/task/31960/stat : 31960 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 42744 0 0 0 10755 20 0 0 25 0 5 0 1133224324 834232320 185293 4294967295 134512640 135237372 3213450208 3061184096 134562351 0 0 0 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 431.84
Current children cumulated vsize (KiB) 814680


################
# More data... #
################

/proc/meminfo: memFree=1705872/3368532 swapFree=1953788/1959888
[pid=31955] ppid=31953 vsize=1348956 CPUtime=908.72
/proc/31955/stat : 31955 (ttsth-5-0) S 31953 31955 20065 0 -1 4202496 313152 0 0 0 90702 170 0 0 25 0 5 0 1133224211 1381330944 311924 4294967295 134512640 135237372 3213450208 3213449804 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/31955/statm: 337239 311924 95 177 0 337061 0
[pid=31955/tid=31957] ppid=31953 vsize=1348956 CPUtime=230.54
/proc/31955/task/31957/stat : 31957 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 86032 0 0 0 23015 39 0 0 25 0 5 0 1133224324 1381330944 311924 4294967295 134512640 135237372 3213450208 3086360144 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=31955/tid=31958] ppid=31953 vsize=1348956 CPUtime=224.81
/proc/31955/task/31958/stat : 31958 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 68717 0 0 0 22433 48 0 0 25 0 5 0 1133224324 1381330944 311925 4294967295 134512640 135237372 3213450208 3077971728 134587833 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=31955/tid=31959] ppid=31953 vsize=1348956 CPUtime=226.21
/proc/31955/task/31959/stat : 31959 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 68616 0 0 0 22576 45 0 0 25 0 5 0 1133224324 1381330944 311925 4294967295 134512640 135237372 3213450208 3069581360 134562338 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=31955/tid=31960] ppid=31953 vsize=1348956 CPUtime=226.04
/proc/31955/task/31960/stat : 31960 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 83223 0 0 0 22569 35 0 0 25 0 5 0 1133224324 1381330944 311925 4294967295 134512640 135237372 3213450208 3061192768 134562338 0 0 0 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 908.72
Current children cumulated vsize (KiB) 1348956

[startup+522.308 s]
/proc/loadavg: 7.34 7.27 7.08 8/116 1202
/proc/meminfo: memFree=1607052/3368532 swapFree=1953788/1959888
[pid=31955] ppid=31953 vsize=1453228 CPUtime=1028.15
/proc/31955/stat : 31955 (ttsth-5-0) S 31953 31955 20065 0 -1 4202496 337921 0 0 0 102628 187 0 0 25 0 5 0 1133224211 1488105472 336693 4294967295 134512640 135237372 3213450208 3213449804 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/31955/statm: 363307 336693 95 177 0 363129 0
[pid=31955/tid=31957] ppid=31953 vsize=1453228 CPUtime=260.63
/proc/31955/task/31957/stat : 31957 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 91841 0 0 0 26019 44 0 0 25 0 5 0 1133224324 1488105472 336693 4294967295 134512640 135237372 3213450208 3086362608 134561428 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=31955/tid=31958] ppid=31953 vsize=1453228 CPUtime=254.62
/proc/31955/task/31958/stat : 31958 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 75518 0 0 0 25409 53 0 0 25 0 5 0 1133224324 1488105472 336693 4294967295 134512640 135237372 3213450208 3077969440 134562351 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=31955/tid=31959] ppid=31953 vsize=1453228 CPUtime=255.94
/proc/31955/task/31959/stat : 31959 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 73765 0 0 0 25545 49 0 0 25 0 5 0 1133224324 1488105472 336693 4294967295 134512640 135237372 3213450208 3069598576 134546034 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=31955/tid=31960] ppid=31953 vsize=1453228 CPUtime=255.84
/proc/31955/task/31960/stat : 31960 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 90232 0 0 0 25546 38 0 0 25 0 5 0 1133224324 1488105472 336693 4294967295 134512640 135237372 3213450208 3061191072 134562351 0 0 0 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1028.15
Current children cumulated vsize (KiB) 1453228

[startup+582.303 s]
/proc/loadavg: 7.45 7.31 7.10 8/116 1391
/proc/meminfo: memFree=1515372/3368532 swapFree=1953788/1959888
[pid=31955] ppid=31953 vsize=1549272 CPUtime=1147.32
/proc/31955/stat : 31955 (ttsth-5-0) S 31953 31955 20065 0 -1 4202496 360751 0 0 0 114527 205 0 0 25 0 5 0 1133224211 1586454528 359523 4294967295 134512640 135237372 3213450208 3213449804 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/31955/statm: 387318 359523 95 177 0 387140 0
[pid=31955/tid=31957] ppid=31953 vsize=1549272 CPUtime=290.44
/proc/31955/task/31957/stat : 31957 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 97120 0 0 0 28995 49 0 0 25 0 5 0 1133224324 1586454528 359523 4294967295 134512640 135237372 3213450208 3086349792 134562336 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=31955/tid=31958] ppid=31953 vsize=1549272 CPUtime=284.42
/proc/31955/task/31958/stat : 31958 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 83100 0 0 0 28387 55 0 0 25 0 5 0 1133224324 1586454528 359523 4294967295 134512640 135237372 3213450208 3077972432 134562444 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=31955/tid=31959] ppid=31953 vsize=1549272 CPUtime=285.7
/proc/31955/task/31959/stat : 31959 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 78108 0 0 0 28515 55 0 0 25 0 5 0 1133224324 1586454528 359523 4294967295 134512640 135237372 3213450208 3069599120 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=31955/tid=31960] ppid=31953 vsize=1549272 CPUtime=285.64
/proc/31955/task/31960/stat : 31960 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 95858 0 0 0 28521 43 0 0 25 0 5 0 1133224324 1586454528 359523 4294967295 134512640 135237372 3213450208 3061209184 134562338 0 0 0 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1147.32
Current children cumulated vsize (KiB) 1549272

[startup+642.308 s]
/proc/loadavg: 7.24 7.27 7.10 8/116 1579
/proc/meminfo: memFree=1412684/3368532 swapFree=1953788/1959888
[pid=31955] ppid=31953 vsize=1658924 CPUtime=1266.37
/proc/31955/stat : 31955 (ttsth-5-0) S 31953 31955 20065 0 -1 4202496 386623 0 0 0 126412 225 0 0 25 0 5 0 1133224211 1698738176 385395 4294967295 134512640 135237372 3213450208 3213449804 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/31955/statm: 414731 385395 95 177 0 414553 0
[pid=31955/tid=31957] ppid=31953 vsize=1658924 CPUtime=320.24
/proc/31955/task/31957/stat : 31957 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 101953 0 0 0 31972 52 0 0 25 0 5 0 1133224324 1698738176 385395 4294967295 134512640 135237372 3213450208 3086358704 134562416 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=31955/tid=31958] ppid=31953 vsize=1658924 CPUtime=314.09
/proc/31955/task/31958/stat : 31958 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 89198 0 0 0 31347 62 0 0 25 0 5 0 1133224324 1698738176 385395 4294967295 134512640 135237372 3213450208 3077974828 134565824 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=31955/tid=31959] ppid=31953 vsize=1658924 CPUtime=315.5
/proc/31955/task/31959/stat : 31959 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 87140 0 0 0 31488 62 0 0 25 0 5 0 1133224324 1698738176 385395 4294967295 134512640 135237372 3213450208 3069598720 134546034 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=31955/tid=31960] ppid=31953 vsize=1658924 CPUtime=315.41
/proc/31955/task/31960/stat : 31960 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 101767 0 0 0 31495 46 0 0 25 0 5 0 1133224324 1698738176 385395 4294967295 134512640 135237372 3213450208 3061199728 134562328 0 0 0 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1266.37
Current children cumulated vsize (KiB) 1658924

[startup+702.303 s]
/proc/loadavg: 7.28 7.29 7.11 8/116 1765
/proc/meminfo: memFree=1300640/3368532 swapFree=1953788/1959888
[pid=31955] ppid=31953 vsize=1773708 CPUtime=1385.51
/proc/31955/stat : 31955 (ttsth-5-0) S 31953 31955 20065 0 -1 4202496 414231 0 0 0 138296 255 0 0 25 0 5 0 1133224211 1816276992 413003 4294967295 134512640 135237372 3213450208 3213449804 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/31955/statm: 443427 413003 95 177 0 443249 0
[pid=31955/tid=31957] ppid=31953 vsize=1773708 CPUtime=350.14
/proc/31955/task/31957/stat : 31957 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 106977 0 0 0 34957 57 0 0 25 0 5 0 1133224324 1816276992 413003 4294967295 134512640 135237372 3213450208 3086359328 134562380 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=31955/tid=31958] ppid=31953 vsize=1773708 CPUtime=343.92
/proc/31955/task/31958/stat : 31958 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 95238 0 0 0 34322 70 0 0 25 0 5 0 1133224324 1816276992 413003 4294967295 134512640 135237372 3213450208 3077972192 134562341 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=31955/tid=31959] ppid=31953 vsize=1773708 CPUtime=345.21
/proc/31955/task/31959/stat : 31959 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 98549 0 0 0 34452 69 0 0 25 0 5 0 1133224324 1816276992 413003 4294967295 134512640 135237372 3213450208 3069596560 134562320 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=31955/tid=31960] ppid=31953 vsize=1773708 CPUtime=345.11
/proc/31955/task/31960/stat : 31960 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 106902 0 0 0 34455 56 0 0 25 0 5 0 1133224324 1816276992 413003 4294967295 134512640 135237372 3213450208 3061192096 134562338 0 0 0 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1385.51
Current children cumulated vsize (KiB) 1773708

[startup+762.318 s]
/proc/loadavg: 7.33 7.31 7.13 8/116 1957
/proc/meminfo: memFree=1203396/3368532 swapFree=1953788/1959888
[pid=31955] ppid=31953 vsize=1874960 CPUtime=1504.85
/proc/31955/stat : 31955 (ttsth-5-0) S 31953 31955 20065 0 -1 4202496 438334 0 0 0 150214 271 0 0 25 0 5 0 1133224211 1919959040 437106 4294967295 134512640 135237372 3213450208 3213449804 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/31955/statm: 468740 437106 95 177 0 468562 0
[pid=31955/tid=31957] ppid=31953 vsize=1874960 CPUtime=380.04
/proc/31955/task/31957/stat : 31957 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 111940 0 0 0 37945 59 0 0 25 0 5 0 1133224324 1919959040 437106 4294967295 134512640 135237372 3213450208 3086350400 134562380 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=31955/tid=31958] ppid=31953 vsize=1874960 CPUtime=373.52
/proc/31955/task/31958/stat : 31958 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 101485 0 0 0 37279 73 0 0 25 0 5 0 1133224324 1919959040 437106 4294967295 134512640 135237372 3213450208 3077971180 134562272 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=31955/tid=31959] ppid=31953 vsize=1874960 CPUtime=375.05
/proc/31955/task/31959/stat : 31959 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 107431 0 0 0 37428 77 0 0 25 0 5 0 1133224324 1919959040 437106 4294967295 134512640 135237372 3213450208 3069597660 134562272 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=31955/tid=31960] ppid=31953 vsize=1874960 CPUtime=375.11
/proc/31955/task/31960/stat : 31960 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 110913 0 0 0 37452 59 0 0 25 0 5 0 1133224324 1919959040 437106 4294967295 134512640 135237372 3213450208 3061199824 134562338 0 0 0 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1504.85
Current children cumulated vsize (KiB) 1874960

[startup+822.314 s]
/proc/loadavg: 7.39 7.32 7.14 9/116 2145
/proc/meminfo: memFree=1101160/3368532 swapFree=1953788/1959888
[pid=31955] ppid=31953 vsize=1984800 CPUtime=1624.64
/proc/31955/stat : 31955 (ttsth-5-0) S 31953 31955 20065 0 -1 4202496 464098 0 0 0 162170 294 0 0 25 0 5 0 1133224211 2032435200 462870 4294967295 134512640 135237372 3213450208 3213449804 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/31955/statm: 496200 462870 95 177 0 496022 0
[pid=31955/tid=31957] ppid=31953 vsize=1984800 CPUtime=409.94
/proc/31955/task/31957/stat : 31957 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 115945 0 0 0 40931 63 0 0 25 0 5 0 1133224324 2032435200 462870 4294967295 134512640 135237372 3213450208 3086357792 134562278 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=31955/tid=31958] ppid=31953 vsize=1984800 CPUtime=403.52
/proc/31955/task/31958/stat : 31958 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 106197 0 0 0 40274 78 0 0 25 0 5 0 1133224324 2032435200 462870 4294967295 134512640 135237372 3213450208 3077968448 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=31955/tid=31959] ppid=31953 vsize=1984800 CPUtime=405.01
/proc/31955/task/31959/stat : 31959 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 119339 0 0 0 40413 88 0 0 25 0 5 0 1133224324 2032435200 462870 4294967295 134512640 135237372 3213450208 3069610044 134549694 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=31955/tid=31960] ppid=31953 vsize=1984800 CPUtime=405.05
/proc/31955/task/31960/stat : 31960 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 116052 0 0 0 40443 62 0 0 25 0 5 0 1133224324 2032435200 462870 4294967295 134512640 135237372 3213450208 3061199504 134562331 0 0 0 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1624.64
Current children cumulated vsize (KiB) 1984800

[startup+882.319 s]
/proc/loadavg: 7.38 7.34 7.16 8/116 2334
/proc/meminfo: memFree=1007296/3368532 swapFree=1953788/1959888
[pid=31955] ppid=31953 vsize=2083500 CPUtime=1743.86
/proc/31955/stat : 31955 (ttsth-5-0) S 31953 31955 20065 0 -1 4202496 487357 0 0 0 174075 311 0 0 25 0 5 0 1133224211 2133504000 486129 4294967295 134512640 135237372 3213450208 3213449804 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/31955/statm: 520875 486129 95 177 0 520697 0
[pid=31955/tid=31957] ppid=31953 vsize=2083500 CPUtime=439.88
/proc/31955/task/31957/stat : 31957 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 120220 0 0 0 43920 68 0 0 25 0 5 0 1133224324 2133504000 486129 4294967295 134512640 135237372 3213450208 3086361120 134562341 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=31955/tid=31958] ppid=31953 vsize=2083500 CPUtime=433.23
/proc/31955/task/31958/stat : 31958 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 110579 0 0 0 43244 79 0 0 25 0 5 0 1133224324 2133504000 486129 4294967295 134512640 135237372 3213450208 3077976384 134546026 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=31955/tid=31959] ppid=31953 vsize=2083500 CPUtime=434.51
/proc/31955/task/31959/stat : 31959 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 128823 0 0 0 43357 94 0 0 25 0 5 0 1133224324 2133504000 486129 4294967295 134512640 135237372 3213450208 3069588272 134562407 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=31955/tid=31960] ppid=31953 vsize=2083500 CPUtime=435.11
/proc/31955/task/31960/stat : 31960 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 121170 0 0 0 43444 67 0 0 25 0 5 0 1133224324 2133504000 486129 4294967295 134512640 135237372 3213450208 3061199920 134562338 0 0 0 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1743.86
Current children cumulated vsize (KiB) 2083500



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+891.708 s]
/proc/loadavg: 7.32 7.33 7.15 8/116 2334
/proc/meminfo: memFree=995028/3368532 swapFree=1953788/1959888
[pid=31955] ppid=31953 vsize=2096204 CPUtime=1762.57
/proc/31955/stat : 31955 (ttsth-5-0) S 31953 31955 20065 0 -1 4202496 490428 0 0 0 175945 312 0 0 25 0 5 0 1133224211 2146512896 489200 4294967295 134512640 135237372 3213450208 3213449804 134629544 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/31955/statm: 524051 489200 95 177 0 523873 0
[pid=31955/tid=31957] ppid=31953 vsize=2096204 CPUtime=444.54
/proc/31955/task/31957/stat : 31957 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 120901 0 0 0 44386 68 0 0 25 0 5 0 1133224324 2146512896 489200 4294967295 134512640 135237372 3213450208 3086358704 134562459 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=31955/tid=31958] ppid=31953 vsize=2096204 CPUtime=437.92
/proc/31955/task/31958/stat : 31958 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 111074 0 0 0 43713 79 0 0 25 0 5 0 1133224324 2146512896 489200 4294967295 134512640 135237372 3213450208 3077967568 134562407 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=31955/tid=31959] ppid=31953 vsize=2096204 CPUtime=439.17
/proc/31955/task/31959/stat : 31959 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 130266 0 0 0 43822 95 0 0 25 0 5 0 1133224324 2146512896 489200 4294967295 134512640 135237372 3213450208 3069605728 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=31955/tid=31960] ppid=31953 vsize=2096204 CPUtime=439.81
/proc/31955/task/31960/stat : 31960 (ttsth-5-0) R 31953 31955 20065 0 -1 4202560 121622 0 0 0 43914 67 0 0 25 0 5 0 1133224324 2146512896 489200 4294967295 134512640 135237372 3213450208 3061208768 134562338 0 0 0 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1762.57
Current children cumulated vsize (KiB) 2096204

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+891.828 s]
/proc/loadavg: 7.32 7.33 7.15 8/116 2334
/proc/meminfo: memFree=995028/3368532 swapFree=1953788/1959888
[pid=31955] ppid=31953 vsize=0 CPUtime=1762.67
/proc/31955/stat : 31955 (ttsth-5-0) Z 31953 31955 20065 0 -1 4203524 490430 0 0 0 175947 320 0 0 18 0 2 0 1133224211 0 0 4294967295 0 0 0 0 0 0 0 0 0 4294967295 0 0 17 0 0 0 0
/proc/31955/statm: 0 0 0 0 0 0 0
[pid=31955/tid=31958] ppid=31953 vsize=0 CPUtime=438
/proc/31955/task/31958/stat : 31958 (ttsth-5-0) R 31953 31955 20065 0 -1 4203588 111074 0 0 0 43713 87 0 0 25 0 2 0 1133224324 0 0 4294967295 0 0 0 0 0 0 0 0 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1762.67
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 891.853
CPU time (s): 1762.69
CPU user time (s): 1759.47
CPU system time (s): 3.22
CPU usage (%): 197.643
Max. virtual memory (cumulated for all children) (KiB): 2096204

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1759.47
system time used= 3.22
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 490430
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1005
involuntary context switches= 71728

runsolver used 0 second user time and 0 second system time

The end

Launcher Data 

Begin job on  at 2009-05-30 14:13:21
IDJOB=1781128
IDBENCH=71098
IDSOLVER=632
FILE ID=/1781128-1243685601
PBS_JOBID=
Free space on /tmp= 6476 MiB

SOLVER NAME= ttsth-5-0 2009-03-19 (4 threads)
BENCH NAME= SAT09/APPLICATIONS/satComp09_BioInstances/rbcl_xits_09_UNKNOWN.cnf
COMMAND LINE= HOME/ttsth-5-0 -num-threads MAXNBTHREAD BENCHNAME
RUNSOLVER COMMAND LINE= runsolver --timestamp  -w /tmp/evaluation-result-1781128-1243685601/watcher-1781128-1243685601 -o /tmp/evaluation-result-1781128-1243685601/solver-1781128-1243685601 -C 10000 -W 1000 -M 2047  HOME/ttsth-5-0 -num-threads 4 HOME/instance-1781128-1243685601.cnf

TIME LIMIT= 10000 seconds
MEMORY LIMIT= 2047 MiB
MAX NB THREAD= 4

MD5SUM BENCH= 18ad1dee1e9d863458e75a20545ae68a
RANDOM SEED= 162756013

Linux (none) 2.6.21.5 #6 SMP Wed Jul 4 17:16:38 CEST 2007 i686 GNU/Linux

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU            5130  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.061
cache size	: 4096 KB
physical id	: 0
siblings	: 2
core id		: 0
cpu cores	: 2
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 3992.41
clflush size	: 64

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU            5130  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.061
cache size	: 4096 KB
physical id	: 0
siblings	: 2
core id		: 1
cpu cores	: 2
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 3990.12
clflush size	: 64

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU            5130  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.061
cache size	: 4096 KB
physical id	: 3
siblings	: 2
core id		: 0
cpu cores	: 2
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 3990.14
clflush size	: 64

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU            5130  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.061
cache size	: 4096 KB
physical id	: 3
siblings	: 2
core id		: 1
cpu cores	: 2
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 3990.14
clflush size	: 64


/proc/meminfo:
MemTotal:      3368532 kB
MemFree:       2954612 kB
Buffers:        149128 kB
Cached:         143312 kB
SwapCached:       2256 kB
Active:         188680 kB
Inactive:       120720 kB
HighTotal:     2489216 kB
HighFree:      2328148 kB
LowTotal:       879316 kB
LowFree:        626464 kB
SwapTotal:     1959888 kB
SwapFree:      1953788 kB
Dirty:            2052 kB
Writeback:           0 kB
AnonPages:       15992 kB
Mapped:           5272 kB
Slab:            96088 kB
SReclaimable:    82780 kB
SUnreclaim:      13308 kB
PageTables:        668 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   3644152 kB
Committed_AS:   112688 kB
VmallocTotal:   114680 kB
VmallocUsed:      1444 kB
VmallocChunk:   113072 kB

Free space on /tmp at the end= 6475 MiB
End job on  at 2009-05-30 14:28:15