Trace number 1776298

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth-5-0 2009-03-19 (4 threads)? (MO) 1734.09 741.548

General information on the benchmark

Nameindustrial/grieu/
vmpc_25.cnf
MD5SUMc5b44b10975f50cd32d40650a6a22d18
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark19.83
Satisfiable
(Un)Satisfiability was proved
Number of variables625
Number of clauses76775
Sum of the clauses size201350
Maximum clause length25
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 230075
Number of clauses of size 346650
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 550

Solver Data 

0.10/0.11	c Command: HOME/ttsth-5-0 -num-threads 4 HOME/instance-1776298-1243674096.cnf 
0.10/0.11	c Using SATTIMEOUT of 10000
0.10/0.11	c Using SATRAM of 2047
0.10/0.11	c Size of subset node = 16
0.10/0.11	c Variables = 625
0.10/0.11	c Clauses = 76775
0.10/0.11	c Literals = 201350
0.10/0.11	c Reduced size:
0.10/0.11	c Variables = 625
0.10/0.11	c Clauses = 76775
0.10/0.11	c Literals = 201350
0.10/0.11	c a2
0.20/0.28	c a1
0.60/0.63	c Independent sub-problems = 1
0.60/0.63	c Sub-problem with 625 variables and 76775 clauses
0.60/0.63	c using permutation method 4
0.60/0.63	c count = 1
0.60/0.63	c Ordering finished 0.600 seconds
0.60/0.63	c mapped 0.630 seconds
0.60/0.63	c permuted 0.630 seconds
0.60/0.63	c about to sort
0.60/0.65	c sorted
1.01/1.08	c clauses now 76775
1.01/1.08	c init finished 0.670 seconds
1.01/1.08	c tree built 1.080 seconds
1.01/1.08	c 
1.01/1.08	c PA progress                      | CPU progress Partial assignments | improvements      CPU time        PA/sec        nodes
1.01/1.08	c ---------------------------------|----------------------------------|------------------------------------------------------
1.01/1.08	c ................................ | ..........                       |                        1.1           0.0             
1.01/1.08	c ................................ | ..........                     1 |                        1.1           0.9             
1.01/1.08	c ................................ | ..........                     2 |                        1.1           1.9             
1.01/1.08	c ................................ | ..........                     3 |                        1.1           2.8             
1.01/1.08	c ................................ | ..........                     4 |                        1.1           3.7             
1.01/1.08	c ................................ | ..........                     5 |                        1.1           4.6             
1.01/1.08	c *............................... | ..........                    11 |                        1.1          10.2           12
1.13/1.19	c *............................... | ..........                  1000 | 9                      1.3         781.2        80509
200.08/93.61	c *............................... | ..........                389000 | 20740                200.1        1944.1     30748682
400.85/186.62	c *............................... | ..........                645000 | 35362                400.9        1609.0     49554902
601.20/281.27	c *............................... | ..........                811000 | 46226                601.4        1348.5     64187389
800.98/361.69	c *............................... | ..........                933000 | 55432                801.2        1164.5     77641204
1000.97/445.38	c *............................... | *.........               1044000 | 63155               1001.2        1042.8     89901731
1113.34/491.00	c ****************................ | *.........               1097262 | 66965               1113.5         985.4     96043752
1113.54/491.10	c ****************................ | *.........               1097263 | 66965               1113.5         985.4     96043752
1113.54/491.10	c ****************................ | *.........               1097264 | 66965               1113.7         985.2     96051640
1113.54/491.10	c ****************................ | *.........               1097265 | 66965               1113.7         985.2     96051640
1113.54/491.10	c ****************................ | *.........               1097266 | 66965               1113.7         985.2     96051640
1113.54/491.10	c ******************.............. | *.........               1097268 | 66965               1113.7         985.2     96051646
1113.54/491.10	c ******************.............. | *.........               1097269 | 66965               1113.7         985.2     96051646
1201.33/527.83	c ******************.............. | *.........               1143000 | 70268               1201.4         951.4    101019593
1400.36/607.79	c ******************.............. | *.........               1250000 | 77703               1400.7         892.4    111955674
1601.83/688.58	c ******************.............. | *.........               1329000 | 84145               1602.0         829.6    121457283

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver version 3.2.9 (svn:503) (c) roussel@cril.univ-artois.fr

command line: runsolver --timestamp -w /tmp/evaluation-result-1776298-1243674096/watcher-1776298-1243674096 -o /tmp/evaluation-result-1776298-1243674096/solver-1776298-1243674096 -C 10000 -W 1000 -M 2047 HOME/ttsth-5-0 -num-threads 4 HOME/instance-1776298-1243674096.cnf 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 10000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 10030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 2096128 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 2147328 KiB
Current StackSize limit: 8192 KiB


[startup+0 s]
/proc/loadavg: 6.16 4.91 4.42 5/104 10670
/proc/meminfo: memFree=2186796/3368532 swapFree=1954076/1959888
[pid=10670] ppid=10668 vsize=1272 CPUtime=0.01
/proc/10670/stat : 10670 (ttsth-5-0) R 10668 10670 2540 0 -1 4202496 218 0 0 0 1 0 0 0 19 0 1 0 1121510740 1302528 174 4294967295 134512640 135237372 3217442736 3217442184 134730961 0 0 0 0 0 0 0 17 3 0 0 0
/proc/10670/statm: 318 174 75 177 0 140 0

[startup+0.097351 s]
/proc/loadavg: 6.16 4.91 4.42 5/104 10670
/proc/meminfo: memFree=2186796/3368532 swapFree=1954076/1959888
[pid=10670] ppid=10668 vsize=5336 CPUtime=0.09
/proc/10670/stat : 10670 (ttsth-5-0) R 10668 10670 2540 0 -1 4202496 1203 0 0 0 9 0 0 0 19 0 1 0 1121510740 5464064 1159 4294967295 134512640 135237372 3217442736 3217442184 134732614 0 0 0 0 0 0 0 17 3 0 0 0
/proc/10670/statm: 1334 1159 75 177 0 1156 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 5336

[startup+0.107352 s]
/proc/loadavg: 6.16 4.91 4.42 5/104 10670
/proc/meminfo: memFree=2186796/3368532 swapFree=1954076/1959888
[pid=10670] ppid=10668 vsize=5600 CPUtime=0.1
/proc/10670/stat : 10670 (ttsth-5-0) R 10668 10670 2540 0 -1 4202496 1263 0 0 0 10 0 0 0 19 0 1 0 1121510740 5734400 1219 4294967295 134512640 135237372 3217442736 3217442280 134732614 0 0 0 0 0 0 0 17 3 0 0 0
/proc/10670/statm: 1400 1219 75 177 0 1222 0
Current children cumulated CPU time (s) 0.1
Current children cumulated vsize (KiB) 5600

[startup+0.307334 s]
/proc/loadavg: 6.16 4.91 4.42 5/104 10670
/proc/meminfo: memFree=2186796/3368532 swapFree=1954076/1959888
[pid=10670] ppid=10668 vsize=11284 CPUtime=0.3
/proc/10670/stat : 10670 (ttsth-5-0) R 10668 10670 2540 0 -1 4202496 2664 0 0 0 29 1 0 0 20 0 1 0 1121510740 11554816 2619 4294967295 134512640 135237372 3217442736 3217442240 134540092 0 0 0 0 0 0 0 17 3 0 0 0
/proc/10670/statm: 2821 2619 76 177 0 2643 0
Current children cumulated CPU time (s) 0.3
Current children cumulated vsize (KiB) 11284

[startup+0.707303 s]
/proc/loadavg: 6.16 4.91 4.42 5/104 10670
/proc/meminfo: memFree=2186796/3368532 swapFree=1954076/1959888
[pid=10670] ppid=10668 vsize=11920 CPUtime=0.7
/proc/10670/stat : 10670 (ttsth-5-0) R 10668 10670 2540 0 -1 4202496 4141 0 0 0 68 2 0 0 21 0 1 0 1121510740 12206080 2828 4294967295 134512640 135237372 3217442736 3217428376 134730102 0 0 0 0 0 0 0 17 3 0 0 0
/proc/10670/statm: 2980 2828 90 177 0 2802 0
Current children cumulated CPU time (s) 0.7
Current children cumulated vsize (KiB) 11920

[startup+1.50726 s]
/proc/loadavg: 5.91 4.87 4.41 4/105 10671
/proc/meminfo: memFree=2173328/3368532 swapFree=1954076/1959888
[pid=10670] ppid=10668 vsize=57756 CPUtime=1.91
/proc/10670/stat : 10670 (ttsth-5-0) S 10668 10670 2540 0 -1 4202496 7154 0 0 0 188 3 0 0 22 0 5 0 1121510740 59142144 5841 4294967295 134512640 135237372 3217442736 3217442332 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/10670/statm: 14439 5841 95 177 0 14261 0
Current children cumulated CPU time (s) 1.91
Current children cumulated vsize (KiB) 57756

[startup+3.10715 s]
/proc/loadavg: 5.91 4.87 4.41 7/109 10675
/proc/meminfo: memFree=2156240/3368532 swapFree=1954076/1959888
[pid=10670] ppid=10668 vsize=77172 CPUtime=5.1
/proc/10670/stat : 10670 (ttsth-5-0) S 10668 10670 2540 0 -1 4202496 12203 0 0 0 505 5 0 0 22 0 5 0 1121510740 79024128 10890 4294967295 134512640 135237372 3217442736 3217442332 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/10670/statm: 19293 10890 95 177 0 19115 0
[pid=10670/tid=10672] ppid=10668 vsize=77172 CPUtime=1.01
/proc/10670/task/10672/stat : 10672 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 1602 0 0 0 100 1 0 0 25 0 5 0 1121510849 79024128 10890 4294967295 134512640 135237372 3217442736 3085973536 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=10670/tid=10673] ppid=10668 vsize=77172 CPUtime=0.91
/proc/10670/task/10673/stat : 10673 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 1619 0 0 0 91 0 0 0 25 0 5 0 1121510849 79024128 10890 4294967295 134512640 135237372 3217442736 3077586324 134634454 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=10670/tid=10674] ppid=10668 vsize=77172 CPUtime=1.15
/proc/10670/task/10674/stat : 10674 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 1959 0 0 0 114 1 0 0 25 0 5 0 1121510850 79024128 10890 4294967295 134512640 135237372 3217442736 3069191024 134588792 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=10670/tid=10675] ppid=10668 vsize=77172 CPUtime=0.95
/proc/10670/task/10675/stat : 10675 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 1546 0 0 0 95 0 0 0 25 0 5 0 1121510850 79024128 10890 4294967295 134512640 135237372 3217442736 3060802464 134546034 0 0 0 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 5.1
Current children cumulated vsize (KiB) 77172

[startup+6.30694 s]
/proc/loadavg: 5.91 4.87 4.41 7/109 10675
/proc/meminfo: memFree=2115476/3368532 swapFree=1954076/1959888
[pid=10670] ppid=10668 vsize=117000 CPUtime=13.04
/proc/10670/stat : 10670 (ttsth-5-0) S 10668 10670 2540 0 -1 4202496 22017 0 0 0 1296 8 0 0 22 0 5 0 1121510740 119808000 20704 4294967295 134512640 135237372 3217442736 3217442332 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/10670/statm: 29250 20704 95 177 0 29072 0
[pid=10670/tid=10672] ppid=10668 vsize=117000 CPUtime=2.49
/proc/10670/task/10672/stat : 10672 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 3167 0 0 0 248 1 0 0 25 0 5 0 1121510849 119808000 20704 4294967295 134512640 135237372 3217442736 3085965792 134546086 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=10670/tid=10673] ppid=10668 vsize=117000 CPUtime=2.37
/proc/10670/task/10673/stat : 10673 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 3754 0 0 0 235 2 0 0 25 0 5 0 1121510849 119808000 20704 4294967295 134512640 135237372 3217442736 3077586208 134546034 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=10670/tid=10674] ppid=10668 vsize=117000 CPUtime=2.95
/proc/10670/task/10674/stat : 10674 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 4167 0 0 0 293 2 0 0 25 0 5 0 1121510850 119808000 20704 4294967295 134512640 135237372 3217442736 3069190472 134728912 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=10670/tid=10675] ppid=10668 vsize=117000 CPUtime=4.15
/proc/10670/task/10675/stat : 10675 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 5452 0 0 0 415 0 0 0 25 0 5 0 1121510850 119808000 20704 4294967295 134512640 135237372 3217442736 3060805728 134546026 0 0 0 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 13.04
Current children cumulated vsize (KiB) 117000

[startup+12.7065 s]
/proc/loadavg: 5.99 4.93 4.43 7/109 10675
/proc/meminfo: memFree=2056400/3368532 swapFree=1954076/1959888
[pid=10670] ppid=10668 vsize=172432 CPUtime=27.96
/proc/10670/stat : 10670 (ttsth-5-0) S 10668 10670 2540 0 -1 4202496 35549 0 0 0 2786 10 0 0 22 0 5 0 1121510740 176570368 34236 4294967295 134512640 135237372 3217442736 3217442332 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/10670/statm: 43108 34236 95 177 0 42930 0
[pid=10670/tid=10672] ppid=10668 vsize=172432 CPUtime=5.72
/proc/10670/task/10672/stat : 10672 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 5849 0 0 0 570 2 0 0 25 0 5 0 1121510849 176570368 34236 4294967295 134512640 135237372 3217442736 3085966096 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=10670/tid=10673] ppid=10668 vsize=172432 CPUtime=5.61
/proc/10670/task/10673/stat : 10673 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 7046 0 0 0 559 2 0 0 25 0 5 0 1121510849 176570368 34237 4294967295 134512640 135237372 3217442736 3077576928 134562338 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=10670/tid=10674] ppid=10668 vsize=172432 CPUtime=6.12
/proc/10670/task/10674/stat : 10674 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 6946 0 0 0 609 3 0 0 25 0 5 0 1121510850 176570368 34237 4294967295 134512640 135237372 3217442736 3069195648 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=10670/tid=10675] ppid=10668 vsize=172432 CPUtime=9.43
/proc/10670/task/10675/stat : 10675 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 10232 0 0 0 943 0 0 0 25 0 5 0 1121510850 176570368 34237 4294967295 134512640 135237372 3217442736 3060803856 134546034 0 0 0 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 27.96
Current children cumulated vsize (KiB) 172432

[startup+25.5056 s]
/proc/loadavg: 5.99 4.96 4.45 7/109 10677
/proc/meminfo: memFree=1977600/3368532 swapFree=1954076/1959888
[pid=10670] ppid=10668 vsize=247460 CPUtime=53.56
/proc/10670/stat : 10670 (ttsth-5-0) S 10668 10670 2540 0 -1 4202496 54297 0 0 0 5339 17 0 0 22 0 5 0 1121510740 253399040 52984 4294967295 134512640 135237372 3217442736 3217442332 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/10670/statm: 61865 52984 95 177 0 61687 0
[pid=10670/tid=10672] ppid=10668 vsize=247460 CPUtime=12.09
/proc/10670/task/10672/stat : 10672 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 10283 0 0 0 1203 6 0 0 25 0 5 0 1121510849 253399040 52984 4294967295 134512640 135237372 3217442736 3085967520 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=10670/tid=10673] ppid=10668 vsize=247460 CPUtime=12.21
/proc/10670/task/10673/stat : 10673 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 12231 0 0 0 1217 4 0 0 25 0 5 0 1121510849 253399040 52984 4294967295 134512640 135237372 3217442736 3077578368 134562338 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=10670/tid=10674] ppid=10668 vsize=247460 CPUtime=12.55
/proc/10670/task/10674/stat : 10674 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 11780 0 0 0 1251 4 0 0 25 0 5 0 1121510850 253399040 52984 4294967295 134512640 135237372 3217442736 3069187792 134562311 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=10670/tid=10675] ppid=10668 vsize=247460 CPUtime=15.63
/proc/10670/task/10675/stat : 10675 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 14526 0 0 0 1563 0 0 0 25 0 5 0 1121510850 253399040 52984 4294967295 134512640 135237372 3217442736 3060804432 134546026 0 0 0 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 53.56
Current children cumulated vsize (KiB) 247460

[startup+51.1039 s]
/proc/loadavg: 6.07 5.06 4.49 7/109 10678
/proc/meminfo: memFree=1853984/3368532 swapFree=1954076/1959888
[pid=10670] ppid=10668 vsize=374168 CPUtime=108.67
/proc/10670/stat : 10670 (ttsth-5-0) S 10668 10670 2540 0 -1 4202496 85402 0 0 0 10837 30 0 0 22 0 5 0 1121510740 383148032 84089 4294967295 134512640 135237372 3217442736 3217442332 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/10670/statm: 93542 84089 95 177 0 93364 0
[pid=10670/tid=10672] ppid=10668 vsize=374168 CPUtime=24.85
/proc/10670/task/10672/stat : 10672 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 17436 0 0 0 2474 11 0 0 25 0 5 0 1121510849 383148032 84089 4294967295 134512640 135237372 3217442736 3085957920 134546026 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=10670/tid=10673] ppid=10668 vsize=374168 CPUtime=25.01
/proc/10670/task/10673/stat : 10673 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 20329 0 0 0 2496 5 0 0 25 0 5 0 1121510849 383148032 84089 4294967295 134512640 135237372 3217442736 3077577004 134562272 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=10670/tid=10674] ppid=10668 vsize=374168 CPUtime=25.05
/proc/10670/task/10674/stat : 10674 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 17972 0 0 0 2495 10 0 0 25 0 5 0 1121510850 383148032 84089 4294967295 134512640 135237372 3217442736 3069192300 134549947 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=10670/tid=10675] ppid=10668 vsize=374168 CPUtime=32.68
/proc/10670/task/10675/stat : 10675 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 24188 0 0 0 3267 1 0 0 25 0 5 0 1121510850 383148032 84089 4294967295 134512640 135237372 3217442736 3060806944 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 108.67
Current children cumulated vsize (KiB) 374168

[startup+102.32 s]
/proc/loadavg: 6.46 5.34 4.62 7/109 10870
/proc/meminfo: memFree=1658032/3368532 swapFree=1954076/1959888
[pid=10670] ppid=10668 vsize=572028 CPUtime=217.49
/proc/10670/stat : 10670 (ttsth-5-0) S 10668 10670 2540 0 -1 4202496 133787 0 0 0 21706 43 0 0 22 0 5 0 1121510740 585756672 132474 4294967295 134512640 135237372 3217442736 3217442332 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/10670/statm: 143007 132474 95 177 0 142829 0
[pid=10670/tid=10672] ppid=10668 vsize=572028 CPUtime=53.34
/proc/10670/task/10672/stat : 10672 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 30191 0 0 0 5321 13 0 0 25 0 5 0 1121510849 585756672 132474 4294967295 134512640 135237372 3217442736 3085970160 134546034 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=10670/tid=10673] ppid=10668 vsize=572028 CPUtime=50.51
/proc/10670/task/10673/stat : 10673 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 32739 0 0 0 5040 11 0 0 25 0 5 0 1121510849 585756672 132474 4294967295 134512640 135237372 3217442736 3077574608 134562401 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=10670/tid=10674] ppid=10668 vsize=572028 CPUtime=54.05
/proc/10670/task/10674/stat : 10674 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 30134 0 0 0 5392 13 0 0 25 0 5 0 1121510850 585756672 132474 4294967295 134512640 135237372 3217442736 3069191904 134562331 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=10670/tid=10675] ppid=10668 vsize=572028 CPUtime=58.51
/proc/10670/task/10675/stat : 10675 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 35246 0 0 0 5848 3 0 0 25 0 5 0 1121510850 585756672 132474 4294967295 134512640 135237372 3217442736 3060799264 134562328 0 0 0 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 217.49
Current children cumulated vsize (KiB) 572028

[startup+162.306 s]
/proc/loadavg: 6.62 5.59 4.75 7/109 11057
/proc/meminfo: memFree=1472620/3368532 swapFree=1954076/1959888
[pid=10670] ppid=10668 vsize=761876 CPUtime=341.69
/proc/10670/stat : 10670 (ttsth-5-0) S 10668 10670 2540 0 -1 4202496 180360 0 0 0 34111 58 0 0 22 0 5 0 1121510740 780161024 179047 4294967295 134512640 135237372 3217442736 3217442332 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/10670/statm: 190469 179047 95 177 0 190291 0
[pid=10670/tid=10672] ppid=10668 vsize=761876 CPUtime=86.29
/proc/10670/task/10672/stat : 10672 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 42552 0 0 0 8615 14 0 0 25 0 5 0 1121510849 780161024 179047 4294967295 134512640 135237372 3217442736 3085962016 134562338 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=10670/tid=10673] ppid=10668 vsize=761876 CPUtime=82.24
/proc/10670/task/10673/stat : 10673 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 45044 0 0 0 8209 15 0 0 25 0 5 0 1121510849 780161024 179047 4294967295 134512640 135237372 3217442736 3077576272 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=10670/tid=10674] ppid=10668 vsize=761876 CPUtime=83.85
/proc/10670/task/10674/stat : 10674 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 40776 0 0 0 8367 18 0 0 25 0 5 0 1121510850 780161024 179047 4294967295 134512640 135237372 3217442736 3069190752 134562338 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=10670/tid=10675] ppid=10668 vsize=761876 CPUtime=88.23
/proc/10670/task/10675/stat : 10675 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 46511 0 0 0 8815 8 0 0 25 0 5 0 1121510850 780161024 179047 4294967295 134512640 135237372 3217442736 3060799312 134562420 0 0 0 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 341.69
Current children cumulated vsize (KiB) 761876

[startup+222.312 s]
/proc/loadavg: 6.85 5.84 4.89 7/109 11246
/proc/meminfo: memFree=1305512/3368532 swapFree=1954076/1959888
[pid=10670] ppid=10668 vsize=930312 CPUtime=472.25
/proc/10670/stat : 10670 (ttsth-5-0) S 10668 10670 2540 0 -1 4202496 221402 0 0 0 47152 73 0 0 22 0 5 0 1121510740 952639488 220089 4294967295 134512640 135237372 3217442736 3217442332 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/10670/statm: 232578 220089 95 177 0 232400 0
[pid=10670/tid=10672] ppid=10668 vsize=930312 CPUtime=127.06
/proc/10670/task/10672/stat : 10672 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 55303 0 0 0 12686 20 0 0 25 0 5 0 1121510849 952639488 220089 4294967295 134512640 135237372 3217442736 3085961808 134546026 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=10670/tid=10673] ppid=10668 vsize=930312 CPUtime=112.03
/proc/10670/task/10673/stat : 10673 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 55574 0 0 0 11184 19 0 0 25 0 5 0 1121510849 952639488 220089 4294967295 134512640 135237372 3217442736 3077581168 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=10670/tid=10674] ppid=10668 vsize=930312 CPUtime=113.75
/proc/10670/task/10674/stat : 10674 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 49789 0 0 0 11353 22 0 0 25 0 5 0 1121510850 952639488 220089 4294967295 134512640 135237372 3217442736 3069192080 134562338 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=10670/tid=10675] ppid=10668 vsize=930312 CPUtime=118.33
/proc/10670/task/10675/stat : 10675 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 55259 0 0 0 11824 9 0 0 25 0 5 0 1121510850 952639488 220089 4294967295 134512640 135237372 3217442736 3060802272 134591086 0 0 0 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 472.25
Current children cumulated vsize (KiB) 930312


################
# More data... #
################

/proc/meminfo: memFree=1160240/3368532 swapFree=1954076/1959888
[pid=10670] ppid=10668 vsize=1082748 CPUtime=603.98
/proc/10670/stat : 10670 (ttsth-5-0) S 10668 10670 2540 0 -1 4202496 258253 0 0 0 60305 93 0 0 22 0 5 0 1121510740 1108733952 256940 4294967295 134512640 135237372 3217442736 3217442332 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/10670/statm: 270687 256940 95 177 0 270509 0
[pid=10670/tid=10672] ppid=10668 vsize=1082748 CPUtime=157.32
/proc/10670/task/10672/stat : 10672 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 63987 0 0 0 15710 22 0 0 25 0 5 0 1121510849 1108733952 256940 4294967295 134512640 135237372 3217442736 3085974688 134561428 0 0 0 0 0 0 0 -1 3 0 0 0
[pid=10670/tid=10673] ppid=10668 vsize=1082748 CPUtime=153.82
/proc/10670/task/10673/stat : 10673 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 67853 0 0 0 15355 27 0 0 25 0 5 0 1121510849 1108733952 256940 4294967295 134512640 135237372 3217442736 3077582080 134546034 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=10670/tid=10674] ppid=10668 vsize=1082748 CPUtime=143.53
/proc/10670/task/10674/stat : 10674 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 57496 0 0 0 14323 30 0 0 25 0 5 0 1121510850 1108733952 256940 4294967295 134512640 135237372 3217442736 3069189648 134562290 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=10670/tid=10675] ppid=10668 vsize=1082748 CPUtime=148.23
/proc/10670/task/10675/stat : 10675 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 63440 0 0 0 14812 11 0 0 25 0 5 0 1121510850 1108733952 256940 4294967295 134512640 135237372 3217442736 3060799008 134562344 0 0 0 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 603.98
Current children cumulated vsize (KiB) 1082748

[startup+342.314 s]
/proc/loadavg: 6.55 6.11 5.10 8/109 11621
/proc/meminfo: memFree=996652/3368532 swapFree=1954076/1959888
[pid=10670] ppid=10668 vsize=1247120 CPUtime=752.8
/proc/10670/stat : 10670 (ttsth-5-0) S 10668 10670 2540 0 -1 4202496 298352 0 0 0 75173 107 0 0 22 0 5 0 1121510740 1277050880 297039 4294967295 134512640 135237372 3217442736 3217442332 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/10670/statm: 311780 297039 95 177 0 311602 0
[pid=10670/tid=10672] ppid=10668 vsize=1247120 CPUtime=187.7
/proc/10670/task/10672/stat : 10672 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 72415 0 0 0 18744 26 0 0 25 0 5 0 1121510849 1277050880 297039 4294967295 134512640 135237372 3217442736 3085972864 134562284 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=10670/tid=10673] ppid=10668 vsize=1247120 CPUtime=184.11
/proc/10670/task/10673/stat : 10673 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 76937 0 0 0 18383 28 0 0 25 0 5 0 1121510849 1277050880 297039 4294967295 134512640 135237372 3217442736 3077575136 134561428 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=10670/tid=10674] ppid=10668 vsize=1247120 CPUtime=173.12
/proc/10670/task/10674/stat : 10674 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 64859 0 0 0 17279 33 0 0 25 0 5 0 1121510850 1277050880 297039 4294967295 134512640 135237372 3217442736 3069195556 134634454 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=10670/tid=10675] ppid=10668 vsize=1247120 CPUtime=206.79
/proc/10670/task/10675/stat : 10675 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 78664 0 0 0 20662 17 0 0 25 0 5 0 1121510850 1277050880 297039 4294967295 134512640 135237372 3217442736 3060802736 134565869 0 0 0 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 752.8
Current children cumulated vsize (KiB) 1247120

[startup+402.31 s]
/proc/loadavg: 6.81 6.27 5.22 8/109 11811
/proc/meminfo: memFree=855064/3368532 swapFree=1954076/1959888
[pid=10670] ppid=10668 vsize=1393656 CPUtime=896.33
/proc/10670/stat : 10670 (ttsth-5-0) S 10668 10670 2540 0 -1 4202496 333871 0 0 0 89510 123 0 0 22 0 5 0 1121510740 1427103744 332558 4294967295 134512640 135237372 3217442736 3217442332 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/10670/statm: 348414 332558 95 177 0 348236 0
[pid=10670/tid=10672] ppid=10668 vsize=1393656 CPUtime=218.9
/proc/10670/task/10672/stat : 10672 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 80051 0 0 0 21859 31 0 0 25 0 5 0 1121510849 1427103744 332559 4294967295 134512640 135237372 3217442736 3085965696 134546034 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=10670/tid=10673] ppid=10668 vsize=1393656 CPUtime=223.51
/proc/10670/task/10673/stat : 10673 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 86497 0 0 0 22321 30 0 0 25 0 5 0 1121510849 1427103744 332559 4294967295 134512640 135237372 3217442736 3077587392 134562338 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=10670/tid=10674] ppid=10668 vsize=1393656 CPUtime=203.61
/proc/10670/task/10674/stat : 10674 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 72614 0 0 0 20322 39 0 0 25 0 5 0 1121510850 1427103744 332559 4294967295 134512640 135237372 3217442736 3069192768 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=10670/tid=10675] ppid=10668 vsize=1393656 CPUtime=249.23
/proc/10670/task/10675/stat : 10675 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 89233 0 0 0 24903 20 0 0 25 0 5 0 1121510850 1427103744 332559 4294967295 134512640 135237372 3217442736 3060803664 134546034 0 0 0 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 896.33
Current children cumulated vsize (KiB) 1393656

[startup+462.316 s]
/proc/loadavg: 6.61 6.32 5.30 7/109 11998
/proc/meminfo: memFree=718940/3368532 swapFree=1954076/1959888
[pid=10670] ppid=10668 vsize=1530608 CPUtime=1043.35
/proc/10670/stat : 10670 (ttsth-5-0) S 10668 10670 2540 0 -1 4202496 367510 0 0 0 104199 136 0 0 22 0 5 0 1121510740 1567342592 366197 4294967295 134512640 135237372 3217442736 3217442332 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/10670/statm: 382652 366197 95 177 0 382474 0
[pid=10670/tid=10672] ppid=10668 vsize=1530608 CPUtime=248.72
/proc/10670/task/10672/stat : 10672 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 86943 0 0 0 24836 36 0 0 25 0 5 0 1121510849 1567342592 366197 4294967295 134512640 135237372 3217442736 3085973584 134562338 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=10670/tid=10673] ppid=10668 vsize=1530608 CPUtime=253.56
/proc/10670/task/10673/stat : 10673 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 93808 0 0 0 25323 33 0 0 25 0 5 0 1121510849 1567342592 366197 4294967295 134512640 135237372 3217442736 3077581904 134562338 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=10670/tid=10674] ppid=10668 vsize=1530608 CPUtime=260.92
/proc/10670/task/10674/stat : 10674 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 86024 0 0 0 26049 43 0 0 25 0 5 0 1121510850 1567342592 366197 4294967295 134512640 135237372 3217442736 3069185168 134562290 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=10670/tid=10675] ppid=10668 vsize=1530608 CPUtime=279.07
/proc/10670/task/10675/stat : 10675 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 95258 0 0 0 27886 21 0 0 25 0 5 0 1121510850 1567342592 366197 4294967295 134512640 135237372 3217442736 3060800752 134562338 0 0 0 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 1043.35
Current children cumulated vsize (KiB) 1530608

[startup+522.312 s]
/proc/loadavg: 6.41 6.32 5.36 7/109 12191
/proc/meminfo: memFree=597300/3368532 swapFree=1954076/1959888
[pid=10670] ppid=10668 vsize=1658660 CPUtime=1187.62
/proc/10670/stat : 10670 (ttsth-5-0) S 10668 10670 2540 0 -1 4202496 398333 0 0 0 118615 147 0 0 22 0 5 0 1121510740 1698467840 397020 4294967295 134512640 135237372 3217442736 3217442332 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/10670/statm: 414665 397020 95 177 0 414487 0
[pid=10670/tid=10672] ppid=10668 vsize=1658660 CPUtime=278.32
/proc/10670/task/10672/stat : 10672 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 92992 0 0 0 27793 39 0 0 25 0 5 0 1121510849 1698467840 397020 4294967295 134512640 135237372 3217442736 3085970144 134562344 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=10670/tid=10673] ppid=10668 vsize=1658660 CPUtime=295.3
/proc/10670/task/10673/stat : 10673 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 103404 0 0 0 29494 36 0 0 25 0 5 0 1121510849 1698467840 397020 4294967295 134512640 135237372 3217442736 3077584528 134546026 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=10670/tid=10674] ppid=10668 vsize=1658660 CPUtime=294.99
/proc/10670/task/10674/stat : 10674 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 92912 0 0 0 29453 46 0 0 25 0 5 0 1121510850 1698467840 397020 4294967295 134512640 135237372 3217442736 3069191840 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=10670/tid=10675] ppid=10668 vsize=1658660 CPUtime=317.93
/proc/10670/task/10675/stat : 10675 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 103548 0 0 0 31770 23 0 0 25 0 5 0 1121510850 1698467840 397020 4294967295 134512640 135237372 3217442736 3060810176 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1187.62
Current children cumulated vsize (KiB) 1658660

[startup+582.308 s]
/proc/loadavg: 6.84 6.44 5.46 8/109 12377
/proc/meminfo: memFree=464348/3368532 swapFree=1954076/1959888
[pid=10670] ppid=10668 vsize=1793584 CPUtime=1337.05
/proc/10670/stat : 10670 (ttsth-5-0) S 10668 10670 2540 0 -1 4202496 431308 0 0 0 133550 155 0 0 22 0 5 0 1121510740 1836630016 429995 4294967295 134512640 135237372 3217442736 3217442332 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/10670/statm: 448396 429995 95 177 0 448218 0
[pid=10670/tid=10672] ppid=10668 vsize=1793584 CPUtime=309.21
/proc/10670/task/10672/stat : 10672 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 99468 0 0 0 30882 39 0 0 25 0 5 0 1121510849 1836630016 429995 4294967295 134512640 135237372 3217442736 3085976016 134590180 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=10670/tid=10673] ppid=10668 vsize=1793584 CPUtime=344.11
/proc/10670/task/10673/stat : 10673 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 115267 0 0 0 34370 41 0 0 25 0 5 0 1121510849 1836630016 429995 4294967295 134512640 135237372 3217442736 3077584464 134562341 0 0 0 0 0 0 0 -1 2 0 0 0
[pid=10670/tid=10674] ppid=10668 vsize=1793584 CPUtime=332.25
/proc/10670/task/10674/stat : 10674 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 100956 0 0 0 33178 47 0 0 25 0 5 0 1121510850 1836630016 429995 4294967295 134512640 135237372 3217442736 3069199440 134749345 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=10670/tid=10675] ppid=10668 vsize=1793584 CPUtime=350.41
/proc/10670/task/10675/stat : 10675 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 110140 0 0 0 35016 25 0 0 25 0 5 0 1121510850 1836630016 429995 4294967295 134512640 135237372 3217442736 3060796464 134562338 0 0 0 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 1337.05
Current children cumulated vsize (KiB) 1793584

[startup+642.304 s]
/proc/loadavg: 6.63 6.45 5.52 8/109 12566
/proc/meminfo: memFree=343692/3368532 swapFree=1954076/1959888
[pid=10670] ppid=10668 vsize=1915848 CPUtime=1486.58
/proc/10670/stat : 10670 (ttsth-5-0) S 10668 10670 2540 0 -1 4202496 460990 0 0 0 148489 169 0 0 22 0 5 0 1121510740 1961828352 459677 4294967295 134512640 135237372 3217442736 3217442332 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/10670/statm: 478962 459677 95 177 0 478784 0
[pid=10670/tid=10672] ppid=10668 vsize=1915848 CPUtime=346.98
/proc/10670/task/10672/stat : 10672 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 107419 0 0 0 34654 44 0 0 25 0 5 0 1121510849 1961828352 459677 4294967295 134512640 135237372 3217442736 3085964608 134562309 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=10670/tid=10673] ppid=10668 vsize=1915848 CPUtime=376.61
/proc/10670/task/10673/stat : 10673 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 121534 0 0 0 37617 44 0 0 25 0 5 0 1121510849 1961828352 459677 4294967295 134512640 135237372 3217442736 3077578912 134562338 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=10670/tid=10674] ppid=10668 vsize=1915848 CPUtime=378.86
/proc/10670/task/10674/stat : 10674 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 110243 0 0 0 37833 53 0 0 25 0 5 0 1121510850 1961828352 459677 4294967295 134512640 135237372 3217442736 3069191440 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=10670/tid=10675] ppid=10668 vsize=1915848 CPUtime=383.05
/proc/10670/task/10675/stat : 10675 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 116317 0 0 0 38280 25 0 0 25 0 5 0 1121510850 1961828352 459677 4294967295 134512640 135237372 3217442736 3060804432 134562338 0 0 0 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 1486.58
Current children cumulated vsize (KiB) 1915848

[startup+702.31 s]
/proc/loadavg: 6.58 6.47 5.59 7/109 12754
/proc/meminfo: memFree=236780/3368532 swapFree=1954076/1959888
[pid=10670] ppid=10668 vsize=2024596 CPUtime=1636.21
/proc/10670/stat : 10670 (ttsth-5-0) S 10668 10670 2540 0 -1 4202496 487680 0 0 0 163437 184 0 0 22 0 5 0 1121510740 2073186304 486367 4294967295 134512640 135237372 3217442736 3217442332 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/10670/statm: 506149 486367 95 177 0 505971 0
[pid=10670/tid=10672] ppid=10668 vsize=2024596 CPUtime=376.72
/proc/10670/task/10672/stat : 10672 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 113036 0 0 0 37621 51 0 0 25 0 5 0 1121510849 2073186304 486367 4294967295 134512640 135237372 3217442736 3085974432 134587921 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=10670/tid=10673] ppid=10668 vsize=2024596 CPUtime=406.86
/proc/10670/task/10673/stat : 10673 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 126390 0 0 0 40639 47 0 0 25 0 5 0 1121510849 2073186304 486367 4294967295 134512640 135237372 3217442736 3077583552 134562370 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=10670/tid=10674] ppid=10668 vsize=2024596 CPUtime=438.72
/proc/10670/task/10674/stat : 10674 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 120738 0 0 0 43816 56 0 0 25 0 5 0 1121510850 2073186304 486367 4294967295 134512640 135237372 3217442736 3069188864 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=10670/tid=10675] ppid=10668 vsize=2024596 CPUtime=412.83
/proc/10670/task/10675/stat : 10675 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 122039 0 0 0 41256 27 0 0 25 0 5 0 1121510850 2073186304 486367 4294967295 134512640 135237372 3217442736 3060800400 134546034 0 0 0 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 1636.21
Current children cumulated vsize (KiB) 2024596



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+741.417 s]
/proc/loadavg: 6.69 6.51 5.63 9/110 12841
/proc/meminfo: memFree=166408/3368532 swapFree=1954076/1959888
[pid=10670] ppid=10668 vsize=2096136 CPUtime=1733.94
/proc/10670/stat : 10670 (ttsth-5-0) S 10668 10670 2540 0 -1 4202496 504868 0 0 0 173185 209 0 0 22 0 5 0 1121510740 2146443264 503555 4294967295 134512640 135237372 3217442736 3217442332 134629544 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/10670/statm: 524034 503555 95 177 0 523856 0
[pid=10670/tid=10672] ppid=10668 vsize=2096136 CPUtime=398.31
/proc/10670/task/10672/stat : 10672 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 117640 0 0 0 39778 53 0 0 25 0 5 0 1121510849 2146443264 503555 4294967295 134512640 135237372 3217442736 3085978448 134562338 0 0 0 0 0 0 0 -1 0 0 0 0
[pid=10670/tid=10673] ppid=10668 vsize=2096136 CPUtime=434.05
/proc/10670/task/10673/stat : 10673 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 130107 0 0 0 43344 61 0 0 25 0 5 0 1121510849 2146443264 503555 4294967295 134512640 135237372 3217442736 3077590672 134546026 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=10670/tid=10674] ppid=10668 vsize=2096136 CPUtime=468.05
/proc/10670/task/10674/stat : 10674 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 125955 0 0 0 46743 62 0 0 25 0 5 0 1121510850 2146443264 503555 4294967295 134512640 135237372 3217442736 3069199120 134562416 0 0 0 0 0 0 0 -1 1 0 0 0
[pid=10670/tid=10675] ppid=10668 vsize=2096136 CPUtime=432.45
/proc/10670/task/10675/stat : 10675 (ttsth-5-0) R 10668 10670 2540 0 -1 4202560 125689 0 0 0 43215 30 0 0 25 0 5 0 1121510850 2146443264 503555 4294967295 134512640 135237372 3217442736 3060794512 134562338 0 0 0 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 1733.94
Current children cumulated vsize (KiB) 2096136

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+741.527 s]
/proc/loadavg: 6.69 6.51 5.63 9/110 12841
/proc/meminfo: memFree=166408/3368532 swapFree=1954076/1959888
[pid=10670] ppid=10668 vsize=0 CPUtime=1734.07
/proc/10670/stat : 10670 (ttsth-5-0) Z 10668 10670 2540 0 -1 4203524 504877 0 0 0 173189 218 0 0 18 0 2 0 1121510740 0 0 4294967295 0 0 0 0 0 0 0 0 0 4294967295 0 0 17 3 0 0 0
/proc/10670/statm: 0 0 0 0 0 0 0
[pid=10670/tid=10672] ppid=10668 vsize=0 CPUtime=398.42
/proc/10670/task/10672/stat : 10672 (ttsth-5-0) R 10668 10670 2540 0 -1 4203588 117640 0 0 0 39780 62 0 0 25 0 2 0 1121510849 0 0 4294967295 0 0 0 0 0 0 0 0 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1734.07
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 741.548
CPU time (s): 1734.09
CPU user time (s): 1731.89
CPU system time (s): 2.2
CPU usage (%): 233.847
Max. virtual memory (cumulated for all children) (KiB): 2096136

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1731.89
system time used= 2.2
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 504877
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 708
involuntary context switches= 48285

runsolver used 0 second user time and 0 second system time

The end

Launcher Data 

Begin job on tipi44 at 2009-05-30 11:01:36
IDJOB=1776298
IDBENCH=20485
IDSOLVER=632
FILE ID=tipi44/1776298-1243674096
PBS_JOBID=
Free space on /tmp= 15893 MiB

SOLVER NAME= ttsth-5-0 2009-03-19 (4 threads)
BENCH NAME= SAT07/industrial/grieu/vmpc_25.cnf
COMMAND LINE= HOME/ttsth-5-0 -num-threads MAXNBTHREAD BENCHNAME
RUNSOLVER COMMAND LINE= runsolver --timestamp  -w /tmp/evaluation-result-1776298-1243674096/watcher-1776298-1243674096 -o /tmp/evaluation-result-1776298-1243674096/solver-1776298-1243674096 -C 10000 -W 1000 -M 2047  HOME/ttsth-5-0 -num-threads 4 HOME/instance-1776298-1243674096.cnf

TIME LIMIT= 10000 seconds
MEMORY LIMIT= 2047 MiB
MAX NB THREAD= 4

MD5SUM BENCH= c5b44b10975f50cd32d40650a6a22d18
RANDOM SEED= 387450205

Linux tipi44 2.6.21.5 #6 SMP Wed Jul 4 17:16:38 CEST 2007 i686 GNU/Linux

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU            5130  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.116
cache size	: 4096 KB
physical id	: 0
siblings	: 2
core id		: 0
cpu cores	: 2
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 3992.42
clflush size	: 64

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU            5130  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.116
cache size	: 4096 KB
physical id	: 0
siblings	: 2
core id		: 1
cpu cores	: 2
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 3990.10
clflush size	: 64

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU            5130  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.116
cache size	: 4096 KB
physical id	: 3
siblings	: 2
core id		: 0
cpu cores	: 2
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 3990.14
clflush size	: 64

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 15
model name	: Intel(R) Xeon(R) CPU            5130  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.116
cache size	: 4096 KB
physical id	: 3
siblings	: 2
core id		: 1
cpu cores	: 2
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 10
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl vmx tm2 ssse3 cx16 xtpr dca lahf_lm
bogomips	: 3990.16
clflush size	: 64


/proc/meminfo:
MemTotal:      3368532 kB
MemFree:       2186796 kB
Buffers:        176424 kB
Cached:         681724 kB
SwapCached:       2152 kB
Active:         775596 kB
Inactive:       296928 kB
HighTotal:     2489216 kB
HighFree:      1596456 kB
LowTotal:       879316 kB
LowFree:        590340 kB
SwapTotal:     1959888 kB
SwapFree:      1954076 kB
Dirty:            2168 kB
Writeback:           0 kB
AnonPages:      213640 kB
Mapped:           5448 kB
Slab:            97756 kB
SReclaimable:    87168 kB
SUnreclaim:      10588 kB
PageTables:        752 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   3644152 kB
Committed_AS:   296360 kB
VmallocTotal:   114680 kB
VmallocUsed:      1444 kB
VmallocChunk:   113072 kB

Free space on /tmp at the end= 15893 MiB
End job on tipi44 at 2009-05-30 11:13:59