Trace number 3265566

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.3 1282.29

General information on the benchmark

NameSAT05/crafted/markstrom05/eulcbip/
eulcbip-7-UNSAT.shuffled-as.sat05-3936.cnf
MD5SUM151ccdb2c52c7f525fde5dca388bb6e6
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark0.083986
Satisfiable
(Un)Satisfiability was proved
Number of variables135
Number of clauses672
Sum of the clauses size2684
Maximum clause length4
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 22
Number of clauses of size 30
Number of clauses of size 4670
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3265566-1304102351.cnf 
0.00/0.00	c Using SATTIMEOUT of 4800
0.00/0.00	c Using SATRAM of 15500
0.00/0.00	c Size of subset node = 32
0.00/0.00	c Variables = 135
0.00/0.00	c Clauses = 672
0.00/0.00	c Literals = 2684
0.00/0.00	c Reduced size:
0.00/0.00	c Variables = 135
0.00/0.00	c Clauses = 672
0.00/0.00	c Literals = 2684
0.00/0.00	c a2
0.00/0.00	c a1
0.29/0.34	c Independent sub-problems = 1
0.29/0.34	c Sub-problem with 135 variables and 672 clauses
0.29/0.34	c using permutation method 4
0.29/0.34	c count = 3
0.29/0.34	c Ordering finished 0.340 seconds
0.29/0.34	c mapped 0.340 seconds
0.29/0.34	c permuted 0.340 seconds
0.29/0.34	c about to sort
0.29/0.34	c sorted
0.29/0.34	c clauses now 672
0.29/0.34	c init finished 0.340 seconds
0.29/0.34	c tree built 0.340 seconds
0.29/0.34	c 
0.29/0.34	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.29/0.34	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.29/0.34	c ................................ | ..........                    |                                  0.3         0.0             
0.29/0.34	c ................................ | ..........                  1 |                                  0.3         2.9             
0.29/0.34	c ................................ | ..........                  2 |                                  0.3         5.9             
0.29/0.34	c ................................ | ..........                  3 |                                  0.3         8.8             
0.29/0.34	c ................................ | ..........                  4 |                                  0.3        11.8             
0.29/0.34	c ................................ | ..........                  5 |                                  0.3        14.7             
0.29/0.34	c ................................ | ..........               1000 | 61           1027                0.3      2857.1        13697
95.78/25.05	c ................................ | ..........            2815000 | 409288       2945783            96.0     29322.9     12684202
191.77/50.18	c ................................ | ..........            5051000 | 746372       5287286           192.1     26299.1     15234795
287.89/75.63	c ................................ | ..........            7602000 | 1101401      7647078           288.0     26394.9     19242308
383.81/101.14	c ................................ | ..........            9880000 | 1432060      9928336           384.0     25729.2     22718388
479.89/126.74	c ................................ | *.........           12078000 | 1761218      12191213          480.0     25160.9     25287974
575.79/152.57	c ................................ | *.........           14414000 | 2083990      14451289          576.0     25023.4     30013157
671.73/178.17	c ................................ | *.........           16634000 | 2396853      16652343          672.0     24752.6     33359907
767.75/203.87	c ................................ | *.........           18683000 | 2706533      18738950          768.0     24325.9     36961645
863.98/229.81	c ................................ | *.........           20708000 | 3010184      20877547          864.0     23967.3     40923164
959.83/255.86	c ................................ | **........           22608000 | 3298156      22756493          960.0     23550.0     43208485
1056.09/281.86	c ................................ | **........           24502000 | 3584535      24702392         1056.0     23202.7     46050323
1151.92/307.31	c ................................ | **........           26418000 | 3851475      26699963         1152.0     22932.3     49036295
1248.01/333.00	c ................................ | **........           28259000 | 4097701      28606250         1248.0     22643.1     51504290
1343.91/358.63	c ................................ | **........           30072000 | 4341592      30518856         1344.0     22375.0     52861762
1440.06/384.30	c ................................ | ***.......           31801000 | 4593139      32408995         1440.0     22083.7     55049418
1535.82/409.96	c ................................ | ***.......           33521000 | 4837881      34265109         1536.0     21823.1     57303851
1631.81/435.76	c ................................ | ***.......           35185000 | 5076792      36025554         1632.0     21558.9     59259955
1727.81/461.37	c ................................ | ***.......           36874000 | 5323735      37718666         1728.0     21339.0     61722749
1823.99/487.01	c ................................ | ***.......           38555000 | 5575187      39448677         1824.0     21137.0     64186117
1919.65/512.49	c ................................ | ****......           40181000 | 5810652      41169122         1920.0     20927.5     65421254
2015.75/538.16	c ................................ | ****......           41890000 | 6043357      42875776         2016.0     20778.8     68136413
2111.73/563.96	c ................................ | ****......           43540000 | 6273299      44601429         2112.0     20615.4     70796943
2207.88/589.74	c ................................ | ****......           45217000 | 6496025      46376228         2208.0     20478.5     72159484
2303.92/615.42	c ................................ | ****......           46925000 | 6722839      48195392         2304.0     20366.6     72889250
2399.66/641.19	c ................................ | *****.....           48636000 | 6952805      49975324         2400.0     20264.7     74478510
2496.09/667.01	c ................................ | *****.....           50229000 | 7170736      51764172         2496.0     20123.7     76486941
2591.69/692.88	c ................................ | *****.....           51740000 | 7381579      53527554         2592.0     19961.1     77456408
2687.67/718.59	c ................................ | *****.....           53271000 | 7595039      55202409         2688.0     19818.0     78602259
2784.78/744.31	c ................................ | *****.....           54840000 | 7820503      56960772         2784.8     19692.4     79044642
2879.75/769.96	c ................................ | ******....           56459000 | 8048980      58709928         2880.0     19603.7     80992761
2975.99/795.51	c ................................ | ******....           57962000 | 8258444      60345249         2976.0     19476.3     81940466
3071.90/821.04	c ................................ | ******....           59571000 | 8489755      62006866         3072.1     19391.3     83876202
3167.65/846.38	c ................................ | ******....           61086000 | 8717072      63648442         3168.0     19282.2     85012764
3263.89/871.94	c ................................ | ******....           62598000 | 8937552      65361611         3264.1     19178.0     86421480
3359.76/898.07	c ................................ | *******...           64293000 | 9169403      66990794         3360.1     19134.5     89448000
3455.84/923.94	c ................................ | *******...           65876000 | 9383484      68567198         3456.0     19061.2     90939461
3551.90/949.62	c ................................ | *******...           67497000 | 9602096      70145014         3552.0     19002.5     92372751
3647.81/975.08	c ................................ | *******...           68992000 | 9817810      71607674         3648.0     18912.1     93391412
3744.35/1000.61	c ................................ | *******...           70451000 | 10016307     73147851         3744.1     18816.6     94585187
3839.72/1026.07	c ................................ | ********..           71970000 | 10230223     74725853         3840.0     18742.0     96227395
3935.87/1051.54	c ................................ | ********..           73422000 | 10449869     76349168         3936.0     18653.9     98044355
4031.93/1077.13	c ................................ | ********..           74888000 | 10659442     77939554         4032.0     18573.3    101305171
4128.08/1103.27	c ................................ | ********..           76330000 | 10876288     79424849         4128.3     18489.5    103410211
4223.86/1128.85	c ................................ | ********..           77807000 | 11103168     80929319         4224.0     18420.0    104742290
4319.76/1154.68	c ................................ | *********.           79281000 | 11323728     82474510         4320.0     18352.0    106796576
4415.75/1180.26	c ................................ | *********.           80713000 | 11524937     84048926         4416.0     18277.4    107915218
4511.75/1205.66	c ................................ | *********.           82186000 | 11734145     85614080         4512.0     18214.9    109525872
4607.66/1230.99	c ................................ | *********.           83697000 | 11937206     87203361         4608.0     18163.4    111052224
4703.76/1256.47	c ................................ | *********.           85191000 | 12133904     88849345         4704.0     18110.3    111552992
4799.88/1282.04	c CPU limit (4800) reached
4799.88/1282.04	c Time = 4800.06 seconds
4799.88/1282.04	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3265566-1304102351/watcher-3265566-1304102351 -o /tmp/evaluation-result-3265566-1304102351/solver-3265566-1304102351 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3265566-1304102351.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.40 4.52 6.09 2/174 25851
/proc/meminfo: memFree=25308724/32951124 swapFree=67051468/67111528
[pid=25851] ppid=25849 vsize=8840 CPUtime=0 cores=1,3,5,7
/proc/25851/stat : 25851 (ttsth-5-4) R 25849 25851 25096 0 -1 4202496 300 0 0 0 0 0 0 0 23 0 1 0 521837827 9052160 243 33554432000 4194304 4338100 140735376996528 18446744073709551615 4295924 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/25851/statm: 2210 243 151 36 0 115 0

[startup+0.0686779 s]
/proc/loadavg: 3.40 4.52 6.09 2/174 25851
/proc/meminfo: memFree=25308724/32951124 swapFree=67051468/67111528
[pid=25851] ppid=25849 vsize=8840 CPUtime=0.06 cores=1,3,5,7
/proc/25851/stat : 25851 (ttsth-5-4) R 25849 25851 25096 0 -1 4202496 301 0 0 0 6 0 0 0 23 0 1 0 521837827 9052160 244 33554432000 4194304 4338100 140735376996528 18446744073709551615 4295699 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/25851/statm: 2210 244 152 36 0 115 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 8840

[startup+0.100671 s]
/proc/loadavg: 3.40 4.52 6.09 2/174 25851
/proc/meminfo: memFree=25308724/32951124 swapFree=67051468/67111528
[pid=25851] ppid=25849 vsize=8840 CPUtime=0.09 cores=1,3,5,7
/proc/25851/stat : 25851 (ttsth-5-4) R 25849 25851 25096 0 -1 4202496 301 0 0 0 9 0 0 0 23 0 1 0 521837827 9052160 244 33554432000 4194304 4338100 140735376996528 18446744073709551615 4296000 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/25851/statm: 2210 244 152 36 0 115 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 8840

[startup+0.300635 s]
/proc/loadavg: 3.40 4.52 6.09 2/174 25851
/proc/meminfo: memFree=25308724/32951124 swapFree=67051468/67111528
[pid=25851] ppid=25849 vsize=8840 CPUtime=0.29 cores=1,3,5,7
/proc/25851/stat : 25851 (ttsth-5-4) R 25849 25851 25096 0 -1 4202496 301 0 0 0 29 0 0 0 25 0 1 0 521837827 9052160 244 33554432000 4194304 4338100 140735376996528 18446744073709551615 4295699 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/25851/statm: 2210 244 152 36 0 115 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 8840

[startup+0.792491 s]
/proc/loadavg: 3.40 4.52 6.09 2/174 25851
/proc/meminfo: memFree=25308724/32951124 swapFree=67051468/67111528
[pid=25851] ppid=25849 vsize=230556 CPUtime=2.44 cores=1,3,5,7
/proc/25851/stat : 25851 (ttsth-5-4) S 25849 25851 25096 0 -1 4202496 11268 0 0 0 241 3 0 0 25 0 5 0 521837827 236089344 11210 33554432000 4194304 4338100 140735376996528 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25851/statm: 57639 11211 173 36 0 55544 0
Current children cumulated CPU time (s) 2.44
Current children cumulated vsize (KiB) 230556

[startup+1.50039 s]
/proc/loadavg: 3.40 4.52 6.09 5/179 25856
/proc/meminfo: memFree=25223380/32951124 swapFree=67051468/67111528
[pid=25851] ppid=25849 vsize=260384 CPUtime=4.91 cores=1,3,5,7
/proc/25851/stat : 25851 (ttsth-5-4) S 25849 25851 25096 0 -1 4202496 18935 0 0 0 486 5 0 0 25 0 5 0 521837827 266633216 18869 33554432000 4194304 4338100 140735376996528 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25851/statm: 65096 18870 175 36 0 63001 0
[pid=25851/tid=25853] ppid=25849 vsize=260384 CPUtime=1.13 cores=1,3,5,7
/proc/25851/task/25853/stat : 25853 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 4584 0 0 0 112 1 0 0 18 0 5 0 521837862 266633216 18870 33554432000 4194304 4338100 140735376996528 18446744073709551615 4254342 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25851/tid=25854] ppid=25849 vsize=260384 CPUtime=1.14 cores=1,3,5,7
/proc/25851/task/25854/stat : 25854 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 4544 0 0 0 113 1 0 0 18 0 5 0 521837862 266633216 18870 33554432000 4194304 4338100 140735376996528 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=25851/tid=25855] ppid=25849 vsize=260384 CPUtime=1.14 cores=1,3,5,7
/proc/25851/task/25855/stat : 25855 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 4615 0 0 0 113 1 0 0 25 0 5 0 521837862 266633216 18870 33554432000 4194304 4338100 140735376996528 18446744073709551615 4256770 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25851/tid=25856] ppid=25849 vsize=260384 CPUtime=1.14 cores=1,3,5,7
/proc/25851/task/25856/stat : 25856 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 4484 0 0 0 113 1 0 0 18 0 5 0 521837862 266633216 18870 33554432000 4194304 4338100 140735376996528 18446744073709551615 4253966 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4.91
Current children cumulated vsize (KiB) 260384

[startup+3.1011 s]
/proc/loadavg: 3.53 4.52 6.08 6/182 25862
/proc/meminfo: memFree=25010072/32951124 swapFree=67051468/67111528
[pid=25851] ppid=25849 vsize=338336 CPUtime=11.24 cores=1,3,5,7
/proc/25851/stat : 25851 (ttsth-5-4) S 25849 25851 25096 0 -1 4202496 39673 0 0 0 1113 11 0 0 25 0 5 0 521837827 346456064 39607 33554432000 4194304 4338100 140735376996528 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25851/statm: 84584 39607 175 36 0 82489 0
[pid=25851/tid=25853] ppid=25849 vsize=338336 CPUtime=2.71 cores=1,3,5,7
/proc/25851/task/25853/stat : 25853 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 11172 0 0 0 268 3 0 0 19 0 5 0 521837862 346456064 39607 33554432000 4194304 4338100 140735376996528 18446744073709551615 4254208 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25851/tid=25854] ppid=25849 vsize=338336 CPUtime=2.72 cores=1,3,5,7
/proc/25851/task/25854/stat : 25854 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 9067 0 0 0 270 2 0 0 20 0 5 0 521837862 346456064 39607 33554432000 4194304 4338100 140735376996528 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=25851/tid=25855] ppid=25849 vsize=338336 CPUtime=2.72 cores=1,3,5,7
/proc/25851/task/25855/stat : 25855 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 9131 0 0 0 269 3 0 0 25 0 5 0 521837862 346456064 39607 33554432000 4194304 4338100 140735376996528 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25851/tid=25856] ppid=25849 vsize=338336 CPUtime=2.72 cores=1,3,5,7
/proc/25851/task/25856/stat : 25856 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 9594 0 0 0 270 2 0 0 18 0 5 0 521837862 346456064 39607 33554432000 4194304 4338100 140735376996528 18446744073709551615 4240537 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 11.24
Current children cumulated vsize (KiB) 338336

[startup+6.30053 s]
/proc/loadavg: 3.53 4.52 6.08 6/182 25862
/proc/meminfo: memFree=24708756/32951124 swapFree=67051468/67111528
[pid=25851] ppid=25849 vsize=412520 CPUtime=23.82 cores=1,3,5,7
/proc/25851/stat : 25851 (ttsth-5-4) S 25849 25851 25096 0 -1 4202496 58134 0 0 0 2364 18 0 0 25 0 5 0 521837827 422420480 58068 33554432000 4194304 4338100 140735376996528 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25851/statm: 103130 58068 175 36 0 101035 0
[pid=25851/tid=25853] ppid=25849 vsize=412520 CPUtime=5.83 cores=1,3,5,7
/proc/25851/task/25853/stat : 25853 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 16241 0 0 0 579 4 0 0 25 0 5 0 521837862 422420480 58068 33554432000 4194304 4338100 140735376996528 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25851/tid=25854] ppid=25849 vsize=412520 CPUtime=5.89 cores=1,3,5,7
/proc/25851/task/25854/stat : 25854 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 13169 0 0 0 585 4 0 0 23 0 5 0 521837862 422420480 58068 33554432000 4194304 4338100 140735376996528 18446744073709551615 4256352 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=25851/tid=25855] ppid=25849 vsize=412520 CPUtime=5.85 cores=1,3,5,7
/proc/25851/task/25855/stat : 25855 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 13718 0 0 0 580 5 0 0 25 0 5 0 521837862 422420480 58068 33554432000 4194304 4338100 140735376996528 18446744073709551615 4254337 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25851/tid=25856] ppid=25849 vsize=412520 CPUtime=5.88 cores=1,3,5,7
/proc/25851/task/25856/stat : 25856 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 14297 0 0 0 584 4 0 0 25 0 5 0 521837862 422420480 58068 33554432000 4194304 4338100 140735376996528 18446744073709551615 4254304 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 23.82
Current children cumulated vsize (KiB) 412520

[startup+12.7004 s]
/proc/loadavg: 3.64 4.53 6.07 6/182 25862
/proc/meminfo: memFree=24047280/32951124 swapFree=67051468/67111528
[pid=25851] ppid=25849 vsize=503380 CPUtime=48.66 cores=1,3,5,7
/proc/25851/stat : 25851 (ttsth-5-4) S 25849 25851 25096 0 -1 4202496 84357 0 0 0 4831 35 0 0 25 0 5 0 521837827 515461120 84291 33554432000 4194304 4338100 140735376996528 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25851/statm: 125845 84291 175 36 0 123750 0
[pid=25851/tid=25853] ppid=25849 vsize=503380 CPUtime=11.96 cores=1,3,5,7
/proc/25851/task/25853/stat : 25853 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 29133 0 0 0 1186 10 0 0 25 0 5 0 521837862 515461120 84291 33554432000 4194304 4338100 140735376996528 18446744073709551615 218382769926 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25851/tid=25854] ppid=25849 vsize=503380 CPUtime=12.11 cores=1,3,5,7
/proc/25851/task/25854/stat : 25854 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 18115 0 0 0 1203 8 0 0 25 0 5 0 521837862 515461120 84291 33554432000 4194304 4338100 140735376996528 18446744073709551615 4254437 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=25851/tid=25855] ppid=25849 vsize=503380 CPUtime=12.08 cores=1,3,5,7
/proc/25851/task/25855/stat : 25855 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 19115 0 0 0 1199 9 0 0 25 0 5 0 521837862 515461120 84291 33554432000 4194304 4338100 140735376996528 18446744073709551615 4254407 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25851/tid=25856] ppid=25849 vsize=503380 CPUtime=12.14 cores=1,3,5,7
/proc/25851/task/25856/stat : 25856 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 17285 0 0 0 1207 7 0 0 21 0 5 0 521837862 515461120 84291 33554432000 4194304 4338100 140735376996528 18446744073709551615 218382771439 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 48.66
Current children cumulated vsize (KiB) 503380

[startup+25.5071 s]
/proc/loadavg: 4.29 4.62 6.08 9/186 25866
/proc/meminfo: memFree=23489384/32951124 swapFree=67051468/67111528
[pid=25851] ppid=25849 vsize=650004 CPUtime=97.68 cores=1,3,5,7
/proc/25851/stat : 25851 (ttsth-5-4) S 25849 25851 25096 0 -1 4202496 124701 0 0 0 9713 55 0 0 25 0 5 0 521837827 665604096 124635 33554432000 4194304 4338100 140735376996528 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25851/statm: 162501 124635 175 36 0 160406 0
[pid=25851/tid=25853] ppid=25849 vsize=650004 CPUtime=23.99 cores=1,3,5,7
/proc/25851/task/25853/stat : 25853 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 49503 0 0 0 2382 17 0 0 18 0 5 0 521837862 665604096 124635 33554432000 4194304 4338100 140735376996528 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25851/tid=25854] ppid=25849 vsize=650004 CPUtime=24.36 cores=1,3,5,7
/proc/25851/task/25854/stat : 25854 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 26316 0 0 0 2423 13 0 0 25 0 5 0 521837862 665604096 124635 33554432000 4194304 4338100 140735376996528 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25851/tid=25855] ppid=25849 vsize=650004 CPUtime=24.42 cores=1,3,5,7
/proc/25851/task/25855/stat : 25855 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 25289 0 0 0 2429 13 0 0 25 0 5 0 521837862 665604096 124635 33554432000 4194304 4338100 140735376996528 18446744073709551615 218370352149 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=25851/tid=25856] ppid=25849 vsize=650004 CPUtime=24.54 cores=1,3,5,7
/proc/25851/task/25856/stat : 25856 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 22884 0 0 0 2443 11 0 0 25 0 5 0 521837862 665604096 124635 33554432000 4194304 4338100 140735376996528 18446744073709551615 4240537 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 97.68
Current children cumulated vsize (KiB) 650004

[startup+51.1006 s]
/proc/loadavg: 5.55 4.89 6.13 9/186 25867
/proc/meminfo: memFree=23318796/32951124 swapFree=67051468/67111528
[pid=25851] ppid=25849 vsize=840828 CPUtime=195.47 cores=1,3,5,7
/proc/25851/stat : 25851 (ttsth-5-4) S 25849 25851 25096 0 -1 4202496 164623 0 0 0 19459 88 0 0 25 0 5 0 521837827 861007872 164557 33554432000 4194304 4338100 140735376996528 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25851/statm: 210207 164557 175 36 0 208112 0
[pid=25851/tid=25853] ppid=25849 vsize=840828 CPUtime=47.87 cores=1,3,5,7
/proc/25851/task/25853/stat : 25853 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 74841 0 0 0 4756 31 0 0 25 0 5 0 521837862 861007872 164557 33554432000 4194304 4338100 140735376996528 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25851/tid=25854] ppid=25849 vsize=840828 CPUtime=48.97 cores=1,3,5,7
/proc/25851/task/25854/stat : 25854 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 31439 0 0 0 4878 19 0 0 25 0 5 0 521837862 861007872 164557 33554432000 4194304 4338100 140735376996528 18446744073709551615 4242651 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25851/tid=25855] ppid=25849 vsize=840828 CPUtime=49.01 cores=1,3,5,7
/proc/25851/task/25855/stat : 25855 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 29717 0 0 0 4882 19 0 0 20 0 5 0 521837862 861007872 164557 33554432000 4194304 4338100 140735376996528 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=25851/tid=25856] ppid=25849 vsize=840828 CPUtime=49.24 cores=1,3,5,7
/proc/25851/task/25856/stat : 25856 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 27917 0 0 0 4907 17 0 0 22 0 5 0 521837862 861007872 164557 33554432000 4194304 4338100 140735376996528 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 195.47
Current children cumulated vsize (KiB) 840828

[startup+102.3 s]
/proc/loadavg: 6.90 5.35 6.22 9/186 25868
/proc/meminfo: memFree=22896252/32951124 swapFree=67051468/67111528
[pid=25851] ppid=25849 vsize=1294500 CPUtime=388.32 cores=1,3,5,7
/proc/25851/stat : 25851 (ttsth-5-4) S 25849 25851 25096 0 -1 4202496 265275 0 0 0 38679 153 0 0 25 0 5 0 521837827 1325568000 265209 33554432000 4194304 4338100 140735376996528 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25851/statm: 323625 265209 175 36 0 321530 0
[pid=25851/tid=25853] ppid=25849 vsize=1294500 CPUtime=94.55 cores=1,3,5,7
/proc/25851/task/25853/stat : 25853 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 142200 0 0 0 9395 60 0 0 18 0 5 0 521837862 1325568000 265209 33554432000 4194304 4338100 140735376996528 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25851/tid=25854] ppid=25849 vsize=1294500 CPUtime=97.54 cores=1,3,5,7
/proc/25851/task/25854/stat : 25854 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 42952 0 0 0 9724 30 0 0 25 0 5 0 521837862 1325568000 265209 33554432000 4194304 4338100 140735376996528 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25851/tid=25855] ppid=25849 vsize=1294500 CPUtime=97.75 cores=1,3,5,7
/proc/25851/task/25855/stat : 25855 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 38925 0 0 0 9742 33 0 0 19 0 5 0 521837862 1325568000 265209 33554432000 4194304 4338100 140735376996528 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=25851/tid=25856] ppid=25849 vsize=1294500 CPUtime=98.12 cores=1,3,5,7
/proc/25851/task/25856/stat : 25856 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 40489 0 0 0 9783 29 0 0 25 0 5 0 521837862 1325568000 265209 33554432000 4194304 4338100 140735376996528 18446744073709551615 4256310 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 388.32
Current children cumulated vsize (KiB) 1294500

[startup+162.301 s]
/proc/loadavg: 7.44 5.80 6.32 9/186 25870
/proc/meminfo: memFree=22425820/32951124 swapFree=67051468/67111528
[pid=25851] ppid=25849 vsize=1683556 CPUtime=612.28 cores=1,3,5,7
/proc/25851/stat : 25851 (ttsth-5-4) S 25849 25851 25096 0 -1 4202496 372041 0 0 0 60998 230 0 0 25 0 5 0 521837827 1723961344 371975 33554432000 4194304 4338100 140735376996528 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25851/statm: 420889 371975 175 36 0 418794 0
[pid=25851/tid=25853] ppid=25849 vsize=1683556 CPUtime=148.76 cores=1,3,5,7
/proc/25851/task/25853/stat : 25853 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 213324 0 0 0 14788 88 0 0 25 0 5 0 521837862 1723961344 371975 33554432000 4194304 4338100 140735376996528 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25851/tid=25854] ppid=25849 vsize=1683556 CPUtime=154.2 cores=1,3,5,7
/proc/25851/task/25854/stat : 25854 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 54194 0 0 0 15375 45 0 0 21 0 5 0 521837862 1723961344 371975 33554432000 4194304 4338100 140735376996528 18446744073709551615 4254352 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=25851/tid=25855] ppid=25849 vsize=1683556 CPUtime=154.29 cores=1,3,5,7
/proc/25851/task/25855/stat : 25855 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 52194 0 0 0 15379 50 0 0 18 0 5 0 521837862 1723961344 371975 33554432000 4194304 4338100 140735376996528 18446744073709551615 4234377 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25851/tid=25856] ppid=25849 vsize=1683556 CPUtime=154.66 cores=1,3,5,7
/proc/25851/task/25856/stat : 25856 (ttsth-5-4) S 25849 25851 25096 0 -1 4202560 51620 0 0 0 15421 45 0 0 25 0 5 0 521837862 1723961344 371975 33554432000 4194304 4338100 140735376996528 18446744073709551615 218382769926 0 0 4096 0 18446604471136816192 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 612.28
Current children cumulated vsize (KiB) 1683556


################
# More data... #
################

/proc/25851/task/25856/stat : 25856 (ttsth-5-4) S 25849 25851 25096 0 -1 4202560 143993 0 0 0 71737 181 0 0 25 0 5 0 521837862 5013561344 1113237 33554432000 4194304 4338100 140735376996528 18446744073709551615 218382769926 0 0 4096 0 18446604471136816192 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2851.61
Current children cumulated vsize (KiB) 4896056

[startup+822.301 s]
/proc/loadavg: 7.96 7.76 7.16 9/186 25887
/proc/meminfo: memFree=18895596/32951124 swapFree=67051468/67111528
[pid=25851] ppid=25849 vsize=5100208 CPUtime=3076.81 cores=1,3,5,7
/proc/25851/stat : 25851 (ttsth-5-4) S 25849 25851 25096 0 -1 4202496 1170809 0 0 0 306766 915 0 0 25 0 5 0 521837827 5222612992 1170743 33554432000 4194304 4338100 140735376996528 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25851/statm: 1275052 1170743 175 36 0 1272957 0
[pid=25851/tid=25853] ppid=25849 vsize=5100208 CPUtime=746.52 cores=1,3,5,7
/proc/25851/task/25853/stat : 25853 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 699956 0 0 0 74291 361 0 0 25 0 5 0 521837862 5222612992 1170743 33554432000 4194304 4338100 140735376996528 18446744073709551615 218370352358 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=25851/tid=25854] ppid=25849 vsize=5100208 CPUtime=776.85 cores=1,3,5,7
/proc/25851/task/25854/stat : 25854 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 159202 0 0 0 77512 173 0 0 23 0 5 0 521837862 5222612992 1170743 33554432000 4194304 4338100 140735376996528 18446744073709551615 4254362 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25851/tid=25855] ppid=25849 vsize=5100208 CPUtime=777.18 cores=1,3,5,7
/proc/25851/task/25855/stat : 25855 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 158584 0 0 0 77532 186 0 0 25 0 5 0 521837862 5222612992 1170743 33554432000 4194304 4338100 140735376996528 18446744073709551615 4289435 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25851/tid=25856] ppid=25849 vsize=5100208 CPUtime=775.9 cores=1,3,5,7
/proc/25851/task/25856/stat : 25856 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 152358 0 0 0 77396 194 0 0 18 0 5 0 521837862 5222612992 1170743 33554432000 4194304 4338100 140735376996528 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3076.81
Current children cumulated vsize (KiB) 5100208

[startup+882.3 s]
/proc/loadavg: 7.86 7.77 7.20 9/186 25889
/proc/meminfo: memFree=18641964/32951124 swapFree=67051468/67111528
[pid=25851] ppid=25849 vsize=5529828 CPUtime=3302.16 cores=1,3,5,7
/proc/25851/stat : 25851 (ttsth-5-4) S 25849 25851 25096 0 -1 4202496 1224780 0 0 0 329242 974 0 0 25 0 5 0 521837827 5662543872 1224714 33554432000 4194304 4338100 140735376996528 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25851/statm: 1382457 1224714 175 36 0 1380362 0
[pid=25851/tid=25853] ppid=25849 vsize=5529828 CPUtime=801.08 cores=1,3,5,7
/proc/25851/task/25853/stat : 25853 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 732335 0 0 0 79725 383 0 0 25 0 5 0 521837862 5662543872 1224714 33554432000 4194304 4338100 140735376996528 18446744073709551615 218370352237 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25851/tid=25854] ppid=25849 vsize=5529828 CPUtime=833.72 cores=1,3,5,7
/proc/25851/task/25854/stat : 25854 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 167150 0 0 0 83191 181 0 0 25 0 5 0 521837862 5662543872 1224714 33554432000 4194304 4338100 140735376996528 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=25851/tid=25855] ppid=25849 vsize=5529828 CPUtime=833.8 cores=1,3,5,7
/proc/25851/task/25855/stat : 25855 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 164581 0 0 0 83182 198 0 0 25 0 5 0 521837862 5662543872 1224714 33554432000 4194304 4338100 140735376996528 18446744073709551615 4254326 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25851/tid=25856] ppid=25849 vsize=5529828 CPUtime=833.19 cores=1,3,5,7
/proc/25851/task/25856/stat : 25856 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 160005 0 0 0 83109 210 0 0 19 0 5 0 521837862 5662543872 1224714 33554432000 4194304 4338100 140735376996528 18446744073709551615 218370352514 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3302.16
Current children cumulated vsize (KiB) 5529828

[startup+942.301 s]
/proc/loadavg: 7.94 7.81 7.25 8/186 25891
/proc/meminfo: memFree=18347536/32951124 swapFree=67051468/67111528
[pid=25851] ppid=25849 vsize=5701244 CPUtime=3524.81 cores=1,3,5,7
/proc/25851/stat : 25851 (ttsth-5-4) S 25849 25851 25096 0 -1 4202496 1292046 0 0 0 351453 1028 0 0 25 0 5 0 521837827 5838073856 1291980 33554432000 4194304 4338100 140735376996528 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25851/statm: 1425311 1291980 175 36 0 1423216 0
[pid=25851/tid=25853] ppid=25849 vsize=5701244 CPUtime=854.82 cores=1,3,5,7
/proc/25851/task/25853/stat : 25853 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 775834 0 0 0 85075 407 0 0 22 0 5 0 521837862 5838073856 1291980 33554432000 4194304 4338100 140735376996528 18446744073709551615 4235651 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25851/tid=25854] ppid=25849 vsize=5701244 CPUtime=889.88 cores=1,3,5,7
/proc/25851/task/25854/stat : 25854 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 176407 0 0 0 88799 189 0 0 25 0 5 0 521837862 5838073856 1291980 33554432000 4194304 4338100 140735376996528 18446744073709551615 4235746 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25851/tid=25855] ppid=25849 vsize=5701244 CPUtime=890.22 cores=1,3,5,7
/proc/25851/task/25855/stat : 25855 (ttsth-5-4) S 25849 25851 25096 0 -1 4202560 172002 0 0 0 88813 209 0 0 19 0 5 0 521837862 5838073856 1291980 33554432000 4194304 4338100 140735376996528 18446744073709551615 218382769926 0 0 4096 0 18446604471136816192 0 0 -1 3 0 0 0
[pid=25851/tid=25856] ppid=25849 vsize=5701244 CPUtime=889.51 cores=1,3,5,7
/proc/25851/task/25856/stat : 25856 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 167094 0 0 0 88730 221 0 0 20 0 5 0 521837862 5838073856 1291980 33554432000 4194304 4338100 140735376996528 18446744073709551615 4235746 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3524.81
Current children cumulated vsize (KiB) 5701244

[startup+1002.31 s]
/proc/loadavg: 7.98 7.84 7.29 9/186 25892
/proc/meminfo: memFree=18094188/32951124 swapFree=67051468/67111528
[pid=25851] ppid=25849 vsize=5929032 CPUtime=3750.81 cores=1,3,5,7
/proc/25851/stat : 25851 (ttsth-5-4) S 25849 25851 25096 0 -1 4202496 1346098 0 0 0 374002 1079 0 0 25 0 5 0 521837827 6071328768 1346032 33554432000 4194304 4338100 140735376996528 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25851/statm: 1482258 1346032 175 36 0 1480163 0
[pid=25851/tid=25853] ppid=25849 vsize=5929032 CPUtime=909.88 cores=1,3,5,7
/proc/25851/task/25853/stat : 25853 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 800140 0 0 0 90562 426 0 0 18 0 5 0 521837862 6071328768 1346032 33554432000 4194304 4338100 140735376996528 18446744073709551615 4235746 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25851/tid=25854] ppid=25849 vsize=5929032 CPUtime=946.56 cores=1,3,5,7
/proc/25851/task/25854/stat : 25854 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 186093 0 0 0 94459 197 0 0 20 0 5 0 521837862 6071328768 1346032 33554432000 4194304 4338100 140735376996528 18446744073709551615 4235651 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25851/tid=25855] ppid=25849 vsize=5929032 CPUtime=947.17 cores=1,3,5,7
/proc/25851/task/25855/stat : 25855 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 179336 0 0 0 94496 221 0 0 25 0 5 0 521837862 6071328768 1346032 33554432000 4194304 4338100 140735376996528 18446744073709551615 4235651 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=25851/tid=25856] ppid=25849 vsize=5929032 CPUtime=946.82 cores=1,3,5,7
/proc/25851/task/25856/stat : 25856 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 179820 0 0 0 94449 233 0 0 25 0 5 0 521837862 6071328768 1346032 33554432000 4194304 4338100 140735376996528 18446744073709551615 4235847 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3750.81
Current children cumulated vsize (KiB) 5929032

[startup+1062.3 s]
/proc/loadavg: 7.86 7.84 7.32 9/186 25894
/proc/meminfo: memFree=17825444/32951124 swapFree=67051468/67111528
[pid=25851] ppid=25849 vsize=6152000 CPUtime=3976.54 cores=1,3,5,7
/proc/25851/stat : 25851 (ttsth-5-4) S 25849 25851 25096 0 -1 4202496 1406046 0 0 0 396524 1130 0 0 25 0 5 0 521837827 6299648000 1405980 33554432000 4194304 4338100 140735376996528 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25851/statm: 1538000 1405981 175 36 0 1535905 0
[pid=25851/tid=25853] ppid=25849 vsize=6152000 CPUtime=964.83 cores=1,3,5,7
/proc/25851/task/25853/stat : 25853 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 839352 0 0 0 96035 448 0 0 25 0 5 0 521837862 6299648000 1405981 33554432000 4194304 4338100 140735376996528 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=25851/tid=25854] ppid=25849 vsize=6152000 CPUtime=1003.55 cores=1,3,5,7
/proc/25851/task/25854/stat : 25854 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 192187 0 0 0 100149 206 0 0 19 0 5 0 521837862 6299648000 1405981 33554432000 4194304 4338100 140735376996528 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25851/tid=25855] ppid=25849 vsize=6152000 CPUtime=1004.17 cores=1,3,5,7
/proc/25851/task/25855/stat : 25855 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 186724 0 0 0 100187 230 0 0 25 0 5 0 521837862 6299648000 1405981 33554432000 4194304 4338100 140735376996528 18446744073709551615 4254239 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=25851/tid=25856] ppid=25849 vsize=6152000 CPUtime=1003.6 cores=1,3,5,7
/proc/25851/task/25856/stat : 25856 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 187075 0 0 0 100116 244 0 0 19 0 5 0 521837862 6299648000 1405981 33554432000 4194304 4338100 140735376996528 18446744073709551615 218370653715 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3976.54
Current children cumulated vsize (KiB) 6152000

[startup+1122.3 s]
/proc/loadavg: 7.94 7.86 7.37 9/186 25896
/proc/meminfo: memFree=17522996/32951124 swapFree=67051468/67111528
[pid=25851] ppid=25849 vsize=6327428 CPUtime=4199.33 cores=1,3,5,7
/proc/25851/stat : 25851 (ttsth-5-4) S 25849 25851 25096 0 -1 4202496 1472995 0 0 0 418752 1181 0 0 25 0 5 0 521837827 6479286272 1472929 33554432000 4194304 4338100 140735376996528 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25851/statm: 1581857 1472929 175 36 0 1579762 0
[pid=25851/tid=25853] ppid=25849 vsize=6327428 CPUtime=1018.69 cores=1,3,5,7
/proc/25851/task/25853/stat : 25853 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 882788 0 0 0 101399 470 0 0 25 0 5 0 521837862 6479286272 1472929 33554432000 4194304 4338100 140735376996528 18446744073709551615 4254226 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=25851/tid=25854] ppid=25849 vsize=6327428 CPUtime=1059.66 cores=1,3,5,7
/proc/25851/task/25854/stat : 25854 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 197381 0 0 0 105749 217 0 0 23 0 5 0 521837862 6479286272 1472929 33554432000 4194304 4338100 140735376996528 18446744073709551615 218370618401 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25851/tid=25855] ppid=25849 vsize=6327428 CPUtime=1060.91 cores=1,3,5,7
/proc/25851/task/25855/stat : 25855 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 194558 0 0 0 105853 238 0 0 25 0 5 0 521837862 6479286272 1472929 33554432000 4194304 4338100 140735376996528 18446744073709551615 4254329 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25851/tid=25856] ppid=25849 vsize=6327428 CPUtime=1059.71 cores=1,3,5,7
/proc/25851/task/25856/stat : 25856 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 197559 0 0 0 105716 255 0 0 25 0 5 0 521837862 6479286272 1472929 33554432000 4194304 4338100 140735376996528 18446744073709551615 218370352609 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4199.33
Current children cumulated vsize (KiB) 6327428

[startup+1182.3 s]
/proc/loadavg: 8.02 7.90 7.40 9/186 25897
/proc/meminfo: memFree=17226980/32951124 swapFree=67051468/67111528
[pid=25851] ppid=25849 vsize=6603532 CPUtime=4423.56 cores=1,3,5,7
/proc/25851/stat : 25851 (ttsth-5-4) S 25849 25851 25096 0 -1 4202496 1539042 0 0 0 441126 1230 0 0 25 0 5 0 521837827 6762016768 1538976 33554432000 4194304 4338100 140735376996528 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25851/statm: 1650883 1538976 175 36 0 1648788 0
[pid=25851/tid=25853] ppid=25849 vsize=6603532 CPUtime=1073.09 cores=1,3,5,7
/proc/25851/task/25853/stat : 25853 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 918708 0 0 0 106819 490 0 0 25 0 5 0 521837862 6762016768 1538976 33554432000 4194304 4338100 140735376996528 18446744073709551615 4241252 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=25851/tid=25854] ppid=25849 vsize=6603532 CPUtime=1116.35 cores=1,3,5,7
/proc/25851/task/25854/stat : 25854 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 206304 0 0 0 111408 227 0 0 22 0 5 0 521837862 6762016768 1538976 33554432000 4194304 4338100 140735376996528 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=25851/tid=25855] ppid=25849 vsize=6603532 CPUtime=1117.37 cores=1,3,5,7
/proc/25851/task/25855/stat : 25855 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 206440 0 0 0 111491 246 0 0 18 0 5 0 521837862 6762016768 1538976 33554432000 4194304 4338100 140735376996528 18446744073709551615 218370352353 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25851/tid=25856] ppid=25849 vsize=6603532 CPUtime=1116.37 cores=1,3,5,7
/proc/25851/task/25856/stat : 25856 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 206881 0 0 0 111372 265 0 0 25 0 5 0 521837862 6762016768 1538976 33554432000 4194304 4338100 140735376996528 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4423.56
Current children cumulated vsize (KiB) 6603532

[startup+1242.3 s]
/proc/loadavg: 6.01 7.39 7.26 5/176 25916
/proc/meminfo: memFree=18923256/32951124 swapFree=67051468/67111528
[pid=25851] ppid=25849 vsize=6805192 CPUtime=4650.54 cores=1,3,5,7
/proc/25851/stat : 25851 (ttsth-5-4) S 25849 25851 25096 0 -1 4202496 1593610 0 0 0 463767 1287 0 0 25 0 5 0 521837827 6968516608 1593544 33554432000 4194304 4338100 140735376996528 18446744073709551615 218382759909 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25851/statm: 1701298 1593544 175 36 0 1699203 0
[pid=25851/tid=25853] ppid=25849 vsize=6805192 CPUtime=1128.56 cores=1,3,5,7
/proc/25851/task/25853/stat : 25853 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 951157 0 0 0 112341 515 0 0 25 0 5 0 521837862 6968516608 1593544 33554432000 4194304 4338100 140735376996528 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=25851/tid=25854] ppid=25849 vsize=6805192 CPUtime=1173.79 cores=1,3,5,7
/proc/25851/task/25854/stat : 25854 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 213894 0 0 0 117141 238 0 0 25 0 5 0 521837862 6968516608 1593544 33554432000 4194304 4338100 140735376996528 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=25851/tid=25855] ppid=25849 vsize=6805192 CPUtime=1174.14 cores=1,3,5,7
/proc/25851/task/25855/stat : 25855 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 212136 0 0 0 117160 254 0 0 25 0 5 0 521837862 6968516608 1593544 33554432000 4194304 4338100 140735376996528 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=25851/tid=25856] ppid=25849 vsize=6805192 CPUtime=1173.69 cores=1,3,5,7
/proc/25851/task/25856/stat : 25856 (ttsth-5-4) R 25849 25851 25096 0 -1 4202560 215714 0 0 0 117090 279 0 0 25 0 5 0 521837862 6968516608 1593544 33554432000 4194304 4338100 140735376996528 18446744073709551615 218370352871 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4650.54
Current children cumulated vsize (KiB) 6805192



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1282.1 s]
/proc/loadavg: 5.03 6.96 7.12 2/173 25917
/proc/meminfo: memFree=20065664/32951124 swapFree=67051468/67111528
[pid=25851] ppid=25849 vsize=0 CPUtime=4800.12 cores=1,3,5,7
/proc/25851/stat : 25851 (ttsth-5-4) Z 25849 25851 25096 0 -1 4203548 1636125 0 0 0 478684 1328 0 0 18 0 2 0 521837827 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25851/statm: 0 0 0 0 0 0 0
[pid=25851/tid=25856] ppid=25849 vsize=0 CPUtime=1211.54 cores=1,3,5,7
/proc/25851/task/25856/stat : 25856 (ttsth-5-4) R 25849 25851 25096 0 -1 4203588 221957 0 0 0 120861 293 0 0 21 0 2 0 521837862 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.12
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1282.21 s]
/proc/loadavg: 5.03 6.96 7.12 2/173 25917
/proc/meminfo: memFree=20065664/32951124 swapFree=67051468/67111528
[pid=25851] ppid=25849 vsize=0 CPUtime=4800.22 cores=1,3,5,7
/proc/25851/stat : 25851 (ttsth-5-4) Z 25849 25851 25096 0 -1 4203548 1636125 0 0 0 478684 1338 0 0 18 0 2 0 521837827 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/25851/statm: 0 0 0 0 0 0 0
[pid=25851/tid=25856] ppid=25849 vsize=0 CPUtime=1211.64 cores=1,3,5,7
/proc/25851/task/25856/stat : 25856 (ttsth-5-4) R 25849 25851 25096 0 -1 4203588 221957 0 0 0 120861 303 0 0 22 0 2 0 521837862 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.22
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 1282.29
CPU time (s): 4800.3
CPU user time (s): 4786.84
CPU system time (s): 13.461
CPU usage (%): 374.355
Max. virtual memory (cumulated for all children) (KiB): 6917788

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4786.84
system time used= 13.461
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1636125
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 366384
involuntary context switches= 32568

runsolver used 1.83772 second user time and 5.10822 second system time

The end

Launcher Data 

Begin job on node101 at 2011-04-29 20:39:11
IDJOB=3265566
IDBENCH=85780
IDSOLVER=1654
FILE ID=node101/3265566-1304102351
RUNJOBID= node101-1304102351-25838
PBS_JOBID= 13170603
Free space on /tmp= 72024 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT05/crafted/markstrom05/eulcbip/eulcbip-7-UNSAT.shuffled-as.sat05-3936.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3265566-1304102351/watcher-3265566-1304102351 -o /tmp/evaluation-result-3265566-1304102351/solver-3265566-1304102351 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3265566-1304102351.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 151ccdb2c52c7f525fde5dca388bb6e6
RANDOM SEED=1883544570

node101.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.841
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      25308988 kB
Buffers:       2028832 kB
Cached:        5134872 kB
SwapCached:      30452 kB
Active:        3085240 kB
Inactive:      4113588 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      25308988 kB
SwapTotal:    67111528 kB
SwapFree:     67051468 kB
Dirty:             104 kB
Writeback:           0 kB
AnonPages:       26360 kB
Mapped:           8624 kB
Slab:           380480 kB
PageTables:       4384 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   193720 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72024 MiB
End job on node101 at 2011-04-29 21:00:35