Trace number 3258790

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.28 1203.34

General information on the benchmark

NameCRAFTED/rbsat/random/forced/
rbsat-v2640c305320gyes10.cnf
MD5SUMe4e4b653016e98f210aa8da2214e783a
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables2640
Number of clauses305320
Sum of the clauses size613120
Maximum clause length33
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2305240
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 580

Solver Data 

0.19/0.21	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3258790-1304053030.cnf 
0.19/0.21	c Using SATTIMEOUT of 4800
0.19/0.21	c Using SATRAM of 15500
0.19/0.21	c Size of subset node = 32
0.19/0.21	c Variables = 2640
0.19/0.21	c Clauses = 305320
0.19/0.21	c Literals = 613120
0.19/0.21	c Reduced size:
0.19/0.21	c Variables = 2640
0.19/0.21	c Clauses = 305320
0.19/0.21	c Literals = 613120
0.19/0.21	c a2
0.29/0.39	c a1
0.79/0.81	c Independent sub-problems = 1
0.79/0.81	c Sub-problem with 2640 variables and 305320 clauses
0.79/0.81	c using permutation method 4
0.79/0.81	c count = 1
0.79/0.81	c Ordering finished 0.760 seconds
0.79/0.81	c mapped 0.800 seconds
0.79/0.81	c permuted 0.800 seconds
0.79/0.81	c about to sort
0.79/0.84	c sorted
1.59/1.61	c clauses now 305320
1.59/1.61	c init finished 0.900 seconds
1.59/1.61	c tree built 1.600 seconds
1.59/1.61	c 
1.59/1.61	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
1.59/1.61	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
1.59/1.61	c ................................ | ..........                    |                                  1.6         0.0             
1.59/1.61	c ................................ | ..........                  1 |                                  1.6         0.6             
1.59/1.61	c ................................ | ..........                  2 |                                  1.6         1.2             
1.59/1.61	c ................................ | ..........                  3 |                                  1.6         1.9             
1.59/1.61	c ................................ | ..........                  4 |                                  1.6         2.5             
1.59/1.61	c ................................ | ..........                  5 |                                  1.6         3.1             
1.59/1.61	c ................................ | ..........               1000 |              79                  1.6       621.1         1324
95.83/25.25	c ................................ | ..........            2834000 | 45746        194183             96.0     29511.6     15100639
191.69/49.28	c ................................ | ..........            5007000 | 85615        338863            192.0     26074.1     25953768
287.95/73.31	c ................................ | ..........            6919000 | 126597       448960            288.0     24024.3     34401888
383.82/97.35	c ................................ | ..........            8563000 | 160292       572208            384.0     22298.3     41518160
479.68/121.38	c ................................ | *.........           10090000 | 195076       675576            480.0     21020.0     48068800
575.94/145.47	c ................................ | *.........           11530000 | 226551       774341            576.2     20009.0     53061536
671.83/169.45	c ................................ | *.........           12812000 | 253600       857263            672.0     19064.9     56854226
767.64/193.49	c ................................ | *.........           14296000 | 281779       944249            768.0     18614.3     62097453
863.91/217.53	c ................................ | *.........           15611000 | 307143       1026227           864.0     18067.5     62097453
959.76/241.56	c ................................ | **........           16920000 | 338277       1090417           960.0     17625.0     64801444
1056.02/265.60	c ................................ | **........           18117000 | 368105       1165610          1056.0     17155.8     69007669
1151.88/289.65	c ................................ | **........           19268000 | 395205       1239263          1152.1     16724.4     72452438
1247.73/313.68	c ................................ | **........           20330000 | 419656       1306810          1248.1     16289.2     73144497
1343.61/337.70	c ................................ | **........           21387000 | 445060       1379581          1344.0     15912.9     73144497
1439.87/361.73	c ................................ | ***.......           22419000 | 470469       1437756          1440.0     15568.4     73938734
1535.72/385.78	c ................................ | ***.......           23541000 | 493137       1504083          1536.1     15325.6     77106962
1631.99/409.82	c ................................ | ***.......           24713000 | 515422       1574350          1632.1     15142.2     78844995
1728.04/433.91	c ................................ | ***.......           25797000 | 536887       1639718          1728.1     14928.0     82013666
1823.71/458.07	c ................................ | ***.......           26908000 | 560035       1707899          1824.0     14752.2     87487405
1919.97/482.12	c ................................ | ****......           28029000 | 582730       1775656          1920.0     14598.1     92537163
2015.84/506.14	c ................................ | ****......           29268000 | 608527       1838777          2016.0     14517.9     97609138
2111.71/530.19	c ................................ | ****......           30314000 | 631514       1905982          2112.1     14352.7     98965042
2207.94/554.22	c ................................ | ****......           31270000 | 655025       1973381          2208.0     14161.9     99350013
2303.81/578.26	c ................................ | ****......           32195000 | 676096       2035957          2304.1     13973.2     99350013
2399.68/602.29	c ................................ | *****.....           33101000 | 699530       2098630          2400.1     13791.7     99350013
2495.93/626.33	c ................................ | *****.....           34068000 | 722310       2152117          2496.1     13648.7    100938677
2591.78/650.36	c ................................ | *****.....           34953000 | 743663       2208255          2592.0     13484.8    100938677
2688.03/674.40	c ................................ | *****.....           35800000 | 763231       2267077          2688.0     13318.3    101132897
2783.91/698.42	c ................................ | *****.....           36702000 | 783544       2324363          2784.0     13183.2    101981013
2879.78/722.47	c ................................ | ******....           37678000 | 804778       2378447          2880.1     13082.3    105666416
2975.64/746.49	c ................................ | ******....           38602000 | 825606       2430854          2976.0     12971.1    108917771
3071.91/770.54	c ................................ | ******....           39452000 | 848642       2478051          3072.1     12842.1    111265447
3167.76/794.58	c ................................ | ******....           40268000 | 868057       2525362          3168.1     12710.5    112882916
3264.03/818.60	c ................................ | ******....           40983000 | 887018       2566063          3264.1     12555.8    115276377
3359.93/842.65	c ................................ | *******...           41785000 | 908024       2615103          3360.1     12435.6    118447425
3455.77/866.66	c ................................ | *******...           42597000 | 928534       2660310          3456.0     12325.4    120114878
3551.65/890.68	c ................................ | *******...           43365000 | 949595       2701130          3552.0     12208.6    120114878
3647.90/914.74	c ................................ | *******...           44117000 | 970498       2751566          3648.1     12093.3    122128799
3743.74/938.78	c ................................ | *******...           44833000 | 991041       2795781          3744.1     11974.5    123956874
3840.00/962.80	c ................................ | ********..           45528000 | 1008761      2840743          3840.0     11856.2    125709460
3935.87/986.86	c ................................ | ********..           46309000 | 1028692      2884903          3936.1     11765.2    127950535
4031.75/1010.88	c ................................ | ********..           47118000 | 1048261      2928110          4032.1     11685.9    130531097
4128.01/1034.91	c ................................ | ********..           47939000 | 1068605      2971024          4128.1     11612.9    131308346
4223.88/1058.96	c ................................ | ********..           48695000 | 1088242      3019636          4224.1     11527.8    133938402
4319.73/1082.97	c ................................ | *********.           49470000 | 1107272      3067466          4320.0     11451.3    136412203
4415.98/1107.04	c ................................ | *********.           50190000 | 1123967      3112766          4416.1     11365.1    139122287
4511.85/1131.05	c ................................ | *********.           51046000 | 1142560      3154809          4512.1     11313.3    141250202
4607.73/1155.09	c ................................ | *********.           51903000 | 1161020      3194018          4608.1     11263.4    142406410
4703.98/1179.12	c ................................ | *********.           52715000 | 1179341      3239512          4704.1     11206.2    143153215
4799.81/1203.17	c CPU limit (4800) reached
4799.81/1203.17	c Time = 4800.10 seconds
4799.81/1203.17	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3258790-1304053030/watcher-3258790-1304053030 -o /tmp/evaluation-result-3258790-1304053030/solver-3258790-1304053030 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3258790-1304053030.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.09 8.08 7.93 1/175 437
/proc/meminfo: memFree=27178672/32951124 swapFree=67088964/67111528
[pid=437] ppid=435 vsize=8980 CPUtime=0 cores=1,3,5,7
/proc/437/stat : 437 (ttsth-5-4) R 435 437 330 0 -1 4202496 299 0 0 0 0 0 0 0 23 0 1 0 516901493 9195520 242 33554432000 4194304 4338100 140735123845648 18446744073709551615 210204308165 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/437/statm: 2245 243 146 36 0 150 0

[startup+0.048579 s]
/proc/loadavg: 8.09 8.08 7.93 1/175 437
/proc/meminfo: memFree=27178672/32951124 swapFree=67088964/67111528
[pid=437] ppid=435 vsize=18140 CPUtime=0.04 cores=1,3,5,7
/proc/437/stat : 437 (ttsth-5-4) R 435 437 330 0 -1 4202496 2593 0 0 0 4 0 0 0 23 0 1 0 516901493 18575360 2536 33554432000 4194304 4338100 140735123845648 18446744073709551615 210204308165 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/437/statm: 4535 2537 148 36 0 2440 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 18140

[startup+0.100574 s]
/proc/loadavg: 8.09 8.08 7.93 1/175 437
/proc/meminfo: memFree=27178672/32951124 swapFree=67088964/67111528
[pid=437] ppid=435 vsize=28676 CPUtime=0.09 cores=1,3,5,7
/proc/437/stat : 437 (ttsth-5-4) R 435 437 330 0 -1 4202496 5166 0 0 0 8 1 0 0 24 0 1 0 516901493 29364224 5110 33554432000 4194304 4338100 140735123845648 18446744073709551615 210204308165 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/437/statm: 7169 5111 148 36 0 5074 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 28676

[startup+0.300541 s]
/proc/loadavg: 8.09 8.08 7.93 1/175 437
/proc/meminfo: memFree=27178672/32951124 swapFree=67088964/67111528
[pid=437] ppid=435 vsize=67988 CPUtime=0.29 cores=1,3,5,7
/proc/437/stat : 437 (ttsth-5-4) R 435 437 330 0 -1 4202496 15038 0 0 0 27 2 0 0 25 0 1 0 516901493 69619712 14980 33554432000 4194304 4338100 140735123845648 18446744073709551615 4206987 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/437/statm: 16997 14980 153 36 0 14902 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 67988

[startup+0.700487 s]
/proc/loadavg: 8.09 8.08 7.93 1/175 437
/proc/meminfo: memFree=27178672/32951124 swapFree=67088964/67111528
[pid=437] ppid=435 vsize=76788 CPUtime=0.69 cores=1,3,5,7
/proc/437/stat : 437 (ttsth-5-4) R 435 437 330 0 -1 4202496 18454 0 0 0 67 2 0 0 25 0 1 0 516901493 78630912 17202 33554432000 4194304 4338100 140735123845648 18446744073709551615 4230244 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/437/statm: 19197 17202 156 36 0 17102 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 76788

[startup+1.50035 s]
/proc/loadavg: 7.53 7.96 7.89 3/173 467
/proc/meminfo: memFree=27084804/32951124 swapFree=67088964/67111528
[pid=437] ppid=435 vsize=179144 CPUtime=1.49 cores=1,3,5,7
/proc/437/stat : 437 (ttsth-5-4) R 435 437 330 0 -1 4202496 44638 0 0 0 144 5 0 0 25 0 1 0 516901493 183443456 42789 33554432000 4194304 4338100 140735123845648 18446744073709551615 210204308165 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/437/statm: 44786 42795 164 36 0 42691 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 179144

[startup+3.10104 s]
/proc/loadavg: 7.53 7.96 7.89 6/186 484
/proc/meminfo: memFree=26830456/32951124 swapFree=67088964/67111528
[pid=437] ppid=435 vsize=429276 CPUtime=7.54 cores=1,3,5,7
/proc/437/stat : 437 (ttsth-5-4) S 435 437 330 0 -1 4202496 63454 0 0 0 733 21 0 0 25 0 5 0 516901493 439578624 61008 33554432000 4194304 4338100 140735123845648 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/437/statm: 107319 61008 176 36 0 105224 0
[pid=437/tid=476] ppid=435 vsize=429276 CPUtime=1.48 cores=1,3,5,7
/proc/437/task/476/stat : 476 (ttsth-5-4) R 435 437 330 0 -1 4202560 4992 0 0 0 147 1 0 0 20 0 5 0 516901654 439578624 61008 33554432000 4194304 4338100 140735123845648 18446744073709551615 4240537 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=437/tid=477] ppid=435 vsize=429276 CPUtime=1.48 cores=1,3,5,7
/proc/437/task/477/stat : 477 (ttsth-5-4) R 435 437 330 0 -1 4202560 2101 0 0 0 142 6 0 0 20 0 5 0 516901654 439578624 61008 33554432000 4194304 4338100 140735123845648 18446744073709551615 210204042646 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=437/tid=478] ppid=435 vsize=429276 CPUtime=1.47 cores=1,3,5,7
/proc/437/task/478/stat : 478 (ttsth-5-4) R 435 437 330 0 -1 4202560 5183 0 0 0 147 0 0 0 20 0 5 0 516901654 439578624 61008 33554432000 4194304 4338100 140735123845648 18446744073709551615 4239191 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=437/tid=479] ppid=435 vsize=429276 CPUtime=1.47 cores=1,3,5,7
/proc/437/task/479/stat : 479 (ttsth-5-4) R 435 437 330 0 -1 4202560 2000 0 0 0 141 6 0 0 20 0 5 0 516901654 439578624 61009 33554432000 4194304 4338100 140735123845648 18446744073709551615 210204042682 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 7.54
Current children cumulated vsize (KiB) 429276

[startup+6.3005 s]
/proc/loadavg: 7.56 7.96 7.89 9/186 484
/proc/meminfo: memFree=26629452/32951124 swapFree=67088964/67111528
[pid=437] ppid=435 vsize=511000 CPUtime=20.32 cores=1,3,5,7
/proc/437/stat : 437 (ttsth-5-4) S 435 437 330 0 -1 4202496 84167 0 0 0 1982 50 0 0 25 0 5 0 516901493 523264000 81721 33554432000 4194304 4338100 140735123845648 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/437/statm: 127750 81721 176 36 0 125655 0
[pid=437/tid=476] ppid=435 vsize=511000 CPUtime=4.67 cores=1,3,5,7
/proc/437/task/476/stat : 476 (ttsth-5-4) R 435 437 330 0 -1 4202560 12773 0 0 0 464 3 0 0 25 0 5 0 516901654 523264000 81721 33554432000 4194304 4338100 140735123845648 18446744073709551615 210204042616 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=437/tid=477] ppid=435 vsize=511000 CPUtime=4.68 cores=1,3,5,7
/proc/437/task/477/stat : 477 (ttsth-5-4) R 435 437 330 0 -1 4202560 5172 0 0 0 449 19 0 0 25 0 5 0 516901654 523264000 81721 33554432000 4194304 4338100 140735123845648 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=437/tid=478] ppid=435 vsize=511000 CPUtime=4.67 cores=1,3,5,7
/proc/437/task/478/stat : 478 (ttsth-5-4) R 435 437 330 0 -1 4202560 11954 0 0 0 466 1 0 0 25 0 5 0 516901654 523264000 81721 33554432000 4194304 4338100 140735123845648 18446744073709551615 210216461571 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=437/tid=479] ppid=435 vsize=511000 CPUtime=4.65 cores=1,3,5,7
/proc/437/task/479/stat : 479 (ttsth-5-4) R 435 437 330 0 -1 4202560 5090 0 0 0 447 18 0 0 25 0 5 0 516901654 523264000 81722 33554432000 4194304 4338100 140735123845648 18446744073709551615 4239450 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 20.32
Current children cumulated vsize (KiB) 511000

[startup+12.7004 s]
/proc/loadavg: 7.60 7.96 7.89 9/186 484
/proc/meminfo: memFree=26322716/32951124 swapFree=67088964/67111528
[pid=437] ppid=435 vsize=643016 CPUtime=45.88 cores=1,3,5,7
/proc/437/stat : 437 (ttsth-5-4) S 435 437 330 0 -1 4202496 117150 0 0 0 4489 99 0 0 25 0 5 0 516901493 658448384 114704 33554432000 4194304 4338100 140735123845648 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/437/statm: 160754 114704 176 36 0 158659 0
[pid=437/tid=476] ppid=435 vsize=643016 CPUtime=11.07 cores=1,3,5,7
/proc/437/task/476/stat : 476 (ttsth-5-4) R 435 437 330 0 -1 4202560 23245 0 0 0 1101 6 0 0 25 0 5 0 516901654 658448384 114704 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254342 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=437/tid=477] ppid=435 vsize=643016 CPUtime=11.08 cores=1,3,5,7
/proc/437/task/477/stat : 477 (ttsth-5-4) R 435 437 330 0 -1 4202560 10112 0 0 0 1067 41 0 0 25 0 5 0 516901654 658448384 114704 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=437/tid=478] ppid=435 vsize=643016 CPUtime=11.07 cores=1,3,5,7
/proc/437/task/478/stat : 478 (ttsth-5-4) R 435 437 330 0 -1 4202560 24448 0 0 0 1104 3 0 0 25 0 5 0 516901654 658448384 114704 33554432000 4194304 4338100 140735123845648 18446744073709551615 210204042363 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=437/tid=479] ppid=435 vsize=643016 CPUtime=11.02 cores=1,3,5,7
/proc/437/task/479/stat : 479 (ttsth-5-4) R 435 437 330 0 -1 4202560 10167 0 0 0 1061 41 0 0 25 0 5 0 516901654 658448384 114704 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254275 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 45.88
Current children cumulated vsize (KiB) 643016

[startup+25.5003 s]
/proc/loadavg: 7.75 7.98 7.90 9/186 484
/proc/meminfo: memFree=25862068/32951124 swapFree=67088964/67111528
[pid=437] ppid=435 vsize=853596 CPUtime=97.01 cores=1,3,5,7
/proc/437/stat : 437 (ttsth-5-4) S 435 437 330 0 -1 4202496 170095 0 0 0 9505 196 0 0 25 0 5 0 516901493 874082304 167649 33554432000 4194304 4338100 140735123845648 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/437/statm: 213399 167650 176 36 0 211304 0
[pid=437/tid=476] ppid=435 vsize=853596 CPUtime=23.88 cores=1,3,5,7
/proc/437/task/476/stat : 476 (ttsth-5-4) R 435 437 330 0 -1 4202560 41338 0 0 0 2377 11 0 0 25 0 5 0 516901654 874082304 167650 33554432000 4194304 4338100 140735123845648 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=437/tid=477] ppid=435 vsize=853596 CPUtime=23.88 cores=1,3,5,7
/proc/437/task/477/stat : 477 (ttsth-5-4) R 435 437 330 0 -1 4202560 18869 0 0 0 2303 85 0 0 25 0 5 0 516901654 874082304 167650 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=437/tid=478] ppid=435 vsize=853596 CPUtime=23.87 cores=1,3,5,7
/proc/437/task/478/stat : 478 (ttsth-5-4) R 435 437 330 0 -1 4202560 42408 0 0 0 2380 7 0 0 25 0 5 0 516901654 874082304 167650 33554432000 4194304 4338100 140735123845648 18446744073709551615 4240516 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=437/tid=479] ppid=435 vsize=853596 CPUtime=23.74 cores=1,3,5,7
/proc/437/task/479/stat : 479 (ttsth-5-4) R 435 437 330 0 -1 4202560 18303 0 0 0 2289 85 0 0 25 0 5 0 516901654 874082304 167650 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254239 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 97.01
Current children cumulated vsize (KiB) 853596

[startup+51.101 s]
/proc/loadavg: 7.84 7.98 7.90 10/186 485
/proc/meminfo: memFree=25121956/32951124 swapFree=67088964/67111528
[pid=437] ppid=435 vsize=1213068 CPUtime=199.28 cores=1,3,5,7
/proc/437/stat : 437 (ttsth-5-4) S 435 437 330 0 -1 4202496 259982 0 0 0 19555 373 0 0 25 0 5 0 516901493 1242181632 257536 33554432000 4194304 4338100 140735123845648 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/437/statm: 303267 257536 176 36 0 301172 0
[pid=437/tid=476] ppid=435 vsize=1213068 CPUtime=49.48 cores=1,3,5,7
/proc/437/task/476/stat : 476 (ttsth-5-4) R 435 437 330 0 -1 4202560 71084 0 0 0 4929 19 0 0 25 0 5 0 516901654 1242181632 257536 33554432000 4194304 4338100 140735123845648 18446744073709551615 210216461571 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=437/tid=477] ppid=435 vsize=1213068 CPUtime=49.48 cores=1,3,5,7
/proc/437/task/477/stat : 477 (ttsth-5-4) R 435 437 330 0 -1 4202560 32485 0 0 0 4783 165 0 0 25 0 5 0 516901654 1242181632 257536 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254342 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=437/tid=478] ppid=435 vsize=1213068 CPUtime=49.47 cores=1,3,5,7
/proc/437/task/478/stat : 478 (ttsth-5-4) R 435 437 330 0 -1 4202560 74611 0 0 0 4934 13 0 0 25 0 5 0 516901654 1242181632 257536 33554432000 4194304 4338100 140735123845648 18446744073709551615 4283592 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=437/tid=479] ppid=435 vsize=1213068 CPUtime=49.21 cores=1,3,5,7
/proc/437/task/479/stat : 479 (ttsth-5-4) R 435 437 330 0 -1 4202560 32624 0 0 0 4753 168 0 0 25 0 5 0 516901654 1242181632 257536 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 199.28
Current children cumulated vsize (KiB) 1213068

[startup+102.306 s]
/proc/loadavg: 7.99 8.00 7.91 9/186 487
/proc/meminfo: memFree=24204960/32951124 swapFree=67088964/67111528
[pid=437] ppid=435 vsize=1726580 CPUtime=403.81 cores=1,3,5,7
/proc/437/stat : 437 (ttsth-5-4) S 435 437 330 0 -1 4202496 389584 0 0 0 39688 693 0 0 25 0 5 0 516901493 1768017920 384162 33554432000 4194304 4338100 140735123845648 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/437/statm: 431645 384162 176 36 0 429550 0
[pid=437/tid=476] ppid=435 vsize=1726580 CPUtime=100.68 cores=1,3,5,7
/proc/437/task/476/stat : 476 (ttsth-5-4) R 435 437 330 0 -1 4202560 125483 0 0 0 10033 35 0 0 25 0 5 0 516901654 1768017920 384162 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254462 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=437/tid=477] ppid=435 vsize=1726580 CPUtime=100.68 cores=1,3,5,7
/proc/437/task/477/stat : 477 (ttsth-5-4) R 435 437 330 0 -1 4202560 47699 0 0 0 9758 310 0 0 25 0 5 0 516901654 1768017920 384162 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=437/tid=478] ppid=435 vsize=1726580 CPUtime=100.68 cores=1,3,5,7
/proc/437/task/478/stat : 478 (ttsth-5-4) R 435 437 330 0 -1 4202560 117500 0 0 0 10044 24 0 0 25 0 5 0 516901654 1768017920 384162 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=437/tid=479] ppid=435 vsize=1726580 CPUtime=100.13 cores=1,3,5,7
/proc/437/task/479/stat : 479 (ttsth-5-4) R 435 437 330 0 -1 4202560 49724 0 0 0 9697 316 0 0 25 0 5 0 516901654 1768017920 384162 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 403.81
Current children cumulated vsize (KiB) 1726580

[startup+162.3 s]
/proc/loadavg: 8.13 8.03 7.92 9/186 488
/proc/meminfo: memFree=23447052/32951124 swapFree=67088964/67111528
[pid=437] ppid=435 vsize=2164220 CPUtime=643.45 cores=1,3,5,7
/proc/437/stat : 437 (ttsth-5-4) S 435 437 330 0 -1 4202496 498102 0 0 0 63317 1028 0 0 25 0 5 0 516901493 2216161280 492680 33554432000 4194304 4338100 140735123845648 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/437/statm: 541055 492680 176 36 0 538960 0
[pid=437/tid=476] ppid=435 vsize=2164220 CPUtime=160.68 cores=1,3,5,7
/proc/437/task/476/stat : 476 (ttsth-5-4) R 435 437 330 0 -1 4202560 175288 0 0 0 16017 51 0 0 25 0 5 0 516901654 2216161280 492680 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254437 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=437/tid=477] ppid=435 vsize=2164220 CPUtime=160.46 cores=1,3,5,7
/proc/437/task/477/stat : 477 (ttsth-5-4) R 435 437 330 0 -1 4202560 56437 0 0 0 15582 464 0 0 25 0 5 0 516901654 2216161280 492680 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=437/tid=478] ppid=435 vsize=2164220 CPUtime=160.67 cores=1,3,5,7
/proc/437/task/478/stat : 478 (ttsth-5-4) R 435 437 330 0 -1 4202560 161319 0 0 0 16034 33 0 0 25 0 5 0 516901654 2216161280 492680 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=437/tid=479] ppid=435 vsize=2164220 CPUtime=160 cores=1,3,5,7
/proc/437/task/479/stat : 479 (ttsth-5-4) R 435 437 330 0 -1 4202560 55880 0 0 0 15528 472 0 0 25 0 5 0 516901654 2216161280 492680 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254208 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 643.45
Current children cumulated vsize (KiB) 2164220

[startup+222.3 s]
/proc/loadavg: 8.17 8.05 7.93 10/186 489
/proc/meminfo: memFree=22863492/32951124 swapFree=67088964/67111528
[pid=437] ppid=435 vsize=2509320 CPUtime=883.08 cores=1,3,5,7
/proc/437/stat : 437 (ttsth-5-4) S 435 437 330 0 -1 4202496 574986 0 0 0 86994 1314 0 0 25 0 5 0 516901493 2569543680 569564 33554432000 4194304 4338100 140735123845648 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/437/statm: 627330 569564 176 36 0 625235 0
[pid=437/tid=476] ppid=435 vsize=2509320 CPUtime=220.66 cores=1,3,5,7
/proc/437/task/476/stat : 476 (ttsth-5-4) R 435 437 330 0 -1 4202560 211724 0 0 0 22001 65 0 0 25 0 5 0 516901654 2569543680 569564 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0

################
# More data... #
################

/proc/437/task/479/stat : 479 (ttsth-5-4) R 435 437 330 0 -1 4202560 84022 0 0 0 68325 1601 0 0 25 0 5 0 516901654 3929063424 893799 33554432000 4194304 4338100 140735123845648 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 2799.48
Current children cumulated vsize (KiB) 3836976

[startup+762.306 s]
/proc/loadavg: 8.15 8.11 8.01 9/186 516
/proc/meminfo: memFree=20064852/32951124 swapFree=67088964/67111528
[pid=437] ppid=435 vsize=4096600 CPUtime=3039.17 cores=1,3,5,7
/proc/437/stat : 437 (ttsth-5-4) S 435 437 330 0 -1 4202496 963666 0 0 0 300294 3623 0 0 25 0 5 0 516901493 4194918400 958244 33554432000 4194304 4338100 140735123845648 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/437/statm: 1024150 958244 176 36 0 1022055 0
[pid=437/tid=476] ppid=435 vsize=4096600 CPUtime=759.44 cores=1,3,5,7
/proc/437/task/476/stat : 476 (ttsth-5-4) R 435 437 330 0 -1 4202560 424293 0 0 0 75793 151 0 0 25 0 5 0 516901654 4194918400 958244 33554432000 4194304 4338100 140735123845648 18446744073709551615 4206902 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=437/tid=477] ppid=435 vsize=4096600 CPUtime=759.28 cores=1,3,5,7
/proc/437/task/477/stat : 477 (ttsth-5-4) R 435 437 330 0 -1 4202560 83513 0 0 0 74233 1695 0 0 25 0 5 0 516901654 4194918400 958244 33554432000 4194304 4338100 140735123845648 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=437/tid=478] ppid=435 vsize=4096600 CPUtime=759.53 cores=1,3,5,7
/proc/437/task/478/stat : 478 (ttsth-5-4) R 435 437 330 0 -1 4202560 322660 0 0 0 75884 69 0 0 25 0 5 0 516901654 4194918400 958244 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=437/tid=479] ppid=435 vsize=4096600 CPUtime=759.26 cores=1,3,5,7
/proc/437/task/479/stat : 479 (ttsth-5-4) R 435 437 330 0 -1 4202560 84022 0 0 0 74227 1699 0 0 25 0 5 0 516901654 4194918400 958244 33554432000 4194304 4338100 140735123845648 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3039.17
Current children cumulated vsize (KiB) 4096600

[startup+822.301 s]
/proc/loadavg: 8.16 8.12 8.02 9/186 517
/proc/meminfo: memFree=19823296/32951124 swapFree=67088964/67111528
[pid=437] ppid=435 vsize=4253132 CPUtime=3278.81 cores=1,3,5,7
/proc/437/stat : 437 (ttsth-5-4) S 435 437 330 0 -1 4202496 1003059 0 0 0 324064 3817 0 0 25 0 5 0 516901493 4355207168 997637 33554432000 4194304 4338100 140735123845648 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/437/statm: 1063283 997637 176 36 0 1061188 0
[pid=437/tid=476] ppid=435 vsize=4253132 CPUtime=819.16 cores=1,3,5,7
/proc/437/task/476/stat : 476 (ttsth-5-4) R 435 437 330 0 -1 4202560 442144 0 0 0 81755 161 0 0 25 0 5 0 516901654 4355207168 997637 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=437/tid=477] ppid=435 vsize=4253132 CPUtime=819.27 cores=1,3,5,7
/proc/437/task/477/stat : 477 (ttsth-5-4) R 435 437 330 0 -1 4202560 83513 0 0 0 80140 1787 0 0 25 0 5 0 516901654 4355207168 997637 33554432000 4194304 4338100 140735123845648 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=437/tid=478] ppid=435 vsize=4253132 CPUtime=819.49 cores=1,3,5,7
/proc/437/task/478/stat : 478 (ttsth-5-4) R 435 437 330 0 -1 4202560 344201 0 0 0 81874 75 0 0 25 0 5 0 516901654 4355207168 997637 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=437/tid=479] ppid=435 vsize=4253132 CPUtime=819.25 cores=1,3,5,7
/proc/437/task/479/stat : 479 (ttsth-5-4) R 435 437 330 0 -1 4202560 84023 0 0 0 80139 1786 0 0 25 0 5 0 516901654 4355207168 997637 33554432000 4194304 4338100 140735123845648 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3278.81
Current children cumulated vsize (KiB) 4253132

[startup+882.301 s]
/proc/loadavg: 8.24 8.14 8.03 10/186 519
/proc/meminfo: memFree=19506076/32951124 swapFree=67088964/67111528
[pid=437] ppid=435 vsize=4387636 CPUtime=3518.49 cores=1,3,5,7
/proc/437/stat : 437 (ttsth-5-4) S 435 437 330 0 -1 4202496 1036257 0 0 0 347827 4022 0 0 25 0 5 0 516901493 4492939264 1030835 33554432000 4194304 4338100 140735123845648 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/437/statm: 1096909 1030835 176 36 0 1094814 0
[pid=437/tid=476] ppid=435 vsize=4387636 CPUtime=878.83 cores=1,3,5,7
/proc/437/task/476/stat : 476 (ttsth-5-4) R 435 437 330 0 -1 4202560 460198 0 0 0 87715 168 0 0 25 0 5 0 516901654 4492939264 1030835 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254342 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=437/tid=477] ppid=435 vsize=4387636 CPUtime=879.27 cores=1,3,5,7
/proc/437/task/477/stat : 477 (ttsth-5-4) R 435 437 330 0 -1 4202560 83513 0 0 0 86044 1883 0 0 25 0 5 0 516901654 4492939264 1030835 33554432000 4194304 4338100 140735123845648 18446744073709551615 210204343728 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=437/tid=478] ppid=435 vsize=4387636 CPUtime=879.49 cores=1,3,5,7
/proc/437/task/478/stat : 478 (ttsth-5-4) R 435 437 330 0 -1 4202560 359345 0 0 0 87871 78 0 0 25 0 5 0 516901654 4492939264 1030835 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=437/tid=479] ppid=435 vsize=4387636 CPUtime=879.26 cores=1,3,5,7
/proc/437/task/479/stat : 479 (ttsth-5-4) R 435 437 330 0 -1 4202560 84023 0 0 0 86041 1885 0 0 25 0 5 0 516901654 4492939264 1030835 33554432000 4194304 4338100 140735123845648 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 3518.49
Current children cumulated vsize (KiB) 4387636

[startup+942.301 s]
/proc/loadavg: 8.16 8.13 8.03 9/186 521
/proc/meminfo: memFree=19135428/32951124 swapFree=67088964/67111528
[pid=437] ppid=435 vsize=4515884 CPUtime=3758.12 cores=1,3,5,7
/proc/437/stat : 437 (ttsth-5-4) S 435 437 330 0 -1 4202496 1068632 0 0 0 371585 4227 0 0 25 0 5 0 516901493 4624265216 1063210 33554432000 4194304 4338100 140735123845648 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/437/statm: 1128971 1063210 176 36 0 1126876 0
[pid=437/tid=476] ppid=435 vsize=4515884 CPUtime=938.65 cores=1,3,5,7
/proc/437/task/476/stat : 476 (ttsth-5-4) R 435 437 330 0 -1 4202560 477030 0 0 0 93690 175 0 0 25 0 5 0 516901654 4624265216 1063210 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=437/tid=477] ppid=435 vsize=4515884 CPUtime=939.26 cores=1,3,5,7
/proc/437/task/477/stat : 477 (ttsth-5-4) R 435 437 330 0 -1 4202560 83514 0 0 0 91945 1981 0 0 25 0 5 0 516901654 4624265216 1063210 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=437/tid=478] ppid=435 vsize=4515884 CPUtime=939.46 cores=1,3,5,7
/proc/437/task/478/stat : 478 (ttsth-5-4) R 435 437 330 0 -1 4202560 374887 0 0 0 93863 83 0 0 25 0 5 0 516901654 4624265216 1063210 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=437/tid=479] ppid=435 vsize=4515884 CPUtime=939.1 cores=1,3,5,7
/proc/437/task/479/stat : 479 (ttsth-5-4) R 435 437 330 0 -1 4202560 84023 0 0 0 91931 1979 0 0 25 0 5 0 516901654 4624265216 1063210 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3758.12
Current children cumulated vsize (KiB) 4515884

[startup+1002.3 s]
/proc/loadavg: 8.06 8.10 8.02 9/186 522
/proc/meminfo: memFree=18828880/32951124 swapFree=67088964/67111528
[pid=437] ppid=435 vsize=4691052 CPUtime=3997.78 cores=1,3,5,7
/proc/437/stat : 437 (ttsth-5-4) S 435 437 330 0 -1 4202496 1112757 0 0 0 395359 4419 0 0 25 0 5 0 516901493 4803637248 1107335 33554432000 4194304 4338100 140735123845648 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/437/statm: 1172763 1107335 176 36 0 1170668 0
[pid=437/tid=476] ppid=435 vsize=4691052 CPUtime=998.65 cores=1,3,5,7
/proc/437/task/476/stat : 476 (ttsth-5-4) R 435 437 330 0 -1 4202560 498620 0 0 0 99683 182 0 0 25 0 5 0 516901654 4803637248 1107335 33554432000 4194304 4338100 140735123845648 18446744073709551615 210204042279 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=437/tid=477] ppid=435 vsize=4691052 CPUtime=999.26 cores=1,3,5,7
/proc/437/task/477/stat : 477 (ttsth-5-4) R 435 437 330 0 -1 4202560 83514 0 0 0 97856 2070 0 0 25 0 5 0 516901654 4803637248 1107335 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254418 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=437/tid=478] ppid=435 vsize=4691052 CPUtime=999.46 cores=1,3,5,7
/proc/437/task/478/stat : 478 (ttsth-5-4) R 435 437 330 0 -1 4202560 397422 0 0 0 99856 90 0 0 25 0 5 0 516901654 4803637248 1107335 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=437/tid=479] ppid=435 vsize=4691052 CPUtime=998.77 cores=1,3,5,7
/proc/437/task/479/stat : 479 (ttsth-5-4) R 435 437 330 0 -1 4202560 84023 0 0 0 97808 2069 0 0 25 0 5 0 516901654 4803637248 1107335 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254437 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3997.78
Current children cumulated vsize (KiB) 4691052

[startup+1062.3 s]
/proc/loadavg: 8.02 8.08 8.01 9/186 524
/proc/meminfo: memFree=18552472/32951124 swapFree=67088964/67111528
[pid=437] ppid=435 vsize=4822560 CPUtime=4237.46 cores=1,3,5,7
/proc/437/stat : 437 (ttsth-5-4) S 435 437 330 0 -1 4202496 1145719 0 0 0 419137 4609 0 0 25 0 5 0 516901493 4938301440 1140297 33554432000 4194304 4338100 140735123845648 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/437/statm: 1205640 1140297 176 36 0 1203545 0
[pid=437/tid=476] ppid=435 vsize=4822560 CPUtime=1058.66 cores=1,3,5,7
/proc/437/task/476/stat : 476 (ttsth-5-4) R 435 437 330 0 -1 4202560 516012 0 0 0 105674 192 0 0 25 0 5 0 516901654 4938301440 1140297 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254285 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=437/tid=477] ppid=435 vsize=4822560 CPUtime=1059.25 cores=1,3,5,7
/proc/437/task/477/stat : 477 (ttsth-5-4) R 435 437 330 0 -1 4202560 83517 0 0 0 103766 2159 0 0 25 0 5 0 516901654 4938301440 1140297 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=437/tid=478] ppid=435 vsize=4822560 CPUtime=1059.46 cores=1,3,5,7
/proc/437/task/478/stat : 478 (ttsth-5-4) R 435 437 330 0 -1 4202560 412982 0 0 0 105852 94 0 0 25 0 5 0 516901654 4938301440 1140297 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=437/tid=479] ppid=435 vsize=4822560 CPUtime=1058.45 cores=1,3,5,7
/proc/437/task/479/stat : 479 (ttsth-5-4) R 435 437 330 0 -1 4202560 84030 0 0 0 103689 2156 0 0 25 0 5 0 516901654 4938301440 1140297 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4237.46
Current children cumulated vsize (KiB) 4822560

[startup+1122.3 s]
/proc/loadavg: 8.00 8.06 8.01 10/186 525
/proc/meminfo: memFree=18318832/32951124 swapFree=67088964/67111528
[pid=437] ppid=435 vsize=5032136 CPUtime=4477.1 cores=1,3,5,7
/proc/437/stat : 437 (ttsth-5-4) S 435 437 330 0 -1 4202496 1197650 0 0 0 442883 4827 0 0 25 0 5 0 516901493 5152907264 1192228 33554432000 4194304 4338100 140735123845648 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/437/statm: 1258034 1192228 176 36 0 1255939 0
[pid=437/tid=476] ppid=435 vsize=5032136 CPUtime=1118.53 cores=1,3,5,7
/proc/437/task/476/stat : 476 (ttsth-5-4) R 435 437 330 0 -1 4202560 543282 0 0 0 111650 203 0 0 25 0 5 0 516901654 5152907264 1192229 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=437/tid=477] ppid=435 vsize=5032136 CPUtime=1119.24 cores=1,3,5,7
/proc/437/task/477/stat : 477 (ttsth-5-4) R 435 437 330 0 -1 4202560 83519 0 0 0 109669 2255 0 0 25 0 5 0 516901654 5152907264 1192229 33554432000 4194304 4338100 140735123845648 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=437/tid=478] ppid=435 vsize=5032136 CPUtime=1119.45 cores=1,3,5,7
/proc/437/task/478/stat : 478 (ttsth-5-4) R 435 437 330 0 -1 4202560 437642 0 0 0 111843 102 0 0 25 0 5 0 516901654 5152907264 1192229 33554432000 4194304 4338100 140735123845648 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=437/tid=479] ppid=435 vsize=5032136 CPUtime=1118.24 cores=1,3,5,7
/proc/437/task/479/stat : 479 (ttsth-5-4) R 435 437 330 0 -1 4202560 84030 0 0 0 109565 2259 0 0 25 0 5 0 516901654 5152907264 1192229 33554432000 4194304 4338100 140735123845648 18446744073709551615 210216473269 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4477.1
Current children cumulated vsize (KiB) 5032136

[startup+1182.3 s]
/proc/loadavg: 8.05 8.06 8.01 9/186 527
/proc/meminfo: memFree=18073292/32951124 swapFree=67088964/67111528
[pid=437] ppid=435 vsize=5113464 CPUtime=4716.77 cores=1,3,5,7
/proc/437/stat : 437 (ttsth-5-4) S 435 437 330 0 -1 4202496 1217870 0 0 0 466670 5007 0 0 25 0 5 0 516901493 5236187136 1212448 33554432000 4194304 4338100 140735123845648 18446744073709551615 210216450021 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/437/statm: 1278366 1212448 176 36 0 1276271 0
[pid=437/tid=476] ppid=435 vsize=5113464 CPUtime=1178.2 cores=1,3,5,7
/proc/437/task/476/stat : 476 (ttsth-5-4) R 435 437 330 0 -1 4202560 561806 0 0 0 117608 212 0 0 25 0 5 0 516901654 5236187136 1212448 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=437/tid=477] ppid=435 vsize=5113464 CPUtime=1179.23 cores=1,3,5,7
/proc/437/task/477/stat : 477 (ttsth-5-4) R 435 437 330 0 -1 4202560 83519 0 0 0 115584 2339 0 0 25 0 5 0 516901654 5236187136 1212448 33554432000 4194304 4338100 140735123845648 18446744073709551615 210204042274 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=437/tid=478] ppid=435 vsize=5113464 CPUtime=1179.46 cores=1,3,5,7
/proc/437/task/478/stat : 478 (ttsth-5-4) R 435 437 330 0 -1 4202560 439337 0 0 0 117843 103 0 0 25 0 5 0 516901654 5236187136 1212448 33554432000 4194304 4338100 140735123845648 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=437/tid=479] ppid=435 vsize=5113464 CPUtime=1178.24 cores=1,3,5,7
/proc/437/task/479/stat : 479 (ttsth-5-4) R 435 437 330 0 -1 4202560 84030 0 0 0 115479 2345 0 0 25 0 5 0 516901654 5236187136 1212448 33554432000 4194304 4338100 140735123845648 18446744073709551615 210204042489 0 0 4096 0 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4716.77
Current children cumulated vsize (KiB) 5113464



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1203.2 s]
/proc/loadavg: 8.04 8.06 8.01 6/183 528
/proc/meminfo: memFree=18649144/32951124 swapFree=67088964/67111528
[pid=437] ppid=435 vsize=0 CPUtime=4800.13 cores=1,3,5,7
/proc/437/stat : 437 (ttsth-5-4) Z 435 437 330 0 -1 4203548 1223733 0 0 0 474949 5064 0 0 18 0 2 0 516901493 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/437/statm: 0 0 0 0 0 0 0
[pid=437/tid=477] ppid=435 vsize=0 CPUtime=1200.12 cores=1,3,5,7
/proc/437/task/477/stat : 477 (ttsth-5-4) R 435 437 330 0 -1 4203588 83520 0 0 0 117645 2367 0 0 25 0 2 0 516901654 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.13
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1203.31 s]
/proc/loadavg: 8.04 8.06 8.01 6/184 529
/proc/meminfo: memFree=21924240/32951124 swapFree=67088964/67111528
[pid=437] ppid=435 vsize=0 CPUtime=4800.24 cores=1,3,5,7
/proc/437/stat : 437 (ttsth-5-4) Z 435 437 330 0 -1 4203548 1223733 0 0 0 474949 5075 0 0 18 0 2 0 516901493 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/437/statm: 0 0 0 0 0 0 0
[pid=437/tid=477] ppid=435 vsize=0 CPUtime=1200.23 cores=1,3,5,7
/proc/437/task/477/stat : 477 (ttsth-5-4) R 435 437 330 0 -1 4203588 83520 0 0 0 117645 2378 0 0 25 0 2 0 516901654 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.24
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 1203.34
CPU time (s): 4800.28
CPU user time (s): 4749.49
CPU system time (s): 50.7833
CPU usage (%): 398.912
Max. virtual memory (cumulated for all children) (KiB): 5135360

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4749.49
system time used= 50.7833
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1223733
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3608
involuntary context switches= 32745

runsolver used 1.93371 second user time and 4.63829 second system time

The end

Launcher Data 

Begin job on node130 at 2011-04-29 06:57:10
IDJOB=3258790
IDBENCH=71425
IDSOLVER=1654
FILE ID=node130/3258790-1304053030
RUNJOBID= node130-1304051825-349
PBS_JOBID= 13170415
Free space on /tmp= 71988 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT09/CRAFTED/rbsat/random/forced/rbsat-v2640c305320gyes10.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3258790-1304053030/watcher-3258790-1304053030 -o /tmp/evaluation-result-3258790-1304053030/solver-3258790-1304053030 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3258790-1304053030.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= e4e4b653016e98f210aa8da2214e783a
RANDOM SEED=927470584

node130.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.81
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.81
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      27178820 kB
Buffers:       1155964 kB
Cached:        4098136 kB
SwapCached:      22432 kB
Active:         361220 kB
Inactive:      4980240 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      27178820 kB
SwapTotal:    67111528 kB
SwapFree:     67088964 kB
Dirty:            3924 kB
Writeback:           0 kB
AnonPages:       64604 kB
Mapped:          15224 kB
Slab:           368224 kB
PageTables:       4260 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   221280 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71988 MiB
End job on node130 at 2011-04-29 07:17:15