Trace number 3264202

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.3 1204.86

General information on the benchmark

NameSAT04/handmade/kexu/frb-0.8/
frb59-26-2.used-as.sat04-892.cnf
MD5SUMfd8d5783ed404e5c6590d627ffdcca50
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables1534
Number of clauses132295
Sum of the clauses size266006
Maximum clause length26
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2132236
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 559

Solver Data 

0.02/0.09	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3264202-1304099103.cnf 
0.02/0.09	c Using SATTIMEOUT of 4800
0.02/0.09	c Using SATRAM of 15500
0.02/0.09	c Size of subset node = 32
0.02/0.09	c Variables = 1534
0.02/0.09	c Clauses = 132295
0.02/0.09	c Literals = 266006
0.02/0.09	c Reduced size:
0.02/0.09	c Variables = 1534
0.02/0.09	c Clauses = 132295
0.02/0.09	c Literals = 266006
0.02/0.09	c a2
0.09/0.16	c a1
0.29/0.33	c Independent sub-problems = 1
0.29/0.33	c Sub-problem with 1534 variables and 132295 clauses
0.29/0.33	c using permutation method 4
0.29/0.33	c count = 1
0.29/0.33	c Ordering finished 0.310 seconds
0.29/0.33	c mapped 0.330 seconds
0.29/0.33	c permuted 0.330 seconds
0.29/0.33	c about to sort
0.29/0.34	c sorted
0.59/0.64	c clauses now 132295
0.59/0.64	c init finished 0.360 seconds
0.59/0.64	c tree built 0.630 seconds
0.59/0.64	c 
0.59/0.64	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.59/0.64	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.59/0.64	c ................................ | ..........                    |                                  0.6         0.0             
0.59/0.64	c ................................ | ..........                  1 |                                  0.6         1.6             
0.59/0.64	c ................................ | ..........                  2 |                                  0.6         3.2             
0.59/0.64	c ................................ | ..........                  3 |                                  0.6         4.8             
0.59/0.64	c ................................ | ..........                  4 |                                  0.6         6.3             
0.59/0.64	c ................................ | ..........                  5 |                                  0.6         7.9             
0.59/0.64	c ................................ | ..........               1000 | 15           90                  0.6      1562.5         4354
95.96/24.61	c ................................ | ..........            3238000 | 94484        314859             96.0     33729.2     15109556
191.91/48.72	c ................................ | ..........            5873000 | 173740       549400            192.0     30588.5     25375706
287.88/72.83	c ................................ | ..........            8172000 | 238761       760748            288.0     28372.0     30650668
383.89/96.93	c ................................ | ..........           10308000 | 300332       955217            384.0     26843.1     38083010
479.92/121.03	c ................................ | *.........           12192000 | 358313       1126515           480.0     25398.9     46223974
575.97/145.11	c ................................ | *.........           13940000 | 411429       1281166           576.0     24201.4     53230344
671.98/169.21	c ................................ | *.........           15664000 | 461948       1435515           672.0     23307.8     56390795
767.98/193.31	c ................................ | *.........           17257000 | 511081       1591501           768.0     22469.5     63336917
864.01/217.40	c ................................ | *.........           19037000 | 563351       1745321           864.0     22033.3     71116965
960.04/241.50	c ................................ | **........           20741000 | 617198       1890742           960.0     21604.3     75058147
1055.70/265.57	c ................................ | **........           22390000 | 670924       2018349          1056.0     21202.2     78679306
1151.71/289.68	c ................................ | **........           23987000 | 723569       2152236          1152.0     20821.5     81302940
1247.75/313.77	c ................................ | **........           25495000 | 774631       2285844          1248.1     20427.7     85012124
1343.81/337.85	c ................................ | **........           27040000 | 824495       2416676          1344.0     20118.7     89504338
1439.87/361.95	c ................................ | ***.......           28396000 | 871488       2533033          1440.0     19718.8     93274233
1535.89/386.03	c ................................ | ***.......           29877000 | 921090       2661617          1536.0     19451.0     97060669
1631.94/410.12	c ................................ | ***.......           31269000 | 962719       2783560          1632.0     19159.5     98310090
1728.00/434.21	c ................................ | ***.......           32592000 | 1005132      2894104          1728.0     18860.6     98511743
1823.67/458.30	c ................................ | ***.......           33890000 | 1046321      3013698          1824.1     18579.4     98511743
1919.72/482.38	c ................................ | ****......           35247000 | 1087089      3133178          1920.0     18357.3    100598000
2015.82/506.45	c ................................ | ****......           36563000 | 1125646      3251026          2016.0     18136.0    103765088
2111.88/530.55	c ................................ | ****......           37785000 | 1159397      3370223          2112.1     17889.8    107429960
2207.98/554.61	c ................................ | ****......           38996000 | 1191705      3476007          2208.0     17661.1    110214572
2303.70/578.68	c ................................ | ****......           40220000 | 1228554      3577483          2304.0     17456.4    110214572
2399.78/602.76	c ................................ | *****.....           41322000 | 1265623      3667789          2400.0     17217.4    110998760
2495.86/626.83	c ................................ | *****.....           42351000 | 1300303      3760042          2496.0     16967.5    115132940
2591.96/650.91	c ................................ | *****.....           43525000 | 1339540      3852019          2592.0     16792.0    119702664
2688.04/675.00	c ................................ | *****.....           44641000 | 1377982      3942090          2688.1     16607.1    121739387
2783.75/699.06	c ................................ | *****.....           45812000 | 1415226      4042715          2784.0     16455.4    124547330
2879.85/723.15	c ................................ | ******....           46870000 | 1452238      4141829          2880.0     16274.1    128110741
2975.95/747.23	c ................................ | ******....           47929000 | 1489873      4238129          2976.1     16104.8    131443118
3072.02/771.30	c ................................ | ******....           49004000 | 1525760      4331288          3072.0     15951.7    134763039
3167.77/795.36	c ................................ | ******....           50104000 | 1562597      4419082          3168.0     15815.6    138480143
3263.91/819.42	c ................................ | ******....           51181000 | 1599416      4509318          3264.0     15680.5    141558022
3360.01/843.51	c ................................ | *******...           52171000 | 1636452      4593283          3360.1     15526.7    142903598
3455.73/867.59	c ................................ | *******...           53180000 | 1669624      4683006          3456.1     15387.3    145721355
3551.83/891.66	c ................................ | *******...           54105000 | 1702842      4766640          3552.1     15232.0    146509922
3647.96/915.73	c ................................ | *******...           55039000 | 1736327      4848783          3648.1     15087.1    149176892
3743.69/939.79	c ................................ | *******...           56037000 | 1769210      4933338          3744.1     14966.9    152796217
3839.80/963.85	c ................................ | ********..           56976000 | 1799575      5015863          3840.0     14837.4    155355878
3935.92/987.92	c ................................ | ********..           57950000 | 1830167      5100989          3936.0     14723.0    158247241
4031.68/1011.99	c ................................ | ********..           58964000 | 1860181      5187293          4032.0     14623.9    163436712
4127.78/1036.05	c ................................ | ********..           59969000 | 1892942      5272730          4128.0     14527.4    168521347
4223.93/1060.12	c ................................ | ********..           60979000 | 1925214      5355124          4224.0     14436.2    173795616
4319.99/1084.22	c ................................ | *********.           61955000 | 1957160      5432434          4320.1     14341.2    178283413
4415.82/1108.24	c ................................ | *********.           63057000 | 1992929      5517984          4416.0     14279.2    181117950
4512.08/1132.30	c ................................ | *********.           64151000 | 2029573      5602090          4512.1     14217.5    182968630
4607.94/1156.32	c ................................ | *********.           65202000 | 2063920      5680497          4608.0     14149.6    185110405
4703.78/1180.55	c ................................ | *********.           66269000 | 2095094      5760517          4704.0     14087.8    185506620
4799.98/1204.61	c CPU limit (4800) reached
4799.98/1204.61	c Time = 4800.04 seconds
4799.98/1204.61	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3264202-1304099103/watcher-3264202-1304099103 -o /tmp/evaluation-result-3264202-1304099103/solver-3264202-1304099103 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3264202-1304099103.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.87 7.82 7.84 5/185 10746
/proc/meminfo: memFree=18849580/32951124 swapFree=67111352/67111528
[pid=10746] ppid=10744 vsize=9764 CPUtime=0 cores=0,2,4,6
/proc/10746/stat : 10746 (ttsth-5-4) R 10744 10746 10639 0 -1 4202496 489 0 0 0 0 0 0 0 25 0 1 0 521508543 9998336 432 33554432000 4194304 4338100 140734051659968 18446744073709551615 246516495045 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/10746/statm: 2441 433 147 36 0 346 0

[startup+0.0295841 s]
/proc/loadavg: 7.87 7.82 7.84 5/185 10746
/proc/meminfo: memFree=18849580/32951124 swapFree=67111352/67111528
[pid=10746] ppid=10744 vsize=14472 CPUtime=0.02 cores=0,2,4,6
/proc/10746/stat : 10746 (ttsth-5-4) R 10744 10746 10639 0 -1 4202496 1669 0 0 0 2 0 0 0 25 0 1 0 521508543 14819328 1612 33554432000 4194304 4338100 140734051659968 18446744073709551615 246516495045 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/10746/statm: 3618 1613 148 36 0 1523 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 14472

[startup+0.100594 s]
/proc/loadavg: 7.87 7.82 7.84 5/185 10746
/proc/meminfo: memFree=18849580/32951124 swapFree=67111352/67111528
[pid=10746] ppid=10744 vsize=30784 CPUtime=0.09 cores=0,2,4,6
/proc/10746/stat : 10746 (ttsth-5-4) R 10744 10746 10639 0 -1 4202496 5744 0 0 0 9 0 0 0 25 0 1 0 521508543 31522816 5687 33554432000 4194304 4338100 140734051659968 18446744073709551615 246516495045 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/10746/statm: 7696 5702 151 36 0 5601 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 30784

[startup+0.300564 s]
/proc/loadavg: 7.87 7.82 7.84 5/185 10746
/proc/meminfo: memFree=18849580/32951124 swapFree=67111352/67111528
[pid=10746] ppid=10744 vsize=38568 CPUtime=0.29 cores=0,2,4,6
/proc/10746/stat : 10746 (ttsth-5-4) R 10744 10746 10639 0 -1 4202496 10226 0 0 0 28 1 0 0 25 0 1 0 521508543 39493632 7666 33554432000 4194304 4338100 140734051659968 18446744073709551615 4295813 0 0 4096 0 0 0 0 17 4 0 0 0
/proc/10746/statm: 9642 7666 156 36 0 7547 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 38568

[startup+0.712251 s]
/proc/loadavg: 7.87 7.82 7.84 5/185 10746
/proc/meminfo: memFree=18849580/32951124 swapFree=67111352/67111528
[pid=10746] ppid=10744 vsize=344828 CPUtime=0.91 cores=0,2,4,6
/proc/10746/stat : 10746 (ttsth-5-4) S 10744 10746 10639 0 -1 4202496 26064 0 0 0 88 3 0 0 25 0 5 0 521508543 353103872 22985 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 6 0 0 0
/proc/10746/statm: 86207 22985 176 36 0 84112 0
Current children cumulated CPU time (s) 0.91
Current children cumulated vsize (KiB) 344828

[startup+1.50027 s]
/proc/loadavg: 7.88 7.83 7.84 10/190 10752
/proc/meminfo: memFree=18745636/32951124 swapFree=67111352/67111528
[pid=10746] ppid=10744 vsize=370776 CPUtime=4.04 cores=0,2,4,6
/proc/10746/stat : 10746 (ttsth-5-4) S 10744 10746 10639 0 -1 4202496 33453 0 0 0 398 6 0 0 25 0 5 0 521508543 379674624 30374 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 6 0 0 0
/proc/10746/statm: 92694 30375 176 36 0 90599 0
[pid=10746/tid=10749] ppid=10744 vsize=370776 CPUtime=0.84 cores=0,2,4,6
/proc/10746/task/10749/stat : 10749 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 3319 0 0 0 84 0 0 0 19 0 5 0 521508607 379674624 30375 33554432000 4194304 4338100 140734051659968 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10746/tid=10750] ppid=10744 vsize=370776 CPUtime=0.82 cores=0,2,4,6
/proc/10746/task/10750/stat : 10750 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 1505 0 0 0 82 0 0 0 18 0 5 0 521508607 379674624 30376 33554432000 4194304 4338100 140734051659968 18446744073709551615 4254254 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10746/tid=10751] ppid=10744 vsize=370776 CPUtime=0.85 cores=0,2,4,6
/proc/10746/task/10751/stat : 10751 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 2830 0 0 0 85 0 0 0 19 0 5 0 521508607 379674624 30376 33554432000 4194304 4338100 140734051659968 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10746/tid=10752] ppid=10744 vsize=370776 CPUtime=0.85 cores=0,2,4,6
/proc/10746/task/10752/stat : 10752 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 1325 0 0 0 84 1 0 0 18 0 5 0 521508607 379674624 30376 33554432000 4194304 4338100 140734051659968 18446744073709551615 4284188 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4.04
Current children cumulated vsize (KiB) 370776

[startup+3.10102 s]
/proc/loadavg: 7.88 7.83 7.84 9/190 10752
/proc/meminfo: memFree=18706328/32951124 swapFree=67111352/67111528
[pid=10746] ppid=10744 vsize=421088 CPUtime=10.41 cores=0,2,4,6
/proc/10746/stat : 10746 (ttsth-5-4) S 10744 10746 10639 0 -1 4202496 45555 0 0 0 1030 11 0 0 25 0 5 0 521508543 431194112 42476 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 6 0 0 0
/proc/10746/statm: 105272 42476 176 36 0 103177 0
[pid=10746/tid=10749] ppid=10744 vsize=421088 CPUtime=2.44 cores=0,2,4,6
/proc/10746/task/10749/stat : 10749 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 7742 0 0 0 243 1 0 0 22 0 5 0 521508607 431194112 42477 33554432000 4194304 4338100 140734051659968 18446744073709551615 4254362 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10746/tid=10750] ppid=10744 vsize=421088 CPUtime=2.39 cores=0,2,4,6
/proc/10746/task/10750/stat : 10750 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 3903 0 0 0 238 1 0 0 18 0 5 0 521508607 431194112 42478 33554432000 4194304 4338100 140734051659968 18446744073709551615 246516229566 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10746/tid=10751] ppid=10744 vsize=421088 CPUtime=2.45 cores=0,2,4,6
/proc/10746/task/10751/stat : 10751 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 6515 0 0 0 244 1 0 0 22 0 5 0 521508607 431194112 42478 33554432000 4194304 4338100 140734051659968 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10746/tid=10752] ppid=10744 vsize=421088 CPUtime=2.45 cores=0,2,4,6
/proc/10746/task/10752/stat : 10752 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 2921 0 0 0 242 3 0 0 19 0 5 0 521508607 431194112 42478 33554432000 4194304 4338100 140734051659968 18446744073709551615 246516230150 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 10.41
Current children cumulated vsize (KiB) 421088

[startup+6.3004 s]
/proc/loadavg: 7.88 7.83 7.84 6/186 10780
/proc/meminfo: memFree=24720260/32951124 swapFree=67111352/67111528
[pid=10746] ppid=10744 vsize=502680 CPUtime=23.14 cores=0,2,4,6
/proc/10746/stat : 10746 (ttsth-5-4) S 10744 10746 10639 0 -1 4202496 66241 0 0 0 2290 24 0 0 25 0 5 0 521508543 514744320 63162 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 6 0 0 0
/proc/10746/statm: 125670 63162 176 36 0 123575 0
[pid=10746/tid=10749] ppid=10744 vsize=502680 CPUtime=5.65 cores=0,2,4,6
/proc/10746/task/10749/stat : 10749 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 15625 0 0 0 561 4 0 0 25 0 5 0 521508607 514744320 63162 33554432000 4194304 4338100 140734051659968 18446744073709551615 246516530736 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10746/tid=10750] ppid=10744 vsize=502680 CPUtime=5.53 cores=0,2,4,6
/proc/10746/task/10750/stat : 10750 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 8601 0 0 0 550 3 0 0 18 0 5 0 521508607 514744320 63162 33554432000 4194304 4338100 140734051659968 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10746/tid=10751] ppid=10744 vsize=502680 CPUtime=5.65 cores=0,2,4,6
/proc/10746/task/10751/stat : 10751 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 12484 0 0 0 563 2 0 0 25 0 5 0 521508607 514744320 63162 33554432000 4194304 4338100 140734051659968 18446744073709551615 246516229566 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10746/tid=10752] ppid=10744 vsize=502680 CPUtime=5.65 cores=0,2,4,6
/proc/10746/task/10752/stat : 10752 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 5056 0 0 0 554 11 0 0 25 0 5 0 521508607 514744320 63162 33554432000 4194304 4338100 140734051659968 18446744073709551615 4254827 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 23.14
Current children cumulated vsize (KiB) 502680

[startup+12.7002 s]
/proc/loadavg: 7.89 7.83 7.84 9/190 10784
/proc/meminfo: memFree=24339140/32951124 swapFree=67111352/67111528
[pid=10746] ppid=10744 vsize=632616 CPUtime=48.6 cores=0,2,4,6
/proc/10746/stat : 10746 (ttsth-5-4) S 10744 10746 10639 0 -1 4202496 98364 0 0 0 4817 43 0 0 25 0 5 0 521508543 647798784 95285 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 6 0 0 0
/proc/10746/statm: 158154 95285 176 36 0 156059 0
[pid=10746/tid=10749] ppid=10744 vsize=632616 CPUtime=12.05 cores=0,2,4,6
/proc/10746/task/10749/stat : 10749 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 29483 0 0 0 1197 8 0 0 25 0 5 0 521508607 647798784 95285 33554432000 4194304 4338100 140734051659968 18446744073709551615 4256000 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10746/tid=10750] ppid=10744 vsize=632616 CPUtime=11.88 cores=0,2,4,6
/proc/10746/task/10750/stat : 10750 (ttsth-5-4) S 10744 10746 10639 0 -1 4202560 12723 0 0 0 1178 10 0 0 25 0 5 0 521508607 647798784 95285 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528659828 0 0 4096 0 18446604446947248064 0 0 -1 0 0 0 0
[pid=10746/tid=10751] ppid=10744 vsize=632616 CPUtime=12.05 cores=0,2,4,6
/proc/10746/task/10751/stat : 10751 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 22871 0 0 0 1201 4 0 0 25 0 5 0 521508607 647798784 95285 33554432000 4194304 4338100 140734051659968 18446744073709551615 4206896 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10746/tid=10752] ppid=10744 vsize=632616 CPUtime=11.96 cores=0,2,4,6
/proc/10746/task/10752/stat : 10752 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 8812 0 0 0 1179 17 0 0 18 0 5 0 521508607 647798784 95285 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528648247 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 48.6
Current children cumulated vsize (KiB) 632616

[startup+25.5009 s]
/proc/loadavg: 7.99 7.85 7.85 9/190 10784
/proc/meminfo: memFree=23800384/32951124 swapFree=67111352/67111528
[pid=10746] ppid=10744 vsize=836424 CPUtime=99.54 cores=0,2,4,6
/proc/10746/stat : 10746 (ttsth-5-4) S 10744 10746 10639 0 -1 4202496 150517 0 0 0 9876 78 0 0 25 0 5 0 521508543 856498176 145955 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 6 0 0 0
/proc/10746/statm: 209106 145955 176 36 0 207011 0
[pid=10746/tid=10749] ppid=10744 vsize=836424 CPUtime=24.85 cores=0,2,4,6
/proc/10746/task/10749/stat : 10749 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 50751 0 0 0 2470 15 0 0 25 0 5 0 521508607 856498176 145955 33554432000 4194304 4338100 140734051659968 18446744073709551615 246516530714 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10746/tid=10750] ppid=10744 vsize=836424 CPUtime=24.58 cores=0,2,4,6
/proc/10746/task/10750/stat : 10750 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 20248 0 0 0 2433 25 0 0 18 0 5 0 521508607 856498176 145955 33554432000 4194304 4338100 140734051659968 18446744073709551615 4254285 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10746/tid=10751] ppid=10744 vsize=836424 CPUtime=24.84 cores=0,2,4,6
/proc/10746/task/10751/stat : 10751 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 38947 0 0 0 2478 6 0 0 25 0 5 0 521508607 856498176 145955 33554432000 4194304 4338100 140734051659968 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10746/tid=10752] ppid=10744 vsize=836424 CPUtime=24.6 cores=0,2,4,6
/proc/10746/task/10752/stat : 10752 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 16096 0 0 0 2433 27 0 0 25 0 5 0 521508607 856498176 145955 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 99.54
Current children cumulated vsize (KiB) 836424

[startup+51.1012 s]
/proc/loadavg: 8.25 7.93 7.87 9/190 10785
/proc/meminfo: memFree=23021860/32951124 swapFree=67111352/67111528
[pid=10746] ppid=10744 vsize=1196408 CPUtime=201.48 cores=0,2,4,6
/proc/10746/stat : 10746 (ttsth-5-4) S 10744 10746 10639 0 -1 4202496 240002 0 0 0 20012 136 0 0 25 0 5 0 521508543 1225121792 235440 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 6 0 0 0
/proc/10746/statm: 299102 235440 176 36 0 297007 0
[pid=10746/tid=10749] ppid=10744 vsize=1196408 CPUtime=50.44 cores=0,2,4,6
/proc/10746/task/10749/stat : 10749 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 92096 0 0 0 5020 24 0 0 25 0 5 0 521508607 1225121792 235440 33554432000 4194304 4338100 140734051659968 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10746/tid=10750] ppid=10744 vsize=1196408 CPUtime=49.71 cores=0,2,4,6
/proc/10746/task/10750/stat : 10750 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 30331 0 0 0 4939 32 0 0 18 0 5 0 521508607 1225121792 235440 33554432000 4194304 4338100 140734051659968 18446744073709551615 4240516 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10746/tid=10751] ppid=10744 vsize=1196408 CPUtime=50.45 cores=0,2,4,6
/proc/10746/task/10751/stat : 10751 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 66219 0 0 0 5034 11 0 0 25 0 5 0 521508607 1225121792 235440 33554432000 4194304 4338100 140734051659968 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10746/tid=10752] ppid=10744 vsize=1196408 CPUtime=50.2 cores=0,2,4,6
/proc/10746/task/10752/stat : 10752 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 26881 0 0 0 4957 63 0 0 25 0 5 0 521508607 1225121792 235440 33554432000 4194304 4338100 140734051659968 18446744073709551615 4284167 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 201.48
Current children cumulated vsize (KiB) 1196408

[startup+102.307 s]
/proc/loadavg: 8.48 8.03 7.91 9/190 10786
/proc/meminfo: memFree=22187060/32951124 swapFree=67111352/67111528
[pid=10746] ppid=10744 vsize=1638148 CPUtime=405.43 cores=0,2,4,6
/proc/10746/stat : 10746 (ttsth-5-4) S 10744 10746 10639 0 -1 4202496 345912 0 0 0 40313 230 0 0 25 0 5 0 521508543 1677463552 341350 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 6 0 0 0
/proc/10746/statm: 409537 341350 176 36 0 407442 0
[pid=10746/tid=10749] ppid=10744 vsize=1638148 CPUtime=101.63 cores=0,2,4,6
/proc/10746/task/10749/stat : 10749 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 143399 0 0 0 10122 41 0 0 25 0 5 0 521508607 1677463552 341350 33554432000 4194304 4338100 140734051659968 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10746/tid=10750] ppid=10744 vsize=1638148 CPUtime=100.91 cores=0,2,4,6
/proc/10746/task/10750/stat : 10750 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 40558 0 0 0 10001 90 0 0 25 0 5 0 521508607 1677463552 341350 33554432000 4194304 4338100 140734051659968 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=10746/tid=10751] ppid=10744 vsize=1638148 CPUtime=101.64 cores=0,2,4,6
/proc/10746/task/10751/stat : 10751 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 103785 0 0 0 10145 19 0 0 25 0 5 0 521508607 1677463552 341350 33554432000 4194304 4338100 140734051659968 18446744073709551615 246516229141 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10746/tid=10752] ppid=10744 vsize=1638148 CPUtime=100.57 cores=0,2,4,6
/proc/10746/task/10752/stat : 10752 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 33695 0 0 0 9983 74 0 0 18 0 5 0 521508607 1677463552 341350 33554432000 4194304 4338100 140734051659968 18446744073709551615 4254400 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 405.43
Current children cumulated vsize (KiB) 1638148

[startup+162.301 s]
/proc/loadavg: 8.50 8.09 7.93 9/190 10788
/proc/meminfo: memFree=20961524/32951124 swapFree=67111352/67111528
[pid=10746] ppid=10744 vsize=2160656 CPUtime=644.49 cores=0,2,4,6
/proc/10746/stat : 10746 (ttsth-5-4) S 10744 10746 10639 0 -1 4202496 477042 0 0 0 64116 333 0 0 25 0 5 0 521508543 2212511744 472480 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 6 0 0 0
/proc/10746/statm: 540164 472480 176 36 0 538069 0
[pid=10746/tid=10749] ppid=10744 vsize=2160656 CPUtime=161.61 cores=0,2,4,6
/proc/10746/task/10749/stat : 10749 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 202194 0 0 0 16104 57 0 0 25 0 5 0 521508607 2212511744 472480 33554432000 4194304 4338100 140734051659968 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10746/tid=10750] ppid=10744 vsize=2160656 CPUtime=160.9 cores=0,2,4,6
/proc/10746/task/10750/stat : 10750 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 52284 0 0 0 15936 154 0 0 25 0 5 0 521508607 2212511744 472480 33554432000 4194304 4338100 140734051659968 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=10746/tid=10751] ppid=10744 vsize=2160656 CPUtime=161.63 cores=0,2,4,6
/proc/10746/task/10751/stat : 10751 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 146701 0 0 0 16135 28 0 0 25 0 5 0 521508607 2212511744 472480 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528646950 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10746/tid=10752] ppid=10744 vsize=2160656 CPUtime=159.67 cores=0,2,4,6
/proc/10746/task/10752/stat : 10752 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 51388 0 0 0 15879 88 0 0 18 0 5 0 521508607 2212511744 472480 33554432000 4194304 4338100 140734051659968 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 644.49
Current children cumulated vsize (KiB) 2160656


################
# More data... #
################

/proc/10746/task/10752/stat : 10752 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 86083 0 0 0 69331 246 0 0 18 0 5 0 521508607 4528435200 1035571 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 2796.89
Current children cumulated vsize (KiB) 4422300

[startup+762.306 s]
/proc/loadavg: 8.22 8.12 8.01 9/190 10804
/proc/meminfo: memFree=15342252/32951124 swapFree=67111352/67111528
[pid=10746] ppid=10744 vsize=4688312 CPUtime=3036.17 cores=0,2,4,6
/proc/10746/stat : 10746 (ttsth-5-4) S 10744 10746 10639 0 -1 4202496 1106175 0 0 0 302493 1124 0 0 25 0 5 0 521508543 4800831488 1101613 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 6 0 0 0
/proc/10746/statm: 1172078 1101613 176 36 0 1169983 0
[pid=10746/tid=10749] ppid=10744 vsize=4688312 CPUtime=761.48 cores=0,2,4,6
/proc/10746/task/10749/stat : 10749 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 531766 0 0 0 75976 172 0 0 25 0 5 0 521508607 4800831488 1101613 33554432000 4194304 4338100 140734051659968 18446744073709551615 246516530673 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10746/tid=10750] ppid=10744 vsize=4688312 CPUtime=759.39 cores=0,2,4,6
/proc/10746/task/10750/stat : 10750 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 78872 0 0 0 75325 614 0 0 25 0 5 0 521508607 4800831488 1101613 33554432000 4194304 4338100 140734051659968 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=10746/tid=10751] ppid=10744 vsize=4688312 CPUtime=759.25 cores=0,2,4,6
/proc/10746/task/10751/stat : 10751 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 384969 0 0 0 75843 82 0 0 25 0 5 0 521508607 4800831488 1101613 33554432000 4194304 4338100 140734051659968 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10746/tid=10752] ppid=10744 vsize=4688312 CPUtime=755.38 cores=0,2,4,6
/proc/10746/task/10752/stat : 10752 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 86093 0 0 0 75287 251 0 0 18 0 5 0 521508607 4800831488 1101613 33554432000 4194304 4338100 140734051659968 18446744073709551615 246516229262 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3036.17
Current children cumulated vsize (KiB) 4688312

[startup+822.3 s]
/proc/loadavg: 8.19 8.12 8.02 9/190 10805
/proc/meminfo: memFree=14874000/32951124 swapFree=67111352/67111528
[pid=10746] ppid=10744 vsize=4957320 CPUtime=3275.47 cores=0,2,4,6
/proc/10746/stat : 10746 (ttsth-5-4) S 10744 10746 10639 0 -1 4202496 1174375 0 0 0 326358 1189 0 0 25 0 5 0 521508543 5076295680 1169813 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 6 0 0 0
/proc/10746/statm: 1239330 1169813 176 36 0 1237235 0
[pid=10746/tid=10749] ppid=10744 vsize=4957320 CPUtime=821.4 cores=0,2,4,6
/proc/10746/task/10749/stat : 10749 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 567512 0 0 0 81956 184 0 0 25 0 5 0 521508607 5076295680 1169813 33554432000 4194304 4338100 140734051659968 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10746/tid=10750] ppid=10744 vsize=4957320 CPUtime=819.37 cores=0,2,4,6
/proc/10746/task/10750/stat : 10750 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 78880 0 0 0 81283 654 0 0 25 0 5 0 521508607 5076295680 1169813 33554432000 4194304 4338100 140734051659968 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=10746/tid=10751] ppid=10744 vsize=4957320 CPUtime=818.96 cores=0,2,4,6
/proc/10746/task/10751/stat : 10751 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 416178 0 0 0 81807 89 0 0 25 0 5 0 521508607 5076295680 1169813 33554432000 4194304 4338100 140734051659968 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10746/tid=10752] ppid=10744 vsize=4957320 CPUtime=815.05 cores=0,2,4,6
/proc/10746/task/10752/stat : 10752 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 87330 0 0 0 81249 256 0 0 18 0 5 0 521508607 5076295680 1169813 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3275.47
Current children cumulated vsize (KiB) 4957320

[startup+882.3 s]
/proc/loadavg: 8.12 8.11 8.02 9/190 10807
/proc/meminfo: memFree=14650036/32951124 swapFree=67111352/67111528
[pid=10746] ppid=10744 vsize=5094952 CPUtime=3514.75 cores=0,2,4,6
/proc/10746/stat : 10746 (ttsth-5-4) S 10744 10746 10639 0 -1 4202496 1208401 0 0 0 350228 1247 0 0 25 0 5 0 521508543 5217230848 1203839 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 6 0 0 0
/proc/10746/statm: 1273738 1203839 176 36 0 1271643 0
[pid=10746/tid=10749] ppid=10744 vsize=5094952 CPUtime=881.34 cores=0,2,4,6
/proc/10746/task/10749/stat : 10749 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 584230 0 0 0 87944 190 0 0 25 0 5 0 521508607 5217230848 1203839 33554432000 4194304 4338100 140734051659968 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10746/tid=10750] ppid=10744 vsize=5094952 CPUtime=879.35 cores=0,2,4,6
/proc/10746/task/10750/stat : 10750 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 78889 0 0 0 87238 697 0 0 25 0 5 0 521508607 5217230848 1203839 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=10746/tid=10751] ppid=10744 vsize=5094952 CPUtime=878.69 cores=0,2,4,6
/proc/10746/task/10751/stat : 10751 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 432359 0 0 0 87776 93 0 0 25 0 5 0 521508607 5217230848 1203839 33554432000 4194304 4338100 140734051659968 18446744073709551615 4254329 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10746/tid=10752] ppid=10744 vsize=5094952 CPUtime=874.69 cores=0,2,4,6
/proc/10746/task/10752/stat : 10752 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 88448 0 0 0 87208 261 0 0 18 0 5 0 521508607 5217230848 1203839 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3514.75
Current children cumulated vsize (KiB) 5094952

[startup+942.3 s]
/proc/loadavg: 8.09 8.11 8.02 9/190 10809
/proc/meminfo: memFree=14415020/32951124 swapFree=67111352/67111528
[pid=10746] ppid=10744 vsize=5307656 CPUtime=3754.06 cores=0,2,4,6
/proc/10746/stat : 10746 (ttsth-5-4) S 10744 10746 10639 0 -1 4202496 1262146 0 0 0 374092 1314 0 0 25 0 5 0 521508543 5435039744 1257584 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 6 0 0 0
/proc/10746/statm: 1326914 1257585 176 36 0 1324819 0
[pid=10746/tid=10749] ppid=10744 vsize=5307656 CPUtime=941.08 cores=0,2,4,6
/proc/10746/task/10749/stat : 10749 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 612652 0 0 0 93908 200 0 0 25 0 5 0 521508607 5435039744 1257585 33554432000 4194304 4338100 140734051659968 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10746/tid=10750] ppid=10744 vsize=5307656 CPUtime=939.36 cores=0,2,4,6
/proc/10746/task/10750/stat : 10750 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 78894 0 0 0 93193 743 0 0 25 0 5 0 521508607 5435039744 1257585 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=10746/tid=10751] ppid=10744 vsize=5307656 CPUtime=938.58 cores=0,2,4,6
/proc/10746/task/10751/stat : 10751 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 456080 0 0 0 93759 99 0 0 25 0 5 0 521508607 5435039744 1257585 33554432000 4194304 4338100 140734051659968 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10746/tid=10752] ppid=10744 vsize=5307656 CPUtime=934.36 cores=0,2,4,6
/proc/10746/task/10752/stat : 10752 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 90046 0 0 0 93170 266 0 0 18 0 5 0 521508607 5435039744 1257585 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3754.06
Current children cumulated vsize (KiB) 5307656

[startup+1002.3 s]
/proc/loadavg: 8.21 8.13 8.03 9/190 10819
/proc/meminfo: memFree=14103928/32951124 swapFree=67111352/67111528
[pid=10746] ppid=10744 vsize=5557896 CPUtime=3993.36 cores=0,2,4,6
/proc/10746/stat : 10746 (ttsth-5-4) S 10744 10746 10639 0 -1 4202496 1323778 0 0 0 397958 1378 0 0 25 0 5 0 521508543 5691285504 1319216 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 6 0 0 0
/proc/10746/statm: 1389474 1319216 176 36 0 1387379 0
[pid=10746/tid=10749] ppid=10744 vsize=5557896 CPUtime=1001.07 cores=0,2,4,6
/proc/10746/task/10749/stat : 10749 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 641880 0 0 0 99898 209 0 0 25 0 5 0 521508607 5691285504 1319216 33554432000 4194304 4338100 140734051659968 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10746/tid=10750] ppid=10744 vsize=5557896 CPUtime=999.35 cores=0,2,4,6
/proc/10746/task/10750/stat : 10750 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 82201 0 0 0 99148 787 0 0 25 0 5 0 521508607 5691285504 1319216 33554432000 4194304 4338100 140734051659968 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=10746/tid=10751] ppid=10744 vsize=5557896 CPUtime=998.24 cores=0,2,4,6
/proc/10746/task/10751/stat : 10751 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 474359 0 0 0 99721 103 0 0 25 0 5 0 521508607 5691285504 1319216 33554432000 4194304 4338100 140734051659968 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10746/tid=10752] ppid=10744 vsize=5557896 CPUtime=994.02 cores=0,2,4,6
/proc/10746/task/10752/stat : 10752 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 100863 0 0 0 99128 274 0 0 18 0 5 0 521508607 5691285504 1319216 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 3993.36
Current children cumulated vsize (KiB) 5557896

[startup+1062.3 s]
/proc/loadavg: 8.18 8.14 8.03 9/190 10821
/proc/meminfo: memFree=13554336/32951124 swapFree=67111352/67111528
[pid=10746] ppid=10744 vsize=5967664 CPUtime=4232.69 cores=0,2,4,6
/proc/10746/stat : 10746 (ttsth-5-4) S 10744 10746 10639 0 -1 4202496 1426215 0 0 0 421820 1449 0 0 25 0 5 0 521508543 6110887936 1421653 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 6 0 0 0
/proc/10746/statm: 1491916 1421653 176 36 0 1489821 0
[pid=10746/tid=10749] ppid=10744 vsize=5967664 CPUtime=1061.07 cores=0,2,4,6
/proc/10746/task/10749/stat : 10749 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 680829 0 0 0 105886 221 0 0 25 0 5 0 521508607 6110887936 1421653 33554432000 4194304 4338100 140734051659968 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10746/tid=10750] ppid=10744 vsize=5967664 CPUtime=1059.36 cores=0,2,4,6
/proc/10746/task/10750/stat : 10750 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 96340 0 0 0 105104 832 0 0 25 0 5 0 521508607 6110887936 1421653 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=10746/tid=10751] ppid=10744 vsize=5967664 CPUtime=1057.93 cores=0,2,4,6
/proc/10746/task/10751/stat : 10751 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 508202 0 0 0 105683 110 0 0 25 0 5 0 521508607 6110887936 1421653 33554432000 4194304 4338100 140734051659968 18446744073709551615 4253954 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10746/tid=10752] ppid=10744 vsize=5967664 CPUtime=1053.67 cores=0,2,4,6
/proc/10746/task/10752/stat : 10752 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 116369 0 0 0 105085 282 0 0 18 0 5 0 521508607 6110887936 1421653 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528660149 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4232.69
Current children cumulated vsize (KiB) 5967664

[startup+1122.3 s]
/proc/loadavg: 8.17 8.14 8.04 9/190 10822
/proc/meminfo: memFree=13069332/32951124 swapFree=67111352/67111528
[pid=10746] ppid=10744 vsize=6227420 CPUtime=4472.14 cores=0,2,4,6
/proc/10746/stat : 10746 (ttsth-5-4) S 10744 10746 10639 0 -1 4202496 1490529 0 0 0 445600 1614 0 0 25 0 5 0 521508543 6376878080 1485967 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 6 0 0 0
/proc/10746/statm: 1556855 1485967 176 36 0 1554760 0
[pid=10746/tid=10749] ppid=10744 vsize=6227420 CPUtime=1121.05 cores=0,2,4,6
/proc/10746/task/10749/stat : 10749 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 711625 0 0 0 111874 231 0 0 25 0 5 0 521508607 6376878080 1485967 33554432000 4194304 4338100 140734051659968 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10746/tid=10750] ppid=10744 vsize=6227420 CPUtime=1119.26 cores=0,2,4,6
/proc/10746/task/10750/stat : 10750 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 102603 0 0 0 111023 903 0 0 25 0 5 0 521508607 6376878080 1485967 33554432000 4194304 4338100 140734051659968 18446744073709551615 4285600 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10746/tid=10751] ppid=10744 vsize=6227420 CPUtime=1117.79 cores=0,2,4,6
/proc/10746/task/10751/stat : 10751 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 527428 0 0 0 111665 114 0 0 25 0 5 0 521508607 6376878080 1485967 33554432000 4194304 4338100 140734051659968 18446744073709551615 4253962 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=10746/tid=10752] ppid=10744 vsize=6227420 CPUtime=1113.37 cores=0,2,4,6
/proc/10746/task/10752/stat : 10752 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 124398 0 0 0 110976 361 0 0 25 0 5 0 521508607 6376878080 1485967 33554432000 4194304 4338100 140734051659968 18446744073709551615 4285386 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4472.14
Current children cumulated vsize (KiB) 6227420

[startup+1182.3 s]
/proc/loadavg: 8.36 8.19 8.06 9/190 10827
/proc/meminfo: memFree=12773660/32951124 swapFree=67111352/67111528
[pid=10746] ppid=10744 vsize=6321260 CPUtime=4710.98 cores=0,2,4,6
/proc/10746/stat : 10746 (ttsth-5-4) S 10744 10746 10639 0 -1 4202496 1513836 0 0 0 469267 1831 0 0 25 0 5 0 521508543 6472970240 1509274 33554432000 4194304 4338100 140734051659968 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 6 0 0 0
/proc/10746/statm: 1580315 1509274 176 36 0 1578220 0
[pid=10746/tid=10749] ppid=10744 vsize=6321260 CPUtime=1181.03 cores=0,2,4,6
/proc/10746/task/10749/stat : 10749 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 728541 0 0 0 117865 238 0 0 25 0 5 0 521508607 6472970240 1509274 33554432000 4194304 4338100 140734051659968 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10746/tid=10750] ppid=10744 vsize=6321260 CPUtime=1178.84 cores=0,2,4,6
/proc/10746/task/10750/stat : 10750 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 102609 0 0 0 116878 1006 0 0 25 0 5 0 521508607 6472970240 1509274 33554432000 4194304 4338100 140734051659968 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10746/tid=10751] ppid=10744 vsize=6321260 CPUtime=1177.78 cores=0,2,4,6
/proc/10746/task/10751/stat : 10751 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 533809 0 0 0 117663 115 0 0 25 0 5 0 521508607 6472970240 1509274 33554432000 4194304 4338100 140734051659968 18446744073709551615 246516229562 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=10746/tid=10752] ppid=10744 vsize=6321260 CPUtime=1172.64 cores=0,2,4,6
/proc/10746/task/10752/stat : 10752 (ttsth-5-4) R 10744 10746 10639 0 -1 4202560 124402 0 0 0 116798 466 0 0 25 0 5 0 521508607 6472970240 1509275 33554432000 4194304 4338100 140734051659968 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4710.98
Current children cumulated vsize (KiB) 6321260



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1204.7 s]
/proc/loadavg: 8.34 8.19 8.06 6/187 10828
/proc/meminfo: memFree=14568044/32951124 swapFree=67111352/67111528
[pid=10746] ppid=10744 vsize=0 CPUtime=4800.13 cores=0,2,4,6
/proc/10746/stat : 10746 (ttsth-5-4) Z 10744 10746 10639 0 -1 4203548 1518334 0 0 0 478095 1918 0 0 18 0 2 0 521508543 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 6 0 0 0
/proc/10746/statm: 0 0 0 0 0 0 0
[pid=10746/tid=10752] ppid=10744 vsize=0 CPUtime=1195.01 cores=0,2,4,6
/proc/10746/task/10752/stat : 10752 (ttsth-5-4) R 10744 10746 10639 0 -1 4203588 124404 0 0 0 118988 513 0 0 25 0 2 0 521508607 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.13
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1204.81 s]
/proc/loadavg: 8.34 8.19 8.06 6/187 10828
/proc/meminfo: memFree=14568044/32951124 swapFree=67111352/67111528
[pid=10746] ppid=10744 vsize=0 CPUtime=4800.24 cores=0,2,4,6
/proc/10746/stat : 10746 (ttsth-5-4) Z 10744 10746 10639 0 -1 4203548 1518334 0 0 0 478095 1929 0 0 18 0 2 0 521508543 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 6 0 0 0
/proc/10746/statm: 0 0 0 0 0 0 0
[pid=10746/tid=10752] ppid=10744 vsize=0 CPUtime=1195.12 cores=0,2,4,6
/proc/10746/task/10752/stat : 10752 (ttsth-5-4) R 10744 10746 10639 0 -1 4203588 124404 0 0 0 118988 524 0 0 25 0 2 0 521508607 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4800.24
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 1204.86
CPU time (s): 4800.3
CPU user time (s): 4780.96
CPU system time (s): 19.3391
CPU usage (%): 398.412
Max. virtual memory (cumulated for all children) (KiB): 6336900

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4780.96
system time used= 19.3391
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1518334
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 12539
involuntary context switches= 71956

runsolver used 2.02669 second user time and 4.55931 second system time

The end

Launcher Data 

Begin job on node134 at 2011-04-29 19:45:04
IDJOB=3264202
IDBENCH=84560
IDSOLVER=1654
FILE ID=node134/3264202-1304099103
RUNJOBID= node134-1304097856-10657
PBS_JOBID= 13170616
Free space on /tmp= 71584 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT04/handmade/kexu/frb-0.8/frb59-26-2.used-as.sat04-892.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3264202-1304099103/watcher-3264202-1304099103 -o /tmp/evaluation-result-3264202-1304099103/solver-3264202-1304099103 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3264202-1304099103.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= fd8d5783ed404e5c6590d627ffdcca50
RANDOM SEED=490529117

node134.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 7186.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      18849868 kB
Buffers:       1491912 kB
Cached:        5857232 kB
SwapCached:         64 kB
Active:        7622464 kB
Inactive:      5946020 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      18849868 kB
SwapTotal:    67111528 kB
SwapFree:     67111352 kB
Dirty:            1576 kB
Writeback:           0 kB
AnonPages:     6219176 kB
Mapped:          15356 kB
Slab:           459040 kB
PageTables:      16976 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:  6551588 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71580 MiB
End job on node134 at 2011-04-29 20:05:10