Trace number 3262134

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.46 1202.06

General information on the benchmark

Namecrafted/kullmann/VanDerWaerden/VanderWaerden_pd_3k/
VanDerWaerden_pd_2-3-19_348.cnf
MD5SUMd56eb1fbc523242359df78e10ae2a44f
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark60.0609
Satisfiable
(Un)Satisfiability was proved
Number of variables174
Number of clauses16458
Sum of the clauses size73267
Maximum clause length19
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 258
Number of clauses of size 314898
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 51502

Solver Data 

0.00/0.01	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3262134-1304091097.cnf 
0.00/0.01	c Using SATTIMEOUT of 4800
0.00/0.01	c Using SATRAM of 15500
0.00/0.01	c Size of subset node = 32
0.00/0.01	c Variables = 174
0.00/0.01	c Clauses = 16458
0.00/0.01	c Literals = 73267
0.00/0.01	c Reduced size:
0.00/0.01	c Variables = 174
0.00/0.01	c Clauses = 16458
0.00/0.01	c Literals = 73267
0.00/0.01	c a2
0.00/0.09	c a1
0.19/0.21	c Independent sub-problems = 1
0.19/0.21	c Sub-problem with 174 variables and 16458 clauses
0.19/0.21	c using permutation method 4
0.19/0.21	c count = 1
0.19/0.21	c Ordering finished 0.200 seconds
0.19/0.21	c mapped 0.200 seconds
0.19/0.21	c permuted 0.200 seconds
0.19/0.21	c about to sort
0.19/0.21	c sorted
0.19/0.26	c clauses now 16458
0.19/0.26	c init finished 0.210 seconds
0.19/0.26	c tree built 0.250 seconds
0.19/0.26	c 
0.19/0.26	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.19/0.26	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.19/0.26	c ................................ | ..........                    |                                  0.2         0.0             
0.19/0.26	c ................................ | ..........                  1 |                                  0.2         4.0             
0.19/0.26	c ................................ | ..........                  2 |                                  0.2         8.0             
0.19/0.26	c ................................ | ..........                  3 |                                  0.2        12.0             
0.19/0.26	c ................................ | ..........                  4 |                                  0.2        16.0             
0.19/0.26	c ................................ | ..........                  5 |                                  0.2        20.0             
0.19/0.27	c ................................ | ..........               1000 | 10           419                 0.3      3225.8        18960
96.27/24.31	c ................................ | ..........             466000 | 34642        223702             96.3      4838.0     20148077
192.16/48.38	c ................................ | ..........             667000 | 56474        331781            192.5      3465.3     31065953
288.42/72.45	c ................................ | ..........             797000 | 73360        411787            288.6      2761.2     39152648
384.28/96.42	c ................................ | ..........             952000 | 86673        479280            384.4      2476.7     45578276
480.56/120.51	c ................................ | *.........            1091000 | 98183        543935            480.6      2269.9     51868543
576.03/144.43	c ................................ | *.........            1216000 | 109739       597201            576.2      2110.5     57815374
672.30/168.51	c ................................ | *.........            1335000 | 121333       642516            672.4      1985.6     63301527
768.20/192.51	c ................................ | *.........            1441000 | 133182       687140            768.2      1875.7     67786240
864.46/216.69	c ................................ | *.........            1540000 | 140615       734460            864.8      1780.7     71667719
960.32/240.63	c ................................ | **........            1631000 | 149533       774764            960.5      1698.1     75601479
1056.19/264.63	c ................................ | **........            1720000 | 158988       811098           1056.3      1628.2     79889345
1152.46/288.72	c ................................ | **........            1793000 | 168839       847237           1152.5      1555.7     83543690
1249.13/312.93	c ................................ | **........            1864000 | 178200       879454           1249.2      1492.1     87110444
1344.61/336.85	c ................................ | **........            1947000 | 187606       909865           1344.8      1447.8     90406278
1440.09/360.76	c ................................ | ***.......            2014000 | 193937       939227           1440.3      1398.3     93823708
1536.76/384.95	c ................................ | ***.......            2080000 | 201461       970324           1537.0      1353.3     95423350
1631.82/408.79	c ................................ | ***.......            2139000 | 208892       1002648          1632.2      1310.5     97958793
1727.70/432.77	c ................................ | ***.......            2199000 | 216631       1034490          1728.0      1272.6    101156108
1824.33/456.94	c ................................ | ***.......            2261000 | 225366       1067420          1824.5      1239.2    104170937
1920.20/480.93	c ................................ | ****......            2335000 | 230558       1100003          1920.3      1215.9    104170937
2016.87/505.17	c ................................ | ****......            2405000 | 235350       1134975          2017.2      1192.3    104170937
2112.35/529.06	c ................................ | ****......            2479000 | 240511       1166957          2112.6      1173.4    104170937
2207.82/552.98	c ................................ | ****......            2557000 | 245689       1196671          2208.2      1158.0    104358856
2304.89/577.21	c ................................ | ****......            2622000 | 251549       1225970          2305.0      1137.5    105243397
2400.76/601.26	c ................................ | *****.....            2693000 | 257174       1254198          2401.0      1121.6    107688328
2497.03/625.32	c ................................ | *****.....            2760000 | 263029       1280270          2497.1      1105.3    109921091
2593.30/649.43	c ................................ | *****.....            2822000 | 267579       1303847          2593.4      1088.1    111993211
2688.77/673.32	c ................................ | *****.....            2875000 | 272163       1329933          2688.9      1069.2    114459539
2785.07/697.41	c ................................ | *****.....            2939000 | 276289       1355812          2785.1      1055.3    116963624
2880.92/721.43	c ................................ | ******....            3009000 | 281193       1382848          2881.1      1044.4    119651960
2976.00/745.25	c ................................ | ******....            3077000 | 286785       1403358          2976.2      1033.9    122400310
3072.66/769.44	c ................................ | ******....            3149000 | 290748       1431478          3072.9      1024.8    125398065
3168.93/793.56	c ................................ | ******....            3211000 | 294637       1459474          3169.2      1013.2    127408288
3264.81/817.58	c ................................ | ******....            3267000 | 298882       1486527          3265.2      1000.6    129813545
3360.70/841.55	c ................................ | *******...            3331000 | 304028       1512376          3360.9       991.1    132588631
3456.10/865.46	c ................................ | *******...            3397000 | 309048       1535438          3456.3       982.8    135131036
3552.37/889.50	c ................................ | *******...            3462000 | 315251       1557122          3552.4       974.6    137238871
3648.25/913.56	c ................................ | *******...            3521000 | 320168       1580771          3648.5       965.1    139503006
3744.91/937.76	c ................................ | *******...            3579000 | 325370       1603042          3745.2       955.6    140588510
3839.98/961.52	c ................................ | ********..            3639000 | 330845       1625621          3840.1       947.6    140602613
3936.28/985.60	c ................................ | ********..            3694000 | 336351       1647329          3936.3       938.4    140602613
4032.13/1009.64	c ................................ | ********..            3742000 | 342103       1664372          4032.3       928.0    140602613
4129.62/1034.02	c ................................ | ********..            3788000 | 346958       1687404          4129.7       917.3    140602613
4224.27/1057.73	c ................................ | ********..            3846000 | 352767       1710944          4224.4       910.4    140602613
4320.15/1081.78	c ................................ | *********.            3896000 | 358504       1733243          4320.5       901.8    140602613
4416.81/1105.99	c ................................ | *********.            3947000 | 362745       1754664          4417.2       893.6    140617910
4513.49/1130.14	c ................................ | *********.            3999000 | 366607       1774646          4513.7       886.0    141665523
4609.76/1154.24	c ................................ | *********.            4049000 | 370533       1795927          4609.9       878.3    143092012
4704.44/1177.99	c ................................ | *********.            4088000 | 374025       1815985          4704.8       868.9    144692137
4799.90/1201.89	c CPU limit (4800) reached
4799.90/1201.89	c Time = 4800.28 seconds
4799.90/1201.89	s UNKNOWN

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: UNKNOWN

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3262134-1304091097/watcher-3262134-1304091097 -o /tmp/evaluation-result-3262134-1304091097/solver-3262134-1304091097 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3262134-1304091097.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.33 6.01 7.01 2/176 1713
/proc/meminfo: memFree=30925892/32951124 swapFree=67105204/67111528
[pid=1711] ppid=1709 vsize=9112 CPUtime=0 cores=1,3,5,7
/proc/1711/stat : 1711 (ttsth-5-4) R 1709 1711 1676 0 -1 4202496 356 0 0 0 0 0 0 0 18 0 1 0 501525631 9330688 298 33554432000 4194304 4338100 140735696134160 18446744073709551615 220815897285 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/1711/statm: 2278 300 146 36 0 183 0

[startup+0.0975849 s]
/proc/loadavg: 3.33 6.01 7.01 2/176 1713
/proc/meminfo: memFree=30925892/32951124 swapFree=67105204/67111528
[pid=1711] ppid=1709 vsize=20876 CPUtime=0.09 cores=1,3,5,7
/proc/1711/stat : 1711 (ttsth-5-4) R 1709 1711 1676 0 -1 4202496 2762 0 0 0 9 0 0 0 18 0 1 0 501525631 21377024 2661 33554432000 4194304 4338100 140735696134160 18446744073709551615 4228108 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/1711/statm: 5219 2662 313 36 0 3124 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 20876

[startup+0.100585 s]
/proc/loadavg: 3.33 6.01 7.01 2/176 1713
/proc/meminfo: memFree=30925892/32951124 swapFree=67105204/67111528
[pid=1711] ppid=1709 vsize=11348 CPUtime=0.09 cores=1,3,5,7
/proc/1711/stat : 1711 (ttsth-5-4) R 1709 1711 1676 0 -1 4202496 2813 0 0 0 9 0 0 0 18 0 1 0 501525631 11620352 841 33554432000 4194304 4338100 140735696134160 18446744073709551615 4245205 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/1711/statm: 2837 841 154 36 0 742 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 11348

[startup+0.300513 s]
/proc/loadavg: 3.33 6.01 7.01 2/176 1713
/proc/meminfo: memFree=30925892/32951124 swapFree=67105204/67111528
[pid=1711] ppid=1709 vsize=144764 CPUtime=0.4 cores=1,3,5,7
/proc/1711/stat : 1711 (ttsth-5-4) S 1709 1711 1676 0 -1 4202496 7015 0 0 0 39 1 0 0 18 0 5 0 501525631 148238336 5010 33554432000 4194304 4338100 140735696134160 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/1711/statm: 36191 5011 175 36 0 34096 0
Current children cumulated CPU time (s) 0.4
Current children cumulated vsize (KiB) 144764

[startup+0.700441 s]
/proc/loadavg: 3.33 6.01 7.01 2/176 1713
/proc/meminfo: memFree=30925892/32951124 swapFree=67105204/67111528
[pid=1711] ppid=1709 vsize=164324 CPUtime=2 cores=1,3,5,7
/proc/1711/stat : 1711 (ttsth-5-4) S 1709 1711 1676 0 -1 4202496 13256 0 0 0 198 2 0 0 18 0 5 0 501525631 168267776 11251 33554432000 4194304 4338100 140735696134160 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/1711/statm: 41081 11251 175 36 0 38986 0
Current children cumulated CPU time (s) 2
Current children cumulated vsize (KiB) 164324

[startup+1.50029 s]
/proc/loadavg: 3.33 6.01 7.01 11/186 1723
/proc/meminfo: memFree=30798480/32951124 swapFree=67105204/67111528
[pid=1711] ppid=1709 vsize=208776 CPUtime=5.19 cores=1,3,5,7
/proc/1711/stat : 1711 (ttsth-5-4) S 1709 1711 1676 0 -1 4202496 23788 0 0 0 515 4 0 0 18 0 5 0 501525631 213786624 21783 33554432000 4194304 4338100 140735696134160 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/1711/statm: 52194 21784 175 36 0 50099 0
[pid=1711/tid=1716] ppid=1709 vsize=208776 CPUtime=1.22 cores=1,3,5,7
/proc/1711/task/1716/stat : 1716 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 4106 0 0 0 122 0 0 0 19 0 5 0 501525657 213786624 21785 33554432000 4194304 4338100 140735696134160 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=1711/tid=1717] ppid=1709 vsize=208776 CPUtime=1.23 cores=1,3,5,7
/proc/1711/task/1717/stat : 1717 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 4355 0 0 0 122 1 0 0 19 0 5 0 501525657 213786624 21786 33554432000 4194304 4338100 140735696134160 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=1711/tid=1718] ppid=1709 vsize=208776 CPUtime=1.23 cores=1,3,5,7
/proc/1711/task/1718/stat : 1718 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 3714 0 0 0 123 0 0 0 19 0 5 0 501525657 213786624 21787 33554432000 4194304 4338100 140735696134160 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=1711/tid=1719] ppid=1709 vsize=208776 CPUtime=1.21 cores=1,3,5,7
/proc/1711/task/1719/stat : 1719 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 5412 0 0 0 121 0 0 0 22 0 5 0 501525657 213786624 21787 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 5.19
Current children cumulated vsize (KiB) 208776

[startup+3.101 s]
/proc/loadavg: 3.33 6.01 7.01 10/186 1723
/proc/meminfo: memFree=30682920/32951124 swapFree=67105204/67111528
[pid=1711] ppid=1709 vsize=271996 CPUtime=11.59 cores=1,3,5,7
/proc/1711/stat : 1711 (ttsth-5-4) S 1709 1711 1676 0 -1 4202496 40341 0 0 0 1151 8 0 0 18 0 5 0 501525631 278523904 38336 33554432000 4194304 4338100 140735696134160 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/1711/statm: 67999 38336 175 36 0 65904 0
[pid=1711/tid=1716] ppid=1709 vsize=271996 CPUtime=2.82 cores=1,3,5,7
/proc/1711/task/1716/stat : 1716 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 8684 0 0 0 281 1 0 0 24 0 5 0 501525657 278523904 38336 33554432000 4194304 4338100 140735696134160 18446744073709551615 4245241 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=1711/tid=1717] ppid=1709 vsize=271996 CPUtime=2.83 cores=1,3,5,7
/proc/1711/task/1717/stat : 1717 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 8610 0 0 0 281 2 0 0 24 0 5 0 501525657 278523904 38336 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=1711/tid=1718] ppid=1709 vsize=271996 CPUtime=2.83 cores=1,3,5,7
/proc/1711/task/1718/stat : 1718 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 6805 0 0 0 282 1 0 0 24 0 5 0 501525657 278523904 38336 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=1711/tid=1719] ppid=1709 vsize=271996 CPUtime=2.81 cores=1,3,5,7
/proc/1711/task/1719/stat : 1719 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 10037 0 0 0 280 1 0 0 25 0 5 0 501525657 278523904 38336 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 11.59
Current children cumulated vsize (KiB) 271996

[startup+6.30041 s]
/proc/loadavg: 3.70 6.05 7.01 10/186 1723
/proc/meminfo: memFree=30393256/32951124 swapFree=67105204/67111528
[pid=1711] ppid=1709 vsize=376276 CPUtime=24.37 cores=1,3,5,7
/proc/1711/stat : 1711 (ttsth-5-4) S 1709 1711 1676 0 -1 4202496 65370 0 0 0 2423 14 0 0 18 0 5 0 501525631 385306624 63365 33554432000 4194304 4338100 140735696134160 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/1711/statm: 94069 63365 175 36 0 91974 0
[pid=1711/tid=1716] ppid=1709 vsize=376276 CPUtime=6.02 cores=1,3,5,7
/proc/1711/task/1716/stat : 1716 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 14597 0 0 0 600 2 0 0 25 0 5 0 501525657 385306624 63366 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254418 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=1711/tid=1717] ppid=1709 vsize=376276 CPUtime=6.03 cores=1,3,5,7
/proc/1711/task/1717/stat : 1717 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 15412 0 0 0 599 4 0 0 25 0 5 0 501525657 385306624 63367 33554432000 4194304 4338100 140735696134160 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=1711/tid=1718] ppid=1709 vsize=376276 CPUtime=6.02 cores=1,3,5,7
/proc/1711/task/1718/stat : 1718 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 12334 0 0 0 600 2 0 0 25 0 5 0 501525657 385306624 63367 33554432000 4194304 4338100 140735696134160 18446744073709551615 220828049102 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=1711/tid=1719] ppid=1709 vsize=376276 CPUtime=6 cores=1,3,5,7
/proc/1711/task/1719/stat : 1719 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 16824 0 0 0 597 3 0 0 25 0 5 0 501525657 385306624 63367 33554432000 4194304 4338100 140735696134160 18446744073709551615 4279536 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 24.37
Current children cumulated vsize (KiB) 376276

[startup+12.7002 s]
/proc/loadavg: 4.04 6.08 7.02 10/186 1723
/proc/meminfo: memFree=29948440/32951124 swapFree=67105204/67111528
[pid=1711] ppid=1709 vsize=546376 CPUtime=49.93 cores=1,3,5,7
/proc/1711/stat : 1711 (ttsth-5-4) S 1709 1711 1676 0 -1 4202496 108772 0 0 0 4971 22 0 0 18 0 5 0 501525631 559489024 106767 33554432000 4194304 4338100 140735696134160 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/1711/statm: 136594 106767 175 36 0 134499 0
[pid=1711/tid=1716] ppid=1709 vsize=546376 CPUtime=12.43 cores=1,3,5,7
/proc/1711/task/1716/stat : 1716 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 24022 0 0 0 1239 4 0 0 25 0 5 0 501525657 559489024 106767 33554432000 4194304 4338100 140735696134160 18446744073709551615 220815897285 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=1711/tid=1717] ppid=1709 vsize=546376 CPUtime=12.43 cores=1,3,5,7
/proc/1711/task/1717/stat : 1717 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 28221 0 0 0 1236 7 0 0 25 0 5 0 501525657 559489024 106767 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=1711/tid=1718] ppid=1709 vsize=546376 CPUtime=12.42 cores=1,3,5,7
/proc/1711/task/1718/stat : 1718 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 22695 0 0 0 1238 4 0 0 25 0 5 0 501525657 559489024 106767 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254221 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=1711/tid=1719] ppid=1709 vsize=546376 CPUtime=12.36 cores=1,3,5,7
/proc/1711/task/1719/stat : 1719 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 27629 0 0 0 1231 5 0 0 25 0 5 0 501525657 559489024 106767 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 49.93
Current children cumulated vsize (KiB) 546376

[startup+25.5009 s]
/proc/loadavg: 4.92 6.17 7.03 10/186 1723
/proc/meminfo: memFree=29295000/32951124 swapFree=67105204/67111528
[pid=1711] ppid=1709 vsize=791680 CPUtime=101.06 cores=1,3,5,7
/proc/1711/stat : 1711 (ttsth-5-4) S 1709 1711 1676 0 -1 4202496 170826 0 0 0 10072 34 0 0 18 0 5 0 501525631 810680320 168821 33554432000 4194304 4338100 140735696134160 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/1711/statm: 197920 168821 175 36 0 195825 0
[pid=1711/tid=1716] ppid=1709 vsize=791680 CPUtime=25.23 cores=1,3,5,7
/proc/1711/task/1716/stat : 1716 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 39638 0 0 0 2516 7 0 0 25 0 5 0 501525657 810680320 168821 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254208 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=1711/tid=1717] ppid=1709 vsize=791680 CPUtime=25.23 cores=1,3,5,7
/proc/1711/task/1717/stat : 1717 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 43107 0 0 0 2513 10 0 0 25 0 5 0 501525657 810680320 168821 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=1711/tid=1718] ppid=1709 vsize=791680 CPUtime=25.23 cores=1,3,5,7
/proc/1711/task/1718/stat : 1718 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 39317 0 0 0 2515 8 0 0 25 0 5 0 501525657 810680320 168821 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=1711/tid=1719] ppid=1709 vsize=791680 CPUtime=25.09 cores=1,3,5,7
/proc/1711/task/1719/stat : 1719 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 42559 0 0 0 2502 7 0 0 25 0 5 0 501525657 810680320 168821 33554432000 4194304 4338100 140735696134160 18446744073709551615 4253966 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 101.06
Current children cumulated vsize (KiB) 791680

[startup+51.1012 s]
/proc/loadavg: 5.97 6.32 7.06 10/186 1724
/proc/meminfo: memFree=28397672/32951124 swapFree=67105204/67111528
[pid=1711] ppid=1709 vsize=1153248 CPUtime=203.33 cores=1,3,5,7
/proc/1711/stat : 1711 (ttsth-5-4) S 1709 1711 1676 0 -1 4202496 260553 0 0 0 20281 52 0 0 18 0 5 0 501525631 1180925952 258548 33554432000 4194304 4338100 140735696134160 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/1711/statm: 288312 258548 175 36 0 286217 0
[pid=1711/tid=1716] ppid=1709 vsize=1153248 CPUtime=50.83 cores=1,3,5,7
/proc/1711/task/1716/stat : 1716 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 61592 0 0 0 5072 11 0 0 25 0 5 0 501525657 1180925952 258548 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=1711/tid=1717] ppid=1709 vsize=1153248 CPUtime=50.83 cores=1,3,5,7
/proc/1711/task/1717/stat : 1717 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 60423 0 0 0 5069 14 0 0 25 0 5 0 501525657 1180925952 258548 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=1711/tid=1718] ppid=1709 vsize=1153248 CPUtime=50.83 cores=1,3,5,7
/proc/1711/task/1718/stat : 1718 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 66578 0 0 0 5069 14 0 0 25 0 5 0 501525657 1180925952 258548 33554432000 4194304 4338100 140735696134160 18446744073709551615 4257000 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=1711/tid=1719] ppid=1709 vsize=1153248 CPUtime=50.55 cores=1,3,5,7
/proc/1711/task/1719/stat : 1719 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 65755 0 0 0 5044 11 0 0 25 0 5 0 501525657 1180925952 258548 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254429 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 203.33
Current children cumulated vsize (KiB) 1153248

[startup+102.307 s]
/proc/loadavg: 7.19 6.60 7.11 10/186 1725
/proc/meminfo: memFree=27020256/32951124 swapFree=67105204/67111528
[pid=1711] ppid=1709 vsize=1628216 CPUtime=407.88 cores=1,3,5,7
/proc/1711/stat : 1711 (ttsth-5-4) S 1709 1711 1676 0 -1 4202496 380566 0 0 0 40708 80 0 0 18 0 5 0 501525631 1667293184 378561 33554432000 4194304 4338100 140735696134160 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/1711/statm: 407054 378561 175 36 0 404959 0
[pid=1711/tid=1716] ppid=1709 vsize=1628216 CPUtime=102.04 cores=1,3,5,7
/proc/1711/task/1716/stat : 1716 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 91642 0 0 0 10187 17 0 0 25 0 5 0 501525657 1667293184 378561 33554432000 4194304 4338100 140735696134160 18446744073709551615 220815631624 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=1711/tid=1717] ppid=1709 vsize=1628216 CPUtime=102.04 cores=1,3,5,7
/proc/1711/task/1717/stat : 1717 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 88968 0 0 0 10183 21 0 0 25 0 5 0 501525657 1667293184 378561 33554432000 4194304 4338100 140735696134160 18446744073709551615 4284177 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=1711/tid=1718] ppid=1709 vsize=1628216 CPUtime=102.03 cores=1,3,5,7
/proc/1711/task/1718/stat : 1718 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 91332 0 0 0 10184 19 0 0 25 0 5 0 501525657 1667293184 378561 33554432000 4194304 4338100 140735696134160 18446744073709551615 4255920 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=1711/tid=1719] ppid=1709 vsize=1628216 CPUtime=101.48 cores=1,3,5,7
/proc/1711/task/1719/stat : 1719 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 102419 0 0 0 10127 21 0 0 25 0 5 0 501525657 1667293184 378561 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 407.88
Current children cumulated vsize (KiB) 1628216

[startup+162.301 s]
/proc/loadavg: 7.77 6.88 7.18 10/186 1727
/proc/meminfo: memFree=25820512/32951124 swapFree=67105204/67111528
[pid=1711] ppid=1709 vsize=2096400 CPUtime=647.53 cores=1,3,5,7
/proc/1711/stat : 1711 (ttsth-5-4) S 1709 1711 1676 0 -1 4202496 497426 0 0 0 64642 111 0 0 18 0 5 0 501525631 2146713600 495421 33554432000 4194304 4338100 140735696134160 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/1711/statm: 524100 495421 175 36 0 522005 0
[pid=1711/tid=1716] ppid=1709 vsize=2096400 CPUtime=162.03 cores=1,3,5,7
/proc/1711/task/1716/stat : 1716 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 121675 0 0 0 16178 25 0 0 25 0 5 0 501525657 2146713600 495421 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=1711/tid=1717] ppid=1709 vsize=2096400 CPUtime=162.03 cores=1,3,5,7
/proc/1711/task/1717/stat : 1717 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 121753 0 0 0 16174 29 0 0 25 0 5 0 501525657 2146713600 495421 33554432000 4194304 4338100 140735696134160 18446744073709551615 4240516 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=1711/tid=1718] ppid=1709 vsize=2096400 CPUtime=161.93 cores=1,3,5,7
/proc/1711/task/1718/stat : 1718 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 107739 0 0 0 16169 24 0 0 25 0 5 0 501525657 2146713600 495421 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=1711/tid=1719] ppid=1709 vsize=2096400 CPUtime=161.24 cores=1,3,5,7
/proc/1711/task/1719/stat : 1719 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 140054 0 0 0 16094 30 0 0 25 0 5 0 501525657 2146713600 495421 33554432000 4194304 4338100 140735696134160 18446744073709551615 4251295 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 647.53
Current children cumulated vsize (KiB) 2096400


################
# More data... #
################

/proc/1711/task/1719/stat : 1719 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 274009 0 0 0 70058 67 0 0 25 0 5 0 501525657 4211666944 969501 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 2804.62
Current children cumulated vsize (KiB) 4112956

[startup+762.307 s]
/proc/loadavg: 8.13 7.92 7.58 10/186 1743
/proc/meminfo: memFree=19468460/32951124 swapFree=67105204/67111528
[pid=1711] ppid=1709 vsize=4329712 CPUtime=3044.32 cores=1,3,5,7
/proc/1711/stat : 1711 (ttsth-5-4) S 1709 1711 1676 0 -1 4202496 1026641 0 0 0 304171 261 0 0 18 0 5 0 501525631 4433625088 1024636 33554432000 4194304 4338100 140735696134160 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/1711/statm: 1082428 1024636 176 36 0 1080333 0
[pid=1711/tid=1716] ppid=1709 vsize=4329712 CPUtime=762.04 cores=1,3,5,7
/proc/1711/task/1716/stat : 1716 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 226561 0 0 0 76148 56 0 0 25 0 5 0 501525657 4433625088 1024636 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=1711/tid=1717] ppid=1709 vsize=4329712 CPUtime=762.06 cores=1,3,5,7
/proc/1711/task/1717/stat : 1717 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 278061 0 0 0 76136 70 0 0 25 0 5 0 501525657 4433625088 1024636 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=1711/tid=1718] ppid=1709 vsize=4329712 CPUtime=758.68 cores=1,3,5,7
/proc/1711/task/1718/stat : 1718 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 229222 0 0 0 75806 62 0 0 25 0 5 0 501525657 4433625088 1024636 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=1711/tid=1719] ppid=1709 vsize=4329712 CPUtime=761.24 cores=1,3,5,7
/proc/1711/task/1719/stat : 1719 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 286592 0 0 0 76054 70 0 0 25 0 5 0 501525657 4433625088 1024636 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3044.32
Current children cumulated vsize (KiB) 4329712

[startup+822.301 s]
/proc/loadavg: 8.13 7.96 7.62 10/186 1745
/proc/meminfo: memFree=19064676/32951124 swapFree=67105204/67111528
[pid=1711] ppid=1709 vsize=4515320 CPUtime=3283.99 cores=1,3,5,7
/proc/1711/stat : 1711 (ttsth-5-4) S 1709 1711 1676 0 -1 4202496 1072594 0 0 0 328122 277 0 0 18 0 5 0 501525631 4623687680 1070589 33554432000 4194304 4338100 140735696134160 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/1711/statm: 1128830 1070589 176 36 0 1126735 0
[pid=1711/tid=1716] ppid=1709 vsize=4515320 CPUtime=822.03 cores=1,3,5,7
/proc/1711/task/1716/stat : 1716 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 240924 0 0 0 82142 61 0 0 25 0 5 0 501525657 4623687680 1070589 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=1711/tid=1717] ppid=1709 vsize=4515320 CPUtime=822.05 cores=1,3,5,7
/proc/1711/task/1717/stat : 1717 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 291467 0 0 0 82132 73 0 0 25 0 5 0 501525657 4623687680 1070589 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=1711/tid=1718] ppid=1709 vsize=4515320 CPUtime=818.35 cores=1,3,5,7
/proc/1711/task/1718/stat : 1718 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 236745 0 0 0 81770 65 0 0 25 0 5 0 501525657 4623687680 1070589 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=1711/tid=1719] ppid=1709 vsize=4515320 CPUtime=821.24 cores=1,3,5,7
/proc/1711/task/1719/stat : 1719 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 297253 0 0 0 82050 74 0 0 25 0 5 0 501525657 4623687680 1070589 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3283.99
Current children cumulated vsize (KiB) 4515320

[startup+882.301 s]
/proc/loadavg: 8.12 7.99 7.65 10/186 1746
/proc/meminfo: memFree=18665624/32951124 swapFree=67105204/67111528
[pid=1711] ppid=1709 vsize=4703660 CPUtime=3523.61 cores=1,3,5,7
/proc/1711/stat : 1711 (ttsth-5-4) S 1709 1711 1676 0 -1 4202496 1120777 0 0 0 352070 291 0 0 18 0 5 0 501525631 4816547840 1118772 33554432000 4194304 4338100 140735696134160 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/1711/statm: 1175915 1118772 176 36 0 1173820 0
[pid=1711/tid=1716] ppid=1709 vsize=4703660 CPUtime=882.03 cores=1,3,5,7
/proc/1711/task/1716/stat : 1716 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 257244 0 0 0 88138 65 0 0 25 0 5 0 501525657 4816547840 1118772 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=1711/tid=1717] ppid=1709 vsize=4703660 CPUtime=882.05 cores=1,3,5,7
/proc/1711/task/1717/stat : 1717 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 300105 0 0 0 88130 75 0 0 25 0 5 0 501525657 4816547840 1118772 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=1711/tid=1718] ppid=1709 vsize=4703660 CPUtime=878.01 cores=1,3,5,7
/proc/1711/task/1718/stat : 1718 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 245547 0 0 0 87732 69 0 0 25 0 5 0 501525657 4816547840 1118772 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254342 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=1711/tid=1719] ppid=1709 vsize=4703660 CPUtime=881.23 cores=1,3,5,7
/proc/1711/task/1719/stat : 1719 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 311676 0 0 0 88044 79 0 0 25 0 5 0 501525657 4816547840 1118772 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254294 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3523.61
Current children cumulated vsize (KiB) 4703660

[startup+942.301 s]
/proc/loadavg: 8.04 7.99 7.67 10/186 1748
/proc/meminfo: memFree=18446724/32951124 swapFree=67105204/67111528
[pid=1711] ppid=1709 vsize=4838824 CPUtime=3763.29 cores=1,3,5,7
/proc/1711/stat : 1711 (ttsth-5-4) S 1709 1711 1676 0 -1 4202496 1153848 0 0 0 376027 302 0 0 18 0 5 0 501525631 4954955776 1151843 33554432000 4194304 4338100 140735696134160 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/1711/statm: 1209706 1151843 176 36 0 1207611 0
[pid=1711/tid=1716] ppid=1709 vsize=4838824 CPUtime=942.03 cores=1,3,5,7
/proc/1711/task/1716/stat : 1716 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 267018 0 0 0 94135 68 0 0 25 0 5 0 501525657 4954955776 1151843 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254362 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=1711/tid=1717] ppid=1709 vsize=4838824 CPUtime=942.05 cores=1,3,5,7
/proc/1711/task/1717/stat : 1717 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 307080 0 0 0 94128 77 0 0 25 0 5 0 501525657 4954955776 1151843 33554432000 4194304 4338100 140735696134160 18446744073709551615 4286315 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=1711/tid=1718] ppid=1709 vsize=4838824 CPUtime=937.69 cores=1,3,5,7
/proc/1711/task/1718/stat : 1718 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 251679 0 0 0 93698 71 0 0 25 0 5 0 501525657 4954955776 1151843 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=1711/tid=1719] ppid=1709 vsize=4838824 CPUtime=941.22 cores=1,3,5,7
/proc/1711/task/1719/stat : 1719 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 321866 0 0 0 94039 83 0 0 25 0 5 0 501525657 4954955776 1151843 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3763.29
Current children cumulated vsize (KiB) 4838824

[startup+1002.3 s]
/proc/loadavg: 8.01 7.99 7.68 10/186 1749
/proc/meminfo: memFree=18316108/32951124 swapFree=67105204/67111528
[pid=1711] ppid=1709 vsize=4839352 CPUtime=4002.97 cores=1,3,5,7
/proc/1711/stat : 1711 (ttsth-5-4) S 1709 1711 1676 0 -1 4202496 1154296 0 0 0 399994 303 0 0 18 0 5 0 501525631 4955496448 1152291 33554432000 4194304 4338100 140735696134160 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/1711/statm: 1209838 1152291 176 36 0 1207743 0
[pid=1711/tid=1716] ppid=1709 vsize=4839352 CPUtime=1002.03 cores=1,3,5,7
/proc/1711/task/1716/stat : 1716 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 267217 0 0 0 100135 68 0 0 25 0 5 0 501525657 4955496448 1152291 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=1711/tid=1717] ppid=1709 vsize=4839352 CPUtime=1002.06 cores=1,3,5,7
/proc/1711/task/1717/stat : 1717 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 307158 0 0 0 100128 78 0 0 25 0 5 0 501525657 4955496448 1152291 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=1711/tid=1718] ppid=1709 vsize=4839352 CPUtime=997.36 cores=1,3,5,7
/proc/1711/task/1718/stat : 1718 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 251743 0 0 0 99665 71 0 0 25 0 5 0 501525657 4955496448 1152291 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=1711/tid=1719] ppid=1709 vsize=4839352 CPUtime=1001.22 cores=1,3,5,7
/proc/1711/task/1719/stat : 1719 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 321973 0 0 0 100039 83 0 0 25 0 5 0 501525657 4955496448 1152291 33554432000 4194304 4338100 140735696134160 18446744073709551615 4239484 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4002.97
Current children cumulated vsize (KiB) 4839352

[startup+1062.3 s]
/proc/loadavg: 8.00 7.99 7.70 10/186 1751
/proc/meminfo: memFree=18247480/32951124 swapFree=67105204/67111528
[pid=1711] ppid=1709 vsize=4839880 CPUtime=4242.64 cores=1,3,5,7
/proc/1711/stat : 1711 (ttsth-5-4) S 1709 1711 1676 0 -1 4202496 1154664 0 0 0 423961 303 0 0 18 0 5 0 501525631 4956037120 1152659 33554432000 4194304 4338100 140735696134160 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/1711/statm: 1209970 1152659 176 36 0 1207875 0
[pid=1711/tid=1716] ppid=1709 vsize=4839880 CPUtime=1062.03 cores=1,3,5,7
/proc/1711/task/1716/stat : 1716 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 267317 0 0 0 106135 68 0 0 25 0 5 0 501525657 4956037120 1152659 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254347 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=1711/tid=1717] ppid=1709 vsize=4839880 CPUtime=1062.06 cores=1,3,5,7
/proc/1711/task/1717/stat : 1717 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 307271 0 0 0 106128 78 0 0 25 0 5 0 501525657 4956037120 1152659 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=1711/tid=1718] ppid=1709 vsize=4839880 CPUtime=1057.04 cores=1,3,5,7
/proc/1711/task/1718/stat : 1718 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 251808 0 0 0 105633 71 0 0 25 0 5 0 501525657 4956037120 1152659 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=1711/tid=1719] ppid=1709 vsize=4839880 CPUtime=1061.22 cores=1,3,5,7
/proc/1711/task/1719/stat : 1719 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 322063 0 0 0 106039 83 0 0 25 0 5 0 501525657 4956037120 1152659 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4242.64
Current children cumulated vsize (KiB) 4839880

[startup+1122.3 s]
/proc/loadavg: 8.11 8.02 7.73 10/186 1753
/proc/meminfo: memFree=18052148/32951124 swapFree=67105204/67111528
[pid=1711] ppid=1709 vsize=4859308 CPUtime=4482.33 cores=1,3,5,7
/proc/1711/stat : 1711 (ttsth-5-4) S 1709 1711 1676 0 -1 4202496 1160236 0 0 0 447927 306 0 0 18 0 5 0 501525631 4975931392 1158231 33554432000 4194304 4338100 140735696134160 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/1711/statm: 1214827 1158231 176 36 0 1212732 0
[pid=1711/tid=1716] ppid=1709 vsize=4859308 CPUtime=1122.03 cores=1,3,5,7
/proc/1711/task/1716/stat : 1716 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 269303 0 0 0 112134 69 0 0 25 0 5 0 501525657 4975931392 1158231 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=1711/tid=1717] ppid=1709 vsize=4859308 CPUtime=1122.06 cores=1,3,5,7
/proc/1711/task/1717/stat : 1717 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 307359 0 0 0 112128 78 0 0 25 0 5 0 501525657 4975931392 1158231 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=1711/tid=1718] ppid=1709 vsize=4859308 CPUtime=1116.71 cores=1,3,5,7
/proc/1711/task/1718/stat : 1718 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 251908 0 0 0 111599 72 0 0 25 0 5 0 501525657 4975931392 1158231 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254429 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=1711/tid=1719] ppid=1709 vsize=4859308 CPUtime=1121.23 cores=1,3,5,7
/proc/1711/task/1719/stat : 1719 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 325461 0 0 0 112039 84 0 0 25 0 5 0 501525657 4975931392 1158231 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4482.33
Current children cumulated vsize (KiB) 4859308

[startup+1182.3 s]
/proc/loadavg: 8.11 8.03 7.75 10/186 1754
/proc/meminfo: memFree=17680612/32951124 swapFree=67105204/67111528
[pid=1711] ppid=1709 vsize=4972444 CPUtime=4722.01 cores=1,3,5,7
/proc/1711/stat : 1711 (ttsth-5-4) S 1709 1711 1676 0 -1 4202496 1189343 0 0 0 471884 317 0 0 18 0 5 0 501525631 5091782656 1187338 33554432000 4194304 4338100 140735696134160 18446744073709551615 220828039141 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/1711/statm: 1243111 1187338 176 36 0 1241016 0
[pid=1711/tid=1716] ppid=1709 vsize=4972444 CPUtime=1182.03 cores=1,3,5,7
/proc/1711/task/1716/stat : 1716 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 278193 0 0 0 118130 73 0 0 25 0 5 0 501525657 5091782656 1187338 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=1711/tid=1717] ppid=1709 vsize=4972444 CPUtime=1182.06 cores=1,3,5,7
/proc/1711/task/1717/stat : 1717 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 315900 0 0 0 118125 81 0 0 25 0 5 0 501525657 5091782656 1187338 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=1711/tid=1718] ppid=1709 vsize=4972444 CPUtime=1176.39 cores=1,3,5,7
/proc/1711/task/1718/stat : 1718 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 251982 0 0 0 117567 72 0 0 25 0 5 0 501525657 5091782656 1187338 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=1711/tid=1719] ppid=1709 vsize=4972444 CPUtime=1181.23 cores=1,3,5,7
/proc/1711/task/1719/stat : 1719 (ttsth-5-4) R 1709 1711 1676 0 -1 4202560 337063 0 0 0 118035 88 0 0 25 0 5 0 501525657 5091782656 1187338 33554432000 4194304 4338100 140735696134160 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4722.01
Current children cumulated vsize (KiB) 4972444



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.9 s]
/proc/loadavg: 8.08 8.03 7.75 7/183 1755
/proc/meminfo: memFree=17625080/32951124 swapFree=67105204/67111528
[pid=1711] ppid=1709 vsize=0 CPUtime=4800.29 cores=1,3,5,7
/proc/1711/stat : 1711 (ttsth-5-4) Z 1709 1711 1676 0 -1 4203548 1200038 0 0 0 479708 321 0 0 18 0 2 0 501525631 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/1711/statm: 0 0 0 0 0 0 0
[pid=1711/tid=1719] ppid=1709 vsize=0 CPUtime=1200.83 cores=1,3,5,7
/proc/1711/task/1719/stat : 1719 (ttsth-5-4) R 1709 1711 1676 0 -1 4203588 339851 0 0 0 119993 90 0 0 25 0 2 0 501525657 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.29
Current children cumulated vsize (KiB) 0

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1202.01 s]
/proc/loadavg: 8.08 8.03 7.75 7/183 1755
/proc/meminfo: memFree=17625080/32951124 swapFree=67105204/67111528
[pid=1711] ppid=1709 vsize=0 CPUtime=4800.4 cores=1,3,5,7
/proc/1711/stat : 1711 (ttsth-5-4) Z 1709 1711 1676 0 -1 4203548 1200038 0 0 0 479708 332 0 0 18 0 2 0 501525631 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/1711/statm: 0 0 0 0 0 0 0
[pid=1711/tid=1719] ppid=1709 vsize=0 CPUtime=1200.94 cores=1,3,5,7
/proc/1711/task/1719/stat : 1719 (ttsth-5-4) R 1709 1711 1676 0 -1 4203588 339851 0 0 0 119993 101 0 0 25 0 2 0 501525657 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.4
Current children cumulated vsize (KiB) 0

Child status: 0
Real time (s): 1202.06
CPU time (s): 4800.46
CPU user time (s): 4797.09
CPU system time (s): 3.37549
CPU usage (%): 399.352
Max. virtual memory (cumulated for all children) (KiB): 5019628

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4797.09
system time used= 3.37549
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1200038
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 648
involuntary context switches= 48204

runsolver used 1.82972 second user time and 4.72528 second system time

The end

Launcher Data 

Begin job on node147 at 2011-04-29 17:31:37
IDJOB=3262134
IDBENCH=83273
IDSOLVER=1654
FILE ID=node147/3262134-1304091097
RUNJOBID= node147-1304091097-1695
PBS_JOBID= 13170578
Free space on /tmp= 72660 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT11/crafted/kullmann/VanDerWaerden/VanderWaerden_pd_3k/VanDerWaerden_pd_2-3-19_348.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3262134-1304091097/watcher-3262134-1304091097 -o /tmp/evaluation-result-3262134-1304091097/solver-3262134-1304091097 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3262134-1304091097.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= d56eb1fbc523242359df78e10ae2a44f
RANDOM SEED=378769244

node147.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      30926328 kB
Buffers:        273196 kB
Cached:        1365216 kB
SwapCached:       2968 kB
Active:         125276 kB
Inactive:      1574060 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      30926328 kB
SwapTotal:    67111528 kB
SwapFree:     67105204 kB
Dirty:             880 kB
Writeback:           0 kB
AnonPages:       60100 kB
Mapped:          14240 kB
Slab:           260508 kB
PageTables:       5044 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   184776 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72656 MiB
End job on node147 at 2011-04-29 17:51:41