Trace number 3250736

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.42 1202.07

General information on the benchmark

Namecrafted/kullmann/VanDerWaerden/
VanderWaerden3k/VanDerWaerden_2-3-15_218.cnf
MD5SUMafc70649cebcc24cd0a7b28def593295
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables218
Number of clauses13362
Sum of the clauses size59166
Maximum clause length15
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 311772
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 51590

Solver Data 

0.00/0.01	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3250736-1304093294.cnf 
0.00/0.01	c Using SATTIMEOUT of 4800
0.00/0.01	c Using SATRAM of 15500
0.00/0.01	c Size of subset node = 32
0.00/0.01	c Variables = 218
0.00/0.01	c Clauses = 13362
0.00/0.01	c Literals = 59166
0.00/0.01	c Reduced size:
0.00/0.01	c Variables = 218
0.00/0.01	c Clauses = 13362
0.00/0.01	c Literals = 59166
0.00/0.01	c a2
0.00/0.07	c a1
0.09/0.14	c Independent sub-problems = 1
0.09/0.14	c Sub-problem with 218 variables and 13362 clauses
0.09/0.14	c using permutation method 4
0.09/0.14	c count = 1
0.09/0.14	c Ordering finished 0.130 seconds
0.09/0.14	c mapped 0.130 seconds
0.09/0.14	c permuted 0.130 seconds
0.09/0.14	c about to sort
0.09/0.14	c sorted
0.09/0.19	c clauses now 13362
0.09/0.19	c init finished 0.140 seconds
0.09/0.19	c tree built 0.180 seconds
0.09/0.19	c 
0.09/0.19	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.09/0.19	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.09/0.19	c ................................ | ..........                    |                                  0.2         0.0             
0.09/0.19	c ................................ | ..........                  1 |                                  0.2         5.6             
0.09/0.19	c ................................ | ..........                  2 |                                  0.2        11.1             
0.09/0.19	c ................................ | ..........                  3 |                                  0.2        16.7             
0.09/0.19	c ................................ | ..........                  4 |                                  0.2        22.2             
0.09/0.19	c ................................ | ..........                  5 |                                  0.2        27.8             
0.21/0.20	c ................................ | ..........               1000 | 34           553                 0.2      4166.7        26568
96.10/24.27	c ................................ | ..........             465000 | 45608        225112             96.4      4824.7     19323898
192.37/48.31	c ................................ | ..........             684000 | 68929        338280            192.4      3554.9     29927659
287.82/72.30	c ................................ | ..........             873000 | 90906        425985            288.2      3028.9     37821574
383.70/96.30	c ................................ | ..........            1020000 | 106863       495397            384.1      2655.6     43867782
480.76/120.50	c ................................ | *.........            1139000 | 120469       560468            480.8      2369.2     49135349
575.83/144.38	c ................................ | *.........            1271000 | 135758       625435            576.2      2205.9     55095891
672.10/168.49	c ................................ | *.........            1390000 | 149330       681066            672.5      2067.0     60679837
768.39/192.57	c ................................ | *.........            1492000 | 159732       731453            768.7      1941.0     66235247
863.84/216.45	c ................................ | *.........            1590000 | 170988       779315            864.1      1840.1     71267323
960.12/240.57	c ................................ | **........            1685000 | 181866       826134            960.4      1754.4     74402022
1056.38/264.64	c ................................ | **........            1769000 | 191797       865053           1056.5      1674.3     78632098
1152.25/288.68	c ................................ | **........            1857000 | 202783       902471           1152.6      1611.2     83685593
1248.93/312.83	c ................................ | **........            1944000 | 213806       945084           1249.1      1556.4     87986545
1344.79/336.80	c ................................ | **........            2034000 | 224141       983139           1344.8      1512.5     91672871
1440.29/360.73	c ................................ | ***.......            2125000 | 234377       1023519          1440.4      1475.3     95703374
1536.95/384.99	c ................................ | ***.......            2199000 | 242415       1060472          1537.3      1430.4     99428908
1632.01/408.76	c ................................ | ***.......            2273000 | 250201       1097488          1632.3      1392.5    103452139
1728.28/432.84	c ................................ | ***.......            2343000 | 257807       1133365          1728.5      1355.5    107094772
1824.95/457.09	c ................................ | ***.......            2404000 | 264781       1167657          1825.3      1317.0    109877760
1920.81/481.05	c ................................ | ****......            2473000 | 273193       1199990          1921.0      1287.3    111552987
2015.90/504.83	c ................................ | ****......            2537000 | 280770       1231866          2016.0      1258.4    114458777
2112.17/528.97	c ................................ | ****......            2598000 | 288641       1261455          2112.5      1229.8    117567998
2208.43/553.05	c ................................ | ****......            2657000 | 295586       1294001          2208.6      1203.0    120475787
2305.11/577.22	c ................................ | ****......            2713000 | 302712       1323736          2305.2      1176.9    123214654
2399.78/600.99	c ................................ | *****.....            2767000 | 309307       1352802          2400.2      1152.8    126284709
2496.44/625.19	c ................................ | *****.....            2824000 | 315419       1378680          2496.8      1131.0    128835051
2592.31/649.18	c ................................ | *****.....            2875000 | 321174       1406371          2592.7      1108.9    131062136
2688.61/673.27	c ................................ | *****.....            2929000 | 327545       1433356          2688.9      1089.3    133492999
2784.06/697.16	c ................................ | *****.....            2982000 | 334597       1460020          2784.3      1071.0    136421154
2880.72/721.30	c ................................ | ******....            3038000 | 340489       1485394          2880.7      1054.6    138948684
2976.99/745.44	c ................................ | ******....            3097000 | 346962       1509603          2977.2      1040.3    141678693
3073.27/769.57	c ................................ | ******....            3150000 | 353340       1535657          3073.6      1024.9    144078066
3169.13/793.53	c ................................ | ******....            3203000 | 359998       1560560          3169.3      1010.6    146221567
3264.23/817.38	c ................................ | ******....            3261000 | 367371       1585528          3264.5       998.9    148844290
3360.49/841.49	c ................................ | *******...            3319000 | 375007       1609951          3360.8       987.6    151554204
3456.34/865.40	c ................................ | *******...            3371000 | 381001       1635644          3456.3       975.3    153426366
3552.22/889.45	c ................................ | *******...            3425000 | 387503       1666142          3552.4       964.1    156112845
3649.28/913.70	c ................................ | *******...            3481000 | 394908       1693600          3649.3       953.9    159097524
3744.35/937.52	c ................................ | *******...            3540000 | 402183       1721345          3744.5       945.4    161773331
3840.62/961.64	c ................................ | ********..            3600000 | 410238       1748686          3840.8       937.3    164527977
3938.49/986.13	c ................................ | ********..            3651000 | 416584       1776324          3938.6       927.0    166399952
4032.36/1009.64	c ................................ | ********..            3690000 | 421564       1797250          4032.5       915.1    168460116
4129.02/1033.89	c ................................ | ********..            3739000 | 427577       1820384          4129.4       905.5    170002383
4225.29/1057.97	c ................................ | ********..            3791000 | 433675       1842231          4225.6       897.1    170193461
4320.36/1081.70	c ................................ | *********.            3835000 | 438785       1864309          4320.4       887.7    170752011
4416.63/1105.89	c ................................ | *********.            3886000 | 444266       1886873          4417.0       879.8    170995413
4512.13/1129.76	c ................................ | *********.            3937000 | 449773       1910674          4512.3       872.5    171476060
4608.39/1153.85	c ................................ | *********.            3989000 | 456817       1934519          4608.6       865.6    172678136
4705.06/1178.08	c ................................ | *********.            4030000 | 461824       1955909          4705.4       856.5    173664882

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3250736-1304093294/watcher-3250736-1304093294 -o /tmp/evaluation-result-3250736-1304093294/solver-3250736-1304093294 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3250736-1304093294.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.17 7.90 8.00 2/177 28617
/proc/meminfo: memFree=17450948/32950940 swapFree=67111528/67111528
[pid=28617] ppid=28615 vsize=9540 CPUtime=0 cores=1,3,5,7
/proc/28617/stat : 28617 (ttsth-5-4) R 28615 28617 27480 0 -1 4202496 443 0 0 0 0 0 0 0 23 0 1 0 599560171 9768960 385 33554432000 4194304 4338100 140733406613248 18446744073709551615 215988253381 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/28617/statm: 2385 385 146 36 0 290 0

[startup+0.00863809 s]
/proc/loadavg: 7.17 7.90 8.00 2/177 28617
/proc/meminfo: memFree=17450948/32950940 swapFree=67111528/67111528
[pid=28617] ppid=28615 vsize=9888 CPUtime=0 cores=1,3,5,7
/proc/28617/stat : 28617 (ttsth-5-4) R 28615 28617 27480 0 -1 4202496 549 0 0 0 0 0 0 0 23 0 1 0 599560171 10125312 491 33554432000 4194304 4338100 140733406613248 18446744073709551615 215988592384 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/28617/statm: 2472 492 146 36 0 377 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 9888

[startup+0.100622 s]
/proc/loadavg: 7.17 7.90 8.00 2/177 28617
/proc/meminfo: memFree=17450948/32950940 swapFree=67111528/67111528
[pid=28617] ppid=28615 vsize=16680 CPUtime=0.09 cores=1,3,5,7
/proc/28617/stat : 28617 (ttsth-5-4) R 28615 28617 27480 0 -1 4202496 1970 0 0 0 9 0 0 0 23 0 1 0 599560171 17080320 1870 33554432000 4194304 4338100 140733406613248 18446744073709551615 4245205 0 0 4096 0 0 0 0 17 1 0 0 0
/proc/28617/statm: 4170 1870 187 36 0 2075 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 16680

[startup+0.300593 s]
/proc/loadavg: 7.17 7.90 8.00 2/177 28617
/proc/meminfo: memFree=17450948/32950940 swapFree=67111528/67111528
[pid=28617] ppid=28615 vsize=146824 CPUtime=0.61 cores=1,3,5,7
/proc/28617/stat : 28617 (ttsth-5-4) S 28615 28617 27480 0 -1 4202496 7828 0 0 0 61 0 0 0 25 0 5 0 599560171 150347776 6929 33554432000 4194304 4338100 140733406613248 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/28617/statm: 36706 6930 175 36 0 34611 0
Current children cumulated CPU time (s) 0.61
Current children cumulated vsize (KiB) 146824

[startup+0.700523 s]
/proc/loadavg: 7.17 7.90 8.00 2/177 28617
/proc/meminfo: memFree=17450948/32950940 swapFree=67111528/67111528
[pid=28617] ppid=28615 vsize=241172 CPUtime=2.22 cores=1,3,5,7
/proc/28617/stat : 28617 (ttsth-5-4) S 28615 28617 27480 0 -1 4202496 14720 0 0 0 220 2 0 0 25 0 5 0 599560171 246960128 13821 33554432000 4194304 4338100 140733406613248 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/28617/statm: 60293 13822 175 36 0 58198 0
Current children cumulated CPU time (s) 2.22
Current children cumulated vsize (KiB) 241172

[startup+1.50038 s]
/proc/loadavg: 7.17 7.90 8.00 9/186 28626
/proc/meminfo: memFree=17292304/32950940 swapFree=67111528/67111528
[pid=28617] ppid=28615 vsize=282496 CPUtime=5.4 cores=1,3,5,7
/proc/28617/stat : 28617 (ttsth-5-4) S 28615 28617 27480 0 -1 4202496 25360 0 0 0 537 3 0 0 25 0 5 0 599560171 289275904 24461 33554432000 4194304 4338100 140733406613248 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/28617/statm: 70624 24461 175 36 0 68529 0
[pid=28617/tid=28623] ppid=28615 vsize=282496 CPUtime=1.29 cores=1,3,5,7
/proc/28617/task/28623/stat : 28623 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 5376 0 0 0 129 0 0 0 25 0 5 0 599560190 289275904 24461 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=28617/tid=28624] ppid=28615 vsize=282496 CPUtime=1.3 cores=1,3,5,7
/proc/28617/task/28624/stat : 28624 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 4648 0 0 0 129 1 0 0 25 0 5 0 599560190 289275904 24461 33554432000 4194304 4338100 140733406613248 18446744073709551615 4229545 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=28617/tid=28625] ppid=28615 vsize=282496 CPUtime=1.3 cores=1,3,5,7
/proc/28617/task/28625/stat : 28625 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 4862 0 0 0 130 0 0 0 19 0 5 0 599560190 289275904 24461 33554432000 4194304 4338100 140733406613248 18446744073709551615 216000418485 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=28617/tid=28626] ppid=28615 vsize=282496 CPUtime=1.3 cores=1,3,5,7
/proc/28617/task/28626/stat : 28626 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 5020 0 0 0 130 0 0 0 19 0 5 0 599560190 289275904 24461 33554432000 4194304 4338100 140733406613248 18446744073709551615 4256989 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 5.4
Current children cumulated vsize (KiB) 282496

[startup+3.1011 s]
/proc/loadavg: 7.17 7.90 8.00 9/186 28626
/proc/meminfo: memFree=17184548/32950940 swapFree=67111528/67111528
[pid=28617] ppid=28615 vsize=348980 CPUtime=11.81 cores=1,3,5,7
/proc/28617/stat : 28617 (ttsth-5-4) S 28615 28617 27480 0 -1 4202496 42176 0 0 0 1174 7 0 0 25 0 5 0 599560171 357355520 41277 33554432000 4194304 4338100 140733406613248 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/28617/statm: 87245 41277 175 36 0 85150 0
[pid=28617/tid=28623] ppid=28615 vsize=348980 CPUtime=2.89 cores=1,3,5,7
/proc/28617/task/28623/stat : 28623 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 9704 0 0 0 288 1 0 0 25 0 5 0 599560190 357355520 41277 33554432000 4194304 4338100 140733406613248 18446744073709551615 216000405198 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=28617/tid=28624] ppid=28615 vsize=348980 CPUtime=2.9 cores=1,3,5,7
/proc/28617/task/28624/stat : 28624 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 8880 0 0 0 289 1 0 0 25 0 5 0 599560190 357355520 41277 33554432000 4194304 4338100 140733406613248 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=28617/tid=28625] ppid=28615 vsize=348980 CPUtime=2.9 cores=1,3,5,7
/proc/28617/task/28625/stat : 28625 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 8485 0 0 0 289 1 0 0 24 0 5 0 599560190 357355520 41277 33554432000 4194304 4338100 140733406613248 18446744073709551615 4283643 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=28617/tid=28626] ppid=28615 vsize=348980 CPUtime=2.9 cores=1,3,5,7
/proc/28617/task/28626/stat : 28626 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 9653 0 0 0 288 2 0 0 24 0 5 0 599560190 357355520 41277 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254254 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 11.81
Current children cumulated vsize (KiB) 348980

[startup+6.30054 s]
/proc/loadavg: 7.24 7.90 8.00 9/186 28626
/proc/meminfo: memFree=16916028/32950940 swapFree=67111528/67111528
[pid=28617] ppid=28615 vsize=459524 CPUtime=24.59 cores=1,3,5,7
/proc/28617/stat : 28617 (ttsth-5-4) S 28615 28617 27480 0 -1 4202496 69428 0 0 0 2446 13 0 0 25 0 5 0 599560171 470552576 68529 33554432000 4194304 4338100 140733406613248 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/28617/statm: 114881 68529 175 36 0 112786 0
[pid=28617/tid=28623] ppid=28615 vsize=459524 CPUtime=6.07 cores=1,3,5,7
/proc/28617/task/28623/stat : 28623 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 16835 0 0 0 605 2 0 0 25 0 5 0 599560190 470552576 68529 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=28617/tid=28624] ppid=28615 vsize=459524 CPUtime=6.1 cores=1,3,5,7
/proc/28617/task/28624/stat : 28624 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 16214 0 0 0 607 3 0 0 25 0 5 0 599560190 470552576 68529 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=28617/tid=28625] ppid=28615 vsize=459524 CPUtime=6.1 cores=1,3,5,7
/proc/28617/task/28625/stat : 28625 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 14279 0 0 0 607 3 0 0 25 0 5 0 599560190 470552576 68529 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=28617/tid=28626] ppid=28615 vsize=459524 CPUtime=6.1 cores=1,3,5,7
/proc/28617/task/28626/stat : 28626 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 16646 0 0 0 607 3 0 0 25 0 5 0 599560190 470552576 68529 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 24.59
Current children cumulated vsize (KiB) 459524

[startup+12.7004 s]
/proc/loadavg: 7.30 7.90 8.00 9/186 28627
/proc/meminfo: memFree=16487972/32950940 swapFree=67111528/67111528
[pid=28617] ppid=28615 vsize=617244 CPUtime=50.15 cores=1,3,5,7
/proc/28617/stat : 28617 (ttsth-5-4) S 28615 28617 27480 0 -1 4202496 109798 0 0 0 4993 22 0 0 25 0 5 0 599560171 632057856 108899 33554432000 4194304 4338100 140733406613248 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/28617/statm: 154311 108900 175 36 0 152216 0
[pid=28617/tid=28623] ppid=28615 vsize=617244 CPUtime=12.43 cores=1,3,5,7
/proc/28617/task/28623/stat : 28623 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 27035 0 0 0 1238 5 0 0 25 0 5 0 599560190 632057856 108901 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=28617/tid=28624] ppid=28615 vsize=617244 CPUtime=12.5 cores=1,3,5,7
/proc/28617/task/28624/stat : 28624 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 26181 0 0 0 1245 5 0 0 25 0 5 0 599560190 632057856 108901 33554432000 4194304 4338100 140733406613248 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=28617/tid=28625] ppid=28615 vsize=617244 CPUtime=12.5 cores=1,3,5,7
/proc/28617/task/28625/stat : 28625 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 23691 0 0 0 1245 5 0 0 25 0 5 0 599560190 632057856 108901 33554432000 4194304 4338100 140733406613248 18446744073709551615 4257007 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=28617/tid=28626] ppid=28615 vsize=617244 CPUtime=12.5 cores=1,3,5,7
/proc/28617/task/28626/stat : 28626 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 27439 0 0 0 1246 4 0 0 25 0 5 0 599560190 632057856 108901 33554432000 4194304 4338100 140733406613248 18446744073709551615 215988258328 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 50.15
Current children cumulated vsize (KiB) 617244

[startup+25.5012 s]
/proc/loadavg: 7.52 7.92 8.00 9/186 28627
/proc/meminfo: memFree=15834856/32950940 swapFree=67111528/67111528
[pid=28617] ppid=28615 vsize=837920 CPUtime=101.28 cores=1,3,5,7
/proc/28617/stat : 28617 (ttsth-5-4) S 28615 28617 27480 0 -1 4202496 165392 0 0 0 10095 33 0 0 25 0 5 0 599560171 858030080 164493 33554432000 4194304 4338100 140733406613248 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/28617/statm: 209480 164493 175 36 0 207385 0
[pid=28617/tid=28623] ppid=28615 vsize=837920 CPUtime=25.16 cores=1,3,5,7
/proc/28617/task/28623/stat : 28623 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 41660 0 0 0 2508 8 0 0 25 0 5 0 599560190 858030080 164493 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=28617/tid=28624] ppid=28615 vsize=837920 CPUtime=25.3 cores=1,3,5,7
/proc/28617/task/28624/stat : 28624 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 42917 0 0 0 2521 9 0 0 25 0 5 0 599560190 858030080 164493 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254342 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=28617/tid=28625] ppid=28615 vsize=837920 CPUtime=25.3 cores=1,3,5,7
/proc/28617/task/28625/stat : 28625 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 36567 0 0 0 2522 8 0 0 25 0 5 0 599560190 858030080 164493 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254418 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=28617/tid=28626] ppid=28615 vsize=837920 CPUtime=25.3 cores=1,3,5,7
/proc/28617/task/28626/stat : 28626 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 38794 0 0 0 2524 6 0 0 25 0 5 0 599560190 858030080 164493 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 101.28
Current children cumulated vsize (KiB) 837920

[startup+51.1007 s]
/proc/loadavg: 7.68 7.93 8.00 9/186 28628
/proc/meminfo: memFree=14823484/32950940 swapFree=67111528/67111528
[pid=28617] ppid=28615 vsize=1191228 CPUtime=203.54 cores=1,3,5,7
/proc/28617/stat : 28617 (ttsth-5-4) S 28615 28617 27480 0 -1 4202496 253350 0 0 0 20302 52 0 0 25 0 5 0 599560171 1219817472 252451 33554432000 4194304 4338100 140733406613248 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/28617/statm: 297807 252451 175 36 0 295712 0
[pid=28617/tid=28623] ppid=28615 vsize=1191228 CPUtime=50.62 cores=1,3,5,7
/proc/28617/task/28623/stat : 28623 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 63358 0 0 0 5050 12 0 0 25 0 5 0 599560190 1219817472 252451 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=28617/tid=28624] ppid=28615 vsize=1191228 CPUtime=50.9 cores=1,3,5,7
/proc/28617/task/28624/stat : 28624 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 66916 0 0 0 5076 14 0 0 25 0 5 0 599560190 1219817472 252451 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254362 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=28617/tid=28625] ppid=28615 vsize=1191228 CPUtime=50.9 cores=1,3,5,7
/proc/28617/task/28625/stat : 28625 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 59241 0 0 0 5076 14 0 0 25 0 5 0 599560190 1219817472 252451 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=28617/tid=28626] ppid=28615 vsize=1191228 CPUtime=50.9 cores=1,3,5,7
/proc/28617/task/28626/stat : 28626 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 58381 0 0 0 5080 10 0 0 25 0 5 0 599560190 1219817472 252451 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254291 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 203.54
Current children cumulated vsize (KiB) 1191228

[startup+102.307 s]
/proc/loadavg: 7.90 7.95 8.00 9/186 28629
/proc/meminfo: memFree=13380104/32950940 swapFree=67111528/67111528
[pid=28617] ppid=28615 vsize=1641336 CPUtime=408.09 cores=1,3,5,7
/proc/28617/stat : 28617 (ttsth-5-4) S 28615 28617 27480 0 -1 4202496 365816 0 0 0 40728 81 0 0 25 0 5 0 599560171 1680728064 364917 33554432000 4194304 4338100 140733406613248 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/28617/statm: 410334 364917 175 36 0 408239 0
[pid=28617/tid=28623] ppid=28615 vsize=1641336 CPUtime=101.81 cores=1,3,5,7
/proc/28617/task/28623/stat : 28623 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 95024 0 0 0 10161 20 0 0 25 0 5 0 599560190 1680728064 364917 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=28617/tid=28624] ppid=28615 vsize=1641336 CPUtime=102.02 cores=1,3,5,7
/proc/28617/task/28624/stat : 28624 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 94146 0 0 0 10182 20 0 0 25 0 5 0 599560190 1680728064 364917 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=28617/tid=28625] ppid=28615 vsize=1641336 CPUtime=102.11 cores=1,3,5,7
/proc/28617/task/28625/stat : 28625 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 87052 0 0 0 10190 21 0 0 25 0 5 0 599560190 1680728064 364917 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=28617/tid=28626] ppid=28615 vsize=1641336 CPUtime=101.92 cores=1,3,5,7
/proc/28617/task/28626/stat : 28626 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 84140 0 0 0 10175 17 0 0 25 0 5 0 599560190 1680728064 364917 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 408.09
Current children cumulated vsize (KiB) 1641336

[startup+162.3 s]
/proc/loadavg: 7.99 7.97 8.00 9/186 28631
/proc/meminfo: memFree=12020784/32950940 swapFree=67111528/67111528
[pid=28617] ppid=28615 vsize=2089288 CPUtime=647.73 cores=1,3,5,7
/proc/28617/stat : 28617 (ttsth-5-4) S 28615 28617 27480 0 -1 4202496 478024 0 0 0 64661 112 0 0 25 0 5 0 599560171 2139430912 477125 33554432000 4194304 4338100 140733406613248 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/28617/statm: 522322 477125 176 36 0 520227 0
[pid=28617/tid=28623] ppid=28615 vsize=2089288 CPUtime=161.8 cores=1,3,5,7
/proc/28617/task/28623/stat : 28623 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 122858 0 0 0 16152 28 0 0 25 0 5 0 599560190 2139430912 477125 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=28617/tid=28624] ppid=28615 vsize=2089288 CPUtime=162.01 cores=1,3,5,7
/proc/28617/task/28624/stat : 28624 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 120826 0 0 0 16171 30 0 0 25 0 5 0 599560190 2139430912 477125 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=28617/tid=28625] ppid=28615 vsize=2089288 CPUtime=162.1 cores=1,3,5,7
/proc/28617/task/28625/stat : 28625 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 113628 0 0 0 16183 27 0 0 25 0 5 0 599560190 2139430912 477125 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=28617/tid=28626] ppid=28615 vsize=2089288 CPUtime=161.59 cores=1,3,5,7
/proc/28617/task/28626/stat : 28626 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 115258 0 0 0 16134 25 0 0 25 0 5 0 599560190 2139430912 477125 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 647.73
Current children cumulated vsize (KiB) 2089288


################
# More data... #
################

/proc/meminfo: memFree=4020732/32950940 swapFree=67111528/67111528
[pid=28617] ppid=28615 vsize=4758904 CPUtime=3044.52 cores=1,3,5,7
/proc/28617/stat : 28617 (ttsth-5-4) S 28615 28617 27480 0 -1 4202496 1146104 0 0 0 304136 316 0 0 25 0 5 0 599560171 4873117696 1145205 33554432000 4194304 4338100 140733406613248 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/28617/statm: 1189726 1145205 176 36 0 1187631 0
[pid=28617/tid=28623] ppid=28615 vsize=4758904 CPUtime=761.82 cores=1,3,5,7
/proc/28617/task/28623/stat : 28623 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 279152 0 0 0 76100 82 0 0 25 0 5 0 599560190 4873117696 1145205 33554432000 4194304 4338100 140733406613248 18446744073709551615 4284426 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=28617/tid=28624] ppid=28615 vsize=4758904 CPUtime=762.03 cores=1,3,5,7
/proc/28617/task/28624/stat : 28624 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 312352 0 0 0 76116 87 0 0 25 0 5 0 599560190 4873117696 1145205 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
[pid=28617/tid=28625] ppid=28615 vsize=4758904 CPUtime=762.11 cores=1,3,5,7
/proc/28617/task/28625/stat : 28625 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 261396 0 0 0 76139 72 0 0 25 0 5 0 599560190 4873117696 1145205 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=28617/tid=28626] ppid=28615 vsize=4758904 CPUtime=758.33 cores=1,3,5,7
/proc/28617/task/28626/stat : 28626 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 287750 0 0 0 75760 73 0 0 25 0 5 0 599560190 4873117696 1145205 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3044.52
Current children cumulated vsize (KiB) 4758904

[startup+822.301 s]
/proc/loadavg: 8.06 8.05 8.00 9/186 28648
/proc/meminfo: memFree=3492724/32950940 swapFree=67111528/67111528
[pid=28617] ppid=28615 vsize=4941252 CPUtime=3284.18 cores=1,3,5,7
/proc/28617/stat : 28617 (ttsth-5-4) S 28615 28617 27480 0 -1 4202496 1193039 0 0 0 328085 333 0 0 25 0 5 0 599560171 5059842048 1192140 33554432000 4194304 4338100 140733406613248 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/28617/statm: 1235313 1192140 176 36 0 1233218 0
[pid=28617/tid=28623] ppid=28615 vsize=4941252 CPUtime=821.82 cores=1,3,5,7
/proc/28617/task/28623/stat : 28623 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 291279 0 0 0 82095 87 0 0 25 0 5 0 599560190 5059842048 1192140 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=28617/tid=28624] ppid=28615 vsize=4941252 CPUtime=822.01 cores=1,3,5,7
/proc/28617/task/28624/stat : 28624 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 324533 0 0 0 82110 91 0 0 25 0 5 0 599560190 5059842048 1192140 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=28617/tid=28625] ppid=28615 vsize=4941252 CPUtime=822.1 cores=1,3,5,7
/proc/28617/task/28625/stat : 28625 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 272384 0 0 0 82135 75 0 0 25 0 5 0 599560190 5059842048 1192140 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=28617/tid=28626] ppid=28615 vsize=4941252 CPUtime=818.02 cores=1,3,5,7
/proc/28617/task/28626/stat : 28626 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 299389 0 0 0 81725 77 0 0 25 0 5 0 599560190 5059842048 1192140 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3284.18
Current children cumulated vsize (KiB) 4941252

[startup+882.3 s]
/proc/loadavg: 8.06 8.05 8.00 9/186 28650
/proc/meminfo: memFree=2961548/32950940 swapFree=67111528/67111528
[pid=28617] ppid=28615 vsize=5126596 CPUtime=3523.85 cores=1,3,5,7
/proc/28617/stat : 28617 (ttsth-5-4) S 28615 28617 27480 0 -1 4202496 1238833 0 0 0 352039 346 0 0 25 0 5 0 599560171 5249634304 1237934 33554432000 4194304 4338100 140733406613248 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/28617/statm: 1281649 1237934 176 36 0 1279554 0
[pid=28617/tid=28623] ppid=28615 vsize=5126596 CPUtime=881.82 cores=1,3,5,7
/proc/28617/task/28623/stat : 28623 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 303184 0 0 0 88092 90 0 0 25 0 5 0 599560190 5249634304 1237934 33554432000 4194304 4338100 140733406613248 18446744073709551615 4284214 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=28617/tid=28624] ppid=28615 vsize=5126596 CPUtime=881.68 cores=1,3,5,7
/proc/28617/task/28624/stat : 28624 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 337375 0 0 0 88073 95 0 0 25 0 5 0 599560190 5249634304 1237934 33554432000 4194304 4338100 140733406613248 18446744073709551615 215987988194 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=28617/tid=28625] ppid=28615 vsize=5126596 CPUtime=882.11 cores=1,3,5,7
/proc/28617/task/28625/stat : 28625 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 280526 0 0 0 88133 78 0 0 25 0 5 0 599560190 5249634304 1237934 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254451 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=28617/tid=28626] ppid=28615 vsize=5126596 CPUtime=878.02 cores=1,3,5,7
/proc/28617/task/28626/stat : 28626 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 312294 0 0 0 87721 81 0 0 25 0 5 0 599560190 5249634304 1237934 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3523.85
Current children cumulated vsize (KiB) 5126596

[startup+942.301 s]
/proc/loadavg: 8.09 8.07 8.01 9/186 28651
/proc/meminfo: memFree=2545860/32950940 swapFree=67111528/67111528
[pid=28617] ppid=28615 vsize=5352736 CPUtime=3763.52 cores=1,3,5,7
/proc/28617/stat : 28617 (ttsth-5-4) S 28615 28617 27480 0 -1 4202496 1295464 0 0 0 375989 363 0 0 25 0 5 0 599560171 5481201664 1294565 33554432000 4194304 4338100 140733406613248 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/28617/statm: 1338184 1294565 176 36 0 1336089 0
[pid=28617/tid=28623] ppid=28615 vsize=5352736 CPUtime=941.73 cores=1,3,5,7
/proc/28617/task/28623/stat : 28623 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 315548 0 0 0 94077 96 0 0 25 0 5 0 599560190 5481201664 1294565 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=28617/tid=28624] ppid=28615 vsize=5352736 CPUtime=941.45 cores=1,3,5,7
/proc/28617/task/28624/stat : 28624 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 356356 0 0 0 94045 100 0 0 25 0 5 0 599560190 5481201664 1294565 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=28617/tid=28625] ppid=28615 vsize=5352736 CPUtime=942.1 cores=1,3,5,7
/proc/28617/task/28625/stat : 28625 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 294729 0 0 0 94129 81 0 0 25 0 5 0 599560190 5481201664 1294565 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=28617/tid=28626] ppid=28615 vsize=5352736 CPUtime=938.02 cores=1,3,5,7
/proc/28617/task/28626/stat : 28626 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 323377 0 0 0 93718 84 0 0 25 0 5 0 599560190 5481201664 1294565 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3763.52
Current children cumulated vsize (KiB) 5352736

[startup+1002.3 s]
/proc/loadavg: 8.18 8.10 8.02 9/186 28653
/proc/meminfo: memFree=2368416/32950940 swapFree=67111528/67111528
[pid=28617] ppid=28615 vsize=5522836 CPUtime=4003.2 cores=1,3,5,7
/proc/28617/stat : 28617 (ttsth-5-4) S 28615 28617 27480 0 -1 4202496 1338929 0 0 0 399941 379 0 0 25 0 5 0 599560171 5655384064 1338030 33554432000 4194304 4338100 140733406613248 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/28617/statm: 1380709 1338030 176 36 0 1378614 0
[pid=28617/tid=28623] ppid=28615 vsize=5522836 CPUtime=1001.6 cores=1,3,5,7
/proc/28617/task/28623/stat : 28623 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 325204 0 0 0 100060 100 0 0 25 0 5 0 599560190 5655384064 1338030 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=28617/tid=28624] ppid=28615 vsize=5522836 CPUtime=1001.45 cores=1,3,5,7
/proc/28617/task/28624/stat : 28624 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 368700 0 0 0 100041 104 0 0 25 0 5 0 599560190 5655384064 1338030 33554432000 4194304 4338100 140733406613248 18446744073709551615 4239472 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=28617/tid=28625] ppid=28615 vsize=5522836 CPUtime=1001.9 cores=1,3,5,7
/proc/28617/task/28625/stat : 28625 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 304615 0 0 0 100105 85 0 0 25 0 5 0 599560190 5655384064 1338030 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=28617/tid=28626] ppid=28615 vsize=5522836 CPUtime=998.02 cores=1,3,5,7
/proc/28617/task/28626/stat : 28626 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 334956 0 0 0 99715 87 0 0 25 0 5 0 599560190 5655384064 1338030 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4003.2
Current children cumulated vsize (KiB) 5522836

[startup+1062.3 s]
/proc/loadavg: 8.25 8.13 8.03 9/186 28655
/proc/meminfo: memFree=2197476/32950940 swapFree=67111528/67111528
[pid=28617] ppid=28615 vsize=5658280 CPUtime=4242.87 cores=1,3,5,7
/proc/28617/stat : 28617 (ttsth-5-4) S 28615 28617 27480 0 -1 4202496 1368657 0 0 0 423898 389 0 0 25 0 5 0 599560171 5794078720 1367758 33554432000 4194304 4338100 140733406613248 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/28617/statm: 1414570 1367758 176 36 0 1412475 0
[pid=28617/tid=28623] ppid=28615 vsize=5658280 CPUtime=1061.6 cores=1,3,5,7
/proc/28617/task/28623/stat : 28623 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 332417 0 0 0 106057 103 0 0 25 0 5 0 599560190 5794078720 1367758 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=28617/tid=28624] ppid=28615 vsize=5658280 CPUtime=1061.44 cores=1,3,5,7
/proc/28617/task/28624/stat : 28624 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 375457 0 0 0 106038 106 0 0 25 0 5 0 599560190 5794078720 1367758 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=28617/tid=28625] ppid=28615 vsize=5658280 CPUtime=1061.57 cores=1,3,5,7
/proc/28617/task/28625/stat : 28625 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 310482 0 0 0 106070 87 0 0 25 0 5 0 599560190 5794078720 1367758 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=28617/tid=28626] ppid=28615 vsize=5658280 CPUtime=1058.02 cores=1,3,5,7
/proc/28617/task/28626/stat : 28626 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 344847 0 0 0 105712 90 0 0 25 0 5 0 599560190 5794078720 1367758 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254404 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4242.87
Current children cumulated vsize (KiB) 5658280

[startup+1122.3 s]
/proc/loadavg: 8.09 8.10 8.02 9/186 28656
/proc/meminfo: memFree=1958744/32950940 swapFree=67111528/67111528
[pid=28617] ppid=28615 vsize=5684096 CPUtime=4482.54 cores=1,3,5,7
/proc/28617/stat : 28617 (ttsth-5-4) S 28615 28617 27480 0 -1 4202496 1375690 0 0 0 447861 393 0 0 25 0 5 0 599560171 5820514304 1374791 33554432000 4194304 4338100 140733406613248 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/28617/statm: 1421024 1374791 176 36 0 1418929 0
[pid=28617/tid=28623] ppid=28615 vsize=5684096 CPUtime=1121.61 cores=1,3,5,7
/proc/28617/task/28623/stat : 28623 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 332504 0 0 0 112057 104 0 0 25 0 5 0 599560190 5820514304 1374791 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=28617/tid=28624] ppid=28615 vsize=5684096 CPUtime=1121.45 cores=1,3,5,7
/proc/28617/task/28624/stat : 28624 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 382175 0 0 0 112036 109 0 0 25 0 5 0 599560190 5820514304 1374791 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254384 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=28617/tid=28625] ppid=28615 vsize=5684096 CPUtime=1121.24 cores=1,3,5,7
/proc/28617/task/28625/stat : 28625 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 310575 0 0 0 112037 87 0 0 25 0 5 0 599560190 5820514304 1374791 33554432000 4194304 4338100 140733406613248 18446744073709551615 215987987472 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=28617/tid=28626] ppid=28615 vsize=5684096 CPUtime=1118.02 cores=1,3,5,7
/proc/28617/task/28626/stat : 28626 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 344982 0 0 0 111712 90 0 0 25 0 5 0 599560190 5820514304 1374791 33554432000 4194304 4338100 140733406613248 18446744073709551615 4240516 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4482.54
Current children cumulated vsize (KiB) 5684096

[startup+1182.3 s]
/proc/loadavg: 8.06 8.09 8.02 9/186 28658
/proc/meminfo: memFree=1722516/32950940 swapFree=67111528/67111528
[pid=28617] ppid=28615 vsize=5775600 CPUtime=4722.22 cores=1,3,5,7
/proc/28617/stat : 28617 (ttsth-5-4) S 28615 28617 27480 0 -1 4202496 1398013 0 0 0 471819 403 0 0 25 0 5 0 599560171 5914214400 1397114 33554432000 4194304 4338100 140733406613248 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/28617/statm: 1443900 1397114 176 36 0 1441805 0
[pid=28617/tid=28623] ppid=28615 vsize=5775600 CPUtime=1181.61 cores=1,3,5,7
/proc/28617/task/28623/stat : 28623 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 343043 0 0 0 118052 109 0 0 25 0 5 0 599560190 5914214400 1397114 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=28617/tid=28624] ppid=28615 vsize=5775600 CPUtime=1181.45 cores=1,3,5,7
/proc/28617/task/28624/stat : 28624 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 392955 0 0 0 118031 114 0 0 25 0 5 0 599560190 5914214400 1397114 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=28617/tid=28625] ppid=28615 vsize=5775600 CPUtime=1180.93 cores=1,3,5,7
/proc/28617/task/28625/stat : 28625 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 311439 0 0 0 118005 88 0 0 25 0 5 0 599560190 5914214400 1397114 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254221 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=28617/tid=28626] ppid=28615 vsize=5775600 CPUtime=1178.02 cores=1,3,5,7
/proc/28617/task/28626/stat : 28626 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 345122 0 0 0 117711 91 0 0 25 0 5 0 599560190 5914214400 1397114 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4722.22
Current children cumulated vsize (KiB) 5775600



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.8 s]
/proc/loadavg: 8.04 8.09 8.02 6/184 28659
/proc/meminfo: memFree=4157416/32950940 swapFree=67111528/67111528
[pid=28617] ppid=28615 vsize=5813268 CPUtime=4800.13 cores=1,3,5,7
/proc/28617/stat : 28617 (ttsth-5-4) S 28615 28617 27480 0 -1 4202496 1407704 0 0 0 479606 407 0 0 25 0 5 0 599560171 5952786432 1406805 33554432000 4194304 4338100 140733406613248 18446744073709551615 216000395237 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/28617/statm: 1453317 1406806 176 36 0 1451222 0
[pid=28617/tid=28623] ppid=28615 vsize=5813268 CPUtime=1201.11 cores=1,3,5,7
/proc/28617/task/28623/stat : 28623 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 345775 0 0 0 120001 110 0 0 25 0 5 0 599560190 5952786432 1406806 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 3 0 0 0
[pid=28617/tid=28624] ppid=28615 vsize=5813268 CPUtime=1200.95 cores=1,3,5,7
/proc/28617/task/28624/stat : 28624 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 396464 0 0 0 119980 115 0 0 25 0 5 0 599560190 5952786432 1406806 33554432000 4194304 4338100 140733406613248 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 1 0 0 0
[pid=28617/tid=28625] ppid=28615 vsize=5813268 CPUtime=1200.32 cores=1,3,5,7
/proc/28617/task/28625/stat : 28625 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 314859 0 0 0 119943 89 0 0 25 0 5 0 599560190 5952786432 1406807 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 7 0 0 0
[pid=28617/tid=28626] ppid=28615 vsize=5813268 CPUtime=1197.53 cores=1,3,5,7
/proc/28617/task/28626/stat : 28626 (ttsth-5-4) R 28615 28617 27480 0 -1 4202560 345153 0 0 0 119662 91 0 0 25 0 5 0 599560190 5952786432 1406807 33554432000 4194304 4338100 140733406613248 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4800.13
Current children cumulated vsize (KiB) 5813268

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1202 s]
/proc/loadavg: 8.04 8.09 8.02 6/184 28659
/proc/meminfo: memFree=4157416/32950940 swapFree=67111528/67111528
[pid=28617] ppid=28615 vsize=0 CPUtime=4800.34 cores=1,3,5,7
/proc/28617/stat : 28617 (ttsth-5-4) Z 28615 28617 27480 0 -1 4203548 1407707 0 0 0 479608 426 0 0 18 0 2 0 599560171 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 1 0 0 0
/proc/28617/statm: 0 0 0 0 0 0 0
[pid=28617/tid=28625] ppid=28615 vsize=0 CPUtime=1200.5 cores=1,3,5,7
/proc/28617/task/28625/stat : 28625 (ttsth-5-4) R 28615 28617 27480 0 -1 4203588 314860 0 0 0 119943 107 0 0 25 0 2 0 599560190 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4800.34
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1202.07
CPU time (s): 4800.42
CPU user time (s): 4796.08
CPU system time (s): 4.33734
CPU usage (%): 399.345
Max. virtual memory (cumulated for all children) (KiB): 5813268

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4796.08
system time used= 4.33734
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1407707
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1075
involuntary context switches= 32243

runsolver used 2.01669 second user time and 4.54931 second system time

The end

Launcher Data 

Begin job on node117 at 2011-04-29 18:08:14
IDJOB=3250736
IDBENCH=83353
IDSOLVER=1654
FILE ID=node117/3250736-1304093294
RUNJOBID= node117-1304093293-28598
PBS_JOBID= 13170554
Free space on /tmp= 71680 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT11/crafted/kullmann/VanDerWaerden/VanderWaerden3k/VanDerWaerden_2-3-15_218.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3250736-1304093294/watcher-3250736-1304093294 -o /tmp/evaluation-result-3250736-1304093294/solver-3250736-1304093294 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3250736-1304093294.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= afc70649cebcc24cd0a7b28def593295
RANDOM SEED=1453078770

node117.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5267.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950940 kB
MemFree:      17451236 kB
Buffers:       2379984 kB
Cached:       12463872 kB
SwapCached:          0 kB
Active:        8150532 kB
Inactive:      6767580 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950940 kB
LowFree:      17451236 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:             892 kB
Writeback:           0 kB
AnonPages:       73820 kB
Mapped:          14884 kB
Slab:           518780 kB
PageTables:       4444 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586996 kB
Committed_AS:   217040 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71680 MiB
End job on node117 at 2011-04-29 18:28:18