Trace number 3262574

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.53 1202.04

General information on the benchmark

Namecrafted/kullmann/VanDerWaerden/
VanderWaerden3k/VanDerWaerden_2-3-13_160.cnf
MD5SUMad5b26aa9760a3817f3ab03e9314a5d6
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark284.707
Satisfiable
(Un)Satisfiability was proved
Number of variables160
Number of clauses7308
Sum of the clauses size31804
Maximum clause length13
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 36320
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 5988

Solver Data 

0.00/0.01	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3262574-1304093010.cnf 
0.00/0.01	c Using SATTIMEOUT of 4800
0.00/0.01	c Using SATRAM of 15500
0.00/0.01	c Size of subset node = 32
0.00/0.01	c Variables = 160
0.00/0.01	c Clauses = 7308
0.00/0.01	c Literals = 31804
0.00/0.01	c Reduced size:
0.00/0.01	c Variables = 160
0.00/0.01	c Clauses = 7308
0.00/0.01	c Literals = 31804
0.00/0.01	c a2
0.00/0.03	c a1
0.00/0.06	c Independent sub-problems = 1
0.00/0.06	c Sub-problem with 160 variables and 7308 clauses
0.00/0.06	c using permutation method 4
0.00/0.06	c count = 1
0.00/0.06	c Ordering finished 0.060 seconds
0.00/0.06	c mapped 0.060 seconds
0.00/0.06	c permuted 0.060 seconds
0.00/0.06	c about to sort
0.00/0.06	c sorted
0.08/0.08	c clauses now 7308
0.08/0.08	c init finished 0.060 seconds
0.08/0.08	c tree built 0.080 seconds
0.08/0.08	c 
0.08/0.08	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.08/0.08	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.08/0.08	c ................................ | ..........                    |                                  0.1         0.0             
0.08/0.08	c ................................ | ..........                  1 |                                  0.1        10.0             
0.08/0.08	c ................................ | ..........                  2 |                                  0.1        20.0             
0.08/0.08	c ................................ | ..........                  3 |                                  0.1        30.0             
0.08/0.08	c ................................ | ..........                  4 |                                  0.1        40.0             
0.08/0.08	c ................................ | ..........                  5 |                                  0.1        50.0             
0.11/0.10	c ................................ | ..........               1000 | 31           792                 0.1      7692.3        35179
96.00/24.16	c ................................ | ..........             508000 | 36883        444637             96.2      5279.0     21764058
191.83/48.19	c ................................ | ..........             760000 | 54957        661249            192.2      3953.8     31681015
288.10/72.21	c ................................ | ..........             957000 | 70458        830195            288.2      3321.0     40214641
383.98/96.24	c ................................ | ..........            1124000 | 84533        973796            384.1      2925.9     47234723
479.83/120.24	c ................................ | *.........            1291000 | 97477        1104044           480.0      2689.5     52888948
576.10/144.39	c ................................ | *.........            1450000 | 111330       1222001           576.5      2515.3     58767894
672.74/168.53	c ................................ | *.........            1604000 | 125246       1330632           672.9      2383.8     60232017
768.22/192.49	c ................................ | *.........            1718000 | 134098       1435483           768.6      2235.3     62194197
864.08/216.48	c ................................ | *.........            1857000 | 143444       1543763           864.4      2148.3     66575028
960.74/240.61	c ................................ | **........            1995000 | 151689       1646330           960.8      2076.4     70826355
1056.21/264.55	c ................................ | **........            2126000 | 160082       1742859          1056.4      2012.5     72594356
1152.48/288.60	c ................................ | **........            2243000 | 167794       1835110          1152.5      1946.2     74837397
1248.36/312.64	c ................................ | **........            2355000 | 175220       1919295          1248.5      1886.2     79156288
1344.63/336.75	c ................................ | **........            2454000 | 182248       2000531          1344.8      1824.8     83360796
1439.70/360.58	c ................................ | ***.......            2550000 | 189305       2080403          1440.0      1770.8     87634677
1536.38/384.78	c ................................ | ***.......            2639000 | 195795       2155951          1536.7      1717.3     91570178
1631.84/408.68	c ................................ | ***.......            2739000 | 203387       2228975          1632.2      1678.1     95284805
1728.11/432.74	c ................................ | ***.......            2830000 | 210154       2299134          1728.3      1637.5     98779664
1824.78/456.90	c ................................ | ***.......            2913000 | 216996       2367149          1824.8      1596.3    102391999
1920.65/480.90	c ................................ | ****......            3003000 | 223281       2435707          1920.7      1563.5    106062974
2016.15/504.85	c ................................ | ****......            3090000 | 230482       2501512          2016.4      1532.5    109489114
2112.01/528.80	c ................................ | ****......            3181000 | 238030       2564371          2112.0      1506.1    112410274
2208.27/552.95	c ................................ | ****......            3266000 | 245699       2628133          2208.5      1478.8    115238358
2303.74/576.87	c ................................ | ****......            3343000 | 252152       2687917          2304.1      1450.9    118458400
2400.82/601.18	c ................................ | *****.....            3432000 | 260466       2747622          2401.2      1429.3    121608830
2496.68/625.16	c ................................ | *****.....            3512000 | 267117       2805332          2496.9      1406.5    124805298
2591.77/648.99	c ................................ | *****.....            3585000 | 273645       2861816          2592.1      1383.0    126757467
2688.03/673.00	c ................................ | *****.....            3662000 | 280503       2916282          2688.0      1362.3    129373434
2784.30/697.10	c ................................ | *****.....            3735000 | 287591       2970955          2784.3      1341.4    129704310
2880.54/721.27	c ................................ | ******....            3804000 | 294126       3022929          2880.8      1320.4    129839173
2976.01/745.13	c ................................ | ******....            3886000 | 301684       3079196          2976.1      1305.7    131464833
3072.27/769.28	c ................................ | ******....            3963000 | 308781       3134023          3072.6      1289.8    133510800
3168.93/793.42	c ................................ | ******....            4040000 | 315972       3184905          3169.0      1274.8    135606108
3264.43/817.32	c ................................ | ******....            4124000 | 323727       3237285          3264.5      1263.3    137345190
3361.07/841.59	c ................................ | *******...            4202000 | 330754       3289417          3361.4      1250.1    138170919
3456.52/865.42	c ................................ | *******...            4277000 | 337331       3336905          3456.6      1237.3    139877338
3552.79/889.58	c ................................ | *******...            4350000 | 343715       3388840          3553.1      1224.3    142666550
3649.46/913.75	c ................................ | *******...            4435000 | 351454       3442463          3649.7      1215.2    145412582
3744.13/937.40	c ................................ | *******...            4500000 | 358034       3492281          3744.2      1201.9    147998376
3840.02/961.41	c ................................ | ********..            4565000 | 364704       3538611          3840.1      1188.8    150452541
3936.69/985.64	c ................................ | ********..            4633000 | 371313       3586618          3936.8      1176.8    153297118
4032.54/1009.63	c ................................ | ********..            4697000 | 377236       3632298          4032.7      1164.7    155812134
4129.62/1034.00	c ................................ | ********..            4754000 | 383665       3677268          4130.0      1151.1    158431275
4223.86/1057.54	c ................................ | ********..            4817000 | 389740       3723518          4224.0      1140.4    159128541
4320.09/1081.69	c ................................ | *********.            4891000 | 396248       3769138          4320.5      1132.1    159128541
4416.36/1105.70	c ................................ | *********.            4950000 | 402408       3817012          4416.4      1120.8    159128541
4511.86/1129.66	c ................................ | *********.            5007000 | 408748       3859707          4512.1      1109.7    159128541
4608.93/1153.98	c ................................ | *********.            5076000 | 415687       3908230          4609.2      1101.3    159128541
4705.59/1178.11	c ................................ | *********.            5130000 | 421907       3949003          4705.6      1090.2    159128541

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3262574-1304093010/watcher-3262574-1304093010 -o /tmp/evaluation-result-3262574-1304093010/solver-3262574-1304093010 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3262574-1304093010.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.64 7.01 7.74 2/177 10244
/proc/meminfo: memFree=25021932/32951124 swapFree=67111352/67111528
[pid=10244] ppid=10241 vsize=9244 CPUtime=0 cores=0,2,4,6
/proc/10244/stat : 10244 (ttsth-5-4) R 10241 10244 9884 0 -1 4202496 385 0 0 0 0 0 0 0 22 0 1 0 520899249 9465856 327 33554432000 4194304 4338100 140733593680688 18446744073709551615 246516495045 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/10244/statm: 2311 328 146 36 0 216 0

[startup+0.0854671 s]
/proc/loadavg: 4.64 7.01 7.74 2/177 10244
/proc/meminfo: memFree=25021932/32951124 swapFree=67111352/67111528
[pid=10244] ppid=10241 vsize=16804 CPUtime=0.08 cores=0,2,4,6
/proc/10244/stat : 10244 (ttsth-5-4) R 10241 10244 9884 0 -1 4202496 2717 0 0 0 8 0 0 0 22 0 1 0 520899249 17207296 2240 33554432000 4194304 4338100 140733593680688 18446744073709551615 246516495045 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/10244/statm: 4234 2243 163 36 0 2139 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 16804

[startup+0.100432 s]
/proc/loadavg: 4.64 7.01 7.74 2/177 10244
/proc/meminfo: memFree=25021932/32951124 swapFree=67111352/67111528
[pid=10244] ppid=10241 vsize=137672 CPUtime=0.11 cores=0,2,4,6
/proc/10244/stat : 10244 (ttsth-5-4) S 10241 10244 9884 0 -1 4202496 3413 0 0 0 11 0 0 0 22 0 5 0 520899249 140976128 2936 33554432000 4194304 4338100 140733593680688 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/10244/statm: 34418 2938 175 36 0 32323 0
Current children cumulated CPU time (s) 0.11
Current children cumulated vsize (KiB) 137672

[startup+0.300399 s]
/proc/loadavg: 4.64 7.01 7.74 2/177 10244
/proc/meminfo: memFree=25021932/32951124 swapFree=67111352/67111528
[pid=10244] ppid=10241 vsize=151108 CPUtime=0.91 cores=0,2,4,6
/proc/10244/stat : 10244 (ttsth-5-4) S 10241 10244 9884 0 -1 4202496 8057 0 0 0 90 1 0 0 22 0 5 0 520899249 154734592 7580 33554432000 4194304 4338100 140733593680688 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/10244/statm: 37777 7580 175 36 0 35682 0
Current children cumulated CPU time (s) 0.91
Current children cumulated vsize (KiB) 151108

[startup+0.700293 s]
/proc/loadavg: 4.64 7.01 7.74 2/177 10244
/proc/meminfo: memFree=25021932/32951124 swapFree=67111352/67111528
[pid=10244] ppid=10241 vsize=183304 CPUtime=2.51 cores=0,2,4,6
/proc/10244/stat : 10244 (ttsth-5-4) S 10241 10244 9884 0 -1 4202496 15879 0 0 0 250 1 0 0 22 0 5 0 520899249 187703296 15402 33554432000 4194304 4338100 140733593680688 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/10244/statm: 45826 15404 175 36 0 43731 0
Current children cumulated CPU time (s) 2.51
Current children cumulated vsize (KiB) 183304

[startup+1.50115 s]
/proc/loadavg: 4.64 7.01 7.74 10/190 10260
/proc/meminfo: memFree=24872552/32951124 swapFree=67111352/67111528
[pid=10244] ppid=10241 vsize=234264 CPUtime=5.7 cores=0,2,4,6
/proc/10244/stat : 10244 (ttsth-5-4) S 10241 10244 9884 0 -1 4202496 28645 0 0 0 566 4 0 0 22 0 5 0 520899249 239886336 28168 33554432000 4194304 4338100 140733593680688 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/10244/statm: 58566 28169 175 36 0 56471 0
[pid=10244/tid=10253] ppid=10241 vsize=234264 CPUtime=1.4 cores=0,2,4,6
/proc/10244/task/10253/stat : 10253 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 6552 0 0 0 140 0 0 0 25 0 5 0 520899258 239886336 28169 33554432000 4194304 4338100 140733593680688 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10244/tid=10254] ppid=10241 vsize=234264 CPUtime=1.4 cores=0,2,4,6
/proc/10244/task/10254/stat : 10254 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 6254 0 0 0 139 1 0 0 20 0 5 0 520899258 239886336 28169 33554432000 4194304 4338100 140733593680688 18446744073709551615 246528646862 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10244/tid=10255] ppid=10241 vsize=234264 CPUtime=1.39 cores=0,2,4,6
/proc/10244/task/10255/stat : 10255 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 6269 0 0 0 138 1 0 0 25 0 5 0 520899258 239886336 28169 33554432000 4194304 4338100 140733593680688 18446744073709551615 4255963 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10244/tid=10256] ppid=10241 vsize=234264 CPUtime=1.4 cores=0,2,4,6
/proc/10244/task/10256/stat : 10256 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 6568 0 0 0 139 1 0 0 25 0 5 0 520899258 239886336 28169 33554432000 4194304 4338100 140733593680688 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 5.7
Current children cumulated vsize (KiB) 234264

[startup+3.10086 s]
/proc/loadavg: 4.64 7.01 7.74 9/190 10260
/proc/meminfo: memFree=24747808/32951124 swapFree=67111352/67111528
[pid=10244] ppid=10241 vsize=313644 CPUtime=12.1 cores=0,2,4,6
/proc/10244/stat : 10244 (ttsth-5-4) S 10241 10244 9884 0 -1 4202496 48862 0 0 0 1202 8 0 0 22 0 5 0 520899249 321171456 48385 33554432000 4194304 4338100 140733593680688 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/10244/statm: 78411 48385 175 36 0 76316 0
[pid=10244/tid=10253] ppid=10241 vsize=313644 CPUtime=3 cores=0,2,4,6
/proc/10244/task/10253/stat : 10253 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 11504 0 0 0 299 1 0 0 25 0 5 0 520899258 321171456 48385 33554432000 4194304 4338100 140733593680688 18446744073709551615 246528648451 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10244/tid=10254] ppid=10241 vsize=313644 CPUtime=2.99 cores=0,2,4,6
/proc/10244/task/10254/stat : 10254 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 11320 0 0 0 298 1 0 0 25 0 5 0 520899258 321171456 48385 33554432000 4194304 4338100 140733593680688 18446744073709551615 246516492550 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10244/tid=10255] ppid=10241 vsize=313644 CPUtime=2.99 cores=0,2,4,6
/proc/10244/task/10255/stat : 10255 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 11433 0 0 0 297 2 0 0 25 0 5 0 520899258 321171456 48385 33554432000 4194304 4338100 140733593680688 18446744073709551615 4239500 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10244/tid=10256] ppid=10241 vsize=313644 CPUtime=3 cores=0,2,4,6
/proc/10244/task/10256/stat : 10256 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 11602 0 0 0 298 2 0 0 25 0 5 0 520899258 321171456 48386 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 12.1
Current children cumulated vsize (KiB) 313644

[startup+6.30027 s]
/proc/loadavg: 4.91 7.02 7.74 10/190 10260
/proc/meminfo: memFree=24448320/32951124 swapFree=67111352/67111528
[pid=10244] ppid=10241 vsize=427820 CPUtime=24.88 cores=0,2,4,6
/proc/10244/stat : 10244 (ttsth-5-4) S 10241 10244 9884 0 -1 4202496 78321 0 0 0 2474 14 0 0 22 0 5 0 520899249 438087680 77844 33554432000 4194304 4338100 140733593680688 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/10244/statm: 106955 77844 175 36 0 104860 0
[pid=10244/tid=10253] ppid=10241 vsize=427820 CPUtime=6.2 cores=0,2,4,6
/proc/10244/task/10253/stat : 10253 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 18344 0 0 0 618 2 0 0 25 0 5 0 520899258 438087680 77844 33554432000 4194304 4338100 140733593680688 18446744073709551615 246516229588 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10244/tid=10254] ppid=10241 vsize=427820 CPUtime=6.2 cores=0,2,4,6
/proc/10244/task/10254/stat : 10254 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 19494 0 0 0 617 3 0 0 25 0 5 0 520899258 438087680 77844 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254244 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10244/tid=10255] ppid=10241 vsize=427820 CPUtime=6.17 cores=0,2,4,6
/proc/10244/task/10255/stat : 10255 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 18247 0 0 0 613 4 0 0 25 0 5 0 520899258 438087680 77844 33554432000 4194304 4338100 140733593680688 18446744073709551615 246528648360 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10244/tid=10256] ppid=10241 vsize=427820 CPUtime=6.19 cores=0,2,4,6
/proc/10244/task/10256/stat : 10256 (ttsth-5-4) S 10241 10244 9884 0 -1 4202560 19233 0 0 0 616 3 0 0 25 0 5 0 520899258 438087680 77845 33554432000 4194304 4338100 140733593680688 18446744073709551615 246528659828 0 0 4096 0 18446604453273217728 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 24.88
Current children cumulated vsize (KiB) 427820

[startup+12.7011 s]
/proc/loadavg: 5.15 7.04 7.75 9/190 10260
/proc/meminfo: memFree=24005180/32951124 swapFree=67111352/67111528
[pid=10244] ppid=10241 vsize=611464 CPUtime=50.45 cores=0,2,4,6
/proc/10244/stat : 10244 (ttsth-5-4) S 10241 10244 9884 0 -1 4202496 124444 0 0 0 5021 24 0 0 22 0 5 0 520899249 626139136 123967 33554432000 4194304 4338100 140733593680688 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/10244/statm: 152866 123967 175 36 0 150771 0
[pid=10244/tid=10253] ppid=10241 vsize=611464 CPUtime=12.6 cores=0,2,4,6
/proc/10244/task/10253/stat : 10253 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 29267 0 0 0 1255 5 0 0 25 0 5 0 520899258 626139136 123967 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10244/tid=10254] ppid=10241 vsize=611464 CPUtime=12.59 cores=0,2,4,6
/proc/10244/task/10254/stat : 10254 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 32271 0 0 0 1254 5 0 0 25 0 5 0 520899258 626139136 123967 33554432000 4194304 4338100 140733593680688 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10244/tid=10255] ppid=10241 vsize=611464 CPUtime=12.54 cores=0,2,4,6
/proc/10244/task/10255/stat : 10255 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 27880 0 0 0 1248 6 0 0 25 0 5 0 520899258 626139136 123967 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254304 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10244/tid=10256] ppid=10241 vsize=611464 CPUtime=12.6 cores=0,2,4,6
/proc/10244/task/10256/stat : 10256 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 32023 0 0 0 1254 6 0 0 25 0 5 0 520899258 626139136 123967 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 50.45
Current children cumulated vsize (KiB) 611464

[startup+25.5008 s]
/proc/loadavg: 5.78 7.09 7.75 9/190 10261
/proc/meminfo: memFree=23342760/32951124 swapFree=67111352/67111528
[pid=10244] ppid=10241 vsize=906968 CPUtime=101.58 cores=0,2,4,6
/proc/10244/stat : 10244 (ttsth-5-4) S 10241 10244 9884 0 -1 4202496 181574 0 0 0 10120 38 0 0 22 0 5 0 520899249 928735232 181089 33554432000 4194304 4338100 140733593680688 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/10244/statm: 226742 181089 176 36 0 224647 0
[pid=10244/tid=10253] ppid=10241 vsize=906968 CPUtime=25.4 cores=0,2,4,6
/proc/10244/task/10253/stat : 10253 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 43001 0 0 0 2532 8 0 0 25 0 5 0 520899258 928735232 181089 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10244/tid=10254] ppid=10241 vsize=906968 CPUtime=25.32 cores=0,2,4,6
/proc/10244/task/10254/stat : 10254 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 48003 0 0 0 2523 9 0 0 25 0 5 0 520899258 928735232 181089 33554432000 4194304 4338100 140733593680688 18446744073709551615 4263499 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10244/tid=10255] ppid=10241 vsize=906968 CPUtime=25.33 cores=0,2,4,6
/proc/10244/task/10255/stat : 10255 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 40043 0 0 0 2524 9 0 0 25 0 5 0 520899258 928735232 181089 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10244/tid=10256] ppid=10241 vsize=906968 CPUtime=25.4 cores=0,2,4,6
/proc/10244/task/10256/stat : 10256 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 47524 0 0 0 2531 9 0 0 25 0 5 0 520899258 928735232 181089 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 101.58
Current children cumulated vsize (KiB) 906968

[startup+51.1011 s]
/proc/loadavg: 6.61 7.18 7.76 9/190 10263
/proc/meminfo: memFree=22405148/32951124 swapFree=67111352/67111528
[pid=10244] ppid=10241 vsize=1245652 CPUtime=203.82 cores=0,2,4,6
/proc/10244/stat : 10244 (ttsth-5-4) S 10241 10244 9884 0 -1 4202496 267475 0 0 0 20326 56 0 0 22 0 5 0 520899249 1275547648 266990 33554432000 4194304 4338100 140733593680688 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/10244/statm: 311413 266990 176 36 0 309318 0
[pid=10244/tid=10253] ppid=10241 vsize=1245652 CPUtime=51 cores=0,2,4,6
/proc/10244/task/10253/stat : 10253 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 64660 0 0 0 5088 12 0 0 25 0 5 0 520899258 1275547648 266990 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10244/tid=10254] ppid=10241 vsize=1245652 CPUtime=50.78 cores=0,2,4,6
/proc/10244/task/10254/stat : 10254 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 69959 0 0 0 5063 15 0 0 25 0 5 0 520899258 1275547648 266990 33554432000 4194304 4338100 140733593680688 18446744073709551615 4256139 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10244/tid=10255] ppid=10241 vsize=1245652 CPUtime=50.93 cores=0,2,4,6
/proc/10244/task/10255/stat : 10255 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 59441 0 0 0 5080 13 0 0 25 0 5 0 520899258 1275547648 266990 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254208 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10244/tid=10256] ppid=10241 vsize=1245652 CPUtime=50.98 cores=0,2,4,6
/proc/10244/task/10256/stat : 10256 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 70412 0 0 0 5085 13 0 0 25 0 5 0 520899258 1275547648 266990 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254254 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 203.82
Current children cumulated vsize (KiB) 1245652

[startup+102.307 s]
/proc/loadavg: 7.61 7.37 7.80 10/190 10266
/proc/meminfo: memFree=20991588/32951124 swapFree=67111352/67111528
[pid=10244] ppid=10241 vsize=1766040 CPUtime=408.36 cores=0,2,4,6
/proc/10244/stat : 10244 (ttsth-5-4) S 10241 10244 9884 0 -1 4202496 397095 0 0 0 40746 90 0 0 22 0 5 0 520899249 1808424960 396610 33554432000 4194304 4338100 140733593680688 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/10244/statm: 441510 396610 176 36 0 439415 0
[pid=10244/tid=10253] ppid=10241 vsize=1766040 CPUtime=102.2 cores=0,2,4,6
/proc/10244/task/10253/stat : 10253 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 95271 0 0 0 10200 20 0 0 25 0 5 0 520899258 1808424960 396610 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10244/tid=10254] ppid=10241 vsize=1766040 CPUtime=101.71 cores=0,2,4,6
/proc/10244/task/10254/stat : 10254 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 108015 0 0 0 10146 25 0 0 25 0 5 0 520899258 1808424960 396610 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10244/tid=10255] ppid=10241 vsize=1766040 CPUtime=102.13 cores=0,2,4,6
/proc/10244/task/10255/stat : 10255 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 86182 0 0 0 10193 20 0 0 25 0 5 0 520899258 1808424960 396610 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10244/tid=10256] ppid=10241 vsize=1766040 CPUtime=102.2 cores=0,2,4,6
/proc/10244/task/10256/stat : 10256 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 104624 0 0 0 10197 23 0 0 25 0 5 0 520899258 1808424960 396610 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 408.36
Current children cumulated vsize (KiB) 1766040

[startup+162.301 s]
/proc/loadavg: 7.85 7.48 7.81 9/190 10267
/proc/meminfo: memFree=19738636/32951124 swapFree=67111352/67111528
[pid=10244] ppid=10241 vsize=2142804 CPUtime=647.98 cores=0,2,4,6
/proc/10244/stat : 10244 (ttsth-5-4) S 10241 10244 9884 0 -1 4202496 489824 0 0 0 64684 114 0 0 22 0 5 0 520899249 2194231296 489339 33554432000 4194304 4338100 140733593680688 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/10244/statm: 535701 489339 176 36 0 533606 0
[pid=10244/tid=10253] ppid=10241 vsize=2142804 CPUtime=161.86 cores=0,2,4,6
/proc/10244/task/10253/stat : 10253 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 120876 0 0 0 16158 28 0 0 25 0 5 0 520899258 2194231296 489340 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10244/tid=10254] ppid=10241 vsize=2142804 CPUtime=161.69 cores=0,2,4,6
/proc/10244/task/10254/stat : 10254 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 132318 0 0 0 16138 31 0 0 25 0 5 0 520899258 2194231296 489340 33554432000 4194304 4338100 140733593680688 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10244/tid=10255] ppid=10241 vsize=2142804 CPUtime=162.13 cores=0,2,4,6
/proc/10244/task/10255/stat : 10255 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 102685 0 0 0 16188 25 0 0 25 0 5 0 520899258 2194231296 489340 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10244/tid=10256] ppid=10241 vsize=2142804 CPUtime=162.19 cores=0,2,4,6
/proc/10244/task/10256/stat : 10256 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 130943 0 0 0 16190 29 0 0 25 0 5 0 520899258 2194231296 489340 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254357 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 647.98
Current children cumulated vsize (KiB) 2142804


################
# More data... #
################

/proc/meminfo: memFree=12820120/32951124 swapFree=67111352/67111528
[pid=10244] ppid=10241 vsize=4482260 CPUtime=3044.73 cores=0,2,4,6
/proc/10244/stat : 10244 (ttsth-5-4) S 10241 10244 9884 0 -1 4202496 1081986 0 0 0 304186 287 0 0 22 0 5 0 520899249 4589834240 1081501 33554432000 4194304 4338100 140733593680688 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/10244/statm: 1120565 1081501 176 36 0 1118470 0
[pid=10244/tid=10253] ppid=10241 vsize=4482260 CPUtime=758.6 cores=0,2,4,6
/proc/10244/task/10253/stat : 10253 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 274476 0 0 0 75786 74 0 0 25 0 5 0 520899258 4589834240 1081501 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10244/tid=10254] ppid=10241 vsize=4482260 CPUtime=761.69 cores=0,2,4,6
/proc/10244/task/10254/stat : 10254 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 290517 0 0 0 76097 72 0 0 25 0 5 0 520899258 4589834240 1081501 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10244/tid=10255] ppid=10241 vsize=4482260 CPUtime=762.11 cores=0,2,4,6
/proc/10244/task/10255/stat : 10255 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 240531 0 0 0 76145 66 0 0 25 0 5 0 520899258 4589834240 1081501 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10244/tid=10256] ppid=10241 vsize=4482260 CPUtime=762.2 cores=0,2,4,6
/proc/10244/task/10256/stat : 10256 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 273459 0 0 0 76148 72 0 0 25 0 5 0 520899258 4589834240 1081501 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3044.73
Current children cumulated vsize (KiB) 4482260

[startup+822.301 s]
/proc/loadavg: 8.23 8.07 7.95 9/190 10285
/proc/meminfo: memFree=12501656/32951124 swapFree=67111352/67111528
[pid=10244] ppid=10241 vsize=4643224 CPUtime=3284.38 cores=0,2,4,6
/proc/10244/stat : 10244 (ttsth-5-4) S 10241 10244 9884 0 -1 4202496 1121667 0 0 0 328138 300 0 0 22 0 5 0 520899249 4754661376 1121182 33554432000 4194304 4338100 140733593680688 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/10244/statm: 1160806 1121182 176 36 0 1158711 0
[pid=10244/tid=10253] ppid=10241 vsize=4643224 CPUtime=818.6 cores=0,2,4,6
/proc/10244/task/10253/stat : 10253 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 284428 0 0 0 81782 78 0 0 25 0 5 0 520899258 4754661376 1121182 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10244/tid=10254] ppid=10241 vsize=4643224 CPUtime=821.69 cores=0,2,4,6
/proc/10244/task/10254/stat : 10254 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 299961 0 0 0 82093 76 0 0 25 0 5 0 520899258 4754661376 1121182 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10244/tid=10255] ppid=10241 vsize=4643224 CPUtime=821.79 cores=0,2,4,6
/proc/10244/task/10255/stat : 10255 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 250328 0 0 0 82110 69 0 0 25 0 5 0 520899258 4754661376 1121182 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10244/tid=10256] ppid=10241 vsize=4643224 CPUtime=822.2 cores=0,2,4,6
/proc/10244/task/10256/stat : 10256 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 283947 0 0 0 82144 76 0 0 25 0 5 0 520899258 4754661376 1121182 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3284.38
Current children cumulated vsize (KiB) 4643224

[startup+882.3 s]
/proc/loadavg: 8.19 8.08 7.96 9/190 10287
/proc/meminfo: memFree=12239472/32951124 swapFree=67111352/67111528
[pid=10244] ppid=10241 vsize=4772648 CPUtime=3524.02 cores=0,2,4,6
/proc/10244/stat : 10244 (ttsth-5-4) S 10241 10244 9884 0 -1 4202496 1154911 0 0 0 352091 311 0 0 22 0 5 0 520899249 4887191552 1154426 33554432000 4194304 4338100 140733593680688 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/10244/statm: 1193162 1154426 176 36 0 1191067 0
[pid=10244/tid=10253] ppid=10241 vsize=4772648 CPUtime=878.59 cores=0,2,4,6
/proc/10244/task/10253/stat : 10253 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 294846 0 0 0 87777 82 0 0 25 0 5 0 520899258 4887191552 1154426 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10244/tid=10254] ppid=10241 vsize=4772648 CPUtime=881.67 cores=0,2,4,6
/proc/10244/task/10254/stat : 10254 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 308190 0 0 0 88089 78 0 0 25 0 5 0 520899258 4887191552 1154426 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10244/tid=10255] ppid=10241 vsize=4772648 CPUtime=881.44 cores=0,2,4,6
/proc/10244/task/10255/stat : 10255 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 260353 0 0 0 88073 71 0 0 25 0 5 0 520899258 4887191552 1154426 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10244/tid=10256] ppid=10241 vsize=4772648 CPUtime=882.2 cores=0,2,4,6
/proc/10244/task/10256/stat : 10256 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 288519 0 0 0 88142 78 0 0 25 0 5 0 520899258 4887191552 1154426 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254404 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3524.02
Current children cumulated vsize (KiB) 4772648

[startup+942.3 s]
/proc/loadavg: 8.22 8.11 7.98 9/190 10288
/proc/meminfo: memFree=11712292/32951124 swapFree=67111352/67111528
[pid=10244] ppid=10241 vsize=4986912 CPUtime=3763.7 cores=0,2,4,6
/proc/10244/stat : 10244 (ttsth-5-4) S 10241 10244 9884 0 -1 4202496 1209461 0 0 0 376046 324 0 0 22 0 5 0 520899249 5106597888 1208976 33554432000 4194304 4338100 140733593680688 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/10244/statm: 1246728 1208976 176 36 0 1244633 0
[pid=10244/tid=10253] ppid=10241 vsize=4986912 CPUtime=938.59 cores=0,2,4,6
/proc/10244/task/10253/stat : 10253 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 311230 0 0 0 93774 85 0 0 25 0 5 0 520899258 5106597888 1208976 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10244/tid=10254] ppid=10241 vsize=4986912 CPUtime=941.67 cores=0,2,4,6
/proc/10244/task/10254/stat : 10254 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 324265 0 0 0 94085 82 0 0 25 0 5 0 520899258 5106597888 1208976 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10244/tid=10255] ppid=10241 vsize=4986912 CPUtime=941.12 cores=0,2,4,6
/proc/10244/task/10255/stat : 10255 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 269411 0 0 0 94038 74 0 0 25 0 5 0 520899258 5106597888 1208976 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254462 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10244/tid=10256] ppid=10241 vsize=4986912 CPUtime=942.2 cores=0,2,4,6
/proc/10244/task/10256/stat : 10256 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 301552 0 0 0 94139 81 0 0 25 0 5 0 520899258 5106597888 1208976 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3763.7
Current children cumulated vsize (KiB) 4986912

[startup+1002.3 s]
/proc/loadavg: 8.12 8.10 7.98 9/190 10290
/proc/meminfo: memFree=11504912/32951124 swapFree=67111352/67111528
[pid=10244] ppid=10241 vsize=5191788 CPUtime=4003.37 cores=0,2,4,6
/proc/10244/stat : 10244 (ttsth-5-4) S 10241 10244 9884 0 -1 4202496 1260306 0 0 0 399998 339 0 0 22 0 5 0 520899249 5316390912 1259821 33554432000 4194304 4338100 140733593680688 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/10244/statm: 1297947 1259821 176 36 0 1295852 0
[pid=10244/tid=10253] ppid=10241 vsize=5191788 CPUtime=998.59 cores=0,2,4,6
/proc/10244/task/10253/stat : 10253 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 324201 0 0 0 99769 90 0 0 25 0 5 0 520899258 5316390912 1259821 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254404 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10244/tid=10254] ppid=10241 vsize=5191788 CPUtime=1001.67 cores=0,2,4,6
/proc/10244/task/10254/stat : 10254 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 337043 0 0 0 100081 86 0 0 25 0 5 0 520899258 5316390912 1259821 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10244/tid=10255] ppid=10241 vsize=5191788 CPUtime=1000.8 cores=0,2,4,6
/proc/10244/task/10255/stat : 10255 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 281384 0 0 0 100003 77 0 0 25 0 5 0 520899258 5316390912 1259821 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10244/tid=10256] ppid=10241 vsize=5191788 CPUtime=1002.19 cores=0,2,4,6
/proc/10244/task/10256/stat : 10256 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 314675 0 0 0 100134 85 0 0 25 0 5 0 520899258 5316390912 1259821 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254342 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4003.37
Current children cumulated vsize (KiB) 5191788

[startup+1062.3 s]
/proc/loadavg: 8.13 8.11 8.00 9/190 10291
/proc/meminfo: memFree=11369588/32951124 swapFree=67111352/67111528
[pid=10244] ppid=10241 vsize=5321348 CPUtime=4243.03 cores=0,2,4,6
/proc/10244/stat : 10244 (ttsth-5-4) S 10241 10244 9884 0 -1 4202496 1293553 0 0 0 423952 351 0 0 22 0 5 0 520899249 5449060352 1293068 33554432000 4194304 4338100 140733593680688 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/10244/statm: 1330337 1293068 176 36 0 1328242 0
[pid=10244/tid=10253] ppid=10241 vsize=5321348 CPUtime=1058.58 cores=0,2,4,6
/proc/10244/task/10253/stat : 10253 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 332265 0 0 0 105765 93 0 0 25 0 5 0 520899258 5449060352 1293068 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10244/tid=10254] ppid=10241 vsize=5321348 CPUtime=1061.67 cores=0,2,4,6
/proc/10244/task/10254/stat : 10254 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 346562 0 0 0 106078 89 0 0 25 0 5 0 520899258 5449060352 1293068 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10244/tid=10255] ppid=10241 vsize=5321348 CPUtime=1060.46 cores=0,2,4,6
/proc/10244/task/10255/stat : 10255 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 289038 0 0 0 105967 79 0 0 25 0 5 0 520899258 5449060352 1293068 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254357 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10244/tid=10256] ppid=10241 vsize=5321348 CPUtime=1062.2 cores=0,2,4,6
/proc/10244/task/10256/stat : 10256 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 322685 0 0 0 106133 87 0 0 25 0 5 0 520899258 5449060352 1293068 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4243.03
Current children cumulated vsize (KiB) 5321348

[startup+1122.3 s]
/proc/loadavg: 8.08 8.10 8.00 9/190 10293
/proc/meminfo: memFree=11286216/32951124 swapFree=67111352/67111528
[pid=10244] ppid=10241 vsize=5446044 CPUtime=4482.67 cores=0,2,4,6
/proc/10244/stat : 10244 (ttsth-5-4) S 10241 10244 9884 0 -1 4202496 1314110 0 0 0 447911 356 0 0 22 0 5 0 520899249 5576749056 1313625 33554432000 4194304 4338100 140733593680688 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/10244/statm: 1361511 1313625 176 36 0 1359416 0
[pid=10244/tid=10253] ppid=10241 vsize=5446044 CPUtime=1118.58 cores=0,2,4,6
/proc/10244/task/10253/stat : 10253 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 340730 0 0 0 111763 95 0 0 25 0 5 0 520899258 5576749056 1313625 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10244/tid=10254] ppid=10241 vsize=5446044 CPUtime=1121.66 cores=0,2,4,6
/proc/10244/task/10254/stat : 10254 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 349894 0 0 0 112076 90 0 0 25 0 5 0 520899258 5576749056 1313625 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254337 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10244/tid=10255] ppid=10241 vsize=5446044 CPUtime=1120.12 cores=0,2,4,6
/proc/10244/task/10255/stat : 10255 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 294633 0 0 0 111931 81 0 0 25 0 5 0 520899258 5576749056 1313625 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10244/tid=10256] ppid=10241 vsize=5446044 CPUtime=1122.18 cores=0,2,4,6
/proc/10244/task/10256/stat : 10256 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 325850 0 0 0 112130 88 0 0 25 0 5 0 520899258 5576749056 1313625 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4482.67
Current children cumulated vsize (KiB) 5446044

[startup+1182.3 s]
/proc/loadavg: 8.03 8.08 8.00 9/190 10295
/proc/meminfo: memFree=11147036/32951124 swapFree=67111352/67111528
[pid=10244] ppid=10241 vsize=5447480 CPUtime=4722.36 cores=0,2,4,6
/proc/10244/stat : 10244 (ttsth-5-4) S 10241 10244 9884 0 -1 4202496 1314944 0 0 0 471877 359 0 0 22 0 5 0 520899249 5578219520 1314459 33554432000 4194304 4338100 140733593680688 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/10244/statm: 1361870 1314459 176 36 0 1359775 0
[pid=10244/tid=10253] ppid=10241 vsize=5447480 CPUtime=1178.58 cores=0,2,4,6
/proc/10244/task/10253/stat : 10253 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 340944 0 0 0 117762 96 0 0 25 0 5 0 520899258 5578219520 1314459 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254433 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10244/tid=10254] ppid=10241 vsize=5447480 CPUtime=1181.66 cores=0,2,4,6
/proc/10244/task/10254/stat : 10254 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 350121 0 0 0 118076 90 0 0 25 0 5 0 520899258 5578219520 1314459 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10244/tid=10255] ppid=10241 vsize=5447480 CPUtime=1179.8 cores=0,2,4,6
/proc/10244/task/10255/stat : 10255 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 294810 0 0 0 117898 82 0 0 25 0 5 0 520899258 5578219520 1314459 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254462 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10244/tid=10256] ppid=10241 vsize=5447480 CPUtime=1182.19 cores=0,2,4,6
/proc/10244/task/10256/stat : 10256 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 326066 0 0 0 118130 89 0 0 25 0 5 0 520899258 5578219520 1314459 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4722.36
Current children cumulated vsize (KiB) 5447480



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.8 s]
/proc/loadavg: 8.02 8.08 8.00 9/190 10295
/proc/meminfo: memFree=11048804/32951124 swapFree=67111352/67111528
[pid=10244] ppid=10241 vsize=5447868 CPUtime=4800.27 cores=0,2,4,6
/proc/10244/stat : 10244 (ttsth-5-4) S 10241 10244 9884 0 -1 4202496 1315178 0 0 0 479668 359 0 0 22 0 5 0 520899249 5578616832 1314693 33554432000 4194304 4338100 140733593680688 18446744073709551615 246528636901 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/10244/statm: 1361967 1314693 176 36 0 1359872 0
[pid=10244/tid=10253] ppid=10241 vsize=5447868 CPUtime=1198.09 cores=0,2,4,6
/proc/10244/task/10253/stat : 10253 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 341010 0 0 0 119713 96 0 0 25 0 5 0 520899258 5578616832 1314693 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=10244/tid=10254] ppid=10241 vsize=5447868 CPUtime=1201.17 cores=0,2,4,6
/proc/10244/task/10254/stat : 10254 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 350171 0 0 0 120027 90 0 0 25 0 5 0 520899258 5578616832 1314693 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254311 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=10244/tid=10255] ppid=10241 vsize=5447868 CPUtime=1199.2 cores=0,2,4,6
/proc/10244/task/10255/stat : 10255 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 294860 0 0 0 119838 82 0 0 25 0 5 0 520899258 5578616832 1314693 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=10244/tid=10256] ppid=10241 vsize=5447868 CPUtime=1201.69 cores=0,2,4,6
/proc/10244/task/10256/stat : 10256 (ttsth-5-4) R 10241 10244 9884 0 -1 4202560 326134 0 0 0 120080 89 0 0 25 0 5 0 520899258 5578616832 1314693 33554432000 4194304 4338100 140733593680688 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.27
Current children cumulated vsize (KiB) 5447868

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1202 s]
/proc/loadavg: 8.02 8.08 8.00 9/190 10295
/proc/meminfo: memFree=11048804/32951124 swapFree=67111352/67111528
[pid=10244] ppid=10241 vsize=0 CPUtime=4800.48 cores=0,2,4,6
/proc/10244/stat : 10244 (ttsth-5-4) Z 10241 10244 9884 0 -1 4203548 1315178 0 0 0 479670 378 0 0 18 0 2 0 520899249 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 2 0 0 0
/proc/10244/statm: 0 0 0 0 0 0 0
[pid=10244/tid=10255] ppid=10241 vsize=0 CPUtime=1199.39 cores=0,2,4,6
/proc/10244/task/10255/stat : 10255 (ttsth-5-4) R 10241 10244 9884 0 -1 4203588 294860 0 0 0 119838 101 0 0 25 0 2 0 520899258 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.48
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1202.04
CPU time (s): 4800.53
CPU user time (s): 4796.7
CPU system time (s): 3.82342
CPU usage (%): 399.364
Max. virtual memory (cumulated for all children) (KiB): 5447868

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4796.7
system time used= 3.82342
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1315178
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1710
involuntary context switches= 66919

runsolver used 2.00469 second user time and 4.56131 second system time

The end

Launcher Data 

Begin job on node134 at 2011-04-29 18:03:31
IDJOB=3262574
IDBENCH=83323
IDSOLVER=1654
FILE ID=node134/3262574-1304093010
RUNJOBID= node134-1304093010-10231
PBS_JOBID= 13170571
Free space on /tmp= 71584 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT11/crafted/kullmann/VanDerWaerden/VanderWaerden3k/VanDerWaerden_2-3-13_160.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3262574-1304093010/watcher-3262574-1304093010 -o /tmp/evaluation-result-3262574-1304093010/solver-3262574-1304093010 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3262574-1304093010.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= ad5b26aa9760a3817f3ab03e9314a5d6
RANDOM SEED=1628799452

node134.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 7186.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.831
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      25022344 kB
Buffers:       1489692 kB
Cached:        5853880 kB
SwapCached:         64 kB
Active:        1460864 kB
Inactive:      5946536 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      25022344 kB
SwapTotal:    67111528 kB
SwapFree:     67111352 kB
Dirty:             448 kB
Writeback:           0 kB
AnonPages:       63508 kB
Mapped:          14652 kB
Slab:           459352 kB
PageTables:       4464 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   191152 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71584 MiB
End job on node134 at 2011-04-29 18:23:34