Trace number 3260022

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ttsth 5-4? (TO) 4800.67 1202.38

General information on the benchmark

Namecrafted/mosoi/
289/289-sat-12x15.cnf
MD5SUMc77d740bbf7d77b30e6053138dd22e73
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark0.083987
Satisfiable
(Un)Satisfiability was proved
Number of variables720
Number of clauses28980
Sum of the clauses size113760
Maximum clause length4
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 21080
Number of clauses of size 30
Number of clauses of size 427900
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.03	c Command: HOME/ttsth-5-4 -num-threads 4 HOME/instance-3260022-1304061536.cnf 
0.00/0.03	c Using SATTIMEOUT of 4800
0.00/0.03	c Using SATRAM of 15500
0.00/0.03	c Size of subset node = 32
0.00/0.03	c Variables = 720
0.00/0.03	c Clauses = 28980
0.00/0.03	c Literals = 113760
0.00/0.03	c Reduced size:
0.00/0.03	c Variables = 720
0.00/0.03	c Clauses = 28980
0.00/0.03	c Literals = 113760
0.00/0.03	c a2
0.07/0.08	c a1
0.09/0.16	c Independent sub-problems = 1
0.09/0.16	c Sub-problem with 720 variables and 28980 clauses
0.09/0.16	c using permutation method 4
0.09/0.16	c count = 1
0.09/0.16	c Ordering finished 0.150 seconds
0.09/0.16	c mapped 0.150 seconds
0.09/0.16	c permuted 0.150 seconds
0.09/0.16	c about to sort
0.09/0.16	c sorted
0.19/0.21	c clauses now 28980
0.19/0.21	c init finished 0.160 seconds
0.19/0.21	c tree built 0.210 seconds
0.19/0.21	c 
0.19/0.21	c PA progress                      | CPU progress Partial assignments | improvements                 ucps  CPU time       PA/sec  nodes
0.19/0.21	c ---------------------------------|----------------------------------|--------------------------------------------------------------------
0.19/0.21	c ................................ | ..........                    |                                  0.2         0.0             
0.19/0.21	c ................................ | ..........                  1 |                                  0.2         4.8             
0.19/0.21	c ................................ | ..........                  2 |                                  0.2         9.5             
0.19/0.21	c ................................ | ..........                  3 |                                  0.2        14.3             
0.19/0.21	c ................................ | ..........                  4 |                                  0.2        19.0             
0.19/0.21	c ................................ | ..........                  5 |                                  0.2        23.8             
0.19/0.22	c ................................ | ..........               1000 | 25           584                 0.2      4347.8         2525
96.02/24.22	c ................................ | ..........            1131000 | 109710       691678             96.1     11770.2     20570623
191.88/48.26	c ................................ | ..........            1677000 | 164459       1007535           192.2      8727.6     32069934
287.76/72.30	c ................................ | ..........            2074000 | 203212       1249132           288.2      7197.4     42856132
384.04/96.32	c ................................ | ..........            2401000 | 237264       1449580           384.1      6250.2     50636693
480.32/120.43	c ................................ | *.........            2663000 | 266271       1656071           480.4      5542.7     59543951
575.81/144.36	c ................................ | *.........            2899000 | 293824       1833136           576.0      5032.6     68081975
671.71/168.38	c ................................ | *.........            3131000 | 315610       1980260           672.0      4659.0     74853774
767.96/192.44	c ................................ | *.........            3346000 | 335053       2085825           768.1      4356.1     79794450
863.75/216.48	c ................................ | *.........            3539000 | 354473       2204682           864.1      4095.6     85359540
960.37/240.65	c ................................ | **........            3737000 | 374860       2322778           960.6      3890.3     89943332
1055.83/264.59	c ................................ | **........            3899000 | 391091       2442064          1056.2      3691.5     94227641
1152.10/288.65	c ................................ | **........            4051000 | 408397       2549014          1152.3      3515.5    100253622
1248.38/312.71	c ................................ | **........            4208000 | 425794       2658692          1248.4      3370.6    106362705
1343.86/336.69	c ................................ | **........            4429000 | 453588       2780208          1344.2      3294.9    113222163
1439.71/360.69	c ................................ | ***.......            4702000 | 488548       2896872          1440.1      3265.1    119603757
1535.99/384.76	c ................................ | ***.......            4963000 | 519880       3009642          1536.2      3230.6    124841726
1632.25/408.88	c ................................ | ***.......            5186000 | 543235       3115433          1632.6      3176.6    129680237
1728.13/432.85	c ................................ | ***.......            5393000 | 567359       3224439          1728.3      3120.3    134806735
1824.01/456.83	c ................................ | ***.......            5606000 | 589327       3332708          1824.1      3073.2    141012613
1919.87/480.83	c ................................ | ****......            5805000 | 609533       3416085          1920.0      3023.4    146810758
2016.51/505.08	c ................................ | ****......            5994000 | 630216       3502586          2016.8      2972.0    152360785
2111.98/528.92	c ................................ | ****......            6202000 | 655212       3572598          2112.1      2936.4    159335016
2207.80/552.97	c ................................ | ****......            6361000 | 672957       3655211          2208.1      2880.8    162999374
2304.07/577.08	c ................................ | ****......            6514000 | 690293       3738741          2304.4      2826.8    167946370
2399.91/601.09	c ................................ | *****.....            6671000 | 707261       3811785          2400.3      2779.3    173586259
2496.18/625.10	c ................................ | *****.....            6829000 | 725814       3890391          2496.2      2735.8    178536769
2592.05/649.17	c ................................ | *****.....            6979000 | 745794       3955258          2592.3      2692.2    183608769
2688.31/673.25	c ................................ | *****.....            7112000 | 761600       4013540          2688.5      2645.3    187986220
2784.15/697.29	c ................................ | *****.....            7229000 | 775135       4072320          2784.5      2596.1    194460039
2880.02/721.23	c ................................ | ******....            7347000 | 788455       4135835          2880.2      2550.9    199617936
2975.90/745.26	c ................................ | ******....            7488000 | 803577       4211010          2976.1      2516.0    204116552
3072.17/769.39	c ................................ | ******....            7606000 | 817551       4273797          3072.5      2475.5    209150114
3168.03/793.39	c ................................ | ******....            7717000 | 832795       4331309          3168.4      2435.6    214032688
3263.93/817.39	c ................................ | ******....            7816000 | 844954       4383311          3264.3      2394.4    219165494
3360.96/841.68	c ................................ | *******...            7930000 | 859599       4432308          3361.3      2359.2    224279868
3456.43/865.51	c ................................ | *******...            8038000 | 873684       4488389          3456.5      2325.5    229576381
3552.30/889.51	c ................................ | *******...            8168000 | 886754       4544625          3552.3      2299.3    233531827
3648.55/913.60	c ................................ | *******...            8299000 | 899823       4600784          3648.6      2274.6    238382977
3743.99/937.52	c ................................ | *******...            8426000 | 913534       4650914          3744.1      2250.5    241651875
3840.63/961.74	c ................................ | ********..            8563000 | 929926       4716803          3840.8      2229.5    242898271
3936.12/985.68	c ................................ | ********..            8658000 | 941569       4757603          3936.4      2199.5    247120991
4031.90/1009.62	c ................................ | ********..            8758000 | 952511       4810076          4032.0      2172.1    250667849
4128.18/1033.70	c ................................ | ********..            8876000 | 966834       4857849          4128.2      2150.1    255671608
4224.04/1057.75	c ................................ | ********..            8994000 | 979424       4910859          4224.2      2129.1    259685016
4320.70/1081.94	c ................................ | *********.            9096000 | 990650       4959769          4320.9      2105.1    264069002
4415.78/1105.76	c ................................ | *********.            9211000 | 1005592      5014408          4416.0      2085.8    268722519
4512.45/1129.95	c ................................ | *********.            9323000 | 1017645      5058059          4512.6      2066.0    273271856
4608.71/1154.06	c ................................ | *********.            9424000 | 1029487      5105260          4609.0      2044.7    278574218
4704.61/1178.03	c ................................ | *********.            9544000 | 1041766      5158347          4704.7      2028.6    282573037

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3260022-1304061536/watcher-3260022-1304061536 -o /tmp/evaluation-result-3260022-1304061536/solver-3260022-1304061536 -C 4800 -W 1300 -M 15500 HOME/ttsth-5-4 -num-threads 4 HOME/instance-3260022-1304061536.cnf 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.12 8.10 8.01 6/181 28539
/proc/meminfo: memFree=12342164/32950928 swapFree=67111528/67111528
[pid=28539] ppid=28537 vsize=9408 CPUtime=0 cores=0,2,4,6
/proc/28539/stat : 28539 (ttsth-5-4) R 28537 28539 28330 0 -1 4202496 407 0 0 0 0 0 0 0 25 0 1 0 596384246 9633792 350 33554432000 4194304 4338100 140735838151664 18446744073709551615 218880225989 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/28539/statm: 2352 351 147 36 0 257 0

[startup+0.0836581 s]
/proc/loadavg: 8.12 8.10 8.01 6/181 28539
/proc/meminfo: memFree=12342164/32950928 swapFree=67111528/67111528
[pid=28539] ppid=28537 vsize=18224 CPUtime=0.07 cores=0,2,4,6
/proc/28539/stat : 28539 (ttsth-5-4) R 28537 28539 28330 0 -1 4202496 2613 0 0 0 7 0 0 0 25 0 1 0 596384246 18661376 2555 33554432000 4194304 4338100 140735838151664 18446744073709551615 218879960080 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/28539/statm: 4556 2555 153 36 0 2461 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 18224

[startup+0.100655 s]
/proc/loadavg: 8.12 8.10 8.01 6/181 28539
/proc/meminfo: memFree=12342164/32950928 swapFree=67111528/67111528
[pid=28539] ppid=28537 vsize=13724 CPUtime=0.09 cores=0,2,4,6
/proc/28539/stat : 28539 (ttsth-5-4) R 28537 28539 28330 0 -1 4202496 2736 0 0 0 9 0 0 0 25 0 1 0 596384246 14053376 1444 33554432000 4194304 4338100 140735838151664 18446744073709551615 218880225989 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/28539/statm: 3431 1447 155 36 0 1336 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 13724

[startup+0.300552 s]
/proc/loadavg: 8.12 8.10 8.01 6/181 28539
/proc/meminfo: memFree=12342164/32950928 swapFree=67111528/67111528
[pid=28539] ppid=28537 vsize=145516 CPUtime=0.54 cores=0,2,4,6
/proc/28539/stat : 28539 (ttsth-5-4) S 28537 28539 28330 0 -1 4202496 6487 0 0 0 53 1 0 0 25 0 5 0 596384246 149008384 5137 33554432000 4194304 4338100 140735838151664 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/28539/statm: 36379 5137 176 36 0 34284 0
Current children cumulated CPU time (s) 0.54
Current children cumulated vsize (KiB) 145516

[startup+0.700483 s]
/proc/loadavg: 8.12 8.10 8.01 6/181 28539
/proc/meminfo: memFree=12342164/32950928 swapFree=67111528/67111528
[pid=28539] ppid=28537 vsize=217968 CPUtime=2.13 cores=0,2,4,6
/proc/28539/stat : 28539 (ttsth-5-4) S 28537 28539 28330 0 -1 4202496 8430 0 0 0 212 1 0 0 25 0 5 0 596384246 223199232 7080 33554432000 4194304 4338100 140735838151664 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/28539/statm: 54492 7080 176 36 0 52397 0
Current children cumulated CPU time (s) 2.13
Current children cumulated vsize (KiB) 217968

[startup+1.50034 s]
/proc/loadavg: 8.12 8.10 8.01 9/186 28544
/proc/meminfo: memFree=12306792/32950928 swapFree=67111528/67111528
[pid=28539] ppid=28537 vsize=231668 CPUtime=5.33 cores=0,2,4,6
/proc/28539/stat : 28539 (ttsth-5-4) S 28537 28539 28330 0 -1 4202496 13573 0 0 0 530 3 0 0 25 0 5 0 596384246 237228032 12223 33554432000 4194304 4338100 140735838151664 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/28539/statm: 57917 12223 176 36 0 55822 0
[pid=28539/tid=28541] ppid=28537 vsize=231668 CPUtime=1.27 cores=0,2,4,6
/proc/28539/task/28541/stat : 28541 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 989 0 0 0 127 0 0 0 19 0 5 0 596384267 237228032 12223 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=28539/tid=28542] ppid=28537 vsize=231668 CPUtime=1.27 cores=0,2,4,6
/proc/28539/task/28542/stat : 28542 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 1660 0 0 0 127 0 0 0 19 0 5 0 596384267 237228032 12223 33554432000 4194304 4338100 140735838151664 18446744073709551615 4256139 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=28539/tid=28543] ppid=28537 vsize=231668 CPUtime=1.27 cores=0,2,4,6
/proc/28539/task/28543/stat : 28543 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 2314 0 0 0 127 0 0 0 25 0 5 0 596384267 237228032 12223 33554432000 4194304 4338100 140735838151664 18446744073709551615 4285854 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=28539/tid=28544] ppid=28537 vsize=231668 CPUtime=1.27 cores=0,2,4,6
/proc/28539/task/28544/stat : 28544 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 2742 0 0 0 127 0 0 0 19 0 5 0 596384267 237228032 12223 33554432000 4194304 4338100 140735838151664 18446744073709551615 4256749 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 5.33
Current children cumulated vsize (KiB) 231668

[startup+3.10107 s]
/proc/loadavg: 8.11 8.10 8.01 9/186 28545
/proc/meminfo: memFree=12271700/32950928 swapFree=67111528/67111528
[pid=28539] ppid=28537 vsize=286228 CPUtime=11.72 cores=0,2,4,6
/proc/28539/stat : 28539 (ttsth-5-4) S 28537 28539 28330 0 -1 4202496 26233 0 0 0 1168 4 0 0 25 0 5 0 596384246 293097472 24883 33554432000 4194304 4338100 140735838151664 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/28539/statm: 71557 24883 176 36 0 69462 0
[pid=28539/tid=28541] ppid=28537 vsize=286228 CPUtime=2.87 cores=0,2,4,6
/proc/28539/task/28541/stat : 28541 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 3800 0 0 0 287 0 0 0 25 0 5 0 596384267 293097472 24883 33554432000 4194304 4338100 140735838151664 18446744073709551615 4256013 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=28539/tid=28542] ppid=28537 vsize=286228 CPUtime=2.87 cores=0,2,4,6
/proc/28539/task/28542/stat : 28542 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 5403 0 0 0 287 0 0 0 23 0 5 0 596384267 293097472 24883 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=28539/tid=28543] ppid=28537 vsize=286228 CPUtime=2.86 cores=0,2,4,6
/proc/28539/task/28543/stat : 28543 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 4232 0 0 0 286 0 0 0 25 0 5 0 596384267 293097472 24883 33554432000 4194304 4338100 140735838151664 18446744073709551615 218879960211 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=28539/tid=28544] ppid=28537 vsize=286228 CPUtime=2.88 cores=0,2,4,6
/proc/28539/task/28544/stat : 28544 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 6930 0 0 0 287 1 0 0 23 0 5 0 596384267 293097472 24883 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254384 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 11.72
Current children cumulated vsize (KiB) 286228

[startup+6.30051 s]
/proc/loadavg: 8.11 8.10 8.01 9/186 28545
/proc/meminfo: memFree=12137852/32950928 swapFree=67111528/67111528
[pid=28539] ppid=28537 vsize=422844 CPUtime=24.5 cores=0,2,4,6
/proc/28539/stat : 28539 (ttsth-5-4) S 28537 28539 28330 0 -1 4202496 61407 0 0 0 2440 10 0 0 25 0 5 0 596384246 432992256 60057 33554432000 4194304 4338100 140735838151664 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/28539/statm: 105711 60057 176 36 0 103616 0
[pid=28539/tid=28541] ppid=28537 vsize=422844 CPUtime=6.08 cores=0,2,4,6
/proc/28539/task/28541/stat : 28541 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 15781 0 0 0 605 3 0 0 25 0 5 0 596384267 432992256 60057 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=28539/tid=28542] ppid=28537 vsize=422844 CPUtime=6.07 cores=0,2,4,6
/proc/28539/task/28542/stat : 28542 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 13920 0 0 0 606 1 0 0 25 0 5 0 596384267 432992256 60057 33554432000 4194304 4338100 140735838151664 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=28539/tid=28543] ppid=28537 vsize=422844 CPUtime=6.04 cores=0,2,4,6
/proc/28539/task/28543/stat : 28543 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 11464 0 0 0 602 2 0 0 25 0 5 0 596384267 432992256 60057 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=28539/tid=28544] ppid=28537 vsize=422844 CPUtime=6.07 cores=0,2,4,6
/proc/28539/task/28544/stat : 28544 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 14374 0 0 0 605 2 0 0 25 0 5 0 596384267 432992256 60058 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 24.5
Current children cumulated vsize (KiB) 422844

[startup+12.7004 s]
/proc/loadavg: 8.09 8.09 8.01 9/186 28545
/proc/meminfo: memFree=11898180/32950928 swapFree=67111528/67111528
[pid=28539] ppid=28537 vsize=618700 CPUtime=50.06 cores=0,2,4,6
/proc/28539/stat : 28539 (ttsth-5-4) S 28537 28539 28330 0 -1 4202496 110874 0 0 0 4986 20 0 0 25 0 5 0 596384246 633548800 109524 33554432000 4194304 4338100 140735838151664 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/28539/statm: 154675 109524 176 36 0 152580 0
[pid=28539/tid=28541] ppid=28537 vsize=618700 CPUtime=12.47 cores=0,2,4,6
/proc/28539/task/28541/stat : 28541 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 33903 0 0 0 1241 6 0 0 25 0 5 0 596384267 633548800 109524 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=28539/tid=28542] ppid=28537 vsize=618700 CPUtime=12.47 cores=0,2,4,6
/proc/28539/task/28542/stat : 28542 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 26124 0 0 0 1243 4 0 0 25 0 5 0 596384267 633548800 109524 33554432000 4194304 4338100 140735838151664 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=28539/tid=28543] ppid=28537 vsize=618700 CPUtime=12.41 cores=0,2,4,6
/proc/28539/task/28543/stat : 28543 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 20968 0 0 0 1237 4 0 0 25 0 5 0 596384267 633548800 109524 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=28539/tid=28544] ppid=28537 vsize=618700 CPUtime=12.47 cores=0,2,4,6
/proc/28539/task/28544/stat : 28544 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 24011 0 0 0 1243 4 0 0 25 0 5 0 596384267 633548800 109524 33554432000 4194304 4338100 140735838151664 18446744073709551615 4256352 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 50.06
Current children cumulated vsize (KiB) 618700

[startup+25.5012 s]
/proc/loadavg: 8.08 8.09 8.01 9/186 28545
/proc/meminfo: memFree=11610960/32950928 swapFree=67111528/67111528
[pid=28539] ppid=28537 vsize=881052 CPUtime=101.2 cores=0,2,4,6
/proc/28539/stat : 28539 (ttsth-5-4) S 28537 28539 28330 0 -1 4202496 176625 0 0 0 10086 34 0 0 25 0 5 0 596384246 902197248 175275 33554432000 4194304 4338100 140735838151664 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/28539/statm: 220263 175275 176 36 0 218168 0
[pid=28539/tid=28541] ppid=28537 vsize=881052 CPUtime=25.27 cores=0,2,4,6
/proc/28539/task/28541/stat : 28541 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 55200 0 0 0 2517 10 0 0 25 0 5 0 596384267 902197248 175275 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254362 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=28539/tid=28542] ppid=28537 vsize=881052 CPUtime=25.27 cores=0,2,4,6
/proc/28539/task/28542/stat : 28542 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 41851 0 0 0 2519 8 0 0 25 0 5 0 596384267 902197248 175276 33554432000 4194304 4338100 140735838151664 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=28539/tid=28543] ppid=28537 vsize=881052 CPUtime=25.13 cores=0,2,4,6
/proc/28539/task/28543/stat : 28543 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 39068 0 0 0 2506 7 0 0 25 0 5 0 596384267 902197248 175277 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=28539/tid=28544] ppid=28537 vsize=881052 CPUtime=25.27 cores=0,2,4,6
/proc/28539/task/28544/stat : 28544 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 34639 0 0 0 2521 6 0 0 25 0 5 0 596384267 902197248 175277 33554432000 4194304 4338100 140735838151664 18446744073709551615 218879960814 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 101.2
Current children cumulated vsize (KiB) 881052

[startup+51.1007 s]
/proc/loadavg: 7.60 7.97 7.97 9/186 28578
/proc/meminfo: memFree=13221416/32950928 swapFree=67111528/67111528
[pid=28539] ppid=28537 vsize=1260992 CPUtime=203.47 cores=0,2,4,6
/proc/28539/stat : 28539 (ttsth-5-4) S 28537 28539 28330 0 -1 4202496 272573 0 0 0 20291 56 0 0 25 0 5 0 596384246 1291255808 271223 33554432000 4194304 4338100 140735838151664 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/28539/statm: 315248 271223 176 36 0 313153 0
[pid=28539/tid=28541] ppid=28537 vsize=1260992 CPUtime=50.87 cores=0,2,4,6
/proc/28539/task/28541/stat : 28541 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 84824 0 0 0 5070 17 0 0 25 0 5 0 596384267 1291255808 271223 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254294 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=28539/tid=28542] ppid=28537 vsize=1260992 CPUtime=50.87 cores=0,2,4,6
/proc/28539/task/28542/stat : 28542 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 60343 0 0 0 5075 12 0 0 25 0 5 0 596384267 1291255808 271223 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254304 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=28539/tid=28543] ppid=28537 vsize=1260992 CPUtime=50.61 cores=0,2,4,6
/proc/28539/task/28543/stat : 28543 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 68846 0 0 0 5048 13 0 0 25 0 5 0 596384267 1291255808 271224 33554432000 4194304 4338100 140735838151664 18446744073709551615 218879960313 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=28539/tid=28544] ppid=28537 vsize=1260992 CPUtime=50.87 cores=0,2,4,6
/proc/28539/task/28544/stat : 28544 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 52693 0 0 0 5076 11 0 0 25 0 5 0 596384267 1291255808 271224 33554432000 4194304 4338100 140735838151664 18446744073709551615 4234407 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 203.47
Current children cumulated vsize (KiB) 1260992

[startup+102.307 s]
/proc/loadavg: 7.84 7.97 7.97 9/186 28579
/proc/meminfo: memFree=12367520/32950928 swapFree=67111528/67111528
[pid=28539] ppid=28537 vsize=1900048 CPUtime=408.03 cores=0,2,4,6
/proc/28539/stat : 28539 (ttsth-5-4) S 28537 28539 28330 0 -1 4202496 432181 0 0 0 40715 88 0 0 25 0 5 0 596384246 1945649152 430831 33554432000 4194304 4338100 140735838151664 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/28539/statm: 475012 430831 176 36 0 472917 0
[pid=28539/tid=28541] ppid=28537 vsize=1900048 CPUtime=102.07 cores=0,2,4,6
/proc/28539/task/28541/stat : 28541 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 128594 0 0 0 10182 25 0 0 25 0 5 0 596384267 1945649152 430831 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=28539/tid=28542] ppid=28537 vsize=1900048 CPUtime=102.07 cores=0,2,4,6
/proc/28539/task/28542/stat : 28542 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 108638 0 0 0 10185 22 0 0 25 0 5 0 596384267 1945649152 430831 33554432000 4194304 4338100 140735838151664 18446744073709551615 4256767 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=28539/tid=28543] ppid=28537 vsize=1900048 CPUtime=101.56 cores=0,2,4,6
/proc/28539/task/28543/stat : 28543 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 97803 0 0 0 10137 19 0 0 25 0 5 0 596384267 1945649152 430831 33554432000 4194304 4338100 140735838151664 18446744073709551615 4284177 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=28539/tid=28544] ppid=28537 vsize=1900048 CPUtime=102.08 cores=0,2,4,6
/proc/28539/task/28544/stat : 28544 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 91278 0 0 0 10188 20 0 0 25 0 5 0 596384267 1945649152 430831 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 408.03
Current children cumulated vsize (KiB) 1900048

[startup+162.3 s]
/proc/loadavg: 7.98 7.99 7.97 9/186 28581
/proc/meminfo: memFree=11560164/32950928 swapFree=67111528/67111528
[pid=28539] ppid=28537 vsize=2533860 CPUtime=647.73 cores=0,2,4,6
/proc/28539/stat : 28539 (ttsth-5-4) S 28537 28539 28330 0 -1 4202496 592015 0 0 0 64651 122 0 0 25 0 5 0 596384246 2594672640 590665 33554432000 4194304 4338100 140735838151664 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/28539/statm: 633465 590665 176 36 0 631370 0
[pid=28539/tid=28541] ppid=28537 vsize=2533860 CPUtime=162.06 cores=0,2,4,6
/proc/28539/task/28541/stat : 28541 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 171628 0 0 0 16172 34 0 0 25 0 5 0 596384267 2594672640 590665 33554432000 4194304 4338100 140735838151664 18446744073709551615 4256013 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=28539/tid=28542] ppid=28537 vsize=2533860 CPUtime=162.06 cores=0,2,4,6
/proc/28539/task/28542/stat : 28542 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 155791 0 0 0 16174 32 0 0 25 0 5 0 596384267 2594672640 590665 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=28539/tid=28543] ppid=28537 vsize=2533860 CPUtime=161.27 cores=0,2,4,6
/proc/28539/task/28543/stat : 28543 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 131288 0 0 0 16102 25 0 0 25 0 5 0 596384267 2594672640 590665 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
[pid=28539/tid=28544] ppid=28537 vsize=2533860 CPUtime=162.07 cores=0,2,4,6
/proc/28539/task/28544/stat : 28544 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 127440 0 0 0 16180 27 0 0 25 0 5 0 596384267 2594672640 590665 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 647.73
Current children cumulated vsize (KiB) 2533860


################
# More data... #
################

/proc/28539/task/28544/stat : 28544 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 322482 0 0 0 76126 70 0 0 25 0 5 0 596384267 6943870976 1654778 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3044.22
Current children cumulated vsize (KiB) 6781124

[startup+822.301 s]
/proc/loadavg: 8.11 8.08 8.01 9/186 28598
/proc/meminfo: memFree=5436044/32950928 swapFree=67111528/67111528
[pid=28539] ppid=28537 vsize=7173048 CPUtime=3283.88 cores=0,2,4,6
/proc/28539/stat : 28539 (ttsth-5-4) S 28537 28539 28330 0 -1 4202496 1753951 0 0 0 328011 377 0 0 25 0 5 0 596384246 7345201152 1752601 33554432000 4194304 4338100 140735838151664 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/28539/statm: 1793262 1752601 176 36 0 1791167 0
[pid=28539/tid=28541] ppid=28537 vsize=7173048 CPUtime=822.02 cores=0,2,4,6
/proc/28539/task/28541/stat : 28541 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 557717 0 0 0 82078 124 0 0 25 0 5 0 596384267 7345201152 1752601 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=28539/tid=28542] ppid=28537 vsize=7173048 CPUtime=822.05 cores=0,2,4,6
/proc/28539/task/28542/stat : 28542 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 468111 0 0 0 82101 104 0 0 25 0 5 0 596384267 7345201152 1752601 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=28539/tid=28543] ppid=28537 vsize=7173048 CPUtime=817.86 cores=0,2,4,6
/proc/28539/task/28543/stat : 28543 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 376364 0 0 0 81714 72 0 0 25 0 5 0 596384267 7345201152 1752601 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=28539/tid=28544] ppid=28537 vsize=7173048 CPUtime=821.71 cores=0,2,4,6
/proc/28539/task/28544/stat : 28544 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 345891 0 0 0 82097 74 0 0 25 0 5 0 596384267 7345201152 1752601 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254439 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3283.88
Current children cumulated vsize (KiB) 7173048

[startup+882.301 s]
/proc/loadavg: 8.08 8.08 8.01 9/186 28600
/proc/meminfo: memFree=4949880/32950928 swapFree=67111528/67111528
[pid=28539] ppid=28537 vsize=7561580 CPUtime=3523.54 cores=0,2,4,6
/proc/28539/stat : 28539 (ttsth-5-4) S 28537 28539 28330 0 -1 4202496 1850650 0 0 0 351958 396 0 0 25 0 5 0 596384246 7743057920 1849300 33554432000 4194304 4338100 140735838151664 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/28539/statm: 1890395 1849300 176 36 0 1888300 0
[pid=28539/tid=28541] ppid=28537 vsize=7561580 CPUtime=882.02 cores=0,2,4,6
/proc/28539/task/28541/stat : 28541 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 585211 0 0 0 88073 129 0 0 25 0 5 0 596384267 7743057920 1849300 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=28539/tid=28542] ppid=28537 vsize=7561580 CPUtime=882.03 cores=0,2,4,6
/proc/28539/task/28542/stat : 28542 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 499726 0 0 0 88093 110 0 0 25 0 5 0 596384267 7743057920 1849300 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=28539/tid=28543] ppid=28537 vsize=7561580 CPUtime=877.86 cores=0,2,4,6
/proc/28539/task/28543/stat : 28543 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 396771 0 0 0 87710 76 0 0 25 0 5 0 596384267 7743057920 1849300 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=28539/tid=28544] ppid=28537 vsize=7561580 CPUtime=881.38 cores=0,2,4,6
/proc/28539/task/28544/stat : 28544 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 363074 0 0 0 88060 78 0 0 25 0 5 0 596384267 7743057920 1849300 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3523.54
Current children cumulated vsize (KiB) 7561580

[startup+942.301 s]
/proc/loadavg: 8.13 8.09 8.01 9/186 28602
/proc/meminfo: memFree=4593124/32950928 swapFree=67111528/67111528
[pid=28539] ppid=28537 vsize=7866644 CPUtime=3763.15 cores=0,2,4,6
/proc/28539/stat : 28539 (ttsth-5-4) S 28537 28539 28330 0 -1 4202496 1926117 0 0 0 375901 414 0 0 25 0 5 0 596384246 8055443456 1924767 33554432000 4194304 4338100 140735838151664 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/28539/statm: 1966661 1924767 176 36 0 1964566 0
[pid=28539/tid=28541] ppid=28537 vsize=7866644 CPUtime=942.01 cores=0,2,4,6
/proc/28539/task/28541/stat : 28541 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 605201 0 0 0 94068 133 0 0 25 0 5 0 596384267 8055443456 1924767 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=28539/tid=28542] ppid=28537 vsize=7866644 CPUtime=942 cores=0,2,4,6
/proc/28539/task/28542/stat : 28542 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 517995 0 0 0 94084 116 0 0 25 0 5 0 596384267 8055443456 1924767 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=28539/tid=28543] ppid=28537 vsize=7866644 CPUtime=937.84 cores=0,2,4,6
/proc/28539/task/28543/stat : 28543 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 415541 0 0 0 93704 80 0 0 25 0 5 0 596384267 8055443456 1924767 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=28539/tid=28544] ppid=28537 vsize=7866644 CPUtime=941.05 cores=0,2,4,6
/proc/28539/task/28544/stat : 28544 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 381512 0 0 0 94022 83 0 0 25 0 5 0 596384267 8055443456 1924767 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3763.15
Current children cumulated vsize (KiB) 7866644

[startup+1002.3 s]
/proc/loadavg: 8.09 8.09 8.01 9/186 28603
/proc/meminfo: memFree=4247840/32950928 swapFree=67111528/67111528
[pid=28539] ppid=28537 vsize=8101772 CPUtime=4002.75 cores=0,2,4,6
/proc/28539/stat : 28539 (ttsth-5-4) S 28537 28539 28330 0 -1 4202496 1984540 0 0 0 399845 430 0 0 25 0 5 0 596384246 8296214528 1983190 33554432000 4194304 4338100 140735838151664 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/28539/statm: 2025443 1983190 176 36 0 2023348 0
[pid=28539/tid=28541] ppid=28537 vsize=8101772 CPUtime=1002 cores=0,2,4,6
/proc/28539/task/28541/stat : 28541 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 617752 0 0 0 100063 137 0 0 25 0 5 0 596384267 8296214528 1983190 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=28539/tid=28542] ppid=28537 vsize=8101772 CPUtime=1001.98 cores=0,2,4,6
/proc/28539/task/28542/stat : 28542 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 532275 0 0 0 100078 120 0 0 25 0 5 0 596384267 8296214528 1983190 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254208 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=28539/tid=28543] ppid=28537 vsize=8101772 CPUtime=997.8 cores=0,2,4,6
/proc/28539/task/28543/stat : 28543 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 430236 0 0 0 99697 83 0 0 25 0 5 0 596384267 8296214528 1983190 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=28539/tid=28544] ppid=28537 vsize=8101772 CPUtime=1000.72 cores=0,2,4,6
/proc/28539/task/28544/stat : 28544 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 398409 0 0 0 99985 87 0 0 25 0 5 0 596384267 8296214528 1983190 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254321 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4002.75
Current children cumulated vsize (KiB) 8101772

[startup+1062.3 s]
/proc/loadavg: 8.07 8.08 8.01 9/186 28605
/proc/meminfo: memFree=3819588/32950928 swapFree=67111528/67111528
[pid=28539] ppid=28537 vsize=8452768 CPUtime=4242.42 cores=0,2,4,6
/proc/28539/stat : 28539 (ttsth-5-4) S 28537 28539 28330 0 -1 4202496 2073274 0 0 0 423789 453 0 0 25 0 5 0 596384246 8655634432 2071924 33554432000 4194304 4338100 140735838151664 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/28539/statm: 2113192 2071924 176 36 0 2111097 0
[pid=28539/tid=28541] ppid=28537 vsize=8452768 CPUtime=1061.99 cores=0,2,4,6
/proc/28539/task/28541/stat : 28541 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 646612 0 0 0 106056 143 0 0 25 0 5 0 596384267 8655634432 2071924 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254321 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=28539/tid=28542] ppid=28537 vsize=8452768 CPUtime=1061.96 cores=0,2,4,6
/proc/28539/task/28542/stat : 28542 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 552364 0 0 0 106071 125 0 0 25 0 5 0 596384267 8655634432 2071924 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=28539/tid=28543] ppid=28537 vsize=8452768 CPUtime=1057.81 cores=0,2,4,6
/proc/28539/task/28543/stat : 28543 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 452697 0 0 0 105692 89 0 0 25 0 5 0 596384267 8655634432 2071924 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254289 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=28539/tid=28544] ppid=28537 vsize=8452768 CPUtime=1060.4 cores=0,2,4,6
/proc/28539/task/28544/stat : 28544 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 415733 0 0 0 105947 93 0 0 25 0 5 0 596384267 8655634432 2071924 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4242.42
Current children cumulated vsize (KiB) 8452768

[startup+1122.3 s]
/proc/loadavg: 8.11 8.09 8.02 9/186 28606
/proc/meminfo: memFree=3371476/32950928 swapFree=67111528/67111528
[pid=28539] ppid=28537 vsize=8803764 CPUtime=4482.08 cores=0,2,4,6
/proc/28539/stat : 28539 (ttsth-5-4) S 28537 28539 28330 0 -1 4202496 2161419 0 0 0 447736 472 0 0 25 0 5 0 596384246 9015054336 2160069 33554432000 4194304 4338100 140735838151664 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/28539/statm: 2200941 2160069 176 36 0 2198846 0
[pid=28539/tid=28541] ppid=28537 vsize=8803764 CPUtime=1122 cores=0,2,4,6
/proc/28539/task/28541/stat : 28541 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 673013 0 0 0 112051 149 0 0 25 0 5 0 596384267 9015054336 2160069 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254315 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=28539/tid=28542] ppid=28537 vsize=8803764 CPUtime=1121.97 cores=0,2,4,6
/proc/28539/task/28542/stat : 28542 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 572004 0 0 0 112066 131 0 0 25 0 5 0 596384267 9015054336 2160069 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=28539/tid=28543] ppid=28537 vsize=8803764 CPUtime=1117.8 cores=0,2,4,6
/proc/28539/task/28543/stat : 28543 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 476835 0 0 0 111686 94 0 0 25 0 5 0 596384267 9015054336 2160069 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254307 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=28539/tid=28544] ppid=28537 vsize=8803764 CPUtime=1120.07 cores=0,2,4,6
/proc/28539/task/28544/stat : 28544 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 433699 0 0 0 111911 96 0 0 25 0 5 0 596384267 9015054336 2160069 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254337 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4482.08
Current children cumulated vsize (KiB) 8803764

[startup+1182.3 s]
/proc/loadavg: 8.08 8.09 8.02 9/186 28608
/proc/meminfo: memFree=2902348/32950928 swapFree=67111528/67111528
[pid=28539] ppid=28537 vsize=9170400 CPUtime=4721.77 cores=0,2,4,6
/proc/28539/stat : 28539 (ttsth-5-4) S 28537 28539 28330 0 -1 4202496 2253739 0 0 0 471685 492 0 0 25 0 5 0 596384246 9390489600 2252389 33554432000 4194304 4338100 140735838151664 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/28539/statm: 2292600 2252389 176 36 0 2290505 0
[pid=28539/tid=28541] ppid=28537 vsize=9170400 CPUtime=1182 cores=0,2,4,6
/proc/28539/task/28541/stat : 28541 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 696815 0 0 0 118045 155 0 0 25 0 5 0 596384267 9390489600 2252389 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=28539/tid=28542] ppid=28537 vsize=9170400 CPUtime=1181.97 cores=0,2,4,6
/proc/28539/task/28542/stat : 28542 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 601317 0 0 0 118060 137 0 0 25 0 5 0 596384267 9390489600 2252389 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=28539/tid=28543] ppid=28537 vsize=9170400 CPUtime=1177.81 cores=0,2,4,6
/proc/28539/task/28543/stat : 28543 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 494414 0 0 0 117682 99 0 0 25 0 5 0 596384267 9390489600 2252389 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254291 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=28539/tid=28544] ppid=28537 vsize=9170400 CPUtime=1179.74 cores=0,2,4,6
/proc/28539/task/28544/stat : 28544 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 455325 0 0 0 117875 99 0 0 25 0 5 0 596384267 9390489600 2252389 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4721.77
Current children cumulated vsize (KiB) 9170400



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1202 s]
/proc/loadavg: 8.19 8.11 8.02 10/187 28612
/proc/meminfo: memFree=2743200/32950928 swapFree=67111528/67111528
[pid=28539] ppid=28537 vsize=9277872 CPUtime=4800.27 cores=0,2,4,6
/proc/28539/stat : 28539 (ttsth-5-4) S 28537 28539 28330 0 -1 4202496 2278120 0 0 0 479528 499 0 0 25 0 5 0 596384246 9500540928 2276770 33554432000 4194304 4338100 140735838151664 18446744073709551615 218892367845 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/28539/statm: 2319468 2276770 176 36 0 2317373 0
[pid=28539/tid=28541] ppid=28537 vsize=9277872 CPUtime=1201.7 cores=0,2,4,6
/proc/28539/task/28541/stat : 28541 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 702246 0 0 0 120014 156 0 0 25 0 5 0 596384267 9500540928 2276770 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 4 0 0 0
[pid=28539/tid=28542] ppid=28537 vsize=9277872 CPUtime=1201.66 cores=0,2,4,6
/proc/28539/task/28542/stat : 28542 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 609455 0 0 0 120027 139 0 0 25 0 5 0 596384267 9500540928 2276770 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254264 0 0 4096 0 0 0 0 -1 2 0 0 0
[pid=28539/tid=28543] ppid=28537 vsize=9277872 CPUtime=1197.46 cores=0,2,4,6
/proc/28539/task/28543/stat : 28543 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 497615 0 0 0 119646 100 0 0 25 0 5 0 596384267 9500540928 2276770 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254208 0 0 4096 0 0 0 0 -1 0 0 0 0
[pid=28539/tid=28544] ppid=28537 vsize=9277872 CPUtime=1199.21 cores=0,2,4,6
/proc/28539/task/28544/stat : 28544 (ttsth-5-4) R 28537 28539 28330 0 -1 4202560 462936 0 0 0 119820 101 0 0 25 0 5 0 596384267 9500540928 2276770 33554432000 4194304 4338100 140735838151664 18446744073709551615 4254388 0 0 4096 0 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4800.27
Current children cumulated vsize (KiB) 9277872

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1202.2 s]
/proc/loadavg: 8.19 8.11 8.02 7/185 28613
/proc/meminfo: memFree=7400988/32950928 swapFree=67111528/67111528
[pid=28539] ppid=28537 vsize=0 CPUtime=4800.49 cores=0,2,4,6
/proc/28539/stat : 28539 (ttsth-5-4) Z 28537 28539 28330 0 -1 4203548 2278120 0 0 0 479531 518 0 0 18 0 2 0 596384246 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/28539/statm: 0 0 0 0 0 0 0
[pid=28539/tid=28544] ppid=28537 vsize=0 CPUtime=1199.4 cores=0,2,4,6
/proc/28539/task/28544/stat : 28544 (ttsth-5-4) R 28537 28539 28330 0 -1 4203588 462936 0 0 0 119820 120 0 0 25 0 2 0 596384267 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4800.49
Current children cumulated vsize (KiB) 0

[startup+1202.31 s]
/proc/loadavg: 8.19 8.11 8.02 7/185 28613
/proc/meminfo: memFree=7400988/32950928 swapFree=67111528/67111528
[pid=28539] ppid=28537 vsize=0 CPUtime=4800.6 cores=0,2,4,6
/proc/28539/stat : 28539 (ttsth-5-4) Z 28537 28539 28330 0 -1 4203548 2278120 0 0 0 479531 529 0 0 18 0 2 0 596384246 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 0 0 0 0
/proc/28539/statm: 0 0 0 0 0 0 0
[pid=28539/tid=28544] ppid=28537 vsize=0 CPUtime=1199.5 cores=0,2,4,6
/proc/28539/task/28544/stat : 28544 (ttsth-5-4) R 28537 28539 28330 0 -1 4203588 462936 0 0 0 119820 130 0 0 25 0 2 0 596384267 0 0 33554432000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 4800.6
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1202.38
CPU time (s): 4800.67
CPU user time (s): 4795.31
CPU system time (s): 5.35919
CPU usage (%): 399.265
Max. virtual memory (cumulated for all children) (KiB): 9277872

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4795.31
system time used= 5.35919
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2278120
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2479
involuntary context switches= 20772

runsolver used 1.88371 second user time and 4.50531 second system time

The end

Launcher Data 

Begin job on node112 at 2011-04-29 09:18:56
IDJOB=3260022
IDBENCH=83164
IDSOLVER=1654
FILE ID=node112/3260022-1304061536
RUNJOBID= node112-1304059146-28348
PBS_JOBID= 13170443
Free space on /tmp= 72564 MiB

SOLVER NAME= ttsth 5-4
BENCH NAME= SAT11/crafted/mosoi/289/289-sat-12x15.cnf
COMMAND LINE= DIR/ttsth-5-4 -num-threads NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3260022-1304061536/watcher-3260022-1304061536 -o /tmp/evaluation-result-3260022-1304061536/solver-3260022-1304061536 -C 4800 -W 1300 -M 15500  HOME/ttsth-5-4 -num-threads 4 HOME/instance-3260022-1304061536.cnf

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= c77d740bbf7d77b30e6053138dd22e73
RANDOM SEED=1262489792

node112.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5261.27
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.93
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      12342460 kB
Buffers:       1995276 kB
Cached:       15797808 kB
SwapCached:          0 kB
Active:        9289140 kB
Inactive:     10758600 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      12342460 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:             828 kB
Writeback:           0 kB
AnonPages:     2253696 kB
Mapped:          15216 kB
Slab:           492332 kB
PageTables:       9064 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:  2459600 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72556 MiB
End job on node112 at 2011-04-29 09:39:00